Many of these finetunes or base models are dumped with little info and private training datasets. How much of those datasets are questionably legal or ethical? How many are accidentally or intentionally contaminated with test data? No 3rd party ever seems to test for contamination outside the rare social media post.
I see a whole lot of frameworks, papers and APIs claiming SOTA transformers/diffusion performance in this or that, but they seem barebones and janky when one actually goes and investigate them, if you can get them to work at all. Or they are grand promises asking for money now and usability later.
...I am no data scientist, but I am a straight up AI fan. I was finetuning ESRGAN 1x models for img2img/vid2vid years ago. GenAI is the best thing since sliced bread as far as I am concerned. Yet I am getting a serious crypto fraud vibe from everything that is happening, albeit one more targeted at businesses and investors than the crypto wave.
Most models on HF are like CrapCoins of the crypto era, so forget about 99.5% of them. There are only a few that can be useful (mostly +70b params).
Don't forget that some of this hype can be driven by nVidia to sell more GPUs. Some of it is driven by crypto enthusiasts who now use their GPUs for GenAI.
Forget about benchmarks and SOTA performance claims. Data leaks and you'll never be 100% sure that the team didn't mess with training data to manipulate the benchmarks.
Is GenAI a hype? Most likely, yes. Can it still be useful? Yes. You can still use GPT-4 and Llama 2 70b models for various tasks. Just don't use Langchain or any frameworks. Make your own API calls. And the smaller models (3-7b)? They are useful for when you don't want to chat with the model, like sentiment analysis.
AI doesn’t have any rivals, perhaps faster and lower-power chips but we haven’t seen all the implications yet.
Instead this seems like the standard tech hype cycle where crowds of investors pounce on the “next big thing” which will inevitably be overhyped and oversold, leading to a bubble pop when expectations outpace actual returns, but the core technology is valid. The dotcom bubble popping didn’t mean the Internet was a fad, the video game crash of 1983 didn’t permanently doom the industry, the railway mania in the 1840s didn’t mean no one ever took a train after it crashed… any new and exciting development attracts grifters but it eventually stabilises once people learn to separate the hype from the real applications of the tech. I remember when VR went through this - it was absolutely overhyped, but today there’s still a solid community around it even if it didn’t change the world.
With open source models and fine-tunes being uploaded on Huggingface, there’s no fraud, they’re perfectly free, and you can test them out and see for yourself that some work better than others even if benchmarks only tell some of the story. There’s absolutely a glut of thin veneers over APIs and models offering questionable benefits, but there’s also real value in having a solid UI on top of a basic model.
Prior to LLaMA in February there were almost no openly licensed language models that were any good at all - BLOOM and GPT-J were pretty much the best of the bunch.
Then LLaMA happened and the flood gates opened - there was now a base model which people could fine-tune against to get really great results.
Llama 2 accelerated things even more because it's licensed for commercial use - so now it's worth spending real money building on top of it.
Llama 2 isn't even 2.5 months old at this point. I'm not at all surprised at the rate of innovation around it - there is SO MUCH scope for improvement, and so many new techniques to be explored.
I expect we'll see a slow down in the release of models at some point, once the low hanging fruit has all been picked off - but it's going to be a while away yet.
My primary concern is the ratio of AI-infrastructure products being built relative to the average utility delta of current end-user products. There are some shining counterexamples to this like Github copilot, but for the most part the utility gain from most text-based generative-AI seems incremental.
Current language models are bad to okay at most tasks - and great at a select few. The capabilities in unstructured ETL and production of labeled datasets fall in the latter category.
I think the next generation of really powerful capabilities to be unlocked with generative-AI will come with 1) an increase in compute availability (training even larger models) 2) advances in multimodal models, and 3) better interfaces to interact with them. Spatial computing will be a really big deal here, and people seem to have forgotten that Apple may have unlocked the next version of interactive computing earlier this year.
The competition for AI is simply any solution that gets a higher quality of answer than a trained approximation. That's actually a lot of things, but often there's a chain of dependencies in the information lifecycle that lead to combinations of technologies, and so our total solutions frontier expands by having the tech as a "drop-in" option that would otherwise be some custom engineered thing. But the AGI hype, to name one "unlikely AI thing", is silly. Based on what we know, larger genai models are likely to be more normal, not more novel. And normal isn't right or good, it just tells us what we want to hear.
Crypto is actually covering a polar-opposite technological context: it lets you get very high quality answers within a very narrow space. Done right, it tells you what you DON'T want to know.
Personally, I haven't found a tool like ChatGPT+ that instantly improved my productivity overnight. Brainstorming, writing code, documentation, marketing copy, transforming data etc all became easier with this tool. I'd rather spend a shorter amount of time providing clear instructions to a language model than writing a lot these things from scratch.
Anecdotally, I've tried all the major models from Google, Meta, Anthropic, Stability etc nothing comes close to GPT-4 in understanding instructions properly.
But I think applied AI, AI saas, and so on built on top of the best models to optimize specific workflows has a pretty bright future. There will be failures and false starts as with any new technology, but there are already a number of useful apps out there with wide adoption, which is more than crypto ever achieved.
With crypto, the product is speculation. There are some potentially interesting applications for crypto, but it's never gotten large-scale traction as anything other than a form of gambling. AI doesn't have speculation built in like crypto does. It has to actually solve problems and do useful things to make money, and this is already happening.
There may be a speculative bubble in the VC realm, but when isn't there? It happens literally every time a new startup trend comes around.
Bitcoin is a solution in search of a much bigger set of problems than it can currently solve now. The ML Renaissance we've hit right now has clear, immediate utility to many. The moves made by the stock art heavyweights is proof enough of how disruptive it is.
Contrast this with how banks and financial institutions have responded to bitcoin. To my knowledge, there's yet to be a Bank of America Coin, a Wells Fargo blockchain solution, or non-fungible tokens offered by RBC. They clearly felt at least somewhat threatened, but not in any meaningful way.
The downrounds and devaulations for the ChatGPT wrappers are coming in [0] and realizing that most of these 'AI startups' are overhyped. Especially the 'Chat with PDF' web apps and basic copywriting apps.
To suggest otherwise is pretty naive, and the large AI providers (OpenAI, Google, etc) do compete against their own partners.
There are a bunch of models because it's easy to grab llama2 etc and do a little bit of fine tuning. There is nothing wrong or bubbly about that.
It has adoption that might not be every person in the world, but the depth and breadth and variety of users is hard to ignore.
Crypto wallets and getting up and running is the failure for most crypto. Yes, there have been improvements to that.
LLM's that took a chat interface went as universal as they could - just a computer anyone can talk to.
One issue with looking at HuggingFace and expecting maturity in 10-12 months is I'm not sure what can be compared as an equal in that number of months in other technologies. I agree it's far from perfect, but also it seems to be improving.
LLM's are just one kind of GenAI as you have pointed out. The AI you were working with before all of this remains relevant.
There's more to this than wanting a prompt to do everything magically. Having to "fine tune" and get each piece working may be the work for a while, and still be usable and beneficial.
Does it need to evolve and get better? It is.
With each passing month are more and more models running on more accessible hardware? It is.
Are people quietly doing a lot with it and not sharing it yet? Sure feels like it.
If you mean the bubble of excitement for AI, then I highly doubt it. LLMs are paradigm shift in how we use computers and what we can accomplish with them, and we've barely scratched the surface of them.
Generative AI as a technology isn't going anywhere and is very much going to increase in scope, scale, and necessity in most workflows.
But the vast majority of AI companies that cropped up riding the swell are going to be out of business in the near future.
Which is pretty standard. 90% of new companies aren't still in business a decade later no matter the field.
It's just that there's been a lot more new companies in a single new space since maybe 1999.
So expect a significant 'pop' of a bubble without there being any underlying issues with the tech or its applications.
(Possibly even more than typical given the rate of change and likelihood any custom product right now is building into its own obsolescence.)
There’s still a lot of evolution to be had with AI; for example gen AI to interactive AI that uses all the bits we’ve seen so far to compose experiences with more than chatbot interaction.
If anything is going to die in AI it’s all the bespoke projects, not AI as a whole.
Am working on a Linux distro that ditches userspace as we know for a local LLM (root login is still available). The goal is no user accounts, /home and the AI encodes user data as vectors to disk. Need to successfully unlock the vectors rather than a user account.
So far it’s Linux From Scratch, Wayland, hyprland which displays the GPU viewport.
There is still a lot of weird stuff to do with AI. What do I use blockchain for except as yet another fiat currency