Does it feel like AI progress is slowing down, or being hidden?

Having worked on developing a GPT-4 based product from the day we got access, which was quite early as a YC company, I would say it's the following:

- GPT-4 is too slow, and GPT-3 is not powerful enough to be 'good enough'

- Therefore, a lot of 'the work' is coming up with UX magic to overcome this

- It just is not reliable enough for business applications (yet) without hardcore QA that is currently being invented and improved materially on a weekly basis

- Therefore, the progress is in the last mile stuff like productionizing, QA, UX idiom invention, and/or coaxing GPT-3 to be good enough and go fast enough

- All of what you're mentioning are LLMs, and GPT-4 seems likely the theoretical maximum LLM based on currently available knowledge

- No doubt some more powerful models are in the works, but then the same process repeats (e.g. assess capabilities, invent UX idioms, QA and wait)

So a conspiracy that incredible things are happening but no one is talking about it to me is not accurate. More like productionizing ChatGPT/GPT-4 is very early and putting new technology into use is always harder than it says on the label.

Well, Google decided to stop publishing their research after losing the battle to ClosedAI, which Google themselves started (the transformer paper that Google researchers wrote in 2017). So it's reasonable to think that companies are not going to compete with ClosedAI by being open about their latest achievements until they have something comparable to show. They might have achieved incredible success in terms of inference speed, but not in the quality of responses.

It's also possible that they're waiting to see how the regulation landscape changes. Big tech like Google and Meta are under much more scrutiny than a small company like ClosedAI, so they might need to solve legal issues before they can unleash their models. For example, Meta didn't release LlaMA for commercial use partly due to legal reasons. I've heard they're working on the next gen LlaMA w/ commercial license. I hope that happens so that ClosedAI realizes they can't enjoy their monopoly for long unless they become more open about their products.

It will continue to slow down. We just don't have the know how to move to the next level. Faster hardware might bring about more gains but there's a limit there too.

I see it as a plus. The current AIs will change the way society functions so it's a good thing we have a bit of time for society to catch up.

The current AI naysayers though have a point to be very careful with AI breakthroughs. There's a non-zero chance that eventually we will hit on the AGI formula and create something. Maybe not in our lifetimes but eventually. It makes sense to start preparing now.

I think the AI hype was a little overblown, a lot of things by people not understanding the field or just out right wrong. Don't get me wrong, AI will come and it will do most of what people are talking about, if not all and more, but there have been a lot of unrealistic time frames thrown about.

Look at self driving cars, I'm pretty sure ten years ago people were talking about it being a huge thing and being almost here and here we are in 2023 and it feels like there is far less buzz around it now and any system in public use seems highly problematic and limited, despite many more years of development.

It's the same in the AI world, although it keeps ticking away, one of the first breakthroughs that got public attention was image generation and in particular midjourney and they have been adding more cool stuff lately, it's all continuing at pace but some people have unrealistic expectations.

People saw ChatGPT and suddenly that was AI for them and it was everywhere and then ChatGPT/ GPT 4 came along and since that was only a few months afterwards, everyone is expecting GPT 5 soon, which I believe OpenAI have said hasn't even started training yet. The reality is that there was a bigger gap between GPT 3 and GPT 4 then people realised as it was less public and potentially the gaps will get bigger as it needs more training/development as it's harder as the low hanging fruit has already been picked.

There is also a bunch of tech floating around at companies that don't know what to do with it yet, facebook for example have the audio generation thing they don't want to release to the public as they said it was too 'dangerious'

It's progressing, but maybe not in the way you think. 1 model will not rule them all, they'll get overtaken by vertical LLM's specialized with niche, high quality data sets.

Also prompt templates are rapidly progressing. "Tree of Thought" is the latest state of the art, but we're not applying predictive probabilistic tree traversal yet.

There's still a ton of low hanging fruit everywhere. Why are we inputting with language instead of code? Seems really backwards to use imprecise freeform language instead of intent templates.

I wonder if there's something like the Doppler effect going on here. The field continues to advance at a constant rate, but it takes time for news to reach us, proportional to the distance between us and the frontier.

To a stationary observer, it would appear as if the time between developments is increasing, and that the field is slowing.

I hope I'm wrong to use this analogy, though. I'd like to continue thinking of myself as relatively informed about technology.

Does the Gartner Hype Cycle not exist in your world?

No not really. It also felt like an AI "winter" after the fairly transformative Talk-To-Transformer released, but we eventually got better stuff in the end.