HACKER Q&A
📣 sottol

GPT/AGI – are we getting of ourselves again?


Transformers and esp chatGPT and llama.cpp seem to have kicked off a new wave of excitement around AI (and not just here).

A lot of the hypothetical discussion of challenges and changes to our society (and how it will be totally different in X<5 years) reminds of the excitement around self-driving cars around 2013-2015, how car ownership and our cities themselves would change in the next 5 years. Looks like we're still 5+ years away from fusio\b\b\b\b... err, self-driving cars.

I'm not the first to say it, but in my experience we really do "tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run." -- is this another case of that.

Also, this would be a great occasion to post your predictions. Imo making public predictions that you can fall back on is a great way to check yourself in the future!

[1] https://news.ycombinator.com/item?id=35182824

[2] https://news.ycombinator.com/item?id=35179539

[3] https://news.ycombinator.com/item?id=35179539

[4] https://news.ycombinator.com/item?id=35184299

[5] https://news.ycombinator.com/item?id=35181139

....

  👤 Satam Accepted Answer ✓
Huh, interesting, just posted almost the opposite of your thoughts. I'm aware we tend to overhype things, so I tried to base my predictions on the development and release cycle we've seen so far. Here's my (admittedly sci-fi sounding) timeline:

1 year from now, GPT-5 or equivalent is released

2 years from now, a more promising LLM architecture XXX is discovered

3 years from now, the new XXX architecture is productionized

4 years from now, XXX V2 is released

5 years from now, anyone that weren't already scared, are now in complete panic after seeing XXX V2's reasoning abilities. World leaders commence to discuss and plan the banning of unlicensed AI research. Access to non-trivial amounts of GPU resources are restricted, research now has to be approved by the government.

What happens after, I'm not sure.

👤 sottol
My prediction is that we're probably soon getting to an inflection point where gains made by transformer-based AI models will soon be again more incremental than revolutionary. The mildly informed reasoning being based on training data availability (GPT3/4 being trained on a good chunk of the internet), training and running costs not coming down as fast as in the past due to diminished Moore's law, and GPT4 going multi-modal (is text-based training close to maxed out?).

I do think that there will be another leap in 5-10 years and equally exciting developments in other applications of AI.