How did we go from wanting AGI to asking a 6 month pause on it?

Question

I know the letter might be fake, but the overall tone seems to agree with the content of the letter. How did we go from desperately looking for a way to build AGI to a point that we plead everyone to just stop/pause achieving AGI level models? I understand the concerns written in the letter, but isn't the genie out of the bottle? How can anyone be sure that governments won't do it behind the scenes? Isn't AGI a first-mover advantage?

giantg2 · Accepted Answer

"We" might not have changed our ask at all. "We" might be referring to two divergent groups with minimal defectors.
Many people have been urging caution for a long time. As we get closer to more features and capabilities those people are going to ride on the publicity of those achievements to call out their warnings. This will garner more interest because the threat seems closer or more real as the capabilities increase (or at least appear to be so due to company hype).
Then there are many others who are excited and still pressing ahead.

duringmath · Answer

AGI isn't even imminent it's just the Twitter AI hype cycle has gone out of hand and is spilling over and producing this kind of silliness.

taf2 · Answer

I believe the idea of a pause is ridiculous right now. AGI is the biggest thing since the iPhone in 2007 and Google in 98&hellip; anyone thinking about pausing is asking to be left in the dust . It&rsquo;s go time like never before.

wmf · Answer

I haven't seen many people change position. The people who want to build AGI and the people who want to prevent AGI are different.

mejutoco · Answer

Asking for a 6 months pause "to protect x" is just the excuse to further the interests of certain individuals.

markus_zhang · Answer

If something may cause a hike of my chance of losing job I don't really care how rosy it sounds. Yes I use it very often but I prefer it to stay where it is. By the same principle I wish graphics tech to be fixed approximately in the level of late 90s. Well but I don't expect to get what I want.
So this is basically different interest groups competing with each other. Usually the more technically advanced ones won towards the end but the more important thing is how long the other groups can drag it down.
Under the current social-economic world order, any large scale tech advance such as AGI will almost surely bring destruction to a whole generation of workers.
You think AGI, nuclear fusion and such will bring prosperity to us? Too naive, mon ami.

inphovore · Answer

These thoughts of a unified &ldquo;we&rdquo; are the first sign of self deception. We must all get along, yet there will not be found a universal consensus among humanity.Keep on going man, the future shall arrive, and not everyone will be ready.We must continue this debate of our human competence, relevance of public narratives, and true potential of outcomes. All the meanwhile the future presses upon us.Humans are still by far the greatest threat to our humanity.

chatmasta · Answer

Some authoritarians wrote a blog post.

Lionga · Answer

AGI (any reasonable definition of it) is not happening in our lifetime.
We have a few very specialised "AI"s and still struggle since 30 years and trillions of USD investment to even just let an AI drive a car (in all conditions not just when everything is easy).
The step from ChatGPT to an real AGI is much bigger then the step from going to the moon to colonising other planets that are lightyears away.
Both AGI and colonising other planets lightyears away will not happen in our lifetime.

TheAlchemist · Answer

One word - hype !That being said, once we really reach the AGI level - I suppose it will be like this - very sudden.

iExploder · Answer

Anyone with a bit of prescience would know well to follow the butlerian doctrine.In any way I think overhype is playing a role here as well. There is a lot of money to be made here, so motivations are high, tends to make people polarize their opinions.

p-e-w · Answer

Who is "we"? There are thousands of nuanced opinions on what AGI is and what it should be.
So someone wrote a letter. That doesn't mean anything. It doesn't even mean that the average opinion has substantially changed.
There is no consensus on anything here. Beware of so-called "experts" on this subject. There are no "AGI experts". There are AI experts, but speculation by humans on what a hypothetical superhuman intelligence might be like will never be more than, well, speculation. It's like ants trying to wrap their minds around what they imagine human philosophy to be.
Your opinion on AGI is as good as mine, or as Elon Musk's, or as Eliezer Yudkowsky's. Nobody knows anything. Get comfortable with uncertainty, and don't jump to the conclusion that anything has happened just because somebody somewhere said something.

How did we go from wanting AGI to asking a 6 month pause on it?

AGI isn't even imminent it's just the Twitter AI hype cycle has gone out of hand and is spilling over and producing this kind of silliness.

I believe the idea of a pause is ridiculous right now. AGI is the biggest thing since the iPhone in 2007 and Google in 98… anyone thinking about pausing is asking to be left in the dust . It’s go time like never before.

I haven't seen many people change position. The people who want to build AGI and the people who want to prevent AGI are different.

Asking for a 6 months pause "to protect x" is just the excuse to further the interests of certain individuals.

Some authoritarians wrote a blog post.

One word - hype !
That being said, once we really reach the AGI level - I suppose it will be like this - very sudden.

Anyone with a bit of prescience would know well to follow the butlerian doctrine.
In any way I think overhype is playing a role here as well. There is a lot of money to be made here, so motivations are high, tends to make people polarize their opinions.