Is September 2021 the new 'eternal September'?

Question

Both GPT-3 and GPT-4 mark this date as their 'knowledge cutoff date'. Maybe this date is chosen on purpose, in order to avoid a negative feedback loop, where the learning process of their own LLM is influenced by the output of bots that are powered by their own engines?A public reminder: September 1993 was the old 'eternal September', where the big collaborative message board of olden times died, due to a sudden influx of new users. See: https://en.wikipedia.org/wiki/Eternal_September

ilaksh · Accepted Answer

Training involved collecting and preparing a massive dataset and they can't just add to it on the fly without restarting the training. Training the model took a long time. Once they had the initial trained model they needed to test it. Then they started a huge additional training effort with human feedback.All of those things take time.