Why doesn't ChatGPT try to learn while chatting with experts?

> Why doesn’t ChatGPT try to learn while chatting with experts?

1. ChatGPT doesn’t try anything, that’s a rather extreme anthropomorphism that doesn’t map to how ChatGPT works.

2. ChatGPT does “learn” from everything said in a conversation (“in-context learning”), and, prompted properly, it will request information from the person it is interacting with.

3. But, unless it is presented in the conversation, ChatGPT has no information about who it is talking to.

4. And, everything it learns through in-context learning applies only to the conversation it is learned in (that is, its only available as long as what it got it from is in the prompt.) The mental model of ChatGPT as a single, persistent entity that can “learn” things in a permanent way (separate from the periodic supplemental training and new model releases by OpenAI) is just a fundamental misunderstanding of what it is.

It might be an interesting use for a long-context LLM to build a chat agent that incorporated persistent, “universal” in context learning by selectively adding material that was expected to have broad utility from conversations into the universal system prompt, but there’s a whole lot of issues there (including with privacy.)

It doesn’t “try” anything… it can’t “try” anything. If prompted to ask a question it can ask a question… indeed if the provided prompt leads to a question being the statistically most likely response then the response will most likely be a question. At no point and in no way is it attempting to gather novel information. You could specialize it towards asking tougher questions of those who prompted it that it was chatting with a famous mathematician, I suppose, though it wouldn’t know a famous mathematician from anyone adequately mimicking (or really just purporting to be) that mathematician. It could be made to learn their response and give it greater than usual future weight, as well, but woe be unto anyone who tries to use what it learned from such a one-off interaction to do anything important.

It will learn. It's called "in-context learning". This learning will be limited to the current context window, because it won't get encoded in the model parameters. In order to make it permanent, you would need to retrain/finetune the model using the new information.

ChatGPT can describe curiosity but isn't curious. ChatGPT can tell you everything about trust, but does not know what it means to trust or distrust. ChatGPT, while it will say to you and me that it doesn't know something about a topic, does not have intellectual humility.

World famous mathematicians will have written all their well-formed novel ideas up in academic papers, which are readily available.

I don't know what you would learn from a conversation, except for the half-finished stuff they're still working on before it gets published.

aside from what others have said, here's another point : it's not easy to know what you don't know.

Every answer given by GPT is the best answer that specific model can give given the context and relational clues, but counting those 'nodes' of context and relation clues isn't actually a good metric for discovering the unknown and finding holes in the model.