Should LLM think before answering?

Question

ChatGPT can often correct it's answer, if a mistake is pointed out. That means it has the correct data, it just doesn't know how to retrieve it.So, what if it would try to automatically challenge statements it has made, before giving a final answer. Do you think, it would be possible to improve LLM's accuracy with this kind of auto validation feedback loop?Like we need to think before speaking, to consider what we are about to say.

novaRom · Accepted Answer

This is what Bing already does, and probably chatGPT when it generates some long text and then finally decides to remove it showing a banner of error occurred. This is trivial to do, I mean you can add this to Alpaca easily.

theGeatZhopa · Answer

Good idea. But the wrong audience.The guys at OpenAI are aware of that. For sure.The scientific community is aware of that. For sure.So. Why is this done, like it is done?

gjvc · Answer

it's done its "thinking" while being trained. I expect it gives wrong answers as a weak anti-plagiarism feature.

ClapperHeid · Answer

>ChatGPT can often correct it's answer, if a mistake is pointed out I've wondered about this myself. As you say, ChatGPT often responds instantly with an incorrect answer to a question. and then, if you "ask it to think again" comes up with a better one. It kind of reminds me of the kid in class at school who would always instantly leap to his* tiptoes with hand shooting in the air, as soon as the teacher began to ask a question. Quite often coming out with the wrong answer because in his haste to get in there first, he hadn't listened to the queston properly.*["His" because it was always a boy. Girls aren't so overtly competitive]By coincidence, here's an example from earlier today when I wanted to know the name of a move I'd played in a few chess games:----ME: In chess is there a term for an opening where black sacrifices his black bishop on F2 or white sacrifices his white bishop on F7. Thus forcing the opponent's king to have to move and therefore preventing him from castling later?ChatGPT: Yes, there is a term for this kind of opening. It is called the "Scholar's Mate" or "Fool's Mate." The moves that lead to this opening are...[I hit "Stop Generating" here]ME: That is not the answer to what I asked. Please read the question againChatGPT: I apologize for misunderstanding your question. The opening you are referring to is called the "Bishop's Gambit" or "King's Gambit Accepted." In this opening, White sacrifices their f-pawn to gain control of the center of the board and open up lines for their pieces, including the bishop on c4.If Black accepts the gambit by capturing the pawn, White can then play Bxf7+, sacrificing the bishop to force the king to move and prevent castling. This move is known as the "King's Bishop Gambit." Similarly, Black can play the "Queen's Bishop Gambit" by sacrificing their bishop on f2 to achieve the same goal.----A trivial example. but it demonstrates the point. It's like "teacher" started to ask 'What is the move called in Chess...?' and "Little Johnny" immediately leapt to his feet and blurted out 'Scholar's Mate!', without even listening to the question.I suppose in a way it makes interacting with ChatGPT seem a bit more human. Although I doubt it's meant to be like that.......or is it? Here's a conspiracy theory for you, that's just occurred to me: What if ChatGPT is programmed this way deliberately so that, in occasionally having to correct it, we feel more like we're collaborating" on a semi-equal footing and less like "the machine" is our infallible invincible superior?