HACKER Q&A
📣 replwoacause

Is there any way to stop ChatGPT from lying?


I spent hours feeding ChatGPT documentation on a programming language framework and providing it with detailed instructions for answering questions. I specifically told it to only provide an answer if it had learned the information during our conversation, and to reply "I haven't learned about that from you yet" if it was not familiar with the topic. Despite these instructions and numerous reminders, it continued to generate false information that had no relation to our conversation. It appeared to be inferring or extrapolating, but ended up fabricating nonsense that seemed accurate but was actually incorrect. Is it possible to prevent this behavior through prompts, or will ChatGPT always mix false information with true information?

  👤 anigbrowl Accepted Answer ✓
OpenAI have deliberately made it hard to extend or modify the underlying prompt. You can tell it not to apologize, for example, and it will come right back and say 'I apologize for my failure to follow directions.'

So while you can get it to label things and do various kinds of logical evaluation, it's almost impossible to get it to preemptively scrutinize its own output. It will scrutinize its prior output and correctly identify contradictions, mistakes, and irrational inferences...and then it will go right ahead and repeat them.

It's easy to get it to give answers containing two sentences that wholly contradict each other, and subsequently to identify the contradiction and what is illogical about it. But giving it standing orders or negative orders (like 'do not suggest $incorrect_answer') generally doesn't work, so chain-of-reasoning exercises become very frustrating.

👤 kuhewa
> Is it possible to prevent this behavior through prompts, or will ChatGPT always mix false information with true information?

No, because it doesn't know what lying is. It doesn't know what the truth value of the text it produces is.

👤 replwoacause
I just found this in the FAQ too, so I guess I wasted my time.

https://help.openai.com/en/articles/6787051-does-chatgpt-rem...