What happens if ChatGPT start giving the wrong answer

Question

ChatGPT is a blackbox. Many people are asking if it could be used to solve many queries people have. What happens when that blackbox has a manipulated result? I'm not picking on OpenAI, or any company, but I could see world where one of these services is used to change what we know about history, because we don't know how they were created.Example: Who won the Stanley Cup in 2019? - San Jose Sharks. (this is incorrect - St, Louis Blues beat SJS in semi finals) What's the duty of a source of data to reference its answers. Search engines at least provide reference to where they got the information.(I use Hockey cause it's less controversial and GPT3 doesn't have that data today)

smadge · Accepted Answer

ChatGPT already gives wrong answers. When I asked it to cite sources for its incorrect information it made up a non-existent journal article and provided fabricated direct quotes from the non-existent article.

gardenhedge · Answer

I guess that is why they called it ChatGPT and not AnswersGPT. It is concerning that some answers can be convincingly wrong, but I see it as a tool. If I have a hammer, I can still mess up hammering a nail into a wall. If I have ChatGPT, it's up to me to take the text response and use it appropriately.

afinlayson · Answer

I should have titled it Wrong answers on purpose ... I know there's some wrong answers ... but it's not on purpose that I can tell. If everyone is using this blackbox as a data source it's really easy to abuse.