Can AI be programmed to fact-check its own responses before sharing?

Totally get where you’re coming from! Some of the answers really miss the mark, especially in niche fields. For instance, when I asked about bassoon reed measurements, it gave me answers that were outright ludicrous! It’s wild that it doesn't always give accurate info and sometimes fabricates details, including false names of users in the bassoon community. It feels like there should be a system where you could run the AI's response through another AI for fact-checking before accepting it.

Unfortunately, there isn't a method to completely ensure that AI can fact-check itself before showing results. The systems already have certain instructions to avoid making up answers when they aren't sure, but this can sometimes make them overly cautious and lead to responses like 'I don’t know' even when they really could provide information. It's frustrating because there are cases where prompting like asking the model to break down its thought process can help a bit, but it’s not a fix-all solution for the inaccuracies.

Actually, you might be onto something! The o-series models are designed with more capabilities, but it still doesn't guarantee perfect verification. I once asked a variant of your question and got the reply that 'prompt engineering can't force self-verification' since every output is essentially a separate task without a guaranteed truth source. It's a bit of a hit-or-miss situation.