I'm in the process of building a chatbot that utilizes the invokeagent API to interact with my Bedrock agent. This agent uses a RAG model alongside a knowledge base, and I've added instructions to ensure it queries solely from this knowledge base. However, despite these precautions, it sometimes pulls in general information that's not from the knowledge base, undermining the purpose of my setup. What steps can I take to ensure the agent only retrieves information from the knowledge base?
1 Answer
It sounds like you might still be getting responses from the language model's own training when there's nothing in the knowledge base. One approach is to adjust your system prompts. You could say something like, 'If there's no relevant info in the knowledge base, please stop generating and report that you don't have an answer.' That way, it won't just make things up.
That's true, but I can't expect every user to know how to fine-tune prompts. Do you have any other suggestions?