I'm working on developing an internal chatbot for an insurance company and need some advice on selecting the best language model on AWS. Since our system is going to handle a lot of heavy database queries, like policy lookups and customer details, I'm looking for a model that meets a few key requirements. It needs to be fully integrated with AWS, must focus on text-to-text interactions, and should be cost-efficient for high usage levels. I've come across Anthropic Claude 3.5 Haiku and Amazon Nova Lite, but I'm really interested in hearing about real-world experiences with AWS Bedrock models. If you've implemented chatbots or high-volume automation with these models, which LLM provided the best combination of cost, performance, and reliability? Any tips or recommendations would be greatly appreciated!
1 Answer
From what I've seen, Nova Lite could work well for you. It's affordable, fast, and decent overall. Just keep in mind that its input token limit is pretty small, which might get filled up quickly with database calls. You might have to consider using Sonnet 3.7 or 4 to balance cost efficiency and functionality.
Have you considered Nova Pro instead? It might offer some benefits.