Hey folks! I'm working on a chatbot that generates diagrams from our data stored in Dataverse. Since I can't directly use Dataverse with Azure AI, I've exported the data into a CSV file and am trying to set up Azure Search to work with this structured data.
I'm facing a few key issues:
1. **Inaccurate Search Results:** When I run simple queries like "Give me all projects", I only get a fraction of the results—sometimes just 5 projects instead of the many I know exist. Other queries also return incorrect or incomplete results.
2. **Lack of Guidance for Structured Data:** There seems to be a scarcity of tutorials or examples for using Azure AI Search with tabular data. Most resources focus on working with unstructured documents like PDFs, which hasn't been helpful.
3. **Costly Workaround:** Right now, I'm sending the entire dataset to the AI whenever the chatbot gets a request, which is obviously not efficient and racks up high token costs. I'm desperate to find a better way for the AI to access just the needed data without this overhead.
I'm looking for advice on how to integrate my CSV data with Azure AI Search effectively, particularly in optimizing indexing and query parameters to get accurate and complete results for my chatbot. Any tips would be greatly appreciated! Also, just so you know, I'm using the Mermaid JavaScript library for creating the diagrams based on the syntax the AI provides. Thanks in advance!
1 Answer
It might be simpler to switch to a regular SQL database instead of relying on Azure AI Search. There are some resources like the 'NLP in a Box' sample that you can check out, which could guide you through using a SQL solution with NLP capabilities. Plus, it should integrate well with Azure services!
I don't have server access and need to stick with Azure Services. If you know other options that fit within those constraints, I'd love to hear them!