Hey everyone! I'm a junior system administrator in the IT department of a medium-sized company, and my team is exploring how to implement AI to optimize some of our backend processes. We're in the early stages of research since none of us are really experts in AI yet, and we want to gather some insights before pitching ideas to management.
Data privacy is a major concern for us, so we're leaning towards self-hosted solutions like Ollama but are worried about the upfront costs, particularly for GPUs which can be super pricey—definitely don't want to shock our CFO with a $10,000+ proposal!
Here are some ways we think AI could help us:
1. Generating reports
2. Document analysis
3. Analyzing marketing trends
4. Sales insights
5. Identifying duplicate customer accounts
6. Monitoring orders to prevent fraud
7. Summarizing VoIP reports
8. Assisting with basic product research
Is there a platform that strikes a good balance between cloud hosting and fine-tuning while keeping our data safe? Or maybe an option for renting GPU power from a third-party in case we want to keep things on-site? I appreciate any advice as I'm still figuring things out! Thanks!
2 Answers
I’d recommend starting with a hybrid RAG setup on Azure OpenAI or AWS Bedrock. This way, you keep prompts within your network, protecting any personal identifiable information. You can avoid the costs of GPUs at first! Here’s a breakdown of how you can implement it:
- For reports/sales, utilize LLM text-to-SQL while having a review step in place.
- For document analysis, chunk your files and embed them, then use RAG for responses.
- For duplicates, use tools like Splink or Dedupe on customer tables and automate the merging to your CRM.
- You can also rent GPUs if you ever find a need for them, via platforms like Runpod or Lambda Cloud.
If you're using Office 365, you might want to check out Microsoft's Copilot since it has built-in enterprise data protection. It could be an easy option without major overhauls!
We don’t actually use Office 365, but I might suggest it to my boss. He's pretty set on individual licenses, but I'll keep an eye on what Microsoft does with that in the future!