Hey everyone! We're a medium-sized resources business dealing with the usual IT headache of ensuring user safety when it comes to handling sensitive documents, especially with the rise of public LLMs. We've got about 200 staff members, but around 50 of them will be heavy users who rely on these tools to rewrite and analyze documents. It's crucial for us to make sure that no sensitive data leaks to these public platforms.
We tried using Copilot, but it didn't meet our needs. Now, we're considering setting up a custom front end for our users, effectively blocking access to public LLMs. We're thinking of using something like Open WebUI and host it on our ESX cluster, managing API access through OpenAI with a fixed budget of tokens for each user.
Also, I'm curious if Azure SSO would be a viable option for us. I've heard about Azure OpenAI, but I'm uncertain about the best direction to take. Can anyone share insights on effective solutions and estimate costs? Just a note that we're based in Australia!
5 Answers
Copilot recently added support for ChatGPT-5, which could be a great fit for you. Sure, you can build your own setup, but remember that tech evolves fast! Part of the challenge is keeping up with things like multi-modal capabilities and memory management. Big companies are using enterprise solutions with strict privacy policies, so that might be worth considering.
I think Azure OpenAI is your best bet, especially with a private endpoint and Open WebUI connected to Entra ID. This way, your data stays safe within your network, you get seamless SSO for your users, and starting everyone on something like 4o-mini can help control costs.
Mistral offers GDPR-compliant APIs, which is great if privacy is a huge concern for you. You could run something like GPT4all locally for document processing while using the API for quick answers. Just keep in mind that managing a local setup for 50 users could be resource-intensive.
This is a common challenge for IT teams these days! Your idea of a custom front end using an API is solid, but be ready for the overhead that comes with managing the VM, user authentication, and tracking token usage for each person. Azure OpenAI seems tailored for enterprise needs and integrates well with Azure AD for SSO access. If you're looking for something a bit simpler, I work with an AI company called eesel. We built a tool that connects seamlessly with existing knowledge sources and provides a secure internal Q&A solution for teams. If you're interested, happy to share more details!
You might want to stick with Copilot if you can implement Enterprise Data Protection. Developing a policy that makes it the go-to choice for everyone could help ease data concerns. It's less about the tech and more about setting the right policies to handle breaches as they come without stifling productivity.
Related Questions
Neural Network Simulation Tool
xAI Grok Token Calculator
DeepSeek Token Calculator
Google Gemini Token Calculator
Meta LLaMA Token Calculator
OpenAI Token Calculator