I've been using Azure OpenAI for hosting fine-tuned models and it works well for adhering to data residency requirements in the EU. However, the content filters can be really frustrating. They seem overly strict, and I run into issues even when the content is legit, such as in injury insurance or medical documents where explicit terms might trigger the filters. The only way to disable these filters is if you're a large company, which leaves smaller businesses like mine at a disadvantage. We can only adjust the filters to be less aggressive, but sometimes they still block necessary content.
I'm searching for alternative LLM providers that offer deployment options at a similar cost, specifically with per-token billing and the ability to adhere to data residency requirements. So far, I haven't found anyone that provides data residency with per-hour billing. This makes it hard to consider a fallback solution without incurring massive additional costs on top of what we're already paying to Azure. I'm looking to hear others' experiences and any potential solutions to this issue. Cheers!
2 Answers
I totally agree! I’m working on a personal project related to food recipes, but the content filter keeps blocking me just for mentioning names in the text. It makes sense to have filters, but it feels excessive at times.
Exactly! If only we could create some kind of workaround, but those restrictions really keep us in check.
I was able to get my content filters lifted by telling them I was moving from OpenAI APIs to Azure for the MACC discount, and they didn't ask any questions. It worked out well for me, as I was able to create an 'empty' content filter for all my models!
That's really helpful to know! Sounds like they are responsive when it involves customer retention. Maybe mentioning migration plans could help others as well.
Interesting approach! I wonder if that would work for others too.
Right? Have you tried adjusting the filter settings to only block higher severity stuff? It seems like they could really use a rephraser tool for prompts!