I've been working with Azure AI Foundry and recently received complaints about the quality of the LLM responses being inaccurate. I decided to test this by asking the same question in both the Azure AI Foundry playground and directly on OpenAI's ChatGPT. In Foundry, when I asked "What version of OpenAI are you?", it responded: "I'm ChatGPT, an AI from OPENAI. In this interface, I don't have visibility into the exact model identifier. I'm a GPT-4 class multimodal model with a knowledge cutoff of October 2024." However, when I asked the same question in OpenAI's ChatGPT, it claimed to be "GPT-5, the latest generation OpenAI model". I'm confused and concerned about this discrepancy. Why does one version seem unsure of its identity while the other does not? Is the model quality different on Azure?
5 Answers
The neural networks used are the same, but the identifiers and how they update or provide information can differ. That might explain the variance you're experiencing. Different content filters and predefined response styles on each platform could also play a role.
Just a heads up, I work on the Azure AI Foundry team. The difference you're seeing primarily comes from comparing Azure's API to OpenAI's ChatGPT interface, not their API. ChatGPT has this hidden system message that sets the context and provides essential details automatically, which is something you can't manipulate in Foundry. If you tweak the system messages in Foundry, you'd likely see similar responses to ChatGPT. Also, remember, ChatGPT has tools like web search that can provide more dynamic context, while Foundry requires you to manually incorporate that if needed.
Honestly, from what you're describing, it seems like you could benefit from a deeper understanding of how these AI systems work. If you're managing AI tools at your job, you might want to get into the details of how they operate.
It sounds like you're not really comparing apples to apples here. In Azure Foundry, you’re working directly with the base GPT-5 model without the additional system messages that ChatGPT has. ChatGPT has pre-defined instructions from OpenAI that give it context, while in Foundry, you need to set all that up yourself.
Essentially, Azure provides the model while ChatGPT combines that model with a host of extra tools and enhancements. That's a big part of the reason you're seeing those differences in responses.
Exactly, the model's the same, but it's all about how you access it.