Why Are Anthropic’s Models Slower on Bedrock Compared to Other Platforms?

Actually, I've found that the Anthropic models on Bedrock can be faster than other providers, including Anthropic's own offerings. Remember, Bedrock has two APIs: non-streaming and streaming versions. By default, Anthropic streams data. You might want to switch your code to use the streaming API; that could boost your performance. Also, make sure you're using global inference if you aren't already!

That's interesting to hear, but what kind of latency and token throughput are you experiencing on Bedrock compared to other platforms? I've noticed my models sometimes get a bit 'congested' too, but AWS seems reluctant to admit there's an issue unless you’re spending a lot. Just keep in mind it's an on-demand service without any performance guarantees.