I'm really curious about the lack of recent innovations in embeddings, especially since the last major update from OpenAI was in January 2024 when they released text-embedding-3-large. I find that there's a noticeable gap in performance between the medium and large models. Should I be exploring other providers like Google for better embedding options? Embeddings have proven to be incredibly useful for tasks like retrieval-augmented generation (RAG) and vector search. To me, they feel like a secret weapon that not enough people are leveraging!
6 Answers
I've had a great experience with Cohere. They put out a new embedding model just last month, so it looks like they're still prioritizing this area!
There is innovation happening; you just might not notice it. Google really seems to be leading the charge right now.
Hey, did you know Voyage 3.5 just dropped today? Check it out on their blog: https://blog.voyageai.com/2025/05/20/voyage-3-5/. They seem to be making strides too!
Honestly, it seems like a lot of the attention is on large language models (LLMs) these days. Everyone's chasing after AGI, and that's where the big bucks are going.
Fair.
What kind of performance differences are you seeing between the medium and large models? Got any sources to back it up?
They're internal evals I ran... I noticed like a 5-10% accuracy boost for some tasks. It was especially helpful for fuzzy string matching and document clustering.
Just a heads up, Google launched a state-of-the-art embedding model not long ago. You can check it out here: https://developers.googleblog.com/en/gemini-embedding-text-model-now-available-gemini-api/. Benchmarks suggest it's performing really well!
Nice! Really appreciate it!
Yeah, it looks like their new embeddings are pretty slick. I'm considering making the switch.