I've been working on a project where I use AI to transcribe musical notation and guitar tabs as a blind individual. I give the AI a tab or notation file and ask it to describe which frets to play. Recently, I tried out the new o3 model, but it seems to struggle quite a bit and often hallucinates wildly. I don't have a deep understanding of AI, but I'm curious why it has such difficulties with tasks that require precise numerical answers. For instance, while it can describe geography well, it doesn't reliably read my microwave settings. Why is the transcription of guitar tabs so problematic?
5 Answers
Really, it's about the availability of data. If you’re open to creating a dataset for better training specifically for tasks related to guitar tabs, that could potentially help. What materials are you looking to input?
Totally! Having structure in your output will help the AIs process it better, especially when it comes to complex notation!
I’ve tried using Songsterr for transcription, and while it gets started pretty well, it still misses a lot of details like slides and legato. Plus, a lot of free tabs out there are just plain wrong, which complicates things. You might consider using Songsterr to convert tabs into Guitar Pro files and then get the model to generate audio instructions from there.
That's smart! It's all about working with what the models can handle, right? Exploring the usage of strong datasets could open more doors.
Exactly! Since models can't process sound directly, it might help to generate files with clear formatting and structure, then have the AI interpret those regularly.
I completely sympathize; it just feels like these models have limitations, especially with topics that are less frequently addressed. You might want to train them on specific examples to improve recognition.
Well, I don’t have the skills per se, but I’d love to see more musicians getting what they need. Sharing our knowledge might spark a breakthrough!
Exactly! Building context around the music could improve results significantly. We should definitely collaborate on this!
Model performance can vary quite a bit. I've heard Gemini 2.5 has improved image recognition, but it's still prone to hallucination when it comes to musical details. Might be beneficial to dig into that for your needs.
It sounds like a tricky situation! Have you thought about how you're phrasing your prompts? With a specialized request like this, providing clear examples of your inputs and what you expect can make a difference. That could help the AI understand what it needs to do better. You know, it might even help to describe guitar tabs in detail as if the AI is learning about them for the first time.
I agree, it really seems like the models can get lost if they don't have a solid understanding from the start. Even describing music in general could be tricky for them.
Adding more context in your prompts could certainly make a big difference! Sometimes less is more, but with something as complex as music, clarity is key.
I'm eager to see something like that happen. There’s such a demand for reliable transcriptions for blind musicians. The inputs could be basic music scores, and yes, figuring out how to format the outputs correctly will be crucial!