How difficult is real-time single-note detection for piano and wind instruments?

FFT is another useful approach: capture the audio, apply a window function, inspect the spectrum, and map the detected frequency to the nearest note. It’s helpful for visualizing what is happening, but simply choosing the strongest frequency bin can fail when harmonics are louder than the fundamental. Also decide whether the app only validates pitch or whether it must judge timing and note duration too. For an education tool, clear feedback about being sharp or flat may be more useful than demanding an exact frequency immediately.

Mobile microphone processing is likely to be one of the biggest practical problems. Browser input commonly enables echo cancellation, noise suppression, and automatic gain control, which can distort an instrument’s harmonics. Try disabling those features for this audio stream when possible. You’ll also need to handle latency, background noise, and unstable readings by requiring a confidence threshold and smoothing results across several consecutive frames. A frame around 2,048 samples at 44.1 kHz is a reasonable starting point, though it introduces roughly 45 ms of analysis latency.

Piano will probably be more difficult than flute or trumpet. Wind instruments tend to have a clearer harmonic structure, while piano strings are inharmonic, decay quickly, and can sometimes have a weaker fundamental than the second harmonic. That can lead to octave mistakes. I’d begin with YIN or a similar time-domain detector in an AudioWorklet, then add octave-error checks and a short median or majority filter. More advanced models can be more robust in noisy phone recordings, but they will cost more processing power.