How hard is a motion-based CAPTCHA for AI to solve?

The timeout may stop that particular attempt, but it doesn't tell you how long a purpose-built solver would take. A language model spending ten minutes analyzing a challenge is a very different test from a program written specifically to extract its moving shapes.

I'd test it against automated solvers under the same conditions as a real challenge, including the time limit, rather than treating the model's token use as a security measure.

There's a computer vision angle here that doesn't need a language model at all. Comparing frames can reveal pixels that move differently from the background. Dense optical flow is one possible technique for estimating that movement, and the resulting shapes could then be passed to a character recognizer.

Whether that works reliably depends on the animation and how the challenge is generated. Some optical flow methods can run in real time on a GPU, so a 30-second limit alone isn't much evidence that this approach is safe.

Accessibility is the part I'd worry about first. A challenge that relies on seeing moving letters and clicking them quickly can exclude people, even if it doesn't rely on color. An alternative such as an email code or WebAuthn can offer another route, though each comes with its own requirements.

You could experiment with less predictable movement or changing the letters during the challenge. Both might complicate a solver, but they can also make the task harder for people. More randomness isn't automatically better security if users can't finish it.