For a Spring application consuming structured output from a model, is provider-side formatting enough, or should the client validate and retry when a response is unusable?
How do you handle self-correcting structured output in Spring?
Asked by
retrofit_routine
Answers
Validate on the application side even if the provider offers structured output. Parse the response into your expected type, then check any constraints that matter before passing it to the rest of the application.
A response can be parseable and still contain values your application can't use. Handwritten validation is useful there because it gives you a predictable boundary between generated content and your own code.
Reply to this answer
Retries aren't free, though. Sending a bad response back for correction means another network round trip and potentially more tokens. I'd put a limit on attempts and have a clear failure path rather than retrying indefinitely.
Reply to this answer
Provider-side formatting and client-side checks solve different problems. Formatting can help get the response into the requested shape, but it doesn't necessarily mean the content is correct or usable. I wouldn't treat a valid format as proof that the answer itself is right.
Reply to this answer
Add your answer
Answers are reviewed before publication.
hingeless__handbook_:
It's also worth checking the provider's billing terms for failed requests and rate-limit errors rather than assuming every failure is free.