I'm designing a local CLI harness for agent-assisted development with OpenCode across distributed repositories, Java services, Kafka contracts, and shared Ansible deployment code. The goal is not an autonomous agent that judges its own work, but a deterministic governor that forces every change through hard, locally verifiable boundaries.
We have strict API limits, so prompts and feedback must be small and dense. The team is skeptical of AI-generated code and will reject anything that disrupts local environments or produces unverified diffs. Developers retain ownership of their branches, run the harness locally, inspect atomic commits, and push manually. Every merge request still requires two human approvals.
The main risks are circular validation, where the model implements a mistaken assumption and writes tests that merely confirm it; loss of compiler and linter friction; and cross-repository contract breakage, such as a service change that compiles but breaks Kafka consumers or shared Ansible expectations.
The harness currently standardizes runtimes through mise, runs versioned Python scripts as binary 0/1 invariant checks, hashes test directories before and after edits, rejects changes outside an explicit path scope, prunes diagnostic output to conserve tokens, and stops after three failed attempts.
I'm particularly interested in whether binary results plus concise diagnostic snippets are enough for a model to correct itself, how to perform lightweight local compatibility checks for downstream consumers without launching full integration builds, and whether presenting the workflow as deterministic safety and anti-cheat protection helps skeptical teams trust it. I'd also appreciate feedback on blind spots, especially around parsing compiler failures and enforcing contract boundaries.
1 Answer
The overall direction makes sense, especially treating the harness as a governor rather than an autonomous reviewer. Binary oracles can work, but an exit code alone usually gives the model too little information. Keep the verdict binary while emitting a very small structured diagnostic: check ID, file and line, violated invariant, and one relevant compiler or schema message. That preserves deterministic evaluation without spending tokens on an entire stack trace.
For Java failures, prefer compiler diagnostics or structured reports when available instead of brittle regex parsing of raw console output. For Kafka and Ansible, offline schema and variable-boundary checks can catch many compatibility problems without running full integration suites. Version those checks alongside the contracts, and make the harness fail closed when a required contract cannot be evaluated.
The test hashing and path restrictions are useful, but they should be paired with ordinary human review rather than treated as proof of correctness. The strongest adoption message is probably reproducibility and containment: the tool cannot silently alter tests, escape its declared scope, or push changes, and every failed check leaves a concise, inspectable reason.
That matches my concern about raw output. Regex parsing across nested Java exceptions is fragile, so I’m leaning toward compiler diagnostics and structured reports wherever possible, with a small fallback parser only for tools that provide no machine-readable output. The goal is to expose one actionable failure without turning the harness into another large dependency.