technical essay

Agentic Software Development Needs a Harness

Reliable agentic development needs deterministic workflow control around probabilistic reasoning, with explicit authority, durable evidence, and safe recovery.

ai agents software-development orchestration governance

The language around AI and software development is still too vague for the systems we are building.

"Generative AI software development" describes where some output came from. A model generated code, a test, a plan, or an explanation. That label says almost nothing about how the work proceeded, who made which decision, what tools were used, or why anyone should trust the result.

"AI-assisted development" describes a familiar interaction: a person operates the process and consults a model along the way. The assistant may propose code or execute a bounded task, but the human remains the continuous control loop.

"Agentic software development" is the more accurate term when a model can pursue a multi-step objective, inspect a repository, use tools, modify files, run commands, delegate work, evaluate results, and decide what to do next within some granted authority. The important change is not better text generation, it is action(s) over time.

Once the model can act, prompt quality stops being the main systems problem. The problem becomes control.

From generator to actor

A code generator can produce a bad patch. A tool-using agent can produce a bad patch, run the wrong validation, misread the output, retry after a partial mutation, skip a review gate, and report completion with a persuasive summary. These are not merely generation errors, they are workflow failures. The distinction matters because fluent output can hide a broken process. A final response can sound coherent while the repository contains unrelated changes. A test claim can be technically true while covering the wrong behavior. A security review can report no findings after inspecting only the happy path. A coordinator can remember a required gate until context compaction removes the relevant instruction.

More capable reasoning does not remove these failure modes. Capability increases the useful work an agent can do, but it also usually increases the amount of state and authority exposed to probabilistic decisions. My current conclusion is simple: agentic software development needs a harness. I do not mean another system prompt with a longer checklist, I mean a deterministic orchestration layer that owns workflow mechanism while agents retain bounded judgment.

What the harness should own

The harness should know and enforce the legal workflow states and transitions. It should persist them outside model memory. It should know which role owns the current stage, which artifacts that role may consume, which outcomes it may return, and what evidence must exist before the workflow advances. At minimum, that means owning:

  • workflow routing and stage ordering
  • durable run, stage, and attempt state
  • dispatch and result contracts
  • workspace and mutation locks
  • bounded retries and remediation loops
  • interruption and restart recovery
  • evidence, finding, and risk records
  • permission narrowing and approval boundaries
  • reconciliation between claimed changes and repository state
  • terminal outcomes such as complete, blocked, needs replanning, or needs a human decision

These are mechanisms, not product judgment. A state machine can enforce that security review happens after implementation and before QA. It cannot decide whether a product requirement is correct on behalf of a person. A schema can require a risk record, but it cannot accept that risk on behalf of a person. This separation is the central design constraint: the harness owns mechanism, and roles (human or agent) own judgment.

Deterministic control around probabilistic reasoning

Agents are useful precisely because many software tasks do not reduce to a fixed algorithm. Understanding an unfamiliar codebase, forming a debugging hypothesis, evaluating an architectural tradeoff, and deciding which evidence would disprove a claim all require judgment under uncertainty. Trying to make that reasoning deterministic would destroy much of its value. Treating the surrounding workflow as equally probabilistic creates a different problem: nobody can reliably answer what happened, what remains, or what the system will do after failure; the boundary should be explicit.

Inside a stage, an authorized role can investigate, reason, and choose among methods. Around that stage, deterministic code should enforce prerequisites, authority, allowed outcomes, evidence requirements, retry limits, and legal next states. For example, an implementation agent may choose how to repair a bounded defect. It should not choose to waive the proof requirement, broaden its own scope, accept a security finding, or skip independent review, as those are different decisions owned by different contracts.

This resembles a transaction coordinator more than an autonomous developer persona. The coordinator does not need to understand every semantic detail, it needs to preserve invariants while specialized participants do work that cannot be reduced to a transaction script.

Roles need authority boundaries, not personalities

Agent roles are often described as personalities: "You are a senior security engineer" or "You are a meticulous reviewer." That can affect style and attention, but it is not an authority model. A useful role contract answers harder questions:

  • May this role mutate the repository?
  • Which files or slices does it own?
  • Which decisions may it make?
  • Which outcomes may it return?
  • What evidence must accompany those outcomes?
  • Which findings send work backward?
  • Which unresolved questions require a human?

A planner should be able to produce a decision-complete implementation brief without quietly implementing it. An implementation owner should be able to mutate its assigned slice without redefining product scope. Security, QA, and review roles should inspect independently without repairing their own findings. Remediation should return through the authorized implementation path. That separation adds friction, and that friction is intentional. A reviewer that fixes its own finding destroys independence, separation-of-duties. A coordinator that can silently perform every role makes stage names decorative rather than meaningful.

One mutating owner at a time is a particularly useful baseline. Concurrent agents sharing a workspace can interfere through files, generated output, locks, uncommitted changes, or planning artifacts even when their nominal source files differ. Read-only concurrency can be safe when the workflow explicitly proves independence. Mutation should begin conservative.

Skills supply methods, never authority

Reusable skills are valuable. They can encode how to build a FastAPI service, review an Alembic migration, inspect logging, operate a browser, or test an ORM contract. They reduce repeated instruction and preserve technical method. They should not grant authority. Loading a security skill must not turn an executor into a security decision-maker. Loading a database skill must not authorize a schema change. Loading a deployment skill must not authorize deployment. Authority should come from the workflow, role, approved brief, and explicit caller constraints. Skills remain subordinate inputs selected because the task needs their method.

This prevents a subtle form of policy drift: skill catalogs grow over time. If capability implies permission, installing new guidance silently expands what agents may do. A harness should instead validate that a selected skill exists, matches the task and stack, and stays inside the role's existing authority.

Contracts make handoffs inspectable

Natural-language transcripts are useful evidence sources but they are poor canonical state. A durable workflow needs typed handoffs. An attempt should receive an immutable envelope containing the objective, role, allowed scope, prohibited actions, input artifacts, prior findings, retry number, required output sections, and allowed outcomes. Its result should identify requirement closure, findings, evidence, changed files, inspected files, commands, coverage gaps, residual risks, and any knowledge worth preserving. The exact serialization format matters less than the invariants:

  • Every assigned requirement closes exactly once.
  • Every claimed command has captured exit status and output metadata.
  • Reported changed files reconcile with the actual repository diff.
  • A read-only role reporting mutations triggers an infrastructure fault unless the changes were demonstrably pre-existing.
  • Unknown or contradictory outcomes stop the workflow.
  • Missing evidence stays missing; it does not become a pass because the prose sounds confident.

Failing closed here will sometimes make the system feel less smooth. That is preferable to treating malformed control output as an invitation for another model to infer what probably happened.

Evidence outranks fluency

Agentic workflows produce an unusual amount of confident narration. The narration can help a human understand the work, but it is not proof that what was written by the model is truth. Evidence should be stored as data: command, working directory, exit code, timing, relevant output artifact, changed path, inspected symbol, rendered state, finding, coverage gap, and residual risk. Claims should point to that evidence. Different claims need different proof. A file's presence does not prove behavior. A type declaration does not prove runtime compatibility. A rendered component does not prove every required viewport or interaction. A security claim needs an identified trust boundary, prerequisites, and plausible attack path. An unavailable environment means untested, not passed.

This does not eliminate judgment, it helps make judgment inspectable. A reviewer can disagree with the interpretation while seeing the underlying artifact. A later run can distinguish a new regression from a previously accepted risk. A human can tell whether "tests passed" means a complete validation floor or one convenient command. Evidence also constrains final reporting. The system should be unable to claim completion when required proof artifacts are absent, even if every participating agent agrees that the change looks correct.

Security, QA, and review belong in the pipeline

Security, QA, and independent review are often appended to a prompt as behavioral reminders: consider security, run tests, review your work. That is better than ignoring them, but it is not equivalent to independent gates.

A role asked to implement and validate its own work shares context, assumptions, and incentives with the implementation. It tends to verify the solution it intended to build. Independent stages receive the resulting repository state and an explicit review contract instead. The proposed normal pipeline I am documenting is proof, implementation, security, QA, independent review, then knowledge capture. Security can return a confirmed in-scope flaw to the implementation owner. QA can return an actionable defect. Review can do the same. Material scope or architecture invalidation should stop for replanning rather than encouraging an executor to improvise a new contract.

Ordering matters. Security should inspect the final implementation state, not an intermediate patch. QA should not begin while known security remediation is still changing the code. Independent review should receive the QA evidence rather than reconstructing the entire run from a transcript. Risk-scaled gates are still possible. A one-line documentation correction does not need the same proof as authentication code. The harness can select a workflow based on declared risk and policy. What it should not do is let an agent omit a required stage because the agent judges its own change to be harmless.

Recovery is part of correctness

Long-running agent work will be interrupted! Processes crash, API calls time out, context compacts, sessions become unreachable, users stop runs. Tools can mutate a repository before the final model response arrives. The last case is the dangerous one.

Suppose an implementation attempt edits three files and runs a migration, but the session fails before returning its result envelope. From the orchestrator's perspective, the response is missing. From the repository's perspective, the operation happened. Automatically replaying the same prompt may duplicate a migration, overwrite a valid partial repair, or apply a second incompatible interpretation. Mutation without a valid final response must therefore receive forensic handling, never automatic replay.

Recovery should compare durable pre-attempt state, session events, messages, tool evidence, Git status, and the live repository. It should classify the attempt: completed, stalled without mutation, infrastructure failure with mutation, interrupted, or unknown. An empty result with no mutation might justify one narrowed retry. An empty result with mutation should route through read-only forensic inspection before any further write. Idempotency keys help identify an attempt, but they do not make arbitrary software mutations idempotent. That distinction is easy to lose when borrowing vocabulary from job queues and distributed systems.

Repository reality must outrank cached workflow state. A journal can say an attempt never completed. The working tree can prove that it changed. Safe resume reconciles both before dispatching anything else.

Humans still own the consequential decisions

An effective harness should reduce repeated supervision without laundering human decisions through agent prose. Humans retain product intent, architecture, ownership, authority changes, security posture, and risk acceptance. Agents can gather facts, identify tradeoffs, propose options, and execute approved work. They should stop when a decision crosses those boundaries. That stop needs structure. "Needs user input" is too broad. A decision request should state the unresolved question, why it blocks progress, available options, consequences, requesting role, and blocked stage. Repository facts should be investigated before escalation. Genuine product or authority choices should not be disguised as technical inevitabilities.

The goal is not a human approval prompt between every stage. Once the contracts are ready, the harness should advance through legal transitions without asking "continue?" repeatedly. Human attention should be reserved for decisions only a human is authorized to make.

What exists, and what can be demonstrated publicly

The harness described here is not hypothetical. Its design is informed by two systems I have built: a personal implementation around OpenCode V2, which I continue to develop and test, and a separate professional implementation built with the Claude Agent SDK for my employer. Neither implementation is available for public inspection. I am deliberately keeping the personal implementation private, and the professional implementation is proprietary. I therefore cannot provide source code, implementation artifacts, or reproducible public evidence for either system.

This essay should be read with that limitation in mind. The architecture, authority boundaries, recovery policies, and workflow constraints described here are derived from implementation experience, but they are generalized rather than complete descriptions of either system. Some details have necessarily been omitted.

What does not exist is a public reference implementation accompanying this essay. The absence of public proof limits independent verification; it does not mean the systems themselves are conceptual.

What a credible implementation must prove

The existence of a harness does not establish that its control model works. A credible implementation should demonstrate that its invariants survive malformed model output, partial mutation, interruption, retry, restart, and recovery, not merely that it can complete a happy-path workflow. The smallest useful validation surface can remain narrow: one workspace, one normal implementation workflow, durable state, explicit locks, bounded retries, and a pinned runtime adapter. That is enough to expose most of the difficult control problems without importing unnecessary machinery.

Deterministic tests should use a controlled runtime capable of forcing pathological states: malformed outputs, repeated findings, permission denial, interruption before mutation, interruption after mutation, profile drift, and recovery after process restart. At least one pinned integration with a real runtime is also necessary because a perfect test double can preserve assumptions that the actual runtime violates.

The acceptance bar should include:

  • restart-safe completion through the normal pipeline
  • rejection of illegal transitions and malformed role output
  • forensic handling when mutation exists without a final result
  • security remediation before QA
  • enforcement of read-only review branches
  • exclusive workspace and planning-file ownership
  • correlated evidence and logs without secret leakage

These criteria are informed by my private implementations, but their validation artifacts cannot be published or independently inspected. This section therefore states the standard against which I believe such systems should be judged; it is not public proof that either private implementation satisfies every criterion.

Passing these tests would support claims about one implementation under the tested conditions. It would not prove the broader thesis. Real use still has to reveal whether the contracts are too rigid, the evidence burden is proportionate, and the role boundaries survive contact with ambiguous work. That uncertainty is healthy; a harness should be treated like any other control system: state its invariants, expose its behavior, test its failure paths, and revise it when evidence contradicts the design.

Agentic software development is not mainly about generating more code. It is about allowing probabilistic systems to take consequential actions inside a software process. That can work, but only if the process has a stronger memory, clearer authority, and less tolerance for unsupported claims than the agents operating within it. The model can reason. The harness must remember what the model is allowed to do, what it actually did, and what must happen next.