Architecture
The writer cannot be the only judge.
The defendant on the bench
The writer cannot be the only judge
Suppose an AI agent writes a feature, rereads its own diff and reports that the work is complete. That report may be right. The problem is that the review begins with the same instructions, assumptions and interpretation that produced the code. A mistaken reading can survive both passes because nothing new has challenged it.
That is why I separate creator from verifier. The creator produces the artifact. The verifier decides whether it meets an acceptance contract, a fixed list of observable conditions written before the work began. Separation does not require pretending that either model has pride, sunk cost or a desire to pass. It simply gives the second pass a better chance of using different evidence instead of repeating the first pass.
Creation and verification need different evidence paths
A second pass helps only when it can challenge the assumptions that produced the first result.
Writer as sole judge
- Shared framing The review begins from the same interpretation.
- Familiar omissions Missing cases can remain invisible twice.
- Confidence reinforcement Polish can be mistaken for correctness.
separate the context
Independent verifier
- Fixed acceptance contract Success is defined before seeing the result.
- Independent evidence Sources, tests and user boundaries can contradict the writer.
- Authority to reject The verifier can block or escalate a consequential failure.
Independence is a property of evidence and authority, not merely a second model call.
Why self-review is not independence
It can re-derive instead of checking
Language models generate a plausible continuation from the context they receive. During implementation, that context fills with the chosen design, intermediate explanations and reasons the approach should work. If the review happens in the same conversation, those choices remain prominent. The model can re-explain the design coherently without testing the disputed behaviour. As I argue in the change looking right is not proof that it works.
Tests written after implementation can still be valuable, but they are easier to fit to the behaviour that already exists. The stronger move is to state observable acceptance conditions first. A green test then means something specific: under these inputs and conditions, the observed output matched the contract. It does not prove the system is correct everywhere, but it is firmer evidence than a second explanation.
Blind, and ideally a stranger
A bug should face two different nets
Useful independence has four parts. Give the verifier a separate context without the creator's reasoning, a separate role whose job is to find unmet conditions, the same acceptance contract the creator received, and access to deterministic evidence such as tests, type checks, database state or browser output. These controls matter more than merely changing the model name.
A different model or provider can add diversity, but it is not automatically independent. Two models can accept the same vague requirement, trust the same faulty fixture or overlook the same domain rule. Conversely, two isolated runs of one capable model can disagree when the second receives a tighter contract and better evidence. The two-net picture is therefore a useful aspiration, not a guarantee: independence is designed through information and authority, not purchased with a model switch.
For consequential work, authority must be separate too. The creator should not be able to mark its own gate complete, rewrite a failing criterion or discard inconvenient output. The verifier reports against the contract, and a deterministic gate decides whether the workflow advances. That turns separation from a polite request into an operating control.
Catch different failures with different nets
No single evaluation surface observes every way a result can be wrong.
- Specification check Did the result address the approved outcome and constraints?
- Executable check Do focused and adjacent tests behave as expected?
- Evidence check Do citations and source states support the claims?
- User-boundary check Does the real workflow work under realistic permissions and data?
- Human judgement Are consequences, trade-offs and residual risks acceptable?
Layered verification reduces blind spots without pretending proof is absolute.
Validation is not the cheap part
Verification is a heavyweight
Consider a creator asked to add subscription cancellation. Before it writes code, the contract says that an authorised customer can cancel an active subscription, a repeated request changes nothing, an unknown subscription returns the agreed error, and exactly one audit event is recorded. The creator receives that contract and implements the endpoint.
The verifier receives the contract, the resulting code and a clean test environment, but not the implementation conversation. It exercises all four cases, inspects the response, subscription record and audit log, then reports each condition as pass or fail with evidence. A test runner can block release if any required case fails. The verifier may suggest more cases, but it cannot silently redefine success.
Suppose the repeated-cancellation case creates a second audit event. The isolated verifier detects two records where the contract permits one and attaches the database evidence to a failed verdict. The release gate blocks the change. The creator then makes the operation idempotent, meaning the same request can be repeated without changing the result, and the verifier reruns all four cases before the gate can pass.
This is real work, not a rubber stamp. Its share varies with risk, system maturity and the quality of existing tests, so there is no universal orchestration-to-validation percentage. The preserved chart is too precise for the evidence behind it and should be read only as a reminder to budget verification as part of delivery, not as a benchmark.
Spend validation by consequence
There is no universal percentage of effort that belongs to review. Allocate it where error is expensive or hard to reverse.
More independent validation
- High consequence Security, finance, identity, safety or customer rights.
- Low reversibility Failure is difficult to undo or discover later.
- Weak observability The system can fail quietly.
calibrate, do not fix a ratio
Lighter validation
- Low consequence A small, reversible internal change.
- Strong deterministic checks Failure is quickly and reliably visible.
- Short feedback loop The result can be corrected before harm spreads.
Validation is sufficient when evidence matches the claim and remaining risk is owned.
The discipline lives in the wiring
Build the split into the rig
The practical pattern is small. Define observable success before the build. Run creation and verification in separate contexts. Give the verifier evidence-producing tools. Prevent the creator from changing the gate. Then record the evidence beside the verdict so a person can inspect why the work advanced. This is one useful application of harness engineering: the workflow, permissions and checks shape reliability around the model.
The boundary matters. Separate agents can share a blind spot in the requirement, test data, training or evaluator. Deterministic tests can also prove the wrong thing with great consistency. High-impact work still needs domain review, adversarial cases, monitoring after release and a path to reverse the change. Separation reduces correlated reasoning; it does not manufacture truth.
As a professional, I want evidence strong enough to support the decision being made. As a student, I also want the verifier to show me which assumption failed. That is where the split earns its keep. Output is cheap; accountable judgement remains the scarce seat. Let the creator propose. Let a separately governed process decide whether the proposal is ready.