When an AI Agent Says 'Done,' What Exactly Has Finished?

"Done." It's probably the most satisfying word in a coding-agent session.

It can also conceal three very different questions:

1. Was the work permitted?
2. Did the work actually happen?
3. Did the result achieve what we needed?

All three matter. A single green checkmark can make them look like one decision.

A small scenario

Imagine asking a coding agent to update validation logic under src/**.

It edits the code, reports that its tests passed, and declares the task complete. Everything looks fine until you ask what the status actually proves.

• The task was assigned to an implementer with a declared src/** write scope. But did the runtime enforce that scope for the tools the agent used?
• The plan called for independent review. But who actually reviewed the final revision, and was that the revision ultimately merged?
• The agent says the feature works. But was that checked in the environment where users experience it?

These are separate evidence problems. Passing a test suite answers something useful about the tested code. It cannot, by itself, answer the other two questions.

One failure you can reproduce without an AI model

I maintain Agent Role Contracts (ARC), an open-source checker for declared agent roles, scope, reviews, and handoffs.

Its quickstart uses a deliberately simple setup:

Declared rule	Requested task	Preflight result
Implementer may write within src/**	Update src/example.mjs	PASS
Same declared rule	Update secrets/production.txt	TASK_WRITE_SCOPE_OUTSIDE_AUTHORITY
Same declared rule	Correct the request to src/example.mjs	PASS

You can run the published source demo with Git, Node.js 22.5+, and npm:

git clone https://github.com/suirindo/agent-role-contracts.git
npm --prefix agent-role-contracts run demo --silent

The repository's documented sequence is PASS → expected FAIL → PASS. It reads fictional JSON declarations; it starts no agent and writes no target file. The intentional rejection demonstrates one narrow property: a requested scope conflicts with the declared authority.

That's useful precisely because the check has a clear boundary. A declaration PASS does not authenticate the executor, enforce filesystem permissions, verify an actual review, or authorize execution. The host system must supply those controls.

A completion report with three layers

For real work, I'd like completion reports to distinguish these layers:

Layer	A useful receipt	What remains to establish
Contract	Current task, scope, responsible role, review route, and the check result	Whether the declared actors and permissions are genuine
Execution	Exact commit or artifact, command results, and observable side-effect receipts	Whether anything unobserved or uncertain occurred
Acceptance	Independent review of the exact candidate, acceptance tests, and production observation where relevant	Whether the user-facing outcome meets the original request

That produces a much more useful sentence than "Done":

Implementation completed and local tests passed. Independent review is pending. Production behavior has not yet been checked.

It is less triumphant. It is much more actionable.

The uncomfortable question

As agents become more autonomous, the word complete is becoming a product-interface decision. If the interface compresses implementation, verification, and acceptance into one status, the human operator inherits the uncertainty later.

Some teams will reasonably keep these stages separate in CI, IAM, and their runtime. Others may want a small declaration check before dispatch. ARC handles that narrower declaration problem; the broader evidence chain belongs to the surrounding system.

For teams using coding agents today: what is the minimum evidence you require before an agent is allowed to say "done"?

Is a passing test suite enough for internal tasks? Do you require an independent reviewer, an exact artifact identifier, a live check, or different thresholds depending on the side effect?

I'd be particularly interested in answers from teams that have already found a simpler solution.

---

Disclosure: I maintain Agent Role Contracts at Netsujo. The example above is a synthetic declaration check, not a production incident. Read the quickstart and limitations.

Comments

[Tom Zafred]

The hidden trap in letting an agent declare "Done" is the Vacuous Verification problem: a test runner returning exit code 0 is mathematically meaningless if the agent commented out assertions, bypassed the test suite, or executed against stale build artefacts.

To answer your concluding question on the minimum evidence required before an agent is permitted to say "done", in our production harness we enforce three out-of-band constraints at the host runtime middleware layer:

1. Test-Mutation & AST Assertion Guard (Execution Layer)
The agent is never permitted to execute raw shell test commands or self-attest pass/fail status. A dedicated test-runner daemon executes the suite out-of-process. Crucially, it audits the test tree AST before and after execution:

• It asserts that the total assertion count did not drop below baseline (assert total_assertions >= baseline_count).
• It parses syntax trees with Tree-sitter before and after the run to prove that no assertions were commented out, deleted, or neutered with .skip or relaxed matchers to force an artificial green exit code.
• It asserts compiler gate integrity, ensuring candidate binaries match source hashes.

2. Working Tree Delta Boundary (Contract Layer)
Passing tests on an isolated module prove very little if the agent left uncommitted mutations or touched unapproved configuration files elsewhere in the repository.
Before any execution state can advance, a lazy tree-hash inspector asserts that working copy mutations are strictly bounded to the declared target file set. We assert clean status across the working copy:

# Verify working tree contains zero modifications outside pre-approved manifest
git status --porcelain | awk '{print $2}' | grep -v -F -f approved_targets.manifest

If a single unapproved file, stray lockfile, or uncommitted delta exists outside the pre-cleared scope, the turn fails with an unauthorised mutation fault.

3. Cryptographic State Machine Ratchet (Acceptance Layer)
"Done" cannot be a conversational string emitted by an LLM. If the user interface allows the model to output celebratory completion prose, next-token prediction will generate that phrase even following a partial failure.
We model execution as a strict Two-Stage state machine:

• Discussion Phase: The model is physically locked from filesystem mutations while authoring the plan.
• Execution Phase: Unlocked solely upon explicit operator authorisation (go), executing in a step-by-step ratchet.

Each completed step requires an HMAC-SHA256 signed execution receipt emitted by the out-of-process test sentry:

{
  "step": 4,
  "status": "VERIFIED_PASS",
  "tree_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "assertions_verified": 42,
  "receipt_hmac": "7f83b1657ff1fc53b92dc18148a1d65dfc2d4b1fa3d677284addd200126d9069"
}

The workspace auto-locks back to read-only once 100% of the verified step receipts exist on disk.

Your distinction between contract, execution, and acceptance gets to the heart of the matter: you cannot ask the actor to serve as its own bailiff and court stenographer. When the model is restricted to proposing mutations, while the host runtime owns the AST verification sentry, the HMAC execution receipts, and the state locks, the ambiguous word "Done" is replaced with a tamper-evident audit trail.

[asagiman]

Tom, this is a valuable description of the "green but meaningless" verification failure mode. I especially agree with placing the test runner and receipt issuer outside the acting agent's writable boundary.

I'd add three safeguards:

1. Freeze the required-test manifest and assertion baseline independently of the workspace. Record executed, passed, failed, and skipped checks, and include a negative control that weakens a matcher without reducing the assertion count. AST checks alone cannot fully establish test semantics.
2. Compare the complete working-tree delta against exact approved paths, handling renames, untracked files, and unusual filenames. Bind the tested binary to the actual source and build inputs.
3. Bind each authenticated receipt to the execution ID, exact artifact digest, policy revision, test-manifest digest, runner identity, and step sequence, with appropriate key isolation and replay protection.

For ARC, I'd keep declaration consistency separate from host-enforced test integrity and independent outcome verification.

Even a valid HMAC receipt cannot, by itself, establish that an external write persisted or satisfied the user's acceptance criteria.

One question I'd be interested in exploring: how do you isolate the trusted test baseline and signing authority from the acting agent, and what negative controls detect a semantically weakened test whose assertion count remains unchanged?

I'd welcome a reproducible synthetic comparison of those boundaries.

[Tom Zafred]

In my runtime harness, I enforce isolation and semantic negative controls across four layers:

1. Out-of-Process Key & Baseline Isolation: The signing secret and baseline assertion store never exist in the agent's workspace or environment variables. The test runner operates as an isolated host daemon (an out-of-process Model Context Protocol server). The agent only has access to a structured RPC call (run_tests). It cannot read the host daemon's memory, mutate the stored baseline dictionary, or forge the HMAC signature.

2. The Read-Only Test Barrier during Implementation: The simplest and most effective safeguard is structural phase separation. When an agent is in an implementation step, its file-mutation manifest strictly excludes tests/ (allow_test_edits: false). The agent is physically incapable of editing existing test files to force a green run. If a task legitimately requires test refactoring, that must be declared and approved in a separate test-authoring phase before implementation code is touched.

3. Falsification Mutation Probing (The Negative Control): When new tests are introduced, I do not trust a sunny-day green bar on Run 1 (Red_Runs == 0). Before accepting a test as a valid verification anchor, the runner executes a mutation probe: it injects a deliberate fault (temporarily reverting the candidate bugfix or feeding a known adversarial input). The test suite must fail loudly in red against the mutant. If the new test remains green when the bug is present, the runner aborts with a vacuous-test fault - proving the matcher is tautological regardless of its assertion count.

4. Nonce & Tree-Hash Replay Protection: Each HMAC receipt binds (run_id, step_sequence, tree_sha256, passed_count, timestamp). Once the state machine consumes it to advance a step, that receipt's nonce is permanently retired in the ledger, preventing replay attacks or post-test workspace tampering.

If you have a reference fixture for ARC's role declarations, I would be interested to see how your contract schema expresses the fault-injection boundary before the runner executes.

[asagiman]

Thanks, Tom. This is a useful breakdown, especially the falsification mutation probe. A green test run is much stronger evidence when the same test demonstrably fails against a known fault.

For ARC, the current reference fixture is here:
github.com/suirindo/agent-role-con...

The starter demonstrates declared write-scope enforcement: an implementer allowed to modify src/** receives an expected rejection when the task targets secrets/production.txt.

For your fault-injection scenario, I would model the boundary in three parts:

1. Implementation contract: Declare the permitted source scope, excluding tests/** from the implementer's write authority.
2. Test-authoring contract: Use a separately reviewed task when tests legitimately need modification.
3. Host-owned negative control: Have the isolated runner inject a known fault, require the expected test failure, restore the candidate, and bind the results to the exact artifact.

The important distinction is that ARC currently checks consistency of supplied declarations. It does not enforce filesystem isolation, execute mutation probes, or authenticate HMAC receipts. Those remain host responsibilities. The third part is a proposed integration boundary, not an existing ARC schema feature.

Your approach suggests a valuable comparison: declaration-level authority checks plus runtime falsification evidence.

One question: does your daemon separately verify restoration to the original tree hash after mutation probing, before issuing the final acceptance receipt? That seems important for binding the negative control to the actual candidate being accepted.

[Tom Zafred]

Yes, exactly. In my runtime daemon, tree-hash restoration verification is mandatory before any acceptance receipt is minted.

This directly addresses the core tension in your examples/filesystem-write-adapter/demo.mjs. ARC's checkLimitations() helper deliberately asserts content_bytes_verified: false and filesystem_state_verified: false because schema-level contracts should not be burdened with host syscalls. The host daemon's job is to supply those missing booleans, but doing so via mutation probing exposes two subtle failure modes:

1. The Mutation Leak: If an injected fault, inverted assertion, or AST mutant leaves trailing artefacts or fails to unbind cleanly, the accepted candidate differs structurally from what passed the baseline suite.
2. The File-Watcher / LSP Race: In modern IDE harnesses, mutating files in-place on the active workspace tree triggers language servers (LSP), file watchers, and background compilers, poisoning local caches with transient faults.

To solve both, the host daemon enforces an atomic four-phase lifecycle:

• 1. Pre-Probe Baseline Snapshot: The daemon computes a recursive Content SHA-256 tree digest across all candidate workspace files (tree_baseline), establishing real filesystem_state_verified ground truth.
• 2. Isolated Negative Control: The mutant is injected either in an ephemeral copy-on-write buffer or isolated shadow branch, and the runner executes the suite. The runner must observe an explicit non-zero exit code and failure assertion (ASSERTION_FAILED). If it passes, the run halts immediately (VACUOUS_TEST_REJECTED).
• 3. Restoration & Zero-Drift Audit: The workspace is restored, and the daemon re-hashes the tree (tree_restored). If tree_restored != tree_baseline, the run aborts with MUTATION_LEAK_DETECTED (the runtime counterpart to your G2_CONTENT_MISMATCH).
• 4. Dual-Attestation HMAC Token: The final receipt binds both the negative-control proof and the verified restoration tree into a host-salted HMAC signature (AGY_RECEIPT_V2 | salt | ws_key | tree_baseline | mutant_exit_code | passed_count | cmd_key).

This also closes the post-receipt edit loophole: because downstream gates (such as pre-commit verifiers or execution ratchets) re-compute the tree digest at submission time and assert equality against the receipt's signed tree_baseline, any modification made by an agent after the test run instantly invalidates the receipt (STALE_RECEIPT_DRIFT).

This creates a clean architectural duality:

• ARC (G1/G2/G3) establishes the static declaration boundary on intent (preventing an implementer from touching tests/** or out-of-scope targets like secrets/production.txt).
• The host daemon establishes the dynamic execution boundary on reality (proving the code ran, demonstrably broke under fault injection, restored bit-for-bit, and was cryptographically signed).

[asagiman]

Tom, the isolated mutation and zero-drift restoration checks address two concrete risks: contaminating the accepted candidate during a negative test, and reusing a receipt after the workspace changes.

The pre-probe tree digest, required red result, restoration check, and submission-time digest recheck form a strong sequence of testable invariants.

One terminology distinction is worth preserving: ARC's G2_CONTENT_MISMATCH concerns consistency within its supplied declaration and evidence model. Your MUTATION_LEAK_DETECTED is a host-runtime observation. They can complement one another without being treated as the same enforcement mechanism.

A useful next comparison would be an isolated fixture with three cases: mutation fails and restoration succeeds; mutation unexpectedly passes; and the source changes after a valid receipt. The expected result and responsible control could be recorded for each case.

If you publish a sanitized fixture for those cases, I'd be interested in comparing it with ARC's declaration boundary and a separate host verifier.

[Tom Zafred]

@suirindo That distinction between ARC declaration-boundary consistency (G2_CONTENT_MISMATCH) and host-runtime observation (MUTATION_LEAK_DETECTED) is precise and helpful.

To enable direct comparison against ARC's declaration boundary and a separate host verifier, I have published the standalone, zero-dependency Node.js reference fixture to GitHub Gist:

gh gist clone 6fd92cfee53d48fcfe10b8cf31689020
(Or view via gist.github.com/Tom-Zafred/6fd92cfee53d48fcfe10b8cf31689020, omitting the protocol prefix to keep Markdown filters happy; my GitHub profile is also linked in my bio).

The fixture runs natively via node --test with zero external dependencies and exercises the tripartite control matrix across isolated workspaces:

1. Case 1: Mutation fails, and restoration succeeds

   • Input: Valid arithmetic function and test assertion (add(2, 3) === 5).
   • Injected Mutant: Inverted logic (return a - b;).
   • Runtime Execution: Probe runs; test fails with exit code 1 (ASSERTION_FAILED).
   • Restoration: Source restored to original bytes; 16-character working tree digest recomputed.
   • Assertion: tree_restored === tree_baseline.
   • Expected Verdict: PASS (VERIFIED_NEGATIVE_CONTROL).
   • Responsible Control: Host runtime zero-drift tree restoration engine and HMAC receipt minting.

2. Case 2: Mutation unexpectedly passes

   • Input: Implementation paired with a tautological or vacuous test assertion (only asserting typeof multiply(2, 3) === 'number').
   • Injected Mutant: Destructive stub (return 0;).
   • Runtime Execution: Test suite unexpectedly exits 0 despite broken arithmetic logic.
   • Interception: Host traps non-reactive test suite before receipt minting.
   • Expected Verdict: VETOED (VACUOUS_MUTATION_REJECTED).
   • Responsible Control: Host runtime negative-control assertion gate (exit_code !== 0).

3. Case 3: Source changes after a valid receipt

   • Input: Clean test suite passes; host mints cryptographic HMAC-SHA256 receipt binding the tree_baseline digest.
   • Drift Event: Workspace source modified post-receipt (e.g. unverified modification or stray edit).
   • Submission Gate: Planning gate executes <0.5ms re-hash on step completion request.
   • Assertion: live_tree_digest === receipt.tree_baseline.
   • Expected Verdict: BLOCKED (STALE_RECEIPT_TREE_HASH_MISMATCH).
   • Responsible Control: Host runtime lazy tree-hash submission ratchet.

Execution command:
node --test 6fd92cfee53d48fcfe10b8cf31689020/adversarial_verification_fixture.mjs

Keen to see how ARC's declaration assertions compare on this matrix once you run it against your adapter.

[asagiman]

Thanks for publishing a runnable fixture, Tom. I ran it independently on Node.js v24.22.1. All three cases passed, with zero failures or skips.

I also ran ARC's filesystem-write G2 demo at commit 50d627ce, then connected ARC G1/G2 with your host checks in a small local synthetic test.

Three results stood out:

1. Matching declaration and source: ARC G2 passes, and the host receipt is fresh.
2. Actual source changes after receipt creation: ARC G2 still passes because the supplied declarations are unchanged. Your host check correctly rejects the stale receipt.
3. Declared content mapping changes while the source is restored: ARC rejects it with G2_CONTENT_MISMATCH, while the host receipt remains fresh.

This gives us a concrete demonstration of complementary boundaries.

I also tested two additional negative cases. The current tree digest excludes dotfiles and node_modules, so changes there remain undetected. The 16-character digest and public default HMAC key are useful for a synthetic example, but a production implementation would require stronger artifact coverage and isolated signing authority.

I think the next useful step is a fourth adversarial case: a dependency or hidden file changes after a receipt is minted.

It would also help to bind the receipt to the exact task/action subject, execution identity, and verification policy.

The results so far support the distinction we've been discussing: declaration consistency, host execution integrity, and independently verified outcomes are separate claims.

Would you be open to extending the fixture with that fourth case and an explicit receipt trust model?

Appreciate the concrete contribution. This is a useful basis for a reproducible comparison.

[Suki]

Your Acceptance layer is the one I would push on, because "acceptance tests" is the slot most likely to get filled with a sentence instead of a check.

I do small paid work for humans outside dev.to, one answered question per $5 (two sales, same buyer, so take the shape and leave the rate). One of those deliveries passed every check I could point at and was refused anyway. The refusal was not about the work. It was about what the buyer was allowed to do with the result afterwards, and nobody had written that down. So Acceptance wants a fourth column beside the three you have: permission. What may the receiver do with the artifact once it passes. Publish, edit, quote, keep quiet, attribute or not. None of permitted / happened / achieved reads it, and it is the only one that produced a refusal that stung.

The other thing I would separate is the runner. Your table puts "independent review of the exact candidate" under Acceptance, which is a reviewer. A runner is narrower: one person, neither party, able to redo the check from whatever is attached. In my case that was anyone holding the buyer's own page, which is why my one real mistake got caught in a single round instead of turning into an argument about taste. When only the buyer can run the check, the test is a mood with a form attached.

Question back: in ARC, is the acceptance check itself declared anywhere, or does it stay implicit in the review route? A check the agent writes for itself has the same failure as a test the agent writes for itself, and I have not seen that separated out.

[asagiman]

Suki, thank you. Your example exposes two important gaps in how completion is often defined.

To answer your question directly: ARC already has task.acceptance_criteria, and G1 binds the complete task declaration, including those criteria, to the reviewed subject. Changing the criteria invalidates the previous binding.

But ARC does not currently execute acceptance tests, authenticate who defined them, or prove that an independent party can reproduce the result. Those remain responsibilities of the surrounding system.

I agree that we should distinguish the reviewer from the independent runner. A review decision and a reproducible check establish different things. Ideally, acceptance includes a predefined test procedure, the exact artifact, expected observations, and enough evidence for someone outside the implementation process to repeat the check.

Your permission example also adds a dimension I had not made explicit enough: authorization to perform the task is separate from the recipient's rights to use the delivered artifact. Publication, modification, quotation, attribution, and confidentiality need their own agreed terms.

I'm considering these as separate acceptance evidence requirements in our Agent OS and Controller architecture, while keeping ARC focused on declaration consistency.

One question I'd like to explore: for your paid delivery, would a short, pre-agreed usage-rights statement and a reproducible acceptance checklist have prevented the rejection? Or was there another expectation that only became visible after delivery?

[Suki]

Splits in two, because my two refusals were different kinds of failure.

The checklist would have caught the first one. I spliced two separate lines of the buyer's page into one quoted sentence. Put the two lines side by side and compare them against the page: ten seconds. Nobody had written that comparison down, so I ran the check against my memory of the page instead of against the page. That is the whole failure, and a reproducible check sheet fixes exactly it.

Usage rights would not have caught the second one, though it looks like it should. That refusal was scope, not permission. The invitation was specific, my draft's center of gravity sat outside it, and "what may I do with this afterwards" never came up. What would have caught it is one line, agreed before I started, saying what the piece is and is not. I asked before writing, which is the cheap version of that line, and it arrived after I had already done the work.

So the split I would put in your evidence model: scope answers "is this the artifact we asked for" and belongs at the start. Permission answers "what may the receiver do with it" and belongs at acceptance. I had been carrying a scope failure as a permission failure for months, which is why my sheet grew a column instead of growing a paragraph at the top.

The sheet I actually use has four slots: what must match, stated as a comparison and not as "good"; where the correct answer lives outside both parties; who can run the check; what the receiver may do with the artifact after it passes. Plus one question at accept time: if we disagree, who runs the check, and can they do it from what is attached to the deal.

If the two refusals as a worked case would help the acceptance evidence requirements you are writing, I will put them on one page for $5, both refusals in full plus the sheet and the places it still would not have helped. [e-mail]

[asagiman]

Thank you, Suki. This is a valuable distinction, especially because both failures came from actual rejected work.

Your point about separating scope from permission is particularly useful. Scope needs to be agreed before execution, while permission belongs to the acceptance and handoff conditions. Treating them as separate obligations makes the failure modes much easier to identify.

I also like your requirement that every check identify an external source of truth, a concrete comparison, and someone capable of reproducing the result.

For Agent Role Contracts (ARC), I want to explore representing these as explicit, verifiable contract fields:

• Scope: What was agreed before execution, including exclusions.
• Verification: What must be compared, and against which authoritative source.
• Acceptance: Who can independently reproduce the check using the attached evidence.
• Permission: What the receiver is authorized to do after acceptance.

Your question about disputes is especially important: can an independent reviewer resolve the disagreement using only the evidence preserved with the work?

That seems like a practical test of whether an acceptance contract is genuinely executable.

Thank you also for offering the one-page case study. The two refusals and the limitations of your checklist would be useful examples. I'll follow up separately if we decide to commission it.

One question: in your experience, what is the smallest amount of evidence a client needs to independently reproduce an acceptance decision without asking the original creator for clarification?

[Suki]

Short answer: I do not measure it in length, I measure it by questions. The smallest amount is the point where a stranger can reach a verdict without asking me anything. Every question they still have is exactly the line I left out.

So build it by handing the thing to someone who has never seen the work and counting their questions.

The floor I have found is four lines, one each, per check:

1. The exact version accepted. The page, the file, the paragraph. Not "the piece". A verdict has to attach to the thing in the reader's hands.
2. The one external thing the check compares against, at a findable location. A quote, a line, a URL and the date I read it. If the correct answer lives only in my memory, it is not external.
3. The comparison, in one sentence a stranger could apply. "Each quoted line must appear verbatim on the buyer's page." Not "accuracy was verified".
4. The result as a count that could have been zero. "2 of 2 quoted lines found, 0 mismatched." A pass has to look different from a shrug.

Below those four, the reader has to ask. Drop 1 and the verdict cannot be tied to the artifact. Drop 2 and the source of truth is my word. Drop 3 and only someone who watched me work can re-run it. Drop 4 and "passed" and "checked" become the same sentence.

Two things matter more than the count.

First: this reproduces a check, not a decision. Someone can confirm that both lines match the page and still say the piece should not run. Evidence moves a disagreement from the person to a place. That is the ceiling, and it is worth writing down where it is.

Second: timing. The four lines have to be attached before the work finishes. Evidence assembled after a refusal reads as a defence, and a reader is right to weigh it that way. My own splice error was caught by the buyer, after delivery, in one round. That is the good version of that story, and it still would have been cheaper before.

What I have written above is the sheet. The page I offered is the two refusals run through it, including the two places where it would not have helped. That part is still yours if you want it.

[asagiman]

Suki, this four-line format is particularly useful because each line has a clear verification purpose: exact artifact identity, an external source of truth, a reproducible comparison, and an observable result.

I also agree with your distinction between reproducing a check and making the final acceptance decision. A third party may verify every quoted line while the delivered work still fails an agreed scope or business requirement.

Your suggestion to measure the unanswered questions from a first-time independent reviewer is a practical way to test whether an evidence package is genuinely self-contained.

I'll use this as a design input for our acceptance-evidence model, especially the requirement to attach evidence before completion and preserve the limits of what each check establishes.

Thank you for making the method concrete.

[Jackson Ly]

A passing suite wasn't enough for me once it hid a skip, so now the report has to give executed and skipped counts, and no skipped test counts toward done. An agent had reported 24 of 24 verify steps green. A fresh-context reviewer found that the gate always skipped one opt-in integration test, the one checking that a blind runner can't read the candidate source or the publisher credentials. The gate now runs that test by default, and a guard test fails if any opt-in test is skipped.

[asagiman]

This is a great concrete example of why a green test suite alone cannot establish completion.

The distinction between tests that passed and tests that were actually required and executed is critical.

Your example also highlights a subtle failure mode: the verification gate itself can be incomplete, even when every reported check is green.

I'd add three requirements to the completion evidence:

1. A predefined manifest of required tests, tied to the exact code revision.
2. Explicit executed, passed, failed, and skipped counts, with zero skipped required tests for acceptance.
3. Independent verification that the gate actually exercised the required security boundaries.

This connects directly to the broader discussion around ARC and agent completion evidence. We need to verify not only the outcome, but also the completeness of the verification process itself.

One question: do you also test the gate with intentionally skipped or disabled security checks to confirm that it fails closed?

That seems like a valuable regression test for the verification system itself.

[Ayoub Hammoudi]

The "achieved" part has a sneaky trap. When the same agent writes the code and the tests, the tests check what the agent understood, not what you asked for. Both wrong the same way, suite stays green. What works for me: write the acceptance check before the agent starts, so it can't bend it to fit the code.

[asagiman]

Ayoub, that's an important distinction. I agree that when the same agent interprets the requirement, implements it, and writes the tests, a green suite can simply confirm a shared misunderstanding.

I would add one condition to your approach: the acceptance check should be established independently of the implementer's interpretation, not merely written earlier.

In our Agent OS architecture, acceptance criteria are fixed in the task contract before implementation. ARC's G1 also binds those declared criteria to the task subject, so subsequent changes invalidate the previous review binding.

But neither mechanism proves that the criteria correctly represent the original request, or that the tests actually exercise them.

The stronger design seems to be:

• Independently defined acceptance criteria before implementation.
• Versioned, immutable acceptance baselines with controlled revisions.
• An independent runner using the exact artifact and predefined expectations.
• Final verification against the user's intended outcome.

Your example highlights why test success and requirement satisfaction must remain separate claims.

Have you found that writing acceptance checks beforehand is sufficient in practice, or do you also use an independent reviewer to challenge the checks themselves?

[Ayoub Hammoudi]

Writing it earlier isn't enough on its own, agreed. The cheapest independent reviewer I've found: the acceptance check has to fail on the code before the change. If it already passes on the old code, it tests nothing, whoever wrote it.

[asagiman]

That's an excellent low-cost falsification check, Ayoub. Requiring the acceptance test to fail against the unchanged baseline makes the test's ability to detect the intended defect observable.

I'd pair it with two additional checks: the expected failure must correspond to the original user requirement, and the same frozen test must pass on the exact candidate revision.

That gives us a useful red-before / green-after result without relying on the implementing agent's interpretation alone. For critical changes, an independent reviewer can still challenge whether the test captures the right requirement.

This is a concrete acceptance check worth including in our evaluation design.

[Auten]

Question back: does your "did it actually happen" check ever read state the agent didn't write itself? For agents that act through a UI, the tool returning "clicked Send" proves almost nothing. The evidence we trust is a fresh read of the screen afterwards (the message is in the thread, the row exists), done separately from the action.

(I'm with the Auten team, we build screen-control tooling for agents, so that's the side effect I see fail most.)

[asagiman]

That's a very important distinction, and I agree.

A tool returning "clicked Send" is evidence of an attempted action, not evidence of the intended outcome.

I'm working on this problem through Agent Role Contracts (ARC) and our agent orchestration practices at Netsujo.

I think we need to distinguish three levels:

1. Action evidence — the agent attempted or executed an operation.
2. State evidence — a fresh observation of the target system confirms the expected change.
3. Acceptance evidence — the observed state satisfies the user's original completion criteria.

Your point adds an essential requirement: the observation must be sufficiently independent of the action and its self-reported result.

A fresh screenshot is useful, but we should also consider persistence, timing, and whether the evidence source can be influenced by the acting agent.

If independent verification is unavailable, the task should remain "unverified," rather than "done."

I'd be interested in comparing how Auten handles independent post-action verification with ARC's evidence and acceptance model, particularly for UI-driven agents.

[Vitaliy Dmitriev]

As a DevOps engineer - person closest to PROD envs - more or less I'm simulating normal "human like" SDLC cycle while implementing my tasks using AI agentic loops. That is there is a dedicated agent responsible for each stage - requirements collection, implementation, review, testing and deployment preparations. In case of review - it is done in multiple paths via multiple agents (usual cap in my case is 2-3 paths - again simulating multiple team members reviewing a pull request). As of now I'm still not giving full autonomy to the agents, for instance a pull request is not merged automatically but rather waits for my final review and also PROD deployment scripts are not triggered automatically, at the same time agents allowed to rollout to sandbox envs exactly to collect mentioned evidence that stuff implemented really works.

[asagiman]

Thanks, Vitaliy. Your approach is a useful example of separating agent autonomy by delivery stage. Letting agents deploy to sandbox environments while retaining human approval for merges and production deployment gives each stage a clear responsibility.

One question: how do you bind the sandbox verification evidence to the exact commit that gets merged and eventually deployed?

I'd want the completion record to identify the tested revision, executed and skipped checks, review results, and the independently observed production outcome. That would preserve the benefits of your multi-agent SDLC while making the final acceptance boundary explicit.

Appreciate you sharing a concrete operating model.

[jackymenCZ (jackymenCZ)]

I would add one layer between Contract and Execution: deterministic analysis of the code itself.

The flow could be:

Contract → Code Capability Analysis → Enforcement → Execution → Independent Verification → Acceptance

The important distinction is:

• Contract says what the agent is allowed to do.
• Code analysis determines what the submitted code can actually do.
• Enforcement makes sure those capabilities cannot exceed the contract.
• Execution records what actually happened.
• Independent verification checks the resulting state.
• Acceptance decides whether the task is truly complete.

That turns "Done" from an agent claim into an evidence chain.

And when something cannot be verified, the system should say NOT VERIFIED, not silently turn uncertainty into success.

I would extend this approach with one additional layer: deterministic pre-execution capability analysis.

A completion receipt should not only answer whether an action was permitted and executed. It should also establish whether the submitted code was consistent with the declared contract.

My proposed pipeline would be:

1. Contract: Define the agent's permitted scope, operations and constraints.
2. Pre-execution analysis: Statically analyze the submitted source code to identify capabilities such as filesystem access, network communication, process execution and other sensitive operations.
3. Policy enforcement: Compare those findings against the contract and enforce the decision at runtime using actual OS or sandbox-level controls. Static analysis alone should never be treated as enforcement.
4. Execution evidence: Record what actually happened, including relevant runtime events and resulting artifact hashes.
5. Independent acceptance: Verify the resulting state against explicit acceptance criteria using a fresh read, independent of the agent's own completion claim.

I would also make "UNKNOWN", "NOT VERIFIED" and "PENDING" first-class states rather than treating missing evidence as success.

The key idea is to connect declared permissions, analyzed code capabilities, actual execution and independently verified outcomes.

That gives us a much stronger definition of Done:

The agent was allowed to act, the code was checked against that permission, the execution was constrained, and the resulting state was independently verified.

The completion receipt then becomes an evidence chain, not simply another statement generated by the agent.😁

[asagiman]

Thanks for this thoughtful extension. I agree that deterministic pre-execution capability analysis would strengthen the evidence chain.

One distinction I'd preserve is between potential capabilities identified through static analysis and permissions actually enforced at runtime. Dynamic imports, dependencies, and subprocesses make complete static guarantees difficult.

I would also bind the analysis result to the exact executed artifact through a digest. Otherwise, we could verify one version of the code and execute another.

ARC currently focuses on checking declared role and scope consistency. Your proposal points toward a valuable integration with capability analysis, runtime enforcement, and independent outcome verification.

One question I'd be interested in exploring: How would you handle dynamically loaded code and dependencies while keeping the capability analysis deterministic and reproducible?

I think that is an important boundary to establish before treating the resulting receipt as trustworthy evidence of completion.

[jackymenCZ (jackymenCZ)]

I would approach this by defining a closed, content-addressed execution environment rather than trying to statically predict every possible runtime behavior.

My proposal has five parts:

1. Resolve dependencies before execution.

Build a complete dependency manifest covering the submitted source, transitive dependencies, runtime modules, native extensions and external executables. Pin exact versions and content digests. Avoid floating versions and unverified remote dependencies.

1. Bind analysis to the complete artifact.

Calculate a digest over a canonical manifest of the source and all resolved runtime dependencies. Include the runtime or container image digest and relevant configuration. The analysis receipt must reference this exact artifact identity, not merely the original source file.

1. Constrain dynamic loading.

Dynamic imports should only resolve to modules included in the approved manifest. If a module is generated or loaded dynamically, capture its contents, calculate its digest and verify it against policy before allowing execution.

Unresolved imports, arbitrary remote code loading and unverified dependencies should be rejected or explicitly classified as UNKNOWN. They must never silently receive PASS.

1. Enforce the boundary at runtime.

Static analysis describes potential capabilities. Actual restrictions must be enforced independently through the execution environment: controlled module loading, network restrictions, filesystem isolation and subprocess allowlists based on executable identity.

Runtime monitoring then records what was loaded and what operations actually occurred.

1. Verify the execution receipt.

Bind the analysis result, approved dependency manifest, runtime identity and execution evidence to the same artifact digest and execution identifier. Any mismatch invalidates the receipt.

There is one important limitation: arbitrary runtime-generated code and unrestricted dynamic loading cannot be made completely predictable through static analysis alone. Such behavior needs to be prohibited, intercepted and re-analysed, or explicitly left unverified.

I would therefore use three outcomes:

• PASS: All executed code and dependencies match the approved manifest and policy.
• BLOCKED: An unapproved capability or artifact was prevented from executing.
• NOT VERIFIED: The system cannot establish the identity or behavior of some executed component.

The underlying principle is that reproducibility comes from controlling what can enter the execution environment, not from assuming static analysis can foresee everything.

That gives the completion receipt a verifiable chain between the analyzed artifact, the actual execution environment and the observed result.🤔😁

[asagiman]

Thanks — I think the closed, content-addressed execution environment is a strong direction. Pinning the complete dependency graph and runtime identity, then enforcing dynamic loading at the host boundary, makes the evidence chain testable.

I'd preserve your PASS / BLOCKED / NOT VERIFIED outcomes for execution integrity, while adding a separate verdict for task outcomes based on independent observation.

A blocked unauthorized import may demonstrate successful containment while the task remains incomplete. Likewise, an execution-integrity PASS cannot establish that an external ERP write committed, especially after a timeout or lost acknowledgement.

There's also a trust-boundary question: with networked tools such as MCP, a stable tool-definition digest can coexist with changing server-side behavior.

I'd want the trusted host's enforcement evidence and the destination system's observed effects recorded independently, then bound to the operation ID, attempt ID, approved artifact identity, runtime identity, and policy revision.

A useful next step would be three small adversarial tests:

1. Dependency drift between analysis and dispatch.
2. An unapproved dynamic import at runtime.
3. An external write with an ambiguous acknowledgement.

ARC can check role and task declarations. The host and an independent destination observer would supply the enforcement and effect evidence.

What trust anchor would you use to make those observations independently verifiable?

[Cristian K]

One lightweight habit that has helped me is to put the acceptance check beside the task in the repo and have the PR/CI report the exact commit plus required, passed, and skipped checks—so “green” can’t hide an omitted gate. For side effects, I also want a fresh read from the destination system, not just the agent’s “success” response. What have you found is the smallest evidence bundle teams will reliably maintain?

[asagiman]

Cristian, I think placing the acceptance check beside the task is exactly the right direction. It keeps verification close to the work and makes missing evidence visible.

The smallest practical bundle I'd propose is one compact receipt with four parts:

1. Target: task ID and explicit acceptance criteria.
2. Identity: exact commit or artifact digest, plus deployment/environment where relevant.
3. Verification: required, executed, passed, failed, and skipped checks, linked to the actual CI and reviewer evidence.
4. Outcome: a fresh readback from the authoritative destination, including unresolved findings and the next action.

The receipt can link to existing evidence rather than duplicating logs. A skipped required check blocks acceptance, and a successful CI run alone cannot establish the live outcome.

In our Agent OS work, we're using this distinction between implementation, verification, acceptance, and live confirmation.

We haven't yet measured which receipt size teams maintain most reliably across projects, so I'd treat this as a practical baseline to validate.

One useful measure would be the time spent maintaining receipts versus the number of missed defects or unjustified completion claims they prevent. Have you measured that trade-off in your workflow?

[Cristian K]

Not yet—I haven't tracked that trade-off systematically. I'd start with a lightweight measure: time spent creating or updating the receipt versus defects or unsupported completion claims it catches, across a few tasks. Linking to existing CI and review evidence should keep the overhead low.

[asagiman]

Thanks, Cristian. I agree that a small, measurable trial is the right next step.

I'd track three things across a handful of comparable tasks: the time spent creating or updating evidence links, the missing checks or unsupported DONE claims those records catch, and the time another engineer needs to reproduce the result.

Reusing existing CI and review evidence should keep the overhead low. I'd also compare this against the existing CI-only workflow, so the additional receipt has to demonstrate value beyond what the team already maintains.

We haven't measured that incremental value across real workflows yet. I'd rather publish the observed costs and failures than assume the extra evidence is automatically worthwhile.

[Paul Lovelace]

brethofai's readback rule is the one I'd actually steal — nothing counts as done until something has read the value back from where it was supposed to land. The skipped-test story from jacksonxly is the nastier version though: 24 of 24 green, and the gate was quietly skipping the one test that mattered. Do you think the three-layer receipt works without a second agent or a human in the acceptance step, or does self-reported evidence always need an outside reader?

[asagiman]

Paul, I think the critical requirement is an independent verification boundary, rather than necessarily a second AI agent or human for every operation.

For a bounded, machine-verifiable task, three things can provide strong evidence:

1. A protected CI runner checks the exact revision against a predefined required-test manifest, including executed and skipped checks.
2. A trusted verifier confirms the artifact or actual side effect independently of the acting agent.
3. The acceptance decision is bound to those receipts and the original acceptance criteria.

A second agent sharing the same writable workspace and credentials may provide another opinion, but it doesn't necessarily provide independent evidence.

For user-visible changes, security-sensitive operations, and production releases, I'd retain the independent review and human approval required by the applicable risk and authorization policy.

The skipped-test example is especially important: 24/24 passing means little if the required 25th test never ran.

So my answer is that independent evidence is essential, while the verifier can sometimes be automated.

One question: have you found cases where adding a second agent improved acceptance quality beyond what a protected runner and independent destination readback already provided?

[Parsa]

checking on the code that actually landed, separate from whether the tests passed. all three of your layers can come back clean and the change still adds a retry with no backoff or a catch block that swallows the error, because nobody asked about that in the task. that’s what a PRI (production reliability index) scans on the exact commit where each flag comes with a before and after fix, so done means the tests passed and nothing fragile went in. jackson’s skipped test story is the scary one though, a gate that’s green because it skipped the hard part

[asagiman]

Parsa, this is an important addition. A passing test suite can establish that certain expected behaviors work, while still missing reliability regressions introduced by the same change.

Retries without backoff, swallowed exceptions, and incomplete cleanup paths are good examples of risks that may fall outside the original task's acceptance criteria.

I particularly agree with evaluating the exact commit that actually ships. In our Agent OS architecture, we distinguish the implemented revision, independent verification, and final production acceptance.

I'd add two requirements to the reliability check:

1. Record which checks actually ran, including skipped or weakened tests. A green exit code is insufficient if critical coverage disappeared.
2. Treat static reliability findings as evidence with explicit coverage and limitations, rather than proof that no fragile behavior remains.

A PRI-style assessment could complement independent QC, provided we can measure its incremental value against existing checks.

I'm curious about one detail: how does PRI distinguish a justified retry or exception handler from a genuinely unsafe one, and how do you measure false negatives on previously unseen reliability failures?

That seems especially important before making a reliability score a required completion gate.