NRT Agentic AI Michigan State University

DEMO 7 · ADVANCED — A CHECK THAT CAN FAIL DIFFERENTLY

Analyst and verifier

One agent does the work, another decides whether it may be recorded. The rigorous version of the checker confirms a confidently wrong answer — and the version that can do less catches it.

Download Demo 7  ↓ ← Demo 6 Demo 8  → All demos

← Back to all demos

Package:

Demo7_two_agent_verifier.zip

This is Demo 5's app and server with two scoped subagents added. One does the work; the other decides whether the work may be recorded.

main agent  ──►  tissue-analyst    measures, labels, writes a report
            ──►  tissue-verifier   says whether that report may be recorded

The subagents never speak to each other. The main agent runs the first, takes its report, and hands that report to the second — which is the only way two agents ever communicate, and the reason results are written to files.

This demo assumes Demos 2, 4 and 5. You need the MCP server from Demo 2, the subagent file format from Demo 4, and the app from Demo 5. If any of that is unfamiliar, those three first.

1. Unzip and look at who is in the room

unzip Demo7_two_agent_verifier.zip
cd Demo7_two_agent_verifier
ls .claude/agents/

tissue-analyst.md
tissue-verifier.md

Two agent files, and a third MCP tool the server did not have before:

grep '"name"' server.py

analyze_tissue
classify_tissue
check_report        <- new

2. Run the case that matters

Start Claude Code and ask for both agents by name:

claude

Use the tissue-analyst subagent to analyze
data/case_fold_artifact_histology.pgm at 20x magnification with
damage_threshold_fraction 0.45.
Then use the tissue-verifier subagent to check the analyst's report
before I record it as a finding.

This is the fold-artifact slide from Demo 1 — the one where a photographic defect gets a confident clinical label. Watch both agents run.

Expect the verifier to confirm it.

analyst   mild_injury, probability 0.70
verifier  CONFIRMED - independent re-run agrees: mild_injury, 0.70

3. Read what the verifier actually did

Ask the main agent:

What tools did the verifier call, and with what arguments?

It called analyze_tissue and classify_tissue — the same two tools, with the same arguments, as the analyst. It re-ran the work and got the same answer, which it reported as independent agreement.

Open the file and see why that was inevitable:

cat .claude/agents/tissue-verifier.md

Its tools: list is the analyst's list. Its instructions say "run the analysis yourself and compare." It is careful, rigorous, and does exactly what it was told. It is also structurally incapable of disagreeing, because it asks the same classifier the same question — and that classifier is confidently wrong about this slide every single time.

A checker that can redo the work will usually agree with the work. Voting reduces variance. It does nothing about bias.

Notice what the verifier never looked at: what the report actually said. It compared its own answer to the analyst's answer. Nobody checked whether the report disclosed that low_confidence was true.

4. Install the other verifier

cp docs/verifier_versions/v2_checks_the_file.md .claude/agents/tissue-verifier.md

Restart Claude Code and run step 2 again, unchanged.

verifier  REJECTED - failed: confidence_disclosed_first, inside_classifier_domain

Same slide, same analyst, same report. Different checker.

5. What changed

The two verifiers differ in one line of frontmatter and the instructions that follow from it.

v1 v2
tools: analyze_tissue, classify_tissue check_report, and nothing else
can it run the analysis? yes no
what it compares its answer vs the analyst's answer the report's claims vs the files on disk
on the fold slide confirmed rejected

v2 cannot form an opinion. It has one tool, and that tool does arithmetic:

label_matches_file          the label in the report is the label in the file
probability_matches_file    the probability matches, to 0.005
measurement_was_usable      an unusable measurement gets no label at all
confidence_disclosed_first  low_confidence true -> the report must say so first
inside_classifier_domain    low_confidence true -> the model card excludes this slide

check_report has no model in it and cannot be persuaded. That is the point of moving the check into a tool rather than into a prompt.

6. The check that cannot be worded around

Run the analyst again and tell it to lead with the caveat this time:

Use the tissue-analyst subagent on the same slide, and have it state the
confidence problem before the label. Then verify it again.

confidence_disclosed_first now passes. inside_classifier_domain still fails.

That is not a bug. The classifier's own model card says clean slides only; folds, tears and pen marks excluded. A label on this slide is outside what the model was trained on, however carefully the sentence is phrased. The honest report is that this slide cannot be classified by this model — not a well-hedged label.

Better wording does not move a case back inside a model's domain.

7. See it in the app

python3 -m venv .demo_venv
source .demo_venv/bin/activate
pip install -r requirements.txt
./run.sh
The Demo 7 app with the Path C tab open: the sidebar shows the slide, magnification and threshold controls and a slide preview; the main panel shows the request naming both subagents, the claude -p command the app will run, and the Run both agents and Replay buttons.
The Path C tab before a run. The request names both subagents; the command below it is exactly what the app will execute. Click to enlarge.

Open the Path C — analyst + verifier tab. It runs both agents and splits the trace into lanes — whichever verifier you have installed:

main agent        2 tool calls     started the two subagents
tissue-analyst    2 tool calls     analyze_tissue, classify_tissue
tissue-verifier   2 tool calls     analyze_tissue, classify_tissue   <- v1

With v1 installed, the two lower lanes are identical. That picture is the whole demo.

The lanes come from one field. In --output-format stream-json, every message a subagent produces carries parent_tool_use_id — the id of the Agent call that started it; main-conversation messages carry null. stream_parse.agent_lanes() groups on exactly that.

Replay a recorded run (no model) renders a saved trace with real tool numbers, so the demo works with no Claude Code and no tokens — useful in a lecture.

8. Design questions for your own project

question what Demo 7 answers
Where should the check live? In a tool, if it can be arithmetic. A prompt that says "be rigorous" is not a check.
What should the checker be allowed to see? Something different from what the worker saw. v2 is weaker than v1 in capability and stronger as a check, because its narrower tool list is what forces it to look at the report.
Verifier, referee or vote? A verifier compares a claim to a source. A referee decides between two disagreeing agents. A vote takes n samples. Only the first one catches a systematic error.
Which model for the checker? The strongest one you have. The check is exactly where you are paying for accuracy — and note that v2 barely needs a model at all, because the judgement is in the tool.
How do agents share data? They do not. The main agent passes an artifact — here, two file paths — which is why the analyst writes out files rather than describing its numbers in prose.

9. What to take away

  1. "Independent" is not the same as "different". v1 was genuinely independent — its own context, its own tool calls, no sight of the analyst's work — and still agreed, because independence of process does not help when the error is in the model.
  2. A narrower agent can be a stronger check. v2 can do less than v1 by every measure, and catches what v1 misses.
  3. Put the rule where it cannot be talked out of. check_report is thirty lines of comparison. It is the same lesson as Demo 2's validate(), one level up: the model proposes, and something that is not a model disposes.