Specimen 04
The count that counted the passes.
“Failed · 3 of 4” when 3 passed and 1 failed — a count must say which way it counts.
This one is ours. It was caught in ThruLine’s own prototype during design review — one of ten sentences the review marked wrong or stale — and fixed in that review: failures first.
Recorded on ThruLine’s checkout-web demo record. No customer data. [swap in a corpus chain before launch]
One row, before and after
The prototype’s whole story of a change — the one page you take away to decide — lists its runs in a table. This is its row for the run at step 26, on version 4b19a0c — a version is one exact commit of the code.
Before
What the prototype said
One row of its runs table, as design review found it.
The record · step 26
What actually ran
Read from the log line. Exit 1 is the command’s own signal that something failed.
pytest -q tests/test_totals.py · version 4b19a0c · 16:02 · exit 1
- 3 passed
- 1 failed
- 4 tests
After
The fix, from the same review
Failures first. Each number carries its own word, and the total is named.
The same word, counted both ways
Later, in the same table, a failing re-run read “Failed · 1 of 5”. There the 1 counted the failures. One table, one word, two opposite counts — and nothing on the screen said which.
| Run | The screen said | The number counted | The record | Written failures first |
|---|---|---|---|---|
| step 26 4b19a0c · 16:02 | the passes | 3 passed, 1 failed | 1 failed, 3 passed (4 tests) | |
| A failing re-run on e71c5b0 | the failures | 1 failed, 4 passed | 1 failed, 4 passed (5 tests) |
The second row comes from the prototype’s failing branch of the demo record, where the re-run on e71c5b0 failed. On the main path, that re-run passes.
Where a count sits in the chain
The chain asks six questions in order, and each answer is a line from the record. A count is read at ② Rests on — and the chain’s own line there names both numbers, so neither can be taken for the other.
Claude Code said “Checkout total fixed.” “Tests pass.” · step 52 · demo record
The first two links
-
① Said
What did the agent claim?
"Tests pass." Claude Code · 16:03 step 52
-
② Rests on
What is that claim based on?
4 passed, 0 failed step 46 cited by the agent
-
③ Ran on · ④ Since · ⑤ Re-checked · ⑥ So
All six links, drawn in full: A file changed after the run
Not drawn on this page.
This specimen is about how a count is written, not about what the chain concludes, so it shows no verdict.
Also looked at by this chain
step 26 on 4b19a0c · 1 failed, 3 passed (4 tests)
An older version, and it failed. It is the row this specimen is about.
What this shows
A number is not a result until it says what it counts. “3 of 4” beside “Failed” reads as three failures; the record holds one. The review’s fix puts the failure first and gives every number its own word: “1 failed, 3 passed (4 tests)”.
- Product · Check it again Six outcomes, one pass Passed · Failed · Ran, checked nothing · Cancelled by you · Interrupted · Tool not found. Only a pass reads as a pass.
- Specimen 01 Ran, checked nothing A run that exits 0, the code for success, having checked nothing. This is not a pass.
- FAQ Does a model decide? No. It reads recorded runs, and the same record always gives the same answer.