prove-guard.sh refuses with exit 1, not the documented 2, so a mistyped find-string accuses the guard of being broken #19

New Issue

Closed

opened 2026-08-17 23:38:16 -05:00 by null · 0 comments

null commented

2026-08-17 23:38:16 -05:00

Owner

The mutation step refuses two ways — the find-string is absent, or it occurs more than once — and both used sys.exit("message") inside the embedded Python. sys.exit with a string prints it to stderr and exits 1.

What it costs

The script's header assigns those codes distinct meanings:

1  the guard stayed GREEN with its target broken. It is not testing what you
   think it is, and you have just learned that for the price of one edit
2  nothing was proven: bad arguments, missing file, or a find-string that is
   absent or ambiguous. **Two is not a pass**

So a typo in the find-string returned the code reserved for "your guard is broken" — a diagnosis about the code under test, from a run that never mutated anything and never executed the guard. The two states it most matters to distinguish were indistinguishable, and the wrong one is the alarming one.

TOOLS.md teaches callers to read these codes ("Exit code 2 is never a pass. These scripts distinguish 'the check ran and found nothing' from 'the check did not run', because those look identical from the outside and only one of them is evidence"). Every other refusal path in this script already exits 2 explicitly; only the Python block did not.

Both codes are non-zero, so CI still goes red — this is a wrong diagnosis rather than a missed failure, hence P2 rather than P1.

What to do

Replace both sys.exit(...) calls with a helper that prints to stderr and raises SystemExit(2).

Traps

The messages must still reach stderr; sys.exit(str) was doing that implicitly.
set -euo pipefail propagates the interpreter's status, so the shell needs no change — but the restore trap must still fire, and the file must come back intact on both refusal paths.

Verify: bash prove-guard.sh <file> <absent-string> X <cmd> and the ambiguous-string case both exit 2, still print their message, and still print "prove-guard: restored "; the 0, 1 and 3 paths are unchanged.

The mutation step refuses two ways — the find-string is absent, or it occurs more than once — and both used `sys.exit("message")` inside the embedded Python. `sys.exit` with a string prints it to stderr and exits **1**. ## What it costs The script's header assigns those codes distinct meanings: ``` 1 the guard stayed GREEN with its target broken. It is not testing what you think it is, and you have just learned that for the price of one edit 2 nothing was proven: bad arguments, missing file, or a find-string that is absent or ambiguous. **Two is not a pass** ``` So a **typo in the find-string** returned the code reserved for "your guard is broken" — a diagnosis about the code under test, from a run that never mutated anything and never executed the guard. The two states it most matters to distinguish were indistinguishable, and the wrong one is the alarming one. `TOOLS.md` teaches callers to read these codes ("Exit code `2` is never a pass. These scripts distinguish 'the check ran and found nothing' from 'the check did not run', because those look identical from the outside and only one of them is evidence"). Every other refusal path in this script already exits 2 explicitly; only the Python block did not. Both codes are non-zero, so CI still goes red — this is a wrong diagnosis rather than a missed failure, hence P2 rather than P1. ## What to do Replace both `sys.exit(...)` calls with a helper that prints to stderr and raises `SystemExit(2)`. ## Traps - The messages must still reach stderr; `sys.exit(str)` was doing that implicitly. - `set -euo pipefail` propagates the interpreter's status, so the shell needs no change — but the `restore` trap must still fire, and the file must come back intact on both refusal paths. Verify: `bash prove-guard.sh <file> <absent-string> X <cmd>` and the ambiguous-string case both exit 2, still print their message, and still print "prove-guard: restored <file>"; the 0, 1 and 3 paths are unchanged.