Project-Template/docs/architecture/scripts/verify-before-done.sh

138 lines
5.7 KiB
Bash
Raw Normal View History

feat(guards): gate the claim of being finished, not only the artifact Every guard in this template fires on an artifact -- `githooks/pre-commit` on a commit, `audit-gate.mjs` and `preflight.sh` on a release, `verify.sh` when somebody runs it. `GUARDS.md` §6 is the rule they follow: guards belong before the artifact exists. An agent that says "done" and does not commit produces no artifact and trips none of them. That gap already has two rules pointing at it -- `WORK_CYCLE.md` opens with *"Done" is not a close*, and the tracker convention refuses a close under fifteen characters of evidence -- and a rule is a thing a reader can skip. This is the same rule with an exit code. `scripts/verify-before-done.sh` is a Claude Code `TaskCompleted` hook. Verified against Claude Code 2.1.251: the event exists and the binary carries the string `TaskCompleted hook prevented continuation`. WIRED TO `verify.sh`, NOT TO THE TEST COMMAND. `npm test` in a repo with no tests exits 0, so a gate built on it reports green having verified nothing -- `GUARDS.md` §4 and §8's whole subject. `verify.sh` already distinguishes the three answers, and this treats exit 2 (nothing was verified) as a failure with its own message rather than a pass. A repository with no checks yet fails this gate loudly, which is the intended answer. IT EXITS 2 TO BLOCK, AND THAT IS NOT THE USUAL CONVENTION. Claude Code reads a `1` from a hook as "the hook broke" and continues, so a gate written the ordinary way fails OPEN and lets through exactly what it was installed to catch, looking identical from the outside. `docs/TOOLS.md` now says this beside the exit-2-is-never-a-pass rule it inverts. THE LIMITATION IS IN THE HEADER IN CAPITALS, because it decides whether this is worth installing: it gates Claude Code and nothing else. A Codex session or a human in the same checkout writes past it. It is a second layer and never the layer -- `.githooks/pre-commit` is what git runs whoever is driving, and if a project's real suite is not wired into that hook, wiring it there is worth more than installing this. Proved rather than assumed, all four outcomes, per §1: no `verify.sh` -> 2 (fails closed); pass -> 0 with empty stderr; fail -> 2 naming the code; verified-nothing -> 2 with the distinct message. Docs in the same commit, per the triggers `doc-triggers.py` reported: `architecture/README.md` gains the row its own "Adding one" rule requires, `TOOLS.md` gains the paragraph under "Which ones can stop you", and `DOC_TRUST_MAP.md` gains the question it now answers. `doc-claims.sh` re-run: 51 claimed paths across the three, all present. Prompted by an XDA piece on Claude Code shipping unverified work. The idea is theirs; the exit-2 contract, the `verify.sh` wiring and the Codex caveat are what it needed to be true here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-29 08:42:00 -05:00
#!/usr/bin/env bash
#
# Gate the CLAIM of being finished, not the artifact. A Claude Code
# `TaskCompleted` hook.
#
# ## The hole this fills
#
# Every other guard in this template fires on an artifact: `githooks/pre-commit`
# on a commit, `audit-gate.mjs` and `preflight.sh` on a release, `verify.sh`
# when somebody runs it. `GUARDS.md` §6 is the rule they follow -- guards belong
# before the artifact exists.
#
# An agent that says "done" and does not commit produces no artifact, so it
# trips none of them. That is not a hypothetical: `WORK_CYCLE.md` opens by
# saying "Done" is not a close, the tracker convention refuses a `close` under
# fifteen characters of evidence, and both exist because the claim arrives
# without the work behind it often enough to need a rule. A rule is a thing a
# reader can skip. This is the same rule with an exit code.
#
# So: one step earlier than §6. Before the artifact exists, and before the
# sentence claiming it does.
#
# ## Why it calls verify.sh and not the test command
#
# `npm test` in a repo with no tests exits 0. A gate wired to it passes
# vacuously and reports green on a repository that verified nothing, which is
# `GUARDS.md` §4 and §8's entire subject -- the guard that cannot fail is worse
# than the guard nobody wrote, because it is believed.
#
# `verify.sh` already answers the right question. It runs every check the
# project actually has, in one command, and its exit codes are built for this:
#
# 0 everything that ran passed
# 1 something failed, or the run could not start
# 2 NOTHING WAS VERIFIED -- no checks detected, or every step skipped
#
# Two is not a pass and this script does not treat it as one. A repository that
# has not got as far as having checks fails this gate, loudly, with the reason.
# That is the intended answer: wire the gate up when there is something for it
# to run, and until then know that there is not.
#
# ## Install
#
# cp docs/architecture/scripts/verify-before-done.sh scripts/
# chmod +x scripts/verify-before-done.sh
#
# and in `.claude/settings.json` at the project root:
#
# {
# "hooks": {
# "TaskCompleted": [
# { "hooks": [ { "type": "command",
# "command": "${CLAUDE_PROJECT_DIR}/scripts/verify-before-done.sh",
# "timeout": 900 } ] }
# ]
# }
# }
#
# **Set `timeout` from this repository's own suite time, not from a number
# copied out of a blog post.** Time `bash scripts/verify.sh` once and give it
# headroom. A hook killed by its own timeout reports as a failure, so a budget
# that is too small turns a passing suite into a blocked task and teaches
# everybody to remove the hook -- `GUARDS.md` §5, a guard that is often wrong is
# worse than none.
#
# ## What it costs, said plainly
#
# It runs the suite every time a task completes, not once at the end of the
# session, so a long session runs it many times. On a repository whose suite
# takes minutes that is real time and real usage. `VERIFY_BEFORE_DONE_ARGS`
# exists for that: set it to `--quick` and the gate runs the checks that are
# cheap and names the test step as skipped rather than pretending it ran.
#
# A skipped step is visible in verify.sh's table. A gate that quietly stopped
# running is not. That is the whole difference.
#
# ## THE LIMITATION, AND IT IS NOT SMALL
#
# This is a Claude Code harness feature. It gates Claude Code and nothing else.
# A Codex session, a human, or any other agent working in the same checkout
# writes past it without noticing it exists -- and at least one repository here
# shares its checkout with Codex.
#
# So this is a second layer and never the layer. The gate that catches every
# writer is `.githooks/pre-commit`, because git runs it whoever is driving. If
# this repository's real suite is not wired into that hook, wiring it there is
# worth more than installing this, and neither one replaces the other.
#
# ## Exit codes -- the hook contract, which is not the usual one
#
# 0 allow the task to complete
# 2 BLOCK it, and feed stderr back to the model as the reason
#
# Two, specifically. A hook exiting 1 is a hook that broke, and Claude Code
# treats it as non-blocking -- so a gate that returns 1 on failure fails OPEN
# and lets exactly the thing it was installed to catch straight through. It
# looks identical from the outside. Prove this one blocks before trusting it:
# break something the suite covers and watch a task refuse to finish.
set -uo pipefail
cd "${CLAUDE_PROJECT_DIR:-.}" || {
echo "verify-before-done: cannot enter ${CLAUDE_PROJECT_DIR:-.}" >&2
exit 2
}
if [ ! -f scripts/verify.sh ]; then
# Fail closed. "There is no verify.sh" is a fact about the repository worth
# blocking on -- the alternative is a gate that silently allows everything on
# exactly the projects with no other checks.
echo "verify-before-done: scripts/verify.sh not found in $PWD." >&2
echo "Install it from the template, or remove this hook from .claude/settings.json." >&2
exit 2
fi
# Word-split on purpose: this is a small set of flags, not a path.
# shellcheck disable=SC2086
out="$(bash scripts/verify.sh ${VERIFY_BEFORE_DONE_ARGS:-} 2>&1)"
code=$?
if [ "$code" -eq 0 ]; then
exit 0
fi
# The tail, because the model reads this and the table verify.sh prints last is
# the part that says which step failed. A full log of a large suite buries it.
{
if [ "$code" -eq 2 ]; then
echo "verify.sh verified NOTHING (exit 2): no checks were detected, or every"
echo "step skipped. That is not a pass. Do not report this task as finished."
else
echo "verify.sh FAILED (exit $code). Do not report this task as finished."
fi
echo "Fix what it names, or say plainly that it is failing and why."
echo
printf '%s\n' "$out" | tail -n 200
} >&2
exit 2