695 lines
30 KiB
Bash
Executable File
695 lines
30 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Run every check this project has, in one command, and print one table saying
|
|
# which passed, which failed and which did not run at all.
|
|
#
|
|
# bash scripts/verify.sh # everything
|
|
# bash scripts/verify.sh --quick # skip the test step, name it as skipped
|
|
# bash scripts/verify.sh --list # show the plan, run nothing
|
|
# bash scripts/verify.sh --only npm # only steps whose id contains 'npm'
|
|
# VERIFY_GUARD_DIR=scripts/verify.d bash scripts/verify.sh
|
|
#
|
|
# Exit codes: 0 everything that ran passed. 1 something failed, or the run
|
|
# could not start at all — a usage error, a bad VERIFY_GUARD_DIR, not a git
|
|
# repository. 2 nothing was verified: either no checks were detected, or every
|
|
# detected step skipped. Two is not a pass and CI must not treat it as one.
|
|
#
|
|
# One and two are both non-zero on purpose, so every refusal fails closed. If
|
|
# you need to tell "a check failed" apart from "it never got to run", read the
|
|
# message — a failed check always prints a table first.
|
|
#
|
|
# ===========================================================================
|
|
# TEMPLATE COPY — configure this before the first run
|
|
# ===========================================================================
|
|
#
|
|
# Copy to `scripts/verify.sh`, `chmod +x` it, and wire it in: for a Node
|
|
# project add `"verify": "bash scripts/verify.sh"` to package.json; for anything
|
|
# else call it from `pre-commit`. Then read the CONFIGURATION block below —
|
|
# two environment variables (VERIFY_GUARD_DIR, VERIFY_SLOW_STEP) and six lists
|
|
# of candidate script and target names. Every one is optional except the one
|
|
# you need, which is GUARD_DIR.
|
|
#
|
|
# Unlike its sibling release.sh this script has no host, image or container to
|
|
# get wrong, so it defaults to DETECTING rather than to refusing. The rule it
|
|
# inherits unchanged is the important half: it never claims to have checked
|
|
# something it did not check. A detector that finds nothing says so and exits
|
|
# non-zero; it does not print a green table.
|
|
#
|
|
# Assumes: bash, coreutils, git. Each detector additionally needs the toolchain
|
|
# it detects, and says so by name when that toolchain is missing instead of
|
|
# quietly dropping the step.
|
|
#
|
|
# ## Why this exists
|
|
#
|
|
# "Did I break anything" should not be a judgement call, and in a project with
|
|
# three separate check commands it always is. The commands live in different
|
|
# places — a package.json script, a Makefile target, a lint you have to
|
|
# remember — so the honest answer to "did you run everything" is usually "I ran
|
|
# the one I remembered". The suite passes, the typecheck was never run, and the
|
|
# breakage is found by the deploy.
|
|
#
|
|
# One entry point removes the judgement. It also removes the excuse: there is
|
|
# no "I ran the important one" when running all of them is the same amount of
|
|
# typing.
|
|
#
|
|
# The second reason is the summary. A chain of `&&` stops at the first failure,
|
|
# so a run tells you about one broken thing at a time and you pay the whole
|
|
# cost again to find the next. This runs every step even after one fails, so a
|
|
# single run tells you everything that is broken.
|
|
#
|
|
# ## Guard tests are the point
|
|
#
|
|
# This project family keeps getting bitten by bugs that are invisible to
|
|
# ordinary tests, because the test and the bug agree with each other:
|
|
#
|
|
# - A Date serialised across a SQL boundary came back shifted by the server's
|
|
# timezone. Every test asserted against the same shifted value, so the
|
|
# suite was green and the dates were wrong.
|
|
# - A React prop was spread over a form field and silently overwrote its
|
|
# `name`. The component rendered, the test rendered it, and the field
|
|
# submitted under the wrong key.
|
|
# - A `redirect()` was called inside a `try` block. Next.js implements
|
|
# redirect by throwing, so the `catch` swallowed it and every successful
|
|
# action reported failure.
|
|
#
|
|
# None of those is a logic error a unit test would catch. All three are SHAPES
|
|
# in the source: a shape you can grep for. The fix is a guard test — a script
|
|
# that greps the source for the shape and exits non-zero when it reappears —
|
|
# and this script is where guard tests belong, because verify.sh is the thing
|
|
# that actually gets run.
|
|
#
|
|
# Write them as small executables in GUARD_DIR. One shape per file, named for
|
|
# the bug, exiting non-zero with a message naming the file and line. They cost
|
|
# milliseconds, they run first here for exactly that reason, and they are the
|
|
# only mechanism in the repository that catches a bug the tests cannot see.
|
|
#
|
|
# ## SKIPPED is not PASS
|
|
#
|
|
# A step that did not run gets its own state in the table and its own colour,
|
|
# and it is never folded into the pass count. This matters more than it looks:
|
|
# the failure this script exists to prevent is a green table produced by
|
|
# checking nothing, and every path to that failure runs through a skip that was
|
|
# reported as a success. So a run where nothing actually executed exits 2 even
|
|
# though nothing failed, and --quick names the step it dropped rather than
|
|
# quietly shortening the table.
|
|
#
|
|
# ## What it deliberately does not do
|
|
#
|
|
# It does not fix anything. No --fix, no formatter writing to your files: this
|
|
# runs immediately before a commit, and a verify that edits the tree changes
|
|
# what you were about to commit into something you have not read.
|
|
#
|
|
# It does not touch git — no staging, no committing, no stash, no branch check.
|
|
# It answers one question about the working tree as it stands.
|
|
#
|
|
# It does not walk into workspaces or sub-packages. Detection runs at the
|
|
# repository root, once. A monorepo wants a guard script per package, or a
|
|
# Makefile target that fans out, and either is a step this will find.
|
|
#
|
|
# It has no --dry-run because it writes nothing OF ITS OWN: no file is created,
|
|
# deleted, truncated or moved anywhere in this script. That is not the same as
|
|
# "changes nothing", and the difference matters. Every step it runs is somebody
|
|
# else's program — a package.json script, a Makefile target, an executable in
|
|
# GUARD_DIR — and those inherit no restraint from here. Dropping a test
|
|
# database is the usual one.
|
|
#
|
|
# --list is the dry-run equivalent: it shows the exact command each step would
|
|
# run, and runs none of them. Read it once, on a new project, before trusting
|
|
# this in a hook. That is the only place the full list is visible before it
|
|
# executes.
|
|
|
|
set -uo pipefail
|
|
|
|
say() { printf '\033[1mverify:\033[0m %s\n' "$*" >&2; }
|
|
die() { printf '\033[1mverify:\033[0m %s\n' "$*" >&2; exit 1; }
|
|
|
|
ROOT=$(git rev-parse --show-toplevel 2>/dev/null) \
|
|
|| die "not inside a git repository. Detection runs from the repository root so every detector looks in one place; cd into the repo and run this again."
|
|
|
|
# Empty is checked separately from failed. `cd ""` succeeds and stays put, so an
|
|
# empty ROOT would not fail here — it would silently detect in whatever
|
|
# directory you happened to be in, which is the wrong answer delivered
|
|
# confidently. A bare repository is the way it happens.
|
|
[ -n "$ROOT" ] \
|
|
|| die "git reported an empty repository root — this looks like a bare repository, which has no working tree to check. Run this from a normal checkout."
|
|
|
|
cd "$ROOT" || die "cannot cd to $ROOT"
|
|
|
|
# Resolved once, with symlinks collapsed, so the guard-directory containment
|
|
# check below compares two paths of the same kind.
|
|
ROOT_ABS=$(pwd -P)
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# CONFIGURATION
|
|
# ---------------------------------------------------------------------------
|
|
|
|
# Where guard scripts live, relative to the repository root. Empty by default:
|
|
# an unset path is a path that cannot point at the wrong thing, and this is the
|
|
# one value worth setting by hand — see "Guard tests are the point" above.
|
|
#
|
|
# Set and missing is a hard failure, not a skip. A guard directory that got
|
|
# renamed is precisely the case where silently running zero guards looks
|
|
# identical to running them all.
|
|
#
|
|
# IT MUST BE A DEDICATED DIRECTORY. Every executable file in it is RUN, with no
|
|
# allowlist — unlike the npm and make candidate lists below, which name the
|
|
# handful of scripts they are willing to invoke. That asymmetry is the whole
|
|
# hazard: point this at scripts/ and verify.sh runs release.sh; point it at a
|
|
# bin directory and it runs every binary there. detect_guards refuses the cases
|
|
# it can prove wrong (outside the repository, the repository root itself, the
|
|
# directory holding this script), but it cannot tell a guard from a deploy
|
|
# script that happens to sit beside one. Give guards their own directory and
|
|
# put nothing else in it.
|
|
GUARD_DIR="${VERIFY_GUARD_DIR:-}"
|
|
|
|
# Which step --quick drops, by exact id (the left column of the table). Empty
|
|
# means "every step classified as a test", which is the right guess almost
|
|
# everywhere and is stated out loud when it is used.
|
|
SLOW_STEP="${VERIFY_SLOW_STEP:-}"
|
|
|
|
# Candidate script and target names, checked against what the project actually
|
|
# has. Nothing here is run unless it exists — these are search terms, not
|
|
# defaults, and the distinction is load-bearing: `npm run lint` on a project
|
|
# without a lint script fails, and a verify that invents work to do fails for
|
|
# reasons that have nothing to do with the code.
|
|
#
|
|
# It is also an allowlist, and that is the other half of why it exists. A
|
|
# package.json contains scripts named `deploy` and `db:reset`. Running
|
|
# everything found would be a verify that publishes.
|
|
NODE_TYPECHECK_SCRIPTS="typecheck type-check tsc types"
|
|
NODE_LINT_SCRIPTS="lint lint:ci eslint format:check fmt:check"
|
|
NODE_TEST_SCRIPTS="test test:ci"
|
|
|
|
# `verify` and `all` are absent on purpose: a Makefile in a repository holding
|
|
# this script very likely has a `verify` target that calls this script, and
|
|
# that is an infinite loop rather than a failed check. The guard below catches
|
|
# it anyway, for the `check` target that turns out to do the same thing.
|
|
MAKE_LINT_TARGETS="lint fmt-check format-check"
|
|
MAKE_TYPECHECK_TARGETS="typecheck type-check types"
|
|
MAKE_TEST_TARGETS="test check"
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Refuse to be re-entered.
|
|
#
|
|
# A detected step that calls this script back — `make check` running
|
|
# scripts/verify.sh is the way it happens — recurses until the machine gives
|
|
# up, and the symptom is a hang rather than an error. Dying on the second entry
|
|
# turns that into one legible failure naming the step that did it.
|
|
# ---------------------------------------------------------------------------
|
|
if [ -n "${VERIFY_RUNNING:-}" ]; then
|
|
die "verify.sh invoked itself — a detected step calls this script back, which would recurse forever. Remove that target from the candidate lists in this file, or stop it calling verify."
|
|
fi
|
|
export VERIFY_RUNNING=1
|
|
|
|
QUICK=""
|
|
LIST=""
|
|
ONLY=""
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--quick) QUICK="yes" ;;
|
|
--list) LIST="yes" ;;
|
|
--only)
|
|
shift
|
|
[ $# -gt 0 ] || die "--only needs a pattern. Usage: --only <substring of a step id>"
|
|
ONLY="$1" ;;
|
|
--only=*) ONLY="${1#--only=}" ;;
|
|
*) die "unknown argument '$1'. Usage: verify.sh [--quick] [--list] [--only <pattern>]" ;;
|
|
esac
|
|
shift
|
|
done
|
|
|
|
if [ -t 2 ]; then
|
|
C_PASS=$'\033[32m'; C_FAIL=$'\033[31;1m'; C_SKIP=$'\033[33m'; C_OFF=$'\033[0m'
|
|
else
|
|
C_PASS=""; C_FAIL=""; C_SKIP=""; C_OFF=""
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The plan. Built completely before anything runs, so that "nothing was
|
|
# detected" is discovered before a single command executes rather than after a
|
|
# five-minute suite.
|
|
#
|
|
# Parallel arrays indexed together, iterated with a counted loop: `${arr[@]}` on
|
|
# an empty array is an unbound-variable error under `set -u` in bash 3.2, which
|
|
# is the bash on every stock macOS, and the empty case is the one that matters
|
|
# most here.
|
|
# ---------------------------------------------------------------------------
|
|
STEP_ID=()
|
|
STEP_CLASS=()
|
|
STEP_CMD=()
|
|
STEP_SKIP=()
|
|
STEP_RESULT=()
|
|
STEP_SECS=()
|
|
|
|
add_step() { STEP_ID+=("$1"); STEP_CLASS+=("$2"); STEP_CMD+=("$3"); STEP_SKIP+=(""); }
|
|
|
|
# A step that was found but cannot run. Recorded rather than dropped, because
|
|
# "this project has a mypy config and mypy is not installed" is information,
|
|
# and a silently shorter table is not.
|
|
add_skip() { STEP_ID+=("$1"); STEP_CLASS+=("skip"); STEP_CMD+=(""); STEP_SKIP+=("$2"); }
|
|
|
|
q() { printf '%q' "$1"; }
|
|
|
|
have() { command -v "$1" >/dev/null 2>&1; }
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Guards first. They are greps: they finish before the toolchain has finished
|
|
# starting, and they are the only steps that catch the bug class described in
|
|
# the header. Putting them last would mean the cheapest answer arrives after
|
|
# the most expensive one.
|
|
# ---------------------------------------------------------------------------
|
|
detect_guards() {
|
|
[ -n "$GUARD_DIR" ] || return 0
|
|
|
|
[ -d "$GUARD_DIR" ] \
|
|
|| die "VERIFY_GUARD_DIR is set to '$GUARD_DIR' but there is no such directory. Create it, or unset the variable — a missing guard directory would otherwise run zero guards and look exactly like running all of them."
|
|
|
|
# -------------------------------------------------------------------------
|
|
# Containment. Everything below this point EXECUTES every file it finds, so a
|
|
# mis-set GUARD_DIR is not a wrong answer — it is arbitrary code with the
|
|
# developer's credentials, launched by the one command they were told is safe
|
|
# to run before every commit.
|
|
#
|
|
# The three refusals below are the cases that can be proven wrong rather than
|
|
# guessed at. Resolved with `cd`+`pwd -P` so that symlinks, `..` and relative
|
|
# paths all collapse to one comparable form before being judged.
|
|
# -------------------------------------------------------------------------
|
|
local guard_abs self_abs
|
|
guard_abs=$(cd "$GUARD_DIR" 2>/dev/null && pwd -P) \
|
|
|| die "VERIFY_GUARD_DIR is set to '$GUARD_DIR' but that directory could not be entered (permissions?)."
|
|
|
|
# Outside the repository. `..`, an absolute path and a symlink pointing out of
|
|
# the tree all land here. A guard is a check on THIS repository's source; a
|
|
# directory outside it holds someone else's programs.
|
|
case "$guard_abs/" in
|
|
"$ROOT_ABS"/*) ;;
|
|
*) die "VERIFY_GUARD_DIR '$GUARD_DIR' resolves to '$guard_abs', which is outside this repository ('$ROOT_ABS'). Every executable file in it would be RUN. Guard scripts belong in a dedicated directory inside the repository." ;;
|
|
esac
|
|
|
|
# The repository root itself. Running every executable at the root means
|
|
# running whatever release, deploy or reset script the project keeps there.
|
|
[ "$guard_abs" != "$ROOT_ABS" ] \
|
|
|| die "VERIFY_GUARD_DIR points at the repository root. Every executable file at the root would be RUN as a guard, including any release, deploy or database script. Put guards in a dedicated subdirectory — scripts/verify.d is the convention."
|
|
|
|
# The directory holding this script. This is the likely mistake, because it is
|
|
# where scripts live and it reads as the obvious answer: this template ships
|
|
# verify.sh beside release.sh, backup.sh and migrate.sh, and pointing the
|
|
# guard directory here would run all three and call the result a passing
|
|
# check. Skipped when $0 cannot be resolved, which loses nothing — the two
|
|
# refusals above still apply.
|
|
self_abs=$(cd "$(dirname -- "$0")" 2>/dev/null && pwd -P) || self_abs=""
|
|
if [ -n "$self_abs" ] && [ "$guard_abs" = "$self_abs" ]; then
|
|
die "VERIFY_GUARD_DIR points at the directory holding verify.sh itself ('$guard_abs'). Every executable file beside this script — release.sh, backup.sh, migrate.sh — would be RUN as a guard. Put guards in a dedicated subdirectory of their own."
|
|
fi
|
|
|
|
local f path found=""
|
|
while IFS= read -r f; do
|
|
[ -n "$f" ] || continue
|
|
found="yes"
|
|
|
|
# Made explicitly relative so the guard runs whether or not its directory is
|
|
# on PATH — but only when it is not already absolute, since './' in front of
|
|
# an absolute path silently resolves somewhere else entirely.
|
|
case "$f" in /*) path="$f" ;; *) path="./$f" ;; esac
|
|
|
|
# A guard that lost its executable bit never runs and nothing notices. That
|
|
# is the same failure the whole script is about, one directory down. So is a
|
|
# guard symlinked in from a shared directory whose target has since moved:
|
|
# both are recorded, neither is dropped.
|
|
if [ -x "$f" ]; then
|
|
add_step "guard:$(basename "$f")" guard "$(q "$path")"
|
|
elif [ ! -e "$f" ]; then
|
|
add_skip "guard:$(basename "$f")" "broken symlink — it points at nothing, so this guard has not run since the target moved"
|
|
else
|
|
add_skip "guard:$(basename "$f")" "not executable — chmod +x it, or it will never run again either"
|
|
fi
|
|
|
|
# -L so a symlinked guard, and a symlinked guard DIRECTORY, are both seen.
|
|
# Sharing one guard across sibling repositories by symlink is the normal way
|
|
# to do it, and plain `-type f` does not match a symlink: those guards were
|
|
# not skipped, not listed and not run, which is the silent-zero failure this
|
|
# script exists to prevent. `! -type d` rather than `-type f` so a broken
|
|
# symlink still surfaces above instead of vanishing again.
|
|
done < <(find -L "$GUARD_DIR" -maxdepth 1 ! -type d 2>/dev/null | LC_ALL=C sort)
|
|
|
|
[ -n "$found" ] || say "note: $GUARD_DIR is configured but empty. See 'Guard tests are the point' at the top of this file for what belongs there."
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Node. The script names come from package.json and nowhere else.
|
|
#
|
|
# Read with node rather than grepped, because a grep for '"test"' matches keys
|
|
# in devDependencies, in a nested tool config, and in lint-staged — and
|
|
# `npm run` on a name that is not a script exits non-zero, so the guess would
|
|
# surface as a failing check with no failing code behind it.
|
|
# ---------------------------------------------------------------------------
|
|
detect_node() {
|
|
[ -f package.json ] || return 0
|
|
|
|
if ! have npm || ! have node; then
|
|
add_skip "npm" "package.json is here but node/npm is not on PATH — none of its checks could run"
|
|
return 0
|
|
fi
|
|
|
|
local scripts rc
|
|
scripts=$(node -e 'try{const s=require(process.cwd()+"/package.json").scripts||{};process.stdout.write(Object.keys(s).join("\n"))}catch(e){process.exit(1)}' 2>/dev/null); rc=$?
|
|
|
|
# Kept separate from "there are no scripts" below. Both end in zero npm steps
|
|
# and they need different answers: one is a project that has no checks yet,
|
|
# the other is a file this script could not read, which is also about to break
|
|
# every npm command anyone else runs today.
|
|
if [ "$rc" -ne 0 ]; then
|
|
add_skip "npm" "package.json could not be parsed by node — fix the JSON; no npm check could be read from it"
|
|
return 0
|
|
fi
|
|
|
|
if [ -z "$scripts" ]; then
|
|
add_skip "npm" "package.json declares no scripts — this Node project was not checked at all"
|
|
return 0
|
|
fi
|
|
|
|
local class list name matched=""
|
|
for class in typecheck lint test; do
|
|
case "$class" in
|
|
typecheck) list="$NODE_TYPECHECK_SCRIPTS" ;;
|
|
lint) list="$NODE_LINT_SCRIPTS" ;;
|
|
test) list="$NODE_TEST_SCRIPTS" ;;
|
|
esac
|
|
|
|
for name in $list; do
|
|
printf '%s\n' "$scripts" | grep -qx -- "$name" || continue
|
|
add_step "npm:$name" "$class" "npm run $(q "$name")"
|
|
matched="yes"
|
|
# One per class. Two matches usually means `test` and `test:ci` are the
|
|
# same suite twice, and paying for a suite twice is how people stop
|
|
# running verify.
|
|
break
|
|
done
|
|
done
|
|
|
|
if [ -z "$matched" ]; then
|
|
say "note: package.json has scripts, but none named like a check. Looked for:"
|
|
say " $NODE_TYPECHECK_SCRIPTS $NODE_LINT_SCRIPTS $NODE_TEST_SCRIPTS"
|
|
say " Add the real names to the candidate lists near the top of this file."
|
|
fi
|
|
}
|
|
|
|
detect_make() {
|
|
local mk="" f
|
|
for f in Makefile makefile GNUmakefile; do
|
|
if [ -f "$f" ]; then mk="$f"; break; fi
|
|
done
|
|
[ -n "$mk" ] || return 0
|
|
|
|
if ! have make; then
|
|
add_skip "make" "$mk is here but make is not on PATH"
|
|
return 0
|
|
fi
|
|
|
|
local class list target matched=""
|
|
for class in typecheck lint test; do
|
|
case "$class" in
|
|
typecheck) list="$MAKE_TYPECHECK_TARGETS" ;;
|
|
lint) list="$MAKE_LINT_TARGETS" ;;
|
|
test) list="$MAKE_TEST_TARGETS" ;;
|
|
esac
|
|
|
|
for target in $list; do
|
|
# Anchored at the start of the line so this matches a rule and not a
|
|
# .PHONY declaration or a variable that happens to contain the word.
|
|
grep -qE "^${target}[[:space:]]*:" "$mk" || continue
|
|
add_step "make:$target" "$class" "make $(q "$target")"
|
|
matched="yes"
|
|
break
|
|
done
|
|
done
|
|
|
|
# Said out loud, for the same reason the npm detector says it. A Makefile
|
|
# whose target is `tests` or `ci` contributes nothing here, and in a project
|
|
# that ALSO has a package.json the run still prints a full green table — one
|
|
# that silently excludes the Makefile's suite. Contributing zero steps is
|
|
# information; contributing zero steps quietly is the failure.
|
|
if [ -z "$matched" ]; then
|
|
say "note: $mk is here, but none of its targets are named like a check. Looked for:"
|
|
say " $MAKE_TYPECHECK_TARGETS $MAKE_LINT_TARGETS $MAKE_TEST_TARGETS"
|
|
say " Add the real names to the candidate lists near the top of this file."
|
|
fi
|
|
}
|
|
|
|
detect_cargo() {
|
|
[ -f Cargo.toml ] || return 0
|
|
|
|
if ! have cargo; then
|
|
add_skip "cargo" "Cargo.toml is here but cargo is not on PATH"
|
|
return 0
|
|
fi
|
|
|
|
if cargo fmt --version >/dev/null 2>&1; then
|
|
add_step "cargo:fmt" lint "cargo fmt --all -- --check"
|
|
else
|
|
add_skip "cargo:fmt" "rustfmt is not installed (rustup component add rustfmt)"
|
|
fi
|
|
|
|
if cargo clippy --version >/dev/null 2>&1; then
|
|
# `-D warnings` is not strictness for its own sake: without it clippy prints
|
|
# its findings and exits 0, so the step passes whatever it found — a check
|
|
# that cannot fail. Loosen it here if this project has warnings it has
|
|
# decided to keep, but loosen it visibly.
|
|
add_step "cargo:clippy" lint "cargo clippy --all-targets -- -D warnings"
|
|
else
|
|
add_skip "cargo:clippy" "clippy is not installed (rustup component add clippy)"
|
|
fi
|
|
|
|
add_step "cargo:test" test "cargo test"
|
|
}
|
|
|
|
detect_python() {
|
|
[ -f pyproject.toml ] || return 0
|
|
|
|
PY_MATCHED=""
|
|
|
|
# Configured in pyproject means the project uses it. A tool that merely
|
|
# happens to be installed on this machine is not a check this project has,
|
|
# and running it would invent a standard the repository never agreed to.
|
|
py_tool() { # section-regex binary class command
|
|
grep -qE "$1" pyproject.toml || return 0
|
|
PY_MATCHED="yes"
|
|
if have "$2"; then
|
|
add_step "py:$2" "$3" "$4"
|
|
else
|
|
add_skip "py:$2" "configured in pyproject.toml but '$2' is not on PATH — activate the virtualenv, or install it"
|
|
fi
|
|
}
|
|
|
|
py_tool '^\[tool\.ruff' ruff lint "ruff check ."
|
|
py_tool '^\[tool\.black' black lint "black --check ."
|
|
py_tool '^\[tool\.mypy' mypy typecheck "mypy ."
|
|
# pytest is matched anywhere in the file, not only as a [tool.pytest] section:
|
|
# most projects configure nothing and simply depend on it.
|
|
py_tool 'pytest' pytest test "pytest"
|
|
|
|
if [ -z "$PY_MATCHED" ]; then
|
|
say "note: pyproject.toml is here, but it configures none of the tools this"
|
|
say " looks for (ruff, black, mypy, pytest). No Python check was run."
|
|
fi
|
|
}
|
|
|
|
detect_go() {
|
|
[ -f go.mod ] || return 0
|
|
|
|
if ! have go; then
|
|
add_skip "go" "go.mod is here but go is not on PATH"
|
|
return 0
|
|
fi
|
|
|
|
add_step "go:vet" lint "go vet ./..."
|
|
add_step "go:test" test "go test ./..."
|
|
}
|
|
|
|
detect_guards
|
|
detect_node
|
|
detect_make
|
|
detect_cargo
|
|
detect_python
|
|
detect_go
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Nothing detected is a failure, and the loudest one here.
|
|
#
|
|
# A verify that checks nothing and prints success is worse than no verify at
|
|
# all: it converts "I have not checked" into "it passed" for everyone
|
|
# downstream, including the hook that trusts the exit code.
|
|
# ---------------------------------------------------------------------------
|
|
if [ "${#STEP_ID[@]}" -eq 0 ]; then
|
|
say "NO CHECKS DETECTED. Nothing was verified and nothing passed."
|
|
say ""
|
|
say "Looked at the repository root for: package.json scripts, a Makefile"
|
|
say "target, Cargo.toml, pyproject.toml, go.mod, and VERIFY_GUARD_DIR"
|
|
say "(currently ${GUARD_DIR:-unset})."
|
|
say ""
|
|
say "Give it something to run: add a check script to package.json, add a"
|
|
say "target to the Makefile, or set VERIFY_GUARD_DIR to a directory of"
|
|
say "executable guard scripts. Then run this again."
|
|
exit 2
|
|
fi
|
|
|
|
if [ -n "$ONLY" ]; then
|
|
n=${#STEP_ID[@]}
|
|
kept=""
|
|
for ((i = 0; i < n; i++)); do
|
|
case "${STEP_ID[$i]}" in
|
|
*"$ONLY"*) kept="yes" ;;
|
|
# An existing reason wins. A step already skipped because its tool is not
|
|
# installed is not skipped BY --only, and overwriting the reason would
|
|
# report that it was filtered out when in fact it could not have run —
|
|
# turning a real gap in the checks into a deliberate-looking choice.
|
|
*) [ -n "${STEP_SKIP[$i]}" ] || STEP_SKIP[$i]="--only $ONLY" ;;
|
|
esac
|
|
done
|
|
[ -n "$kept" ] || die "--only '$ONLY' matched none of the detected steps: $(printf '%s ' "${STEP_ID[@]}")"
|
|
fi
|
|
|
|
# --quick, resolved against the plan rather than assumed. A SLOW_STEP naming a
|
|
# step this project does not have is a stale config that would silently skip
|
|
# nothing while reporting that it skipped, so it is refused.
|
|
if [ -n "$QUICK" ]; then
|
|
n=${#STEP_ID[@]}
|
|
hit=""
|
|
for ((i = 0; i < n; i++)); do
|
|
if [ -n "$SLOW_STEP" ]; then
|
|
[ "${STEP_ID[$i]}" = "$SLOW_STEP" ] || continue
|
|
else
|
|
[ "${STEP_CLASS[$i]}" = "test" ] || continue
|
|
fi
|
|
[ -n "${STEP_SKIP[$i]}" ] || STEP_SKIP[$i]="--quick"
|
|
hit="yes"
|
|
done
|
|
|
|
if [ -z "$hit" ]; then
|
|
if [ -n "$SLOW_STEP" ]; then
|
|
die "VERIFY_SLOW_STEP is '$SLOW_STEP' but no detected step has that id: $(printf '%s ' "${STEP_ID[@]}")"
|
|
fi
|
|
say "note: --quick had no effect — no detected step is a test step."
|
|
elif [ -z "$SLOW_STEP" ]; then
|
|
say "--quick: skipping the test steps (set VERIFY_SLOW_STEP to an id to pick a different one)."
|
|
fi
|
|
fi
|
|
|
|
n=${#STEP_ID[@]}
|
|
|
|
if [ -n "$LIST" ]; then
|
|
say "$n step(s) detected. Nothing was run:"
|
|
for ((i = 0; i < n; i++)); do
|
|
if [ -n "${STEP_SKIP[$i]}" ]; then
|
|
say " $(printf '%-18s' "${STEP_ID[$i]}") SKIPPED — ${STEP_SKIP[$i]}"
|
|
else
|
|
say " $(printf '%-18s' "${STEP_ID[$i]}") ${STEP_CMD[$i]}"
|
|
fi
|
|
done
|
|
exit 0
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Run everything. No early exit on failure: the point of one entry point is one
|
|
# answer, and an answer that stops at the first broken thing makes you pay the
|
|
# whole runtime again for the second.
|
|
#
|
|
# Output is streamed rather than captured, so a step that hangs is visible
|
|
# while it hangs instead of after it is killed.
|
|
# ---------------------------------------------------------------------------
|
|
started=$SECONDS
|
|
failed=0
|
|
passed=0
|
|
skipped=0
|
|
|
|
for ((i = 0; i < n; i++)); do
|
|
if [ -n "${STEP_SKIP[$i]}" ]; then
|
|
STEP_RESULT[$i]="SKIPPED"
|
|
STEP_SECS[$i]="-"
|
|
skipped=$((skipped + 1))
|
|
continue
|
|
fi
|
|
|
|
say "── ${STEP_ID[$i]} — ${STEP_CMD[$i]}"
|
|
t0=$SECONDS
|
|
eval "${STEP_CMD[$i]}"
|
|
code=$?
|
|
STEP_SECS[$i]=$((SECONDS - t0))
|
|
|
|
if [ "$code" -eq 0 ]; then
|
|
STEP_RESULT[$i]="PASS"
|
|
passed=$((passed + 1))
|
|
else
|
|
STEP_RESULT[$i]="FAIL"
|
|
failed=$((failed + 1))
|
|
say "${STEP_ID[$i]} failed (exit $code) — continuing so this run reports everything."
|
|
fi
|
|
done
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# The table. Three states, three words, three colours, and the word alone is
|
|
# enough when the colour is gone — this gets piped into files and hook output
|
|
# where nothing is a terminal.
|
|
# ---------------------------------------------------------------------------
|
|
say ""
|
|
say "───────────────────────────────────────────────────────────"
|
|
|
|
for ((i = 0; i < n; i++)); do
|
|
case "${STEP_RESULT[$i]}" in
|
|
PASS) colour="$C_PASS" ;;
|
|
FAIL) colour="$C_FAIL" ;;
|
|
*) colour="$C_SKIP" ;;
|
|
esac
|
|
|
|
line="$(printf ' %s%-8s%s %-26s %5s' \
|
|
"$colour" "${STEP_RESULT[$i]}" "$C_OFF" "${STEP_ID[$i]}" "${STEP_SECS[$i]}")"
|
|
|
|
if [ "${STEP_RESULT[$i]}" = "SKIPPED" ]; then
|
|
line="${line} ${STEP_SKIP[$i]}"
|
|
else
|
|
line="${line}s"
|
|
fi
|
|
|
|
say "$line"
|
|
done
|
|
|
|
say "───────────────────────────────────────────────────────────"
|
|
|
|
elapsed=$((SECONDS - started))
|
|
|
|
# Counts are stated only for things that were measured. "0 failed" alongside
|
|
# four skips is true and misleading, so the skip count is never omitted and the
|
|
# exit code below refuses to call an all-skipped run a pass.
|
|
summary="${passed} passed"
|
|
[ "$failed" -eq 0 ] || summary="${summary}, ${failed} FAILED"
|
|
[ "$skipped" -eq 0 ] || summary="${summary}, ${skipped} SKIPPED (not run — not verified)"
|
|
|
|
say "${summary} [${elapsed}s]"
|
|
|
|
if [ "$failed" -gt 0 ]; then
|
|
say ""
|
|
say "Fix the FAILED step(s) above. To re-run one on its own:"
|
|
for ((i = 0; i < n; i++)); do
|
|
# Both halves shell-quoted, because both routinely contain spaces: this
|
|
# family of projects lives under directories like "Privacy LLC", and a step
|
|
# id is a guard's filename. Printed unquoted, the command this suggests
|
|
# parses as extra arguments and dies on the copy-paste.
|
|
[ "${STEP_RESULT[$i]}" = "FAIL" ] \
|
|
&& say " bash $(q "$0") --only $(q "${STEP_ID[$i]}")"
|
|
done
|
|
exit 1
|
|
fi
|
|
|
|
# Nothing failed and nothing ran. That is not a pass, and the exit code is the
|
|
# only part of this output a hook or a CI job will read.
|
|
if [ "$passed" -eq 0 ]; then
|
|
say ""
|
|
say "NOTHING WAS VERIFIED — every detected step skipped. Exiting non-zero:"
|
|
say "a clean table over an empty run is the one outcome this script exists"
|
|
say "to prevent."
|
|
exit 2
|
|
fi
|
|
|
|
exit 0
|