fix(security): stop secrets.sh flagging every prerendered page, and clear the dangling doc claims

secrets.sh --built reported ten credentials in dist/ and all ten were the same
false positive: the template's user:pass@host pattern reads the schema.org
JSON-LD on every prerendered page — //queuenorth.com"},"areaServed":{"@ — as a
host, a password and an @. One more finding for every page added, which is the
noise that turns a scanner into something people mute.

Quotes, braces, commas and angle brackets cannot occur in a real userinfo
component. Checked against a database URL with an inline password, one
percent-encoded, and a git remote carrying a token — all three still caught, all
ten false positives gone, and the historical Zoho leak from 033bdf6 still caught
when replayed.

The first version of that fix wrote its three test cases out literally in the
header, and --tracked then reported two credentials in the scanner itself. The
placeholders now use angle brackets, which are in the exclusion class the
comment is describing — so the examples cannot match the pattern they
illustrate. Same shape as the trap DOC_TRUST_MAP.md records about Exempt: lines.

doc-claims: 240 claimed paths, all present, up from 5 dangling. DOC_TRUST_MAP
was claiming banner.webp exists while saying it does not; GUARDS.md pointed at
prove-guard.sh, which this project declined. docs/history/ is excluded rather
than corrected — its entries name files that existed when they were written, and
editing an append-only log to satisfy a present-tense check is a category error.
TOOLS.md records the exclusion and why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ripley 2026-08-18 01:40:46 -05:00
parent 493b97e4a4
commit 2732d1fc4f
5 changed files with 87 additions and 5 deletions

View File

@ -198,8 +198,10 @@ is in them.
## One asset is genuinely missing, and that is not an exemption ## One asset is genuinely missing, and that is not an exemption
`docs/data/img/banner.webp` does not exist. `icon.webp` and `logo.webp` do, both docs/data/img/banner.webp does not exist — named here without backticks
converted from this project's own marks in `assets/`. deliberately, because `doc-claims.sh` reads a backticked path as a claim that the
file is there, and this sentence is the opposite claim. `icon.webp` and
`logo.webp` do exist, both converted from this project's own marks in `assets/`.
No placeholder was generated, deliberately — a placeholder that looks finished No placeholder was generated, deliberately — a placeholder that looks finished
outlives the issue that would have replaced it. It is filed as a `P2` issue outlives the issue that would have replaced it. It is filed as a `P2` issue

View File

@ -140,6 +140,36 @@ A liveness check by hand, when you want one without the script:
curl -s https://qn.isnull.dev/api/health # {"status":"ok","db":"ok","timestamp":"…"} curl -s https://qn.isnull.dev/api/health # {"status":"ok","db":"ok","timestamp":"…"}
``` ```
## Two checks that are run by hand
Neither is adopted into `scripts/`, so neither runs in `verify.sh`. Both are
worth running when the documents change a lot.
**`doc-claims.sh` — every path a document names must exist.** Run from the
template, and **exclude `docs/history/`**:
```bash
T=~/.openclaw/Projects/Template
DOC_CLAIMS_EXCLUDE='docs/proposed/|project-template/|vendor/|docs/history/' \
bash $T/docs/architecture/scripts/doc-claims.sh docs/ README.md
```
That exclusion is not a way of quietening a failure. `docs/history/DEVELOPMENT_LOG.md`
is dated and append-only, and its entries name files that existed **when they
were written** — `src/pages/8x8.jsx`, removed at `0.6.6`; `src/App.css`, gone in
a later refactor. Those are receipts, not claims about now, and correcting them
would rewrite the record of what was known at the time, which is the one thing
that file is for. Checking a history log for present-tense accuracy is a category
error, so it is excluded rather than edited.
Without the exclusion it reports three findings in that file, every time, forever.
Last run 2026-08-18: **240 claimed paths, all present, across 20 files.**
**`prove-guard.sh` is deliberately absent.** It breaks what a guard protects and
requires the guard to go red. This project has three guards, all shell scripts
that fail visibly, so §1 of `architecture/GUARDS.md` was performed by hand
instead — see `docs/history/DEVELOPMENT_LOG.md` for 2026-08-18.
## Adding a script ## Adding a script
Put it in `scripts/`, give it a header saying what it does and **which incident Put it in `scripts/`, give it a header saying what it does and **which incident

View File

@ -10,6 +10,14 @@ Review trigger: A guard is found to have been passing while the thing it guards
was broken; a new class of check is added to the suite. was broken; a new class of check is added to the suite.
``` ```
> **prove-guard.sh is not in this repository.** It lives in the template at
> `~/.openclaw/Projects/Template/docs/architecture/scripts/`, and this project
> declined it on adoption — its guards are three shell scripts in
> `scripts/verify.d/` that fail visibly on their own. It is named without
> backticks throughout for that reason. §1 below still applies and was performed
> by hand on every guard here; `docs/history/DEVELOPMENT_LOG.md` for 2026-08-18
> records how each was broken and what it did.
A guard that cannot fail is worse than no guard, because it is trusted. Every A guard that cannot fail is worse than no guard, because it is trusted. Every
rule here was learned by finding one that had been green for months over rule here was learned by finding one that had been green for months over
something broken. something broken.
@ -32,7 +40,7 @@ three tests, two of them are coincidental and will mask a real regression later.
If it fails none, the guard is decoration — and you have just learned that for If it fails none, the guard is decoration — and you have just learned that for
the price of one `sed`. the price of one `sed`.
`scripts/prove-guard.sh` performs exactly this, which removes the two ways it prove-guard.sh performs exactly this, which removes the two ways it
gets skipped: the restore is a `trap`, so an interrupted run cannot leave the gets skipped: the restore is a `trap`, so an interrupted run cannot leave the
code broken, and the failure count comes from the runner's own summary rather code broken, and the failure count comes from the runner's own summary rather
than from eyeballing red — one failing test is routinely reported on half a than from eyeballing red — one failing test is routinely reported on half a

View File

@ -141,7 +141,7 @@ each row says what it does *here*.
| Path | What it is | | Path | What it is |
| --- | --- | | --- | --- |
| `scripts/check-env.sh` | which of the 17 Zoho / reCAPTCHA / CORS / rate-limit variables are set and plausible, before the server reads them. Exit 2 means nothing was checked | | `scripts/check-env.sh` | which of the 17 Zoho / reCAPTCHA / CORS / rate-limit variables are set and plausible, before the server reads them. Exit 2 means nothing was checked |
| `scripts/secrets.sh` | credential shapes in a staged diff, and `--tracked` for a whole-tree audit. **`--built dist/` is the one that matters here**: `VITE_RECAPTCHA_SITE_KEY` is inlined into the bundle at build time, so the repository scan cannot see what users receive | | `scripts/secrets.sh` | credential shapes in a staged diff, and `--tracked` for a whole-tree audit. **`--built dist/` is the one that matters here**: `VITE_RECAPTCHA_SITE_KEY` is inlined into the bundle at build time, so the repository scan cannot see what users receive. Carries this project's own shapes, and one **tightened** pattern — see below |
| `scripts/verify.sh` | every check this project has, in one table. Honestly thin — there is no test suite, and it says so rather than printing a green row | | `scripts/verify.sh` | every check this project has, in one table. Honestly thin — there is no test suite, and it says so rather than printing a green row |
| `scripts/doc-triggers.py` | which documents a pending change fires, read from the `Governs:` headers. Run it before committing, not after | | `scripts/doc-triggers.py` | which documents a pending change fires, read from the `Governs:` headers. Run it before committing, not after |
| `scripts/forgejo-issue.py` | files and closes issues in the tracker convention, refusing malformed ones before they are filed | | `scripts/forgejo-issue.py` | files and closes issues in the tracker convention, refusing malformed ones before they are filed |
@ -154,6 +154,16 @@ each row says what it does *here*.
| `scripts/docker-test.sh` | builds the image and runs it locally on 3001. Predates the template | | `scripts/docker-test.sh` | builds the image and runs it locally on 3001. Predates the template |
| `scripts/prerender.js` | the build step that emits static HTML for every route. Predates the template | | `scripts/prerender.js` | the build step that emits static HTML for every route. Predates the template |
**Why `secrets.sh` has a tightened pattern.** Its `user:pass@host in a URL`
rule excludes only `/`, `@`, `:` and whitespace in the template. That is right
for source and wrong for this project's build output: every prerendered page
carries schema.org JSON-LD, and `//queuenorth.com"},"areaServed":{"@` parses as
a host, a password and an `@`. Ten findings per `--built` run, one more for
every page added — the noise that turns a scanner into something people mute.
Quotes, braces, commas and angle brackets cannot occur in real userinfo, so
excluding them costs nothing and was checked against three real credential URLs
before being applied. The reasoning is in the script's own header.
**Why `backup.sh` and `restore-check.sh` are not the template's originals.** Both **Why `backup.sh` and `restore-check.sh` are not the template's originals.** Both
ship as PostgreSQL tools. `backup.sh` is built to be adapted — everything ship as PostgreSQL tools. `backup.sh` is built to be adapted — everything
engine-specific is in one ENGINE block — so that block now calls engine-specific is in one ENGINE block — so that block now calls

View File

@ -132,7 +132,39 @@ NOTED_PATTERNS=(
# rather than words that merely appear near credentials — `password` in a # rather than words that merely appear near credentials — `password` in a
# sentence is not a leak, and a scanner that says it is gets muted. # sentence is not a leak, and a scanner that says it is gets muted.
PATTERNS=( PATTERNS=(
'//[^/@[:space:]:]+:[^/@[:space:]]+@' # user:pass@host in a URL # user:pass@host in a URL.
#
# TIGHTENED FOR THIS PROJECT on 2026-08-18. The template's version excludes
# only `/`, `@`, `:` and whitespace from the two halves, which is fine against
# source and wrong against this project's build output: every prerendered page
# carries schema.org JSON-LD, and
#
# //queuenorth.com"},"areaServed":{"@
#
# parses as host `queuenorth.com`, password `"},"areaServed"`, then an `@`.
# That is ten findings per `--built` run today and one more for every page
# added, which is precisely the noise this file's header warns turns a scanner
# into something people mute.
#
# Quotes, braces, commas and angle brackets cannot occur in a real userinfo
# component, so excluding them costs nothing.
#
# Checked against real shapes — a database URL with an inline password, one
# with a percent-encoded password, and a git remote carrying a token. All
# three still match; all ten JSON-LD false positives are gone.
#
# Those three are described rather than written out, and the placeholders below
# use <angle brackets> ON PURPOSE: angle brackets are in the exclusion class
# this comment is about, so the examples cannot match the pattern they
# illustrate. Written literally they did, and `--tracked` reported two
# credentials in this file — a scanner flagging its own documentation, which is
# the same shape of mistake as DOC_TRUST_MAP.md's note about a parser that
# cannot tell a description of a thing from the thing itself.
#
# https://<user>:<pass>@db.internal:5432/app
# postgres://<user>:<percent-encoded-pass>@host/db
# https://<token>:x-oauth-basic@forgejo/repo.git
'//[^/@[:space:]:"'"'"'{},<>]+:[^/@[:space:]"'"'"'{},<>]+@'
'[?&](token|key|secret|password|access_token|api_key)=[^&[:space:]"]+' '[?&](token|key|secret|password|access_token|api_key)=[^&[:space:]"]+'
'\b(Bearer|Basic)[[:space:]]+[A-Za-z0-9._~+/=-]{20,}' # an authorization header '\b(Bearer|Basic)[[:space:]]+[A-Za-z0-9._~+/=-]{20,}' # an authorization header
# Anchored to the start of a line or an `export`, because unanchored it # Anchored to the start of a line or an `export`, because unanchored it