900 lines
41 KiB
Bash
Executable File
900 lines
41 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
#
|
|
# Assert that the environment this project needs is present AND plausible,
|
|
# before the thing that needs it starts. Read-only: it measures and it reports.
|
|
#
|
|
# bash scripts/check-env.sh # check the process environment
|
|
# bash scripts/check-env.sh --file .env # check a file instead — only the file
|
|
# bash scripts/check-env.sh --quiet # failures only, for calling from a script
|
|
# bash scripts/check-env.sh --list # print what is declared, check nothing
|
|
#
|
|
# Exit codes, because the caller is usually another script deciding whether to
|
|
# continue:
|
|
#
|
|
# 0 every required variable is present, and everything that IS set has the
|
|
# right shape
|
|
# 1 problems found — they are listed, grouped, with the source named
|
|
# 2 NOTHING WAS CHECKED — the spec is empty, or the source could not be
|
|
# read. This is not a pass and CI must not treat it as one.
|
|
#
|
|
# All output goes to stderr, so `check-env.sh --quiet` can be called from the
|
|
# middle of another script without polluting that script's stdout.
|
|
#
|
|
# ===========================================================================
|
|
# TEMPLATE COPY — configure this before the first run
|
|
# ===========================================================================
|
|
#
|
|
# Copy to `scripts/check-env.sh`, `chmod +x` it, and fill in the SPEC block
|
|
# below — one line per variable. Nothing else needs editing.
|
|
#
|
|
# SPEC ships empty and the script exits 2 until it has an entry. An empty spec
|
|
# would examine nothing and have nothing to report, and the natural way to
|
|
# print that is "ok" — a green result from a checker that measured nothing.
|
|
# That is the worst possible output, because the whole value of this script is
|
|
# that everything downstream stops re-checking and starts trusting it.
|
|
#
|
|
# There are no default variable names, deliberately, for the reason release.sh
|
|
# gives at length: a list inherited from another project is a list of the wrong
|
|
# names. It passes — every variable it knows about really is set, in your shell,
|
|
# by that other project's tooling — while every variable this project actually
|
|
# reads goes unexamined. A checker that is confidently green about the wrong
|
|
# environment is worse than no checker.
|
|
#
|
|
# Assumes: bash, coreutils. It reads nothing from the repository, and the only
|
|
# thing it shells out to is `tr` for case folding — no docker, no git, no
|
|
# network client — so it runs unchanged inside a scratch container.
|
|
#
|
|
# ## Why this exists
|
|
#
|
|
# Absence is the cheap failure. A variable that is not set fails at the first
|
|
# line that reads it, loudly, usually before the process finishes starting.
|
|
#
|
|
# The expensive failure is a variable that is present and the wrong shape,
|
|
# because nothing refuses to start:
|
|
#
|
|
# PORT=3000/tcp a compose port mapping pasted into a port field
|
|
# DATABASE_URL=postgres://localhost/app
|
|
# right on a laptop; inside a container `localhost`
|
|
# is the container, and the database is elsewhere
|
|
# NODE_ENV=Production matches no `=== "production"` anywhere, so every
|
|
# branch takes its development arm in production
|
|
# JWT_SECRET=devsecret the eight characters someone typed to get the dev
|
|
# server up, now signing real sessions
|
|
# API_URL=https://api.example.com/
|
|
# one trailing slash; every joined path is `//v1/...`
|
|
#
|
|
# All five start. Most of them serve traffic. That is nearly the whole of
|
|
# "works on my machine, broken in the container": not a missing variable — a
|
|
# variable with the wrong shape, in an environment nobody ever compared against
|
|
# the one it was written for. So this checks SHAPE, not just presence.
|
|
#
|
|
# ## Every problem, in one pass
|
|
#
|
|
# The alternative is the loop everyone knows: start, crash on one variable, fix
|
|
# it, start, crash on the next. Each turn of that loop costs a full boot, and
|
|
# on a deploy target it costs a rollback. So nothing here short-circuits — the
|
|
# whole spec is evaluated and the report names everything that is wrong at
|
|
# once, grouped, with optional-and-absent shown separately because it is not a
|
|
# problem and must never read as one.
|
|
#
|
|
# ## Secrets are measured, never printed
|
|
#
|
|
# Anything whose name looks like a credential, and anything declared with the
|
|
# `secret-min-length` kind, is reported as "set, N characters" and never as its
|
|
# value — including inside failure messages, which is where a careless
|
|
# validator leaks: the whole point of a failure message is to show you what it
|
|
# saw. CI keeps its logs for months and shows them to everyone with read access
|
|
# to the repository, so a validator that echoes what it validated has published
|
|
# the credential to a wider audience than the breach it was guarding against.
|
|
#
|
|
# The name pattern errs toward over-matching. A false positive costs one
|
|
# unprinted value; a false negative costs a credential in a log.
|
|
#
|
|
# ## Where to call it
|
|
#
|
|
# From the container entrypoint before `exec`, from `predev`/`prestart`, and
|
|
# from CI before the deploy step. A checker nobody calls is decoration, and the
|
|
# environment is exactly the thing that differs between the place you tested
|
|
# and the place it broke.
|
|
#
|
|
# ## What it deliberately does not do
|
|
#
|
|
# It does not connect to anything. It never opens DATABASE_URL or curls
|
|
# API_URL. Reachability depends on a VPN, a network namespace and a machine
|
|
# that may not be this one, so a reachability check fails on laptops for
|
|
# reasons that are nobody's fault — and a check that cries wolf gets an
|
|
# `|| true` appended within a week, taking the shape checks down with it. This
|
|
# says the values are plausible. It does not say the services are up.
|
|
#
|
|
# It does not write, export, repair or default anything, so there is no
|
|
# --dry-run: there is no outward-facing act to preview. A checker that fixes
|
|
# what it finds is one you stop reading, and then it starts inventing the
|
|
# values that reach production.
|
|
#
|
|
# It does not merge sources. With --file it reads the file and the process
|
|
# environment is not consulted at all, which is the point: a variable that is
|
|
# set in your shell and missing from the .env is precisely the bug you came
|
|
# here to find, and merging hides it.
|
|
#
|
|
# It does not know your framework and never guesses a name. Every variable it
|
|
# checks is one you declared.
|
|
|
|
set -uo pipefail
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Snapshot the environment before this script assigns a single variable of its
|
|
# own. Nothing above this line may set a variable.
|
|
#
|
|
# `${!name}` reads a SHELL variable, and this script's own — QUIET, VALUE,
|
|
# SOURCE_LABEL, NAME_WIDTH, LIST_ONLY — live in the same namespace as the
|
|
# environment it is measuring. Read live rather than from a snapshot, the
|
|
# checker measures itself, in both directions and both are wrong:
|
|
#
|
|
# NAME_WIDTH declared in SPEC, unset in a scrubbed environment, reported
|
|
# "ok — present and the right shape: 14" and exited 0. Green for
|
|
# a variable nobody set, which is the one failure this file's
|
|
# header spends forty lines arguing is worse than no checker.
|
|
# QUIET=yes exported by the caller, reported "set, but empty" and exited 1,
|
|
# because the assignment below overwrote the value while keeping
|
|
# the export attribute. A failure nobody caused, on a variable
|
|
# that was correct.
|
|
#
|
|
# Taken here the snapshot is the environment as this process received it, and
|
|
# nothing this script does afterwards can alter what it reports. The _CE_ prefix
|
|
# is reserved from SPEC below so the snapshot cannot collide in its turn.
|
|
# ---------------------------------------------------------------------------
|
|
# `compgen` is a bash builtin, but it belongs to programmable completion and a
|
|
# bash built with --disable-progcomp does not have it. Without this the snapshot
|
|
# would come back empty and every declared variable would be reported missing:
|
|
# a confident, total, wrong answer. Refuse instead — that is a reason this
|
|
# script could not measure the environment, which is what exit 2 means. die()
|
|
# is not defined this early, so the message is written out longhand.
|
|
if ! type -t compgen >/dev/null 2>&1; then
|
|
printf '\033[1mcheck-env:\033[0m this bash has no `compgen`, so the environment cannot be\n' >&2
|
|
printf '\033[1mcheck-env:\033[0m read reliably and nothing was checked. That is not a pass.\n' >&2
|
|
exit 2
|
|
fi
|
|
|
|
_CE_ENV_NAMES=()
|
|
_CE_ENV_VALS=()
|
|
while IFS= read -r _CE_N; do
|
|
[ -n "$_CE_N" ] || continue
|
|
# `export FOO` with no value is exported-but-unset: it is not in a child's
|
|
# environment either, so it is absence rather than an empty value. `+x`
|
|
# keeps set-to-empty, which `docker run -e FOO` produces and which is a
|
|
# different bug from unset.
|
|
[ -n "${!_CE_N+x}" ] || continue
|
|
_CE_ENV_NAMES+=("$_CE_N")
|
|
_CE_ENV_VALS+=("${!_CE_N}")
|
|
done <<<"$(compgen -e)"
|
|
unset _CE_N
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# SPEC — the variables this project needs. One line each, then delete this
|
|
# banner. Adding a variable later is a one-line edit and nothing else.
|
|
#
|
|
# "NAME|required|kind|what breaks when it is missing or wrong"
|
|
#
|
|
# The description is not documentation. It is the sentence printed next to the
|
|
# failure at 03:00, so write the CONSEQUENCE ("sessions cannot be signed, every
|
|
# login 500s") rather than a restatement of the name ("the JWT secret").
|
|
#
|
|
# required | optional — spelled exactly. A typo is refused rather than read as
|
|
# "optional", because silently downgrading a required
|
|
# variable is the one mistake this script cannot survive.
|
|
#
|
|
# Kinds:
|
|
# nonempty any non-empty string; for values whose shape is
|
|
# genuinely unknowable. Not a lazy default — it is the
|
|
# honest one, and it still catches unset and empty.
|
|
# int a whole number
|
|
# int:MIN..MAX a whole number within bounds, inclusive
|
|
# port 1-65535
|
|
# url scheme://host[...]; rejects a bare `host:5432`
|
|
# path a filesystem path; existence is reported only when the
|
|
# source is this machine's own environment
|
|
# one-of:a,b,c exactly one of these, compared case-sensitively
|
|
# secret-min-length:N at least N characters; the value is never printed, and
|
|
# it is checked against well-known placeholders
|
|
#
|
|
# Examples of the FORM. Delete them — they are shapes, not your variables:
|
|
#
|
|
# "DATABASE_URL|required|url|nothing can read or write; every request 500s"
|
|
# "PORT|optional|port|the server binds its own default, which the proxy is not pointed at"
|
|
# "NODE_ENV|required|one-of:development,production,test|the wrong branch of every environment check"
|
|
# "SESSION_SECRET|required|secret-min-length:32|sessions are forgeable"
|
|
# "UPLOAD_DIR|optional|path|uploads land somewhere nothing serves and nothing backs up"
|
|
# ---------------------------------------------------------------------------
|
|
|
|
SPEC=(
|
|
# --- Always. Wrong here and the site is wrong for everybody. ---
|
|
"NODE_ENV|required|one-of:development,production,test|CSP relaxes in dev and the HTTP-to-HTTPS redirect only fires in production; the wrong value ships dev CSP to the public origin, or redirects a local dev server into a loop"
|
|
"SERVER_PORT|required|port|Express binds 3001 by default and the reverse proxy in front of nebula is pointed at whatever this says; a mismatch is a site that is running and unreachable"
|
|
"CORS_ORIGIN|required|url|the production origin, https://queuenorth.com, with NO trailing slash — a slash matches no real browser origin and silently blocks every cross-origin form post while the server logs nothing wrong. Both public hostnames reach the same container, so the site's own forms are same-origin and unaffected; this is about anything that is not"
|
|
"LOG_LEVEL|optional|one-of:error,warn,info,debug|defaults to info. At error you lose the request log, which is the only record of traffic this project keeps"
|
|
"RATE_LIMIT_PER_MINUTE|optional|int:1..1000|defaults to 5 per IP across all of /api. Unset is fine; set to something huge and the two POST endpoints are open to a bot overnight"
|
|
|
|
# --- Anti-abuse. The only thing between two open POST endpoints and a bot. ---
|
|
"RECAPTCHA_ENABLED|required|one-of:true,false|anything other than the literal string true disables verification silently. Declared required so that switching it off is a decision somebody typed, not a variable somebody forgot"
|
|
"RECAPTCHA_SECRET_KEY|optional|secret-min-length:32|required in practice whenever RECAPTCHA_ENABLED=true — without it every verification fails open and the forms have no protection at all. Optional here only because this script cannot express the conditional"
|
|
"RECAPTCHA_MIN_SCORE|optional|nonempty|defaults to 0.5. A float, so it is not range-checked here; set it to 0 and every bot passes"
|
|
"VITE_RECAPTCHA_SITE_KEY|optional|nonempty|BUILD TIME, not runtime — Vite inlines it into dist/. Absent at build and the widget never loads, and no restart fixes it. It must be the SITE key; the secret key here ships the secret to every visitor"
|
|
|
|
# --- CRM forwarding. Which half matters depends on the mode. ---
|
|
"ZOHO_FORWARDING_MODE|required|one-of:webtolead,api|selects which of the two blocks below is actually read. Production is webtolead; the api path is a configured standby. Wrong value and leads are written to SQLite and forwarded nowhere, silently, because the forward is fire-and-forget by design"
|
|
|
|
"ZOHO_WEBTOLEAD_ENABLED|optional|one-of:true,false|the on switch for the mode production actually uses. False means no lead reaches the CRM and the visitor still sees success"
|
|
"ZOHO_WEBTOLEAD_URL|optional|url|defaults to the Zoho form endpoint. A wrong host means every forward posts into the void"
|
|
"ZOHO_WEBTOLEAD_XNQSJSDP|optional|secret-min-length:32|the form identifier. Missing or wrong and Zoho rejects the post; leads survive in SQLite and never appear in the CRM"
|
|
"ZOHO_WEBTOLEAD_XMIWTLD|optional|secret-min-length:32|the second form identifier, same consequence"
|
|
"ZOHO_WEBTOLEAD_ACTION_TYPE|optional|nonempty|base64 for the target module. Defaults to Leads; wrong and records land in the wrong module"
|
|
"ZOHO_WEBTOLEAD_RETURN_URL|optional|nonempty|the literal string null is correct here — server-to-server, nothing redirects. Not url-checked for that reason"
|
|
"ZOHO_WEBTOLEAD_ZC_GAD|optional|nonempty|Zoho ad-tracking field, legitimately empty"
|
|
|
|
"ZOHO_ENABLED|optional|one-of:true,false|the on switch for the OAuth/REST standby path. Irrelevant while the mode is webtolead"
|
|
"ZOHO_CASES_ENABLED|optional|one-of:true,false|forwards support requests as Zoho Cases. Off means support tickets exist only in SQLite, where nobody is looking at them"
|
|
"ZOHO_API_DOMAIN|optional|url|the API host for the standby path. Datacenter-specific"
|
|
"ZOHO_ACCOUNTS_DOMAIN|optional|url|the OAUTH TOKEN host, which is a DIFFERENT domain from the API host. Pointing this at the API domain is the bug that shipped once and failed every token refresh in production"
|
|
"ZOHO_CLIENT_ID|optional|nonempty|standby path only"
|
|
"ZOHO_CLIENT_SECRET|optional|secret-min-length:16|standby path only"
|
|
"ZOHO_REFRESH_TOKEN|optional|secret-min-length:16|standby path only. Grants write access to the CRM"
|
|
)
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Redaction. See "Secrets are measured, never printed" above.
|
|
#
|
|
# Matched case-insensitively against the whole variable name, so KEY catches
|
|
# API_KEY, KEYCLOAK_SECRET and MONKEY_HOST alike. The last is a false positive
|
|
# and costs nothing: its value is described rather than shown.
|
|
# ---------------------------------------------------------------------------
|
|
SECRET_NAME_PATTERN='SECRET|TOKEN|PASSWORD|PASSWD|PWD|CREDENTIAL|PRIVATE|SALT|SIGNATURE|SIGNING|AUTH|KEY|DSN|COOKIE'
|
|
|
|
# Values a `secret-min-length` variable must not be, compared in lower case and
|
|
# never echoed. These are the strings typed to make the dev server start, which
|
|
# then travel to production inside a copied .env and satisfy every length rule.
|
|
PLACEHOLDER_SECRETS='changeme change-me change_me changethis secret mysecret supersecret password passwd hunter2 test testing example placeholder todo tbd xxx xxxx admin dev devsecret development your-secret-here your_secret_here notasecret 123456 12345678 abc123'
|
|
|
|
# Redaction applies to NAMES read out of a file too, not only to values.
|
|
#
|
|
# A multi-line secret pasted into a .env — a PEM body, a wrapped base64 blob —
|
|
# has continuation lines carrying '=' padding, so `KEY=VALUE` parsing splits one
|
|
# into a "name" and a "value" and the name is a slice of the credential. Every
|
|
# message that prints a name out of a file therefore prints its LENGTH once it
|
|
# is longer than any name could plausibly be, which is the same rule values
|
|
# already follow. 40 covers the longest real variable names (the framework
|
|
# ones reach the low forties); a 64-column PEM line does not come close.
|
|
NAME_PRINT_MAX=40
|
|
|
|
say() { printf '\033[1mcheck-env:\033[0m %s\n' "$*" >&2; }
|
|
|
|
# die is 2, not the 1 its counterpart in release.sh uses: everything that calls
|
|
# it is a reason this script could not measure the environment, and a caller
|
|
# must be able to tell "your configuration is wrong" from "I never found out".
|
|
die() { printf '\033[1mcheck-env:\033[0m %s\n' "$*" >&2; exit 2; }
|
|
|
|
usage() {
|
|
say "usage: check-env.sh [--file PATH] [--quiet] [--list] [--help]"
|
|
say " --file PATH check that file instead of the process environment"
|
|
say " --quiet, -q print only failures (exit code still says everything)"
|
|
say " --list print the declared spec and check nothing"
|
|
say " --help, -h this message"
|
|
}
|
|
|
|
trim() {
|
|
local s="$1"
|
|
s="${s#"${s%%[![:space:]]*}"}"
|
|
s="${s%"${s##*[![:space:]]}"}"
|
|
printf '%s' "$s"
|
|
}
|
|
|
|
SOURCE_FILE=""
|
|
QUIET=""
|
|
LIST_ONLY=""
|
|
|
|
while [ $# -gt 0 ]; do
|
|
case "$1" in
|
|
--file) shift; [ $# -gt 0 ] || die "--file needs a path."; SOURCE_FILE="$1" ;;
|
|
--file=*) SOURCE_FILE="${1#--file=}"; [ -n "$SOURCE_FILE" ] || die "--file needs a path." ;;
|
|
--quiet|-q) QUIET="yes" ;;
|
|
--list) LIST_ONLY="yes" ;;
|
|
-h|--help) usage; exit 0 ;;
|
|
*) usage; die "unknown argument '$1'." ;;
|
|
esac
|
|
shift
|
|
done
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Read the spec, and refuse anything ambiguous.
|
|
#
|
|
# A malformed spec entry is a configuration error in this script, not a finding
|
|
# about the environment, so it exits 2 and reports nothing about the variables:
|
|
# a partial report from a spec that is half-understood is the same lie as a
|
|
# green one from an empty spec.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
if [ "${#SPEC[@]}" -eq 0 ]; then
|
|
say "SPEC is empty, so nothing was checked. That is not a pass."
|
|
say ""
|
|
say "This is a fresh template copy: declare the variables this project reads"
|
|
say "in the SPEC block near the top of this file, one line each —"
|
|
say ' "DATABASE_URL|required|url|nothing can read or write; every request 500s"'
|
|
die "then run this again."
|
|
fi
|
|
|
|
SPEC_NAME=()
|
|
SPEC_REQ=()
|
|
SPEC_KIND=()
|
|
SPEC_PARAM=()
|
|
SPEC_DESC=()
|
|
NAME_WIDTH=4
|
|
|
|
entry_no=0
|
|
for entry in "${SPEC[@]}"; do
|
|
entry_no=$((entry_no + 1))
|
|
|
|
# A description may legitimately contain '|', so read takes only four fields
|
|
# and the fourth keeps the rest.
|
|
IFS='|' read -r f_name f_req f_kind f_desc <<<"$entry"
|
|
|
|
f_name=$(trim "${f_name:-}")
|
|
f_req=$(trim "${f_req:-}")
|
|
f_kind=$(trim "${f_kind:-}")
|
|
f_desc=$(trim "${f_desc:-}")
|
|
|
|
[ -n "$f_name" ] && [ -n "$f_req" ] && [ -n "$f_kind" ] && [ -n "$f_desc" ] \
|
|
|| die "SPEC entry ${entry_no} is not NAME|required|kind|description: '${entry}'"
|
|
|
|
# The shell cannot export a name it cannot parse, so a name that fails this
|
|
# could never have been set in the first place and every run would report it
|
|
# missing forever.
|
|
[[ "$f_name" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] \
|
|
|| die "SPEC entry ${entry_no}: '${f_name}' is not a usable variable name."
|
|
|
|
# The environment snapshot has to live somewhere, and a variable it could
|
|
# collide with is one this script would measure against itself. Refused
|
|
# rather than measured wrong.
|
|
case "$f_name" in
|
|
_CE_*) die "SPEC entry ${entry_no}: '${f_name}' uses the _CE_ prefix, which this script reserves for the environment snapshot it takes before it defines anything. Rename the variable, or rename the snapshot at the top of this file." ;;
|
|
esac
|
|
|
|
case "$f_req" in
|
|
required|optional) : ;;
|
|
*) die "SPEC entry ${entry_no} (${f_name}): expected 'required' or 'optional', got '${f_req}'. Spelling matters here — see the SPEC block." ;;
|
|
esac
|
|
|
|
kind_name="${f_kind%%:*}"
|
|
kind_param=""
|
|
case "$f_kind" in *:*) kind_param="${f_kind#*:}" ;; esac
|
|
|
|
case "$kind_name" in
|
|
nonempty|url|port|path)
|
|
[ -z "$kind_param" ] || die "SPEC entry ${entry_no} (${f_name}): kind '${kind_name}' takes no ':' argument." ;;
|
|
int)
|
|
if [ -n "$kind_param" ]; then
|
|
[[ "$kind_param" =~ ^[+-]?[0-9]+\.\.[+-]?[0-9]+$ ]] \
|
|
|| die "SPEC entry ${entry_no} (${f_name}): int bounds must look like 1..100, got '${kind_param}'."
|
|
fi ;;
|
|
one-of)
|
|
[ -n "$kind_param" ] || die "SPEC entry ${entry_no} (${f_name}): one-of needs a comma-separated list, e.g. one-of:development,production." ;;
|
|
secret-min-length)
|
|
[[ "$kind_param" =~ ^[0-9]+$ ]] && [ "$kind_param" -gt 0 ] \
|
|
|| die "SPEC entry ${entry_no} (${f_name}): secret-min-length needs a positive number, e.g. secret-min-length:32." ;;
|
|
*)
|
|
die "SPEC entry ${entry_no} (${f_name}): unknown kind '${kind_name}'. Valid: nonempty, int, port, url, path, one-of:a,b, secret-min-length:N." ;;
|
|
esac
|
|
|
|
# A name declared twice gets two descriptions, and the report would print
|
|
# both — one of which is now out of date and neither of which is marked.
|
|
for existing in ${SPEC_NAME[@]+"${SPEC_NAME[@]}"}; do
|
|
[ "$existing" = "$f_name" ] && die "SPEC declares ${f_name} twice. Keep one line."
|
|
done
|
|
|
|
SPEC_NAME+=("$f_name")
|
|
SPEC_REQ+=("$f_req")
|
|
SPEC_KIND+=("$kind_name")
|
|
SPEC_PARAM+=("$kind_param")
|
|
SPEC_DESC+=("$f_desc")
|
|
|
|
[ "${#f_name}" -gt "$NAME_WIDTH" ] && NAME_WIDTH="${#f_name}"
|
|
done
|
|
|
|
if [ -n "$LIST_ONLY" ]; then
|
|
say "declared in SPEC — nothing was checked:"
|
|
for (( i = 0; i < ${#SPEC_NAME[@]}; i++ )); do
|
|
kind="${SPEC_KIND[$i]}"
|
|
[ -n "${SPEC_PARAM[$i]}" ] && kind="${kind}:${SPEC_PARAM[$i]}"
|
|
printf ' %-*s %-8s %-22s %s\n' \
|
|
"$NAME_WIDTH" "${SPEC_NAME[$i]}" "${SPEC_REQ[$i]}" "$kind" "${SPEC_DESC[$i]}" >&2
|
|
done
|
|
exit 0
|
|
fi
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Display. Every message that mentions a value goes through show(), and that is
|
|
# the only place the redaction rule is enforced — so nothing else in this
|
|
# script may interpolate a value into a message. If you add a check, use it.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
is_secret() {
|
|
case "$2" in secret-min-length) return 0 ;; esac
|
|
local upper
|
|
upper=$(printf '%s' "$1" | tr '[:lower:]' '[:upper:]')
|
|
[[ "$upper" =~ $SECRET_NAME_PATTERN ]]
|
|
}
|
|
|
|
show() {
|
|
local name="$1" kind="$2" value="$3"
|
|
|
|
if is_secret "$name" "$kind"; then
|
|
printf 'set, %d characters (value withheld)' "${#value}"
|
|
return 0
|
|
fi
|
|
|
|
case "$kind" in
|
|
url)
|
|
# Scheme and host are what you need to spot the wrong environment; the
|
|
# path and query are where webhook and signed-URL secrets live, so they
|
|
# are summarised rather than shown.
|
|
local rest authority tail=""
|
|
case "$value" in
|
|
*://*)
|
|
rest="${value#*://}"
|
|
authority="${rest%%/*}"
|
|
authority="${authority%%\?*}"
|
|
case "$authority" in *@*) authority="***@${authority##*@}" ;; esac
|
|
case "$rest" in */*|*\?*) tail="/…" ;; esac
|
|
printf '%s://%s%s' "${value%%://*}" "$authority" "$tail"
|
|
return 0 ;;
|
|
esac
|
|
;;
|
|
# Bounded, enumerable and not credential-shaped by construction. `literal`
|
|
# is the mode a failure message asks for when the value IS the finding —
|
|
# a port of '3000/tcp' cannot be explained without quoting it.
|
|
int|port|one-of|path|literal) : ;;
|
|
*)
|
|
# Unknown shape, so nothing is known about what might be inside it.
|
|
printf 'set, %d characters' "${#value}"
|
|
return 0 ;;
|
|
esac
|
|
|
|
if [ "${#value}" -gt 60 ]; then
|
|
printf '%s…' "${value:0:59}"
|
|
else
|
|
printf '%s' "$value"
|
|
fi
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Load the source.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
FILE_KEYS=()
|
|
FILE_VALS=()
|
|
FILE_LINES=()
|
|
FILE_BAD=()
|
|
FILE_NOTES=()
|
|
|
|
load_file() {
|
|
local path="$1" lineno=0 line trimmed key val
|
|
|
|
# Redirection, not `cat "$path" |`: a piped while-loop runs in a subshell and
|
|
# every array appended inside it is discarded at the `done`.
|
|
while IFS= read -r line || [ -n "$line" ]; do
|
|
lineno=$((lineno + 1))
|
|
|
|
case "$line" in
|
|
*$'\r')
|
|
line="${line%$'\r'}"
|
|
FILE_NOTES+=("line ${lineno}|has a CRLF line ending; most loaders keep the carriage return, making the value one invisible character longer than it looks") ;;
|
|
esac
|
|
|
|
trimmed=$(trim "$line")
|
|
case "$trimmed" in ''|'#'*) continue ;; esac
|
|
|
|
case "$trimmed" in
|
|
export\ *|export$'\t'*)
|
|
trimmed=$(trim "${trimmed#export}")
|
|
FILE_NOTES+=("line ${lineno}|uses 'export'; a shell sourcing this file is fine, but docker --env-file reads the name as 'export NAME'") ;;
|
|
esac
|
|
|
|
case "$trimmed" in
|
|
*=*) key="${trimmed%%=*}"; val="${trimmed#*=}" ;;
|
|
*) FILE_BAD+=("line ${lineno}|no '=', so it sets nothing at all"); continue ;;
|
|
esac
|
|
|
|
if [[ ! "$key" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then
|
|
if [[ "$(trim "$key")" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then
|
|
FILE_BAD+=("line ${lineno}|has spaces around '='; outside a shell those become part of the name and of the value")
|
|
elif [ "${#key}" -gt "$NAME_PRINT_MAX" ]; then
|
|
FILE_BAD+=("line ${lineno}|has ${#key} characters before its '=', so that is not a name and nothing can read it; the text is withheld in case this is one line of a wrapped multi-line secret")
|
|
else
|
|
FILE_BAD+=("line ${lineno}|'${key}' is not a usable variable name, so nothing can read it")
|
|
fi
|
|
continue
|
|
fi
|
|
|
|
# Quotes are stripped here because that is what dotenv-style loaders do —
|
|
# but docker --env-file does not, so the two see different values. Said out
|
|
# loud rather than resolved, because only the operator knows which loader
|
|
# will read this file.
|
|
case "$val" in
|
|
\"*\"|\'*\')
|
|
val="${val:1:${#val}-2}"
|
|
FILE_NOTES+=("${key}|is quoted; dotenv-style loaders strip the quotes and docker --env-file keeps them, so the two disagree about this value") ;;
|
|
esac
|
|
|
|
case "$val" in
|
|
*'${'*|*'$('*)
|
|
FILE_NOTES+=("${key}|contains a \$ expansion; dotenv-style loaders interpolate it and docker --env-file passes it through literally") ;;
|
|
esac
|
|
|
|
local j
|
|
for (( j = 0; j < ${#FILE_KEYS[@]}; j++ )); do
|
|
if [ "${FILE_KEYS[$j]}" = "$key" ]; then
|
|
FILE_NOTES+=("${key}|is set twice (lines ${FILE_LINES[$j]} and ${lineno}); loaders disagree about which one wins")
|
|
break
|
|
fi
|
|
done
|
|
|
|
FILE_KEYS+=("$key")
|
|
FILE_VALS+=("$val")
|
|
FILE_LINES+=("$lineno")
|
|
done < "$path"
|
|
}
|
|
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
# Refused rather than fallen back on. You asked about a file; answering about
|
|
# the process environment instead would be a confident pass from a source you
|
|
# did not name — see status.sh on why a tool that answers about the wrong
|
|
# thing is worse than one that answers nothing.
|
|
[ -e "$SOURCE_FILE" ] || die "--file '${SOURCE_FILE}' does not exist. Refusing to check the process environment instead."
|
|
[ -f "$SOURCE_FILE" ] || die "--file '${SOURCE_FILE}' is not a regular file."
|
|
[ -r "$SOURCE_FILE" ] || die "--file '${SOURCE_FILE}' is not readable by this user."
|
|
load_file "$SOURCE_FILE"
|
|
SOURCE_LABEL="${SOURCE_FILE}"
|
|
else
|
|
SOURCE_LABEL="the process environment"
|
|
fi
|
|
|
|
# The empty string means "not set here". Callers must not read it as "set to
|
|
# nothing" — FOUND says which.
|
|
FOUND=""
|
|
VALUE=""
|
|
|
|
# The locals are prefixed for the same reason the snapshot is: `lookup`'s own
|
|
# `want` and `j` would shadow a SPEC entry of either name, and this is the one
|
|
# function that resolves a variable name at runtime.
|
|
lookup() {
|
|
local _ce_want="$1" _ce_j
|
|
FOUND=""
|
|
VALUE=""
|
|
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
# No break: the last assignment wins, which is what a loader reading the
|
|
# file top to bottom into a map does.
|
|
for (( _ce_j = 0; _ce_j < ${#FILE_KEYS[@]}; _ce_j++ )); do
|
|
if [ "${FILE_KEYS[$_ce_j]}" = "$_ce_want" ]; then FOUND="yes"; VALUE="${FILE_VALS[$_ce_j]}"; fi
|
|
done
|
|
else
|
|
# The snapshot taken at the top, never the live variable: see the reasoning
|
|
# there. A name absent from the snapshot was absent from the environment,
|
|
# whatever this script may since have set in its own namespace.
|
|
for (( _ce_j = 0; _ce_j < ${#_CE_ENV_NAMES[@]}; _ce_j++ )); do
|
|
if [ "${_CE_ENV_NAMES[$_ce_j]}" = "$_ce_want" ]; then
|
|
FOUND="yes"; VALUE="${_CE_ENV_VALS[$_ce_j]}"; break
|
|
fi
|
|
done
|
|
fi
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Shape checks.
|
|
#
|
|
# check_shape prints zero or more lines, each 'fail:<reason>' or 'note:<reason>'
|
|
# — a protocol rather than array appends because it is called in a command
|
|
# substitution, and an array appended inside a subshell is gone at the closing
|
|
# paren.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
check_shape() {
|
|
local name="$1" kind="$2" param="$3" value="$4"
|
|
|
|
case "$value" in
|
|
' '*|$'\t'*) printf 'fail:begins with whitespace, which is part of the value everywhere except a shell\n' ;;
|
|
esac
|
|
case "$value" in
|
|
*' '|*$'\t') printf 'fail:ends with whitespace, which is part of the value and breaks every comparison against it\n' ;;
|
|
esac
|
|
|
|
case "$kind" in
|
|
url|port|int|one-of|path)
|
|
case "$value" in
|
|
*$'\n'*) printf 'fail:contains a newline, so nothing downstream will parse it\n'; return 0 ;;
|
|
esac ;;
|
|
esac
|
|
|
|
case "$kind" in
|
|
nonempty)
|
|
# Presence and non-emptiness are the same test, already done by the
|
|
# caller. Reaching here means it passed; there is nothing else to know.
|
|
: ;;
|
|
|
|
int)
|
|
if [[ ! "$value" =~ ^[+-]?[0-9]+$ ]]; then
|
|
printf 'fail:is not a whole number: %s\n' "$(show "$name" "$kind" "$value")"
|
|
return 0
|
|
fi
|
|
local n="${value#+}"
|
|
# Bash arithmetic is 64-bit and errors past it, so an absurd number is
|
|
# rejected by length before anything tries to compare it.
|
|
if [ "${#n}" -gt 18 ]; then
|
|
printf 'fail:has %d digits; nothing downstream will read that as a number\n' "${#n}"
|
|
return 0
|
|
fi
|
|
if [ -n "$param" ]; then
|
|
local lo="${param%%..*}" hi="${param##*..}"
|
|
if [ "$n" -lt "${lo#+}" ] || [ "$n" -gt "${hi#+}" ]; then
|
|
# Through show(), not the bare value: an out-of-range number under a
|
|
# credential-shaped name — TOTP_KEY, AUTH_TOKEN_TTL — is still a
|
|
# credential, and this is a failure message, which is precisely where
|
|
# the header says a careless validator leaks. The bounds come from the
|
|
# spec, so they are always safe to print.
|
|
printf 'fail:is %s, outside the allowed %s..%s\n' \
|
|
"$(show "$name" "$kind" "$value")" "${lo#+}" "${hi#+}"
|
|
fi
|
|
fi ;;
|
|
|
|
port)
|
|
case "$value" in
|
|
*://*) printf 'fail:is a URL, not a port number: %s\n' "$(show "$name" url "$value")"; return 0 ;;
|
|
*/*) printf 'fail:looks like a compose port mapping (%s); a port field takes the number alone\n' "$(show "$name" literal "$value")"; return 0 ;;
|
|
esac
|
|
if [[ ! "$value" =~ ^[0-9]{1,5}$ ]] || [ "$value" -lt 1 ] || [ "$value" -gt 65535 ]; then
|
|
printf 'fail:is not a port; ports are 1-65535\n'
|
|
return 0
|
|
fi ;;
|
|
|
|
url)
|
|
case "$value" in
|
|
*://*) : ;;
|
|
*)
|
|
printf 'fail:has no scheme — %s is a host, not a URL; write it as scheme://host\n' "$(show "$name" literal "$value")"
|
|
return 0 ;;
|
|
esac
|
|
|
|
local scheme="${value%%://*}" rest="${value#*://}" authority
|
|
if [[ ! "$scheme" =~ ^[A-Za-z][A-Za-z0-9+.-]*$ ]]; then
|
|
printf 'fail:has a scheme that is not a scheme\n'
|
|
return 0
|
|
fi
|
|
|
|
authority="${rest%%/*}"
|
|
authority="${authority%%\?*}"
|
|
case "$authority" in *@*) authority="${authority##*@}" ;; esac
|
|
|
|
if [ -z "$authority" ]; then
|
|
printf 'fail:has no host between // and the path\n'
|
|
return 0
|
|
fi
|
|
case "$authority" in
|
|
*' '*) printf 'fail:has a space in the host\n'; return 0 ;;
|
|
esac
|
|
|
|
case "$authority" in
|
|
localhost|localhost:*|127.0.0.1|127.0.0.1:*|0.0.0.0|0.0.0.0:*|'[::1]'|'[::1]':*)
|
|
printf 'note:points at %s — correct on a laptop; inside a container that is the container itself, not the host\n' "$authority" ;;
|
|
host.docker.internal*)
|
|
printf 'note:points at host.docker.internal, which resolves on Docker Desktop and not on a Linux daemon without an extra_hosts entry\n' ;;
|
|
esac
|
|
|
|
case "$value" in
|
|
*/) printf 'note:ends with a slash, so anything joining a path onto it produces a double slash\n' ;;
|
|
esac ;;
|
|
|
|
path)
|
|
case "$value" in
|
|
'~'*) printf 'fail:starts with ~, and nothing expands a tilde inside an environment variable — a process reading this opens a directory literally named ~\n' ;;
|
|
esac
|
|
case "$value" in
|
|
/*) : ;;
|
|
*) printf 'note:is relative, so it resolves against the working directory of whatever starts the process rather than the one you typed it in\n' ;;
|
|
esac
|
|
# Existence is a fact about THIS machine. Checking it against a file that
|
|
# describes another machine would report a fault that is not one, so in
|
|
# --file mode it is not checked and the report says so rather than
|
|
# leaving a silent gap.
|
|
if [ -z "$SOURCE_FILE" ] && [ ! -e "$value" ]; then
|
|
printf 'note:does not exist on this machine (this may be correct if the volume is mounted later)\n'
|
|
fi ;;
|
|
|
|
one-of)
|
|
local rest="$param" opt matched="" ci_match=""
|
|
local lower_value lower_opt
|
|
lower_value=$(printf '%s' "$value" | tr '[:upper:]' '[:lower:]')
|
|
while [ -n "$rest" ]; do
|
|
opt="${rest%%,*}"
|
|
[ "$value" = "$opt" ] && matched="yes"
|
|
lower_opt=$(printf '%s' "$opt" | tr '[:upper:]' '[:lower:]')
|
|
[ "$lower_value" = "$lower_opt" ] && ci_match="$opt"
|
|
case "$rest" in *,*) rest="${rest#*,}" ;; *) rest="" ;; esac
|
|
done
|
|
if [ -z "$matched" ]; then
|
|
if [ -n "$ci_match" ]; then
|
|
printf 'fail:is %s, and the comparison is case-sensitive: write %s\n' "$(show "$name" one-of "$value")" "$ci_match"
|
|
else
|
|
printf 'fail:is %s; allowed: %s\n' "$(show "$name" one-of "$value")" "${param//,/, }"
|
|
fi
|
|
fi ;;
|
|
|
|
secret-min-length)
|
|
# Length is the only property reported. Nothing below prints the value,
|
|
# and the comparison below pipes it to `tr` rather than passing it as an
|
|
# argument, so it never appears in this process's command line either.
|
|
if [ "${#value}" -lt "$param" ]; then
|
|
printf 'fail:is %d characters and the spec requires %d (value withheld)\n' "${#value}" "$param"
|
|
fi
|
|
local lower placeholder
|
|
lower=$(printf '%s' "$value" | tr '[:upper:]' '[:lower:]')
|
|
for placeholder in $PLACEHOLDER_SECRETS; do
|
|
if [ "$lower" = "$placeholder" ]; then
|
|
printf 'fail:is a well-known placeholder, not a secret — anyone can guess it (value withheld)\n'
|
|
break
|
|
fi
|
|
done ;;
|
|
esac
|
|
}
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Measure everything, then report. Nothing exits early: one run must name every
|
|
# problem, or fixing them costs one restart each.
|
|
# ---------------------------------------------------------------------------
|
|
|
|
MISSING=()
|
|
MALFORMED=()
|
|
NOTES=()
|
|
OPT_ABSENT=()
|
|
PRESENT_OK=()
|
|
|
|
for (( i = 0; i < ${#SPEC_NAME[@]}; i++ )); do
|
|
name="${SPEC_NAME[$i]}"
|
|
req="${SPEC_REQ[$i]}"
|
|
kind="${SPEC_KIND[$i]}"
|
|
param="${SPEC_PARAM[$i]}"
|
|
desc="${SPEC_DESC[$i]}"
|
|
|
|
lookup "$name"
|
|
|
|
if [ -z "$FOUND" ]; then
|
|
if [ "$req" = "required" ]; then
|
|
MISSING+=("${name}|${desc}")
|
|
else
|
|
OPT_ABSENT+=("${name}|${desc}")
|
|
fi
|
|
continue
|
|
fi
|
|
|
|
if [ -z "$VALUE" ]; then
|
|
if [ "$req" = "required" ]; then
|
|
MISSING+=("${name}|set, but empty — ${desc}")
|
|
else
|
|
# Set-to-empty is not unset, and code tests it both ways: a truthiness
|
|
# check sees nothing, a key-presence check sees it. Reported so the
|
|
# ambiguity is the operator's to resolve rather than this script's.
|
|
OPT_ABSENT+=("${name}|${desc}")
|
|
NOTES+=("${name}|is set to the empty string, which is not the same as unset to code that checks whether the key is present")
|
|
fi
|
|
continue
|
|
fi
|
|
|
|
findings=$(check_shape "$name" "$kind" "$param" "$VALUE")
|
|
had_fail=""
|
|
|
|
while IFS= read -r rline; do
|
|
[ -n "$rline" ] || continue
|
|
case "$rline" in
|
|
fail:*) MALFORMED+=("${name}|${rline#fail:}"); had_fail="yes" ;;
|
|
note:*) NOTES+=("${name}|${rline#note:}") ;;
|
|
esac
|
|
done <<<"$findings"
|
|
|
|
[ -n "$had_fail" ] || PRESENT_OK+=("${name}|$(show "$name" "$kind" "$VALUE")")
|
|
done
|
|
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
# Only for a file. The process environment carries hundreds of variables from
|
|
# the shell, the init system and every tool that ever exported one, so
|
|
# "undeclared" there would be noise; in a file every line was written on
|
|
# purpose, and one that matches nothing in the spec is usually a typo in a
|
|
# name — the thing that makes people swear the value is right there.
|
|
for (( j = 0; j < ${#FILE_KEYS[@]}; j++ )); do
|
|
fkey="${FILE_KEYS[$j]}"
|
|
known=""
|
|
for name in ${SPEC_NAME[@]+"${SPEC_NAME[@]}"}; do
|
|
[ "$name" = "$fkey" ] && known="yes" && break
|
|
done
|
|
if [ -z "$known" ]; then
|
|
if [ "${#fkey}" -gt "$NAME_PRINT_MAX" ]; then
|
|
FILE_NOTES+=("line ${FILE_LINES[$j]}|sets a ${#fkey}-character name that is not declared in SPEC and is too long to be a name at all; the text is withheld in case this is one line of a wrapped multi-line secret")
|
|
else
|
|
FILE_NOTES+=("${fkey}|is set on line ${FILE_LINES[$j]} but is not declared in SPEC — a misspelled name, or a spec that has fallen behind")
|
|
fi
|
|
fi
|
|
done
|
|
fi
|
|
|
|
group() {
|
|
local heading="$1"; shift
|
|
[ $# -gt 0 ] || return 0
|
|
say ""
|
|
say "$heading"
|
|
local e
|
|
for e in "$@"; do
|
|
printf ' %-*s %s\n' "$NAME_WIDTH" "${e%%|*}" "${e#*|}" >&2
|
|
done
|
|
}
|
|
|
|
n_missing="${#MISSING[@]}"
|
|
n_malformed="${#MALFORMED[@]}"
|
|
n_badlines="${#FILE_BAD[@]}"
|
|
n_problems=$(( n_missing + n_malformed + n_badlines ))
|
|
|
|
# `${arr[@]+"${arr[@]}"}` throughout: under `set -u` a plain "${arr[@]}" on an
|
|
# empty array is an error in bash before 4.4, and macOS still ships 3.2.
|
|
if [ -z "$QUIET" ] || [ "$n_problems" -gt 0 ]; then
|
|
say "source: ${SOURCE_LABEL}"
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
say " the process environment was NOT consulted, on purpose."
|
|
fi
|
|
fi
|
|
|
|
if [ -z "$QUIET" ]; then
|
|
group "ok — present and the right shape:" ${PRESENT_OK[@]+"${PRESENT_OK[@]}"}
|
|
group "optional, not set — this is fine:" ${OPT_ABSENT[@]+"${OPT_ABSENT[@]}"}
|
|
group "worth knowing — not failures:" ${NOTES[@]+"${NOTES[@]}"}
|
|
|
|
# Kept apart from the value notes above: these are facts about how the FILE
|
|
# is written, and they apply to whichever loader reads it rather than to any
|
|
# one value being wrong.
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
group "about ${SOURCE_FILE} itself:" ${FILE_NOTES[@]+"${FILE_NOTES[@]}"}
|
|
fi
|
|
|
|
if [ -n "$SOURCE_FILE" ]; then
|
|
for (( i = 0; i < ${#SPEC_KIND[@]}; i++ )); do
|
|
if [ "${SPEC_KIND[$i]}" = "path" ]; then
|
|
say ""
|
|
say "note: path existence was not checked — a file can describe a machine"
|
|
say " that is not this one. Run without --file to check paths here."
|
|
break
|
|
fi
|
|
done
|
|
fi
|
|
fi
|
|
|
|
group "MALFORMED — set, but the wrong shape:" ${MALFORMED[@]+"${MALFORMED[@]}"}
|
|
group "UNREADABLE LINES — these set nothing:" ${FILE_BAD[@]+"${FILE_BAD[@]}"}
|
|
group "MISSING — required, not set:" ${MISSING[@]+"${MISSING[@]}"}
|
|
|
|
if [ "$n_problems" -gt 0 ]; then
|
|
say ""
|
|
say "${n_missing} missing, ${n_malformed} malformed, ${n_badlines} unreadable line(s) in ${SOURCE_LABEL}."
|
|
# Malformed counts as a failure whether the variable was required or
|
|
# optional: not setting an optional variable is a choice, and setting it to
|
|
# something unusable is a mistake that reads as a working configuration.
|
|
say "fix the entries above and run this again — nothing that reads this"
|
|
say "environment can be trusted until then."
|
|
# Exit 1, not die's 2. The difference is the whole contract: 1 means this
|
|
# measured the environment and it is wrong, 2 means it never got far enough
|
|
# to find out. A caller that cannot tell those apart will retry the wrong one.
|
|
exit 1
|
|
fi
|
|
|
|
if [ -z "$QUIET" ]; then
|
|
say ""
|
|
say "ok — ${#PRESENT_OK[@]} set and well-formed, ${#OPT_ABSENT[@]} optional and absent, ${#SPEC_NAME[@]} declared."
|
|
say "shapes only: nothing was connected to, so this says the values are"
|
|
say "plausible, not that the services behind them are up."
|
|
fi
|
|
|
|
exit 0
|