diff --git a/.dockerignore b/.dockerignore index f6d21ec..9f6e408 100644 --- a/.dockerignore +++ b/.dockerignore @@ -26,6 +26,27 @@ db logs *.log +# Client material and operator credentials. None of it belongs in the build +# context, which is sent to the Docker daemon in full: .drop/ holds the original +# site drop and the owner's copy sheets, zoho.md holds the live reCAPTCHA secret +# and the Zoho tokens, and the zips are 30 MB each. The final image copies only +# built output, so none of this has ever shipped, but one careless COPY would +# change that, and every build sends it across for no reason. +.drop +zoho.md +Levi.md +*.zip +*.eml +*.pdf + +# Agent workspaces and local editor state. Gitignored, and no more use to a +# build than they are to the image. +.claude +.codex +.agents +.learnings +*.code-workspace + # Private docs (ignored per requirements) DEVELOPMENT_LOG.md FUTURE.md diff --git a/.githooks/pre-commit b/.githooks/pre-commit index ceffb10..3d6bc94 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -132,9 +132,17 @@ else fi # Said last so it is the thing still on screen when the editor opens. -if ! git diff --quiet; then - say "NOTE: unstaged changes are present. The guards ran against the working" - say " tree, so they did not verify this commit in isolation." +# +# Untracked files count here too, and used not to. The build above runs against +# the WORKING TREE: a new module that is imported but never added makes the +# build pass and the commit itself unbuildable, and `docker build .` would ship +# the file anyway while the tagged commit could not produce it. +untracked=$(git ls-files --others --exclude-standard | wc -l | tr -d ' ') +if ! git diff --quiet || [ "$untracked" != "0" ]; then + say "NOTE: the guards ran against the working tree, so they did not verify" + say " this commit in isolation." + git diff --quiet || say " unstaged changes to tracked files are present." + [ "$untracked" = "0" ] || say " ${untracked} untracked file(s) are present, and the build saw them." fi exit 0 diff --git a/Dockerfile b/Dockerfile index 8fe8219..b72f6a5 100644 --- a/Dockerfile +++ b/Dockerfile @@ -19,6 +19,13 @@ COPY . . ARG VITE_RECAPTCHA_SITE_KEY= ENV VITE_RECAPTCHA_SITE_KEY=$VITE_RECAPTCHA_SITE_KEY +# Sitemap dates, computed by scripts/release.sh where git exists. This build +# has no git: .dockerignore excludes .git and this image ships no git binary. +# Without this the sitemap goes out with no lastmod at all, which is what +# production served for months. +ARG SITEMAP_LASTMOD= +ENV SITEMAP_LASTMOD=$SITEMAP_LASTMOD + # Build the frontend RUN npm run build diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index 852b995..3c07ff8 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -246,6 +246,21 @@ npm run deploy # deploy: move stack 58 to what :dev now points a Both take `--dry-run`, and both refuse rather than guess. Run the dry runs first; they print exactly what would change. +**The image build cannot see git, and the sitemap needs it.** `.dockerignore` +excludes `.git` and the build image has no git binary, so the `lastmod` dates +that come from commit history silently came out empty: **the production sitemap +carried no dates at all** until 2026-09-10. `release.sh` now computes the dates +where git exists and passes them in as the `SITEMAP_LASTMOD` build arg, which the +builder stage of the `Dockerfile` reads. It then asks the built image whether its +sitemap has dates, and refuses to publish one that does not. Search engines +schedule recrawls partly on `lastmod`, and Bing's index feeds Copilot and ChatGPT +search, so an undated sitemap costs visibility on exactly the surfaces this site +was rewritten for. + +`release.sh` also refuses to run while **untracked** files are present, not only +uncommitted ones. `docker build` packs the working tree, so an untracked file +produces an image that works and a tag that cannot rebuild it. + | | | | --- | --- | | Portainer | `https://192.168.1.11:9443` (nebula), API key in `~/.openclaw/credentials/portainer.md` | diff --git a/docs/TOOLS.md b/docs/TOOLS.md index 67de206..3f99424 100644 --- a/docs/TOOLS.md +++ b/docs/TOOLS.md @@ -110,6 +110,13 @@ when `src/`, `server/`, `index.html`, `vite.config.js` or `package.json` is staged. **That is a build, not a test.** It catches a broken import and will not catch a broken behaviour. +It builds the **working tree**, not the commit, so it says so when they differ, +counting untracked files as well as unstaged edits. An untracked module that the +staged code imports makes the build pass and the commit itself unbuildable, and +`docker build` would ship the file anyway while the tag could not rebuild it. +`npm run release` refuses outright while untracked files are present, for the +same reason. + `post-commit` pushes, and that is the intent — but it has a consequence worth holding on to: whatever documentation was not in that commit is now behind the code by one push. That is the mechanical reason `docs/WORK_CYCLE.md` asks for doc diff --git a/docs/architecture/GUARDS.md b/docs/architecture/GUARDS.md index 2479806..c01f0e1 100644 --- a/docs/architecture/GUARDS.md +++ b/docs/architecture/GUARDS.md @@ -109,6 +109,14 @@ tr '\0' '\n' < /proc/$PID/environ | grep -c . # 0 here means "could not read" This is the confident-absence failure one level up: the same trap as a screen rendering a failed query as a count of zero, applied to your own diagnosis. +**The same trap inside a data source.** `scripts/prerender.js` read sitemap dates +from `git log` inside a `try` whose `catch` was empty and commented *"git +unavailable or file untracked"*. Inside the image build git is always +unavailable, so every date came out empty and the sitemap shipped with none at +all, for months, while the build printed a cheerful success line. The fix is two +parts and both matter: pass the data in from where it exists, and **say the +count out loud on every run**, so "18 URLs, 0 dated" cannot read as success. + **The same trap inside a scanner.** A matcher that discards its errors turns "could not run" into "found nothing". `grep "$pattern" 2>/dev/null` answers a pattern it cannot read with exit 2 and no output, and a loop reading its matches diff --git a/scripts/lib/routes.js b/scripts/lib/routes.js index ddbf29e..cb61afa 100644 --- a/scripts/lib/routes.js +++ b/scripts/lib/routes.js @@ -9,9 +9,14 @@ // // Plain JavaScript with no side effects, because prerender imports it in Node // before Vite exists, and so does the validator runner. +import { execFileSync } from 'child_process' +import path from 'path' +import { fileURLToPath } from 'url' import { services } from '../../src/data/services.js' import { industries } from '../../src/data/industries.js' +const repoRoot = path.join(path.dirname(fileURLToPath(import.meta.url)), '../..') + export const STATIC_ROUTES = [ '/', '/about', @@ -46,7 +51,82 @@ export const routerPaths = (table, base = '') => export const routeDrift = (table) => [ ...new Set( routerPaths(table) - .filter((path) => !path.includes(':') && !path.includes('*')) - .filter((path) => !ROUTES.includes(path)), + .filter((route) => !route.includes(':') && !route.includes('*')) + .filter((route) => !ROUTES.includes(route)), ), ] + +// --- sitemap dates ----------------------------------------------------------- +// +// The files that produce each page. A page's `lastmod` is the newest commit +// date among them, never the build timestamp: a sitemap that marks every page +// as changed on every deploy is one search engines learn to ignore. + +const ROUTE_SOURCES = { + '/': ['src/pages/Home.jsx'], + '/about': ['src/pages/About.jsx'], + '/services': ['src/pages/Services.jsx', 'src/data/services.js'], + '/industries': ['src/pages/Industries.jsx', 'src/data/industries.js'], + '/contact': ['src/pages/Contact.jsx'], + '/support': ['src/pages/Support.jsx'], + '/privacy-policy': ['src/pages/PrivacyPolicy.jsx', 'src/data/privacyPolicy.js'], +} + +export const sourcesFor = (url) => { + if (ROUTE_SOURCES[url]) return ROUTE_SOURCES[url] + if (url.startsWith('/services/')) { + // A page with owner-approved copy has its own content file, so editing that + // copy moves that page's date and no other. + const slug = url.slice('/services/'.length) + return ['src/pages/ServiceDetail.jsx', 'src/data/services.js', `src/data/serviceContent/${slug}.js`] + } + return ['src/pages/IndustryDetail.jsx', 'src/data/industries.js'] +} + +const gitLastModified = (files) => { + let newest = null + for (const file of files) { + try { + const iso = execFileSync('git', ['log', '-1', '--format=%cI', '--', file], { + cwd: repoRoot, + encoding: 'utf8', + stdio: ['ignore', 'pipe', 'ignore'], + }).trim() + if (iso && (!newest || iso > newest)) newest = iso + } catch { + // git is unavailable, or the file is untracked. Either way, no date from + // this file. Whether that leaves the ROUTE undated is the caller's + // problem to report, and it must not pass silently: the production + // sitemap carried no dates at all for months because this catch was the + // end of the story. + } + } + return newest ? newest.slice(0, 10) : null +} + +/** + * Route to YYYY-MM-DD, for the sitemap. + * + * The image build has no git: `.dockerignore` excludes `.git` and node:alpine + * ships no git binary. So a map computed where git DOES exist can be injected + * through SITEMAP_LASTMOD, which is what scripts/release.sh does. + */ +export const lastModByRoute = () => { + const injected = process.env.SITEMAP_LASTMOD + if (injected) { + try { + const parsed = JSON.parse(injected) + if (parsed && typeof parsed === 'object') return parsed + console.warn('routes: SITEMAP_LASTMOD is not an object, so falling back to git.') + } catch (error) { + console.warn(`routes: SITEMAP_LASTMOD is not valid JSON (${error.message}), so falling back to git.`) + } + } + + const dates = {} + for (const route of ROUTES) { + const date = gitLastModified(sourcesFor(route)) + if (date) dates[route] = date + } + return dates +} diff --git a/scripts/prerender.js b/scripts/prerender.js index f89015c..7a4f188 100644 --- a/scripts/prerender.js +++ b/scripts/prerender.js @@ -13,7 +13,6 @@ // // Run automatically as part of `npm run build`. -import { execFileSync } from 'child_process' import { mkdirSync, readFileSync, writeFileSync } from 'fs' import path from 'path' import { fileURLToPath } from 'url' @@ -21,7 +20,7 @@ import { fileURLToPath } from 'url' import { render, routes as routerTable } from '../dist-ssr/entry-server.js' import { services } from '../src/data/services.js' import { industries } from '../src/data/industries.js' -import { ROUTES as routes, STATIC_ROUTES, routeDrift } from './lib/routes.js' +import { ROUTES as routes, lastModByRoute, routeDrift } from './lib/routes.js' import { validateContent } from './lib/content.js' const __dirname = path.dirname(fileURLToPath(import.meta.url)) @@ -210,36 +209,6 @@ for (const [url, size] of written) { const SITE_URL = 'https://queuenorth.com' -// lastmod comes from git history for the files that produce each route. A build -// timestamp would mark every page as changed on every deploy, which search engines -// learn to distrust. -const lastModifiedFor = (files) => { - let newest = null - for (const file of files) { - try { - const iso = execFileSync('git', ['log', '-1', '--format=%cI', '--', file], { - cwd: path.join(__dirname, '..'), - encoding: 'utf8', - stdio: ['ignore', 'pipe', 'ignore'], - }).trim() - if (iso && (!newest || iso > newest)) newest = iso - } catch { - // git unavailable or file untracked — fall through - } - } - return newest ? newest.slice(0, 10) : null -} - -const ROUTE_SOURCES = { - '/': ['src/pages/Home.jsx'], - '/about': ['src/pages/About.jsx'], - '/services': ['src/pages/Services.jsx', 'src/data/services.js'], - '/industries': ['src/pages/Industries.jsx', 'src/data/industries.js'], - '/contact': ['src/pages/Contact.jsx'], - '/support': ['src/pages/Support.jsx'], - '/privacy-policy': ['src/pages/PrivacyPolicy.jsx', 'src/data/privacyPolicy.js'], -} - const PRIORITY = { '/': '1.0', '/services': '0.9', @@ -252,15 +221,13 @@ const PRIORITY = { const CHANGEFREQ = { '/': 'weekly', '/privacy-policy': 'yearly' } -const sourcesFor = (url) => { - if (ROUTE_SOURCES[url]) return ROUTE_SOURCES[url] - if (url.startsWith('/services/')) return ['src/pages/ServiceDetail.jsx', 'src/data/services.js'] - return ['src/pages/IndustryDetail.jsx', 'src/data/industries.js'] -} +// Dates come from git where git exists, and from the map release.sh injects +// where it does not. See scripts/lib/routes.js. +const lastmodByRoute = lastModByRoute() const entries = routes.map((url) => { const loc = url === '/' ? SITE_URL : `${SITE_URL}${url}` - const lastmod = lastModifiedFor(sourcesFor(url)) + const lastmod = lastmodByRoute[url] ?? null return [ ' ', ` ${loc}`, @@ -282,4 +249,15 @@ const sitemap = [ ].join('\n') writeFileSync(path.join(distDir, 'sitemap.xml'), sitemap) -console.log(`\nGenerated sitemap.xml with ${routes.length} URLs`) + +// Say how many pages carry a date, every time. The production sitemap carried +// none at all for months: the image build has no git, and the failure to read +// it was swallowed. Silence is what let that run. +const dated = routes.filter((url) => lastmodByRoute[url]).length +console.log(`\nGenerated sitemap.xml with ${routes.length} URLs, ${dated} carrying a lastmod`) +if (dated < routes.length) { + console.warn( + `prerender: ${routes.length - dated} URL(s) have no lastmod. git history is not readable here, which is ` + + 'normal inside the image build. Pass SITEMAP_LASTMOD, as scripts/release.sh does.', + ) +} diff --git a/scripts/release.sh b/scripts/release.sh index 075b6cd..fb63f75 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -215,6 +215,18 @@ if ! git diff --quiet || ! git diff --cached --quiet; then cannot tell your work in progress from a release." fi +# Untracked files are checked separately, and they matter more than they look. +# `docker build .` packs the WORKING TREE, not the commit: an untracked module +# that the code imports produces an image that works and a tagged commit that +# cannot be rebuilt. The tag is supposed to be the record of what shipped. +untracked=$(git ls-files --others --exclude-standard) +if [ -n "$untracked" ]; then + say "the working tree has untracked files:" + printf '%s\n' "$untracked" | sed 's/^/ /' >&2 + die "add or remove them first. docker build packs the working tree, so these + would go into the image while the tag could not rebuild it." +fi + current=$(node -p "require('./package.json').version" 2>/dev/null) \ || stop "could not read the version from package.json." @@ -265,8 +277,9 @@ if [ -n "$DRY_RUN" ]; then say "--dry-run: nothing was changed. It would have:" say " set version ${next} in ${FILES[*]}" say " bash scripts/verify.sh" - say " docker build --build-arg APP_VERSION=${next} -t ${IMAGE}:${TAG} ." + say " docker build --build-arg APP_VERSION=${next} --build-arg SITEMAP_LASTMOD= -t ${IMAGE}:${TAG} ." say " verify the image's org.opencontainers.image.version label reads ${next}" + say " verify the image's sitemap carries dates" say " verify the built bundle contains the reCAPTCHA site key" say " docker push ${IMAGE}:${TAG}" say " git commit -m 'chore(release): ${TAG}' (post-commit then pushes)" @@ -327,9 +340,21 @@ say " loaded a page in a browser." # --------------------------------------------------------------------------- say "building ${IMAGE}:${TAG}…" +# Sitemap dates come from git history, and the image build cannot see it: +# .dockerignore excludes .git and node:alpine ships no git binary. Computing the +# map here, where git exists, and passing it in is what puts dates in the +# production sitemap. Without it every URL goes out undated, which is exactly +# what production served until 2026-09-10. +sitemap_lastmod=$(node -e "import('./scripts/lib/routes.js').then(m => console.log(JSON.stringify(m.lastModByRoute())))" 2>/dev/null) +if [ -z "$sitemap_lastmod" ] || [ "$sitemap_lastmod" = "{}" ]; then + die "could not compute sitemap dates from git history, so the image would ship an + undated sitemap. Run this from a full clone, not an export." +fi + if ! docker build \ --build-arg "APP_VERSION=${next}" \ --build-arg "VITE_RECAPTCHA_SITE_KEY=${VITE_RECAPTCHA_SITE_KEY:-}" \ + --build-arg "SITEMAP_LASTMOD=${sitemap_lastmod}" \ -t "${IMAGE}:${TAG}" . ; then say "build failed. The version bump is in your working tree and NOTHING was" say " published or committed. 'git checkout -- ${FILES[*]}' undoes it." @@ -360,6 +385,25 @@ fi # makes, for the value that actually stops the product working. This survives # somebody editing the Dockerfile's ARG/ENV pair or adding dist to # .dockerignore, neither of which would fail the build. +say "verifying the image's sitemap carries dates…" + +# Ask the artifact, not the wiring. A missing build arg, a typo in the +# Dockerfile's ARG/ENV pair, or a .dockerignore change would each leave the +# sitemap undated while the build still succeeds. +dated=$(docker run --rm --entrypoint sh "${IMAGE}:${TAG}" -c \ + "grep -c '' /app/dist/sitemap.xml" 2>/dev/null | tr -d '\r\n') + +if [ "${dated:-0}" -lt 1 ]; then + docker rmi "${IMAGE}:${TAG}" >/dev/null 2>&1 + say "the image's sitemap carries no lastmod at all. Search engines schedule" + say " recrawls on it, and Bing's index feeds Copilot and ChatGPT search." + say " Check ARG/ENV SITEMAP_LASTMOD in the builder stage of the" + say " Dockerfile. Nothing was published or committed; the local image" + say " was removed." + exit 1 +fi +say " ${dated} URL(s) dated." + say "verifying the bundle carries the reCAPTCHA site key…" if ! docker run --rm --entrypoint sh "${IMAGE}:${TAG}" -c \