BillTracker/docs/Engineering_Reference_Manual.md

# Engineering Reference Manual — Bill Tracker

**Status:** Current code reference
**Last Updated:** 2026-07-05
**Version:** 0.40.0 (package version; the in-flight Track A/B/C/D commits target **v0.41.0** — see HISTORY.md and the `Last Updated` note below)
**Primary stack:** Node.js 22 + Express 4 (CommonJS), React 19 + Vite 5 (TypeScript, `strict` + `noUncheckedIndexedAccess`), Tailwind CSS 3 + shadcn/ui, TanStack Query 5, Sonner, SQLite via `better-sqlite3` 12, `@simplewebauthn/*` 13, `otplib` 13, `openid-client` 5

This manual reflects the current application code in `server.js`, `routes/`, `services/`, `middleware/`, `db/`, `client/`, `package.json`, `Dockerfile`, `docker-compose.yml`, `tsconfig.json`, `eslint.config.mjs`, and `vite.config.mjs`. It is written as a current-state reference, not a changelog. The previous version of this manual (v0.28.1) is fully superseded.

Notable changes since the previous manual:

- **Client migrated JSX → TSX.** Full strict TypeScript with `noUncheckedIndexedAccess`. `allowJs` + `checkJs:false` keep legacy `.js` running while `.ts/.tsx` are type-checked. `npm run typecheck` wired into CI.
- **Branded money types.** `client/lib/money.ts` exposes `Cents` / `Dollars` branded types — a typed caller can no longer format cents as dollars (the 100× bug) by accident.
- **React 18 → React 19** with `babel-plugin-react-compiler` for automatic memoization. Vite 5 with `vite-plugin-pwa` for service-worker / installable PWA.
- **Node 18 → Node 22** in Dockerfile; `node:22-bookworm` in CI.
- **New features:** SimpleFIN bank sync, WebAuthn + TOTP 2FA, calendar feed (`/api/calendar/feed.ics`), subscription catalog + recommendations, spending categories / budgets, snowball plans (lifecycle), push notifications (ntfy / Gotify / Discord / Telegram), push notification column on users, encrypted secrets at rest with HKDF-derived key, money-cents migration (in progress), category groups, banker-rule auto-attribute.
- **Schema:** migrations `v0.2` → `v1.06`. Rollback map covers `v0.44` → `v1.04`.

Since the v0.40.0 manual shipped, **8 in-flight commits** targeting v0.41.0 have been added locally and are documented in this manual ahead of push:

- **Track A (money integrity):** every payment balance-mutation now runs in a single transaction. `POST /payments` returns `409 { code: 'DUPLICATE_SUSPECTED' }` instead of silently deduping when a live payment matches `(bill_id, paid_date, amount)`; the client surfaces a sonner "Add anyway?" toast via `client/lib/paymentActions.ts → createPaymentOrConfirm` and resends with `allow_duplicate: true` to confirm.
- **Track B (tests):** `tests/paymentsRoute.test.js` (route + transactional behavior) and `tests/reconciliation.test.js` (cross-surface reconciliation harness).
- **Track C (API response typing):** high-traffic domains (`auth`, `snowball`, `subscriptions`, `spending`, `bank-ledger`) promoted page-local shapes into `client/types.ts` and wired through `get<T>` / `post<T>`. `User` moved from `@/hooks/useAuth` to `@/types` (re-exported for 7 importers). The single-consumer non-money casts (`adminUsers` and the one-off `version` / `about` / `privacy` / `health` / `importHistory` doc shapes) are deliberately left as-is.
- **Track D (server error-shape consistency):** the global 500 handler in `server.js` now emits `{ error, code: 'INTERNAL_ERROR' }` so an unexpected server error carries the same `{error, code}` field the client can switch on as every structured 4xx.

---

## 1. System Overview

Bill Tracker is a self-hosted bill management application. It supports:

- Local username/password authentication, optional single-user mode, optional Authentik/OIDC login, and optional WebAuthn / FIDO2 + TOTP two-factor authentication.
- User-scoped bills, categories (with `category_groups` and `spending_enabled` flag), payments, monthly bill overrides, monthly income, and starting cash buckets.
- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, migration rollback, and SimpleFIN bank-sync admin configuration.
- Spreadsheet, user-SQLite, OFX, and CSV transaction import workflows with preview/apply split.
- CSV, Excel, and user-SQLite export workflows; an iCalendar feed (`/api/calendar/feed.ics`) of upcoming bills gated by per-user tokens.
- SMTP-based bill due notifications plus ntfy / Gotify / Discord / Telegram push notifications.
- SimpleFIN-based bank sync (data sources, financial accounts, transactions, match suggestions, auto-categorize, merchant rules, merchant-store matches).
- Subscription catalog (top-200 known services), per-user recommendations, declined-subscription hints, recommendation feedback.
- Spending module: categorized transactions, rules, monthly budgets, income breakdown, copy-month workflow.
- Snowball plans: lifecycle (active / paused / completed / abandoned), debt-order projection (snowball/avalanche/minimum-only), payoff scenarios.
- React 19 SPA in TypeScript with branded `Cents` / `Dollars` money types, lazy-loaded routes, global QueryCache error toasts, and CSRF-protected JSON APIs.

Runtime flow:

1. `server.js` initializes SQLite through `db/database.js`, runs `db/migrations/versionedMigrations.js` plus `legacyReconcileMigrations.js`, seeds defaults/admin user, cleans expired sessions, then starts Express.
2. Express applies `securityHeaders`, optional CORS, `express.json({ limit: '100kb' })`, `cookieParser`, `csrfTokenProvider`, the patched `res.json` error formatter, and an unauthenticated `GET /api/health` probe.
3. Route files under `routes/` validate input, enforce ownership through `req.user.id`, and use service modules for business logic. WebAuthn and TOTP challenges use a 5–15 minute TTL in `webauthn_challenges` and `totp_challenges`.
4. React 19 under `client/` calls `/api/*` through `client/api.ts` with `credentials: include` and CSRF headers on mutating methods; the CSRF token is fetched once from `/api/auth/csrf-token` and cached in memory.

---

## 2. Project Layout

- `server.js` — Express entry point and route mounting.
- `routes/` — HTTP API handlers:
  - `auth.js` — login, logout, logout-all, password change, TOTP setup/enable/disable/challenge/status, WebAuthn status, login history, CSRF token endpoint, mode, acknowledge-version/privacy.
  - `authOidc.js` — OIDC login and callback.
  - `bills.js` — bills CRUD, auto-mark paid, history ranges, soft-delete, recently-deleted restore.
  - `payments.js` — payments CRUD, status matching, snowball handling, bank-override accounting, bulk/quick.
  - `categories.js` — category CRUD, sort order, spending/bill split, category groups.
  - `tracker.js` — monthly tracker data, bucket resolution, cycle handling, drift, autopay suggestions, amount suggestions, safe-to-spend, three-month averages.
  - `summary.js` — summary stats, income, starting amounts, bank-tracking overlay, skip override.
  - `analytics.js` — expense reports, category breakdown, month-window filtering.
  - `settings.js` — user settings, admin config, notification settings.
  - `notifications.js` — notification management (admin SMTP + push).
  - `profile.js` — user profile, notification settings, change-password, export URLs, import history, login history.
  - `import.js` — CSV / Excel / user-SQLite / OFX import workflows.
  - `export.js` — CSV, Excel, SQLite export.
  - `status.js` — system status, health, runtime.
  - `dataSources.js` — data sources CRUD with sync status (manual / file_import / provider_sync).
  - `transactions.js` — transaction CRUD, match/ignore/commit actions, manual entry, pending flag, geolocation.
  - `matches.js` — match suggestions, rejection tracking.
  - `snowball.js` — snowball projection, settings, debt order, **plan lifecycle** (create / list / active / pause / resume / complete / abandon).
  - `subscriptions.js` — user subscriptions, **recommendations** (list / decline / match-bill / create), transaction matches, catalog links, custom descriptors.
  - `spending.js` — categorized spending summary, transactions, budgets (PUT / copy / month-rollover), category rules, income.
  - `calendarFeed.js` — public iCalendar feed (`GET /feed.ics`) per-user token-gated.
  - `admin.js` — users (incl. username change), backups, cleanup, auth-mode, **bank-sync-config**, migrations rollback.
  - `about.js`, `aboutAdmin.js` — public + admin markdown about view.
  - `privacy.js` — public privacy page.
  - `version.js` — public version, history, update-status.
- `services/` — business logic modules. See §4.
- `middleware/` — auth guards, CSRF, rate limits, security headers, error formatting.
- `db/schema.sql` — base SQLite schema.
- `db/database.js` — DB connection, migrations runner, defaults, settings, rollback support.
- `db/migrations/versionedMigrations.js` — ordered, dependency-aware migrations `v0.2` → `v1.06`.
- `db/migrations/legacyReconcileMigrations.js` — column-level reconciliation for unversioned columns that pre-date the migration framework.
- `db/subscriptionCatalogSeed.js` — top-200 US subscriptions dataset used by `v0.65` / `v0.69` / `v0.95`.
- `workers/dailyWorker.js` — scheduled notification/cleanup worker entry.
- `client/` — React 19 SPA in TypeScript (see §7).
- `dist/` — generated Vite build output served by Express.
- `Dockerfile`, `docker-compose.yml`, `docker-entrypoint.sh` — container deployment.
- `tsconfig.json` — TypeScript config (strict + `noUncheckedIndexedAccess`, `allowJs`).
- `eslint.config.mjs` — flat ESLint config, `react-hooks` + `react-refresh` + `typescript-eslint`.
- `vite.config.mjs` — Vite + React Compiler + PWA + Vitest config.

---

## 3. Backend Entry Point and Middleware

### `server.js`

Server defaults:

- Port: `PORT || 3000`.
- Bind host: `BIND_HOST || '0.0.0.0'`.
- Static frontend: `dist/`.
- Optional CORS: enabled when `CORS_ORIGIN` is set; comma-separated allowlist with `credentials: true`.
- `TRUST_PROXY` (env): when set to `true`/`yes`/`on` trusts 1 hop; numeric → trust that many hops; otherwise the raw value (e.g. `loopback`, CIDR). Unset/false = no proxy trust. Required behind nginx/Traefik so `req.ip` and `req.secure` reflect the real client.
- Session cleanup: runs on startup and every `SESSION_CLEANUP_INTERVAL_MS || 86400000` ms.
- Admin seed: `INIT_ADMIN_USER || admin`; `INIT_ADMIN_PASS` or generated/default behavior in `db/database.js`.
- Environment variables `INIT_ADMIN_USER` and `INIT_ADMIN_PASS` (or `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`) skip the first-login flow entirely by pre-seeding users with `first_login=0` flags via `setup/firstRun.js`.
- Optional regular-user seed: `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`; password must be at least 8 chars.
- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked.
- After `app.listen`, `cleanupExpiredSessions` is scheduled, `bankSyncWorker.start()` runs, the backup scheduler is started, and `workers/dailyWorker.start()` schedules the daily jobs.

Global middleware order:

1. `securityHeaders`
2. optional `cors` (only if `CORS_ORIGIN`)
3. `express.json({ limit: '100kb' })` — import routes override this per-endpoint
4. `cookieParser()`
5. `csrfTokenProvider`
6. `errorFormatter` (patches `res.json`)
7. `GET /api/health` — unauthenticated liveness probe
8. mounted API routers with route-level rate-limit/auth/CSRF middleware
9. `GET /login.html` → 302 `/login`, `express.static(DIST)`
10. SPA fallback `GET *` serving `dist/index.html` after ensuring a CSRF token cookie
11. final JSON error handler for malformed JSON / body-size / runtime errors

### Authentication middleware

`middleware/requireAuth.js` exports:

- `requireAuth`: attaches `req.user` from `bt_session`; in single-user mode attaches the configured active regular user without a session.
- `requireUser`: permits `user` and `admin` roles but blocks the default admin account from tracker access.
- `requireAdmin`: requires `req.user.role === 'admin'`.

### CSRF middleware

`middleware/csrf.js`:

- Cookie name: `CSRF_COOKIE_NAME || bt_csrf_token`.
- Header name: `x-csrf-token`.
- Defaults: `CSRF_HTTP_ONLY === true` only when explicitly configured, `CSRF_SAME_SITE || strict`, `CSRF_SECURE !== false`.
- The SPA expects `CSRF_HTTP_ONLY=false` so `client/api.ts` can read the token cookie and send `x-csrf-token`; do not enable httpOnly CSRF cookies unless token delivery changes.
- The SPA also calls `GET /api/auth/csrf-token` (handled by `routes/auth.js`) on first request and caches the token in memory; the cookie is the source of truth and the body mirrors it.
- `csrfTokenProvider` sets a token cookie on responses.
- `csrfMiddleware` validates mutating requests unless `req.csrfSkip` is set. Token may come from header, query, or body and must match cookie.
- Failures return 403 and audit `csrf.failure`.

### Rate limits

Defined in `middleware/rateLimiter.js`:

- Login: 10 / 15 min.
- Login-by-username (extra layer): separate limiter, applied alongside `loginLimiter`.
- Password changes: 5 / 15 min.
- Import: 20 / 15 min.
- Export: 30 / 15 min.
- Admin actions: 30 / 15 min.
- OIDC: 20 / 15 min.
- Backup operations: 5 / 60 min.
- Demo-data clear: 3 / 15 min.

A `skipRateLimitIfNoUsers(limiter)` wrapper is applied to `/api/auth/login` so a brand-new install with zero users is not locked out by the login limiter.

Rate-limit responses are JSON: `{ "error": "..." }`.

### Security headers

`middleware/securityHeaders.js` sets CSP with a per-request nonce plus common hardening headers. Static SPA scripts/styles must comply with the generated CSP.

### Error formatting

`middleware/errorFormatter.js` standardizes route responses into JSON with `error`, `code`, and optional `field`. Common statuses: 400 validation, 401 auth, 403 forbidden, 404 not found, 409 conflict, 429 rate limit, 500 server error.

---

## 4. Services

### `services/authService.js`

- Cookie: `bt_session`.
- Session lifetime: 7 days. Session tokens are **hashed at rest** in `sessions` (column populated in `v0.94`); the cookie value is a random opaque ID and is looked up by hash.
- Password hashing: bcrypt, cost 12.
- Functions:
  - `login(username, password)` — verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updates `last_login_at`, and returns `{sessionId, user}` or `null`.
  - `createSession(userId)` — creates a session for an OIDC-provisioned or existing local user after validating that the user is active.
  - `logout(sessionId)` — deletes one session.
  - `getSessionUser(sessionId)` — returns public user fields when the session exists, is unexpired, and belongs to an active user.
  - `rotateSessionId(oldSessionId, userId)` — validates the current session, deletes it, and inserts a replacement session in a transaction. Password changes use this to prevent session fixation.
  - `invalidateOtherSessions(userId, keepSessionId)` — deletes all sessions for a user except the supplied current session; when `keepSessionId` is `null`, deletes every session for that user.
  - `pruneExpiredSessions()` — deletes expired sessions.
  - `publicUser(user)` — strips password/session secrets and normalizes booleans.

Password changes through `/api/auth/change-password` update the password hash, clear `must_change_password`, stamp `last_password_change_at`, invalidate all other sessions, rotate the current session ID when a valid session cookie is present, set a replacement `bt_session` cookie, and audit `password.change`.

`/api/auth/logout-all` calls `invalidateOtherSessions(req.user.id, null)`, deletes the current cookie session if present, audits `logout.all`, clears `bt_session`, and returns `{success:true}`.

### `services/totpService.js`

TOTP / RFC-6238 authenticator-app 2FA:

- `newSecret()` — `otplib` 20-char base32 secret.
- `generateQrCode(secret, username)` — returns `{uri, qr_data_url}` for the SPA to display.
- `verifyToken(encryptedSecret, token)` — decrypts the per-user secret, strips whitespace, calls `verifySync`.
- Setup flow: `getTotpSetup` → user scans QR → `enableTotp(plainSecret, token)` validates a sample token, encrypts and stores the secret, returns 8 recovery codes.
- `disableTotp(userId)` — clears `totp_enabled`, `totp_secret`, recovery codes.
- Recovery codes are stored hashed in `users.totp_recovery_codes` (JSON).
- `totpStatus(userId)` returns `{enabled, hasSecret, recoveryCodesRemaining}`.
- Login flow: `POST /api/auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id` stored in `totp_challenges`; the SPA submits it with the 6-digit code on the next request.

### `services/webauthnService.js`

WebAuthn / FIDO2 security-key 2FA via `@simplewebauthn/server`:

- `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`).
- `createRegistrationChallenge(db, userId, username)` — generates a 32-byte `webauthn_user_id` (base64url), excludes already-registered credentials, stores the challenge in `webauthn_challenges` with 15-minute TTL, returns `{options, challengeId}`.
- `verifyRegistration(db, userId, challengeId, response, credentialName)` — verifies the registration, stores the credential (`credential_id`, public key, sign count, transports, backup flags, AAGUID, friendly name) and returns the credential id.
- `createAuthenticationChallenge(db, userId)` — for login: returns assertion options + challengeId.
- `verifyAuthentication(db, userId, challengeId, response)` — verifies, updates sign count, audits `webauthn.login`.
- Supports `attestationType: 'none'`, `residentKey: 'preferred'`, `userVerification: 'preferred'`, algorithms `-7` (ES256) and `-257` (RS256).
- `WEBAUTHN_RP_ID` must match the browser-visible hostname, or registration/assertion will fail.

### `services/oidcService.js`

Handles Authentik/OIDC login with `openid-client`:

- Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag.
- The OIDC client secret is **encrypted at rest** (migration `v0.79`) using `services/encryptionService` (HKDF-derived key from `v0.78`).
- Builds PKCE login state in `oidc_states`.
- Discovers provider and caches OIDC client for 1 hour.
- Exchanges callback code, verifies nonce/state, maps claims to local user.
- Role mapping uses configured admin group; default role is `user`.
- Client secret is write-only through admin settings.

### `services/encryptionService.js`

- `assertEncryptionReady()` — throws if no master key is configured.
- `encryptSecret(plain)` / `decryptSecret(cipher)` — symmetric encryption with HKDF-derived key.
- The key derivation was upgraded from SHA-256 to HKDF in `v0.78`; migrations `v0.77` (SMTP password) and `v0.79` (OIDC secret) wrapped any pre-existing plaintexts.
- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected.

### `services/loginFingerprint.js`

- Captures `ip`, `user_agent`, `browser`, `os`, `device_type`, `device_fingerprint` on every login.
- The `ip` and `user_agent` columns are **encrypted at rest** (migration `v0.84`).
- `success` (`v0.85`) tracks failed-attempt history separately from successful logins.
- A `session_fingerprint` is stored on each successful login so the SPA can detect a different device for the same session and trigger a re-auth challenge.

### `services/backupService.js`

- Backup directory: `BACKUP_PATH || backups/`.
- Valid backup IDs match managed prefixes only: `bill-tracker-backup`, `pre-restore`, `imported-backup`, `scheduled-backup`.
- Creates SQLite backups with checksum and metadata.
- Validates uploads by SQLite `integrity_check` and optional SHA-256 checksum.
- Restore creates a pre-restore backup before swapping DB.
- Path traversal is prevented by ID regex and `path.relative` checks.

### `services/backupScheduler.js`

- Validates daily/weekly schedule, `HH:MM` time, retention count 1-365.
- Stores schedule settings in `settings`.
- Uses `node-cron`, supports run-now, reload, next-run status, and retention cleanup.

### `services/cleanupService.js`

Cleanup tasks:

- Expired import sessions.
- Stale temp SQLite export files in OS temp dir.
- Orphan `.partial`/`.upload` backup files.
- Optional old import-history pruning.

Settings are stored in `settings`; run results are stored as JSON.

### `services/notificationService.js`

- Builds SMTP transport from global notification settings (SMTP password is **encrypted at rest**, migration `v0.77`).
- Sends test email to an admin-provided address.
- Runs due-bill notifications for 3 days, 1 day, due today, and overdue.
- De-duplicates sends via `notifications` unique key.
- Recipients come from user notification settings when enabled/allowed or global recipient settings.
- Push notifications: ntfy, Gotify, Discord webhook, Telegram bot — configured per-user in `users` columns (migration `v0.80`); dispatched alongside or in place of SMTP when configured.

### `services/spreadsheetImportService.js`

- Accepts XLSX buffers only, max 10 MB, max 5,000 rows.
- Detects monthly sheets, headers, categories, bills, amounts, and ambiguous rows.
- Creates `import_sessions` preview records with 24-hour TTL.
- Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history.

### `services/ofxImportService.js`

- OFX / QFX bank-statement import (separate from CSV/Excel). Uses `node-ofx-parser`; falls back to manual buffer parsing when the package is not installed.
- Produces `import_sessions` previews and commits to `transactions` with `source_type='file_import'`.
- 24-hour TTL on preview sessions; results write to `import_history`.

### `services/userDbImportService.js`

- Accepts user SQLite export files up to 50 MB.
- Requires export metadata and known tables.
- Sanitizes categories, bills, payments, monthly state, and starting amounts.
- Preview stores an import session; apply maps export IDs to current user-owned IDs.

### `services/transactionService.js`

Transaction data source and transaction row helpers:

- `ensureManualDataSource(db, userId)` creates/retrieves a user-specific manual data source (`type='manual', provider='manual', name='Manual Entry'`).
- `decorateDataSource(row)` removes `encrypted_secret`, adds `source_label` and `source_type_label`.
- `decorateTransaction(row)` adds `source_label`, `source_type_label`, and embedded `data_source` object with safe fields.
- `getSourceTypeLabel(type)` returns labels: `manual` → Manual, `file_import` → File import, `provider_sync` → Provider sync.
- `sourceLabel(source)` constructs human-readable source labels for manual entries or provider names.
- `transactions` schema carries a `pending` flag (`v1.01`) for SimpleFIN pending transactions.

### `services/transactionMatchService.js` + `services/transactionMatchState.js`

- `matchTransactionToBill(transactionId, billId, userId)` — links a transaction to a bill, creates a corresponding `payments` row with `payment_source='provider_sync'`, runs accounting override logic.
- `unmatchTransaction(transactionId, userId)` — reverses the link and deletes the auto-created payment (or marks it `accounting_excluded`).
- `ignoreTransaction` / `unignoreTransaction` — set/match-status transitions.
- `transactionMatchState.js` is the state-machine helper that powers match/unmatch/ignore/unignore.

### `services/matchSuggestionService.js`

- `listMatchSuggestions(userId, transactionId, billId, limit, offset)` — returns scored suggestions.
- `rejectMatchSuggestion(userId, transactionId, billId)` — records the rejection in `match_suggestion_rejections`.
- `suggestionCounts(userId)` — dashboard counts.
- Rejections are stored in `match_suggestion_rejections` (created in `v0.62`) so a transaction/bill pair is never re-suggested after rejection.
- After sync, `autoMatchForUser(userId)` runs merchant-rule + merchant-store-match + match-suggestion passes in order.

### `services/csvTransactionImportService.js`

CSV import workflow for transactions:

- Parses CSV with quoted-field support and quote doubling.
- `previewCsvTransactions(userId, buffer, options)` returns headers, sample rows, suggested field mapping, errors, and creates a 24-hour TTL import session (max 25k rows).
- `suggestMapping(headers)` auto-detects field mappings from header names against `posted_date`, `transacted_at`, `amount`, `description`, `payee`, `memo`, `category`, `account`, `transaction_type`, `currency`, etc.
- `commitCsvTransactions(userId, importSessionId, mapping)` imports rows into the `transactions` table with `source_type='file_import'`, auto-creates a CSV data source and financial accounts per unique account name.
- Stable deduplication via SHA-256 hash: `csv:id:` prefix for explicit transaction IDs, `csv:hash:` prefix from date+amount+description+payee+account.
- Imports record to `import_history` with counts and details.
- `FIELD_LABELS` maps field keys to user-friendly labels for validation messages.

### `services/bankSyncService.js` + `services/simplefinService.js` + `services/bankSyncConfigService.js` + `services/bankSyncWorker.js`

- **SimpleFIN bridge** (`simplefinService.js`): wraps the SimpleFIN protocol — `claimSetupToken`, `fetchAccountsAndTransactions`, `normalizeAccount`, `normalizeTransaction`, `sanitizeErrorMessage`. `assertEncryptionReady` is invoked before any secret operation.
- **Per-user sync** (`bankSyncService.js`): upserts `financial_accounts` rows keyed by `(data_source_id, provider_account_id)`; upserts `transactions` keyed by `(data_source_id, provider_transaction_id)`. Honors the `monitored` flag (`v0.64`) so the user can exclude a credit-card account from bill matching. Applies merchant rules, spending-category rules, merchant-store matches, then `autoMatchForUser` in sequence.
- **Config** (`bankSyncConfigService.js`): `getBankSyncConfig(userId)`, `SYNC_DAYS_EFFECTIVE`, `SYNC_DAYS_DEFAULT`. Per-user default-lookback window.
- **Worker** (`bankSyncWorker.js`): scheduled background sync, started from `server.js` after `app.listen`. `bankSyncConfigService` admin endpoints live at `/api/admin/bank-sync-config`.

### `services/billMerchantRuleService.js`

- Persistent merchant→bill auto-match rules (table created in `v0.67`).
- `applyMerchantRules(transactions, userId)` — returns transactions with `matched_bill_id` and `match_status='matched'` filled in.
- `auto_attribute_late` (`v0.83`) marks rules whose bills always post after month-end, so amount-suggestion math skips the “looks like last month’s bill” signal.

### `services/merchantStoreMatchService.js`

- 5k merchant/store matching pack seeded by migration `v1.05`.
- `applyMerchantStoreMatches(transactions, userId)` — broad merchant-to-store normalization for spending categorization.

### `services/advisoryFilterService.js`

- 5k advisory non-bill patterns + bill-like override terms seeded by migration `v0.68`.
- Used to suppress “this looks like a bill but isn’t” false positives in tracker suggestions.

### `services/subscriptionService.js` + `services/amountSuggestionService.js`

- `subscriptionService.js` — manages the user’s subscription bills, recommendations, declined hints (`v0.66`), feedback (`v0.97`), and the top-200 catalog seeding. Exposes `normalizeMerchant` used by migration `v0.90` to re-normalize stored rules.
- `amountSuggestionService.js` — `computeAmountSuggestion` (per-bill) and `computeAmountSuggestionsBatch` (all-bills, used by the Tracker P1 fix to kill the N+1).
- The batched suggestion is byte-identical to the per-bill function; pinned by `tests/amountSuggestionService.test.js`.

### `services/snowballService.js` + `services/calendarFeedService.js` + `services/driftService.js` + `services/statusRuntime.js`

- `snowballService.js` — Debt snowball/avalanche calculations, Ramsey mode support, order updates, plan lifecycle math.
- `calendarFeedService.js` — generates iCalendar (`.ics`) text for upcoming bills, gated by `calendar_tokens` (`v1.00`).
- `driftService.js` — detects when an autopay/posted amount drifts from the bill’s expected amount; surfaces the “drift_snoozed_until” field (`v0.71`).
- `statusRuntime.js` — `recordError(err, req)`, runtime worker state, health JSON consumed by `/api/status`.

### `services/spendingService.js` + `services/userSettings.js` + `services/userDataService.js` + `services/eraseUserData.js` (in `utils`/tests) + `services/billsService.js` + `services/aprService.js`

- `spendingService.js` — spending categories, rules (`v0.87`), budgets, default category seeding, income breakdown, `applySpendingCategoryRules` used by the bank-sync pipeline.
- `userSettings.js` — `getUserSettings(userId)` returns a typed settings object; consumed by the bank-sync pipeline.
- `userDataService.js` — user-scoped read helpers (e.g. `getCategoriesForUser`).
- `billsService.js` — bill CRUD helpers, soft-delete + `RecentlyDeletedBills` restore support.
- `aprService.js` — annual-percentage-rate / monthly interest accrual used by `v0.93` (`interest_accrued_month`, `interest_delta`).

### `services/paymentAccountingService.js` + `services/paymentValidation.js`

- `paymentAccountingService.js` — source-of-truth for “manual vs. bank” payments. The bank-backed predicate, the accounting-active SQL fragment, and the core invariant: a bank payment overrides a provisional manual payment so it isn’t double-counted, restores the balance, and the manual payment counts again with its balance re-applied when the override is removed. Pinned by `tests/paymentAccountingService.test.js`.
- `paymentValidation.js` — `PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match']` (auto_match was normalized to provider_sync in `v0.82`), `validatePaymentSource()`, `validatePaymentInput()`.

### `services/auditService.js`

Writes `audit_log` rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, migration rollback attempts, WebAuthn/TOTP events, bank-sync admin changes, and 2FA enable/disable.

`db/database.js` does not import `logAudit` at module load time. It uses a lazy `getLogAudit()` helper so migration code can write audit rows without creating a circular dependency: `auditService` imports `database.js`, and `database.js` needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors.

---

## 5. API Reference

All routes are prefixed with `/api` unless stated otherwise. Most mutating endpoints require CSRF token header `x-csrf-token`. Auth is cookie-based (`bt_session`). User routes are scoped to the authenticated user unless noted. `GET /api/health` is the unauthenticated liveness probe and is the only public non-meta endpoint.

Response conventions:

- Success: JSON object/array unless endpoint downloads a file.
- Validation: `400 {error, code?, field?}`.
- Unauthenticated: `401`.
- Forbidden: `403`.
- Missing resource: `404`.
- Conflict: `409` (also used for `DUPLICATE_SUSPECTED` on `POST /payments` — see §5.5).
- Rate-limited: `429`.
- **Server error (Track D):** the global 500 handler in `server.js` emits `{ error: 'Internal server error', code: 'INTERNAL_ERROR' }` so an unexpected server error carries the same `{error, code}` field the client can switch on as every structured 4xx.

### 5.1 Auth (`/api/auth`)

- `POST /auth/login` — `{username, password}`. Sets `bt_session`. Rate-limited (skipped if zero users). Skips CSRF (no session yet).
- `POST /auth/logout` — clears `bt_session`.
- `POST /auth/logout-all` — invalidates every session for the user.
- `GET /auth/me` — public user object (single-user mode supplies user without session).
- `GET /auth/mode` — auth mode, local-login flag, OIDC public info, single-user status.
- `POST /auth/restore-multi-user-mode` — restores multi-user mode where allowed.
- `POST /auth/acknowledge-privacy` — updates first-login/privacy acknowledgement flags.
- `POST /auth/acknowledge-version` — stamps `users.last_seen_version`.
- `POST /auth/change-password` — `{current_password, new_password}`. Password-limited, CSRF-skip. Rotates session id, invalidates others.
- `GET /auth/csrf-token` — returns `{token}` mirroring the cookie; SPA caches in memory.
- `GET /auth/has-users` — whether non-default users exist.
- `GET /auth/users` — admin-only safe user list.
- `POST /auth/users` — admin-only create.
- `GET /auth/login-history` — recent login/failed-login history for the current user (encrypted columns decrypted in-process).
- `GET /auth/webauthn/status` — `{enabled, credentials:[{id, name, createdAt, lastUsedAt}]}`.
- **TOTP** (`/api/auth/totp`):
  - `POST /auth/totp/challenge` — `{username, password}` → `{challenge_id}` (5-min TTL).
  - `GET /auth/totp/setup` — admin-aware: returns `{secret, uri, qr_data_url}` for a new enrolment.
  - `POST /auth/totp/enable` — `{secret, token}` → verifies, encrypts, stores, returns recovery codes.
  - `POST /auth/totp/disable` — clears secret + recovery codes.
  - `GET /auth/totp/status` — `{enabled, hasSecret, recoveryCodesRemaining}`.

### 5.2 OIDC Auth (`/api/auth/oidc`)

OIDC rate limiter applies.

- `GET /auth/oidc/login?redirect_to=/path` — public when OIDC active; builds PKCE state and redirects.
- `GET /auth/oidc/callback?code=&state=` — validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root. Error redirects use query errors such as `access_denied` or `authentication_failed`.

### 5.3 Admin (`/api/admin`)

Mounted under `/api/admin`; all require `requireAuth + requireAdmin + adminActionLimiter`. Backup subroutes also use `backupOperationLimiter`.

- `GET /admin/has-users` — `{has_users:boolean}`.
- `GET /admin/users` — safe users ordered by default-admin / role / username.
- `POST /admin/users` — `{username, password}`.
- `PUT /admin/users/:id/password` — `{password}`. Invalidates target sessions; requires password change.
- `PUT /admin/users/:id/role` — `{role:"admin"|"user"}`. Cannot change own role; cannot remove last admin.
- `PUT /admin/users/:id/active` — `{active:boolean}`. Cannot deactivate self.
- `PUT /admin/users/:id/username` — change a user’s username (admin only).
- `DELETE /admin/users/:id` — transaction deletes import sessions/history, sessions, and the user.
- **Backups**: `GET/POST /admin/backups`, `GET /admin/backups/settings`, `PUT /admin/backups/settings`, `POST /admin/backups/run-scheduled-now`, `POST /admin/backups/import` (raw SQLite, max 100 MB, optional `X-Checksum-Sha256`), `GET /admin/backups/:id/download`, `POST /admin/backups/:id/restore`, `DELETE /admin/backups/:id`.
- **Cleanup**: `GET/PUT /admin/cleanup`, `POST /admin/cleanup/run`.
- **Auth mode**: `GET /admin/auth-mode`, `POST /admin/auth-mode/oidc-test`, `PUT /admin/auth-mode`.
- **Bank sync config**: `GET /admin/bank-sync-config`, `PUT /admin/bank-sync-config` (default-lookback days, monitored-account default, etc.).
- **Migrations**: `POST /admin/migrations/rollback` with `{version}`. Maps `NOT_APPLIED` → 404, `ROLLBACK_NOT_SUPPORTED` → 422, other failures → 500.

### 5.4 Bills (`/api/bills`)

User/admin tracker access. Standard bill fields plus `cycle_type`, `cycle_day`, `is_subscription`, `subscription_type`, `catalog_id`, `autopay_verified_at`, `inactive_reason`, `drift_snoozed_until`, `sort_order`, `interest_accrued_month`, `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at`.

Endpoints (subset): `GET /bills?inactive=true`, `GET /bills/:id`, `POST /bills`, `PUT /bills/:id`, `DELETE /bills/:id` (hard delete; cascades payments, monthly state, history ranges, merchant rules; returns warning), `GET/PUT /bills/:id/monthly-state?year=&month=`, `GET /bills/:id/payments`, `POST /bills/:id/toggle-paid`, `GET/POST/PUT/DELETE /bills/:id/history-ranges`, `GET /bills/recently-deleted` (admin), `POST /bills/:id/restore`.

### 5.5 Payments (`/api/payments`)

User/admin tracker access. Soft delete via `deleted_at`. `payment_source` ∈ `{manual, file_import, provider_sync, transaction_match}`. Hard-dedupe unique index on `transaction_id` (`v0.61`).

Endpoints: `GET /payments?bill_id=&year=&month=`, `GET /payments/:id`, `POST /payments`, `POST /payments/quick`, `POST /payments/bulk` (max 50), `PUT /payments/:id`, `DELETE /payments/:id`, `POST /payments/:id/restore`.

**Track A money-integrity notes:**

- `POST /payments` body now accepts an optional `allow_duplicate: true` flag. When omitted and a live payment already matches the same `(bill_id, paid_date, amount)`, the route returns `409 { error, code: 'DUPLICATE_SUSPECTED' }` instead of silently deduping — silently dropping a legitimately-identical second payment would be a money-integrity bug. The client uses `client/lib/paymentActions.ts → createPaymentOrConfirm` to surface a sonner "Add anyway?" toast; confirming resends with `allow_duplicate: true`.
- Every balance-mutation runs in a single transaction so a partial failure cannot leave `bills.current_balance` half-updated relative to the new payment.
- New tests: `tests/paymentsRoute.test.js` (route + transactional behavior) and `tests/reconciliation.test.js` (cross-surface reconciliation harness).

### 5.6 Categories (`/api/categories`)

User/admin tracker access. Categories have a `spending_enabled` flag (`v0.88`) and a `sort_order` (`v0.75`). Categories can be grouped via `category_groups` (`v1.06`).

Endpoints: `GET /categories` (seeds defaults, includes `spending_enabled` + counts), `POST /categories`, `PUT /categories/:id`, `DELETE /categories/:id`, plus category-group endpoints under `/api/categories/groups` (list/create/update/delete/reorder).

### 5.7 Tracker and Calendar (`/api/tracker`, `/api/calendar`)

- `GET /tracker?year=&month=` — returns `year`, `month`, tracker `rows`, totals, starting-amount info, previous-month paid total, three-month averages/trends, **drift / autopay / amount-suggestion** objects, **safe-to-spend** (cashflow timeline), `generated_at`.
- `GET /tracker/upcoming?days=30` — upcoming active bills.
- `GET /tracker/health` — dedicated health endpoint for the home page (subset of `/tracker`).
- `GET /calendar?year=&month=` — month days with payment entries, bills/due-status entries, totals.
- `GET /calendar/feed.ics?token=...` — public iCalendar feed of upcoming bills, gated by `calendar_tokens` (`v1.00`).

### 5.8 Summary and Starting Amounts (`/api/summary`, `/api/monthly-starting-amounts`)

- `GET /summary?year=&month=` — returns `{year, month, income, expenses, starting_amounts, previous_month, summary, chart, bank_tracking, generated_at}`.
- `PUT /summary/income` — `{year, month, amount, label?}`.
- `GET /monthly-starting-amounts?year=&month=` / `PUT /monthly-starting-amounts` — bucket response.

### 5.9 Snowball (`/api/snowball`)

- `GET /snowball` — current user's debt bills pre-sorted by `snowball_order`.
- `GET /snowball/settings` / `PATCH /snowball/settings` — extra payment, ramsey mode, ready-current, ready-emergency-fund.
- `GET /snowball/projection` — `{snowball, avalanche, minimum_only, comparison}` with enriched debt arrays (APR snapshots, interest-accrual month).
- `PATCH /snowball/order` — `[{id, snowball_order}]` batch update.
- **Plans** (lifecycle, `v0.73`): `POST /snowball/plans`, `GET /snowball/plans`, `GET /snowball/plans/active`, `PATCH /snowball/plans/:id`, `POST /snowball/plans/:id/pause|resume|complete|abandon`.

### 5.10 Spending (`/api/spending`)

- `GET /spending/summary?year=&month=` — monthly spending summary, by category / by bill.
- `GET /spending/transactions` — paginated, filterable.
- `PATCH /spending/transactions/:id/category` — manual reassignment.
- `GET /spending/budgets` / `PUT /spending/budgets` / `POST /spending/budgets/copy` — monthly budgets with month-rollover.
- `GET /spending/category-rules` / `POST /spending/category-rules` / `DELETE /spending/category-rules/:id` — automation rules seeded in `v0.87`.
- `GET /spending/income` — per-user monthly income.

### 5.11 Subscriptions (`/api/subscriptions`)

- `GET /subscriptions` — user’s subscription bills.
- `GET /subscriptions/recommendations` — bank-derived candidates (declined hints suppressed).
- `POST /subscriptions/recommendations/decline` — record a decline.
- `POST /subscriptions/recommendations/match-bill` — link a recommendation to an existing bill.
- `POST /subscriptions/recommendations/create` — create a new bill from a recommendation.
- `GET /subscriptions/transaction-matches` — candidate transactions for the user’s subscriptions.
- `GET /subscriptions/catalog` — top-200 catalog with bank descriptors.
- `PUT /subscriptions/:id/catalog-link` / `POST /subscriptions/catalog/:catalogId/descriptors` / `DELETE /subscriptions/catalog/descriptors/:id` — manage custom bank descriptors.
- `PATCH /subscriptions/:id` — update subscription fields (price, cycle, descriptor, etc.).

### 5.12 Analytics (`/api/analytics`)

- `GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=&include_skipped=` — monthly spending, expected vs actual, category totals, bill totals, filters, `generated_at`.

### 5.13 Settings (`/api/settings`)

- `GET /settings` / `PUT /settings` — user-visible settings.
- `POST /settings/seed-demo-data` — seed demo data.

### 5.14 Notifications (`/api/notifications`)

Mounted under `/api/notifications`; `requireAuth` at mount.

- `GET/PUT /notifications/admin` — SMTP + push (ntfy / Gotify / Discord / Telegram) global settings.
- `POST /notifications/test` — `{to}`.
- `GET/PUT /notifications/me` — per-user notification email, push channels, lead times.

### 5.15 Profile (`/api/profile`)

- `GET /profile` — safe profile, notification settings, export URLs, import-history URL.
- `PATCH /profile` — `{display_name}`.
- `GET/PATCH /profile/settings` — per-user notification prefs.
- `POST /profile/change-password` — rotates session, invalidates others.
- `GET /profile/exports` / `GET /profile/import-history`.

### 5.16 User Demo Data (`/api/user`)

- `POST /user/seed-demo-data`, `POST /user/clear-demo-data` (rate-limited; deletes `is_seeded=1` bills/categories for current user).

### 5.17 Import (`/api/import`, `/api/imports`)

`requireAuth + requireUser + importLimiter` at mount. Both prefixes are accepted.

- `POST /import/spreadsheet/preview` (XLSX, max 10 MB, optional `parse_all_sheets`, `year`, `month`).
- `POST /import/spreadsheet/apply`.
- `POST /import/user-db/preview` (SQLite, max 50 MB) / `apply`.
- `POST /import/csv/preview` / `POST /import/csv/commit`.
- `POST /import/ofx/preview` / `POST /import/ofx/commit` (OFX/QFX bank statements).
- `GET /import/history`.

### 5.18 Data Sources (`/api/data-sources`)

- `GET /data-sources?type=&status=` — `type` ∈ `manual, file_import, provider_sync`; `status` ∈ `active, inactive, error`. Returns sources with `account_count`, `transaction_count`, sync info.

### 5.19 Transactions (`/api/transactions`)

- `GET /transactions?limit=&offset=&match_status=&ignored=&source_type=&start_date=&end_date=&q=&data_source_id=&account_id=&matched_bill_id=`. Free-text search across description/payee/memo/category.
- `POST /transactions/manual` — manual entry, returns created row with `source_type='manual'`.
- `PUT /transactions/:id` — partial update.
- `DELETE /transactions/:id` — soft-unmatch then hard delete.
- `POST /transactions/:id/match` / `unmatch` / `ignore` / `unignore`.
- The `pending` flag (`v1.01`) marks SimpleFIN pending transactions separately from posted ones.

### 5.20 Match Suggestions (`/api/matches`)

- `GET /matches/suggestions?transaction_id=&bill_id=&limit=&offset=`.
- `POST /matches/:id/reject` — records rejection so the pair is never re-suggested.

### 5.21 Export (`/api/export`)

- `GET /export?year=YYYY&format=csv|xlsx` — payment/bill history for the year.
- `GET /export/user-excel` — user workbook.
- `GET /export/user-db` — portable SQLite with metadata.

### 5.22 Status, About, Version, Privacy, Health

- `GET /status` — admin-only: app version, uptime, runtime worker state, DB health, SMTP/push config status, backup status, current-month tracker health, recent errors.
- `GET /about` — public; `{version, about}`.
- `GET /about-admin` — admin-only, file allowlist `FUTURE.md` / `DEVELOPMENT_LOG.md`.
- `GET /version` — public; package version + latest structured notes from `HISTORY.md`.
- `GET /version/history` — raw history text.
- `GET /version/update-status` — async, checks upstream for new releases (uses `services/updateCheckService`).
- `GET /privacy` — public privacy text.
- `GET /health` — public liveness probe `{ok:true}`.

## 6. Database Reference

SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/database.js` applies migrations to reach the current schema.

### Tables and columns

#### `users`

- `id INTEGER PRIMARY KEY`
- `username TEXT NOT NULL UNIQUE COLLATE NOCASE`
- `password_hash TEXT NOT NULL`
- `role TEXT NOT NULL DEFAULT 'user'` (`admin` or `user`)
- `must_change_password INTEGER NOT NULL DEFAULT 0`
- `first_login INTEGER NOT NULL DEFAULT 1`
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- `notification_email TEXT`
- `notifications_enabled INTEGER NOT NULL DEFAULT 0`
- `notify_3d INTEGER NOT NULL DEFAULT 1`
- `notify_1d INTEGER NOT NULL DEFAULT 1`
- `notify_due INTEGER NOT NULL DEFAULT 1`
- `notify_overdue INTEGER NOT NULL DEFAULT 1`
- `display_name TEXT`
- `last_password_change_at TEXT`
- `auth_provider TEXT NOT NULL DEFAULT 'local'`
- `external_subject TEXT`
- `email TEXT`
- `last_login_at TEXT`
- `active INTEGER NOT NULL DEFAULT 1`
- `is_default_admin INTEGER NOT NULL DEFAULT 0`
- `snowball_extra_payment REAL` (`v0.48`)
- `last_seen_version TEXT` (`v0.50`) — release-notes notification stamp
- `webauthn_user_id TEXT` (`v0.92`); `webauthn_enabled INTEGER NOT NULL DEFAULT 0` (`v0.92`)
- `totp_enabled INTEGER NOT NULL DEFAULT 0` (`v0.86`); `totp_secret` (encrypted, `v0.86`); `totp_recovery_codes` (JSON array of hashes, `v0.86`)
- Push notification columns (`v0.80`): ntfy / Gotify / Discord / Telegram (topic + URL + enabled flag each)
- `geolocation_enabled INTEGER NOT NULL DEFAULT 0` (`v1.02`; was a global admin setting)

#### `sessions`

- `id TEXT PRIMARY KEY` — **session token hash**, not the raw cookie value (`v0.94`). The cookie carries a random opaque ID; the DB stores its SHA-256 so a leaked DB does not yield usable session tokens.
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `expires_at TEXT NOT NULL`
- `created_at TEXT DEFAULT datetime('now')` (`v0.43`)

#### `webauthn_credentials` (`v0.92`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `credential_id TEXT NOT NULL UNIQUE`
- `public_key TEXT NOT NULL`
- `sign_count INTEGER NOT NULL DEFAULT 0`
- `transports TEXT` (JSON)
- `backup_eligible INTEGER NOT NULL DEFAULT 0`
- `backup_state INTEGER NOT NULL DEFAULT 0`
- `credential_name TEXT`
- `aaguid TEXT`
- `last_used_at TEXT`
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`

#### `webauthn_challenges` (`v0.92`)

- `id TEXT PRIMARY KEY`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `challenge_type TEXT NOT NULL CHECK ('registration' OR 'authentication')`
- `challenge TEXT NOT NULL`
- `expires_at TEXT NOT NULL` (15-min TTL)

#### `totp_challenges` (`v0.86`)

- `id TEXT PRIMARY KEY`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `challenge TEXT NOT NULL`
- `expires_at TEXT NOT NULL` (5-min TTL)

#### `categories`

- `id INTEGER PRIMARY KEY`
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
- `name TEXT NOT NULL`
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- `is_seeded INTEGER NOT NULL DEFAULT 0`
- `deleted_at TEXT` (`v0.54`)
- `sort_order INTEGER` (`v0.75`) — persistent order
- `spending_enabled INTEGER NOT NULL DEFAULT 0` (`v0.88`) — separates bill vs spending categories
- `group_id INTEGER REFERENCES category_groups(id) ON DELETE SET NULL` (`v1.06`)

#### `category_groups` (`v1.06`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `name TEXT NOT NULL`
- `sort_order INTEGER`
- `created_at`, `updated_at`
- Unique: `(user_id, name)`

#### `bills`

- `id INTEGER PRIMARY KEY`
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
- `name TEXT NOT NULL`
- `category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL`
- `due_day INTEGER NOT NULL CHECK 1-31`
- `override_due_date TEXT`
- `bucket TEXT CHECK ('1st','15th')`
- `expected_amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`)
- `interest_rate REAL CHECK null or 0-100`
- `billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')` (canonicalized in `v0.76`)
- `autopay_enabled INTEGER NOT NULL DEFAULT 0`
- `autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')`
- `website TEXT`
- `username TEXT`
- `account_info TEXT`
- `has_2fa INTEGER NOT NULL DEFAULT 0`
- `active INTEGER NOT NULL DEFAULT 1`
- `notes TEXT`
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- `history_visibility TEXT NOT NULL DEFAULT 'default'`
- `is_seeded INTEGER NOT NULL DEFAULT 0`
- `cycle_type TEXT NOT NULL DEFAULT 'monthly'`
- `cycle_day TEXT`
- `current_balance REAL` (dollars; converted to integer cents by `v1.03`)
- `minimum_payment REAL` (dollars; converted to integer cents by `v1.03`)
- `snowball_order INTEGER`
- `snowball_include INTEGER NOT NULL DEFAULT 0`
- `snowball_exempt INTEGER NOT NULL DEFAULT 0`
- `auto_mark_paid INTEGER NOT NULL DEFAULT 0`
- `deleted_at TEXT`
- `is_subscription INTEGER NOT NULL DEFAULT 0`, `subscription_type TEXT` (`v0.63`)
- `interest_accrued_month TEXT` (`v0.93`) — calendar month of last applied interest
- `drift_snoozed_until TEXT` (`v0.71`) — suppresses drift notifications until this date
- `sort_order INTEGER` (`v0.72`) — persistent tracker order
- `catalog_id INTEGER REFERENCES subscription_catalog(id) ON DELETE SET NULL` (`v0.96`)
- `autopay_verified_at TEXT`, `inactive_reason TEXT` (`v0.99`) — autopay trust / lifecycle fields

#### `payments`

- `id INTEGER PRIMARY KEY`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `amount REAL NOT NULL` (dollars; converted to integer cents by `v1.03`)
- `paid_date TEXT NOT NULL`
- `method TEXT`
- `notes TEXT`
- `balance_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.49`)
- `payment_source TEXT NOT NULL DEFAULT 'manual'` (`v0.59`; `auto_match` normalized to `provider_sync` in `v0.82`)
- `transaction_id INTEGER` (`v0.59`; unique-per-active index added in `v0.61`)
- `interest_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.93`)
- `accounting_excluded INTEGER NOT NULL DEFAULT 0` (`v0.98`) — bank-override metadata for provisional manual payments
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- `deleted_at TEXT`

#### `monthly_bill_state`

- `id INTEGER PRIMARY KEY`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `year INTEGER NOT NULL CHECK 2000-2100`
- `month INTEGER NOT NULL CHECK 1-12`
- `actual_amount REAL` (dollars; converted to integer cents by `v1.03`)
- `notes TEXT`
- `is_skipped INTEGER NOT NULL DEFAULT 0`
- `snoozed_until TEXT` (`v0.70`) — overdue command-center snooze
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- Unique: `(bill_id, year, month)`

#### `monthly_income`

- `id INTEGER PRIMARY KEY`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `year INTEGER NOT NULL`
- `month INTEGER NOT NULL`
- `label TEXT NOT NULL DEFAULT 'Salary'`
- `amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`)
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- Unique: `(user_id, year, month)`

#### `monthly_starting_amounts`

- `id INTEGER PRIMARY KEY`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `year INTEGER NOT NULL`
- `month INTEGER NOT NULL`
- `first_amount REAL NOT NULL DEFAULT 0`
- `fifteenth_amount REAL NOT NULL DEFAULT 0`
- `other_amount REAL NOT NULL DEFAULT 0`
- `notes TEXT`
- `created_at TEXT DEFAULT datetime('now')`
- `updated_at TEXT DEFAULT datetime('now')`
- Unique: `(user_id, year, month)`

#### `bill_history_ranges`

- `id INTEGER PRIMARY KEY`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `start_year INTEGER NOT NULL`
- `start_month INTEGER NOT NULL`
- `end_year INTEGER`
- `end_month INTEGER`
- `label TEXT`
- `description TEXT NOT NULL`
- `applied_at TEXT NOT NULL DEFAULT datetime('now')`

#### `data_sources`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `type TEXT NOT NULL` (`manual`, `file_import`, `provider_sync`)
- `provider TEXT`
- `name TEXT NOT NULL`
- `status TEXT NOT NULL DEFAULT 'active'` (`active`, `inactive`, `error`)
- `config_json TEXT`
- `encrypted_secret TEXT`
- `last_sync_at TEXT`
- `last_error TEXT`
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- Unique partial index: `(user_id, type, provider)` WHERE `type='manual' AND provider='manual'`

#### `financial_accounts`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL`
- `provider_account_id TEXT`
- `name TEXT NOT NULL`
- `org_name TEXT`
- `account_type TEXT`
- `currency TEXT`
- `balance INTEGER`
- `available_balance INTEGER`
- `raw_data TEXT`
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- Unique: `(data_source_id, provider_account_id)`

#### `transactions`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL`
- `account_id INTEGER REFERENCES financial_accounts(id) ON DELETE SET NULL`
- `provider_transaction_id TEXT`
- `source_type TEXT NOT NULL` (`manual`, `file_import`, `provider_sync`)
- `transaction_type TEXT`
- `posted_date TEXT`
- `transacted_at TEXT`
- `amount INTEGER NOT NULL`
- `currency TEXT`
- `description TEXT`
- `payee TEXT`
- `memo TEXT`
- `category TEXT`
- `raw_data TEXT`
- `matched_bill_id INTEGER REFERENCES bills(id) ON DELETE SET NULL`
- `match_status TEXT NOT NULL DEFAULT 'unmatched'` (`unmatched`, `matched`, `ignored`)
- `ignored INTEGER NOT NULL DEFAULT 0`
- `spending_category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL` (`v0.87`)
- `pending INTEGER NOT NULL DEFAULT 0` (`v1.01`) — SimpleFIN pending transactions
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- Unique partial index: `(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL`
- Indexes: `(user_id, posted_date, transacted_at)`, `(user_id, match_status, ignored)`, `(account_id)`, `(matched_bill_id)`

### Indexes

Important indexes include:

- `idx_bills_active(active)`
- `idx_bills_user_active(user_id, active)`
- `idx_bills_user_name(user_id, name)`
- `idx_bills_user_deleted(user_id, deleted_at)` (`v0.91`)
- `idx_bills_user_sort(user_id, active, sort_order, due_day)` (`v0.72`)
- `idx_categories_user_name(user_id, name)`
- `idx_categories_user_name_unique(user_id, name COLLATE NOCASE)`
- `idx_categories_user_deleted(user_id, deleted_at)` (`v0.91`)
- `idx_categories_user_sort(user_id, sort_order, name)` (`v0.75`)
- `idx_payments_bill_id(bill_id)`
- `idx_payments_paid_date(paid_date)`
- `idx_payments_bill_date_del(bill_id, paid_date, deleted_at)`
- `idx_payments_deleted(deleted_at)`
- `idx_payments_method(method)`
- `idx_payments_transaction_active` — unique-per-active on `transaction_id` (`v0.61`)
- `idx_sessions_user_id(user_id)`
- `idx_sessions_expires(expires_at)`
- `idx_monthly_bill_state_lookup(bill_id, year, month)`
- `idx_monthly_income_user_month(user_id, year, month)`
- `idx_monthly_starting_amounts_user(user_id)`
- `idx_monthly_starting_amounts_user_month(user_id, year, month)`
- `idx_notifications_lookup(bill_id, user_id, year, month)`
- `idx_import_sessions_user(user_id)`, `idx_import_sessions_expires(expires_at)`
- `idx_import_history_user(user_id)`, `idx_import_history_imported_at(imported_at)`
- `idx_oidc_states_expires(expires_at)`
- `idx_bill_history_ranges_bill(bill_id)`
- `idx_audit_log_user(user_id, created_at)`, `idx_audit_log_action(action, created_at)`
- `idx_data_sources_user_type(user_id, type, status)`
- `idx_data_sources_user_manual(user_id, type, provider)` WHERE `type='manual' AND provider='manual'`
- `idx_financial_accounts_user_source(user_id, data_source_id)`
- `idx_transactions_user_date(user_id, posted_date, transacted_at)`
- `idx_transactions_user_match(user_id, match_status, ignored)`
- `idx_transactions_account(account_id)`
- `idx_transactions_matched_bill(matched_bill_id)`
- `idx_transactions_provider_dedupe(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL`
- `idx_bill_merchant_rules_user_bill(user_id, bill_id)` (`v0.81`)

#### `import_sessions`

- `id TEXT PRIMARY KEY`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `created_at TEXT NOT NULL`
- `expires_at TEXT NOT NULL`
- `preview_json TEXT NOT NULL`

#### `import_history`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `imported_at TEXT NOT NULL`
- `source_filename TEXT`
- `file_type TEXT DEFAULT 'csv_transactions'`
- `rows_parsed INTEGER DEFAULT 0`
- `rows_created INTEGER DEFAULT 0`
- `rows_updated INTEGER DEFAULT 0`
- `rows_skipped INTEGER DEFAULT 0`
- `rows_errored INTEGER DEFAULT 0`
- `options_json TEXT`
- `summary_json TEXT`
- `created_at TEXT DEFAULT (datetime('now'))`

#### `autopay_suggestion_dismissals`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `year INTEGER NOT NULL CHECK(year BETWEEN 2000 AND 2100)`
- `month INTEGER NOT NULL CHECK(month BETWEEN 1 AND 12)`
- `dismissed_at TEXT NOT NULL DEFAULT (datetime('now'))`
- Unique: `(user_id, bill_id, year, month)`

#### `bill_templates`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `name TEXT NOT NULL`
- `data TEXT NOT NULL`
- `created_at TEXT DEFAULT (datetime('now'))`
- `updated_at TEXT DEFAULT (datetime('now'))`
- Unique: `(user_id, name)`

#### `match_suggestion_rejections`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `transaction_id INTEGER NOT NULL REFERENCES transactions(id) ON DELETE CASCADE`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `rejected_at TEXT NOT NULL DEFAULT (datetime('now'))`
- Unique: `(user_id, transaction_id, bill_id)`

#### `user_login_history`

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `logged_in_at TEXT NOT NULL DEFAULT (datetime('now'))`
- `ip_address TEXT` (encrypted at rest, `v0.84`)
- `user_agent TEXT` (encrypted at rest, `v0.84`)
- `browser TEXT`
- `os TEXT`
- `device_type TEXT`
- `device_fingerprint TEXT`
- `location TEXT` (`v0.84`) — IP-geolocation city/region/country when geolocation opt-in is on
- `success INTEGER NOT NULL DEFAULT 1` (`v0.85`) — failed attempts when 0
- `session_fingerprint TEXT` (`v0.85`) — for current-session detection

#### `subscription_catalog` (`v0.65`, expanded `v0.69`, `v0.95`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `name TEXT NOT NULL`
- `category TEXT`
- `subcategory TEXT` (`v0.95`)
- `starting_monthly_usd REAL` (`v0.95`)
- `bank_descriptors TEXT` (JSON array; populated by `v0.95` from a 2026 researched dataset)
- `created_at`, `updated_at`

#### `declined_subscription_hints` (`v0.66`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `descriptor TEXT NOT NULL`
- `source TEXT`
- `declined_at TEXT NOT NULL DEFAULT (datetime('now'))`
- Unique: `(user_id, descriptor, source)`

#### `bill_merchant_rules` (`v0.67`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
- `normalized_merchant TEXT NOT NULL`
- `match_mode TEXT NOT NULL DEFAULT 'contains'` (e.g. `contains`, `equals`, `regex`)
- `priority INTEGER NOT NULL DEFAULT 0`
- `auto_attribute_late INTEGER NOT NULL DEFAULT 0` (`v0.83`)
- `created_at`, `updated_at`
- Composite index `(user_id, bill_id)` for fast `EXISTS` lookups (`v0.81`)

#### `user_catalog_descriptors` (`v0.96`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `catalog_id INTEGER NOT NULL REFERENCES subscription_catalog(id) ON DELETE CASCADE`
- `descriptor TEXT NOT NULL`
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`

#### `subscription_recommendation_feedback` (`v0.97`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `recommendation_key TEXT NOT NULL`
- `verdict TEXT NOT NULL` (`accept` | `decline` | `mute`)
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`

#### `snowball_plans` (`v0.73`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `name TEXT NOT NULL`
- `strategy TEXT NOT NULL` (`snowball` | `avalanche` | `minimum_only`)
- `extra_payment REAL NOT NULL DEFAULT 0`
- `status TEXT NOT NULL DEFAULT 'active'` (`active` | `paused` | `completed` | `abandoned`)
- `created_at`, `updated_at`, `completed_at`
- Lifecycle transitions are recorded with audit + plan-row updates; `pause`/`resume` flip `status`.

#### `calendar_tokens` (`v1.00`)

- `id INTEGER PRIMARY KEY AUTOINCREMENT`
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
- `token TEXT NOT NULL UNIQUE` (random opaque)
- `label TEXT`
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
- `last_used_at TEXT`
- `revoked_at TEXT`

#### `spending_rules` / `spending_budgets` (`v0.87`)

- `spending_rules`: per-user regex/keyword rules mapping a transaction pattern to a `spending_category_id`. Auto-applied during bank sync by `applySpendingCategoryRules`.
- `spending_budgets`: per-user monthly budget rows; month-rollover uses the copy endpoint.

#### `merchant_store_matches` (seeded by `v1.05`)

- Static 5k merchant/store matching pack; used by `applyMerchantStoreMatches` during bank sync to normalize merchant strings.

#### `advisory_non_bill_filters` (seeded by `v0.68`)

- 5k advisory non-bill patterns + bill-like override terms.

### Migration system

`db/database.js`:

- Reads `db/schema.sql` in `initSchema()`.
- Creates `schema_migrations`.
- Detects and reconciles legacy databases via `db/migrations/legacyReconcileMigrations.js`.
- Applies ordered migrations from `db/migrations/versionedMigrations.js` only when not already recorded.
- Validates dependency chains (`dependsOn: ['v0.71']`) before applying dependent migrations.
- Uses a column whitelist for dynamic `ALTER TABLE` statements.
- Wraps versioned migrations in transactions, with special handling for `v0.40` because it uses PRAGMA-driven table rebuild work.
- Supports rollback SQL for selected migrations through `ROLLBACK_SQL_MAP` and `rollbackMigration(version)`.

Current migration set (`v0.2` → `v1.06`):

- `v0.2` payments soft delete.
- `v0.3` tracker payment compound index.
- `v0.4` monthly bill state.
- `v0.13` user profile columns.
- `v0.14` bill history visibility.
- `v0.14.4` bill interest rate.
- `v0.15` import sessions/history.
- `v0.17` OIDC/external identity columns and state table.
- `v0.18.1` monthly income.
- `v0.18.2` monthly starting amounts.
- `v0.18.3` other starting amount bucket.
- `v0.38` per-user import audit history.
- `v0.40` ownership for bills/categories.
- `v0.41` seeded demo-data flags.
- `v0.42` bill history ranges.
- `v0.43` session `created_at`.
- `v0.44` performance indexes.
- `v0.45` audit log.
- `v0.46` bill `cycle_type` and `cycle_day`.
- `v0.47` bills: `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at` columns.
- `v0.48` users: `snowball_extra_payment` column.
- `v0.49` payments: `balance_delta` column for debt payoff tracking.
- `v0.50` users: `last_seen_version` for release-notes notifications.
- `v0.51` user_login_history table.
- `v0.52`–`v0.53` ad-hoc columns (legacy reconcile).
- `v0.54` bills/categories: soft-delete columns (`deleted_at`).
- `v0.55` autopay: `auto_mark_paid` and suggestion dismissals table.
- `v0.56` bills: saved bill templates table.
- `v0.57` match_suggestion_rejections table (replaced by `v0.62`).
- `v0.58` import_sessions and import_history tables.
- `v0.59` payments: `payment_source` and `transaction_id` columns.
- `v0.60` data_sources, financial_accounts, transactions tables.
- `v0.61` payments: one active payment per linked transaction (unique index on `transaction_id`).
- `v0.62` match_suggestion_rejections table (canonical).
- `v0.63` bills: subscription metadata fields (`is_subscription`, `subscription_type`).
- `v0.64` financial_accounts: `monitored` flag for bill matching; transactions: partial unique index on `(data_source_id, provider_transaction_id)` and supporting indexes.
- `v0.65` subscription_catalog: top-200 known subscription services.
- `v0.66` declined_subscription_hints: per-user dismissed recommendation store.
- `v0.67` bill_merchant_rules: persistent merchant→bill auto-match rules.
- `v0.68` advisory_non_bill_filters: 5k advisory patterns + bill-like override terms.
- `v0.69` subscription_catalog v2: 90 new services + category fixes.
- `v0.70` monthly_bill_state: `snoozed_until` for overdue command center.
- `v0.71` bills: `drift_snoozed_until`; users: `notify_amount_change`.
- `v0.72` bills: persistent tracker sort order.
- `v0.73` snowball_plans table for plan lifecycle + history.
- `v0.74` subscription_catalog: Claude.ai Anthropic matching.
- `v0.75` categories: persistent sort order.
- `v0.76` bills: canonical billing schedule cleanup.
- `v0.77` encrypt SMTP password at rest.
- `v0.78` re-encrypt secrets from SHA-256 to HKDF key derivation.
- `v0.79` encrypt OIDC client secret at rest.
- `v0.80` users: push notification columns (ntfy / Gotify / Discord / Telegram).
- `v0.81` bill_merchant_rules: composite index on `(user_id, bill_id)`.
- `v0.82` payments: normalise `auto_match` source to `provider_sync`.
- `v0.83` bill_merchant_rules: `auto_attribute_late` flag.
- `v0.84` user_login_history: encrypt ip/useragent at rest + add `location` + keep 10 records.
- `v0.85` user_login_history: failed-attempt tracking + `session_fingerprint`.
- `v0.86` users: TOTP/authenticator 2FA columns + `totp_challenges` table.
- `v0.87` spending: category assignment on transactions + rules + budgets + default categories.
- `v0.88` categories: `spending_enabled` flag to separate bill vs spending categories.
- `v0.89` categories: seed spending defaults for users who had existing categories before `v0.87`.
- `v0.90` re-normalize merchant rules after `&` fix; ensure rejection expiry column.
- `v0.91` performance: composite indexes on `user_id+deleted_at` for categories, bills, payments.
- `v0.92` auth: WebAuthn/FIDO2 security key support — `webauthn_credentials` + `webauthn_challenges` tables.
- `v0.93` bills: `interest_accrued_month`; payments: `interest_delta`; transactions: stable provider key + dedupe index.
- `v0.94` security: session token hashing + geolocation opt-in setting (seeded into `settings`).
- `v0.95` subscription_catalog: bank descriptors + pricing from 2026 researched dataset.
- `v0.96` bills: `catalog_id` FK; `user_catalog_descriptors` for custom bank descriptors.
- `v0.97` subscription recommendation feedback: per-user learning signals.
- `v0.98` payments: `accounting_excluded` for bank-override metadata.
- `v0.99` bills: autopay trust indicators (`autopay_verified_at`, `inactive_reason`) + lifecycle fields; payments: autopay failure flag.
- `v1.00` calendar feed subscription tokens (`calendar_tokens`).
- `v1.01` transactions: `pending` flag for SimpleFIN pending transactions.
- `v1.02` users: per-user geolocation opt-in (was global admin setting).
- `v1.03` money columns: dollars (REAL) → integer cents for `bills.expected_amount/current_balance/minimum_payment`, `payments.amount/balance_delta/interest_delta`, `monthly_bill_state.actual_amount`, `monthly_income.amount`, plus relevant index updates.
- `v1.04` `bill_templates.data` JSON: money fields dollars → integer cents (column-only `v1.03` does not touch the JSON blob; `serializeTemplateData` reads cents).
- `v1.05` `merchant_store_matches`: 5k merchant/store matching pack for bank transaction categorization.
- `v1.06` `category_groups`: organize spending categories into named groups.
- Unversioned user notification columns are also reconciled by `legacyReconcileMigrations.js`.

Migration logging is both console-based and audit-backed:

- `runMigrations()` logs start, dependency status, transaction begin/commit, per-migration elapsed time, skips for already-applied migrations, failures with rollback messages, and total elapsed time.
- Audit events use the lazy `getLogAudit()` helper to avoid the `auditService -> database.js -> auditService` circular dependency.
- Audit actions include `migration.start`, `migration.complete`, and `migration.failure`.
- Rollback paths audit `migration.rollback` and `migration.rollback.failure`.

Rollback support is defined by `ROLLBACK_SQL_MAP` in `db/database.js` and currently covers `v0.44` through `v1.04` (the most recent migrations `v1.05` and `v1.06` are **not** yet rollbackable):

- `v0.44` — drops selected performance indexes: `idx_bills_user_name`, `idx_payments_method`, `idx_monthly_starting_amounts_user`, and `idx_import_history_imported_at`.
- `v0.45` — drops `idx_audit_log_user`, `idx_audit_log_action`, and the `audit_log` table.
- `v0.46` — drops `bills.cycle_day` and `bills.cycle_type`.
- `v0.47` — drops `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at` columns from bills.
- `v0.48` — drops `snowball_extra_payment` column from users.
- `v0.49` — drops `balance_delta` column from payments.
- `v0.50` — drops `last_seen_version` column from users.
- `v0.51` — drops `user_login_history` table.
- `v0.54` — removes soft-delete columns (`deleted_at`) from bills and categories.
- `v0.55` — drops autopay suggestion dismissals table.
- `v0.56` — drops `bill_templates` table.
- `v0.57` — drops `match_suggestion_rejections` table.
- `v0.58` — drops `import_sessions` and `import_history` tables.
- `v0.59` — drops `payment_source` and `transaction_id` columns from payments.
- `v0.60` — drops `data_sources`, `financial_accounts`, and `transactions` tables.
- `v0.61` — drops unique index on `transaction_id` from payments.
- `v0.62` — drops `match_suggestion_rejections` table.
- `v0.63` — drops partial unique index on `data_sources`.
- `v0.64` — drops partial unique index and indexes on `transactions`.
- `v0.72` — drops `bills.sort_order` and `idx_bills_user_sort`.
- `v0.75` — drops `categories.sort_order` and `idx_categories_user_sort`.
- `v0.76` — drops `bills.billing_cycle` (canonical cleanup).
- `v0.77`–`v0.79` — revert SMTP/OIDC secret encryption (re-store plaintext).
- `v0.81` — drops `idx_bill_merchant_rules_user_bill`.
- `v0.82` — no-op rollback (data migration).
- `v0.83` — drops `bill_merchant_rules.auto_attribute_late`.
- `v0.84`–`v0.85` — revert `user_login_history` encryption and drop `success`/`session_fingerprint`/`location`.
- `v0.86` — drops `users.totp_*` columns and `totp_challenges`.
- `v0.87` — drops `transactions.spending_category_id`, `spending_rules`, `spending_budgets`, related defaults.
- `v0.88` — drops `categories.spending_enabled`.
- `v0.89` — data-only (seeding); no schema rollback.
- `v0.90` — data-only (re-normalization); no schema rollback.
- `v0.91` — drops `idx_categories_user_deleted`, `idx_bills_user_deleted`, `idx_payments_user_deleted`.
- `v0.92` — drops `webauthn_credentials`, `webauthn_challenges`, and the `webauthn_*` columns on `users`.
- `v0.93` — drops `bills.interest_accrued_month`, `payments.interest_delta`, transactions dedupe index + indexes.
- `v0.94` — reverts session token hashing (drops the hash; cookie value is now the raw ID again); removes `geolocation_enabled` setting row.
- `v0.95` — drops `subscription_catalog.bank_descriptors` / `subcategory` / `starting_monthly_usd`.
- `v0.96` — drops `bills.catalog_id` and `user_catalog_descriptors` table.
- `v0.97` — drops `subscription_recommendation_feedback` table.
- `v0.98` — drops `payments.accounting_excluded`.
- `v0.99` — drops `bills.autopay_verified_at`/`inactive_reason` (and any payments autopay-failure column added in the same migration).
- `v1.00` — drops `calendar_tokens` table.
- `v1.01` — drops `transactions.pending`.
- `v1.02` — drops `users.geolocation_enabled` and re-seeds the global admin setting key.
- `v1.03` — reverts the integer-cents conversion (back to dollar REALs) for the listed columns.
- `v1.04` — reverts `bill_templates.data` money fields back to dollars.

`rollbackMigration(version)` requires an initialized database, verifies the version exists in `schema_migrations`, looks up rollback SQL in `ROLLBACK_SQL_MAP`, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns `{success:true, version, description, elapsed_ms}`. If the migration is not recorded, it throws `NOT_APPLIED`. If no rollback definition exists, it throws `ROLLBACK_NOT_SUPPORTED`. Execution failures roll back the transaction and are audited as `migration.rollback.failure`.

The admin API exposes rollback through `POST /api/admin/migrations/rollback`. The route requires admin auth through the `/api/admin` mount. It maps `NOT_APPLIED` to HTTP 404, `ROLLBACK_NOT_SUPPORTED` to HTTP 422, and unexpected rollback failures to HTTP 500.

The lazy audit helper in `db/database.js` is:

```javascript
let _logAudit = null;
function getLogAudit() {
  if (!_logAudit) {
    try { _logAudit = require('../services/auditService').logAudit; } catch { _logAudit = () => {}; }
  }
  return _logAudit;
}
```

Use this pattern for database-layer audit calls instead of a top-level `require('../services/auditService')`.

---

## 7. Frontend Reference

### Frontend stack

- React `^19.2.7` (with `babel-plugin-react-compiler` for automatic memoization)
- TypeScript `^6.0.3` with `strict: true` + `noUncheckedIndexedAccess: true`; `allowJs: true`, `checkJs: false` for the gradual JS → TS migration
- Vite `^5.4.10` with `@vitejs/plugin-react` and `vite-plugin-pwa` (autoUpdate service worker, manifest, runtime caching for tracker/bills/calendar/summary/analytics/snowball/categories)
- React Router `^6.26.2`
- TanStack Query `^5.100.9` + `@tanstack/react-query-devtools` + `@tanstack/react-virtual` for the virtualized Subscriptions list
- Tailwind CSS `^3.4.14` (content glob: `./client/**/*.{js,jsx,ts,tsx}` — must include `ts,tsx` or `.tsx`-only classes generate no CSS)
- shadcn/ui component primitives, backed by Radix UI where applicable
- Sonner toast notifications via `sonner`
- `framer-motion` `^12.40.0` for transitions
- `react-markdown`, `remark-gfm`, `rehype-sanitize` for markdown rendering
- `vitest` `^4.1.8` + `@testing-library/react` for client unit tests (server tests stay on `node --test`)

### `client/main.tsx`

Creates the React root and wraps `App` with global providers. Theme is provided by `client/contexts/ThemeContext.tsx`.

### `client/App.tsx`

- Creates a TanStack `QueryClient` with:
  - Global `QueryCache.onError` — toasts a `sonner` error when a *background* refetch fails while stale data is on screen. Pages render inline errors on the initial load so the toast only fires for refetches.
  - `staleTime: 2 * 60 * 1000`, `retry: 1`, `refetchOnWindowFocus: false`.
- Uses lazy loading and `Suspense` with `PageLoader` for most pages.
- Wraps route elements in `ErrorBoundary`; pages further wrap in `PageTransition`.
- Exposes `ReactQueryDevtools`.
- Provides skip link for keyboard users.
- Mounts the `CommandPalette` and `ReleaseNotesDialog` globally.
- `RequireAuth` (`{children, role}`) behavior:
  - Shows loading while auth state is `undefined`.
  - Single-user mode bypasses the auth gate for `role: 'user'`.
  - Redirects unauthenticated users to `/login` (preserves `state.from`).
  - Default admin cannot access user routes — redirected to `/admin`.
  - Role mismatches redirect to `/admin` (for admin role) or `/` (for user role).
- `AdminShell` wraps admin routes so the user-shell nav does not appear inside admin.

Routes (all rendered through `Routes` with `ErrorBoundary`, lazy + `Suspense` for non-critical pages):

- `/login` → `LoginPage` (public; supports local + OIDC + TOTP challenge)
- `/about` → `AboutPage` (public)
- `/privacy` → `PrivacyPage` (public)
- `/release-notes` → `ReleaseNotesPage` (public)
- `/admin` → `AdminPage` (admin only)
- `/admin/about` → admin shell + `AboutPage admin` (admin only)
- `/admin/roadmap` → admin shell + `RoadmapPage` (admin only)
- `/admin/status` → admin shell + `StatusPage` (admin only)
- `/status` → redirects admin to `/admin/status`
- `/` (index under user layout) → `TrackerPage`
- `/calendar` → `CalendarPage`
- `/summary` → `SummaryPage`
- `/bills` → `BillsPage`
- `/subscriptions` → `SubscriptionsPage` (virtualized list, optimistic reorder, drag-reorder)
- `/subscriptions/catalog` → `SubscriptionCatalogPage` (admin view of the catalog)
- `/categories` → `CategoriesPage`
- `/health` → `HealthPage` (bill + payment health snapshot)
- `/analytics` → `AnalyticsPage`
- `/settings` → `SettingsPage`
- `/snowball` → `SnowballPage` (projection + plans)
- `/payoff` → `PayoffPage` (payoff scenarios)
- `/spending` → `SpendingPage` (categorized transactions, rules, budgets, income)
- `/bank-transactions` → `BankTransactionsPage` (the bank ledger, `Cents`-based)
- `/data` → `DataPage` (export, CSV/OFX/XLSX/SQLite imports, bank sync, transaction matching)
- `/profile` → `ProfilePage`
- `*` → `NotFoundPage`

### API client

`client/api.ts` (TypeScript; converted from `api.js` in the TypeScript migration):

- `_fetch<T>(method, path, body, opts)` calls `/api${path}` with JSON headers and `credentials: include`.
- Reads CSRF token from the `bt_csrf_token` cookie **or** fetches once from `/api/auth/csrf-token` and caches the token in memory; sends `x-csrf-token` on `POST`/`PUT`/`PATCH`/`DELETE`.
- Non-OK responses throw an `Error` whose `status`/`data`/`details`/`code` match the server's structured `errorFormatter`.
- `ApiError` interface carries those fields; `TogglePaidResult` and `PaymentRecord` provide typed payment responses; `QueryParams` types the query-string builder.
- Exposes grouped functions for `auth`, `admin`, `notifications`, `profile`, `tracker`, `calendar`, `summary`, `bills`, `payments`, `categories`, `settings`, `analytics`, `status`, `version`/`about`, `import`, `export`, `data-sources`, `transactions`, `matches`, `snowball`, `subscriptions`, `spending`, `monthlyStartingAmounts`, `csvImport`, `matchSuggestions`, `bankTransactionsLedger`, `matchTransaction`, `unmatchTransaction`, `ignoreTransaction`, `unignoreTransaction`, `applyTransactionMerchantMatch`, `autoCategorizeTransactions`, `categoryGroups`, `copySpendingBudgets`, `spendingIncome`, `spendingCategoryRules`, `createPaymentOrConfirm` (via `client/lib/paymentActions.ts`).
- File download/upload endpoints use raw `fetch` because responses/bodies are blobs or octet streams.
- **All 16 import sites** that referenced `@/api.js` were normalized to `@/api` so Vite/TS resolve the `.ts`.
- **Track C response typing** — see §7 *Domain types* for the full list. In short, the high-traffic domains (`auth`, `bills`, `payments`, `categories`, `tracker`, `snowball`, `subscriptions`, `spending`, `bank-ledger`) all return typed responses via `get<T>` / `post<T>`; the remaining single-consumer non-money casts (adminUsers, version/about/privacy/health/importHistory) are deliberately left as-is.
- **Server error-shape consistency (Track D).** The global 500 handler in `server.js` emits `{ error, code: 'INTERNAL_ERROR' }` so an unexpected server error carries the same `{error, code}` field the client can switch on as every structured 4xx. See §5 *Response conventions*.

### Domain types — `client/types.ts`

- Branded `Dollars` and `Cents` money types (from `client/lib/money.ts`).
- `User` (moved from `@/hooks/useAuth` in Track C; re-exported from `useAuth` for the 7 importers).
- `Bill`, `Payment`, `Category`, `TrackerResponse` (with `summary` / `bank_tracking` / `cashflow` / `rows`).
- `DriftBill`, `AutopaySuggestion`, `AmountSuggestion`, `AutopayStats`.
- `BankTransaction` (integer cents on the wire).
- `BankLedgerTransaction extends BankTransaction` (Track C: bank-ledger domain types) — preserves the `Cents` brand.
- `BankAccount`, `BankLedger`, `BankLedgerSummary`, `BankCategoryBreakdown`, `AutoCategorizePreview`, `AutoCategorizeChange`.
- `SnowballProjection` (Track C) + `SnowballProjectionDetail` / `AvalancheProjectionDetail` / `SnowballDebtProjection`.
- `SnowballSettings` (Track C).
- `Subscription`, `SubscriptionTopType`, `SubscriptionSummary`, `SubscriptionsResponse`, `CatalogMatch`, `ExistingBillMatch`, `Recommendation` (Track C: subscriptions domain types) + `RecommendationEvidence` / `RecommendationTransaction` / `SubscriptionTx`.
- `SpendingCategoryEntry`, `SpendingSummary`, `SpendingTransaction`, `SpendingTransactionsResponse`, `SpendingIncomeTx`, `SpendingIncomeResponse`, `SpendingRule`, `CategoryGroup`, `CopyBudgetsResponse` (Track C: spending domain types).
- `TimelineBill` for the Safe-to-Spend SVG timeline.
- Money fields are typed as `Dollars` for bill/payment/tracker/summary amounts and as `Cents` for raw bank transactions — so the unit mixup bug class is unrepresentable in typed code.
- A `money.type-test.ts` never-imported guard uses `@ts-expect-error` to assert each unit mixup is a real compile error, so `tsc --noEmit` fails loudly if the branding ever regresses.
- **Single-consumer non-money casts left in place** (deliberate Track C stop): `adminUsers` with its page-local `AdminUser`, and the one-off `version` / `about` / `privacy` / `health` / `importHistory` doc shapes. Central types add ~nothing for a single reader; typing them is the low-value tail.

### Auth state

`client/hooks/useAuth.tsx`:

- `AuthContext` typed (a `createContext(null)` would have made `useContext` return `never`).
- `User` interface lives in `client/types.ts` (moved there in Track C; `useAuth` re-exports it for the 7 importers that previously imported it from `@/hooks/useAuth`).
- Maintains `user`, `singleUserMode`, `loading`.
- Calls `api.authMode()` and `api.me()` on startup; `api.me()` is now `get<User>` (typed).
- Exposes `logout()`, `logoutAll()`, `refresh()`.
- `singleUserMode` is sourced from `/auth/mode`; `RequireAuth` consults it.

### Query hooks

`client/hooks/useQueries.ts` (TypeScript; converted from `useQueries.js`):

- `useTracker(year, month)`, `useTrackerHealth(year, month)`, `usePrefetchTracker()` — warms the adjacent-month cache on hover/focus.
- `useBills()`, `useCategories()`, `useSnowball*`, `useSpending*`, `useSummary`, `useBankLedger`, `useCategoriesWithSpending`, `useSubscriptions`, `useSubscriptionCatalog`, `useSubscriptionRecommendations`, `useMatchSuggestions`, `useTransactions`, `useTransactionsCategory`, `useTransactionCategories`, `useBankDataSources`, `useMatchBillDialog`, `useBankAccountDataSources`, `useBankSyncConfig`, `useAdminUserList`, `useAdminOIDCTest`, `useAdminBackups`, `useAdminCleanup`, `useAdminMigrations`, `useAdminAuthMode`, `useAdminNotifications`, `useReleaseNotes`, `useLoginHistory`, `useSnowballPlans`, `useWebAuthnStatus`, `useTOTPStatus`, `useUpcomingBills`.
- All hooks typed (params + `QueryParams`-typed query builders) and exported from `api.ts`.
- React Query v5: caches use `gcTime` (not `cacheTime`, which the v5 rename silently ignored). All mutation hooks invalidate / set query data via `setQueryData<T>(key, updater)`.
- Mutations: `useQuickPay` (shared across desktop + mobile tracker rows; `isPending` replaces manual busy flags; tracker/badge caches invalidate on settle), `useTogglePaid`, `useMarkPaid`, `useMatch`, `useUnmatch`, `useIgnore`, `useUnignore`, `useCategorizeTransaction`, `useBillCreate`, `useBillUpdate`, `useBillDelete`, `useBillReorder`, `useBudgetPut`, `useBudgetCopy`, `useCategoryRuleCreate`, `useCategoryRuleDelete`, `usePlanCreate`, `usePlanUpdate`, `usePlanPause`, `usePlanResume`, `usePlanComplete`, `usePlanAbandon`, `useSubscriptionLink`, `useSubscriptionDecline`, `useSubscriptionCreate`, `useSubscriptionUpdate`, `useImportCsv`, `useImportOfx`, `useBankSyncNow`, etc.
- `useAutoSave<T>(value, options)` is a generic debounced-save hook with an `AutoSaveStatus` union (`idle` | `pending` | `saving` | `saved` | `error`).
- `usePaymentActions.ts` — shared quick-pay/toggle-paid mutation hooks; typed payloads with `Dollars` on money amounts.
- `useSearchPanelPreference.ts` — typed `[boolean, setter]` tuple.

### Pages

- `LoginPage.tsx` — local login, OIDC login button, TOTP challenge, WebAuthn assertion, recovery code entry.
- `TrackerPage.tsx` — monthly tracker, payment interactions, upcoming bills, starting-amount awareness, safe-to-spend, drift/autopay/amount-suggestion badges, bulk/session logout, month-nav prefetch.
- `CalendarPage.tsx` — calendar grid backed by `/calendar`.
- `SummaryPage.tsx` — monthly plan, income, starting amounts, expenses, chart, bank-tracking overlay, skip-override.
- `BillsPage.tsx` — bill CRUD, categories, monthly state/history range controls, soft-delete + recently-deleted restore, merchant rules manager, bill historical-import dialog.
- `CategoriesPage.tsx` — category list/create/update/delete, group/category split, sort order, spending flag.
- `AnalyticsPage.tsx` — analytics summary filters and charts.
- `SettingsPage.tsx` — user settings and demo data seed.
- `DataPage.tsx` — export, spreadsheet import, user DB import, import history, CSV transaction import with preview + commit flow (`ImportTransactionCsvSection`), OFX import, bank sync.
- `ProfilePage.tsx` — display name, notification preferences, password change, TOTP setup, WebAuthn enrollment, login history, export/import-history links.
- `AdminPage.tsx` — users, auth mode/OIDC, WebAuthn, TOTP, backups, cleanup, notifications, migrations, admin settings, bank-sync config.
- `StatusPage.tsx` — admin system status.
- `AboutPage.tsx` — public or admin markdown/about view; admin mode uses `/about-admin`; markdown is sanitized.
- `RoadmapPage.tsx` — admin roadmap.
- `ReleaseNotesPage.tsx` — release history display; the `ReleaseNotesDialog` is shown to the user on login when their `last_seen_version` is older than the current package version.
- `PrivacyPage.tsx` — public privacy text.
- `SubscriptionsPage.tsx` (1814 lines) — `useOptimistic<Subscription[]>` reducer, virtualizer's `FlatItem` discriminated union, drag-reorder typed via the shared `DragProps`/`MoveControls`.
- `SubscriptionCatalogPage.tsx` — admin view of the top-200 catalog with bank descriptors.
- `SnowballPage.tsx` — projection, settings, plans lifecycle.
- `PayoffPage.tsx` — payoff scenarios.
- `SpendingPage.tsx` — categorized transactions, rules, monthly budgets with month-rollover, income.
- `BankTransactionsPage.tsx` — the `Cents`-based bank ledger, with a local `Tx` structurally compatible with `BankTransaction` so it still satisfies `MatchBillDialog`.
- `HealthPage.tsx` — bill + payment health snapshot.
- `NotFoundPage.tsx` — 404 fallback.

### Components

- **Layout**: `Layout`, `Sidebar`, `BrandBlock`, `NavPill`, `CommandPalette`, `PageTransition`.
- **Admin**: full `components/admin/` directory — users, auth methods, OIDC, WebAuthn, TOTP, bank-sync, backup manager, cleanup, notifications, migrations, settings cards.
- **Bill modal**: full `components/bill-modal/` directory plus `BillModal.tsx` (1092-line add/edit form, reached from 9 entry points).
- **Tracker**: full `components/tracker/` directory (18 files — the home page) where `fmt(row.expected_amount)` type-checks against `Dollars`.
- **Transactions**: `components/transactions/` directory.
- **Snowball**: `components/snowball/` directory (plan list, projection, settings).
- **Data**: full `components/data/` directory (14 files) — the Data page's SimpleFIN connect/sync workbench, transaction matching, and CSV / OFX / XLSX-spreadsheet / SQLite import flows, with `Cents`-branded bank-transaction amounts flowing through.
- **Domain UI**: `BillModal`, `BillsTableInner`, `MobileBillRow`, `MobileTrackerRow`, `StatusBadge`, `SummaryCard`, `MarkdownText`, `ReleaseNotesDialog`, `IncomeBreakdownModal`, `BillHistoricalImportDialog`, `BillRulesManager`, `BillMerchantRules`, `RecentlyDeletedBillsDialog`, `CalendarFeedManager`, `SearchFilterPanel`, `SubscriptionCatalogSection`.
- **Reliability**: `ErrorBoundary` (typed class component), `PageLoader`, `PageTransition`.
- **UI primitives** (`components/ui/`): `alert-dialog`, `badge`, `button`, `card`, `checkbox`, `collapsible`, `confirm-dialog`, `dialog`, `dropdown-menu`, `input`, `input-dialog`, `label`, `save-status`, `select`, `separator`, `skeleton`, `sonner`, `switch`, `table`, `tabs`, `theme-toggle`, `tooltip` — all `.tsx`.

### `client/lib/`

- `money.ts` — branded `Cents`/`Dollars` types, `asCents`/`asDollars`/`centsToDollars`/`dollarsToCents`, formatters. A `formatUSD(dollars)` and `formatCentsUSD(cents)` split mirrors the server's `formatUSD`/`formatCentsUSD` split. Inputs are coerced defensively so null / `''` / undefined / NaN never render as `$NaN`.
- `utils.ts` — `cn`, `fmt`, date/format helpers, `errMessage(unknown, fallback)` for narrowing `unknown` caught errors into strings. `fmt` inherits `formatUSD`'s branded-dollars input.
- `trackerUtils.ts` — row/status/sort helpers, `TrackerRow` domain interface, `TrackerStatus` union.
- `billDrafts.ts` — `SourceBill` / `Template` / `Category` shapes for `makeBillDraft`.
- `billingSchedule.ts` — `Schedule` union and helpers.
- `cashflowUtils.ts` — typed SVG timeline geometry for the Safe-to-Spend card.
- `trackerTableColumns.ts` — typed column descriptors.
- `reorder.ts` — generic `moveInArray<T>`.
- `version.ts` — typed release-notes shapes; the Vite-injected `__APP_VERSION__` is declared inline.
- `paymentActions.ts` — `createPaymentOrConfirm(payload, onSuccess)` (Track A). The deliberate manual-add path treats the server's `409 DUPLICATE_SUSPECTED` as a question rather than an error: losing a legitimately-identical second payment (same bill, date, amount) is itself a money-integrity bug, so the server returns 409 instead of silently deduping and the helper surfaces a sonner "Add anyway?" toast whose confirm action resends with `allow_duplicate: true`. `onSuccess` runs after any real creation (immediate or confirmed retry). Non-duplicate errors re-throw for the caller's existing `catch`.

### Frontend build & lint

- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`).
- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`. Wired into `npm run ci`.
- `npm run lint` — `eslint client` (flat config; `react-hooks/rules-of-hooks`, `react-hooks/exhaustive-deps`, `react-refresh/only-export-components`, typescript-eslint parser for `.ts/.tsx`).
- `npm run test` — server `node --test tests/*.test.js`.
- `npm run test:client` — `vitest run` (client unit tests).
- `npm run test:all` — both.
- `npm run test:e2e` — Playwright (chromium desktop + mobile); `prepare-db.js` reseeds the test DB.
- `npm run check:server` — `node --check` on every server JS file.
- `npm run check` — `check:server` + `build`.
- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`.

### Frontend security notes

- CSRF header is sent on `POST`/`PUT`/`PATCH`/`DELETE`; the token is fetched once from `/api/auth/csrf-token` and cached in memory.
- Auth is cookie/session based; no tokens are stored in localStorage. Session token hashing (`v0.94`) means a leaked DB does not yield usable session IDs.
- Admin routes are client-guarded and server-guarded; the default admin is redirected away from user routes.
- Markdown rendering uses `rehype-sanitize`; CSP headers in `middleware/securityHeaders.js` cover the built assets.
- Error boundaries prevent route crashes from taking down the whole SPA.
- PWA: `vite-plugin-pwa` registers a service worker (autoUpdate) with manifest + icons. Runtime caching uses `NetworkFirst` for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` with a 5-second network timeout, 30-entry max, 5-minute max-age.

---

## 8. Infrastructure and Deployment

### `package.json`

Version: `0.40.0`. `"type": "commonjs"`.

Scripts:

- `npm run dev:api` — `node --watch server.js`.
- `npm run dev:ui` — Vite dev server.
- `npm run dev` — `concurrently` runs API and UI.
- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`).
- `npm run check:server` — `node --check` every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils`.
- `npm run lint` — `eslint client` (flat config; `react-hooks` + `react-refresh` + `typescript-eslint`).
- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`.
- `npm run check` — `check:server` + `build`.
- `npm run test` — `node --test tests/*.test.js` (server).
- `npm run test:client` — `vitest run` (client).
- `npm run test:all` — both.
- `npm run test:e2e` / `test:e2e:ui` / `test:e2e:update` / `test:e2e:probe` — Playwright (chromium desktop + mobile + axe), with `e2e/setup/prepare-db.js` reseeding the test DB.
- `npm run smoke:prod` — `bash scripts/prod-smoke.sh`.
- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`. Runs in Forgejo Actions on every push and PR (`.forgejo/workflows/ci.yml`, container `node:22-bookworm`).
- `npm start` — `node server.js`.

Key runtime dependencies:

- Express 4, cookie-parser, cors, express-rate-limit.
- better-sqlite3 12.
- bcryptjs.
- openid-client 5.
- @simplewebauthn/server 13, @simplewebauthn/browser 13.
- otplib 13, qrcode 1.
- nodemailer 8.
- node-cron 4.
- React 19, React DOM 19, React Router 6, TanStack Query 5, TanStack Virtual 3, framer-motion 12.
- shadcn/ui component primitives, Radix UI primitives, lucide-react, Tailwind utilities, Sonner toasts.
- xlsx for spreadsheet import/export.

Key dev dependencies:

- typescript 6, @types/react 19, @types/react-dom 19, typescript-eslint 8.
- eslint 9, @eslint/js, eslint-plugin-react-hooks 5, eslint-plugin-react-refresh 0.4, globals 17.
- vite 5, @vitejs/plugin-react 4, vite-plugin-pwa 1, babel-plugin-react-compiler 1.
- vitest 4, @testing-library/react 16, @axe-core/playwright 4, @playwright/test 1.
- tailwindcss 3, autoprefixer 10, postcss 8.

### Dockerfile

Multi-stage build:

1. Builder: `node:22-alpine`, installs build deps `python3 make g++`, runs `npm install`, copies source, runs `npm run build`.
2. Runtime: `node:22-alpine`, installs `bash nano su-exec`, creates non-root `bill` user, copies built app, creates `/data/db`, `/data/backups`, `/app/backups`, sets ownership and restrictive permissions.

Runtime environment:

- `NODE_ENV=production`
- `PORT=3000`
- `DB_PATH=/data/db/bills.db`
- `BACKUP_PATH=/data/backups`

Exposes port 3000, declares volume `/data`, entrypoint `docker-entrypoint.sh`, command `node server.js`.

### `docker-compose.yml`

Service: `bill-tracker`.

- Image: `dream.scheller.ltd/null/billtracker:latest`.
- Container name: `bill-tracker`.
- Ports: host `3030` to container `3000`.
- Volume: `/portainer/hosting/bill-tracker/data:/data`.
- Restart: `unless-stopped`.
- Environment includes `INIT_ADMIN_USER`, `INIT_ADMIN_PASS`, and CSRF cookie settings (`CSRF_HTTP_ONLY: "false"`, `CSRF_SAME_SITE: "strict"`, `CSRF_SECURE: "true"`, `CSRF_COOKIE_NAME: "bt_csrf_token"`).

Important deployment note: the compose file currently sets `CSRF_SECURE: "true"`; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to `false` only in local/dev.

### CI (`.forgejo/workflows/ci.yml`)

Forgejo Actions runs on every push and PR: `actions/checkout@v4` on `node:22-bookworm` (matches Dockerfile runtime so better-sqlite3 prebuilds resolve identically), then `npm ci` and `npm run ci`. The workflow is the only place a build is fully re-verified end-to-end (lint + typecheck + server tests + client tests + build).

### Vite (`vite.config.mjs`)

- `@vitejs/plugin-react` with `babel-plugin-react-compiler` enabled (React 19 auto-memoization). The codebase is rules-of-hooks clean (enforced by `eslint-plugin-react-hooks`).
- `vite-plugin-pwa` with `registerType: 'autoUpdate'`, manifest, icons in `client/public/img/`, runtime caching for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` via `NetworkFirst` (5s network timeout, 30-entry max, 5-min max-age).
- `__APP_VERSION__` is injected at build time from `package.json`; the SPA reads it through `client/lib/version.ts`.
- Manual chunk splitting: `vendor-react` (react / react-dom / react-router-dom), `vendor-radix` (dialog / select / dropdown-menu / tabs / tooltip / alert-dialog), `vendor-motion` (framer-motion), `vendor-query` (@tanstack/react-query + @tanstack/react-virtual).
- `resolve.alias['@']` → `client/`, matches the Vite alias and the `tsconfig.json` `paths` mapping.
- `server.port: 5173`, proxies `/api` to `http://localhost:${API_PORT || PORT || 3000}`.
- `test.environment: 'node'`; tests opt into jsdom via `@vitest-environment` per-file.
- `test.include: ['client/**/*.test.{js,jsx}']`; **note**: client tests on `.ts` are currently run via the `tests/*.test.js` server suite (which still exercises the same behaviors). The Vitest `include` is restricted to `.js/.jsx` to keep current test coverage green during the gradual TS migration.

### TypeScript (`tsconfig.json`)

- `target: ES2022`, `lib: [ES2023, DOM, DOM.Iterable]`.
- `module: ESNext`, `moduleResolution: bundler`, `jsx: react-jsx`.
- `paths: { "@/*": ["./client/*"] }`, `resolveJsonModule`, `esModuleInterop`, `isolatedModules`, `skipLibCheck`, `noEmit`.
- `allowJs: true`, `checkJs: false` — legacy `.js`/`.jsx` keep resolving and running untouched; only `.ts`/`.tsx` are type-checked. Convert file-by-file.
- `strict: true`, `noUncheckedIndexedAccess: true`.
- `types: ["vite/client"]` — provides `import.meta.env` and `__APP_VERSION__` typings.
- `include: ["client/**/*"]`, `exclude: ["node_modules", "dist", "client/**/*.test.*"]`.

### ESLint (`eslint.config.mjs`)

Flat config scoped to the React client. The point is enforcement of the two rules that catch real React bugs — `rules-of-hooks` (conditional hooks) and `exhaustive-deps` (stale closures) — which nothing previously checked. Server code (CommonJS Node) is out of scope here; `npm run check:server` covers it.

- JS/JSX block (`client/**/*.{js,jsx}`): `js.configs.recommended` + `react-hooks` + `react-refresh`. `react-hooks/rules-of-hooks: error`, `react-hooks/exhaustive-deps: warn`, `react-refresh/only-export-components: warn` (with `allowConstantExport: true`).
- TS/TSX block (`client/**/*.{ts,tsx}`): same react rules via the `typescript-eslint` parser. `no-undef` and `no-unused-vars` off (TypeScript handles them); `@typescript-eslint/no-unused-vars: warn` with `^[A-Z_]` ignore pattern.
- `no-empty: warn` with `allowEmptyCatch: true`.
- Ignores: `dist/**`, `node_modules/**`, `coverage/**`, `client/**/*.test.*`.

---

## 9. Auth and Security Flows

### Local login

1. Client calls `GET /auth/mode` to determine local/OIDC visibility.
2. Client submits `POST /auth/login`.
3. Server checks `local_login_enabled`, validates credentials through bcrypt (cost 12), rejects inactive users, cleans expired sessions, creates a 7-day session, **hashes the session token** for storage (`v0.94`), and sets the `bt_session` cookie with the raw opaque ID.
4. Server sets `bt_session` cookie using `cookieOpts(req)`.
5. Client calls `GET /auth/me` to populate auth state.

### TOTP 2FA

1. User enables TOTP via `GET /auth/totp/setup` → scans QR code → `POST /auth/totp/enable` with `{secret, token}`. Server encrypts the secret with `services/encryptionService` and stores 8 recovery codes (hashed).
2. On subsequent logins, after a successful `POST /auth/login`, the server returns `{ totp_required: true, challenge_id }`. The SPA prompts for the 6-digit code and submits it (or a recovery code) with the challenge id.
3. `POST /auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id`; the SPA submits it with the token on the next request.
4. `disableTotp(userId)` clears the encrypted secret and recovery codes.

### WebAuthn 2FA

1. User enrolls a security key: `GET /auth/webauthn/status` (or the admin equivalent) returns current credentials. The SPA calls an internal helper that triggers a registration ceremony; the server stores the credential in `webauthn_credentials` (AAGUID, transports, backup flags, friendly name).
2. On login, the SPA requests an assertion via `webauthn_login` flow; `createAuthenticationChallenge` returns options + a 15-minute `challenge_id`; `verifyAuthentication` updates the sign count and audits `webauthn.login`.
3. `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`) must match the browser-visible hostname, or registration/assertion will fail.

### OIDC login

1. Client navigates to `/api/auth/oidc/login`.
2. Server verifies active OIDC config (the OIDC client secret is decrypted from `settings.oidc_client_secret`, encrypted at rest by `v0.79`), creates PKCE state, redirects to provider.
3. Provider returns to `/api/auth/oidc/callback`.
4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie (token hashed in `sessions.id`), redirects back into SPA.

### Password change

1. Current password and matching new password are required.
2. New password must be at least 8 chars.
3. Server updates hash, clears `must_change_password`, sets `last_password_change_at`.
4. Other sessions are invalidated and current session is rotated when possible.

### Login history + device fingerprint

- Every login (successful or failed) writes a `user_login_history` row.
- `ip_address` and `user_agent` are encrypted at rest (`v0.84`).
- `location` (IP-geolocation city/region/country) is captured only when the user has `users.geolocation_enabled = 1` (`v1.02`).
- `success` (`v0.85`) tracks failed-attempt history separately.
- `session_fingerprint` lets the SPA detect a different device for the same session and trigger a re-auth challenge.

### Authorization

- All user data routes enforce owner scope by `req.user.id` in SQL.
- Admin-only routes require `requireAdmin` on server.
- Default admin cannot use tracker routes.
- Role changes invalidate target sessions.
- Deactivation invalidates target sessions.

### Secret encryption at rest

- `services/encryptionService.js` provides `encryptSecret` / `decryptSecret` / `assertEncryptionReady` using a master key derived with HKDF (`v0.78`; the prior SHA-256 derivation was migrated in-place).
- Encrypted at rest: `oidc_client_secret` (`v0.79`), SMTP password (`v0.77`), `data_sources.encrypted_secret` (SimpleFIN access URLs), `user_login_history.ip_address` / `user_agent` (`v0.84`), `users.totp_secret` (`v0.86`).
- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected.

### Backup safety

- Managed filename regex and path checks prevent traversal.
- Uploads are written to temp paths first, validated by SQLite `integrity_check` + optional SHA-256 checksum, then moved.
- Restore creates a pre-restore backup.

### Import safety

- Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions.
- User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership.
- CSV import preview/apply is split; ambiguous rows are surfaced in `errors` for the user to resolve.
- OFX/QFX import is rate-limited via the import limiter; previews have a 24-hour TTL.

### Bank sync safety

- SimpleFIN access URLs are encrypted at rest (`data_sources.encrypted_secret`).
- `assertEncryptionReady` is called before any secret operation.
- The `monitored` flag (`v0.64`) lets users exclude a credit-card account from bill matching.
- Transactions are deduplicated by `provider_transaction_id` (unique partial index).
- Account balance updates run in transactions; partial failures roll back.

---

## 10. Operational Notes

### Startup behavior

- DB path is `DB_PATH` or `db/bills.db`.
- DB open logging prints only `path.basename(DB_PATH)` instead of the full database path, so startup logs identify the file without exposing the full filesystem location.
- SQLite WAL and foreign keys are enabled.
- Schema and migrations run automatically (`db/migrations/versionedMigrations.js` + `db/migrations/legacyReconcileMigrations.js`).
- Default categories/settings are seeded (incl. spending defaults for new users in `v0.87`).
- Expired sessions are purged at startup; a periodic `SESSION_CLEANUP_INTERVAL_MS` cleanup interval is scheduled.
- `bankSyncWorker.start()` is started; `backupScheduler` is started; `workers/dailyWorker.start()` runs the daily jobs.
- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked.

Environment-seeded regular users use `INIT_REGULAR_USER` and `INIT_REGULAR_PASS`. New seeded users are inserted with `first_login = 0` and `must_change_password = 0`. Existing seeded regular users have their password hash updated and both flags reset to `0`, then the server audits `seed.flag_reset` with the username, reset flags, and `source: "server-seed"`. This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes.

### Environment variables

Common variables used by current code:

- `PORT` — server port (default 3000)
- `BIND_HOST` — server bind (default `0.0.0.0`)
- `DB_PATH`
- `BACKUP_PATH`
- `INIT_ADMIN_USER`
- `INIT_ADMIN_PASS`
- `INIT_REGULAR_USER`
- `INIT_REGULAR_PASS`
- `SESSION_CLEANUP_INTERVAL_MS`
- `TRUST_PROXY` — `true` / `1` / numeric / `loopback` / CIDR; required behind nginx/Traefik
- `CORS_ORIGIN` — comma-separated allowlist
- `COOKIE_SECURE`
- `HTTPS`
- `CSRF_HTTP_ONLY`
- `CSRF_SAME_SITE`
- `CSRF_SECURE`
- `CSRF_COOKIE_NAME`
- `OIDC_ISSUER_URL`
- `OIDC_CLIENT_ID`
- `OIDC_CLIENT_SECRET` — encrypted at rest (`v0.79`); the settings-table copy is the source of truth
- `OIDC_REDIRECT_URI`
- `OIDC_TOKEN_AUTH_METHOD`
- `WEBAUTHN_RP_ID` — default `localhost`
- `WEBAUTHN_ORIGIN` — default `http://localhost:${PORT}`
- `ENCRYPTION_KEY` — master key for at-rest encryption (HKDF, `v0.78`)
- `API_PORT` — Vite dev-server proxy target (defaults to `PORT || 3000`)
- `DATA_IMPORT_ENABLED` — gates CSV import (defaults to true)

Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the `settings` table and managed from Admin UI.

### Known code characteristics to preserve

- Use transactions for multi-step destructive or bulk DB changes.
- Keep user-owned SQL scoped by `req.user.id`.
- Keep admin lockout protection before changing login methods.
- Do not expose `password_hash`, session IDs (raw or hashed), OIDC client secret, encrypted secret blobs, or internal backup paths in API responses.
- Keep import preview/apply separated so users can resolve ambiguous spreadsheet / CSV / OFX / user-DB data before DB writes.
- Branded `Dollars` / `Cents` types in `client/lib/money.ts` are the source of truth for client money handling. A bare number at a `fmt()` / `formatUSD` / `formatCentsUSD` boundary is a compile error in typed code. The `money.type-test.ts` guard uses `@ts-expect-error` so a regression in the branding fails `tsc --noEmit`.
- Tailwind's `content` glob must include `ts,tsx` (`./client/**/*.{js,jsx,ts,tsx}`). Without it, classes used only in `.tsx` files generate no CSS and produce a runtime-only failure that the e2e probe (not typecheck/build/tests) is the only gate that catches.
- DB path support: `db/database.js` uses `path.basename(DB_PATH)` in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported.
- React Compiler (`babel-plugin-react-compiler`) requires a rules-of-hooks-clean codebase; do not introduce conditional hooks.
- React Query v5: use `gcTime`, not `cacheTime` (silently ignored in v5).

---

## 11. Verification Checklist Used for This Reference

Reviewed current code sources:

- `server.js` (including `TRUST_PROXY` handling, health probe, route mount order, post-`listen` worker startup).
- All 26 route files under `routes/`.
- All 41 service files under `services/` (incl. `bankSync*`, `simplefinService`, `ofxImportService`, `merchantStoreMatchService`, `advisoryFilterService`, `totpService`, `webauthnService`, `loginFingerprint`, `driftService`, `aprService`, `paymentAccountingService`, `encryptionService`, `calendarFeedService`).
- All middleware files (`csrf`, `errorFormatter`, `rateLimiter`, `requireAuth`, `securityHeaders`).
- `db/schema.sql`, `db/database.js`, `db/migrations/versionedMigrations.js`, `db/migrations/legacyReconcileMigrations.js`, `db/subscriptionCatalogSeed.js`.
- Actual initialized SQLite schema via `better-sqlite3` introspection (`PRAGMA table_info`, `PRAGMA index_list`).
- `client/main.tsx`, `client/App.tsx`, `client/api.ts`, `client/types.ts`, `client/contexts/ThemeContext.tsx`.
- `client/hooks/useAuth.tsx`, `client/hooks/useQueries.ts`, `client/hooks/usePaymentActions.ts`, `client/hooks/useAutoSave.ts`, `client/hooks/useSearchPanelPreference.ts`.
- Page inventory under `client/pages/` (24 files) and component inventory under `client/components/` (incl. `admin/`, `bill-modal/`, `tracker/`, `transactions/`, `snowball/`, `data/`, `layout/`, `ui/`).
- `client/lib/` (`money.ts`, `utils.ts`, `trackerUtils.ts`, `billDrafts.ts`, `billingSchedule.ts`, `cashflowUtils.ts`, `trackerTableColumns.ts`, `reorder.ts`, `version.ts`).
- `package.json` (version 0.40.0; `"type": "commonjs"`), `tsconfig.json`, `eslint.config.mjs`, `vite.config.mjs`.
- `Dockerfile` (node:22-alpine), `docker-compose.yml`, `docker-entrypoint.sh`, `.forgejo/workflows/ci.yml`.

End-to-end checks run against the codebase:

- `find client -name '*.jsx'` returns 0 — the JSX → TSX migration is complete.
- `find client -name '*.ts'` and `find client -name '*.tsx'` together cover every previously `.jsx` file in `components/`, `pages/`, `hooks/`, `contexts/`, plus the API client and `lib/`.
- `grep` on `package.json` for React 19, TypeScript 6, Vite 5, TanStack Query 5, `@simplewebauthn/*` 13, `otplib` 13.
- `npm run typecheck` was green at the time of the v0.41.0 entry in `HISTORY.md` (0 errors); 48 client unit tests pass; 17/17 e2e probe (every page renders, all API paths respond).
- `node --check` on every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils` (`npm run check:server`).

Track A/B/C/D additions (targeting v0.41.0, un-pushed as of this update):

- `6084896` (Track A) — `fix(money): make every payment balance-mutation atomic`
- `dd5bf92` (Track A) — `fix(money): manual payment path flags suspected dupes (409) w/o losing legit ones`
- `c223f62` (Track B) — `test(money): cross-surface reconciliation harness`
- `b267599` (Track C) — `refactor(ts): type the bank-ledger API responses`
- `8265b4a` (Track C) — `refactor(ts): type the spending API responses`
- `6bb8c63` (Track C) — `refactor(ts): type the subscriptions API responses`
- `65a477c` (Track C) — `refactor(ts): type snowball projection + settings responses`
- `1df4b1b` (Track C/D) — `refactor(ts): move User to @/types + type auth endpoints; add code to 500`

Diff stats across the 8 commits: 19 files changed, 813 insertions(+), 415 deletions(-). Notable: `client/types.ts` grew by 240 lines; new file `client/lib/paymentActions.ts` (~40 lines, `createPaymentOrConfirm`); `client/api.ts` re-exports 9 new typed methods; `routes/payments.js` grew by ~150 lines for the transactional + duplicate-detection logic; `server.js` gained a single line (`code: 'INTERNAL_ERROR'`).

The previous manual (v0.28.1) contained stale route/page descriptions and missed major features added between v0.28 and v0.40 (subscriptions, spending, snowball plans, SimpleFIN, WebAuthn, TOTP, push notifications, category groups, calendar feed, money-cents migration in progress, React 18 → 19 + TypeScript migration, branded money types). This version replaces that with a current-state engineering reference.