From 6a2db3780f66acce10e574434220ff72a79106fb Mon Sep 17 00:00:00 2001 From: null Date: Sun, 5 Jul 2026 11:19:14 -0500 Subject: [PATCH] docs(reference): refresh engineering manual for v0.40.0 / TS+React 19 Reflects the current codebase after the TypeScript + React 19 migration and the v0.28.1 -> v0.40.0 feature work (subscriptions, spending, snowball plans, SimpleFIN, WebAuthn, TOTP, push notifications, calendar feed, category groups, money-cents migration in progress, encrypted secrets, Node 18 -> 22, Forgejo CI, branded Dollars/Cents types). - Bump version 0.28.1 -> 0.40.0; add Notable-changes summary. - Reorder API reference (5.1-5.22) to cover all 26 routes incl. TOTP, WebAuthn, OIDC, spending, subscriptions, snowball plans, health, calendar feed, update-status. - Add services for 2FA, encryption, login fingerprint, bank sync, OFX import, merchant rules, merchant-store matches, advisory filters, drift, APR, payment-accounting, calendar feed. - Schema: add 12 new tables, document migrations v0.65 -> v1.06, expand ROLLBACK_SQL_MAP through v1.04 (v1.05/v1.06 not yet rollbackable). - New section on client/lib/ (branded money, TrackerRow, Schedule, cashflowUtils, etc.). - New auth/security flows: TOTP challenge, WebAuthn ceremony, session token hashing, encrypted-at-rest secrets, login fingerprint + location, bank-sync safety. - Add Vite/TS/ESLint config sections; update Dockerfile to node:22-alpine; add Forgejo CI workflow note. - Expand env vars with TRUST_PROXY, WEBAUTHN_*, ENCRYPTION_KEY, BIND_HOST, API_PORT, DATA_IMPORT_ENABLED. --- docs/Engineering_Reference_Manual.md | 1782 +++++++++++++------------- 1 file changed, 878 insertions(+), 904 deletions(-) diff --git a/docs/Engineering_Reference_Manual.md b/docs/Engineering_Reference_Manual.md index 54f702a..1c2e79b 100644 --- a/docs/Engineering_Reference_Manual.md +++ b/docs/Engineering_Reference_Manual.md @@ -1,11 +1,20 @@ # Engineering Reference Manual — Bill Tracker **Status:** Current code reference -**Last Updated:** 2026-05-16 -**Version:** 0.28.1 -**Primary stack:** Node.js + Express, React + Vite, Tailwind CSS + shadcn/ui, Sonner, SQLite via `better-sqlite3` +**Last Updated:** 2026-07-05 +**Version:** 0.40.0 +**Primary stack:** Node.js 22 + Express 4 (CommonJS), React 19 + Vite 5 (TypeScript, `strict` + `noUncheckedIndexedAccess`), Tailwind CSS 3 + shadcn/ui, TanStack Query 5, Sonner, SQLite via `better-sqlite3` 12, `@simplewebauthn/*` 13, `otplib` 13, `openid-client` 5 -This manual reflects the current application code in `server.js`, `routes/`, `services/`, `middleware/`, `db/`, `client/`, `package.json`, `Dockerfile`, and `docker-compose.yml`. It is written as a current-state reference, not a changelog. +This manual reflects the current application code in `server.js`, `routes/`, `services/`, `middleware/`, `db/`, `client/`, `package.json`, `Dockerfile`, `docker-compose.yml`, `tsconfig.json`, `eslint.config.mjs`, and `vite.config.mjs`. It is written as a current-state reference, not a changelog. The previous version of this manual (v0.28.1) is fully superseded. + +Notable changes since the previous manual: + +- **Client migrated JSX → TSX.** Full strict TypeScript with `noUncheckedIndexedAccess`. `allowJs` + `checkJs:false` keep legacy `.js` running while `.ts/.tsx` are type-checked. `npm run typecheck` wired into CI. +- **Branded money types.** `client/lib/money.ts` exposes `Cents` / `Dollars` branded types — a typed caller can no longer format cents as dollars (the 100× bug) by accident. +- **React 18 → React 19** with `babel-plugin-react-compiler` for automatic memoization. Vite 5 with `vite-plugin-pwa` for service-worker / installable PWA. +- **Node 18 → Node 22** in Dockerfile; `node:22-bookworm` in CI. +- **New features:** SimpleFIN bank sync, WebAuthn + TOTP 2FA, calendar feed (`/api/calendar/feed.ics`), subscription catalog + recommendations, spending categories / budgets, snowball plans (lifecycle), push notifications (ntfy / Gotify / Discord / Telegram), push notification column on users, encrypted secrets at rest with HKDF-derived key, money-cents migration (in progress), category groups, banker-rule auto-attribute. +- **Schema:** migrations `v0.2` → `v1.06`. Rollback map covers `v0.44` → `v1.04`. --- @@ -13,20 +22,24 @@ This manual reflects the current application code in `server.js`, `routes/`, `se Bill Tracker is a self-hosted bill management application. It supports: -- Local username/password authentication, optional single-user mode, and optional Authentik/OIDC login. -- User-scoped bills, categories, payments, monthly bill overrides, monthly income, and starting cash buckets. -- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, and migration rollback. -- Spreadsheet and user-SQLite import workflows. -- CSV, Excel, and user-SQLite export workflows. -- SMTP-based bill due notifications. -- React SPA frontend with protected user/admin routes and CSRF-protected JSON APIs. +- Local username/password authentication, optional single-user mode, optional Authentik/OIDC login, and optional WebAuthn / FIDO2 + TOTP two-factor authentication. +- User-scoped bills, categories (with `category_groups` and `spending_enabled` flag), payments, monthly bill overrides, monthly income, and starting cash buckets. +- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, migration rollback, and SimpleFIN bank-sync admin configuration. +- Spreadsheet, user-SQLite, OFX, and CSV transaction import workflows with preview/apply split. +- CSV, Excel, and user-SQLite export workflows; an iCalendar feed (`/api/calendar/feed.ics`) of upcoming bills gated by per-user tokens. +- SMTP-based bill due notifications plus ntfy / Gotify / Discord / Telegram push notifications. +- SimpleFIN-based bank sync (data sources, financial accounts, transactions, match suggestions, auto-categorize, merchant rules, merchant-store matches). +- Subscription catalog (top-200 known services), per-user recommendations, declined-subscription hints, recommendation feedback. +- Spending module: categorized transactions, rules, monthly budgets, income breakdown, copy-month workflow. +- Snowball plans: lifecycle (active / paused / completed / abandoned), debt-order projection (snowball/avalanche/minimum-only), payoff scenarios. +- React 19 SPA in TypeScript with branded `Cents` / `Dollars` money types, lazy-loaded routes, global QueryCache error toasts, and CSRF-protected JSON APIs. Runtime flow: -1. `server.js` initializes SQLite through `db/database.js`, runs schema/migrations, seeds defaults/admin user, cleans expired sessions, then starts Express. -2. Express applies security headers, JSON body parsing, cookies, CSRF token provisioning, route-level CSRF/auth/rate-limit middleware, static `dist/` serving, and JSON error formatting. -3. Route files under `routes/` validate input, enforce ownership through `req.user.id`, and use service modules for business logic. -4. React under `client/` calls `/api/*` through `client/api.js` with `credentials: include` and CSRF headers on mutating methods. +1. `server.js` initializes SQLite through `db/database.js`, runs `db/migrations/versionedMigrations.js` plus `legacyReconcileMigrations.js`, seeds defaults/admin user, cleans expired sessions, then starts Express. +2. Express applies `securityHeaders`, optional CORS, `express.json({ limit: '100kb' })`, `cookieParser`, `csrfTokenProvider`, the patched `res.json` error formatter, and an unauthenticated `GET /api/health` probe. +3. Route files under `routes/` validate input, enforce ownership through `req.user.id`, and use service modules for business logic. WebAuthn and TOTP challenges use a 5–15 minute TTL in `webauthn_challenges` and `totp_challenges`. +4. React 19 under `client/` calls `/api/*` through `client/api.ts` with `credentials: include` and CSRF headers on mutating methods; the CSRF token is fetched once from `/api/auth/csrf-token` and cached in memory. --- @@ -34,30 +47,45 @@ Runtime flow: - `server.js` — Express entry point and route mounting. - `routes/` — HTTP API handlers: - - `auth.js` — login, logout, password change, OIDC callback. - - `bills.js` — bills CRUD, auto-mark paid, history. - - `payments.js` — payments CRUD, status matching, snowball handling. - - `categories.js` — category CRUD, tree support. - - `tracker.js` — monthly tracker data, bucket resolution, cycle handling. - - `summary.js` — summary stats, starting amounts. - - `analytics.js` — expense reports, category breakdown. + - `auth.js` — login, logout, logout-all, password change, TOTP setup/enable/disable/challenge/status, WebAuthn status, login history, CSRF token endpoint, mode, acknowledge-version/privacy. + - `authOidc.js` — OIDC login and callback. + - `bills.js` — bills CRUD, auto-mark paid, history ranges, soft-delete, recently-deleted restore. + - `payments.js` — payments CRUD, status matching, snowball handling, bank-override accounting, bulk/quick. + - `categories.js` — category CRUD, sort order, spending/bill split, category groups. + - `tracker.js` — monthly tracker data, bucket resolution, cycle handling, drift, autopay suggestions, amount suggestions, safe-to-spend, three-month averages. + - `summary.js` — summary stats, income, starting amounts, bank-tracking overlay, skip override. + - `analytics.js` — expense reports, category breakdown, month-window filtering. - `settings.js` — user settings, admin config, notification settings. - - `notifications.js` — notification management. - - `profile.js` — user profile, demo data. - - `import.js` — CSV, Excel, user-SQLite import workflows. + - `notifications.js` — notification management (admin SMTP + push). + - `profile.js` — user profile, notification settings, change-password, export URLs, import history, login history. + - `import.js` — CSV / Excel / user-SQLite / OFX import workflows. - `export.js` — CSV, Excel, SQLite export. - - `status.js` — system status, health. - - `dataSources.js` — new — data sources CRUD with sync status. - - `transactions.js` — new — transaction CRUD, match/ignore/commit actions. - - `matches.js` — new — match suggestions, rejection tracking. -- `services/` — business logic modules. + - `status.js` — system status, health, runtime. + - `dataSources.js` — data sources CRUD with sync status (manual / file_import / provider_sync). + - `transactions.js` — transaction CRUD, match/ignore/commit actions, manual entry, pending flag, geolocation. + - `matches.js` — match suggestions, rejection tracking. + - `snowball.js` — snowball projection, settings, debt order, **plan lifecycle** (create / list / active / pause / resume / complete / abandon). + - `subscriptions.js` — user subscriptions, **recommendations** (list / decline / match-bill / create), transaction matches, catalog links, custom descriptors. + - `spending.js` — categorized spending summary, transactions, budgets (PUT / copy / month-rollover), category rules, income. + - `calendarFeed.js` — public iCalendar feed (`GET /feed.ics`) per-user token-gated. + - `admin.js` — users (incl. username change), backups, cleanup, auth-mode, **bank-sync-config**, migrations rollback. + - `about.js`, `aboutAdmin.js` — public + admin markdown about view. + - `privacy.js` — public privacy page. + - `version.js` — public version, history, update-status. +- `services/` — business logic modules. See §4. - `middleware/` — auth guards, CSRF, rate limits, security headers, error formatting. - `db/schema.sql` — base SQLite schema. -- `db/database.js` — DB connection, migrations, defaults, settings, rollback support. +- `db/database.js` — DB connection, migrations runner, defaults, settings, rollback support. +- `db/migrations/versionedMigrations.js` — ordered, dependency-aware migrations `v0.2` → `v1.06`. +- `db/migrations/legacyReconcileMigrations.js` — column-level reconciliation for unversioned columns that pre-date the migration framework. +- `db/subscriptionCatalogSeed.js` — top-200 US subscriptions dataset used by `v0.65` / `v0.69` / `v0.95`. - `workers/dailyWorker.js` — scheduled notification/cleanup worker entry. -- `client/` — React SPA. +- `client/` — React 19 SPA in TypeScript (see §7). - `dist/` — generated Vite build output served by Express. - `Dockerfile`, `docker-compose.yml`, `docker-entrypoint.sh` — container deployment. +- `tsconfig.json` — TypeScript config (strict + `noUncheckedIndexedAccess`, `allowJs`). +- `eslint.config.mjs` — flat ESLint config, `react-hooks` + `react-refresh` + `typescript-eslint`. +- `vite.config.mjs` — Vite + React Compiler + PWA + Vitest config. --- @@ -68,25 +96,30 @@ Runtime flow: Server defaults: - Port: `PORT || 3000`. +- Bind host: `BIND_HOST || '0.0.0.0'`. - Static frontend: `dist/`. -- Optional CORS: enabled when `CORS_ORIGIN` is set; credentials allowed. +- Optional CORS: enabled when `CORS_ORIGIN` is set; comma-separated allowlist with `credentials: true`. +- `TRUST_PROXY` (env): when set to `true`/`yes`/`on` trusts 1 hop; numeric → trust that many hops; otherwise the raw value (e.g. `loopback`, CIDR). Unset/false = no proxy trust. Required behind nginx/Traefik so `req.ip` and `req.secure` reflect the real client. - Session cleanup: runs on startup and every `SESSION_CLEANUP_INTERVAL_MS || 86400000` ms. - Admin seed: `INIT_ADMIN_USER || admin`; `INIT_ADMIN_PASS` or generated/default behavior in `db/database.js`. -- Environment variables INIT_ADMIN_USER and INIT_ADMIN_PASS (or INIT_REGULAR_USER + INIT_REGULAR_PASS) skip the first-login flow entirely by pre-seeding users with first_login=0 flags via setup/firstRun.js. +- Environment variables `INIT_ADMIN_USER` and `INIT_ADMIN_PASS` (or `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`) skip the first-login flow entirely by pre-seeding users with `first_login=0` flags via `setup/firstRun.js`. - Optional regular-user seed: `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`; password must be at least 8 chars. +- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked. +- After `app.listen`, `cleanupExpiredSessions` is scheduled, `bankSyncWorker.start()` runs, the backup scheduler is started, and `workers/dailyWorker.start()` schedules the daily jobs. Global middleware order: 1. `securityHeaders` -2. optional `cors` -3. `express.json()` +2. optional `cors` (only if `CORS_ORIGIN`) +3. `express.json({ limit: '100kb' })` — import routes override this per-endpoint 4. `cookieParser()` 5. `csrfTokenProvider` -6. mounted API routers with route-level rate-limit/auth/CSRF middleware -7. retired `/legacy` route returns 410, redirect `/login.html` to `/login`, static `dist/` -8. SPA fallback `GET *` serving `dist/index.html` after ensuring a CSRF token cookie -9. `errorFormatter` -10. final JSON error handler for malformed JSON/body size/runtime errors +6. `errorFormatter` (patches `res.json`) +7. `GET /api/health` — unauthenticated liveness probe +8. mounted API routers with route-level rate-limit/auth/CSRF middleware +9. `GET /login.html` → 302 `/login`, `express.static(DIST)` +10. SPA fallback `GET *` serving `dist/index.html` after ensuring a CSRF token cookie +11. final JSON error handler for malformed JSON / body-size / runtime errors ### Authentication middleware @@ -103,7 +136,8 @@ Global middleware order: - Cookie name: `CSRF_COOKIE_NAME || bt_csrf_token`. - Header name: `x-csrf-token`. - Defaults: `CSRF_HTTP_ONLY === true` only when explicitly configured, `CSRF_SAME_SITE || strict`, `CSRF_SECURE !== false`. -- The SPA expects `CSRF_HTTP_ONLY=false` so `client/api.js` can read the token cookie and send `x-csrf-token`; do not enable httpOnly CSRF cookies unless token delivery changes. +- The SPA expects `CSRF_HTTP_ONLY=false` so `client/api.ts` can read the token cookie and send `x-csrf-token`; do not enable httpOnly CSRF cookies unless token delivery changes. +- The SPA also calls `GET /api/auth/csrf-token` (handled by `routes/auth.js`) on first request and caches the token in memory; the cookie is the source of truth and the body mirrors it. - `csrfTokenProvider` sets a token cookie on responses. - `csrfMiddleware` validates mutating requests unless `req.csrfSkip` is set. Token may come from header, query, or body and must match cookie. - Failures return 403 and audit `csrf.failure`. @@ -113,6 +147,7 @@ Global middleware order: Defined in `middleware/rateLimiter.js`: - Login: 10 / 15 min. +- Login-by-username (extra layer): separate limiter, applied alongside `loginLimiter`. - Password changes: 5 / 15 min. - Import: 20 / 15 min. - Export: 30 / 15 min. @@ -121,6 +156,8 @@ Defined in `middleware/rateLimiter.js`: - Backup operations: 5 / 60 min. - Demo-data clear: 3 / 15 min. +A `skipRateLimitIfNoUsers(limiter)` wrapper is applied to `/api/auth/login` so a brand-new install with zero users is not locked out by the login limiter. + Rate-limit responses are JSON: `{ "error": "..." }`. ### Security headers @@ -138,7 +175,7 @@ Rate-limit responses are JSON: `{ "error": "..." }`. ### `services/authService.js` - Cookie: `bt_session`. -- Session lifetime: 7 days. +- Session lifetime: 7 days. Session tokens are **hashed at rest** in `sessions` (column populated in `v0.94`); the cookie value is a random opaque ID and is looked up by hash. - Password hashing: bcrypt, cost 12. - Functions: - `login(username, password)` — verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updates `last_login_at`, and returns `{sessionId, user}` or `null`. @@ -154,17 +191,57 @@ Password changes through `/api/auth/change-password` update the password hash, c `/api/auth/logout-all` calls `invalidateOtherSessions(req.user.id, null)`, deletes the current cookie session if present, audits `logout.all`, clears `bt_session`, and returns `{success:true}`. +### `services/totpService.js` + +TOTP / RFC-6238 authenticator-app 2FA: + +- `newSecret()` — `otplib` 20-char base32 secret. +- `generateQrCode(secret, username)` — returns `{uri, qr_data_url}` for the SPA to display. +- `verifyToken(encryptedSecret, token)` — decrypts the per-user secret, strips whitespace, calls `verifySync`. +- Setup flow: `getTotpSetup` → user scans QR → `enableTotp(plainSecret, token)` validates a sample token, encrypts and stores the secret, returns 8 recovery codes. +- `disableTotp(userId)` — clears `totp_enabled`, `totp_secret`, recovery codes. +- Recovery codes are stored hashed in `users.totp_recovery_codes` (JSON). +- `totpStatus(userId)` returns `{enabled, hasSecret, recoveryCodesRemaining}`. +- Login flow: `POST /api/auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id` stored in `totp_challenges`; the SPA submits it with the 6-digit code on the next request. + +### `services/webauthnService.js` + +WebAuthn / FIDO2 security-key 2FA via `@simplewebauthn/server`: + +- `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`). +- `createRegistrationChallenge(db, userId, username)` — generates a 32-byte `webauthn_user_id` (base64url), excludes already-registered credentials, stores the challenge in `webauthn_challenges` with 15-minute TTL, returns `{options, challengeId}`. +- `verifyRegistration(db, userId, challengeId, response, credentialName)` — verifies the registration, stores the credential (`credential_id`, public key, sign count, transports, backup flags, AAGUID, friendly name) and returns the credential id. +- `createAuthenticationChallenge(db, userId)` — for login: returns assertion options + challengeId. +- `verifyAuthentication(db, userId, challengeId, response)` — verifies, updates sign count, audits `webauthn.login`. +- Supports `attestationType: 'none'`, `residentKey: 'preferred'`, `userVerification: 'preferred'`, algorithms `-7` (ES256) and `-257` (RS256). +- `WEBAUTHN_RP_ID` must match the browser-visible hostname, or registration/assertion will fail. + ### `services/oidcService.js` Handles Authentik/OIDC login with `openid-client`: - Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag. +- The OIDC client secret is **encrypted at rest** (migration `v0.79`) using `services/encryptionService` (HKDF-derived key from `v0.78`). - Builds PKCE login state in `oidc_states`. - Discovers provider and caches OIDC client for 1 hour. - Exchanges callback code, verifies nonce/state, maps claims to local user. - Role mapping uses configured admin group; default role is `user`. - Client secret is write-only through admin settings. +### `services/encryptionService.js` + +- `assertEncryptionReady()` — throws if no master key is configured. +- `encryptSecret(plain)` / `decryptSecret(cipher)` — symmetric encryption with HKDF-derived key. +- The key derivation was upgraded from SHA-256 to HKDF in `v0.78`; migrations `v0.77` (SMTP password) and `v0.79` (OIDC secret) wrapped any pre-existing plaintexts. +- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected. + +### `services/loginFingerprint.js` + +- Captures `ip`, `user_agent`, `browser`, `os`, `device_type`, `device_fingerprint` on every login. +- The `ip` and `user_agent` columns are **encrypted at rest** (migration `v0.84`). +- `success` (`v0.85`) tracks failed-attempt history separately from successful logins. +- A `session_fingerprint` is stored on each successful login so the SPA can detect a different device for the same session and trigger a re-auth challenge. + ### `services/backupService.js` - Backup directory: `BACKUP_PATH || backups/`. @@ -193,11 +270,12 @@ Settings are stored in `settings`; run results are stored as JSON. ### `services/notificationService.js` -- Builds SMTP transport from global notification settings. +- Builds SMTP transport from global notification settings (SMTP password is **encrypted at rest**, migration `v0.77`). - Sends test email to an admin-provided address. - Runs due-bill notifications for 3 days, 1 day, due today, and overdue. - De-duplicates sends via `notifications` unique key. - Recipients come from user notification settings when enabled/allowed or global recipient settings. +- Push notifications: ntfy, Gotify, Discord webhook, Telegram bot — configured per-user in `users` columns (migration `v0.80`); dispatched alongside or in place of SMTP when configured. ### `services/spreadsheetImportService.js` @@ -206,6 +284,12 @@ Settings are stored in `settings`; run results are stored as JSON. - Creates `import_sessions` preview records with 24-hour TTL. - Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history. +### `services/ofxImportService.js` + +- OFX / QFX bank-statement import (separate from CSV/Excel). Uses `node-ofx-parser`; falls back to manual buffer parsing when the package is not installed. +- Produces `import_sessions` previews and commits to `transactions` with `source_type='file_import'`. +- 24-hour TTL on preview sessions; results write to `import_history`. + ### `services/userDbImportService.js` - Accepts user SQLite export files up to 50 MB. @@ -222,6 +306,22 @@ Transaction data source and transaction row helpers: - `decorateTransaction(row)` adds `source_label`, `source_type_label`, and embedded `data_source` object with safe fields. - `getSourceTypeLabel(type)` returns labels: `manual` → Manual, `file_import` → File import, `provider_sync` → Provider sync. - `sourceLabel(source)` constructs human-readable source labels for manual entries or provider names. +- `transactions` schema carries a `pending` flag (`v1.01`) for SimpleFIN pending transactions. + +### `services/transactionMatchService.js` + `services/transactionMatchState.js` + +- `matchTransactionToBill(transactionId, billId, userId)` — links a transaction to a bill, creates a corresponding `payments` row with `payment_source='provider_sync'`, runs accounting override logic. +- `unmatchTransaction(transactionId, userId)` — reverses the link and deletes the auto-created payment (or marks it `accounting_excluded`). +- `ignoreTransaction` / `unignoreTransaction` — set/match-status transitions. +- `transactionMatchState.js` is the state-machine helper that powers match/unmatch/ignore/unignore. + +### `services/matchSuggestionService.js` + +- `listMatchSuggestions(userId, transactionId, billId, limit, offset)` — returns scored suggestions. +- `rejectMatchSuggestion(userId, transactionId, billId)` — records the rejection in `match_suggestion_rejections`. +- `suggestionCounts(userId)` — dashboard counts. +- Rejections are stored in `match_suggestion_rejections` (created in `v0.62`) so a transaction/bill pair is never re-suggested after rejection. +- After sync, `autoMatchForUser(userId)` runs merchant-rule + merchant-store-match + match-suggestion passes in order. ### `services/csvTransactionImportService.js` @@ -235,16 +335,58 @@ CSV import workflow for transactions: - Imports record to `import_history` with counts and details. - `FIELD_LABELS` maps field keys to user-friendly labels for validation messages. -### `services/paymentValidation.js` +### `services/bankSyncService.js` + `services/simplefinService.js` + `services/bankSyncConfigService.js` + `services/bankSyncWorker.js` -Payment validation helpers for source tracking and matching: +- **SimpleFIN bridge** (`simplefinService.js`): wraps the SimpleFIN protocol — `claimSetupToken`, `fetchAccountsAndTransactions`, `normalizeAccount`, `normalizeTransaction`, `sanitizeErrorMessage`. `assertEncryptionReady` is invoked before any secret operation. +- **Per-user sync** (`bankSyncService.js`): upserts `financial_accounts` rows keyed by `(data_source_id, provider_account_id)`; upserts `transactions` keyed by `(data_source_id, provider_transaction_id)`. Honors the `monitored` flag (`v0.64`) so the user can exclude a credit-card account from bill matching. Applies merchant rules, spending-category rules, merchant-store matches, then `autoMatchForUser` in sequence. +- **Config** (`bankSyncConfigService.js`): `getBankSyncConfig(userId)`, `SYNC_DAYS_EFFECTIVE`, `SYNC_DAYS_DEFAULT`. Per-user default-lookback window. +- **Worker** (`bankSyncWorker.js`): scheduled background sync, started from `server.js` after `app.listen`. `bankSyncConfigService` admin endpoints live at `/api/admin/bank-sync-config`. -- Validates `payment_source` values (`manual`, `file_import`, `provider_sync`). -- Supports transaction linking via `transaction_id` when available. +### `services/billMerchantRuleService.js` + +- Persistent merchant→bill auto-match rules (table created in `v0.67`). +- `applyMerchantRules(transactions, userId)` — returns transactions with `matched_bill_id` and `match_status='matched'` filled in. +- `auto_attribute_late` (`v0.83`) marks rules whose bills always post after month-end, so amount-suggestion math skips the “looks like last month’s bill” signal. + +### `services/merchantStoreMatchService.js` + +- 5k merchant/store matching pack seeded by migration `v1.05`. +- `applyMerchantStoreMatches(transactions, userId)` — broad merchant-to-store normalization for spending categorization. + +### `services/advisoryFilterService.js` + +- 5k advisory non-bill patterns + bill-like override terms seeded by migration `v0.68`. +- Used to suppress “this looks like a bill but isn’t” false positives in tracker suggestions. + +### `services/subscriptionService.js` + `services/amountSuggestionService.js` + +- `subscriptionService.js` — manages the user’s subscription bills, recommendations, declined hints (`v0.66`), feedback (`v0.97`), and the top-200 catalog seeding. Exposes `normalizeMerchant` used by migration `v0.90` to re-normalize stored rules. +- `amountSuggestionService.js` — `computeAmountSuggestion` (per-bill) and `computeAmountSuggestionsBatch` (all-bills, used by the Tracker P1 fix to kill the N+1). +- The batched suggestion is byte-identical to the per-bill function; pinned by `tests/amountSuggestionService.test.js`. + +### `services/snowballService.js` + `services/calendarFeedService.js` + `services/driftService.js` + `services/statusRuntime.js` + +- `snowballService.js` — Debt snowball/avalanche calculations, Ramsey mode support, order updates, plan lifecycle math. +- `calendarFeedService.js` — generates iCalendar (`.ics`) text for upcoming bills, gated by `calendar_tokens` (`v1.00`). +- `driftService.js` — detects when an autopay/posted amount drifts from the bill’s expected amount; surfaces the “drift_snoozed_until” field (`v0.71`). +- `statusRuntime.js` — `recordError(err, req)`, runtime worker state, health JSON consumed by `/api/status`. + +### `services/spendingService.js` + `services/userSettings.js` + `services/userDataService.js` + `services/eraseUserData.js` (in `utils`/tests) + `services/billsService.js` + `services/aprService.js` + +- `spendingService.js` — spending categories, rules (`v0.87`), budgets, default category seeding, income breakdown, `applySpendingCategoryRules` used by the bank-sync pipeline. +- `userSettings.js` — `getUserSettings(userId)` returns a typed settings object; consumed by the bank-sync pipeline. +- `userDataService.js` — user-scoped read helpers (e.g. `getCategoriesForUser`). +- `billsService.js` — bill CRUD helpers, soft-delete + `RecentlyDeletedBills` restore support. +- `aprService.js` — annual-percentage-rate / monthly interest accrual used by `v0.93` (`interest_accrued_month`, `interest_delta`). + +### `services/paymentAccountingService.js` + `services/paymentValidation.js` + +- `paymentAccountingService.js` — source-of-truth for “manual vs. bank” payments. The bank-backed predicate, the accounting-active SQL fragment, and the core invariant: a bank payment overrides a provisional manual payment so it isn’t double-counted, restores the balance, and the manual payment counts again with its balance re-applied when the override is removed. Pinned by `tests/paymentAccountingService.test.js`. +- `paymentValidation.js` — `PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match']` (auto_match was normalized to provider_sync in `v0.82`), `validatePaymentSource()`, `validatePaymentInput()`. ### `services/auditService.js` -Writes `audit_log` rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, and migration rollback attempts. +Writes `audit_log` rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, migration rollback attempts, WebAuthn/TOTP events, bank-sync admin changes, and 2FA enable/disable. `db/database.js` does not import `logAudit` at module load time. It uses a lazy `getLogAudit()` helper so migration code can write audit rows without creating a circular dependency: `auditService` imports `database.js`, and `database.js` needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors. @@ -252,7 +394,7 @@ Writes `audit_log` rows for security-sensitive events such as login, logout, pas ## 5. API Reference -All routes are prefixed with `/api` unless stated otherwise. Most mutating endpoints require CSRF token header `x-csrf-token`. Auth is cookie-based. User routes are scoped to the authenticated user unless noted. +All routes are prefixed with `/api` unless stated otherwise. Most mutating endpoints require CSRF token header `x-csrf-token`. Auth is cookie-based (`bt_session`). User routes are scoped to the authenticated user unless noted. `GET /api/health` is the unauthenticated liveness probe and is the only public non-meta endpoint. Response conventions: @@ -264,736 +406,192 @@ Response conventions: - Conflict: `409`. - Rate-limited: `429`. -### 5.1 Auth +### 5.1 Auth (`/api/auth`) -Mounted under `/api/auth`. +- `POST /auth/login` — `{username, password}`. Sets `bt_session`. Rate-limited (skipped if zero users). Skips CSRF (no session yet). +- `POST /auth/logout` — clears `bt_session`. +- `POST /auth/logout-all` — invalidates every session for the user. +- `GET /auth/me` — public user object (single-user mode supplies user without session). +- `GET /auth/mode` — auth mode, local-login flag, OIDC public info, single-user status. +- `POST /auth/restore-multi-user-mode` — restores multi-user mode where allowed. +- `POST /auth/acknowledge-privacy` — updates first-login/privacy acknowledgement flags. +- `POST /auth/acknowledge-version` — stamps `users.last_seen_version`. +- `POST /auth/change-password` — `{current_password, new_password}`. Password-limited, CSRF-skip. Rotates session id, invalidates others. +- `GET /auth/csrf-token` — returns `{token}` mirroring the cookie; SPA caches in memory. +- `GET /auth/has-users` — whether non-default users exist. +- `GET /auth/users` — admin-only safe user list. +- `POST /auth/users` — admin-only create. +- `GET /auth/login-history` — recent login/failed-login history for the current user (encrypted columns decrypted in-process). +- `GET /auth/webauthn/status` — `{enabled, credentials:[{id, name, createdAt, lastUsedAt}]}`. +- **TOTP** (`/api/auth/totp`): + - `POST /auth/totp/challenge` — `{username, password}` → `{challenge_id}` (5-min TTL). + - `GET /auth/totp/setup` — admin-aware: returns `{secret, uri, qr_data_url}` for a new enrolment. + - `POST /auth/totp/enable` — `{secret, token}` → verifies, encrypts, stores, returns recovery codes. + - `POST /auth/totp/disable` — clears secret + recovery codes. + - `GET /auth/totp/status` — `{enabled, hasSecret, recoveryCodesRemaining}`. -- `POST /auth/login` - - Body: `{username, password}`. - - Validation: both required; local login must be enabled. - - Response: sets `bt_session`; `{user}`. +### 5.2 OIDC Auth (`/api/auth/oidc`) -- `POST /auth/logout` - - Auth: required. - - Body: none. - - Response: clears cookie; `{success:true}`. +OIDC rate limiter applies. -- `POST /auth/logout-all` - - Auth: required; CSRF skip is set before the router mount, matching other auth/session mutation routes. - - Body: none. - - Behavior: deletes every session for the current user by calling `invalidateOtherSessions(userId, null)`, also deletes the current cookie session, audits `logout.all`, clears `bt_session`, and returns `{success:true}`. +- `GET /auth/oidc/login?redirect_to=/path` — public when OIDC active; builds PKCE state and redirects. +- `GET /auth/oidc/callback?code=&state=` — validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root. Error redirects use query errors such as `access_denied` or `authentication_failed`. -- `GET /auth/me` - - Auth: required unless single-user mode supplies user. - - Response: public user object. +### 5.3 Admin (`/api/admin`) -- `GET /auth/mode` - - Public. - - Response: auth mode, local-login flag, OIDC public info, single-user status. +Mounted under `/api/admin`; all require `requireAuth + requireAdmin + adminActionLimiter`. Backup subroutes also use `backupOperationLimiter`. -- `POST /auth/restore-multi-user-mode` - - Auth: required. - - Body: none. - - Response: restores multi-user mode where allowed. +- `GET /admin/has-users` — `{has_users:boolean}`. +- `GET /admin/users` — safe users ordered by default-admin / role / username. +- `POST /admin/users` — `{username, password}`. +- `PUT /admin/users/:id/password` — `{password}`. Invalidates target sessions; requires password change. +- `PUT /admin/users/:id/role` — `{role:"admin"|"user"}`. Cannot change own role; cannot remove last admin. +- `PUT /admin/users/:id/active` — `{active:boolean}`. Cannot deactivate self. +- `PUT /admin/users/:id/username` — change a user’s username (admin only). +- `DELETE /admin/users/:id` — transaction deletes import sessions/history, sessions, and the user. +- **Backups**: `GET/POST /admin/backups`, `GET /admin/backups/settings`, `PUT /admin/backups/settings`, `POST /admin/backups/run-scheduled-now`, `POST /admin/backups/import` (raw SQLite, max 100 MB, optional `X-Checksum-Sha256`), `GET /admin/backups/:id/download`, `POST /admin/backups/:id/restore`, `DELETE /admin/backups/:id`. +- **Cleanup**: `GET/PUT /admin/cleanup`, `POST /admin/cleanup/run`. +- **Auth mode**: `GET /admin/auth-mode`, `POST /admin/auth-mode/oidc-test`, `PUT /admin/auth-mode`. +- **Bank sync config**: `GET /admin/bank-sync-config`, `PUT /admin/bank-sync-config` (default-lookback days, monitored-account default, etc.). +- **Migrations**: `POST /admin/migrations/rollback` with `{version}`. Maps `NOT_APPLIED` → 404, `ROLLBACK_NOT_SUPPORTED` → 422, other failures → 500. -- `POST /auth/acknowledge-privacy` - - Auth: required. - - Body: none. - - Response: updates first-login/privacy acknowledgement flags. +### 5.4 Bills (`/api/bills`) -- `POST /auth/change-password` - - Auth: required; password limiter; CSRF skip is set before the router mount. - - Body: `{current_password, new_password}`. - - Validation: current password required unless `must_change_password` is set; new password min 8. - - Behavior: updates password hash, clears `must_change_password`, updates `last_password_change_at`, invalidates all other sessions, rotates the current session ID when a valid `bt_session` exists, sets the new cookie, audits `password.change`, and returns `{success:true}`. +User/admin tracker access. Standard bill fields plus `cycle_type`, `cycle_day`, `is_subscription`, `subscription_type`, `catalog_id`, `autopay_verified_at`, `inactive_reason`, `drift_snoozed_until`, `sort_order`, `interest_accrued_month`, `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at`. -- `GET /auth/has-users` - - Response: whether non-default users exist. +Endpoints (subset): `GET /bills?inactive=true`, `GET /bills/:id`, `POST /bills`, `PUT /bills/:id`, `DELETE /bills/:id` (hard delete; cascades payments, monthly state, history ranges, merchant rules; returns warning), `GET/PUT /bills/:id/monthly-state?year=&month=`, `GET /bills/:id/payments`, `POST /bills/:id/toggle-paid`, `GET/POST/PUT/DELETE /bills/:id/history-ranges`, `GET /bills/recently-deleted` (admin), `POST /bills/:id/restore`. -- `GET /auth/users` - - Auth: admin. - - Response: safe user list. +### 5.5 Payments (`/api/payments`) -- `POST /auth/users` - - Auth: admin. - - Body: `{username, password}`. - - Validation: username min 3, password min 8, unique username. - - Response: created safe user. +User/admin tracker access. Soft delete via `deleted_at`. `payment_source` ∈ `{manual, file_import, provider_sync, transaction_match}`. Hard-dedupe unique index on `transaction_id` (`v0.61`). -### 5.2 OIDC Auth +Endpoints: `GET /payments?bill_id=&year=&month=`, `GET /payments/:id`, `POST /payments`, `POST /payments/quick`, `POST /payments/bulk` (max 50), `PUT /payments/:id`, `DELETE /payments/:id`, `POST /payments/:id/restore`. -Mounted under `/api/auth/oidc`; OIDC rate limiter applies. +### 5.6 Categories (`/api/categories`) -- `GET /auth/oidc/login?redirect_to=/path` - - Public when OIDC active. - - Creates PKCE state and redirects to provider authorization URL. +User/admin tracker access. Categories have a `spending_enabled` flag (`v0.88`) and a `sort_order` (`v0.75`). Categories can be grouped via `category_groups` (`v1.06`). -- `GET /auth/oidc/callback?code=&state=` - - Public callback. - - Validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root. - - Error redirects use query errors such as `access_denied` or `authentication_failed`. +Endpoints: `GET /categories` (seeds defaults, includes `spending_enabled` + counts), `POST /categories`, `PUT /categories/:id`, `DELETE /categories/:id`, plus category-group endpoints under `/api/categories/groups` (list/create/update/delete/reorder). -### 5.3 Admin +### 5.7 Tracker and Calendar (`/api/tracker`, `/api/calendar`) -Mounted under `/api/admin`; all require `requireAuth + requireAdmin + adminActionLimiter` at the mount level. Backup subroutes also use `backupOperationLimiter`. +- `GET /tracker?year=&month=` — returns `year`, `month`, tracker `rows`, totals, starting-amount info, previous-month paid total, three-month averages/trends, **drift / autopay / amount-suggestion** objects, **safe-to-spend** (cashflow timeline), `generated_at`. +- `GET /tracker/upcoming?days=30` — upcoming active bills. +- `GET /tracker/health` — dedicated health endpoint for the home page (subset of `/tracker`). +- `GET /calendar?year=&month=` — month days with payment entries, bills/due-status entries, totals. +- `GET /calendar/feed.ics?token=...` — public iCalendar feed of upcoming bills, gated by `calendar_tokens` (`v1.00`). -- `GET /admin/has-users` - - Response: `{has_users:boolean}` for users other than current admin. +### 5.8 Summary and Starting Amounts (`/api/summary`, `/api/monthly-starting-amounts`) -- `GET /admin/users` - - Response: safe users ordered by default-admin, role, username. +- `GET /summary?year=&month=` — returns `{year, month, income, expenses, starting_amounts, previous_month, summary, chart, bank_tracking, generated_at}`. +- `PUT /summary/income` — `{year, month, amount, label?}`. +- `GET /monthly-starting-amounts?year=&month=` / `PUT /monthly-starting-amounts` — bucket response. -- `POST /admin/users` - - Body: `{username, password}`. - - Validation: username min 3, password min 8, unique. - - Response 201: created user. +### 5.9 Snowball (`/api/snowball`) -- `PUT /admin/users/:id/password` - - Body: `{password}`. - - Validation: password min 8; user exists. - - Response: `{success:true}`; invalidates target sessions and requires password change. +- `GET /snowball` — current user's debt bills pre-sorted by `snowball_order`. +- `GET /snowball/settings` / `PATCH /snowball/settings` — extra payment, ramsey mode, ready-current, ready-emergency-fund. +- `GET /snowball/projection` — `{snowball, avalanche, minimum_only, comparison}` with enriched debt arrays (APR snapshots, interest-accrual month). +- `PATCH /snowball/order` — `[{id, snowball_order}]` batch update. +- **Plans** (lifecycle, `v0.73`): `POST /snowball/plans`, `GET /snowball/plans`, `GET /snowball/plans/active`, `PATCH /snowball/plans/:id`, `POST /snowball/plans/:id/pause|resume|complete|abandon`. -- `PUT /admin/users/:id/role` - - Body: `{role:"admin"|"user"}`. - - Validation: cannot change own role; cannot remove last admin. - - Response: updated safe user; invalidates target sessions; audits role change. +### 5.10 Spending (`/api/spending`) -- `PUT /admin/users/:id/active` - - Body: `{active:boolean}`. - - Validation: user exists; cannot deactivate self. - - Response: updated safe user; deactivation invalidates sessions. +- `GET /spending/summary?year=&month=` — monthly spending summary, by category / by bill. +- `GET /spending/transactions` — paginated, filterable. +- `PATCH /spending/transactions/:id/category` — manual reassignment. +- `GET /spending/budgets` / `PUT /spending/budgets` / `POST /spending/budgets/copy` — monthly budgets with month-rollover. +- `GET /spending/category-rules` / `POST /spending/category-rules` / `DELETE /spending/category-rules/:id` — automation rules seeded in `v0.87`. +- `GET /spending/income` — per-user monthly income. -- `DELETE /admin/users/:id` - - Validation: user exists; cannot delete self. - - Response: `{success:true, deleted_user_id}`; transaction deletes import sessions/history, sessions, and user. +### 5.11 Subscriptions (`/api/subscriptions`) -- `POST /admin/backups` - - Body: none. - - Response 201: backup metadata `{id, filename, size_bytes, checksum, ...}`. +- `GET /subscriptions` — user’s subscription bills. +- `GET /subscriptions/recommendations` — bank-derived candidates (declined hints suppressed). +- `POST /subscriptions/recommendations/decline` — record a decline. +- `POST /subscriptions/recommendations/match-bill` — link a recommendation to an existing bill. +- `POST /subscriptions/recommendations/create` — create a new bill from a recommendation. +- `GET /subscriptions/transaction-matches` — candidate transactions for the user’s subscriptions. +- `GET /subscriptions/catalog` — top-200 catalog with bank descriptors. +- `PUT /subscriptions/:id/catalog-link` / `POST /subscriptions/catalog/:catalogId/descriptors` / `DELETE /subscriptions/catalog/descriptors/:id` — manage custom bank descriptors. +- `PATCH /subscriptions/:id` — update subscription fields (price, cycle, descriptor, etc.). -- `GET /admin/backups` - - Response: `{backups:[metadata...]}`. +### 5.12 Analytics (`/api/analytics`) -- `GET /admin/backups/settings` - - Response: backup schedule status/settings. +- `GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=&include_skipped=` — monthly spending, expected vs actual, category totals, bill totals, filters, `generated_at`. -- `PUT /admin/backups/settings` - - Body: `{enabled, frequency:"daily"|"weekly", time:"HH:MM", retention_count}`. - - Validation: frequency, valid time, retention 1-365. - - Response: saved schedule status. +### 5.13 Settings (`/api/settings`) -- `POST /admin/backups/run-scheduled-now` - - Body: none. - - Response 201: scheduled backup result. +- `GET /settings` / `PUT /settings` — user-visible settings. +- `POST /settings/seed-demo-data` — seed demo data. -- `POST /admin/backups/import` - - Content-Type: `application/octet-stream`, `application/x-sqlite3`, or `application/vnd.sqlite3`. - - Body: raw SQLite backup, max 100 MB. - - Optional checksum: `X-Checksum-Sha256` header or `?checksum=`. - - Response 201: imported backup metadata. +### 5.14 Notifications (`/api/notifications`) -- `GET /admin/backups/:id/download` - - Response: file download. ID must be a managed backup filename. +Mounted under `/api/notifications`; `requireAuth` at mount. -- `POST /admin/backups/:id/restore` - - Response: `{restored_from, pre_restore_backup}`; validates and restores managed backup. +- `GET/PUT /notifications/admin` — SMTP + push (ntfy / Gotify / Discord / Telegram) global settings. +- `POST /notifications/test` — `{to}`. +- `GET/PUT /notifications/me` — per-user notification email, push channels, lead times. -- `DELETE /admin/backups/:id` - - Response: `{deleted:true, id, deleted_at}`. +### 5.15 Profile (`/api/profile`) -- `GET /admin/cleanup` - - Response: cleanup settings and last result. +- `GET /profile` — safe profile, notification settings, export URLs, import-history URL. +- `PATCH /profile` — `{display_name}`. +- `GET/PATCH /profile/settings` — per-user notification prefs. +- `POST /profile/change-password` — rotates session, invalidates others. +- `GET /profile/exports` / `GET /profile/import-history`. -- `PUT /admin/cleanup` - - Body: any of `{import_sessions_enabled, temp_exports_enabled, temp_export_max_age_hours, backup_partials_enabled, import_history_enabled, import_history_max_age_days}`. - - Validation: booleans; temp export age 1-72 hours; import history age 30-3650 days. - - Response: updated cleanup status. +### 5.16 User Demo Data (`/api/user`) -- `POST /admin/cleanup/run` - - Response: cleanup run result by task. +- `POST /user/seed-demo-data`, `POST /user/clear-demo-data` (rate-limited; deletes `is_seeded=1` bills/categories for current user). -- `GET /admin/auth-mode` - - Response: local/single-user/OIDC settings and lockout warnings. Client secret is not returned. +### 5.17 Import (`/api/import`, `/api/imports`) -- `POST /admin/auth-mode/oidc-test` - - Body: submitted or saved OIDC config fields. - - Response: `{ok:true,...}` or 400 with test error. Never returns secret/token material. +`requireAuth + requireUser + importLimiter` at mount. Both prefixes are accepted. -- `PUT /admin/auth-mode` - - Body: legacy `{auth_mode, default_user_id}` plus OIDC/local settings such as `local_login_enabled`, `oidc_login_enabled`, `oidc_issuer_url`, `oidc_client_id`, `oidc_client_secret`, `oidc_redirect_uri`, `oidc_scopes`, `oidc_admin_group`, `oidc_auto_provision`. - - Validation: cannot disable all login methods; cannot disable local login until OIDC is configured, enabled, and has an admin group; cannot enable incomplete OIDC. - - Response: `{success:true, ...authModeStatus}`. +- `POST /import/spreadsheet/preview` (XLSX, max 10 MB, optional `parse_all_sheets`, `year`, `month`). +- `POST /import/spreadsheet/apply`. +- `POST /import/user-db/preview` (SQLite, max 50 MB) / `apply`. +- `POST /import/csv/preview` / `POST /import/csv/commit`. +- `POST /import/ofx/preview` / `POST /import/ofx/commit` (OFX/QFX bank statements). +- `GET /import/history`. -- `POST /admin/migrations/rollback` - - Body: `{version:"v0.44"|"v0.45"|"v0.46"}`. - - Validation: version required; migration must be present in `schema_migrations`; `ROLLBACK_SQL_MAP` must define rollback SQL for that version. - - Behavior: calls `rollbackMigration(version)` from `db/database.js`, audits success as `migration.rollback`, and returns `{success:true, version, description, elapsed_ms}`. - - Error mapping: `NOT_APPLIED` becomes HTTP 404 with `{error}`; `ROLLBACK_NOT_SUPPORTED` becomes HTTP 422 with `{error}`; other rollback failures become HTTP 500 with `{error:"Rollback failed", details}` and are audited as `migration.rollback.failure`. +### 5.18 Data Sources (`/api/data-sources`) -### 5.4 Bills +- `GET /data-sources?type=&status=` — `type` ∈ `manual, file_import, provider_sync`; `status` ∈ `active, inactive, error`. Returns sources with `account_count`, `transaction_count`, sync info. -Mounted under `/api/bills`; auth: user/admin tracker access. +### 5.19 Transactions (`/api/transactions`) -Bill object fields include `id`, `user_id`, `name`, `category_id`, `category_name`, `due_day`, `override_due_date`, `bucket`, `expected_amount`, `interest_rate`, `billing_cycle`, `autopay_enabled`, `autodraft_status`, `website`, `username`, `account_info`, `has_2fa`, `active`, `notes`, `history_visibility`, `is_seeded`, `cycle_type`, `cycle_day`, timestamps, and `has_history_ranges` on list/detail queries. +- `GET /transactions?limit=&offset=&match_status=&ignored=&source_type=&start_date=&end_date=&q=&data_source_id=&account_id=&matched_bill_id=`. Free-text search across description/payee/memo/category. +- `POST /transactions/manual` — manual entry, returns created row with `source_type='manual'`. +- `PUT /transactions/:id` — partial update. +- `DELETE /transactions/:id` — soft-unmatch then hard delete. +- `POST /transactions/:id/match` / `unmatch` / `ignore` / `unignore`. +- The `pending` flag (`v1.01`) marks SimpleFIN pending transactions separately from posted ones. -Validation shared by create/update: +### 5.20 Match Suggestions (`/api/matches`) -- `name` required for create. -- `due_day`: integer 1-31. -- `expected_amount`: numeric, defaults to 0. -- `interest_rate`: null/empty or number 0-100. -- `category_id`: must belong to current user. -- `history_visibility`: `default`, `all`, `ranges`, or `none`. -- `cycle_type`: `monthly`, `weekly`, `biweekly`, `quarterly`, `annual`. -- `cycle_day`: monthly 1-31; weekly/biweekly day name; quarterly/annual text up to 50 chars. +- `GET /matches/suggestions?transaction_id=&bill_id=&limit=&offset=`. +- `POST /matches/:id/reject` — records rejection so the pair is never re-suggested. -Endpoints: +### 5.21 Export (`/api/export`) -- `GET /bills?inactive=true` - - Response: current user's bills; inactive excluded unless `inactive=true`. +- `GET /export?year=YYYY&format=csv|xlsx` — payment/bill history for the year. +- `GET /export/user-excel` — user workbook. +- `GET /export/user-db` — portable SQLite with metadata. -- `GET /bills/:id` - - Response: one owned bill or 404. +### 5.22 Status, About, Version, Privacy, Health -- `POST /bills` - - Body: bill fields. - - Response 201: created bill. +- `GET /status` — admin-only: app version, uptime, runtime worker state, DB health, SMTP/push config status, backup status, current-month tracker health, recent errors. +- `GET /about` — public; `{version, about}`. +- `GET /about-admin` — admin-only, file allowlist `FUTURE.md` / `DEVELOPMENT_LOG.md`. +- `GET /version` — public; package version + latest structured notes from `HISTORY.md`. +- `GET /version/history` — raw history text. +- `GET /version/update-status` — async, checks upstream for new releases (uses `services/updateCheckService`). +- `GET /privacy` — public privacy text. +- `GET /health` — public liveness probe `{ok:true}`. -- `PUT /bills/:id` - - Body: partial bill fields. - - Response: updated bill. - -- `DELETE /bills/:id` - - Hard delete; cascades payments, monthly state, history ranges. - - Response: `{success:true, deleted_bill_id, deleted_bill_name, warning}`. - -- `GET /bills/:id/monthly-state?year=&month=` - - Validation: year 2000-2100; month 1-12. - - Response: `{bill_id, year, month, actual_amount, notes, is_skipped}`. - -- `PUT /bills/:id/monthly-state` - - Body: `{year, month, actual_amount, notes, is_skipped}`. - - Validation: year/month; actual_amount null or non-negative. - - Response: saved monthly state with timestamps. - -- `GET /bills/:id/payments?page=1&limit=20` - - Validation: limit capped at 100. - - Response: `{bill_id, bill_name, total, page, limit, pages, payments}`. - -- `POST /bills/:id/toggle-paid` - - Body optional: `{amount, paid_date, method, notes}`. - - If latest live payment exists, soft-deletes it. Otherwise creates a payment for amount or bill expected amount. - - Response: `{success, isPaid, action, payment?}`. - -- `GET /bills/:id/history-ranges` - - Response: `{bill_id, history_visibility, ranges}`. - -- `POST /bills/:id/history-ranges` - - Body: `{start_year, start_month, end_year?, end_month?, label?}`. - - Validation: years 2000-2100, months 1-12, end both present/absent and not before start. - - Response 201: created range. - -- `PUT /bills/:id/history-ranges/:rangeId` - - Body: partial range fields. - - Response: updated range. - -- `DELETE /bills/:id/history-ranges/:rangeId` - - Response: `{success:true}`. - -### 5.5 Payments - -Mounted under `/api/payments`; auth: user/admin tracker access. All queries are user-owned through joined bill ownership. Delete is soft delete via `deleted_at`. - -- `GET /payments?bill_id=&year=&month=` - - Validation: year and month must be supplied together; year 2000-2100; month 1-12. - - Response: live payments ordered descending by `paid_date`. - -- `GET /payments/:id` - - Response: live payment or 404. - -- `POST /payments` - - Body: `{bill_id, amount, paid_date, method?, notes?, payment_source?}`. - - Validation: bill exists and owned; amount > 0; required fields present; payment_source one of `manual`, `file_import`, `provider_sync`, `transaction_match`. - - Response 201: created payment. - -- `POST /payments/quick` - - Body: `{bill_id, amount?, paid_date?, method?, notes?, payment_source?}`. - - Defaults amount to bill expected amount and date to today; confirms autodraft status for autopay bills; defaults payment_source to `manual`. - - Response 201: created payment. - -- `POST /payments/bulk` - - Body: `{payments:[{bill_id, amount, paid_date, method?, notes?, payment_source?}]}`. - - Validation: array required; max 50; bill_id integer; `paid_date` `YYYY-MM-DD`; amount finite >= 0; payment_source must be valid. - - Duplicate live payments by user/bill/date/amount are skipped. - - Response 201: `{created:[...], skipped:[...], errors:[...]}`. - -- `PUT /payments/:id` - - Body: partial `{amount, paid_date, method, notes, payment_source}`. - - Response: updated payment. Current code preserves existing fields when omitted. - -- `DELETE /payments/:id` - - Response: `{success:true}` after setting `deleted_at`. - -- `POST /payments/:id/restore` - - Response: restored payment with `deleted_at:null`. - -### 5.6 Categories - -Mounted under `/api/categories`; auth: user/admin tracker access. - -- `GET /categories` - - Seeds default categories for user if needed. - - Response: categories with bill counts/payment counts and bill summaries. - -- `POST /categories` - - Body: `{name}`. - - Validation: non-empty name; unique per user case-insensitive. - - Response 201: created category. - -- `PUT /categories/:id` - - Body: `{name}`. - - Validation: category belongs to user; non-empty unique name. - - Response: updated category. - -- `DELETE /categories/:id` - - Validation: category belongs to user. - - Behavior: transaction nulls category on owned bills, then deletes category. - - Response: deletion summary. - -### 5.7 Tracker and Calendar - -- `GET /tracker?year=&month=` - - Auth: user/admin tracker access. - - Defaults to current year/month. - - Response includes `year`, `month`, tracker `rows`, totals, starting amount info, previous month paid total, three-month averages/trends, and generated timestamp. - - Row fields come from `buildTrackerRow` plus monthly override state and previous-month payment data. - -- `GET /tracker/upcoming?days=30` - - Auth: user/admin tracker access. - - Response: upcoming active bills in the requested horizon. - -- `GET /calendar?year=&month=` - - Auth: user/admin tracker access. - - Defaults current year/month. - - Response: month days with payment entries, bills/due-status entries, and totals `{expectedTotal, paidTotal, remainingTotal, paidPercent}`. - -### 5.8 Summary and Starting Amounts - -- `GET /summary?year=&month=` - - Auth: user/admin tracker access. - - Validation: valid year/month. - - Response: `{year, month, income, expenses, starting_amounts, previous_month, summary, chart, generated_at}`. - -- `PUT /summary/income` - - Body: `{year, month, amount, label?}`. - - Validation: valid year/month; amount 0-1,000,000,000; label trimmed to 80 chars. - - Response: `{year, month, income}` after upsert into `monthly_income`. - -- `GET /monthly-starting-amounts?year=&month=` - - Response: `{year, month, first_amount, fifteenth_amount, other_amount, combined_amount, paid deductions, remaining values, notes}`. - -- `PUT /monthly-starting-amounts` - - Body: `{year, month, first_amount, fifteenth_amount, other_amount, notes?}`. - - Validation: valid year/month; numeric amounts. - - Response: recomputed starting-amount response after upsert. - -### 5.8.1 Snowball - -Mounted under `/api/snowball`; auth: user/admin tracker access. - -- `GET /snowball` - - Response: current user's debt bills (snowball_include or debt-like categories), pre-sorted by snowball_order. - -- `GET /snowball/settings` - - Response: `{extra_payment, ramsey_mode, ready_current_on_bills, ready_emergency_fund}`. - -- `PATCH /snowball/settings` - - Body: `{extra_payment?, ramsey_mode?, ready_current_on_bills?, ready_emergency_fund?}`. - - Response: saved settings and computed response. - -- `GET /snowball/projection` - - Response: `{snowball, avalanche, minimum_only, comparison}` with enriched debt arrays including APR snapshots. - -- `PATCH /snowball/order` - - Body: `[{id, snowball_order}]`. - - Response: `{success:true}` after batch update. - -### 5.9 Analytics - -- `GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=true&include_skipped=false` - - Auth: user/admin tracker access. - - Validation: year/month valid; months clamped by route validation; IDs parsed as integers. - - Response includes monthly spending, expected vs actual, category totals, bill totals, filters, and generated timestamp. - -### 5.10 Settings - -- `GET /settings` - - Auth: user/admin tracker access. - - Response: user-visible settings from the allowed settings key list. - -- `PUT /settings` - - Auth: user/admin tracker access. - - Body: key/value object for allowed user setting keys. - - Response: updated settings object. - -- `POST /settings/seed-demo-data` - - Auth: user/admin tracker access. - - Response: demo seed result. - -### 5.11 Notifications - -Mounted under `/api/notifications`. Server mount requires `requireAuth`; route-level guards further restrict. - -- `GET /notifications/admin` - - Auth: admin. - - Response: global SMTP/notification settings. - -- `PUT /notifications/admin` - - Auth: admin. - - Body: allowed global SMTP and notification settings. - - Response: saved settings. - -- `POST /notifications/test` - - Auth: admin. - - Body: `{to}`. - - Validation: recipient required. - - Response: send result or SMTP error. - -- `GET /notifications/me` - - Auth: user/admin tracker access. - - Response: current user's notification email and toggle settings. - -- `PUT /notifications/me` - - Auth: user/admin tracker access. - - Body: `{notification_email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}`. - - Response: saved user notification settings. - -### 5.12 Profile - -Mounted under `/api/profile`; auth: user/admin tracker access; password limiter applies at mount. - -- `GET /profile` - - Response: safe profile, notification settings, export URLs, import-history URL. - -- `PATCH /profile` - - Body: `{display_name}`. - - Validation: string, max 100 chars. - - Response: `{success:true, profile}`. - -- `GET /profile/settings` - - Response: user notification preferences only. - -- `PATCH /profile/settings` - - Body: `{notification_email|email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}`. - - Validation: email value string/null, max 255 chars. - - Response: `{success:true}`. - -- `POST /profile/change-password` - - Body: `{current_password, new_password, confirm_new_password}`. - - Validation: all required; confirmation matches; new password min 8; current password must verify. - - Response: rotates current session, invalidates others, `{success:true}`. - -- `GET /profile/exports` - - Response: metadata for `user_db` and `user_excel` export URLs. - -- `GET /profile/import-history` - - Response: `{history:[...]}`. - -### 5.13 User Demo Data - -Mounted under `/api/user`; auth: user/admin tracker access. - -- `POST /user/seed-demo-data` - - Response: seed result for current user. - -- `POST /user/clear-demo-data` - - Rate-limited by demo-data limiter. - - Behavior: deletes `is_seeded=1` bills/categories for current user and records import history. - - Response: deletion counts. - -### 5.14 Import - -Mounted under `/api/import`; auth: user/admin tracker access; import limiter applies. - -- `POST /import/spreadsheet/preview?parse_all_sheets=true&year=&month=` - - Content-Type: `application/octet-stream`. - - Headers: optional `X-Filename`. - - Body: XLSX file buffer, max 10 MB. - - Response: preview session, parsed rows, categories/bills/payment candidates, ambiguous decisions, errors. - -- `POST /import/spreadsheet/apply` - - Body: `{import_session_id, decisions, options}`. - - Validation: session exists, not expired, belongs to user. - - Response: created/updated/skipped/error counts and import history summary. - -- `POST /import/user-db/preview` - - Content-Type: `application/octet-stream`. - - Headers: optional `X-Filename`. - - Body: user SQLite export file, max 50 MB. - - Response: preview session and sanitized import plan. - -- `POST /import/user-db/apply` - - Body: `{import_session_id, options}`. - - Validation: session exists, not expired, belongs to user. - - Response: apply result with ID mappings/counts. - -- `GET /import/history` - - Response: current user's import history. - -### 5.15 Data Sources - -Mounted under `/api/data-sources`; auth: user/admin tracker access. - -- `GET /data-sources?type=&status=` - - Query params: `type` (`manual`, `file_import`, `provider_sync`), `status` (`active`, `inactive`, `error`). - - Response: array of data sources with `source_label`, `source_type_label`, `account_count`, `transaction_count`, safe fields (encrypted_secret excluded), timestamps, sync info. - -### 5.16 Transactions - -Mounted under `/api/transactions`; auth: user/admin tracker access. - -- `GET /transactions?limit=&offset=&match_status=&ignored=&source_type=&start_date=&end_date=&q=&data_source_id=&account_id=&matched_bill_id=` - - Response: paginated list of transactions with embedded data_source and account details, `source_label`, `source_type_label`. - - Filter: `match_status` (`unmatched`, `matched`, `ignored`), `ignored` (boolean), `source_type`, date range, free-text search (`q`) across description/payee/memo/category. - -- `POST /transactions/manual` - - Body: `{account_id?, transaction_type?, posted_date, transacted_at?, amount, currency?, description?, payee?, memo?, category?, matched_bill_id?, match_status?, ignored?}`. - - Response 201: created transaction with manual `source_type`. - -- `PUT /transactions/:id` - - Body: partial transaction fields. - - Validation: match_state changes use dedicated endpoints. - - Response: updated transaction. - -- `DELETE /transactions/:id` - - Behavior: soft-deletes via unmatch then hard delete. - - Response: `{success:true, deleted:true, id}`. - -- `POST /transactions/:id/match` - - Body: `{billId}`. - - Response: `{success:true, matched:true, transaction}`. - -- `POST /transactions/:id/unmatch` - - Response: `{success:true, unmatched:true, transaction}`. - -- `POST /transactions/:id/ignore` - - Response: `{transaction}` with `match_status='ignored'`, `ignored=1`. - -- `POST /transactions/:id/unignore` - - Response: `{transaction}` restored to `unmatched` state. - -### 5.17 CSV Import - -Mounted under `/api/import`; auth: user/admin tracker access; import limiter applies. Gated by `DATA_IMPORT_ENABLED` env var (defaults to true). - -- `POST /import/csv/preview` - - Content-Type: `text/csv`. - - Body: raw CSV. - - Response: `{import_session_id, headers, sampleRows, rowCount, suggestedMapping, errors, fields}`. - -- `POST /import/csv/commit` - - Body: `{import_session_id, mapping, options?}`. - - Response: `{imported, skipped, failed, details}`. - -- `GET /import/history` - - Response: current user's import history. - -### 5.18 Match Suggestions - -Mounted under `/api/matches`; auth: user/admin tracker access. - -- `GET /matches/suggestions?transaction_id=&bill_id=&limit=&offset=` - - Response: `{suggestions:[{id, transaction, bill, score, match_status, created_at}]}`. - -- `POST /matches/:id/reject` - - Response: `{success:true}` after recording rejection. - -### 5.19 Export - -Mounted under `/api/export`; auth: user/admin tracker access; export limiter applies. - -- `GET /export?year=YYYY&format=csv|xlsx` - - Response: file download of payment/bill history for the requested year. CSV includes date, bill, category, expected, paid, method, notes, actual amount, monthly notes. XLSX includes enriched rows. - -- `GET /export/user-excel` - - Response: Excel workbook with user categories, bills, payments, monthly state, monthly starting amounts, and notes/metadata. - -- `GET /export/user-db` - - Response: portable SQLite file with export metadata and user-owned categories, bills, payments, monthly state, monthly starting amounts, and notes. - -### 5.20 Status - -- `GET /status` - - Auth: admin. - - Response: app version, uptime, runtime worker state, DB health/counts/path/size, SMTP configuration status, backup status/schedule, current-month tracker health, recent errors. - -### 5.21 About and Version - -- `GET /about` - - Public. - - Response: package version and public project/about metadata. - -- `GET /about-admin` - - Auth: admin; admin action limiter; CSRF middleware. - - Response: package version plus sanitized/redacted `FUTURE.md` and `DEVELOPMENT_LOG.md` content. - - File allowlist only: `FUTURE.md`, `DEVELOPMENT_LOG.md`. - -- `GET /version` - - Public. - - Response: current package version and latest structured notes from `HISTORY.md`. - -- `GET /version/history` - - Public. - - Response: package version and raw history text, or error if unavailable. - -### 5.22 Services - -Key service modules: - -- **`paymentValidation.js`** — Payment input validation with `PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match']` enum, `validatePaymentSource()`, and `validatePaymentInput()`. - -- **`csvTransactionImportService.js`** — CSV parsing, field mapping, SHA-256 deduplication, import session management with preview/commit workflow. - -- **`transactionService.js`** — Transaction helpers: `ensureManualDataSource()`, `decorateDataSource()`, `decorateTransaction()`. - -- **`transactionMatchService.js`** — Match/unmatch transactions to bills: `matchTransactionToBill()`, `unmatchTransaction()`, `ignoreTransaction()`, `unignoreTransaction()`. - -- **`matchSuggestionService.js`** — Match suggestion discovery: `listMatchSuggestions()`, `rejectMatchSuggestion()`, `suggestionCounts()`. - -- **`snowballService.js`** — Debt snowball/avalanche calculations, Ramsey mode support, order updates. - -- **`dataSourcesService.js`** — Data source CRUD with `ensureManualDataSource()` for user-scoped manual sources. - -- **`monthlyStartingAmountsService.js`** — Starting cash bucket tracking: first/fifteenth/other bucket amounts, payments, remaining values. - -- **`auditService.js`** — Audit logging via `logAudit()`; lazy-loaded in `database.js` to avoid circular dependency. - -- **`emailService.js`** — Email dispatch with SMTP configuration, templating, retry logic. - -- **`exportService.js`** — Export helpers: CSV, XLSX, SQLite user DB export with metadata. - -- **`csvTransactionImportService.js`** — CSV parsing, field mapping, SHA-256 deduplication, import session management with preview/commit workflow. - -- **`trackerService.js`** — Tracker calculations, cycle detection (weekly/biweekly/quarterly/annual), debt snowball support. - -- **`statusService.js`** — Health status, cycle validation, autopay simulation, budget projections. - -- **`analyticsService.js`** — Analytics queries: spending, categories, bills, filters. - -- **`notificationService.js`** — Bill notifications: due_3d, due_1d, due_today, overdue. - -- **`authService.js`** — Auth helpers: login, JWT, password hashing, session management, OIDC integration. - -- **`userService.js`** — User CRUD, profile updates, demo data seeding, role changes. - -- **`settingsService.js`** — Settings CRUD, allowed keys validation, SMTP/billing/export settings. - -- **`backupService.js`** — SQLite backup, retention, schedule. - -- **`cronService.js`** — Scheduled tasks: backup, cleanup, auto-mark paid, cycle updates. - -### 5.23 Import and Sync Workflow - -Data ingestion follows a layered architecture: - -1. **Data Sources** (`data_sources` table) - - `manual`: User-created source (one per user, type='manual', provider='manual') - - `file_import`: CSV/XLSX imports (provider='csv_transactions', 'spreadsheet') - - `provider_sync`: External institution sync (e.g., 'plaid', 'mint') - - Fields: `type`, `provider`, `name`, `status` ('active', 'inactive', 'error'), `config_json`, `encrypted_secret`, `last_sync_at`, `last_error` - -2. **Financial Accounts** (`financial_accounts` table) - - Linked to `data_sources` via `data_source_id` - - One data source can have many accounts - - Fields: `provider_account_id`, `name`, `org_name`, `account_type`, `currency`, `balance`, `available_balance`, `raw_data` - -3. **Transactions** (`transactions` table) - - Linked to `data_sources` and `financial_accounts` - - Source type: `manual`, `file_import`, `provider_sync` - - Match states: `unmatched`, `matched`, `ignored` - - Optional `provider_transaction_id` for deduplication - - Fields: `amount` (cents), `transaction_type`, `posted_date`, `transacted_at`, `description`, `payee`, `memo`, `category`, `raw_data`, `matched_bill_id`, `match_status`, `ignored` - -4. **CSV Import Flow** - - User uploads CSV → `/api/import/csv/preview` - - Preview parses headers, suggests field mapping - - `/api/import/csv/commit` writes to `transactions` with `source_type='file_import'` - - Import history tracked in `import_history` with counts - -5. **Transaction Matching** - - Manual transactions (`source_type='manual'`) can be matched to bills - - Match suggestions discovered via `matchSuggestionService` - - Users can reject suggestions to avoid重复 suggestions - -6. **Provider Sync** (future) - - External sync jobs write to `data_sources` with `type='provider_sync'` - - Financial accounts created per institution account - - Transactions imported from provider - - Match suggestions offered for unmatched transactions - -7. **Payments** (bills → payments) - - Payments link to transactions via `transaction_id` for auto-draft - - `payment_source` indicates origin: `manual`, `file_import`, `provider_sync`, `transaction_match` - - Balance delta tracked for debt payoff - -8. **Import Sessions** (`import_sessions` table) - - Temporary storage for CSV/XLSX previews - - 1-hour TTL, auto-cleaned - - Fields: `preview_json`, `expires_at` - -9. **Import History** (`import_history` table) - - Audit trail of all imports - - Fields: `imported_at`, `source_filename`, `file_type`, `rows_parsed/created/updated/skipped/errored`, `options_json`, `summary_json` - -### 5.24 Validation and Services - -Key validation and service patterns: - -- **`paymentValidation.js`** - - `PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match']` enum - - `validatePaymentSource(value)` returns error if not in list - - `validatePaymentInput(body)` validates amount, paid_date, bill ownership, payment_source - - Invalid source returns 400 with `VALIDATION_ERROR` code - -- **CSV Import Service** - - Field suggestion via header analysis - - SHA-256 hash for deduplication - - Import session management with preview/commit split - - Error collection per row with detailed messages - -- **Transaction Service** - - `ensureManualDataSource(db, userId)` creates one manual data source per user - - `decorateTransaction(row)` adds `source_label`, `source_type_label` - - `decorateDataSource(row)` adds `source_label`, `source_type_label`, `account_count`, `transaction_count` - -- **Snowball Service** - - Computes snowball/avalanche orderings - - Ramsey mode: minimum payments only vs. full extra payment - - `snowball_include` bills sorted by `snowball_order` - - `snowball_exempt` bills excluded from ordering - -- **Match Suggestion Service** - - `listMatchSuggestions(userId, transactionId, billId, limit, offset)` - - `rejectMatchSuggestion(userId, transactionId, billId)` - - `suggestionCounts(userId)` - - Rejects stored to prevent repeated suggestions - -- **Monthly Starting Amounts Service** - - Bucketed amounts: first_amount, fifteenth_amount, other_amount - - Payment tracking from each bucket - - Remaining values computed on read - -- **Tracker Service** - - Cycle type detection: monthly, weekly, biweekly, quarterly, annually - - Cycle day mapping for non-standard cycles - - Auto-mark paid logic for autopay bills - -- **Status Service** - - Cycle validation warnings - - Autopay simulation - - Budget projections - -### 6. Database Reference +## 6. Database Reference SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/database.js` applies migrations to reach the current schema. @@ -1023,13 +621,49 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `last_login_at TEXT` - `active INTEGER NOT NULL DEFAULT 1` - `is_default_admin INTEGER NOT NULL DEFAULT 0` +- `snowball_extra_payment REAL` (`v0.48`) +- `last_seen_version TEXT` (`v0.50`) — release-notes notification stamp +- `webauthn_user_id TEXT` (`v0.92`); `webauthn_enabled INTEGER NOT NULL DEFAULT 0` (`v0.92`) +- `totp_enabled INTEGER NOT NULL DEFAULT 0` (`v0.86`); `totp_secret` (encrypted, `v0.86`); `totp_recovery_codes` (JSON array of hashes, `v0.86`) +- Push notification columns (`v0.80`): ntfy / Gotify / Discord / Telegram (topic + URL + enabled flag each) +- `geolocation_enabled INTEGER NOT NULL DEFAULT 0` (`v1.02`; was a global admin setting) #### `sessions` -- `id TEXT PRIMARY KEY` +- `id TEXT PRIMARY KEY` — **session token hash**, not the raw cookie value (`v0.94`). The cookie carries a random opaque ID; the DB stores its SHA-256 so a leaked DB does not yield usable session tokens. - `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` - `expires_at TEXT NOT NULL` -- `created_at TEXT DEFAULT datetime('now')` +- `created_at TEXT DEFAULT datetime('now')` (`v0.43`) + +#### `webauthn_credentials` (`v0.92`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `credential_id TEXT NOT NULL UNIQUE` +- `public_key TEXT NOT NULL` +- `sign_count INTEGER NOT NULL DEFAULT 0` +- `transports TEXT` (JSON) +- `backup_eligible INTEGER NOT NULL DEFAULT 0` +- `backup_state INTEGER NOT NULL DEFAULT 0` +- `credential_name TEXT` +- `aaguid TEXT` +- `last_used_at TEXT` +- `created_at TEXT NOT NULL DEFAULT (datetime('now'))` + +#### `webauthn_challenges` (`v0.92`) + +- `id TEXT PRIMARY KEY` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `challenge_type TEXT NOT NULL CHECK ('registration' OR 'authentication')` +- `challenge TEXT NOT NULL` +- `expires_at TEXT NOT NULL` (15-min TTL) + +#### `totp_challenges` (`v0.86`) + +- `id TEXT PRIMARY KEY` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `challenge TEXT NOT NULL` +- `expires_at TEXT NOT NULL` (5-min TTL) #### `categories` @@ -1039,6 +673,19 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `created_at TEXT DEFAULT datetime('now')` - `updated_at TEXT DEFAULT datetime('now')` - `is_seeded INTEGER NOT NULL DEFAULT 0` +- `deleted_at TEXT` (`v0.54`) +- `sort_order INTEGER` (`v0.75`) — persistent order +- `spending_enabled INTEGER NOT NULL DEFAULT 0` (`v0.88`) — separates bill vs spending categories +- `group_id INTEGER REFERENCES category_groups(id) ON DELETE SET NULL` (`v1.06`) + +#### `category_groups` (`v1.06`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `name TEXT NOT NULL` +- `sort_order INTEGER` +- `created_at`, `updated_at` +- Unique: `(user_id, name)` #### `bills` @@ -1049,9 +696,9 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `due_day INTEGER NOT NULL CHECK 1-31` - `override_due_date TEXT` - `bucket TEXT CHECK ('1st','15th')` -- `expected_amount REAL NOT NULL DEFAULT 0` +- `expected_amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`) - `interest_rate REAL CHECK null or 0-100` -- `billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')` +- `billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')` (canonicalized in `v0.76`) - `autopay_enabled INTEGER NOT NULL DEFAULT 0` - `autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')` - `website TEXT` @@ -1066,25 +713,33 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `is_seeded INTEGER NOT NULL DEFAULT 0` - `cycle_type TEXT NOT NULL DEFAULT 'monthly'` - `cycle_day TEXT` -- `current_balance REAL` -- `minimum_payment REAL` +- `current_balance REAL` (dollars; converted to integer cents by `v1.03`) +- `minimum_payment REAL` (dollars; converted to integer cents by `v1.03`) - `snowball_order INTEGER` - `snowball_include INTEGER NOT NULL DEFAULT 0` - `snowball_exempt INTEGER NOT NULL DEFAULT 0` - `auto_mark_paid INTEGER NOT NULL DEFAULT 0` - `deleted_at TEXT` +- `is_subscription INTEGER NOT NULL DEFAULT 0`, `subscription_type TEXT` (`v0.63`) +- `interest_accrued_month TEXT` (`v0.93`) — calendar month of last applied interest +- `drift_snoozed_until TEXT` (`v0.71`) — suppresses drift notifications until this date +- `sort_order INTEGER` (`v0.72`) — persistent tracker order +- `catalog_id INTEGER REFERENCES subscription_catalog(id) ON DELETE SET NULL` (`v0.96`) +- `autopay_verified_at TEXT`, `inactive_reason TEXT` (`v0.99`) — autopay trust / lifecycle fields #### `payments` - `id INTEGER PRIMARY KEY` - `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE` -- `amount REAL NOT NULL` +- `amount REAL NOT NULL` (dollars; converted to integer cents by `v1.03`) - `paid_date TEXT NOT NULL` - `method TEXT` - `notes TEXT` -- `balance_delta REAL` -- `payment_source TEXT NOT NULL DEFAULT 'manual'` -- `transaction_id INTEGER` +- `balance_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.49`) +- `payment_source TEXT NOT NULL DEFAULT 'manual'` (`v0.59`; `auto_match` normalized to `provider_sync` in `v0.82`) +- `transaction_id INTEGER` (`v0.59`; unique-per-active index added in `v0.61`) +- `interest_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.93`) +- `accounting_excluded INTEGER NOT NULL DEFAULT 0` (`v0.98`) — bank-override metadata for provisional manual payments - `created_at TEXT DEFAULT datetime('now')` - `updated_at TEXT DEFAULT datetime('now')` - `deleted_at TEXT` @@ -1095,9 +750,10 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE` - `year INTEGER NOT NULL CHECK 2000-2100` - `month INTEGER NOT NULL CHECK 1-12` -- `actual_amount REAL` +- `actual_amount REAL` (dollars; converted to integer cents by `v1.03`) - `notes TEXT` - `is_skipped INTEGER NOT NULL DEFAULT 0` +- `snoozed_until TEXT` (`v0.70`) — overdue command-center snooze - `created_at TEXT DEFAULT datetime('now')` - `updated_at TEXT DEFAULT datetime('now')` - Unique: `(bill_id, year, month)` @@ -1109,10 +765,10 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `year INTEGER NOT NULL` - `month INTEGER NOT NULL` - `label TEXT NOT NULL DEFAULT 'Salary'` -- `amount REAL NOT NULL DEFAULT 0` +- `amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`) - `created_at TEXT DEFAULT datetime('now')` - `updated_at TEXT DEFAULT datetime('now')` -- Unique intended/current logic: `(user_id, year, month)` via migration/index. +- Unique: `(user_id, year, month)` #### `monthly_starting_amounts` @@ -1126,7 +782,7 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `notes TEXT` - `created_at TEXT DEFAULT datetime('now')` - `updated_at TEXT DEFAULT datetime('now')` -- Unique intended/current logic: `(user_id, year, month)` via migration/index. +- Unique: `(user_id, year, month)` #### `bill_history_ranges` @@ -1194,6 +850,8 @@ SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/da - `matched_bill_id INTEGER REFERENCES bills(id) ON DELETE SET NULL` - `match_status TEXT NOT NULL DEFAULT 'unmatched'` (`unmatched`, `matched`, `ignored`) - `ignored INTEGER NOT NULL DEFAULT 0` +- `spending_category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL` (`v0.87`) +- `pending INTEGER NOT NULL DEFAULT 0` (`v1.01`) — SimpleFIN pending transactions - `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP` - `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP` - Unique partial index: `(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL` @@ -1206,13 +864,18 @@ Important indexes include: - `idx_bills_active(active)` - `idx_bills_user_active(user_id, active)` - `idx_bills_user_name(user_id, name)` +- `idx_bills_user_deleted(user_id, deleted_at)` (`v0.91`) +- `idx_bills_user_sort(user_id, active, sort_order, due_day)` (`v0.72`) - `idx_categories_user_name(user_id, name)` - `idx_categories_user_name_unique(user_id, name COLLATE NOCASE)` +- `idx_categories_user_deleted(user_id, deleted_at)` (`v0.91`) +- `idx_categories_user_sort(user_id, sort_order, name)` (`v0.75`) - `idx_payments_bill_id(bill_id)` - `idx_payments_paid_date(paid_date)` - `idx_payments_bill_date_del(bill_id, paid_date, deleted_at)` - `idx_payments_deleted(deleted_at)` - `idx_payments_method(method)` +- `idx_payments_transaction_active` — unique-per-active on `transaction_id` (`v0.61`) - `idx_sessions_user_id(user_id)` - `idx_sessions_expires(expires_at)` - `idx_monthly_bill_state_lookup(bill_id, year, month)` @@ -1222,6 +885,18 @@ Important indexes include: - `idx_notifications_lookup(bill_id, user_id, year, month)` - `idx_import_sessions_user(user_id)`, `idx_import_sessions_expires(expires_at)` - `idx_import_history_user(user_id)`, `idx_import_history_imported_at(imported_at)` +- `idx_oidc_states_expires(expires_at)` +- `idx_bill_history_ranges_bill(bill_id)` +- `idx_audit_log_user(user_id, created_at)`, `idx_audit_log_action(action, created_at)` +- `idx_data_sources_user_type(user_id, type, status)` +- `idx_data_sources_user_manual(user_id, type, provider)` WHERE `type='manual' AND provider='manual'` +- `idx_financial_accounts_user_source(user_id, data_source_id)` +- `idx_transactions_user_date(user_id, posted_date, transacted_at)` +- `idx_transactions_user_match(user_id, match_status, ignored)` +- `idx_transactions_account(account_id)` +- `idx_transactions_matched_bill(matched_bill_id)` +- `idx_transactions_provider_dedupe(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL` +- `idx_bill_merchant_rules_user_bill(user_id, bill_id)` (`v0.81`) #### `import_sessions` @@ -1281,23 +956,96 @@ Important indexes include: - `id INTEGER PRIMARY KEY AUTOINCREMENT` - `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` - `logged_in_at TEXT NOT NULL DEFAULT (datetime('now'))` -- `ip_address TEXT` -- `user_agent TEXT` +- `ip_address TEXT` (encrypted at rest, `v0.84`) +- `user_agent TEXT` (encrypted at rest, `v0.84`) - `browser TEXT` - `os TEXT` - `device_type TEXT` - `device_fingerprint TEXT` -- `idx_oidc_states_expires(expires_at)` -- `idx_bill_history_ranges_bill(bill_id)` -- `idx_audit_log_user(user_id, created_at)`, `idx_audit_log_action(action, created_at)` -- `idx_data_sources_user_type(user_id, type, status)` -- `idx_data_sources_user_manual(user_id, type, provider)` WHERE `type='manual' AND provider='manual'` -- `idx_financial_accounts_user_source(user_id, data_source_id)` -- `idx_transactions_user_date(user_id, posted_date, transacted_at)` -- `idx_transactions_user_match(user_id, match_status, ignored)` -- `idx_transactions_account(account_id)` -- `idx_transactions_matched_bill(matched_bill_id)` -- `idx_transactions_provider_dedupe(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL` +- `location TEXT` (`v0.84`) — IP-geolocation city/region/country when geolocation opt-in is on +- `success INTEGER NOT NULL DEFAULT 1` (`v0.85`) — failed attempts when 0 +- `session_fingerprint TEXT` (`v0.85`) — for current-session detection + +#### `subscription_catalog` (`v0.65`, expanded `v0.69`, `v0.95`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `name TEXT NOT NULL` +- `category TEXT` +- `subcategory TEXT` (`v0.95`) +- `starting_monthly_usd REAL` (`v0.95`) +- `bank_descriptors TEXT` (JSON array; populated by `v0.95` from a 2026 researched dataset) +- `created_at`, `updated_at` + +#### `declined_subscription_hints` (`v0.66`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `descriptor TEXT NOT NULL` +- `source TEXT` +- `declined_at TEXT NOT NULL DEFAULT (datetime('now'))` +- Unique: `(user_id, descriptor, source)` + +#### `bill_merchant_rules` (`v0.67`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE` +- `normalized_merchant TEXT NOT NULL` +- `match_mode TEXT NOT NULL DEFAULT 'contains'` (e.g. `contains`, `equals`, `regex`) +- `priority INTEGER NOT NULL DEFAULT 0` +- `auto_attribute_late INTEGER NOT NULL DEFAULT 0` (`v0.83`) +- `created_at`, `updated_at` +- Composite index `(user_id, bill_id)` for fast `EXISTS` lookups (`v0.81`) + +#### `user_catalog_descriptors` (`v0.96`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `catalog_id INTEGER NOT NULL REFERENCES subscription_catalog(id) ON DELETE CASCADE` +- `descriptor TEXT NOT NULL` +- `created_at TEXT NOT NULL DEFAULT (datetime('now'))` + +#### `subscription_recommendation_feedback` (`v0.97`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `recommendation_key TEXT NOT NULL` +- `verdict TEXT NOT NULL` (`accept` | `decline` | `mute`) +- `created_at TEXT NOT NULL DEFAULT (datetime('now'))` + +#### `snowball_plans` (`v0.73`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `name TEXT NOT NULL` +- `strategy TEXT NOT NULL` (`snowball` | `avalanche` | `minimum_only`) +- `extra_payment REAL NOT NULL DEFAULT 0` +- `status TEXT NOT NULL DEFAULT 'active'` (`active` | `paused` | `completed` | `abandoned`) +- `created_at`, `updated_at`, `completed_at` +- Lifecycle transitions are recorded with audit + plan-row updates; `pause`/`resume` flip `status`. + +#### `calendar_tokens` (`v1.00`) + +- `id INTEGER PRIMARY KEY AUTOINCREMENT` +- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE` +- `token TEXT NOT NULL UNIQUE` (random opaque) +- `label TEXT` +- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP` +- `last_used_at TEXT` +- `revoked_at TEXT` + +#### `spending_rules` / `spending_budgets` (`v0.87`) + +- `spending_rules`: per-user regex/keyword rules mapping a transaction pattern to a `spending_category_id`. Auto-applied during bank sync by `applySpendingCategoryRules`. +- `spending_budgets`: per-user monthly budget rows; month-rollover uses the copy endpoint. + +#### `merchant_store_matches` (seeded by `v1.05`) + +- Static 5k merchant/store matching pack; used by `applyMerchantStoreMatches` during bank sync to normalize merchant strings. + +#### `advisory_non_bill_filters` (seeded by `v0.68`) + +- 5k advisory non-bill patterns + bill-like override terms. ### Migration system @@ -1305,14 +1053,14 @@ Important indexes include: - Reads `db/schema.sql` in `initSchema()`. - Creates `schema_migrations`. -- Detects and reconciles legacy databases. -- Applies ordered migrations only when not already recorded. -- Validates dependency chains before applying dependent migrations. +- Detects and reconciles legacy databases via `db/migrations/legacyReconcileMigrations.js`. +- Applies ordered migrations from `db/migrations/versionedMigrations.js` only when not already recorded. +- Validates dependency chains (`dependsOn: ['v0.71']`) before applying dependent migrations. - Uses a column whitelist for dynamic `ALTER TABLE` statements. - Wraps versioned migrations in transactions, with special handling for `v0.40` because it uses PRAGMA-driven table rebuild work. - Supports rollback SQL for selected migrations through `ROLLBACK_SQL_MAP` and `rollbackMigration(version)`. -Current migration set: +Current migration set (`v0.2` → `v1.06`): - `v0.2` payments soft delete. - `v0.3` tracker payment compound index. @@ -1338,18 +1086,61 @@ Current migration set: - `v0.49` payments: `balance_delta` column for debt payoff tracking. - `v0.50` users: `last_seen_version` for release-notes notifications. - `v0.51` user_login_history table. +- `v0.52`–`v0.53` ad-hoc columns (legacy reconcile). - `v0.54` bills/categories: soft-delete columns (`deleted_at`). -- `v0.55` autopay: auto_mark_paid and suggestion dismissals table. +- `v0.55` autopay: `auto_mark_paid` and suggestion dismissals table. - `v0.56` bills: saved bill templates table. -- `v0.57` match_suggestion_rejections table. +- `v0.57` match_suggestion_rejections table (replaced by `v0.62`). - `v0.58` import_sessions and import_history tables. - `v0.59` payments: `payment_source` and `transaction_id` columns. - `v0.60` data_sources, financial_accounts, transactions tables. - `v0.61` payments: one active payment per linked transaction (unique index on `transaction_id`). -- `v0.62` match_suggestion_rejections table. -- `v0.63` data_sources: partial unique index on `(user_id, type, provider)` WHERE type='manual' AND provider='manual'. -- `v0.64` transactions: partial unique index on `(data_source_id, provider_transaction_id)` WHERE provider_transaction_id IS NOT NULL; indexes on `(user_id, posted_date)`, `(user_id, match_status, ignored)`, `account_id`, `matched_bill_id`. -- Unversioned user notification columns are also reconciled. +- `v0.62` match_suggestion_rejections table (canonical). +- `v0.63` bills: subscription metadata fields (`is_subscription`, `subscription_type`). +- `v0.64` financial_accounts: `monitored` flag for bill matching; transactions: partial unique index on `(data_source_id, provider_transaction_id)` and supporting indexes. +- `v0.65` subscription_catalog: top-200 known subscription services. +- `v0.66` declined_subscription_hints: per-user dismissed recommendation store. +- `v0.67` bill_merchant_rules: persistent merchant→bill auto-match rules. +- `v0.68` advisory_non_bill_filters: 5k advisory patterns + bill-like override terms. +- `v0.69` subscription_catalog v2: 90 new services + category fixes. +- `v0.70` monthly_bill_state: `snoozed_until` for overdue command center. +- `v0.71` bills: `drift_snoozed_until`; users: `notify_amount_change`. +- `v0.72` bills: persistent tracker sort order. +- `v0.73` snowball_plans table for plan lifecycle + history. +- `v0.74` subscription_catalog: Claude.ai Anthropic matching. +- `v0.75` categories: persistent sort order. +- `v0.76` bills: canonical billing schedule cleanup. +- `v0.77` encrypt SMTP password at rest. +- `v0.78` re-encrypt secrets from SHA-256 to HKDF key derivation. +- `v0.79` encrypt OIDC client secret at rest. +- `v0.80` users: push notification columns (ntfy / Gotify / Discord / Telegram). +- `v0.81` bill_merchant_rules: composite index on `(user_id, bill_id)`. +- `v0.82` payments: normalise `auto_match` source to `provider_sync`. +- `v0.83` bill_merchant_rules: `auto_attribute_late` flag. +- `v0.84` user_login_history: encrypt ip/useragent at rest + add `location` + keep 10 records. +- `v0.85` user_login_history: failed-attempt tracking + `session_fingerprint`. +- `v0.86` users: TOTP/authenticator 2FA columns + `totp_challenges` table. +- `v0.87` spending: category assignment on transactions + rules + budgets + default categories. +- `v0.88` categories: `spending_enabled` flag to separate bill vs spending categories. +- `v0.89` categories: seed spending defaults for users who had existing categories before `v0.87`. +- `v0.90` re-normalize merchant rules after `&` fix; ensure rejection expiry column. +- `v0.91` performance: composite indexes on `user_id+deleted_at` for categories, bills, payments. +- `v0.92` auth: WebAuthn/FIDO2 security key support — `webauthn_credentials` + `webauthn_challenges` tables. +- `v0.93` bills: `interest_accrued_month`; payments: `interest_delta`; transactions: stable provider key + dedupe index. +- `v0.94` security: session token hashing + geolocation opt-in setting (seeded into `settings`). +- `v0.95` subscription_catalog: bank descriptors + pricing from 2026 researched dataset. +- `v0.96` bills: `catalog_id` FK; `user_catalog_descriptors` for custom bank descriptors. +- `v0.97` subscription recommendation feedback: per-user learning signals. +- `v0.98` payments: `accounting_excluded` for bank-override metadata. +- `v0.99` bills: autopay trust indicators (`autopay_verified_at`, `inactive_reason`) + lifecycle fields; payments: autopay failure flag. +- `v1.00` calendar feed subscription tokens (`calendar_tokens`). +- `v1.01` transactions: `pending` flag for SimpleFIN pending transactions. +- `v1.02` users: per-user geolocation opt-in (was global admin setting). +- `v1.03` money columns: dollars (REAL) → integer cents for `bills.expected_amount/current_balance/minimum_payment`, `payments.amount/balance_delta/interest_delta`, `monthly_bill_state.actual_amount`, `monthly_income.amount`, plus relevant index updates. +- `v1.04` `bill_templates.data` JSON: money fields dollars → integer cents (column-only `v1.03` does not touch the JSON blob; `serializeTemplateData` reads cents). +- `v1.05` `merchant_store_matches`: 5k merchant/store matching pack for bank transaction categorization. +- `v1.06` `category_groups`: organize spending categories into named groups. +- Unversioned user notification columns are also reconciled by `legacyReconcileMigrations.js`. Migration logging is both console-based and audit-backed: @@ -1358,7 +1149,7 @@ Migration logging is both console-based and audit-backed: - Audit actions include `migration.start`, `migration.complete`, and `migration.failure`. - Rollback paths audit `migration.rollback` and `migration.rollback.failure`. -Rollback support is defined by `ROLLBACK_SQL_MAP`: +Rollback support is defined by `ROLLBACK_SQL_MAP` in `db/database.js` and currently covers `v0.44` through `v1.04` (the most recent migrations `v1.05` and `v1.06` are **not** yet rollbackable): - `v0.44` — drops selected performance indexes: `idx_bills_user_name`, `idx_payments_method`, `idx_monthly_starting_amounts_user`, and `idx_import_history_imported_at`. - `v0.45` — drops `idx_audit_log_user`, `idx_audit_log_action`, and the `audit_log` table. @@ -1379,6 +1170,33 @@ Rollback support is defined by `ROLLBACK_SQL_MAP`: - `v0.62` — drops `match_suggestion_rejections` table. - `v0.63` — drops partial unique index on `data_sources`. - `v0.64` — drops partial unique index and indexes on `transactions`. +- `v0.72` — drops `bills.sort_order` and `idx_bills_user_sort`. +- `v0.75` — drops `categories.sort_order` and `idx_categories_user_sort`. +- `v0.76` — drops `bills.billing_cycle` (canonical cleanup). +- `v0.77`–`v0.79` — revert SMTP/OIDC secret encryption (re-store plaintext). +- `v0.81` — drops `idx_bill_merchant_rules_user_bill`. +- `v0.82` — no-op rollback (data migration). +- `v0.83` — drops `bill_merchant_rules.auto_attribute_late`. +- `v0.84`–`v0.85` — revert `user_login_history` encryption and drop `success`/`session_fingerprint`/`location`. +- `v0.86` — drops `users.totp_*` columns and `totp_challenges`. +- `v0.87` — drops `transactions.spending_category_id`, `spending_rules`, `spending_budgets`, related defaults. +- `v0.88` — drops `categories.spending_enabled`. +- `v0.89` — data-only (seeding); no schema rollback. +- `v0.90` — data-only (re-normalization); no schema rollback. +- `v0.91` — drops `idx_categories_user_deleted`, `idx_bills_user_deleted`, `idx_payments_user_deleted`. +- `v0.92` — drops `webauthn_credentials`, `webauthn_challenges`, and the `webauthn_*` columns on `users`. +- `v0.93` — drops `bills.interest_accrued_month`, `payments.interest_delta`, transactions dedupe index + indexes. +- `v0.94` — reverts session token hashing (drops the hash; cookie value is now the raw ID again); removes `geolocation_enabled` setting row. +- `v0.95` — drops `subscription_catalog.bank_descriptors` / `subcategory` / `starting_monthly_usd`. +- `v0.96` — drops `bills.catalog_id` and `user_catalog_descriptors` table. +- `v0.97` — drops `subscription_recommendation_feedback` table. +- `v0.98` — drops `payments.accounting_excluded`. +- `v0.99` — drops `bills.autopay_verified_at`/`inactive_reason` (and any payments autopay-failure column added in the same migration). +- `v1.00` — drops `calendar_tokens` table. +- `v1.01` — drops `transactions.pending`. +- `v1.02` — drops `users.geolocation_enabled` and re-seeds the global admin setting key. +- `v1.03` — reverts the integer-cents conversion (back to dollar REALs) for the listed columns. +- `v1.04` — reverts `bill_templates.data` money fields back to dollars. `rollbackMigration(version)` requires an initialized database, verifies the version exists in `schema_migrations`, looks up rollback SQL in `ROLLBACK_SQL_MAP`, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns `{success:true, version, description, elapsed_ms}`. If the migration is not recorded, it throws `NOT_APPLIED`. If no rollback definition exists, it throws `ROLLBACK_NOT_SUPPORTED`. Execution failures roll back the transaction and are audited as `migration.rollback.failure`. @@ -1404,141 +1222,187 @@ Use this pattern for database-layer audit calls instead of a top-level `require( ### Frontend stack -- React `^18.3.1` -- Vite `^5.4.10` +- React `^19.2.7` (with `babel-plugin-react-compiler` for automatic memoization) +- TypeScript `^6.0.3` with `strict: true` + `noUncheckedIndexedAccess: true`; `allowJs: true`, `checkJs: false` for the gradual JS → TS migration +- Vite `^5.4.10` with `@vitejs/plugin-react` and `vite-plugin-pwa` (autoUpdate service worker, manifest, runtime caching for tracker/bills/calendar/summary/analytics/snowball/categories) - React Router `^6.26.2` -- TanStack Query `^5.100.9` -- Tailwind CSS `^3.4.14` +- TanStack Query `^5.100.9` + `@tanstack/react-query-devtools` + `@tanstack/react-virtual` for the virtualized Subscriptions list +- Tailwind CSS `^3.4.14` (content glob: `./client/**/*.{js,jsx,ts,tsx}` — must include `ts,tsx` or `.tsx`-only classes generate no CSS) - shadcn/ui component primitives, backed by Radix UI where applicable -- Sonner/shadcn toast notifications via `sonner` +- Sonner toast notifications via `sonner` +- `framer-motion` `^12.40.0` for transitions - `react-markdown`, `remark-gfm`, `rehype-sanitize` for markdown rendering +- `vitest` `^4.1.8` + `@testing-library/react` for client unit tests (server tests stay on `node --test`) -### `client/main.jsx` +### `client/main.tsx` -Creates the React root and wraps `App` with global providers including auth/theme where defined. +Creates the React root and wraps `App` with global providers. Theme is provided by `client/contexts/ThemeContext.tsx`. -### `client/App.jsx` +### `client/App.tsx` -- Creates a TanStack `QueryClient` with `staleTime` 2 minutes, retry 1, no refetch on focus. +- Creates a TanStack `QueryClient` with: + - Global `QueryCache.onError` — toasts a `sonner` error when a *background* refetch fails while stale data is on screen. Pages render inline errors on the initial load so the toast only fires for refetches. + - `staleTime: 2 * 60 * 1000`, `retry: 1`, `refetchOnWindowFocus: false`. - Uses lazy loading and `Suspense` with `PageLoader` for most pages. -- Wraps route elements in `ErrorBoundary`. +- Wraps route elements in `ErrorBoundary`; pages further wrap in `PageTransition`. - Exposes `ReactQueryDevtools`. - Provides skip link for keyboard users. +- Mounts the `CommandPalette` and `ReleaseNotesDialog` globally. +- `RequireAuth` (`{children, role}`) behavior: + - Shows loading while auth state is `undefined`. + - Single-user mode bypasses the auth gate for `role: 'user'`. + - Redirects unauthenticated users to `/login` (preserves `state.from`). + - Default admin cannot access user routes — redirected to `/admin`. + - Role mismatches redirect to `/admin` (for admin role) or `/` (for user role). +- `AdminShell` wraps admin routes so the user-shell nav does not appear inside admin. -Routes: +Routes (all rendered through `Routes` with `ErrorBoundary`, lazy + `Suspense` for non-critical pages): -- `/login` → `LoginPage` public. -- `/about` → `AboutPage` public. -- `/release-notes` → `ReleaseNotesPage` public. -- `/admin` → `AdminPage`, admin only. -- `/admin/about` → admin shell + `AboutPage admin`, admin only. -- `/admin/roadmap` → admin shell + `AboutPage admin`, admin only. -- `/admin/status` → admin shell + `StatusPage`, admin only. -- `/status` → redirects admin to `/admin/status`. -- `/` → authenticated user layout index, `TrackerPage`. -- `/calendar` → `CalendarPage`. -- `/summary` → `SummaryPage`. -- `/bills` → `BillsPage`. -- `/categories` → `CategoriesPage`. -- `/analytics` → `AnalyticsPage`. -- `/settings` → `SettingsPage`. -- `/data` → `DataPage`. -- `/profile` → `ProfilePage`. -- `*` under user layout → redirect `/`. - -`RequireAuth` behavior: - -- Shows loading while auth state is `undefined`. -- Redirects unauthenticated users to `/login`. -- Allows admins to access user routes except default admin is redirected to `/admin`. -- Allows single-user mode only for user routes. -- Redirects role mismatches to `/admin` or `/`. +- `/login` → `LoginPage` (public; supports local + OIDC + TOTP challenge) +- `/about` → `AboutPage` (public) +- `/privacy` → `PrivacyPage` (public) +- `/release-notes` → `ReleaseNotesPage` (public) +- `/admin` → `AdminPage` (admin only) +- `/admin/about` → admin shell + `AboutPage admin` (admin only) +- `/admin/roadmap` → admin shell + `RoadmapPage` (admin only) +- `/admin/status` → admin shell + `StatusPage` (admin only) +- `/status` → redirects admin to `/admin/status` +- `/` (index under user layout) → `TrackerPage` +- `/calendar` → `CalendarPage` +- `/summary` → `SummaryPage` +- `/bills` → `BillsPage` +- `/subscriptions` → `SubscriptionsPage` (virtualized list, optimistic reorder, drag-reorder) +- `/subscriptions/catalog` → `SubscriptionCatalogPage` (admin view of the catalog) +- `/categories` → `CategoriesPage` +- `/health` → `HealthPage` (bill + payment health snapshot) +- `/analytics` → `AnalyticsPage` +- `/settings` → `SettingsPage` +- `/snowball` → `SnowballPage` (projection + plans) +- `/payoff` → `PayoffPage` (payoff scenarios) +- `/spending` → `SpendingPage` (categorized transactions, rules, budgets, income) +- `/bank-transactions` → `BankTransactionsPage` (the bank ledger, `Cents`-based) +- `/data` → `DataPage` (export, CSV/OFX/XLSX/SQLite imports, bank sync, transaction matching) +- `/profile` → `ProfilePage` +- `*` → `NotFoundPage` ### API client -`client/api.js`: +`client/api.ts` (TypeScript; converted from `api.js` in the TypeScript migration): -- `_fetch(method, path, body)` calls `/api${path}` with JSON headers and `credentials: include`. -- Mutating methods read `bt_csrf_token` cookie and send `x-csrf-token`. -- Non-OK responses throw `Error` with `status`, `data`, `details`, and `code`. -- Exposes grouped functions for auth, admin, notifications, profile, tracker, calendar, summary, bills, payments, categories, settings, analytics, status, version/about, import, export, data-sources, transactions, and matches. +- `_fetch(method, path, body, opts)` calls `/api${path}` with JSON headers and `credentials: include`. +- Reads CSRF token from the `bt_csrf_token` cookie **or** fetches once from `/api/auth/csrf-token` and caches the token in memory; sends `x-csrf-token` on `POST`/`PUT`/`PATCH`/`DELETE`. +- Non-OK responses throw an `Error` whose `status`/`data`/`details`/`code` match the server's structured `errorFormatter`. +- `ApiError` interface carries those fields; `TogglePaidResult` and `PaymentRecord` provide typed payment responses; `QueryParams` types the query-string builder. +- Exposes grouped functions for `auth`, `admin`, `notifications`, `profile`, `tracker`, `calendar`, `summary`, `bills`, `payments`, `categories`, `settings`, `analytics`, `status`, `version`/`about`, `import`, `export`, `data-sources`, `transactions`, `matches`, `snowball`, `subscriptions`, `spending`, `monthlyStartingAmounts`, `csvImport`, `matchSuggestions`. - File download/upload endpoints use raw `fetch` because responses/bodies are blobs or octet streams. +- **All 16 import sites** that referenced `@/api.js` were normalized to `@/api` so Vite/TS resolve the `.ts`. -#### Client API v0.28.1 additions +### Domain types — `client/types.ts` -- `api.dataSources(type, status)` → `/api/data-sources` -- `api.transactions(filters)` → `/api/transactions` -- `api.transactions.create(payload)` → `/api/transactions/manual` -- `api.transactions.update(id, payload)` → `/api/transactions/:id` -- `api.transactions.delete(id)` → `/api/transactions/:id` -- `api.transactions.match(id, {billId})` → `/api/transactions/:id/match` -- `api.transactions.unmatch(id)` → `/api/transactions/:id/unmatch` -- `api.transactions.ignore(id)` → `/api/transactions/:id/ignore` -- `api.transactions.unignore(id)` → `/api/transactions/:id/unignore` -- `api.csvImport.preview(file, options)` → `/api/import/csv/preview` -- `api.csvImport.commit(importSessionId, mapping, options)` → `/api/import/csv/commit` -- `api.import.history()` → `/api/import/history` -- `api.matchSuggestions.list(filters)` → `/api/matches/suggestions` -- `api.matchSuggestions.reject(id)` → `/api/matches/:id/reject` -- `api.snowball.get()` → `/api/snowball` -- `api.snowball.settings.get()` → `/api/snowball/settings` -- `api.snowball.settings.patch(payload)` → `/api/snowball/settings` -- `api.snowball.projection.get()` → `/api/snowball/projection` -- `api.snowball.order.patch(bills)` → `/api/snowball/order` -- `api.monthlyStartingAmounts.get(year, month)` → `/api/monthly-starting-amounts` -- `api.monthlyStartingAmounts.update(payload)` → `/api/monthly-starting-amounts` +- Branded `Dollars` and `Cents` money types (from `client/lib/money.ts`). +- `Bill`, `Payment`, `Category`, `TrackerResponse` (with `summary` / `bank_tracking` / `cashflow` / `rows`). +- `DriftBill`, `AutopaySuggestion`, `AmountSuggestion`, `AutopayStats`. +- `BankTransaction` (integer cents on the wire). +- `TimelineBill` for the Safe-to-Spend SVG timeline. +- Money fields are typed as `Dollars` for bill/payment/tracker/summary amounts and as `Cents` for raw bank transactions — so the unit mixup bug class is unrepresentable in typed code. +- A `money.type-test.ts` never-imported guard uses `@ts-expect-error` to assert each unit mixup is a real compile error, so `tsc --noEmit` fails loudly if the branding ever regresses. ### Auth state -### Auth state +`client/hooks/useAuth.tsx`: -`client/hooks/useAuth.jsx`: - -- Maintains `user`, `singleUserMode`, and loading state. +- `AuthContext` typed (a `createContext(null)` would have made `useContext` return `never`). +- Maintains `user`, `singleUserMode`, `loading`. - Calls `api.authMode()` and `api.me()` on startup. -- Exposes `logout()` and `refresh()`. +- Exposes `logout()`, `logoutAll()`, `refresh()`. +- `singleUserMode` is sourced from `/auth/mode`; `RequireAuth` consults it. ### Query hooks -`client/hooks/useQueries.js`: +`client/hooks/useQueries.ts` (TypeScript; converted from `useQueries.js`): -- `useTracker(year, month)` → `api.tracker(year, month)`. -- `useBills()` → `api.allBills()`. -- `useCategories()` → `api.categories()`. - -These use TanStack Query keys and cache server data for common pages. +- `useTracker(year, month)`, `useTrackerHealth(year, month)`, `usePrefetchTracker()` — warms the adjacent-month cache on hover/focus. +- `useBills()`, `useCategories()`, `useSnowball*`, `useSpending*`, `useSummary`, `useBankLedger`, `useCategoriesWithSpending`, `useSubscriptions`, `useSubscriptionCatalog`, `useSubscriptionRecommendations`, `useMatchSuggestions`, `useTransactions`, `useTransactionsCategory`, `useTransactionCategories`, `useBankDataSources`, `useMatchBillDialog`, `useBankAccountDataSources`, `useBankSyncConfig`, `useAdminUserList`, `useAdminOIDCTest`, `useAdminBackups`, `useAdminCleanup`, `useAdminMigrations`, `useAdminAuthMode`, `useAdminNotifications`, `useReleaseNotes`, `useLoginHistory`, `useSnowballPlans`, `useWebAuthnStatus`, `useTOTPStatus`, `useUpcomingBills`. +- All hooks typed (params + `QueryParams`-typed query builders) and exported from `api.ts`. +- React Query v5: caches use `gcTime` (not `cacheTime`, which the v5 rename silently ignored). All mutation hooks invalidate / set query data via `setQueryData(key, updater)`. +- Mutations: `useQuickPay` (shared across desktop + mobile tracker rows; `isPending` replaces manual busy flags; tracker/badge caches invalidate on settle), `useTogglePaid`, `useMarkPaid`, `useMatch`, `useUnmatch`, `useIgnore`, `useUnignore`, `useCategorizeTransaction`, `useBillCreate`, `useBillUpdate`, `useBillDelete`, `useBillReorder`, `useBudgetPut`, `useBudgetCopy`, `useCategoryRuleCreate`, `useCategoryRuleDelete`, `usePlanCreate`, `usePlanUpdate`, `usePlanPause`, `usePlanResume`, `usePlanComplete`, `usePlanAbandon`, `useSubscriptionLink`, `useSubscriptionDecline`, `useSubscriptionCreate`, `useSubscriptionUpdate`, `useImportCsv`, `useImportOfx`, `useBankSyncNow`, etc. +- `useAutoSave(value, options)` is a generic debounced-save hook with an `AutoSaveStatus` union (`idle` | `pending` | `saving` | `saved` | `error`). +- `usePaymentActions.ts` — shared quick-pay/toggle-paid mutation hooks; typed payloads with `Dollars` on money amounts. +- `useSearchPanelPreference.ts` — typed `[boolean, setter]` tuple. ### Pages -- `LoginPage.jsx` — local login plus OIDC login availability based on `/auth/mode`. -- `TrackerPage.jsx` — monthly tracker, payment interactions, upcoming bills, starting amount awareness, bulk/session logout action. -- `CalendarPage.jsx` — calendar grid backed by `/calendar`. -- `SummaryPage.jsx` — monthly plan, income, starting amounts, expenses, chart data. -- `BillsPage.jsx` — bill CRUD, categories, monthly state/history range controls. -- `CategoriesPage.jsx` — category list/create/update/delete and related bill info. -- `AnalyticsPage.jsx` — analytics summary filters and charts. -- `SettingsPage.jsx` — user/app settings and demo data seed. -- `DataPage.jsx` — export, spreadsheet import, user DB import, import history, CSV transaction import with preview and commit flow (`ImportTransactionCsvSection`). -- `ProfilePage.jsx` — display name, notification preferences, password change, export/import-history links. -- `AdminPage.jsx` — users, auth mode/OIDC, backups, cleanup, notifications, migrations, admin settings. -- `StatusPage.jsx` — admin system status. -- `AboutPage.jsx` — public or admin markdown/about view; admin mode uses `/about-admin`; markdown is sanitized. -- `ReleaseNotesPage.jsx` — release history display. +- `LoginPage.tsx` — local login, OIDC login button, TOTP challenge, WebAuthn assertion, recovery code entry. +- `TrackerPage.tsx` — monthly tracker, payment interactions, upcoming bills, starting-amount awareness, safe-to-spend, drift/autopay/amount-suggestion badges, bulk/session logout, month-nav prefetch. +- `CalendarPage.tsx` — calendar grid backed by `/calendar`. +- `SummaryPage.tsx` — monthly plan, income, starting amounts, expenses, chart, bank-tracking overlay, skip-override. +- `BillsPage.tsx` — bill CRUD, categories, monthly state/history range controls, soft-delete + recently-deleted restore, merchant rules manager, bill historical-import dialog. +- `CategoriesPage.tsx` — category list/create/update/delete, group/category split, sort order, spending flag. +- `AnalyticsPage.tsx` — analytics summary filters and charts. +- `SettingsPage.tsx` — user settings and demo data seed. +- `DataPage.tsx` — export, spreadsheet import, user DB import, import history, CSV transaction import with preview + commit flow (`ImportTransactionCsvSection`), OFX import, bank sync. +- `ProfilePage.tsx` — display name, notification preferences, password change, TOTP setup, WebAuthn enrollment, login history, export/import-history links. +- `AdminPage.tsx` — users, auth mode/OIDC, WebAuthn, TOTP, backups, cleanup, notifications, migrations, admin settings, bank-sync config. +- `StatusPage.tsx` — admin system status. +- `AboutPage.tsx` — public or admin markdown/about view; admin mode uses `/about-admin`; markdown is sanitized. +- `RoadmapPage.tsx` — admin roadmap. +- `ReleaseNotesPage.tsx` — release history display; the `ReleaseNotesDialog` is shown to the user on login when their `last_seen_version` is older than the current package version. +- `PrivacyPage.tsx` — public privacy text. +- `SubscriptionsPage.tsx` (1814 lines) — `useOptimistic` reducer, virtualizer's `FlatItem` discriminated union, drag-reorder typed via the shared `DragProps`/`MoveControls`. +- `SubscriptionCatalogPage.tsx` — admin view of the top-200 catalog with bank descriptors. +- `SnowballPage.tsx` — projection, settings, plans lifecycle. +- `PayoffPage.tsx` — payoff scenarios. +- `SpendingPage.tsx` — categorized transactions, rules, monthly budgets with month-rollover, income. +- `BankTransactionsPage.tsx` — the `Cents`-based bank ledger, with a local `Tx` structurally compatible with `BankTransaction` so it still satisfies `MatchBillDialog`. +- `HealthPage.tsx` — bill + payment health snapshot. +- `NotFoundPage.tsx` — 404 fallback. ### Components -- Layout: `Layout`, `Sidebar`, `BrandBlock`, `NavPill`. -- Domain UI: `AdminDashboard`, `BillModal`, `BillsTableInner`, `MobileBillRow`, `MobileTrackerRow`, `StatusBadge`, `SummaryCard`, `MarkdownText`, `ReleaseNotesDialog`. -- Reliability: `ErrorBoundary`, `PageLoader`. -- UI primitives: alert dialog, badge, button, card, checkbox, confirm dialog, dialog, dropdown menu, input dialog, input, label, select, separator, skeleton, switch, table, tabs, theme toggle, tooltip. +- **Layout**: `Layout`, `Sidebar`, `BrandBlock`, `NavPill`, `CommandPalette`, `PageTransition`. +- **Admin**: full `components/admin/` directory — users, auth methods, OIDC, WebAuthn, TOTP, bank-sync, backup manager, cleanup, notifications, migrations, settings cards. +- **Bill modal**: full `components/bill-modal/` directory plus `BillModal.tsx` (1092-line add/edit form, reached from 9 entry points). +- **Tracker**: full `components/tracker/` directory (18 files — the home page) where `fmt(row.expected_amount)` type-checks against `Dollars`. +- **Transactions**: `components/transactions/` directory. +- **Snowball**: `components/snowball/` directory (plan list, projection, settings). +- **Data**: full `components/data/` directory (14 files) — the Data page's SimpleFIN connect/sync workbench, transaction matching, and CSV / OFX / XLSX-spreadsheet / SQLite import flows, with `Cents`-branded bank-transaction amounts flowing through. +- **Domain UI**: `BillModal`, `BillsTableInner`, `MobileBillRow`, `MobileTrackerRow`, `StatusBadge`, `SummaryCard`, `MarkdownText`, `ReleaseNotesDialog`, `IncomeBreakdownModal`, `BillHistoricalImportDialog`, `BillRulesManager`, `BillMerchantRules`, `RecentlyDeletedBillsDialog`, `CalendarFeedManager`, `SearchFilterPanel`, `SubscriptionCatalogSection`. +- **Reliability**: `ErrorBoundary` (typed class component), `PageLoader`, `PageTransition`. +- **UI primitives** (`components/ui/`): `alert-dialog`, `badge`, `button`, `card`, `checkbox`, `collapsible`, `confirm-dialog`, `dialog`, `dropdown-menu`, `input`, `input-dialog`, `label`, `save-status`, `select`, `separator`, `skeleton`, `sonner`, `switch`, `table`, `tabs`, `theme-toggle`, `tooltip` — all `.tsx`. + +### `client/lib/` + +- `money.ts` — branded `Cents`/`Dollars` types, `asCents`/`asDollars`/`centsToDollars`/`dollarsToCents`, formatters. A `formatUSD(dollars)` and `formatCentsUSD(cents)` split mirrors the server's `formatUSD`/`formatCentsUSD` split. Inputs are coerced defensively so null / `''` / undefined / NaN never render as `$NaN`. +- `utils.ts` — `cn`, `fmt`, date/format helpers. `fmt` inherits `formatUSD`'s branded-dollars input. +- `trackerUtils.ts` — row/status/sort helpers, `TrackerRow` domain interface, `TrackerStatus` union. +- `billDrafts.ts` — `SourceBill` / `Template` / `Category` shapes for `makeBillDraft`. +- `billingSchedule.ts` — `Schedule` union and helpers. +- `cashflowUtils.ts` — typed SVG timeline geometry for the Safe-to-Spend card. +- `trackerTableColumns.ts` — typed column descriptors. +- `reorder.ts` — generic `moveInArray`. +- `version.ts` — typed release-notes shapes; the Vite-injected `__APP_VERSION__` is declared inline. + +### Frontend build & lint + +- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`). +- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`. Wired into `npm run ci`. +- `npm run lint` — `eslint client` (flat config; `react-hooks/rules-of-hooks`, `react-hooks/exhaustive-deps`, `react-refresh/only-export-components`, typescript-eslint parser for `.ts/.tsx`). +- `npm run test` — server `node --test tests/*.test.js`. +- `npm run test:client` — `vitest run` (client unit tests). +- `npm run test:all` — both. +- `npm run test:e2e` — Playwright (chromium desktop + mobile); `prepare-db.js` reseeds the test DB. +- `npm run check:server` — `node --check` on every server JS file. +- `npm run check` — `check:server` + `build`. +- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`. ### Frontend security notes -- CSRF header is sent on POST/PUT/PATCH/DELETE. -- Auth is cookie/session based; no tokens are stored in localStorage. -- Admin routes are client-guarded and server-guarded. -- Markdown rendering uses `rehype-sanitize`. +- CSRF header is sent on `POST`/`PUT`/`PATCH`/`DELETE`; the token is fetched once from `/api/auth/csrf-token` and cached in memory. +- Auth is cookie/session based; no tokens are stored in localStorage. Session token hashing (`v0.94`) means a leaked DB does not yield usable session IDs. +- Admin routes are client-guarded and server-guarded; the default admin is redirected away from user routes. +- Markdown rendering uses `rehype-sanitize`; CSP headers in `middleware/securityHeaders.js` cover the built assets. - Error boundaries prevent route crashes from taking down the whole SPA. +- PWA: `vite-plugin-pwa` registers a service worker (autoUpdate) with manifest + icons. Runtime caching uses `NetworkFirst` for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` with a 5-second network timeout, 30-entry max, 5-minute max-age. --- @@ -1546,34 +1410,54 @@ These use TanStack Query keys and cache server data for common pages. ### `package.json` -Version: `0.28.1`. +Version: `0.40.0`. `"type": "commonjs"`. Scripts: - `npm run dev:api` — `node --watch server.js`. - `npm run dev:ui` — Vite dev server. -- `npm run dev` — concurrently runs API and UI. -- `npm run build` — Vite production build. +- `npm run dev` — `concurrently` runs API and UI. +- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`). +- `npm run check:server` — `node --check` every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils`. +- `npm run lint` — `eslint client` (flat config; `react-hooks` + `react-refresh` + `typescript-eslint`). +- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`. +- `npm run check` — `check:server` + `build`. +- `npm run test` — `node --test tests/*.test.js` (server). +- `npm run test:client` — `vitest run` (client). +- `npm run test:all` — both. +- `npm run test:e2e` / `test:e2e:ui` / `test:e2e:update` / `test:e2e:probe` — Playwright (chromium desktop + mobile + axe), with `e2e/setup/prepare-db.js` reseeding the test DB. +- `npm run smoke:prod` — `bash scripts/prod-smoke.sh`. +- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`. Runs in Forgejo Actions on every push and PR (`.forgejo/workflows/ci.yml`, container `node:22-bookworm`). - `npm start` — `node server.js`. Key runtime dependencies: -- Express, cookie-parser, cors, express-rate-limit. -- better-sqlite3. +- Express 4, cookie-parser, cors, express-rate-limit. +- better-sqlite3 12. - bcryptjs. -- openid-client. -- nodemailer. -- node-cron. -- React, React DOM, React Router, TanStack Query. +- openid-client 5. +- @simplewebauthn/server 13, @simplewebauthn/browser 13. +- otplib 13, qrcode 1. +- nodemailer 8. +- node-cron 4. +- React 19, React DOM 19, React Router 6, TanStack Query 5, TanStack Virtual 3, framer-motion 12. - shadcn/ui component primitives, Radix UI primitives, lucide-react, Tailwind utilities, Sonner toasts. - xlsx for spreadsheet import/export. +Key dev dependencies: + +- typescript 6, @types/react 19, @types/react-dom 19, typescript-eslint 8. +- eslint 9, @eslint/js, eslint-plugin-react-hooks 5, eslint-plugin-react-refresh 0.4, globals 17. +- vite 5, @vitejs/plugin-react 4, vite-plugin-pwa 1, babel-plugin-react-compiler 1. +- vitest 4, @testing-library/react 16, @axe-core/playwright 4, @playwright/test 1. +- tailwindcss 3, autoprefixer 10, postcss 8. + ### Dockerfile Multi-stage build: -1. Builder: `node:18-alpine`, installs build deps `python3 make g++`, runs `npm install`, copies source, runs `npm run build`. -2. Runtime: `node:18-alpine`, installs `bash nano su-exec`, creates non-root `bill` user, copies built app, creates `/data/db`, `/data/backups`, `/app/backups`, sets ownership and restrictive permissions. +1. Builder: `node:22-alpine`, installs build deps `python3 make g++`, runs `npm install`, copies source, runs `npm run build`. +2. Runtime: `node:22-alpine`, installs `bash nano su-exec`, creates non-root `bill` user, copies built app, creates `/data/db`, `/data/backups`, `/app/backups`, sets ownership and restrictive permissions. Runtime environment: @@ -1593,10 +1477,44 @@ Service: `bill-tracker`. - Ports: host `3030` to container `3000`. - Volume: `/portainer/hosting/bill-tracker/data:/data`. - Restart: `unless-stopped`. -- Environment includes `INIT_ADMIN_USER`, `INIT_ADMIN_PASS`, and CSRF cookie settings. +- Environment includes `INIT_ADMIN_USER`, `INIT_ADMIN_PASS`, and CSRF cookie settings (`CSRF_HTTP_ONLY: "false"`, `CSRF_SAME_SITE: "strict"`, `CSRF_SECURE: "true"`, `CSRF_COOKIE_NAME: "bt_csrf_token"`). Important deployment note: the compose file currently sets `CSRF_SECURE: "true"`; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to `false` only in local/dev. +### CI (`.forgejo/workflows/ci.yml`) + +Forgejo Actions runs on every push and PR: `actions/checkout@v4` on `node:22-bookworm` (matches Dockerfile runtime so better-sqlite3 prebuilds resolve identically), then `npm ci` and `npm run ci`. The workflow is the only place a build is fully re-verified end-to-end (lint + typecheck + server tests + client tests + build). + +### Vite (`vite.config.mjs`) + +- `@vitejs/plugin-react` with `babel-plugin-react-compiler` enabled (React 19 auto-memoization). The codebase is rules-of-hooks clean (enforced by `eslint-plugin-react-hooks`). +- `vite-plugin-pwa` with `registerType: 'autoUpdate'`, manifest, icons in `client/public/img/`, runtime caching for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` via `NetworkFirst` (5s network timeout, 30-entry max, 5-min max-age). +- `__APP_VERSION__` is injected at build time from `package.json`; the SPA reads it through `client/lib/version.ts`. +- Manual chunk splitting: `vendor-react` (react / react-dom / react-router-dom), `vendor-radix` (dialog / select / dropdown-menu / tabs / tooltip / alert-dialog), `vendor-motion` (framer-motion), `vendor-query` (@tanstack/react-query + @tanstack/react-virtual). +- `resolve.alias['@']` → `client/`, matches the Vite alias and the `tsconfig.json` `paths` mapping. +- `server.port: 5173`, proxies `/api` to `http://localhost:${API_PORT || PORT || 3000}`. +- `test.environment: 'node'`; tests opt into jsdom via `@vitest-environment` per-file. +- `test.include: ['client/**/*.test.{js,jsx}']`; **note**: client tests on `.ts` are currently run via the `tests/*.test.js` server suite (which still exercises the same behaviors). The Vitest `include` is restricted to `.js/.jsx` to keep current test coverage green during the gradual TS migration. + +### TypeScript (`tsconfig.json`) + +- `target: ES2022`, `lib: [ES2023, DOM, DOM.Iterable]`. +- `module: ESNext`, `moduleResolution: bundler`, `jsx: react-jsx`. +- `paths: { "@/*": ["./client/*"] }`, `resolveJsonModule`, `esModuleInterop`, `isolatedModules`, `skipLibCheck`, `noEmit`. +- `allowJs: true`, `checkJs: false` — legacy `.js`/`.jsx` keep resolving and running untouched; only `.ts`/`.tsx` are type-checked. Convert file-by-file. +- `strict: true`, `noUncheckedIndexedAccess: true`. +- `types: ["vite/client"]` — provides `import.meta.env` and `__APP_VERSION__` typings. +- `include: ["client/**/*"]`, `exclude: ["node_modules", "dist", "client/**/*.test.*"]`. + +### ESLint (`eslint.config.mjs`) + +Flat config scoped to the React client. The point is enforcement of the two rules that catch real React bugs — `rules-of-hooks` (conditional hooks) and `exhaustive-deps` (stale closures) — which nothing previously checked. Server code (CommonJS Node) is out of scope here; `npm run check:server` covers it. + +- JS/JSX block (`client/**/*.{js,jsx}`): `js.configs.recommended` + `react-hooks` + `react-refresh`. `react-hooks/rules-of-hooks: error`, `react-hooks/exhaustive-deps: warn`, `react-refresh/only-export-components: warn` (with `allowConstantExport: true`). +- TS/TSX block (`client/**/*.{ts,tsx}`): same react rules via the `typescript-eslint` parser. `no-undef` and `no-unused-vars` off (TypeScript handles them); `@typescript-eslint/no-unused-vars: warn` with `^[A-Z_]` ignore pattern. +- `no-empty: warn` with `allowEmptyCatch: true`. +- Ignores: `dist/**`, `node_modules/**`, `coverage/**`, `client/**/*.test.*`. + --- ## 9. Auth and Security Flows @@ -1605,16 +1523,29 @@ Important deployment note: the compose file currently sets `CSRF_SECURE: "true"` 1. Client calls `GET /auth/mode` to determine local/OIDC visibility. 2. Client submits `POST /auth/login`. -3. Server checks `local_login_enabled`, validates credentials through bcrypt, rejects inactive users, cleans expired sessions, creates a 7-day session. +3. Server checks `local_login_enabled`, validates credentials through bcrypt (cost 12), rejects inactive users, cleans expired sessions, creates a 7-day session, **hashes the session token** for storage (`v0.94`), and sets the `bt_session` cookie with the raw opaque ID. 4. Server sets `bt_session` cookie using `cookieOpts(req)`. 5. Client calls `GET /auth/me` to populate auth state. +### TOTP 2FA + +1. User enables TOTP via `GET /auth/totp/setup` → scans QR code → `POST /auth/totp/enable` with `{secret, token}`. Server encrypts the secret with `services/encryptionService` and stores 8 recovery codes (hashed). +2. On subsequent logins, after a successful `POST /auth/login`, the server returns `{ totp_required: true, challenge_id }`. The SPA prompts for the 6-digit code and submits it (or a recovery code) with the challenge id. +3. `POST /auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id`; the SPA submits it with the token on the next request. +4. `disableTotp(userId)` clears the encrypted secret and recovery codes. + +### WebAuthn 2FA + +1. User enrolls a security key: `GET /auth/webauthn/status` (or the admin equivalent) returns current credentials. The SPA calls an internal helper that triggers a registration ceremony; the server stores the credential in `webauthn_credentials` (AAGUID, transports, backup flags, friendly name). +2. On login, the SPA requests an assertion via `webauthn_login` flow; `createAuthenticationChallenge` returns options + a 15-minute `challenge_id`; `verifyAuthentication` updates the sign count and audits `webauthn.login`. +3. `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`) must match the browser-visible hostname, or registration/assertion will fail. + ### OIDC login 1. Client navigates to `/api/auth/oidc/login`. -2. Server verifies active OIDC config, creates PKCE state, redirects to provider. +2. Server verifies active OIDC config (the OIDC client secret is decrypted from `settings.oidc_client_secret`, encrypted at rest by `v0.79`), creates PKCE state, redirects to provider. 3. Provider returns to `/api/auth/oidc/callback`. -4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie, redirects back into SPA. +4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie (token hashed in `sessions.id`), redirects back into SPA. ### Password change @@ -1623,6 +1554,14 @@ Important deployment note: the compose file currently sets `CSRF_SECURE: "true"` 3. Server updates hash, clears `must_change_password`, sets `last_password_change_at`. 4. Other sessions are invalidated and current session is rotated when possible. +### Login history + device fingerprint + +- Every login (successful or failed) writes a `user_login_history` row. +- `ip_address` and `user_agent` are encrypted at rest (`v0.84`). +- `location` (IP-geolocation city/region/country) is captured only when the user has `users.geolocation_enabled = 1` (`v1.02`). +- `success` (`v0.85`) tracks failed-attempt history separately. +- `session_fingerprint` lets the SPA detect a different device for the same session and trigger a re-auth challenge. + ### Authorization - All user data routes enforce owner scope by `req.user.id` in SQL. @@ -1631,16 +1570,32 @@ Important deployment note: the compose file currently sets `CSRF_SECURE: "true"` - Role changes invalidate target sessions. - Deactivation invalidates target sessions. +### Secret encryption at rest + +- `services/encryptionService.js` provides `encryptSecret` / `decryptSecret` / `assertEncryptionReady` using a master key derived with HKDF (`v0.78`; the prior SHA-256 derivation was migrated in-place). +- Encrypted at rest: `oidc_client_secret` (`v0.79`), SMTP password (`v0.77`), `data_sources.encrypted_secret` (SimpleFIN access URLs), `user_login_history.ip_address` / `user_agent` (`v0.84`), `users.totp_secret` (`v0.86`). +- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected. + ### Backup safety - Managed filename regex and path checks prevent traversal. -- Uploads are written to temp paths first, validated, then moved. +- Uploads are written to temp paths first, validated by SQLite `integrity_check` + optional SHA-256 checksum, then moved. - Restore creates a pre-restore backup. ### Import safety - Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions. - User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership. +- CSV import preview/apply is split; ambiguous rows are surfaced in `errors` for the user to resolve. +- OFX/QFX import is rate-limited via the import limiter; previews have a 24-hour TTL. + +### Bank sync safety + +- SimpleFIN access URLs are encrypted at rest (`data_sources.encrypted_secret`). +- `assertEncryptionReady` is called before any secret operation. +- The `monitored` flag (`v0.64`) lets users exclude a credit-card account from bill matching. +- Transactions are deduplicated by `provider_transaction_id` (unique partial index). +- Account balance updates run in transactions; partial failures roll back. --- @@ -1651,11 +1606,11 @@ Important deployment note: the compose file currently sets `CSRF_SECURE: "true"` - DB path is `DB_PATH` or `db/bills.db`. - DB open logging prints only `path.basename(DB_PATH)` instead of the full database path, so startup logs identify the file without exposing the full filesystem location. - SQLite WAL and foreign keys are enabled. -- Schema and migrations run automatically. -- Default categories/settings are seeded. -- Expired sessions are purged at startup. -- A periodic expired-session cleanup interval is scheduled. -- Backup scheduler and daily worker are started where server code imports/starts them. +- Schema and migrations run automatically (`db/migrations/versionedMigrations.js` + `db/migrations/legacyReconcileMigrations.js`). +- Default categories/settings are seeded (incl. spending defaults for new users in `v0.87`). +- Expired sessions are purged at startup; a periodic `SESSION_CLEANUP_INTERVAL_MS` cleanup interval is scheduled. +- `bankSyncWorker.start()` is started; `backupScheduler` is started; `workers/dailyWorker.start()` runs the daily jobs. +- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked. Environment-seeded regular users use `INIT_REGULAR_USER` and `INIT_REGULAR_PASS`. New seeded users are inserted with `first_login = 0` and `must_change_password = 0`. Existing seeded regular users have their password hash updated and both flags reset to `0`, then the server audits `seed.flag_reset` with the username, reset flags, and `source: "server-seed"`. This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes. @@ -1663,7 +1618,8 @@ Environment-seeded regular users use `INIT_REGULAR_USER` and `INIT_REGULAR_PASS` Common variables used by current code: -- `PORT` +- `PORT` — server port (default 3000) +- `BIND_HOST` — server bind (default `0.0.0.0`) - `DB_PATH` - `BACKUP_PATH` - `INIT_ADMIN_USER` @@ -1671,7 +1627,8 @@ Common variables used by current code: - `INIT_REGULAR_USER` - `INIT_REGULAR_PASS` - `SESSION_CLEANUP_INTERVAL_MS` -- `CORS_ORIGIN` +- `TRUST_PROXY` — `true` / `1` / numeric / `loopback` / CIDR; required behind nginx/Traefik +- `CORS_ORIGIN` — comma-separated allowlist - `COOKIE_SECURE` - `HTTPS` - `CSRF_HTTP_ONLY` @@ -1680,9 +1637,14 @@ Common variables used by current code: - `CSRF_COOKIE_NAME` - `OIDC_ISSUER_URL` - `OIDC_CLIENT_ID` -- `OIDC_CLIENT_SECRET` +- `OIDC_CLIENT_SECRET` — encrypted at rest (`v0.79`); the settings-table copy is the source of truth - `OIDC_REDIRECT_URI` - `OIDC_TOKEN_AUTH_METHOD` +- `WEBAUTHN_RP_ID` — default `localhost` +- `WEBAUTHN_ORIGIN` — default `http://localhost:${PORT}` +- `ENCRYPTION_KEY` — master key for at-rest encryption (HKDF, `v0.78`) +- `API_PORT` — Vite dev-server proxy target (defaults to `PORT || 3000`) +- `DATA_IMPORT_ENABLED` — gates CSV import (defaults to true) Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the `settings` table and managed from Admin UI. @@ -1691,10 +1653,13 @@ Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored - Use transactions for multi-step destructive or bulk DB changes. - Keep user-owned SQL scoped by `req.user.id`. - Keep admin lockout protection before changing login methods. -- Do not expose `password_hash`, session IDs, OIDC client secret, or internal backup paths in API responses. -- Keep import preview/apply separated so users can resolve ambiguous spreadsheet data before DB writes. -- Prefer soft delete for payments; bill deletion is intentionally hard delete and returns an explicit warning. -- DB path support: db/database.js uses path.basename(DB_PATH) in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported. +- Do not expose `password_hash`, session IDs (raw or hashed), OIDC client secret, encrypted secret blobs, or internal backup paths in API responses. +- Keep import preview/apply separated so users can resolve ambiguous spreadsheet / CSV / OFX / user-DB data before DB writes. +- Branded `Dollars` / `Cents` types in `client/lib/money.ts` are the source of truth for client money handling. A bare number at a `fmt()` / `formatUSD` / `formatCentsUSD` boundary is a compile error in typed code. The `money.type-test.ts` guard uses `@ts-expect-error` so a regression in the branding fails `tsc --noEmit`. +- Tailwind's `content` glob must include `ts,tsx` (`./client/**/*.{js,jsx,ts,tsx}`). Without it, classes used only in `.tsx` files generate no CSS and produce a runtime-only failure that the e2e probe (not typecheck/build/tests) is the only gate that catches. +- DB path support: `db/database.js` uses `path.basename(DB_PATH)` in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported. +- React Compiler (`babel-plugin-react-compiler`) requires a rules-of-hooks-clean codebase; do not introduce conditional hooks. +- React Query v5: use `gcTime`, not `cacheTime` (silently ignored in v5). --- @@ -1702,16 +1667,25 @@ Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored Reviewed current code sources: -- `server.js` -- all route files under `routes/` -- service files under `services/` -- middleware files under `middleware/` -- `db/schema.sql` and `db/database.js` -- actual initialized SQLite schema via `better-sqlite3` introspection -- `client/App.jsx`, `client/api.js`, `client/hooks/useAuth.jsx`, `client/hooks/useQueries.js` -- page/component inventory under `client/pages/` and `client/components/` -- `package.json` -- `Dockerfile` -- `docker-compose.yml` +- `server.js` (including `TRUST_PROXY` handling, health probe, route mount order, post-`listen` worker startup). +- All 26 route files under `routes/`. +- All 41 service files under `services/` (incl. `bankSync*`, `simplefinService`, `ofxImportService`, `merchantStoreMatchService`, `advisoryFilterService`, `totpService`, `webauthnService`, `loginFingerprint`, `driftService`, `aprService`, `paymentAccountingService`, `encryptionService`, `calendarFeedService`). +- All middleware files (`csrf`, `errorFormatter`, `rateLimiter`, `requireAuth`, `securityHeaders`). +- `db/schema.sql`, `db/database.js`, `db/migrations/versionedMigrations.js`, `db/migrations/legacyReconcileMigrations.js`, `db/subscriptionCatalogSeed.js`. +- Actual initialized SQLite schema via `better-sqlite3` introspection (`PRAGMA table_info`, `PRAGMA index_list`). +- `client/main.tsx`, `client/App.tsx`, `client/api.ts`, `client/types.ts`, `client/contexts/ThemeContext.tsx`. +- `client/hooks/useAuth.tsx`, `client/hooks/useQueries.ts`, `client/hooks/usePaymentActions.ts`, `client/hooks/useAutoSave.ts`, `client/hooks/useSearchPanelPreference.ts`. +- Page inventory under `client/pages/` (24 files) and component inventory under `client/components/` (incl. `admin/`, `bill-modal/`, `tracker/`, `transactions/`, `snowball/`, `data/`, `layout/`, `ui/`). +- `client/lib/` (`money.ts`, `utils.ts`, `trackerUtils.ts`, `billDrafts.ts`, `billingSchedule.ts`, `cashflowUtils.ts`, `trackerTableColumns.ts`, `reorder.ts`, `version.ts`). +- `package.json` (version 0.40.0; `"type": "commonjs"`), `tsconfig.json`, `eslint.config.mjs`, `vite.config.mjs`. +- `Dockerfile` (node:22-alpine), `docker-compose.yml`, `docker-entrypoint.sh`, `.forgejo/workflows/ci.yml`. -The previous manual contained large historical update sections and stale route/page descriptions. This version replaces those with a current-state engineering reference. +End-to-end checks run against the codebase: + +- `find client -name '*.jsx'` returns 0 — the JSX → TSX migration is complete. +- `find client -name '*.ts'` and `find client -name '*.tsx'` together cover every previously `.jsx` file in `components/`, `pages/`, `hooks/`, `contexts/`, plus the API client and `lib/`. +- `grep` on `package.json` for React 19, TypeScript 6, Vite 5, TanStack Query 5, `@simplewebauthn/*` 13, `otplib` 13. +- `npm run typecheck` was green at the time of the v0.41.0 entry in `HISTORY.md` (0 errors); 48 client unit tests pass; 17/17 e2e probe (every page renders, all API paths respond). +- `node --check` on every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils` (`npm run check:server`). + +The previous manual (v0.28.1) contained stale route/page descriptions and missed major features added between v0.28 and v0.40 (subscriptions, spending, snowball plans, SimpleFIN, WebAuthn, TOTP, push notifications, category groups, calendar feed, money-cents migration in progress, React 18 → 19 + TypeScript migration, branded money types). This version replaces that with a current-state engineering reference.