1692 lines
105 KiB
Markdown
1692 lines
105 KiB
Markdown
# Engineering Reference Manual — Bill Tracker
|
||
|
||
**Status:** Current code reference
|
||
**Last Updated:** 2026-07-05
|
||
**Version:** 0.40.0
|
||
**Primary stack:** Node.js 22 + Express 4 (CommonJS), React 19 + Vite 5 (TypeScript, `strict` + `noUncheckedIndexedAccess`), Tailwind CSS 3 + shadcn/ui, TanStack Query 5, Sonner, SQLite via `better-sqlite3` 12, `@simplewebauthn/*` 13, `otplib` 13, `openid-client` 5
|
||
|
||
This manual reflects the current application code in `server.js`, `routes/`, `services/`, `middleware/`, `db/`, `client/`, `package.json`, `Dockerfile`, `docker-compose.yml`, `tsconfig.json`, `eslint.config.mjs`, and `vite.config.mjs`. It is written as a current-state reference, not a changelog. The previous version of this manual (v0.28.1) is fully superseded.
|
||
|
||
Notable changes since the previous manual:
|
||
|
||
- **Client migrated JSX → TSX.** Full strict TypeScript with `noUncheckedIndexedAccess`. `allowJs` + `checkJs:false` keep legacy `.js` running while `.ts/.tsx` are type-checked. `npm run typecheck` wired into CI.
|
||
- **Branded money types.** `client/lib/money.ts` exposes `Cents` / `Dollars` branded types — a typed caller can no longer format cents as dollars (the 100× bug) by accident.
|
||
- **React 18 → React 19** with `babel-plugin-react-compiler` for automatic memoization. Vite 5 with `vite-plugin-pwa` for service-worker / installable PWA.
|
||
- **Node 18 → Node 22** in Dockerfile; `node:22-bookworm` in CI.
|
||
- **New features:** SimpleFIN bank sync, WebAuthn + TOTP 2FA, calendar feed (`/api/calendar/feed.ics`), subscription catalog + recommendations, spending categories / budgets, snowball plans (lifecycle), push notifications (ntfy / Gotify / Discord / Telegram), push notification column on users, encrypted secrets at rest with HKDF-derived key, money-cents migration (in progress), category groups, banker-rule auto-attribute.
|
||
- **Schema:** migrations `v0.2` → `v1.06`. Rollback map covers `v0.44` → `v1.04`.
|
||
|
||
---
|
||
|
||
## 1. System Overview
|
||
|
||
Bill Tracker is a self-hosted bill management application. It supports:
|
||
|
||
- Local username/password authentication, optional single-user mode, optional Authentik/OIDC login, and optional WebAuthn / FIDO2 + TOTP two-factor authentication.
|
||
- User-scoped bills, categories (with `category_groups` and `spending_enabled` flag), payments, monthly bill overrides, monthly income, and starting cash buckets.
|
||
- Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, migration rollback, and SimpleFIN bank-sync admin configuration.
|
||
- Spreadsheet, user-SQLite, OFX, and CSV transaction import workflows with preview/apply split.
|
||
- CSV, Excel, and user-SQLite export workflows; an iCalendar feed (`/api/calendar/feed.ics`) of upcoming bills gated by per-user tokens.
|
||
- SMTP-based bill due notifications plus ntfy / Gotify / Discord / Telegram push notifications.
|
||
- SimpleFIN-based bank sync (data sources, financial accounts, transactions, match suggestions, auto-categorize, merchant rules, merchant-store matches).
|
||
- Subscription catalog (top-200 known services), per-user recommendations, declined-subscription hints, recommendation feedback.
|
||
- Spending module: categorized transactions, rules, monthly budgets, income breakdown, copy-month workflow.
|
||
- Snowball plans: lifecycle (active / paused / completed / abandoned), debt-order projection (snowball/avalanche/minimum-only), payoff scenarios.
|
||
- React 19 SPA in TypeScript with branded `Cents` / `Dollars` money types, lazy-loaded routes, global QueryCache error toasts, and CSRF-protected JSON APIs.
|
||
|
||
Runtime flow:
|
||
|
||
1. `server.js` initializes SQLite through `db/database.js`, runs `db/migrations/versionedMigrations.js` plus `legacyReconcileMigrations.js`, seeds defaults/admin user, cleans expired sessions, then starts Express.
|
||
2. Express applies `securityHeaders`, optional CORS, `express.json({ limit: '100kb' })`, `cookieParser`, `csrfTokenProvider`, the patched `res.json` error formatter, and an unauthenticated `GET /api/health` probe.
|
||
3. Route files under `routes/` validate input, enforce ownership through `req.user.id`, and use service modules for business logic. WebAuthn and TOTP challenges use a 5–15 minute TTL in `webauthn_challenges` and `totp_challenges`.
|
||
4. React 19 under `client/` calls `/api/*` through `client/api.ts` with `credentials: include` and CSRF headers on mutating methods; the CSRF token is fetched once from `/api/auth/csrf-token` and cached in memory.
|
||
|
||
---
|
||
|
||
## 2. Project Layout
|
||
|
||
- `server.js` — Express entry point and route mounting.
|
||
- `routes/` — HTTP API handlers:
|
||
- `auth.js` — login, logout, logout-all, password change, TOTP setup/enable/disable/challenge/status, WebAuthn status, login history, CSRF token endpoint, mode, acknowledge-version/privacy.
|
||
- `authOidc.js` — OIDC login and callback.
|
||
- `bills.js` — bills CRUD, auto-mark paid, history ranges, soft-delete, recently-deleted restore.
|
||
- `payments.js` — payments CRUD, status matching, snowball handling, bank-override accounting, bulk/quick.
|
||
- `categories.js` — category CRUD, sort order, spending/bill split, category groups.
|
||
- `tracker.js` — monthly tracker data, bucket resolution, cycle handling, drift, autopay suggestions, amount suggestions, safe-to-spend, three-month averages.
|
||
- `summary.js` — summary stats, income, starting amounts, bank-tracking overlay, skip override.
|
||
- `analytics.js` — expense reports, category breakdown, month-window filtering.
|
||
- `settings.js` — user settings, admin config, notification settings.
|
||
- `notifications.js` — notification management (admin SMTP + push).
|
||
- `profile.js` — user profile, notification settings, change-password, export URLs, import history, login history.
|
||
- `import.js` — CSV / Excel / user-SQLite / OFX import workflows.
|
||
- `export.js` — CSV, Excel, SQLite export.
|
||
- `status.js` — system status, health, runtime.
|
||
- `dataSources.js` — data sources CRUD with sync status (manual / file_import / provider_sync).
|
||
- `transactions.js` — transaction CRUD, match/ignore/commit actions, manual entry, pending flag, geolocation.
|
||
- `matches.js` — match suggestions, rejection tracking.
|
||
- `snowball.js` — snowball projection, settings, debt order, **plan lifecycle** (create / list / active / pause / resume / complete / abandon).
|
||
- `subscriptions.js` — user subscriptions, **recommendations** (list / decline / match-bill / create), transaction matches, catalog links, custom descriptors.
|
||
- `spending.js` — categorized spending summary, transactions, budgets (PUT / copy / month-rollover), category rules, income.
|
||
- `calendarFeed.js` — public iCalendar feed (`GET /feed.ics`) per-user token-gated.
|
||
- `admin.js` — users (incl. username change), backups, cleanup, auth-mode, **bank-sync-config**, migrations rollback.
|
||
- `about.js`, `aboutAdmin.js` — public + admin markdown about view.
|
||
- `privacy.js` — public privacy page.
|
||
- `version.js` — public version, history, update-status.
|
||
- `services/` — business logic modules. See §4.
|
||
- `middleware/` — auth guards, CSRF, rate limits, security headers, error formatting.
|
||
- `db/schema.sql` — base SQLite schema.
|
||
- `db/database.js` — DB connection, migrations runner, defaults, settings, rollback support.
|
||
- `db/migrations/versionedMigrations.js` — ordered, dependency-aware migrations `v0.2` → `v1.06`.
|
||
- `db/migrations/legacyReconcileMigrations.js` — column-level reconciliation for unversioned columns that pre-date the migration framework.
|
||
- `db/subscriptionCatalogSeed.js` — top-200 US subscriptions dataset used by `v0.65` / `v0.69` / `v0.95`.
|
||
- `workers/dailyWorker.js` — scheduled notification/cleanup worker entry.
|
||
- `client/` — React 19 SPA in TypeScript (see §7).
|
||
- `dist/` — generated Vite build output served by Express.
|
||
- `Dockerfile`, `docker-compose.yml`, `docker-entrypoint.sh` — container deployment.
|
||
- `tsconfig.json` — TypeScript config (strict + `noUncheckedIndexedAccess`, `allowJs`).
|
||
- `eslint.config.mjs` — flat ESLint config, `react-hooks` + `react-refresh` + `typescript-eslint`.
|
||
- `vite.config.mjs` — Vite + React Compiler + PWA + Vitest config.
|
||
|
||
---
|
||
|
||
## 3. Backend Entry Point and Middleware
|
||
|
||
### `server.js`
|
||
|
||
Server defaults:
|
||
|
||
- Port: `PORT || 3000`.
|
||
- Bind host: `BIND_HOST || '0.0.0.0'`.
|
||
- Static frontend: `dist/`.
|
||
- Optional CORS: enabled when `CORS_ORIGIN` is set; comma-separated allowlist with `credentials: true`.
|
||
- `TRUST_PROXY` (env): when set to `true`/`yes`/`on` trusts 1 hop; numeric → trust that many hops; otherwise the raw value (e.g. `loopback`, CIDR). Unset/false = no proxy trust. Required behind nginx/Traefik so `req.ip` and `req.secure` reflect the real client.
|
||
- Session cleanup: runs on startup and every `SESSION_CLEANUP_INTERVAL_MS || 86400000` ms.
|
||
- Admin seed: `INIT_ADMIN_USER || admin`; `INIT_ADMIN_PASS` or generated/default behavior in `db/database.js`.
|
||
- Environment variables `INIT_ADMIN_USER` and `INIT_ADMIN_PASS` (or `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`) skip the first-login flow entirely by pre-seeding users with `first_login=0` flags via `setup/firstRun.js`.
|
||
- Optional regular-user seed: `INIT_REGULAR_USER` + `INIT_REGULAR_PASS`; password must be at least 8 chars.
|
||
- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked.
|
||
- After `app.listen`, `cleanupExpiredSessions` is scheduled, `bankSyncWorker.start()` runs, the backup scheduler is started, and `workers/dailyWorker.start()` schedules the daily jobs.
|
||
|
||
Global middleware order:
|
||
|
||
1. `securityHeaders`
|
||
2. optional `cors` (only if `CORS_ORIGIN`)
|
||
3. `express.json({ limit: '100kb' })` — import routes override this per-endpoint
|
||
4. `cookieParser()`
|
||
5. `csrfTokenProvider`
|
||
6. `errorFormatter` (patches `res.json`)
|
||
7. `GET /api/health` — unauthenticated liveness probe
|
||
8. mounted API routers with route-level rate-limit/auth/CSRF middleware
|
||
9. `GET /login.html` → 302 `/login`, `express.static(DIST)`
|
||
10. SPA fallback `GET *` serving `dist/index.html` after ensuring a CSRF token cookie
|
||
11. final JSON error handler for malformed JSON / body-size / runtime errors
|
||
|
||
### Authentication middleware
|
||
|
||
`middleware/requireAuth.js` exports:
|
||
|
||
- `requireAuth`: attaches `req.user` from `bt_session`; in single-user mode attaches the configured active regular user without a session.
|
||
- `requireUser`: permits `user` and `admin` roles but blocks the default admin account from tracker access.
|
||
- `requireAdmin`: requires `req.user.role === 'admin'`.
|
||
|
||
### CSRF middleware
|
||
|
||
`middleware/csrf.js`:
|
||
|
||
- Cookie name: `CSRF_COOKIE_NAME || bt_csrf_token`.
|
||
- Header name: `x-csrf-token`.
|
||
- Defaults: `CSRF_HTTP_ONLY === true` only when explicitly configured, `CSRF_SAME_SITE || strict`, `CSRF_SECURE !== false`.
|
||
- The SPA expects `CSRF_HTTP_ONLY=false` so `client/api.ts` can read the token cookie and send `x-csrf-token`; do not enable httpOnly CSRF cookies unless token delivery changes.
|
||
- The SPA also calls `GET /api/auth/csrf-token` (handled by `routes/auth.js`) on first request and caches the token in memory; the cookie is the source of truth and the body mirrors it.
|
||
- `csrfTokenProvider` sets a token cookie on responses.
|
||
- `csrfMiddleware` validates mutating requests unless `req.csrfSkip` is set. Token may come from header, query, or body and must match cookie.
|
||
- Failures return 403 and audit `csrf.failure`.
|
||
|
||
### Rate limits
|
||
|
||
Defined in `middleware/rateLimiter.js`:
|
||
|
||
- Login: 10 / 15 min.
|
||
- Login-by-username (extra layer): separate limiter, applied alongside `loginLimiter`.
|
||
- Password changes: 5 / 15 min.
|
||
- Import: 20 / 15 min.
|
||
- Export: 30 / 15 min.
|
||
- Admin actions: 30 / 15 min.
|
||
- OIDC: 20 / 15 min.
|
||
- Backup operations: 5 / 60 min.
|
||
- Demo-data clear: 3 / 15 min.
|
||
|
||
A `skipRateLimitIfNoUsers(limiter)` wrapper is applied to `/api/auth/login` so a brand-new install with zero users is not locked out by the login limiter.
|
||
|
||
Rate-limit responses are JSON: `{ "error": "..." }`.
|
||
|
||
### Security headers
|
||
|
||
`middleware/securityHeaders.js` sets CSP with a per-request nonce plus common hardening headers. Static SPA scripts/styles must comply with the generated CSP.
|
||
|
||
### Error formatting
|
||
|
||
`middleware/errorFormatter.js` standardizes route responses into JSON with `error`, `code`, and optional `field`. Common statuses: 400 validation, 401 auth, 403 forbidden, 404 not found, 409 conflict, 429 rate limit, 500 server error.
|
||
|
||
---
|
||
|
||
## 4. Services
|
||
|
||
### `services/authService.js`
|
||
|
||
- Cookie: `bt_session`.
|
||
- Session lifetime: 7 days. Session tokens are **hashed at rest** in `sessions` (column populated in `v0.94`); the cookie value is a random opaque ID and is looked up by hash.
|
||
- Password hashing: bcrypt, cost 12.
|
||
- Functions:
|
||
- `login(username, password)` — verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updates `last_login_at`, and returns `{sessionId, user}` or `null`.
|
||
- `createSession(userId)` — creates a session for an OIDC-provisioned or existing local user after validating that the user is active.
|
||
- `logout(sessionId)` — deletes one session.
|
||
- `getSessionUser(sessionId)` — returns public user fields when the session exists, is unexpired, and belongs to an active user.
|
||
- `rotateSessionId(oldSessionId, userId)` — validates the current session, deletes it, and inserts a replacement session in a transaction. Password changes use this to prevent session fixation.
|
||
- `invalidateOtherSessions(userId, keepSessionId)` — deletes all sessions for a user except the supplied current session; when `keepSessionId` is `null`, deletes every session for that user.
|
||
- `pruneExpiredSessions()` — deletes expired sessions.
|
||
- `publicUser(user)` — strips password/session secrets and normalizes booleans.
|
||
|
||
Password changes through `/api/auth/change-password` update the password hash, clear `must_change_password`, stamp `last_password_change_at`, invalidate all other sessions, rotate the current session ID when a valid session cookie is present, set a replacement `bt_session` cookie, and audit `password.change`.
|
||
|
||
`/api/auth/logout-all` calls `invalidateOtherSessions(req.user.id, null)`, deletes the current cookie session if present, audits `logout.all`, clears `bt_session`, and returns `{success:true}`.
|
||
|
||
### `services/totpService.js`
|
||
|
||
TOTP / RFC-6238 authenticator-app 2FA:
|
||
|
||
- `newSecret()` — `otplib` 20-char base32 secret.
|
||
- `generateQrCode(secret, username)` — returns `{uri, qr_data_url}` for the SPA to display.
|
||
- `verifyToken(encryptedSecret, token)` — decrypts the per-user secret, strips whitespace, calls `verifySync`.
|
||
- Setup flow: `getTotpSetup` → user scans QR → `enableTotp(plainSecret, token)` validates a sample token, encrypts and stores the secret, returns 8 recovery codes.
|
||
- `disableTotp(userId)` — clears `totp_enabled`, `totp_secret`, recovery codes.
|
||
- Recovery codes are stored hashed in `users.totp_recovery_codes` (JSON).
|
||
- `totpStatus(userId)` returns `{enabled, hasSecret, recoveryCodesRemaining}`.
|
||
- Login flow: `POST /api/auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id` stored in `totp_challenges`; the SPA submits it with the 6-digit code on the next request.
|
||
|
||
### `services/webauthnService.js`
|
||
|
||
WebAuthn / FIDO2 security-key 2FA via `@simplewebauthn/server`:
|
||
|
||
- `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`).
|
||
- `createRegistrationChallenge(db, userId, username)` — generates a 32-byte `webauthn_user_id` (base64url), excludes already-registered credentials, stores the challenge in `webauthn_challenges` with 15-minute TTL, returns `{options, challengeId}`.
|
||
- `verifyRegistration(db, userId, challengeId, response, credentialName)` — verifies the registration, stores the credential (`credential_id`, public key, sign count, transports, backup flags, AAGUID, friendly name) and returns the credential id.
|
||
- `createAuthenticationChallenge(db, userId)` — for login: returns assertion options + challengeId.
|
||
- `verifyAuthentication(db, userId, challengeId, response)` — verifies, updates sign count, audits `webauthn.login`.
|
||
- Supports `attestationType: 'none'`, `residentKey: 'preferred'`, `userVerification: 'preferred'`, algorithms `-7` (ES256) and `-257` (RS256).
|
||
- `WEBAUTHN_RP_ID` must match the browser-visible hostname, or registration/assertion will fail.
|
||
|
||
### `services/oidcService.js`
|
||
|
||
Handles Authentik/OIDC login with `openid-client`:
|
||
|
||
- Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag.
|
||
- The OIDC client secret is **encrypted at rest** (migration `v0.79`) using `services/encryptionService` (HKDF-derived key from `v0.78`).
|
||
- Builds PKCE login state in `oidc_states`.
|
||
- Discovers provider and caches OIDC client for 1 hour.
|
||
- Exchanges callback code, verifies nonce/state, maps claims to local user.
|
||
- Role mapping uses configured admin group; default role is `user`.
|
||
- Client secret is write-only through admin settings.
|
||
|
||
### `services/encryptionService.js`
|
||
|
||
- `assertEncryptionReady()` — throws if no master key is configured.
|
||
- `encryptSecret(plain)` / `decryptSecret(cipher)` — symmetric encryption with HKDF-derived key.
|
||
- The key derivation was upgraded from SHA-256 to HKDF in `v0.78`; migrations `v0.77` (SMTP password) and `v0.79` (OIDC secret) wrapped any pre-existing plaintexts.
|
||
- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected.
|
||
|
||
### `services/loginFingerprint.js`
|
||
|
||
- Captures `ip`, `user_agent`, `browser`, `os`, `device_type`, `device_fingerprint` on every login.
|
||
- The `ip` and `user_agent` columns are **encrypted at rest** (migration `v0.84`).
|
||
- `success` (`v0.85`) tracks failed-attempt history separately from successful logins.
|
||
- A `session_fingerprint` is stored on each successful login so the SPA can detect a different device for the same session and trigger a re-auth challenge.
|
||
|
||
### `services/backupService.js`
|
||
|
||
- Backup directory: `BACKUP_PATH || backups/`.
|
||
- Valid backup IDs match managed prefixes only: `bill-tracker-backup`, `pre-restore`, `imported-backup`, `scheduled-backup`.
|
||
- Creates SQLite backups with checksum and metadata.
|
||
- Validates uploads by SQLite `integrity_check` and optional SHA-256 checksum.
|
||
- Restore creates a pre-restore backup before swapping DB.
|
||
- Path traversal is prevented by ID regex and `path.relative` checks.
|
||
|
||
### `services/backupScheduler.js`
|
||
|
||
- Validates daily/weekly schedule, `HH:MM` time, retention count 1-365.
|
||
- Stores schedule settings in `settings`.
|
||
- Uses `node-cron`, supports run-now, reload, next-run status, and retention cleanup.
|
||
|
||
### `services/cleanupService.js`
|
||
|
||
Cleanup tasks:
|
||
|
||
- Expired import sessions.
|
||
- Stale temp SQLite export files in OS temp dir.
|
||
- Orphan `.partial`/`.upload` backup files.
|
||
- Optional old import-history pruning.
|
||
|
||
Settings are stored in `settings`; run results are stored as JSON.
|
||
|
||
### `services/notificationService.js`
|
||
|
||
- Builds SMTP transport from global notification settings (SMTP password is **encrypted at rest**, migration `v0.77`).
|
||
- Sends test email to an admin-provided address.
|
||
- Runs due-bill notifications for 3 days, 1 day, due today, and overdue.
|
||
- De-duplicates sends via `notifications` unique key.
|
||
- Recipients come from user notification settings when enabled/allowed or global recipient settings.
|
||
- Push notifications: ntfy, Gotify, Discord webhook, Telegram bot — configured per-user in `users` columns (migration `v0.80`); dispatched alongside or in place of SMTP when configured.
|
||
|
||
### `services/spreadsheetImportService.js`
|
||
|
||
- Accepts XLSX buffers only, max 10 MB, max 5,000 rows.
|
||
- Detects monthly sheets, headers, categories, bills, amounts, and ambiguous rows.
|
||
- Creates `import_sessions` preview records with 24-hour TTL.
|
||
- Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history.
|
||
|
||
### `services/ofxImportService.js`
|
||
|
||
- OFX / QFX bank-statement import (separate from CSV/Excel). Uses `node-ofx-parser`; falls back to manual buffer parsing when the package is not installed.
|
||
- Produces `import_sessions` previews and commits to `transactions` with `source_type='file_import'`.
|
||
- 24-hour TTL on preview sessions; results write to `import_history`.
|
||
|
||
### `services/userDbImportService.js`
|
||
|
||
- Accepts user SQLite export files up to 50 MB.
|
||
- Requires export metadata and known tables.
|
||
- Sanitizes categories, bills, payments, monthly state, and starting amounts.
|
||
- Preview stores an import session; apply maps export IDs to current user-owned IDs.
|
||
|
||
### `services/transactionService.js`
|
||
|
||
Transaction data source and transaction row helpers:
|
||
|
||
- `ensureManualDataSource(db, userId)` creates/retrieves a user-specific manual data source (`type='manual', provider='manual', name='Manual Entry'`).
|
||
- `decorateDataSource(row)` removes `encrypted_secret`, adds `source_label` and `source_type_label`.
|
||
- `decorateTransaction(row)` adds `source_label`, `source_type_label`, and embedded `data_source` object with safe fields.
|
||
- `getSourceTypeLabel(type)` returns labels: `manual` → Manual, `file_import` → File import, `provider_sync` → Provider sync.
|
||
- `sourceLabel(source)` constructs human-readable source labels for manual entries or provider names.
|
||
- `transactions` schema carries a `pending` flag (`v1.01`) for SimpleFIN pending transactions.
|
||
|
||
### `services/transactionMatchService.js` + `services/transactionMatchState.js`
|
||
|
||
- `matchTransactionToBill(transactionId, billId, userId)` — links a transaction to a bill, creates a corresponding `payments` row with `payment_source='provider_sync'`, runs accounting override logic.
|
||
- `unmatchTransaction(transactionId, userId)` — reverses the link and deletes the auto-created payment (or marks it `accounting_excluded`).
|
||
- `ignoreTransaction` / `unignoreTransaction` — set/match-status transitions.
|
||
- `transactionMatchState.js` is the state-machine helper that powers match/unmatch/ignore/unignore.
|
||
|
||
### `services/matchSuggestionService.js`
|
||
|
||
- `listMatchSuggestions(userId, transactionId, billId, limit, offset)` — returns scored suggestions.
|
||
- `rejectMatchSuggestion(userId, transactionId, billId)` — records the rejection in `match_suggestion_rejections`.
|
||
- `suggestionCounts(userId)` — dashboard counts.
|
||
- Rejections are stored in `match_suggestion_rejections` (created in `v0.62`) so a transaction/bill pair is never re-suggested after rejection.
|
||
- After sync, `autoMatchForUser(userId)` runs merchant-rule + merchant-store-match + match-suggestion passes in order.
|
||
|
||
### `services/csvTransactionImportService.js`
|
||
|
||
CSV import workflow for transactions:
|
||
|
||
- Parses CSV with quoted-field support and quote doubling.
|
||
- `previewCsvTransactions(userId, buffer, options)` returns headers, sample rows, suggested field mapping, errors, and creates a 24-hour TTL import session (max 25k rows).
|
||
- `suggestMapping(headers)` auto-detects field mappings from header names against `posted_date`, `transacted_at`, `amount`, `description`, `payee`, `memo`, `category`, `account`, `transaction_type`, `currency`, etc.
|
||
- `commitCsvTransactions(userId, importSessionId, mapping)` imports rows into the `transactions` table with `source_type='file_import'`, auto-creates a CSV data source and financial accounts per unique account name.
|
||
- Stable deduplication via SHA-256 hash: `csv:id:` prefix for explicit transaction IDs, `csv:hash:` prefix from date+amount+description+payee+account.
|
||
- Imports record to `import_history` with counts and details.
|
||
- `FIELD_LABELS` maps field keys to user-friendly labels for validation messages.
|
||
|
||
### `services/bankSyncService.js` + `services/simplefinService.js` + `services/bankSyncConfigService.js` + `services/bankSyncWorker.js`
|
||
|
||
- **SimpleFIN bridge** (`simplefinService.js`): wraps the SimpleFIN protocol — `claimSetupToken`, `fetchAccountsAndTransactions`, `normalizeAccount`, `normalizeTransaction`, `sanitizeErrorMessage`. `assertEncryptionReady` is invoked before any secret operation.
|
||
- **Per-user sync** (`bankSyncService.js`): upserts `financial_accounts` rows keyed by `(data_source_id, provider_account_id)`; upserts `transactions` keyed by `(data_source_id, provider_transaction_id)`. Honors the `monitored` flag (`v0.64`) so the user can exclude a credit-card account from bill matching. Applies merchant rules, spending-category rules, merchant-store matches, then `autoMatchForUser` in sequence.
|
||
- **Config** (`bankSyncConfigService.js`): `getBankSyncConfig(userId)`, `SYNC_DAYS_EFFECTIVE`, `SYNC_DAYS_DEFAULT`. Per-user default-lookback window.
|
||
- **Worker** (`bankSyncWorker.js`): scheduled background sync, started from `server.js` after `app.listen`. `bankSyncConfigService` admin endpoints live at `/api/admin/bank-sync-config`.
|
||
|
||
### `services/billMerchantRuleService.js`
|
||
|
||
- Persistent merchant→bill auto-match rules (table created in `v0.67`).
|
||
- `applyMerchantRules(transactions, userId)` — returns transactions with `matched_bill_id` and `match_status='matched'` filled in.
|
||
- `auto_attribute_late` (`v0.83`) marks rules whose bills always post after month-end, so amount-suggestion math skips the “looks like last month’s bill” signal.
|
||
|
||
### `services/merchantStoreMatchService.js`
|
||
|
||
- 5k merchant/store matching pack seeded by migration `v1.05`.
|
||
- `applyMerchantStoreMatches(transactions, userId)` — broad merchant-to-store normalization for spending categorization.
|
||
|
||
### `services/advisoryFilterService.js`
|
||
|
||
- 5k advisory non-bill patterns + bill-like override terms seeded by migration `v0.68`.
|
||
- Used to suppress “this looks like a bill but isn’t” false positives in tracker suggestions.
|
||
|
||
### `services/subscriptionService.js` + `services/amountSuggestionService.js`
|
||
|
||
- `subscriptionService.js` — manages the user’s subscription bills, recommendations, declined hints (`v0.66`), feedback (`v0.97`), and the top-200 catalog seeding. Exposes `normalizeMerchant` used by migration `v0.90` to re-normalize stored rules.
|
||
- `amountSuggestionService.js` — `computeAmountSuggestion` (per-bill) and `computeAmountSuggestionsBatch` (all-bills, used by the Tracker P1 fix to kill the N+1).
|
||
- The batched suggestion is byte-identical to the per-bill function; pinned by `tests/amountSuggestionService.test.js`.
|
||
|
||
### `services/snowballService.js` + `services/calendarFeedService.js` + `services/driftService.js` + `services/statusRuntime.js`
|
||
|
||
- `snowballService.js` — Debt snowball/avalanche calculations, Ramsey mode support, order updates, plan lifecycle math.
|
||
- `calendarFeedService.js` — generates iCalendar (`.ics`) text for upcoming bills, gated by `calendar_tokens` (`v1.00`).
|
||
- `driftService.js` — detects when an autopay/posted amount drifts from the bill’s expected amount; surfaces the “drift_snoozed_until” field (`v0.71`).
|
||
- `statusRuntime.js` — `recordError(err, req)`, runtime worker state, health JSON consumed by `/api/status`.
|
||
|
||
### `services/spendingService.js` + `services/userSettings.js` + `services/userDataService.js` + `services/eraseUserData.js` (in `utils`/tests) + `services/billsService.js` + `services/aprService.js`
|
||
|
||
- `spendingService.js` — spending categories, rules (`v0.87`), budgets, default category seeding, income breakdown, `applySpendingCategoryRules` used by the bank-sync pipeline.
|
||
- `userSettings.js` — `getUserSettings(userId)` returns a typed settings object; consumed by the bank-sync pipeline.
|
||
- `userDataService.js` — user-scoped read helpers (e.g. `getCategoriesForUser`).
|
||
- `billsService.js` — bill CRUD helpers, soft-delete + `RecentlyDeletedBills` restore support.
|
||
- `aprService.js` — annual-percentage-rate / monthly interest accrual used by `v0.93` (`interest_accrued_month`, `interest_delta`).
|
||
|
||
### `services/paymentAccountingService.js` + `services/paymentValidation.js`
|
||
|
||
- `paymentAccountingService.js` — source-of-truth for “manual vs. bank” payments. The bank-backed predicate, the accounting-active SQL fragment, and the core invariant: a bank payment overrides a provisional manual payment so it isn’t double-counted, restores the balance, and the manual payment counts again with its balance re-applied when the override is removed. Pinned by `tests/paymentAccountingService.test.js`.
|
||
- `paymentValidation.js` — `PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match']` (auto_match was normalized to provider_sync in `v0.82`), `validatePaymentSource()`, `validatePaymentInput()`.
|
||
|
||
### `services/auditService.js`
|
||
|
||
Writes `audit_log` rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, migration rollback attempts, WebAuthn/TOTP events, bank-sync admin changes, and 2FA enable/disable.
|
||
|
||
`db/database.js` does not import `logAudit` at module load time. It uses a lazy `getLogAudit()` helper so migration code can write audit rows without creating a circular dependency: `auditService` imports `database.js`, and `database.js` needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors.
|
||
|
||
---
|
||
|
||
## 5. API Reference
|
||
|
||
All routes are prefixed with `/api` unless stated otherwise. Most mutating endpoints require CSRF token header `x-csrf-token`. Auth is cookie-based (`bt_session`). User routes are scoped to the authenticated user unless noted. `GET /api/health` is the unauthenticated liveness probe and is the only public non-meta endpoint.
|
||
|
||
Response conventions:
|
||
|
||
- Success: JSON object/array unless endpoint downloads a file.
|
||
- Validation: `400 {error, code?, field?}`.
|
||
- Unauthenticated: `401`.
|
||
- Forbidden: `403`.
|
||
- Missing resource: `404`.
|
||
- Conflict: `409`.
|
||
- Rate-limited: `429`.
|
||
|
||
### 5.1 Auth (`/api/auth`)
|
||
|
||
- `POST /auth/login` — `{username, password}`. Sets `bt_session`. Rate-limited (skipped if zero users). Skips CSRF (no session yet).
|
||
- `POST /auth/logout` — clears `bt_session`.
|
||
- `POST /auth/logout-all` — invalidates every session for the user.
|
||
- `GET /auth/me` — public user object (single-user mode supplies user without session).
|
||
- `GET /auth/mode` — auth mode, local-login flag, OIDC public info, single-user status.
|
||
- `POST /auth/restore-multi-user-mode` — restores multi-user mode where allowed.
|
||
- `POST /auth/acknowledge-privacy` — updates first-login/privacy acknowledgement flags.
|
||
- `POST /auth/acknowledge-version` — stamps `users.last_seen_version`.
|
||
- `POST /auth/change-password` — `{current_password, new_password}`. Password-limited, CSRF-skip. Rotates session id, invalidates others.
|
||
- `GET /auth/csrf-token` — returns `{token}` mirroring the cookie; SPA caches in memory.
|
||
- `GET /auth/has-users` — whether non-default users exist.
|
||
- `GET /auth/users` — admin-only safe user list.
|
||
- `POST /auth/users` — admin-only create.
|
||
- `GET /auth/login-history` — recent login/failed-login history for the current user (encrypted columns decrypted in-process).
|
||
- `GET /auth/webauthn/status` — `{enabled, credentials:[{id, name, createdAt, lastUsedAt}]}`.
|
||
- **TOTP** (`/api/auth/totp`):
|
||
- `POST /auth/totp/challenge` — `{username, password}` → `{challenge_id}` (5-min TTL).
|
||
- `GET /auth/totp/setup` — admin-aware: returns `{secret, uri, qr_data_url}` for a new enrolment.
|
||
- `POST /auth/totp/enable` — `{secret, token}` → verifies, encrypts, stores, returns recovery codes.
|
||
- `POST /auth/totp/disable` — clears secret + recovery codes.
|
||
- `GET /auth/totp/status` — `{enabled, hasSecret, recoveryCodesRemaining}`.
|
||
|
||
### 5.2 OIDC Auth (`/api/auth/oidc`)
|
||
|
||
OIDC rate limiter applies.
|
||
|
||
- `GET /auth/oidc/login?redirect_to=/path` — public when OIDC active; builds PKCE state and redirects.
|
||
- `GET /auth/oidc/callback?code=&state=` — validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root. Error redirects use query errors such as `access_denied` or `authentication_failed`.
|
||
|
||
### 5.3 Admin (`/api/admin`)
|
||
|
||
Mounted under `/api/admin`; all require `requireAuth + requireAdmin + adminActionLimiter`. Backup subroutes also use `backupOperationLimiter`.
|
||
|
||
- `GET /admin/has-users` — `{has_users:boolean}`.
|
||
- `GET /admin/users` — safe users ordered by default-admin / role / username.
|
||
- `POST /admin/users` — `{username, password}`.
|
||
- `PUT /admin/users/:id/password` — `{password}`. Invalidates target sessions; requires password change.
|
||
- `PUT /admin/users/:id/role` — `{role:"admin"|"user"}`. Cannot change own role; cannot remove last admin.
|
||
- `PUT /admin/users/:id/active` — `{active:boolean}`. Cannot deactivate self.
|
||
- `PUT /admin/users/:id/username` — change a user’s username (admin only).
|
||
- `DELETE /admin/users/:id` — transaction deletes import sessions/history, sessions, and the user.
|
||
- **Backups**: `GET/POST /admin/backups`, `GET /admin/backups/settings`, `PUT /admin/backups/settings`, `POST /admin/backups/run-scheduled-now`, `POST /admin/backups/import` (raw SQLite, max 100 MB, optional `X-Checksum-Sha256`), `GET /admin/backups/:id/download`, `POST /admin/backups/:id/restore`, `DELETE /admin/backups/:id`.
|
||
- **Cleanup**: `GET/PUT /admin/cleanup`, `POST /admin/cleanup/run`.
|
||
- **Auth mode**: `GET /admin/auth-mode`, `POST /admin/auth-mode/oidc-test`, `PUT /admin/auth-mode`.
|
||
- **Bank sync config**: `GET /admin/bank-sync-config`, `PUT /admin/bank-sync-config` (default-lookback days, monitored-account default, etc.).
|
||
- **Migrations**: `POST /admin/migrations/rollback` with `{version}`. Maps `NOT_APPLIED` → 404, `ROLLBACK_NOT_SUPPORTED` → 422, other failures → 500.
|
||
|
||
### 5.4 Bills (`/api/bills`)
|
||
|
||
User/admin tracker access. Standard bill fields plus `cycle_type`, `cycle_day`, `is_subscription`, `subscription_type`, `catalog_id`, `autopay_verified_at`, `inactive_reason`, `drift_snoozed_until`, `sort_order`, `interest_accrued_month`, `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at`.
|
||
|
||
Endpoints (subset): `GET /bills?inactive=true`, `GET /bills/:id`, `POST /bills`, `PUT /bills/:id`, `DELETE /bills/:id` (hard delete; cascades payments, monthly state, history ranges, merchant rules; returns warning), `GET/PUT /bills/:id/monthly-state?year=&month=`, `GET /bills/:id/payments`, `POST /bills/:id/toggle-paid`, `GET/POST/PUT/DELETE /bills/:id/history-ranges`, `GET /bills/recently-deleted` (admin), `POST /bills/:id/restore`.
|
||
|
||
### 5.5 Payments (`/api/payments`)
|
||
|
||
User/admin tracker access. Soft delete via `deleted_at`. `payment_source` ∈ `{manual, file_import, provider_sync, transaction_match}`. Hard-dedupe unique index on `transaction_id` (`v0.61`).
|
||
|
||
Endpoints: `GET /payments?bill_id=&year=&month=`, `GET /payments/:id`, `POST /payments`, `POST /payments/quick`, `POST /payments/bulk` (max 50), `PUT /payments/:id`, `DELETE /payments/:id`, `POST /payments/:id/restore`.
|
||
|
||
### 5.6 Categories (`/api/categories`)
|
||
|
||
User/admin tracker access. Categories have a `spending_enabled` flag (`v0.88`) and a `sort_order` (`v0.75`). Categories can be grouped via `category_groups` (`v1.06`).
|
||
|
||
Endpoints: `GET /categories` (seeds defaults, includes `spending_enabled` + counts), `POST /categories`, `PUT /categories/:id`, `DELETE /categories/:id`, plus category-group endpoints under `/api/categories/groups` (list/create/update/delete/reorder).
|
||
|
||
### 5.7 Tracker and Calendar (`/api/tracker`, `/api/calendar`)
|
||
|
||
- `GET /tracker?year=&month=` — returns `year`, `month`, tracker `rows`, totals, starting-amount info, previous-month paid total, three-month averages/trends, **drift / autopay / amount-suggestion** objects, **safe-to-spend** (cashflow timeline), `generated_at`.
|
||
- `GET /tracker/upcoming?days=30` — upcoming active bills.
|
||
- `GET /tracker/health` — dedicated health endpoint for the home page (subset of `/tracker`).
|
||
- `GET /calendar?year=&month=` — month days with payment entries, bills/due-status entries, totals.
|
||
- `GET /calendar/feed.ics?token=...` — public iCalendar feed of upcoming bills, gated by `calendar_tokens` (`v1.00`).
|
||
|
||
### 5.8 Summary and Starting Amounts (`/api/summary`, `/api/monthly-starting-amounts`)
|
||
|
||
- `GET /summary?year=&month=` — returns `{year, month, income, expenses, starting_amounts, previous_month, summary, chart, bank_tracking, generated_at}`.
|
||
- `PUT /summary/income` — `{year, month, amount, label?}`.
|
||
- `GET /monthly-starting-amounts?year=&month=` / `PUT /monthly-starting-amounts` — bucket response.
|
||
|
||
### 5.9 Snowball (`/api/snowball`)
|
||
|
||
- `GET /snowball` — current user's debt bills pre-sorted by `snowball_order`.
|
||
- `GET /snowball/settings` / `PATCH /snowball/settings` — extra payment, ramsey mode, ready-current, ready-emergency-fund.
|
||
- `GET /snowball/projection` — `{snowball, avalanche, minimum_only, comparison}` with enriched debt arrays (APR snapshots, interest-accrual month).
|
||
- `PATCH /snowball/order` — `[{id, snowball_order}]` batch update.
|
||
- **Plans** (lifecycle, `v0.73`): `POST /snowball/plans`, `GET /snowball/plans`, `GET /snowball/plans/active`, `PATCH /snowball/plans/:id`, `POST /snowball/plans/:id/pause|resume|complete|abandon`.
|
||
|
||
### 5.10 Spending (`/api/spending`)
|
||
|
||
- `GET /spending/summary?year=&month=` — monthly spending summary, by category / by bill.
|
||
- `GET /spending/transactions` — paginated, filterable.
|
||
- `PATCH /spending/transactions/:id/category` — manual reassignment.
|
||
- `GET /spending/budgets` / `PUT /spending/budgets` / `POST /spending/budgets/copy` — monthly budgets with month-rollover.
|
||
- `GET /spending/category-rules` / `POST /spending/category-rules` / `DELETE /spending/category-rules/:id` — automation rules seeded in `v0.87`.
|
||
- `GET /spending/income` — per-user monthly income.
|
||
|
||
### 5.11 Subscriptions (`/api/subscriptions`)
|
||
|
||
- `GET /subscriptions` — user’s subscription bills.
|
||
- `GET /subscriptions/recommendations` — bank-derived candidates (declined hints suppressed).
|
||
- `POST /subscriptions/recommendations/decline` — record a decline.
|
||
- `POST /subscriptions/recommendations/match-bill` — link a recommendation to an existing bill.
|
||
- `POST /subscriptions/recommendations/create` — create a new bill from a recommendation.
|
||
- `GET /subscriptions/transaction-matches` — candidate transactions for the user’s subscriptions.
|
||
- `GET /subscriptions/catalog` — top-200 catalog with bank descriptors.
|
||
- `PUT /subscriptions/:id/catalog-link` / `POST /subscriptions/catalog/:catalogId/descriptors` / `DELETE /subscriptions/catalog/descriptors/:id` — manage custom bank descriptors.
|
||
- `PATCH /subscriptions/:id` — update subscription fields (price, cycle, descriptor, etc.).
|
||
|
||
### 5.12 Analytics (`/api/analytics`)
|
||
|
||
- `GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=&include_skipped=` — monthly spending, expected vs actual, category totals, bill totals, filters, `generated_at`.
|
||
|
||
### 5.13 Settings (`/api/settings`)
|
||
|
||
- `GET /settings` / `PUT /settings` — user-visible settings.
|
||
- `POST /settings/seed-demo-data` — seed demo data.
|
||
|
||
### 5.14 Notifications (`/api/notifications`)
|
||
|
||
Mounted under `/api/notifications`; `requireAuth` at mount.
|
||
|
||
- `GET/PUT /notifications/admin` — SMTP + push (ntfy / Gotify / Discord / Telegram) global settings.
|
||
- `POST /notifications/test` — `{to}`.
|
||
- `GET/PUT /notifications/me` — per-user notification email, push channels, lead times.
|
||
|
||
### 5.15 Profile (`/api/profile`)
|
||
|
||
- `GET /profile` — safe profile, notification settings, export URLs, import-history URL.
|
||
- `PATCH /profile` — `{display_name}`.
|
||
- `GET/PATCH /profile/settings` — per-user notification prefs.
|
||
- `POST /profile/change-password` — rotates session, invalidates others.
|
||
- `GET /profile/exports` / `GET /profile/import-history`.
|
||
|
||
### 5.16 User Demo Data (`/api/user`)
|
||
|
||
- `POST /user/seed-demo-data`, `POST /user/clear-demo-data` (rate-limited; deletes `is_seeded=1` bills/categories for current user).
|
||
|
||
### 5.17 Import (`/api/import`, `/api/imports`)
|
||
|
||
`requireAuth + requireUser + importLimiter` at mount. Both prefixes are accepted.
|
||
|
||
- `POST /import/spreadsheet/preview` (XLSX, max 10 MB, optional `parse_all_sheets`, `year`, `month`).
|
||
- `POST /import/spreadsheet/apply`.
|
||
- `POST /import/user-db/preview` (SQLite, max 50 MB) / `apply`.
|
||
- `POST /import/csv/preview` / `POST /import/csv/commit`.
|
||
- `POST /import/ofx/preview` / `POST /import/ofx/commit` (OFX/QFX bank statements).
|
||
- `GET /import/history`.
|
||
|
||
### 5.18 Data Sources (`/api/data-sources`)
|
||
|
||
- `GET /data-sources?type=&status=` — `type` ∈ `manual, file_import, provider_sync`; `status` ∈ `active, inactive, error`. Returns sources with `account_count`, `transaction_count`, sync info.
|
||
|
||
### 5.19 Transactions (`/api/transactions`)
|
||
|
||
- `GET /transactions?limit=&offset=&match_status=&ignored=&source_type=&start_date=&end_date=&q=&data_source_id=&account_id=&matched_bill_id=`. Free-text search across description/payee/memo/category.
|
||
- `POST /transactions/manual` — manual entry, returns created row with `source_type='manual'`.
|
||
- `PUT /transactions/:id` — partial update.
|
||
- `DELETE /transactions/:id` — soft-unmatch then hard delete.
|
||
- `POST /transactions/:id/match` / `unmatch` / `ignore` / `unignore`.
|
||
- The `pending` flag (`v1.01`) marks SimpleFIN pending transactions separately from posted ones.
|
||
|
||
### 5.20 Match Suggestions (`/api/matches`)
|
||
|
||
- `GET /matches/suggestions?transaction_id=&bill_id=&limit=&offset=`.
|
||
- `POST /matches/:id/reject` — records rejection so the pair is never re-suggested.
|
||
|
||
### 5.21 Export (`/api/export`)
|
||
|
||
- `GET /export?year=YYYY&format=csv|xlsx` — payment/bill history for the year.
|
||
- `GET /export/user-excel` — user workbook.
|
||
- `GET /export/user-db` — portable SQLite with metadata.
|
||
|
||
### 5.22 Status, About, Version, Privacy, Health
|
||
|
||
- `GET /status` — admin-only: app version, uptime, runtime worker state, DB health, SMTP/push config status, backup status, current-month tracker health, recent errors.
|
||
- `GET /about` — public; `{version, about}`.
|
||
- `GET /about-admin` — admin-only, file allowlist `FUTURE.md` / `DEVELOPMENT_LOG.md`.
|
||
- `GET /version` — public; package version + latest structured notes from `HISTORY.md`.
|
||
- `GET /version/history` — raw history text.
|
||
- `GET /version/update-status` — async, checks upstream for new releases (uses `services/updateCheckService`).
|
||
- `GET /privacy` — public privacy text.
|
||
- `GET /health` — public liveness probe `{ok:true}`.
|
||
|
||
## 6. Database Reference
|
||
|
||
SQLite uses WAL mode and foreign keys. Base schema is in `db/schema.sql`; `db/database.js` applies migrations to reach the current schema.
|
||
|
||
### Tables and columns
|
||
|
||
#### `users`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `username TEXT NOT NULL UNIQUE COLLATE NOCASE`
|
||
- `password_hash TEXT NOT NULL`
|
||
- `role TEXT NOT NULL DEFAULT 'user'` (`admin` or `user`)
|
||
- `must_change_password INTEGER NOT NULL DEFAULT 0`
|
||
- `first_login INTEGER NOT NULL DEFAULT 1`
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- `notification_email TEXT`
|
||
- `notifications_enabled INTEGER NOT NULL DEFAULT 0`
|
||
- `notify_3d INTEGER NOT NULL DEFAULT 1`
|
||
- `notify_1d INTEGER NOT NULL DEFAULT 1`
|
||
- `notify_due INTEGER NOT NULL DEFAULT 1`
|
||
- `notify_overdue INTEGER NOT NULL DEFAULT 1`
|
||
- `display_name TEXT`
|
||
- `last_password_change_at TEXT`
|
||
- `auth_provider TEXT NOT NULL DEFAULT 'local'`
|
||
- `external_subject TEXT`
|
||
- `email TEXT`
|
||
- `last_login_at TEXT`
|
||
- `active INTEGER NOT NULL DEFAULT 1`
|
||
- `is_default_admin INTEGER NOT NULL DEFAULT 0`
|
||
- `snowball_extra_payment REAL` (`v0.48`)
|
||
- `last_seen_version TEXT` (`v0.50`) — release-notes notification stamp
|
||
- `webauthn_user_id TEXT` (`v0.92`); `webauthn_enabled INTEGER NOT NULL DEFAULT 0` (`v0.92`)
|
||
- `totp_enabled INTEGER NOT NULL DEFAULT 0` (`v0.86`); `totp_secret` (encrypted, `v0.86`); `totp_recovery_codes` (JSON array of hashes, `v0.86`)
|
||
- Push notification columns (`v0.80`): ntfy / Gotify / Discord / Telegram (topic + URL + enabled flag each)
|
||
- `geolocation_enabled INTEGER NOT NULL DEFAULT 0` (`v1.02`; was a global admin setting)
|
||
|
||
#### `sessions`
|
||
|
||
- `id TEXT PRIMARY KEY` — **session token hash**, not the raw cookie value (`v0.94`). The cookie carries a random opaque ID; the DB stores its SHA-256 so a leaked DB does not yield usable session tokens.
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `expires_at TEXT NOT NULL`
|
||
- `created_at TEXT DEFAULT datetime('now')` (`v0.43`)
|
||
|
||
#### `webauthn_credentials` (`v0.92`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `credential_id TEXT NOT NULL UNIQUE`
|
||
- `public_key TEXT NOT NULL`
|
||
- `sign_count INTEGER NOT NULL DEFAULT 0`
|
||
- `transports TEXT` (JSON)
|
||
- `backup_eligible INTEGER NOT NULL DEFAULT 0`
|
||
- `backup_state INTEGER NOT NULL DEFAULT 0`
|
||
- `credential_name TEXT`
|
||
- `aaguid TEXT`
|
||
- `last_used_at TEXT`
|
||
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
|
||
#### `webauthn_challenges` (`v0.92`)
|
||
|
||
- `id TEXT PRIMARY KEY`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `challenge_type TEXT NOT NULL CHECK ('registration' OR 'authentication')`
|
||
- `challenge TEXT NOT NULL`
|
||
- `expires_at TEXT NOT NULL` (15-min TTL)
|
||
|
||
#### `totp_challenges` (`v0.86`)
|
||
|
||
- `id TEXT PRIMARY KEY`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `challenge TEXT NOT NULL`
|
||
- `expires_at TEXT NOT NULL` (5-min TTL)
|
||
|
||
#### `categories`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
|
||
- `name TEXT NOT NULL`
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- `is_seeded INTEGER NOT NULL DEFAULT 0`
|
||
- `deleted_at TEXT` (`v0.54`)
|
||
- `sort_order INTEGER` (`v0.75`) — persistent order
|
||
- `spending_enabled INTEGER NOT NULL DEFAULT 0` (`v0.88`) — separates bill vs spending categories
|
||
- `group_id INTEGER REFERENCES category_groups(id) ON DELETE SET NULL` (`v1.06`)
|
||
|
||
#### `category_groups` (`v1.06`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `name TEXT NOT NULL`
|
||
- `sort_order INTEGER`
|
||
- `created_at`, `updated_at`
|
||
- Unique: `(user_id, name)`
|
||
|
||
#### `bills`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `user_id INTEGER REFERENCES users(id) ON DELETE CASCADE`
|
||
- `name TEXT NOT NULL`
|
||
- `category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL`
|
||
- `due_day INTEGER NOT NULL CHECK 1-31`
|
||
- `override_due_date TEXT`
|
||
- `bucket TEXT CHECK ('1st','15th')`
|
||
- `expected_amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`)
|
||
- `interest_rate REAL CHECK null or 0-100`
|
||
- `billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')` (canonicalized in `v0.76`)
|
||
- `autopay_enabled INTEGER NOT NULL DEFAULT 0`
|
||
- `autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')`
|
||
- `website TEXT`
|
||
- `username TEXT`
|
||
- `account_info TEXT`
|
||
- `has_2fa INTEGER NOT NULL DEFAULT 0`
|
||
- `active INTEGER NOT NULL DEFAULT 1`
|
||
- `notes TEXT`
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- `history_visibility TEXT NOT NULL DEFAULT 'default'`
|
||
- `is_seeded INTEGER NOT NULL DEFAULT 0`
|
||
- `cycle_type TEXT NOT NULL DEFAULT 'monthly'`
|
||
- `cycle_day TEXT`
|
||
- `current_balance REAL` (dollars; converted to integer cents by `v1.03`)
|
||
- `minimum_payment REAL` (dollars; converted to integer cents by `v1.03`)
|
||
- `snowball_order INTEGER`
|
||
- `snowball_include INTEGER NOT NULL DEFAULT 0`
|
||
- `snowball_exempt INTEGER NOT NULL DEFAULT 0`
|
||
- `auto_mark_paid INTEGER NOT NULL DEFAULT 0`
|
||
- `deleted_at TEXT`
|
||
- `is_subscription INTEGER NOT NULL DEFAULT 0`, `subscription_type TEXT` (`v0.63`)
|
||
- `interest_accrued_month TEXT` (`v0.93`) — calendar month of last applied interest
|
||
- `drift_snoozed_until TEXT` (`v0.71`) — suppresses drift notifications until this date
|
||
- `sort_order INTEGER` (`v0.72`) — persistent tracker order
|
||
- `catalog_id INTEGER REFERENCES subscription_catalog(id) ON DELETE SET NULL` (`v0.96`)
|
||
- `autopay_verified_at TEXT`, `inactive_reason TEXT` (`v0.99`) — autopay trust / lifecycle fields
|
||
|
||
#### `payments`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `amount REAL NOT NULL` (dollars; converted to integer cents by `v1.03`)
|
||
- `paid_date TEXT NOT NULL`
|
||
- `method TEXT`
|
||
- `notes TEXT`
|
||
- `balance_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.49`)
|
||
- `payment_source TEXT NOT NULL DEFAULT 'manual'` (`v0.59`; `auto_match` normalized to `provider_sync` in `v0.82`)
|
||
- `transaction_id INTEGER` (`v0.59`; unique-per-active index added in `v0.61`)
|
||
- `interest_delta REAL` (dollars; converted to integer cents by `v1.03`; `v0.93`)
|
||
- `accounting_excluded INTEGER NOT NULL DEFAULT 0` (`v0.98`) — bank-override metadata for provisional manual payments
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- `deleted_at TEXT`
|
||
|
||
#### `monthly_bill_state`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `year INTEGER NOT NULL CHECK 2000-2100`
|
||
- `month INTEGER NOT NULL CHECK 1-12`
|
||
- `actual_amount REAL` (dollars; converted to integer cents by `v1.03`)
|
||
- `notes TEXT`
|
||
- `is_skipped INTEGER NOT NULL DEFAULT 0`
|
||
- `snoozed_until TEXT` (`v0.70`) — overdue command-center snooze
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- Unique: `(bill_id, year, month)`
|
||
|
||
#### `monthly_income`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `year INTEGER NOT NULL`
|
||
- `month INTEGER NOT NULL`
|
||
- `label TEXT NOT NULL DEFAULT 'Salary'`
|
||
- `amount REAL NOT NULL DEFAULT 0` (dollars; converted to integer cents by `v1.03`)
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- Unique: `(user_id, year, month)`
|
||
|
||
#### `monthly_starting_amounts`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `year INTEGER NOT NULL`
|
||
- `month INTEGER NOT NULL`
|
||
- `first_amount REAL NOT NULL DEFAULT 0`
|
||
- `fifteenth_amount REAL NOT NULL DEFAULT 0`
|
||
- `other_amount REAL NOT NULL DEFAULT 0`
|
||
- `notes TEXT`
|
||
- `created_at TEXT DEFAULT datetime('now')`
|
||
- `updated_at TEXT DEFAULT datetime('now')`
|
||
- Unique: `(user_id, year, month)`
|
||
|
||
#### `bill_history_ranges`
|
||
|
||
- `id INTEGER PRIMARY KEY`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `start_year INTEGER NOT NULL`
|
||
- `start_month INTEGER NOT NULL`
|
||
- `end_year INTEGER`
|
||
- `end_month INTEGER`
|
||
- `label TEXT`
|
||
- `description TEXT NOT NULL`
|
||
- `applied_at TEXT NOT NULL DEFAULT datetime('now')`
|
||
|
||
#### `data_sources`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `type TEXT NOT NULL` (`manual`, `file_import`, `provider_sync`)
|
||
- `provider TEXT`
|
||
- `name TEXT NOT NULL`
|
||
- `status TEXT NOT NULL DEFAULT 'active'` (`active`, `inactive`, `error`)
|
||
- `config_json TEXT`
|
||
- `encrypted_secret TEXT`
|
||
- `last_sync_at TEXT`
|
||
- `last_error TEXT`
|
||
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- Unique partial index: `(user_id, type, provider)` WHERE `type='manual' AND provider='manual'`
|
||
|
||
#### `financial_accounts`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL`
|
||
- `provider_account_id TEXT`
|
||
- `name TEXT NOT NULL`
|
||
- `org_name TEXT`
|
||
- `account_type TEXT`
|
||
- `currency TEXT`
|
||
- `balance INTEGER`
|
||
- `available_balance INTEGER`
|
||
- `raw_data TEXT`
|
||
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- Unique: `(data_source_id, provider_account_id)`
|
||
|
||
#### `transactions`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL`
|
||
- `account_id INTEGER REFERENCES financial_accounts(id) ON DELETE SET NULL`
|
||
- `provider_transaction_id TEXT`
|
||
- `source_type TEXT NOT NULL` (`manual`, `file_import`, `provider_sync`)
|
||
- `transaction_type TEXT`
|
||
- `posted_date TEXT`
|
||
- `transacted_at TEXT`
|
||
- `amount INTEGER NOT NULL`
|
||
- `currency TEXT`
|
||
- `description TEXT`
|
||
- `payee TEXT`
|
||
- `memo TEXT`
|
||
- `category TEXT`
|
||
- `raw_data TEXT`
|
||
- `matched_bill_id INTEGER REFERENCES bills(id) ON DELETE SET NULL`
|
||
- `match_status TEXT NOT NULL DEFAULT 'unmatched'` (`unmatched`, `matched`, `ignored`)
|
||
- `ignored INTEGER NOT NULL DEFAULT 0`
|
||
- `spending_category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL` (`v0.87`)
|
||
- `pending INTEGER NOT NULL DEFAULT 0` (`v1.01`) — SimpleFIN pending transactions
|
||
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- `updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- Unique partial index: `(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL`
|
||
- Indexes: `(user_id, posted_date, transacted_at)`, `(user_id, match_status, ignored)`, `(account_id)`, `(matched_bill_id)`
|
||
|
||
### Indexes
|
||
|
||
Important indexes include:
|
||
|
||
- `idx_bills_active(active)`
|
||
- `idx_bills_user_active(user_id, active)`
|
||
- `idx_bills_user_name(user_id, name)`
|
||
- `idx_bills_user_deleted(user_id, deleted_at)` (`v0.91`)
|
||
- `idx_bills_user_sort(user_id, active, sort_order, due_day)` (`v0.72`)
|
||
- `idx_categories_user_name(user_id, name)`
|
||
- `idx_categories_user_name_unique(user_id, name COLLATE NOCASE)`
|
||
- `idx_categories_user_deleted(user_id, deleted_at)` (`v0.91`)
|
||
- `idx_categories_user_sort(user_id, sort_order, name)` (`v0.75`)
|
||
- `idx_payments_bill_id(bill_id)`
|
||
- `idx_payments_paid_date(paid_date)`
|
||
- `idx_payments_bill_date_del(bill_id, paid_date, deleted_at)`
|
||
- `idx_payments_deleted(deleted_at)`
|
||
- `idx_payments_method(method)`
|
||
- `idx_payments_transaction_active` — unique-per-active on `transaction_id` (`v0.61`)
|
||
- `idx_sessions_user_id(user_id)`
|
||
- `idx_sessions_expires(expires_at)`
|
||
- `idx_monthly_bill_state_lookup(bill_id, year, month)`
|
||
- `idx_monthly_income_user_month(user_id, year, month)`
|
||
- `idx_monthly_starting_amounts_user(user_id)`
|
||
- `idx_monthly_starting_amounts_user_month(user_id, year, month)`
|
||
- `idx_notifications_lookup(bill_id, user_id, year, month)`
|
||
- `idx_import_sessions_user(user_id)`, `idx_import_sessions_expires(expires_at)`
|
||
- `idx_import_history_user(user_id)`, `idx_import_history_imported_at(imported_at)`
|
||
- `idx_oidc_states_expires(expires_at)`
|
||
- `idx_bill_history_ranges_bill(bill_id)`
|
||
- `idx_audit_log_user(user_id, created_at)`, `idx_audit_log_action(action, created_at)`
|
||
- `idx_data_sources_user_type(user_id, type, status)`
|
||
- `idx_data_sources_user_manual(user_id, type, provider)` WHERE `type='manual' AND provider='manual'`
|
||
- `idx_financial_accounts_user_source(user_id, data_source_id)`
|
||
- `idx_transactions_user_date(user_id, posted_date, transacted_at)`
|
||
- `idx_transactions_user_match(user_id, match_status, ignored)`
|
||
- `idx_transactions_account(account_id)`
|
||
- `idx_transactions_matched_bill(matched_bill_id)`
|
||
- `idx_transactions_provider_dedupe(data_source_id, provider_transaction_id)` WHERE `provider_transaction_id IS NOT NULL`
|
||
- `idx_bill_merchant_rules_user_bill(user_id, bill_id)` (`v0.81`)
|
||
|
||
#### `import_sessions`
|
||
|
||
- `id TEXT PRIMARY KEY`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `created_at TEXT NOT NULL`
|
||
- `expires_at TEXT NOT NULL`
|
||
- `preview_json TEXT NOT NULL`
|
||
|
||
#### `import_history`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `imported_at TEXT NOT NULL`
|
||
- `source_filename TEXT`
|
||
- `file_type TEXT DEFAULT 'csv_transactions'`
|
||
- `rows_parsed INTEGER DEFAULT 0`
|
||
- `rows_created INTEGER DEFAULT 0`
|
||
- `rows_updated INTEGER DEFAULT 0`
|
||
- `rows_skipped INTEGER DEFAULT 0`
|
||
- `rows_errored INTEGER DEFAULT 0`
|
||
- `options_json TEXT`
|
||
- `summary_json TEXT`
|
||
- `created_at TEXT DEFAULT (datetime('now'))`
|
||
|
||
#### `autopay_suggestion_dismissals`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `year INTEGER NOT NULL CHECK(year BETWEEN 2000 AND 2100)`
|
||
- `month INTEGER NOT NULL CHECK(month BETWEEN 1 AND 12)`
|
||
- `dismissed_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
- Unique: `(user_id, bill_id, year, month)`
|
||
|
||
#### `bill_templates`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `name TEXT NOT NULL`
|
||
- `data TEXT NOT NULL`
|
||
- `created_at TEXT DEFAULT (datetime('now'))`
|
||
- `updated_at TEXT DEFAULT (datetime('now'))`
|
||
- Unique: `(user_id, name)`
|
||
|
||
#### `match_suggestion_rejections`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `transaction_id INTEGER NOT NULL REFERENCES transactions(id) ON DELETE CASCADE`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `rejected_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
- Unique: `(user_id, transaction_id, bill_id)`
|
||
|
||
#### `user_login_history`
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `logged_in_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
- `ip_address TEXT` (encrypted at rest, `v0.84`)
|
||
- `user_agent TEXT` (encrypted at rest, `v0.84`)
|
||
- `browser TEXT`
|
||
- `os TEXT`
|
||
- `device_type TEXT`
|
||
- `device_fingerprint TEXT`
|
||
- `location TEXT` (`v0.84`) — IP-geolocation city/region/country when geolocation opt-in is on
|
||
- `success INTEGER NOT NULL DEFAULT 1` (`v0.85`) — failed attempts when 0
|
||
- `session_fingerprint TEXT` (`v0.85`) — for current-session detection
|
||
|
||
#### `subscription_catalog` (`v0.65`, expanded `v0.69`, `v0.95`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `name TEXT NOT NULL`
|
||
- `category TEXT`
|
||
- `subcategory TEXT` (`v0.95`)
|
||
- `starting_monthly_usd REAL` (`v0.95`)
|
||
- `bank_descriptors TEXT` (JSON array; populated by `v0.95` from a 2026 researched dataset)
|
||
- `created_at`, `updated_at`
|
||
|
||
#### `declined_subscription_hints` (`v0.66`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `descriptor TEXT NOT NULL`
|
||
- `source TEXT`
|
||
- `declined_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
- Unique: `(user_id, descriptor, source)`
|
||
|
||
#### `bill_merchant_rules` (`v0.67`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE`
|
||
- `normalized_merchant TEXT NOT NULL`
|
||
- `match_mode TEXT NOT NULL DEFAULT 'contains'` (e.g. `contains`, `equals`, `regex`)
|
||
- `priority INTEGER NOT NULL DEFAULT 0`
|
||
- `auto_attribute_late INTEGER NOT NULL DEFAULT 0` (`v0.83`)
|
||
- `created_at`, `updated_at`
|
||
- Composite index `(user_id, bill_id)` for fast `EXISTS` lookups (`v0.81`)
|
||
|
||
#### `user_catalog_descriptors` (`v0.96`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `catalog_id INTEGER NOT NULL REFERENCES subscription_catalog(id) ON DELETE CASCADE`
|
||
- `descriptor TEXT NOT NULL`
|
||
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
|
||
#### `subscription_recommendation_feedback` (`v0.97`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `recommendation_key TEXT NOT NULL`
|
||
- `verdict TEXT NOT NULL` (`accept` | `decline` | `mute`)
|
||
- `created_at TEXT NOT NULL DEFAULT (datetime('now'))`
|
||
|
||
#### `snowball_plans` (`v0.73`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `name TEXT NOT NULL`
|
||
- `strategy TEXT NOT NULL` (`snowball` | `avalanche` | `minimum_only`)
|
||
- `extra_payment REAL NOT NULL DEFAULT 0`
|
||
- `status TEXT NOT NULL DEFAULT 'active'` (`active` | `paused` | `completed` | `abandoned`)
|
||
- `created_at`, `updated_at`, `completed_at`
|
||
- Lifecycle transitions are recorded with audit + plan-row updates; `pause`/`resume` flip `status`.
|
||
|
||
#### `calendar_tokens` (`v1.00`)
|
||
|
||
- `id INTEGER PRIMARY KEY AUTOINCREMENT`
|
||
- `user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE`
|
||
- `token TEXT NOT NULL UNIQUE` (random opaque)
|
||
- `label TEXT`
|
||
- `created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP`
|
||
- `last_used_at TEXT`
|
||
- `revoked_at TEXT`
|
||
|
||
#### `spending_rules` / `spending_budgets` (`v0.87`)
|
||
|
||
- `spending_rules`: per-user regex/keyword rules mapping a transaction pattern to a `spending_category_id`. Auto-applied during bank sync by `applySpendingCategoryRules`.
|
||
- `spending_budgets`: per-user monthly budget rows; month-rollover uses the copy endpoint.
|
||
|
||
#### `merchant_store_matches` (seeded by `v1.05`)
|
||
|
||
- Static 5k merchant/store matching pack; used by `applyMerchantStoreMatches` during bank sync to normalize merchant strings.
|
||
|
||
#### `advisory_non_bill_filters` (seeded by `v0.68`)
|
||
|
||
- 5k advisory non-bill patterns + bill-like override terms.
|
||
|
||
### Migration system
|
||
|
||
`db/database.js`:
|
||
|
||
- Reads `db/schema.sql` in `initSchema()`.
|
||
- Creates `schema_migrations`.
|
||
- Detects and reconciles legacy databases via `db/migrations/legacyReconcileMigrations.js`.
|
||
- Applies ordered migrations from `db/migrations/versionedMigrations.js` only when not already recorded.
|
||
- Validates dependency chains (`dependsOn: ['v0.71']`) before applying dependent migrations.
|
||
- Uses a column whitelist for dynamic `ALTER TABLE` statements.
|
||
- Wraps versioned migrations in transactions, with special handling for `v0.40` because it uses PRAGMA-driven table rebuild work.
|
||
- Supports rollback SQL for selected migrations through `ROLLBACK_SQL_MAP` and `rollbackMigration(version)`.
|
||
|
||
Current migration set (`v0.2` → `v1.06`):
|
||
|
||
- `v0.2` payments soft delete.
|
||
- `v0.3` tracker payment compound index.
|
||
- `v0.4` monthly bill state.
|
||
- `v0.13` user profile columns.
|
||
- `v0.14` bill history visibility.
|
||
- `v0.14.4` bill interest rate.
|
||
- `v0.15` import sessions/history.
|
||
- `v0.17` OIDC/external identity columns and state table.
|
||
- `v0.18.1` monthly income.
|
||
- `v0.18.2` monthly starting amounts.
|
||
- `v0.18.3` other starting amount bucket.
|
||
- `v0.38` per-user import audit history.
|
||
- `v0.40` ownership for bills/categories.
|
||
- `v0.41` seeded demo-data flags.
|
||
- `v0.42` bill history ranges.
|
||
- `v0.43` session `created_at`.
|
||
- `v0.44` performance indexes.
|
||
- `v0.45` audit log.
|
||
- `v0.46` bill `cycle_type` and `cycle_day`.
|
||
- `v0.47` bills: `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at` columns.
|
||
- `v0.48` users: `snowball_extra_payment` column.
|
||
- `v0.49` payments: `balance_delta` column for debt payoff tracking.
|
||
- `v0.50` users: `last_seen_version` for release-notes notifications.
|
||
- `v0.51` user_login_history table.
|
||
- `v0.52`–`v0.53` ad-hoc columns (legacy reconcile).
|
||
- `v0.54` bills/categories: soft-delete columns (`deleted_at`).
|
||
- `v0.55` autopay: `auto_mark_paid` and suggestion dismissals table.
|
||
- `v0.56` bills: saved bill templates table.
|
||
- `v0.57` match_suggestion_rejections table (replaced by `v0.62`).
|
||
- `v0.58` import_sessions and import_history tables.
|
||
- `v0.59` payments: `payment_source` and `transaction_id` columns.
|
||
- `v0.60` data_sources, financial_accounts, transactions tables.
|
||
- `v0.61` payments: one active payment per linked transaction (unique index on `transaction_id`).
|
||
- `v0.62` match_suggestion_rejections table (canonical).
|
||
- `v0.63` bills: subscription metadata fields (`is_subscription`, `subscription_type`).
|
||
- `v0.64` financial_accounts: `monitored` flag for bill matching; transactions: partial unique index on `(data_source_id, provider_transaction_id)` and supporting indexes.
|
||
- `v0.65` subscription_catalog: top-200 known subscription services.
|
||
- `v0.66` declined_subscription_hints: per-user dismissed recommendation store.
|
||
- `v0.67` bill_merchant_rules: persistent merchant→bill auto-match rules.
|
||
- `v0.68` advisory_non_bill_filters: 5k advisory patterns + bill-like override terms.
|
||
- `v0.69` subscription_catalog v2: 90 new services + category fixes.
|
||
- `v0.70` monthly_bill_state: `snoozed_until` for overdue command center.
|
||
- `v0.71` bills: `drift_snoozed_until`; users: `notify_amount_change`.
|
||
- `v0.72` bills: persistent tracker sort order.
|
||
- `v0.73` snowball_plans table for plan lifecycle + history.
|
||
- `v0.74` subscription_catalog: Claude.ai Anthropic matching.
|
||
- `v0.75` categories: persistent sort order.
|
||
- `v0.76` bills: canonical billing schedule cleanup.
|
||
- `v0.77` encrypt SMTP password at rest.
|
||
- `v0.78` re-encrypt secrets from SHA-256 to HKDF key derivation.
|
||
- `v0.79` encrypt OIDC client secret at rest.
|
||
- `v0.80` users: push notification columns (ntfy / Gotify / Discord / Telegram).
|
||
- `v0.81` bill_merchant_rules: composite index on `(user_id, bill_id)`.
|
||
- `v0.82` payments: normalise `auto_match` source to `provider_sync`.
|
||
- `v0.83` bill_merchant_rules: `auto_attribute_late` flag.
|
||
- `v0.84` user_login_history: encrypt ip/useragent at rest + add `location` + keep 10 records.
|
||
- `v0.85` user_login_history: failed-attempt tracking + `session_fingerprint`.
|
||
- `v0.86` users: TOTP/authenticator 2FA columns + `totp_challenges` table.
|
||
- `v0.87` spending: category assignment on transactions + rules + budgets + default categories.
|
||
- `v0.88` categories: `spending_enabled` flag to separate bill vs spending categories.
|
||
- `v0.89` categories: seed spending defaults for users who had existing categories before `v0.87`.
|
||
- `v0.90` re-normalize merchant rules after `&` fix; ensure rejection expiry column.
|
||
- `v0.91` performance: composite indexes on `user_id+deleted_at` for categories, bills, payments.
|
||
- `v0.92` auth: WebAuthn/FIDO2 security key support — `webauthn_credentials` + `webauthn_challenges` tables.
|
||
- `v0.93` bills: `interest_accrued_month`; payments: `interest_delta`; transactions: stable provider key + dedupe index.
|
||
- `v0.94` security: session token hashing + geolocation opt-in setting (seeded into `settings`).
|
||
- `v0.95` subscription_catalog: bank descriptors + pricing from 2026 researched dataset.
|
||
- `v0.96` bills: `catalog_id` FK; `user_catalog_descriptors` for custom bank descriptors.
|
||
- `v0.97` subscription recommendation feedback: per-user learning signals.
|
||
- `v0.98` payments: `accounting_excluded` for bank-override metadata.
|
||
- `v0.99` bills: autopay trust indicators (`autopay_verified_at`, `inactive_reason`) + lifecycle fields; payments: autopay failure flag.
|
||
- `v1.00` calendar feed subscription tokens (`calendar_tokens`).
|
||
- `v1.01` transactions: `pending` flag for SimpleFIN pending transactions.
|
||
- `v1.02` users: per-user geolocation opt-in (was global admin setting).
|
||
- `v1.03` money columns: dollars (REAL) → integer cents for `bills.expected_amount/current_balance/minimum_payment`, `payments.amount/balance_delta/interest_delta`, `monthly_bill_state.actual_amount`, `monthly_income.amount`, plus relevant index updates.
|
||
- `v1.04` `bill_templates.data` JSON: money fields dollars → integer cents (column-only `v1.03` does not touch the JSON blob; `serializeTemplateData` reads cents).
|
||
- `v1.05` `merchant_store_matches`: 5k merchant/store matching pack for bank transaction categorization.
|
||
- `v1.06` `category_groups`: organize spending categories into named groups.
|
||
- Unversioned user notification columns are also reconciled by `legacyReconcileMigrations.js`.
|
||
|
||
Migration logging is both console-based and audit-backed:
|
||
|
||
- `runMigrations()` logs start, dependency status, transaction begin/commit, per-migration elapsed time, skips for already-applied migrations, failures with rollback messages, and total elapsed time.
|
||
- Audit events use the lazy `getLogAudit()` helper to avoid the `auditService -> database.js -> auditService` circular dependency.
|
||
- Audit actions include `migration.start`, `migration.complete`, and `migration.failure`.
|
||
- Rollback paths audit `migration.rollback` and `migration.rollback.failure`.
|
||
|
||
Rollback support is defined by `ROLLBACK_SQL_MAP` in `db/database.js` and currently covers `v0.44` through `v1.04` (the most recent migrations `v1.05` and `v1.06` are **not** yet rollbackable):
|
||
|
||
- `v0.44` — drops selected performance indexes: `idx_bills_user_name`, `idx_payments_method`, `idx_monthly_starting_amounts_user`, and `idx_import_history_imported_at`.
|
||
- `v0.45` — drops `idx_audit_log_user`, `idx_audit_log_action`, and the `audit_log` table.
|
||
- `v0.46` — drops `bills.cycle_day` and `bills.cycle_type`.
|
||
- `v0.47` — drops `current_balance`, `minimum_payment`, `snowball_order`, `snowball_include`, `snowball_exempt`, `auto_mark_paid`, `deleted_at` columns from bills.
|
||
- `v0.48` — drops `snowball_extra_payment` column from users.
|
||
- `v0.49` — drops `balance_delta` column from payments.
|
||
- `v0.50` — drops `last_seen_version` column from users.
|
||
- `v0.51` — drops `user_login_history` table.
|
||
- `v0.54` — removes soft-delete columns (`deleted_at`) from bills and categories.
|
||
- `v0.55` — drops autopay suggestion dismissals table.
|
||
- `v0.56` — drops `bill_templates` table.
|
||
- `v0.57` — drops `match_suggestion_rejections` table.
|
||
- `v0.58` — drops `import_sessions` and `import_history` tables.
|
||
- `v0.59` — drops `payment_source` and `transaction_id` columns from payments.
|
||
- `v0.60` — drops `data_sources`, `financial_accounts`, and `transactions` tables.
|
||
- `v0.61` — drops unique index on `transaction_id` from payments.
|
||
- `v0.62` — drops `match_suggestion_rejections` table.
|
||
- `v0.63` — drops partial unique index on `data_sources`.
|
||
- `v0.64` — drops partial unique index and indexes on `transactions`.
|
||
- `v0.72` — drops `bills.sort_order` and `idx_bills_user_sort`.
|
||
- `v0.75` — drops `categories.sort_order` and `idx_categories_user_sort`.
|
||
- `v0.76` — drops `bills.billing_cycle` (canonical cleanup).
|
||
- `v0.77`–`v0.79` — revert SMTP/OIDC secret encryption (re-store plaintext).
|
||
- `v0.81` — drops `idx_bill_merchant_rules_user_bill`.
|
||
- `v0.82` — no-op rollback (data migration).
|
||
- `v0.83` — drops `bill_merchant_rules.auto_attribute_late`.
|
||
- `v0.84`–`v0.85` — revert `user_login_history` encryption and drop `success`/`session_fingerprint`/`location`.
|
||
- `v0.86` — drops `users.totp_*` columns and `totp_challenges`.
|
||
- `v0.87` — drops `transactions.spending_category_id`, `spending_rules`, `spending_budgets`, related defaults.
|
||
- `v0.88` — drops `categories.spending_enabled`.
|
||
- `v0.89` — data-only (seeding); no schema rollback.
|
||
- `v0.90` — data-only (re-normalization); no schema rollback.
|
||
- `v0.91` — drops `idx_categories_user_deleted`, `idx_bills_user_deleted`, `idx_payments_user_deleted`.
|
||
- `v0.92` — drops `webauthn_credentials`, `webauthn_challenges`, and the `webauthn_*` columns on `users`.
|
||
- `v0.93` — drops `bills.interest_accrued_month`, `payments.interest_delta`, transactions dedupe index + indexes.
|
||
- `v0.94` — reverts session token hashing (drops the hash; cookie value is now the raw ID again); removes `geolocation_enabled` setting row.
|
||
- `v0.95` — drops `subscription_catalog.bank_descriptors` / `subcategory` / `starting_monthly_usd`.
|
||
- `v0.96` — drops `bills.catalog_id` and `user_catalog_descriptors` table.
|
||
- `v0.97` — drops `subscription_recommendation_feedback` table.
|
||
- `v0.98` — drops `payments.accounting_excluded`.
|
||
- `v0.99` — drops `bills.autopay_verified_at`/`inactive_reason` (and any payments autopay-failure column added in the same migration).
|
||
- `v1.00` — drops `calendar_tokens` table.
|
||
- `v1.01` — drops `transactions.pending`.
|
||
- `v1.02` — drops `users.geolocation_enabled` and re-seeds the global admin setting key.
|
||
- `v1.03` — reverts the integer-cents conversion (back to dollar REALs) for the listed columns.
|
||
- `v1.04` — reverts `bill_templates.data` money fields back to dollars.
|
||
|
||
`rollbackMigration(version)` requires an initialized database, verifies the version exists in `schema_migrations`, looks up rollback SQL in `ROLLBACK_SQL_MAP`, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns `{success:true, version, description, elapsed_ms}`. If the migration is not recorded, it throws `NOT_APPLIED`. If no rollback definition exists, it throws `ROLLBACK_NOT_SUPPORTED`. Execution failures roll back the transaction and are audited as `migration.rollback.failure`.
|
||
|
||
The admin API exposes rollback through `POST /api/admin/migrations/rollback`. The route requires admin auth through the `/api/admin` mount. It maps `NOT_APPLIED` to HTTP 404, `ROLLBACK_NOT_SUPPORTED` to HTTP 422, and unexpected rollback failures to HTTP 500.
|
||
|
||
The lazy audit helper in `db/database.js` is:
|
||
|
||
```javascript
|
||
let _logAudit = null;
|
||
function getLogAudit() {
|
||
if (!_logAudit) {
|
||
try { _logAudit = require('../services/auditService').logAudit; } catch { _logAudit = () => {}; }
|
||
}
|
||
return _logAudit;
|
||
}
|
||
```
|
||
|
||
Use this pattern for database-layer audit calls instead of a top-level `require('../services/auditService')`.
|
||
|
||
---
|
||
|
||
## 7. Frontend Reference
|
||
|
||
### Frontend stack
|
||
|
||
- React `^19.2.7` (with `babel-plugin-react-compiler` for automatic memoization)
|
||
- TypeScript `^6.0.3` with `strict: true` + `noUncheckedIndexedAccess: true`; `allowJs: true`, `checkJs: false` for the gradual JS → TS migration
|
||
- Vite `^5.4.10` with `@vitejs/plugin-react` and `vite-plugin-pwa` (autoUpdate service worker, manifest, runtime caching for tracker/bills/calendar/summary/analytics/snowball/categories)
|
||
- React Router `^6.26.2`
|
||
- TanStack Query `^5.100.9` + `@tanstack/react-query-devtools` + `@tanstack/react-virtual` for the virtualized Subscriptions list
|
||
- Tailwind CSS `^3.4.14` (content glob: `./client/**/*.{js,jsx,ts,tsx}` — must include `ts,tsx` or `.tsx`-only classes generate no CSS)
|
||
- shadcn/ui component primitives, backed by Radix UI where applicable
|
||
- Sonner toast notifications via `sonner`
|
||
- `framer-motion` `^12.40.0` for transitions
|
||
- `react-markdown`, `remark-gfm`, `rehype-sanitize` for markdown rendering
|
||
- `vitest` `^4.1.8` + `@testing-library/react` for client unit tests (server tests stay on `node --test`)
|
||
|
||
### `client/main.tsx`
|
||
|
||
Creates the React root and wraps `App` with global providers. Theme is provided by `client/contexts/ThemeContext.tsx`.
|
||
|
||
### `client/App.tsx`
|
||
|
||
- Creates a TanStack `QueryClient` with:
|
||
- Global `QueryCache.onError` — toasts a `sonner` error when a *background* refetch fails while stale data is on screen. Pages render inline errors on the initial load so the toast only fires for refetches.
|
||
- `staleTime: 2 * 60 * 1000`, `retry: 1`, `refetchOnWindowFocus: false`.
|
||
- Uses lazy loading and `Suspense` with `PageLoader` for most pages.
|
||
- Wraps route elements in `ErrorBoundary`; pages further wrap in `PageTransition`.
|
||
- Exposes `ReactQueryDevtools`.
|
||
- Provides skip link for keyboard users.
|
||
- Mounts the `CommandPalette` and `ReleaseNotesDialog` globally.
|
||
- `RequireAuth` (`{children, role}`) behavior:
|
||
- Shows loading while auth state is `undefined`.
|
||
- Single-user mode bypasses the auth gate for `role: 'user'`.
|
||
- Redirects unauthenticated users to `/login` (preserves `state.from`).
|
||
- Default admin cannot access user routes — redirected to `/admin`.
|
||
- Role mismatches redirect to `/admin` (for admin role) or `/` (for user role).
|
||
- `AdminShell` wraps admin routes so the user-shell nav does not appear inside admin.
|
||
|
||
Routes (all rendered through `Routes` with `ErrorBoundary`, lazy + `Suspense` for non-critical pages):
|
||
|
||
- `/login` → `LoginPage` (public; supports local + OIDC + TOTP challenge)
|
||
- `/about` → `AboutPage` (public)
|
||
- `/privacy` → `PrivacyPage` (public)
|
||
- `/release-notes` → `ReleaseNotesPage` (public)
|
||
- `/admin` → `AdminPage` (admin only)
|
||
- `/admin/about` → admin shell + `AboutPage admin` (admin only)
|
||
- `/admin/roadmap` → admin shell + `RoadmapPage` (admin only)
|
||
- `/admin/status` → admin shell + `StatusPage` (admin only)
|
||
- `/status` → redirects admin to `/admin/status`
|
||
- `/` (index under user layout) → `TrackerPage`
|
||
- `/calendar` → `CalendarPage`
|
||
- `/summary` → `SummaryPage`
|
||
- `/bills` → `BillsPage`
|
||
- `/subscriptions` → `SubscriptionsPage` (virtualized list, optimistic reorder, drag-reorder)
|
||
- `/subscriptions/catalog` → `SubscriptionCatalogPage` (admin view of the catalog)
|
||
- `/categories` → `CategoriesPage`
|
||
- `/health` → `HealthPage` (bill + payment health snapshot)
|
||
- `/analytics` → `AnalyticsPage`
|
||
- `/settings` → `SettingsPage`
|
||
- `/snowball` → `SnowballPage` (projection + plans)
|
||
- `/payoff` → `PayoffPage` (payoff scenarios)
|
||
- `/spending` → `SpendingPage` (categorized transactions, rules, budgets, income)
|
||
- `/bank-transactions` → `BankTransactionsPage` (the bank ledger, `Cents`-based)
|
||
- `/data` → `DataPage` (export, CSV/OFX/XLSX/SQLite imports, bank sync, transaction matching)
|
||
- `/profile` → `ProfilePage`
|
||
- `*` → `NotFoundPage`
|
||
|
||
### API client
|
||
|
||
`client/api.ts` (TypeScript; converted from `api.js` in the TypeScript migration):
|
||
|
||
- `_fetch<T>(method, path, body, opts)` calls `/api${path}` with JSON headers and `credentials: include`.
|
||
- Reads CSRF token from the `bt_csrf_token` cookie **or** fetches once from `/api/auth/csrf-token` and caches the token in memory; sends `x-csrf-token` on `POST`/`PUT`/`PATCH`/`DELETE`.
|
||
- Non-OK responses throw an `Error` whose `status`/`data`/`details`/`code` match the server's structured `errorFormatter`.
|
||
- `ApiError` interface carries those fields; `TogglePaidResult` and `PaymentRecord` provide typed payment responses; `QueryParams` types the query-string builder.
|
||
- Exposes grouped functions for `auth`, `admin`, `notifications`, `profile`, `tracker`, `calendar`, `summary`, `bills`, `payments`, `categories`, `settings`, `analytics`, `status`, `version`/`about`, `import`, `export`, `data-sources`, `transactions`, `matches`, `snowball`, `subscriptions`, `spending`, `monthlyStartingAmounts`, `csvImport`, `matchSuggestions`.
|
||
- File download/upload endpoints use raw `fetch` because responses/bodies are blobs or octet streams.
|
||
- **All 16 import sites** that referenced `@/api.js` were normalized to `@/api` so Vite/TS resolve the `.ts`.
|
||
|
||
### Domain types — `client/types.ts`
|
||
|
||
- Branded `Dollars` and `Cents` money types (from `client/lib/money.ts`).
|
||
- `Bill`, `Payment`, `Category`, `TrackerResponse` (with `summary` / `bank_tracking` / `cashflow` / `rows`).
|
||
- `DriftBill`, `AutopaySuggestion`, `AmountSuggestion`, `AutopayStats`.
|
||
- `BankTransaction` (integer cents on the wire).
|
||
- `TimelineBill` for the Safe-to-Spend SVG timeline.
|
||
- Money fields are typed as `Dollars` for bill/payment/tracker/summary amounts and as `Cents` for raw bank transactions — so the unit mixup bug class is unrepresentable in typed code.
|
||
- A `money.type-test.ts` never-imported guard uses `@ts-expect-error` to assert each unit mixup is a real compile error, so `tsc --noEmit` fails loudly if the branding ever regresses.
|
||
|
||
### Auth state
|
||
|
||
`client/hooks/useAuth.tsx`:
|
||
|
||
- `AuthContext` typed (a `createContext(null)` would have made `useContext` return `never`).
|
||
- Maintains `user`, `singleUserMode`, `loading`.
|
||
- Calls `api.authMode()` and `api.me()` on startup.
|
||
- Exposes `logout()`, `logoutAll()`, `refresh()`.
|
||
- `singleUserMode` is sourced from `/auth/mode`; `RequireAuth` consults it.
|
||
|
||
### Query hooks
|
||
|
||
`client/hooks/useQueries.ts` (TypeScript; converted from `useQueries.js`):
|
||
|
||
- `useTracker(year, month)`, `useTrackerHealth(year, month)`, `usePrefetchTracker()` — warms the adjacent-month cache on hover/focus.
|
||
- `useBills()`, `useCategories()`, `useSnowball*`, `useSpending*`, `useSummary`, `useBankLedger`, `useCategoriesWithSpending`, `useSubscriptions`, `useSubscriptionCatalog`, `useSubscriptionRecommendations`, `useMatchSuggestions`, `useTransactions`, `useTransactionsCategory`, `useTransactionCategories`, `useBankDataSources`, `useMatchBillDialog`, `useBankAccountDataSources`, `useBankSyncConfig`, `useAdminUserList`, `useAdminOIDCTest`, `useAdminBackups`, `useAdminCleanup`, `useAdminMigrations`, `useAdminAuthMode`, `useAdminNotifications`, `useReleaseNotes`, `useLoginHistory`, `useSnowballPlans`, `useWebAuthnStatus`, `useTOTPStatus`, `useUpcomingBills`.
|
||
- All hooks typed (params + `QueryParams`-typed query builders) and exported from `api.ts`.
|
||
- React Query v5: caches use `gcTime` (not `cacheTime`, which the v5 rename silently ignored). All mutation hooks invalidate / set query data via `setQueryData<T>(key, updater)`.
|
||
- Mutations: `useQuickPay` (shared across desktop + mobile tracker rows; `isPending` replaces manual busy flags; tracker/badge caches invalidate on settle), `useTogglePaid`, `useMarkPaid`, `useMatch`, `useUnmatch`, `useIgnore`, `useUnignore`, `useCategorizeTransaction`, `useBillCreate`, `useBillUpdate`, `useBillDelete`, `useBillReorder`, `useBudgetPut`, `useBudgetCopy`, `useCategoryRuleCreate`, `useCategoryRuleDelete`, `usePlanCreate`, `usePlanUpdate`, `usePlanPause`, `usePlanResume`, `usePlanComplete`, `usePlanAbandon`, `useSubscriptionLink`, `useSubscriptionDecline`, `useSubscriptionCreate`, `useSubscriptionUpdate`, `useImportCsv`, `useImportOfx`, `useBankSyncNow`, etc.
|
||
- `useAutoSave<T>(value, options)` is a generic debounced-save hook with an `AutoSaveStatus` union (`idle` | `pending` | `saving` | `saved` | `error`).
|
||
- `usePaymentActions.ts` — shared quick-pay/toggle-paid mutation hooks; typed payloads with `Dollars` on money amounts.
|
||
- `useSearchPanelPreference.ts` — typed `[boolean, setter]` tuple.
|
||
|
||
### Pages
|
||
|
||
- `LoginPage.tsx` — local login, OIDC login button, TOTP challenge, WebAuthn assertion, recovery code entry.
|
||
- `TrackerPage.tsx` — monthly tracker, payment interactions, upcoming bills, starting-amount awareness, safe-to-spend, drift/autopay/amount-suggestion badges, bulk/session logout, month-nav prefetch.
|
||
- `CalendarPage.tsx` — calendar grid backed by `/calendar`.
|
||
- `SummaryPage.tsx` — monthly plan, income, starting amounts, expenses, chart, bank-tracking overlay, skip-override.
|
||
- `BillsPage.tsx` — bill CRUD, categories, monthly state/history range controls, soft-delete + recently-deleted restore, merchant rules manager, bill historical-import dialog.
|
||
- `CategoriesPage.tsx` — category list/create/update/delete, group/category split, sort order, spending flag.
|
||
- `AnalyticsPage.tsx` — analytics summary filters and charts.
|
||
- `SettingsPage.tsx` — user settings and demo data seed.
|
||
- `DataPage.tsx` — export, spreadsheet import, user DB import, import history, CSV transaction import with preview + commit flow (`ImportTransactionCsvSection`), OFX import, bank sync.
|
||
- `ProfilePage.tsx` — display name, notification preferences, password change, TOTP setup, WebAuthn enrollment, login history, export/import-history links.
|
||
- `AdminPage.tsx` — users, auth mode/OIDC, WebAuthn, TOTP, backups, cleanup, notifications, migrations, admin settings, bank-sync config.
|
||
- `StatusPage.tsx` — admin system status.
|
||
- `AboutPage.tsx` — public or admin markdown/about view; admin mode uses `/about-admin`; markdown is sanitized.
|
||
- `RoadmapPage.tsx` — admin roadmap.
|
||
- `ReleaseNotesPage.tsx` — release history display; the `ReleaseNotesDialog` is shown to the user on login when their `last_seen_version` is older than the current package version.
|
||
- `PrivacyPage.tsx` — public privacy text.
|
||
- `SubscriptionsPage.tsx` (1814 lines) — `useOptimistic<Subscription[]>` reducer, virtualizer's `FlatItem` discriminated union, drag-reorder typed via the shared `DragProps`/`MoveControls`.
|
||
- `SubscriptionCatalogPage.tsx` — admin view of the top-200 catalog with bank descriptors.
|
||
- `SnowballPage.tsx` — projection, settings, plans lifecycle.
|
||
- `PayoffPage.tsx` — payoff scenarios.
|
||
- `SpendingPage.tsx` — categorized transactions, rules, monthly budgets with month-rollover, income.
|
||
- `BankTransactionsPage.tsx` — the `Cents`-based bank ledger, with a local `Tx` structurally compatible with `BankTransaction` so it still satisfies `MatchBillDialog`.
|
||
- `HealthPage.tsx` — bill + payment health snapshot.
|
||
- `NotFoundPage.tsx` — 404 fallback.
|
||
|
||
### Components
|
||
|
||
- **Layout**: `Layout`, `Sidebar`, `BrandBlock`, `NavPill`, `CommandPalette`, `PageTransition`.
|
||
- **Admin**: full `components/admin/` directory — users, auth methods, OIDC, WebAuthn, TOTP, bank-sync, backup manager, cleanup, notifications, migrations, settings cards.
|
||
- **Bill modal**: full `components/bill-modal/` directory plus `BillModal.tsx` (1092-line add/edit form, reached from 9 entry points).
|
||
- **Tracker**: full `components/tracker/` directory (18 files — the home page) where `fmt(row.expected_amount)` type-checks against `Dollars`.
|
||
- **Transactions**: `components/transactions/` directory.
|
||
- **Snowball**: `components/snowball/` directory (plan list, projection, settings).
|
||
- **Data**: full `components/data/` directory (14 files) — the Data page's SimpleFIN connect/sync workbench, transaction matching, and CSV / OFX / XLSX-spreadsheet / SQLite import flows, with `Cents`-branded bank-transaction amounts flowing through.
|
||
- **Domain UI**: `BillModal`, `BillsTableInner`, `MobileBillRow`, `MobileTrackerRow`, `StatusBadge`, `SummaryCard`, `MarkdownText`, `ReleaseNotesDialog`, `IncomeBreakdownModal`, `BillHistoricalImportDialog`, `BillRulesManager`, `BillMerchantRules`, `RecentlyDeletedBillsDialog`, `CalendarFeedManager`, `SearchFilterPanel`, `SubscriptionCatalogSection`.
|
||
- **Reliability**: `ErrorBoundary` (typed class component), `PageLoader`, `PageTransition`.
|
||
- **UI primitives** (`components/ui/`): `alert-dialog`, `badge`, `button`, `card`, `checkbox`, `collapsible`, `confirm-dialog`, `dialog`, `dropdown-menu`, `input`, `input-dialog`, `label`, `save-status`, `select`, `separator`, `skeleton`, `sonner`, `switch`, `table`, `tabs`, `theme-toggle`, `tooltip` — all `.tsx`.
|
||
|
||
### `client/lib/`
|
||
|
||
- `money.ts` — branded `Cents`/`Dollars` types, `asCents`/`asDollars`/`centsToDollars`/`dollarsToCents`, formatters. A `formatUSD(dollars)` and `formatCentsUSD(cents)` split mirrors the server's `formatUSD`/`formatCentsUSD` split. Inputs are coerced defensively so null / `''` / undefined / NaN never render as `$NaN`.
|
||
- `utils.ts` — `cn`, `fmt`, date/format helpers. `fmt` inherits `formatUSD`'s branded-dollars input.
|
||
- `trackerUtils.ts` — row/status/sort helpers, `TrackerRow` domain interface, `TrackerStatus` union.
|
||
- `billDrafts.ts` — `SourceBill` / `Template` / `Category` shapes for `makeBillDraft`.
|
||
- `billingSchedule.ts` — `Schedule` union and helpers.
|
||
- `cashflowUtils.ts` — typed SVG timeline geometry for the Safe-to-Spend card.
|
||
- `trackerTableColumns.ts` — typed column descriptors.
|
||
- `reorder.ts` — generic `moveInArray<T>`.
|
||
- `version.ts` — typed release-notes shapes; the Vite-injected `__APP_VERSION__` is declared inline.
|
||
|
||
### Frontend build & lint
|
||
|
||
- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`).
|
||
- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`. Wired into `npm run ci`.
|
||
- `npm run lint` — `eslint client` (flat config; `react-hooks/rules-of-hooks`, `react-hooks/exhaustive-deps`, `react-refresh/only-export-components`, typescript-eslint parser for `.ts/.tsx`).
|
||
- `npm run test` — server `node --test tests/*.test.js`.
|
||
- `npm run test:client` — `vitest run` (client unit tests).
|
||
- `npm run test:all` — both.
|
||
- `npm run test:e2e` — Playwright (chromium desktop + mobile); `prepare-db.js` reseeds the test DB.
|
||
- `npm run check:server` — `node --check` on every server JS file.
|
||
- `npm run check` — `check:server` + `build`.
|
||
- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`.
|
||
|
||
### Frontend security notes
|
||
|
||
- CSRF header is sent on `POST`/`PUT`/`PATCH`/`DELETE`; the token is fetched once from `/api/auth/csrf-token` and cached in memory.
|
||
- Auth is cookie/session based; no tokens are stored in localStorage. Session token hashing (`v0.94`) means a leaked DB does not yield usable session IDs.
|
||
- Admin routes are client-guarded and server-guarded; the default admin is redirected away from user routes.
|
||
- Markdown rendering uses `rehype-sanitize`; CSP headers in `middleware/securityHeaders.js` cover the built assets.
|
||
- Error boundaries prevent route crashes from taking down the whole SPA.
|
||
- PWA: `vite-plugin-pwa` registers a service worker (autoUpdate) with manifest + icons. Runtime caching uses `NetworkFirst` for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` with a 5-second network timeout, 30-entry max, 5-minute max-age.
|
||
|
||
---
|
||
|
||
## 8. Infrastructure and Deployment
|
||
|
||
### `package.json`
|
||
|
||
Version: `0.40.0`. `"type": "commonjs"`.
|
||
|
||
Scripts:
|
||
|
||
- `npm run dev:api` — `node --watch server.js`.
|
||
- `npm run dev:ui` — Vite dev server.
|
||
- `npm run dev` — `concurrently` runs API and UI.
|
||
- `npm run build` — `vite build` (output: `dist/`, `emptyOutDir: true`).
|
||
- `npm run check:server` — `node --check` every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils`.
|
||
- `npm run lint` — `eslint client` (flat config; `react-hooks` + `react-refresh` + `typescript-eslint`).
|
||
- `npm run typecheck` — `tsc --noEmit -p tsconfig.json`.
|
||
- `npm run check` — `check:server` + `build`.
|
||
- `npm run test` — `node --test tests/*.test.js` (server).
|
||
- `npm run test:client` — `vitest run` (client).
|
||
- `npm run test:all` — both.
|
||
- `npm run test:e2e` / `test:e2e:ui` / `test:e2e:update` / `test:e2e:probe` — Playwright (chromium desktop + mobile + axe), with `e2e/setup/prepare-db.js` reseeding the test DB.
|
||
- `npm run smoke:prod` — `bash scripts/prod-smoke.sh`.
|
||
- `npm run ci` — `lint` + `typecheck` + `check:server` + `test:all` + `build`. Runs in Forgejo Actions on every push and PR (`.forgejo/workflows/ci.yml`, container `node:22-bookworm`).
|
||
- `npm start` — `node server.js`.
|
||
|
||
Key runtime dependencies:
|
||
|
||
- Express 4, cookie-parser, cors, express-rate-limit.
|
||
- better-sqlite3 12.
|
||
- bcryptjs.
|
||
- openid-client 5.
|
||
- @simplewebauthn/server 13, @simplewebauthn/browser 13.
|
||
- otplib 13, qrcode 1.
|
||
- nodemailer 8.
|
||
- node-cron 4.
|
||
- React 19, React DOM 19, React Router 6, TanStack Query 5, TanStack Virtual 3, framer-motion 12.
|
||
- shadcn/ui component primitives, Radix UI primitives, lucide-react, Tailwind utilities, Sonner toasts.
|
||
- xlsx for spreadsheet import/export.
|
||
|
||
Key dev dependencies:
|
||
|
||
- typescript 6, @types/react 19, @types/react-dom 19, typescript-eslint 8.
|
||
- eslint 9, @eslint/js, eslint-plugin-react-hooks 5, eslint-plugin-react-refresh 0.4, globals 17.
|
||
- vite 5, @vitejs/plugin-react 4, vite-plugin-pwa 1, babel-plugin-react-compiler 1.
|
||
- vitest 4, @testing-library/react 16, @axe-core/playwright 4, @playwright/test 1.
|
||
- tailwindcss 3, autoprefixer 10, postcss 8.
|
||
|
||
### Dockerfile
|
||
|
||
Multi-stage build:
|
||
|
||
1. Builder: `node:22-alpine`, installs build deps `python3 make g++`, runs `npm install`, copies source, runs `npm run build`.
|
||
2. Runtime: `node:22-alpine`, installs `bash nano su-exec`, creates non-root `bill` user, copies built app, creates `/data/db`, `/data/backups`, `/app/backups`, sets ownership and restrictive permissions.
|
||
|
||
Runtime environment:
|
||
|
||
- `NODE_ENV=production`
|
||
- `PORT=3000`
|
||
- `DB_PATH=/data/db/bills.db`
|
||
- `BACKUP_PATH=/data/backups`
|
||
|
||
Exposes port 3000, declares volume `/data`, entrypoint `docker-entrypoint.sh`, command `node server.js`.
|
||
|
||
### `docker-compose.yml`
|
||
|
||
Service: `bill-tracker`.
|
||
|
||
- Image: `dream.scheller.ltd/null/billtracker:latest`.
|
||
- Container name: `bill-tracker`.
|
||
- Ports: host `3030` to container `3000`.
|
||
- Volume: `/portainer/hosting/bill-tracker/data:/data`.
|
||
- Restart: `unless-stopped`.
|
||
- Environment includes `INIT_ADMIN_USER`, `INIT_ADMIN_PASS`, and CSRF cookie settings (`CSRF_HTTP_ONLY: "false"`, `CSRF_SAME_SITE: "strict"`, `CSRF_SECURE: "true"`, `CSRF_COOKIE_NAME: "bt_csrf_token"`).
|
||
|
||
Important deployment note: the compose file currently sets `CSRF_SECURE: "true"`; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to `false` only in local/dev.
|
||
|
||
### CI (`.forgejo/workflows/ci.yml`)
|
||
|
||
Forgejo Actions runs on every push and PR: `actions/checkout@v4` on `node:22-bookworm` (matches Dockerfile runtime so better-sqlite3 prebuilds resolve identically), then `npm ci` and `npm run ci`. The workflow is the only place a build is fully re-verified end-to-end (lint + typecheck + server tests + client tests + build).
|
||
|
||
### Vite (`vite.config.mjs`)
|
||
|
||
- `@vitejs/plugin-react` with `babel-plugin-react-compiler` enabled (React 19 auto-memoization). The codebase is rules-of-hooks clean (enforced by `eslint-plugin-react-hooks`).
|
||
- `vite-plugin-pwa` with `registerType: 'autoUpdate'`, manifest, icons in `client/public/img/`, runtime caching for `/api/(tracker|bills|calendar|summary|analytics|snowball|categories)` via `NetworkFirst` (5s network timeout, 30-entry max, 5-min max-age).
|
||
- `__APP_VERSION__` is injected at build time from `package.json`; the SPA reads it through `client/lib/version.ts`.
|
||
- Manual chunk splitting: `vendor-react` (react / react-dom / react-router-dom), `vendor-radix` (dialog / select / dropdown-menu / tabs / tooltip / alert-dialog), `vendor-motion` (framer-motion), `vendor-query` (@tanstack/react-query + @tanstack/react-virtual).
|
||
- `resolve.alias['@']` → `client/`, matches the Vite alias and the `tsconfig.json` `paths` mapping.
|
||
- `server.port: 5173`, proxies `/api` to `http://localhost:${API_PORT || PORT || 3000}`.
|
||
- `test.environment: 'node'`; tests opt into jsdom via `@vitest-environment` per-file.
|
||
- `test.include: ['client/**/*.test.{js,jsx}']`; **note**: client tests on `.ts` are currently run via the `tests/*.test.js` server suite (which still exercises the same behaviors). The Vitest `include` is restricted to `.js/.jsx` to keep current test coverage green during the gradual TS migration.
|
||
|
||
### TypeScript (`tsconfig.json`)
|
||
|
||
- `target: ES2022`, `lib: [ES2023, DOM, DOM.Iterable]`.
|
||
- `module: ESNext`, `moduleResolution: bundler`, `jsx: react-jsx`.
|
||
- `paths: { "@/*": ["./client/*"] }`, `resolveJsonModule`, `esModuleInterop`, `isolatedModules`, `skipLibCheck`, `noEmit`.
|
||
- `allowJs: true`, `checkJs: false` — legacy `.js`/`.jsx` keep resolving and running untouched; only `.ts`/`.tsx` are type-checked. Convert file-by-file.
|
||
- `strict: true`, `noUncheckedIndexedAccess: true`.
|
||
- `types: ["vite/client"]` — provides `import.meta.env` and `__APP_VERSION__` typings.
|
||
- `include: ["client/**/*"]`, `exclude: ["node_modules", "dist", "client/**/*.test.*"]`.
|
||
|
||
### ESLint (`eslint.config.mjs`)
|
||
|
||
Flat config scoped to the React client. The point is enforcement of the two rules that catch real React bugs — `rules-of-hooks` (conditional hooks) and `exhaustive-deps` (stale closures) — which nothing previously checked. Server code (CommonJS Node) is out of scope here; `npm run check:server` covers it.
|
||
|
||
- JS/JSX block (`client/**/*.{js,jsx}`): `js.configs.recommended` + `react-hooks` + `react-refresh`. `react-hooks/rules-of-hooks: error`, `react-hooks/exhaustive-deps: warn`, `react-refresh/only-export-components: warn` (with `allowConstantExport: true`).
|
||
- TS/TSX block (`client/**/*.{ts,tsx}`): same react rules via the `typescript-eslint` parser. `no-undef` and `no-unused-vars` off (TypeScript handles them); `@typescript-eslint/no-unused-vars: warn` with `^[A-Z_]` ignore pattern.
|
||
- `no-empty: warn` with `allowEmptyCatch: true`.
|
||
- Ignores: `dist/**`, `node_modules/**`, `coverage/**`, `client/**/*.test.*`.
|
||
|
||
---
|
||
|
||
## 9. Auth and Security Flows
|
||
|
||
### Local login
|
||
|
||
1. Client calls `GET /auth/mode` to determine local/OIDC visibility.
|
||
2. Client submits `POST /auth/login`.
|
||
3. Server checks `local_login_enabled`, validates credentials through bcrypt (cost 12), rejects inactive users, cleans expired sessions, creates a 7-day session, **hashes the session token** for storage (`v0.94`), and sets the `bt_session` cookie with the raw opaque ID.
|
||
4. Server sets `bt_session` cookie using `cookieOpts(req)`.
|
||
5. Client calls `GET /auth/me` to populate auth state.
|
||
|
||
### TOTP 2FA
|
||
|
||
1. User enables TOTP via `GET /auth/totp/setup` → scans QR code → `POST /auth/totp/enable` with `{secret, token}`. Server encrypts the secret with `services/encryptionService` and stores 8 recovery codes (hashed).
|
||
2. On subsequent logins, after a successful `POST /auth/login`, the server returns `{ totp_required: true, challenge_id }`. The SPA prompts for the 6-digit code and submits it (or a recovery code) with the challenge id.
|
||
3. `POST /auth/totp/challenge` accepts `{username, password}` and returns a 5-minute `challenge_id`; the SPA submits it with the token on the next request.
|
||
4. `disableTotp(userId)` clears the encrypted secret and recovery codes.
|
||
|
||
### WebAuthn 2FA
|
||
|
||
1. User enrolls a security key: `GET /auth/webauthn/status` (or the admin equivalent) returns current credentials. The SPA calls an internal helper that triggers a registration ceremony; the server stores the credential in `webauthn_credentials` (AAGUID, transports, backup flags, friendly name).
|
||
2. On login, the SPA requests an assertion via `webauthn_login` flow; `createAuthenticationChallenge` returns options + a 15-minute `challenge_id`; `verifyAuthentication` updates the sign count and audits `webauthn.login`.
|
||
3. `WEBAUTHN_RP_ID` (env, default `localhost`) and `WEBAUTHN_ORIGIN` (env, default `http://localhost:${PORT}`) must match the browser-visible hostname, or registration/assertion will fail.
|
||
|
||
### OIDC login
|
||
|
||
1. Client navigates to `/api/auth/oidc/login`.
|
||
2. Server verifies active OIDC config (the OIDC client secret is decrypted from `settings.oidc_client_secret`, encrypted at rest by `v0.79`), creates PKCE state, redirects to provider.
|
||
3. Provider returns to `/api/auth/oidc/callback`.
|
||
4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie (token hashed in `sessions.id`), redirects back into SPA.
|
||
|
||
### Password change
|
||
|
||
1. Current password and matching new password are required.
|
||
2. New password must be at least 8 chars.
|
||
3. Server updates hash, clears `must_change_password`, sets `last_password_change_at`.
|
||
4. Other sessions are invalidated and current session is rotated when possible.
|
||
|
||
### Login history + device fingerprint
|
||
|
||
- Every login (successful or failed) writes a `user_login_history` row.
|
||
- `ip_address` and `user_agent` are encrypted at rest (`v0.84`).
|
||
- `location` (IP-geolocation city/region/country) is captured only when the user has `users.geolocation_enabled = 1` (`v1.02`).
|
||
- `success` (`v0.85`) tracks failed-attempt history separately.
|
||
- `session_fingerprint` lets the SPA detect a different device for the same session and trigger a re-auth challenge.
|
||
|
||
### Authorization
|
||
|
||
- All user data routes enforce owner scope by `req.user.id` in SQL.
|
||
- Admin-only routes require `requireAdmin` on server.
|
||
- Default admin cannot use tracker routes.
|
||
- Role changes invalidate target sessions.
|
||
- Deactivation invalidates target sessions.
|
||
|
||
### Secret encryption at rest
|
||
|
||
- `services/encryptionService.js` provides `encryptSecret` / `decryptSecret` / `assertEncryptionReady` using a master key derived with HKDF (`v0.78`; the prior SHA-256 derivation was migrated in-place).
|
||
- Encrypted at rest: `oidc_client_secret` (`v0.79`), SMTP password (`v0.77`), `data_sources.encrypted_secret` (SimpleFIN access URLs), `user_login_history.ip_address` / `user_agent` (`v0.84`), `users.totp_secret` (`v0.86`).
|
||
- `reEncryptWithEnvKey()` is invoked by the migration runner when an admin sets `ENCRYPTION_KEY` after install; called from the runtime error handler when a legacy ciphertext is detected.
|
||
|
||
### Backup safety
|
||
|
||
- Managed filename regex and path checks prevent traversal.
|
||
- Uploads are written to temp paths first, validated by SQLite `integrity_check` + optional SHA-256 checksum, then moved.
|
||
- Restore creates a pre-restore backup.
|
||
|
||
### Import safety
|
||
|
||
- Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions.
|
||
- User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership.
|
||
- CSV import preview/apply is split; ambiguous rows are surfaced in `errors` for the user to resolve.
|
||
- OFX/QFX import is rate-limited via the import limiter; previews have a 24-hour TTL.
|
||
|
||
### Bank sync safety
|
||
|
||
- SimpleFIN access URLs are encrypted at rest (`data_sources.encrypted_secret`).
|
||
- `assertEncryptionReady` is called before any secret operation.
|
||
- The `monitored` flag (`v0.64`) lets users exclude a credit-card account from bill matching.
|
||
- Transactions are deduplicated by `provider_transaction_id` (unique partial index).
|
||
- Account balance updates run in transactions; partial failures roll back.
|
||
|
||
---
|
||
|
||
## 10. Operational Notes
|
||
|
||
### Startup behavior
|
||
|
||
- DB path is `DB_PATH` or `db/bills.db`.
|
||
- DB open logging prints only `path.basename(DB_PATH)` instead of the full database path, so startup logs identify the file without exposing the full filesystem location.
|
||
- SQLite WAL and foreign keys are enabled.
|
||
- Schema and migrations run automatically (`db/migrations/versionedMigrations.js` + `db/migrations/legacyReconcileMigrations.js`).
|
||
- Default categories/settings are seeded (incl. spending defaults for new users in `v0.87`).
|
||
- Expired sessions are purged at startup; a periodic `SESSION_CLEANUP_INTERVAL_MS` cleanup interval is scheduled.
|
||
- `bankSyncWorker.start()` is started; `backupScheduler` is started; `workers/dailyWorker.start()` runs the daily jobs.
|
||
- First-run flow: if `userCount === 0` after migrations, `setup/firstRun.run(db)` is invoked.
|
||
|
||
Environment-seeded regular users use `INIT_REGULAR_USER` and `INIT_REGULAR_PASS`. New seeded users are inserted with `first_login = 0` and `must_change_password = 0`. Existing seeded regular users have their password hash updated and both flags reset to `0`, then the server audits `seed.flag_reset` with the username, reset flags, and `source: "server-seed"`. This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes.
|
||
|
||
### Environment variables
|
||
|
||
Common variables used by current code:
|
||
|
||
- `PORT` — server port (default 3000)
|
||
- `BIND_HOST` — server bind (default `0.0.0.0`)
|
||
- `DB_PATH`
|
||
- `BACKUP_PATH`
|
||
- `INIT_ADMIN_USER`
|
||
- `INIT_ADMIN_PASS`
|
||
- `INIT_REGULAR_USER`
|
||
- `INIT_REGULAR_PASS`
|
||
- `SESSION_CLEANUP_INTERVAL_MS`
|
||
- `TRUST_PROXY` — `true` / `1` / numeric / `loopback` / CIDR; required behind nginx/Traefik
|
||
- `CORS_ORIGIN` — comma-separated allowlist
|
||
- `COOKIE_SECURE`
|
||
- `HTTPS`
|
||
- `CSRF_HTTP_ONLY`
|
||
- `CSRF_SAME_SITE`
|
||
- `CSRF_SECURE`
|
||
- `CSRF_COOKIE_NAME`
|
||
- `OIDC_ISSUER_URL`
|
||
- `OIDC_CLIENT_ID`
|
||
- `OIDC_CLIENT_SECRET` — encrypted at rest (`v0.79`); the settings-table copy is the source of truth
|
||
- `OIDC_REDIRECT_URI`
|
||
- `OIDC_TOKEN_AUTH_METHOD`
|
||
- `WEBAUTHN_RP_ID` — default `localhost`
|
||
- `WEBAUTHN_ORIGIN` — default `http://localhost:${PORT}`
|
||
- `ENCRYPTION_KEY` — master key for at-rest encryption (HKDF, `v0.78`)
|
||
- `API_PORT` — Vite dev-server proxy target (defaults to `PORT || 3000`)
|
||
- `DATA_IMPORT_ENABLED` — gates CSV import (defaults to true)
|
||
|
||
Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the `settings` table and managed from Admin UI.
|
||
|
||
### Known code characteristics to preserve
|
||
|
||
- Use transactions for multi-step destructive or bulk DB changes.
|
||
- Keep user-owned SQL scoped by `req.user.id`.
|
||
- Keep admin lockout protection before changing login methods.
|
||
- Do not expose `password_hash`, session IDs (raw or hashed), OIDC client secret, encrypted secret blobs, or internal backup paths in API responses.
|
||
- Keep import preview/apply separated so users can resolve ambiguous spreadsheet / CSV / OFX / user-DB data before DB writes.
|
||
- Branded `Dollars` / `Cents` types in `client/lib/money.ts` are the source of truth for client money handling. A bare number at a `fmt()` / `formatUSD` / `formatCentsUSD` boundary is a compile error in typed code. The `money.type-test.ts` guard uses `@ts-expect-error` so a regression in the branding fails `tsc --noEmit`.
|
||
- Tailwind's `content` glob must include `ts,tsx` (`./client/**/*.{js,jsx,ts,tsx}`). Without it, classes used only in `.tsx` files generate no CSS and produce a runtime-only failure that the e2e probe (not typecheck/build/tests) is the only gate that catches.
|
||
- DB path support: `db/database.js` uses `path.basename(DB_PATH)` in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported.
|
||
- React Compiler (`babel-plugin-react-compiler`) requires a rules-of-hooks-clean codebase; do not introduce conditional hooks.
|
||
- React Query v5: use `gcTime`, not `cacheTime` (silently ignored in v5).
|
||
|
||
---
|
||
|
||
## 11. Verification Checklist Used for This Reference
|
||
|
||
Reviewed current code sources:
|
||
|
||
- `server.js` (including `TRUST_PROXY` handling, health probe, route mount order, post-`listen` worker startup).
|
||
- All 26 route files under `routes/`.
|
||
- All 41 service files under `services/` (incl. `bankSync*`, `simplefinService`, `ofxImportService`, `merchantStoreMatchService`, `advisoryFilterService`, `totpService`, `webauthnService`, `loginFingerprint`, `driftService`, `aprService`, `paymentAccountingService`, `encryptionService`, `calendarFeedService`).
|
||
- All middleware files (`csrf`, `errorFormatter`, `rateLimiter`, `requireAuth`, `securityHeaders`).
|
||
- `db/schema.sql`, `db/database.js`, `db/migrations/versionedMigrations.js`, `db/migrations/legacyReconcileMigrations.js`, `db/subscriptionCatalogSeed.js`.
|
||
- Actual initialized SQLite schema via `better-sqlite3` introspection (`PRAGMA table_info`, `PRAGMA index_list`).
|
||
- `client/main.tsx`, `client/App.tsx`, `client/api.ts`, `client/types.ts`, `client/contexts/ThemeContext.tsx`.
|
||
- `client/hooks/useAuth.tsx`, `client/hooks/useQueries.ts`, `client/hooks/usePaymentActions.ts`, `client/hooks/useAutoSave.ts`, `client/hooks/useSearchPanelPreference.ts`.
|
||
- Page inventory under `client/pages/` (24 files) and component inventory under `client/components/` (incl. `admin/`, `bill-modal/`, `tracker/`, `transactions/`, `snowball/`, `data/`, `layout/`, `ui/`).
|
||
- `client/lib/` (`money.ts`, `utils.ts`, `trackerUtils.ts`, `billDrafts.ts`, `billingSchedule.ts`, `cashflowUtils.ts`, `trackerTableColumns.ts`, `reorder.ts`, `version.ts`).
|
||
- `package.json` (version 0.40.0; `"type": "commonjs"`), `tsconfig.json`, `eslint.config.mjs`, `vite.config.mjs`.
|
||
- `Dockerfile` (node:22-alpine), `docker-compose.yml`, `docker-entrypoint.sh`, `.forgejo/workflows/ci.yml`.
|
||
|
||
End-to-end checks run against the codebase:
|
||
|
||
- `find client -name '*.jsx'` returns 0 — the JSX → TSX migration is complete.
|
||
- `find client -name '*.ts'` and `find client -name '*.tsx'` together cover every previously `.jsx` file in `components/`, `pages/`, `hooks/`, `contexts/`, plus the API client and `lib/`.
|
||
- `grep` on `package.json` for React 19, TypeScript 6, Vite 5, TanStack Query 5, `@simplewebauthn/*` 13, `otplib` 13.
|
||
- `npm run typecheck` was green at the time of the v0.41.0 entry in `HISTORY.md` (0 errors); 48 client unit tests pass; 17/17 e2e probe (every page renders, all API paths respond).
|
||
- `node --check` on every server JS file under `server.js`, `db`, `middleware`, `routes`, `services`, `utils` (`npm run check:server`).
|
||
|
||
The previous manual (v0.28.1) contained stale route/page descriptions and missed major features added between v0.28 and v0.40 (subscriptions, spending, snowball plans, SimpleFIN, WebAuthn, TOTP, push notifications, category groups, calendar feed, money-cents migration in progress, React 18 → 19 + TypeScript migration, branded money types). This version replaces that with a current-state engineering reference.
|