null/BillTracker
null
/
BillTracker
1
0
Fork 0
Code Issues 4 Pull Requests Packages Projects Releases 1 Wiki Activity
BillTracker/docs/Engineering_Reference_Manua...

72 KiB
Raw Blame History

Engineering Reference Manual — Bill Tracker

Status: Current code reference Last Updated: 2026-05-16 Version: 0.28.1 Primary stack: Node.js + Express, React + Vite, Tailwind CSS + shadcn/ui, Sonner, SQLite via better-sqlite3

This manual reflects the current application code in server.js, routes/, services/, middleware/, db/, client/, package.json, Dockerfile, and docker-compose.yml. It is written as a current-state reference, not a changelog.

1. System Overview

Bill Tracker is a self-hosted bill management application. It supports:

  • Local username/password authentication, optional single-user mode, and optional Authentik/OIDC login.
  • User-scoped bills, categories, payments, monthly bill overrides, monthly income, and starting cash buckets.
  • Admin user management, backup/restore, cleanup, auth-mode/OIDC configuration, status checks, and migration rollback.
  • Spreadsheet and user-SQLite import workflows.
  • CSV, Excel, and user-SQLite export workflows.
  • SMTP-based bill due notifications.
  • React SPA frontend with protected user/admin routes and CSRF-protected JSON APIs.

Runtime flow:

  1. server.js initializes SQLite through db/database.js, runs schema/migrations, seeds defaults/admin user, cleans expired sessions, then starts Express.
  2. Express applies security headers, JSON body parsing, cookies, CSRF token provisioning, route-level CSRF/auth/rate-limit middleware, static dist/ serving, and JSON error formatting.
  3. Route files under routes/ validate input, enforce ownership through req.user.id, and use service modules for business logic.
  4. React under client/ calls /api/* through client/api.js with credentials: include and CSRF headers on mutating methods.

2. Project Layout

  • server.js — Express entry point and route mounting.
  • routes/ — HTTP API handlers:
    • auth.js — login, logout, password change, OIDC callback.
    • bills.js — bills CRUD, auto-mark paid, history.
    • payments.js — payments CRUD, status matching, snowball handling.
    • categories.js — category CRUD, tree support.
    • tracker.js — monthly tracker data, bucket resolution, cycle handling.
    • summary.js — summary stats, starting amounts.
    • analytics.js — expense reports, category breakdown.
    • settings.js — user settings, admin config, notification settings.
    • notifications.js — notification management.
    • profile.js — user profile, demo data.
    • import.js — CSV, Excel, user-SQLite import workflows.
    • export.js — CSV, Excel, SQLite export.
    • status.js — system status, health.
    • dataSources.js — new — data sources CRUD with sync status.
    • transactions.js — new — transaction CRUD, match/ignore/commit actions.
    • matches.js — new — match suggestions, rejection tracking.
  • services/ — business logic modules.
  • middleware/ — auth guards, CSRF, rate limits, security headers, error formatting.
  • db/schema.sql — base SQLite schema.
  • db/database.js — DB connection, migrations, defaults, settings, rollback support.
  • workers/dailyWorker.js — scheduled notification/cleanup worker entry.
  • client/ — React SPA.
  • dist/ — generated Vite build output served by Express.
  • Dockerfile, docker-compose.yml, docker-entrypoint.sh — container deployment.

3. Backend Entry Point and Middleware

server.js

Server defaults:

  • Port: PORT || 3000.
  • Static frontend: dist/.
  • Optional CORS: enabled when CORS_ORIGIN is set; credentials allowed.
  • Session cleanup: runs on startup and every SESSION_CLEANUP_INTERVAL_MS || 86400000 ms.
  • Admin seed: INIT_ADMIN_USER || admin; INIT_ADMIN_PASS or generated/default behavior in db/database.js.
  • Environment variables INIT_ADMIN_USER and INIT_ADMIN_PASS (or INIT_REGULAR_USER + INIT_REGULAR_PASS) skip the first-login flow entirely by pre-seeding users with first_login=0 flags via setup/firstRun.js.
  • Optional regular-user seed: INIT_REGULAR_USER + INIT_REGULAR_PASS; password must be at least 8 chars.

Global middleware order:

  1. securityHeaders
  2. optional cors
  3. express.json()
  4. cookieParser()
  5. csrfTokenProvider
  6. mounted API routers with route-level rate-limit/auth/CSRF middleware
  7. static legacy/, redirect /login.html to /login, static dist/
  8. SPA fallback GET * serving dist/index.html after ensuring a CSRF token cookie
  9. errorFormatter
  10. final JSON error handler for malformed JSON/body size/runtime errors

Authentication middleware

middleware/requireAuth.js exports:

  • requireAuth: attaches req.user from bt_session; in single-user mode attaches the configured active regular user without a session.
  • requireUser: permits user and admin roles but blocks the default admin account from tracker access.
  • requireAdmin: requires req.user.role === 'admin'.

CSRF middleware

middleware/csrf.js:

  • Cookie name: CSRF_COOKIE_NAME || bt_csrf_token.
  • Header name: x-csrf-token.
  • Defaults: CSRF_HTTP_ONLY === true only when explicitly configured, CSRF_SAME_SITE || strict, CSRF_SECURE !== false.
  • The SPA expects CSRF_HTTP_ONLY=false so client/api.js can read the token cookie and send x-csrf-token; do not enable httpOnly CSRF cookies unless token delivery changes.
  • csrfTokenProvider sets a token cookie on responses.
  • csrfMiddleware validates mutating requests unless req.csrfSkip is set. Token may come from header, query, or body and must match cookie.
  • Failures return 403 and audit csrf.failure.

Rate limits

Defined in middleware/rateLimiter.js:

  • Login: 10 / 15 min.
  • Password changes: 5 / 15 min.
  • Import: 20 / 15 min.
  • Export: 30 / 15 min.
  • Admin actions: 30 / 15 min.
  • OIDC: 20 / 15 min.
  • Backup operations: 5 / 60 min.
  • Demo-data clear: 3 / 15 min.

Rate-limit responses are JSON: { "error": "..." }.

Security headers

middleware/securityHeaders.js sets CSP with a per-request nonce plus common hardening headers. Static SPA scripts/styles must comply with the generated CSP.

Error formatting

middleware/errorFormatter.js standardizes route responses into JSON with error, code, and optional field. Common statuses: 400 validation, 401 auth, 403 forbidden, 404 not found, 409 conflict, 429 rate limit, 500 server error.

4. Services

services/authService.js

  • Cookie: bt_session.
  • Session lifetime: 7 days.
  • Password hashing: bcrypt, cost 12.
  • Functions:
    • login(username, password) — verifies active local user, rejects OIDC-only accounts, cleans expired sessions for that user, creates a new session, updates last_login_at, and returns {sessionId, user} or null.
    • createSession(userId) — creates a session for an OIDC-provisioned or existing local user after validating that the user is active.
    • logout(sessionId) — deletes one session.
    • getSessionUser(sessionId) — returns public user fields when the session exists, is unexpired, and belongs to an active user.
    • rotateSessionId(oldSessionId, userId) — validates the current session, deletes it, and inserts a replacement session in a transaction. Password changes use this to prevent session fixation.
    • invalidateOtherSessions(userId, keepSessionId) — deletes all sessions for a user except the supplied current session; when keepSessionId is null, deletes every session for that user.
    • pruneExpiredSessions() — deletes expired sessions.
    • publicUser(user) — strips password/session secrets and normalizes booleans.

Password changes through /api/auth/change-password update the password hash, clear must_change_password, stamp last_password_change_at, invalidate all other sessions, rotate the current session ID when a valid session cookie is present, set a replacement bt_session cookie, and audit password.change.

/api/auth/logout-all calls invalidateOtherSessions(req.user.id, null), deletes the current cookie session if present, audits logout.all, clears bt_session, and returns {success:true}.

services/oidcService.js

Handles Authentik/OIDC login with openid-client:

  • Reads settings first, then env fallbacks: issuer URL, client ID/secret, redirect URI, token auth method, scopes, provider name, admin group, auto-provision flag.
  • Builds PKCE login state in oidc_states.
  • Discovers provider and caches OIDC client for 1 hour.
  • Exchanges callback code, verifies nonce/state, maps claims to local user.
  • Role mapping uses configured admin group; default role is user.
  • Client secret is write-only through admin settings.

services/backupService.js

  • Backup directory: BACKUP_PATH || backups/.
  • Valid backup IDs match managed prefixes only: bill-tracker-backup, pre-restore, imported-backup, scheduled-backup.
  • Creates SQLite backups with checksum and metadata.
  • Validates uploads by SQLite integrity_check and optional SHA-256 checksum.
  • Restore creates a pre-restore backup before swapping DB.
  • Path traversal is prevented by ID regex and path.relative checks.

services/backupScheduler.js

  • Validates daily/weekly schedule, HH:MM time, retention count 1-365.
  • Stores schedule settings in settings.
  • Uses node-cron, supports run-now, reload, next-run status, and retention cleanup.

services/cleanupService.js

Cleanup tasks:

  • Expired import sessions.
  • Stale temp SQLite export files in OS temp dir.
  • Orphan .partial/.upload backup files.
  • Optional old import-history pruning.

Settings are stored in settings; run results are stored as JSON.

services/notificationService.js

  • Builds SMTP transport from global notification settings.
  • Sends test email to an admin-provided address.
  • Runs due-bill notifications for 3 days, 1 day, due today, and overdue.
  • De-duplicates sends via notifications unique key.
  • Recipients come from user notification settings when enabled/allowed or global recipient settings.

services/spreadsheetImportService.js

  • Accepts XLSX buffers only, max 10 MB, max 5,000 rows.
  • Detects monthly sheets, headers, categories, bills, amounts, and ambiguous rows.
  • Creates import_sessions preview records with 24-hour TTL.
  • Apply step creates/updates user-owned categories, bills, payments, monthly state, and import history.

services/userDbImportService.js

  • Accepts user SQLite export files up to 50 MB.
  • Requires export metadata and known tables.
  • Sanitizes categories, bills, payments, monthly state, and starting amounts.
  • Preview stores an import session; apply maps export IDs to current user-owned IDs.

services/transactionService.js

Transaction data source and transaction row helpers:

  • ensureManualDataSource(db, userId) creates/retrieves a user-specific manual data source (type='manual', provider='manual', name='Manual Entry').
  • decorateDataSource(row) removes encrypted_secret, adds source_label and source_type_label.
  • decorateTransaction(row) adds source_label, source_type_label, and embedded data_source object with safe fields.
  • getSourceTypeLabel(type) returns labels: manual → Manual, file_import → File import, provider_sync → Provider sync.
  • sourceLabel(source) constructs human-readable source labels for manual entries or provider names.

services/csvTransactionImportService.js

CSV import workflow for transactions:

  • Parses CSV with quoted-field support and quote doubling.
  • previewCsvTransactions(userId, buffer, options) returns headers, sample rows, suggested field mapping, errors, and creates a 24-hour TTL import session (max 25k rows).
  • suggestMapping(headers) auto-detects field mappings from header names against posted_date, transacted_at, amount, description, payee, memo, category, account, transaction_type, currency, etc.
  • commitCsvTransactions(userId, importSessionId, mapping) imports rows into the transactions table with source_type='file_import', auto-creates a CSV data source and financial accounts per unique account name.
  • Stable deduplication via SHA-256 hash: csv:id: prefix for explicit transaction IDs, csv:hash: prefix from date+amount+description+payee+account.
  • Imports record to import_history with counts and details.
  • FIELD_LABELS maps field keys to user-friendly labels for validation messages.

services/paymentValidation.js

Payment validation helpers for source tracking and matching:

  • Validates payment_source values (manual, file_import, provider_sync).
  • Supports transaction linking via transaction_id when available.

services/auditService.js

Writes audit_log rows for security-sensitive events such as login, logout, password changes, role changes, CSRF failures, user seed flag resets, migration runs, and migration rollback attempts.

db/database.js does not import logAudit at module load time. It uses a lazy getLogAudit() helper so migration code can write audit rows without creating a circular dependency: auditService imports database.js, and database.js needs audit logging during migrations. If the lazy require fails, the helper degrades to a no-op audit function while console logging still records migration progress/errors.

5. API Reference

All routes are prefixed with /api unless stated otherwise. Most mutating endpoints require CSRF token header x-csrf-token. Auth is cookie-based. User routes are scoped to the authenticated user unless noted.

Response conventions:

  • Success: JSON object/array unless endpoint downloads a file.
  • Validation: 400 {error, code?, field?}.
  • Unauthenticated: 401.
  • Forbidden: 403.
  • Missing resource: 404.
  • Conflict: 409.
  • Rate-limited: 429.

5.1 Auth

Mounted under /api/auth.

  • POST /auth/login

    • Body: {username, password}.
    • Validation: both required; local login must be enabled.
    • Response: sets bt_session; {user}.

  • POST /auth/logout

    • Auth: required.
    • Body: none.
    • Response: clears cookie; {success:true}.

  • POST /auth/logout-all

    • Auth: required; CSRF skip is set before the router mount, matching other auth/session mutation routes.
    • Body: none.
    • Behavior: deletes every session for the current user by calling invalidateOtherSessions(userId, null), also deletes the current cookie session, audits logout.all, clears bt_session, and returns {success:true}.

  • GET /auth/me

    • Auth: required unless single-user mode supplies user.
    • Response: public user object.

  • GET /auth/mode

    • Public.
    • Response: auth mode, local-login flag, OIDC public info, single-user status.

  • POST /auth/restore-multi-user-mode

    • Auth: required.
    • Body: none.
    • Response: restores multi-user mode where allowed.

  • POST /auth/acknowledge-privacy

    • Auth: required.
    • Body: none.
    • Response: updates first-login/privacy acknowledgement flags.

  • POST /auth/change-password

    • Auth: required; password limiter; CSRF skip is set before the router mount.
    • Body: {current_password, new_password}.
    • Validation: current password required unless must_change_password is set; new password min 8.
    • Behavior: updates password hash, clears must_change_password, updates last_password_change_at, invalidates all other sessions, rotates the current session ID when a valid bt_session exists, sets the new cookie, audits password.change, and returns {success:true}.

  • GET /auth/has-users

    • Response: whether non-default users exist.

  • GET /auth/users

    • Auth: admin.
    • Response: safe user list.

  • POST /auth/users

    • Auth: admin.
    • Body: {username, password}.
    • Validation: username min 3, password min 8, unique username.
    • Response: created safe user.

5.2 OIDC Auth

Mounted under /api/auth/oidc; OIDC rate limiter applies.

  • GET /auth/oidc/login?redirect_to=/path

    • Public when OIDC active.
    • Creates PKCE state and redirects to provider authorization URL.

  • GET /auth/oidc/callback?code=&state=

    • Public callback.
    • Validates state/nonce, exchanges code, provisions/fetches user, creates session, redirects to saved path or app root.
    • Error redirects use query errors such as access_denied or authentication_failed.

5.3 Admin

Mounted under /api/admin; all require requireAuth + requireAdmin + adminActionLimiter at the mount level. Backup subroutes also use backupOperationLimiter.

  • GET /admin/has-users

    • Response: {has_users:boolean} for users other than current admin.

  • GET /admin/users

    • Response: safe users ordered by default-admin, role, username.

  • POST /admin/users

    • Body: {username, password}.
    • Validation: username min 3, password min 8, unique.
    • Response 201: created user.

  • PUT /admin/users/:id/password

    • Body: {password}.
    • Validation: password min 8; user exists.
    • Response: {success:true}; invalidates target sessions and requires password change.

  • PUT /admin/users/:id/role

    • Body: {role:"admin"|"user"}.
    • Validation: cannot change own role; cannot remove last admin.
    • Response: updated safe user; invalidates target sessions; audits role change.

  • PUT /admin/users/:id/active

    • Body: {active:boolean}.
    • Validation: user exists; cannot deactivate self.
    • Response: updated safe user; deactivation invalidates sessions.

  • DELETE /admin/users/:id

    • Validation: user exists; cannot delete self.
    • Response: {success:true, deleted_user_id}; transaction deletes import sessions/history, sessions, and user.

  • POST /admin/backups

    • Body: none.
    • Response 201: backup metadata {id, filename, size_bytes, checksum, ...}.

  • GET /admin/backups

    • Response: {backups:[metadata...]}.

  • GET /admin/backups/settings

    • Response: backup schedule status/settings.

  • PUT /admin/backups/settings

    • Body: {enabled, frequency:"daily"|"weekly", time:"HH:MM", retention_count}.
    • Validation: frequency, valid time, retention 1-365.
    • Response: saved schedule status.

  • POST /admin/backups/run-scheduled-now

    • Body: none.
    • Response 201: scheduled backup result.

  • POST /admin/backups/import

    • Content-Type: application/octet-stream, application/x-sqlite3, or application/vnd.sqlite3.
    • Body: raw SQLite backup, max 100 MB.
    • Optional checksum: X-Checksum-Sha256 header or ?checksum=.
    • Response 201: imported backup metadata.

  • GET /admin/backups/:id/download

    • Response: file download. ID must be a managed backup filename.

  • POST /admin/backups/:id/restore

    • Response: {restored_from, pre_restore_backup}; validates and restores managed backup.

  • DELETE /admin/backups/:id

    • Response: {deleted:true, id, deleted_at}.

  • GET /admin/cleanup

    • Response: cleanup settings and last result.

  • PUT /admin/cleanup

    • Body: any of {import_sessions_enabled, temp_exports_enabled, temp_export_max_age_hours, backup_partials_enabled, import_history_enabled, import_history_max_age_days}.
    • Validation: booleans; temp export age 1-72 hours; import history age 30-3650 days.
    • Response: updated cleanup status.

  • POST /admin/cleanup/run

    • Response: cleanup run result by task.

  • GET /admin/auth-mode

    • Response: local/single-user/OIDC settings and lockout warnings. Client secret is not returned.

  • POST /admin/auth-mode/oidc-test

    • Body: submitted or saved OIDC config fields.
    • Response: {ok:true,...} or 400 with test error. Never returns secret/token material.

  • PUT /admin/auth-mode

    • Body: legacy {auth_mode, default_user_id} plus OIDC/local settings such as local_login_enabled, oidc_login_enabled, oidc_issuer_url, oidc_client_id, oidc_client_secret, oidc_redirect_uri, oidc_scopes, oidc_admin_group, oidc_auto_provision.
    • Validation: cannot disable all login methods; cannot disable local login until OIDC is configured, enabled, and has an admin group; cannot enable incomplete OIDC.
    • Response: {success:true, ...authModeStatus}.

  • POST /admin/migrations/rollback

    • Body: {version:"v0.44"|"v0.45"|"v0.46"}.
    • Validation: version required; migration must be present in schema_migrations; ROLLBACK_SQL_MAP must define rollback SQL for that version.
    • Behavior: calls rollbackMigration(version) from db/database.js, audits success as migration.rollback, and returns {success:true, version, description, elapsed_ms}.
    • Error mapping: NOT_APPLIED becomes HTTP 404 with {error}; ROLLBACK_NOT_SUPPORTED becomes HTTP 422 with {error}; other rollback failures become HTTP 500 with {error:"Rollback failed", details} and are audited as migration.rollback.failure.

5.4 Bills

Mounted under /api/bills; auth: user/admin tracker access.

Bill object fields include id, user_id, name, category_id, category_name, due_day, override_due_date, bucket, expected_amount, interest_rate, billing_cycle, autopay_enabled, autodraft_status, website, username, account_info, has_2fa, active, notes, history_visibility, is_seeded, cycle_type, cycle_day, timestamps, and has_history_ranges on list/detail queries.

Validation shared by create/update:

  • name required for create.
  • due_day: integer 1-31.
  • expected_amount: numeric, defaults to 0.
  • interest_rate: null/empty or number 0-100.
  • category_id: must belong to current user.
  • history_visibility: default, all, ranges, or none.
  • cycle_type: monthly, weekly, biweekly, quarterly, annual.
  • cycle_day: monthly 1-31; weekly/biweekly day name; quarterly/annual text up to 50 chars.

Endpoints:

  • GET /bills?inactive=true

    • Response: current user's bills; inactive excluded unless inactive=true.

  • GET /bills/:id

    • Response: one owned bill or 404.

  • POST /bills

    • Body: bill fields.
    • Response 201: created bill.

  • PUT /bills/:id

    • Body: partial bill fields.
    • Response: updated bill.

  • DELETE /bills/:id

    • Hard delete; cascades payments, monthly state, history ranges.
    • Response: {success:true, deleted_bill_id, deleted_bill_name, warning}.

  • GET /bills/:id/monthly-state?year=&month=

    • Validation: year 2000-2100; month 1-12.
    • Response: {bill_id, year, month, actual_amount, notes, is_skipped}.

  • PUT /bills/:id/monthly-state

    • Body: {year, month, actual_amount, notes, is_skipped}.
    • Validation: year/month; actual_amount null or non-negative.
    • Response: saved monthly state with timestamps.

  • GET /bills/:id/payments?page=1&limit=20

    • Validation: limit capped at 100.
    • Response: {bill_id, bill_name, total, page, limit, pages, payments}.

  • POST /bills/:id/toggle-paid

    • Body optional: {amount, paid_date, method, notes}.
    • If latest live payment exists, soft-deletes it. Otherwise creates a payment for amount or bill expected amount.
    • Response: {success, isPaid, action, payment?}.

  • GET /bills/:id/history-ranges

    • Response: {bill_id, history_visibility, ranges}.

  • POST /bills/:id/history-ranges

    • Body: {start_year, start_month, end_year?, end_month?, label?}.
    • Validation: years 2000-2100, months 1-12, end both present/absent and not before start.
    • Response 201: created range.

  • PUT /bills/:id/history-ranges/:rangeId

    • Body: partial range fields.
    • Response: updated range.

  • DELETE /bills/:id/history-ranges/:rangeId

    • Response: {success:true}.

5.5 Payments

Mounted under /api/payments; auth: user/admin tracker access. All queries are user-owned through joined bill ownership. Delete is soft delete via deleted_at.

  • GET /payments?bill_id=&year=&month=

    • Validation: year and month must be supplied together; year 2000-2100; month 1-12.
    • Response: live payments ordered descending by paid_date.

  • GET /payments/:id

    • Response: live payment or 404.

  • POST /payments

    • Body: {bill_id, amount, paid_date, method?, notes?, payment_source?}.
    • Validation: bill exists and owned; amount > 0; required fields present; payment_source one of manual, file_import, provider_sync, transaction_match.
    • Response 201: created payment.

  • POST /payments/quick

    • Body: {bill_id, amount?, paid_date?, method?, notes?, payment_source?}.
    • Defaults amount to bill expected amount and date to today; confirms autodraft status for autopay bills; defaults payment_source to manual.
    • Response 201: created payment.

  • POST /payments/bulk

    • Body: {payments:[{bill_id, amount, paid_date, method?, notes?, payment_source?}]}.
    • Validation: array required; max 50; bill_id integer; paid_date YYYY-MM-DD; amount finite >= 0; payment_source must be valid.
    • Duplicate live payments by user/bill/date/amount are skipped.
    • Response 201: {created:[...], skipped:[...], errors:[...]}.

  • PUT /payments/:id

    • Body: partial {amount, paid_date, method, notes, payment_source}.
    • Response: updated payment. Current code preserves existing fields when omitted.

  • DELETE /payments/:id

    • Response: {success:true} after setting deleted_at.

  • POST /payments/:id/restore

    • Response: restored payment with deleted_at:null.

5.6 Categories

Mounted under /api/categories; auth: user/admin tracker access.

  • GET /categories

    • Seeds default categories for user if needed.
    • Response: categories with bill counts/payment counts and bill summaries.

  • POST /categories

    • Body: {name}.
    • Validation: non-empty name; unique per user case-insensitive.
    • Response 201: created category.

  • PUT /categories/:id

    • Body: {name}.
    • Validation: category belongs to user; non-empty unique name.
    • Response: updated category.

  • DELETE /categories/:id

    • Validation: category belongs to user.
    • Behavior: transaction nulls category on owned bills, then deletes category.
    • Response: deletion summary.

5.7 Tracker and Calendar

  • GET /tracker?year=&month=

    • Auth: user/admin tracker access.
    • Defaults to current year/month.
    • Response includes year, month, tracker rows, totals, starting amount info, previous month paid total, three-month averages/trends, and generated timestamp.
    • Row fields come from buildTrackerRow plus monthly override state and previous-month payment data.

  • GET /tracker/upcoming?days=30

    • Auth: user/admin tracker access.
    • Response: upcoming active bills in the requested horizon.

  • GET /calendar?year=&month=

    • Auth: user/admin tracker access.
    • Defaults current year/month.
    • Response: month days with payment entries, bills/due-status entries, and totals {expectedTotal, paidTotal, remainingTotal, paidPercent}.

5.8 Summary and Starting Amounts

  • GET /summary?year=&month=

    • Auth: user/admin tracker access.
    • Validation: valid year/month.
    • Response: {year, month, income, expenses, starting_amounts, previous_month, summary, chart, generated_at}.

  • PUT /summary/income

    • Body: {year, month, amount, label?}.
    • Validation: valid year/month; amount 0-1,000,000,000; label trimmed to 80 chars.
    • Response: {year, month, income} after upsert into monthly_income.

  • GET /monthly-starting-amounts?year=&month=

    • Response: {year, month, first_amount, fifteenth_amount, other_amount, combined_amount, paid deductions, remaining values, notes}.

  • PUT /monthly-starting-amounts

    • Body: {year, month, first_amount, fifteenth_amount, other_amount, notes?}.
    • Validation: valid year/month; numeric amounts.
    • Response: recomputed starting-amount response after upsert.

5.8.1 Snowball

Mounted under /api/snowball; auth: user/admin tracker access.

  • GET /snowball

    • Response: current user's debt bills (snowball_include or debt-like categories), pre-sorted by snowball_order.

  • GET /snowball/settings

    • Response: {extra_payment, ramsey_mode, ready_current_on_bills, ready_emergency_fund}.

  • PATCH /snowball/settings

    • Body: {extra_payment?, ramsey_mode?, ready_current_on_bills?, ready_emergency_fund?}.
    • Response: saved settings and computed response.

  • GET /snowball/projection

    • Response: {snowball, avalanche, minimum_only, comparison} with enriched debt arrays including APR snapshots.

  • PATCH /snowball/order

    • Body: [{id, snowball_order}].
    • Response: {success:true} after batch update.

5.9 Analytics

  • GET /analytics/summary?year=&month=&months=&category_id=&bill_id=&include_inactive=true&include_skipped=false
    • Auth: user/admin tracker access.
    • Validation: year/month valid; months clamped by route validation; IDs parsed as integers.
    • Response includes monthly spending, expected vs actual, category totals, bill totals, filters, and generated timestamp.

5.10 Settings

  • GET /settings

    • Auth: user/admin tracker access.
    • Response: user-visible settings from the allowed settings key list.

  • PUT /settings

    • Auth: user/admin tracker access.
    • Body: key/value object for allowed user setting keys.
    • Response: updated settings object.

  • POST /settings/seed-demo-data

    • Auth: user/admin tracker access.
    • Response: demo seed result.

5.11 Notifications

Mounted under /api/notifications. Server mount requires requireAuth; route-level guards further restrict.

  • GET /notifications/admin

    • Auth: admin.
    • Response: global SMTP/notification settings.

  • PUT /notifications/admin

    • Auth: admin.
    • Body: allowed global SMTP and notification settings.
    • Response: saved settings.

  • POST /notifications/test

    • Auth: admin.
    • Body: {to}.
    • Validation: recipient required.
    • Response: send result or SMTP error.

  • GET /notifications/me

    • Auth: user/admin tracker access.
    • Response: current user's notification email and toggle settings.

  • PUT /notifications/me

    • Auth: user/admin tracker access.
    • Body: {notification_email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}.
    • Response: saved user notification settings.

5.12 Profile

Mounted under /api/profile; auth: user/admin tracker access; password limiter applies at mount.

  • GET /profile

    • Response: safe profile, notification settings, export URLs, import-history URL.

  • PATCH /profile

    • Body: {display_name}.
    • Validation: string, max 100 chars.
    • Response: {success:true, profile}.

  • GET /profile/settings

    • Response: user notification preferences only.

  • PATCH /profile/settings

    • Body: {notification_email|email, notifications_enabled, notify_3d, notify_1d, notify_due, notify_overdue}.
    • Validation: email value string/null, max 255 chars.
    • Response: {success:true}.

  • POST /profile/change-password

    • Body: {current_password, new_password, confirm_new_password}.
    • Validation: all required; confirmation matches; new password min 8; current password must verify.
    • Response: rotates current session, invalidates others, {success:true}.

  • GET /profile/exports

    • Response: metadata for user_db and user_excel export URLs.

  • GET /profile/import-history

    • Response: {history:[...]}.

5.13 User Demo Data

Mounted under /api/user; auth: user/admin tracker access.

  • POST /user/seed-demo-data

    • Response: seed result for current user.

  • POST /user/clear-demo-data

    • Rate-limited by demo-data limiter.
    • Behavior: deletes is_seeded=1 bills/categories for current user and records import history.
    • Response: deletion counts.

5.14 Import

Mounted under /api/import; auth: user/admin tracker access; import limiter applies.

  • POST /import/spreadsheet/preview?parse_all_sheets=true&year=&month=

    • Content-Type: application/octet-stream.
    • Headers: optional X-Filename.
    • Body: XLSX file buffer, max 10 MB.
    • Response: preview session, parsed rows, categories/bills/payment candidates, ambiguous decisions, errors.

  • POST /import/spreadsheet/apply

    • Body: {import_session_id, decisions, options}.
    • Validation: session exists, not expired, belongs to user.
    • Response: created/updated/skipped/error counts and import history summary.

  • POST /import/user-db/preview

    • Content-Type: application/octet-stream.
    • Headers: optional X-Filename.
    • Body: user SQLite export file, max 50 MB.
    • Response: preview session and sanitized import plan.

  • POST /import/user-db/apply

    • Body: {import_session_id, options}.
    • Validation: session exists, not expired, belongs to user.
    • Response: apply result with ID mappings/counts.

  • GET /import/history

    • Response: current user's import history.

5.15 Data Sources

Mounted under /api/data-sources; auth: user/admin tracker access.

  • GET /data-sources?type=&status=
    • Query params: type (manual, file_import, provider_sync), status (active, inactive, error).
    • Response: array of data sources with source_label, source_type_label, account_count, transaction_count, safe fields (encrypted_secret excluded), timestamps, sync info.

5.16 Transactions

Mounted under /api/transactions; auth: user/admin tracker access.

  • GET /transactions?limit=&offset=&match_status=&ignored=&source_type=&start_date=&end_date=&q=&data_source_id=&account_id=&matched_bill_id=

    • Response: paginated list of transactions with embedded data_source and account details, source_label, source_type_label.
    • Filter: match_status (unmatched, matched, ignored), ignored (boolean), source_type, date range, free-text search (q) across description/payee/memo/category.

  • POST /transactions/manual

    • Body: {account_id?, transaction_type?, posted_date, transacted_at?, amount, currency?, description?, payee?, memo?, category?, matched_bill_id?, match_status?, ignored?}.
    • Response 201: created transaction with manual source_type.

  • PUT /transactions/:id

    • Body: partial transaction fields.
    • Validation: match_state changes use dedicated endpoints.
    • Response: updated transaction.

  • DELETE /transactions/:id

    • Behavior: soft-deletes via unmatch then hard delete.
    • Response: {success:true, deleted:true, id}.

  • POST /transactions/:id/match

    • Body: {billId}.
    • Response: {success:true, matched:true, transaction}.

  • POST /transactions/:id/unmatch

    • Response: {success:true, unmatched:true, transaction}.

  • POST /transactions/:id/ignore

    • Response: {transaction} with match_status='ignored', ignored=1.

  • POST /transactions/:id/unignore

    • Response: {transaction} restored to unmatched state.

5.17 CSV Import

Mounted under /api/import; auth: user/admin tracker access; import limiter applies. Gated by DATA_IMPORT_ENABLED env var (defaults to true).

  • POST /import/csv/preview

    • Content-Type: text/csv.
    • Body: raw CSV.
    • Response: {import_session_id, headers, sampleRows, rowCount, suggestedMapping, errors, fields}.

  • POST /import/csv/commit

    • Body: {import_session_id, mapping, options?}.
    • Response: {imported, skipped, failed, details}.

  • GET /import/history

    • Response: current user's import history.

5.18 Match Suggestions

Mounted under /api/matches; auth: user/admin tracker access.

  • GET /matches/suggestions?transaction_id=&bill_id=&limit=&offset=

    • Response: {suggestions:[{id, transaction, bill, score, match_status, created_at}]}.

  • POST /matches/:id/reject

    • Response: {success:true} after recording rejection.

5.19 Export

Mounted under /api/export; auth: user/admin tracker access; export limiter applies.

  • GET /export?year=YYYY&format=csv|xlsx

    • Response: file download of payment/bill history for the requested year. CSV includes date, bill, category, expected, paid, method, notes, actual amount, monthly notes. XLSX includes enriched rows.

  • GET /export/user-excel

    • Response: Excel workbook with user categories, bills, payments, monthly state, monthly starting amounts, and notes/metadata.

  • GET /export/user-db

    • Response: portable SQLite file with export metadata and user-owned categories, bills, payments, monthly state, monthly starting amounts, and notes.

5.20 Status

  • GET /status
    • Auth: admin.
    • Response: app version, uptime, runtime worker state, DB health/counts/path/size, SMTP configuration status, backup status/schedule, current-month tracker health, recent errors.

5.21 About and Version

  • GET /about

    • Public.
    • Response: package version and public project/about metadata.

  • GET /about-admin

    • Auth: admin; admin action limiter; CSRF middleware.
    • Response: package version plus sanitized/redacted FUTURE.md and DEVELOPMENT_LOG.md content.
    • File allowlist only: FUTURE.md, DEVELOPMENT_LOG.md.

  • GET /version

    • Public.
    • Response: current package version and latest structured notes from HISTORY.md.

  • GET /version/history

    • Public.
    • Response: package version and raw history text, or error if unavailable.

5.22 Services

Key service modules:

  • paymentValidation.js — Payment input validation with PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match'] enum, validatePaymentSource(), and validatePaymentInput().

  • csvTransactionImportService.js — CSV parsing, field mapping, SHA-256 deduplication, import session management with preview/commit workflow.

  • transactionService.js — Transaction helpers: ensureManualDataSource(), decorateDataSource(), decorateTransaction().

  • transactionMatchService.js — Match/unmatch transactions to bills: matchTransactionToBill(), unmatchTransaction(), ignoreTransaction(), unignoreTransaction().

  • matchSuggestionService.js — Match suggestion discovery: listMatchSuggestions(), rejectMatchSuggestion(), suggestionCounts().

  • snowballService.js — Debt snowball/avalanche calculations, Ramsey mode support, order updates.

  • dataSourcesService.js — Data source CRUD with ensureManualDataSource() for user-scoped manual sources.

  • monthlyStartingAmountsService.js — Starting cash bucket tracking: first/fifteenth/other bucket amounts, payments, remaining values.

  • auditService.js — Audit logging via logAudit(); lazy-loaded in database.js to avoid circular dependency.

  • emailService.js — Email dispatch with SMTP configuration, templating, retry logic.

  • exportService.js — Export helpers: CSV, XLSX, SQLite user DB export with metadata.

  • csvTransactionImportService.js — CSV parsing, field mapping, SHA-256 deduplication, import session management with preview/commit workflow.

  • trackerService.js — Tracker calculations, cycle detection (weekly/biweekly/quarterly/annual), debt snowball support.

  • statusService.js — Health status, cycle validation, autopay simulation, budget projections.

  • analyticsService.js — Analytics queries: spending, categories, bills, filters.

  • notificationService.js — Bill notifications: due_3d, due_1d, due_today, overdue.

  • authService.js — Auth helpers: login, JWT, password hashing, session management, OIDC integration.

  • userService.js — User CRUD, profile updates, demo data seeding, role changes.

  • settingsService.js — Settings CRUD, allowed keys validation, SMTP/billing/export settings.

  • backupService.js — SQLite backup, retention, schedule.

  • cronService.js — Scheduled tasks: backup, cleanup, auto-mark paid, cycle updates.

5.23 Import and Sync Workflow

Data ingestion follows a layered architecture:

  1. Data Sources (data_sources table)

    • manual: User-created source (one per user, type='manual', provider='manual')
    • file_import: CSV/XLSX imports (provider='csv_transactions', 'spreadsheet')
    • provider_sync: External institution sync (e.g., 'plaid', 'mint')
    • Fields: type, provider, name, status ('active', 'inactive', 'error'), config_json, encrypted_secret, last_sync_at, last_error

  2. Financial Accounts (financial_accounts table)

    • Linked to data_sources via data_source_id
    • One data source can have many accounts
    • Fields: provider_account_id, name, org_name, account_type, currency, balance, available_balance, raw_data

  3. Transactions (transactions table)

    • Linked to data_sources and financial_accounts
    • Source type: manual, file_import, provider_sync
    • Match states: unmatched, matched, ignored
    • Optional provider_transaction_id for deduplication
    • Fields: amount (cents), transaction_type, posted_date, transacted_at, description, payee, memo, category, raw_data, matched_bill_id, match_status, ignored

  4. CSV Import Flow

    • User uploads CSV → /api/import/csv/preview
    • Preview parses headers, suggests field mapping
    • /api/import/csv/commit writes to transactions with source_type='file_import'
    • Import history tracked in import_history with counts

  5. Transaction Matching

    • Manual transactions (source_type='manual') can be matched to bills
    • Match suggestions discovered via matchSuggestionService
    • Users can reject suggestions to avoid重复 suggestions

  6. Provider Sync (future)

    • External sync jobs write to data_sources with type='provider_sync'
    • Financial accounts created per institution account
    • Transactions imported from provider
    • Match suggestions offered for unmatched transactions

  7. Payments (bills → payments)

    • Payments link to transactions via transaction_id for auto-draft
    • payment_source indicates origin: manual, file_import, provider_sync, transaction_match
    • Balance delta tracked for debt payoff

  8. Import Sessions (import_sessions table)

    • Temporary storage for CSV/XLSX previews
    • 1-hour TTL, auto-cleaned
    • Fields: preview_json, expires_at

  9. Import History (import_history table)

    • Audit trail of all imports
    • Fields: imported_at, source_filename, file_type, rows_parsed/created/updated/skipped/errored, options_json, summary_json

5.24 Validation and Services

Key validation and service patterns:

  • paymentValidation.js

    • PAYMENT_SOURCES = ['manual', 'file_import', 'provider_sync', 'transaction_match'] enum
    • validatePaymentSource(value) returns error if not in list
    • validatePaymentInput(body) validates amount, paid_date, bill ownership, payment_source
    • Invalid source returns 400 with VALIDATION_ERROR code

  • CSV Import Service

    • Field suggestion via header analysis
    • SHA-256 hash for deduplication
    • Import session management with preview/commit split
    • Error collection per row with detailed messages

  • Transaction Service

    • ensureManualDataSource(db, userId) creates one manual data source per user
    • decorateTransaction(row) adds source_label, source_type_label
    • decorateDataSource(row) adds source_label, source_type_label, account_count, transaction_count

  • Snowball Service

    • Computes snowball/avalanche orderings
    • Ramsey mode: minimum payments only vs. full extra payment
    • snowball_include bills sorted by snowball_order
    • snowball_exempt bills excluded from ordering

  • Match Suggestion Service

    • listMatchSuggestions(userId, transactionId, billId, limit, offset)
    • rejectMatchSuggestion(userId, transactionId, billId)
    • suggestionCounts(userId)
    • Rejects stored to prevent repeated suggestions

  • Monthly Starting Amounts Service

    • Bucketed amounts: first_amount, fifteenth_amount, other_amount
    • Payment tracking from each bucket
    • Remaining values computed on read

  • Tracker Service

    • Cycle type detection: monthly, weekly, biweekly, quarterly, annually
    • Cycle day mapping for non-standard cycles
    • Auto-mark paid logic for autopay bills

  • Status Service

    • Cycle validation warnings
    • Autopay simulation
    • Budget projections

6. Database Reference

SQLite uses WAL mode and foreign keys. Base schema is in db/schema.sql; db/database.js applies migrations to reach the current schema.

Tables and columns

users

  • id INTEGER PRIMARY KEY
  • username TEXT NOT NULL UNIQUE COLLATE NOCASE
  • password_hash TEXT NOT NULL
  • role TEXT NOT NULL DEFAULT 'user' (admin or user)
  • must_change_password INTEGER NOT NULL DEFAULT 0
  • first_login INTEGER NOT NULL DEFAULT 1
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • notification_email TEXT
  • notifications_enabled INTEGER NOT NULL DEFAULT 0
  • notify_3d INTEGER NOT NULL DEFAULT 1
  • notify_1d INTEGER NOT NULL DEFAULT 1
  • notify_due INTEGER NOT NULL DEFAULT 1
  • notify_overdue INTEGER NOT NULL DEFAULT 1
  • display_name TEXT
  • last_password_change_at TEXT
  • auth_provider TEXT NOT NULL DEFAULT 'local'
  • external_subject TEXT
  • email TEXT
  • last_login_at TEXT
  • active INTEGER NOT NULL DEFAULT 1
  • is_default_admin INTEGER NOT NULL DEFAULT 0

sessions

  • id TEXT PRIMARY KEY
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • expires_at TEXT NOT NULL
  • created_at TEXT DEFAULT datetime('now')

categories

  • id INTEGER PRIMARY KEY
  • user_id INTEGER REFERENCES users(id) ON DELETE CASCADE
  • name TEXT NOT NULL
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • is_seeded INTEGER NOT NULL DEFAULT 0

bills

  • id INTEGER PRIMARY KEY
  • user_id INTEGER REFERENCES users(id) ON DELETE CASCADE
  • name TEXT NOT NULL
  • category_id INTEGER REFERENCES categories(id) ON DELETE SET NULL
  • due_day INTEGER NOT NULL CHECK 1-31
  • override_due_date TEXT
  • bucket TEXT CHECK ('1st','15th')
  • expected_amount REAL NOT NULL DEFAULT 0
  • interest_rate REAL CHECK null or 0-100
  • billing_cycle TEXT DEFAULT 'monthly' CHECK ('monthly','quarterly','annually','irregular')
  • autopay_enabled INTEGER NOT NULL DEFAULT 0
  • autodraft_status TEXT NOT NULL DEFAULT 'none' CHECK ('none','pending','assumed_paid','confirmed')
  • website TEXT
  • username TEXT
  • account_info TEXT
  • has_2fa INTEGER NOT NULL DEFAULT 0
  • active INTEGER NOT NULL DEFAULT 1
  • notes TEXT
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • history_visibility TEXT NOT NULL DEFAULT 'default'
  • is_seeded INTEGER NOT NULL DEFAULT 0
  • cycle_type TEXT NOT NULL DEFAULT 'monthly'
  • cycle_day TEXT
  • current_balance REAL
  • minimum_payment REAL
  • snowball_order INTEGER
  • snowball_include INTEGER NOT NULL DEFAULT 0
  • snowball_exempt INTEGER NOT NULL DEFAULT 0
  • auto_mark_paid INTEGER NOT NULL DEFAULT 0
  • deleted_at TEXT

payments

  • id INTEGER PRIMARY KEY
  • bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE
  • amount REAL NOT NULL
  • paid_date TEXT NOT NULL
  • method TEXT
  • notes TEXT
  • balance_delta REAL
  • payment_source TEXT NOT NULL DEFAULT 'manual'
  • transaction_id INTEGER
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • deleted_at TEXT

monthly_bill_state

  • id INTEGER PRIMARY KEY
  • bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE
  • year INTEGER NOT NULL CHECK 2000-2100
  • month INTEGER NOT NULL CHECK 1-12
  • actual_amount REAL
  • notes TEXT
  • is_skipped INTEGER NOT NULL DEFAULT 0
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • Unique: (bill_id, year, month)

monthly_income

  • id INTEGER PRIMARY KEY
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • year INTEGER NOT NULL
  • month INTEGER NOT NULL
  • label TEXT NOT NULL DEFAULT 'Salary'
  • amount REAL NOT NULL DEFAULT 0
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • Unique intended/current logic: (user_id, year, month) via migration/index.

monthly_starting_amounts

  • id INTEGER PRIMARY KEY
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • year INTEGER NOT NULL
  • month INTEGER NOT NULL
  • first_amount REAL NOT NULL DEFAULT 0
  • fifteenth_amount REAL NOT NULL DEFAULT 0
  • other_amount REAL NOT NULL DEFAULT 0
  • notes TEXT
  • created_at TEXT DEFAULT datetime('now')
  • updated_at TEXT DEFAULT datetime('now')
  • Unique intended/current logic: (user_id, year, month) via migration/index.

bill_history_ranges

  • id INTEGER PRIMARY KEY
  • bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE
  • start_year INTEGER NOT NULL
  • start_month INTEGER NOT NULL
  • end_year INTEGER
  • end_month INTEGER
  • label TEXT
  • description TEXT NOT NULL
  • applied_at TEXT NOT NULL DEFAULT datetime('now')

data_sources

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • type TEXT NOT NULL (manual, file_import, provider_sync)
  • provider TEXT
  • name TEXT NOT NULL
  • status TEXT NOT NULL DEFAULT 'active' (active, inactive, error)
  • config_json TEXT
  • encrypted_secret TEXT
  • last_sync_at TEXT
  • last_error TEXT
  • created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • Unique partial index: (user_id, type, provider) WHERE type='manual' AND provider='manual'

financial_accounts

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL
  • provider_account_id TEXT
  • name TEXT NOT NULL
  • org_name TEXT
  • account_type TEXT
  • currency TEXT
  • balance INTEGER
  • available_balance INTEGER
  • raw_data TEXT
  • created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • Unique: (data_source_id, provider_account_id)

transactions

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • data_source_id INTEGER REFERENCES data_sources(id) ON DELETE SET NULL
  • account_id INTEGER REFERENCES financial_accounts(id) ON DELETE SET NULL
  • provider_transaction_id TEXT
  • source_type TEXT NOT NULL (manual, file_import, provider_sync)
  • transaction_type TEXT
  • posted_date TEXT
  • transacted_at TEXT
  • amount INTEGER NOT NULL
  • currency TEXT
  • description TEXT
  • payee TEXT
  • memo TEXT
  • category TEXT
  • raw_data TEXT
  • matched_bill_id INTEGER REFERENCES bills(id) ON DELETE SET NULL
  • match_status TEXT NOT NULL DEFAULT 'unmatched' (unmatched, matched, ignored)
  • ignored INTEGER NOT NULL DEFAULT 0
  • created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • updated_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP
  • Unique partial index: (data_source_id, provider_transaction_id) WHERE provider_transaction_id IS NOT NULL
  • Indexes: (user_id, posted_date, transacted_at), (user_id, match_status, ignored), (account_id), (matched_bill_id)

Indexes

Important indexes include:

  • idx_bills_active(active)
  • idx_bills_user_active(user_id, active)
  • idx_bills_user_name(user_id, name)
  • idx_categories_user_name(user_id, name)
  • idx_categories_user_name_unique(user_id, name COLLATE NOCASE)
  • idx_payments_bill_id(bill_id)
  • idx_payments_paid_date(paid_date)
  • idx_payments_bill_date_del(bill_id, paid_date, deleted_at)
  • idx_payments_deleted(deleted_at)
  • idx_payments_method(method)
  • idx_sessions_user_id(user_id)
  • idx_sessions_expires(expires_at)
  • idx_monthly_bill_state_lookup(bill_id, year, month)
  • idx_monthly_income_user_month(user_id, year, month)
  • idx_monthly_starting_amounts_user(user_id)
  • idx_monthly_starting_amounts_user_month(user_id, year, month)
  • idx_notifications_lookup(bill_id, user_id, year, month)
  • idx_import_sessions_user(user_id), idx_import_sessions_expires(expires_at)
  • idx_import_history_user(user_id), idx_import_history_imported_at(imported_at)

import_sessions

  • id TEXT PRIMARY KEY
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • created_at TEXT NOT NULL
  • expires_at TEXT NOT NULL
  • preview_json TEXT NOT NULL

import_history

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • imported_at TEXT NOT NULL
  • source_filename TEXT
  • file_type TEXT DEFAULT 'csv_transactions'
  • rows_parsed INTEGER DEFAULT 0
  • rows_created INTEGER DEFAULT 0
  • rows_updated INTEGER DEFAULT 0
  • rows_skipped INTEGER DEFAULT 0
  • rows_errored INTEGER DEFAULT 0
  • options_json TEXT
  • summary_json TEXT
  • created_at TEXT DEFAULT (datetime('now'))

autopay_suggestion_dismissals

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE
  • year INTEGER NOT NULL CHECK(year BETWEEN 2000 AND 2100)
  • month INTEGER NOT NULL CHECK(month BETWEEN 1 AND 12)
  • dismissed_at TEXT NOT NULL DEFAULT (datetime('now'))
  • Unique: (user_id, bill_id, year, month)

bill_templates

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • name TEXT NOT NULL
  • data TEXT NOT NULL
  • created_at TEXT DEFAULT (datetime('now'))
  • updated_at TEXT DEFAULT (datetime('now'))
  • Unique: (user_id, name)

match_suggestion_rejections

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • transaction_id INTEGER NOT NULL REFERENCES transactions(id) ON DELETE CASCADE
  • bill_id INTEGER NOT NULL REFERENCES bills(id) ON DELETE CASCADE
  • rejected_at TEXT NOT NULL DEFAULT (datetime('now'))
  • Unique: (user_id, transaction_id, bill_id)

user_login_history

  • id INTEGER PRIMARY KEY AUTOINCREMENT
  • user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE
  • logged_in_at TEXT NOT NULL DEFAULT (datetime('now'))
  • ip_address TEXT
  • user_agent TEXT
  • browser TEXT
  • os TEXT
  • device_type TEXT
  • device_fingerprint TEXT
  • idx_oidc_states_expires(expires_at)
  • idx_bill_history_ranges_bill(bill_id)
  • idx_audit_log_user(user_id, created_at), idx_audit_log_action(action, created_at)
  • idx_data_sources_user_type(user_id, type, status)
  • idx_data_sources_user_manual(user_id, type, provider) WHERE type='manual' AND provider='manual'
  • idx_financial_accounts_user_source(user_id, data_source_id)
  • idx_transactions_user_date(user_id, posted_date, transacted_at)
  • idx_transactions_user_match(user_id, match_status, ignored)
  • idx_transactions_account(account_id)
  • idx_transactions_matched_bill(matched_bill_id)
  • idx_transactions_provider_dedupe(data_source_id, provider_transaction_id) WHERE provider_transaction_id IS NOT NULL

Migration system

db/database.js:

  • Reads db/schema.sql in initSchema().
  • Creates schema_migrations.
  • Detects and reconciles legacy databases.
  • Applies ordered migrations only when not already recorded.
  • Validates dependency chains before applying dependent migrations.
  • Uses a column whitelist for dynamic ALTER TABLE statements.
  • Wraps versioned migrations in transactions, with special handling for v0.40 because it uses PRAGMA-driven table rebuild work.
  • Supports rollback SQL for selected migrations through ROLLBACK_SQL_MAP and rollbackMigration(version).

Current migration set:

  • v0.2 payments soft delete.
  • v0.3 tracker payment compound index.
  • v0.4 monthly bill state.
  • v0.13 user profile columns.
  • v0.14 bill history visibility.
  • v0.14.4 bill interest rate.
  • v0.15 import sessions/history.
  • v0.17 OIDC/external identity columns and state table.
  • v0.18.1 monthly income.
  • v0.18.2 monthly starting amounts.
  • v0.18.3 other starting amount bucket.
  • v0.38 per-user import audit history.
  • v0.40 ownership for bills/categories.
  • v0.41 seeded demo-data flags.
  • v0.42 bill history ranges.
  • v0.43 session created_at.
  • v0.44 performance indexes.
  • v0.45 audit log.
  • v0.46 bill cycle_type and cycle_day.
  • v0.47 bills: current_balance, minimum_payment, snowball_order, snowball_include, snowball_exempt, auto_mark_paid, deleted_at columns.
  • v0.48 users: snowball_extra_payment column.
  • v0.49 payments: balance_delta column for debt payoff tracking.
  • v0.50 users: last_seen_version for release-notes notifications.
  • v0.51 user_login_history table.
  • v0.54 bills/categories: soft-delete columns (deleted_at).
  • v0.55 autopay: auto_mark_paid and suggestion dismissals table.
  • v0.56 bills: saved bill templates table.
  • v0.57 match_suggestion_rejections table.
  • v0.58 import_sessions and import_history tables.
  • v0.59 payments: payment_source and transaction_id columns.
  • v0.60 data_sources, financial_accounts, transactions tables.
  • v0.61 payments: one active payment per linked transaction (unique index on transaction_id).
  • v0.62 match_suggestion_rejections table.
  • v0.63 data_sources: partial unique index on (user_id, type, provider) WHERE type='manual' AND provider='manual'.
  • v0.64 transactions: partial unique index on (data_source_id, provider_transaction_id) WHERE provider_transaction_id IS NOT NULL; indexes on (user_id, posted_date), (user_id, match_status, ignored), account_id, matched_bill_id.
  • Unversioned user notification columns are also reconciled.

Migration logging is both console-based and audit-backed:

  • runMigrations() logs start, dependency status, transaction begin/commit, per-migration elapsed time, skips for already-applied migrations, failures with rollback messages, and total elapsed time.
  • Audit events use the lazy getLogAudit() helper to avoid the auditService -> database.js -> auditService circular dependency.
  • Audit actions include migration.start, migration.complete, and migration.failure.
  • Rollback paths audit migration.rollback and migration.rollback.failure.

Rollback support is defined by ROLLBACK_SQL_MAP:

  • v0.44 — drops selected performance indexes: idx_bills_user_name, idx_payments_method, idx_monthly_starting_amounts_user, and idx_import_history_imported_at.
  • v0.45 — drops idx_audit_log_user, idx_audit_log_action, and the audit_log table.
  • v0.46 — drops bills.cycle_day and bills.cycle_type.
  • v0.47 — drops current_balance, minimum_payment, snowball_order, snowball_include, snowball_exempt, auto_mark_paid, deleted_at columns from bills.
  • v0.48 — drops snowball_extra_payment column from users.
  • v0.49 — drops balance_delta column from payments.
  • v0.50 — drops last_seen_version column from users.
  • v0.51 — drops user_login_history table.
  • v0.54 — removes soft-delete columns (deleted_at) from bills and categories.
  • v0.55 — drops autopay suggestion dismissals table.
  • v0.56 — drops bill_templates table.
  • v0.57 — drops match_suggestion_rejections table.
  • v0.58 — drops import_sessions and import_history tables.
  • v0.59 — drops payment_source and transaction_id columns from payments.
  • v0.60 — drops data_sources, financial_accounts, and transactions tables.
  • v0.61 — drops unique index on transaction_id from payments.
  • v0.62 — drops match_suggestion_rejections table.
  • v0.63 — drops partial unique index on data_sources.
  • v0.64 — drops partial unique index and indexes on transactions.

rollbackMigration(version) requires an initialized database, verifies the version exists in schema_migrations, looks up rollback SQL in ROLLBACK_SQL_MAP, executes all rollback statements inside a transaction, deletes the migration record, logs elapsed time, audits success, and returns {success:true, version, description, elapsed_ms}. If the migration is not recorded, it throws NOT_APPLIED. If no rollback definition exists, it throws ROLLBACK_NOT_SUPPORTED. Execution failures roll back the transaction and are audited as migration.rollback.failure.

The admin API exposes rollback through POST /api/admin/migrations/rollback. The route requires admin auth through the /api/admin mount. It maps NOT_APPLIED to HTTP 404, ROLLBACK_NOT_SUPPORTED to HTTP 422, and unexpected rollback failures to HTTP 500.

The lazy audit helper in db/database.js is:

let _logAudit = null;
function getLogAudit() {
  if (!_logAudit) {
    try { _logAudit = require('../services/auditService').logAudit; } catch { _logAudit = () => {}; }
  }
  return _logAudit;
}

Use this pattern for database-layer audit calls instead of a top-level require('../services/auditService').

7. Frontend Reference

Frontend stack

  • React ^18.3.1
  • Vite ^5.4.10
  • React Router ^6.26.2
  • TanStack Query ^5.100.9
  • Tailwind CSS ^3.4.14
  • shadcn/ui component primitives, backed by Radix UI where applicable
  • Sonner/shadcn toast notifications via sonner
  • react-markdown, remark-gfm, rehype-sanitize for markdown rendering

client/main.jsx

Creates the React root and wraps App with global providers including auth/theme where defined.

client/App.jsx

  • Creates a TanStack QueryClient with staleTime 2 minutes, retry 1, no refetch on focus.
  • Uses lazy loading and Suspense with PageLoader for most pages.
  • Wraps route elements in ErrorBoundary.
  • Exposes ReactQueryDevtools.
  • Provides skip link for keyboard users.

Routes:

  • /login → LoginPage public.
  • /about → AboutPage public.
  • /release-notes → ReleaseNotesPage public.
  • /admin → AdminPage, admin only.
  • /admin/about → admin shell + AboutPage admin, admin only.
  • /admin/roadmap → admin shell + AboutPage admin, admin only.
  • /admin/status → admin shell + StatusPage, admin only.
  • /status → redirects admin to /admin/status.
  • / → authenticated user layout index, TrackerPage.
  • /calendar → CalendarPage.
  • /summary → SummaryPage.
  • /bills → BillsPage.
  • /categories → CategoriesPage.
  • /analytics → AnalyticsPage.
  • /settings → SettingsPage.
  • /data → DataPage.
  • /profile → ProfilePage.
  • * under user layout → redirect /.

RequireAuth behavior:

  • Shows loading while auth state is undefined.
  • Redirects unauthenticated users to /login.
  • Allows admins to access user routes except default admin is redirected to /admin.
  • Allows single-user mode only for user routes.
  • Redirects role mismatches to /admin or /.

API client

client/api.js:

  • _fetch(method, path, body) calls /api${path} with JSON headers and credentials: include.
  • Mutating methods read bt_csrf_token cookie and send x-csrf-token.
  • Non-OK responses throw Error with status, data, details, and code.
  • Exposes grouped functions for auth, admin, notifications, profile, tracker, calendar, summary, bills, payments, categories, settings, analytics, status, version/about, import, export, data-sources, transactions, and matches.
  • File download/upload endpoints use raw fetch because responses/bodies are blobs or octet streams.

Client API v0.28.1 additions

  • api.dataSources(type, status) → /api/data-sources
  • api.transactions(filters) → /api/transactions
  • api.transactions.create(payload) → /api/transactions/manual
  • api.transactions.update(id, payload) → /api/transactions/:id
  • api.transactions.delete(id) → /api/transactions/:id
  • api.transactions.match(id, {billId}) → /api/transactions/:id/match
  • api.transactions.unmatch(id) → /api/transactions/:id/unmatch
  • api.transactions.ignore(id) → /api/transactions/:id/ignore
  • api.transactions.unignore(id) → /api/transactions/:id/unignore
  • api.csvImport.preview(file, options) → /api/import/csv/preview
  • api.csvImport.commit(importSessionId, mapping, options) → /api/import/csv/commit
  • api.import.history() → /api/import/history
  • api.matchSuggestions.list(filters) → /api/matches/suggestions
  • api.matchSuggestions.reject(id) → /api/matches/:id/reject
  • api.snowball.get() → /api/snowball
  • api.snowball.settings.get() → /api/snowball/settings
  • api.snowball.settings.patch(payload) → /api/snowball/settings
  • api.snowball.projection.get() → /api/snowball/projection
  • api.snowball.order.patch(bills) → /api/snowball/order
  • api.monthlyStartingAmounts.get(year, month) → /api/monthly-starting-amounts
  • api.monthlyStartingAmounts.update(payload) → /api/monthly-starting-amounts

Auth state

Auth state

client/hooks/useAuth.jsx:

  • Maintains user, singleUserMode, and loading state.
  • Calls api.authMode() and api.me() on startup.
  • Exposes logout() and refresh().

Query hooks

client/hooks/useQueries.js:

  • useTracker(year, month) → api.tracker(year, month).
  • useBills() → api.allBills().
  • useCategories() → api.categories().

These use TanStack Query keys and cache server data for common pages.

Pages

  • LoginPage.jsx — local login plus OIDC login availability based on /auth/mode.
  • TrackerPage.jsx — monthly tracker, payment interactions, upcoming bills, starting amount awareness, bulk/session logout action.
  • CalendarPage.jsx — calendar grid backed by /calendar.
  • SummaryPage.jsx — monthly plan, income, starting amounts, expenses, chart data.
  • BillsPage.jsx — bill CRUD, categories, monthly state/history range controls.
  • CategoriesPage.jsx — category list/create/update/delete and related bill info.
  • AnalyticsPage.jsx — analytics summary filters and charts.
  • SettingsPage.jsx — user/app settings and demo data seed.
  • DataPage.jsx — export, spreadsheet import, user DB import, import history, CSV transaction import with preview and commit flow (ImportTransactionCsvSection).
  • ProfilePage.jsx — display name, notification preferences, password change, export/import-history links.
  • AdminPage.jsx — users, auth mode/OIDC, backups, cleanup, notifications, migrations, admin settings.
  • StatusPage.jsx — admin system status.
  • AboutPage.jsx — public or admin markdown/about view; admin mode uses /about-admin; markdown is sanitized.
  • ReleaseNotesPage.jsx — release history display.

Components

  • Layout: Layout, Sidebar, BrandBlock, NavPill.
  • Domain UI: AdminDashboard, BillModal, BillsTableInner, MobileBillRow, MobileTrackerRow, StatusBadge, SummaryCard, MarkdownText, ReleaseNotesDialog.
  • Reliability: ErrorBoundary, PageLoader.
  • UI primitives: alert dialog, badge, button, card, checkbox, confirm dialog, dialog, dropdown menu, input dialog, input, label, select, separator, skeleton, switch, table, tabs, theme toggle, tooltip.

Frontend security notes

  • CSRF header is sent on POST/PUT/PATCH/DELETE.
  • Auth is cookie/session based; no tokens are stored in localStorage.
  • Admin routes are client-guarded and server-guarded.
  • Markdown rendering uses rehype-sanitize.
  • Error boundaries prevent route crashes from taking down the whole SPA.

8. Infrastructure and Deployment

package.json

Version: 0.28.1.

Scripts:

  • npm run dev:api — node --watch server.js.
  • npm run dev:ui — Vite dev server.
  • npm run dev — concurrently runs API and UI.
  • npm run build — Vite production build.
  • npm start — node server.js.

Key runtime dependencies:

  • Express, cookie-parser, cors, express-rate-limit.
  • better-sqlite3.
  • bcryptjs.
  • openid-client.
  • nodemailer.
  • node-cron.
  • React, React DOM, React Router, TanStack Query.
  • shadcn/ui component primitives, Radix UI primitives, lucide-react, Tailwind utilities, Sonner toasts.
  • xlsx for spreadsheet import/export.

Dockerfile

Multi-stage build:

  1. Builder: node:18-alpine, installs build deps python3 make g++, runs npm install, copies source, runs npm run build.
  2. Runtime: node:18-alpine, installs bash nano su-exec, creates non-root bill user, copies built app, creates /data/db, /data/backups, /app/backups, sets ownership and restrictive permissions.

Runtime environment:

  • NODE_ENV=production
  • PORT=3000
  • DB_PATH=/data/db/bills.db
  • BACKUP_PATH=/data/backups

Exposes port 3000, declares volume /data, entrypoint docker-entrypoint.sh, command node server.js.

docker-compose.yml

Service: bill-tracker.

  • Image: dream.scheller.ltd/null/billtracker:latest.
  • Container name: bill-tracker.
  • Ports: host 3030 to container 3000.
  • Volume: /portainer/hosting/bill-tracker/data:/data.
  • Restart: unless-stopped.
  • Environment includes INIT_ADMIN_USER, INIT_ADMIN_PASS, and CSRF cookie settings.

Important deployment note: the compose file currently sets CSRF_SECURE: "true"; for plain HTTP development this prevents CSRF cookies from being sent by browsers. Use HTTPS or override to false only in local/dev.

9. Auth and Security Flows

Local login

  1. Client calls GET /auth/mode to determine local/OIDC visibility.
  2. Client submits POST /auth/login.
  3. Server checks local_login_enabled, validates credentials through bcrypt, rejects inactive users, cleans expired sessions, creates a 7-day session.
  4. Server sets bt_session cookie using cookieOpts(req).
  5. Client calls GET /auth/me to populate auth state.

OIDC login

  1. Client navigates to /api/auth/oidc/login.
  2. Server verifies active OIDC config, creates PKCE state, redirects to provider.
  3. Provider returns to /api/auth/oidc/callback.
  4. Server validates state/nonce, exchanges code, maps/provisions user, creates local session cookie, redirects back into SPA.

Password change

  1. Current password and matching new password are required.
  2. New password must be at least 8 chars.
  3. Server updates hash, clears must_change_password, sets last_password_change_at.
  4. Other sessions are invalidated and current session is rotated when possible.

Authorization

  • All user data routes enforce owner scope by req.user.id in SQL.
  • Admin-only routes require requireAdmin on server.
  • Default admin cannot use tracker routes.
  • Role changes invalidate target sessions.
  • Deactivation invalidates target sessions.

Backup safety

  • Managed filename regex and path checks prevent traversal.
  • Uploads are written to temp paths first, validated, then moved.
  • Restore creates a pre-restore backup.

Import safety

  • Spreadsheet import accepts only XLSX and validates size, sheets, rows, cells, headers, and decisions.
  • User DB import validates SQLite magic, size, required metadata/tables, and maps all data to current user ownership.

10. Operational Notes

Startup behavior

  • DB path is DB_PATH or db/bills.db.
  • DB open logging prints only path.basename(DB_PATH) instead of the full database path, so startup logs identify the file without exposing the full filesystem location.
  • SQLite WAL and foreign keys are enabled.
  • Schema and migrations run automatically.
  • Default categories/settings are seeded.
  • Expired sessions are purged at startup.
  • A periodic expired-session cleanup interval is scheduled.
  • Backup scheduler and daily worker are started where server code imports/starts them.

Environment-seeded regular users use INIT_REGULAR_USER and INIT_REGULAR_PASS. New seeded users are inserted with first_login = 0 and must_change_password = 0. Existing seeded regular users have their password hash updated and both flags reset to 0, then the server audits seed.flag_reset with the username, reset flags, and source: "server-seed". This lets ENV-managed users skip first-login/privacy/password-change gates after seed refreshes.

Environment variables

Common variables used by current code:

  • PORT
  • DB_PATH
  • BACKUP_PATH
  • INIT_ADMIN_USER
  • INIT_ADMIN_PASS
  • INIT_REGULAR_USER
  • INIT_REGULAR_PASS
  • SESSION_CLEANUP_INTERVAL_MS
  • CORS_ORIGIN
  • COOKIE_SECURE
  • HTTPS
  • CSRF_HTTP_ONLY
  • CSRF_SAME_SITE
  • CSRF_SECURE
  • CSRF_COOKIE_NAME
  • OIDC_ISSUER_URL
  • OIDC_CLIENT_ID
  • OIDC_CLIENT_SECRET
  • OIDC_REDIRECT_URI
  • OIDC_TOKEN_AUTH_METHOD

Most notification, OIDC, backup, cleanup, and auth-mode settings are also stored in the settings table and managed from Admin UI.

Known code characteristics to preserve

  • Use transactions for multi-step destructive or bulk DB changes.
  • Keep user-owned SQL scoped by req.user.id.
  • Keep admin lockout protection before changing login methods.
  • Do not expose password_hash, session IDs, OIDC client secret, or internal backup paths in API responses.
  • Keep import preview/apply separated so users can resolve ambiguous spreadsheet data before DB writes.
  • Prefer soft delete for payments; bill deletion is intentionally hard delete and returns an explicit warning.
  • DB path support: db/database.js uses path.basename(DB_PATH) in logging to anonymize the DB path while still providing useful diagnostic information. Absolute and relative paths are both supported.

11. Verification Checklist Used for This Reference

Reviewed current code sources:

  • server.js
  • all route files under routes/
  • service files under services/
  • middleware files under middleware/
  • db/schema.sql and db/database.js
  • actual initialized SQLite schema via better-sqlite3 introspection
  • client/App.jsx, client/api.js, client/hooks/useAuth.jsx, client/hooks/useQueries.js
  • page/component inventory under client/pages/ and client/components/
  • package.json
  • Dockerfile
  • docker-compose.yml

The previous manual contained large historical update sections and stale route/page descriptions. This version replaces those with a current-state engineering reference.