BillTracker/HISTORY.md

781 lines
52 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Bill Tracker — Changelog
## v0.18
### Branding
- Replaced the top-navbar dollar-sign placeholder and duplicate text/version brand stack with the selected `/img/logo.png` BillTracker logo.
- The logo now serves as the BillTracker brand in the top navigation while preserving the existing navbar height and route behavior.
- Login now uses the BillTracker logo, shows linked build/version information near the login actions, and uses the authentik icon for OIDC login.
- Admin Authentication Methods now uses subtle authentik branding in the OIDC toggle/configuration/test-login controls.
- Cropped transparent padding from the BillTracker logo asset so it renders larger and more readably in the unchanged-height navbar.
- Promoted the transparent `logo_cut.png` artwork to the served `/img/logo.png` asset and enlarged the login-page logo while keeping the login card layout compact.
- Login logo sizing now follows the login form width so the brand grows and shrinks with the sign-in column instead of rendering too small.
- Legacy `/login.html` now redirects to the modern React `/login` screen so the old static login page is no longer served by stale links.
- Vite now copies only modern React public assets from `client/public`, preventing legacy `public/*.html`, CSS, and JS files from being emitted into `dist`.
- No backend, auth, tracker, bills, categories, settings, status, admin, or navigation-link behavior was changed.
### Security
- **OIDC ID token signature verification** now uses `openid-client@5` for full cryptographic validation via JWKS: signature, issuer, audience, expiry, nonce, and `sub` presence — tokens without a valid signature are rejected
- **OIDC client cache** invalidation path added; cache is keyed by issuer/client/redirect so Admin panel credential changes pick up a fresh client
- OIDC-provisioned accounts (empty `password_hash`, `auth_provider='oidc'`) continue to be blocked from local password login
- Tokens, auth codes, and client secrets are never logged at any point in the OIDC flow
- Session cookies no longer become `Secure` solely because `NODE_ENV=production`; this preserves login on plain-HTTP Docker deployments while still supporting `COOKIE_SECURE=true`, `HTTPS=true`, and HTTPS reverse-proxy detection.
### Added
- **Docker startup volume repair**: runtime now starts through `docker-entrypoint.sh`, creates `/data/db` and `/data/backups`, fixes `/data` ownership for the non-root `bill` user, then drops privileges before launching Node. This prevents SQLite migrations from failing with `SQLITE_READONLY` on mounted volumes.
- **Docker startup migrations**: entrypoint now runs `scripts/migrate-db.js` as the non-root app user before starting the server, so required SQLite schema migrations and seeded defaults complete before the app listens for requests. Set `RUN_DB_MIGRATIONS=false` only for special maintenance runs.
- **Database writability preflight**: startup now checks that `DB_PATH` and its parent directory are writable before opening SQLite, producing a clearer error if a bind mount or volume is genuinely read-only.
- **Release notes Markdown rendering**: the release notes viewer now renders inline Markdown such as `**bold**`, backticked code, and HTTPS links instead of showing the raw markers.
- **authentik configuration testing**: Admin Authentication Methods now includes a live OIDC discovery test for the entered issuer/client/redirect settings and a direct authentik login test button once OIDC is enabled.
- **authentik setup guardrails**: the Admin form now fills the Redirect URI from the current app origin, offers a "Use Current" reset, and warns when the Issuer URL looks like an authorize/token/userinfo endpoint instead of the provider issuer.
- **authentik client auth method**: Admin OIDC settings now include an advanced `client_secret_basic` / `client_secret_post` token endpoint authentication method selector. The default remains `client_secret_basic`, matching the previous `openid-client` behavior.
- **Admin user role management**: Admin Users table now lets an admin promote another user to `admin` or demote an admin back to `user`, with protections against changing your own role or removing the last admin account.
- **Single-user mode recovery**: User Settings now shows a Login Mode section while single-user mode is active, allowing the default user to restore multi-user login without needing access to Admin routes.
- **Admin navigation parity**: Admin users now keep the normal app navigation and get an Admin link after Status; `/admin` uses the same top nav so admins can return to Tracker/Bills/Categories/Profile/Settings/Status without typing a URL. Backend `/admin` protection remains unchanged.
- **Admin-controlled auth method toggles** in Admin panel (Authentication Methods card):
- `local_login_enabled` — enable/disable local username/password login (default: enabled)
- `oidc_login_enabled` — enable/disable OIDC/authentik login (default: disabled)
- Database-backed authentik/OIDC provider settings: provider name, issuer URL, client ID, client secret, redirect URI, scopes, auto-provision, admin group, default role
- Lockout protection: admin cannot disable all login methods; cannot disable local login unless OIDC is configured, enabled, and has an admin group mapping
- Client secret is write-only in the UI/API; Admin GET returns only `oidc_client_secret_set`
- **`GET /api/admin/auth-mode`** extended: returns `local_login_enabled`, `oidc_login_enabled`, `oidc_configured`, `can_disable_local`, `warnings`, safe OIDC settings, and the client-secret marker
- **`PUT /api/admin/auth-mode`** extended: accepts all new provider settings, allows setting a new client secret, keeps the existing secret when blank, and supports explicit saved-secret clearing
- **`GET /api/auth/mode`** extended: returns `local_enabled`; returns OIDC provider name and login URL only when OIDC is enabled and fully configured
- **`POST /api/auth/login`** now checks `local_login_enabled` setting and returns 403 if admin has disabled local login
- **Login page OIDC button** uses `/api/auth/mode` so local-only, OIDC-only, and mixed login states are reflected safely
- **OIDC login and callback routes** now check both DB-backed effective OIDC config and `oidc_login_enabled` before proceeding
- Eleven auth settings keys are seeded: `local_login_enabled`, `oidc_login_enabled`, `oidc_provider_name`, `oidc_issuer_url`, `oidc_client_id`, `oidc_client_secret`, `oidc_redirect_uri`, `oidc_scopes`, `oidc_auto_provision`, `oidc_admin_group`, `oidc_default_role`
- **`scripts/test-oidc-smoke.js`** — 42 smoke tests covering PKCE, redirect sanitization, DB/env OIDC config precedence, safe secret handling, incomplete config behavior, provisioning, email linking, role/group mapping, OIDC-only local-login denial, and lockout logic; all pass
### Changed
- `services/oidcService.js` rewritten to use `openid-client@5` throughout:
- `getOidcClient(config)` — builds and caches an openid-client `Client` after OIDC discovery
- `buildAuthorizationUrl()` uses `client.authorizationUrl()` (uses discovered `authorization_endpoint`)
- `exchangeAndVerifyTokens()` replaces manual exchange + claims-only validation with `client.callback()` which does the full PKCE exchange, JWKS signature verification, and all claim checks in one call
- `getOidcConfig()` resolves provider config from DB settings first, env fallback second, safe defaults last
- `mapRoleFromClaims()` reads the effective admin group at runtime; default role remains `user`, and admin is granted only by explicit group match
- `findOrProvisionUser()` uses effective `oidc_auto_provision` and creates local OIDC users on first valid authentik login when enabled
### Notes
- Local username/password login remains supported and protected by lockout checks
- OIDC environment variables remain optional fallback/bootstrap values only; once a DB field is set, the DB value takes precedence
- Client secret is stored in the existing `settings` table, never returned through public endpoints, and never displayed in the UI
- Valid authentik users auto-provision local app users when enabled; unknown users receive a safe 403 when disabled
- Admin role is never granted by default; requires explicit OIDC admin group membership or local admin account
- OIDC signature verification/security is preserved through `openid-client@5` JWKS-backed validation, issuer/audience/expiry/nonce checks, PKCE, and state replay protection
- Live authentik SSO cannot be verified without a running authentik instance; all local testable code paths are covered by the smoke test script
- Admin single/multi user mode behavior was not changed
---
## v0.17
### Security
- **Rate limiting** added via `express-rate-limit`: login (10/15 min), password change (5/15 min), import (20/15 min), export (30/15 min), admin mutations (30/15 min), OIDC (20/15 min) — all per-IP, in-memory
- **Security headers** added globally: `X-Content-Type-Options: nosniff`, `X-Frame-Options: SAMEORIGIN`, `Referrer-Policy: strict-origin-when-cross-origin`, `X-Permitted-Cross-Domain-Policies: none`; `X-Powered-By` removed; `Strict-Transport-Security` added when `HTTPS=true`
- **CORS locked down**: `cors({ credentials: true })` (wide-open) replaced with opt-in via `CORS_ORIGIN` env var; without it, no CORS headers are sent and the browser's same-origin policy applies
- **Cookie `secure` flag**: session cookie now sets `secure: true` in production (`NODE_ENV=production` or `HTTPS=true`), preventing transmission over plain HTTP in deployed environments
- **Settings endpoint hardened**: `GET /api/settings` now returns only the four user-facing keys (`currency`, `date_format`, `grace_period_days`, `notify_days_before`); previously returned all settings rows including SMTP password hash and backup paths
- **DB path removed from status response**: `database.path`/`database.file` fields removed from `GET /api/status`; filesystem paths are no longer exposed to any authenticated user
- **OIDC-only account protection**: `login()` now rejects local-password login attempts for accounts provisioned via external OIDC, preventing bypass of SSO-only accounts
- **Error handler hardened**: global Express error handler logs `err.message` internally but no longer calls `console.error(err)` (which could log full stack traces with paths); user-facing errors remain useful but safe
- **CSP deferred**: Content-Security-Policy requires auditing inline styles from Tailwind and Radix event handlers; deferred to a dedicated hardening pass
### Added
- **Backend OIDC/authentik preparation** — disabled by default; activated only when `OIDC_ENABLED=true` plus all required env vars are present:
- `GET /api/auth/oidc/login` — generates PKCE code verifier + challenge, stores one-time state in `oidc_states` DB table, redirects to the identity provider's authorization endpoint
- `GET /api/auth/oidc/callback` — validates state, exchanges authorization code for tokens (PKCE), validates ID token claims (issuer, audience, expiry, nonce), maps to local user, creates session, redirects to frontend
- `GET /api/auth/mode` now includes `oidc_enabled`, `oidc_provider_name`, and `oidc_login_url` when OIDC is configured
- `services/oidcService.js`: OIDC discovery caching, PKCE helpers, login-state management, token exchange, claim validation, role/group mapping, user auto-provisioning
- `middleware/rateLimiter.js`: rate limiter factory for all endpoint categories
- `middleware/securityHeaders.js`: global security headers middleware
- `authService.createSession(userId)`: creates a server-side session for a user who has already been authenticated externally (used by OIDC callback)
- **Schema migrations** (v0.17, additive — safe on existing databases):
- `users.auth_provider TEXT DEFAULT 'local'` — tracks identity source (`local` | `oidc`)
- `users.external_subject TEXT` — stores OIDC `sub` claim for stable identity mapping
- `users.email TEXT` — stores email for OIDC-linked accounts and optional local-user email linking
- `users.last_login_at TEXT` — updated on every successful login (local or OIDC)
- `oidc_states` table: short-lived (5 min TTL) PKCE/nonce state for in-flight OIDC logins; pruned on each new login attempt
### Changed
- `authService.login()` now updates `last_login_at` on each successful local login
- `requireAuth` single-user-mode query now includes `display_name` so the top nav shows correctly in single-user mode
### Notes
- Local username/password authentication continues to work regardless of OIDC configuration
- OIDC requires `OIDC_ENABLED=true`, `OIDC_ISSUER_URL`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`, `OIDC_REDIRECT_URI`
- JWT signature verification (JWKS) is not implemented in this pass — ID token *claims* are validated but the cryptographic signature is not. Install `openid-client@5` and upgrade `validateIdToken` in `services/oidcService.js` for full production-grade signature verification
- Admin full-database backup/restore was not changed
- No frontend SSO login button was added in this pass
---
## v0.16.2
### Added
- User SQLite data import backend for exports created by this app:
- `POST /api/import/user-db/preview`
- `POST /api/import/user-db/apply`
- SQLite import preview validates the uploaded file, confirms it is a BillTracker user data export, summarizes counts, warnings, and proposed create/skip/conflict actions, and writes no live data
- SQLite import apply uses the preview session, imports only into the signed-in user's account, creates missing user-owned records, skips duplicates/conflicts by default, and records import history
- Regression coverage for user SQLite preview, apply, conflict skipping, and invalid file rejection
### Changed
- Profile > My Data now shows Import Spreadsheet History and Import SQLite Data Export side by side on desktop and stacked on mobile
- Export My Data now appears below the two import tools
- The SQLite import UI includes file selection, preview summary, warnings/conflicts, confirmation before apply, and result/error states
### Security
- User SQLite import does not use admin backup/restore endpoints and does not perform a full system restore
- User SQLite import does not import users, password hashes, sessions, cookies, admin/global settings, SMTP credentials, backup files, server paths, or other users' data
- Uploaded SQLite files are read as data through parameterized queries, stored temporarily only for preview parsing, and cleaned up without exposing server paths
---
## v0.16.1
### Added
- Bills now support an optional `interest_rate` field for credit-card APR values; blank remains allowed for non-credit-card bills
- The bills database schema, API create/update routes, bill responses, and user exports now preserve `interest_rate`
- Bills page bill names now open the shared Edit Bill dialog directly
### Changed
- Edit Bill due-date editing now uses a recurring day-of-month number instead of a full calendar date picker
- Removed the old due-date wording from the shared Edit Bill UI; the field is now labeled "Due day of month"
- Bills page editing no longer relies on a separate Edit button as the primary action
- Bills page and Tracker both use the same shared Edit Bill dialog with the corrected due-day and APR fields
- Tracker due-date calculation now uses the recurring `due_day` field and clamps to shorter months using the existing month-end handling
### Notes
- The legacy `override_due_date` column remains in the database for compatibility, but the shared Edit Bill dialog no longer edits it and current tracker due-date calculation ignores it
- Payment, monthly state, bill delete, deactivate, reactivate, and global grace-period behavior were not changed
- Per-bill grace periods were not added
---
## v0.16
### Added
- Admin Cleanup / Maintenance panel in the Admin area with settings for all four cleanup tasks, last-run summary, Save Settings, and Run Cleanup Now button
- Import history trimming shows an explicit destructive-action warning when enabled
- Read-only Maintenance card on the user Status page showing last cleanup run timestamp and per-task result counts (import sessions pruned, temp files removed, backup partials removed); no admin controls exposed
- Tracker page: clicking a bill name opens the existing Edit Bill dialog for that bill; saving refreshes the tracker for the current month; monthly gear (⚙) still opens monthly state, not global bill edit
- Edit Bill dialog is now a shared component (`components/BillModal.jsx`); Bills page and Tracker page both import it — no duplicate dialog
### Changed
- `GET /api/auth/me` now returns `display_name` so the top-nav user menu always shows the current display name after Profile save without requiring logout/login
- `GET /api/status` now includes a safe read-only `cleanup` section: `last_run_at` and `last_result` task counts
- Edit Bill due-date helper text clarified that grace period is a global Setting
### Notes
- Per-bill grace period days are not a backend-supported field; grace period remains a global app setting in Settings
- Admin cleanup controls (Save Settings, Run Cleanup Now) are admin-only and do not appear on the user Status page
- Profile `display_name` save already merged correctly into local state; the auth session fix ensures `refresh()` also returns the updated value
---
## v0.15
### Added
- `services/cleanupService.js` — new cleanup service with four independent tasks:
- **Expired import sessions** — deletes `import_sessions` rows past their 24-hour expiry (previously only pruned on next preview request; now also runs daily)
- **Stale export temp files** — removes orphaned `bill-tracker-user-*.sqlite` files from the OS temp directory that were not deleted after an interrupted download; configurable max age (default 2 hours)
- **Orphaned backup partials** — removes `.partial` and `.upload` files left in the backup directory after a server crash; uses a fixed 2-hour safety cutoff so in-progress operations are never interrupted
- **Import history trimming** — optionally deletes `import_history` rows older than a configurable threshold; disabled by default
- Daily worker now runs all enabled cleanup tasks each morning after notifications; cleanup errors are caught and logged but do not fail the worker
- Admin cleanup API:
- `GET /api/admin/cleanup` — returns current cleanup settings and last run result
- `PUT /api/admin/cleanup` — update any combination of cleanup settings (partial updates supported)
- `POST /api/admin/cleanup/run` — trigger all enabled cleanup tasks immediately and return the result
- Eight new `cleanup_*` keys in the `settings` table with safe defaults (seeded via `INSERT OR IGNORE`)
### Notes
- No frontend admin UI for cleanup settings in this pass — backend and APIs only
- All cleanup tasks are independently toggled; disabling one does not affect others
- Import history trimming is off by default to preserve audit history; enable explicitly via `PUT /api/admin/cleanup`
- Backup partial pruning always uses a 2-hour minimum age regardless of settings so a live backup in progress is never removed
---
## v0.14.3
### Changed
- Added the shadcn Material Design theme registry setup and made Material Design the default light theme for the app
- `:root` now represents the Material Design light tokens used by the existing Tailwind/shadcn CSS variable system
- Aligned dark mode to the same Material-inspired design language so light and dark mode feel related
- Modernized shared theme surfaces and app shell styling, including the top navigation, cards, dialogs, dropdowns, buttons, inputs, badges, tables, and focus/hover/disabled states
### Notes
- The app still uses the existing Tailwind, shadcn, and Radix framework; no new UI framework was introduced
- Material Design is not an optional user-selected variant; it is the default light theme
- Backend behavior, routes, tracker, bills, payments, import/export logic, and database behavior were not changed
---
## v0.14.2
### Added
- Inactive bills now have a History Visibility editor on the Bills page
- History Visibility supports Default, Show all history, Show no history, and Show only selected date ranges
- Selected ranges mode supports adding, editing, deleting, labeling, and saving multiple year/month ranges
### Changed
- Bills with non-default history visibility or saved history ranges continue to show the history visibility indicator after saving
### Notes
- The editor is available for inactive bills only; active bills may show the indicator but do not expose the editor
- Delete, deactivate, and reactivate behavior was not changed
- No backend behavior changed
---
## v0.14.1
### Added
- Bills page now exposes permanent bill deletion behind a strong confirmation dialog
- Delete confirmation explains that payments, monthly history, notes, and history ranges are permanently deleted and cannot be undone
- Delete confirmation requires an explicit acknowledgement checkbox before the destructive action is enabled
- Bills with historical visibility metadata now show a small history visibility indicator in the Bills table
### Notes
- Deactivate/reactivate remains the safe non-destructive option and still uses the existing active/inactive update behavior
- The delete dialog offers Deactivate instead for active bills and Activate instead for inactive bills
- No backend delete, deactivate, reactivate, payment, monthly state, or history range behavior changed
---
## v0.14
### Added
- Bill hard-delete: `DELETE /api/bills/:id` now permanently removes the bill and all associated payments, monthly state, and history ranges — inactivation (`PUT` with `active: 0`) remains the safer non-destructive alternative
- Bill history visibility: bills now carry a `history_visibility` field (`default`, `all`, `ranges`, `none`) for future UI control over which historical data is shown for inactive bills
- `bill_history_ranges` table: per-bill, multi-range date records for fine-grained history visibility control
- `GET /api/bills/:id/history-ranges` — list all history ranges for a bill
- `POST /api/bills/:id/history-ranges` — add a date range (start year/month, optional end year/month, optional label)
- `PUT /api/bills/:id/history-ranges/:rangeId` — update a history range
- `DELETE /api/bills/:id/history-ranges/:rangeId` — remove a history range
- Bills list and detail responses now include `history_visibility` and `has_history_ranges` flag for future UI icon support
### Changed
- `DELETE /api/bills/:id` changed from soft-delete (set active=0) to hard-delete; clients that need deactivation should use `PUT /api/bills/:id` with `{ active: 0 }` (unchanged behavior)
- AP flag badge on the Bills page is now emerald/green and uses boolean coercion to prevent the SQLite integer `0` from rendering as a visible "0" next to the badge
### Notes
- No delete-confirmation UI added in this pass — backend only for the delete/history changes
- No history-visibility UI added — backend and data model only
- All history-range queries are scoped to the bill's owning user
- Deleting a bill cascades automatically via database foreign keys (`ON DELETE CASCADE`)
---
## v0.13.3
### Changed
- Moved authenticated navigation from a left-sidebar-first shell to a top-navigation-first app header
- Aligned authenticated pages under the new shared top-nav shell with consistent page width, padding, background, and card surface styling
- Modernized the shared app background, top nav active states, user menu, theme toggle placement, and mobile navigation menu
### Notes
- Profile remains the user/account hub, and user Data tools remain under Profile > My Data
- Settings remains focused on app-level preferences
- Admin/system controls remain separated from regular user Profile and continue to be shown only in the admin area
- No backend import/export, tracker, bill, category, or payment behavior changed
---
## v0.13.2
### Changed
- User-owned spreadsheet import, user data export, user data import placeholder, and import history tools now live under Profile > My Data
- Removed the top-level Data item from the regular sidebar; `/data` now redirects to `/profile` for backward-compatible deep links
- Settings is narrowed to app-level preferences: appearance, currency, date format, and billing grace period
### Notes
- Admin/system backup and restore controls remain separate from regular user Profile
- Status remains a standalone operational/system health page
---
## v0.13.1
### Added
- Profile page frontend at `/profile` with profile summary, display-name editing, notification preferences, password change, user-owned exports, and import history
- Sidebar signed-in user name now links to the Profile page
- Profile API helpers for profile details, profile settings, password changes, export metadata, and import history
- User export download buttons for SQLite and Excel exports from the Profile page
### Fixed
- Startup crash: `UPDATE categories SET user_id = ?` now removes orphaned NULL-owner categories whose names already exist for the target user before assigning ownership, preventing a `UNIQUE(user_id, name)` constraint violation on databases that had previously completed the v0.12 migration
### Notes
- Profile page uses user-owned export endpoints only and does not include admin backup controls or backup paths
- Password values are only kept in local form state for submission and are cleared after successful password change
---
## v0.13
### Added
- Profile backend foundation: `GET /api/profile` returns safe user data (id, username, display_name, role, created_at, updated_at, last_password_change_at, notification preferences, export links)
- `PATCH /api/profile` — updates `display_name` (only safe user-owned field)
- `GET /api/profile/settings` — returns user-owned notification preferences from the users table
- `PATCH /api/profile/settings` — updates user notification preferences (partial update; omitted fields are preserved)
- `POST /api/profile/change-password` — strict password change requiring `current_password`, `new_password`, and `confirm_new_password`; always verifies the current password regardless of account state; records `last_password_change_at` on success
- `GET /api/profile/exports` — returns metadata and links for user data export actions (SQLite and Excel)
- `GET /api/profile/import-history` — returns the signed-in user's import history (delegates to existing import service)
- `display_name` and `last_password_change_at` columns added to the users table via additive migration
### Changed
- `PUT /api/settings` no longer accepts backup-related keys (`backup_enabled`, `backup_frequency_days`, `backup_keep_count`, `backup_path`) from regular users; those settings are admin-only and remain manageable through the admin backup routes
### Security
- All `/api/profile/*` endpoints require authentication; user identity is always derived from the session — `user_id` is never accepted from the request body
- Profile responses never include password hashes, session tokens, SMTP credentials, backup paths, or other users' data
- `POST /api/profile/change-password` always requires the current password; no bypass path exists
### Notes
- No frontend Profile UI in this pass — backend APIs only
- Admin full-database backup/restore behavior was not changed
- Existing `POST /api/auth/change-password` preserved for the first-login forced-reset flow
---
## v0.12
### Added
- Bills and categories now have database ownership fields so imported and manually created bills belong to the signed-in user
- User data exports are now enabled for SQLite and Excel formats
- User exports include only the signed-in user's bills, payments, categories, monthly bill state, notes, and export metadata
### Fixed
- Bill, category, payment, tracker, spreadsheet import, and export routes now filter user-owned data instead of using global bill/category records
### Notes
- Existing unowned bills/categories are assigned to the first regular user during migration when one exists
- Admin full-system backup/export behavior was not changed
---
## v0.11.4
### Added
- Creating a new bill from an XLSX import row now also imports other paid months for the same detected bill name from the current preview
- Related paid months create monthly state and payment records on the newly created bill
### Notes
- Related-month import is limited to exact normalized bill-name matches in the reviewed XLSX preview
- Rows for other bill names remain skipped and are not imported automatically
---
## v0.11.3.2
### Fixed
- Login now updates the shared auth state before navigating, preventing the app from sitting on the login flow after successful sign-in
- First-login password change now sends the backend field name expected by the auth route
- Privacy acknowledgment refreshes auth state before continuing to the tracker
---
## v0.11.3.1
### Fixed
- XLSX import history and apply summary now record the number of rows skipped during review even though skipped rows are not sent as apply decisions
- Import review still applies only confirmed non-skipped rows, preserving the safer focused apply payload
### Tests
- Added regression coverage that omitted reviewed skips are counted in both apply results and import history
---
## v0.11.3
### Added
- XLSX import now detects `Paid Date` / `Date Paid` columns separately from due-date columns
- Confirmed XLSX imports now create payment records from detected paid dates and paid amounts so imported bills can appear paid in the tracker
- Import review rows now show and allow editing detected paid date and paid amount before Apply
### Notes
- Payment creation still only happens after the user confirms and applies the reviewed row
- Duplicate payment records with the same bill, amount, and paid date are skipped unless overwrite is enabled
---
## v0.11.2.5
### Fixed
- XLSX import apply now sends only rows the user chose to import instead of also sending every skipped preview row
- Import request parser failures on `/api/import/*` now return safe import-specific JSON errors instead of the generic `Internal server error`
### Notes
- Skipped preview rows remain visible and editable on the review screen, but they are not applied or created silently
- User confirmation is still required before creating a new bill, matching an existing bill, or applying any XLSX import row
---
## v0.11.2.4
### Fixed
- Fixed XLSX import month detection for real workbook tabs that combine month and year without a separator, such as `July2017`, `August2017`, and `September2017`
- Added tolerance for known month-name typos found in the test workbook, including `Januaru`, `Febuary`, and `Novevmber`
- All-sheets XLSX preview now skips obvious non-bill tabs such as `info`, tax/debt summary sheets, generic `Sheet13`-style tabs, and `home ownership expenses`
### Notes
- Electric rows from `Test_Data/monthly bills.xlsx` now resolve to the correct 2017 month values instead of becoming ambiguous because of tab-name parsing
- Import recommendations and apply remain confirmation-only; no auto-apply behavior was added
---
## v0.11.2.3
### Fixed
- Fixed remaining XLSX import apply error paths by normalizing and validating import decision fields before SQL writes
- Import apply now returns structured validation errors with row/field details when possible instead of generic Internal Server Error responses
- Import review now keeps the preview visible after apply failure so decisions can be corrected and retried
### Tests
- Added regression coverage for unsupported actions, unknown row IDs, missing create-new-bill names, invalid year/month values, bulk-style create-new-bill payloads without category IDs, frontend/backend match payloads, and skip rows without bill/month fields
---
## v0.11.2.2
### Fixed
- Fixed XLSX import apply validation for matching rows to existing bills
- Invalid match-existing-bill decisions now return clear validation errors instead of falling through to generic server errors
### Tests
- Added regression coverage for matched existing bill applies with numeric and string bill IDs, invalid match payloads, monthly state writes, bill template preservation, create-new-bill, and skip-row behavior
---
## v0.11.2.1
### Changed
- XLSX import bulk row tools are now visually scoped inside the XLSX Review Table, directly above the preview rows
- Bulk controls no longer use page-level sticky positioning, making it clearer they belong only to the current XLSX preview
### Notes
- Bulk actions still affect only selected XLSX preview rows and do not auto-apply imports
---
## v0.11.2
### Added
- Import review now supports bulk row selection and bulk row actions for the current XLSX preview
- Users can bulk mark selected preview rows as Skip
- Users can bulk mark selected preview rows as Create New Bill with editable prefilled name, category, due day, expected amount, actual amount, and notes when available
- Selected rows can be reset back to their recommendation/default decision
### Notes
- Bulk actions only update review decisions; no auto-apply or silent bill creation was added
- Per-row confirmation and editing remain available before Apply
- Rows without usable bill names remain unresolved after bulk Create New Bill until the user enters a name or skips the row
- Ambiguous rows are not auto-matched by bulk actions
---
## v0.11.1.1
### Changed
- Import review rows now let users override recommended matches and choose Create New Bill even when a possible existing bill match is suggested
- Create New Bill action switching now keeps the row expanded, hides the existing-bill selector, and sends a create-new-bill payload without the recommended bill ID
### Notes
- Recommendations remain confirmation-only and are not auto-applied
- Ambiguous rows still block Apply until the user chooses a valid action or skips the row
---
## v0.11.1
### Added
- XLSX import preview rows now include a `recommendation` object for pre-filling the review screen without applying changes automatically
- Smarter bill matching tolerates casing, punctuation, token order, and common abbreviations, including examples like `Capital` / `Cap One``Capital One` and `Discover Austin``Austin Discover`
- Create-new-bill recommendations now prefill likely bill name, due day, expected amount, and detected monthly amount when available
- Category suggestions use existing categories only, from explicit category columns, obvious keywords, or strongly matched similar bills
- Due-day suggestions are parsed from spreadsheet date/due-date cells when a valid day can be detected
- Amount recommendations carry detected spreadsheet amounts into confirmed monthly state decisions, and create-new-bill expected amounts when appropriate
### Changed
- Import review UI now shows the recommendation action, confidence, reason, and warnings before apply
- Apply payloads now include confirmed `category_id`, `due_day`, `expected_amount`, `actual_amount`, month/year, and notes when the user accepts or adjusts recommendations
- Weak or multiple possible bill matches remain unresolved until the user chooses a bill, creates a new bill, or skips the row
### Notes
- Recommendations are never auto-applied; the user must still review the preview screen and press Apply
- Ambiguous rows are not auto-applied and continue to block Apply until resolved
- Categories are not auto-created in this pass
- Existing bills are not silently updated, including due days or expected amounts; mismatches are shown as warnings
- Date intent is still heuristic: columns that look like payment dates lower confidence and warn rather than treating the date as a definite due date
---
## v0.11
### Added
- New dedicated **Data** page (`/data`) accessible from the sidebar with a FolderInput icon
- **Import Spreadsheet History** section: XLSX file picker with parse-all-sheets toggle, default year/month inputs, and a Preview Import workflow
- Preview UI shows workbook summary (sheet names, detected year/month, row counts, status badges), grouped by sheet tab in multi-sheet mode
- Per-row decision controls: auto-preselects high-confidence bill matches; ambiguous rows surface as "Needs decision" and block apply until resolved; user can choose match existing bill, create new bill, update monthly record, add monthly note, record as payment, or skip
- Suggested-match quick buttons and bill selector using optgroup (suggested matches / all bills) for fast selection
- Apply bar shows live summary of rows to apply, skipped, and unresolved; Apply button disabled until all rows are resolved
- Post-apply result card with created/updated/skipped/error counts; "New Import" button to start over
- **Download My Data** section (moved from Settings): SQLite and Excel export cards (Coming soon badges while backend endpoints are pending)
- **Import My Data Export** section: placeholder card explaining user-scoped SQLite import with Coming Soon state; clearly labelled as distinct from admin DB restore
- **Import History** section: table of all past imports for the current user with timestamps, file names, sheet names, and row outcome counts
- API helpers added: `api.previewSpreadsheetImport()`, `api.applySpreadsheetImport()`, `api.importHistory()`
### Changed
- Removed "Download My Data" section from Settings page — now lives exclusively on the Data page
- Settings page icon imports trimmed to only what the page uses
### Notes
- Admin full-database backup/restore remains exclusively in the Admin panel and is not accessible from the Data page
- User SQL import (user-scoped SQLite restore) shows a Coming Soon card; backend endpoints `POST /api/import/user-db/preview` and `POST /api/import/user-db/apply` are still needed
- User data exports (SQLite + Excel) remain Coming Soon; backend endpoints `GET /api/export/user-db` and `GET /api/export/user-excel` are still needed
---
## v0.10.1
### Added
- Multi-sheet preview mode: pass `?parse_all_sheets=true` to `POST /api/import/spreadsheet/preview` to parse every tab in one XLSX upload
- `parseSheetName()` — detects year/month from worksheet tab names supporting: `Jan 2026`, `January 2026`, `2026-01`, `01-2026`, `May`, `May Bills`, `Bills May 2026`, `2026 May`, and any combination of month name (full or abbreviated) with an optional 4-digit year
- Known non-data sheet names (Summary, Totals, Dashboard, Notes, Categories, Settings, Overview, Template, etc.) are automatically skipped in multi-sheet mode with `status: "skipped"` in the response
- `resolveYearMonth()` — tracks where each row's year/month came from; new `year_month_source` field on every preview row: `row_date`, `sheet_name`, `default`, or `ambiguous`
- Every row now includes `sheet_name` identifying which worksheet it came from
- Multi-sheet workbook response includes a `sheets` array with per-tab metadata: detected year, detected month, status (`parsed`, `parsed_month_only`, `ambiguous`, `skipped`), and row count
- Rows on ambiguous tabs (no detectable month) carry `year_month_source: "ambiguous"` and a warning; apply treats them normally using decision-level year/month override if provided
- `resolveYearMonth` and `parseSheetName` exported from service module for direct testing without DB
- Multi-sheet XLSX fixture (`scripts/test-import-multi-fixture.xlsx`) with Jan 2026, Feb 2026, Summary (skipped), May (month-only), and Misc Data (ambiguous) tabs
### Changed
- Single-sheet preview response now includes `parse_mode: "single_sheet"` in workbook metadata for consistency
- `parseXlsxBuffer()` now returns the full workbook object; raw-row extraction moved to `getSheetRows()`; `parseSheetRows()` extracted as reusable helper
- All preview rows now include `sheet_name` and `year_month_source` fields (single-sheet mode gets the selected sheet name and correct source)
- Row IDs in multi-sheet mode use `s{sheetIndex}_r{rowNumber}` format to guarantee uniqueness across tabs; single-sheet mode keeps existing `row_{N}` format
### Notes
- Apply behavior unchanged — `previewRow.detected_year` and `detected_month` already carried sheet-derived values through the session; no apply-path code changes needed
- Single-sheet preview behaviour is fully backward-compatible
- "1st14th" style bucket names are treated as ambiguous (no month detected); provide `?month=N` default if needed
---
## v0.10
### Added
- XLSX spreadsheet import backend for importing historical bill data from Google Sheets exports (no direct Sheets API connection; user uploads an exported XLSX file)
- `POST /api/import/spreadsheet/preview` — parses an uploaded XLSX file safely, classifies rows, detects bill names/amounts/dates/labels, matches against existing bills, and returns a structured preview with proposed actions; writes no data
- `POST /api/import/spreadsheet/apply` — accepts confirmed import decisions from a preview session and applies only those decisions in a single transaction; ambiguous, conflicting, and skipped rows are never applied without explicit user confirmation
- `GET /api/import/history` — returns the authenticated user's import history (last 100 imports)
- Import session table (`import_sessions`) stores temporary preview state scoped to the uploading user; sessions expire after 24 hours and are cleaned up on next preview
- Import history table (`import_history`) records a per-user audit log of every apply: filename, sheet, row counts by outcome, and a decision summary
- Bill matching: exact normalized-name matches proposed automatically; partial/token-overlap matches require user confirmation; multiple matches mark row as requires_user_decision
- Duplicate detection for payments and monthly bill state; existing records are preserved by default unless `overwrite: true` is passed
- XLSX magic-bytes validation and formula parsing disabled (`cellFormula: false`) to prevent formula execution
- Test script (`scripts/test-import.js`) covering parseAmount, parseDate, detectLabels, normalizeName, row classification, XLSX round-trip, and fixture generation; also saves a test XLSX file for manual API testing
### Security
- Import endpoint requires authenticated user session; user_id is always taken from session, never from request body
- XLSX formula parsing disabled; all cells treated as plain string data
- File size limited to 10 MB; magic-bytes check rejects non-XLSX uploads
- Import sessions and history are scoped by user_id; sessions validate user ownership on load
- Temp/session data is not the original binary file; parsed row data is stored in the session table and cleaned up after apply or expiry
### Notes
- No frontend UI yet; this is the backend foundation for the import workflow
- No direct Google Sheets API integration in this pass; input is a user-exported XLSX file
- The `bills` table does not have a `user_id` column (shared household design); imported bills enter the shared pool, consistent with existing app behavior. Import history and sessions are per-user
- The `xlsx` (SheetJS) Community Edition has known prototype-pollution and ReDoS CVEs with no available OSS patch; mitigations applied (formula parsing off, size cap, magic-bytes check, authenticated-only access). Consider migrating to `exceljs` if stricter isolation is required
---
## v0.9
### Added
- "Download My Data" section in Settings with user-facing export cards for SQLite and Excel formats
- Export cards display "Coming soon" status pills and disabled buttons until backend endpoints are implemented
- "What's included" and "What's not included" info panels clarify that exports contain only the signed-in user's own data, not system backups
- Placeholder comments in `api.js` documenting the needed `GET /api/export/user-db` and `GET /api/export/user-excel` endpoints
---
## v0.8.2
### Fixed
- Docker runtime image now creates writable `/data/db`, `/data/backups`, and fallback `/app/backups` directories for the non-root app user
- Docker Compose now builds the project Dockerfile and mounts persistent storage at `/data` instead of bypassing the image setup with a plain Node image
- Docker Compose no longer requires a present `.env` file; explicit service environment defaults remain in the compose file
- Docker Compose now stores the SQLite database at `/data/db/bills.db`, matching the persistent `/data` volume and Dockerfile defaults
---
## v0.8.1
### Fixed
- Backup storage now respects `BACKUP_PATH` and otherwise derives a writable backup directory from the configured database path, preventing container permission errors from attempts to create `/app/backups`
---
## v0.8
### Added
- Admin backup management UI for creating, importing, listing, downloading, restoring, and deleting managed SQLite backups
- Admin scheduled-backup controls for enabling daily or weekly scheduled backups, choosing run time, setting scheduled-backup retention, saving settings, and running a scheduled backup immediately
- Scheduled backup worker using existing cron support; scheduled backups use managed `scheduled-backup-...sqlite` IDs and the same safe backup directory
- Admin backup settings endpoints for reading/saving schedule settings and running a scheduled backup on demand
### Changed
- Backup metadata now includes a backup type: manual, scheduled, imported, or pre-restore
- Backup status includes scheduled backup settings, next run, retention count, and last error when available
- Settings writes now update `updated_at` correctly so scheduled backup settings can be saved
### Security
- Backup delete is admin-only and only deletes managed backup IDs inside the controlled backup directory
- Scheduled retention only deletes old scheduled backups, never manual, imported, or pre-restore backups
---
## v0.7
### Added
- Admin-only `POST /api/admin/backups/import` endpoint for importing uploaded SQLite backup files into the managed backup directory
- Imported backups use server-generated `imported-backup-...sqlite` IDs, checksum metadata, and the same path validation as managed backups
### Security
- Uploaded backup imports are written to controlled temporary files, SQLite integrity-checked, and only promoted after validation succeeds
- Invalid or empty uploaded backup files are rejected without leaving temporary artifacts behind
---
## v0.6.1
### Security
- Backup status metadata now uses managed backup IDs instead of exposing filesystem paths
- Invalid SQLite backup files now fail restore validation with a safe HTTP 400 error
### Changed
- Backup status counts now use the backup service's managed backup list instead of scanning arbitrary files in the backup directory
---
## v0.6
### Added
- Admin-only SQLite database backup endpoints for creating, listing, downloading, and restoring backups
- Secure backup service with generated timestamped backup IDs, checksum metadata, SQLite integrity validation, and controlled backup directory handling
- Restore flow that creates a pre-restore safety backup before replacing the live database
### Security
- Backup download and restore IDs are strictly validated to prevent path traversal, absolute paths, nested paths, and arbitrary file access
- Backup API returns safe metadata only and does not expose backup directory paths
---
## v0.5
### Added
- `GET /api/version/history` returns the full `HISTORY.md` contents, current package version, and changelog last-modified timestamp
- Dedicated Release Notes page at `/release-notes` with loading, error, empty, and full-history display states
- Status page Release Notes card with current version, changelog last-updated timestamp, preview, and link to the full Release Notes page
- Frontend API helper `releaseHistory()` for fetching full release history
### Changed
- Status page release notes area now stays compact and links to the dedicated full-history view
---
## v0.4
### Added
- Status page operations cards for daily worker, notifications, backups, server clock, tracker health, and recent errors
- `/api/status` now returns backward-compatible operational status sections: `worker`, `notifications`, `backups`, `server`, `tracker`, and `recent_errors`
- Lightweight in-memory status runtime for worker state, notification test/error state, and recent backend errors
### Changed
- Quick Pay and inline payment creation on the Tracker page now scope new payments to the selected tracker month/year
- Status page runtime memory now reads `runtime.memory_mb` from the backend
- Status backend now includes database `last_modified`, server time, timezone, and current-month tracker counts
### Fixed
- Quick Pay no longer records payments against today's real month when viewing a previous or future tracker month
---
## v0.3
### Added
- `monthly_bill_state` table: stores per-bill, per-month overrides for actual_amount, notes, is_skipped
- `GET /api/bills/:id/monthly-state?year=&month=` — retrieve monthly state for a bill (returns defaults if no state set)
- `PUT /api/bills/:id/monthly-state` — upsert monthly state (actual_amount, notes, is_skipped) for a specific bill+month
- Tracker response now includes `actual_amount`, `monthly_notes`, and `is_skipped` fields per row from monthly_bill_state
- Export CSV now includes `Actual Amount` and `Monthly Notes` columns
### Changed
- `GET /api/tracker` rows now include monthly override fields when set; fall back to bill defaults when not set
- `GET /api/export` CSV output includes two new columns (backward-compatible, new columns appended)
---
## v0.2.1
### Fixed
- `dailyWorker.js`: Payment query for auto-draft status detection was missing `deleted_at IS NULL`, causing soft-deleted payments to count toward bill status in the daily worker
- `notificationService.js`: Payment query for notification suppression was missing `deleted_at IS NULL`, allowing a soft-deleted payment to incorrectly suppress due/overdue email notifications
- `payments.js GET /`: Year and month query parameters were interpolated directly into SQL string instead of using parameterized queries (SQL injection risk); replaced with bound parameters and proper validation
- `payments.js GET /`: End-of-month boundary was hardcoded as day 31 for all months; now computed using actual days-in-month per year/month
### Changed
- `payments.js POST /`, `POST /quick`, `POST /bulk`: Amount is now validated as a positive number (> 0); zero and negative amounts are rejected with HTTP 400
- `tracker.js GET /`: Year and month query parameters are now validated (year 20002100, month 112); invalid values return HTTP 400 instead of silently computing an invalid date range
- `payments.js GET /`: Year and month query parameters are now validated with the same rules; partial provision of only one is rejected with HTTP 400
- `export.js GET /`: Year query parameter is now validated (20002100); invalid values return HTTP 400
- DB schema: Added compound index `idx_payments_bill_date_del ON payments(bill_id, paid_date, deleted_at)` to accelerate the core tracker query pattern (`WHERE bill_id = ? AND paid_date BETWEEN ? AND ? AND deleted_at IS NULL`)
---
## v0.2
### Added
- `GET /api/version` — serves version and structured release notes from HISTORY.md
- **Paid Date column** in tracker table — shows payment date in green for paid bills
- `POST /api/payments/bulk` — batch payment recording in a single request
- Soft-delete for payments: `deleted_at` column + `POST /api/payments/:id/restore` endpoint
- `GET /api/tracker/upcoming?days=30` — upcoming bills feed sorted by due date
- `GET /api/bills/:id/payments?page=&limit=` — paginated payment history per bill
- `GET /api/export?year=YYYY&format=csv` — CSV export of all payments for a year
- Status page now displays current version and release notes from HISTORY.md
### Changed
- Payments schema: `deleted_at` column added (migration runs automatically on startup)
- `DELETE /api/payments/:id` now soft-deletes (sets `deleted_at`) instead of hard-deleting
- All payment queries now exclude soft-deleted records
---
## v0.1
### Added
- Initial release: React + Vite + Tailwind CSS + shadcn/ui frontend
- Three-theme system: Light, Dark, Dark Purple — persisted to localStorage
- Collapsible sidebar with Ctrl+B / ⌘+B keyboard shortcut
- Stripe-style layered layout with centered max-width container
- Full page set: Tracker, Bills, Categories, Settings, Status
- Admin panel with user management and onboarding wizard
- First-run terminal wizard + env-var non-interactive setup
- Single-user mode (bypass login for household use)
- Email notification system via SMTP (per-user or global recipient)
- Three-level auth: admin (user management only), user (full tracker access)
- First-login privacy notice informing users of admin limitations
- Docker deployment with persistent volume for DB and backups
- Legacy UI preserved at /legacy ("Remember When" mode)
- Release notes one-time dialog on version upgrade