|
|
|
|
@ -1,10 +1,58 @@
|
|
|
|
|
# Bill Tracker — Changelog
|
|
|
|
|
|
|
|
|
|
## v0.37.0
|
|
|
|
|
|
|
|
|
|
### ✨ Added
|
|
|
|
|
|
|
|
|
|
- **Auto-match review panel** — Merchant-rule auto-matches (payment_source = provider_sync) are now surfaced in a collapsible "Auto-matched — review" panel in the Data → Bank Sync section. Each entry shows the payee, date, amount, and the bill it was matched to. An Undo button reverses the match: the payment is soft-deleted, the bill balance is restored (including any interest), and the transaction reverts to unmatched. The panel appears only when there are reviewable items in the last 7 days, disappears when the list is empty, and automatically refreshes after each Sync Now or Backfill. New endpoints: `GET /api/payments/recent-auto` (fetch the review list), `POST /api/payments/:id/undo-auto` (reverse one match).
|
|
|
|
|
|
|
|
|
|
- **Auto-learn merchant rules from manual matches** — When a user explicitly confirms a transaction→bill match (via the confirm-match button or the transaction match endpoint with `learnMerchant: true`), a normalized merchant rule is automatically created so future synced transactions from the same payee auto-match without manual intervention. The derived rule is gated by `learnableMerchantFromTransaction()` which rejects payee text that is too short (<4 chars) or composed entirely of generic financial tokens (ach, atm, transfer, fee, etc.), preventing a rule like "ACH Payment" from becoming a catch-all. Background auto-matching (via `applyMerchantRules`) never creates rules — only explicit user actions do. Generic API endpoint tokens are also now collapsed in `normalizeMerchant` (apostrophes stripped so "Sam's Club" → "sams club").
|
|
|
|
|
|
|
|
|
|
- **Ambiguous merchant-rule matching protection** — `applyMerchantRules` now checks whether the most-specific tier of matching merchant rules maps to more than one distinct bill (e.g., two bills both named "Amazon" with matching rules). When ambiguous, the transaction is skipped and left for manual review rather than silently attributing to the wrong bill.
|
|
|
|
|
|
|
|
|
|
- **Session token hashing** — Session tokens are no longer stored in plaintext in the `sessions` table. The database now stores SHA-256(token); only the cookie retains the raw token. Existing sessions are invalidated on first startup after this version (all users must re-login once). New admin privacy page allows opting into login location recording.
|
|
|
|
|
|
|
|
|
|
- **Geolocation opt-in privacy setting** — Login IP geolocation (city, country, region, ISP) was previously always-on for all logins. It now requires an explicit privacy setting toggle (default off). Admin → Privacy card controls the setting. When disabled, no geolocation requests are made and no location data is stored.
|
|
|
|
|
|
|
|
|
|
- **TOKEN_ENCRYPTION_KEY env var** — Encryption key separation added as a security option. See changed section.
|
|
|
|
|
|
|
|
|
|
- **Auto-match review panel** — Merchant-rule auto-matches (payment_source = provider_sync) are now surfaced in a collapsible "Auto-matched — review" panel in the Data → Bank Sync section. Each entry shows the payee, date, amount, and the bill it was matched to. An Undo button reverses the match: the payment is soft-deleted, the bill balance is restored (including any interest), and the transaction reverts to unmatched. The panel appears only when there are reviewable items in the last 7 days, disappears when the list is empty, and automatically refreshes after each Sync Now or Backfill. New endpoints: `GET /api/payments/recent-auto` (fetch the review list), `POST /api/payments/:id/undo-auto` (reverse one match).
|
|
|
|
|
|
|
|
|
|
### 🔧 Changed
|
|
|
|
|
|
|
|
|
|
- **SimpleFIN transaction deduplication stable across disconnect/reconnect** — `provider_transaction_id` was built as `simplefin:{data_source_id}:{account_id}:{tx_id}`. When a user disconnected (deleting the data source) and reconnected (creating a new data source with a new ID), the ID in the key changed, so nothing matched the old orphaned rows and the full transaction history was duplicated. Changed the key format to `simplefin:{account_id}:{tx_id}` (no data_source_id) — the SimpleFIN account ID and transaction ID are stable identifiers assigned by the financial institution. The unique dedupe index was changed from `(data_source_id, provider_transaction_id)` to `(user_id, provider_transaction_id)` to match the new scope. Migration v0.93 rewrites all existing keys (stripping the numeric data_source_id segment), deduplicates any rows that are now identical after the key change (preserving the linked row over the orphan), and replaces the index atomically.
|
|
|
|
|
|
|
|
|
|
- **Transaction currency from account, not hardcoded USD** — `normalizeTransaction` hardcoded `currency: 'USD'` for every imported transaction even though `normalizeAccount` correctly reads `rawAccount.currency`. Non-USD users' transactions were always mislabeled. The currency is now read from the account's own currency field; the `'USD'` fallback only applies when the account has no currency data.
|
|
|
|
|
|
|
|
|
|
- **Interest on debt bills charged once per calendar month, not once per payment** — `computeBalanceDelta` applied a full month of interest on every call. Making two payments in the same month on a credit-card or loan bill charged interest twice, inflating the tracked balance. `computeBalanceDelta` now checks `bill.interest_accrued_month` against the current month (YYYY-MM) and skips the interest component when they match. The month is updated atomically in `bills` whenever interest accrues (via the new `applyBalanceDelta` helper, which uses `COALESCE` to leave the column alone when no interest is charged). `interest_delta` is now stored on each payment row so the delete/restore/edit paths can correctly reverse only the payment component, not the already-charged interest. Migration v0.93 adds `bills.interest_accrued_month TEXT` and `payments.interest_delta REAL`. All payment-writing paths updated: `routes/payments.js` (create, quick, autopay-confirm, bulk, edit, delete, restore), `routes/matches.js`, `routes/bills.js`, `services/trackerService.js`, `services/billMerchantRuleService.js`, `services/transactionMatchService.js`, `services/spreadsheetImportService.js`. For backward compatibility, payment rows with `interest_delta IS NULL` (written before this version) fall back to the prior full-reversal behavior on edit/delete.
|
|
|
|
|
|
|
|
|
|
- **Sync lookback window — single source of truth, accurate UI copy** — The SimpleFIN lookback window was described with three different wrong numbers, none of which matched the code: the Data page showed a "90d Backfill" button, a "pulled from the last 90 days" toast, and a 90-day "History window" stat; the Admin → Bank Sync card's "Initial connect & backfill" panel showed a "6 days" badge with body copy reading "60 days" and "60-day hard limit" twice. Actual behavior is a **44-day** seed/backfill window (one day under SimpleFIN Bridge's **45-day** hard limit). Root cause was duplicated constants: `bankSyncService` defined its own `SEED_SYNC_DAYS = 44` / `ROUTINE_SYNC_DAYS = 30` independently of `bankSyncConfigService`'s `SYNC_DAYS_EFFECTIVE` / `SYNC_DAYS_DEFAULT` / `SYNC_DAYS_MAX`, and the UI strings were hardcoded and drifted. `bankSyncConfigService` is now the single source of truth — it exports the three constants and `getBankSyncConfig()` returns `seed_days` (44) and `sync_days_max` (45) alongside `sync_days`. `bankSyncService` imports the shared constants instead of redefining them. The user-facing `GET /simplefin/status` now returns `seed_days`, and the admin config endpoint already spreads the full config, so the backfill button label/title/toast, the admin badge and hard-limit copy, the routine-lookback input `max`/clamp, the at-limit warning, and the validation messages all render the backend values. Stale `90`/`30` UI fallbacks corrected so no wrong number is ever shown.
|
|
|
|
|
|
|
|
|
|
- **Encryption key separation — TOKEN_ENCRYPTION_KEY restored** — The encryption key was previously auto-generated and stored in the database under `_auto_encryption_key`, co-located with the ciphertext it protects. Anyone with a database backup or file-level read could decrypt SimpleFIN access URLs, SMTP passwords, TOTP secrets, push notification tokens, and login history geolocation — encryption-at-rest provided no protection against backup theft. `TOKEN_ENCRYPTION_KEY` env var support is restored: when set, new encryptions use it (prefix `e2:`); when absent, the DB key is used (prefix `v2:`). On startup, if `TOKEN_ENCRYPTION_KEY` is set, all DB-key-encrypted secrets are automatically re-encrypted with the env key in a single transaction — covering `data_sources.encrypted_secret`, `users.totp_secret/totp_recovery_codes/push_url/push_token`, `settings.notify_smtp_password/oidc_client_secret`, and all `user_login_history` encrypted columns. Migration is idempotent (already-migrated `e2:` values are skipped). The Admin → Bank Sync card now shows which key source is active: a warning when the DB key is in use, a confirmation when the env key is loaded.
|
|
|
|
|
|
|
|
|
|
- **Bank balance freshness timestamp on TrackerPage** — The bank budget tracking card's pulsing indicator label is renamed from "Live" to "Live Sync", and the card and compact status bar now both show "as of [date, time]" sourced from `bank_tracking.last_updated` (already returned by the summary API) so users can see how fresh the balance is. `CalendarPage` already showed this field; `TrackerPage` now matches.
|
|
|
|
|
|
|
|
|
|
- **SimpleFIN fetch retries transient failures with backoff** — A single network blip or 5xx response previously flipped the connection to `status: 'error'` immediately. `fetchAccountsAndTransactions` now retries up to 3 attempts with 1 s / 2 s delays between them. Network errors and timeouts retry unconditionally; 5xx responses retry; 403 (revoked credentials) and other 4xx responses fail immediately since retrying won't help. `claimSetupToken` is not retried — a setup token is single-use and returns 403 on re-claim.
|
|
|
|
|
|
|
|
|
|
- **SimpleFIN requests now time out after 30 seconds** — `claimSetupToken` and `fetchAccountsAndTransactions` used bare `fetch` with no timeout, so a hung SimpleFIN response would stall the sync indefinitely and hold the worker's `running` flag, blocking all subsequent auto-sync cycles. Both calls now pass `signal: AbortSignal.timeout(30000)`; a `TimeoutError` propagates through the existing `sanitizeError` handler in each function.
|
|
|
|
|
|
|
|
|
|
- **Manual sync and auto-sync now produce identical match results** — `autoMatchForUser` (score-based auto-matching) was only called by the background worker after each sync, not by the routes used for manual "Sync Now". `runSync` in `bankSyncService` now calls `autoMatchForUser` directly alongside `applyMerchantRules` and `applySpendingCategoryRules`, so every sync path — manual, sync-all, initial connect, and the timer — runs all three matching passes. The redundant `autoMatchForUser` call removed from `bankSyncWorker`.
|
|
|
|
|
|
|
|
|
|
- **Match suggestion rejections now expire correctly** — `match_suggestion_rejections` has always stored `rejected_at`, but two queries incorrectly referenced `created_at` (added by a v0.90 migration that should never have been needed). `matchSuggestionService.loadRejections` was throwing and falling back to loading all rejections with no time filter, so rejected suggestions were suppressed forever instead of resurfacing after 90 days. `cleanupService` prune was also throwing and silently catching, so old rejection rows were never deleted. Both queries corrected to use `rejected_at`; the fallback dead-code in `loadRejections` removed.
|
|
|
|
|
|
|
|
|
|
### 🔒 Security
|
|
|
|
|
|
|
|
|
|
- **Session tokens hashed in the database** — `sessions.id` previously stored the raw UUID that was set as the session cookie. Any attacker with read access to the database file (backup theft, direct file access) could extract those UUIDs and replay them as valid session cookies. Sessions now store `SHA-256(token)` as the primary key; the raw token stays only in the cookie and is never written to disk. All session operations in `authService` hash the cookie value before querying or mutating the table: `getSessionUser`, `logout`, `rotateSessionId`, `invalidateOtherSessions`, and the two `INSERT` paths in `login`/`createSession`. Migration v0.94 deletes all existing plaintext sessions, forcing a one-time re-login for all users. This matches the pattern already used for `session_fingerprint` in `user_login_history`.
|
|
|
|
|
|
|
|
|
|
- **IP geolocation made opt-in, disabled by default** — `recordLogin` called `http://ip-api.com/json/{ip}` (plain HTTP, no opt-out) on every new-device login, sending the user's IP to a third party without notice. The call is now guarded by a `geolocation_enabled` admin setting (default: `false`). When disabled, no outbound request is made and the `location_*` columns in `user_login_history` are simply left null. The toggle is exposed in Admin → Privacy. Migration v0.94 seeds the setting at `false` for all installations including existing ones.
|
|
|
|
|
|
|
|
|
|
- **Rate limiting on sync/backfill endpoints** — `POST /:id/sync`, `POST /sync-all`, and `POST /:id/backfill` had no rate limit despite being able to trigger outbound SimpleFIN requests on every call. A new `syncLimiter` (10 requests per 15 minutes, keyed by authenticated user ID rather than IP) is applied inline to all three routes. GET routes on the same router are unaffected. The limiter is included in `allLimiters` so its store is reset alongside the others in tests.
|
|
|
|
|
|
|
|
|
|
- **WebAuthn / FIDO2 hardware security key 2FA** — Migration v0.92 adds `webauthn_enabled` and `webauthn_user_id` columns to `users`, a `webauthn_credentials` table (per-user, multiple keys supported — stores credential ID, CBOR public key as base64url, sign counter, transports, backup eligibility, friendly name, and AAGUID), and a `webauthn_challenges` table for short-lived registration, authentication, and login challenges. The new `webauthnService.js` handles the full lifecycle via `@simplewebauthn/server`: generating registration options (with `excludeCredentials` to prevent re-registering existing keys), verifying attestation responses, generating authentication options (passing allowed credentials and transports), verifying assertion responses (updating the sign counter on each use to detect cloned authenticators), and issuing/consuming login challenge tokens. The login flow mirrors TOTP exactly — after password verification succeeds, if `webauthn_enabled` is set, the server returns `requires_webauthn: true` alongside a `challenge_token` (a short-lived login challenge) and `webauthn_options` (the pre-generated assertion options); the client calls `startAuthentication()` from `@simplewebauthn/browser`, and `POST /api/auth/webauthn/challenge` verifies the assertion and creates a session. Six new endpoints added to `routes/auth.js`: `GET /webauthn/status` (enabled flag + credential count), `GET /webauthn/credentials` (list registered keys with name, AAGUID, backup flags, and timestamps), `GET /webauthn/setup` (begin registration — returns options + challengeId), `POST /webauthn/enable` (complete registration — verifies attestation, stores credential, sets `webauthn_enabled = 1`), `DELETE /webauthn/credentials/:credentialId` (remove one key — requires password confirmation; auto-disables WebAuthn when last key is removed), `POST /webauthn/disable` (remove all keys — requires password confirmation). RP ID and origin are configurable via `WEBAUTHN_RP_ID` and `WEBAUTHN_ORIGIN` env vars (default to `localhost` for dev). `publicUser()` in `authService.js` now includes `webauthn_enabled` so the frontend login flow knows to prompt for a security key tap. Expired WebAuthn challenges are pruned in the daily worker alongside expired sessions. OIDC and single-user mode are unaffected. `@simplewebauthn/server` and `@simplewebauthn/browser` v13 added to dependencies.
|
|
|
|
|
### Release Image
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
## v0.36.0
|
|
|
|
|
|
|
|
|
|
|
null