fix: bank matching returns bill names, reactive Sync button in BillModal, error handling in merchant rule service

HISTORY.md

@@ -10,6 +10,16 @@
 
 - **Migration version sync assertion** — `_runMigrationVersions` module-level variable is now populated by `runMigrations()` before its loop runs. `reconcileLegacyMigrations()` — which runs after `runMigrations()` on legacy-DB upgrade paths — compares its own version array against the stored list and throws a descriptive error if any version appears in one array but not the other. Catches drift between the two migration arrays at startup rather than silently misconfiguring a legacy schema.
 
+- **Encryption key fully app-managed — no env var required** — `TOKEN_ENCRYPTION_KEY` environment variable support removed entirely. The auto-generated DB key (`_auto_encryption_key` in the settings table) is now the primary mechanism, not a fallback. The `[security]` warning that fired on every startup when no env var was set is gone. On first startup a 48-byte cryptographically random key is generated and persisted to the database; subsequent restarts reuse it. All existing encrypted data (SMTP password, OIDC secret, SimpleFIN tokens, push notification tokens) continues to decrypt correctly.
+
+- **TrackerPage crash fixed — `activeTotalExpected` temporal dead zone** — The `cashflow` block added in an earlier change referenced `activeTotalExpected` and `activePaidTowardDue` before their `const` declarations. JavaScript's temporal dead zone caused `Cannot access 'activeTotalExpected' before initialization` on every tracker load. Fixed by moving the four `active*` declarations above the cashflow block that depends on them.
+
+- **Bank merchant rule matching — `balance_delta`, `current_balance`, and error handling** — `billMerchantRuleService.js` was the only payment creation path still missing `balance_delta` computation and `current_balance` update after the #49 fix. Both `applyMerchantRules` (full-user batch matching, called on every sync) and `syncBillPaymentsFromSimplefin` (single-bill retroactive match) now read the bill fresh before each payment, compute `computeBalanceDelta`, include `balance_delta` in the INSERT, and update `bills.current_balance`. `payment_source` corrected from `'auto_match'` (not in `VALID_PAYMENT_SOURCES`) to `'provider_sync'`. DB migration `v0.82` updates all historical `auto_match` records. Both functions also gained full error handling — `txRows` queries, the fallback notes query, and `db.transaction()` blocks are all wrapped with try/catch that logs to console and returns safe zero-match defaults instead of propagating exceptions to the sync worker or API caller.
+
+- **`applyMerchantRules` returns matched bill names** — The function now returns `{ matched, matched_bills: ['AT&T', 'Netflix'] }` instead of just `{ matched }`. Bill name is fetched via JOIN in the rules query. The name set is deduplicated via `Map` keyed by bill_id (so one bill matched by multiple transactions still appears once). The name list is bubbled through `bankSyncService.runSync` and the `POST /api/data-sources/sync-all` endpoint (using a `Set` to dedupe across multiple SimpleFIN sources). TrackerPage Sync Bank toast now reads **"Synced — AT&T, Netflix ✓"** instead of "3 payments matched", with a `(+N more)` suffix when there are more matched bills than bill names to display. Toast duration extended to 5 seconds.
+
+- **BillModal — bank matching Sync button moved, fixed, and made reactive** — The old "SimpleFIN payment history" sync button in the Subscriptions tab was removed (it didn't call `loadLinkedTransactions()` after success and was less discoverable). A new **Sync** button lives in the Bank Matching Rules section header of the Transactions tab, visible whenever `localHasRules` is true. A local `localHasRules` state initialises from `sourceBill.has_merchant_rule` but flips to `true` immediately when `onRulesChanged` fires — so the button appears right after adding the first rule without closing and reopening the modal. After a successful sync, `loadLinkedTransactions()` is called to refresh the Linked transactions list below, and `refetch()` updates the parent tracker view.
+
 - **Pin Due — urgent bills float to top of tracker** — A "Pin Due" toggle button in the TrackerPage header sorts overdue and due-soon bills to the top of each bucket when enabled. Priority order: `missed` → `late` → `due_soon` → `upcoming` → everything else; ties broken by `due_day`. The sort runs after filtering but before the bucket split, so each half-month bucket is sorted independently. The button uses `variant="default"` (solid) when active and `variant="outline"` when off so the current mode is always visible. Preference persists across sessions via `localStorage` under `tracker_pin_upcoming`. Drag reorder is automatically disabled while the toggle is on (`reorderEnabled` now also requires `!pinUpcoming`) since the two modes conflict.
 
 - **Tracker row keyboard navigation** — Tracker rows (desktop table view) are now keyboard navigable. Each row has `tabIndex={0}`, `data-tracker-row`, `aria-rowindex`, and an `aria-label` announcing the bill name, status, and due day. A `focus-visible:ring-2 ring-primary/60 ring-inset` focus ring appears on keyboard focus only. Key bindings: `↓`/`j` focuses the next row, `↑`/`k` the previous (both cross bucket boundaries via `querySelectorAll('[data-tracker-row]')`), `Enter` opens the edit modal, `P` toggles paid/unpaid (skipped bills ignored, `Ctrl+P`/`Cmd+P` passes through to the browser), `Esc` blurs the row. The `onKeyDown` handler guards against firing on nested interactive elements with `if (e.target !== e.currentTarget) return`.

client/components/BillModal.jsx

@@ -143,6 +143,9 @@ export default function BillModal({ bill, initialBill, categories, onClose, onSa
   const [snowballInclude, setSnowballInclude] = useState(!!sourceBill?.snowball_include);
   const [snowballExempt, setSnowballExempt] = useState(!!sourceBill?.snowball_exempt);
   const [syncingPayments, setSyncingPayments] = useState(false);
+  // Track whether rules exist locally so the Sync button appears immediately
+  // after the first rule is added without waiting for sourceBill to refetch.
+  const [localHasRules, setLocalHasRules] = useState(!!sourceBill?.has_merchant_rule);
   const [showDebtSection, setShowDebtSection] = useState(
     () => isDebtCat(categories, sourceBill?.category_id ? String(sourceBill.category_id) : CAT_NONE)
       || !!sourceBill?.snowball_include
@@ -694,44 +697,7 @@ export default function BillModal({ bill, initialBill, categories, onClose, onSa
                     />
                     <p className="text-[10px] text-muted-foreground/70">0-30 days before renewal.</p>
                   </div>
-                  {!isNew && (sourceBill?.has_merchant_rule || ['simplefin_recommendation', 'catalog_match'].includes(sourceBill?.subscription_source)) ? (
-                    <div className="col-span-2 border-t border-border/40 pt-3">
-                      <div className="flex items-center justify-between gap-3">
-                        <div>
-                          <p className="text-xs font-medium text-muted-foreground">SimpleFIN payment history</p>
-                          <p className="text-[10px] text-muted-foreground/70 mt-0.5">
-                            Scan unmatched bank transactions and backfill any missing payments.
-                          </p>
-                        </div>
-                        <Button
-                          type="button"
-                          size="sm"
-                          variant="outline"
-                          className="shrink-0 gap-1.5"
-                          disabled={syncingPayments}
-                          onClick={async () => {
-                            setSyncingPayments(true);
-                            try {
-                              const result = await api.syncBillSimplefinPayments(sourceBill.id);
-                              if (result.added > 0) {
-                                toast.success(`Synced ${result.added} payment${result.added !== 1 ? 's' : ''} from SimpleFIN.`);
-                              } else {
-                                toast.info('No new payments found in SimpleFIN history.');
-                              }
-                            } catch (err) {
-                              toast.error(err.message || 'Sync failed.');
-                            } finally {
-                              setSyncingPayments(false);
-                            }
-                          }}
-                        >
-                          {syncingPayments
-                            ? <><Loader2 className="h-3.5 w-3.5 animate-spin" />Syncing…</>
-                            : <><RefreshCw className="h-3.5 w-3.5" />Sync payments</>}
-                        </Button>
-                      </div>
-                    </div>
-                  ) : null}
+                  {/* Bank sync button moved to the Transactions tab → Bank Matching Rules section */}
                 </div>
               )}
             </div>
@@ -1050,7 +1016,7 @@ export default function BillModal({ bill, initialBill, categories, onClose, onSa
                       Transactions whose description contains these patterns are automatically imported as payments.
                     </p>
                   </div>
-                  {sourceBill?.has_merchant_rule && (
+                  {localHasRules && (
                     <Button
                       type="button"
                       size="sm"
@@ -1086,6 +1052,7 @@ export default function BillModal({ bill, initialBill, categories, onClose, onSa
                   <BillMerchantRules
                     billId={sourceBill?.id}
                     onRulesChanged={() => {
+                      setLocalHasRules(true);
                       refetch?.();
                       loadLinkedTransactions?.();
                     }}

client/pages/TrackerPage.jsx

@@ -101,10 +101,17 @@ export default function TrackerPage() {
     setBankSyncing(true);
     try {
       const result = await api.syncAllSources();
-      const matched = result.auto_matched ?? 0;
-      const newTx   = result.transactions_new ?? 0;
-      if (matched > 0) {
-        toast.success(`Synced — ${matched} payment${matched === 1 ? '' : 's'} matched${newTx > matched ? `, ${newTx} new transactions` : ''}`);
+      const matched      = result.auto_matched ?? 0;
+      const newTx        = result.transactions_new ?? 0;
+      const billNames    = result.matched_bills ?? [];
+      if (matched > 0 && billNames.length > 0) {
+        toast.success(
+          `Synced — ${billNames.join(', ')} ✓` +
+          (matched > billNames.length ? ` (+${matched - billNames.length} more)` : ''),
+          { duration: 5000 }
+        );
+      } else if (matched > 0) {
+        toast.success(`Synced — ${matched} payment${matched === 1 ? '' : 's'} matched`);
       } else if (newTx > 0) {
         toast.success(`Synced — ${newTx} new transaction${newTx === 1 ? '' : 's'}, no automatic matches`);
       } else {

db/database.js

@@ -2694,6 +2694,17 @@ function runMigrations() {
         `);
         console.log('[v0.81] bill_merchant_rules composite index added');
       }
+    },
+    {
+      version: 'v0.82',
+      description: 'payments: normalise auto_match source to provider_sync',
+      dependsOn: ['v0.81'],
+      run: function() {
+        const result = db.prepare(
+          "UPDATE payments SET payment_source = 'provider_sync' WHERE payment_source = 'auto_match'"
+        ).run();
+        console.log(`[v0.82] Normalised ${result.changes} auto_match payment(s) to provider_sync`);
+      }
     }
   ];
 

routes/dataSources.js

@@ -249,6 +249,7 @@ router.post('/sync-all', async (req, res) => {
     let transactionsNew  = 0;
     let transactionsSkip = 0;
     let autoMatched      = 0;
+    const matchedBillSet = new Set(); // dedupe across multiple sources
     const errors = [];
 
     for (const source of sources) {
@@ -258,6 +259,7 @@ router.post('/sync-all', async (req, res) => {
         transactionsNew  += result.transactionsNew  ?? 0;
         transactionsSkip += result.transactionsSkip ?? 0;
         autoMatched      += result.autoMatched      ?? 0;
+        for (const name of result.matched_bills ?? []) matchedBillSet.add(name);
       } catch (err) {
         errors.push(sanitizeErrorMessage(err?.message || 'Sync failed'));
       }
@@ -268,6 +270,7 @@ router.post('/sync-all', async (req, res) => {
       transactions_new:  transactionsNew,
       transactions_skip: transactionsSkip,
       auto_matched:      autoMatched,
+      matched_bills:     [...matchedBillSet],
       errors,
     });
   } catch (err) {

services/bankSyncService.js

@@ -130,9 +130,9 @@ async function runSync(db, userId, dataSource, { days } = {}) {
   `).run(partialError, dataSource.id, userId);
 
   // Apply any stored merchant→bill rules to newly synced transactions
-  const { matched: autoMatched } = applyMerchantRules(db, userId);
+  const { matched: autoMatched, matched_bills: matchedBills } = applyMerchantRules(db, userId);
 
-  return { accountsUpserted, transactionsNew, transactionsSkip, autoMatched, errlist: raw._errlistSummary || null };
+  return { accountsUpserted, transactionsNew, transactionsSkip, autoMatched, matched_bills: matchedBills || [], errlist: raw._errlistSummary || null };
 }
 
 // ─── Public API ───────────────────────────────────────────────────────────────

services/billMerchantRuleService.js

@@ -25,7 +25,7 @@ function applyMerchantRules(db, userId) {
   let rules;
   try {
     rules = db.prepare(`
-      SELECT bmr.bill_id, bmr.merchant
+      SELECT bmr.bill_id, bmr.merchant, b.name AS bill_name
       FROM bill_merchant_rules bmr
       JOIN bills b ON b.id = bmr.bill_id AND b.user_id = bmr.user_id AND b.deleted_at IS NULL
       WHERE bmr.user_id = ?
@@ -34,64 +34,75 @@ function applyMerchantRules(db, userId) {
     return { matched: 0 };
   }
 
-  if (rules.length === 0) return { matched: 0 };
+  if (rules.length === 0) return { matched: 0, matched_bills: [] };
 
-  const txRows = db.prepare(`
-    SELECT t.id, t.amount, t.payee, t.description, t.memo, t.posted_date, t.transacted_at
-    FROM transactions t
-    LEFT JOIN financial_accounts fa ON fa.id = t.account_id AND fa.user_id = t.user_id
-    WHERE t.user_id = ?
-      AND t.match_status = 'unmatched'
-      AND t.ignored = 0
-      AND t.amount < 0
-      AND (t.account_id IS NULL OR fa.id IS NULL OR fa.monitored = 1)
-  `).all(userId);
+  let txRows;
+  try {
+    txRows = db.prepare(`
+      SELECT t.id, t.amount, t.payee, t.description, t.memo, t.posted_date, t.transacted_at
+      FROM transactions t
+      LEFT JOIN financial_accounts fa ON fa.id = t.account_id AND fa.user_id = t.user_id
+      WHERE t.user_id = ?
+        AND t.match_status = 'unmatched'
+        AND t.ignored = 0
+        AND t.amount < 0
+        AND (t.account_id IS NULL OR fa.id IS NULL OR fa.monitored = 1)
+    `).all(userId);
+  } catch (err) {
+    console.error('[applyMerchantRules] Failed to fetch transactions:', err.message);
+    return { matched: 0, matched_bills: [] };
+  }
 
-  if (txRows.length === 0) return { matched: 0 };
+  if (txRows.length === 0) return { matched: 0, matched_bills: [] };
 
-  const getBill      = db.prepare('SELECT current_balance, interest_rate FROM bills WHERE id = ? AND deleted_at IS NULL');
+  const getBill       = db.prepare('SELECT current_balance, interest_rate FROM bills WHERE id = ? AND deleted_at IS NULL');
   const insertPayment = db.prepare(`
     INSERT OR IGNORE INTO payments (bill_id, amount, paid_date, payment_source, transaction_id, balance_delta)
     VALUES (?, ?, ?, 'provider_sync', ?, ?)
   `);
   const updateBalance = db.prepare("UPDATE bills SET current_balance = ?, updated_at = datetime('now') WHERE id = ?");
-  const updateTx = db.prepare(`
+  const updateTx      = db.prepare(`
     UPDATE transactions
     SET matched_bill_id = ?, match_status = 'matched', updated_at = datetime('now')
     WHERE id = ? AND user_id = ? AND match_status = 'unmatched'
   `);
 
   let matched = 0;
+  const matchedBills = new Map(); // bill_id → bill_name for the summary
 
-  db.transaction(() => {
-    for (const tx of txRows) {
-      const txMerchant = normalizeMerchant(tx.payee || tx.description || tx.memo || '');
-      if (!txMerchant) continue;
+  try {
+    db.transaction(() => {
+      for (const tx of txRows) {
+        const txMerchant = normalizeMerchant(tx.payee || tx.description || tx.memo || '');
+        if (!txMerchant) continue;
 
-      const rule = rules.find(r =>
-        txMerchant.includes(r.merchant) || r.merchant.includes(txMerchant)
-      );
-      if (!rule) continue;
+        const rule = rules.find(r =>
+          txMerchant.includes(r.merchant) || r.merchant.includes(txMerchant)
+        );
+        if (!rule) continue;
 
-      const paidDate = tx.posted_date || (tx.transacted_at ? String(tx.transacted_at).slice(0, 10) : null);
-      if (!paidDate) continue;
+        const paidDate = tx.posted_date || (tx.transacted_at ? String(tx.transacted_at).slice(0, 10) : null);
+        if (!paidDate) continue;
 
-      const amount   = Math.round(Math.abs(tx.amount)) / 100;
-      // Read bill fresh so sequential matches for the same bill chain correctly
-      const bill     = getBill.get(rule.bill_id);
-      const balCalc  = bill ? computeBalanceDelta(bill, amount) : null;
+        const amount  = Math.round(Math.abs(tx.amount)) / 100;
+        const bill    = getBill.get(rule.bill_id);
+        const balCalc = bill ? computeBalanceDelta(bill, amount) : null;
 
-      const result = insertPayment.run(rule.bill_id, amount, paidDate, tx.id, balCalc?.balance_delta ?? null);
-      if (result.changes > 0) {
-        // Only update balance and mark matched if the payment was actually inserted
-        if (balCalc) updateBalance.run(balCalc.new_balance, rule.bill_id);
-        updateTx.run(rule.bill_id, tx.id, userId);
-        matched++;
+        const result = insertPayment.run(rule.bill_id, amount, paidDate, tx.id, balCalc?.balance_delta ?? null);
+        if (result.changes > 0) {
+          if (balCalc) updateBalance.run(balCalc.new_balance, rule.bill_id);
+          updateTx.run(rule.bill_id, tx.id, userId);
+          matched++;
+          matchedBills.set(rule.bill_id, rule.bill_name || `Bill #${rule.bill_id}`);
+        }
       }
-    }
-  })();
+    })();
+  } catch (err) {
+    console.error('[applyMerchantRules] Transaction failed, no payments recorded:', err.message);
+    return { matched: 0, matched_bills: [] };
+  }
 
-  return { matched };
+  return { matched, matched_bills: [...matchedBills.values()] };
 }
 
 // Sync all unmatched SimpleFIN transactions for a single bill using its stored
@@ -112,64 +123,79 @@ function syncBillPaymentsFromSimplefin(db, userId, billId) {
 
   // Fallback: extract merchant from notes "Detected from recurring merchant: X"
   if (rules.length === 0) {
-    const bill = db.prepare('SELECT notes FROM bills WHERE id = ? AND user_id = ? AND deleted_at IS NULL').get(billId, userId);
-    const match = bill?.notes?.match(/Detected from recurring merchant:\s*(.+)/i);
-    if (match) {
-      const extracted = normalizeMerchant(match[1].trim());
-      if (extracted && extracted.length >= 3) {
-        addMerchantRule(db, userId, billId, extracted);
-        rules = [extracted];
+    try {
+      const bill = db.prepare('SELECT notes FROM bills WHERE id = ? AND user_id = ? AND deleted_at IS NULL').get(billId, userId);
+      const match = bill?.notes?.match(/Detected from recurring merchant:\s*(.+)/i);
+      if (match) {
+        const extracted = normalizeMerchant(match[1].trim());
+        if (extracted && extracted.length >= 3) {
+          addMerchantRule(db, userId, billId, extracted);
+          rules = [extracted];
+        }
       }
+    } catch (err) {
+      console.error('[syncBillPaymentsFromSimplefin] Failed to read bill notes:', err.message);
     }
     if (rules.length === 0) return { added: 0 };
   }
 
-  const txRows = db.prepare(`
-    SELECT t.id, t.amount, t.payee, t.description, t.memo, t.posted_date, t.transacted_at
-    FROM transactions t
-    LEFT JOIN financial_accounts fa ON fa.id = t.account_id AND fa.user_id = t.user_id
-    WHERE t.user_id = ?
-      AND t.match_status = 'unmatched'
-      AND t.ignored = 0
-      AND t.amount < 0
-      AND (t.account_id IS NULL OR fa.id IS NULL OR fa.monitored = 1)
-  `).all(userId);
+  let txRows;
+  try {
+    txRows = db.prepare(`
+      SELECT t.id, t.amount, t.payee, t.description, t.memo, t.posted_date, t.transacted_at
+      FROM transactions t
+      LEFT JOIN financial_accounts fa ON fa.id = t.account_id AND fa.user_id = t.user_id
+      WHERE t.user_id = ?
+        AND t.match_status = 'unmatched'
+        AND t.ignored = 0
+        AND t.amount < 0
+        AND (t.account_id IS NULL OR fa.id IS NULL OR fa.monitored = 1)
+    `).all(userId);
+  } catch (err) {
+    console.error('[syncBillPaymentsFromSimplefin] Failed to fetch transactions:', err.message);
+    return { added: 0 };
+  }
 
   if (txRows.length === 0) return { added: 0 };
 
-  const getBill      = db.prepare('SELECT current_balance, interest_rate FROM bills WHERE id = ? AND deleted_at IS NULL');
+  const getBill       = db.prepare('SELECT current_balance, interest_rate FROM bills WHERE id = ? AND deleted_at IS NULL');
   const insertPayment = db.prepare(`
     INSERT OR IGNORE INTO payments (bill_id, amount, paid_date, payment_source, transaction_id, balance_delta)
     VALUES (?, ?, ?, 'provider_sync', ?, ?)
   `);
   const updateBalance = db.prepare("UPDATE bills SET current_balance = ?, updated_at = datetime('now') WHERE id = ?");
-  const updateTx = db.prepare(`
+  const updateTx      = db.prepare(`
     UPDATE transactions
     SET matched_bill_id = ?, match_status = 'matched', updated_at = datetime('now')
     WHERE id = ? AND user_id = ? AND match_status = 'unmatched'
   `);
 
   let added = 0;
-  db.transaction(() => {
-    for (const tx of txRows) {
-      const txMerchant = normalizeMerchant(tx.payee || tx.description || tx.memo || '');
-      if (!txMerchant) continue;
-      const matches = rules.some(r => txMerchant.includes(r) || r.includes(txMerchant));
-      if (!matches) continue;
-      const paidDate = tx.posted_date || (tx.transacted_at ? String(tx.transacted_at).slice(0, 10) : null);
-      if (!paidDate) continue;
-      const amount  = Math.round(Math.abs(tx.amount)) / 100;
-      const bill    = getBill.get(billId);
-      const balCalc = bill ? computeBalanceDelta(bill, amount) : null;
+  try {
+    db.transaction(() => {
+      for (const tx of txRows) {
+        const txMerchant = normalizeMerchant(tx.payee || tx.description || tx.memo || '');
+        if (!txMerchant) continue;
+        const matches = rules.some(r => txMerchant.includes(r) || r.includes(txMerchant));
+        if (!matches) continue;
+        const paidDate = tx.posted_date || (tx.transacted_at ? String(tx.transacted_at).slice(0, 10) : null);
+        if (!paidDate) continue;
+        const amount  = Math.round(Math.abs(tx.amount)) / 100;
+        const bill    = getBill.get(billId);
+        const balCalc = bill ? computeBalanceDelta(bill, amount) : null;
 
-      const result = insertPayment.run(billId, amount, paidDate, tx.id, balCalc?.balance_delta ?? null);
-      if (result.changes > 0) {
-        if (balCalc) updateBalance.run(balCalc.new_balance, billId);
-        updateTx.run(billId, tx.id, userId);
-        added++;
+        const result = insertPayment.run(billId, amount, paidDate, tx.id, balCalc?.balance_delta ?? null);
+        if (result.changes > 0) {
+          if (balCalc) updateBalance.run(balCalc.new_balance, billId);
+          updateTx.run(billId, tx.id, userId);
+          added++;
+        }
       }
-    }
-  })();
+    })();
+  } catch (err) {
+    console.error('[syncBillPaymentsFromSimplefin] Transaction failed, no payments recorded:', err.message);
+    return { added: 0 };
+  }
 
   return { added };
 }