perf: optimize bills list query, add merchant rule composite index (v0.81)

HISTORY.md

@@ -30,6 +30,8 @@
 
 ### 🐛 Fixed
 
+- **Bills list query optimised and merchant rule index added** — The issue requested pagination on `GET /api/bills`, `GET /api/categories`, `GET /api/tracker`, and `GET /api/subscriptions`. Pagination was not implemented — the entire UI (tracker buckets, snowball list, drag reorder, BillModal) depends on all bills being loaded at once, and a personal bill tracker with 20–50 bills has no performance problem at this scale. The genuine fix was in two parts. First, `GET /api/bills` replaced a correlated `EXISTS(SELECT 1 FROM bill_history_ranges WHERE bill_id = b.id)` per bill row with a single `LEFT JOIN (SELECT DISTINCT bill_id FROM bill_history_ranges) hr` — same result, one scan instead of N subqueries. Second, DB migration `v0.81` adds a composite index `idx_bill_merchant_rules_user_bill ON bill_merchant_rules(user_id, bill_id)` — the existing index only covered `user_id`, so the EXISTS check in `GET /api/bills/:id` and the snowball debt query had to filter by user then scan for bill_id; the composite index makes it a direct point lookup.
+
 - **Imported payments use correct `payment_source = 'file_import'`** — When issue #49 was fixed (imported payments not updating debt balance), `payment_source` was set to `'import'` — a value not in the canonical `VALID_PAYMENT_SOURCES` set (`manual`, `file_import`, `provider_sync`). Corrected to `'file_import'` in both INSERT paths in `spreadsheetImportService.js`. Payment history now shows the correct source for imported payments and the value round-trips correctly through the import/export and user DB import paths.
 
 - **Mortgage and housing categories now auto-detected as debt** — `DEBT_LIKE_CLAUSES` in `routes/snowball.js` matched `%credit%`, `%loan%`, and `%debt%` category names but not `%mortgage%` or `%housing%`. A real user who created a bill under a "Mortgage" or "Housing" category would never see it on the Snowball page unless they manually toggled `snowball_include`. Demo data hid the bug because the seed bill has `snowball_include` pre-set. The frontend's `mortgageIncluded` warning in `SnowballPage.jsx` already checked for mortgage/housing in category and bill name — it just never fired because those bills were filtered out before reaching the page. Added both patterns to `DEBT_LIKE_CLAUSES`; the warning now works as intended, correctly flagging when a mortgage is present so users see the Ramsey Baby Step 2 note about excluding the house.

db/database.js

@@ -2658,6 +2658,18 @@ function runMigrations() {
         add('push_chat_id',        'TEXT');
         console.log('[v0.80] push notification columns added');
       }
+    },
+    {
+      version: 'v0.81',
+      description: 'bill_merchant_rules: composite index on (user_id, bill_id) for faster EXISTS lookups',
+      dependsOn: ['v0.80'],
+      run: function() {
+        db.exec(`
+          CREATE INDEX IF NOT EXISTS idx_bill_merchant_rules_user_bill
+            ON bill_merchant_rules(user_id, bill_id)
+        `);
+        console.log('[v0.81] bill_merchant_rules composite index added');
+      }
     }
   ];
 

routes/bills.js

@@ -22,13 +22,13 @@ router.get('/', (req, res) => {
   const db = getDb();
   ensureUserDefaultCategories(req.user.id);
   const includeInactive = req.query.inactive === 'true';
+  // LEFT JOIN on a pre-grouped subquery is one query instead of N+1 correlated EXISTS lookups.
   const bills = db.prepare(`
     SELECT b.*, c.name AS category_name,
-      CASE WHEN EXISTS(
-        SELECT 1 FROM bill_history_ranges WHERE bill_id = b.id
-      ) THEN 1 ELSE 0 END AS has_history_ranges
+      CASE WHEN hr.bill_id IS NOT NULL THEN 1 ELSE 0 END AS has_history_ranges
     FROM bills b
     LEFT JOIN categories c ON b.category_id = c.id AND c.deleted_at IS NULL
+    LEFT JOIN (SELECT DISTINCT bill_id FROM bill_history_ranges) hr ON hr.bill_id = b.id
     WHERE b.user_id = ?
       AND b.deleted_at IS NULL
       ${includeInactive ? '' : 'AND b.active = 1'}