Swap the engine binding and prove the replacement beats the baseline #14

New Issue

Closed

opened 2026-08-18 03:04:46 -05:00 by null · 0 comments

null commented

2026-08-18 03:04:46 -05:00

Owner

DataModule.predictionEngine() binds BaselinePredictionEngine, and every other module depends on the PredictionEngine interface rather than an implementation — so this swap is one line. It is filed separately because it is the moment the product's core claim changes, and it should not happen as a side effect of another commit.

Before the swap:

run both engines over the §51 fixtures and record mean absolute error, median absolute error and window width for each
the new engine must be better on error without simply widening its windows to be safe. A wider window is trivially more often correct and less useful, which is why both numbers are recorded

Keep BaselinePredictionEngine afterwards. It stops being the product and becomes the control — the thing that makes the next engine's improvement measurable too.

Also update modelVersion, which is stored on every PredictionRecord: accuracy figures compared across engine versions without it are meaningless, and §16's history will span the swap.

Verify: DataModule binds the new engine; a comparison test asserts lower mean absolute error AND no wider mean window than BaselinePredictionEngine across the §51 fixtures; and modelVersion on newly stored predictions differs from baseline-1.

`DataModule.predictionEngine()` binds `BaselinePredictionEngine`, and every other module depends on the `PredictionEngine` interface rather than an implementation — so this swap is one line. It is filed separately because it is the moment the product's core claim changes, and it should not happen as a side effect of another commit. Before the swap: - run both engines over the §51 fixtures and record mean absolute error, median absolute error and window width for each - the new engine must be better on error **without** simply widening its windows to be safe. A wider window is trivially more often correct and less useful, which is why both numbers are recorded Keep `BaselinePredictionEngine` afterwards. It stops being the product and becomes the control — the thing that makes the next engine's improvement measurable too. Also update `modelVersion`, which is stored on every `PredictionRecord`: accuracy figures compared across engine versions without it are meaningless, and §16's history will span the swap. Verify: `DataModule` binds the new engine; a comparison test asserts lower mean absolute error AND no wider mean window than `BaselinePredictionEngine` across the §51 fixtures; and `modelVersion` on newly stored predictions differs from `baseline-1`.