Windows tighten faster than the history earns, and coverage dips mid-learning #52

New Issue

Closed

opened 2026-08-20 16:13:30 -05:00 by null · 0 comments

null commented

2026-08-20 16:13:30 -05:00

Owner

Measured by the learning-curve simulation at 93ec5b7.

What is true now. The forecast window promises to contain the actual start 80% of the time (WINDOW_MASS = 0.80). Measured coverage holds for stable users late, but dips to its worst around five cycles of history: 57.5% for a variable user, 65% drifting, 77.5-82.5% stable. The window is a promise, and mid-learning it is broken roughly a third of the time for the users least able to absorb a surprise.

Why. With two to five intervals the weighted MAD and mean-absolute-deviation both underestimate the population spread — small-sample bias, made worse by recency weighting shrinking the effective sample. 80% of a distribution that is too narrow covers well under 80%.

What to do. Inflate the estimate that is biased, not the promise: multiply fromSpread in scaleOf by 1 + EARLY_WIDENING * max(0, EARLY_FULL_HISTORY - n) / EARLY_FULL_HISTORY, starting at 0.5 and 6.0 (x1.33 at two intervals, ramping to x1.0 at six and beyond).

Traps. Do not touch WINDOW_MASS — that is the semantic promise, and moving it changes what the window means rather than making it true. Do not touch MINIMUM_SCALE — that is policy. The ramp reaching exactly 1.0 at six intervals is the safety argument: every six-interval §51 fixture stays bit-identical, and the five-interval stable fixtures land below the scale floor anyway. Tune upward against the harness only until the mid-learning coverage floor passes; stop before a stable user gets high confidence and a tight window (window <= 4 days) or EngineComparisonTest's mean-window ceiling of 8.0 complains.

Verify: LearningCurveTest's coverage assertion for k in 3..6 (red before at 57.5%, green after), with the §51 acceptance suite and EngineComparisonTest unchanged.

Measured by the learning-curve simulation at 93ec5b7. **What is true now.** The forecast window promises to contain the actual start 80% of the time (`WINDOW_MASS` = 0.80). Measured coverage holds for stable users late, but dips to its worst around five cycles of history: 57.5% for a variable user, 65% drifting, 77.5-82.5% stable. The window is a promise, and mid-learning it is broken roughly a third of the time for the users least able to absorb a surprise. **Why.** With two to five intervals the weighted MAD and mean-absolute-deviation both underestimate the population spread — small-sample bias, made worse by recency weighting shrinking the effective sample. 80% of a distribution that is too narrow covers well under 80%. **What to do.** Inflate the estimate that is biased, not the promise: multiply `fromSpread` in `scaleOf` by `1 + EARLY_WIDENING * max(0, EARLY_FULL_HISTORY - n) / EARLY_FULL_HISTORY`, starting at 0.5 and 6.0 (x1.33 at two intervals, ramping to x1.0 at six and beyond). **Traps.** Do not touch `WINDOW_MASS` — that is the semantic promise, and moving it changes what the window means rather than making it true. Do not touch `MINIMUM_SCALE` — that is policy. The ramp reaching exactly 1.0 at six intervals is the safety argument: every six-interval §51 fixture stays bit-identical, and the five-interval stable fixtures land below the scale floor anyway. Tune upward against the harness only until the mid-learning coverage floor passes; stop before `a stable user gets high confidence and a tight window` (window <= 4 days) or EngineComparisonTest's mean-window ceiling of 8.0 complains. Verify: `LearningCurveTest`'s coverage assertion for k in 3..6 (red before at 57.5%, green after), with the §51 acceptance suite and EngineComparisonTest unchanged.