From 1761bf33a01d03ad5e533d1c0da74382d338b7b8 Mon Sep 17 00:00:00 2001 From: null Date: Thu, 20 Aug 2026 16:30:00 -0500 Subject: [PATCH] fix: account for not knowing her centre, not just her spread MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The window promises to hold the actual start four times in five. For a user whose cycles genuinely disagree it was delivering closer to three in four, and the missing piece was structural rather than a mistuned knob: the spread estimate measured how far her cycles fall from a centre and never asked how well that centre is known. With four intervals it is not known well. The correction is the textbook one -- a future draw around an ESTIMATED centre is wider than around the true centre by sqrt(1 + 1/n) -- taken over the effective sample size, because recency weighting means eight intervals do not contribute eight intervals' worth of evidence. Applied only to the branch that actually estimates from a sample. The zero- and one-interval branches are hand-tuned constants standing in for total uncertainty when there is nothing to estimate from, and multiplying those would have counted the same doubt twice -- measured, that mistake widened a first-cycle window by a day and a half and bought nothing. Not a knob: 1.22x at two intervals, 1.06x at eight, 1.03x at fifteen, fixed by the arithmetic rather than chosen to reach a number. The issue this closes was filed on a wrong measurement, and that is worth recording. A 40-seed run showed mid-learning coverage collapsing to 57.5%; at 400 seeds the same cells sit at 73-85%, and the collapse was one unlucky cell rather than a defect. What survived the larger sample is smaller and real, and is what this fixes: variable-user coverage 75.6% -> 76.8% late, 72.2% -> 73.3% mid-learning. The floor in the guard stays below the promise on purpose. For an intrinsically unpredictable cycle the last few points would cost width she cannot afford, and the width ceiling in the same file is the other jaw of that vice. Fixture window returns to 4.56 days, exactly where it was before this batch -- so across the three calibration commits the engine now predicts the §51 fleet at MAE 0.44 instead of 0.67, with 9/9 coverage, for the same window it always had. closes #52 Co-Authored-By: Claude Opus 5 (1M context) --- .../prediction/PersonalPredictionEngine.kt | 26 ++++++++++++++++++- .../domain/prediction/LearningCurveTest.kt | 23 ++++++++++++++-- 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/domain/prediction/src/main/kotlin/dev/privacyllc/period/domain/prediction/PersonalPredictionEngine.kt b/domain/prediction/src/main/kotlin/dev/privacyllc/period/domain/prediction/PersonalPredictionEngine.kt index c99b9bf..eb1e932 100644 --- a/domain/prediction/src/main/kotlin/dev/privacyllc/period/domain/prediction/PersonalPredictionEngine.kt +++ b/domain/prediction/src/main/kotlin/dev/privacyllc/period/domain/prediction/PersonalPredictionEngine.kt @@ -5,6 +5,7 @@ import kotlin.math.abs import kotlin.math.exp import kotlin.math.ln import kotlin.math.roundToLong +import kotlin.math.sqrt /** * The engine PRODUCT_PLAN.md §12 specifies. @@ -298,7 +299,30 @@ class PersonalPredictionEngine : PredictionEngine { val meanAd = if (totalWeight <= 0.0) 0.0 else { deviations.sumOf { (d, w) -> d * w } / totalWeight * MEAN_AD_TO_SCALE } - maxOf(mad, meanAd) + // Her cycles vary, AND the centre they vary around is only an + // estimate. Everything above measures the first; this is the + // second, and leaving it out is why an early window promised 80% + // and delivered less. + // + // The correction is the textbook one — a future draw around an + // ESTIMATED centre is wider than around the true centre by + // sqrt(1 + 1/n) — over the EFFECTIVE sample size, because + // recency weighting means eight intervals do not contribute + // eight intervals' worth of evidence. Kish's formula says what + // they do contribute. + // + // Applied here and not to the branches above, deliberately: + // those are hand-tuned constants standing in for total + // uncertainty when there is nothing to estimate from, so + // multiplying them would count the same doubt twice. + // + // Not a knob. 1.22x at two intervals, 1.06x at eight, 1.03x at + // fifteen — the size is fixed by the arithmetic rather than + // chosen to reach a number. + val totalWeight2 = intervals.sumOf { it.weight } + val effectiveN = totalWeight2 * totalWeight2 / + intervals.sumOf { it.weight * it.weight } + maxOf(mad, meanAd) * sqrt(1.0 + 1.0 / effectiveN) } } diff --git a/domain/prediction/src/test/kotlin/dev/privacyllc/period/domain/prediction/LearningCurveTest.kt b/domain/prediction/src/test/kotlin/dev/privacyllc/period/domain/prediction/LearningCurveTest.kt index 06cc601..865f947 100644 --- a/domain/prediction/src/test/kotlin/dev/privacyllc/period/domain/prediction/LearningCurveTest.kt +++ b/domain/prediction/src/test/kotlin/dev/privacyllc/period/domain/prediction/LearningCurveTest.kt @@ -324,14 +324,33 @@ class LearningCurveTest { } } - /** Guards the small-sample widening of the spread estimate. Was 57.5% at worst. */ + /** + * Guards the estimation-uncertainty term in the spread estimate. + * + * Worth recording what this row is *not*, because the first measurement said + * otherwise. A 40-seed run showed mid-learning coverage collapsing to 57.5% + * and looked like a serious calibration fault; at 400 seeds the same cells + * sit at 73-85%, and the collapse turns out to have been one unlucky + * (profile, seed, cycle-count) cell rather than a defect. It is the reason + * every assertion in this file quantifies over a range. + * + * What survived the larger sample is smaller and real: a user whose cycles + * genuinely disagree is covered a few points under the 80% the window + * promises, because the engine knows her *centre* only approximately and the + * spread estimate did not account for that. It does now. + * + * The floor is deliberately below the promise. For an intrinsically + * unpredictable cycle, closing the last few points would cost width she + * cannot afford — see the width ceiling above, which is the other jaw of + * this vice. + */ @Test fun `the window does not tighten faster than the history earns`() { profiles.forEach { profile -> val c = cell(profile.name) { it in 3..6 } assertTrue( "${profile.name} covered ${"%.1f".format(c.coverage * 100)}% mid-learning", - c.coverage >= 0.55, + c.coverage >= 0.65, ) } }