fix: stop charging a drifting cycle for being followed correctly

A user whose cycle lengthens steadily was tracked to within 1.2 days and
told the app was losing confidence: her window grew from 3.1 to 5.1 days
and read Low 31% of the time, the longer it followed her correctly.

The spread estimate measured every interval's distance from a single
static centre. Under a real trend those distances grow with the length of
the history however well the trend is being followed, so the engine was
reading its own success as her variability. Residuals are now measured
against the line the centre was built on. The slope is zero unless a
trend actually fires, so every history without a drift in it is
arithmetically the previous computation -- which is what protects the §51
fixtures rather than a promise to be careful.

Two things fell out of that, and both are the same discovery.

Detrending alone took drift coverage from 96% to 77%: honest residuals
around a line cannot see that the engine deliberately under-follows the
slope, so its centre lags by design. A scale floor applies while
extrapolating a trend -- following one is still an extrapolation.

And the damping itself was paying for the measurement error. While spread
came from a static centre, following a trend inflated the number that
decided how uncertain the forecast was, so the engine had to under-follow
to stay honest. With residuals read against the line that tax is gone:
raising damping 0.50 -> 0.70 halves the error on both §51 drift fixtures
(2 days -> 1), takes fleet MAE from 0.67 to 0.44, returns window coverage
to 9/9, and *narrows* the mean window from 4.56 to 4.33.

Everything improved at once, which is the signature of removing a
distortion rather than trading one fault for another.

Measured at 400 seeds, drifting user at ten or more cycles: MAE 1.19 ->
0.97, window 5.06 -> 3.31 days, coverage 96.1% -> 94.2%, High 6.6% ->
68.1%, Low 31% -> 0.2%. The guard now pins the narrower window together
with a coverage floor, because a tighter window that stopped holding the
answer would be the trade this must not make.

closes #53

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

domain/prediction/src/main/kotlin/dev/privacyllc/period/domain/prediction/PersonalPredictionEngine.kt

@@ -76,7 +76,7 @@ class PersonalPredictionEngine : PredictionEngine {
         )
 
         val distribution = distribution(
-            centreDate = lastStart.plusDays(centre.roundToLong()),
+            centreDate = lastStart.plusDays(centre.days.roundToLong()),
             scale = scale,
             ruledOutThrough = ruledOutThrough,
         ) ?: return null
@@ -127,8 +127,28 @@ class PersonalPredictionEngine : PredictionEngine {
      * The questionable down-weighting from `IntervalAnalysis` applies either
      * way. Doubt about a data point is not the same as doubt about recency.
      */
-    private fun centreOf(intervals: List<Interval>): Double {
+    /**
-        if (intervals.isEmpty()) return POPULATION_DEFAULT_DAYS
+     * The forecast centre, and the per-cycle slope it was built with.
+     *
+     * The slope travels with the centre rather than being recomputed, because
+     * [scaleOf] needs to know whether a trend is being followed: deviations from
+     * a static centre and deviations from a line are different quantities, and
+     * only one of them is this user's actual variability. Zero whenever no trend
+     * fired, which is what makes the spread computation reduce exactly to its
+     * previous form for every history without a drift in it.
+     */
+    private data class CentreEstimate(
+        val days: Double,
+        val trendPerCycle: Double,
+        /** Median cycle length of the recent half, and its mean age — the line's anchor. */
+        val anchorDays: Double,
+        val anchorAge: Double,
+    )
+
+    private fun centreOf(intervals: List<Interval>): CentreEstimate {
+        if (intervals.isEmpty()) {
+            return CentreEstimate(POPULATION_DEFAULT_DAYS, 0.0, POPULATION_DEFAULT_DAYS, 0.0)
+        }
 
         // Weights carrying the questionable damping but no recency decay.
         val flat = intervals.map { interval ->
@@ -149,7 +169,13 @@ class PersonalPredictionEngine : PredictionEngine {
         val consistency = 1.0 / (1.0 + rawSpread / RECENCY_TRUST_SENSITIVITY)
 
         val base = consistency * recencyLed + (1.0 - consistency) * wholeHistory
-        return base + dampedTrend(intervals, base, rawSpread)
+        val trend = trendOf(intervals, base, rawSpread)
+        return CentreEstimate(
+            days = base + trend.damped,
+            trendPerCycle = trend.perCycle,
+            anchorDays = trend.anchorDays.takeIf { trend.perCycle != 0.0 } ?: base,
+            anchorAge = trend.anchorAge,
+        )
     }
 
     /**
@@ -161,17 +187,29 @@ class PersonalPredictionEngine : PredictionEngine {
      * and do not overfit one cycle, and a model that chases every wobble
      * produces a forecast that moves for no reason the user can see.
      */
-    private fun dampedTrend(intervals: List<Interval>, base: Double, spread: Double): Double {
+    private data class Trend(
-        if (intervals.size < MINIMUM_FOR_TREND) return 0.0
+        /** How much to move the forecast, damped. */
+        val damped: Double,
+        /** Undamped slope in days per cycle. Zero when no trend fired. */
+        val perCycle: Double,
+        val anchorDays: Double,
+        val anchorAge: Double,
+    )
+
+    private val noTrend = Trend(0.0, 0.0, 0.0, 0.0)
+
+    private fun trendOf(intervals: List<Interval>, base: Double, spread: Double): Trend {
+        if (intervals.size < MINIMUM_FOR_TREND) return noTrend
 
         val ordered = intervals.sortedBy { it.ageIndex }          // newest first
         val half = ordered.size / 2
         val recent = ordered.take(half).filterNot { it.isQuestionable }
         val older = ordered.drop(half).filterNot { it.isQuestionable }
-        if (recent.isEmpty() || older.isEmpty()) return 0.0
+        if (recent.isEmpty() || older.isEmpty()) return noTrend
 
-        val shift = IntervalAnalysis.median(recent.map { it.days.toDouble() }) -
+        val recentMedian = IntervalAnalysis.median(recent.map { it.days.toDouble() })
-            IntervalAnalysis.median(older.map { it.days.toDouble() })
+        val olderMedian = IntervalAnalysis.median(older.map { it.days.toDouble() })
+        val shift = recentMedian - olderMedian
 
         // Ignore drift smaller than the noise it would be indistinguishable
         // from — and "noise" is this user's own spread, not a constant.
@@ -180,10 +218,23 @@ class PersonalPredictionEngine : PredictionEngine {
         // differed by a single day, turning an exact forecast into a wrong one.
         // One day is a real trend at a cycle of 28 and rounding error at 42.
         val floor = maxOf(MINIMUM_TREND_DAYS, spread * TREND_SIGNAL_RATIO)
-        if (abs(shift) <= floor) return 0.0
+        if (abs(shift) <= floor) return noTrend
+
+        // The same two medians, read as a line rather than a step: the shift
+        // between the halves divided by the gap between their mean ages. Two
+        // resistant points, for the same reason the shift uses medians at all.
+        val recentAge = recent.map { it.ageIndex.toDouble() }.average()
+        val olderAge = older.map { it.ageIndex.toDouble() }.average()
+        val ageGap = olderAge - recentAge
+        val perCycle = if (ageGap <= 0.0) 0.0 else shift / ageGap
 
         val damped = shift * TREND_DAMPING
-        return damped.coerceIn(-base * MAX_TREND_FRACTION, base * MAX_TREND_FRACTION)
+        return Trend(
+            damped = damped.coerceIn(-base * MAX_TREND_FRACTION, base * MAX_TREND_FRACTION),
+            perCycle = perCycle,
+            anchorDays = recentMedian,
+            anchorAge = recentAge,
+        )
     }
 
     // -----------------------------------------------------------------------
@@ -198,12 +249,33 @@ class PersonalPredictionEngine : PredictionEngine {
      * uncertainty, and averaging them lets good-looking intervals hide a run of
      * bad forecasts. Being wrong is evidence, and it outranks looking tidy.
      */
-    private fun scaleOf(intervals: List<Interval>, centre: Double, recentErrors: List<Int>): Double {
+    private fun scaleOf(intervals: List<Interval>, centre: CentreEstimate, recentErrors: List<Int>): Double {
         val fromSpread = when {
             intervals.isEmpty() -> NO_HISTORY_SCALE
             intervals.size == 1 -> SINGLE_INTERVAL_SCALE
             else -> {
-                val deviations = intervals.map { abs(it.days - centre) to it.weight }
+                // Deviations from the line the centre was built on, not from a
+                // point on it.
+                //
+                // When a cycle is genuinely lengthening, every interval sits at
+                // a predictable distance from a static centre and that distance
+                // grows with the length of the history. The engine was reading
+                // its own successful tracking as this user's variability:
+                // measured, a drifting user's window grew from 3.1 to 5.1 days
+                // while her forecast stayed accurate to within a day and a
+                // quarter, and she was told the app was less sure the longer it
+                // followed her correctly.
+                //
+                // `trendPerCycle` is zero unless a trend actually fired, so for
+                // every history without a drift in it this is arithmetically the
+                // previous computation — deviations from `centre.days`, since
+                // the anchor is then the base itself and the slope contributes
+                // nothing. That equivalence is what protects the §51 fixtures.
+                val deviations = intervals.map { interval ->
+                    val expected = centre.anchorDays +
+                        centre.trendPerCycle * (centre.anchorAge - interval.ageIndex)
+                    abs(interval.days - expected) to interval.weight
+                }
 
                 // Two spread estimates, and the larger wins.
                 //
@@ -230,12 +302,27 @@ class PersonalPredictionEngine : PredictionEngine {
             }
         }
 
+        // Following a trend is still an extrapolation.
+        //
+        // Detrending measures how tightly the cycles hug the line, and for a
+        // steadily drifting user that is very tightly indeed — which is true and
+        // not the whole truth. The engine deliberately under-follows the slope
+        // (TREND_DAMPING), so its centre lags the drift by design, and residuals
+        // around the line cannot see that lag at all. Measured, detrending alone
+        // took the drifting user from covering 96% of actual starts to 77%: an
+        // honest window turned into an overconfident one in a single step.
+        //
+        // So a trend that is being followed carries a floor under its scale.
+        // Only when a trend fires — every other history is untouched.
+        val trending = intervals.isNotEmpty() && centre.trendPerCycle != 0.0
+        val floored = if (trending) maxOf(fromSpread, TREND_FOLLOW_MINIMUM_SCALE) else fromSpread
+
         val fromError = if (recentErrors.isEmpty()) 0.0 else {
             val recent = recentErrors.take(ERROR_WINDOW)
             IntervalAnalysis.median(recent.map { it.toDouble() }) * ERROR_TO_SCALE
         }
 
-        return maxOf(fromSpread, fromError).coerceIn(MINIMUM_SCALE, MAXIMUM_SCALE)
+        return maxOf(floored, fromError).coerceIn(MINIMUM_SCALE, MAXIMUM_SCALE)
     }
 
     // -----------------------------------------------------------------------
@@ -397,12 +484,31 @@ class PersonalPredictionEngine : PredictionEngine {
         const val MINIMUM_FOR_TREND = 4
         const val MINIMUM_TREND_DAYS = 1.0
         const val TREND_SIGNAL_RATIO = 1.0
-        // Raised from 0.35 after measurement: the §51 drift fixtures were the
+        // 0.35 -> 0.50 -> 0.70, each step measured.
-        // two the engine won, and it was leaving accuracy on the table by
+        //
-        // following a real signal too timidly. Still well under 1.0, because
+        // The last step came with detrended residuals, and the two belong
-        // §12's "do not overfit one cycle" is the other half of the rule.
+        // together. While spread was measured from a static centre, following a
-        const val TREND_DAMPING = 0.50
+        // trend inflated the very number that decided how uncertain the forecast
+        // was, so the engine had to under-follow to stay honest — the damping
+        // was paying for a measurement error. Once residuals are read against
+        // the line, that tax is gone: at 0.70 the §51 drift fixtures halve their
+        // error (2 days -> 1), the fleet MAE falls from 0.67 to 0.44, window
+        // coverage returns to 9/9 and the mean window *narrows* from 4.56 to
+        // 4.33. Everything improved at once, which is the signature of removing
+        // a distortion rather than trading one fault for another.
+        //
+        // Still under 1.0, and still capped by MAX_TREND_FRACTION, because §12's
+        // "do not overfit one cycle" is the other half of the rule. 0.80 was
+        // measured too: same fixtures, slightly worse simulated coverage.
+        const val TREND_DAMPING = 0.70
         const val MAX_TREND_FRACTION = 0.15
+        /**
+         * The narrowest scale a forecast may claim while extrapolating a trend.
+         *
+         * Set by measurement: without it the drifting user's coverage falls to
+         * 77% against a window that promises 80%. See the comment in `scaleOf`.
+         */
+        const val TREND_FOLLOW_MINIMUM_SCALE = 1.2
 
         // Confidence.
         const val NO_HISTORY_CONFIDENCE = 0.10

domain/prediction/src/test/kotlin/dev/privacyllc/period/domain/prediction/LearningCurveTest.kt

@@ -336,17 +336,31 @@ class LearningCurveTest {
         }
     }
 
-    /** Guards the detrended residuals. Was 35% Low and 5.6 days wide at fifteen cycles. */
+    /**
+     * Guards the detrended residuals and the damping they paid for.
+     *
+     * Before them a drifting user was tracked to within 1.2 days and told the
+     * app was unsure: her window grew to 5.06 days and read Low 31% of the time,
+     * because every interval's distance from a static centre grew with the
+     * history no matter how well the trend was being followed. She was being
+     * charged for the engine's own success.
+     */
     @Test
     fun `following a drift is not punished as if it were spread`() {
         val c = cell("drifting") { it >= 10 }
         assertTrue(
             "drifting read Low ${"%.1f".format(c.low * 100)}% of the time while tracking well",
-            c.low <= 0.40,
+            c.low <= 0.05,
         )
         assertTrue(
             "drifting's window reached ${"%.2f".format(c.width)} days while tracking well",
-            c.width <= 6.0,
+            c.width <= 4.0,
+        )
+        // The other half: a narrower window is only honest if it still holds the
+        // answer. Tightening coverage away would be the trade this must not make.
+        assertTrue(
+            "drifting covered only ${"%.1f".format(c.coverage * 100)}% with its narrower window",
+            c.coverage >= 0.85,
         )
     }
 }