feat: detect a probable missed period without touching the history
IntervalAnalysis turns confirmed starts into weighted intervals and decides what looks questionable — relative to this user's own history, never to a constant. That distinction is the whole point. A global "over 40 days is suspicious" rule gets exactly one group wrong, and it is the group whose cycles are already unusual: the person this product exists for, and the one most tired of apps assuming she is average. 45 days is unremarkable at a usual of 43 and worth questioning at a usual of 29. Both are tests. Two flags, because they earn different responses. A gap near a whole multiple of the usual is a probable missed entry and produces §14's question. A gap merely far from usual is down-weighted and left alone — asking would be the over-questioning §25 warns against, and §51's 45-against-29 outlier is exactly that case. Nothing is ever dropped. §12 step 2: unusual data is marked for review or given less influence, never silently deleted. A questionable interval keeps a quarter of its recency weight, and a test asserts it is still present and still counts. Recency decay is here too, ready for the centre in #10: newest cycle weight 1.0, each older one 0.85 of the last. 12 new tests. The §51 acceptance cases still pass unchanged. closes #13
This commit is contained in:
parent
eb3bebcf2f
commit
b0b4f47a67
|
|
@ -198,6 +198,7 @@ each exists and what must not happen to it.
|
||||||
| `CycleRecord` | derived interval between two confirmed starts | derived, never stored as truth — `toCycles()` recomputes from the period records on every read, so an edit cannot leave a stale interval behind it |
|
| `CycleRecord` | derived interval between two confirmed starts | derived, never stored as truth — `toCycles()` recomputes from the period records on every read, so an edit cannot leave a stale interval behind it |
|
||||||
| `PredictionRecord` | a snapshot taken *before* the outcome is known | this is what makes accuracy measurable at all; never overwritten in place |
|
| `PredictionRecord` | a snapshot taken *before* the outcome is known | this is what makes accuracy measurable at all; never overwritten in place |
|
||||||
| `NotYetObservation` | the user said the period had not started by a date | a censoring observation — the forecast is re-conditioned on it, not shifted by +1 day |
|
| `NotYetObservation` | the user said the period had not started by a date | a censoring observation — the forecast is re-conditioned on it, not shifted by +1 day |
|
||||||
|
| `Interval` | one gap between confirmed starts, with its recency weight and whether it looks questionable | derived per calculation, never stored. A questionable interval is **down-weighted, never dropped** — §12 step 2 |
|
||||||
| `UserPreferences` | notification privacy, reminder time, lock, theme, ads entitlement | lives in DataStore, **never** in the cycle database — see below |
|
| `UserPreferences` | notification privacy, reminder time, lock, theme, ads entitlement | lives in DataStore, **never** in the cycle database — see below |
|
||||||
|
|
||||||
### Why settings are not in the database
|
### Why settings are not in the database
|
||||||
|
|
@ -217,6 +218,20 @@ is what lets its tests run on the JVM against a temporary file. The Android
|
||||||
instance is supplied by DI at the app layer — the only place that should know
|
instance is supplied by DI at the app layer — the only place that should know
|
||||||
where a file lives.
|
where a file lives.
|
||||||
|
|
||||||
|
### Unusual is relative to the user, never to a constant
|
||||||
|
|
||||||
|
`IntervalAnalysis` decides whether a gap is odd by comparing it to a robust
|
||||||
|
centre of **this user's own** intervals. A global rule — "over 40 days is
|
||||||
|
suspicious" — gets exactly one group wrong, and it is the group whose cycles are
|
||||||
|
already unusual: the person this product exists for, and the one most tired of
|
||||||
|
apps assuming she is average. 45 days is unremarkable at a usual of 43 and worth
|
||||||
|
questioning at a usual of 29.
|
||||||
|
|
||||||
|
Two flags, because they earn different responses. A gap near a whole *multiple*
|
||||||
|
of the usual is a probable missed entry and produces §14's question. A gap merely
|
||||||
|
far from usual is down-weighted and left alone — asking about it would be the
|
||||||
|
over-questioning §25 warns against.
|
||||||
|
|
||||||
**Never secretly modify health history.** A gap that looks like a missing entry
|
**Never secretly modify health history.** A gap that looks like a missing entry
|
||||||
([§14](../planning/PRODUCT_PLAN.md)) produces a question, not a correction. That
|
([§14](../planning/PRODUCT_PLAN.md)) produces a question, not a correction. That
|
||||||
is an architectural constraint as much as a UX one: nothing in the data layer
|
is an architectural constraint as much as a UX one: nothing in the data layer
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,149 @@
|
||||||
|
package dev.privacyllc.period.domain.prediction
|
||||||
|
|
||||||
|
import java.time.LocalDate
|
||||||
|
import kotlin.math.abs
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One observed gap between confirmed period starts, and what the model thinks
|
||||||
|
* of it.
|
||||||
|
*
|
||||||
|
* [weight] is how much this interval should influence the forecast: recency
|
||||||
|
* decay, reduced further when the interval is [questionable]. It is never zero
|
||||||
|
* for real data — PRODUCT_PLAN.md §12 step 2 is explicit that unusual data is
|
||||||
|
* marked for review or down-weighted, **never silently discarded**.
|
||||||
|
*/
|
||||||
|
data class Interval(
|
||||||
|
val days: Int,
|
||||||
|
val startedOn: LocalDate,
|
||||||
|
val endedOn: LocalDate,
|
||||||
|
/** 0 is the most recent interval. */
|
||||||
|
val ageIndex: Int,
|
||||||
|
val weight: Double,
|
||||||
|
val questionable: QuestionableReason? = null,
|
||||||
|
) {
|
||||||
|
val isQuestionable: Boolean get() = questionable != null
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Why an interval was flagged.
|
||||||
|
*
|
||||||
|
* Carried rather than reduced to a boolean because the two produce different
|
||||||
|
* questions: a gap roughly twice the usual is *"did you forget to record a
|
||||||
|
* period?"*, while one merely far from usual is not a question worth asking at
|
||||||
|
* all — §25 warns against over-questioning the user.
|
||||||
|
*/
|
||||||
|
enum class QuestionableReason {
|
||||||
|
/** Close to a whole multiple of this user's usual cycle: a period was probably missed. */
|
||||||
|
POSSIBLE_MISSED_PERIOD,
|
||||||
|
|
||||||
|
/** Far from this user's usual, but not a plausible multiple. Kept, down-weighted, not queried. */
|
||||||
|
UNUSUAL_LENGTH,
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Turn confirmed start dates into weighted intervals.
|
||||||
|
*
|
||||||
|
* ## Why detection is relative to the user, never to a constant
|
||||||
|
*
|
||||||
|
* A 45-day gap is unremarkable for somebody whose cycles run 40 and very odd for
|
||||||
|
* somebody at 28. A global threshold gets one of those two wrong, and the one it
|
||||||
|
* gets wrong is the user whose cycle is already unusual — exactly the person
|
||||||
|
* this product exists for, and the one most tired of apps assuming she is
|
||||||
|
* average.
|
||||||
|
*
|
||||||
|
* So the reference is a robust centre of the user's *other* intervals, and a gap
|
||||||
|
* is only called a possible missed period when it sits near a whole multiple of
|
||||||
|
* that. 61 against a usual of 29 is two cycles with a period unrecorded between
|
||||||
|
* them; 45 against a usual of 29 is odd but not a clean multiple, so it is
|
||||||
|
* down-weighted and left alone rather than turned into a question.
|
||||||
|
*/
|
||||||
|
object IntervalAnalysis {
|
||||||
|
|
||||||
|
/** Newest cycle counts fully; each older one a little less. Tuned by test, not picked. */
|
||||||
|
const val RECENCY_DECAY = 0.85
|
||||||
|
|
||||||
|
/** How much a questionable interval still counts. Not zero — see §12 step 2. */
|
||||||
|
const val QUESTIONABLE_WEIGHT_FACTOR = 0.25
|
||||||
|
|
||||||
|
/** Below this many intervals there is no "usual" to compare against. */
|
||||||
|
const val MINIMUM_FOR_DETECTION = 3
|
||||||
|
|
||||||
|
/** Within this fraction of a whole multiple counts as "near" it. */
|
||||||
|
private const val MULTIPLE_TOLERANCE = 0.18
|
||||||
|
|
||||||
|
/** Beyond this fraction from the centre is unusual for this user. */
|
||||||
|
private const val UNUSUAL_TOLERANCE = 0.35
|
||||||
|
|
||||||
|
/** Nothing below this is a cycle; it is a re-entry or a data error. */
|
||||||
|
private const val MINIMUM_PLAUSIBLE_DAYS = 10
|
||||||
|
|
||||||
|
fun intervals(confirmedStarts: List<LocalDate>): List<Interval> {
|
||||||
|
val starts = confirmedStarts.distinct().sorted()
|
||||||
|
if (starts.size < 2) return emptyList()
|
||||||
|
|
||||||
|
val raw = starts.zipWithNext { a, b ->
|
||||||
|
a to (b.toEpochDay() - a.toEpochDay()).toInt()
|
||||||
|
}.filter { it.second >= MINIMUM_PLAUSIBLE_DAYS }
|
||||||
|
|
||||||
|
if (raw.isEmpty()) return emptyList()
|
||||||
|
|
||||||
|
val lengths = raw.map { it.second }
|
||||||
|
// The reference excludes nothing yet — a robust centre is already
|
||||||
|
// resistant to the one value we are about to judge against it, which is
|
||||||
|
// the reason for using a median rather than a mean here.
|
||||||
|
val reference = median(lengths.map { it.toDouble() })
|
||||||
|
|
||||||
|
val newestFirst = raw.indices.reversed().toList()
|
||||||
|
return raw.mapIndexed { index, (start, days) ->
|
||||||
|
val ageIndex = newestFirst.indexOf(index)
|
||||||
|
val reason = classify(days, reference, lengths.size)
|
||||||
|
val recency = Math.pow(RECENCY_DECAY, ageIndex.toDouble())
|
||||||
|
Interval(
|
||||||
|
days = days,
|
||||||
|
startedOn = start,
|
||||||
|
endedOn = start.plusDays(days.toLong()),
|
||||||
|
ageIndex = ageIndex,
|
||||||
|
weight = recency * if (reason != null) QUESTIONABLE_WEIGHT_FACTOR else 1.0,
|
||||||
|
questionable = reason,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun classify(days: Int, reference: Double, sampleSize: Int): QuestionableReason? {
|
||||||
|
if (sampleSize < MINIMUM_FOR_DETECTION || reference <= 0) return null
|
||||||
|
|
||||||
|
val ratio = days / reference
|
||||||
|
|
||||||
|
// Near a whole multiple of two or more: a period was probably missed.
|
||||||
|
// Checked before the generic "unusual" test, because it is the one that
|
||||||
|
// earns a question.
|
||||||
|
for (multiple in 2..4) {
|
||||||
|
if (abs(ratio - multiple) <= MULTIPLE_TOLERANCE) return QuestionableReason.POSSIBLE_MISSED_PERIOD
|
||||||
|
}
|
||||||
|
|
||||||
|
if (abs(ratio - 1.0) > UNUSUAL_TOLERANCE) return QuestionableReason.UNUSUAL_LENGTH
|
||||||
|
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
internal fun median(values: List<Double>): Double {
|
||||||
|
if (values.isEmpty()) return 0.0
|
||||||
|
val s = values.sorted()
|
||||||
|
val mid = s.size / 2
|
||||||
|
return if (s.size % 2 == 1) s[mid] else (s[mid - 1] + s[mid]) / 2.0
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Weighted median: the value where cumulative weight first reaches half the total. */
|
||||||
|
internal fun weightedMedian(values: List<Pair<Double, Double>>): Double {
|
||||||
|
if (values.isEmpty()) return 0.0
|
||||||
|
val sorted = values.sortedBy { it.first }
|
||||||
|
val total = sorted.sumOf { it.second }
|
||||||
|
if (total <= 0.0) return median(sorted.map { it.first })
|
||||||
|
var cumulative = 0.0
|
||||||
|
for ((value, weight) in sorted) {
|
||||||
|
cumulative += weight
|
||||||
|
if (cumulative >= total / 2.0) return value
|
||||||
|
}
|
||||||
|
return sorted.last().first
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,127 @@
|
||||||
|
package dev.privacyllc.period.domain.prediction
|
||||||
|
|
||||||
|
import org.junit.Assert.assertEquals
|
||||||
|
import org.junit.Assert.assertNull
|
||||||
|
import org.junit.Assert.assertTrue
|
||||||
|
import org.junit.Test
|
||||||
|
import java.time.LocalDate
|
||||||
|
|
||||||
|
/**
|
||||||
|
* §14 and §12 step 2, which together say something narrower than "find
|
||||||
|
* outliers": detect what is unusual **for this user**, keep it, reduce its
|
||||||
|
* influence, and ask only when the shape of the gap actually suggests a missed
|
||||||
|
* entry.
|
||||||
|
*/
|
||||||
|
class IntervalAnalysisTest {
|
||||||
|
|
||||||
|
private fun startsFrom(vararg gaps: Long, from: String = "2026-01-01"): List<LocalDate> {
|
||||||
|
var cursor = LocalDate.parse(from)
|
||||||
|
val out = mutableListOf(cursor)
|
||||||
|
gaps.forEach { cursor = cursor.plusDays(it); out += cursor }
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun analyse(vararg gaps: Long) = IntervalAnalysis.intervals(startsFrom(*gaps))
|
||||||
|
|
||||||
|
@Test fun `intervals carry the gaps in order with the newest aged zero`() {
|
||||||
|
val i = analyse(29, 28, 30)
|
||||||
|
assertEquals(listOf(29, 28, 30), i.map { it.days })
|
||||||
|
assertEquals(listOf(2, 1, 0), i.map { it.ageIndex })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `recency decay makes the newest cycle count most`() {
|
||||||
|
val i = analyse(29, 28, 30, 29)
|
||||||
|
val byAge = i.sortedBy { it.ageIndex }
|
||||||
|
assertEquals(1.0, byAge.first().weight, 0.0001)
|
||||||
|
assertTrue("weights must decrease with age", byAge.zipWithNext().all { (a, b) -> a.weight > b.weight })
|
||||||
|
}
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// §14 — a probable missed period
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
@Test fun `a gap near double the usual is flagged as a probable missed period`() {
|
||||||
|
// §14's worked example: 29, 28, 30, 29, 61, 29.
|
||||||
|
val i = analyse(29, 28, 30, 29, 61, 29)
|
||||||
|
val flagged = i.filter { it.isQuestionable }
|
||||||
|
|
||||||
|
assertEquals(1, flagged.size)
|
||||||
|
assertEquals(61, flagged.single().days)
|
||||||
|
assertEquals(QuestionableReason.POSSIBLE_MISSED_PERIOD, flagged.single().questionable)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `a flagged interval is kept and down-weighted, never dropped`() {
|
||||||
|
val i = analyse(29, 28, 30, 29, 61, 29)
|
||||||
|
|
||||||
|
// §12 step 2: do not silently delete unusual data.
|
||||||
|
assertEquals("the interval must still be there", 6, i.size)
|
||||||
|
val odd = i.single { it.days == 61 }
|
||||||
|
assertTrue("and must still carry some weight", odd.weight > 0.0)
|
||||||
|
val comparable = i.single { it.ageIndex == odd.ageIndex - 1 }
|
||||||
|
assertTrue("but much less than an ordinary neighbour", odd.weight < comparable.weight / 2)
|
||||||
|
}
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// The half of §14 that is easy to get wrong
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
@Test fun `a long cycle is normal for a user whose cycles are long`() {
|
||||||
|
// The whole product promise is that this user is not treated as average.
|
||||||
|
// A global "over 40 days is suspicious" rule would flag every one of
|
||||||
|
// these, which is exactly the app she has already deleted once.
|
||||||
|
val i = analyse(41, 44, 42, 45, 43)
|
||||||
|
assertTrue("nothing here is unusual for her", i.none { it.isQuestionable })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `45 against a usual of 29 is unusual but not a missed period`() {
|
||||||
|
// §51's outlier case. 45/29 is 1.55 — nowhere near a whole multiple, so
|
||||||
|
// it is not a forgotten entry and asking would be over-questioning (§25).
|
||||||
|
val i = analyse(29, 29, 28, 30, 45, 29)
|
||||||
|
val odd = i.single { it.days == 45 }
|
||||||
|
assertEquals(QuestionableReason.UNUSUAL_LENGTH, odd.questionable)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `nothing is flagged before there is a usual to compare against`() {
|
||||||
|
// Two intervals do not establish a pattern, and calling one of them
|
||||||
|
// questionable is the model asserting something it cannot know.
|
||||||
|
assertTrue(analyse(29, 61).none { it.isQuestionable })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `a steady history flags nothing`() {
|
||||||
|
assertTrue(analyse(28, 29, 28, 29, 28, 29).none { it.isQuestionable })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `a variable but genuine history is not carpet-flagged`() {
|
||||||
|
// §51's variable user: 25, 34, 29, 37, 26, 32. She is hard to predict,
|
||||||
|
// not wrong. Flagging most of her history would down-weight the data the
|
||||||
|
// model most needs and make her forecast worse.
|
||||||
|
val i = analyse(25, 34, 29, 37, 26, 32)
|
||||||
|
assertTrue("at most one of six may be questioned, got ${i.count { it.isQuestionable }}",
|
||||||
|
i.count { it.isQuestionable } <= 1)
|
||||||
|
assertTrue("and none as a missed period",
|
||||||
|
i.none { it.questionable == QuestionableReason.POSSIBLE_MISSED_PERIOD })
|
||||||
|
}
|
||||||
|
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
// Boundaries
|
||||||
|
// -----------------------------------------------------------------------
|
||||||
|
|
||||||
|
@Test fun `implausibly short gaps are not cycles at all`() {
|
||||||
|
// Two starts three days apart is a re-entry or a typo, not a 3-day
|
||||||
|
// cycle, and letting it into the weighting would wreck the centre.
|
||||||
|
val i = analyse(29, 3, 28)
|
||||||
|
assertTrue("the 3-day gap must not appear", i.none { it.days == 3 })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `fewer than two starts yields nothing rather than failing`() {
|
||||||
|
assertTrue(IntervalAnalysis.intervals(emptyList()).isEmpty())
|
||||||
|
assertTrue(IntervalAnalysis.intervals(listOf(LocalDate.of(2026, 1, 1))).isEmpty())
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test fun `the weighted median respects weights`() {
|
||||||
|
// Equal weights: ordinary median.
|
||||||
|
assertEquals(29.0, IntervalAnalysis.weightedMedian(listOf(28.0 to 1.0, 29.0 to 1.0, 30.0 to 1.0)), 0.0001)
|
||||||
|
// The 35 carries almost all the weight, so it wins despite being extreme.
|
||||||
|
assertEquals(35.0, IntervalAnalysis.weightedMedian(listOf(28.0 to 0.05, 29.0 to 0.05, 35.0 to 10.0)), 0.0001)
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
Reference in New Issue