169 lines
11 KiB
Markdown
169 lines
11 KiB
Markdown
# Claude QA Coverage — Privacy: Period Tracker
|
||
|
||
```
|
||
Status: Current
|
||
Owner: _null
|
||
Last reviewed: 2026-08-18
|
||
Governs: what each QA pass actually reached
|
||
Review trigger: Any QA round run
|
||
```
|
||
|
||
> Pass by pass, what was reached and what was not. The point of this file is the
|
||
> **Blocked** and **Not run** rows: a pass left out of a report reads exactly
|
||
> like a pass that succeeded, and that is how untested code ships believing it
|
||
> was tested.
|
||
|
||
## Round 3 — 2026-08-18 at `0451fbe`, partial
|
||
|
||
Batches 04 and 05 landed: fertility estimates and the reminder system. Pass F
|
||
became runnable for the first time.
|
||
|
||
**Environment:** emulator `PeriodQA`, API 36, Pixel 6 profile, debug build, plus
|
||
6 instrumented tests on the same device — 4 in `NotificationPrivacyTest`, 2 in
|
||
`PeriodCrudTest`.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Re-run after the brand change; onboarding reaches a forecast, relaunch skips it. |
|
||
| B — Core loop | **Pass** | Unchanged and re-driven. Fertility appears on Today and the calendar once the forecast is tight enough. |
|
||
| C — Failure paths | **Partial** | As Round 2. Airplane mode, denied notification permission and a killed process still untried. |
|
||
| D — Persistence | **Partial** | As Round 2. Reboot and update-over-install untried — the second matters more now that WorkManager holds scheduled work. |
|
||
| E — Forecast under hard histories | **Partial** | Unit-tested against both engines; the §51 histories still have not been entered by hand. |
|
||
| F — Notification privacy | **Partial, and this is the important row** | Four instrumented tests on a device assert what a lock screen *would* render: every kind × both private modes attaches a public version and is marked `VISIBILITY_PRIVATE`, no public title or body carries a health word, Direct is the only mode marked public, no channel name in any of the three modes carries one either, and Discreet cannot pop over the screen. Action labels are checked by the JVM unit tests in `NotificationCopyTest`, not on the device. **Nobody has yet looked at an actual locked screen.** That is a real gap: the tests check the notification object, and the last mile is what the system chooses to draw. |
|
||
| G — Accessibility | **Partial** | Unchanged. Calendar verified in real greyscale again after the palette change. TalkBack and font scaling still never run. |
|
||
| H — Data ownership | **Partial** | Unchanged. |
|
||
|
||
### What this round found
|
||
|
||
Two defects, both in how Android behaves rather than in the app's logic, and
|
||
both found by running on a device:
|
||
|
||
1. **A notification channel is immutable after creation.** Importance and
|
||
lock-screen visibility cannot be changed once set, so a single shared channel
|
||
would have kept whatever the user's first privacy mode chose — switching from
|
||
Direct to Maximum privacy would have appeared to work and changed nothing.
|
||
One channel per mode now.
|
||
2. **`checkPermissions` was green over its own target**, reading a stale merged
|
||
manifest because it did not depend on the task that writes one. The third
|
||
guard in this project to fail its first proof.
|
||
|
||
Also confirmed: a seventeen-day "fertile window" was on screen for a user one
|
||
cycle in. Arithmetically correct, useless, and only visible by looking.
|
||
|
||
## Round 2 — 2026-08-18 at `19edf4c`, partial
|
||
|
||
Batch 03 built every screen the app has, and each was driven by hand on the
|
||
emulator as it landed rather than in one pass at the end. Same environment:
|
||
`PeriodQA`, API 36, Pixel 6 profile, debug build.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Clean install reaches onboarding; all seven screens walked; the flow produces a forecast and a relaunch goes straight to Today. |
|
||
| B — Core loop | **Pass** | Two-tap logging confirmed on the device: "Started period" → "Yes — today" → "Logged ✓". Editing, ending, and spotting reclassification all exercised. |
|
||
| C — Failure paths | **Partial** | Duplicate-date logging, an end date before a start, and a future date in the picker all handled. Airplane mode, denied permissions and a process killed mid-write still untried. |
|
||
| D — Persistence | **Partial** | Onboarding completion, records and settings all survive force-stop and relaunch. Reboot and update-over-install untried. |
|
||
| E — Forecast under hard histories | **Partial** | The §51 cases pass as unit tests against both engines, and the not-yet path was driven through the UI. The variable and outlier histories have still not been entered by hand. |
|
||
| F — Notification privacy | **Not run** | Nothing sends a notification yet — Batch 05. The onboarding *choice* was verified: Discreet is selected before the user touches anything. |
|
||
| G — Accessibility | **Partial** | Every calendar day, the confidence indicator and the hero countdown carry content descriptions, checked by reading the view hierarchy. **The calendar was verified in actual greyscale** and all five marks remain distinguishable. TalkBack itself, and font scaling, still untried. |
|
||
| H — Data ownership | **Partial** | Delete-all covered by an instrumented test. Export, app lock and artifact inspection do not exist yet. |
|
||
|
||
### What driving it found that tests did not
|
||
|
||
Three defects, none of which any unit test would have caught:
|
||
|
||
1. **Dark mode was broken for the whole of Batch 01.** `PeriodTheme` never
|
||
wrapped its content in a `Surface`, so text without an explicit colour
|
||
inherited black and the app background never painted. Light mode looked
|
||
correct by accident.
|
||
2. **"Period ended" appeared to do nothing.** The logic was right — a period
|
||
ending today still includes today — but the screen was identical afterwards,
|
||
so the button read as broken.
|
||
3. **Today's underline collided with the spotting dot**, on the one day that was
|
||
both. Invisible in the colour screenshot, obvious in greyscale.
|
||
|
||
The pattern is now two rounds old and worth stating as a rule: **the defects in
|
||
this project are found by opening it, not by reading it.**
|
||
|
||
## Round 1 — 2026-08-18 at `adc5075`, partial
|
||
|
||
Not a full round. Batch 01 produced the first build, and this covered the two
|
||
passes that build could support. Recorded as partial rather than left out,
|
||
because a pass omitted from a report reads exactly like a pass that succeeded.
|
||
|
||
**Environment:** emulator `PeriodQA`, API 36 (`sdk_gphone64_x86_64`), Pixel 6
|
||
profile, debug build.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Clean install, cold start, empty state reads correctly ("No forecast yet", "No periods logged yet"), all four tabs reachable. No notification permission prompt yet — none is requested until Batch 05. |
|
||
| B — Core loop | **Pass** | Logged a period; cycle day, countdown, window and confidence appeared. Edited the start date twice; forecast moved from 15 Sep to 13 Sep and the record was marked `edited`. Deletion covered by instrumented test rather than by hand. |
|
||
| C — Failure paths | **Partial** | Only one path exercised, and it found a crash — see below. Airplane mode, denied permissions and a killed process mid-entry were not tried. |
|
||
| D — Persistence and migration | **Partial** | Covered by an instrumented test that closes and reopens a file-backed database, which is what a force-stop does. A real force-stop, a reboot and an update-over-install were not tried. |
|
||
| E — Forecast under hard histories | **Not run** | The §51 cases pass as unit tests; none has been driven through the UI, which is what this pass is for. |
|
||
| F — Notification privacy on a lock screen | **Not run** | Nothing sends a notification yet — Batch 05. |
|
||
| G — Accessibility | **Not run** | TalkBack, font scaling and greyscale legibility untried. The working surface is not the designed screen, so this is worth deferring to Batch 03 rather than testing a screen that is about to be replaced. |
|
||
| H — Data ownership and leakage | **Partial** | Delete-all covered by an instrumented test. Export, app lock and the built-artifact inspection do not exist yet. |
|
||
|
||
### What pass C found
|
||
|
||
Tapping **Started today** twice on the same day killed the app:
|
||
|
||
```text
|
||
FATAL EXCEPTION: main
|
||
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed:
|
||
period_records.startDate
|
||
```
|
||
|
||
Not filed as an issue: it was found and fixed inside the same batch, before any
|
||
build left this repository, and a tracker issue closed by the commit that
|
||
introduced the code would be bookkeeping rather than a record. It is written
|
||
down here, in the commit that fixed it, and in
|
||
[`../architecture/README.md`](../architecture/README.md), with four regression
|
||
tests pinning the behaviour.
|
||
|
||
The interesting part is not the crash. It is that **the first thing tried by
|
||
hand broke immediately**, on the app's primary button, with 70 unit tests
|
||
green — which is the argument for pass C existing at all.
|
||
|
||
### What lint found, that no pass would have
|
||
|
||
Wiring the boundary guard into `./gradlew check` ran Android lint for the first
|
||
time, on `f5e9fbe`:
|
||
|
||
```text
|
||
NewApi: java.time.LocalDate#ofInstant requires API 34 (minSdk is 26)
|
||
NewApi: java.time.LocalDate#EPOCH requires API 34 (minSdk is 26)
|
||
```
|
||
|
||
Both on the recalculation path — a crash on every device below Android 14.
|
||
Invisible to the unit tests and invisible to an API 36 emulator. Fixed, and
|
||
worth recording as a standing gap below: **this project has no test on a device
|
||
at its own `minSdk`.**
|
||
|
||
## Standing gaps
|
||
|
||
Things no round has ever covered, carried forward until they are. This list
|
||
existing is not a failure; it not existing while the gaps do is.
|
||
|
||
- **No lock screen has been looked at.** Pass F's instrumented tests assert the
|
||
notification object is built correctly, which is most of the risk and not all
|
||
of it. What the system actually draws on a locked device — including heads-up
|
||
behaviour and any launcher's own preview — has never been seen by anybody.
|
||
- **TalkBack has never actually been run.** Content descriptions are written and
|
||
were checked by reading the view hierarchy, which proves they exist and not
|
||
that they make sense in sequence. Pass G is not complete until somebody
|
||
navigates a screen with their eyes shut.
|
||
- **Font scaling untried.** The hero number is 72sp; at the largest accessibility
|
||
scale it may not fit beside anything.
|
||
- **No device at `minSdk`.** Everything so far ran on API 36. The two NewApi
|
||
bugs above would have crashed on Android 8 through 13 and nothing except lint
|
||
could see them. Lint is a good guard and it is not a substitute for one run on
|
||
an old API level.
|
||
- **Physical-device coverage is undecided.** Passes F and G need a real device
|
||
with a lock screen and TalkBack; which device that is has not been chosen, and
|
||
an emulator is not a substitute for either.
|
||
- **Long-horizon accuracy** — whether predictions measurably improve at 3, 6 and
|
||
12 confirmed cycles — cannot be reached by a QA round at all. It needs either
|
||
a simulated history harness or real elapsed time, and until one exists the
|
||
product's headline claim is tested only at the unit level.
|