168 lines
11 KiB
Markdown
168 lines
11 KiB
Markdown
# Claude QA Coverage — Privacy: Period Tracker
|
||
|
||
```
|
||
Status: Current
|
||
Owner: _null
|
||
Last reviewed: 2026-08-18
|
||
Governs: what each QA pass actually reached
|
||
Review trigger: Any QA round run
|
||
```
|
||
|
||
> Pass by pass, what was reached and what was not. The point of this file is the
|
||
> **Blocked** and **Not run** rows: a pass left out of a report reads exactly
|
||
> like a pass that succeeded, and that is how untested code ships believing it
|
||
> was tested.
|
||
|
||
## Round 3 — 2026-08-18 at `0451fbe`, partial
|
||
|
||
Batches 04 and 05 landed: fertility estimates and the reminder system. Pass F
|
||
became runnable for the first time.
|
||
|
||
**Environment:** emulator `PeriodQA`, API 36, Pixel 6 profile, debug build, plus
|
||
8 instrumented tests on the same device.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Re-run after the brand change; onboarding reaches a forecast, relaunch skips it. |
|
||
| B — Core loop | **Pass** | Unchanged and re-driven. Fertility appears on Today and the calendar once the forecast is tight enough. |
|
||
| C — Failure paths | **Partial** | As Round 2. Airplane mode, denied notification permission and a killed process still untried. |
|
||
| D — Persistence | **Partial** | As Round 2. Reboot and update-over-install untried — the second matters more now that WorkManager holds scheduled work. |
|
||
| E — Forecast under hard histories | **Partial** | Unit-tested against both engines; the §51 histories still have not been entered by hand. |
|
||
| F — Notification privacy | **Partial, and this is the important row** | Eight instrumented tests on a device assert what a lock screen *would* render: every kind × both private modes attaches a public version, is marked `VISIBILITY_PRIVATE`, and leaks no health word — including the action labels. **Nobody has yet looked at an actual locked screen.** That is a real gap: the tests check the notification object, and the last mile is what the system chooses to draw. |
|
||
| G — Accessibility | **Partial** | Unchanged. Calendar verified in real greyscale again after the palette change. TalkBack and font scaling still never run. |
|
||
| H — Data ownership | **Partial** | Unchanged. |
|
||
|
||
### What this round found
|
||
|
||
Two defects, both in how Android behaves rather than in the app's logic, and
|
||
both found by running on a device:
|
||
|
||
1. **A notification channel is immutable after creation.** Importance and
|
||
lock-screen visibility cannot be changed once set, so a single shared channel
|
||
would have kept whatever the user's first privacy mode chose — switching from
|
||
Direct to Maximum privacy would have appeared to work and changed nothing.
|
||
One channel per mode now.
|
||
2. **`checkPermissions` was green over its own target**, reading a stale merged
|
||
manifest because it did not depend on the task that writes one. The third
|
||
guard in this project to fail its first proof.
|
||
|
||
Also confirmed: a seventeen-day "fertile window" was on screen for a user one
|
||
cycle in. Arithmetically correct, useless, and only visible by looking.
|
||
|
||
## Round 2 — 2026-08-18 at `19edf4c`, partial
|
||
|
||
Batch 03 built every screen the app has, and each was driven by hand on the
|
||
emulator as it landed rather than in one pass at the end. Same environment:
|
||
`PeriodQA`, API 36, Pixel 6 profile, debug build.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Clean install reaches onboarding; all seven screens walked; the flow produces a forecast and a relaunch goes straight to Today. |
|
||
| B — Core loop | **Pass** | Two-tap logging confirmed on the device: "Started period" → "Yes — today" → "Logged ✓". Editing, ending, and spotting reclassification all exercised. |
|
||
| C — Failure paths | **Partial** | Duplicate-date logging, an end date before a start, and a future date in the picker all handled. Airplane mode, denied permissions and a process killed mid-write still untried. |
|
||
| D — Persistence | **Partial** | Onboarding completion, records and settings all survive force-stop and relaunch. Reboot and update-over-install untried. |
|
||
| E — Forecast under hard histories | **Partial** | The §51 cases pass as unit tests against both engines, and the not-yet path was driven through the UI. The variable and outlier histories have still not been entered by hand. |
|
||
| F — Notification privacy | **Not run** | Nothing sends a notification yet — Batch 05. The onboarding *choice* was verified: Discreet is selected before the user touches anything. |
|
||
| G — Accessibility | **Partial** | Every calendar day, the confidence indicator and the hero countdown carry content descriptions, checked by reading the view hierarchy. **The calendar was verified in actual greyscale** and all five marks remain distinguishable. TalkBack itself, and font scaling, still untried. |
|
||
| H — Data ownership | **Partial** | Delete-all covered by an instrumented test. Export, app lock and artifact inspection do not exist yet. |
|
||
|
||
### What driving it found that tests did not
|
||
|
||
Three defects, none of which any unit test would have caught:
|
||
|
||
1. **Dark mode was broken for the whole of Batch 01.** `PeriodTheme` never
|
||
wrapped its content in a `Surface`, so text without an explicit colour
|
||
inherited black and the app background never painted. Light mode looked
|
||
correct by accident.
|
||
2. **"Period ended" appeared to do nothing.** The logic was right — a period
|
||
ending today still includes today — but the screen was identical afterwards,
|
||
so the button read as broken.
|
||
3. **Today's underline collided with the spotting dot**, on the one day that was
|
||
both. Invisible in the colour screenshot, obvious in greyscale.
|
||
|
||
The pattern is now four rounds old and worth stating as a rule: **the defects in
|
||
this project are found by opening it, not by reading it.**
|
||
|
||
## Round 1 — 2026-08-18 at `adc5075`, partial
|
||
|
||
Not a full round. Batch 01 produced the first build, and this covered the two
|
||
passes that build could support. Recorded as partial rather than left out,
|
||
because a pass omitted from a report reads exactly like a pass that succeeded.
|
||
|
||
**Environment:** emulator `PeriodQA`, API 36 (`sdk_gphone64_x86_64`), Pixel 6
|
||
profile, debug build.
|
||
|
||
| Pass | Result | Notes |
|
||
| --- | --- | --- |
|
||
| A — First run | **Pass** | Clean install, cold start, empty state reads correctly ("No forecast yet", "No periods logged yet"), all four tabs reachable. No notification permission prompt yet — none is requested until Batch 05. |
|
||
| B — Core loop | **Pass** | Logged a period; cycle day, countdown, window and confidence appeared. Edited the start date twice; forecast moved from 15 Sep to 13 Sep and the record was marked `edited`. Deletion covered by instrumented test rather than by hand. |
|
||
| C — Failure paths | **Partial** | Only one path exercised, and it found a crash — see below. Airplane mode, denied permissions and a killed process mid-entry were not tried. |
|
||
| D — Persistence and migration | **Partial** | Covered by an instrumented test that closes and reopens a file-backed database, which is what a force-stop does. A real force-stop, a reboot and an update-over-install were not tried. |
|
||
| E — Forecast under hard histories | **Not run** | The §51 cases pass as unit tests; none has been driven through the UI, which is what this pass is for. |
|
||
| F — Notification privacy on a lock screen | **Not run** | Nothing sends a notification yet — Batch 05. |
|
||
| G — Accessibility | **Not run** | TalkBack, font scaling and greyscale legibility untried. The working surface is not the designed screen, so this is worth deferring to Batch 03 rather than testing a screen that is about to be replaced. |
|
||
| H — Data ownership and leakage | **Partial** | Delete-all covered by an instrumented test. Export, app lock and the built-artifact inspection do not exist yet. |
|
||
|
||
### What pass C found
|
||
|
||
Tapping **Started today** twice on the same day killed the app:
|
||
|
||
```text
|
||
FATAL EXCEPTION: main
|
||
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed:
|
||
period_records.startDate
|
||
```
|
||
|
||
Not filed as an issue: it was found and fixed inside the same batch, before any
|
||
build left this repository, and a tracker issue closed by the commit that
|
||
introduced the code would be bookkeeping rather than a record. It is written
|
||
down here, in the commit that fixed it, and in
|
||
[`../architecture/README.md`](../architecture/README.md), with four regression
|
||
tests pinning the behaviour.
|
||
|
||
The interesting part is not the crash. It is that **the first thing tried by
|
||
hand broke immediately**, on the app's primary button, with 70 unit tests
|
||
green — which is the argument for pass C existing at all.
|
||
|
||
### What lint found, that no pass would have
|
||
|
||
Wiring the boundary guard into `./gradlew check` ran Android lint for the first
|
||
time, on `f5e9fbe`:
|
||
|
||
```text
|
||
NewApi: java.time.LocalDate#ofInstant requires API 34 (minSdk is 26)
|
||
NewApi: java.time.LocalDate#EPOCH requires API 34 (minSdk is 26)
|
||
```
|
||
|
||
Both on the recalculation path — a crash on every device below Android 14.
|
||
Invisible to the unit tests and invisible to an API 36 emulator. Fixed, and
|
||
worth recording as a standing gap below: **this project has no test on a device
|
||
at its own `minSdk`.**
|
||
|
||
## Standing gaps
|
||
|
||
Things no round has ever covered, carried forward until they are. This list
|
||
existing is not a failure; it not existing while the gaps do is.
|
||
|
||
- **No lock screen has been looked at.** Pass F's instrumented tests assert the
|
||
notification object is built correctly, which is most of the risk and not all
|
||
of it. What the system actually draws on a locked device — including heads-up
|
||
behaviour and any launcher's own preview — has never been seen by anybody.
|
||
- **TalkBack has never actually been run.** Content descriptions are written and
|
||
were checked by reading the view hierarchy, which proves they exist and not
|
||
that they make sense in sequence. Pass G is not complete until somebody
|
||
navigates a screen with their eyes shut.
|
||
- **Font scaling untried.** The hero number is 72sp; at the largest accessibility
|
||
scale it may not fit beside anything.
|
||
- **No device at `minSdk`.** Everything so far ran on API 36. The two NewApi
|
||
bugs above would have crashed on Android 8 through 13 and nothing except lint
|
||
could see them. Lint is a good guard and it is not a substitute for one run on
|
||
an old API level.
|
||
- **Physical-device coverage is undecided.** Passes F and G need a real device
|
||
with a lock screen and TalkBack; which device that is has not been chosen, and
|
||
an emulator is not a substitute for either.
|
||
- **Long-horizon accuracy** — whether predictions measurably improve at 3, 6 and
|
||
12 confirmed cycles — cannot be reached by a QA round at all. It needs either
|
||
a simulated history harness or real elapsed time, and until one exists the
|
||
product's headline claim is tested only at the unit level.
|