Privacy-Period-Tracker/docs/qa/ClaudeQAPlan.md

160 lines
7.9 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Claude QA Plan — Privacy: Period Tracker
```
Status: Current
Owner: _null
Last reviewed: 2026-08-18
Governs: what a QA round consists of
Review trigger: Any new user-facing surface, or a defect class that got through
```
> The playbook. What a round *is*, so two rounds are comparable and a gap is
> visible rather than assumed covered.
## Before a round
- Build from a clean checkout at a known SHA, and **record that SHA**. A finding
without one cannot be re-tested, and a finding that cannot be re-tested cannot
be closed.
- Build from a detached worktree if other work is in flight, so uncommitted
changes cannot contaminate what is under test.
- Note the environment: device or emulator, Android version and API level,
display size, font scale, and whether the device has a lock screen set. The
last one matters more here than anywhere — half of pass F depends on it.
- Seed the cycle history deliberately. A round run against three cycles and a
round run against twelve are not comparable, and the forecast is the product.
### The emulators, by name
Rounds cite these by name, so they are defined here rather than in the round
that happened to use one. Both are Pixel 6 profiles, x86_64, no Play Store.
| AVD | API | Why it exists |
| --- | --- | --- |
| `PeriodQA` | 36 | The everyday round. Current Android, and what Rounds 13 ran on. |
| `PeriodMinSdk26` | 26 | **`minSdk`.** The floor the app claims to support, and the level lint's `NewApi` findings are about — two of which were real crashes on every device below Android 14, invisible to the unit tests and to an API 36 emulator. |
Recreate either with:
```bash
SDK=$ANDROID_HOME
$SDK/cmdline-tools/latest/bin/sdkmanager --install "system-images;android-26;google_apis;x86_64"
$SDK/cmdline-tools/latest/bin/avdmanager create avd \
-n PeriodMinSdk26 -k "system-images;android-26;google_apis;x86_64" -d pixel_6
```
**Launch headless, and target it by serial.** Other projects on this machine keep
their own emulators running, so `adb` regularly has three devices attached and a
bare `adb shell` fails with *more than one device/emulator*. Resolve the serial
by AVD name rather than assuming `emulator-5554`:
```bash
$SDK/emulator/emulator -avd PeriodMinSdk26 -no-window -no-audio -no-boot-anim &
D=$(adb devices | awk '/emulator-/{print $1}' \
| while read d; do [ "$(adb -s $d emu avd name | head -1 | tr -d '\r')" \
= PeriodMinSdk26 ] && echo $d; done)
adb -s "$D" install -r app/build/outputs/apk/debug/app-debug.apk
```
`-no-window` is not only for headless convenience: a windowed emulator dies with
`XIO: fatal IO error on X server ":0"` if the launching session's display goes
away, which killed one mid-round. And the debug build's application id carries a
`.debug` suffix, so it is `dev.privacyllc.period.debug` that must be launched —
`monkey -p dev.privacyllc.period` aborts with *No activities found to run*.
## The passes
Each pass gets a letter, so `ClaudeQACoverage.md` can report per pass and a
skipped one is visible.
| Pass | What it covers |
| --- | --- |
| A | First run: install on a clean profile, onboarding end to end, notification permission prompt, empty states |
| B | The core loop as a real user: log a period, see the forecast update, log the next one |
| C | Things going wrong: airplane mode, notification permission denied, invalid or duplicate dates, period logged in the future, app killed mid-entry |
| D | Persistence: force-stop and relaunch, reboot, background for days, app update over an existing install with a Room migration |
| E | **The forecast under the histories that break naive engines** — the §51 acceptance cases exercised through the UI rather than only in unit tests: stable 35-day user, highly variable user, 45-day outlier, repeated "Not yet" |
| F | **Notification privacy at every mode**, on a real lock screen: Discreet, Maximum privacy, Direct. What is visible without unlocking is the finding |
| G | Accessibility: TalkBack through the core loop, largest font scale, calendar states distinguishable in greyscale, touch targets, focus order, reduced motion |
| H | Data ownership and leakage: export, Delete My Data, biometric/PIN gate, incognito launcher, and the built artifact inspected for anything health-derived reaching the ads or analytics path |
Add, remove and rename to fit the product. A pass that never applies is noise; a
pass that is always skipped is a lie — which is why the template's money and
authorisation passes were deleted here rather than carried as permanently
skipped rows; *What is deliberately not here*, below, says why.
### Why E is separate from B
Pass B walks the loop as a satisfied user — a plausible cycle history, a
forecast that looks right. It cannot see the defect the product is most likely
to actually ship, because that defect only appears with a history B would never
generate.
[`../planning/PRODUCT_PLAN.md`](../planning/PRODUCT_PLAN.md) names it as a
**core product defect**: a user recording 34, 35, 36, 34, 35 who is predicted a
28- or 29-day cycle. No amount of walking the happy path finds that. E exists to
run the histories in §51 and check the *window and the confidence*, not just the
date — a right date with a wrong window is still wrong.
### Why F is a pass and not a checkbox
The most likely real breach in this product is not a database compromise. It is
a lock screen in a shared room.
Discreet is the default and Maximum privacy exists for people who need it, which
means both are load-bearing and both are only testable by looking at an actual
locked device. A unit test can assert the string; it cannot tell you Android
expanded the notification, or that a heads-up popup showed the private text on
its way past. Run F on hardware with a lock screen set, at every mode, for every
notification type in §29.
### What is deliberately not here
**No pass for money flowing backwards.** The template carries one, and it is
deleted rather than carried as permanently skipped: the only money here is a
one-time Play purchase, refunds are handled by Google, and there is no
entitlement of ours to claw back beyond what Play reports. If a subscription is
ever added, this pass comes back with it.
**No pass for authorisation.** There are no accounts and no server, so there is
no entitlement to confuse with authentication. That is a property of the
architecture, and if it changes this section is the trigger to re-add the pass.
## What counts as a finding
A finding needs: what was done, what happened, what should have happened, and
the build SHA. Without the SHA it cannot be re-tested, and a finding that cannot
be re-tested cannot be closed.
## Severity
Findings are filed as issues, labelled:
- **P0** — ships broken, or loses data
- **P1** — materially wrong, but shippable
- **P2** — cosmetic or low impact
- **release-blocker** — a release built today would be wrong rather than merely
incomplete
Exactly these label names: the Command Center queries them by name, and a
repository that spells them differently has its defects reported as *not
adopted* rather than counted wrongly.
Severity is what it costs, not how annoying it is to fix.
## After a round
File each finding as a labelled issue. Update `ClaudeReport.md`'s run-state
block, its overall sentence, and its **Round notes** — one entry per round, the
verdict only, because the pass-by-pass detail belongs in `ClaudeQACoverage.md`
and a second copy of it will drift. Then update `ClaudeQACoverage.md` with what
each pass actually reached. A pass that could not be run is recorded as blocked,
with what blocks it — never quietly left out, which reads identically to
"passed".
Then **push, and reconcile**. The verdict on the project screen at
privacyllc.dev is read out of `ClaudeReport.md` in the pushed repository, so a
round whose report is committed but not pushed — or pushed but not reconciled —
leaves a stakeholder reading the previous round's judgment with no indication
that a newer one exists. The rest of the cycle is in `docs/WORK_CYCLE.md`.