160 lines
7.9 KiB
Markdown
160 lines
7.9 KiB
Markdown
# Claude QA Plan — Privacy: Period Tracker
|
||
|
||
```
|
||
Status: Current
|
||
Owner: _null
|
||
Last reviewed: 2026-08-18
|
||
Governs: what a QA round consists of
|
||
Review trigger: Any new user-facing surface, or a defect class that got through
|
||
```
|
||
|
||
> The playbook. What a round *is*, so two rounds are comparable and a gap is
|
||
> visible rather than assumed covered.
|
||
|
||
## Before a round
|
||
|
||
- Build from a clean checkout at a known SHA, and **record that SHA**. A finding
|
||
without one cannot be re-tested, and a finding that cannot be re-tested cannot
|
||
be closed.
|
||
- Build from a detached worktree if other work is in flight, so uncommitted
|
||
changes cannot contaminate what is under test.
|
||
- Note the environment: device or emulator, Android version and API level,
|
||
display size, font scale, and whether the device has a lock screen set. The
|
||
last one matters more here than anywhere — half of pass F depends on it.
|
||
- Seed the cycle history deliberately. A round run against three cycles and a
|
||
round run against twelve are not comparable, and the forecast is the product.
|
||
|
||
### The emulators, by name
|
||
|
||
Rounds cite these by name, so they are defined here rather than in the round
|
||
that happened to use one. Both are Pixel 6 profiles, x86_64, no Play Store.
|
||
|
||
| AVD | API | Why it exists |
|
||
| --- | --- | --- |
|
||
| `PeriodQA` | 36 | The everyday round. Current Android, and what Rounds 1–3 ran on. |
|
||
| `PeriodMinSdk26` | 26 | **`minSdk`.** The floor the app claims to support, and the level lint's `NewApi` findings are about — two of which were real crashes on every device below Android 14, invisible to the unit tests and to an API 36 emulator. |
|
||
|
||
Recreate either with:
|
||
|
||
```bash
|
||
SDK=$ANDROID_HOME
|
||
$SDK/cmdline-tools/latest/bin/sdkmanager --install "system-images;android-26;google_apis;x86_64"
|
||
$SDK/cmdline-tools/latest/bin/avdmanager create avd \
|
||
-n PeriodMinSdk26 -k "system-images;android-26;google_apis;x86_64" -d pixel_6
|
||
```
|
||
|
||
**Launch headless, and target it by serial.** Other projects on this machine keep
|
||
their own emulators running, so `adb` regularly has three devices attached and a
|
||
bare `adb shell` fails with *more than one device/emulator*. Resolve the serial
|
||
by AVD name rather than assuming `emulator-5554`:
|
||
|
||
```bash
|
||
$SDK/emulator/emulator -avd PeriodMinSdk26 -no-window -no-audio -no-boot-anim &
|
||
D=$(adb devices | awk '/emulator-/{print $1}' \
|
||
| while read d; do [ "$(adb -s $d emu avd name | head -1 | tr -d '\r')" \
|
||
= PeriodMinSdk26 ] && echo $d; done)
|
||
adb -s "$D" install -r app/build/outputs/apk/debug/app-debug.apk
|
||
```
|
||
|
||
`-no-window` is not only for headless convenience: a windowed emulator dies with
|
||
`XIO: fatal IO error on X server ":0"` if the launching session's display goes
|
||
away, which killed one mid-round. And the debug build's application id carries a
|
||
`.debug` suffix, so it is `dev.privacyllc.period.debug` that must be launched —
|
||
`monkey -p dev.privacyllc.period` aborts with *No activities found to run*.
|
||
|
||
## The passes
|
||
|
||
Each pass gets a letter, so `ClaudeQACoverage.md` can report per pass and a
|
||
skipped one is visible.
|
||
|
||
| Pass | What it covers |
|
||
| --- | --- |
|
||
| A | First run: install on a clean profile, onboarding end to end, notification permission prompt, empty states |
|
||
| B | The core loop as a real user: log a period, see the forecast update, log the next one |
|
||
| C | Things going wrong: airplane mode, notification permission denied, invalid or duplicate dates, period logged in the future, app killed mid-entry |
|
||
| D | Persistence: force-stop and relaunch, reboot, background for days, app update over an existing install with a Room migration |
|
||
| E | **The forecast under the histories that break naive engines** — the §51 acceptance cases exercised through the UI rather than only in unit tests: stable 35-day user, highly variable user, 45-day outlier, repeated "Not yet" |
|
||
| F | **Notification privacy at every mode**, on a real lock screen: Discreet, Maximum privacy, Direct. What is visible without unlocking is the finding |
|
||
| G | Accessibility: TalkBack through the core loop, largest font scale, calendar states distinguishable in greyscale, touch targets, focus order, reduced motion |
|
||
| H | Data ownership and leakage: export, Delete My Data, biometric/PIN gate, incognito launcher, and the built artifact inspected for anything health-derived reaching the ads or analytics path |
|
||
|
||
Add, remove and rename to fit the product. A pass that never applies is noise; a
|
||
pass that is always skipped is a lie — which is why the template's money and
|
||
authorisation passes were deleted here rather than carried as permanently
|
||
skipped rows; *What is deliberately not here*, below, says why.
|
||
|
||
### Why E is separate from B
|
||
|
||
Pass B walks the loop as a satisfied user — a plausible cycle history, a
|
||
forecast that looks right. It cannot see the defect the product is most likely
|
||
to actually ship, because that defect only appears with a history B would never
|
||
generate.
|
||
|
||
[`../planning/PRODUCT_PLAN.md`](../planning/PRODUCT_PLAN.md) names it as a
|
||
**core product defect**: a user recording 34, 35, 36, 34, 35 who is predicted a
|
||
28- or 29-day cycle. No amount of walking the happy path finds that. E exists to
|
||
run the histories in §51 and check the *window and the confidence*, not just the
|
||
date — a right date with a wrong window is still wrong.
|
||
|
||
### Why F is a pass and not a checkbox
|
||
|
||
The most likely real breach in this product is not a database compromise. It is
|
||
a lock screen in a shared room.
|
||
|
||
Discreet is the default and Maximum privacy exists for people who need it, which
|
||
means both are load-bearing and both are only testable by looking at an actual
|
||
locked device. A unit test can assert the string; it cannot tell you Android
|
||
expanded the notification, or that a heads-up popup showed the private text on
|
||
its way past. Run F on hardware with a lock screen set, at every mode, for every
|
||
notification type in §29.
|
||
|
||
### What is deliberately not here
|
||
|
||
**No pass for money flowing backwards.** The template carries one, and it is
|
||
deleted rather than carried as permanently skipped: the only money here is a
|
||
one-time Play purchase, refunds are handled by Google, and there is no
|
||
entitlement of ours to claw back beyond what Play reports. If a subscription is
|
||
ever added, this pass comes back with it.
|
||
|
||
**No pass for authorisation.** There are no accounts and no server, so there is
|
||
no entitlement to confuse with authentication. That is a property of the
|
||
architecture, and if it changes this section is the trigger to re-add the pass.
|
||
|
||
## What counts as a finding
|
||
|
||
A finding needs: what was done, what happened, what should have happened, and
|
||
the build SHA. Without the SHA it cannot be re-tested, and a finding that cannot
|
||
be re-tested cannot be closed.
|
||
|
||
## Severity
|
||
|
||
Findings are filed as issues, labelled:
|
||
|
||
- **P0** — ships broken, or loses data
|
||
- **P1** — materially wrong, but shippable
|
||
- **P2** — cosmetic or low impact
|
||
- **release-blocker** — a release built today would be wrong rather than merely
|
||
incomplete
|
||
|
||
Exactly these label names: the Command Center queries them by name, and a
|
||
repository that spells them differently has its defects reported as *not
|
||
adopted* rather than counted wrongly.
|
||
|
||
Severity is what it costs, not how annoying it is to fix.
|
||
|
||
## After a round
|
||
|
||
File each finding as a labelled issue. Update `ClaudeReport.md`'s run-state
|
||
block, its overall sentence, and its **Round notes** — one entry per round, the
|
||
verdict only, because the pass-by-pass detail belongs in `ClaudeQACoverage.md`
|
||
and a second copy of it will drift. Then update `ClaudeQACoverage.md` with what
|
||
each pass actually reached. A pass that could not be run is recorded as blocked,
|
||
with what blocks it — never quietly left out, which reads identically to
|
||
"passed".
|
||
|
||
Then **push, and reconcile**. The verdict on the project screen at
|
||
privacyllc.dev is read out of `ClaudeReport.md` in the pushed repository, so a
|
||
round whose report is committed but not pushed — or pushed but not reconciled —
|
||
leaves a stakeholder reading the previous round's judgment with no indication
|
||
that a newer one exists. The rest of the cycle is in `docs/WORK_CYCLE.md`.
|