docs(qa): senior-QA review additions — Pass F, env/matrix, migration, iOS-native dims
- Pass F (cross-cutting): concurrency/realtime races, lifecycle/process-death, network resilience, idempotency/rapid-input, time-dependent (daily rollover/streaks/capsules), account/couple lifecycle, crash reporting. - Methodology: prefer Firebase emulator/staging over prod; device/OS matrix; automate the smoke; test-data hygiene. - Pass D7: encryptionVersion 0->1->2 migration. Reporting/re-QA now A-F. - iOS: iOS-native QA dims (Dynamic Type/VoiceOver/safe-area/edge-swipe-back/sizes), real-device/sandbox needs (App Attest/APNs/StoreKit), crypto golden vectors. - Logged D-OBS: PERMISSION_DENIED on outcomes/challenges/capsules to investigate in Round 2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
e907453f3f
commit
84dd5f1152
|
|
@ -53,6 +53,17 @@ confirms + enumerates this; the fix phase applies couple-shared everywhere.
|
||||||
- Premium toggled via `scratchpad/set_premium.js` (admin, **user-authorized each time**).
|
- Premium toggled via `scratchpad/set_premium.js` (admin, **user-authorized each time**).
|
||||||
- Theme toggled via **Settings → Appearance (Light/Dark)** (`MainActivity` `ThemeMode`).
|
- Theme toggled via **Settings → Appearance (Light/Dark)** (`MainActivity` `ThemeMode`).
|
||||||
- **REPORT-ONLY during passes — never fix mid-pass.**
|
- **REPORT-ONLY during passes — never fix mid-pass.**
|
||||||
|
- **Environment (senior-QA rec):** prefer the **Firebase Local Emulator Suite or a dedicated staging project** over
|
||||||
|
production — isolates test data, makes seeding / entitlement toggles / D3 negative tests **free** (no gated prod
|
||||||
|
writes), and avoids polluting real users. Caveat: App Check, RevenueCat IAP, and real FCM/APNs push need real
|
||||||
|
services — run those against staging/prod with test accounts. (We've been on prod with test accounts — works, but
|
||||||
|
every seed/toggle/deploy hits the gate.)
|
||||||
|
- **Device/OS matrix:** don't certify on one emulator only — cover **minSdk + targetSdk**, a **small** and a **large**
|
||||||
|
screen, and at least one **physical device** (App Check / Play Integrity behave differently on emulators).
|
||||||
|
- **Automate the regression smoke:** capture the smoke checklist as a runnable script (adb/Maestro) so every round
|
||||||
|
re-checks it cheaply instead of by hand.
|
||||||
|
- **Test-data hygiene:** keep known test accounts; clean up artifacts (stray messages/reactions/sessions) between
|
||||||
|
rounds so they don't masquerade as bugs.
|
||||||
|
|
||||||
## Continuity & resumability (this effort WILL span many context windows — don't lose state)
|
## Continuity & resumability (this effort WILL span many context windows — don't lose state)
|
||||||
State lives in **files**, not memory:
|
State lives in **files**, not memory:
|
||||||
|
|
@ -152,6 +163,9 @@ Account); Paywall; Your Progress/Activity; Recovery.
|
||||||
files deleted.
|
files deleted.
|
||||||
- **D6 Leak vectors:** no private content in analytics/crash; `allowBackup=false` + backup rules exclude sensitive data;
|
- **D6 Leak vectors:** no private content in analytics/crash; `allowBackup=false` + backup rules exclude sensitive data;
|
||||||
deep links re-check membership; clipboard user-initiated; consider `FLAG_SECURE`; repo scan for committed secrets.
|
deep links re-check membership; clipboard user-initiated; consider `FLAG_SECURE`; repo scan for committed secrets.
|
||||||
|
- **D7 Encryption migration:** test the `encryptionVersion` paths (0 plaintext → 1 migrating → 2 strict) on a legacy
|
||||||
|
couple — migration completes without exposing plaintext or losing/garbling old content, and a half-migrated couple
|
||||||
|
is safe (no mixed read failures, no downgrade). This is the riskiest data path for existing users.
|
||||||
|
|
||||||
### Pass E — Notifications (every type delivers, deep-links, leaks nothing)
|
### Pass E — Notifications (every type delivers, deep-links, leaks nothing)
|
||||||
For each: trigger fires → delivered to the **right partner (never self)** → in **foreground/background/killed** →
|
For each: trigger fires → delivered to the **right partner (never self)** → in **foreground/background/killed** →
|
||||||
|
|
@ -174,9 +188,25 @@ relationship settings), `memory_capsule_unlocked`(scheduled→capsule), `challen
|
||||||
- Build a delivery matrix (type × {foreground,background,killed}) in ClaudeQACoverage.md. Missed delivery or wrong
|
- Build a delivery matrix (type × {foreground,background,killed}) in ClaudeQACoverage.md. Missed delivery or wrong
|
||||||
deep-link = P1; private content in any payload = P0.
|
deep-link = P1; private content in any payload = P0.
|
||||||
|
|
||||||
|
### Pass F — Resilience, concurrency, lifecycle & time (cross-cutting; a 2-user realtime app needs these)
|
||||||
|
- **Concurrency / realtime races (two partners at once):** both answer the daily question simultaneously; both start
|
||||||
|
a game / swipe a date / react at the same time; partner acts while you're mid-flow. No lost writes, no stuck state,
|
||||||
|
no duplicate sessions, reveal still correct. (This is where a couples app breaks.)
|
||||||
|
- **Lifecycle / process death:** background mid-flow + return; force-kill the app and relaunch (Android may kill the
|
||||||
|
process) — state/auth/draft restore sanely; deep-link/notification after process death still loads (verified for
|
||||||
|
chat — extend to all). Rotation/config-change doesn't lose Compose state. Low-memory.
|
||||||
|
- **Network resilience:** offline / flaky / airplane mid-action across answers, games, dates (not just chat media) —
|
||||||
|
graceful failure + retry/queue, no crash, no silent data loss, recovery on reconnect.
|
||||||
|
- **Idempotency / rapid input:** double-tap send/submit, rapid nav, double-start — guarded (no double-send, no crash).
|
||||||
|
- **Time-dependent behavior:** daily-question rollover (6 PM CST assignment), streak day-boundary + repair window,
|
||||||
|
capsule unlock times, reminder schedules — test across a date change (manipulate device clock / trigger functions).
|
||||||
|
- **Account/couple lifecycle:** brand-new (empty) account; unpaired state; pair → unpair → re-pair; partner leaves
|
||||||
|
mid-session; account deletion cascade; same account on two devices. No orphaned/broken state.
|
||||||
|
- **Crash reporting:** confirm crashes/ANRs are actually captured (Crashlytics) so field issues surface.
|
||||||
|
|
||||||
## Reporting → ClaudeReport.md (living QA report)
|
## Reporting → ClaudeReport.md (living QA report)
|
||||||
- Header: date, build, devices, round number + run-state header.
|
- Header: date, build, devices, round number + run-state header.
|
||||||
- One section per pass (A/B/C/D/E), each a table: **ID | Area | Screen/Route | Mode | Severity | Description | Repro
|
- One section per pass (A/B/C/D/E/F), each a table: **ID | Area | Screen/Route | Mode | Severity | Description | Repro
|
||||||
| Evidence | Suggested fix | Status**.
|
| Evidence | Suggested fix | Status**.
|
||||||
- Summary: counts by severity. Report only during passes — no fixes recorded until the fix phase.
|
- Summary: counts by severity. Report only during passes — no fixes recorded until the fix phase.
|
||||||
|
|
||||||
|
|
@ -197,5 +227,5 @@ five passes are done; **flawless** = one full round with **zero open P0–P2 and
|
||||||
(P3s optional). Don't re-open a clean pass within the same round.
|
(P3s optional). Don't re-open a clean pass within the same round.
|
||||||
|
|
||||||
## Re-QA loop (until flawless)
|
## Re-QA loop (until flawless)
|
||||||
After the fix phase, re-run Pass A/B/C/D/E (regression + confirm fixes). Repeat **fix → re-QA** rounds until a full
|
After the fix phase, re-run Pass A/B/C/D/E/F (regression + confirm fixes). Repeat **fix → re-QA** rounds until a full
|
||||||
round yields zero P0–P2 and Passes D+E fully clean.
|
round yields zero P0–P2 and Passes D+E fully clean.
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# Claude QA Report — Full-App QA (living report)
|
# Claude QA Report — Full-App QA (living report)
|
||||||
|
|
||||||
> **RUN-STATE: Round 2 (re-QA + deferred coverage) NEXT | NEXT ACTION: re-verify A-001 + E-001 fixes; **play each game ONE complete time through on both devices** (Pass B was launch-only — full playthroughs still owed); then Pass C deep/stateful screens (reveal, wheel session, dates, bucket list, auth/onboarding) in both themes + **navigation from every entry point & back-stack/double-back checks**, full live notification matrix, D3 live non-member test.**
|
> **RUN-STATE: Round 2 (re-QA + deferred coverage) NEXT | NEXT ACTION: re-verify A-001 + E-001 fixes; **play each game ONE complete time through on both devices** (Pass B was launch-only — full playthroughs still owed); then Pass C deep/stateful screens (reveal, wheel session, dates, bucket list, auth/onboarding) in both themes + **navigation from every entry point & back-stack/double-back checks**, full live notification matrix, D3 live non-member test; **Pass F (resilience/concurrency/lifecycle/time/migration)**; investigate **D-OBS** PERMISSION_DENIED on outcomes/challenges/capsules.**
|
||||||
> Round 1 complete (all 5 passes run report-only; P0–P2 found were fixed in-line). Fixes: A-001 (e8892a9), E-001 (ce12abb). Open P3: A-003, B-001, E-002.
|
> Round 1 complete (all 5 passes run report-only; P0–P2 found were fixed in-line). Fixes: A-001 (e8892a9), E-001 (ce12abb). Open P3: A-003, B-001, E-002.
|
||||||
> **EXECUTION MODE: autonomous run-to-completion — do NOT stop; fix anything that blocks progress and continue; keep cycling fix→re-QA until a flawless round. Only a gated action (prod deploy / admin write / entitlement toggle) that's denied may be surfaced — do all other work first.**
|
> **EXECUTION MODE: autonomous run-to-completion — do NOT stop; fix anything that blocks progress and continue; keep cycling fix→re-QA until a flawless round. Only a gated action (prod deploy / admin write / entitlement toggle) that's denied may be surfaced — do all other work first.**
|
||||||
> Playbook: `ClaudeQAPlan.md`. Coverage matrix: `ClaudeQACoverage.md`. Report-only during passes (no fixes until the fix phase).
|
> Playbook: `ClaudeQAPlan.md`. Coverage matrix: `ClaudeQACoverage.md`. Report-only during passes (no fixes until the fix phase).
|
||||||
|
|
@ -65,6 +65,10 @@ _Deep/stateful screens (answer reveal, wheel session/complete, date match/builde
|
||||||
|
|
||||||
_Follow-ups (not blockers): live **non-member negative test** (D3) needs a fresh 3rd account (rule logic verified member-scoped); a fresh Storage-bytes spot-check of chat media._
|
_Follow-ups (not blockers): live **non-member negative test** (D3) needs a fresh 3rd account (rule logic verified member-scoped); a fresh Storage-bytes spot-check of chat media._
|
||||||
|
|
||||||
|
| ID | Area | Severity | Description | Status |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| D-OBS | Rules / data load | **P2?** (investigate) | During Pass B, logcat showed `PERMISSION_DENIED` for client listeners on `couples/{id}/outcomes`, `couples/{id}/challenges (where status==active)`, and `couples/{id}/capsules`. Either a rules gap or queries firing for features the (free) user can't read → console errors / possibly broken Connection Challenges + Memory Lane data load. **Round 2: confirm whether these features load correctly + fix the rule or guard the query.** | Open |
|
||||||
|
|
||||||
## Pass E — Notifications
|
## Pass E — Notifications
|
||||||
- **Copy carries no private content:** all function notification bodies are generic ("Tap to read and reply", "Answer together before it expires", etc.); `${title}` refers to public question/game titles, not user answers. ✓ (ties to D6)
|
- **Copy carries no private content:** all function notification bodies are generic ("Tap to read and reply", "Answer together before it expires", etc.); `${title}` refers to public question/game titles, not user answers. ✓ (ties to D6)
|
||||||
- **Routing:** centralized in `PartnerNotificationType` (`fromRemoteType` → `routeFor`); chat opens the exact conversation, reveal→answerReveal(questionId), games→Play, capsule→Memory Lane, etc.
|
- **Routing:** centralized in `PartnerNotificationType` (`fromRemoteType` → `routeFor`); chat opens the exact conversation, reveal→answerReveal(questionId), games→Play, capsule→Memory Lane, etc.
|
||||||
|
|
|
||||||
|
|
@ -56,6 +56,10 @@ Implement byte-compatible Swift crypto for every Android wire format:
|
||||||
`keybox:v1:`) on iOS; (b) have Android decrypt iOS-produced ciphertext; (c) verify an Android-generated recovery
|
`keybox:v1:`) on iOS; (b) have Android decrypt iOS-produced ciphertext; (c) verify an Android-generated recovery
|
||||||
phrase unlocks on iOS and vice versa. Wire formats must match byte-for-byte. If a format can't be matched with
|
phrase unlocks on iOS and vice versa. Wire formats must match byte-for-byte. If a format can't be matched with
|
||||||
CryptoKit, use a vetted Swift lib (e.g. swift-sodium/SwiftArgon2) — never a non-interoperable shortcut.
|
CryptoKit, use a vetted Swift lib (e.g. swift-sodium/SwiftArgon2) — never a non-interoperable shortcut.
|
||||||
|
- **Golden vectors** for the deterministic primitives (checked into both repos): `AnswerCommitment` SHA-256 and
|
||||||
|
`RecoveryKeyManager` Argon2id (fixed salt/params) must produce **identical bytes** on both platforms — assert in unit
|
||||||
|
tests on each side. AEAD/ECIES use random IVs so they can't be golden-matched; cover those with the **round-trip**
|
||||||
|
harness above. Generate the Android fixtures now (on Linux) so iOS has them ready.
|
||||||
|
|
||||||
### 2.4 Screens & features to parity (~48 + new messaging)
|
### 2.4 Screens & features to parity (~48 + new messaging)
|
||||||
All routes from the refreshed audit's screen map, **including the NEW Messages experience** (inbox + conversation
|
All routes from the refreshed audit's screen map, **including the NEW Messages experience** (inbox + conversation
|
||||||
|
|
@ -89,6 +93,12 @@ Run all passes on iOS:
|
||||||
- **NEW — cross-platform pass (X):** a **mixed couple (Android + iOS)** — messaging, answers, dates, premium, and
|
- **NEW — cross-platform pass (X):** a **mixed couple (Android + iOS)** — messaging, answers, dates, premium, and
|
||||||
notifications work cross-platform, and **E2EE decrypts both ways** (the Part 2 interop gate, verified live on real
|
notifications work cross-platform, and **E2EE decrypts both ways** (the Part 2 interop gate, verified live on real
|
||||||
paired devices). This is the make-or-break for the cornerstone.
|
paired devices). This is the make-or-break for the cornerstone.
|
||||||
|
- **iOS-native dimensions (add to the passes):** Dynamic Type (largest sizes), **VoiceOver**, safe-area / notch /
|
||||||
|
Dynamic Island, multiple device sizes (SE → Pro Max; iPad if supported), **edge-swipe-back gesture** + interactive
|
||||||
|
pop (the iOS "double-back"/nav-stack analog), `scenePhase` background/foreground, dark/light.
|
||||||
|
- **Real-device / sandbox needs:** **App Attest/DeviceCheck** and **APNs push** don't fully work on the Simulator —
|
||||||
|
use a real device (or `xcrun simctl push` with a payload for local notif routing); **RevenueCat IAP** needs a
|
||||||
|
StoreKit config file or a sandbox Apple ID. Plan Pass A/E around this.
|
||||||
|
|
||||||
## Definition of done
|
## Definition of done
|
||||||
- **Part 2:** iOS builds green + runs on Simulator + device at feature parity with current Android; E2EE interop
|
- **Part 2:** iOS builds green + runs on Simulator + device at feature parity with current Android; E2EE interop
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue