110 lines
5.0 KiB
Markdown
110 lines
5.0 KiB
Markdown
# Claude QA Plan — <Project>
|
|
|
|
```
|
|
Status: Current
|
|
Owner: <who maintains this>
|
|
Last reviewed: <YYYY-MM-DD>
|
|
Governs: what a QA round consists of
|
|
Review trigger: Any new user-facing surface, or a defect class that got through
|
|
```
|
|
|
|
> The playbook. What a round *is*, so two rounds are comparable and a gap is
|
|
> visible rather than assumed covered.
|
|
|
|
## Before a round
|
|
|
|
- Build from a clean checkout at a known SHA, and record that SHA.
|
|
- Run from a detached worktree if other work is in flight, so uncommitted
|
|
changes cannot contaminate what is under test.
|
|
- Note the environment: device, OS version, browser, screen size — whatever the
|
|
product's behaviour actually depends on.
|
|
|
|
## The passes
|
|
|
|
Each pass gets a letter, so `ClaudeQACoverage.md` can report per pass and a
|
|
skipped one is visible.
|
|
|
|
| Pass | What it covers |
|
|
| --- | --- |
|
|
| A | First run: install or load, cold start, permissions, empty states |
|
|
| B | The core flow, end to end, as a real user would do it |
|
|
| C | The core flow with things going wrong: no network, denied permission, invalid input |
|
|
| D | Persistence: quit and return, background and resume, restart |
|
|
| E | The end of the loop — the state that is hardest to reach on purpose |
|
|
| F | Accessibility: keyboard only, screen reader labels, contrast, text scaling |
|
|
| G | Performance under the load this product will actually see |
|
|
| H | Authorization and abuse: every authenticated endpoint called logged-out; a list endpoint checked for rows the caller should not see; User A requesting User B's object by id; a privileged route opened as an ordinary user; the most expensive endpoint hit repeatedly; the built bundle and `localStorage` inspected for anything secret |
|
|
| I | *(only where money moves)* Money flowing backwards: refund, chargeback, cancellation and failed renewal each take away what the purchase gave |
|
|
|
|
<Add, remove and rename to fit. A pass that never applies is noise; a pass that
|
|
is always skipped is a lie.>
|
|
|
|
### Why H is separate from B
|
|
|
|
**Authenticated is not the same as owning, and neither is the same as
|
|
permitted.** Login is the front door; every room inside still needs its own
|
|
lock. Pass B walks the flow as a real user, which means it walks it as a user
|
|
who is entitled to everything it touches — so B cannot see the entire class of
|
|
defect where the entitlement was never checked.
|
|
|
|
This has already cost this portfolio once. `requireCoupleContext` never verified
|
|
that the caller belonged to the couple whose data was being returned:
|
|
authentication was present, correct, and proved nothing. No pass A-G would have
|
|
looked for it, because every one of them asks a legitimate user to do legitimate
|
|
things.
|
|
|
|
Run H as somebody who is not entitled: logged out, logged in as the wrong
|
|
person, and logged in as the right person asking too often.
|
|
|
|
### Why I is conditional, and why it is here at all
|
|
|
|
Pass B buys. Pass I unbuys, and it is the pass most often missing: builds
|
|
implement the checkout-success webhook and leave every reverse path unhandled,
|
|
so access is granted once and never revoked. Refunded, charged back, cancelled,
|
|
failed renewal — the money went back and the entitlement did not.
|
|
|
|
*(precautionary)* — this one is **not** learned here. It comes from people who
|
|
audit these applications for a living, one of whom calls it the single thing
|
|
they most often fix, and from a report of a refund defect costing a financial
|
|
institution six figures a month. Borrowed evidence is still evidence, but the
|
|
marker says which it is so nobody mistakes it for our own scar.
|
|
|
|
**Delete this pass outright if the project takes no money.** Keeping it as a
|
|
permanently skipped row is the lie this file warns about two paragraphs up.
|
|
|
|
## What counts as a finding
|
|
|
|
A finding needs: what was done, what happened, what should have happened, and
|
|
the build SHA. Without the SHA it cannot be re-tested, and a finding that cannot
|
|
be re-tested cannot be closed.
|
|
|
|
## Severity
|
|
|
|
Findings are filed as issues, labelled:
|
|
|
|
- **P0** — ships broken, or loses data
|
|
- **P1** — materially wrong, but shippable
|
|
- **P2** — cosmetic or low impact
|
|
- **release-blocker** — a release built today would be wrong rather than merely
|
|
incomplete
|
|
|
|
Exactly these label names: the Command Center queries them by name, and a
|
|
repository that spells them differently has its defects reported as *not
|
|
adopted* rather than counted wrongly.
|
|
|
|
Severity is what it costs, not how annoying it is to fix.
|
|
|
|
## After a round
|
|
|
|
File each finding as a labelled issue. Update `ClaudeReport.md`'s run-state
|
|
block and its overall sentence, and `ClaudeQACoverage.md` with what each pass
|
|
actually reached. A pass that could
|
|
not be run is recorded as blocked, with what blocks it — never quietly left out,
|
|
which reads identically to "passed".
|
|
|
|
Then **push, and reconcile**. The verdict on the project screen at
|
|
privacyllc.dev is read out of `ClaudeReport.md` in the pushed repository, so a
|
|
round whose report is committed but not pushed — or pushed but not reconciled —
|
|
leaves a stakeholder reading the previous round's judgment with no indication
|
|
that a newer one exists. The rest of the cycle is in `docs/WORK_CYCLE.md`.
|