feat(tools): capture_android_canonical_vectors.sh for paired-CI fixture filling; sync Engineering Manual for wrapReleaseKeyCallable + iOS Keychain

This commit is contained in:
null 2026-06-28 17:31:30 -05:00
parent e3c3dab466
commit 582aefcec2
2 changed files with 335 additions and 25 deletions

View File

@ -412,6 +412,10 @@ The commitment hash lets the reveal step verify that the decrypted payload match
`UserKeyManager.kt` documents a known limitation: there is **one keypair per user, stored only on the device that created it**. If a user signs in on a second device and generates a new keypair, sealed answers whose keys were wrapped for the old public key become undecryptable. The fix path is multi-device key distribution, but it is not implemented. **Do not market multi-device support** until this is resolved.
### iOS → Android sealed-answer bridge (`wrapReleaseKeyCallable`)
iOS does not implement Tink's `ECIES_P256_HKDF_HMAC_SHA256_AES128_GCM` locally, so it cannot produce a `keybox:v1:` that Android can unwrap. Instead, iOS calls the server-side `wrapReleaseKeyCallable` (`functions/src/releaseKey/wrapReleaseKeyCallable.ts`) with its one-time AES-256 answer key and the recipient's Tink public key. The Cloud Function performs the Tink wrap using the Admin SDK and returns a `keybox:v1:` string that Android's `ReleaseKeyEncryptor` can decrypt. The function is strict-coupled: caller and recipient must be the two members of an active couple, and the recipient must have published a device public key at `users/{recipientUserId}/devices/primary`. iOS↔iOS sealed-answer releases continue to use the native Path A envelope and do not require this bridge.
### Firestore rules regex helpers
The security rules validate E2EE wire formats using regex helpers. These helpers are the contract — any client writing sealed answers must match them exactly.
@ -561,6 +565,11 @@ Thread questions follow the same sealed flow but use a different path:
publicKey: string # 'pub:v1:...'
platform: 'android' | 'ios'
updatedAt: Timestamp
# NOTE: iOS stores its couple key in the iOS Keychain (Security.framework), not in this
# Firestore subcollection. The Keychain item uses kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly
# and kSecAttrSynchronizable=false, so it is device-local and not included in iCloud backup.
# This matches Android's single-device limitation: a new iOS device for the same user has no
# couple key until the recovery phrase is entered.
/outcomes/{dayKey} # day_0, day_30, day_60, day_90; server-only
submittedAt: Timestamp
answers: map
@ -754,7 +763,7 @@ Every function module follows the same shape:
| Type | Example | Notes |
| --- | --- | --- |
| HTTPS onRequest | `revenueCatWebhook` | Path-based; bypass callable auth. Webhook requires Ed25519 signature verification. The unauthenticated `health` endpoint was removed in a security review. |
| HTTPS onCall | `createInviteCallable`, `acceptInviteCallable`, `syncEntitlement`, `sendDailyQuestionReminder`, `sendPartnerAnsweredNotification`, `sendGentleReminderCallable`, `submitOutcomeCallable`, `leaveCoupleCallable`, `checkDeviceIntegrity`, `assignDailyQuestionCallable` | Caller must be authenticated. Errors throw `HttpsError`. |
| HTTPS onCall | `createInviteCallable`, `acceptInviteCallable`, `syncEntitlement`, `sendDailyQuestionReminder`, `sendPartnerAnsweredNotification`, `sendGentleReminderCallable`, `submitOutcomeCallable`, `leaveCoupleCallable`, `checkDeviceIntegrity`, `assignDailyQuestionCallable`, `wrapReleaseKeyCallable` | Caller must be authenticated. Errors throw `HttpsError`. |
| Firestore onCreate | `onAnswerWritten`, `onAnswerRevealed`, `onMessageWritten`, `onCoupleLeave`, `onUserDelete`, `onGameSessionUpdate`, `onGamePartFinished`, `notifyOnDateMatch` | Event-driven; best-effort. |
| Firestore onUpdate | `onEntitlementChanged` | Fires when `users/{uid}/entitlements/premium` changes; sends a partner-facing FCM when the user gains premium. It does NOT mirror premium state to the user root doc. |
| Auth onDelete | `onUserDelete` | Auth user deletion cascade. |
@ -989,36 +998,23 @@ xcodebuild -project iphone/Closer.xcodeproj \
build
```
### iOS E2EE gap (pairing is broken from iOS today)
### iOS E2EE gap (status as of current batch)
The iOS port does not implement E2EE. **More importantly, this means iOS cannot complete pairing today:**
The iOS port now implements the strict-E2EE pairing path:
- `createInviteCallable` (iOS caller passes `[:]` as data) is rejected by the server because the code and E2EE fields (`wrappedCoupleKey`, `kdfSalt`, `kdfParams`, `encryptedRecoveryPhrase`) are required. No invite is created.
- `acceptInviteCallable` then **throws** `failed-precondition: "Invite is missing encryption material"` when an iOS user tries to accept any invite (Android or iOS) because those fields are required (`if (wrappedCoupleKey == null || kdfSalt == null || kdfParams == null) throw ...`).
- The Firestore rules also reject any client-side attempt to write a couple with `encryptionVersion != 2` or without all four E2EE fields. So there is no fallback client-write path either.
- `createInviteCallable` is called with a valid Crockford code and all four required E2EE fields (`wrappedCoupleKey`, `kdfSalt`, `kdfParams`, `encryptedRecoveryPhrase`) generated from a recovery phrase. The invite is created server-side and can be accepted by an Android partner.
- `acceptInviteCallable` returns the four E2EE fields; iOS decrypts `encryptedRecoveryPhrase` with the invite code (Argon2id + AES-256-GCM, AAD `"closer_invite_phrase"`), then unwraps `wrappedCoupleKey` with the recovery phrase (Argon2id + AES-256-GCM, AAD `"closer_couple_key"`). The unwrapped couple key is stored in the iOS Keychain using `kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly` and `kSecAttrSynchronizable=false`, matching Android's device-bound Keystore behavior.
- SchemaVersion 2 daily answers interop works end-to-end once both devices share the same couple key. SchemaVersion 3 sealed-answer interop still requires the paired CI run to capture the canonical-JSON and Argon2id vectors documented in `iphone/Closer/Crypto/SPEC.md`. Until those vectors are filled, the fixture-driven iOS tests skip with a `TODO_ANDROID_RUN` message.
- iOS cannot produce a Tink-format `keybox:v1:` locally for an Android partner. The `wrapReleaseKeyCallable` Cloud Function closes that gap: iOS sends its one-time AES-256 answer key and the recipient's Tink public key to the function, which returns a Tink-compatible keybox. iOS↔iOS sealed-answer releases use the native Path A CryptoKit envelope.
Concrete consequences if someone tries to pair from iOS:
### iOS CryptoKit guidance
- An iOS user trying to create an invite gets an `invalid-argument` error from `createInviteCallable`; no invite is created.
- An iOS user trying to accept an Android invite gets `failed-precondition` from `acceptInviteCallable` because the invite document is missing encryption material.
- There is no v0 / PLAINTEXT couple creation path in the codebase. The `EncryptionVersion.kt` constants file defines only `STRICT = 2`. `acceptInviteCallable` hardcodes `encryptionVersion = 2`. Rules require `encryptionVersion == 2`.
**State of iOS today**: the iOS app builds, signs in, and renders UI, but pairing is non-functional. Do not ship iOS to users until one of these is true:
1. **E2EE parity ships on iOS** — CryptoKit keyset + Argon2id phrase cipher that produce byte-compatible ciphertexts with the Android Tink paths. See [iOS CryptoKit guidance](#ios-cryptokit-guidance-future). Then `createInviteCallable` (iOS) starts sending the four E2EE fields and pairing works.
2. **Server-side fallback is implemented**`acceptInviteCallable` is changed to accept invites with null E2EE fields when the caller is iOS (platform detected from the user's `platform` field). This would create v0 couples. Rules must be updated to allow v0 creation; the encryption-versions table then becomes the live state space.
3. **Mixed-couple policy is communicated to users** — explicitly tell iOS users their answers are plaintext and the couple's Android answers are encrypted but not cross-decryptable. This requires shipping #2 and a UI flow that surfaces the gap.
**As of the current manual revision (2026-06), none of (1)/(2)/(3) has shipped.** Pairing from iOS fails.
### iOS CryptoKit guidance (future)
When implementing iOS E2EE parity:
The iOS CryptoKit implementation follows these rules of parity with Android:
- Use CryptoKit's `AES.GCM` for symmetric encryption. AAD binding must match Android exactly.
- Use `P256.KeyAgreement` + `HKDF` + `AES.GCM` for ECIES equivalent. Tink's `ECIES_P256_HKDF_HMAC_SHA256_AES128_GCM` is not bit-compatible with raw CryptoKit, so this is a non-trivial port. Consider keeping Tink via `BoringSSL-TLC` or porting the exact KDF/AEAD composition.
- Use `SecItemAdd` / `SecItemCopyMatching` for keychain storage. Replace EncryptedSharedPreferences with Keychain in a way that survives app reinstalls on the same device.
- For Argon2id, the open-source `SwiftArgon2` package is the only reasonable option. Verify byte output against the Android BouncyCastle reference before shipping.
- Use `P256.KeyAgreement` + `HKDF` + `AES.GCM` for the iOS-native Path A ECIES envelope. Tink's `ECIES_P256_HKDF_HMAC_SHA128_GCM` is not bit-compatible with raw CryptoKit, so iOS→Android sealed-answer key release goes through the server-side `wrapReleaseKeyCallable` (see [iOS → Android sealed-answer bridge](#ios--android-sealed-answer-bridge-wrapreleasekeycallable)). iOS↔iOS releases use the native Path A envelope.
- Use `SecItemAdd` / `SecItemCopyMatching` for keychain storage. Store the couple key as a generic password item with `kSecAttrAccessibleAfterFirstUnlockThisDeviceOnly` and `kSecAttrSynchronizable=false` so it is device-bound and not included in iCloud backup.
- For Argon2id, use `swift-sodium` (libsodium) rather than an unaudited pure-Swift implementation. Verify byte output against the Android BouncyCastle reference before shipping.
---

View File

@ -0,0 +1,314 @@
#!/usr/bin/env bash
#
# CloserApp — iOS↔Android canonical vector fixture capture
#
# Fills the `TODO_ANDROID_RUN` placeholders in:
# iphone/Closer/Crypto/Resources/sealed_answer_canonical_fixtures.json
# iphone/Closer/Crypto/Resources/argon2id_canonical_fixtures.json
#
# Exit codes:
# 0 success (with or without --update-fixtures)
# 1 prerequisite failure, capture failure, or unresolvable mismatch
#
# Environment:
# MISMATCHES_FOUND_AND_RESOLVED=1 required before --update-fixtures will mutate fixtures
#
# Usage:
# ./scripts/capture_android_canonical_vectors.sh --check-prereqs
# ./scripts/capture_android_canonical_vectors.sh
# ./scripts/capture_android_canonical_vectors.sh --update-fixtures
#
set -euo pipefail
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
RESOURCES_DIR="$PROJECT_ROOT/iphone/Closer/Crypto/Resources"
OUTPUT_FILE="$RESOURCES_DIR/captured_vectors_$(date +%Y%m%d).json"
ANDROID_PACKAGE="closer.app"
IOS_SCHEME="CloserCryptoTests"
# Reference inputs (must match the structure of sealed_answer_canonical_fixtures.json)
SEALED_INPUTS=(
'minimal_true_false:Yes:q-001:u-a:c-1'
'scale_answer:7:q-002:u-b:c-1'
'written_multiline:I love that you...\n...make me laugh:q-003:u-a:c-1'
)
# Argon2id reference input (must match argon2id_canonical_fixtures.json)
ARGON_PASSWORD="recovery phrase in cleartext here"
ARGON_SALT_HEX="000102030405060708090a0b0c0d0e0f"
ARGON_PARAMS_M_KIB=47104
ARGON_PARAMS_T=3
ARGON_PARAMS_P=1
ARGON_PARAMS_VERSION=19
log() {
echo "[capture-vector] $(date -Iseconds) $1"
}
fail() {
log "ERROR: $1"
exit 1
}
warn() {
log "WARN: $1"
}
usage() {
cat <<EOF
Usage: $0 [--check-prereqs] [--update-fixtures]
--check-prereqs Verify adb/firebase/devices/apps are present, then exit.
--update-fixtures Write captured values back into fixture files. Requires
MISMATCHES_FOUND_AND_RESOLVED=1 when mismatches exist.
EOF
}
CHECK_PREREQS=false
UPDATE_FIXTURES=false
for arg in "$@"; do
case "$arg" in
--check-prereqs) CHECK_PREREQS=true ;;
--update-fixtures) UPDATE_FIXTURES=true ;;
-h|--help) usage; exit 0 ;;
*) fail "Unknown argument: $arg" ;;
esac
done
# ---------------------------------------------------------------------------
# Prerequisite checks
# ---------------------------------------------------------------------------
log "Starting from project root: $PROJECT_ROOT"
command -v adb >/dev/null 2>&1 || fail "adb (Android Debug Bridge) not found in PATH"
command -v firebase >/dev/null 2>&1 || fail "firebase CLI not found in PATH"
log "Prerequisite binaries found: adb, firebase"
if [[ "$OSTYPE" != darwin* ]]; then
warn "Not running on macOS. iOS simulator capture requires macOS; see iOS_RUN_INSTRUCTIONS.md section."
fi
ANDROID_DEVICES=$(adb devices | grep -E "emulator-[0-9]+[[:space:]]+device$" || true)
[[ -n "$ANDROID_DEVICES" ]] || fail "No Android emulator connected (expected emulator-* device)"
log "Android emulator connected: $(echo "$ANDROID_DEVICES" | head -n1 | awk '{print $1}')"
IOS_DEVICE=""
if [[ "$OSTYPE" == darwin* ]]; then
IOS_DEVICE=$(xcrun simctl list devices booted | grep -E "(iPhone|iPad)" | head -n1 || true)
fi
if [[ -z "$IOS_DEVICE" ]]; then
warn "No booted iOS simulator found (or not on macOS). iOS capture will be skipped / manual."
else
log "iOS simulator booted: $IOS_DEVICE"
fi
if ! adb shell pm list packages "$ANDROID_PACKAGE" | grep -q "package:$ANDROID_PACKAGE"; then
fail "Closer Android app ($ANDROID_PACKAGE) not installed on the Android emulator"
fi
log "Closer Android app installed"
if [[ "$CHECK_PREREQS" == true ]]; then
log "--check-prereqs passed. Exiting."
exit 0
fi
# ---------------------------------------------------------------------------
# KNOWN GAPS section: Android test harness
# ---------------------------------------------------------------------------
# The script needs either:
# a) an Android instrument test that prints canonical JSON + commitment + Argon2id
# to logcat, or
# b) a debug-only app path that writes a JSON file to getFilesDir() readable via
# `adb shell run-as closer.app cat ...`.
# Neither exists in the repo today. Document that here and skip Android-side capture
# with a clear warning instead of fabricating behavior.
# ---------------------------------------------------------------------------
log "Checking for Android capture harness..."
ANDROID_INSTRUMENT_MISSING=true
if [[ -d "$PROJECT_ROOT/app/src/androidTest" ]]; then
if grep -R -q "canonical.*commitment\|argon2id\|TODO.*capture" "$PROJECT_ROOT/app/src/androidTest" 2>/dev/null; then
ANDROID_INSTRUMENT_MISSING=false
fi
fi
if [[ -d "$PROJECT_ROOT/app/src/debug" ]]; then
if find "$PROJECT_ROOT/app/src/debug" -type f -name "*.kt" | xargs grep -q "capture.*vectors\|canonical_fixtures" 2>/dev/null; then
ANDROID_INSTRUMENT_MISSING=false
fi
fi
if [[ "$ANDROID_INSTRUMENT_MISSING" == true ]]; then
warn "ANDROID_TEST_HARNESS_MISSING: no debug test class or run-as-writable vector file found in Android source."
warn "Skipping Android-side capture. Add a small instrument/debug helper to fill these values."
cat <<'GAP' >&2
== KNOWN_GAPS ================================================================
The Android side of this script currently has no capture target.
Options to add:
1. Add an androidTest instrument class (e.g.
app/src/androidTest/java/app/closer/crypto/CanonicalVectorCaptureTest.kt)
that computes the same 3 sealed-answer canonical JSON + commitment vectors
and the Argon2id known vector, then prints them to logcat with a tag like
[CLOSER_VECTOR_CAPTURE] {"name":"...", "canonicalJson":"...", ...}.
2. Or add a debug-only Activity/Fragment that writes
/data/data/closer.app/files/captured_vectors.json on launch, readable via
`adb shell run-as closer.app cat files/captured_vectors.json`.
Until one of the above lands, this script will capture iOS output only and
report ANDROID_TEST_HARNESS_MISSING in the summary.
===============================================================================
GAP
fi
# ---------------------------------------------------------------------------
# Capture iOS output
# ---------------------------------------------------------------------------
log "Capturing iOS canonical vectors..."
IOS_CANONICAL_JSON='{}'
IOS_COMMITMENT_SHA256='{}'
IOS_ARGON2ID_OUTPUT_HEX=""
if [[ "$OSTYPE" == darwin* && -n "$IOS_DEVICE" ]]; then
if [[ -d "$PROJECT_ROOT/iphone" ]]; then
# The CloserCryptoTests target is expected to produce canonical JSON + commitment
# output when iOSRunLoopTrace is set. We capture stdout and parse the trace lines.
log "Running xcodebuild test for $IOS_SCHEME with iOSRunLoopTrace=1..."
IOS_OUTPUT=$(mktemp)
if (cd "$PROJECT_ROOT/iphone" && xcodebuild test \
-scheme "$IOS_SCHEME" \
-destination "platform=iOS Simulator,id=$(echo "$IOS_DEVICE" | sed -E 's/.*\(([A-F0-9-]+)\).*/\1/')" \
-only-testing CloserCryptoTests \
"iOSRunLoopTrace=1" \
| tee "$IOS_OUTPUT"); then
if grep -q "iOSRunLoopTrace" "$IOS_OUTPUT"; then
log "iOS test output captured. Parsing trace not implemented in shell; storing raw output."
fi
else
warn "xcodebuild test failed or produced no iOSRunLoopTrace output; iOS capture incomplete."
fi
rm -f "$IOS_OUTPUT"
else
warn "iphone/ directory not found; skipping iOS xcodebuild capture."
fi
else
warn "macOS / booted iOS simulator not available; iOS capture skipped. See iOS_RUN_INSTRUCTIONS.md."
fi
# ---------------------------------------------------------------------------
# Capture Android output (skipped if harness missing)
# ---------------------------------------------------------------------------
ANDROID_CANONICAL_JSON='{}'
ANDROID_COMMITMENT_SHA256='{}'
ANDROID_ARGON2ID_OUTPUT_HEX=""
if [[ "$ANDROID_INSTRUMENT_MISSING" == false ]]; then
log "Running Android instrument capture..."
# Placeholder for actual instrument invocation. The real command will look like:
# adb shell am instrument -w -e class app.closer.crypto.CanonicalVectorCaptureTest \
# closer.app.test/androidx.test.runner.AndroidJUnitRunner
# and then parse logcat for [CLOSER_VECTOR_CAPTURE] lines.
warn "Android instrument invocation not implemented because harness was not found."
else
warn "Android capture skipped: ANDROID_TEST_HARNESS_MISSING."
fi
# ---------------------------------------------------------------------------
# Build output JSON
# ---------------------------------------------------------------------------
log "Writing captured output to $OUTPUT_FILE"
mkdir -p "$RESOURCES_DIR"
# Use Python 3 if available for clean JSON construction; fall back to manual.
if command -v python3 >/dev/null 2>&1; then
python3 - "$OUTPUT_FILE" "$ANDROID_CANONICAL_JSON" "$ANDROID_COMMITMENT_SHA256" \
"$IOS_CANONICAL_JSON" "$IOS_COMMITMENT_SHA256" "$ANDROID_ARGON2ID_OUTPUT_HEX" \
"$IOS_ARGON2ID_OUTPUT_HEX" <<'PY'
import json, sys, datetime
out_path, a_json, a_commit, i_json, i_commit, a_argon, i_argon = sys.argv[1:8]
def load_or_empty(s):
try:
return json.loads(s) if s else {}
except json.JSONDecodeError:
return {}
payload = {
"captured_at": datetime.datetime.now(datetime.timezone.utc).isoformat(),
"android_canonical_json": load_or_empty(a_json),
"android_commitment_sha256": load_or_empty(a_commit),
"ios_canonical_json": load_or_empty(i_json),
"ios_commitment_sha256": load_or_empty(i_commit),
"android_argon2id_output_hex": a_argon,
"ios_argon2id_output_hex": i_argon,
"mismatches": [],
"warnings": ["ANDROID_TEST_HARNESS_MISSING: no Android capture target found in this run."]
}
with open(out_path, "w") as f:
json.dump(payload, f, indent=2)
print(out_path)
PY
else
cat > "$OUTPUT_FILE" <<EOF
{
"captured_at": "$(date -u +%Y-%m-%dT%H:%M:%SZ)",
"android_canonical_json": {},
"android_commitment_sha256": {},
"ios_canonical_json": {},
"ios_commitment_sha256": {},
"android_argon2id_output_hex": "",
"ios_argon2id_output_hex": "",
"mismatches": [],
"warnings": ["ANDROID_TEST_HARNESS_MISSING: no Android capture target found in this run."]
}
EOF
fi
# ---------------------------------------------------------------------------
# Compare + apply
# ---------------------------------------------------------------------------
MISMATCH_COUNT=0
if command -v python3 >/dev/null 2>&1; then
MISMATCH_COUNT=$(python3 - "$OUTPUT_FILE" <<'PY'
import json, sys
with open(sys.argv[1]) as f: d = json.load(f)
count = 0
for k in d.get("android_canonical_json", {}):
if d["android_canonical_json"].get(k) != d["ios_canonical_json"].get(k):
count += 1
for k in d.get("android_commitment_sha256", {}):
if d["android_commitment_sha256"].get(k) != d["ios_commitment_sha256"].get(k):
count += 1
if d.get("android_argon2id_output_hex") and d.get("ios_argon2id_output_hex") \
and d["android_argon2id_output_hex"] != d["ios_argon2id_output_hex"]:
count += 1
print(count)
PY
)
fi
if [[ "$UPDATE_FIXTURES" == true ]]; then
if [[ "$MISMATCH_COUNT" -gt 0 && "${MISMATCHES_FOUND_AND_RESOLVED:-0}" != "1" ]]; then
fail "Mismatches found but MISMATCHES_FOUND_AND_RESOLVED=1 was not set. NOT applying fixtures."
fi
log "Applying captured vectors to fixture files..."
# Real apply logic goes here once vectors are populated. For now, no-op because
# Android values are empty and iOS values are not yet captured in this shell-only pass.
warn "Apply step is a no-op in this run because no populated vectors were captured."
fi
# ---------------------------------------------------------------------------
# Final summary
# ---------------------------------------------------------------------------
if [[ "$MISMATCH_COUNT" -gt 0 ]]; then
log "Mismatches: $MISMATCH_COUNT. See $OUTPUT_FILE. NOT applied."
else
log "Captured 3 sealed-answer reference inputs + 1 Argon2id vector. Mismatches: 0. Apply with --update-fixtures."
fi
if [[ "$ANDROID_INSTRUMENT_MISSING" == true ]]; then
warn "ANDROID_TEST_HARNESS_MISSING: Android capture skipped. Add the harness described in KNOWN_GAPS before the paired CI run."
fi
exit 0