AI-generated analysisPublished automatically and not human-verified. Validated context appears in community notes below.
← Watch feed
Low 27 Monero

QA: report-only e2e lanes (android / iOS / cache nightly) (#3700)

Public commit record

What the developer wrote

Authored by Seth For Privacy

100/100 · Strong
QA: report-only e2e lanes (android / iOS / cache nightly) (#3700)

* QA lanes: e2e_android (report-only, artifact reuse), e2e_ios (self-hosted), e2e_cache nightly (QA handoff §6 4-a/4-b/4-c)

* e2e_android: wait for the pr_test_build artifact instead of racing it

* e2e_android: 90min artifact wait (pr_test_build includes integration tests), 240min job cap

* e2e_android: qa_ref input (test an arbitrary cake-qa branch; 4-e broken-label leg)

* e2e_android + nightly: disable animations in-script with a settle/retry (action's step broke with 'Broken pipe' after a 764s boot)

* e2e lanes: emulator-boot-timeout 1200s + 4 cores (600s default flaked after a 764s boot)

* e2e lanes: POSIX-sh script (action runs dash: no pipefail, no arrays)

* e2e lanes: single-line scripts (parseScript runs each line as its own dash -c)

* e2e lanes: self-contained script lines (per-line sh -c: cd does not persist; proxy path root-relative, npx gets 'cd cake-qa &&')

* e2e lanes: derive CW_PACKAGE from the installed APK (PR builds use com.cakewallet.test_<branch>, config default is the RC id — caused ENGINE_FAILURE on app.clearState)

* e2e lanes: adb daemon reset before the suite (hosted adb wedged mid-run: 'Unable to connect to adb daemon' -> 39 cascading failures)

* e2e lanes: wait for adb reconnect after daemon reset (fresh server exited the package query in 6ms -> device offline)

* e2e_android: split artifact-wait into its own job (the shared 4h cap canceled the suite twice before it started)

* e2e_android: 190min artifact wait (builds run 60-190min), suite gated on resolve run_id, resolve job timeout 215

* e2e_android: run ONLY the no-funds slice (bare 'npx e2e run' ran the whole project — map verification + the expected-fail known-defects — and the job was canceled mid-suite)

* e2e lanes: KVM udev rule (action fell back to -accel off) + 3072M AVD memory like the funds lane

* nightly: KVM udev rule + 3072M memory (same as the android lane)

* e2e_android resolve: auto-retry the build's failed jobs once on a flaky packageRelease failure (3rd occurrence; stops the push-per-fix churn)

* e2e_android: actions:write so the build auto-retry actually runs (403 was swallowed)

* e2e_android: suite input (default = no-funds slice; the 4-e broken-label run selects its fixture)

* e2e_android: retry only the packageRelease flake; fail fast on manual runs with no build

The resolve step re-ran every failed pr_test_build when no artifact appeared, so a PR that does not compile had its failed build jobs re-run on every push before the lane gave up. It now reads the failed jobs' logs and retries only when they contain the gradle packageRelease IncrementalSplitterRunnable failure; anything else fails the lane at once.

A workflow_dispatch without build_run on a SHA that has no pr_test_build run never matched the early exit and polled for the full 190-minute deadline. It now fails immediately with a hint to pass build_run. The comment that still said the wait gives up after an hour is corrected.

* e2e_ios: one fresh qa-e2e simulator per run, deleted afterwards, and the suite targets it

Every run created another qa-e2e device (simctl allows duplicate names) and booted it, and nothing shut it down, so booted simulators piled up on the persistent runner. The step now deletes any existing qa-e2e devices, creates one, and an always() step shuts down and deletes it. The get_app_container fallback, which could never return a UDID, is gone.

The device type came from sorting all device types by name, which picks iPhone Xr on current Xcode and fails against the iOS 27 runtime; it now comes from the chosen runtime's supported iPhone types. The suite step also sets CW_IOS_DEVICE=qa-e2e: cake-qa's sim target otherwise defaults to the simulator named iPhone 18 Pro, not the device the build was installed on.

* e2e_android: grep input, redact seed-like word runs before upload, 4GB emulator

* e2e lanes: use the llm.cakewallet.com router

dspark.privacy.family is deprecated. The router address is not a secret, so
it is set in the workflow; the key is read from LLM_API_KEY, falling back to
the existing DSPARK_API_KEY (the same key works on the new router).

* e2e lanes: find the x86_64 APK under the build's per-branch artifact name

The build now uploads <branch>_x86.apk instead of one 'android apk' artifact.
The lanes accept either and normalise the file name for the install step.

* e2e iOS lane: call flutter by path in the install step

GITHUB_PATH only applies to later steps, so 'flutter --version' in the same
step failed with 'command not found' on every run.

* e2e iOS lane: do not GPG-sign the dependency patch commits

The runner account signs commits by default and gpg is not on the runner's
PATH, so 'git am' in apply_patches.sh failed.

* e2e iOS lane: put Homebrew and ~/go/bin on the runner's PATH

The runner service starts with a bare PATH, so go, cmake, pod and gomobile
were not found by the simulator build.

* e2e iOS lane: keep the pinned Flutter ahead of Homebrew on PATH

Homebrew's standalone dart shadowed Flutter's, so 'dart run ffigen' in
cw_decred failed with 'requires the Flutter SDK'.

* e2e iOS lane: qa_ref, suite and grep inputs for manual runs

Same inputs as the Android lane, so a cake-qa branch or a single charter can
be run on the simulator without merging first.

* e2e lanes: run the test runner with node, default the iOS lane to the no-funds slice

- npx (npm exec) rewrites its process title; on macOS the overflow exposes the
job's environment, including the wallet seed secrets, in the process list.
The lanes now call node node_modules/.bin/e2e directly.
- The iOS lane passed no suite, so it ran every test file (293 tests, 143 min)
instead of the 17-test no-funds slice its step is named after.

* e2e iOS lane: tell the suite where the simulator build is

The suite reinstalls the app between tests on iOS, where wiping app data
leaves preferences and the Keychain behind.

* e2e lanes: keep the cake-qa key off disk, publish only redacted junit and summary, build dev for the nightly

* e2e iOS lane: dispatch-only until it runs in an ephemeral VM

* e2e lanes: retry the flake-log read, wait for the simulator, stop the proxy, let the nightly survive a red build test

* e2e iOS lane: build without secrets and test in separate throwaway VMs

* e2e iOS lane: install protoc for the build, run the suite only when an app was built

* e2e iOS lane: download the simulator runtime unless an available one exists

* e2e Android lane: a cancelled run no longer overwrites the PR comment

* e2e Android lane: a failed run listing while waiting for the APK is retried
✓ Specific, descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context✓ Explains rationale or failure mode✓ Mentions testing or verification✓ Links an issue, advisory, or supporting reference✓ Names security-relevant behavior explicitly
The short version

What changed, and why it matters

This commit adds three new automated testing pipelines for the Cake Wallet mobile app: one for Android, one for iOS, and a weekly overnight test. These are explicitly marked as 'reply-only' experiments, meaning a failure will not block releases. The pipelines run real app builds in cloud test environments, send test commands to the app, and use sensitive test wallet seed phrases and API keys. The commit also includes several safety measures: it keeps secrets out of logs, uses encrypted storage for detailed test recordings, and avoids a subtle iOS issue where a common test runner could accidentally expose secrets in the process list. Because this is new infrastructure handling real secrets and running real app code, it deserves a security review, but it is not a known vulnerability in the app itself.

Recommended action

Treat this as a security-sensitive infrastructure change, not an app vulnerability. Reviewers should verify: (1) secrets are not echoed in workflow logs or uploaded unredacted; (2) the `OLLAMA_API_KEY`/`OLLAMA_API_KEY` values are scoped and rotated; (3) the deploy key for `cake-wallet-qa` has only the minimum required permissions; (4) the encrypted artifact passphrase is stored separately from the artifacts; (5) the `NO_REPLY` setting is intentional and documented so a failing security-relevant test does not get ignored; (6) the iOS build VM is genuinely isolated from the test VM. No user-facing patch is required.

Security signals we found

01

New CI/CD pipeline handling wallet seed phrases and API keys

02

Use of `NO_REPLY` / `continue-on-error: true` to avoid blocking merges

03

Explicit mitigation: avoids `npx` on macOS to prevent environment-variable leakage via process title

04

Explicit mitigation: redacts seed-like strings from uploaded artifacts

05

Explicit mitigation: encrypts detailed test recordings with a passphrase before upload

06

iOS build stage runs without secrets; secrets only enter the test stage

07

Overnight job uses a deploy key to push replay data to a separate repo

08

No changes to application source code; purely infrastructure/test automation

Risk score

Why this scored 27/100

Our methodology →
Potential impact 4/30
Exploitability 3/25
Stealth signal 5/15
Affected reach 6/15
Confidence 6/10
Evidence quality 3/5
Human-validated context

Community notes

Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.

No validated notes yet.

The AI analysis stands alone for now. Submit a note if you can add evidence or important context.