test(clear_signing): run the cases without display format
What changed, and why it matters
This commit only changes test scripts and documentation for the Trezor firmware clear-signing test suite. It makes the tests run more registry cases—specifically those where the developer definitions do not yet have a display format—so the nightly summary can list them as 'blind-signed, no format.' It does not change the firmware itself, the signing logic, or any user-facing behavior. There is no security fix or vulnerability here.
No action needed. This is a test-infrastructure change with no security relevance.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The commit modifies the clear-signing test harness to stop skipping registry cases that lack a display format in the dev definitions. It adds a has_display_format field to RegistryCase, updates the skip logic in registry.py, adds a new ‘blind, no format’ status in nightly.py, and updates the summary template and docs. The firmware behavior is unchanged: it already either clear-signs with a built-in format or blind-signs these cases. The change is purely test coverage and reporting.
Changed components
tests/clear_signing_tests/nightly.pytests/clear_signing_tests/registry.pytests/clear_signing_tests/test_registry.pytests/clear_signing_tests/summary.md.makodocs/tests/clear-signing-tests.mdInspect captured patch +63 / −22
### docs/tests/clear-signing-tests.md
@@ -33,22 +33,31 @@ deploy.
## What fails
-Each registry case with a chain ID 1 transaction is one test. A case is skipped when the
-dev definitions have no display format for its contract and function (the definitions
-converter dropped it, or it is not deployed there), or when its calldata is longer than
-the firmware clear-signs (6144 bytes).
+Each registry case with a chain ID 1 transaction is one test, of every provider, whether
+trezor/definitions supports it or not. A case is skipped only when its calldata is longer
+than the firmware clear-signs (6144 bytes), or when the test cannot sign its transaction
+(e.g. contract creation, transaction types other than legacy and EIP-1559).
+
+A case runs even when the dev definitions have no display format for its contract and
+function: trezor/definitions does not convert that provider, the converter dropped the
+format (an unsupported feature, an override by another descriptor), or the descriptor does
+not cover that contract on mainnet. The firmware then clear-signs it only with a built-in
+display format (ERC-20 `approve` and `transfer`), and otherwise blind-signs it. The summary
+lists the latter as "blind-signed, no display format": the cases that the definitions do
+not cover.
A test fails when:
- signing fails,
- its screens differ from those of the last nightly run, or the test did not run then
(`UI_FAILED`, `UI_MISSING`),
-- a test of the last nightly run no longer runs, e.g. because a display format disappeared.
+- a test of the last nightly run no longer runs, e.g. because its registry case was
+ removed.
-Blind signing (the firmware shows the `confirm_data` screen although the dev
-definitions have a display format) is not a failure. It is recorded like clear signing,
-and listed in the summary. A case that changes from one to the other fails on its
-screens.
+Blind signing (the firmware shows the `confirm_data` screen), with or without a display
+format in the dev definitions, is not a failure. It is recorded like clear signing, and
+listed in the summary. A case that changes from one to the other fails on its screens,
+e.g. when a display format is dropped from, or added to, the definitions.
Since the next run compares with this one, a UI change fails only the one run that
brings it. In the UI report, a new test shows *missing* in place of the previous screens,
@@ -85,7 +94,7 @@ each failed test shows its screens next to those of the last run, with a diff, a
## Running locally
From the internal network, the dev definitions can be used by their URL. Check out the
-registry at the commit from `resolve_inputs.py`:
+registry at the commit from `nightly.py resolve-inputs`:
```sh
tests/clear_signing_tests/nightly.py resolve-inputs # prints registry_sha
### tests/clear_signing_tests/nightly.py
@@ -395,8 +395,10 @@ def summarize(
rows: list[_ModelRow] = []
# test_id -> ["T3W1: error DataError: ...", ...]
problems: dict[str, list[str]] = defaultdict(list)
- # test_id -> models
+ # test_id -> models; blind-signed although the dev definitions have a display format
blind: dict[str, list[str]] = defaultdict(list)
+ # test_id -> models; blind-signed, the dev definitions have no display format
+ no_format: dict[str, list[str]] = defaultdict(list)
# test_id -> (models, missing texts)
not_found: dict[str, tuple[list[str], list[str]]] = {}
for model in models:
@@ -405,12 +407,16 @@ def summarize(
for case in to_run:
result = model_results.get(case.test_id)
status = result["status"] if result else "not run"
+ if status == "blind" and not case.has_display_format:
+ status = "blind, no format"
counts[status] += 1
if status == "clear_signed":
if result and (missing := result.get("missing")):
counts["text not found"] += 1
entry = not_found.setdefault(case.test_id, ([], missing))
entry[0].append(model)
+ elif status == "blind, no format":
+ no_format[case.test_id].append(model)
elif status == "blind":
blind[case.test_id].append(model)
else:
@@ -433,6 +439,7 @@ def summarize(
skipped=skipped.most_common(),
problems=sorted(problems.items()),
blind=sorted(blind.items()),
+ no_format=sorted(no_format.items()),
not_found=sorted(not_found.items()),
max_lines=MAX_LINES,
)
### tests/clear_signing_tests/registry.py
@@ -18,7 +18,8 @@
The cases come from the `registry/<entity>/testsv2/calldata-*.tests.json` files of
https://github.com/ethereum/clear-signing-erc7730-registry, at the commit that the last
-unsigned dev deploy of trezor/definitions used. The network, display formats and tokens
+unsigned dev deploy of trezor/definitions used, of all providers (entities), whether the
+definitions have display formats for them or not. The network, display formats and tokens
come from that deploy: a local copy of its `eth/chain-id/<id>/...` tree (CI), or its URL
(only reachable from the internal network).
@@ -267,28 +268,32 @@ class RegistryCase:
expected: dict[str, t.Any]
tx: Transaction | None
skip_reason: str | None
+ # Whether the dev definitions have a display format for the contract and function.
+ # A case without one still runs: the firmware blind-signs it.
+ has_display_format: bool = False
def _slug(text: str) -> str:
return re.sub(r"[^A-Za-z0-9_.-]+", "_", text).strip("_")
-def _skip_reason(tx: Transaction, source: Source) -> str | None:
+def _skip_reason(tx: Transaction) -> str | None:
if len(tx.data) < 4:
return "calldata has no function selector"
if len(tx.data) > MAX_CLEAR_SIGNED_DATA:
return (
f"calldata longer than {MAX_CLEAR_SIGNED_DATA} bytes is never clear-signed"
)
- if source.get_eth_display_format(tx.chain_id, tx.to, tx.data[:4]) is None:
- return "no display format in the dev definitions"
return None
def load_cases(registry_root: Path, source: Source) -> list[RegistryCase]:
- """Load the mainnet calldata test cases, in a stable order.
+ """Load the mainnet calldata test cases of all providers, in a stable order.
- A case that cannot run gets a `skip_reason`. Cases on other chains are left out.
+ A case that cannot run gets a `skip_reason`. Cases on other chains are left out. A
+ case without a display format in the dev definitions runs too (and blind-signs,
+ unless the firmware has a built-in one), so that the summary shows which cases the
+ definitions do not cover.
"""
cases: list[RegistryCase] = []
seen_ids: set[str] = set()
@@ -344,7 +349,19 @@ def add(case_id: str, **kwargs: t.Any) -> None:
continue
if tx.chain_id not in CHAIN_IDS:
continue
- add(case_id, tx=tx, skip_reason=_skip_reason(tx, source), **case)
+ skip_reason = _skip_reason(tx)
+ has_display_format = (
+ skip_reason is None
+ and source.get_eth_display_format(tx.chain_id, tx.to, tx.data[:4])
+ is not None
+ )
+ add(
+ case_id,
+ tx=tx,
+ skip_reason=skip_reason,
+ has_display_format=has_display_format,
+ **case,
+ )
return cases
### tests/clear_signing_tests/summary.md.mako
@@ -33,12 +33,12 @@ Definitions: \
% endif
· Registry: [${short(inputs.get("registry_sha"))}](https://github.com/${inputs.get("registry_repo")}/tree/${inputs.get("registry_sha")})
-${total} mainnet cases: ${run} run on each model, ${total - run} skipped.
+${total} mainnet cases of all registry providers: ${run} run on each model, ${total - run} skipped.
-| Model | Clear-signed | Blind-signed | Error | Not run | Text not found | UI report | Compared with |
-|---|---:|---:|---:|---:|---:|---|---|
+| Model | Clear-signed | Blind-signed | Blind-signed, no format | Error | Not run | Text not found | UI report | Compared with |
+|---|---:|---:|---:|---:|---:|---:|---|---|
% for row in rows:
-| ${row.model} | ${row.counts["clear_signed"]} | ${row.counts["blind"]} | ${row.counts["error"]} | ${row.counts["not run"]} | ${row.counts["text not found"]} | [report](${row.report_url}) · [differing screens](${row.differing_url}) | ${baseline_link(row.baseline)} |
+| ${row.model} | ${row.counts["clear_signed"]} | ${row.counts["blind"]} | ${row.counts["blind, no format"]} | ${row.counts["error"]} | ${row.counts["not run"]} | ${row.counts["text not found"]} | [report](${row.report_url}) · [differing screens](${row.differing_url}) | ${baseline_link(row.baseline)} |
% endfor
The UI report compares the screens of each model with those of the last nightly run (`fixtures.json` when there is none). There is nothing to accept: the screens of this run are what the next one compares with.
@@ -58,6 +58,13 @@ The UI report compares the screens of each model with those of the last nightly
${capped([f"- `{test_id}`: {'; '.join(lines)}" for test_id, lines in problems])}
</details>
+<details><summary>Blind-signed, no display format (${len(no_format)})</summary>
+
+The dev definitions have no display format for the contract and function: trezor/definitions does not convert this provider, the converter dropped the format (an unsupported feature, an override by another descriptor), or the descriptor does not cover this contract on mainnet. Not a failure.
+
+${capped([f"- `{test_id}` ({', '.join(models)})" for test_id, models in no_format])}
+</details>
+
<details><summary>Blind-signed (${len(blind)})</summary>
The firmware fell back to blind signing although the dev definitions have a display format. Not a failure: the UI report shows when a case changes between clear and blind signing.
### tests/clear_signing_tests/test_registry.py
@@ -89,6 +89,7 @@ def test_registry_case(session: Session, case: RegistryCase) -> None:
"description": case.description,
"to": tx.to,
"selector": tx.data[:4].hex(),
+ "has_display_format": case.has_display_format,
}
recorder = FlowRecorder()
definitions = messages.EthereumDefinitions(Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.