common: add brace hack for jsonrpc_async_parse.
What the developer wrote
Authored by Rusty Russell
common: add brace hack for jsonrpc_async_parse.
This is a trick from bcli: we ask bitcoind for the block, and it hands
us a 2MB hex blob (which we read in multiple parts). Our parser wades
through it all, but a quick search for '}' makes it much faster.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
What changed, and why it matters
This commit is a performance optimization, not a security fix. It speeds up how Core Lightning handles very large JSON responses from Bitcoin by first checking for a closing brace before running the full parser. The test change is just to match the new behavior.
No security action required; treat as routine performance improvement.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The change adds a fast-path check in jsonrpc_io_parse() that skips calling json_parse_input() if the incoming buffer does not yet contain a ‘}’ character. This avoids repeatedly parsing a large incomplete hex blob (e.g., from getrawblock) as more chunks arrive. The test is updated to include a ‘}’ so the parser still attempts parsing of malformed input.
Changed components
common/jsonrpc_io.ctests/test_misc.pyInspect captured patch +11 / −2
diff --git a/common/jsonrpc_io.c b/common/jsonrpc_io.c
index b127b677..179b3c1f 100644
--- a/common/jsonrpc_io.c
+++ b/common/jsonrpc_io.c
@@ -64,6 +64,15 @@ const char *jsonrpc_io_parse(const tal_t *ctx,
*toks = NULL;
*buf = NULL;
+ /* Our JSON parser is pretty good at incremental parsing, but
+ * `getrawblock` gives a giant 2MB token, which forces it to re-parse
+ * every time until we have all of it. However, we can't complete a
+ * JSON object without a '}', so we do a cheaper check here.
+ */
+ if (!memchr(membuf_elems(&json_in->membuf), '}',
+ membuf_num_elems(&json_in->membuf)))
+ return NULL;
+
if (!json_parse_input(&json_in->parser, &json_in->toks,
membuf_elems(&json_in->membuf),
membuf_num_elems(&json_in->membuf),
diff --git a/tests/test_misc.py b/tests/test_misc.py
index 9f61d51c..dc5e0a9f 100644
--- a/tests/test_misc.py
+++ b/tests/test_misc.py
@@ -951,8 +951,8 @@ def test_malformed_rpc(node_factory):
obj, _ = l1.rpc._readobj(sock, b'')
assert obj['error']['code'] == -32600
- # Complete crap (this makes it hang up!)
- sock.sendall(b'[]')
+ # Complete crap: needs } to even try parsing, and also this makes it hang up!
+ sock.sendall(b'[]}')
obj, _ = l1.rpc._readobj(sock, b'')
assert obj['error']['code'] == -32600
Why this scored 18/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.