What the developer wrote
Authored by Rusty Russell
common: optimize json parsing.
We would keep parsing if we were out of tokens, even if we had actually
finished one object!
These are comparison against the "xpay: use filtering on rpc_command
so we only get called on "pay"." not the disasterous previous one!
tests/test_coinmoves.py::test_generate_coinmoves (2,000,000, sqlite3):
Time (from start to end of l2 node): 126 seconds (was 135)
Worst latency: 5.1 seconds **WAS 12.1**
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
What changed, and why it matters
This is a performance optimization for JSON parsing in Core Lightning. The change fixes a bug where the parser would keep re-parsing unnecessarily when it ran out of tokens, even though it had already successfully parsed a complete JSON object. The commit message frames this as an optimization, not a security fix, and shows benchmark improvements (worst latency dropping from 12.1 seconds to 5.1 seconds).
No security action required. Treat as a normal performance optimization. If backporting, include it with other performance improvements rather than as a security fix.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The patch modifies common/json_parse_simple.c in the json_parse_input function. Previously, when jsmn_parse returned JSMN_ERROR_NOMEM, the code immediately resized the token array and jumped back to parse again, without first checking whether a complete root element had already been parsed. The new logic first checks if the first token is a complete element (type defined and end != -1). If a complete element exists, it proceeds to count tokens. Only if no complete element exists and ret == JSMN_ERROR_NOMEM does it resize and retry. This avoids redundant re-parsing loops once a valid JSON object is already in hand. The change is purely an optimization/correctness improvement; no security boundary is crossed.
Changed components
common/json_parse_simple.cjson_parse_input() functionInspect captured patch +8 / −8
diff --git a/common/json_parse_simple.c b/common/json_parse_simple.c
index f3b3da80..348be9ef 100644
--- a/common/json_parse_simple.c
+++ b/common/json_parse_simple.c
@@ -486,24 +486,24 @@ bool json_parse_input(jsmn_parser *parser,
again:
ret = jsmn_parse(parser, input, len, *toks, tal_count(*toks) - 1);
-
- switch (ret) {
- case JSMN_ERROR_INVAL:
+ if (ret == JSMN_ERROR_INVAL)
return false;
- case JSMN_ERROR_NOMEM:
- tal_resize(toks, tal_count(*toks) * 2);
- goto again;
- }
/* Check whether we read at least one full root element, i.e., root
* element has its end set. */
if ((*toks)[0].type == JSMN_UNDEFINED || (*toks)[0].end == -1) {
+ /* If it ran out of tokens, provide more. */
+ if (ret == JSMN_ERROR_NOMEM) {
+ tal_resize(toks, tal_count(*toks) * 2);
+ goto again;
+ }
+ /* Otherwise, must be incomplete */
*complete = false;
return true;
}
/* If we read a partial element at the end of the stream we'll get a
- * ret=JSMN_ERROR_PART, but due to the previous check we know we read at
+ * errro, but due to the previous check we know we read at
* least one full element, so count tokens that are part of this root
* element. */
ret = json_next(*toks) - *toks;
Why this scored 15/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.