bookkeeper: only read listchannelmoves 1000 entries at a time.
What the developer wrote
Authored by Rusty Russell
bookkeeper: only read listchannelmoves 1000 entries at a time.
If we read all of them, we might get 1.6M at once (after initial
migration). Then we submit a few hundred thousand simultaneous
requests to lightningd, and it gets upset, queueing them all on the
xpay command hook and running out of memory.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Fixed: plugins: bookkeeper first invocation after migration from prior to 25.09 with very large databases will not crash.
What changed, and why it matters
This update fixes a bug in the bookkeeper plugin of Core Lightning. After a large database migration, the plugin could ask for too many records at once, flood the main lightningd process with hundreds of thousands of requests, and cause it to run out of memory and crash. The fix reads records in batches of 1,000 instead of all at once.
Apply the patch. Operators with large nodes that were running versions before 25.09 should upgrade before restarting bookkeeper after migration to avoid an out-of-memory crash. No immediate incident response is required unless a crash has already occurred.
Security signals we found
Denial-of-service via resource exhaustion (memory exhaustion from unbounded request fan-out)
Unbounded batch read replaced with paginated/limit-based read
Plugin-induced crash of the main daemon (lightningd)
Crash triggered by normal operational state (large migrated database), not malformed input
Evidence from the diff
The bookkeeper plugin previously called listchannelmoves without a limit, then issued a separate lightningd request for every returned move. After a migration from versions before 25.09, this could mean ~1.6 million moves and a corresponding burst of requests queued on the xpay command hook, exhausting memory. The patch introduces limited_listchannelmoves(), which requests moves in batches of 1,000 and recurses through listchannelmoves_done() until no more moves are returned. This bounds concurrency and memory use during the first post-migration refresh.
Changed components
plugins/bkpr/bookkeeper.cbookkeeper pluginlistchannelmoves RPC handlingxpay command hook (indirectly affected by request flood)Inspect captured patch +27 / −8
diff --git a/plugins/bkpr/bookkeeper.c b/plugins/bkpr/bookkeeper.c
index 1312924d..75fb76ad 100644
--- a/plugins/bkpr/bookkeeper.c
+++ b/plugins/bkpr/bookkeeper.c
@@ -53,6 +53,10 @@ static struct refresh_info *use_rinfo(struct refresh_info *rinfo)
return rinfo;
}
+/* Recursion */
+static struct command_result *limited_listchannelmoves(struct command *cmd,
+ struct refresh_info *rinfo);
+
static struct command_result *rinfo_one_done(struct command *cmd,
struct refresh_info *rinfo)
{
@@ -123,16 +127,37 @@ static struct command_result *listchannelmoves_done(struct command *cmd,
&be_index, sizeof(be_index),
"create-or-replace",
datastore_done, NULL, use_rinfo(rinfo));
+
+ /* If there might be more, try asking for more */
+ if (moves->size != 0)
+ limited_listchannelmoves(cmd, rinfo);
+
return rinfo_one_done(cmd, rinfo);
}
+/* We do 1000 at a time to avoid overwhelming lightningd */
+static struct command_result *limited_listchannelmoves(struct command *cmd,
+ struct refresh_info *rinfo)
+{
+ struct bkpr *bkpr = bkpr_of(cmd->plugin);
+ struct out_req *req;
+
+ req = jsonrpc_request_start(cmd, "listchannelmoves",
+ listchannelmoves_done,
+ plugin_broken_cb,
+ use_rinfo(rinfo));
+ json_add_string(req->js, "index", "created");
+ json_add_u64(req->js, "start", bkpr->channelmoves_index + 1);
+ json_add_u64(req->js, "limit", 1000);
+ return send_outreq(req);
+}
+
static struct command_result *listchainmoves_done(struct command *cmd,
const char *method,
const char *buf,
const jsmntok_t *result,
struct refresh_info *rinfo)
{
- struct out_req *req;
const jsmntok_t *moves, *t;
size_t i;
struct bkpr *bkpr = bkpr_of(cmd->plugin);
@@ -148,13 +173,7 @@ static struct command_result *listchainmoves_done(struct command *cmd,
"create-or-replace",
datastore_done, NULL, use_rinfo(rinfo));
- req = jsonrpc_request_start(cmd, "listchannelmoves",
- listchannelmoves_done,
- plugin_broken_cb,
- use_rinfo(rinfo));
- json_add_string(req->js, "index", "created");
- json_add_u64(req->js, "start", bkpr->channelmoves_index + 1);
- send_outreq(req);
+ limited_listchannelmoves(cmd, rinfo);
return rinfo_one_done(cmd, rinfo);
}
Why this scored 53/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.