gossipd: compact when gossip store is 80% deleted records.
What the developer wrote
Authored by Rusty Russell
gossipd: compact when gossip store is 80% deleted records.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Changelog-Added: `gossipd` now uses a `lightning_gossip_compactd` helper to compact the gossip_store on demand, keeping it under about 210MB.
What changed, and why it matters
This change makes the Lightning node's gossip database automatically shrink itself when it becomes bloated with old, deleted records. It also fixes a small bug where the wrong file descriptor was being closed during that cleanup process. There is no direct evidence this is a security fix, but uncontrolled file growth and incorrect descriptor handling can contribute to reliability or minor resource problems.
Treat as a routine maintenance/reliability improvement. Review whether the descriptor bug could leak a pipe fd or cause helper misbehavior under error paths, but no immediate security response is indicated.
Security signals we found
Resource-consumption mitigation: prevents unbounded growth of gossip_store
Descriptor-handling bug fix: closing wrong pipe fd in child process
No vendor security framing: commit message and changelog describe performance/maintenance, not security
Evidence from the diff
The commit adds an on-demand compaction trigger for the gossip_store. A new helper gossmap_manage_maybe_compact() is called after each handled gossip message. It compacts when the store exceeds 10 MB and dead records outnumber live ones by 4:1 (i.e., ~80% deleted). The previously UNNEEDED static gossmap_compact() is now used. A bug fix changes close(childin[2]) to close(childout[0]) in the child process setup, which appears to correct a wrong-fd-close after dup2. No security relevance is stated by the vendor.
Changed components
gossipd/gossipd.cgossipd/gossmap_manage.cgossipd/gossmap_manage.hlightning_gossip_compactd helperInspect captured patch +35 / −2
diff --git a/gossipd/gossipd.c b/gossipd/gossipd.c
index db0f7716..5cb2541c 100644
--- a/gossipd/gossipd.c
+++ b/gossipd/gossipd.c
@@ -282,6 +282,8 @@ handled_msg_errmsg:
handled_msg:
if (err)
queue_peer_msg(daemon, &source, take(err));
+ /* We need to keep gossmap to reasonable size */
+ gossmap_manage_maybe_compact(daemon->gm);
}
/*~ connectd's input handler is very simple. */
diff --git a/gossipd/gossmap_manage.c b/gossipd/gossmap_manage.c
index 9f9571a1..61ce44d8 100644
--- a/gossipd/gossmap_manage.c
+++ b/gossipd/gossmap_manage.c
@@ -1662,7 +1662,7 @@ static struct io_plan *init_compactd_conn_in(struct io_conn *conn,
compactd_read_done, gm);
}
/* Returns false if already running */
-static UNNEEDED bool gossmap_compact(struct gossmap_manage *gm)
+static bool gossmap_compact(struct gossmap_manage *gm)
{
int childin[2], execfail[2], childout[2];
int saved_errno;
@@ -1717,7 +1717,7 @@ static UNNEEDED bool gossmap_compact(struct gossmap_manage *gm)
close(childin[1]);
if (dup2(childout[0], STDIN_FILENO) == -1)
err(1, "Failed to duplicate fd to stdin");
- close(childin[2]);
+ close(childout[0]);
closefrom_limit(0);
closefrom(3);
/* Tell compactd helper what we read so far. */
@@ -1755,3 +1755,28 @@ static UNNEEDED bool gossmap_compact(struct gossmap_manage *gm)
io_set_finish(gm->compactd->in_conn, compactd_done, gm);
return true;
}
+
+void gossmap_manage_maybe_compact(struct gossmap_manage *gm)
+{
+ u64 num_live, num_dead;
+ struct gossmap *gossmap = gossmap_manage_get_gossmap(gm);
+ bool compact_started;
+
+ gossmap_stats(gossmap, &num_live, &num_dead);
+
+ /* Don't get out of bed for less that 10MB */
+ if (gossip_store_len_written(gm->gs) < 10000000)
+ return;
+
+ /* Compact when the density would be 5x better */
+ if (num_dead < 4 * num_live)
+ return;
+
+ compact_started = gossmap_compact(gm);
+ status_debug("%s gossmap compaction:"
+ " %"PRIu64" with"
+ " %"PRIu64" live records and %"PRIu64" dead records",
+ compact_started ? "Beginning" : "Already running",
+ gossip_store_len_written(gm->gs),
+ num_live, num_dead);
+}
diff --git a/gossipd/gossmap_manage.h b/gossipd/gossmap_manage.h
index 74781d55..c1357227 100644
--- a/gossipd/gossmap_manage.h
+++ b/gossipd/gossmap_manage.h
@@ -112,6 +112,12 @@ void gossmap_manage_tell_lightningd_locals(struct daemon *daemon,
*/
bool gossmap_manage_populated(const struct gossmap_manage *gm);
+/**
+ * gossmap_manage_maybe_compact: rewrite gossmap if it's getting giant.
+ * @gm: the gossmap_manage context
+ */
+void gossmap_manage_maybe_compact(struct gossmap_manage *gm);
+
/* For memleak to see inside of maps */
void gossmap_manage_memleak(struct htable *memtable, const struct gossmap_manage *gm);
#endif /* LIGHTNING_GOSSIPD_GOSSMAP_MANAGE_H */
Why this scored 19/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.