`NetworkGraph`: Determine pre-allocations using actual numbers when reading
What the developer wrote
Authored by Elias Rohrer
`NetworkGraph`: Determine pre-allocations using actual numbers when reading
When reading a persisted network graph, we previously pre-allocated our
default node/channels estimate count for the respective `IndexedMap`
capacities. However, this might unnecessarily allocate memory on
reading, for example if we have an (almost) empty network graph for one
reason or another. As we have the actual counts of persisted nodes and
channels available, we here simply opt to allocate these numbers (plus
15%). This will also ensure that our pre-allocations will keep
up-to-date over time as the network grows or shrinks.
What changed, and why it matters
This commit changes how a saved Lightning network graph is loaded back into memory. Instead of always reserving a large fixed amount of memory, it now reserves memory based on the actual number of nodes and channels being loaded, plus a small buffer. This is a performance and resource-usage improvement, not a security fix.
No security action required. Treat as a routine performance/refactoring change.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The deserialization code for NetworkGraph now pre-allocates IndexedMap capacity from the persisted channel/node counts (capped at 100M channels / 10M nodes and scaled by 115%) rather than using hard-coded estimates (CHAN_COUNT_ESTIMATE, NODE_COUNT_ESTIMATE). It also rejects values that fail u128 -> usize conversion with DecodeError::InvalidValue. The change reduces memory pressure for small graphs and avoids repeated reallocations for large graphs.
Changed components
lightning/src/routing/gossip.rsNetworkGraph deserialization/read pathInspect captured patch +15 / −2
diff --git a/lightning/src/routing/gossip.rs b/lightning/src/routing/gossip.rs
index ea72d97..42eab6d 100644
--- a/lightning/src/routing/gossip.rs
+++ b/lightning/src/routing/gossip.rs
@@ -1682,9 +1682,17 @@ where
fn read<R: io::Read>(reader: &mut R, logger: L) -> Result<NetworkGraph<L>, DecodeError> {
let _ver = read_ver_prefix!(reader, SERIALIZATION_VERSION);
+ const MAX_CHAN_COUNT_LIMIT: usize = 100_000_000;
+ const MAX_NODE_COUNT_LIMIT: usize = 10_000_000;
+
let chain_hash: ChainHash = Readable::read(reader)?;
let channels_count: u64 = Readable::read(reader)?;
- let mut channels = IndexedMap::with_capacity(CHAN_COUNT_ESTIMATE);
+ // Pre-allocate 115% of the known channel count to avoid unnecessary reallocations.
+ let channels_map_capacity = (channels_count as u128 * 115 / 100)
+ .try_into()
+ .map(|v: usize| v.min(MAX_CHAN_COUNT_LIMIT))
+ .map_err(|_| DecodeError::InvalidValue)?;
+ let mut channels = IndexedMap::with_capacity(channels_map_capacity);
for _ in 0..channels_count {
let chan_id: u64 = Readable::read(reader)?;
let chan_info: ChannelInfo = Readable::read(reader)?;
@@ -1696,7 +1704,12 @@ where
if nodes_count > u32::max_value() as u64 / 2 {
return Err(DecodeError::InvalidValue);
}
- let mut nodes = IndexedMap::with_capacity(NODE_COUNT_ESTIMATE);
+ // Pre-allocate 115% of the known channel count to avoid unnecessary reallocations.
+ let nodes_map_capacity: usize = (nodes_count as u128 * 115 / 100)
+ .try_into()
+ .map(|v: usize| v.min(MAX_NODE_COUNT_LIMIT))
+ .map_err(|_| DecodeError::InvalidValue)?;
+ let mut nodes = IndexedMap::with_capacity(nodes_map_capacity);
for i in 0..nodes_count {
let node_id = Readable::read(reader)?;
let mut node_info: NodeInfo = Readable::read(reader)?;
Why this scored 19/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.