discovery: update ApplyGossipFilter to use lazy iterator with Pull2
What the developer wrote
Authored by Olaoluwa Osuntokun
discovery: update ApplyGossipFilter to use lazy iterator with Pull2
In this commit, we update ApplyGossipFilter to leverage the new
iterator-based UpdatesInHorizon method. The key innovation here is using
iter.Pull2 to create a pull-based iterator that allows us to check if
any updates exist before launching the background goroutine.
This approach provides several benefits over the previous implementation.
First, we avoid the overhead of launching a goroutine when there are no
updates to send, which was previously unavoidable without materializing
the entire result set. Second, we maintain lazy loading throughout the
sending process, only pulling messages from the database as they're
needed for transmission.
The implementation uses Pull2 to peek at the first message, determining
whether to proceed with sending updates. If updates exist, ownership of
the iterator is transferred to the goroutine, which continues pulling
and sending messages until exhausted. This design ensures memory usage
remains bounded regardless of the number of updates being synchronized.
What changed, and why it matters
This commit refactors how LND's gossip syncer sends old channel updates to a peer. It switches from eagerly reading all updates into memory and launching a goroutine every time, to a lazy pull-iterator that only starts a goroutine if there is at least one update. The main benefit is lower memory and CPU usage, but the change also touches concurrency, error handling, and iterator ownership, which could hide subtle bugs.
Review the iterator lifecycle to confirm stop() is always called exactly once and that firstMsg is not used after being passed to the goroutine. Verify that iterator errors do not leak resources or leave the syncer in an inconsistent state. Consider adding tests for the zero-update path and for iterator-error handling. No immediate security patch is indicated, but treat as a concurrency-sensitive refactor.
Security signals we found
Concurrency/iterator ownership transfer to background goroutine
New error path: iterator errors are now logged and skipped rather than terminating the sync
Early return path now stops iterator and releases semaphore before goroutine launch
Deferred cleanup moved between caller and goroutine depending on whether updates exist
Use of Go 1.23 iter.Pull2 with (value, error, ok) semantics
Evidence from the diff
ApplyGossipFilter in discovery/syncer.go now wraps the UpdatesInHorizon result with iter.Pull2. It peeks the first (msg, err, ok) triplet before deciding whether to start the background goroutine. If no updates exist, it stops the iterator, releases the semaphore, and returns early. If updates exist, it transfers iterator ownership to the goroutine, sends the already-pulled first message, then loops over the remaining iterator values. Error handling is split: iterator errors are logged and iteration continues, while send errors for ErrGossipSyncerExiting/ErrPeerExiting return and other send errors are logged but iteration continues. The deferred stop() and returnSema() are now inside the goroutine instead of the caller path when no updates exist.
Changed components
discovery/syncer.goGossipSyncer.ApplyGossipFiltergossip backlog / peer catch-up synchronizationInspect captured patch +56 / −3
diff --git a/discovery/syncer.go b/discovery/syncer.go
index 478e7ac..b8c51a9 100644
--- a/discovery/syncer.go
+++ b/discovery/syncer.go
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
+ "iter"
"math"
"math/rand"
"sort"
@@ -1446,8 +1447,29 @@ func (g *GossipSyncer) ApplyGossipFilter(ctx context.Context,
g.cfg.chainHash, startTime, endTime,
)
+ // Create a pull-based iterator so we can check if there are any
+ // updates before launching the goroutine.
+ next, stop := iter.Pull2(newUpdatestoSend)
+
+ // Check if we have any updates to send by attempting to get the first
+ // message.
+ firstMsg, firstErr, ok := next()
+ if firstErr != nil {
+ stop()
+ returnSema()
+ return firstErr
+ }
+
log.Infof("GossipSyncer(%x): applying new remote update horizon: "+
- "start=%v, end=%v", g.cfg.peerPub[:], startTime, endTime)
+ "start=%v, end=%v, has_updates=%v", g.cfg.peerPub[:],
+ startTime, endTime, ok)
+
+ // If we don't have any to send, then we can return early.
+ if !ok {
+ stop()
+ returnSema()
+ return nil
+ }
// Set the atomic flag to indicate we're starting to send the backlog.
// If the swap fails, it means another goroutine is already active, so
@@ -1461,14 +1483,45 @@ func (g *GossipSyncer) ApplyGossipFilter(ctx context.Context,
}
// We'll conclude by launching a goroutine to send out any updates.
+ // The goroutine takes ownership of the iterator.
g.cg.WgAdd(1)
go func() {
defer g.cg.WgDone()
defer returnSema()
defer g.isSendingBacklog.Store(false)
+ defer stop()
+
+ // Send the first message we already pulled.
+ err := g.sendToPeerSync(ctx, firstMsg)
+ switch {
+ case errors.Is(err, ErrGossipSyncerExiting):
+ return
+
+ case errors.Is(err, lnpeer.ErrPeerExiting):
+ return
+
+ case err != nil:
+ log.Errorf("Unable to send message for "+
+ "peer catch up: %v", err)
+ }
+
+ // Continue with the rest of the messages using the same pull
+ // iterator.
+ for {
+ msg, err, ok := next()
+ if !ok {
+ return
+ }
+
+ // If the iterator yielded an error, log it and
+ // continue.
+ if err != nil {
+ log.Errorf("Error fetching update for peer "+
+ "catch up: %v", err)
+ continue
+ }
- for msg := range newUpdatestoSend {
- err := g.sendToPeerSync(ctx, msg)
+ err = g.sendToPeerSync(ctx, msg)
switch {
case err == ErrGossipSyncerExiting:
return
Why this scored 27/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.