Commit message · auto-pr-botStrictly limit in-flight gossip validations to 32
When the routing message handler's processing queue is high we so far
relied on pausing reads to limit the number of in-flight gossip
validations, but messages which have already been read are still
processed, leaving us handling up to a socket buffer worth of gossip
messages per peer beyond the limit. Thus the number of in-flight gossip
validations was not strictly bounded.
@matt asked:
> Now that we've merged
> https://git.rust-bitcoin.org/lightningdevkit/rust-lightning/pulls/5241
> we should more tightly limit the number of in-flight gossip validations
> in the same infrastructure - @auto-pr-bot when we're processing messages
> from a peer and the next message is a gossip message and we have
> `processing_queue_high` on the routing message handler, dont process the
> message and use the `stashed_read_data` logic. This should strictly
> limit our in-flight gossip validations to 32. To avoid allowing 1000
> malicious peers to starve an honest peer that mostly exchanges channel
> messages with us but occasionally sends us gossip we'll have to keep a
> `VecDeque` queue of gossip-blocked peers and make sure we process them
> in order.
Instead of processing a gossip message when the queue is high, we stash
the plaintext bytes of the message on the peer, stash any raw bytes which
followed it on the wire in the peer's `stashed_read_data`, and queue the
peer in a FIFO `VecDeque`. `process_events` handles stashed messages in
the order peers became blocked, and only once the processing queue has
drained, so one peer cannot starve others of gossip processing and no
gossip is lost. When a blocked peer's stash is handled, the stashed
message is re-decoded and dispatched by the same read loop as regular
messages, followed by any bytes which had been read before it became
blocked; anything it sends while blocked is not read until then, as its
reads stay paused while a message is stashed.
We cannot stash the ciphertext, as the decryption key stream has already
been rolled past it, and stashing the decoded message rather than its
plaintext bytes would make `PeerManager`'s automatically derived
`Send`/`Sync` require `CustomMessageReader::CustomMessage: Send`, a
public API change which would also complicate backporting.
@matt asked:
> Argh, no, the review bot is right - we should go back to the byte
> storage so that we don't change the public API and this can be
> trivially backported.
Because no messages are handled from a gossip-blocked peer, including
the reply to any ping we send it, it would otherwise be disconnected by
`timer_tick_occurred`'s liveness check a tick or two after being
blocked. Instead we pause liveness checking while a peer is
gossip-blocked, as its silence is our own backpressure rather than a
dead connection.
Note that raw bytes for the peer may be delivered by another thread
while its stash is being processed, so the stash points merge any
concurrently stashed bytes into the remaining read data, preserving
stream order.
Requested-by: @matt
Authored-By: matt/GLM-5.3-Flash-GGUF:Q4_K_XL <goose@users.noreply.local>
88/100 · StrongMessage clarity
✓ Descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context✓ Explains rationale or failure mode✓ Links an issue, advisory, or supporting reference