rust-lightning

Speed up `remove_stale_channels_and_tracking` nontrivially

During startup, the lightning protocol forces us to fetch a ton of
gossip for channels where there is a `channel_update` in only one
direction. We then have to wait around a while until we can prune
the crap cause we don't know when the gossip sync has completed.

Sadly, doing a large prune via `remove_stale_channels_and_tracking`
is somewhat slow. Removing a large portion of our graph currently
takes a bit more than 7.5 seconds on an i9-14900K, which can
ultimately ~hang a node with a few less GHz ~forever.

The bulk of this time is in our `IndexedMap` removals, where we
walk the entire `keys` `Vec` to remove the entry, then shift it
down after removing.

In the previous commit we shifted to a bulk removal model for
channels, here we do the same for nodes. This reduces the same test
to around 340 milliseconds on the same hardware.

Speed up `remove_stale_channels_and_tracking` nontrivially

During startup, the lightning protocol forces us to fetch a ton of
gossip for channels where there is a `channel_update` in only one
direction. We then have to wait around a while until we can prune
the crap cause we don't know when the gossip sync has completed.

Sadly, doing a large prune via `remove_stale_channels_and_tracking`
is somewhat slow. Removing a large portion of our graph currently
takes a bit more than 7.5 seconds on an i9-14900K, which can
ultimately ~hang a node with a few less GHz ~forever.

The bulk of this time is in our `IndexedMap` removals, where we
walk the entire `keys` `Vec` to remove the entry, then shift it
down after removing.

Here we shift to a bulk removal model when removing channels, doing
a single `Vec` iterate + shift. This reduces the same test to
around 1.38 seconds on the same hardware.

Don't auto-fail offers payments pre-HTLC lock in

Previously, we had a bug that particularly affected async payments where if an
outbound payment was in the state {Static}InvoiceReceived and there was a call
to process_pending_htlc_forwards, the payment would be automatically abandoned.
We would behave correctly and avoid abandoning if the payment was awaiting an
invoice, but not if the payment had an invoice but the HTLCs weren't yet locked
in.

ln: remove test flag from `negotiate_anchor_zero_fee_commitments`

Drop `RevokedOutput::is_counterparty_balance_on_anchors`

When we added anchors support we declined to aggregate claims
across different types of outputs for anchor channels, requiring
that we track whether a `RevokedOutput` claim came from an anchor
channel or not. This was only ever used to decide whether the claim
was aggregable or not.

In 0fe90c6f7c3325935b18dbc809be14afd8fe067f we started aggregating
by determining whether a claim is pinnable or not, rather than the
much-too-coarse aggregable flag, leaving
`is_counterparty_balance_on_anchors` unused.

Here we remove it entirely (as it would become ambiguous with
zero-commitment-fee anchor channels).

We always write `Some(())`, marking the claim as an anchor claim
and resulting in LDK prior to 0.1 refusing to aggregate the claim
(which is the more conservative stance). This breaks downgrade to
LDK 0.0.115 and earlier.

Mention the `bitcoin-payment-instructions` crate in docs

Move reasonable-default arguments out of `pay_for_offer`.

`ChannelManager::pay_for_offer` and
`ChannelManager::pay_for_offer_from_human_readable_name` have
accumulated a handful of arguments, most of which we expect users
to never actually set.

Instead, here, we move `quantity` and `payer_note` (which we expect
users to never set) as well as `route_params_config` and
`retry_strategy` (which we expect users to generally stick with the
defaults on) into a new struct, which implements `Default`.

This cleans up a good bit of cruft on payment calls.

Allow setting an HRN in invoice_requests built by `pay_for_offer`

If a user did their own BIP 353 lookup to fetch an offer from a
human readable name, we still want them to be able to use
`ChannelManager::pay_for_offer`.

Because BIP 353 offer payments require that the `invoice_request`
include the human readable name, we need to add an argument to set
the `invoice_request` HRN to `pay_for_offer`, which we do here.

Drop spurious limitations section on `pay_for_offer`

For some time we've automatically opened a connection to the
blinded path introduction point when we need to send a message we
generated. However, the "Limitations" section in
`ChannelManager::pay_for_offer` lists having a direct connection
as required for the payment to succeed, which is not true.

Instead, we simply remove the section from both `pay_for_offer` and
`pay_for_offer_from_human_readable_name`.

Add a `OfferFromHrn` type to wrap `Offer`s which came from HRNs

In a coming commit we'll provide users a different `ChannelManager`
method for sending to `Offer`s which were fetched by resolving an
HRN. Here, we first ad a wrapper type which can be used by HRN
resolution methods to nudge users towards calling the correct
payment method on `ChannelManager`.

Somewhat DRY `pay_for_offer_intern` callback method

Miscellanous CI fixes

Add basic end-to-end splice tests

This adds a new test for both splice-in and splice-out in favor of
maintaining the existing test. Helpers have been added to DRY up a lot
of the logic necessary for driving the splice state machine forward.

Set current block locktime for coinbase in provide_anchor_reserves

This guarantees we get a unique txid when calling
`provide_anchor_reserves` successively as we're immediately mining (and
therefore advancing the chain) the transaction after creating it.

Switch TestWalletSource to use P2WPKH script

We plan to reuse it for dual-funding/splicing, and those require
standard SegWit inputs only.

Implement `KVStore` for `TestStore`

We implement the async `KVStore` trait for `TestStore`. This is mostly
useful in LDK Node tests, where we use `TestStore` and we now require
all stores to implement both variants.

Move `TestStore` logic to `_internal` methods

.. to be easier reusable by different trait implementations.

Remove redundant `startup_replay` arg to `claim_funds_internal`

`startup_replay` should always exactly mirror
`background_events_processed_since_startup`, so we should just use
that rather than having a dedicated argument for it.

Stop re-hydrating pending payments once they are fully resolved

`MonitorEvent`s aren't delivered to the `ChannelManager` in a
durable fasion - if the `ChannelManager` fetches the pending
`MonitorEvent`s, then the `ChannelMonitor` gets persisted (i.e. due
to a block update) then the node crashes, prior to persisting the
`ChannelManager` again, the `MonitorEvent` and its effects on the
`ChannelManger` will be lost. This isn't likely in a sync persist
environment, but in an async one this could be an issue.

Note that this is only an issue for closed channels -
`MonitorEvent`s only inform the `ChannelManager` that a channel is
closed (which the `ChannelManager` will learn on startup or when it
next tries to advance the channel state), that
`ChannelMonitorUpdate` writes completed (which the `ChannelManager`
will detect on startup), or that HTLCs resolved on-chain post
closure. Of the three, only the last is problematic to lose prior
to a reload.

In previous commits we ensured that HTLC resolutions which came to
`ChannelManager` via a `MonitorEvent` were replayed on startup if
the `MonitorEvent` was lost. However, in cases where the
`ChannelManager` was so stale that it didn't have the payment state
for an HTLC at all, we only re-add it in cases where
`ChannelMonitor::get_pending_or_resolved_outbound_htlcs` includes
it.

Because constantly re-adding a payment state and then failing it
would generate lots of noise for users on startup (not to mention
risk of confusing stale payment events for the latest state of a
payment when the `PaymentId` has been reused to retry a payment).
Thus, `get_pending_or_resolved_outbound_htlcs` does not include
state for HTLCs which were resolved on chain with a preimage or
HTLCs which were resolved on chain with a timeout after
`ANTI_REORG_DELAY` confirmations.

This critera matches the critera for generating a `MonitorEvent`,
and works great under the assumption that `MonitorEvent`s are
reliably delivered. However, if they are not, and our
`ChannelManager` is lost or substantially old (or, in a future
where we do not persist `ChannelManager` at all), we will not end
up seeing payment resolution events for an HTLC.

Instead, we really want to tell our `ChannelMonitor`s when the
resolution of an HTLC is complete. Note that we don't particularly
care about non-payment HTLCs, as there is no re-hydration of state
to do there - `ChannelManager` load ignores forwarded HTLCs coming
back from `get_pending_or_resolved_outbound_htlcs` as there's
nothing to do - we always attempt to replay the success/failure and
figure out if it mattered based on whether there was still an HTLC
to claim/fail.

Here we, finally, begin actually using the new
`ChannelMonitorUpdateStep::ReleasePaymentComplete` updates,
skipping re-hydration of pending payments once they have been fully
resolved through to a user `Event`.

Properly provide `PaymentPathSuccessful` event for replay claims

When a payment was sent and ultimately completed through an
on-chain HTLC claim which we discover during startup, we
deliberately break the payment tracking logic to keep it around
forever, declining to send a `PaymentPathSuccessful` event but
ensuring that we don't constantly replay the claim on every
startup.

However, now that we now have logic to complete a claim by marking
it as completed in a `ChannelMonitor` and not replaying information
about the claim on every startup. Thus, we no longer need to take
the conservative stance and can correctly replay claims now,
generating `PaymentPathSuccessful` events and allowing the state to
be removed.

Generate new `ReleasePaymentComplete` monitor updates

`MonitorEvent`s aren't delivered to the `ChannelManager` in a
durable fasion - if the `ChannelManager` fetches the pending
`MonitorEvent`s, then the `ChannelMonitor` gets persisted (i.e. due
to a block update) then the node crashes, prior to persisting the
`ChannelManager` again, the `MonitorEvent` and its effects on the
`ChannelManger` will be lost. This isn't likely in a sync persist
environment, but in an async one this could be an issue.

Note that this is only an issue for closed channels -
`MonitorEvent`s only inform the `ChannelManager` that a channel is
closed (which the `ChannelManager` will learn on startup or when it
next tries to advance the channel state), that
`ChannelMonitorUpdate` writes completed (which the `ChannelManager`
will detect on startup), or that HTLCs resolved on-chain post
closure. Of the three, only the last is problematic to lose prior
to a reload.

In previous commits we ensured that HTLC resolutions which came to
`ChannelManager` via a `MonitorEvent` were replayed on startup if
the `MonitorEvent` was lost. However, in cases where the
`ChannelManager` was so stale that it didn't have the payment state
for an HTLC at all, we only re-add it in cases where
`ChannelMonitor::get_pending_or_resolved_outbound_htlcs` includes
it.

Because constantly re-adding a payment state and then failing it
would generate lots of noise for users on startup (not to mention
risk of confusing stale payment events for the latest state of a
payment when the `PaymentId` has been reused to retry a payment).
Thus, `get_pending_or_resolved_outbound_htlcs` does not include
state for HTLCs which were resolved on chain with a preimage or
HTLCs which were resolved on chain with a timeout after
`ANTI_REORG_DELAY` confirmations.

This critera matches the critera for generating a `MonitorEvent`,
and works great under the assumption that `MonitorEvent`s are
reliably delivered. However, if they are not, and our
`ChannelManager` is lost or substantially old (or, in a future
where we do not persist `ChannelManager` at all), we will not end
up seeing payment resolution events for an HTLC.

Instead, we really want to tell our `ChannelMonitor`s when the
resolution of an HTLC is complete. Note that we don't particularly
care about non-payment HTLCs, as there is no re-hydration of state
to do there - `ChannelManager` load ignores forwarded HTLCs coming
back from `get_pending_or_resolved_outbound_htlcs` as there's
nothing to do - we always attempt to replay the success/failure and
figure out if it mattered based on whether there was still an HTLC
to claim/fail.

Here we begin generating the new
`ChannelMonitorUpdateStep::ReleasePaymentComplete` updates,
updating functional tests for the new `ChannelMonitorUpdate`s where
required.

Address ChannelState inconsistency throughout splicing

Once a channel open has become locked (i.e., we've entered
`ChannelState::ChannelReady`), the channel is intended to remain within
this state for the rest of its lifetime until shutdown. Previously, we
had assumed a channel being spliced would go through the `ChannelState`
lifecycle again starting from `NegotiatingFunding` but skipping
`AwaitingChannelReady`. This inconsistency departs from what we strive
to achieve with `ChannelState` and also makes the state of a channel
harder to reason about.

This commit ensures a channel undergoing a splice remains in
`ChannelReady`, clearing the quiescent flag once the negotiation is
complete. Dual funding is unaffected by this change as the channel is
being opened and we want to maintain the same `ChannelState` lifecycle.

Remove tx_signatures flags for interactive signing ChannelState

This commit addresses an overlap of state between
`InteractiveTxSigningSession` and `ChannelState::FundingNegotiated`. The
signing session already tracks whether both holder and counterparty
`tx_signatures` have been produced, so tracking the state duplicatively
at the `ChannelState` level is unnecessary.

Move interactive signing session into ChannelContext

Since the `InteractiveTxSigningSession` already tracks everything we
need, we can remove the duplicate interactive signing state tracking
within `ChannelState`.

Remove incomplete dual funding tests

The tests have become a bit painful to maintain, and they don't even
fully test the production code paths, so we opt to just remove them for
now. In the future, we plan to unify the dual funding code paths with
splicing.