rust-lightning

Tests: take netgraph locks after peer_state

Previously we were taking the NetworkGraph::{channels,nodes} locks before
ChannelManager::per_peer_state's peer_state locks in some tests, which violated
a lock order requirement we have in ChannelManager to take netgraph locks
*after* peer_state locks. See OffersMessageFlow::path_for_release_htlc which is
called while a peer_state lock is held and takes netgraph locks while creating
blinded paths.

Conditionally advertise htlc_hold feature

Now that we support the feature of sending payments as an often-offline sender
to an often-offline recipient, including the sender's LSP-side, we can start
conditionally advertising the feature bit to other nodes on the network.

Also remove an outdated comment, in a previous version of the code we had more
logic to check if we're an always-online node before advertising the feature
bit but it's since been removed.

Support held_htlc_available counterparty reply path

As part of supporting sending payments as an often-offline sender, the sender
needs to send held_htlc_available onion messages such that the reply path to
the message terminates at their always-online channel counterparty that is
holding the HTLC. That way when the recipient responds with release_held_htlc,
the sender's counterparty will receive that message.

Here we lay some groundwork for using a counterparty-created reply path when
sending held_htlc_available as an async sender in the next commit.

Set UpdateAddHTLC::hold_htlc for offline payees

As part of supporting sending payments as an often-offline sender, the sender
needs to be able to set a flag in their update_add_htlc message indicating that
the HTLC should be held until receipt of a release_held_htlc onion message from
the often-offline payment recipient.

The prior commits laid groundwork to finally set the flag here in this commit.

See-also BOLTs PR 989

Send held_htlc_available with counterparty reply path

As part of supporting sending payments as an often-offline sender, the sender
needs to send held_htlc_available onion messages such that the reply path to
the message terminates at their always-online channel counterparty that is
holding the HTLC. That way when the recipient responds with release_held_htlc,
the sender's counterparty will receive that message.

After laying groundwork over some past commits, here we as an async sender send
held_htlc_available messages using reply paths created by our always-online
channel counterparty.

Test async send

Add testing for sending payments from an often-offline sender to an
often-offline recipient.

Add util to gather channels that support htlc_hold

Returns a list of channels where our counterparty supports
InitFeatures::supports_htlc_hold, or an error if there are none. Useful for
sending async payments to StaticInvoices.

Add config flag for held outbound HTLCs

If the flag is set to true, then if we as an often-offline payer receive a
static invoice to pay, we will attempt to hold the corresponding outbound HTLCs
with our next-hop channel counterparty(s) that support the feature. This
allows our node to go offline once the HTLCs are locked in even though the
recipient may not yet be online to receive them.

Immediately archive `ChannelMonitor`s for inbound unfuned channels

If a peer opens a channel to us, but never actually broadcasts the
funding transaction, we'll still keep a `ChannelMonitor` around for
the channel. While we maybe shouldn't do this either, when the
channel ultimately times out 2016 blocks later, we should at least
immediately archive the `ChannelMonitor`, which we do here.

Fixes #3384

Extract send_to_static_invoice util

Makes an upcoming commit cleaner.

In the util we allow manually passing in the first_hops that should be used
when paying a static invoice, which is useful for only passing in next-hop
channels that support the hold_htlcs feature.

Add hold_htlcs param to pay_route_internal

As part of supporting sending payments as an often-offline sender, the sender
needs to be able to set a flag in their update_add_htlc message indicating that
the HTLC should be held until receipt of a release_held_htlc onion message from
the often-offline payment recipient.

We don't yet ever set this flag, but lay the groundwork by including the
parameter in the pay_route method.

See-also <https://github.com/lightning/bolts/pull/989>

Add hold_htlc param to internal Channel APIs

As part of supporting sending payments as an often-offline sender, the sender
needs to be able to set a flag in their update_add_htlc message indicating that
the HTLC should be held until receipt of a release_held_htlc onion message from
the often-offline payment recipient.

We don't yet ever set this flag, but lay the groundwork by including a
parameter for it in internal channel send_htlc APIs, including serializing the
field when the parameter is set.

See-also BOLTs PR 989

Persist hold_htlc flags in Channel

As part of supporting sending payments as an often-offline sender, the sender
needs to be able to set a flag in their update_add_htlc message indicating that
the HTLC should be held until receipt of a release_held_htlc onion message from
the often-offline payment recipient.

We don't yet ever set this flag, but lay the groundwork by including the field
in Channel serialization.

See-also BOLTs PR 989

Add init_features to list_channels filter callback

Useful to filter for channel peers that support a specific feature, in this
case the hold_htlc feature, in upcoming commits.

Actually remove forward<>intercepted htlcs lock dep

8513341b1f7c39da3e6ad4521d4161362b0f4562 purported to remove a lock dependency
between ChannelManager::pending_intercepted_htlcs and ::forward_htlcs. However,
in practice the dependency remained because the locks of the two maps were
still held at the same time in some cases. Here we fix this.

Send `update_add_htlc` messages after HTLC removal messages

While nodes are generally supposed to validate commitment
transactions after the `commitent_signed` and not while HTLCs are
being added/removed, we don't. This can make a commitment update
where we use HTLC balance claimed with a fulfill to send new HTLCs,
which is perfectly valid, being rejected. While we shouldn't
currently generate any such commitments, we might want to in the
future, and on the off-chance that we do, or where such a
commitment would result in a dust threshold overrun, its always
safter to add new HTLCs to a commitment only after we've removed
any HTLCs we're going to remove, which we do here.

Deprecate `pay_for_offer_from_human_readable_name`

While its cool to have a super trivial API to initiate payment to
a `HumanReadableName`, most wallets ultimately support multiple
payment instruction formats (eg on-chain) or multiple HRN
resolution schemes (eg LN-Address). Thus, we generally don't expect
this API to get a lot of use.

Meanwhile, we've now ended up with both a
`ChannelManager::pay_for_offer_from_hrn` and
`ChannelManager::pay_for_ofer_from_human_readable_name`, which is
incredibly confusing.

Here, we deprecate `pay_for_offer_from_human_readable_name`,
pushing folks to the `bitcoin-payment-instructions` crate instead.
Because it has outbound payment states associated with it it may be
some time before we can fully remove all the pieces of it, sadly.

Add struct-level logging to OutboundPayments

Previously, OutboundPayments methods required borrowed logger
arguments since Logger lacked a Clone bound. This prevented
logging in all desired places, limiting trace statements for
debugging. This change initializes OutboundPayments with a
cloned Logger, simplifying method signatures and improving
logging flexibility.

Speed up `remove_stale_channels_and_tracking` nontrivially

During startup, the lightning protocol forces us to fetch a ton of
gossip for channels where there is a `channel_update` in only one
direction. We then have to wait around a while until we can prune
the crap cause we don't know when the gossip sync has completed.

Sadly, doing a large prune via `remove_stale_channels_and_tracking`
is somewhat slow. Removing a large portion of our graph currently
takes a bit more than 7.5 seconds on an i9-14900K, which can
ultimately ~hang a node with a few less GHz ~forever.

The bulk of this time is in our `IndexedMap` removals, where we
walk the entire `keys` `Vec` to remove the entry, then shift it
down after removing.

In the previous commit we shifted to a bulk removal model for
channels, here we do the same for nodes. This reduces the same test
to around 340 milliseconds on the same hardware.

Speed up `remove_stale_channels_and_tracking` nontrivially

During startup, the lightning protocol forces us to fetch a ton of
gossip for channels where there is a `channel_update` in only one
direction. We then have to wait around a while until we can prune
the crap cause we don't know when the gossip sync has completed.

Sadly, doing a large prune via `remove_stale_channels_and_tracking`
is somewhat slow. Removing a large portion of our graph currently
takes a bit more than 7.5 seconds on an i9-14900K, which can
ultimately ~hang a node with a few less GHz ~forever.

The bulk of this time is in our `IndexedMap` removals, where we
walk the entire `keys` `Vec` to remove the entry, then shift it
down after removing.

Here we shift to a bulk removal model when removing channels, doing
a single `Vec` iterate + shift. This reduces the same test to
around 1.38 seconds on the same hardware.

Don't auto-fail offers payments pre-HTLC lock in

Previously, we had a bug that particularly affected async payments where if an
outbound payment was in the state {Static}InvoiceReceived and there was a call
to process_pending_htlc_forwards, the payment would be automatically abandoned.
We would behave correctly and avoid abandoning if the payment was awaiting an
invoice, but not if the payment had an invoice but the HTLCs weren't yet locked
in.

ln: remove test flag from `negotiate_anchor_zero_fee_commitments`

Drop `RevokedOutput::is_counterparty_balance_on_anchors`

When we added anchors support we declined to aggregate claims
across different types of outputs for anchor channels, requiring
that we track whether a `RevokedOutput` claim came from an anchor
channel or not. This was only ever used to decide whether the claim
was aggregable or not.

In 0fe90c6f7c3325935b18dbc809be14afd8fe067f we started aggregating
by determining whether a claim is pinnable or not, rather than the
much-too-coarse aggregable flag, leaving
`is_counterparty_balance_on_anchors` unused.

Here we remove it entirely (as it would become ambiguous with
zero-commitment-fee anchor channels).

We always write `Some(())`, marking the claim as an anchor claim
and resulting in LDK prior to 0.1 refusing to aggregate the claim
(which is the more conservative stance). This breaks downgrade to
LDK 0.0.115 and earlier.

Move reasonable-default arguments out of `pay_for_offer`.

`ChannelManager::pay_for_offer` and
`ChannelManager::pay_for_offer_from_human_readable_name` have
accumulated a handful of arguments, most of which we expect users
to never actually set.

Instead, here, we move `quantity` and `payer_note` (which we expect
users to never set) as well as `route_params_config` and
`retry_strategy` (which we expect users to generally stick with the
defaults on) into a new struct, which implements `Default`.

This cleans up a good bit of cruft on payment calls.

Allow setting an HRN in invoice_requests built by `pay_for_offer`

If a user did their own BIP 353 lookup to fetch an offer from a
human readable name, we still want them to be able to use
`ChannelManager::pay_for_offer`.

Because BIP 353 offer payments require that the `invoice_request`
include the human readable name, we need to add an argument to set
the `invoice_request` HRN to `pay_for_offer`, which we do here.