rust-lightning

ln/test: add new test vector for v3 duplicate dust

Add support for `Testnet4`

We simply add support for `bitcoin::Network::Testnet4`, which was added
with release v0.32.4, which we hence deem the new minimal requirement
for `lightning` and `lightning-invoice`.

As other Lightning implementations seem to simply map `Testnet4` coins
to be `Testnet` (also reusing the `tb` prefix), we here follow their
lead and choose not to introduce a separate `Currecy` variant, even
though that could lead to checks failing if someone validated the
returned `fallback_addresses` for the given `Network`.

Inline error variable

Functionize the FUNDED cases in `convert_channel_err`

`convert_channel_err` is used extensively in `channelmanager.rs`
(often indirectly via `try_channel_entry`) and generates nontrivial
code (especially once you include `locked_close_channel`). Here we
take the `FUNDED` cases of it and move them to an internal function
reducing the generated code size.

On the same machine as described two commits ago, this further
reduces build times by about another second.

Functionize the UNFUNDED case in `convert_channel_err`

`convert_channel_err` is used extensively in `channelmanager.rs`
(often indirectly via `try_channel_entry`) and generates nontrivial
code (especially once you include `locked_close_channel`). Here we
take the `UNFUNDED` case of it and move them to an internal
function reducing the generated code size.

On the same machine as described three commits ago, this further
reduces build times from the previous commit by about another
second.

Inline `locked_close_channel` in the new `convert_*_channel_err`

Now that `convert_channel_err` is primarily in two functions,
`locked_close_channel` is just a dumb macro that has two forms,
each only called once. Instead, drop it and inline its contents
into `convert_*_channel_err`.

Move `ChannelError`-handling logic down in `channelmanager.rs`

In the next commit we'll functionize some of the
`ChannelError`-handling logic, requiring it have access to
`handle_new_monitor_update_locked_actions_handled_by_caller!` which
thus needs to be defined above it.

Format `convert_*_channel_err` code moved out of macro

Format the code moved in the previous commit

Move unlocked part of `handle_monitor_update_completion` into an fn

The `handle_monitor_update_completion` macro is called from a
number of places in `channelmanager.rs` and dumps quite a bit of
code into each callsite. Here we take the unlocked half of
`handle_monitor_update_completion` and move it into a function in
`ChannelManager`.

As a result, building the `lightning` crate tests after a
single-line `println` change in `channelmanager.rs` was reduced by
around a full second (out of ~28.5 originally) on my machine using
rustc 1.85.0. Memory usage of the `expand_crate` step went from
+554MB to +548MB in the same test.

Log structured peer id, channel id and payment hash

Since most log messages no longer include the channel ID in their text,
this commit updates the test logger to output the structured channel ID
instead. The ID is truncated for readability. Similarly peer id and
payment hash are logged.

Co-authored-by: Matt Corallo <git@bluematt.me>

Remove redundant channel ID logging

Previously, log messages often included the channel ID both in the
message text and in the structured `channel_id` field. This led to
redundant information and made it difficult to quickly identify which
log lines pertain to a given channel, as the ID could appear in
different positions or sometimes not at all.

This change removes the channel ID from the message in all cases where
it is already present in the structured field. A test run was used to
verify that the structured field always appeared in these log calls.
Some exceptions remain—for example, calls where the structured field
contains a temporary channel ID and the message contains the final ID
were left unchanged.

Align log levels

Makes sure that 4 and 5 character log levels line up.

Unify test logger output

Now that we have Display implemented on Record, the various
test logger implementations can make use of that.

Implement Display for Record

As a preparatory step for unifying test log output.

Remove `check_closed_event` macro

Replace calls to `check_closed_event` macro
to the identically-named function.

Cleanup: Remove redundant (hmac, nonce) from codebase

Now that we have introduced an alternate mechanism for authentication
in the codebase, we can safely remove the now redundant (hmac, nonce)
fields from the Payment ReceiveTlvs's while maintaining the security
of the onion messages.

Introduce ReceiveAuthKey-based verification in Blinded Payment Paths

Extends the work started in
[PR#3917](https://github.com/lightningdevkit/rust-lightning/pull/3917)
by adding ReceiveAuthKey-based verification for Blinded Payment Paths.

This reduces space previously taken by individual ReceiveTlvs and
aligns the verification logic with that used for Blinded Message Paths.

Pipe channel node information through to `forward_broadcast_msg`

Sadly, the lightning gossip protocol operates by always flooding
peers with all the latest gossip they receive. For nodes with many
peers, this can result in lots of duplicative gossip as they
receive every message from every peer. As a results, some lightning
implementations disable gossip with new peers after some threshold.
This should mostly work as these peers expect to receive the latest
gossip from their many other peers.

However, in some cases an LDK node may wish to open public
channels but only has a single connection to the bulk of the rest
of the network - with one such peer which requests that it not
receive any gossip. In that case, LDK would dutifully never send
any gossip to its only connection to the outside world. We would
still send gossip for channels with that peer as it would be sent
as unicast gossip, but if we then open another connection to
another peer which doesn't have any connection to the outside world
any information on that channel wouldn't propagate.

We've seen this setup on some LSPs, where they have a public node
and then an LSP which only connects through that public node, but
expects to open public channels to its LSP clients.

In the next commit we'll start forwarding such gossip to all peers
by ignoring peer gossip limitations about all of our own channels.
Here we do the final prep work, passing the two sides of each
channel back from `handle_channel_update` calls and into
`forward_broadcast_msg`.

Always forward gossip for all our channels to all our peers

Sadly, the lightning gossip protocol operates by always flooding
peers with all the latest gossip they receive. For nodes with many
peers, this can result in lots of duplicative gossip as they
receive every message from every peer. As a results, some lightning
implementations disable gossip with new peers after some threshold.
This should mostly work as these peers expect to receive the latest
gossip from their many other peers.

However, in some cases an LDK node may wish to open public
channels but only has a single connection to the bulk of the rest
of the network - with one such peer which requests that it not
receive any gossip. In that case, LDK would dutifully never send
any gossip to its only connection to the outside world. We would
still send gossip for channels with that peer as it would be sent
as unicast gossip, but if we then open another connection to
another peer which doesn't have any connection to the outside world
any information on that channel wouldn't propagate.

We've seen this setup on some LSPs, where they have a public node
and then an LSP which only connects through that public node, but
expects to open public channels to its LSP clients.

Here we finally forward all gossip that is about either side of any
of our channels to all our peers, irrespective of their requested
gossip filtering.

Include both `node_id`s in `BroadcastChannelUpdate`

Sadly, the lightning gossip protocol operates by always flooding
peers with all the latest gossip they receive. For nodes with many
peers, this can result in lots of duplicative gossip as they
receive every message from every peer. As a results, some lightning
implementations disable gossip with new peers after some threshold.
This should mostly work as these peers expect to receive the latest
gossip from their many other peers.

However, in some cases an LDK node may wish to open public
channels but only has a single connection to the bulk of the rest
of the network - with one such peer which requests that it not
receive any gossip. In that case, LDK would dutifully never send
any gossip to its only connection to the outside world. We would
still send gossip for channels with that peer as it would be sent
as unicast gossip, but if we then open another connection to
another peer which doesn't have any connection to the outside world
any information on that channel wouldn't propagate.

We've seen this setup on some LSPs, where they have a public node
and then an LSP which only connects through that public node, but
expects to open public channels to its LSP clients.

In a coming commit we'll start forwarding such gossip to all peers
by ignoring peer gossip limitations about all of our own channels.
While other gossip has the `node_id`s of both sides of the channel
explicitly in the message, `ChannelUpdate`s do not, so here we add
the `node_id`s to `BroadcastChannelUpdate` giving us the
information we need when we go to broadcast updates.

ln/test: add test vectors for v3 channels

ln: pass preimage into test_commitment_common

In v3 test vectors, we're using real preimages so won't be able to
use the workaround here.

ln/test: test_channel_common zero fee commitment anchor handling

ln/refactor: remove rustfmt skip from outbound_commitment_test