Eclair

Improve channel and payment events (#3246)

We improve the following events:

- `TransactionPublished` includes more details about mining fees and
an optional liquidity purchase
- all channel events include the latest `channel_type`
- `PaymentRelayed` exposes the `relayFee` earned

Add event for failed payment relay (#3244)

We add an event when a payment could not be relayed and indicates that
we may need to add liquidity towards the next node. It is really hard
to figure it out in the context of a single payment though, so this
event does not by itself mean that liquidity should be allocated. The
listeners should collect several events and regularly query the state
of existing channels with our peers, and network graph data for remote
nodes, to create good heuristics for allocating liquidity. Otherwise,
it would be trivial for malicious senders to game routing nodes into
allocating liquidity "for free" towards them, which could result in
financial loss.

Include the `node_id` of channel peers in payment events (#3243)

When scoring peers, we don't really care about individual channels but
rather about the peer itself, who is identified by its `node_id`. We
update payment events to include the `node_id` of our channel peer, to
make it easier to compute statistics about each of our peers.

We also rework the events in `PaymentEvents.scala` to be more consistent
now that we've added support for trampoline, splicing, liquidity ads and
on-the-fly funding, which are all the features we had planned that can
impact peer scoring.

Note that we don't yet update the schema of the `AuditDb`, which means
that some of the data isn't stored and is currently filled with a dummy
value. We will include *all* updates to the DB schema in a future PR and
will fix this.

Add duration information to payment events (#3241)

We now record start and end time for payment events, and for remote
failures, which lets us easily identify slower paths and faulty nodes.
We properly track the settlement of each individual payment now.

We also clean-up field names and harmonize those events. Note that we
don't update the DB schema at that point: this will be done in another
PR that batches all DB schema changes.

Add test vector for Bolt12 invalid bech32 padding (#3242)

See https://github.com/lightning/bolts/pull/1312 for more details.

More tests for accountability (#3240)

These tests are also added to lightning-kmp (https://github.com/ACINQ/lightning-kmp/pull/846) to check compatibility.

Use fallback feerates on testnets (#3233)

When using testnet3 or testnet4, Bitcoin Core may fail to estimate fees
because there isn't enough block data. We now use the configured default
feerates when that happens to make it easier to run on testnets.

We also update the default feerates to better match current values.

Fixes #3105

CI: fix test with latest bitcoind (#3239)

Add flag to build without multiprocess architecture which we don't use and requires and additional dependency (capnproto).

Rework channel lifecyle events (#3237)

We emit several events during the channel lifecycle, which have become
a bit of a mess over the years, especially with the addition of 0-conf,
splicing and on-the-fly funding.

We now use the following events:

- `ChannelCreated` once the funding transaction is created
- `ChannelFundingConfirmed` once the funding transaction is confirmed,
which is also emitted for splice transactions
- `ChannelReadyForPayments` once the channel is ready for payments,
after exchanging `channel_ready` for the channel creation or
`splice_locked` for splice transactions

The order between `ChannelFundingConfirmed` and `ChannelReadyForPayments`
depends on whether 0-conf is used or not.

We remove `ChannelOpened`, which was actually a subset of the existing
`ChannelReadyForPayments` event (which was added afterwards).

We add a few fields to existing channel events, which we don't yet
store in the DB to avoid modifying it, but will store later when we
modify the schema of the `AuditDb`.

We now store an entry in the `AuditDb` whenever a splice transaction
confirms, which allows tracking the full history of a channel's changes.

Stop storing channel errors in `AuditDb` (#3236)

We stop storing every `ChannelErrorOccured` event in the `AuditDb`.
This is very spammy because there are a lot of non-fatal events,
and they turned out to be unnecessary for auditing (they make more
sense in the logs for debugging).

Such events include for example notifying that we tried to relay a
payment to a channel that didn't have enough balance, or commands
sent to channels that are offline, closing or reconnecting.

Note that we stop creating the corresponding table, but we don't
delete old data: it is up to node operators to decide whether they
want to keep it or not. We still support migrating older databases
including data from this table.

We also remove tests from very old migrations (more than 2 years old)
which are spammy but not very interesting anymore.

Don't rebroadcast announcements for spent channels (#3235)

* Don't rebroadcast spent channels

We introduced a mechanism to keep spent channels in the `Router` until
the spending transaction has enough confirmations to avoid removing
spliced channels from our graph too eagerly. An unwanted side-effect
is that while we're waiting for confirmations, we may broadcast channel
announcements for these spent channels, which looks like spam to our
peers.

We now filter out channels that have been spent for which we're waiting
for confirmations when sending out gossip.

* Notify front nodes immediately when channel is spent

Another side-effect of the splice change is that we didn't notify front
nodes that the channel was spent before it reached enough confirmations
so we kept sending channel gossip to our peers.

We now immediately notify the front nodes so that they stop sending this
channel to our peers.

Accountable HTLCs (#3217)

Add accountability signal for HTLCs, it replaces endorsement.
See https://github.com/lightning/bolts/pull/1280

Validate Bolt 11 fallback addresses (#3232)

Validate fallback addresses when decoding Bolt 11 invoices.
Note that we must remove an existing test that used invalid addresses.

Fixes #3135

Add `maxCltvExpiryDelta` parameter to `findRoute*` APIs (#3234)

We add a parameter to `findroute` API variants to limit the total CLTV
expiry delta of the route(s) returned.

Fixes #2617

fixup! Add API methods to spend funds sent to taproot channel addresses (#3220) (#3228)

Use bitcoin-lib 0.46 (taproot tweak refactor) (#3225)

* Use bitcoin-lib 0.46 (taproot tweak refactor)

* Update eclair-core/src/main/scala/fr/acinq/eclair/transactions/Transactions.scala

Co-authored-by: Bastien Teinturier <31281497+t-bast@users.noreply.github.com>

Don't scan the blockchain for spent external channels (#3226)

When an external channel is spent, we don't immediately remove it from
our network graph in case the spending transaction is a splice (see
https://github.com/lightning/bolts/pull/1270 for more details).

A side-effect of this change, introduced in #2936, is that when we
start watching a channel after receiving its `channel_announcement`,
we will scan the blockchain if it is actually already spent. This can
be expensive if peers send us `channel_announcement`s for channels
that have been spent a long time ago since `bitcoind` doesn't provide
an index for spending transactions. It is also misleading, because if
we give up after scanning X blocks of the blockchain, we will create
a log line saying that funds are at risk: they're never at risk since
those are not our channels.

This commit fixes this issue by only checking whether the channel is
already spent by a confirmed transaction or not when setting the watch
(which is an inexpensive and efficient RPC call to `bitcoind`), without
scanning the blockchain to find the spending transaction. If it is
already spent, we immediately remove it from our network graph, even
if the spending transaction was actually a splice. This is fine, since
that channel will be re-added to our graph whenever we receive the
`channel_announcement` for the splice. In the worst case, we will simply
not route through an actually available channels for a few blocks while
its splice transaction is confirming.

Co-authored-by: pm47 <pm.padiou@gmail.com>

Allow remote `dust_limit_satoshis` up to 5000 sats (#3227)

Set allow remote `dust_limit_satoshis` up to 5000 sats, which means that
HTLC outputs should be claimable at feerates lower than `25 sat/kw`.

Given that our default `dust-tolerance` is set to 50 000 sats, this
allows at least 10 pending dust HTLCs before we start failing dust HTLCs
back instead of relaying them.

Identify failing node by its index (#3224)

When a HTLC is failed remotely, the failing node was previously identified by its node id.
However this is not enough if the same node appears multiple times in the payment route, for instance for circular rebalancing.
We now identify the failing node using its index in the payment route.

Unwatch previous funding tx after splice (#3218)

After a splice transaction confirms, we don't need to keep watching the
previous funding output: it has been irrevocably spent and needlessly
consumes resources in the `Watcher` actor.

Those watches are cleaned up when the `Channel` actor dies, which does
not happen if the channel isn't closed and the `Peer` actor is kept
alive while disconnected.

Whenever we receive a new block, we log the number of watches we have.
This lets us detect whether we're missing some clean-up of old watches
in edge cases.

Add API methods to spend funds sent to taproot channel addresses (#3220)

Add API methods to spend funds sent to taproot channel addresses

This PR adds new API calls that extend spendFromChannelAddress* calls to taproot channels.

---------

Co-authored-by: pm47 <pm.padiou@gmail.com>

Fix links to 'Reaching The Ground With Lightning' (#3219)

Monitor the internal state of `ReputationRecorder` (#3216)

It is quite hard to reason about the internal state of this actor, and
it tracks every pending HTLC. If for some reason we're not removing
HTLCs correctly, this would create a memory leak.

We log the number of pending HTLCs every 5 minutes to help detect if
clean-up is correctly implemented.

Update default configuration for revoked HTLC clean-up (#3212)

The default configuration for removing old rows containing revoked HTLC
data after a channel close was too slow and inadequate for most nodes.
We update it to use smaller batches without a much shorter interval.

Thanks @DerEwige for providing data about their experiments.

Fixes #3211

Allow high remote `dust_limit_satoshis` (#3215)

Now that we only accept anchor output channels and beyong, we should
let our peers use high `dust_limit_satoshis` values to ensure that the
transaction outputs are economically spendable.

We can't yet raise our `dust_limit_satoshis`: we must first accept high
values, and in the next release (and once other implementations support
it as well), then we'll be able to increase our default value.