Core Lightning

pytest: don't invoke the pcap test every session.

Only do it when the tcp_capture fixture is invoked, which is only for one test.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

schemas: fix ping requirements documentation

Changelog-None

gitignore: add lightningd/lightning_gossip_compactd

Changelog-None

lightningd: add channel_id filter to listpeerchannels

Sometimes we don't know the short_channel_id yet (e.g the channel has
not been broadcasted), but we still know the channel_id from the funding
transaction. Add a channel_id filter to listpeerchannels and clarify
that id, short_channel_id and channel_id are mutually exclusive.

Changelog-Added: `listpeerchannels` now accepts a `channel_id` filter,
for cases where the `short_channel_id` is not known yet.

Signed-off-by: Peter Neuroth <pet.v.ne@gmail.com>

CI: use nproc + 1 for pytest parallelism..

More future proof, better for self-hosted runners.

Example run times before:

4s CI completion
2m 36s Check we can downgrade the node (compile-gcc, sqlite3, liquid-regtest)
2m 52s Check we can downgrade the node (compile-gcc, postgres, regtest)
3m 20s Check we can downgrade the node (compile-gcc, sqlite3, regtest, 1)
3m 20s Check we can downgrade the node (compile-gcc, sqlite3, regtest, 1)
8m 1s Run fuzz regression tests
11m 46s Run unit tests (compile-clang-sanitizers, 0)
12m 59s Compile CLN compile-gcc
13m 42s Compile CLN compile-clang
14m 10s Compile CLN compile-gcc-O3
14m 15s Compile CLN compile-clang-sanitizers
15m 23s Pre-build checks
19m 37s Test CLN dual-fund
26m 49s Run unit tests (compile-gcc, 1)
43m 19s Valgrind Test CLN Valgrind (06/10)
43m 21s Valgrind Test CLN Valgrind (08/10)
43m 40s Valgrind Test CLN Valgrind (05/10)
43m 59s Valgrind Test CLN Valgrind (03/10)
44m 4s Valgrind Test CLN Valgrind (07/10)
47m 28s Valgrind Test CLN Valgrind (09/10)
47m 37s Valgrind Test CLN Valgrind (02/10)
47m 45s Valgrind Test CLN Valgrind (01/10)
52m 51s Valgrind Test CLN Valgrind (04/10)
54m 6s Valgrind Test CLN Valgrind (10/10)
1h 14m 13s Sanitizers Test CLN (ASan/UBSan (03/12), 3, --test-group=3 --test-group-count=12)
1h 14m 6s Sanitizers Test CLN (ASan/UBSan (06/12), 6, --test-group=6 --test-group-count=12)
1h 15m 23s Sanitizers Test CLN (ASan/UBSan (01/12), 1, --test-group=1 --test-group-count=12)
1h 20m 30s Sanitizers Test CLN (ASan/UBSan (08/12), 8, --test-group=8 --test-group-count=12)
1h 22m 35s Sanitizers Test CLN (ASan/UBSan (12/12), 12, --test-group=12 --test-group-count=12)
1h 23m 49s Sanitizers Test CLN (ASan/UBSan (04/12), 4, --test-group=4 --test-group-count=12)
1h 26m 11s Sanitizers Test CLN (ASan/UBSan (07/12), 7, --test-group=7 --test-group-count=12)
1h 28m 44s Sanitizers Test CLN (ASan/UBSan (05/12), 5, --test-group=5 --test-group-count=12)
1h 29m 1s Sanitizers Test CLN (ASan/UBSan (02/12), 2, --test-group=2 --test-group-count=12)
1h 29m 4s Sanitizers Test CLN (ASan/UBSan (11/12), 11, --test-group=11 --test-group-count=12)
1h 31m 36s Sanitizers Test CLN (ASan/UBSan (09/12), 9, --test-group=9 --test-group-count=12)
1h 38m 54s Sanitizers Test CLN (ASan/UBSan (10/12), 10, --test-group=10 --test-group-count=12)
1h 39m 22s Test CLN liquid
1h 52m 23s Test minimum supported BTC v25.0 with clang
1h 52m 28s Test CLN gcc
1h 53m 57s Test CLN splicing
1h 57m 25s Test CLN postgres
1h 59m 25s Test CLN clang

2d 17h 56m 16s TOTAL

Example runtimes afterwards:



Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: make downgrade and slow integration tests depend on check-compiled-source step.

As pointed out by Alex Myers: we don't want to waste time running the
complex steps if the generated files are not up-to-date.

But we still run the faster "integration" tests (not valgrind or
sanitizer ones), since they often reveal early failures.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

connectd: teach dev_report_fds about Tor fds

Our CI didn't run the Tor tests, and when we do, they fail
(particularly tests/test_gossip.py::test_tor_port_onions and
test_static_tor_onions):

```
lightningd-2 2026-02-16T01:06:28.607Z INFO connectd: dev_report_fds: 3 -> hsm fd
lightningd-2 2026-02-16T01:06:28.607Z INFO connectd: dev_report_fds: 4 -> gossipd fd
lightningd-2 2026-02-16T01:06:28.607Z INFO connectd: dev_report_fds: 5 -> listener (connection_in)
lightningd-2 2026-02-16T01:06:28.607Z INFO connectd: dev_report_fds: 5 name IPv4 socket 127.0.0.1:46045
lightningd-2 2026-02-16T01:06:28.607Z **BROKEN** connectd: dev_report_fds: 6 open but unowned? fd mode: socket
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Makefile: mark more things as intermediaries.

This means we don't have to include them in our bundle.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: use more optimized builds sometimes.

1. Don't --enable-debugbuild for sanitizer runs (slow) or -O3 compilations.
2. Use -O3 build for splicing and postgres tests runs (speed them a
little), make sure builds without --enable-debugbuild work.
3. Remove clang test run, because we already use that for
minimum-btc-version test: this saves another 2 hour CI job.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

plugins: neaten rust plugins, make them intermediate objects.

This keeps it simpler for future expansion, and also means we won't
rebuild them as much.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: Don't explicitly invoke .github/scripts/install-bitcoind.sh

It's done by setup.sh, so simply set the env var for that. Don't try
to reinstall valgrind or call `sudo apt-get update -qq` either, since
setup.sh does that too.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: Don't do release target build, create "small" profile

Using RUST_TARGET=release is slower than RUST_TARGET=debug, *but*
the images are much bigger.

On my laptop, but restricted to 4 CPUs. I added a target to build all
the prerequisited but not compress the bzip2:

RUST_TARGET Time to build Size (testpack.tar.bz2)
release 4m49s 133MB
debug 2m34s 343MB
small 2m9s 155MB

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: use -O1 optimized builds for sanitizer and valgrind runs.

ChatGPT suggested this for sanitizer runs, but valgrind gains a little
too.

Sanitizer before: 74.1-98.9(84.5+/-7.3) minutes
Sanitizer after: 11.6-15.2(13.3+/-1.2) minutes

Valgrind before: 43.3-54.1(46.8+/-3.8) minutes
Valgrind after: 40.9-49.4(43.4+/-2.5) minutes

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Makefile: don't build the sources in make check-source.

check-source-bolt needs devtools, and check-python needs a full build.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Makefile: make testpack.tar.gz from (almost) all changed files.

Here are the speeds and sizes of different compressors (just
compressing the raw tar file):

bzip2: 34.15s 412MB

gzip -1: 9.96s 420MB
gzip -2: 10.39s 417MB
gzip -3: 11.63s 414MB
gzip -4: 12.29s 409MB
gzip -5: 14.41s 406MB
gzip -6: 19.02s 405MB
gzip -7: 22.96s 405MB
gzip -8: 39.63s 404MB
gzip -9: 72.19s 404MB

xz: 72.92s 407MB

Uncompressing time:

xz: 2.3 seconds
bzip2: 20 seconds
gzip -5: 3 seconds

Note that this target is only useful on a *freshly built* tree: it
will include everything config.vars and newer.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

ci: Enable sccache compiler cache for Rust and C

[ Cherry-picked from another PR, and read the docs which say you have
to set SCCACHE_GHA_ENABLED to get inter-job caching! --RR ]

CI: Use variable to make group counts neater.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: Use -O3 builds for Liquid integration tests.

The other integration tests are taking under an hour, and this is
taking 1hr 30m.

The downside is that backtraces are far less clear, but that's why the
main run uses the full debug version.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Makefile: clean up a little.

Make cdump-enumstr use libccan.a, rather than explicit ccan object: it's the
last one.

And mark submodcheck phony, as it is.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

pytest: mark test_partial_payment_restart slow.

We can actually be slow enough that we get a (60 second) MPP timeout
under valgrind:

```
# Restart, and make sure it's reconnected to l2.
l3.restart()
print(l2.rpc.listpeers())
wait_for(lambda: [p['connected'] for p in l2.rpc.listpeers()['peers']] == [True, True])

# Pay second part.
l1.rpc.sendpay(
route=route,
payment_hash=inv['payment_hash'],
amount_msat=1000,
bolt11=inv['bolt11'],
payment_secret=paysecret,
partid=2,
groupid=1,
)

> l1.rpc.waitsendpay(payment_hash=inv['payment_hash'], timeout=TIMEOUT, partid=1)

tests/test_pay.py:3352:
...
E pyln.client.lightning.RpcError: RPC call failed: method: waitsendpay, payload: {'payment_hash': 'd73e10b604f53afb05da052d1fc90c45269321169196e31b7ccc312eabc26557', 'timeout': 180, 'partid': 1}, error: {'code': 204, 'message': 'failed: WIRE_MPP_TIMEOUT (reply from remote)', 'data': {'created_index': 1, 'id': 1, 'payment_hash': 'd73e10b604f53afb05da052d1fc90c45269321169196e31b7ccc312eabc26557', 'groupid': 1, 'partid': 1, 'destination': '03cecbfdc68544cc596223b68ce0710c9e5d2c9cb317ee07822d95079acc703d31', 'amount_msat': 500, 'amount_sent_msat': 501, 'created_at': 1771551338, 'status': 'pending', 'bolt11': 'lnbcrt10n1p5e0wnfsp5vpgn0adhmyjecp47yfc6dtefett7d3qqzh4j908ane7tezyy0evqpp56ulppdsy75a0kpw6q5k3ljgvg5nfxggkjxtwxxmuescja27zv4tsdq9d9h8vxqyjw5qcqp9rzjqvuytqpdyk6wqaxvl47d3vee5swuwklej79qxjqqg394r4ptqaue5qqqvuqqqqgqqqqqqqqpqqqqqzsqqc9qxpqysgq6ur4cclfhnzs0tgenmxhvx8glw3q9eylp3pdck73x58cukwtfxj5zjg6g0a4p4plgj6qd0s0uxkuntzcn4l6vvndgrjj7gndma3t7pcqmr9q3j', 'erring_index': 2, 'failcode': 23, 'failcodename': 'WIRE_MPP_TIMEOUT', 'erring_node': '03cecbfdc68544cc596223b68ce0710c9e5d2c9cb317ee07822d95079acc703d31', 'erring_channel': '103x1x0', 'erring_direction': 0, 'raw_message': '0017'}}
```

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: avoid rebuilds on downloads using testpack.tar.gz.

We use the same fragment everywhere for consistency, even though some
stages don't actually call make.

The magic is:
CARGO=false CC=devtools/cc-nobuild SUPPRESS_GENERATION=1

Which causes us to fail if we want to rebuild.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: Make 12 not 10 valgrind tests, and 6 not 12 sanitizer tests.

Given the new timings, this should even things out a little.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: split gcc+sqlite3 tests into 6 runners.

It takes 1h43m, and it's our "basic" integration test. So split 6
ways to get faster results, then if that passes do the other tests
(Postgres, clang, Liquid, splicing, dual-funding).

Now they take about 20 minutes each.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

Makefile: support overriding of CC and CARGO for catching unintended builds.

For CI we want to make sure we don't rebuild. CC=false doesn't work because
we reply on the output of "$CC -dumpmachine" for the external build directory.

So we would use:

make CC=devtools/cc-nobuild CARGO=false

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

CI: do cargo test --all as part of post-build step

No point spending 1m24 seconds for *each* different compiler variant.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>