Bitcoin Core

Merge bitcoin/bitcoin#35952: kernel: prevent dangling iterators from temporary ranges

fc0dcf950f97ef30930960c39a04ac28dcdadee1 kernel: keep range iterators tied to their owner (Lőrinc)
0936c55f626c21a3788545a7fbd6be4424ae3350 test: characterize kernel range iterators (Lőrinc)

Pull request description:

**Problem:** Kernel wrapper methods return `Range` views by value, but their iterators point to the `Range` object.
Saving an iterator from a temporary view, such as `block.Transactions().begin()`, leaves it pointing to the destroyed view, so later use has undefined behavior.

**Fix:** Make range iterators point to the underlying Kernel wrapper object and use the range's compile-time getter for element access.
Remove `operator->`, which returned elements by value and could not easily support arrow expressions.

ACKs for top commit:
purpleKarrot:
ACK fc0dcf950f97ef30930960c39a04ac28dcdadee1
yuvicc:
ACK fc0dcf950f97ef30930960c39a04ac28dcdadee1
sedited:
ACK fc0dcf950f97ef30930960c39a04ac28dcdadee1

Tree-SHA512: 85ae1f8d4c62a762a10c546ebb312f126efc5ef349557dd235c81b585bef92cbb6c4a565734d85d315641a169f48b6032b859f5a9f070bc347b1424b05473e5d

Merge bitcoin/bitcoin#35680: private broadcast: bound rebroadcast attempts to 1,000

fe7d475d450b9aabd549627c6e61024ae45e7a8c private broadcast: bound broadcast attempts per tx to 1k (Gregory Sanders)

Pull request description:

Since rebroacasts introduce additional state, bound the state growth by capping the number of rebroadcasts. With ~72 bytes per record, 10k transactions rebroadcasting for ~42 hours will result about 703 MiB allocated with overhead.

ACKs for top commit:
andrewtoth:
ACK fe7d475d450b9aabd549627c6e61024ae45e7a8c
frankomosh:
ReACK fe7d475d450b9aabd549627c6e61024ae45e7a8c
sedited:
ACK fe7d475d450b9aabd549627c6e61024ae45e7a8c

Tree-SHA512: e4ec5156b90ad24d68b561df03ad09bdf0ac7535886ff56891cb698cf64ff0e1e484075b76040bba6194baf874c9237028c82debf7405136447ba5b5faee589c

Merge bitcoin/bitcoin#35993: guix: build glibc with `--enable-kernel=3.17.0`

5548818115c02d1e452a3240593b6e929c387d1d guix: build glibc with --enable-kernel=3.17.0 (fanquake)

Pull request description:

Our minimum required kernel version is documented as `3.17.0`. Pass `--enable-kernel=3.17.0` when building glibc, so that version is reflected in the binary, and the version checked in the symbol-check script, aligns with the expected minimum.

ACKs for top commit:
hebasto:
ACK 5548818115c02d1e452a3240593b6e929c387d1d, tested on Ubuntu 24.04:
willcl-ark:
ACK 5548818115c02d1e452a3240593b6e929c387d1d

Tree-SHA512: fc23561d77da80f53cf7564bdb87f5f3c0b23401de79e0801168190d7a99c96e6a60e9437779680b43173572aededf4242b0fba0fe255970f83e6529e3149870

Merge bitcoin/bitcoin#36007: http: Make HTTPRequest::m_client a weak_ptr

979a42ec17369b55a3a20c9e45965e6733fbeb19 http: Make HTTPRequest::m_client a weak_ptr (Hodlinator)

Pull request description:

Removes the need for `HTTPClient::ReleaseRequest()` as the client<->request cycle is broken. Not having to remember to call `ReleaseRequest()` reduces cognitive load.

Follow-up to #35735.

ACKs for top commit:
pinheadmz:
untested ACK 979a42ec17369b55a3a20c9e45965e6733fbeb19

Tree-SHA512: b740a765ffe0592055819e71df8654614e9e140bb77e4a6d146045255f1db9b470ae4a1a77aa16a0d3f3519b7c0d1e9f8c9fbc4c29aab28a2d55ea828ca912c6

Merge bitcoin/bitcoin#35797: psbt: support output metadata updates before inputs are added

c0792889673da175661f29138a318614b33636cd psbt: update output metadata without inputs (Lőrinc)
4f5712476a3cf66d3b4f225b883eeca2deee657b test: characterize P2WSH miniscript output (Lőrinc)
e24e8fa2a689d7479262540dba6da3f3e21fbaa8 test: characterize PSBT output metadata (Lőrinc)

Pull request description:

**Problem:** PSBTv2 permits outputs to be added before inputs.
An authenticated `descriptorprocesspsbt` request can abort the node while updating metadata for one of those outputs because `UpdatePSBTOutput()` traverses the output script with a signature creator for input index 0.
ECDSA signing or a miniscript timelock check can then access the missing input.

**Fix:** Make `UpdatePSBTOutput()` traverse output scripts with a temporary one-input transaction while continuing to take the output from the PSBT's unsigned transaction.
`MutableTransactionSignatureCreator` continues to require a valid input index.
Output metadata traversal still records scripts and key origins, allowing outputs to be updated before inputs are added.

ACKs for top commit:
jeanpablojp:
tACK c0792889673da175661f29138a318614b33636cd
achow101:
ACK c0792889673da175661f29138a318614b33636cd
w0xlt:
ACK c0792889673da175661f29138a318614b33636cd
polespinasa:
ACK c0792889673da175661f29138a318614b33636cd

Tree-SHA512: 0d8cda74b8a56c0f4713b2669e5a3e5b0551ecda4fdfceb38a80e5b98a9d208d447f2045a1cc9fee74fe33b2fc8f7a60997cd60b2961de5cea871f53831895fe

Merge bitcoin/bitcoin#35986: p2p: reconsider orphans when missing inputs are mined

9cc7dc50bdc9867d079ab7a111d39487a4566767 p2p: reconsider orphans when missing inputs are mined (Greg Sanders)

Pull request description:

We reconsider for mempool entry of missing inputs, we should reconsider for mining of them too.

ACKs for top commit:
yuvicc:
ACK 9cc7dc50bdc9867d079ab7a111d39487a4566767
l0rinc:
Lightly tested code review ACK 9cc7dc50bdc9867d079ab7a111d39487a4566767
marcofleon:
ACK 9cc7dc50bdc9867d079ab7a111d39487a4566767

Tree-SHA512: 9acfb6898e3b286ce23bc2ca3369ae951fadee5f175baad634a6bd23039108d97283814e47972fb857ae459505535f621866f2514b705e94cf841979c37a3933

Merge bitcoin/bitcoin#35954: qa: Disable Qt's glib event dispatcher for GUI tests on OpenBSD

de2adc308a421ea57414c4e28f9a75350df54f25 qa: Disable Qt's glib event dispatcher for GUI tests on OpenBSD (Hennadii Stepanov)

Pull request description:

When `bitcoin-gui` is built against OpenBSD's system Qt packages (which have GLib support), shutdown emits "GLib-CRITICAL **: g_main_context_pop_thread_default: assertion 'stack != NULL' failed" messages on `stderr`, which the test framework treats as a failure.

Set `QT_NO_GLIB=1` so Qt falls back to its poll-based event dispatcher, which avoids the GLib thread-default context entirely.

Fixes https://github.com/bitcoin/bitcoin/issues/35851.

See the CI log here: https://github.com/hebasto/bitcoin-core-nightly/actions/runs/31510478034.

ACKs for top commit:
maflcko:
lgtm ACK de2adc308a421ea57414c4e28f9a75350df54f25

Tree-SHA512: edc991c7a174bc304a4da0ca29ec97bcaece463289de3da5350a046f1133ce6d830c37d5188536ca2ce238d462e56de8f2167fdeeb1d1e5ecca38d60c8495cce

Merge bitcoin/bitcoin#35963: doc : update cjdns docs to discourage using onlynet option

beefda21be3f7904f29bcc81d4cf3e05e8df41df doc : update cjdns docs to discourage using onlynet option (naiyoma)

Pull request description:

Currently, the number of CJDNS addresses is still very small and may not be sufficient to fill all outbound connection slots. When running with the `-cjdnsonly` and `-cjdnsreachable` options, `ThreadOpenConnections()` repeatedly calls `Select()`, which returns the same few addresses over and over. The connection attempts may fail, the addresses remain in `AddrMan`, and the loop continually restarts.

The documentation does mention running CJDNS alongside other networks, but we should explicitly explain why using only CJDNS is discouraged, since running with these options alone is supported.

ACKs for top commit:
achow101:
ACK beefda21be3f7904f29bcc81d4cf3e05e8df41df
jonatack:
ACK beefda21be3f7904f29bcc81d4cf3e05e8df41df
hodlinator:
ACK beefda21be3f7904f29bcc81d4cf3e05e8df41df
brunoerg:
ACK beefda21be3f7904f29bcc81d4cf3e05e8df41df

Tree-SHA512: 46126291ceb1c36384f9b3deaccdcf20baf8cd4a68491d2ad9fc4a35ec71ad4194b38c4a22c9853bd3f329e2ecb9a0452813d1eea121eb3712876e73dfd4b3f0

Merge bitcoin/bitcoin#35995: doc: fix outdated URL in hash_tests.cpp

158efbc723d37eefb06939b7f7e851c8168342bd doc: fix outdated URL in hash_tests.cpp (cyb3ralbert)

Pull request description:

The old URL still 301-redirects, but to the site root, not to the file — and the file itself is gone: the same path on the new domain returns 404. The author's page at https://www.aumasson.jp/siphash/ says that the SipHash page and documentation have moved to github.com/veorq/SipHash.

The vectors now live in vectors.h in that repository. All 64 values match the array below.

ACKs for top commit:
maflcko:
lgtm ACK 158efbc723d37eefb06939b7f7e851c8168342bd
l0rinc:
tested ACK 158efbc723d37eefb06939b7f7e851c8168342bd

Tree-SHA512: e45d6872787474e50d51f408ae496e816adbaa8ff263f34df6af7070cfc94bc4f289c5d43cf79461055a0e9738340eb3088837e42991640a253853a5c9984ec6

Merge bitcoin/bitcoin#35946: rpc: Improve some type specs for openrpc

e07d826e0ebd9507793fe033236e5f0f12ba5732 rpc: Fix type in ApplyTypeStrOverride (Shuvam Pandey)
c94074fa1b1396e310ab94955f5d04c9bda61b64 rpc: Surface OBJ_USER_KEYS description for openrpc (sedited)
c020c21d543a14268b98995d1a9d1878f3d95ec2 rpc: Handle skip type args for openrpc (sedited)

Pull request description:

This was initially motivated by testing the dump of the schema against open-rpc-generator, which crashed with:

```
open-rpc-generator generate -t client -l rust -n bitcoin_client -d ./openrpc.gen.json -o ./generated
There was error at generator runtime:
TypeError: Cannot convert undefined or null to object
```

The changes here fix this crash (albeit perfectly valid existing schema), but I think creating a more complete output is helpful on its own. The openrpc schema dumps can eventually be re-used for the rpc docs and to track rpc interface changes more accurately. Adding the CreateTxDoc outputs section seems useful for that.

Also includes a type tightening from number to integer in `ApplyTypeStrOverride` to reflect the actual behaviour in the rpc calls, where only integers are accepted.

ACKs for top commit:
achow101:
ACK e07d826e0ebd9507793fe033236e5f0f12ba5732
willcl-ark:
ACK e07d826e0ebd9507793fe033236e5f0f12ba5732

Tree-SHA512: d0454a71b4f1dab1daf8a0d5b1e5bf1c1b8f1a16d26638d4a64a2652402ad74230366cabf0cf4135a16d0bdab584d3d4b2a47f2968a4eff605348ce85e8dbadb

Merge bitcoin/bitcoin#35955: wallet: remove orphaned GetAffectedKeys and LegacyScriptPubKeyMan declarations

02de12b1e614035a73cbae58250920c843a17196 wallet: remove remaining LegacyScriptPubKeyMan references (Laxman Acharya)
d194be69d601322a79444e79ffebe475e58e5f77 wallet: remove orphaned GetAffectedKeys declaration (Laxman Acharya)

Pull request description:

Follow-up to #28710, which removed `GetAffectedKeys()` and `LegacyScriptPubKeyMan` but left their declarations behind.

Remove both orphaned declarations, rename `SetupLegacyScriptPubKeyMan()` to `SetupLegacyDataSPKM()`, and update related comments and logging to reflect the minimal `LegacyDataSPKM` retained for legacy wallet loading and migration.

## Testing

```bash
cmake --build build --target bitcoin_wallet -j 8
cmake --build build --target test_bitcoin -j 8
build/bin/test_bitcoin --run_test=scriptpubkeyman_tests,walletdb_tests,wallet_tests --catch_system_errors=no --log_level=error
```

ACKs for top commit:
achow101:
ACK 02de12b1e614035a73cbae58250920c843a17196
polespinasa:
ACK 02de12b1e614035a73cbae58250920c843a17196

Tree-SHA512: 0d340291c969f013fd6ec55158f6797d4a1478ff7a819c2d250d1b3f345131db99a8b7b4b3ff0da1fa0e06182701abed31dcd317f8ef12c81837c1aed57b21a5

Merge bitcoin/bitcoin#35972: fuzz: Fix assertion in `txorphan`

01dde6b20578616af7620a2180ee6b5db0ca4793 fuzz: Fix assertion in txorphan (marcofleon)

Pull request description:

`EraseTx()` calls `LimitOrphans()`, which may evict announcements from a peer that didn't announce the erased transaction, causing that peer's usage to decrease. Relax the assertion in the `EraseTx()` branch that claimed usage of a non-announcer peer should be unchanged. Also, add assertions for the other cases.

ACKs for top commit:
dergoegge:
utACK 01dde6b20578616af7620a2180ee6b5db0ca4793
instagibbs:
ACK 01dde6b20578616af7620a2180ee6b5db0ca4793

Tree-SHA512: 2e597b85fd41058c2fa79fa55f0d37e12505065b5e27aba7b9680e0c249a5450e6fa97b45394d6ffe1318f42538134ffa9c423b126c455f6f8e6d8ca59eed4b6

Merge bitcoin/bitcoin#35735: Add state to HTTPRequest

9954aa77280ecd67816e784815c6478a973f6635 http: don't parse any new requests from a client if m_req_busy = true (Matthew Zipkin)
c7db3ae1f907a10748e608cd69899c7586bfc210 test: cover HTTPRequest state machine (Matthew Zipkin)
90676e24ad1aa44346aed586f19ba1c68c477394 Add state to HTTPRequest to avoid duplicate work over I/O cycles (Matthew Zipkin)
507e528e845034583dd21b884e6debb1ff5307e3 http: reuse HTTPHeaders to parse chunked trailer (Matthew Zipkin)
902d8908c94dbe837d712a408d63dcbac634d4c5 http: only read one HTTPRequest at a time per client (Matthew Zipkin)

Pull request description:

This PR reduces the memory consumption of the HTTP Server when reading data from connected clients, and improves performance especially when requests are large (i.e. requiring multiple TCP packets).

In https://github.com/bitcoin/bitcoin/pull/35182 the server copies as much data as it can from the socket into application memory, and then tries to parse as many complete HTTP requests as possible from that data. If a request is discovered to be incomplete, the in-progress request is abandoned. The server tries again on the next I/O cycle to read the same data from the buffer, duplicating work as many times as it takes before the client finishes sending the request (or times out).

This PR implements two improvements to this:
1. Only parse one request at a time from the receive buffer. The server processes requests from each client in series anyway.
2. Add state to `HTTPRequest` so it can be filled with data from the receive buffer over multiple I/O loop iterations without losing progress.

If a client sends large or multiple requests, that data will sit in the kernel's socket buffer instead of the application memory. Eventually the socket buffer will fill up and TCP backpressure will kick in, dropping the TCP window to 0 and blocking the client from sending any more.

A state machine for `HTTPRemoteClient` was [discussed previously](https://github.com/bitcoin/bitcoin/pull/35182#pullrequestreview-4322490068) to control resource consumption. Another nice benefit of this model (for a follow-up PR) will be to insert the RPC authentication check after reading 8kB-limited headers but before the 32MB-limited request body.

ACKs for top commit:
winterrdog:
re-ACK 9954aa77280ecd67816e784815c6478a973f6635
janb84:
re ACK 9954aa77280ecd67816e784815c6478a973f6635
frankomosh:
ACK 9954aa77280ecd67816e784815c6478a973f6635.
fjahr:
ACK 9954aa77280ecd67816e784815c6478a973f6635

Tree-SHA512: b7c913114283fbf1f360b40f6c65a01390a26731bf3b166f460ec260f9206f25d738b3a06887bfa839911c1c6aaf634448181da47a752a9a881aebd907e44868

Merge bitcoin/bitcoin#35846: test: Use throwing config parser getters without fallback

fabe100c2b33e03195c157a2eb267029e8532784 test: Use throwing config parser getters without fallback (MarcoFalke)
fa8acd57cd1b9fcd62abc1eca0cab57b04a52a83 test: Write true/false values in config.ini (MarcoFalke)

Pull request description:

Currently, the called `getboolean` member function is *not* the throwing https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.getboolean, but a non-throwing member function on a dict-like proxy object.

This is confusing and brittle, because tests shouldn't silently skip when a config key is missing. Instead, tests should loudly fail, e.g. when the config key is renamed in one place, but not the other.

ACKs for top commit:
jeanpablojp:
tACK fabe100c2b33e03195c157a2eb267029e8532784
willcl-ark:
ACK fabe100c2b33e03195c157a2eb267029e8532784

Tree-SHA512: a970d74ad285372b8adcce8e2a52b01f5a3b563899dfc5262e6ffbf3d8aba43e72f7b03111e8d5188924c7d3d789992407cddb5d182d9be5e42f07896d8ad4a3

Merge bitcoin/bitcoin#35982: Update minisketch subtree to latest master

461e3be8d3d1cc618d3a9f5b3bfcfd7e0d249f9f Squashed 'src/minisketch/' changes from d1bd01e189..4a179c61e3 (fanquake)

Pull request description:

Includes:
* https://github.com/bitcoin-core/minisketch/pull/102

ACKs for top commit:
hebasto:
ACK 2dcb2c6df20bf97aef312d2d71c9ebdea66156c1.

Tree-SHA512: 295e217cb6d32e8d0bb4ea84bd0d6682f98735029c659ad06a6c588fa2865176d80adfca32a1a3a339ffb601e5d1cc99c81266914225e27726a8a859fee3549e

Merge bitcoin/bitcoin#35976: test: Speedup fee estimation functional test with batching

b3d77ea0275e56ad5aca442100b66789b5c08ec5 test: Speedup fee estimation functional test with batching (sedited)

Pull request description:

The fee estimation functional test is currently the slowest one by a good margin. It is a bit annoying, because it also increases the total runtime of the functional tests.

It seems like most of the slowness comes from the transactions propagating between the nodes. This patch helps them do that by submitting them directly to all the nodes. Also take this opportunity to batch the transaction submissions.

On my machine this speeds up the fee estimation functional test from around 71 seconds to 25 seconds.

ACKs for top commit:
151henry151:
tACK b3d77ea0275e56ad5aca442100b66789b5c08ec5
maflcko:
review ACK b3d77ea0275e56ad5aca442100b66789b5c08ec5 🐇
ismaelsadeeq:
ACK b3d77ea0275e56ad5aca442100b66789b5c08ec5

Tree-SHA512: f76415dca7997577ca39ac6b95dfdf32b930dd64b4311e3da34e34adb16107ff4ea2d9fa679f3ca50540e80c38af7f9390b44f21ad1b8107fbbc3586edb2ef19

Merge bitcoin/bitcoin#35531: txindex: hash keys and pack positions to reduce disk usage

25bed560bed370045bead21c27ea21aa228cd963 test: add forward-compat functional test for txindex (sedited)
703304ed8c19d4bc7a2bb7ff0800e5047ccc12c4 doc: add release notes for txindex disk usage and downgrading (Andrew Toth)
8e5320a2d2463131a6442f27d335c0a0f6b577fd tests: cover txindex hash prefix collisions and legacy fallback (Andrew Toth)
b75efa19ba806fbd13604a6bf67ee96a3f9eccf1 txindex: skip bloom filters and legacy lookups for new databases (Andrew Toth)
004d7c098cadecdf1915c77a95d0f856ef0c2ca5 txindex: hash key prefixes and pack block positions (Andrew Toth)
5a255970fd1b02052f669abe064ffa4d4776e40f refactor: move txindex db constants and legacy key to txindex_key.h (Andrew Toth)
327660134cb1629e31ecccb6b5a589e1e27609b2 txindex: pass the full block to DB::WriteTxs (Andrew Toth)
42771e79980cdcedd4876a168d6164c0fd26f5d2 txindex: use a new block locator for downgrade safety (Andrew Toth)
4b08baed72c4dba8d17fe68a85de459fd273d065 txindex: return optional tx and block hash from FindTx (Andrew Toth)

Pull request description:

The current txindex uses the full 32-byte txid as keys, which takes up about 66 GB of disk space today on mainnet. Using a 5-byte key prefix instead drops the disk usage to 26 GB - cutting the size to less than half.

Using the full 32-bytes is unnecessary since a 5-byte salted siphash will produce collisions in about 1 in 1.1 trillion. Some collisions will occur, but the penalty is just an extra disk read, deserialization and hash.
The tx position can be appended to the key instead of used as a value, and a LevelDB iterator can seek to the prefix and then scan for the correct tx. This is an almost identical approach to `txospenderindex`.

Also instead of storing the file position of the block, we can store only the sequence of the connected block and offset of the transaction in the block. This can be packed into a 6-byte key suffix using 3-byte representations of the sequence and offset in the block. The block file can be recovered by the CBlockIndex that is already in memory. The sequence is mapped to the block hash in the db, so we can lookup the block hash to find the CBlockIndex during reads.

If a tx is not found with this method, we fallback to looking up the legacy entry. With this method a user with an existing db can opt to erase the `indexes/txindex` folder and reindex, or keep the current index and new entries will be appended with the smaller footprint.

The time to index was faster on my machine with this method, 1h19m vs current 1h50m.
Lookups are roughly the same, around 0.2ms per lookup with `getrawtransaction`.
When testing on mainnet, I got 894,549 2-way collisions, 395 3-way collision, and 1 4-way collision that worst case could cause an extra 3 false positives when reading.

ACKs for top commit:
l0rinc:
diff reACK 25bed560bed370045bead21c27ea21aa228cd963
sedited:
ACK 25bed560bed370045bead21c27ea21aa228cd963
ajtowns:
ACK 25bed560bed370045bead21c27ea21aa228cd963

Tree-SHA512: a25c79ca7e722e2f372b65f5fc11c8b194ad49f2240b4881c7e606306aabbd3604aede3f1c33606b467486affac3a3f503638f513c896935cebbc02709cb60d8

Merge bitcoin-core/gui#957: fix: add .dat file extension automatically when exporting watchonly

75a4e6c67881a791d6cbc81008c0d70d12c26710 gui: fix allow restore wallets without .dat file extension (Pol Espinasa)
6ed7e05e20da874fe13e4ff74b7a534ca1b2b141 gui: fix add .dat file extension automatically when exporting watchonly (Pol Espinasa)

Pull request description:

fixes https://github.com/bitcoin-core/gui/issues/956

Unlike `backup wallet`, `export watch-only wallet` was not automatically adding the file extension to the exported file, making restoring difficult if the user doesn't manually add the file extension after exporting.

Allows also to restore a wallet from a non specified `.dat` file extension. This is achieved by removing the filter in the select file screen, matching the RPC behavior.

ACKs for top commit:
hebasto:
ACK 75a4e6c67881a791d6cbc81008c0d70d12c26710.

Tree-SHA512: 7c45d51205f9abf2b67233e8abd3297e49a4230eb32aa4118b37ab9da0a8d692aae4b67a8880881e5fab42256d1cf52ccf1b289b8f23f85930354130c192b33d

Merge bitcoin/bitcoin#32784: wallet: derivehdkey RPC to get xpub at arbitrary path

c3945bfd2bf87ae6fe7be1c0cee58b21c4269aec doc: use derivehdkey in multisig tutorial (Sjors Provoost)
3662e3366978e1ecff1924da100cd2606899126e test: use derivehdkey in M-of-N multisig demo (Sjors Provoost)
d9570f0838355bca638ac3287fd334905fb29e16 rpc: add derivehdkey (Sjors Provoost)
62da9f9614508f6bfd30126a73c59d0c4004066e wallet: add GetExtKey helper (Sjors Provoost)
aaf1548475ded41889d5041fc2d60e8b2ca9ddcd wallet: generalize GetActiveHDPubKeys helper (Sjors Provoost)
3821452c4af1440484ddd5ccc83f83cc4c307af3 refactor: add hardened derivation helper (Sjors Provoost)
0ab61caafd10d0539da626bd4354cb7e6df15e54 rpc: ParsePathBIP32 helper (Sjors Provoost)
e36c4b76e198751c14da8d32655e47bd5678abb6 util: reject out-of-range BIP32 keypath indices (Sjors Provoost)
ba78c31a00c912ec440dfb677997b51c72058bab fuzz: check ParseHDKeypath/WriteHDKeypath round-trip (Sjors Provoost)
8cce969085dbe3137c738cbe691008974059af1e Have ParseHDKeypath handle h derivation marker (Sjors Provoost)
fc5307776236c6f08aec94978672626f5aae1aaa test: move parse_hd_keypath test to bip32_tests (Sjors Provoost)
dab525eb7717af3b246a5aa32acea174cd9bd453 key: add DeriveExtKey() helper (Sjors Provoost)

Pull request description:

Adds a `derivehdkey` RPC that returns an xpub, or optionally the xprv, at an arbitrary BIP32 path (with at least one hardened step), derived from a wallet HD key.

The main use case is coordinating a multisig setup, where each participant shares an xpub derived at a hardened path (e.g. `m/87h/0h/0h`) distinct from their default single-signature descriptors. See the (updated) `doc/multisig-tutorial.md` and (updated) functional test to see how that workflow improves.

The first commits are some helpful helpers:

- _key: add DeriveExtKey() helper_ - performs the actual derivation
- _test: move parse_hd_keypath test to bip32_tests_ - from `psbt_wallet_tests`
- _Have ParseHDKeypath handle h derivation marker_
- _util: reject out-of-range BIP32 keypath indices_ - `ParseHDKeypath` would previously map overflowing values without `h` to hardened.
- _fuzz: check ParseHDKeypath/WriteHDKeypath round-trip_
- _rpc: ParsePathBIP32 helper_
- _refactor: add hardened derivation helper_ - `HasHardenedDerivation()`, to enforce the "at least one hardened step" rule
- _wallet: generalize GetActiveHDPubKeys helper_ - extracts code from `gethdkeys` which `derivehdkey` needs
- _wallet: add GetExtKey helper_ - reconstruct an xprv from a wallet xpub (analog of `GetKey()`); behavior-preserving prep, also simplifies `gethdkeys`.

Meat and potatoes:
- _rpc: add derivehdkey_ - the RPC itself, plus the `UnusedKey` filter on `GetHDPubKeys` that drives key selection.
- _test: use derivehdkey in M-of-N multisig demo_ - rewrites the functional multisig test to use the RPC and `<0;1>` syntax.
- _doc: use derivehdkey in multisig tutorial_ - same for the prose tutorial.

ACKs for top commit:
pseudoramdom:
code review ACK c3945bfd2bf87ae6fe7be1c0cee58b21c4269aec
achow101:
ACK c3945bfd2bf87ae6fe7be1c0cee58b21c4269aec
w0xlt:
That being the case, ACK c3945bfd2b

Tree-SHA512: 661f17c9bfe26017eb14c27ba7af37093387100d3baa25f5d29bba9c1aedc40d19afe1bdfc126a18d018857bb02f1fc84386f10b8f4f4b8e9d6f4b0691d9e302

Merge bitcoin/bitcoin#35889: rpc: avoid quadratic `gettxspendingprevout` work and preserve order

ae36e2ef798fa154d54dbe108feea583ebcee48d rpc: avoid quadratic prevout resolution (Lőrinc)
da1eaeb35075c9699da81153759fbc0b257f08bb rpc: preserve `gettxspendingprevout` order (Lőrinc)
f98753e76210b43ad63cfeaca15e10327c21f93f refactor: identify prevouts by request index (Lőrinc)
221a3fe5cfbee8d90eb6ef588ababa1c0a4b4a19 test: cover mixed `gettxspendingprevout` order (Lőrinc)

Pull request description:

**Problem:** `gettxspendingprevout` erases each mempool result from a vector while holding `mempool.cs`, shifting the remaining requests every time and making large calls quadratic in the critical section.
For 10,000 mempool matches, an [operation-count model](https://godbolt.org/z/nzch7McPG) reaches nearly 50 million moves.
For mixed requests, the RPC returns mempool results before `txospenderindex` results instead of following request order.
#34749 introduced both regressions.

**Fix:** `gettxspendingprevout` stores each result at its request position and collects unresolved requests in a reserved worklist for the `txospenderindex` lookup.
The mempool pass is linear, the response follows request order, and Clang can verify the lock requirement on `GetConflictTx`.

**Benchmark:** The [functional benchmark](https://gist.github.com/l0rinc/c3231e287cacfdefd100dbf95cd0c3ad) sends mempool-only requests ranging from 8,000 to 128,000 entries ten times per size.
Using the same settings for the unfixed and fixed commits:

```text
AMD Ryzen 7 3700X (8 cores)
unfixed ██████████████████████████████ 90 s
fixed ███▒░░░░░░░░░░░░░░░░░░░░░░░░░░ 10 s (-80 s, 9.0x faster)

Raspberry Pi 5 (4 cores)
unfixed ██████████████████████████████ 685 s
fixed ▓░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 22 s (-663 s, 31.1x faster)
```

The unfixed run timed out after ~9 minutes on a Raspberry Pi 4 with 1 GB RAM.

<details><summary>Benchmark command</summary>

```bash
for commit in 963b061358b489b2ff4ff64895f2b895b3b89844 46e7173550a93cbe9d4e8ea28cfe7216286d8197; do \
git fetch origin "$commit" && git checkout --detach "$commit" && \
rm -rfd build && cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTS=OFF -DENABLE_WALLET=OFF >/dev/null 2>&1 && \
ninja -C build -j1 bitcoind >/dev/null 2>&1 && \
build/test/functional/test_runner.py rpc_gettxspendingprevout_quadratic.py --repeats=10 || break; \
done
```
</details>

ACKs for top commit:
andrewtoth:
ACK ae36e2ef798fa154d54dbe108feea583ebcee48d
sedited:
Re-ACK ae36e2ef798fa154d54dbe108feea583ebcee48d

Tree-SHA512: c7734cae481f6638228c8fd3cc6d4c3fbc26cec6981dae7902292af08eae37455d37ff196da2cca5c3694271c99154838431ff21c12855f81f5f10edf85a793a

Merge bitcoin/bitcoin#35971: net_processing: remove unused code

0cff3cc5187918b4151f3b84676f7e24365babc5 net_processing: Remove redundant porphanTx in ProcessOrphanTx (fanquake)
c7eacbd45b08199b85b021764d1e84819e39ee94 net_processing: remove Peer& from UpdatePeerStateForReceivedHeaders (fanquake)

Pull request description:

Remove unused `peer` param from `UpdatePeerStateForReceivedHeaders`.
Remove redundant `porphanTx` declaration from `ProcessOrphanTx`.

ACKs for top commit:
marcofleon:
ACK 0cff3cc5187918b4151f3b84676f7e24365babc5
sedited:
ACK 0cff3cc5187918b4151f3b84676f7e24365babc5

Tree-SHA512: d71684052f9f6c788b4d551886e7f6022d76300e9288089eea5fd0c87f9466dbcff88c9004899a399c669477988437e935aad06960cec721c650b66508c7a16a

Merge bitcoin/bitcoin#35852: scripted-diff: Use inline const(expr) over static constexpr in headers

fab74a0e922c6eb6501c3852ae08a24aade3850d refactor: Use C++14 digit separator for large int literals (MarcoFalke)
fae759be793043565f93957d150e482226c8ebb7 scripted-diff: Use inline constexpr over plain constexpr (MarcoFalke)
fa74f58a262096f25d5a3a4ec4951f4ce2a31792 scripted-diff: Use inline const over (static) const (MarcoFalke)
fab1a62c870e5bac92ee89b8444fd5910dfd5034 refactor: Use inline constexpr for string literals in headers (MarcoFalke)
fa08bbed8dd2e31ff1d6c6f015063b07008701fc contrib: Adjust generate-seeds.py to write inline constexpr (MarcoFalke)
fad753611b5c074f38120fd9c1a87e37cc44bf6a scripted-diff: Use inline constexpr over (static) const (MarcoFalke)
faedb52583e1c1eb6feb9cc03f4caf0bf52a8555 refactor: Make CFeeRate(integral) ctor constexpr (MarcoFalke)
5555d5dcb55969f6b5eefcf8733b628b2f36f8c7 scripted-diff: Use inline constexpr over static constexpr (MarcoFalke)
fa6e1a1e85e2e3ee6ec8fedfb5ee27f1b5bf13c4 refactor: Remove static from constexpr functions in headers (MarcoFalke)

Pull request description:

Both are fine and this refactor doesn't change any behavior.

However, `inline constexpr` from C++17 will ensure each symbol has a single address
across all TU, making the release binary minimally smaller. (For me it is smaller by about 1kB)

ACKs for top commit:
l0rinc:
reACK fab74a0e922c6eb6501c3852ae08a24aade3850d
rustaceanrob:
ACK fab74a0e922c6eb6501c3852ae08a24aade3850d
hebasto:
ACK fab74a0e922c6eb6501c3852ae08a24aade3850d, I have reviewed the code and it looks OK.

Tree-SHA512: 6ec94136c12bcbf696812d0661c9857318a69e367c79fc00b9ca0b4068f269d10e5548d95c9ba2070225308c12d7a54fe8cb8447de7e0979cba99f48892b35f9

Merge bitcoin/bitcoin#35960: common: remove `::runtime_error` from `RunCommandParseJSON`

8b5da677d7b0df5ef00b503b506bbfb68f9d436d common: remove ::runtime_error from RunCommandParseJSON (fanquake)

Pull request description:

I don't think there's a code path that can reach `RunCommandParseJSON` if we compile with `ENABLE_EXTERNAL_SIGNER=OFF`. If there is a reason for having the code this way, it could be good to document.

This also requires more workarounds in #35911.

ACKs for top commit:
stickies-v:
re-ACK 8b5da677d7b0df5ef00b503b506bbfb68f9d436d
sedited:
ACK 8b5da677d7b0df5ef00b503b506bbfb68f9d436d
willcl-ark:
ACK 8b5da677d7b0df5ef00b503b506bbfb68f9d436d

Tree-SHA512: b0c50372fed35afe47713310851f0b58cd1803fbe87a3a5a75877772172a3881283394a91663251de22a9972f56b46d84ddc868686dec8b970474cfaf5dc0d32

Merge bitcoin/bitcoin#35847: test: move more tests to `baseindex_tests` and run them for all indexes

34c03075a5a01ced1211d33362b555edf43bbfd7 test: run generic baseindex tests against every index type (Martin Zumsande)
11b3e251c488a5239ed8b527ef17dabee0fee6b4 test: make baseindex flush test chain-length agnostic (Martin Zumsande)
8b959f4c6ab6e03b128828c9ca56be372f218e8e test: move unclean_shutdown test to baseindex_tests (Martin Zumsande)
2232d6afbe2179b4c903995146161f9aa3553238 test: move index_reorg_crash to baseindex_tests (Martin Zumsande)
a3597e268376bb16bfd23d92c40d7d261bf7f5a5 test: move BuildChain helper into test mining util (Martin Zumsande)
954985e6a3807e4b7fd35cca3e94e869c728ef26 test: simplify blockfilter test's BuildChain helper (Martin Zumsande)

Pull request description:

In #34897, the `baseindex_tests` unit test was introduced, meant for tests that test basic index functionality (e.g. reorg or unclean shutdown behavior) that should work regardless of the particular index type.
This PR moves two more of these tests (`index_reorg_crash`, `coinstatsindex_unclean_shutdown`) from test files of specific indexes into that folder.
In the second part, tests are executed sequentially for all index types instead of just one particular one, where applicable.

Before moving `index_reorg_crash` I extracted the `BuildChain` helper to `util/mining` so that it can be used by multiple tests. While doing that, I simplified the helper a bit.

ACKs for top commit:
jeanpablojp:
tACK 34c03075a5a01ced1211d33362b555edf43bbfd7
sedited:
ACK 34c03075a5a01ced1211d33362b555edf43bbfd7

Tree-SHA512: 1d7a43160a9b7ec3c75a8c806967f2031da4855fe449c9c8aac8e44b1940e5ee28fde9473406666e74a682cf135b87f8c0eb9ddba50fce156dce9fc54a7763eb

Replace CTransaction::operator== with Equals that has options

CTransaction::operator== is only used in a few places. In a few
instances of checking transaction equality, we want to control which
fields are actually being compared, so use a custom Equals() function
which takes a EqualsOptions struct to control the checks.

As suggested in https://github.com/bitcoin/bitcoin/pull/32723#issuecomment-3028112892

Co-Authored-By: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>