save silent payments found by a scan when fetching their history fails, and load wallets whose silent payment indexes are not contiguous
What changed, and why it matters
This commit fixes two related bugs in Sparrow Wallet's handling of silent payments. Silent payments are a privacy-preserving way to receive Bitcoin where each payment uses a unique address derived by the sender. The wallet scans the blockchain to find payments made to it. The first bug meant that if the wallet found silent payments during a scan but then failed to fetch their transaction history, those newly found payments could be lost instead of saved. The second bug meant that wallets with silent payment indexes that were not sequential (which is normal for silent payments) could fail to load or refresh properly because the code tried to fill in missing indexes as if they were ordinary derived addresses. The fix ensures found silent payments are saved even when history fetching fails, and that the wallet does not try to force silent payment indexes to be contiguous.
Users relying on silent payments should upgrade to a release containing this commit. Until upgraded, be aware that silent payment transactions discovered during scans that encounter server or network errors may not be persisted, and wallets with non-contiguous silent payment indexes may fail to refresh correctly. No immediate key-rotation or incident response is indicated because no key material is compromised.
Security signals we found
Data loss / state inconsistency: silent payment outputs discovered by scan could be discarded if subsequent history fetch failed or was skipped
Denial of wallet load / refresh: non-contiguous silent payment indexes triggered incorrect gap-limit filling behavior
Fix ensures wallet state changes made before a failure are persisted without advancing stored block height
No cryptographic, authentication, or network protocol changes; this is a wallet state management bug
Evidence from the diff
The patch modifies ElectrumServer.getHistoryToGapLimit to break out of the gap-limit filling loop when fillToIndex returns empty, which occurs for silent payment (SP) purpose nodes because their children are added by scan discovery rather than deterministic derivation. Previously the loop would continue indefinitely or behave incorrectly for non-contiguous SP indexes. It also modifies WalletForm.refreshHistorySP and WalletForm.handleHistoryFailed to call updateWallets(null, previousWallet) when a scan succeeds but history fetch is skipped, or when history fetch fails, so that silent payment child nodes added to the wallet before the history fetch are persisted. The block height is intentionally left unchanged (null passed) so the next scan resumes from the same starting point. Tests are added to verify non-contiguous SP node history fetching, deterministic wallet gap-limit behavior, and persistence of nodes added by a failed refresh.
Changed components
src/main/java/com/sparrowwallet/sparrow/net/ElectrumServer.javasrc/main/java/com/sparrowwallet/sparrow/wallet/WalletForm.javaSilent payments scanning and history fetchingWallet refresh and persistence logicInspect captured patch +153 / −2
### src/main/java/com/sparrowwallet/sparrow/net/ElectrumServer.java
@@ -752,7 +752,11 @@ private void getHistoryToGapLimit(Wallet wallet, Map<WalletNode, Set<BlockTransa
//The gap limit size takes the highest used index in the retrieved history and adds the gap limit (plus one to be comparable to the number of children since index is zero based)
int gapLimitSize = getGapLimitSize(wallet, nodeTransactionMap, purposeNode);
while(historySize < gapLimitSize) {
- purposeNode.fillToIndex(wallet, gapLimitSize - 1);
+ //Silent payments nodes are added as payments are found rather than derived, so their indexes need not be contiguous and none can be added here
+ if(purposeNode.fillToIndex(wallet, gapLimitSize - 1).isEmpty()) {
+ break;
+ }
+
subscribeWalletNodes(wallet, getAddressNodes(wallet, purposeNode), nodeTransactionMap, historySize);
getReferences(wallet, nodeTransactionMap.keySet(), nodeTransactionMap, historySize);
getReferencedTransactions(wallet, nodeTransactionMap);
### src/main/java/com/sparrowwallet/sparrow/wallet/WalletForm.java
@@ -246,6 +246,9 @@ private void refreshHistorySP(Wallet previousWallet, Integer blockHeight) {
if(scanService.getValue()) {
EventManager.get().post(new WalletHistoryFinishedEvent(wallet));
updateWallets(blockHeight, previousWallet);
+ } else {
+ //The payments a scan finds are added to the wallet before their history is fetched, and must be saved even where that fetch is not made
+ updateWallets(null, previousWallet);
}
firePendingRefreshIfRequested();
});
@@ -275,7 +278,7 @@ private void firePendingRefreshIfRequested() {
}
}
- private void handleHistoryFailed(Wallet previousWallet, Throwable exception) {
+ void handleHistoryFailed(Wallet previousWallet, Throwable exception) {
if(exception instanceof AllHistoryChangedException) {
if(getWallet().isMasterWallet() && getWallet().getKeystores().stream().anyMatch(Keystore::needsPassphrase)) {
Optional<ButtonType> optType = AppServices.showWarningDialog("Reopen " + getWallet().getMasterName() + "?",
@@ -309,6 +312,10 @@ private void handleHistoryFailed(Wallet previousWallet, Throwable exception) {
}
EventManager.get().post(new WalletHistoryFailedEvent(wallet, exception));
+
+ //A refresh can change the wallet before failing, as a silent payments scan does in adding the payments it finds before fetching their history.
+ //The next refresh copies a wallet already holding those changes and would not report them, so they are saved here, without advancing the block height
+ updateWallets(null, previousWallet);
}
}
### src/test/java/com/sparrowwallet/sparrow/net/SilentPaymentsBatchTest.java
@@ -2,6 +2,7 @@
import com.github.arteam.simplejsonrpc.client.Transport;
import com.google.common.net.HostAndPort;
+import com.sparrowwallet.drongo.ExtendedKey;
import com.sparrowwallet.drongo.KeyDerivation;
import com.sparrowwallet.drongo.KeyPurpose;
import com.sparrowwallet.drongo.Network;
@@ -17,6 +18,7 @@
import com.sparrowwallet.drongo.silentpayments.SilentPayment;
import com.sparrowwallet.drongo.silentpayments.SilentPaymentScanAddress;
import com.sparrowwallet.drongo.silentpayments.SilentPaymentUtils;
+import com.sparrowwallet.drongo.wallet.BlockTransactionHash;
import com.sparrowwallet.drongo.wallet.BlockTransactionHashIndex;
import com.sparrowwallet.drongo.wallet.Keystore;
import com.sparrowwallet.drongo.wallet.Wallet;
@@ -31,13 +33,16 @@
import java.math.BigInteger;
import java.nio.file.Path;
+import java.time.Duration;
+import java.util.HashSet;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTimeoutPreemptively;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
@@ -54,6 +59,7 @@ public class SilentPaymentsBatchTest {
private CloseableTransport previousTransport;
private ServerCapability previousServerCapability;
private final Map<String, Transaction> served = new LinkedHashMap<>();
+ private final Set<String> usedScriptHashes = new HashSet<>();
@BeforeAll
public static void setUpAll() {
@@ -80,6 +86,20 @@ public Map<String, String> getTransactions(Transport transport, Wallet wallet, S
txids.forEach(txid -> result.put(txid, served.containsKey(txid) ? Utils.bytesToHex(served.get(txid).bitcoinSerialize()) : Sha256Hash.ZERO_HASH.toString()));
return result;
}
+
+ @Override
+ public Map<String, String> subscribeScriptHashes(Transport transport, Wallet wallet, Map<String, String> pathScriptHashes) {
+ Map<String, String> result = new LinkedHashMap<>();
+ pathScriptHashes.forEach((path, scriptHash) -> result.put(path, usedScriptHashes.contains(scriptHash) ? Sha256Hash.of(scriptHash.getBytes()).toString() : null));
+ return result;
+ }
+
+ @Override
+ public Map<String, ScriptHashTx[]> getScriptHashHistory(Transport transport, Wallet wallet, Map<String, String> pathScriptHashes, boolean failOnError) {
+ Map<String, ScriptHashTx[]> result = new LinkedHashMap<>();
+ pathScriptHashes.keySet().forEach(path -> result.put(path, new ScriptHashTx[0]));
+ return result;
+ }
};
}
@@ -152,6 +172,34 @@ public void fetchesAgainATransactionTheWalletHoldsAsUnfetchable() throws Excepti
assertEquals(10000L, walletBalance(wallet));
}
+ /**
+ * Silent payments nodes are not derived, so the wallet may hold a node without those below it. Fetching its history does not try to fill to it.
+ */
+ @Test
+ public void fetchesTheHistoryOfNodesWithoutContiguousIndexes() {
+ Wallet wallet = silentPaymentsWallet();
+ WalletNode node = wallet.getNode(KeyPurpose.RECEIVE).addSilentPaymentChild(wallet, 1, Utils.hexToBytes("c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"));
+ usedScriptHashes.add(ElectrumServer.getScriptHash(node));
+
+ Map<WalletNode, Set<BlockTransactionHash>> history = assertTimeoutPreemptively(Duration.ofSeconds(10), () -> new ElectrumServer().getHistory(wallet));
+ assertTrue(history.containsKey(node));
+ assertEquals(1, wallet.getNode(KeyPurpose.RECEIVE).getChildren().size());
+ }
+
+ /**
+ * A deterministic wallet still derives and fetches nodes to the gap limit past the highest used one.
+ */
+ @Test
+ public void fetchesTheHistoryOfDeterministicNodesToTheGapLimit() {
+ Wallet wallet = deterministicWallet();
+ WalletNode receiveNode = wallet.getNode(KeyPurpose.RECEIVE);
+ receiveNode.fillToIndex(wallet, 1);
+ usedScriptHashes.add(ElectrumServer.getScriptHash(receiveNode.getChildren().stream().filter(node -> node.getIndex() == 1).findFirst().orElseThrow()));
+
+ assertTimeoutPreemptively(Duration.ofSeconds(10), () -> new ElectrumServer().getHistory(wallet));
+ assertEquals(2 + wallet.getGapLimit(), receiveNode.getChildren().size());
+ }
+
/**
* Builds a payment to the wallet's silent payment address, which the server then holds in its mempool, returning the entry a subscription reports for it.
*/
@@ -189,4 +237,16 @@ private static Wallet silentPaymentsWallet() {
wallet.setDefaultPolicy(Policy.getPolicy(PolicyType.SINGLE_SP, ScriptType.P2TR, wallet.getKeystores(), 1));
return wallet;
}
+
+ private static Wallet deterministicWallet() {
+ Wallet wallet = new Wallet("deterministic");
+ wallet.setPolicyType(PolicyType.SINGLE_HD);
+ wallet.setScriptType(ScriptType.P2WPKH);
+ Keystore keystore = new Keystore();
+ keystore.setKeyDerivation(new KeyDerivation("00000000", "m/84'/1'/0'"));
+ keystore.setExtendedPublicKey(ExtendedKey.fromDescriptor("tpubD6NzVbkrYhZ4XzL5Dhayo67Gorv1YMS7j8pRUvVMd5odC2LBPLAygka9p7748JtSq82FNGPppFEz5xxZUdasBRCqJqXvUHq6xpnsMcYJzeh"));
+ wallet.getKeystores().add(keystore);
+ wallet.setDefaultPolicy(Policy.getPolicy(PolicyType.SINGLE_HD, ScriptType.P2WPKH, wallet.getKeystores(), 1));
+ return wallet;
+ }
}
### src/test/java/com/sparrowwallet/sparrow/wallet/WalletFormTest.java
@@ -1,18 +1,25 @@
package com.sparrowwallet.sparrow.wallet;
+import com.google.common.eventbus.Subscribe;
import com.sparrowwallet.drongo.ExtendedKey;
import com.sparrowwallet.drongo.KeyDerivation;
import com.sparrowwallet.drongo.KeyPurpose;
import com.sparrowwallet.drongo.Network;
+import com.sparrowwallet.drongo.Utils;
+import com.sparrowwallet.drongo.crypto.ECKey;
import com.sparrowwallet.drongo.policy.Policy;
import com.sparrowwallet.drongo.policy.PolicyType;
import com.sparrowwallet.drongo.protocol.ScriptType;
import com.sparrowwallet.drongo.protocol.Sha256Hash;
+import com.sparrowwallet.drongo.silentpayments.SilentPaymentScanAddress;
import com.sparrowwallet.drongo.wallet.BlockTransaction;
import com.sparrowwallet.drongo.wallet.BlockTransactionHashIndex;
import com.sparrowwallet.drongo.wallet.Keystore;
import com.sparrowwallet.drongo.wallet.Wallet;
import com.sparrowwallet.drongo.wallet.WalletNode;
+import com.sparrowwallet.sparrow.EventManager;
+import com.sparrowwallet.sparrow.event.WalletHistoryFailedEvent;
+import com.sparrowwallet.sparrow.net.ServerException;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -21,8 +28,10 @@
import java.util.Date;
import java.util.List;
import java.util.Map;
+import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
@@ -164,6 +173,38 @@ public void doesNotReportANodeTwice() {
assertEquals(List.of(node), changedNodes);
}
+ /**
+ * A silent payments scan adds the payments it finds to the wallet before fetching their history. Where that fetch fails, the nodes are reported
+ * against the copy taken before the refresh so that they are saved, since the next refresh copies a wallet already holding them. The stored block
+ * height, where the next scan starts, is left unchanged.
+ */
+ @Test
+ public void reportsTheNodesAddedByARefreshThatFailed() {
+ Wallet wallet = silentPaymentsWallet();
+ wallet.setStoredBlockHeight(HEIGHT - 1000);
+ WalletForm walletForm = new WalletForm(null, wallet);
+
+ Wallet previousWallet = wallet.copy();
+ wallet.updateTransactions(Map.of(TXID, blockTransaction(new Date(1600000000000L), ORPHANED_BLOCK)));
+ WalletNode node = wallet.getNode(KeyPurpose.RECEIVE).addSilentPaymentChild(wallet, 0, Utils.hexToBytes("c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"));
+ node.updateTransactionOutputs(wallet, Set.of(new BlockTransactionHashIndex(TXID, HEIGHT, new Date(1600000000000L), 0L, 0, 50000)));
+
+ HistoryFailedListener listener = new HistoryFailedListener();
+ EventManager.get().register(listener);
+ try {
+ assertTrue(reportsChanges(() -> walletForm.handleHistoryFailed(previousWallet, new ServerException("Failed to subscribe"))));
+ assertEquals(1, listener.events);
+ assertEquals(HEIGHT - 1000, wallet.getStoredBlockHeight());
+
+ //A refresh that changed nothing before failing reports the failure alone
+ Wallet unchangedWallet = wallet.copy();
+ assertFalse(reportsChanges(() -> walletForm.handleHistoryFailed(unchangedWallet, new ServerException("Failed to subscribe"))));
+ assertEquals(2, listener.events);
+ } finally {
+ EventManager.get().unregister(listener);
+ }
+ }
+
/**
* A labelled address with no history has already been given out, so the desktop and terminal receive views both pass over it rather than handing it to
* a second payer.
@@ -211,6 +252,29 @@ public String getWalletId() {
assertEquals(gapLimit + 6, wallet.getGapLimit());
}
+ /**
+ * Returns whether the wallet form reported changed nodes. They are posted through Platform.runLater, and no test here starts the JavaFX toolkit,
+ * so reaching that call is observable as its refusal.
+ */
+ private static boolean reportsChanges(Runnable refreshHandler) {
+ try {
+ refreshHandler.run();
+ return false;
+ } catch(IllegalStateException e) {
+ assertEquals("Toolkit not initialized", e.getMessage());
+ return true;
+ }
+ }
+
+ private static class HistoryFailedListener {
+ private int events;
+
+ @Subscribe
+ public void walletHistoryFailed(WalletHistoryFailedEvent event) {
+ events++;
+ }
+ }
+
private static NodeEntry entryAt(WalletForm walletForm, int index) {
Wallet wallet = walletForm.getWallet();
wallet.getNode(KeyPurpose.RECEIVE).fillToIndex(wallet, index);
@@ -225,6 +289,22 @@ private static WalletNode receiveNode(Wallet wallet, int index) {
return wallet.getNode(KeyPurpose.RECEIVE).getChildren().stream().filter(node -> node.getIndex() == index).findFirst().orElseThrow();
}
+ private static Wallet silentPaymentsWallet() {
+ ECKey scanKey = ECKey.fromPrivate(Utils.hexToBytes("a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2"));
+ ECKey spendKey = ECKey.fromPrivate(Utils.hexToBytes("b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3"));
+
+ Wallet wallet = new Wallet();
+ wallet.setPolicyType(PolicyType.SINGLE_SP);
+ wallet.setScriptType(ScriptType.P2TR);
+ Keystore keystore = new Keystore();
+ keystore.setSilentPaymentScanAddress(new SilentPaymentScanAddress(scanKey, ECKey.fromPublicOnly(spendKey)));
+ keystore.setKeyDerivation(new KeyDerivation("00000000", "m/352'/0'/0'"));
+ wallet.getKeystores().add(keystore);
+ wallet.setDefaultPolicy(Policy.getPolicy(PolicyType.SINGLE_SP, ScriptType.P2TR, wallet.getKeystores(), 1));
+
+ return wallet;
+ }
+
private static Wallet testWallet() {
Wallet wallet = new Wallet();
wallet.setPolicyType(PolicyType.SINGLE_HD);Why this scored 42/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.