← Watch feed
wallet: cache paid-invoice keys to avoid prevout rescan
Public commit record
85/100 · Strong
What the developer wrote
Authored by Sasha Zykov
wallet: cache paid-invoice keys to avoid prevout rescan
On wallets with many outgoing invoices that share output scriptpubkeys,
broadcasting a new transaction froze the GUI for several seconds — up
to a minute on bigger wallets. The hot loops in set_broadcasting() and
_update_onchain_invoice_paid_detection() iterate every invoice touched
by the new tx's outputs, which is large when scriptpubkeys are shared
via _invoices_from_scriptpubkey_map. For each touched invoice they call
_is_onchain_invoice_paid(), which scans all of the invoice's output
scripthashes against _prevouts_by_scripthash and does a get_tx_height
per prevout. Both inner dimensions blow up together when invoices share
inputs/outputs across many past payments.
Add _paid_invoice_keys: Set[str], an in-memory cache of outgoing
invoice ids known to be PR_PAID, maintained incrementally:
- populated at wallet load by _prepare_onchain_invoice_paid_detection
- updated in _update_onchain_invoice_paid_detection (discard then
recompute, so reorgs still demote PAID->UNPAID)
- updated in save_invoice, delete_invoice, clear_invoices
- new on_event_invoice_status listener keeps it in sync with
LN-driven transitions from LNWallet.set_invoice_status
get_invoice_status() short-circuits to PR_PAID on cache hit, skipping
the prevout scan. set_broadcasting() skips already-paid invoices
entirely — broadcasting_status has no effect on a paid invoice and the
callback churn is what made the GUI freeze.
✓ Specific, descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context✓ Explains rationale or failure mode
What changed, and why it matters
This commit is a performance optimization, not a security fix. It adds an in-memory cache of which outgoing invoices are already paid, so Electrum can skip expensive re-scanning of previous transaction outputs when many invoices share the same Bitcoin address. The goal is to stop the user interface from freezing for seconds or minutes when broadcasting a transaction. There is no indication this change fixes a vulnerability or changes security-critical behavior beyond making status lookups faster.
Recommended action
No security action required. Treat as a normal performance improvement. Reviewers may want to confirm that all cache-invalidation paths (reorg, history clear, LN status transitions, invoice deletion) are covered, which the included tests appear to do.
Security signals we found
01
No security-relevant signal: the change is a performance optimization with explicit cache-invalidation paths for reorgs and history clears.
02
Cache consistency is maintained across save_invoice, delete_invoice, clear_invoices, clear_history, and LN-driven invoice_status events.
03
Tests verify that reorgs and clear_history correctly demote cached PR_PAID status, reducing risk of stale-state bugs.
Evidence from the diff
The patch introduces _paid_invoice_keys, a Set[str] cache of outgoing invoice IDs known to be PR_PAID. It is populated at wallet load, updated on invoice save/delete/clear, kept in sync with Lightning status changes via on_event_invoice_status, and deliberately discarded/recomputed in _update_onchain_invoice_paid_detection so blockchain reorgs can demote PR_PAID back to unpaid. get_invoice_status() short-circuits to PR_PAID on cache hit, and set_broadcasting() skips already-paid invoices to avoid callback churn. The change is purely a cache layer over existing status logic; correctness invariants (reorg handling, history clear, LN status sync) are explicitly addressed and tested.
Changed components
electrum/wallet.pytests/test_invoices.pyInspect captured patch +269 / −5
diff --git a/electrum/wallet.py b/electrum/wallet.py
index 70573de..bdfcf36 100644
--- a/electrum/wallet.py
+++ b/electrum/wallet.py
@@ -445,6 +445,8 @@ class Abstract_Wallet(ABC, Logger, EventListener):
self.txbatcher = TxBatcher(self)
self._init_lnworker()
self._init_requests_rhash_index()
+ # cache of outgoing-invoice ids known to be PR_PAID, to skip the prevout scan
+ self._paid_invoice_keys = set() # type: Set[str]
self._prepare_onchain_invoice_paid_detection()
self._calc_unused_change_addresses()
# save wallet type the first time
@@ -656,8 +658,20 @@ class Abstract_Wallet(ABC, Logger, EventListener):
return
self._update_invoices_and_reqs_touched_by_tx(tx_hash)
+ @event_listener
+ def on_event_invoice_status(self, wallet, key, status):
+ # keep _paid_invoice_keys in sync with LN-driven status changes
+ if wallet != self:
+ return
+ if status == PR_PAID:
+ self._paid_invoice_keys.add(key)
+ else:
+ self._paid_invoice_keys.discard(key)
+
def clear_history(self):
self.adb.clear_history()
+ # the paid-keys cache is derived from on-chain prevouts that adb.clear_history just wiped
+ self._paid_invoice_keys.clear()
self.save_db()
def start_network(self, network: 'Network'):
@@ -1278,18 +1292,29 @@ class Abstract_Wallet(ABC, Logger, EventListener):
def save_invoice(self, invoice: Invoice, *, write_to_disk: bool = True) -> None:
key = invoice.get_id()
- if not invoice.is_lightning():
- if self.is_onchain_invoice_paid(invoice)[0]:
+ is_paid = False
+ if invoice.is_lightning():
+ if self.lnworker and self.lnworker.get_invoice_status(invoice) == PR_PAID:
+ is_paid = True
+ else:
+ is_paid_onchain, conf = self.is_onchain_invoice_paid(invoice)
+ if is_paid_onchain:
_logger.info("saving invoice... but it is already paid!")
with self.lock:
for txout in invoice.get_outputs():
self._invoices_from_scriptpubkey_map[txout.scriptpubkey].add(key)
+ is_paid = is_paid_onchain and conf is not None and conf >= 1
self._invoices[key] = invoice
+ if is_paid:
+ self._paid_invoice_keys.add(key)
+ else:
+ self._paid_invoice_keys.discard(key)
if write_to_disk:
self.save_db()
def clear_invoices(self):
self._invoices.clear()
+ self._paid_invoice_keys.clear()
self.save_db()
def clear_requests(self):
@@ -1363,10 +1388,17 @@ class Abstract_Wallet(ABC, Logger, EventListener):
for invoice_key in invoice_keys:
invoice = self._invoices.get(invoice_key)
if not invoice:
+ self._paid_invoice_keys.discard(invoice_key)
continue
+ # clear the cache first so get_invoice_status takes the slow path,
+ # which is needed to detect paid->unpaid transitions (e.g. reorgs)
+ self._paid_invoice_keys.discard(invoice_key)
if invoice.is_lightning() and not invoice.get_address():
+ if self.lnworker and self.lnworker.get_invoice_status(invoice) == PR_PAID:
+ self._paid_invoice_keys.add(invoice_key)
continue
if invoice.is_lightning() and self.lnworker and self.lnworker.get_invoice_status(invoice) == PR_PAID:
+ self._paid_invoice_keys.add(invoice_key)
continue
is_paid, conf_needed, relevant_txs = self._is_onchain_invoice_paid(invoice)
if is_paid:
@@ -1374,8 +1406,17 @@ class Abstract_Wallet(ABC, Logger, EventListener):
self._invoices_from_txid_map[txid].add(invoice_key)
for txout in invoice.get_outputs():
self._invoices_from_scriptpubkey_map[txout.scriptpubkey].add(invoice_key)
- # update invoice status
- status = self.get_invoice_status(invoice)
+ # derive status from the values just computed; avoids a second prevout scan
+ if not is_paid:
+ status = invoice.get_broadcasting_status() or PR_UNPAID
+ elif conf_needed == 0:
+ status = PR_UNCONFIRMED
+ else:
+ assert conf_needed is not None and conf_needed >= 1, conf_needed
+ status = PR_PAID
+ status = self.check_expired_status(invoice, status)
+ if status == PR_PAID:
+ self._paid_invoice_keys.add(invoice_key)
util.trigger_callback('invoice_status', self, invoice_key, status)
def _is_onchain_invoice_paid(self, invoice: BaseInvoice) -> Tuple[bool, Optional[int], Sequence[str]]:
@@ -2869,6 +2910,9 @@ class Abstract_Wallet(ABC, Logger, EventListener):
def get_invoice_status(self, invoice: BaseInvoice):
"""Returns status of (incoming) request or (outgoing) invoice."""
+ # PR_PAID is terminal (reorgs clear the cache in _update_onchain_invoice_paid_detection)
+ if isinstance(invoice, Invoice) and invoice.get_id() in self._paid_invoice_keys:
+ return PR_PAID
# lightning invoices can be paid onchain
if invoice.is_lightning() and self.lnworker:
status = self.lnworker.get_invoice_status(invoice)
@@ -2999,6 +3043,9 @@ class Abstract_Wallet(ABC, Logger, EventListener):
def set_broadcasting(self, tx: Transaction, *, broadcasting_status: Optional[int]):
request_keys, invoice_keys = self.get_invoices_and_requests_touched_by_tx(tx)
for key in invoice_keys:
+ if key in self._paid_invoice_keys:
+ # already-paid invoices ignore _broadcasting_status; skip the prevout scan
+ continue
invoice = self._invoices.get(key)
if not invoice:
continue
@@ -3081,6 +3128,7 @@ class Abstract_Wallet(ABC, Logger, EventListener):
inv = self._invoices.pop(invoice_id, None)
if inv is None:
return
+ self._paid_invoice_keys.discard(invoice_id)
if inv.is_lightning() and self.lnworker:
self.lnworker.delete_payment_info(inv.rhash, direction=SENT)
if write_to_disk:
diff --git a/tests/test_invoices.py b/tests/test_invoices.py
index 89a7fda..e91903b 100644
--- a/tests/test_invoices.py
+++ b/tests/test_invoices.py
@@ -1,9 +1,10 @@
import os
import time
+from electrum import util
from electrum.simple_config import SimpleConfig
from electrum.wallet import Standard_Wallet, Abstract_Wallet
-from electrum.invoices import PR_UNPAID, PR_PAID, PR_UNCONFIRMED, BaseInvoice, Invoice, LN_EXPIRY_NEVER
+from electrum.invoices import PR_UNPAID, PR_PAID, PR_UNCONFIRMED, PR_BROADCASTING, BaseInvoice, Invoice, LN_EXPIRY_NEVER
from electrum.address_synchronizer import TX_HEIGHT_UNCONFIRMED
from electrum.transaction import Transaction, PartialTxOutput
from electrum.util import TxMinedInfo, InvoiceError
@@ -255,3 +256,218 @@ class TestBaseInvoice(ElectrumTestCase):
with self.assertRaises(TypeError):
invoice.exp = "asd"
+
+class TestOutgoingInvoicesPaidCache(ElectrumTestCase):
+ """test caching of paid-status for outgoing invoices"""
+ TESTNET = True
+
+ def setUp(self):
+ super().setUp()
+ self.config = SimpleConfig({'electrum_path': self.electrum_path})
+ self.wallet_path = os.path.join(self.electrum_path, "outgoing_inv_wallet")
+
+ def _make_wallet(self):
+ # Seed matches the funding tx output below (see TestWalletPaymentRequests.create_wallet2).
+ text = 'cross end slow expose giraffe fuel track awake turtle capital ranch pulp'
+ d = restore_wallet_from_text__for_unittest(text, path=self.wallet_path, config=self.config)
+ wallet = d['wallet'] # type: Standard_Wallet
+ funding_tx = Transaction('0200000000010132515e6aade1b79ec7dd3bac0896d8b32c56195d23d07d48e21659cef24301560100000000fdffffff0112841e000000000016001477fe6d2a27e8860c278d4d2cd90bad716bb9521a02473044022041ed68ef7ef122813ac6a5e996b8284f645c53fbe6823b8e430604a8915a867802203233f5f4d347a687eb19b2aa570829ab12aeeb29a24cc6d6d20b8b3d79e971ae012102bee0ee043817e50ac1bb31132770f7c41e35946ccdcb771750fb9696bdd1b307ad951d00')
+ wallet.adb.receive_tx_callback(funding_tx, tx_height=TX_HEIGHT_UNCONFIRMED)
+ return wallet
+
+ def _make_outgoing_invoice(self, dest_addr: str, amount_sat: int, *, t: int = 1700000000):
+ outputs = [PartialTxOutput.from_address_and_value(dest_addr, amount_sat)]
+ return Invoice(
+ amount_msat=amount_sat * 1000,
+ message="",
+ time=t,
+ exp=LN_EXPIRY_NEVER,
+ outputs=outputs,
+ height=0,
+ lightning_invoice=None,
+ )
+
+ async def test_paid_keys_empty_for_fresh_unpaid_invoice(self):
+ wallet = self._make_wallet()
+ inv = self._make_outgoing_invoice("tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385", 5_000)
+ wallet.save_invoice(inv, write_to_disk=False)
+ self.assertNotIn(inv.get_id(), wallet._paid_invoice_keys)
+ self.assertEqual(PR_UNPAID, wallet.get_invoice_status(inv))
+ self.assertEqual([inv], wallet.get_unpaid_invoices())
+
+ async def test_paid_keys_populated_when_invoice_gets_paid(self):
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ amount_sat = 5_000
+ inv = self._make_outgoing_invoice(dest, amount_sat)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Pay the invoice with a confirmed outgoing tx so it becomes PR_PAID.
+ outputs = [PartialTxOutput.from_address_and_value(dest, amount_sat)]
+ tx = wallet.make_unsigned_transaction(outputs=outputs, fee_policy=FixedFeePolicy(1000))
+ wallet.sign_transaction(tx, password=None)
+ wallet.adb.receive_tx_callback(tx, tx_height=TX_HEIGHT_UNCONFIRMED)
+ wallet.db.put('stored_height', 1010)
+ wallet.adb.add_verified_tx(tx.txid(), TxMinedInfo(_height=1001, timestamp=1700000001, txpos=1, header_hash="01"*32))
+ self.assertEqual(PR_PAID, wallet.get_invoice_status(inv))
+ self.assertIn(inv.get_id(), wallet._paid_invoice_keys)
+ self.assertEqual([], wallet.get_unpaid_invoices())
+
+ async def test_paid_keys_removed_on_delete_and_clear(self):
+ wallet = self._make_wallet()
+ inv = self._make_outgoing_invoice("tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385", 5_000)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Force into the cache via the internal hook so we don't depend on the slow path here.
+ wallet._paid_invoice_keys.add(inv.get_id())
+ wallet.delete_invoice(inv.get_id(), write_to_disk=False)
+ self.assertNotIn(inv.get_id(), wallet._paid_invoice_keys)
+ # Re-add and clear all
+ wallet.save_invoice(inv, write_to_disk=False)
+ wallet._paid_invoice_keys.add(inv.get_id())
+ wallet.clear_invoices()
+ self.assertEqual(set(), wallet._paid_invoice_keys)
+
+ async def test_get_invoice_status_short_circuits_on_cache_hit(self):
+ wallet = self._make_wallet()
+ inv = self._make_outgoing_invoice("tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385", 5_000)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Seed the cache and assert the slow path is not taken.
+ wallet._paid_invoice_keys.add(inv.get_id())
+ called = []
+ orig = wallet._is_onchain_invoice_paid
+ def spy(invoice):
+ called.append(invoice.get_id())
+ return orig(invoice)
+ wallet._is_onchain_invoice_paid = spy
+ self.assertEqual(PR_PAID, wallet.get_invoice_status(inv))
+ self.assertEqual([], called, "cache hit must avoid _is_onchain_invoice_paid")
+
+ async def test_set_broadcasting_skips_already_paid_invoices(self):
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ # Two invoices share the same output scriptpubkey but only one is unpaid.
+ inv_paid = self._make_outgoing_invoice(dest, 5_000, t=1700000000)
+ inv_unpaid = self._make_outgoing_invoice(dest, 5_000, t=1700000005)
+ wallet.save_invoice(inv_paid, write_to_disk=False)
+ wallet.save_invoice(inv_unpaid, write_to_disk=False)
+ wallet._paid_invoice_keys.add(inv_paid.get_id())
+
+ events = []
+ def on_status(w, key, status):
+ events.append((key, status))
+ util.register_callback(on_status, ['invoice_status'])
+ try:
+ # Build a tx whose output matches the shared scriptpubkey.
+ tx = wallet.make_unsigned_transaction(
+ outputs=[PartialTxOutput.from_address_and_value(dest, 5_000)],
+ fee_policy=FixedFeePolicy(1000),
+ )
+ wallet.set_broadcasting(tx, broadcasting_status=PR_BROADCASTING)
+ finally:
+ util.unregister_callback(on_status)
+
+ notified_keys = {key for key, _ in events}
+ self.assertIn(inv_unpaid.get_id(), notified_keys)
+ self.assertNotIn(inv_paid.get_id(), notified_keys,
+ "invoice_status callback should not fire for already-paid invoices")
+
+ async def test_set_broadcasting_does_not_rescan_paid_invoices_with_shared_outputs(self):
+ """Regression for the freeze scenario: many already-paid invoices share an
+ output scriptpubkey, so a new tx paying that scriptpubkey touches all of them.
+ set_broadcasting must not call _is_onchain_invoice_paid for the paid ones."""
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ # Many paid invoices, all sharing the same destination scriptpubkey.
+ paid_ids = set()
+ for i in range(50):
+ inv = self._make_outgoing_invoice(dest, 1_000 + i, t=1700000000 + i)
+ wallet.save_invoice(inv, write_to_disk=False)
+ paid_ids.add(inv.get_id())
+ wallet._paid_invoice_keys.add(inv.get_id())
+ # One fresh unpaid invoice with the same destination.
+ unpaid = self._make_outgoing_invoice(dest, 9_999, t=1700001000)
+ wallet.save_invoice(unpaid, write_to_disk=False)
+
+ # Spy on the slow path.
+ called = []
+ orig = wallet._is_onchain_invoice_paid
+ def spy(invoice):
+ called.append(invoice.get_id())
+ return orig(invoice)
+ wallet._is_onchain_invoice_paid = spy
+
+ tx = wallet.make_unsigned_transaction(
+ outputs=[PartialTxOutput.from_address_and_value(dest, 1_000)],
+ fee_policy=FixedFeePolicy(1000),
+ )
+ wallet.set_broadcasting(tx, broadcasting_status=PR_BROADCASTING)
+
+ # The slow path may run for the unpaid invoice (via get_invoice_status),
+ # but must NOT run for any of the cached-paid ones.
+ called_set = set(called)
+ self.assertFalse(called_set & paid_ids,
+ f"slow path ran for paid invoices: {called_set & paid_ids}")
+
+ async def test_paid_keys_populated_on_wallet_load(self):
+ """After the initial _prepare_onchain_invoice_paid_detection pass, the cache
+ should reflect what _is_onchain_invoice_paid found, so that subsequent
+ get_invoice_status calls take the fast path."""
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ amount_sat = 5_000
+ inv = self._make_outgoing_invoice(dest, amount_sat)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Pay it (confirmed)
+ outputs = [PartialTxOutput.from_address_and_value(dest, amount_sat)]
+ tx = wallet.make_unsigned_transaction(outputs=outputs, fee_policy=FixedFeePolicy(1000))
+ wallet.sign_transaction(tx, password=None)
+ wallet.adb.receive_tx_callback(tx, tx_height=TX_HEIGHT_UNCONFIRMED)
+ wallet.db.put('stored_height', 1010)
+ wallet.adb.add_verified_tx(tx.txid(), TxMinedInfo(_height=1001, timestamp=1700000001, txpos=1, header_hash="01"*32))
+ # Force a rebuild of the cache as would happen at wallet load.
+ wallet._paid_invoice_keys.clear()
+ wallet._prepare_onchain_invoice_paid_detection()
+ self.assertIn(inv.get_id(), wallet._paid_invoice_keys)
+
+ async def test_paid_keys_demoted_on_reorg(self):
+ """A reorg unverifying the paying tx must remove the invoice from the cache,
+ otherwise get_invoice_status would keep returning a stale PR_PAID."""
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ amount_sat = 5_000
+ inv = self._make_outgoing_invoice(dest, amount_sat)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Pay and confirm.
+ outputs = [PartialTxOutput.from_address_and_value(dest, amount_sat)]
+ tx = wallet.make_unsigned_transaction(outputs=outputs, fee_policy=FixedFeePolicy(1000))
+ wallet.sign_transaction(tx, password=None)
+ wallet.adb.receive_tx_callback(tx, tx_height=TX_HEIGHT_UNCONFIRMED)
+ wallet.db.put('stored_height', 1010)
+ wallet.adb.add_verified_tx(tx.txid(), TxMinedInfo(_height=1001, timestamp=1700000001, txpos=1, header_hash="01"*32))
+ self.assertEqual(PR_PAID, wallet.get_invoice_status(inv))
+ self.assertIn(inv.get_id(), wallet._paid_invoice_keys)
+ # Simulate reorg: unverify the tx and fire the same event the verifier would.
+ wallet.adb.db.remove_verified_tx(tx.txid())
+ wallet.on_event_adb_removed_verified_tx(wallet.adb, tx.txid())
+ self.assertNotIn(inv.get_id(), wallet._paid_invoice_keys)
+ self.assertNotEqual(PR_PAID, wallet.get_invoice_status(inv))
+
+ async def test_clear_history_resets_paid_keys(self):
+ """clear_history() wipes _prevouts_by_scripthash; the cache must follow,
+ otherwise get_invoice_status would keep returning a stale PR_PAID."""
+ wallet = self._make_wallet()
+ dest = "tb1qmjzmg8nd4z56ar4fpngzsr6euktrhnjg9td385"
+ amount_sat = 5_000
+ inv = self._make_outgoing_invoice(dest, amount_sat)
+ wallet.save_invoice(inv, write_to_disk=False)
+ # Pay and confirm.
+ outputs = [PartialTxOutput.from_address_and_value(dest, amount_sat)]
+ tx = wallet.make_unsigned_transaction(outputs=outputs, fee_policy=FixedFeePolicy(1000))
+ wallet.sign_transaction(tx, password=None)
+ wallet.adb.receive_tx_callback(tx, tx_height=TX_HEIGHT_UNCONFIRMED)
+ wallet.db.put('stored_height', 1010)
+ wallet.adb.add_verified_tx(tx.txid(), TxMinedInfo(_height=1001, timestamp=1700000001, txpos=1, header_hash="01"*32))
+ self.assertIn(inv.get_id(), wallet._paid_invoice_keys)
+ # Wipe history.
+ wallet.clear_history()
+ self.assertNotIn(inv.get_id(), wallet._paid_invoice_keys)
+ self.assertNotEqual(PR_PAID, wallet.get_invoice_status(inv))
Risk score
Our methodology →Why this scored 19/100
Human-validated context
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
No validated notes yet.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.