coins: compact chainstate in background
What changed, and why it matters
This change moves Bitcoin Core's full chainstate database compaction from the main validation thread into a one-shot background thread named 'utxocompact'. The goal is to avoid blocking validation for minutes on large databases. The patch adds a destructor that waits for the background job to finish, a mutex to prevent the database from being resized while compaction is running, and catches exceptions so compaction failures don't crash the node. It is a performance and robustness improvement, not a fix for an active security vulnerability.
No immediate security action required. Treat as a normal performance/reliability improvement. Reviewers may want to confirm that m_db_mutex ordering with cs_main cannot deadlock and that std::async exception handling covers all paths.
Security signals we found
Concurrency control added: new Mutex m_db_mutex guards m_db during compaction and cache resize
Exception handling added in background compaction thread and in validation caller
Destructor now blocks on pending compaction to avoid use-after-free or data races during shutdown
Potential concern: std::async with std::launch::async may use an implementation-defined thread pool; destructor wait could delay shutdown, but this is by design
No evidence of memory corruption, remote trigger, or consensus bug introduced by the diff
Evidence from the diff
CCoinsViewDB::CompactFull() is refactored to launch std::async(std::launch::async) and return a std::shared_future
Changed components
src/txdb.cppsrc/txdb.hsrc/validation.cppsrc/test/coins_tests.cppInspect captured patch +43 / −10
diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
index 4c5ed748..3487a178 100644
--- a/src/test/coins_tests.cpp
+++ b/src/test/coins_tests.cpp
@@ -1078,7 +1078,7 @@ BOOST_FIXTURE_TEST_CASE(coins_db_leveldb_layout, FlushTest)
cache.Sync();
BOOST_CHECK_EQUAL(level2_files(base), 0);
- WITH_LOCK(::cs_main, base.CompactFull());
+ WITH_LOCK(::cs_main, return base.CompactFull()).wait();
BOOST_CHECK_EQUAL(level2_files(base), 1);
BOOST_CHECK(*Assert(base.GetCoin(outpoint)) == coin);
diff --git a/src/txdb.cpp b/src/txdb.cpp
index 62dc49a5..632c554c 100644
--- a/src/txdb.cpp
+++ b/src/txdb.cpp
@@ -14,10 +14,14 @@
#include <uint256.h>
#include <util/byte_units.h>
#include <util/log.h>
+#include <util/threadnames.h>
#include <util/vector.h>
#include <cassert>
+#include <chrono>
#include <cstdlib>
+#include <exception>
+#include <future>
#include <iterator>
#include <utility>
@@ -56,11 +60,22 @@ CCoinsViewDB::CCoinsViewDB(DBParams db_params, CoinsViewOptions options) :
m_options{std::move(options)},
m_db{std::make_unique<CDBWrapper>(m_db_params)} { }
+CCoinsViewDB::~CCoinsViewDB()
+{
+ if (m_compaction.valid()) {
+ if (m_compaction.wait_for(std::chrono::seconds{0}) != std::future_status::ready) {
+ LogInfo("Waiting for background chainstate compaction of %s", fs::PathToString(m_db_params.path));
+ }
+ m_compaction.wait();
+ }
+}
+
void CCoinsViewDB::ResizeCache(size_t new_cache_size)
{
// We can't do this operation with an in-memory DB since we'll lose all the coins upon
// reset.
if (!m_db_params.memory_only) {
+ LOCK(m_db_mutex);
// Have to do a reset first to get the original `m_db` state to release its
// filesystem lock.
m_db.reset();
@@ -180,12 +195,23 @@ std::optional<std::string> CCoinsViewDB::GetDBProperty(const std::string& proper
return m_db->GetProperty(property);
}
-void CCoinsViewDB::CompactFull()
+std::shared_future<void> CCoinsViewDB::CompactFull()
{
AssertLockHeld(::cs_main);
- LogDebug(BCLog::COINDB, "Starting chainstate compaction of %s", fs::PathToString(m_db_params.path));
- m_db->CompactFull();
- LogDebug(BCLog::COINDB, "Finished chainstate compaction of %s", fs::PathToString(m_db_params.path));
+ if (m_compaction.valid() && m_compaction.wait_for(std::chrono::seconds{0}) != std::future_status::ready) return m_compaction;
+ m_compaction = std::async(std::launch::async, [this] {
+ try {
+ util::ThreadRename("utxocompact");
+ LOCK(m_db_mutex);
+
+ LogDebug(BCLog::COINDB, "Starting chainstate compaction of %s", fs::PathToString(m_db_params.path));
+ m_db->CompactFull();
+ LogDebug(BCLog::COINDB, "Finished chainstate compaction of %s", fs::PathToString(m_db_params.path));
+ } catch (const std::exception& e) {
+ LogWarning("Failed chainstate compaction (%s)", e.what());
+ }
+ }).share();
+ return m_compaction;
}
/** Specialization of CCoinsViewCursor to iterate over a CCoinsViewDB */
diff --git a/src/txdb.h b/src/txdb.h
index 648adc85..e8986421 100644
--- a/src/txdb.h
+++ b/src/txdb.h
@@ -15,6 +15,7 @@
#include <cstddef>
#include <cstdint>
+#include <future>
#include <memory>
#include <optional>
#include <string>
@@ -37,9 +38,13 @@ class CCoinsViewDB final : public CCoinsView
protected:
DBParams m_db_params;
CoinsViewOptions m_options;
+ //! Prevents CompactFull() from using m_db while ResizeCache() replaces it.
+ Mutex m_db_mutex;
std::unique_ptr<CDBWrapper> m_db;
+ std::shared_future<void> m_compaction;
public:
explicit CCoinsViewDB(DBParams db_params, CoinsViewOptions options);
+ ~CCoinsViewDB() override;
std::optional<Coin> GetCoin(const COutPoint& outpoint) const override;
std::optional<Coin> PeekCoin(const COutPoint& outpoint) const override;
@@ -54,10 +59,10 @@ public:
size_t EstimateSize() const override;
//! Dynamically alter the underlying leveldb cache size.
- void ResizeCache(size_t new_cache_size) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
+ void ResizeCache(size_t new_cache_size) EXCLUSIVE_LOCKS_REQUIRED(cs_main, !m_db_mutex);
- //! Perform a blocking full compaction of the underlying LevelDB.
- void CompactFull() EXCLUSIVE_LOCKS_REQUIRED(cs_main);
+ //! Perform a full compaction of the underlying LevelDB on a one-shot background thread.
+ std::shared_future<void> CompactFull() EXCLUSIVE_LOCKS_REQUIRED(cs_main, !m_db_mutex);
//! Return an underlying LevelDB property value, if available.
std::optional<std::string> GetDBProperty(const std::string& property);
diff --git a/src/validation.cpp b/src/validation.cpp
index df4bd867..0836b307 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp
@@ -2838,9 +2838,11 @@ bool Chainstate::FlushStateToDisk(
m_chainman.m_options.signals->ChainStateFlushed(this->GetRole(), GetLocator(m_chain.Tip()));
}
- if (!m_chainman.m_interrupt && m_chainman.m_chainstates.size() == 1) { // Skip AssumeUTXO
- if (ShouldCompactChainstate(m_chainman.IsInitialBlockDownload())) {
+ if (!m_chainman.m_interrupt && ShouldCompactChainstate(m_chainman.IsInitialBlockDownload())) {
+ try {
CoinsDB().CompactFull();
+ } catch (const std::exception& e) {
+ LogWarning("Failed to start chainstate compaction (%s)", e.what());
}
}
}
Why this scored 18/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.