blockstorage: allow reading partial block data from storage
What the developer wrote
Authored by Roman Zeyde
blockstorage: allow reading partial block data from storage
It will allow fetching specific transactions using an external index,
following https://github.com/bitcoin/bitcoin/pull/32541#issuecomment-3267485313.
No logging takes place in case of an invalid offset/size (to avoid spamming the log),
by using a new `ReadRawError::BadPartRange` error variant.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
What changed, and why it matters
This commit adds a new feature to Bitcoin Core that lets the node read only a slice of a stored block (a specific byte range) instead of the whole block. It is meant to support future REST API improvements for fetching specific transactions. The commit includes careful range checks and avoids logging invalid requests to prevent log spam. There is no direct evidence in the commit that this fixes an active security bug, but it does touch untrusted REST input handling, so defensive care is warranted.
Review the REST endpoint wiring that will eventually consume this new parameter to ensure the offset/size are parsed safely and that BadPartRange results in an appropriate HTTP error rather than silent behavior. Verify that the size_t arithmetic cannot wrap on unusual platforms and that the seek+read path does not leak block size information via timing.
Security signals we found
New untrusted input path: REST block-part offset/size reaches block storage layer
Explicit no-logging decision to avoid log spam from untrusted REST input
Range validation added before seek/read: size==0, offset>=blk_size, size>blk_size-offset rejected
New error enum variant BadPartRange for invalid partial-block requests
REST handler switch falls through silently on BadPartRange (no error response)
Unit tests exercise boundary and overflow-like ranges
Evidence from the diff
The change extends BlockManager::ReadRawBlock() with an optional block_part parameter (offset + size). It validates the requested range against the actual block size and returns a new ReadRawError::BadPartRange error for invalid ranges without logging, because the offset/size may come from untrusted REST input. The REST handler now silently ignores BadPartRange (breaks from the error switch). Tests cover valid and invalid ranges, including edge cases near size_t max. The implementation appears to prevent out-of-bounds reads and integer-overflow-style range errors.
Changed components
src/node/blockstorage.cppsrc/node/blockstorage.hsrc/rest.cppsrc/test/blockmanager_tests.cppInspect captured patch +75 / −2
diff --git a/src/node/blockstorage.cpp b/src/node/blockstorage.cpp
index 0927eb85..5ba00fcf 100644
--- a/src/node/blockstorage.cpp
+++ b/src/node/blockstorage.cpp
@@ -1048,7 +1048,7 @@ bool BlockManager::ReadBlock(CBlock& block, const CBlockIndex& index) const
return ReadBlock(block, block_pos, index.GetBlockHash());
}
-BlockManager::ReadRawBlockResult BlockManager::ReadRawBlock(const FlatFilePos& pos) const
+BlockManager::ReadRawBlockResult BlockManager::ReadRawBlock(const FlatFilePos& pos, std::optional<std::pair<size_t, size_t>> block_part) const
{
if (pos.nPos < STORAGE_HEADER_BYTES) {
// If nPos is less than STORAGE_HEADER_BYTES, we can't read the header that precedes the block data
@@ -1081,6 +1081,15 @@ BlockManager::ReadRawBlockResult BlockManager::ReadRawBlock(const FlatFilePos& p
return util::Unexpected{ReadRawError::IO};
}
+ if (block_part) {
+ const auto [offset, size]{*block_part};
+ if (size == 0 || offset >= blk_size || size > blk_size - offset) {
+ return util::Unexpected{ReadRawError::BadPartRange}; // Avoid logging - offset/size come from untrusted REST input
+ }
+ filein.seek(offset, SEEK_CUR);
+ blk_size = size;
+ }
+
std::vector<std::byte> data(blk_size); // Zeroing of memory is intentional here
filein.read(data);
return data;
diff --git a/src/node/blockstorage.h b/src/node/blockstorage.h
index 1ce7e8f6..e3f9c445 100644
--- a/src/node/blockstorage.h
+++ b/src/node/blockstorage.h
@@ -172,6 +172,7 @@ std::ostream& operator<<(std::ostream& os, const BlockfileCursor& cursor);
enum class ReadRawError {
IO,
+ BadPartRange,
};
/**
@@ -460,7 +461,7 @@ public:
/** Functions for disk access for blocks */
bool ReadBlock(CBlock& block, const FlatFilePos& pos, const std::optional<uint256>& expected_hash) const;
bool ReadBlock(CBlock& block, const CBlockIndex& index) const;
- ReadRawBlockResult ReadRawBlock(const FlatFilePos& pos) const;
+ ReadRawBlockResult ReadRawBlock(const FlatFilePos& pos, std::optional<std::pair<size_t, size_t>> block_part = std::nullopt) const;
bool ReadBlockUndo(CBlockUndo& blockundo, const CBlockIndex& index) const;
diff --git a/src/rest.cpp b/src/rest.cpp
index 4883e26d..26312633 100644
--- a/src/rest.cpp
+++ b/src/rest.cpp
@@ -420,6 +420,7 @@ static bool rest_block(const std::any& context,
if (!block_data) {
switch (block_data.error()) {
case node::ReadRawError::IO: return RESTERR(req, HTTP_INTERNAL_SERVER_ERROR, "I/O error reading " + hashStr);
+ case node::ReadRawError::BadPartRange: break; // can happen only when reading a block part
}
assert(false);
}
diff --git a/src/test/blockmanager_tests.cpp b/src/test/blockmanager_tests.cpp
index a1551994..4a326458 100644
--- a/src/test/blockmanager_tests.cpp
+++ b/src/test/blockmanager_tests.cpp
@@ -138,6 +138,68 @@ BOOST_FIXTURE_TEST_CASE(blockmanager_block_data_availability, TestChain100Setup)
BOOST_CHECK(!blockman.CheckBlockDataAvailability(tip, *last_pruned_block));
}
+BOOST_FIXTURE_TEST_CASE(blockmanager_block_data_part, TestChain100Setup)
+{
+ LOCK(::cs_main);
+ auto& chainman{m_node.chainman};
+ auto& blockman{chainman->m_blockman};
+ const CBlockIndex& tip{*chainman->ActiveTip()};
+ const FlatFilePos tip_block_pos{tip.GetBlockPos()};
+
+ auto block{blockman.ReadRawBlock(tip_block_pos)};
+ BOOST_REQUIRE(block);
+ BOOST_REQUIRE_GE(block->size(), 200);
+
+ const auto expect_part{[&](size_t offset, size_t size) {
+ auto res{blockman.ReadRawBlock(tip_block_pos, std::pair{offset, size})};
+ BOOST_CHECK(res);
+ const auto& part{res.value()};
+ BOOST_CHECK_EQUAL_COLLECTIONS(part.begin(), part.end(), block->begin() + offset, block->begin() + offset + size);
+ }};
+
+ expect_part(0, 20);
+ expect_part(0, block->size() - 1);
+ expect_part(0, block->size() - 10);
+ expect_part(0, block->size());
+ expect_part(1, block->size() - 1);
+ expect_part(10, 20);
+ expect_part(block->size() - 1, 1);
+}
+
+BOOST_FIXTURE_TEST_CASE(blockmanager_block_data_part_error, TestChain100Setup)
+{
+ LOCK(::cs_main);
+ auto& chainman{m_node.chainman};
+ auto& blockman{chainman->m_blockman};
+ const CBlockIndex& tip{*chainman->ActiveTip()};
+ const FlatFilePos tip_block_pos{tip.GetBlockPos()};
+
+ auto block{blockman.ReadRawBlock(tip_block_pos)};
+ BOOST_REQUIRE(block);
+ BOOST_REQUIRE_GE(block->size(), 200);
+
+ const auto expect_part_error{[&](size_t offset, size_t size) {
+ auto res{blockman.ReadRawBlock(tip_block_pos, std::pair{offset, size})};
+ BOOST_CHECK(!res);
+ BOOST_CHECK_EQUAL(res.error(), node::ReadRawError::BadPartRange);
+ }};
+
+ expect_part_error(0, 0);
+ expect_part_error(0, block->size() + 1);
+ expect_part_error(0, std::numeric_limits<size_t>::max());
+ expect_part_error(1, block->size());
+ expect_part_error(2, block->size() - 1);
+ expect_part_error(block->size() - 1, 2);
+ expect_part_error(block->size() - 2, 3);
+ expect_part_error(block->size() + 1, 0);
+ expect_part_error(block->size() + 1, 1);
+ expect_part_error(block->size() + 2, 2);
+ expect_part_error(block->size(), 0);
+ expect_part_error(block->size(), 1);
+ expect_part_error(std::numeric_limits<size_t>::max(), 1);
+ expect_part_error(std::numeric_limits<size_t>::max(), std::numeric_limits<size_t>::max());
+}
+
BOOST_FIXTURE_TEST_CASE(blockmanager_readblock_hash_mismatch, TestingSetup)
{
CBlockIndex index;
Why this scored 29/100
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.