← Watch feed
Use usize for CompactSizeDecoder and add new_with_limit
Public commit record
73/100 · Adequate
What the developer wrote
Authored by Mitchell Bagot
Use usize for CompactSizeDecoder and add new_with_limit
Following the change to CompactSizeEncoder::new to take a usize, the
return types and ranges of the CompactSizeEncoder and CompactSizeDecoder
now differ. Since a device's memory addressing is limited to usize,
returning values beyond that size is largely useless. Converting the
CompactSizeDecoder to return a usize also allows the
cast_to_usize_if_valid function to be merged into the decoder itself.
Convert the CompactSizeDecoder to return a usize. Remove
cast_to_usize_if_valid and introduce its functionality into
CompactSizeDecoder.
✓ Specific, descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context
What changed, and why it matters
This commit tightens how the library decodes Bitcoin 'compact size' length fields. Previously the decoder could return a 64-bit value, which callers then had to convert to a machine-usable size. Now the decoder itself returns a usize and enforces a configurable upper limit. This is a defensive hardening change: it reduces the chance that a maliciously large length value causes memory problems or integer-conversion bugs later on. It is not a fix for a known active exploit.
Recommended action
Review downstream callers that previously handled u64 compact-size values or LengthPrefixInvalid errors; update them for the new usize output and unified CompactSizeDecoderError. Consider using new_with_limit in contexts where a lower bound than 4 MB is appropriate. No urgent patch is required solely on the basis of this commit.
Security signals we found
01
Bounds checking on decoded length prefixes moved earlier in the decode pipeline
02
Removal of a separate cast/validation step that could be skipped or misused by callers
03
New configurable limit constructor allows per-context stricter caps
04
No new cryptographic or memory-safety primitive introduced; existing DoS cap preserved
Evidence from the diff
CompactSizeDecoder’s Output type changes from u64 to usize, and a new constructor new_with_limit(limit: usize) is added. The old free function cast_to_usize_if_valid is removed and its logic (check against MAX_VEC_SIZE and usize::try_from) is folded into CompactSizeDecoder::end. ByteVecDecoder, VecDecoder and WitnessDecoder now receive a usize directly and drop their separate LengthPrefixInvalid error arms. The default limit remains 4,000,000 bytes. This is an API-breaking refactor that centralises length validation, but it does not change the effective maximum for the default decoder and does not by itself fix a concrete vulnerability.
Changed components
bitcoin_consensus_encoding::CompactSizeDecoderbitcoin_consensus_encoding::ByteVecDecoderbitcoin_consensus_encoding::VecDecoderprimitives::witness::WitnessDecoderLengthPrefixExceedsMaxErrorInspect captured patch +103 / −89
diff --git a/api/consensus_encoding/all-features.txt b/api/consensus_encoding/all-features.txt
index 80251ed4..ee67d857 100644
--- a/api/consensus_encoding/all-features.txt
+++ b/api/consensus_encoding/all-features.txt
@@ -292,6 +292,7 @@ pub const fn bitcoin_consensus_encoding::ArrayEncoder<N>::without_length_prefix(
pub const fn bitcoin_consensus_encoding::ByteVecDecoder::new() -> Self
pub const fn bitcoin_consensus_encoding::BytesEncoder<'sl>::without_length_prefix(sl: &'sl [u8]) -> Self
pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new() -> Self
+pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new_with_limit(limit: usize) -> Self
pub const fn bitcoin_consensus_encoding::Decoder2<A, B>::new(first: A, second: B) -> Self
pub const fn bitcoin_consensus_encoding::Decoder3<A, B, C>::new(dec_1: A, dec_2: B, dec_3: C) -> Self
pub const fn bitcoin_consensus_encoding::Decoder4<A, B, C, D>::new(dec_1: A, dec_2: B, dec_3: C, dec_4: D) -> Self
@@ -332,6 +333,7 @@ pub fn bitcoin_consensus_encoding::CompactSizeDecoder::read_limit(&self) -> usiz
pub fn bitcoin_consensus_encoding::CompactSizeDecoderError::clone(&self) -> bitcoin_consensus_encoding::CompactSizeDecoderError
pub fn bitcoin_consensus_encoding::CompactSizeDecoderError::eq(&self, other: &bitcoin_consensus_encoding::CompactSizeDecoderError) -> bool
pub fn bitcoin_consensus_encoding::CompactSizeDecoderError::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
+pub fn bitcoin_consensus_encoding::CompactSizeDecoderError::source(&self) -> core::option::Option<&(dyn core::error::Error + 'static)>
pub fn bitcoin_consensus_encoding::CompactSizeEncoder::advance(&mut self) -> bool
pub fn bitcoin_consensus_encoding::CompactSizeEncoder::current_chunk(&self) -> &[u8]
pub fn bitcoin_consensus_encoding::CompactSizeEncoder::new(value: usize) -> Self
@@ -401,7 +403,6 @@ pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::eq(&self, other: &bitco
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::from(never: core::convert::Infallible) -> Self
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::source(&self) -> core::option::Option<&(dyn core::error::Error + 'static)>
-pub fn bitcoin_consensus_encoding::cast_to_usize_if_valid(n: u64) -> core::result::Result<usize, bitcoin_consensus_encoding::LengthPrefixExceedsMaxError>
pub fn bitcoin_consensus_encoding::decode_from_read<T, R>(reader: R) -> core::result::Result<T, bitcoin_consensus_encoding::ReadError<<<T as bitcoin_consensus_encoding::Decodable>::Decoder as bitcoin_consensus_encoding::Decoder>::Error>> where T: bitcoin_consensus_encoding::Decodable, R: std::io::BufRead
pub fn bitcoin_consensus_encoding::decode_from_read_unbuffered<T, R>(reader: R) -> core::result::Result<T, bitcoin_consensus_encoding::ReadError<<<T as bitcoin_consensus_encoding::Decodable>::Decoder as bitcoin_consensus_encoding::Decoder>::Error>> where T: bitcoin_consensus_encoding::Decodable, R: std::io::Read
pub fn bitcoin_consensus_encoding::decode_from_read_unbuffered_with<T, R, const BUFFER_SIZE: usize>(reader: R) -> core::result::Result<T, bitcoin_consensus_encoding::ReadError<<<T as bitcoin_consensus_encoding::Decodable>::Decoder as bitcoin_consensus_encoding::Decoder>::Error>> where T: bitcoin_consensus_encoding::Decodable, R: std::io::Read
@@ -443,7 +444,7 @@ pub type bitcoin_consensus_encoding::ArrayDecoder<N>::Output = [u8; N]
pub type bitcoin_consensus_encoding::ByteVecDecoder::Error = bitcoin_consensus_encoding::ByteVecDecoderError
pub type bitcoin_consensus_encoding::ByteVecDecoder::Output = alloc::vec::Vec<u8>
pub type bitcoin_consensus_encoding::CompactSizeDecoder::Error = bitcoin_consensus_encoding::CompactSizeDecoderError
-pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = u64
+pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = usize
pub type bitcoin_consensus_encoding::Decodable::Decoder: bitcoin_consensus_encoding::Decoder<Output = Self>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Error = bitcoin_consensus_encoding::Decoder2Error<<A as bitcoin_consensus_encoding::Decoder>::Error, <B as bitcoin_consensus_encoding::Decoder>::Error>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Output = (<A as bitcoin_consensus_encoding::Decoder>::Output, <B as bitcoin_consensus_encoding::Decoder>::Output)
diff --git a/api/consensus_encoding/alloc-only.txt b/api/consensus_encoding/alloc-only.txt
index faf0c9f1..9b859154 100644
--- a/api/consensus_encoding/alloc-only.txt
+++ b/api/consensus_encoding/alloc-only.txt
@@ -271,6 +271,7 @@ pub const fn bitcoin_consensus_encoding::ArrayEncoder<N>::without_length_prefix(
pub const fn bitcoin_consensus_encoding::ByteVecDecoder::new() -> Self
pub const fn bitcoin_consensus_encoding::BytesEncoder<'sl>::without_length_prefix(sl: &'sl [u8]) -> Self
pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new() -> Self
+pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new_with_limit(limit: usize) -> Self
pub const fn bitcoin_consensus_encoding::Decoder2<A, B>::new(first: A, second: B) -> Self
pub const fn bitcoin_consensus_encoding::Decoder3<A, B, C>::new(dec_1: A, dec_2: B, dec_3: C) -> Self
pub const fn bitcoin_consensus_encoding::Decoder4<A, B, C, D>::new(dec_1: A, dec_2: B, dec_3: C, dec_4: D) -> Self
@@ -370,7 +371,6 @@ pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::clone(&self) -> bitcoin
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::eq(&self, other: &bitcoin_consensus_encoding::VecDecoderError<Err>) -> bool
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
pub fn bitcoin_consensus_encoding::VecDecoderError<Err>::from(never: core::convert::Infallible) -> Self
-pub fn bitcoin_consensus_encoding::cast_to_usize_if_valid(n: u64) -> core::result::Result<usize, bitcoin_consensus_encoding::LengthPrefixExceedsMaxError>
pub fn bitcoin_consensus_encoding::decode_from_slice<T>(bytes: &[u8]) -> core::result::Result<T, <<T as bitcoin_consensus_encoding::Decodable>::Decoder as bitcoin_consensus_encoding::Decoder>::Error> where T: bitcoin_consensus_encoding::Decodable
pub fn bitcoin_consensus_encoding::encode_to_vec<T>(object: &T) -> alloc::vec::Vec<u8> where T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized
pub fn core::option::Option<T>::advance(&mut self) -> bool
@@ -408,7 +408,7 @@ pub type bitcoin_consensus_encoding::ArrayDecoder<N>::Output = [u8; N]
pub type bitcoin_consensus_encoding::ByteVecDecoder::Error = bitcoin_consensus_encoding::ByteVecDecoderError
pub type bitcoin_consensus_encoding::ByteVecDecoder::Output = alloc::vec::Vec<u8>
pub type bitcoin_consensus_encoding::CompactSizeDecoder::Error = bitcoin_consensus_encoding::CompactSizeDecoderError
-pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = u64
+pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = usize
pub type bitcoin_consensus_encoding::Decodable::Decoder: bitcoin_consensus_encoding::Decoder<Output = Self>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Error = bitcoin_consensus_encoding::Decoder2Error<<A as bitcoin_consensus_encoding::Decoder>::Error, <B as bitcoin_consensus_encoding::Decoder>::Error>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Output = (<A as bitcoin_consensus_encoding::Decoder>::Output, <B as bitcoin_consensus_encoding::Decoder>::Output)
diff --git a/api/consensus_encoding/no-features.txt b/api/consensus_encoding/no-features.txt
index 6e089c51..a11b4608 100644
--- a/api/consensus_encoding/no-features.txt
+++ b/api/consensus_encoding/no-features.txt
@@ -214,6 +214,7 @@ pub const fn bitcoin_consensus_encoding::ArrayDecoder<N>::new() -> Self
pub const fn bitcoin_consensus_encoding::ArrayEncoder<N>::without_length_prefix(arr: [u8; N]) -> Self
pub const fn bitcoin_consensus_encoding::BytesEncoder<'sl>::without_length_prefix(sl: &'sl [u8]) -> Self
pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new() -> Self
+pub const fn bitcoin_consensus_encoding::CompactSizeDecoder::new_with_limit(limit: usize) -> Self
pub const fn bitcoin_consensus_encoding::Decoder2<A, B>::new(first: A, second: B) -> Self
pub const fn bitcoin_consensus_encoding::Decoder3<A, B, C>::new(dec_1: A, dec_2: B, dec_3: C) -> Self
pub const fn bitcoin_consensus_encoding::Decoder4<A, B, C, D>::new(dec_1: A, dec_2: B, dec_3: C, dec_4: D) -> Self
@@ -322,7 +323,7 @@ pub trait bitcoin_consensus_encoding::Encoder
pub type bitcoin_consensus_encoding::ArrayDecoder<N>::Error = bitcoin_consensus_encoding::UnexpectedEofError
pub type bitcoin_consensus_encoding::ArrayDecoder<N>::Output = [u8; N]
pub type bitcoin_consensus_encoding::CompactSizeDecoder::Error = bitcoin_consensus_encoding::CompactSizeDecoderError
-pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = u64
+pub type bitcoin_consensus_encoding::CompactSizeDecoder::Output = usize
pub type bitcoin_consensus_encoding::Decodable::Decoder: bitcoin_consensus_encoding::Decoder<Output = Self>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Error = bitcoin_consensus_encoding::Decoder2Error<<A as bitcoin_consensus_encoding::Decoder>::Error, <B as bitcoin_consensus_encoding::Decoder>::Error>
pub type bitcoin_consensus_encoding::Decoder2<A, B>::Output = (<A as bitcoin_consensus_encoding::Decoder>::Output, <B as bitcoin_consensus_encoding::Decoder>::Output)
diff --git a/consensus_encoding/src/decode/decoders.rs b/consensus_encoding/src/decode/decoders.rs
index 0e9d0a07..fe0d49ad 100644
--- a/consensus_encoding/src/decode/decoders.rs
+++ b/consensus_encoding/src/decode/decoders.rs
@@ -15,8 +15,10 @@ use super::Decodable;
use super::Decoder;
/// Maximum size, in bytes, of a vector we are allowed to decode.
-#[cfg(feature = "alloc")]
-const MAX_VEC_SIZE: u64 = 4_000_000;
+///
+/// This is also the default value limit that can be decoded with a decoder from
+/// [`CompactSizeDecoder::new`].
+const MAX_VEC_SIZE: usize = 4_000_000;
/// Maximum amount of memory (in bytes) to allocate at once when deserializing vectors.
#[cfg(feature = "alloc")]
@@ -83,11 +85,8 @@ impl Decoder for ByteVecDecoder {
self.prefix_decoder = Some(decoder);
return Ok(true);
}
- let length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
-
+ self.bytes_expected = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
self.prefix_decoder = None;
- self.bytes_expected =
- cast_to_usize_if_valid(length).map_err(|e| E(Inner::LengthPrefixInvalid(e)))?;
// For DoS prevention, let's not allocate all memory upfront.
}
@@ -191,14 +190,12 @@ impl<T: Decodable> Decoder for VecDecoder<T> {
self.prefix_decoder = Some(decoder);
return Ok(true);
}
- let length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
- if length == 0 {
+ self.length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
+ if self.length == 0 {
return Ok(false);
}
self.prefix_decoder = None;
- self.length =
- cast_to_usize_if_valid(length).map_err(|e| E(Inner::LengthPrefixInvalid(e)))?;
// For DoS prevention, let's not allocate all memory upfront.
}
@@ -257,24 +254,6 @@ impl<T: Decodable> Decoder for VecDecoder<T> {
}
}
-/// Cast a decoded length prefix to a `usize`.
-///
-/// Consensus encoded vectors can be up to 4,000,000 bytes long.
-///
-/// This is a theoretical max since block size is 4 meg wu and minimum vector element is one byte.
-///
-/// # Errors
-///
-/// Errors if `n` is greater than 4,000,000 or won't fit in a `usize`.
-#[cfg(feature = "alloc")]
-pub fn cast_to_usize_if_valid(n: u64) -> Result<usize, LengthPrefixExceedsMaxError> {
- if n > MAX_VEC_SIZE {
- return Err(LengthPrefixExceedsMaxError { value: n });
- }
-
- usize::try_from(n).map_err(|_| LengthPrefixExceedsMaxError { value: n })
-}
-
/// A decoder that expects exactly N bytes and returns them as an array.
pub struct ArrayDecoder<const N: usize> {
buffer: [u8; N],
@@ -623,11 +602,24 @@ where
#[derive(Debug, Clone)]
pub struct CompactSizeDecoder {
buf: internals::array_vec::ArrayVec<u8, 9>,
+ limit: usize,
}
impl CompactSizeDecoder {
/// Constructs a new compact size decoder.
- pub const fn new() -> Self { Self { buf: internals::array_vec::ArrayVec::new() } }
+ ///
+ /// Consensus encoded vectors can be up to 4,000,000 bytes long.
+ /// This is a theoretical max since block size is 4 meg wu and minimum vector element is one byte.
+ ///
+ /// The final call to [`CompactSizeDecoder::end`] on this decoder will fail if the
+ /// decoded value exceeds 4,000,000 or won't fit in a `usize`.
+ pub const fn new() -> Self { Self { buf: internals::array_vec::ArrayVec::new(), limit: MAX_VEC_SIZE } }
+
+ /// Constructs a new compact size decoder with encoded value limited to the provided usize.
+ ///
+ /// The final call to [`CompactSizeDecoder::end`] on this decoder will fail if the
+ /// decoded value exceeds `limit` or won't fit in a `usize`.
+ pub const fn new_with_limit(limit: usize) -> Self { Self { buf: internals::array_vec::ArrayVec::new(), limit } }
}
impl Default for CompactSizeDecoder {
@@ -635,7 +627,7 @@ impl Default for CompactSizeDecoder {
}
impl Decoder for CompactSizeDecoder {
- type Output = u64;
+ type Output = usize;
type Error = CompactSizeDecoderError;
fn push_bytes(&mut self, bytes: &mut &[u8]) -> Result<bool, Self::Error> {
@@ -674,7 +666,7 @@ impl Decoder for CompactSizeDecoder {
.split_first()
.ok_or(CompactSizeDecoderError(E::UnexpectedEof { required: 1, received: 0 }))?;
- match *first {
+ let dec_value = match *first {
0xFF => {
let x = u64::from_le_bytes(arr(payload)?);
if x < 0x100_000_000 {
@@ -700,7 +692,28 @@ impl Decoder for CompactSizeDecoder {
}
}
n => Ok(n.into()),
- }
+ }?;
+
+ // This error is returned if dec_value is outside of the usize range, or
+ // if it is above the given limit.
+ let make_err = || {
+ CompactSizeDecoderError(
+ E::ValueExceedsLimit(LengthPrefixExceedsMaxError {
+ value: dec_value,
+ limit: self.limit,
+ })
+ )
+ };
+
+ usize::try_from(dec_value)
+ .map_err(|_| make_err())
+ .and_then(|nsize| {
+ if nsize > self.limit {
+ Err(make_err())
+ } else {
+ Ok(nsize)
+ }
+ })
}
fn read_limit(&self) -> usize {
@@ -734,6 +747,8 @@ enum CompactSizeDecoderErrorInner {
/// The encoded value.
value: u64,
},
+ /// Returned when the encoded value exceeds the decoder's limit.
+ ValueExceedsLimit(LengthPrefixExceedsMaxError),
}
impl fmt::Display for CompactSizeDecoderError {
@@ -753,12 +768,22 @@ impl fmt::Display for CompactSizeDecoderError {
required, received
),
E::NonMinimal { value } => write!(f, "the value {} was not encoded minimally", value),
+ E::ValueExceedsLimit(ref e) => write_err!(f, "value exceeds limit"; e),
}
}
}
#[cfg(feature = "std")]
-impl std::error::Error for CompactSizeDecoderError {}
+impl std::error::Error for CompactSizeDecoderError {
+ fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+ use CompactSizeDecoderErrorInner as E;
+
+ match self {
+ Self(E::ValueExceedsLimit(ref e)) => Some(e),
+ _ => None,
+ }
+ }
+}
/// The error returned by the [`ByteVecDecoder`].
#[cfg(feature = "alloc")]
@@ -770,8 +795,6 @@ pub struct ByteVecDecoderError(ByteVecDecoderErrorInner);
enum ByteVecDecoderErrorInner {
/// Error decoding the byte vector length prefix.
LengthPrefixDecode(CompactSizeDecoderError),
- /// Length prefix exceeds 4,000,000.
- LengthPrefixInvalid(LengthPrefixExceedsMaxError),
/// Not enough bytes given to decoder.
UnexpectedEof(UnexpectedEofError),
}
@@ -788,7 +811,6 @@ impl fmt::Display for ByteVecDecoderError {
match self.0 {
E::LengthPrefixDecode(ref e) => write_err!(f, "byte vec decoder error"; e),
- E::LengthPrefixInvalid(ref e) => write_err!(f, "byte vec decoder error"; e),
E::UnexpectedEof(ref e) => write_err!(f, "byte vec decoder error"; e),
}
}
@@ -801,7 +823,6 @@ impl std::error::Error for ByteVecDecoderError {
match self.0 {
E::LengthPrefixDecode(ref e) => Some(e),
- E::LengthPrefixInvalid(ref e) => Some(e),
E::UnexpectedEof(ref e) => Some(e),
}
}
@@ -817,8 +838,6 @@ pub struct VecDecoderError<Err>(VecDecoderErrorInner<Err>);
enum VecDecoderErrorInner<Err> {
/// Error decoding the vector length prefix.
LengthPrefixDecode(CompactSizeDecoderError),
- /// Length prefix exceeds 4,000,000.
- LengthPrefixInvalid(LengthPrefixExceedsMaxError),
/// Error while decoding an item.
Item(Err),
/// Not enough bytes given to decoder.
@@ -840,7 +859,6 @@ where
match self.0 {
E::LengthPrefixDecode(ref e) => write_err!(f, "vec decoder error"; e),
- E::LengthPrefixInvalid(ref e) => write_err!(f, "vec decoder error"; e),
E::Item(ref e) => write_err!(f, "vec decoder error"; e),
E::UnexpectedEof(ref e) => write_err!(f, "vec decoder error"; e),
}
@@ -857,35 +875,28 @@ where
match self.0 {
E::LengthPrefixDecode(ref e) => Some(e),
- E::LengthPrefixInvalid(ref e) => Some(e),
E::Item(ref e) => Some(e),
E::UnexpectedEof(ref e) => Some(e),
}
}
}
-/// Length prefix exceeds max value (4,000,000).
-#[cfg(feature = "alloc")]
+/// Length prefix exceeds the configured limit.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct LengthPrefixExceedsMaxError {
/// Decoded value of the compact encoded length prefix.
value: u64,
+ /// The value limit that the length prefix exceeds.
+ limit: usize,
}
-#[cfg(feature = "alloc")]
impl core::fmt::Display for LengthPrefixExceedsMaxError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
- let max = match mem::size_of::<usize>() {
- 1 => u32::from(u8::MAX),
- 2 => u32::from(u16::MAX),
- _ => 4_000_000,
- };
-
- write!(f, "length prefix {} exceeds max value {}", self.value, max)
+ write!(f, "length prefix {} exceeds max value {}", self.value, self.limit)
}
}
-#[cfg(all(feature = "std", feature = "alloc"))]
+#[cfg(feature = "std")]
impl std::error::Error for LengthPrefixExceedsMaxError {}
/// Not enough bytes given to decoder.
@@ -1105,12 +1116,12 @@ mod tests {
// Stress test the push_bytes impl by passing in a single byte slice repeatedly.
macro_rules! check_decode_one_byte_at_a_time {
- ($decoder:ident $($test_name:ident, $want:expr, $array:expr);* $(;)?) => {
+ ($decoder:expr; $($test_name:ident, $want:expr, $array:expr);* $(;)?) => {
$(
#[test]
#[allow(non_snake_case)]
fn $test_name() {
- let mut decoder = $decoder::default();
+ let mut decoder = $decoder;
for (i, _) in $array.iter().enumerate() {
if i < $array.len() - 1 {
@@ -1132,14 +1143,35 @@ mod tests {
}
check_decode_one_byte_at_a_time! {
- CompactSizeDecoder
+ CompactSizeDecoder::new_with_limit(0xF0F0_F0F0);
decode_compact_size_0x10, 0x10, [0x10];
decode_compact_size_0xFC, 0xFC, [0xFC];
decode_compact_size_0xFD, 0xFD, [0xFD, 0xFD, 0x00];
decode_compact_size_0x100, 0x100, [0xFD, 0x00, 0x01];
decode_compact_size_0xFFF, 0x0FFF, [0xFD, 0xFF, 0x0F];
decode_compact_size_0x0F0F_0F0F, 0x0F0F_0F0F, [0xFE, 0xF, 0xF, 0xF, 0xF];
- decode_compact_size_0xF0F0_F0F0_F0E0, 0xF0F0_F0F0_F0E0, [0xFF, 0xE0, 0xF0, 0xF0, 0xF0, 0xF0, 0xF0, 0, 0];
+ }
+
+ #[test]
+ #[cfg(target_pointer_width = "64")]
+ #[allow(non_snake_case)]
+ fn decode_compact_size_0xF0F0_F0F0_F0E0() {
+ let mut decoder = CompactSizeDecoder::new_with_limit(0xF0F0_F0F0_F0EF);
+ let array = [0xFF, 0xE0, 0xF0, 0xF0, 0xF0, 0xF0, 0xF0, 0, 0];
+
+ for (i, _) in array.iter().enumerate() {
+ if i < array.len() - 1 {
+ let mut p = &array[i..=i];
+ assert!(decoder.push_bytes(&mut p).unwrap());
+ } else {
+ // last byte: `push_bytes` should return false since no more bytes required.
+ let mut p = &array[i..];
+ assert!(!decoder.push_bytes(&mut p).unwrap());
+ }
+ }
+
+ let got = decoder.end().unwrap();
+ assert_eq!(got, 0xF0F0_F0F0_F0E0);
}
#[test]
@@ -1168,7 +1200,7 @@ mod tests {
#[cfg(feature = "alloc")]
check_decode_one_byte_at_a_time! {
- ByteVecDecoder
+ ByteVecDecoder::default();
decode_byte_vec, alloc::vec![0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef],
[0x08, 0x01, 0x23, 0x45, 0x67, 0x89, 0xab, 0xcd, 0xef];
decode_byte_vec_multi_byte_length_prefix, [0xff; 256], two_fifty_six_bytes_encoded();
@@ -1393,7 +1425,7 @@ mod tests {
#[cfg(feature = "alloc")]
check_decode_one_byte_at_a_time! {
- TestDecoder
+ TestDecoder::default();
decode_vec, Test(vec![Inner(0xDEAD_BEEF), Inner(0xCAFE_BABE)]),
vec![0x02, 0xEF, 0xBE, 0xAD, 0xDE, 0xBE, 0xBA, 0xFE, 0xCA];
decode_vec_multi_byte_length_prefix, two_fifty_six_elements(), two_fifty_six_elements_encoded();
diff --git a/consensus_encoding/src/lib.rs b/consensus_encoding/src/lib.rs
index 965636a8..999b3889 100644
--- a/consensus_encoding/src/lib.rs
+++ b/consensus_encoding/src/lib.rs
@@ -22,7 +22,7 @@ mod encode;
#[cfg(feature = "alloc")]
pub use self::decode::decoders::{
- cast_to_usize_if_valid, ByteVecDecoder, ByteVecDecoderError, LengthPrefixExceedsMaxError,
+ ByteVecDecoder, ByteVecDecoderError, LengthPrefixExceedsMaxError,
VecDecoder, VecDecoderError,
};
pub use self::decode::decoders::{
diff --git a/consensus_encoding/tests/decode.rs b/consensus_encoding/tests/decode.rs
index a5edc98d..b1c0fed9 100644
--- a/consensus_encoding/tests/decode.rs
+++ b/consensus_encoding/tests/decode.rs
@@ -167,18 +167,6 @@ fn decode_compact_size_single_byte_read_limit() {
assert_eq!(result, 66);
}
-#[test]
-#[cfg(feature = "alloc")]
-fn decode_cast_to_usize_boundary_conditions() {
- // Test the 4MB boundary and some edge cases.
- use bitcoin_consensus_encoding::cast_to_usize_if_valid;
-
- assert!(cast_to_usize_if_valid(4_000_000).is_ok());
- assert!(cast_to_usize_if_valid(4_000_001).is_err());
- assert!(cast_to_usize_if_valid(u64::MAX).is_err());
- assert_eq!(cast_to_usize_if_valid(0).unwrap(), 0);
-}
-
#[test]
#[cfg(feature = "alloc")]
fn decode_byte_vec_decoder_empty() {
diff --git a/fuzz/fuzz_targets/consensus_encoding/decode_compact_size.rs b/fuzz/fuzz_targets/consensus_encoding/decode_compact_size.rs
index a357fe04..e747495c 100644
--- a/fuzz/fuzz_targets/consensus_encoding/decode_compact_size.rs
+++ b/fuzz/fuzz_targets/consensus_encoding/decode_compact_size.rs
@@ -34,7 +34,7 @@ fn do_test(data: &[u8]) {
1 => assert!(value < 0xFD),
3 => assert!((0xFD..=0xFFFF).contains(&value)),
5 => assert!((0x10000..=0xFFFFFFFF).contains(&value)),
- 9 => assert!(value >= 0x100000000),
+ 9 => assert!((value as u64) >= 0x100000000), // Decoded values should only ever fit into a u64
_ => panic!("invalid compact size encoding length: {}", consumed),
}
diff --git a/primitives/src/witness.rs b/primitives/src/witness.rs
index ca275b5f..20542794 100644
--- a/primitives/src/witness.rs
+++ b/primitives/src/witness.rs
@@ -14,7 +14,7 @@ use arbitrary::{Arbitrary, Unstructured};
use encoding::Decoder4;
use encoding::{
self, BytesEncoder, CompactSizeDecoder, CompactSizeDecoderError, CompactSizeEncoder, Decoder,
- Encodable, Encoder, Encoder2, LengthPrefixExceedsMaxError,
+ Encodable, Encoder, Encoder2,
};
#[cfg(feature = "hex")]
use hex::DecodeVariableLengthBytesError;
@@ -350,9 +350,7 @@ impl Decoder for WitnessDecoder {
}
// Take ownership of the decoder in order to consume it.
let decoder = core::mem::take(&mut self.witness_count_decoder);
- let length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
- let witness_elements = encoding::cast_to_usize_if_valid(length)
- .map_err(|e| E(Inner::LengthPrefixInvalid(e)))?;
+ let witness_elements = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
self.witness_elements = Some(witness_elements);
// Short circuit for zero witness elements.
@@ -419,9 +417,7 @@ impl Decoder for WitnessDecoder {
// Take ownership of the decoder so we can consume it.
let decoder = core::mem::take(&mut self.element_length_decoder);
- let length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
- let element_length = encoding::cast_to_usize_if_valid(length)
- .map_err(|e| E(Inner::LengthPrefixInvalid(e)))?;
+ let element_length = decoder.end().map_err(|e| E(Inner::LengthPrefixDecode(e)))?;
// Store the element position in the index.
let position_after_rotation = self.cursor - witness_index_space;
@@ -808,8 +804,6 @@ pub struct WitnessDecoderError(WitnessDecoderErrorInner);
enum WitnessDecoderErrorInner {
/// Error decoding the vector length prefix.
LengthPrefixDecode(CompactSizeDecoderError),
- /// Length prefix exceeds 4,000,000.
- LengthPrefixInvalid(LengthPrefixExceedsMaxError),
/// Not enough bytes given to decoder.
UnexpectedEof(UnexpectedEofError),
}
@@ -824,7 +818,6 @@ impl fmt::Display for WitnessDecoderError {
match self.0 {
E::LengthPrefixDecode(ref e) => write_err!(f, "vec decoder error"; e),
- E::LengthPrefixInvalid(ref e) => write_err!(f, "vec decoder error"; e),
E::UnexpectedEof(ref e) => write_err!(f, "decoder error"; e),
}
}
@@ -837,7 +830,6 @@ impl std::error::Error for WitnessDecoderError {
match self.0 {
E::LengthPrefixDecode(ref e) => Some(e),
- E::LengthPrefixInvalid(ref e) => Some(e),
E::UnexpectedEof(ref e) => Some(e),
}
}
@@ -1338,7 +1330,7 @@ mod test {
let err = decoder.push_bytes(&mut slice).unwrap_err();
assert!(matches!(
err,
- WitnessDecoderError(WitnessDecoderErrorInner::LengthPrefixInvalid(_))
+ WitnessDecoderError(WitnessDecoderErrorInner::LengthPrefixDecode(_))
));
}
Risk score
Our methodology →Why this scored 33/100
Human-validated context
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
No validated notes yet.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.