← Watch feed
Add a few helpers to `EncodableByteIter`
Public commit record
68/100 · Adequate
What the developer wrote
Authored by Martin Habovstiak
Add a few helpers to `EncodableByteIter`
The `EncodableByteIter` internally tracks the position within the
current chunk which would theoretically allow using it to implement
`Future` or other state machine. However, to do this efficiently, it
needs to provide a few methods.
This commit implements those methods:
* `nth` - this is just an idiomatic override to improve performance in
general but can be also used to advance the iterator when tracking the
state.
* `peek_chunk` - this is analogous to `core::slice::Iter` having
`as_slice` method, though we need to take `&mut self` to skip the
empty chunks. In principle, `Option<NonEmpty<[u8]>>` would better
express the intent of empty slice being special but such is not used
yet.
✓ Descriptive subject✓ Names a concrete action or component✓ Provides detailed explanatory context
What changed, and why it matters
This commit adds two convenience methods, `nth` and `peek_chunk`, to a Rust iterator that walks over encoded bytes. It is a routine feature addition with no apparent security relevance. The code includes comments warning callers that using the iterator after it has finished is a bug, but that is normal API documentation, not a vulnerability disclosure.
Recommended action
No security action required. Review as normal code-quality/feature addition if desired.
Security signals we found
No strong security signals were identified.
Evidence from the diff
The patch extends EncodableByteIter in bitcoin-consensus-encoding with an idiomatic Iterator::nth override and a peek_chunk helper that exposes the remaining bytes of the current encoder chunk. The implementation uses checked arithmetic (checked_add) and slice indexing to avoid out-of-bounds access. API snapshot files and unit tests are updated accordingly. No bug fix, bounds-check bypass, cryptographic change, or unsafe code is introduced.
Changed components
consensus_encoding/src/encode/mod.rsconsensus_encoding/tests/iter.rsInspect captured patch +125 / −1
diff --git a/consensus_encoding/api/all-features.txt b/consensus_encoding/api/all-features.txt
index 386f4b9f..e6cd3255 100644
--- a/consensus_encoding/api/all-features.txt
+++ b/consensus_encoding/api/all-features.txt
@@ -1435,11 +1435,13 @@ pub fn bitcoin_consensus_encoding::Decoder6<A, B, C, D, E, F>::from(t: T) -> T
pub struct bitcoin_consensus_encoding::EncodableByteIter<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e>
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::new(encodable: &'e T) -> Self
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::peek_chunk(&mut self) -> &[u8]
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::clone::Clone for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::clone::Clone
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::clone(&self) -> Self
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::iter::traits::iterator::Iterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub type bitcoin_consensus_encoding::EncodableByteIter<'e, T>::Item = u8
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::next(&mut self) -> core::option::Option<Self::Item>
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::nth(&mut self, n: usize) -> core::option::Option<Self::Item>
impl<'e, T: core::fmt::Debug + bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::fmt::Debug for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::fmt::Debug
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
impl<'e, T> core::iter::traits::exact_size::ExactSizeIterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e, <T as bitcoin_consensus_encoding::Encodable>::Encoder: bitcoin_consensus_encoding::ExactSizeEncoder
diff --git a/consensus_encoding/api/alloc-only.txt b/consensus_encoding/api/alloc-only.txt
index 6e73e8f6..1521d4c1 100644
--- a/consensus_encoding/api/alloc-only.txt
+++ b/consensus_encoding/api/alloc-only.txt
@@ -1327,11 +1327,13 @@ pub fn bitcoin_consensus_encoding::Decoder6<A, B, C, D, E, F>::from(t: T) -> T
pub struct bitcoin_consensus_encoding::EncodableByteIter<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e>
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::new(encodable: &'e T) -> Self
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::peek_chunk(&mut self) -> &[u8]
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::clone::Clone for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::clone::Clone
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::clone(&self) -> Self
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::iter::traits::iterator::Iterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub type bitcoin_consensus_encoding::EncodableByteIter<'e, T>::Item = u8
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::next(&mut self) -> core::option::Option<Self::Item>
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::nth(&mut self, n: usize) -> core::option::Option<Self::Item>
impl<'e, T: core::fmt::Debug + bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::fmt::Debug for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::fmt::Debug
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
impl<'e, T> core::iter::traits::exact_size::ExactSizeIterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e, <T as bitcoin_consensus_encoding::Encodable>::Encoder: bitcoin_consensus_encoding::ExactSizeEncoder
diff --git a/consensus_encoding/api/no-features.txt b/consensus_encoding/api/no-features.txt
index 795a4566..f55d55a9 100644
--- a/consensus_encoding/api/no-features.txt
+++ b/consensus_encoding/api/no-features.txt
@@ -1016,11 +1016,13 @@ pub fn bitcoin_consensus_encoding::Decoder6<A, B, C, D, E, F>::from(t: T) -> T
pub struct bitcoin_consensus_encoding::EncodableByteIter<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e>
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::new(encodable: &'e T) -> Self
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::peek_chunk(&mut self) -> &[u8]
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::clone::Clone for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::clone::Clone
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::clone(&self) -> Self
impl<'e, T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::iter::traits::iterator::Iterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T>
pub type bitcoin_consensus_encoding::EncodableByteIter<'e, T>::Item = u8
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::next(&mut self) -> core::option::Option<Self::Item>
+pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::nth(&mut self, n: usize) -> core::option::Option<Self::Item>
impl<'e, T: core::fmt::Debug + bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e> core::fmt::Debug for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where <T as bitcoin_consensus_encoding::Encodable>::Encoder: core::fmt::Debug
pub fn bitcoin_consensus_encoding::EncodableByteIter<'e, T>::fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result
impl<'e, T> core::iter::traits::exact_size::ExactSizeIterator for bitcoin_consensus_encoding::EncodableByteIter<'e, T> where T: bitcoin_consensus_encoding::Encodable + ?core::marker::Sized + 'e, <T as bitcoin_consensus_encoding::Encodable>::Encoder: bitcoin_consensus_encoding::ExactSizeEncoder
diff --git a/consensus_encoding/src/encode/mod.rs b/consensus_encoding/src/encode/mod.rs
index edd61877..d8ac721a 100644
--- a/consensus_encoding/src/encode/mod.rs
+++ b/consensus_encoding/src/encode/mod.rs
@@ -184,6 +184,33 @@ pub struct EncodableByteIter<'e, T: Encodable + ?Sized + 'e> {
impl<'e, T: Encodable + ?Sized + 'e> EncodableByteIter<'e, T> {
/// Constructs a new byte iterator around a provided encodable.
pub fn new(encodable: &'e T) -> Self { Self { enc: encodable.encoder(), position: 0 } }
+
+ /// Returns the remaining bytes in the next non-empty chunk.
+ ///
+ /// The returned value is either a non-empty chunk of bytes that were not yielded yet,
+ /// immediately following the already-yielded bytes or empty slice if the encoder finished.
+ ///
+ /// This call can be paired with `nth` to mark bytes as processed.
+ ///
+ /// Just like with encoders or this iterator, attempting to use this type after this method
+ /// returned an empty slice will lead to unspecified behavior and is considered a bug in the
+ /// caller.
+ pub fn peek_chunk(&mut self) -> &[u8] {
+ // Can't use `.get(self.position..)` due to borrowck bug.
+ if self.position < self.enc.current_chunk().len() {
+ &self.enc.current_chunk()[self.position..]
+ } else {
+ loop {
+ if !self.enc.advance() {
+ return &[];
+ }
+ if !self.enc.current_chunk().is_empty() {
+ self.position = 0;
+ return self.enc.current_chunk();
+ }
+ }
+ }
+ }
}
// Manual impl rather than #[derive(Clone)] because derive would constrain `where T: Clone`,
@@ -201,6 +228,8 @@ impl<'e, T: Encodable + ?Sized + 'e> Iterator for EncodableByteIter<'e, T> {
fn next(&mut self) -> Option<Self::Item> {
loop {
if let Some(b) = self.enc.current_chunk().get(self.position) {
+ // length of slice is guaranteed to be at most `isize::MAX` thus is `n` so this cannot
+ // overflow.
self.position += 1;
return Some(*b);
} else if !self.enc.advance() {
@@ -209,6 +238,31 @@ impl<'e, T: Encodable + ?Sized + 'e> Iterator for EncodableByteIter<'e, T> {
self.position = 0;
}
}
+
+ fn nth(&mut self, mut n: usize) -> Option<Self::Item> {
+ // This could be in a loop but we intentionally unroll one iteration so that addition is
+ // only required at the beginning.
+ if let Some(b) = self.position.checked_add(n).and_then(|pos| self.enc.current_chunk().get(pos)) {
+ // length of slice is guaranteed to be at most `isize::MAX` thus is `n` so this cannot
+ // overflow.
+ self.position += n + 1;
+ return Some(*b);
+ }
+ n -= self.enc.current_chunk().len() - self.position;
+ if !self.enc.advance() {
+ return None;
+ }
+ loop {
+ if let Some(b) = self.enc.current_chunk().get(n) {
+ self.position = n + 1;
+ return Some(*b);
+ }
+ n -= self.enc.current_chunk().len();
+ if !self.enc.advance() {
+ return None;
+ }
+ }
+ }
}
impl<'e, T> ExactSizeIterator for EncodableByteIter<'e, T>
diff --git a/consensus_encoding/tests/iter.rs b/consensus_encoding/tests/iter.rs
index 5b83cc27..ff4fff4b 100644
--- a/consensus_encoding/tests/iter.rs
+++ b/consensus_encoding/tests/iter.rs
@@ -1,4 +1,4 @@
-use bitcoin_consensus_encoding::{ArrayEncoder, Encodable, EncodableByteIter, Encoder2};
+use bitcoin_consensus_encoding::{ArrayEncoder, Encodable, EncodableByteIter, Encoder2, Encoder3};
use hex::BytesToHexIter;
struct TestArray<const N: usize>([u8; N]);
@@ -28,6 +28,23 @@ impl<const N: usize, const M: usize> Encodable for TestCatArray<N, M> {
}
}
+struct TestCatArray3<const N: usize, const M: usize, const L: usize>([u8; N], [u8; M], [u8; L]);
+
+impl<const N: usize, const M: usize, const L: usize> Encodable for TestCatArray3<N, M, L> {
+ type Encoder<'e>
+ = Encoder3<ArrayEncoder<N>, ArrayEncoder<M>, ArrayEncoder<L>>
+ where
+ Self: 'e;
+
+ fn encoder(&self) -> Self::Encoder<'_> {
+ Encoder3::new(
+ ArrayEncoder::without_length_prefix(self.0),
+ ArrayEncoder::without_length_prefix(self.1),
+ ArrayEncoder::without_length_prefix(self.2),
+ )
+ }
+}
+
#[test]
fn hex_iter() {
let data = TestArray([255u8, 240, 9, 135]);
@@ -57,3 +74,50 @@ fn hex_iter_cat_encoder() {
let none = iter.next();
assert_eq!(none, None);
}
+
+// This is purposely testing that nth(0) works correctly.
+#[allow(clippy::iter_nth_zero)]
+#[test]
+fn nth() {
+ let data = TestArray([255u8, 240, 9, 135]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.nth(2).unwrap(), 9);
+ assert_eq!(byte_iter.nth(0).unwrap(), 135);
+ assert!(byte_iter.nth(42).is_none());
+
+ let data = TestCatArray([222u8, 173], [190u8, 239]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.nth(1).unwrap(), 173);
+ assert_eq!(byte_iter.nth(1).unwrap(), 239);
+ assert!(byte_iter.nth(42).is_none());
+
+ let data = TestCatArray3([0, 1, 2, 3], [4, 5, 6, 7], [8, 9, 10, 11]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.nth(5).unwrap(), 5);
+ assert_eq!(byte_iter.peek_chunk(), [6, 7]);
+}
+
+#[test]
+fn peek_chunk() {
+ let data = TestArray([255u8, 240, 9, 135]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.peek_chunk(), [255u8, 240, 9, 135]);
+ assert_eq!(byte_iter.next().unwrap(), 255);
+ assert_eq!(byte_iter.peek_chunk(), [240, 9, 135]);
+
+ let data = TestCatArray([222u8, 173], [190u8, 239]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.peek_chunk(), [222u8, 173]);
+ assert_eq!(byte_iter.next().unwrap(), 222);
+ assert_eq!(byte_iter.peek_chunk(), [173]);
+ assert_eq!(byte_iter.next().unwrap(), 173);
+ assert_eq!(byte_iter.peek_chunk(), [190, 239]);
+
+ let data = TestCatArray([], [21u8, 42]);
+ let mut byte_iter = EncodableByteIter::new(&data);
+ assert_eq!(byte_iter.peek_chunk(), [21, 42]);
+ assert_eq!(byte_iter.next().unwrap(), 21);
+ assert_eq!(byte_iter.peek_chunk(), [42]);
+ assert_eq!(byte_iter.next().unwrap(), 42);
+ assert!(byte_iter.next().is_none());
+}
Risk score
Our methodology →Why this scored 13/100
Human-validated context
Community notes
Notes can correct, qualify, or add evidence to the AI analysis. Every note shown here has been validated by a human moderator.
No validated notes yet.
The AI analysis stands alone for now. Submit a note if you can add evidence or important context.