Describes something that can slice objects in a defined, repeatable manner
Chunkers must meet the following properties:
1.) Data must be split into one or more chunks
2.) Data must be identical to original after a simple reconstruction by concatenation
3.) The same data and settings must produce the same slices every time
4.) Chunkers (that have a max size) should not produce any chunks larger than their
5.) Chunkers (that have a min size) should produce, at most, 1 slice smaller than its
and should only do as such when there is not enough data left to produce a min size chunk
For the time being given the lack of existential types, Chunkers use Box<dyn Read + 'static>.
If/when existential types get stabilized in a way that helps, this will be switched to an existential type, to drop the dynamic dispatch.
Chunkers should, ideally, contain only a small number of settings for the chunking algorithm, and should there for be cloneable with minimal overhead. Ideally, they should implement copy, but that is not supplied as a bound to increase the flexibility in implementation
The Send bound on the Read is likely temporary, it is currently required to make the streams feature work properly.
The return type of the functions in this trait is an iterator over the chunks of their input.
The returned iterator must be owned, hence the 'static bound.
fn chunk_boxed(&self, read: Box<dyn Read + 'static + Send>) -> Self::Chunks
Core function, takes a boxed owned Read and produces an iterator of Vec
Convenience function that boxes a bare Read for you, and passes it to
This will be the primary source of interaction wth the API for most use cases
Convenience function that boxes an AsRef<[u8]> wrapped in a cursor and passes it to
chunk_boxed. Implementations are encouraged to overwrite when sensible.
This method is provided to ensure API compatibility when implementations are using memory mapped io or the like. When chunkers can sensibly override this, they are encouraged to, as it would otherwise result in a performance overhead for consumers using memmaped IO.