mod compression

module compression

Functions

fn detect_compression(bytes: &[u8]) -> Compression

Detect compression format from the first bytes of a file.

• 1f 8b = gzip

• 28 b5 2f fd = zstd

• Otherwise = uncompressed

fn detect_compression_from_extension(path: &Path) -> Compression

Detect compression format from a file extension.

Returns Compression::Gzip for .gz, Compression::Zstd for .zst, Compression::None otherwise.

fn gzip_writer(path: &Path) -> io::Result<flate2::write::GzEncoder<std::fs::File>>

Creates a gzip-compressed writer wrapping a file at the given path.

fn read_file_contents(path: &Path) -> Result<FileContents, Box<dyn std::error::Error>>

Reads file contents, decompressing if needed.

Detection strategy: 1. Read first 2 bytes to check magic bytes. 2. If gzip: decompress entire file to a String. 3. If uncompressed and < 64 KiB: read_to_string. 4. If uncompressed and >= 64 KiB: memory-mapped I/O.

fn zstd_writer(path: &Path) -> io::Result<zstd::stream::write::AutoFinishEncoder<'static, std::fs::File>>

Creates a zstd-compressed writer wrapping a file at the given path.

Returns a finished writer ready for streaming line writes; the caller is responsible for dropping the writer to flush the final frame. Available only when the zstd Cargo feature is enabled.

Enums

enum Compression

Detected compression format based on magic bytes.

Zstd is only constructed when the zstd Cargo feature is enabled. Builds without the feature treat .zst files as opaque bytes and return an error from read_file_contents indicating the feature is required.

None

Gzip

Zstd

zstd frame, magic 28 B5 2F FD. Build with --features zstd.

enum FileContents

Holds file contents either as an owned String or a memory-mapped region.

Owned(String)

Mapped(memmap2::Mmap)

Implementations

impl FileContents

Functions

fn as_str(&self) -> Result<&str, std::str::Utf8Error>