API Reference

Walrus exposes a small set of ergonomic constructors and topic-centric read/write APIs. This page summarizes the most frequently used functions and data types. For tuning guidance, pair it with the Configuration overview.

Constructors

Walrus::new() -> std::io::Result<Self>

Creates a WAL instance with ReadConsistency::StrictlyAtOnce and the default 1 s fsync cadence.

Walrus::with_consistency(mode: ReadConsistency) -> std::io::Result<Self>

Configures the read checkpointing strategy while keeping the default fsync schedule.

Walrus::with_consistency_and_schedule(mode: ReadConsistency, schedule: FsyncSchedule) -> std::io::Result<Self>

Gives full control over read persistence and fsync cadence.

Walrus::new_for_key(key: &str) -> std::io::Result<Self>

Creates a namespaced WAL rooted at wal_files/<sanitized-key>/. Ideal when different workloads need independent durability surfaces.

Walrus::with_consistency_for_key(key: &str, mode: ReadConsistency) -> std::io::Result<Self>

Combines keyed storage with a custom read consistency mode.

Walrus::with_consistency_and_schedule_for_key(key: &str, mode: ReadConsistency, schedule: FsyncSchedule) -> std::io::Result<Self>

Full configuration control plus per-key isolation. Pair with WALRUS_INSTANCE_KEY to apply the same behaviour to default constructors.

Write APIs

append_for_topic(&self, topic: &str, data: &[u8]) -> std::io::Result<()>

Appends a single entry. Topics are created lazily. Returns ErrorKind::WouldBlock if a batch write is in flight for the topic.

batch_append_for_topic(&self, topic: &str, batch: &[&[u8]]) -> std::io::Result<()>

Writes up to 2,000 entries atomically (bounded to ~10 GB including metadata). Uses io_uring on Linux when the fd backend is active and falls back to sequential writes under the mmap backend.

Read APIs

read_next(&self, topic: &str, checkpoint: bool) -> std::io::Result<Option<Entry>>

Returns the next entry for the topic. Passing checkpoint = true advances and persists the cursor according to the configured consistency mode; false leaves offsets untouched for peek semantics.

batch_read_for_topic(&self, topic: &str, max_bytes: usize, checkpoint: bool) -> std::io::Result<Vec<Entry>>

Streams up to max_bytes of payload (and never more than 2,000 entries). Honors the same checkpoint semantics as read_next. Validation ensures every entry passes checksum verification before it is returned to the caller.

Backend Toggles

use walrus_rust::{enable_fd_backend, disable_fd_backend};

enable_fd_backend();  // default on Linux
disable_fd_backend(); // forces mmap paths

• FD backend unlocks io_uring fast paths for batch operations.
• Mmap backend keeps compatibility across non-Linux platforms.

Set WALRUS_QUIET=1 to silence debug output when switching.

Data Types

Entry

pub struct Entry {
    pub data: Vec<u8>,
}

Entries contain opaque payload bytes; metadata is stripped during read parsing after checksum validation.

Storage Layout

wal_files/
├── 1700000000                  # Default instance log file (preallocated to 1 GB)
├── read_offset_idx_index.db    # Default instance read-offset index
├── analytics/                  # Keyed instance for "analytics"
│   ├── 1700000100
│   └── read_offset_idx_index.db
└── transactions/               # Keyed instance for "transactions"
    ├── 1700000200
    └── read_offset_idx_index.db

• Blocks are 10 MB each, 100 per file, yielding 1 GB log segments.
• Index files persist reader cursors for crash recovery.

Benchmarks & Tooling

• make bench-and-show-reads runs read benchmarks and opens plots (requires pandas and matplotlib).
• make show-writes, make show-reads, and make show-scaling visualize CSV output generated during benchmarking.
• Environment variables such as WALRUS_FSYNC, WALRUS_THREADS, and WALRUS_DURATION offer quick tuning without editing code.

Refer back to Getting Started for installation details and environment variable descriptions.