Walrus

Walrus is a high-performance Log based storage engine for Rust applications that need durable, topic-aware streams with predictable latency.

Highlights

  • High throughput append and batch pipelines tuned for streaming workloads
  • Topic isolation with independent read/write offsets and checkpointing
  • Configurable durability via fsync scheduling and read consistency modes
  • Batch APIs that combine atomic multi-entry writes with capped batch reads
  • Benchmark suite and visualization helpers for repeatable performance testing

Blog posts

walrus v0.2.0: beating kafka at their own game

walrus v0.1.0: ingesting data at memory speeds

Quick Start

Add Walrus to your project by declaring the dependency in Cargo.toml:

[dependencies]
walrus-rust = "0.1.0"

Create a WAL instance and start reading and writing topics:

use walrus_rust::{Walrus, ReadConsistency};

let wal = Walrus::new()?;                     // StrictlyAtOnce by default
wal.append_for_topic("my-topic", b"hello")?;  // append a payload

if let Some(entry) = wal.read_next("my-topic", true)? {
    println!("read {:?}", String::from_utf8_lossy(&entry.data));
}

For peek semantics, pass false to read_next so the cursor stays put. Prefer Walrus::with_consistency_and_schedule when you need to tune read persistence or fsync cadence.

Where to Next