Configuring Walrus

Walrus keeps the API surface small while letting you tune durability, storage layout, and performance characteristics. This guide gathers the knobs you can adjust and explains when each option fits.

Instance Constructors

Use the provided constructors to set defaults for read consistency and fsync cadence at startup:

• Walrus::new() — strict read checkpoints, 1 s fsync cadence.
• Walrus::with_consistency(ReadConsistency) — pick the read model while keeping the default fsync schedule.
• Walrus::with_consistency_and_schedule(ReadConsistency, FsyncSchedule) — full control over both read checkpoints and fsync behaviour.
• Walrus::new_for_key(&str) and the _for_key variants — place the instance under wal_files/<sanitized-key>/ for workload isolation.

You can also set WALRUS_INSTANCE_KEY=<key> before constructing Walrus to namespace the default constructors without changing call-sites.

Read Consistency Modes

Walrus persists read offsets according to the configured consistency:

ReadConsistency::StrictlyAtOnce

• Persists the cursor on every checkpointed read.
• Guarantees at-most-once replays after restarts.
• Suitable when duplicate processing is unacceptable.

let wal = Walrus::with_consistency(ReadConsistency::StrictlyAtOnce)?;

ReadConsistency::AtLeastOnce { persist_every: u32 }

• Buffers cursor updates and flushes them every N reads.
• Higher throughput, but the last window of entries may repeat after a crash.
• Works well for streaming pipelines that can tolerate occasional duplicates.

let wal = Walrus::with_consistency(
    ReadConsistency::AtLeastOnce { persist_every: 5_000 },
)?;

Fsync Scheduling

Durability behaviour is controlled via FsyncSchedule:

Schedule	Behaviour	Typical Use
FsyncSchedule::SyncEach	Flush to disk after every write; slowest but safest	Financial or safety-critical systems
FsyncSchedule::Milliseconds(n)	Background thread flushes every n milliseconds (default: 1000)	Balanced throughput vs. durability
FsyncSchedule::NoFsync	Leave writes in the OS page cache	Performance testing or workloads with external durability

use walrus_rust::{FsyncSchedule, ReadConsistency, Walrus};

let wal = Walrus::with_consistency_and_schedule(
    ReadConsistency::AtLeastOnce { persist_every: 1_000 },
    FsyncSchedule::Milliseconds(2_000),
)?;

Backend Selection

Walrus ships with two storage backends:

• FD backend (enable_fd_backend()): default on Linux; unlocks io_uring acceleration for batch operations.
• Mmap backend (disable_fd_backend()): portable fallback when file descriptor-backed io_uring is unavailable.

Toggling the backend is a process-wide decision. Set WALRUS_QUIET=1 to suppress log noise while switching.

use walrus_rust::{enable_fd_backend, disable_fd_backend};

enable_fd_backend();  // Linux fast-path
// ...
disable_fd_backend(); // enforce mmap-only behaviour

Data Directory & Namespacing

• Files live under wal_files/ by default.
• Set WALRUS_DATA_DIR=/custom/path to relocate the entire tree.
• Use the _for_key constructors or WALRUS_INSTANCE_KEY to isolate workloads under wal_files/<sanitized-key>/.

Example layout:

wal_files/
├── 1700000000                  # default instance log (1 GB preallocated)
├── read_offset_idx_index.db    # default read-offset index
├── analytics/
│   ├── 1700000100
│   └── read_offset_idx_index.db
└── transactions/
    ├── 1700000200
    └── read_offset_idx_index.db

Runtime Environment Variables

Variable	Purpose
WALRUS_DATA_DIR	Override the root directory for WAL files and indexes.
WALRUS_INSTANCE_KEY	Namespace instances created via default constructors.
WALRUS_QUIET	Suppress debug output when flipping backends.

Benchmark & Tooling Variables

Walrus includes benchmarking harnesses under Makefile targets. Tune them by exporting the following variables before running a benchmark:

Variable	Description
WALRUS_FSYNC / FSYNC	Set fsync cadence (sync-each, no-fsync, async, <n>ms).
WALRUS_THREADS / THREADS	Control thread counts (<n> or <start-end>).
WALRUS_DURATION	Total benchmark duration (30s, 2m, 1h, …).
WALRUS_WRITE_DURATION	Override write phase duration.
WALRUS_READ_DURATION	Override read phase duration.
WALRUS_BATCH_SIZE / BATCH	Entries per batch for batch benchmarks.

Example:

FSYNC=sync-each THREADS=16 WALRUS_DURATION=5m make bench-scaling

Choosing a Profile

Goal	Suggested Settings
Maximum throughput	ReadConsistency::AtLeastOnce { persist_every: 10_000 }, FsyncSchedule::NoFsync, FD backend enabled.
Balanced production	ReadConsistency::AtLeastOnce { persist_every: 1_000 }, FsyncSchedule::Milliseconds(1_000).
Maximum durability	ReadConsistency::StrictlyAtOnce, FsyncSchedule::SyncEach, consider smaller persist_every if using AtLeastOnce.

Combine these with keyed instances when separate workloads need different durability guarantees on the same host.