hybrid-log · shipping · also HybridLog, FASTER hybrid log

Hybrid log

A hybrid log combines an in-memory region supporting in-place updates with read-only records backed by persistent storage.

A mutable region permits repeated updates without a fresh append each time; updates to read-only records create tail copies. An epoch-coordinated safe read-only boundary defers read-modify-write copying while an older in-place update remains possible. [1]

Checkpoint choice: Fold-over freezes and flushes the main log through its tail, so subsequent updates need new records. A snapshot instead writes the in-memory state to a separate checkpoint file. In-memory completion does not itself establish durability. [2]

Recovery position: Concurrent prefix recovery identifies a committed prefix per session rather than one global issue-time cutoff. [4] The documented API also reports an exception list for pending operations excluded from a checkpoint; a sequence-number maximum alone is insufficient. [2]

Cleaning copies live records forward; truncation expires an address range. Scanning log records can return old versions and tombstones, unlike iteration over current keys. [3]

Relations

related

Key-value separation, Value log, Database checkpoint

Advantages

• Repeated hot-record updates can avoid index-pointer replacement and whole-record copying. [1]

Limitations

• The 2018 throughput evaluation excludes checkpointing, recovery and expiration-based garbage-collection costs. [1]
• Compaction is bounded by the safe read-only address, not simply by the tail. [3]

Citations

1. Chandramouli et al. FASTER: A Concurrent Key-Value Store with In-Place Updates. SIGMOD 2018, sections 2.1, 6 and 7.1. (accessed 2026-11-02)
2. FASTER documentation: FasterKV Checkpointing and Recovery, snapshot, fold-over and commit-point semantics. (accessed 2026-11-02)
3. FASTER documentation: FasterKV Basics, Log Scan and Log Compaction. (accessed 2026-11-02)
4. Prasaad, Chandramouli and Kossmann. Concurrent Prefix Recovery: Performing CPR on a Database. SIGMOD 2019, section 2. (accessed 2026-11-02)

Further reading

5. Key-value separation: the cleaning bill moves elsewhere (accessed 2026-11-02)

JSON: /data/terms/hybrid-log.json