hesela.devSpace amplification
Database space amplification quantifies storage overhead relative to logical data size under an explicitly stated total-space or excess-space convention.
For total-space accounting, divide stored database bytes by represented logical data bytes. State whether stored bytes include SST files, WAL, metadata, retained versions and temporary compaction output, and whether the denominator is uncompressed live keys and values. Report steady-state footprint separately from peak space during a merge. [1] Compression and compaction affect different parts of this accounting. [2] RocksDB Universal's max_size_amplification_percent estimates additional space relative to its oldest run; it is not the total-space ratio or a guaranteed disk-space cap. An insert-only workload can violate the heuristic's overwrite assumption. [3] Sarkar et al. instead define excess-space amplification using invalidated-entry bytes over unique-entry bytes; those conventions are not interchangeable. [4]
Relations
- related
- lsm-tree, leveled-compaction, tiered-compaction, database-write-amplification
Citations
- RocksDB project. Tuning Guide: amplification factors and their accounting boundaries.
- RocksDB project. Space Tuning: compression, compaction and space usage.
- RocksDB project. Universal Compaction: run selection, temporary space and amplification heuristic.
- Sarkar, Staratzis, Zhu and Athanassoulis. Constructing and Analyzing the LSM Compaction Design Space. PVLDB 14(11), 2216-2229 (2021), DOI: 10.14778/3476249.3476274; updated arXiv:2202.04522v2 (2022), sections 2-5.
- Hesela. Bloom filters: optimize the lookup, not each filter; compaction and measurement boundaries.