content-defined-chunking · shipping · also CDC

Content-defined chunking

Content-defined chunking divides a byte stream using content-dependent boundary rules rather than only fixed offsets.

The aim is to retain reusable chunks despite shifted offsets; minimum-size suppression and maximum-size forced cuts can disrupt resynchronization. Boundary selection is separate from checking whole-chunk identity. [lbfs]

Relations

related

deduplication, fastcdc, rolling-hash, chunk-fragmentation

Citations

1. Athicha Muthitacharoen, Benjie Chen and David Mazieres. A Low-bandwidth Network File System. SOSP 2001, sections 3.1.1-3.1.3. (accessed 2026-10-12)
2. Content-defined chunking: boundaries are not identity (accessed 2026-10-12)

JSON: /data/terms/content-defined-chunking.json