end-to-end-data-protection · shipping

End-to-end data protection

End-to-end data protection is an integrity scheme whose generation and verification points span an explicitly defined data path.

Name both endpoints. Metadata generated below the application cannot by itself validate what happened before generation; a consistently sealed wrong payload can pass. This is distinct from persistence, freshness and successful recovery after damage. Saltzer, Reed and Clark's file-transfer argument distinguishes endpoint verification from useful lower-layer checks that reduce retries. A full-object checksum defines byte coverage, not the endpoints of the protected path. [4]

A successful verification return need not mean every interval was checked: PI escape handling is a concrete counterexample. Recording performed and skipped checks separately distinguishes acceptance from evaluation. [5]

A separate checksum pass can read the page cache rather than the storage device. FIVER's shared-buffer hashing and hybrid verification illustrate why received-buffer agreement, filesystem readback and power-loss recovery have different boundaries. Cache behavior alone does not certify all write-fault detection. [6]

Relations

related
Enterprise SSD, Protection information (PI), Lost write, Misdirected write, Checksum, Merkle tree, Full-object checksum, Data Integrity Extensions (DIX)

Citations

  1. Saltzer, Reed and Clark (1984): End-to-End Arguments in System Design, ACM TOCS 2(4), 277-288; MIT reprint (accessed 2026-10-16)
  2. Hesela: PI escape conditions, source hashes and sixteen control-flow fixtures (accessed 2026-10-17)
  3. Arslan and Alhussen (2018): Fast Integrity Verification for High-Speed File Transfers; arXiv:1811.01161v1, sections III and IV.B; DOI 10.48550/arXiv.1811.01161 (accessed 2026-10-23)

Further reading

  1. End-to-end data protection (accessed 2026-03-26)
  2. Linux kernel documentation: Data Integrity, protection boundaries and automatic metadata generation. (accessed 2026-10-11)
  3. Hesela: checksum, identity and freshness boundaries with executable synthetic examples (accessed 2026-10-11)
Sources & caveats
  • PI escape evidence is version-scoped: historical NVMe 1.4c plus extracted Linux v6.12 control flow with a checksum stub; not a controller or running-kernel test.
  • The cited experiment executes sixteen selected fixtures, including zero/eight-byte metadata prefixes, against an unchanged verifier with user-space adapters and a fixed-value checksum stub; call counts are not counts of checked intervals.

JSON: /data/terms/end-to-end-data-protection.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. End-to-end data protection. Hesela. 2026-11-13. https://hesela.dev/terms/end-to-end-data-protection/