End-to-end data protection
End-to-end data protection is an integrity scheme whose generation and verification points span an explicitly defined data path.
Name both endpoints. Metadata generated below the application cannot by itself validate what happened before generation; a consistently sealed wrong payload can pass. This is distinct from persistence, freshness and successful recovery after damage. Saltzer, Reed and Clark's file-transfer argument distinguishes endpoint verification from useful lower-layer checks that reduce retries. A full-object checksum defines byte coverage, not the endpoints of the protected path. [4]
A successful verification return need not mean every interval was checked: PI escape handling is a concrete counterexample. Recording performed and skipped checks separately distinguishes acceptance from evaluation. [5]
A separate checksum pass can read the page cache rather than the storage device. FIVER's shared-buffer hashing and hybrid verification illustrate why received-buffer agreement, filesystem readback and power-loss recovery have different boundaries. Cache behavior alone does not certify all write-fault detection. [6]
Relations
- related
- Enterprise SSD, Protection information (PI), Lost write, Misdirected write, Checksum, Merkle tree, Full-object checksum, Data Integrity Extensions (DIX)
Citations
- Saltzer, Reed and Clark (1984): End-to-End Arguments in System Design, ACM TOCS 2(4), 277-288; MIT reprint
- Hesela: PI escape conditions, source hashes and sixteen control-flow fixtures
- Arslan and Alhussen (2018): Fast Integrity Verification for High-Speed File Transfers; arXiv:1811.01161v1, sections III and IV.B; DOI 10.48550/arXiv.1811.01161
Further reading
- End-to-end data protection
- Linux kernel documentation: Data Integrity, protection boundaries and automatic metadata generation.
- Hesela: checksum, identity and freshness boundaries with executable synthetic examples
Sources & caveats
- PI escape evidence is version-scoped: historical NVMe 1.4c plus extracted Linux v6.12 control flow with a checksum stub; not a controller or running-kernel test.
- The cited experiment executes sixteen selected fixtures, including zero/eight-byte metadata prefixes, against an unchanged verifier with user-space adapters and a fixed-value checksum stub; call counts are not counts of checked intervals.