# A chunk audit does not measure restore blast radius **Result:** two deduplicated catalogs can have the same chunks, chunk sizes, occurrence counts and audit-detection probability, yet differ in how many files cannot be completely restored after the same damage. Even per-chunk file counts do not determine the union of files affected by several damaged chunks. This is an exact synthetic counterexample plus a small native file-I/O probe. It is not a new theorem, a benchmark, a replication of a paper's private dataset, or a reliability measurement of a backup product. ## Why This Belongs In Hesela The existing [storage-proof sampling analysis][sampling] asks whether a selected sample encounters damage. The [scrubbing analysis][scrub] distinguishes detection from durability. The [deduplication restore analysis][restore] separates space savings from restore cost. This packet connects those questions to the file-to-chunk dependency graph, without creating another article or concept ID. ## Four Quantities To Keep Separate Let the catalog contain N unique chunk identities. A file recipe is an ordered list of identities; repeated positions count as logical references. Let D be a fixed set of damaged identities, after other replicas or coding recovery have been exhausted. All chunks have equal payload size in this example. - Physical damage counts distinct damaged identities: M = |D|. - Missing logical payload counts every occurrence of an identity in D. - Incomplete files are the union of files referencing at least one identity in D. - Audit detection is the probability that the selected checks intersect D. For C distinct identities chosen uniformly without replacement: `P(detect | fixed D) = 1 - choose(N-M,C) / choose(N,C)`. This counts samples avoiding D and subtracts from one. The recipe graph does not appear in that expression. It is not the probability that a passing repository is intact: no prior distribution of damage has been supplied. ## Same Audit, One Or Four Incomplete Files A, B, C and D denote four distinct equal-size chunks. Each row below describes four separate files, not a single concatenated stream. The failed identity is A. The check selects two of the four unique identities uniformly. | Catalog | Four file recipes | Logical references to A | Audit detects A | Incomplete files | | --- | --- | --- | --- | --- | | Concentrated | AAAA; BBBB; CCCC; DDDD | 4 of 16 | 1/2 | 1 of 4 | | Spread | ABCD; ABCD; ABCD; ABCD | 4 of 16 | 1/2 | 4 of 4 | Both catalogs have four occurrences of each chunk and four equally long files. A uniform two-position sample of the 16 logical references also has the same detection probability in both: `1 - choose(12,2)/choose(16,2) = 9/20`. Changing the sampling unit does not recover information missing from the graph. The logical lengths of incomplete files total 4 chunk units in the first catalog and 16 in the second. **That is not 16 units of destroyed payload.** Only four logical positions refer to the missing identity in either catalog; other content may be salvageable. A full-file recovery contract and a recoverable-byte fraction are different measures. An intact-store coverage question also changes: selecting two unique chunks includes every dependency of exactly two concentrated files, but of no spread file. This says which recipes are fully selected, not which checks pass. Metadata, keys and application-level consistency remain additional requirements. ## Even Fan-Out Counts Lose Overlap Now every chunk occurs twice and is referenced by two different files. Both catalogs have four two-chunk files. The damaged set is {A, B}. | Catalog | Four file recipes | Fan-out of every chunk | Audit detection, C=2 | Incomplete files | | --- | --- | --- | --- | --- | | Paired | AB; AB; CD; CD | 2 files | 5/6 | 2 of 4 | | Cycle | AB; BC; CD; DA | 2 files | 5/6 | 3 of 4 | Adding the fan-outs of A and B gives four in both cases, but double-counts files that reference both. Formally, if F(c) is the set of file IDs referencing c, the affected set is `union(F(c) for c in D)`, not a sum of its cardinalities. Inclusion-exclusion explains the difference: the intersection has size two for the paired catalog and size one for the cycle. Thus a duplication histogram does not identify restore impact; neither does a table containing only each chunk's file count. A catalog graph or an appropriate joint dependency summary is needed for a specified damaged set. ## The Older Literature Behind The Distinction Bhagwat and colleagues [S1, sections 4-6] distinguished weights based on dependent file counts and dependent file sizes when assigning replicas. Their historical experiments used Internet Archive and newspaper corpora, artificially small devices and randomized device-loss scenarios without repair. The reported robustness was available data conditional on a fraction of unavailable devices, not an annual loss rate. Their warning to align the evaluation measure with the chosen weighting matters here: affected-file bytes are not the same as bytes belonging to missing chunks. Li, Lillibridge and Uysal [S2, section 7, Figures 7-8] separately showed how deduplication can change loss likelihood and severity. Their illustrative identical-block comparison has equal expected lost-block count while its all-loss outcome changes. Section 3's fixed-horizon model assumes independent disk failures. We use their metric distinction, not their assumptions as evidence about modern deployments. The packet's catalog-and-sampling witnesses are Hesela constructions, not results extracted from either paper. Restic's version-pinned documentation [S3] provides an operational boundary: `check --read-data-subset` selects pack files. A pack may contain multiple chunks, so neither chunk-uniform formula above describes that command merely because a percentage appears in its argument. The documentation also warns that repeated random subsets do not ensure complete coverage. This is a documentation comparison, not a restic fault test. ## Reproduction And Evidence Run in this directory using Python 3.12 with no third-party packages: ```sh python3 model.py --check results.json python3 -m unittest -v test_model.py python3 -O -m unittest -v test_model.py python3 native_probe.py --output replay-native.json ``` The exact model emits reduced rational fractions and its source hash. `native_probe.py` creates four real 64-byte content files addressed by SHA-256, round-trips a JSON recipe catalog and confirms all intact restores. It then either deletes a chunk file or overwrites it with a wrong 64-byte payload. Every selected chunk is read and checked against its trusted expected digest. Every recipe is actually assembled or rejected. Temporary fixture files are automatically removed. The eight recorded configurations cover four catalogs and both fault modes: 48 complete two-identity sample enumerations, 32 baseline restores and 32 post-injection restore attempts. Native outcomes are checked against the exact model. This probe validates the deliberately tiny fixture only; it does not implement PDP, PoR, replica repair, cryptographic catalog authentication, crash durability or a production deduplication engine. Independent oracles test 1,792 finite-population/damage/sample combinations, 144 expected whole-recipe selection calculations and 288 dependency-union calculations. These are exhaustive finite checks, not statistical trial counts or confidence intervals. No seed is needed. Frozen native output includes its runtime and source hashes; its case observations must replay exactly even when environment labels differ. ## Practical Use And Limits Record the audit's population and unit (packs, unique chunks, logical positions or complete restores), selection rule, completed checks and object version. Retain the recipe relationships needed to map damaged identities to affected restore objects. Report distinct missing bytes, missing logical payload, incomplete-object count and their logical sizes separately. This does not establish an optimal sampling or replication policy. A high-impact chunk might merit additional checks, but changing the design also changes inclusion probabilities and the interpretation of a clean result. Placement, uneven chunk sizes, metadata loss, keys, repair, correlated failures, updates and malicious responses require additional models or native tests. Identical recipes are separate requested objects here; no extra file-level deduplication of their identities is assumed. **Publication scope:** executable evidence accompanying the existing storage-proof sampling analysis. The code and observations document a synthetic fixture; publication does not validate a production backup product or cryptographic protocol. ## Sources [S1]: https://www.crss.us/media/pubs/1bd42e936db567ec1c9a5d30b8ba7b46d4ad8a26.pdf [S2]: https://shiftleft.com/mirrors/www.hpl.hp.com/personal/Mark_Lillibridge/Reliability/journal.pdf [S3]: https://restic.readthedocs.io/en/v0.18.0/045_working_with_repos.html#checking-integrity-and-consistency [sampling]: https://hesela.com/analyses/encrypted-deduplication-trust-boundaries/#sampling [scrub]: https://hesela.com/analyses/scrubbing-detection-not-durability/ [restore]: https://hesela.com/analyses/deduplication-restore-fragmentation/ - [S1] Bhagwat et al. Providing High Reliability in a Minimum Redundancy Archival Storage System. MASCOTS 2006, pp. 413-421. - [S2] Li, Lillibridge and Uysal. Reliability Analysis of Deduplicated and Erasure-Coded Storage. SIGMETRICS Performance Evaluation Review 38(3), pp. 4-9, 2010. Journal author copy on an HP Labs archive mirror. - [S3] Restic 0.18.0, Working with repositories, Checking integrity and consistency. Specific locations, access date, primary PDF hashes, limitations and rights are recorded in `sources.json`. PDFs are retained privately for verification, not redistributed. No DOI was guessed.