DedupSearch
DedupSearch is a two-phase research algorithm that scans unique stored chunks for byte strings and then resolves matches to logical files.
The physical phase records complete matches and boundary prefixes and suffixes; the logical phase follows file recipes to join partial matches in file order. Physical adjacency alone cannot establish a cross-chunk match. [1]
Unlike a persistent inverted index, the result database is built for the submitted keywords. Recipe traversal and partial-result storage remain costs even when duplicate payload is scanned only once. The evaluation includes text, source code and binary VM backups, not just natural-language terms. [1]
Relations
- related
- Data deduplication, Destor, IDEA index
Citations
Further reading
Sources & caveats
- Full-text source review, especially sections 3, 5 and 6; no native DedupSearch benchmark or paper-result reproduction.