idea-index · research · also Inverted Deduplication-Aware Index

IDEA index

IDEA is a research inverted index that maps terms to shared chunks and then maps those chunks to the files that reference them.

Whitespace-aligned chunking keeps textual terms inside chunks. The index separates term-to-chunk, chunk-to-file and file-to-path mappings. Its prototype integrates Lucene-based indexing with Destor. [1]

Already-chunked inputs and compressed documents need additional handling; arbitrary byte-string search is not interchangeable with whitespace-aligned term indexing. The evaluation uses Linux source versions and Wikipedia dumps. Its ranking prototype supports single-term queries; proposed multi-term ranking is not an implemented result. [1]

Relations

related
Data deduplication, Destor, DedupSearch

Citations

  1. Levi et al. Physical vs. Logical Indexing with IDEA: Inverted Deduplication-Aware Index. FAST 2024, pp. 243-258. (accessed 2026-11-17)

Further reading

  1. Hesela: searching shared chunks and reconstructing file boundaries. (accessed 2026-11-17)
Sources & caveats
  • Full-text source review, especially sections 4-8 and the artifact appendix; no native IDEA benchmark or paper-result reproduction.

JSON: /data/terms/idea-index.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. IDEA index. Hesela. 2026-11-17. https://hesela.dev/terms/idea-index/