{
  "slug": "idedup",
  "term": "iDedup",
  "aliases": [
    "latency-aware inline deduplication"
  ],
  "definition": "iDedup is a research inline deduplication design that trades some capacity savings for longer duplicate-block sequences and memory-resident fingerprint lookup.",
  "note": "The FAST 2012 design matches runs consecutive in both file-block order and existing disk-block locations. A tunable sequence-length threshold limits fragmentation. Its LRU fingerprint cache can miss duplicates and competes with the file buffer cache. Deduplication runs during NVRAM destaging, before disk allocation. [paper]\n\nMemory-resident lookup does not make all metadata volatile. Shared blocks retain persistent reference counts, whose updates can require disk I/O. A fingerprint-cache miss is not proof that no duplicate exists. [paper]\n\nThe evaluation replayed two enterprise CIFS traces on three spinning disks. The headline 60-70% concerns achievable deduplication relative to a maximum, not a 60-70% reduction of all input bytes. The reported 2-4% mean-response-time differences occur at Threshold-8 with a 1 GB metadata cache, not as a universal bound or p99 guarantee. [paper]\n\nA Hesela illustration reads the same twelve block addresses in three orders: one run, three runs or twelve runs. Unique-block count stays twelve. Runs describe address adjacency, not measured seeks or elapsed time; caching, scheduling and device geometry are excluded. This is not an iDedup implementation or paper reproduction. [layout]",
  "status": "research",
  "advantages": [],
  "limitations": [],
  "related": [
    "deduplication",
    "chunk-fragmentation",
    "sparse-indexing",
    "gogetafs"
  ],
  "supersedes": [],
  "supersededBy": [],
  "firstSeen": null,
  "lastSeen": null,
  "updated": "2026-11-21",
  "citations": [
    {
      "id": "paper",
      "title": "Srinivasan et al. iDedup: Latency-aware, Inline Data Deduplication for Primary Storage. FAST 2012, updated 10 February.",
      "url": "https://www.usenix.org/legacy/events/fast12/tech/full_papers/Srinivasan2-10-12.pdf",
      "accessed": "2026-11-17"
    },
    {
      "id": "record",
      "title": "USENIX FAST 2012 publication record and updated-paper link.",
      "url": "https://www.usenix.org/conference/fast12/idedup-latency-aware-inline-data-deduplication-primary-storage",
      "accessed": "2026-11-17"
    },
    {
      "id": "layout",
      "title": "Hesela: reproducible synthetic block-layout examples (JSON).",
      "url": "https://hesela.dev/research/idedup-layout.json",
      "accessed": "2026-11-17"
    },
    {
      "id": "code",
      "title": "Hesela: block-layout illustration source (JavaScript).",
      "url": "https://hesela.dev/research/idedup-layout.mjs",
      "accessed": "2026-11-17"
    },
    {
      "id": "hesela-dedup-restore",
      "title": "Hesela: Deduplication ratio does not predict restore cost.",
      "url": "https://hesela.com/analyses/deduplication-restore-fragmentation/",
      "accessed": "2026-11-17"
    }
  ],
  "provenance": [
    "Paper sections 3.2-3.4, 4.1.4, 4.2.3 and 5.2-5.7. Historical research, not a claim about present NetApp products. The full updated paper and evaluation figures were inspected; no native benchmark or trace replay was performed.",
    "The synthetic illustration uses an explicit adjacency rule and no fitted parameters. Execute the linked JavaScript with Node.js to print its JSON; it does not infer disk latency."
  ]
}
