{
  "slug": "theta-sketch",
  "term": "Theta sketch",
  "aliases": [],
  "definition": "A Theta sketch retains hashed distinct values below a sampling threshold to estimate set sizes and combinations of sets.",
  "note": "The state includes a threshold theta and the retained hashes. The estimate is retained count divided by theta; theta=1 gives an exact count in the hashed domain. Compatible sketches can estimate unions, intersections and set differences. The framework's unbiasedness result assumes a suitable threshold rule and ideal random hashing. [framework]\n\nAn intersection uses the smaller input threshold and retains common hashes below it. A small overlap can leave no retained hash. In that state estimate=0 does not certify disjoint inputs: empty=false and theta<1 preserve uncertainty. Query the result's bounds instead of borrowing the input sketch's relative error. [corner] [accuracy]\n\nIn a native 5.2.0 example, two sets of 10000 integer identities share one value. With lg_k=5 and seed=9001, the intersection estimates zero, but its 2-standard-deviation upper bound is 1092. With lg_k=12 it still estimates zero, with upper bound 5. These are approximate library bounds, not hard maxima. [replay]\n\nPreserve theta, the empty flag and hash/encoding compatibility through serialization. Replacing a zero-retained sampled result with a fresh empty sketch discards its uncertainty; the native compact serialization preserves it. A cardinality sketch cannot authorize deletion of individual blocks or establish ingestion completeness. [native] [replay]",
  "related": [
    "hyperloglog",
    "deduplication",
    "sampling-design",
    "minhash",
    "jaccard-similarity"
  ],
  "status": "shipping",
  "advantages": [],
  "limitations": [],
  "supersedes": [],
  "supersededBy": [],
  "firstSeen": null,
  "lastSeen": null,
  "updated": "2026-11-20",
  "citations": [
    {
      "id": "framework",
      "title": "Dasgupta, Lang, Rhodes and Thaler. A Framework for Estimating Stream Expression Cardinalities. ICDT 2016, sections 2 and 3.7.",
      "url": "https://doi.org/10.4230/LIPIcs.ICDT.2016.6",
      "accessed": "2026-11-20"
    },
    {
      "id": "corner",
      "title": "Apache DataSketches. Theta Sketch and Tuple Sketch Set Operation Corner Cases: empty and degenerate states.",
      "url": "https://datasketches.apache.org/docs/Theta/ThetaSetOpsCornerCases.html",
      "accessed": "2026-11-20"
    },
    {
      "id": "accuracy",
      "title": "Apache DataSketches. Theta accuracy plots: error after intersection and difference.",
      "url": "https://datasketches.apache.org/docs/Theta/ThetaAccuracyPlots.html",
      "accessed": "2026-11-20"
    },
    {
      "id": "native",
      "title": "Apache DataSketches C++ 5.2.0: pinned intersection implementation, empty-state and threshold handling.",
      "url": "https://github.com/apache/datasketches-cpp/blob/de8553ba372e618382c2e7b44b0ffc9422b9458c/theta/include/theta_intersection_base_impl.hpp",
      "accessed": "2026-11-20"
    },
    {
      "id": "replay",
      "title": "Hesela. Theta intersections: preserving uncertainty when the sample is empty. Native controls, results and replay.",
      "url": "https://hesela.dev/research/theta-empty/README.md",
      "accessed": "2026-11-20"
    }
  ],
  "provenance": [
    "Executed Apache DataSketches Python 5.2.0 with synthetic integer ranges; the release source references C++ 5.2.0. Wheel and source hashes are recorded in the replay packet.",
    "The 768-run sweep varies a fixed list of seeds over three known overlaps. It is not a random fleet sample, a confidence-bound coverage study or a reproduction of the paper's performance results.",
    "Exact mode concerns offered identities and finite hashes. Source completeness, intended identity encoding and cryptographic authenticity need separate checks."
  ]
}
