# A Precise HLL Estimate Can Describe the Wrong Time Window Hesela research report, 20 November 2026. Native Redis 7.2.4 evidence, not a newly discovered algorithmic limitation or an estimator benchmark. **Main finding:** rebuilding a rolling HyperLogLog aggregate into an existing Redis destination can retain data from retired buckets. In a deterministic fixture, the requested active union contains 20 distinct identities, but a reused destination reports 120. A fresh destination reports 20. The difference is the population being represented, not an unusually inaccurate estimate. A second failure mode runs in the opposite direction: a missing active bucket is accepted as empty. A successful merge then reports 10 for an intended 20-identity window. Increasing the number of registers does not repair either selection error. The observations are reproduced in sparse and forced-dense representations, with matching decoded register vectors. ## Why This Matters for Storage Unique chunk identifiers, working-set estimates and distinct-device telemetry are often aggregated over time. A compact count cannot certify which retained objects contributed, whether a bucket is complete, or whether an omitted key was genuinely empty. A plausible count and an OK response are insufficient evidence for capacity planning or retention decisions. This complements Hesela's existing Counter Stacks material, which separates distinct-count summaries from cache history, and the [HyperLogLog definition](https://hesela.dev/terms/hyperloglog/). It does not create another HLL identity or claim to replace an exact chunk-reference index. ## Mechanism: Three Contracts, Not One **Register algebra.** For compatible HLLs, a union takes the maximum in each register. The original [HLL paper, Figure 2 and footnote 4][paper] describes this operation. A maximum does not retain the identity or time of the winning input. A register-only aggregate cannot generally retract a retired bucket. This does not prohibit sliding-window designs that retain additional state. **Redis command behavior.** [PFMERGE][merge] includes an existing destination as an input. The pinned [implementation][code] starts its input loop at the destination and skips absent source keys. Thus an update has the form ```text R_next = max(R_old_destination, R_retained_bucket_1, ..., R_new_bucket) ``` not a fresh fold of only the currently selected buckets. Idempotence makes repeating the same merge harmless, but it does not make it reversible. **Window completeness.** An application must decide which buckets belong to a window and establish that each selected bucket is complete. Key existence is one necessary check, not proof of coverage. A legitimately empty sealed bucket must be distinguished from an absent bucket caused by expiry, eviction, failure to ingest, a naming error or data loss. ## Native Controls The generator uses byte identities `object:0` through `object:119`, split into disjoint old (100), kept (10) and new (10) sets. After forming old+kept, it gives old a positive 10 ms TTL and waits until a lookup confirms absence. It does not infer expiry from elapsed time alone or measure active-expiry scheduling. New is outside the first intended window and enters in the next step. The full generated table is [TABLE.md](TABLE.md); its rows are checked against [results.json](results.json). No measured data are hand-entered into the table. The stale destination has exactly the register vector produced by directly inserting all 120 historical identities. The fresh vector matches a repeated fresh union and its count agrees with transient multi-key PFCOUNT. This separates a wrong-input-window explanation from a claim that the estimator mysteriously overshot by six times. After deleting the still-needed kept key, a fresh merge returns OK and the same registers as merging an explicit empty key with new. These two histories are indistinguishable to that union operation although their coverage differs. ## No Change Is Not Proof of a Duplicate The runner inserts the distinct strings `member:0`, `member:1`, and so on, stopping at the first no-op within a declared 20,000-candidate limit. At `member:183`, the exact set grows from 183 to 184, while PFADD returns zero and the estimate stays 183. Both the registers and the complete cached serialization remain identical. Repeating the same identity also returns zero. Those two responses cannot distinguish a new identity from a duplicate. The [PFADD reference][add] describes register changes, not an exact membership test. The empty-key case needs care: PFADD without elements creates a missing key and returns one in the pinned implementation, but its count is zero. Repeating the empty creation returns zero. Do not use these return values to authorize deletion of supposedly duplicate storage objects. The search is a constructed witness, not a sample estimating the probability of a no-op at a given cardinality. We do not claim a full-hash collision. ## Tested Replacement Pattern and Its Boundary [rebuild.lua](rebuild.lua) demonstrates a bounded correction for sealed source buckets on one Redis server. It requires every listed source to exist, rejects destination/source aliasing and occupied scratch keys, validates the HLL inputs, builds into a previously absent private key, then renames it over the old target. The native probe confirms a result of 20, removal of the scratch name and no destination change for the tested missing-source, alias, missing-argument, wrong-type and occupied-scratch errors. A deliberately empty source is accepted. The rebuilt destination is persistent: it does not preserve an old target TTL. [Redis script isolation][lua] prevents other commands interleaving with this operation. It is not rollback or durability. The example is not a production window protocol: it cannot detect omitted manifest entries, incomplete ingestion, late data, wrong identity encoding, a corrupted-but-valid sketch, cross-server inconsistencies or resource failures. All source buckets must be sealed and their ownership/window manifest must already be validated. Blocking cost, Redis Cluster routing, replication and crash recovery are not tested. Do not broadly deploy this historical Redis version; it is an isolated reference build with TCP disabled. ## Estimation Is Not Reconstruction [Ertl's section 6][ertl] studies joint estimates of intersection and relative-complement sizes from two sketches. This is richer than subtracting two rounded counts, but it still does not restore item identities or recreate a discarded time partition. A better estimator cannot decide whether the application supplied the intended complete window. Practical acceptance should keep separate: the selected population, bucket coverage and identity conventions, estimator uncertainty, and the downstream decision that consumes the estimate. None of these tests measures physical deduplication savings, SSD performance or a fleet failure rate. ## Reproduce [Download the complete replay package](reproduction.zip). [SHA-256 manifest](manifest.json) covers the individual files and ZIP. Python 3.12+, a C compiler and make are required for a fresh build. No Python package installation is needed. Sources are pinned to Redis commit `d2c8a4b91e8c0e6aefd1f5bc0bf582cddbe046b7`, version 7.2.4. ```sh python3 -B -m unittest -v test_probe.py python3 -B -O -m unittest -v test_probe.py python3 -B replay.py --receipt /absolute/new-receipt.json ``` The last command downloads the commit archive, verifies its SHA-256, builds Redis in an automatically removed temporary directory, starts only private Unix sockets, compares freshly generated JSON/table byte-for-byte, and runs the tests normally and with optimization. Pass `--archive /path/redis.tar.gz` to reuse the exact pinned archive offline. Every test server is terminated and waited for, including on failure. Use `probe.py --redis /path/redis-server` for replay with a prebuilt binary; use `--write` only to intentionally replace the baseline, followed by review of its diff. `environment.json` records the original host and binary. The source archive, compiler and build receipts bind the reconstruction; a source-tar build reports zeroes in Redis's git-sha version field, so that field is not used as provenance. The upstream build emits compiler warnings; we do not label it warning-free. ## Evidence and Rights This is same-author verification on one Linux x86-64 host. The fixtures use no random seed, stochastic population or empirical confidence interval. Native behavior is checked twice, but this is not independent-person review. The source ledger is [sources.json](sources.json), with exact claim locations. Full third-party papers and source captures stay outside the Git package. Hesela-authored code, prose and constructed results in this directory are offered under CC0-1.0. Third-party Redis code remains under the pinned source's BSD-3-Clause license; papers and documentation retain their own rights. The package contains links and capture hashes, not redistributed papers. [paper]: https://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf [code]: https://github.com/redis/redis/blob/d2c8a4b91e8c0e6aefd1f5bc0bf582cddbe046b7/src/hyperloglog.c#L1324 [merge]: https://redis.io/docs/latest/commands/pfmerge/ [add]: https://redis.io/docs/latest/commands/pfadd/ [lua]: https://redis.io/docs/latest/develop/programmability/eval-intro/ [ertl]: https://arxiv.org/pdf/1706.07290v1