confidence-sequence · research · also time-uniform confidence sequence, anytime-valid confidence sequence

Confidence sequence

A confidence sequence is a sequence of confidence sets whose simultaneous coverage of the target at all monitored times is at least a stated level, under specified assumptions.

Related Observatory article: Read on hesela.com

For a fixed target theta, the guarantee is P(theta belongs to C_t for every t >= 1) >= 1 - alpha. Separate coverage at each fixed sample size is weaker. Optional stopping can use observations collected so far without invalidating a suitable confidence sequence. [2]

Time-uniform coverage also protects selecting an earlier interval from the same valid sequence after seeing the history: miscoverage at that selected time implies miscoverage at some time. This is unconditional coverage over repeated procedures, not a posterior probability or guaranteed coverage conditional on a favorable stopping signal. [1]

Selecting a favorable method or device-model target after inspecting results needs separate justification. The observation model still matters: a method calibrated for independent, constant-risk endpoints does not automatically handle shared failures, censoring or changing populations. [2]

A finite set of planned looks can instead allocate error levels whose sum is at most alpha. This controls that set by the union bound without independent looks; it does not by itself authorize unlimited further looks. [3]

Relations

related
Confidence interval, Binomial proportion, P-value

Limitations

  • Time-uniform validity does not require the interval width to shrink to zero. [1]

Citations

  1. Howard et al. (2021): Time-uniform, nonparametric, nonasymptotic confidence sequences; equation 1, Lemma 3 (accessed 2026-11-19)
  2. Ramdas et al. (2023): Game-Theoretic Statistics and Safe Anytime-Valid Inference; sections 2.8 and 6.4 (accessed 2026-11-19)
  3. NIST e-Handbook 7.4.7.3: Bonferroni's method, general inequality (accessed 2026-11-19)
Sources & caveats
  • Methodological definition, not a device measurement or an implemented confidence-sequence estimator. The related AFR analysis retains its separate exact finite-look example.

JSON: /data/terms/confidence-sequence.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. Confidence sequence. Hesela. 2026-11-19. https://hesela.dev/terms/confidence-sequence/