Confidence sequence
A confidence sequence is a sequence of confidence sets whose simultaneous coverage of the target at all monitored times is at least a stated level, under specified assumptions.
For a fixed target theta, the guarantee is P(theta belongs to C_t for every t >= 1) >= 1 - alpha. Separate coverage at each fixed sample size is weaker. Optional stopping can use observations collected so far without invalidating a suitable confidence sequence. [2]
Time-uniform coverage also protects selecting an earlier interval from the same valid sequence after seeing the history: miscoverage at that selected time implies miscoverage at some time. This is unconditional coverage over repeated procedures, not a posterior probability or guaranteed coverage conditional on a favorable stopping signal. [1]
Selecting a favorable method or device-model target after inspecting results needs separate justification. The observation model still matters: a method calibrated for independent, constant-risk endpoints does not automatically handle shared failures, censoring or changing populations. [2]
A finite set of planned looks can instead allocate error levels whose sum is at most alpha. This controls that set by the union bound without independent looks; it does not by itself authorize unlimited further looks. [3]
Relations
Limitations
- Time-uniform validity does not require the interval width to shrink to zero. [1]
Citations
- Howard et al. (2021): Time-uniform, nonparametric, nonasymptotic confidence sequences; equation 1, Lemma 3
- Ramdas et al. (2023): Game-Theoretic Statistics and Safe Anytime-Valid Inference; sections 2.8 and 6.4
- NIST e-Handbook 7.4.7.3: Bonferroni's method, general inequality
Sources & caveats
- Methodological definition, not a device measurement or an implemented confidence-sequence estimator. The related AFR analysis retains its separate exact finite-look example.