statistical-power · shipping · also Test power, Power analysis

Statistical power

Statistical power is the probability that a specified test rejects its null hypothesis when a specified alternative is true.

Related Observatory article: Read on hesela.com

Power equals 1 minus the Type II error probability for a specified alternative. It depends on the test, significance level, effect size, sample size and sampling assumptions; it is not a probability that a hypothesis is true. Plan against a practically important alternative, not an effect selected after seeing the results. [3]

A fixed-horizon binomial design counts each independent, equal-risk device once as failed or not failed after complete follow-up. With a known 1% probability benchmark and nominal 5% upper-tail test, the synthetic example detects a true 3% probability with 35.3% power for 100 devices and 79.7% for 300. Discrete thresholds keep actual null rejection below 5%. These are horizon-specific probabilities, not AFR values. [2] [1] [5]

Non-rejection does not establish equivalence. Equivalence requires prespecified practical bounds and a suitable test; observed-effect post-hoc power does not resolve an inconclusive result. Shared failure causes, unequal device risks or incomplete follow-up require a design that models those features. [4] [3] [2]

Exact-test power need not rise at every larger sample size. Raising the integer rejection threshold can lower it. In the same 1% versus 3% example, n=301 first reaches 80% power, n=330..339 falls below, and every prespecified n>=340 reaches it. This fixed-design guarantee does not permit outcome-dependent stopping. [6] [7]

Relations

related
Binomial proportion, Confidence interval, Prediction horizon

Citations

  1. NIST e-Handbook 6.2.3.2: Binomial sampling plans and operating-characteristic probabilities (accessed 2026-11-19)
  2. NIST: Binomial distribution, independent Bernoulli experiments with common probability (accessed 2026-11-19)
  3. Hoenig and Heisey (2001): The Abuse of Power, The American Statistician 55(1), 19-24 (accessed 2026-11-19)
  4. Lakens (2017): Equivalence Tests: A Practical Primer, DOI 10.1177/1948550617697177 (accessed 2026-11-19)
  5. Hesela: Fixed-horizon statistical-power example, exact arithmetic and assumptions (accessed 2026-11-19)
  6. Stata: power oneproportion, Example 7, discrete thresholds and nonmonotonic power (accessed 2026-11-19)
  7. Hesela: Exact fixed-design power floor, finite enumeration and rational tail certificate (accessed 2026-11-19)
Sources & caveats
  • The 100- and 300-device examples are synthetic exact-binomial calculations, not observations of a storage fleet. [5]
  • The equivalence source develops general inference principles and t-test examples; it is not a validated storage-specific equivalence protocol. [4]
  • For this synthetic design, exact enumeration covers n=1..849 and a rational exponential-Markov bound covers every n>=850. The proof cutoff is not the required sample size. [7]

JSON: /data/terms/statistical-power.json

Published by Hesela team · Updated

Report a correction · Editorial standards · Reuse guidance

Cite this page
Hesela team. Statistical power. Hesela. 2026-11-19. https://hesela.dev/terms/statistical-power/