Statistical power
Statistical power is the probability that a specified test rejects its null hypothesis when a specified alternative is true.
Power equals 1 minus the Type II error probability for a specified alternative. It depends on the test, significance level, effect size, sample size and sampling assumptions; it is not a probability that a hypothesis is true. Plan against a practically important alternative, not an effect selected after seeing the results. [3]
A fixed-horizon binomial design counts each independent, equal-risk device once as failed or not failed after complete follow-up. With a known 1% probability benchmark and nominal 5% upper-tail test, the synthetic example detects a true 3% probability with 35.3% power for 100 devices and 79.7% for 300. Discrete thresholds keep actual null rejection below 5%. These are horizon-specific probabilities, not AFR values. [2] [1] [5]
Non-rejection does not establish equivalence. Equivalence requires prespecified practical bounds and a suitable test; observed-effect post-hoc power does not resolve an inconclusive result. Shared failure causes, unequal device risks or incomplete follow-up require a design that models those features. [4] [3] [2]
Exact-test power need not rise at every larger sample size. Raising the integer rejection threshold can lower it. In the same 1% versus 3% example, n=301 first reaches 80% power, n=330..339 falls below, and every prespecified n>=340 reaches it. This fixed-design guarantee does not permit outcome-dependent stopping. [6] [7]
Relations
Citations
- NIST e-Handbook 6.2.3.2: Binomial sampling plans and operating-characteristic probabilities
- NIST: Binomial distribution, independent Bernoulli experiments with common probability
- Hoenig and Heisey (2001): The Abuse of Power, The American Statistician 55(1), 19-24
- Lakens (2017): Equivalence Tests: A Practical Primer, DOI 10.1177/1948550617697177
- Hesela: Fixed-horizon statistical-power example, exact arithmetic and assumptions
- Stata: power oneproportion, Example 7, discrete thresholds and nonmonotonic power
- Hesela: Exact fixed-design power floor, finite enumeration and rational tail certificate
Sources & caveats
- The 100- and 300-device examples are synthetic exact-binomial calculations, not observations of a storage fleet. [5]
- The equivalence source develops general inference principles and t-test examples; it is not a validated storage-specific equivalence protocol. [4]
- For this synthetic design, exact enumeration covers n=1..849 and a rational exponential-Markov bound covers every n>=850. The proof cutoff is not the required sample size. [7]