brier-score · shipping · also Brier loss, binary Brier score

Brier score

For a binary event, the Brier score is the mean squared difference between each forecast probability and its observed zero-or-one outcome.

The convention here is BS = sum((p_i-y_i)^2)/N for N observed outcomes, y_i in {0,1} and event probabilities p_i in [0,1]; lower is better and the range is [0,1]. State normalization: the unhalved sum over class-probability components instead ranges up to 2. [1]

Brier loss reflects both reliability and the separation of different event risks, so a lower value does not by itself establish better calibration. Compare forecasts on the same outcomes and include a declared constant-probability reference. [2]

Relations

related

probability-calibration, class-prevalence, prediction-horizon, average-precision

Limitations

• Editorial implication: An unobserved prediction horizon is not a zero outcome; a score cannot replace documentation of follow-up, event labels, population and any weighting. [1]

Citations

1. scikit-learn: brier_score_loss, binary normalization and range. (accessed 2026-10-17)
2. scikit-learn: probability calibration, reliability diagrams and limits of Brier loss. (accessed 2026-10-17)

Further reading

3. Hesela: synthetic ranking, decision and calibration example; not measured drive results. (accessed 2026-10-17)

JSON: /data/terms/brier-score.json