hesela.devGrouped cross-validation
Grouped cross-validation evaluates a predictor using splits that keep all observations from each specified group together in either training or testing within a split.
GroupKFold holds each group out once across its test folds. [1] In storage evaluation, group by physical drive to test unseen drives or by site to test unseen sites; these are different generalization questions. Group separation alone does not impose chronological order. [2]
Relations
- related
- temporal-validation, prediction-horizon, class-prevalence, smart
Limitations
- Choose groups for the deployment question; grouping does not prevent future information or globally fitted preprocessing from entering training. [2]