hesela.devTemporal validation
Temporal validation evaluates a predictor on later observations using training information available before the evaluation boundary.
Training on earlier observations and testing on later ones approximates a future-prediction task; a random split can expose training to future conditions. [1] For disk errors, Xu et al. report a setting-specific gap between random and time-forward evaluation, not a universal correction factor. [2] Chronological separation is different from grouping by drive identity: it does not by itself test generalization to unseen drives. Feature selection and preprocessing must be fitted using training data only. [3]
Relations
- related
- smart, class-prevalence, precision, recall, f1-score, drive-day
Limitations
- Storage evaluation should state prediction time, outcome horizon, label availability, drive overlap and test prevalence; a chronological split alone is not proof that all leakage is absent. [2, 1, 3]