Hierarchical Proxy Modeling for Improved HPO in Time Series Forecasting

Abstract

Selecting the right set of hyperparameters is crucial in time series forecasting. The classical temporal cross-validation framework for hyperparameter optimization (HPO) often leads to poor test performance because of a possible mis- match between validation and test periods. To address this test-validation mismatch, we propose a novel technique, H-Pro to drive HPO via test proxies by exploiting data hi- erarchies often associated with time series datasets. Since higher-level aggregated time series often show less irregular- ity and better predictability as compared to the lowest-level time series which can be sparse and intermittent, we opti- mize the hyperparameters of the lowest-level base-forecaster by leveraging the proxy forecasts for the test period generated from the forecasters at higher levels. We validate the efficacy of our technique with extensive empirical evaluation on five publicly available hierarchical forecasting datasets. Our ap- proach consistently outperforms the existing state-of-the-art approaches, including the winning method of the M5 forecast accuracy competition.