Impacts of nested forward validation techniques on machine learning and regression waste disposal time series models

Abstract

Dataset partitioning and validation techniques are required in all artificial neural network based waste models. However, there is currently no consensual approach on the validation techniques. This study examines the effects of three time series nested forward validation techniques (rolling origin - RO, rolling window - RW, and growing window - GW) on total municipal waste disposal estimates using recurrent neural network (RNN) models, and benchmarks model performance with respect to multiple linear regression (MLR) models. Validation selection techniques appear important to waste disposal time series model construction and evaluation. Sample size is found as an important factor on model accuracy for both RNN and MLR models. Better performance in Trial RW4 is observed, probably due to a more consistent testing set in 2019. Overall, the MAPE of the waste disposal models ranging from 10.4% to 12.7%. Both GW and RO validation techniques appear appropriate for RNN waste models. However, MLR waste models are more sensitive to the dataset characteristics, and RO validation technique appears more suitable to MLR models. It is found that data characteristics are more important than training period duration. It is recommended data set normality and skewness be examined for waste disposal modeling.