Developing a Robust Prediction Interval Based Criterion for Neural Network Model Selection

This paper studies how an optimal Neural Network (NN) can be selected that is later used for constructing the highest quality delta-based Prediction Intervals (PIs). It is argued that traditional assessment criteria, including RMSE, MAPE, BIC, and AIC, are not the most appropriate tools for selecting NNs from a PI-based perspective. A new NN model selection criterion is proposed using the specific features of the delta method. Using two synthetic and real case studies, it is demonstrated that this criterion outperforms all traditional model selection criteria in terms of picking the most appropriate NN. NNs selected using this criterion generate high quality PIs evaluated by their length and coverage probability.