Which regression method to use? Making informed decisions in “data-rich/knowledge poor” scenarios – The Predictive Analytics Comparison framework (PAC)

Abstract In the big data and Manufacturing 4.0 era, there is a growing interest in using advanced analytical platforms to develop predictive modeling approaches that take advantage of the wealthy of data available. Typically, practitioners have their own favorite methods to address the modeling task, as a result of their technical background, past experience or software available, among other possible reasons. However, the importance of this task in the future justifies and requires more informed decisions about the predictive solution to adopt. Therefore, a wider variety of methods should be considered and assessed before taking the final decision. Having passed through this process many times and in different application scenarios (chemical industry, biofuels, drink and food, shipping industry, etc.), the authors developed a software framework that is able to speed up the selection process, while securing a rigorous and robust assessment: the Predictive Analytics Comparison framework (PAC). PAC is a systematic and robust framework for model screening and development that was developed in Matlab, but its implementation can be carried out on other software platforms. It comprises four essential blocks: i) Analytics Domain; ii) Data Domain; iii) Comparison Engine; iv) Results Report. PAC was developed for the case of a single response variable, but can be extended to multiple responses by considering each one separately. Some case studies will be presented in this article in order to illustrate PAC's efficiency and robustness for problem-specific methods screening, in the absence of prior knowledge. For instance, the analysis of a real world dataset reveals that, even when addressing the same predictive problem and using the same response variable, the best modeling approach may not be the one foreseen a priori and may not even be always the same when different predictor sets are used. With an increasing frequency, situations like these raise considerable challenges to practitioners, underlining the importance of having a tool such as PAC to assist them in making more informed decisions and to benefit from the availability of data in Manufacturing 4.0 environments.