A recursively updated Map-Reduce based PCA for monitoring the time-varying fluorochemical engineering processes with big data

Abstract The hypertoxic materials in fluorochemical engineering process make it very critical to public safety. Unfortunately, the complicated and time-varying characteristics of it badly hinder the wide applications of advanced process monitoring strategies, especially the big data techniques, in its monitoring system. Therefore, a recursively updated Map-Reduce based principle component analysis, RMPCA, was proposed. A variable-width bin histogram was proposed to speed up the corresponding mutual information (MI) calculating procedure, which was used to informatively split variables into smaller pieces for running on the Map-Reduce framework. Afterwards, a forgetting factor strategy was proposed to recursively update the distributed PCA model, Bayesian decision fusion and hierarchical fault diagnosis scheme with new data for further monitoring. Applications on a practical R-22, a common propellant and refrigerant, producing process and on the Tennessee Eastman process strongly confirmed the superiority of RMPCA in both fault detection and diagnosis for time-varying processes with big data.