Exploiting Expertise Rules for Statistical Data-Driven Modeling

A variety of real-world applications such as complex industry process usually are lack of abundant training samples since the data acquiring process is time and labor consuming. Hence, it is important to utilize the limited training samples to build a sophisticated data-driven model, which may improve industry productivity. Recently, nonlinear learning models such as artificial neural networks and support vector machines have shown to be effective in modeling small-scale data by their strong modeling ability. However, these nonlinear learning models work as a black box and are often not human understandable and are difficult to be interpreted. In addition, in many applications, domain experts could provide us valuable expertise knowledge which may help further improve the modeling process. In this paper, we propose to integrate expertise knowledge to the nonlinear learning model to advance the data-driven modeling process in real-world applications. Experimental results on six benchmark datasets and a real-world industry application validate the effectiveness of the proposed model.

[1]  Yong Xu,et al.  Neuro-Fuzzy Ensemble Approach for Microarray Cancer Gene Expression Data Analysis , 2006, 2006 International Symposium on Evolving Fuzzy Systems.

[2]  Zhaojie Yin,et al.  Fault diagnosis for rotating machinery based on Local Mean Decomposition morphology filtering and Least Square Support Vector Machine , 2017, J. Intell. Fuzzy Syst..

[3]  Hong Wang,et al.  Data-Driven Nonlinear Subspace Modeling for Prediction and Control of Molten Iron Quality Indices in Blast Furnace Ironmaking , 2017, IEEE Transactions on Control Systems Technology.

[4]  Bill Fulkerson,et al.  Machine Learning, Neural and Statistical Classification , 1995 .

[5]  Olvi L. Mangasarian,et al.  Nonlinear Knowledge-Based Classification , 2008, IEEE Transactions on Neural Networks.

[6]  Francesco Palmieri,et al.  Data Fusion with Entropic Priors , 2010, WIRN.

[7]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[8]  Bogdan Gabrys,et al.  Combining neuro-fuzzy classifiers for improved generalisation and reliability , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[9]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[10]  Kush R. Varshney,et al.  Screening for learning classification rules via Boolean compressed sensing , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[11]  Kush R. Varshney,et al.  Learning sparse two-level boolean rules , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[12]  Chuanhou Gao,et al.  Data-Driven Time Discrete Models for Dynamic Prediction of the Hot Metal Silicon Content in the Blast Furnace—A Review , 2013, IEEE Transactions on Industrial Informatics.

[13]  Gunnar Rätsch,et al.  Large Scale Multiple Kernel Learning , 2006, J. Mach. Learn. Res..

[14]  Bao-Gang Hu,et al.  Generalized Constraint Neural Network Regression Model Subject to Linear Priors , 2011, IEEE Transactions on Neural Networks.

[15]  U. Kruger,et al.  Regression-based analysis of multivariate non-Gaussian datasets for diagnosing abnormal situations in chemical processes , 2014 .

[16]  Francesco Palmieri,et al.  Objective priors from maximum entropy in data classification , 2013, Inf. Fusion.

[17]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[18]  Jude W. Shavlik,et al.  A Simple and Effective Method for Incorporating Advice into Kernel Methods , 2006, AAAI.

[19]  Guozhu Dong,et al.  Pattern-Aided Regression Modeling and Prediction Model Analysis , 2016, IEEE Transactions on Knowledge and Data Engineering.

[20]  Johan A. K. Suykens,et al.  L2-norm multiple kernel learning and its application to biomedical data fusion , 2010, BMC Bioinformatics.

[21]  Chuanhou Gao,et al.  Modeling of the Thermal State Change of Blast Furnace Hearth With Support Vector Machines , 2012, IEEE Transactions on Industrial Electronics.

[22]  Klaus-Robert Müller,et al.  Efficient and Accurate Lp-Norm Multiple Kernel Learning , 2009, NIPS.

[23]  Nello Cristianini,et al.  A statistical framework for genomic data fusion , 2004, Bioinform..

[24]  Deyong You,et al.  WPD-PCA-Based Laser Welding Process Monitoring and Defects Diagnosis by Using FNN and SVM , 2015, IEEE Transactions on Industrial Electronics.

[25]  Tao Jiang,et al.  Lower Bounds on Learning Decision Lists and Trees , 1995, Inf. Comput..

[26]  Huijun Gao,et al.  Data-Based Techniques Focused on Modern Industry: An Overview , 2015, IEEE Transactions on Industrial Electronics.

[27]  Adam Prügel-Bennett,et al.  A Bayesian Framework for Extracting Human Gait Using Strong Prior Knowledge , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Kush R. Varshney,et al.  Exact Rule Learning via Boolean Compressed Sensing , 2013, ICML.

[29]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[30]  Lior Rokach,et al.  Decision forest: Twenty years of research , 2016, Inf. Fusion.

[31]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[32]  R. Preuss,et al.  Maximum entropy and Bayesian data analysis: Entropic prior distributions. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[33]  Osmar R. Zaïane,et al.  Associative Classification with Statistically Significant Positive and Negative Rules , 2015, CIKM.

[34]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[35]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[36]  Hsuan-Tien Lin,et al.  A note on Platt’s probabilistic outputs for support vector machines , 2007, Machine Learning.

[37]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[38]  Fred A. Hamprecht,et al.  Principal Component Imagery for the Quality Monitoring of Dynamic Laser Welding Processes , 2009, IEEE Transactions on Industrial Electronics.