Improving Data Quality through Deep Learning and Statistical Models

Traditional data quality control methods are based on users’ experience or previously established business rules, and this limits performance in addition to being a very time consuming process with lower than desirable accuracy. Utilizing deep learning, we can leverage computing resources and advanced techniques to overcome these challenges and provide greater value to users.

[1]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[2]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[3]  Diane M. Strong,et al.  Data quality in context , 1997, CACM.

[4]  Jun Long,et al.  Data Profiling Technology of Data Governance Regarding Big Data: Review and Rethinking , 2016 .

[5]  Antonio Gomariz,et al.  SPMF: a Java open-source pattern mining library , 2014, J. Mach. Learn. Res..

[6]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[7]  B. Natarajan.  Machine Learning: A Theoretical Approach , 1992 .

[8]  Brian Kingsbury,et al.  New types of deep neural network learning for speech recognition and related applications: an overview , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Bashar Zogheib.  Elementary Statistics: A Step by Step Approach , 2012 .

[10]  Eugene L. Grant,et al.  Statistical Quality Control , 1946 .

[11]  Thorsten Meinl,et al.  KNIME: The Konstanz Information Miner , 2007, GfKl.

[12]  Douglas B. Kell,et al.  Software review: the KNIME workflow environment and its applications in genetic programming and machine learning , 2015, Genetic Programming and Evolvable Machines.

[13]  Richard E. DeVor,et al.  Statistical Quality Design and Control: Contemporary Concepts and Methods , 1992 .

[14]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[15]  Douglas C. Montgomery,et al.  Statistical Quality Control , 2008 .

[16]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.

[17]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[18]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .