A flexible machine vision system for small part inspection based on a hybrid SVM/ANN approach

Machine vision inspection systems are often used for part classification applications to confirm that correct parts are available in manufacturing or assembly operations. Support vector machines (SVMs) and artificial neural networks (ANNs) are popular choices for classifiers. These supervised classifiers perform well when developed for specific applications and trained with known class images. Their drawback is that they cannot be easily applied to different applications without extensive retuning. Moreover, for the same application, they do not perform well if there are unknown class images. This paper proposes a novel solution to the above limitations of SVMs and ANNs, with the development of a hybrid approach that combines supervised and semi-supervised layers. To illustrate its performance, the system is applied to three different small part identification and sorting applications: (1) solid plastic gears, (2) clear plastic wire connectors and (3) metallic Indian coins. The ability of the system to work with different applications with minimal tuning and user inputs illustrates its flexibility. The robustness of the system is demonstrated by its ability to reject unknown class images. Four hybrid classification methods were developed and tested: (1) SSVM–USVM, (2) USVM–SSVM, (3) USVM–SANN and (4) SANN–USVM. It was found that SANN–USVM gave the best results with an accuracy of over 95% for all three applications. A software package known as FlexMVS for flexible machine vision system was written to illustrate the hybrid approach that enabled easy execution of the image conditioning, feature extraction and classification steps. The image library and database used in this study is available at http://my.me.queensu.ca/People/Surgenor/Laboratory/Database.html .

[1]  Qinghua Hu,et al.  A novel measure for evaluating classifiers , 2010, Expert Syst. Appl..

[2]  Sergio Escalera,et al.  Separability of ternary codes for sparse designs of error-correcting output codes , 2009, Pattern Recognit. Lett..

[3]  Martin Kampel,et al.  Coarse-grained ancient coin classification using image-based reverse side motif recognition , 2015, Machine Vision and Applications.

[4]  Cédric Richard,et al.  Abnormal events detection using unsupervised One-Class SVM - Application to audio surveillance and evaluation - , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[5]  Der-Baau Perng,et al.  Automatic optical inspection system for IC molding surface , 2016, J. Intell. Manuf..

[6]  A. Bachelor GLOSSARY OF TERMS GLOSSARY OF TERMS , 2010 .

[7]  Hua Huang,et al.  No-reference image quality assessment based on spatial and spectral entropies , 2014, Signal Process. Image Commun..

[8]  Khalil Khalili,et al.  Flexible Automation in Porcelain Edge Polishing Using Machine Vision , 2016 .

[9]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[10]  Vedang Chauhan,et al.  Machine Vision for Coin Recognition with ANNs: Effect of Training and Testing Parameters , 2017, EANN.

[11]  Hyoung-Keun Park,et al.  Machine Vision System for Effective Semiconductor Package Sorting , 2015 .

[12]  Edwin Lughofer,et al.  On improving performance of surface inspection systems by online active learning and flexible classifier updates , 2015, Machine Vision and Applications.

[13]  Minoru Fukumi,et al.  Rotation-invariant neural pattern recognition system with application to coin recognition , 1992, IEEE Trans. Neural Networks.

[14]  R. Keys Cubic convolution interpolation for digital image processing , 1981 .

[15]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[16]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Cesare Furlanello,et al.  A Comparison of MCC and CEN Error Measures in Multi-Class Prediction , 2010, PloS one.

[18]  P. Niklaus,et al.  Automated Resistor Classification , 2015 .

[19]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[20]  M. Abdullah,et al.  QUALITY INSPECTION OF BAKERY PRODUCTS USING A COLOR-BASED MACHINE VISION SYSTEM , 2000 .

[21]  Zhen-Jun Shi,et al.  From Line Search Method to Trust Region Method ∗ , 2006 .

[22]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[23]  Zhen Li,et al.  Learning Locally-Adaptive Decision Functions for Person Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Keyur D. Joshi,et al.  Analysis of methods for the recognition of Indian coins: A challenging application of machine vision to automated inspection , 2016, 2016 23rd International Conference on Mechatronics and Machine Vision in Practice (M2VIP).

[25]  Richard P. Wildes,et al.  A machine-vision system for iris recognition , 2005, Machine Vision and Applications.

[26]  Szu-Hao Huang,et al.  Automated visual inspection in the semiconductor industry: A survey , 2015, Comput. Ind..

[27]  Hao Feng,et al.  Size-Sorting and Measurement System of Safety Belt Pin Based on Machine Vision , 2015 .

[28]  Franco Scarselli,et al.  On the complexity of shallow and deep neural network classifiers , 2014, ESANN.

[29]  Scott T. Acton,et al.  Scale space classification using area morphology , 2000, IEEE Trans. Image Process..

[30]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[31]  D. Kerbyson,et al.  Using phase to represent radius in the coherent circle Hough transform , 1993 .

[32]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[33]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[34]  P M Panchal,et al.  A Comparison of SIFT and SURF , 2013 .

[35]  Kontorn Chamniprasart,et al.  Using Machine Vision for Flexible Automatic Assembly System , 2016, KES.

[36]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[37]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[38]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[39]  Maqc Consortium The MicroArray Quality Control ( MAQC )-II study of common practices for the development and validation of microarray-based predictive models , 2012 .

[40]  Pierre Payeur,et al.  Feature selection for a real-time vision-based food inspection system , 2008, 2008 International Workshop on Robotic and Sensors Environments.

[41]  B. John Oommen,et al.  On achieving semi-supervised pattern recognition by utilizing tree-based SOMs , 2013, Pattern Recognit..

[42]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[43]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[44]  Anders A. Jensen,et al.  Classifying Proximity Soundings with Self-Organizing Maps toward Improving Supercell and Tornado Forecasting , 2013 .

[45]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[46]  Salvatore J. Stolfo,et al.  One Class Support Vector Machines for Detecting Anomalous Windows Registry Accesses , 2003 .

[47]  A. Chitra,et al.  A New Approach to Coin Recognition using Neural Pattern Analysis , 2005, 2005 Annual IEEE India Conference - Indicon.

[48]  De Xu,et al.  Automatic gear sorting system based on monocular vision , 2015 .

[49]  Gianni D'Angelo,et al.  Automated Eddy Current non-destructive testing through low definition lissajous figures , 2016, 2016 IEEE Metrology for Aerospace (MetroAeroSpace).

[50]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[51]  E D Megaw,et al.  Factors affecting visual inspection accuracy. , 1979, Applied ergonomics.

[52]  S. Nashat,et al.  Original paper: Support vector machine approach to real-time inspection of biscuits on moving conveyor belt , 2011 .

[53]  Nobuyuki Otsu,et al.  ATlreshold Selection Method fromGray-Level Histograms , 1979 .

[54]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[55]  Vishal M. Patel,et al.  Learning Deep Features for One-Class Classification , 2018, IEEE Transactions on Image Processing.

[56]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Young Shik Moon,et al.  Visual inspection system for the classification of solder joints , 1999, Pattern Recognit..

[58]  Sri Lanka,et al.  VISUAL-BASED AUTOMATIC COIN COUNTING SYSTEM , 2011 .

[59]  Jinchang Ren,et al.  ANN vs. SVM: Which one performs better in classification of MCCs in mammogram imaging , 2012, Knowl. Based Syst..

[60]  R. Real,et al.  AUC: a misleading measure of the performance of predictive distribution models , 2008 .

[61]  Seema Bawa,et al.  Image Processing based Systems and Techniques for the Recognition of Ancient and Modern Coins , 2013, ArXiv.

[62]  Xin Li,et al.  Adaptive Active Learning for Image Classification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  P. Payeur,et al.  Automated tuning of a vision-based inspection system for industrial food manufacturing , 2012, 2012 IEEE International Instrumentation and Measurement Technology Conference Proceedings.

[64]  Shuxiao Li,et al.  Bearing defect inspection based on machine vision , 2012 .

[65]  Seema Bawa,et al.  Automated Coin Recognition System using ANN , 2013, ArXiv.

[66]  Vedang Chauhan,et al.  Fault detection and classification in automated assembly machines using machine vision , 2017 .

[67]  M VeluC,et al.  Indian Coin Recognition and Sum Counting System of Image Data Mining Using Artificial Neural Networks , 2011 .

[68]  Maurizio Faccio,et al.  Fully flexible assembly systems (F‐FAS): a new concept in flexible automation , 2013 .

[69]  Euripides G. M. Petrakis,et al.  A survey on industrial vision systems, applications, tools , 2003, Image Vis. Comput..

[70]  Davide Chicco,et al.  Ten quick tips for machine learning in computational biology , 2017, BioData Mining.

[71]  B. Matthews Comparison of the predicted and observed secondary structure of T4 phage lysozyme. , 1975, Biochimica et biophysica acta.

[72]  Zixiang Xiong,et al.  Optimal number of features as a function of sample size for various classification rules , 2005, Bioinform..

[73]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[74]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[75]  Marc Schlipsing,et al.  Adaptive pattern recognition in real-time video-based soccer analysis , 2017, Journal of Real-Time Image Processing.

[76]  Justin Beneke,et al.  The influence of perceived product quality, relative price and risk on customer value and willingness to buy: a study of private label merchandise , 2013 .

[77]  Rita Noumeir,et al.  Limitations of the SSIM quality metric in the context of diagnostic imaging , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[78]  M. Antkowiak Artificial Neural Networks vs . Support Vector Machines for Skin Diseases Recognition , 2006 .

[79]  Te-Hsiu Sun,et al.  Automated thermal fuse inspection using machine vision and artificial neural networks , 2016, J. Intell. Manuf..

[80]  Jayson Tessier,et al.  A machine vision approach to on-line estimation of run-of-mine ore composition on conveyor belts , 2007 .

[81]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[82]  Yunsong Li,et al.  Adaptive maximum margin analysis for image recognition , 2017, Pattern Recognit..

[83]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[84]  Hsiao-Wei Liu,et al.  White light triangulation sensor for flexible inspection system , 2015, 2015 9th International Conference on Sensing Technology (ICST).

[85]  Bernard Widrow,et al.  Layered neural nets for pattern recognition , 1988, IEEE Trans. Acoust. Speech Signal Process..

[86]  Jan Gorodkin,et al.  Comparing two K-category assignments by a K-category correlation coefficient , 2004, Comput. Biol. Chem..

[87]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[88]  Mehryar Mohri,et al.  Confidence Intervals for the Area Under the ROC Curve , 2004, NIPS.

[89]  Shoab Ahmad Khan,et al.  Automated Plant Disease Analysis (APDA): Performance Comparison of Machine Learning Techniques , 2013, 2013 11th International Conference on Frontiers of Information Technology.

[90]  Rajesh C. Sanghvi,et al.  Image Retrieval System Using Intuitive Descriptors , 2014 .

[91]  Darren J. Kerbyson,et al.  Size invariant circle detection , 1999, Image Vis. Comput..

[92]  J D Gould,et al.  Studies of Visual Inspection. , 1973, Ergonomics.

[93]  Vincent Leemans,et al.  Regular ArticleAE—Automation and Emerging Technologies: On-line Fruit Grading according to their External Quality using Machine Vision , 2002 .

[94]  Julian Florez,et al.  Color machine vision system for process control in the ceramics industry , 1997, Other Conferences.

[95]  Brian Surgenor,et al.  A Quantitative Study of Illumination Techniques for Machine Vision Based Inspection , 2011 .

[96]  Vedang Chauhan,et al.  A Comparative Study of Machine Vision Based Methods for Fault Detection in an Automated Assembly Machine , 2015 .

[97]  J. Wilder Industrial Applications of Machine Vision , 1989 .

[98]  Charles X. Ling,et al.  AUC: A Better Measure than Accuracy in Comparing Learning Algorithms , 2003, Canadian Conference on AI.

[99]  Y. S. Tarng,et al.  Automated visual inspection for surface appearance defects of varistors using an adaptive neuro-fuzzy inference system , 2008 .

[100]  C G Drury The effect of speed of working on industrial inspection accuracy. , 1973, Applied ergonomics.

[101]  Frederick M. Waltz,et al.  Machine vision for industrial applications , 2001 .

[102]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[103]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..

[104]  Andrew Zisserman,et al.  A Visual Vocabulary for Flower Classification , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[105]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.