Scale insensitive and focus driven mobile screen defect detection in industry

Abstract With the wide-spread of smartphones, mobile phone screen has become an important IO device in HCI and its quality is of great matter in interaction. Traditional defect detection process involves heavy labor cost or relies on unstable low-level features and suffers from both scale and model sensitive problems. Screen defect varies in size, shape, intensity and is hard to be described. Efficient and accurate detection system remains an urgent need in mobile phone screen manufacturing. In this paper, we propose an end-to-end screen defect detection framework. A defect detection network with merging and splitting strategies (MSDDN) to deal with multiple size and shape variations of defect image patches is firstly designed. After training, feature maps of the last layer before the output of MSDDN can be regarded as good representations of a screen image patch. These feature maps are concatenated into a unified feature vector. We then train a recurrent neural network (SCN) to decide which input screen image patch in a sequence is the most likely to contain defects, where the patches are cropped from the same image, and the feature maps are used as input. As SCN emphasizes on the comparison of image patches from one image, it is less sensitive to different screen batches. The patch with the highest probability of containing defects, or called focus area, is further processed with a sliding window to fill in the MSDDN to produce the final results. Finally, to improve the efficiency of the calculation process to fulfill the real industrial demand, we perform both filter selection and weight quantization on the weights in MSDDN under the purpose of building a low-precision version network without great loss in accuracy (MSDDN-l). Experimental results show MSDDN can better handle the defect variations than traditional models and general purpose convolutional neural networks. Meanwhile, SCN can accurately predict the focus area, and MSDDN-l can greatly improve the efficiency.

[1]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[3]  Tomaso A. Poggio,et al.  A general framework for object detection , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  David Malah,et al.  A study of edge detection algorithms , 1982, Comput. Graph. Image Process..

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Min Han,et al.  Remote sensing image classification based on neural network ensemble algorithm , 2012, Neurocomputing.

[7]  Ryusuke Miyamoto,et al.  Partially Parallel Architecture for AdaBoost-Based Detection With Haar-Like Features , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Jae Yeong Lee,et al.  Automatic Detection of Region-Mura Defect in TFT-LCD , 2004, IEICE Trans. Inf. Syst..

[9]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[10]  Li Lin,et al.  Remaining useful life estimation of engineered systems using vanilla LSTM neural networks , 2018, Neurocomputing.

[11]  Larry S. Davis,et al.  A survey of edge detection techniques , 1975 .

[12]  Bo Liu,et al.  Image classification based on effective extreme learning machine , 2013, Neurocomputing.

[13]  Ramakrishna Kakarala.  A Signal Processing Approach to Fourier Analysis of Ranking Data: The Importance of Phase , 2011, IEEE Transactions on Signal Processing.

[14]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Dacheng Tao,et al.  DCT Regularized Extreme Visual Recovery , 2017, IEEE Transactions on Image Processing.

[16]  Rainer Lienhart,et al.  Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection , 2003, DAGM-Symposium.

[17]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[18]  Dacheng Tao,et al.  Tag Disentangled Generative Adversarial Network for Object Image Re-rendering , 2017, IJCAI.

[19]  Liang-Chia Chen,et al.  Automatic TFT-LCD mura defect inspection using discrete cosine transform-based background filtering and 'just noticeable difference' quantification strategies , 2008 .

[20]  Sen Jia,et al.  Convolutional neural networks for hyperspectral image classification , 2017, Neurocomputing.

[21]  A.V. Oppenheim,et al.  The importance of phase in signals , 1980, Proceedings of the IEEE.

[22]  Dacheng Tao,et al.  Packing Convolutional Neural Networks in the Frequency Domain , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Du-Ming Tsai,et al.  Automatic defect inspection for LCDs using singular value decomposition , 2005 .

[24]  Mengjie Zhang,et al.  Encoding subcomponents in cooperative co-evolutionary recurrent neural networks , 2011, Neurocomputing.

[25]  Changyin Sun,et al.  Discriminative Multi-View Interactive Image Re-Ranking , 2017, IEEE Transactions on Image Processing.

[26]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.