A critical review for machining positioning based on computer vision

Abstract With the rapid development of science and technology, the manufacturing industry has to cope with increasingly stricter requirements in terms of the quality of processed products. To improve production flexibility and automation, computer vision is widely used in machining due to its safety, reliability, continuity, high accuracy, and real-time performance. In this study, a comprehensive review of positioning methods for workpieces in machining is presented from the perspective of computer vision technology. First, the key technologies in image acquisition are described in detail, and a analysis of different lighting modes is conducted. Second, image preprocessing is described by summarizing enhancement and image segmentation methods. Third, from the perspectives of accuracy and speed, feature extraction methods are compared and evaluated. Next, the existing applications of visual positioning technology in machining are discussed. Finally, the existing problems are summarized, and future research directions technology suggested.

[1]  Biao Mei,et al.  Accurate positioning of a drilling and riveting cell for aircraft assembly , 2021, Robotics Comput. Integr. Manuf..

[2]  Du-Ming Tsai,et al.  Fast and Precise Positioning in PCBs Using Deep Neural Network Regression , 2020, IEEE Transactions on Instrumentation and Measurement.

[3]  Anurag Jain,et al.  A Review: Image Segmentation Using Genetic Algorithm , 2014 .

[4]  张志龙,et al.  Remote Sensing Image Feature Extracting Based Multiple Ant Colonies Cooperation , 2014 .

[5]  Zhen Zhang,et al.  Integrated ANN-LWPA for cutting parameter optimization in WEDM , 2015 .

[6]  XiaoQi Chen,et al.  Laser visual sensing for seam tracking in robotic arc welding of titanium alloys , 2005 .

[7]  Zaifang Zhang,et al.  Methodology and implementation of a vision-oriented open CNC system for profile grinding , 2019 .

[8]  Xiaogang Wang,et al.  Visual Tracking with Fully Convolutional Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[9]  Ying Chen,et al.  Image enhancement under low luminance with strong light weakening , 2016, 2016 8th International Conference on Wireless Communications & Signal Processing (WCSP).

[10]  P S Malge,et al.  PCB Defect Detection, Classification and Localization using Mathematical Morphology and Image Processing Tools , 2014 .

[11]  D. F. Dauw,et al.  High-precision wire-EDM by online wire positioning control , 1994 .

[12]  Chen Wang,et al.  Research on the effect of image size on real-time performance of robot vision positioning , 2018, EURASIP J. Image Video Process..

[13]  Youhui Tian,et al.  Artificial Intelligence Image Recognition Method Based on Convolutional Neural Network Algorithm , 2020, IEEE Access.

[14]  Jussi Paakkari,et al.  Machine-vision-guided waterjet cutting , 1999, Industrial Lasers and Inspection.

[15]  Jia Sun,et al.  Precision Work-piece Detection and Measurement Combining Top-down and Bottom-up Saliency , 2018, Int. J. Autom. Comput..

[16]  Pravin P. Rathod,et al.  Ultrasonic vibration–assisted electric discharge machining: A research review , 2016 .

[17]  Yu-Liang Hsu,et al.  The Application of Deep Learning and Image Processing Technology in Laser Positioning , 2018, Applied Sciences.

[18]  Chih-Hung G. Li,et al.  Automated visual positioning and precision placement of a workpiece using deep learning , 2019, The International Journal of Advanced Manufacturing Technology.

[19]  Ji Zhao,et al.  HyNet: Hyper-scale object detection network framework for multiple spatial resolution remote sensing imagery , 2020 .

[20]  Qi Zhang,et al.  Research on Tire Marking Point Completeness Evaluation Based on K-Means Clustering Image Segmentation , 2020, Sensors.

[21]  S. Sidhu,et al.  Study of Magnetic Field-Assisted ED Machining of Metal Matrix Composites , 2016 .

[22]  Yong Yu,et al.  The Image Positioning and Segmentation Techniques about Gauge Visual Detection System between High Signals and Contact Net Based on Target Feature , 2012 .

[23]  Chen Liang.  Automatic Illumination of Microscopic Machine Vision Optimization for Precision Miniature Parts Automatic Assembly System , 2012 .

[24]  Xiaodiao Huang,et al.  Visual positioning for round pin chips based on a parametric deformable template , 2014 .

[25]  Wei,et al.  Adaptive Threshold Median Filter for Multiple-Impulse Noise , 2007 .

[26]  Shihua Gong,et al.  Error analysis and improved calibration algorithm for LED chip localization system based on visual feedback , 2017 .

[27]  H. H. Shahabi,et al.  In-cycle monitoring of tool nose wear and surface roughness of turned parts using machine vision , 2009 .

[28]  Lining Sun,et al.  Surface defect detection method for glass substrate using improved Otsu segmentation. , 2015, Applied optics.

[29]  Xingyu Gao,et al.  Design of apochromatic lens with large field and high definition for machine vision. , 2016, Applied optics.

[30]  Guolei Wang,et al.  High-precise Monocular Positioning with Infrared LED Visual Target , 2019, 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR).

[31]  Khalil Khalili,et al.  Flexible Automation in Porcelain Edge Polishing Using Machine Vision , 2016 .

[32]  Khan Muhammad,et al.  Adaptive Node Clustering Technique for Smart Ocean under Water Sensor Network (SOSNET) , 2019, Sensors.

[33]  Zonghua Zhang,et al.  3D shape measurement of discontinuous specular objects based on advanced PMD with bi-telecentric lens. , 2018, Optics express.

[34]  Zhi Yang,et al.  Semantic recognition of workpiece using computer vision for shape feature extraction and classification based on learning databases , 2017 .

[35]  Shun Cheng,et al.  Design on Control System of Cutting Machine for Chip Component Based on Vision Positioning , 2010 .

[36]  Yue Yang,et al.  Using distortion correction to improve the precision of camera calibration , 2019 .

[37]  N. Ramesh Babu,et al.  An adaptive control strategy for the abrasive waterjet cutting process with the integration of vision-based monitoring and a neuro-genetic control strategy , 2008 .

[38]  Zhou Lipin.  Precision Positioning of a 3-PRR Planar Parallel Manipulator Driven by Linear Ultrasonic Motors Based on Machine Vision , 2014 .

[39]  Xiaochun Wang,et al.  A scaled-MST-based clustering algorithm and application on image segmentation , 2019, Journal of Intelligent Information Systems.

[40]  Fei Han,et al.  Study on Machine Vision Orientation of Incremental Sheet Forming Process , 2011 .

[41]  Guofeng Wang,et al.  A Robotic grinding station based on an industrial manipulator and vision system , 2021, PloS one.

[42]  Ghassan A. Al-Kindi,et al.  Towards computer vision feedback for enhanced CNC machining , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[43]  Zhijun Chen,et al.  A comprehensive review of defect detection in 3C glass components , 2020 .

[44]  Yun Chen,et al.  Visual Positioning Algorithm Based on Micro Assembly Line , 2020, Journal of Physics: Conference Series.

[45]  Qiong-Hua Wang,et al.  1 × 2 optical switch based on electrowetting , 2014 .

[46]  Janez Kopac,et al.  New approach in tool wear measuring technique using CCD vision system , 2005 .

[47]  Lei Yang,et al.  Research on an Image-Based Visual Positioning System in the PDP Lighting Inspection Machine , 2011 .

[48]  Chung-Feng Jeffrey Kuo,et al.  Automatic marking point positioning of printed circuit boards based on template matching technique , 2019, J. Intell. Manuf..

[49]  Kenji Shimada,et al.  Automated workpiece localization for robotic welding , 2013, 2013 IEEE International Conference on Automation Science and Engineering (CASE).

[50]  Yong Jin Park,et al.  A new algorithm on the automatic TFT‐LCD mura defects inspection based on an effective background reconstruction , 2017 .

[51]  Fang Wang,et al.  Study on the Measuring and Positioning of the Rim Valve Hole Based on Machine Vision , 2014 .

[52]  A. Jullien-Corrigan,et al.  Measurement of high-frequency milling forces using piezoelectric dynamometers with dynamic compensation , 2020 .

[53]  Wuyi Ming,et al.  Comparative study of energy efficiency and environmental impact in magnetic field assisted and conventional electrical discharge machining , 2019, Journal of Cleaner Production.

[54]  Wei Zeng,et al.  Night video enhancement using improved dark channel prior , 2013, 2013 IEEE International Conference on Image Processing.

[55]  Yi Zhang,et al.  An Application of Improved RANSAC Algorithm in Visual Positioning , 2019, 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC).

[56]  Du-ming Tsai,et al.  Machine Vision-Based Positioning and Inspection Using Expectation–Maximization Technique , 2017, IEEE Transactions on Instrumentation and Measurement.

[57]  David Kerr,et al.  Assessment and visualisation of machine tool wear using computer vision , 2006 .

[58]  Chang You Li,et al.  Research on Microelectronic Packaging Position Detection Based on Machine Vision , 2013 .

[59]  Dominique Ginhac,et al.  Hardware-based smart camera for recovering high dynamic range video from multiple exposures , 2014 .

[60]  Chao-Ching Ho,et al.  On-line monitoring of laser-drilling process based on coaxial machine vision , 2014 .

[61]  Zhong Zhenyu,et al.  Edge detection technique combined with mathematic morphology and LoG operator , 2011 .

[62]  Hu Zhou,et al.  A Fast and Precise Measuring Method for Large Workpiece Based on Machine Vision , 2010 .

[63]  Holger Braun,et al.  Process Sensor Systems for Laser Beam Welding , 2012 .

[64]  Yongan Huang,et al.  Polygonal-feature-based shape context for flexible surface vision positioning , 2019, Measurement Science and Technology.

[65]  Xiang Yu,et al.  Target image matching algorithm based on pyramid model and higher moments , 2017, J. Comput. Sci..

[66]  Wenlong Song,et al.  Dam Burst: A region-merging-based image segmentation method , 2020, ArXiv.

[67]  Rohit Goel,et al.  Survey on Image Processing With Edge Detection Techniques MATHEMATICS , 2012 .

[68]  Xiangjun Zou,et al.  High-accuracy multi-camera reconstruction enhanced by adaptive point cloud correction algorithm , 2019, Optics and Lasers in Engineering.

[69]  Yonghong Chen,et al.  A survey on heliumspeech communications in saturation diving , 2020, China Communications.

[70]  Guofeng Wang,et al.  Visual Positioning and Grasping Application of Industrial Robot for Casting Parts , 2019, Automation, Control and Intelligent Systems.

[71]  Wei Ding,et al.  Development of a calibrating algorithm for Delta Robot’s visual positioning based on artificial neural network , 2016 .

[72]  Bo Tao,et al.  A vision-based fast base frame calibration method for coordinated mobile manipulators , 2021 .

[73]  Yue Cheng,et al.  Corners positioning for binocular ultra-wide angle long-wave infrared camera calibration , 2020 .

[74]  Wei Liu,et al.  Monocular-vision-based contouring error detection and compensation for CNC machine tools , 2019 .

[75]  Fatih Murat Porikli,et al.  LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement , 2018, Pattern Recognit. Lett..

[76]  Ping Zhu,et al.  The segmentation algorithm of improvement a two-dimensional Otsu and application research , 2010, 2010 2nd International Conference on Software Technology and Engineering.

[77]  Yanling Xu,et al.  An adaptive feature extraction algorithm for multiple typical seam tracking based on vision sensor in robotic arc welding , 2019, Sensors and Actuators A: Physical.

[78]  Xing Zhe Hou,et al.  Based on Machine Vision of Smart Meter Recognition and Accurate Positioning Technology Research , 2013 .

[79]  Qingchun Tang,et al.  Automated vision positioning system for dicing semiconductor chips using improved template matching method , 2018, The International Journal of Advanced Manufacturing Technology.

[80]  Wei Li,et al.  Research on Drilling Bit Positioning Strategy Based on SINS MWD System , 2019, IEEE Access.

[81]  Ujjwal Maulik,et al.  Understanding Deep Learning Techniques for Image Segmentation , 2019, ACM Comput. Surv..

[83]  K Kotsanidis,et al.  Rolling element bearings fault classification based on feature extraction from acceleration data and artificial neural networks , 2021 .

[84]  Marcelo Ricardo Stemmer,et al.  Automated PCB inspection in small series production based on SIFT algorithm , 2015, 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE).

[85]  Kuo-Yi Huang.  A Fast Inspection of Tool Electrode and Drilling Depth in EDM Drilling by Detection Line Algorithm , 2008, Sensors.

[86]  Gulraiz Khan,et al.  Segmentation of Crowd into Multiple Constituents Using Modified Mask R-CNN Based on Mutual Positioning of Human , 2019, 2019 International Conference on Communication Technologies (ComTech).

[87]  Hao Huang,et al.  Soft computing models and intelligent optimization system in electro-discharge machining of SiC/Al composites , 2016 .

[88]  Sergey Shevchenko,et al.  The Effect of a Rotating Medium on Bulk Acoustic Wave Polarization: From Theoretical Considerations to Perspective Angular Motion Sensor Design , 2020, Sensors.

[89]  Jianbo Yu,et al.  Chisel edge wear measurement of high-speed steel twist drills based on machine vision , 2021, Comput. Ind..

[90]  Stefan Domek,et al.  Calibration of Cameras and Fringe Pattern Projectors in the Vision System for Positioning of Workpieces on the CNC Machines , 2013 .

[91]  Shanwen Zhang,et al.  Vision Guiding System Based on Strong Positioning and Hand-eye Calibration , 2020 .

[92]  Xuesong Mei,et al.  Positioning and navigation system based on machine vision intended for laser-electrochemical micro-hole processing , 2018 .

[93]  Edward Chlebus,et al.  Examples of Laser Processing Control with Machine Vision Feedback , 2014 .

[94]  Joonki Paik,et al.  Low-light image restoration using bright channel prior-based variational Retinex model , 2017, EURASIP J. Image Video Process..

[95]  Surjya K. Pal,et al.  Application of digital image processing in tool condition monitoring: A review , 2013 .

[96]  Ashish Kumar Bhandari,et al.  A local contrast fusion based 3D Otsu algorithm for multilevel image segmentation , 2020, IEEE/CAA Journal of Automatica Sinica.

[97]  Steven Mills,et al.  Optimising Light Source Positions to Minimise Illumination Variation for 3D Vision , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[98]  Jiayu Xu,et al.  Optimization of image segmentation based on a visual positioning system for dispensing machines , 2015, 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP).

[99]  Dafeng Gong,et al.  Target Recognition of Industrial Robots Using Machine Vision in 5G Environment , 2021, Frontiers in Neurorobotics.

[100]  Kazuo Yamazaki,et al.  A study on three-dimensional vision system for machining setup verification , 2010 .

[101]  Hitoshi Kiya,et al.  Generation of Gradient-Preserving Images allowing HOG Feature Extraction , 2021, 2021 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW).

[102]  Jing Zhang,et al.  Hybrid particle swarm optimization with differential evolution for numerical and engineering optimization , 2018, Int. J. Autom. Comput..

[103]  A. Mendikute,et al.  Automated Raw Part Alignment by a Novel Machine Vision Approach , 2013 .

[104]  Shih-Ming Wang,et al.  An on-machine and vision-based depth-error measurement method for micro machine tools , 2011 .

[105]  Chunhong Pan,et al.  Enhancement of Low Light Level Images with coupled dictionary learning , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[106]  Petros Maragos,et al.  COGNIMUSE: a multimodal video database annotated with saliency, events, semantics and emotion with application to summarization , 2017, EURASIP J. Image Video Process..

[107]  Xinde Li,et al.  A Visual Kinematics Calibration Method for Manipulator Based on Nonlinear Optimization , 2020, ArXiv.

[108]  Martin Tomko,et al.  Infrastructure-Independent Indoor Localization and Navigation , 2019, ACM Comput. Surv..

[109]  Teruaki Hayashi,et al.  Feature Extraction of Laser Machining Data by Using Deep Multi-Task Learning , 2020, Inf..

[110]  Zengshun Zhao,et al.  Research on Image Segmentation based on Full Convolutional Neural Network , 2020 .

[111]  Zhenzhou Wang.  A laser back-lighting based metal transfer monitoring system for robotic gas metal arc welding , 2016 .