Combined Use of 3D and HSI for the Classification of Printed Circuit Board Components

Successful recycling of electronic waste requires accurate separation of materials such as plastics, PCBs and electronic components on PCBs (capacitors, transistors, etc.). This article therefore proposes a vision approach based on a combination of 3D and HSI data, relying on the mutual support of the datasets to compensate existing weaknesses when using single 3D- and HSI-Sensors. The combined dataset serves as a basis for the extraction of geometric and spectral features. The classification is performed and evaluated based on these extracted features which are exploited through rules. The efficiency of the proposed approach is demonstrated using real electronic waste and leads to convincing results with an overall accuracy (OA) of 98.24%. To illustrate that the addition of 3D data has added value, a comparison is also performed with an SVM classification based only on hyperspectral data.

[1]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[2]  Pedram Ghamisi,et al.  Object Detection Routine for Material Streams Combining RGB and Hyperspectral Reflectance Data Based on Guided Object Localization , 2020, IEEE Sensors Journal.

[3]  Alessandro Ulrici,et al.  Growing applications of hyperspectral and multispectral imaging , 2020 .

[4]  Diego Viejo,et al.  3D geological modelling using laser and hyperspectral data , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[5]  Stefan Hinz,et al.  Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers , 2015 .

[6]  Andrzej Sioma.  3D imaging methods in quality inspection systems , 2019, Symposium on Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA).

[7]  Damien Arvor,et al.  Ontology-based classification of remote sensing images using spectral rules , 2017, Comput. Geosci..

[8]  Reinhard Klette,et al.  Detection of Red-Meat Adulteration by Deep Spectral-Spatial Features in Hyperspectral Images , 2018, J. Imaging.

[9]  Zhangquan Shen,et al.  Review of shadow detection and de-shadowing methods in remote sensing , 2013, Chinese Geographical Science.

[10]  Boris Jutzi,et al.  Feature relevance assessment for the semantic interpretation of 3D point cloud data , 2013 .

[11]  Hamid Saeed Khan,et al.  Modern Trends in Hyperspectral Image Analysis: A Review , 2018, IEEE Access.

[12]  Karsten Rebner,et al.  Hyperspectral Imaging: A Review of Best Practice, Performance and Pitfalls for in-line and on-line Applications , 2012 .

[13]  Offer Rozenstein,et al.  Development of a new approach based on midwave infrared spectroscopy for post-consumer black plastic waste sorting in the recycling industry. , 2017, Waste management.

[14]  Olena Dubovyk,et al.  A rule-based approach for crop identification using multi-temporal and multi-sensor phenological metrics , 2018 .

[15]  Da-Wen Sun,et al.  Advanced Techniques for Hyperspectral Imaging in the Food Industry: Principles and Recent Applications. , 2019, Annual review of food science and technology.

[16]  Alexander Wendel,et al.  Extrinsic Parameter Calibration for Line Scanning Cameras on Ground Vehicles with Navigation Systems Using a Calibration Pattern , 2017, Sensors.

[17]  Yasser Mostafa,et al.  A Review on Various Shadow Detection and Compensation Techniques in Remote Sensing Images , 2017 .

[18]  S. J. Sutley,et al.  Imaging spectroscopy: Earth and planetary remote sensing with the USGS Tetracorder and expert systems , 2003 .

[19]  Silvia Serranti,et al.  Recycling-oriented characterization of plastic frames and printed circuit boards from mobile phones by electronic and chemical imaging. , 2014, Waste management.

[20]  Joachim Hertzberg,et al.  Markerless Ad-Hoc Calibration of a Hyperspectral Camera and a 3D Laser Scanner , 2018, IAS.

[21]  Qian Shi,et al.  Multitask Deep Learning With Spectral Knowledge for Hyperspectral Image Classification , 2019, IEEE Geoscience and Remote Sensing Letters.

[22]  Qiusheng Wu,et al.  Decision-Tree, Rule-Based, and Random Forest Classification of High-Resolution Multispectral Imagery for Wetland Mapping and Inventory , 2018, Remote. Sens..

[23]  Meng Yao,et al.  Knowledge and Geo-Object Based Graph Convolutional Network for Remote Sensing Semantic Segmentation , 2021, Sensors.

[24]  R. Clark,et al.  Reflectance spectroscopy: Quantitative analysis techniques for remote sensing applications , 1984 .

[25]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[26]  Takahiko Horiuchi,et al.  Spectral imaging method for material classification and inspection of printed circuit boards , 2010 .

[27]  Frank Boochs,et al.  Object Semantic Segmentation in Point Clouds - Comparison of a Deep Learning and a Knowledge-Based Method , 2021, ISPRS Int. J. Geo Inf..

[28]  Andrzej Sioma.  Automated Control of Surface Defects on Ceramic Tiles Using 3D Image Analysis , 2020, Materials.

[29]  Ribana Roscher,et al.  DETECTION OF DISEASE SYMPTOMS ON HYPERSPECTRAL 3D PLANT MODELS , 2016 .

[30]  William K. Michener,et al.  Detecting Wetland Change: A Rule-Based Approach Using NWI and SPOT-XS Data , 2000 .

[31]  Ahmed El-Rabbany,et al.  Multispectral LiDAR Data for Land Cover Classification of Urban Areas , 2017, Sensors.

[32]  Chengjun Chen,et al.  Semantic Segmentation of a Printed Circuit Board for Component Recognition Based on Depth Images , 2020, Sensors.

[33]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..