Object Detection Routine for Material Streams Combining RGB and Hyperspectral Reflectance Data Based on Guided Object Localization

Electronic waste is the fastest growing type of scrap globally and is an important challenge due to its heterogeneity, intrinsic toxicity and potential environmental impact. With an objective of obtaining information on the composition of printed circuit boards (PCBs) through non-invasive analysis to aid in recycling and recovery of precious waste, the goal of this paper is to propose a scheme towards the fusion of RGB and hyperspectral data in object detection. State-of-art detectors come with their own set of challenges which make them inapplicable to PCB recycling. We introduce a method which promises to achieve object detection based on multi-sensor data by utilizing the hyperspectral data to localize components and compare the results to a conventional single-sensor (RGB) based approach.

[1]  Saurabh Maheshwari,et al.  Contrast limited adaptive histogram equalization based enhancement for real time video system , 2014, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[2]  Wei Li,et al.  Recycle deep features for better object detection , 2016, ArXiv.

[3]  Wei Li,et al.  Text recognition for information retrieval in images of printed circuit boards , 2014, IECON 2014 - 40th Annual Conference of the IEEE Industrial Electronics Society.

[4]  Bernt Schiele,et al.  Learning Non-maximum Suppression , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[6]  Mahdi Khodadadzadeh,et al.  Multi-Sensor Spectral Imaging of Geological Samples: A Data Fusion Approach Using Spatio-Spectral Feature Extraction , 2019, Sensors.

[7]  Wei Wei,et al.  Salient object detection in hyperspectral imagery using multi-scale spectral-spatial gradient , 2018, Neurocomputing.

[8]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[9]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[10]  Wolfram Burgard,et al.  Multimodal deep learning for robust RGB-D object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Abhinav Vishnu,et al.  Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction , 2018, ArXiv.

[12]  Jun Li,et al.  Advanced Spectral Classifiers for Hyperspectral Images: A review , 2017, IEEE Geoscience and Remote Sensing Magazine.

[13]  R. Younes,et al.  Dimensionality reduction on hyperspectral images: A comparative review based on artificial datas , 2011, 2011 4th International Congress on Image and Signal Processing.

[14]  J. Chanussot,et al.  On the influence of feature reduction for the classification of hyperspectral images based on the extended morphological profile , 2010 .

[15]  Zsolt Kira,et al.  Data-Efficient Graph Embedding Learning for PCB Component Detection , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[16]  Wei Li,et al.  Segmentation and classification of THCs on PCBAs , 2013, 2013 11th IEEE International Conference on Industrial Informatics (INDIN).

[17]  Dan Zecha,et al.  A closer look: Small object detection in faster R-CNN , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[18]  Pedram Ghamisi,et al.  An Integrated Multi-Sensor System for the In-Line Monitoring of Material Streams , 2019, 2019 IEEE SENSORS.

[19]  Jon Atli Benediktsson,et al.  Multisource and Multitemporal Data Fusion in Remote Sensing , 2018, ArXiv.

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[21]  G. Mercier,et al.  Support vector machines for hyperspectral image classification with spectral-based kernels , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[22]  Wolfram Burgard,et al.  Choosing smartly: Adaptive multimodal fusion for object detection in changing environments , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Cesare Pautasso,et al.  Mean Average Precision , 2009, Encyclopedia of Database Systems.

[26]  Thomas B. Moeslund,et al.  Multimodal Neural Network for Overhead Person Re-Identification , 2017, 2017 International Conference of the Biometrics Special Interest Group (BIOSIG).

[27]  Xavier Bresson,et al.  Segmentation for Hyperspectral Images with Priors , 2010, ISVC.

[28]  Jon Atli Benediktsson,et al.  Classification and feature extraction for remote sensing images from urban areas based on morphological transformations , 2003, IEEE Trans. Geosci. Remote. Sens..

[29]  Jon Atli Benediktsson,et al.  Generalized Differential Morphological Profiles for Remote Sensing Image Classification , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Martin Kampel,et al.  Pre-Analysis of Printed Circuit Boards Based On Multispectral Imaging for Vision Based Recognition of Electronics Waste , 2015 .

[32]  Nikos Komodakis,et al.  Building detection in very high resolution multispectral data with deep learning features , 2015, 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).