Multispectral Imaging for Fine-Grained Recognition of Powders on Complex Backgrounds

Hundreds of materials, such as drugs, explosives, makeup, food additives, are in the form of powder. Recognizing such powders is important for security checks, criminal identification, drug control, and quality assessment. However, powder recognition has drawn little attention in the computer vision community. Powders are hard to distinguish: they are amorphous, appear matte, have little color or texture variation and blend with surfaces they are deposited on in complex ways. To address these challenges, we present the first comprehensive dataset and approach for powder recognition using multi-spectral imaging. By using Shortwave Infrared (SWIR) multi-spectral imaging together with visible light (RGB) and Near Infrared (NIR), powders can be discriminated with reasonable accuracy. We present a method to select discriminative spectral bands to significantly reduce acquisition time while improving recognition accuracy. We propose a blending model to synthesize images of powders of various thickness deposited on a wide range of surfaces. Incorporating band selection and image synthesis, we conduct fine-grained recognition of 100 powders on complex backgrounds, and achieve 60%~70% accuracy on recognition with known powder location, and over 40% mean IoU without known location.

[1]  Z. Bodó,et al.  Some optical properties of luminescent powders , 1951 .

[2]  Peter D. Johnson.  Absolute Optical Absorption from Diffuse Reflectance , 1952 .

[3]  N. Melamed.  Optical Properties of Powders. Part I. Optical Absorption Coefficients and the Absolute Value of the Diffuse Reflectance. Part II. Properties of Luminescent Powders , 1963 .

[4]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[5]  E. L. Simmons,et al.  An Equation Relating the Diffuse Reflectance of Weakly Absorbing Powdered Samples to the Fundamental Optical Parameters , 1971 .

[6]  Harry G. Hecht,et al.  The Interpretation of Diffuse Reflectance Spectra , 1976, Journal of research of the National Bureau of Standards. Section A, Physics and chemistry.

[7]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.

[8]  Qian Du,et al.  A joint band prioritization and band-decorrelation approach to band selection for hyperspectral image classification , 1999, IEEE Trans. Geosci. Remote. Sens..

[9]  Edward H. Adelson,et al.  On seeing stuff: the perception of materials by humans and machines , 2001, IS&T/SPIE Electronic Imaging.

[10]  Alexei A. Efros,et al.  Image quilting for texture synthesis and transfer , 2001, SIGGRAPH.

[11]  Derek Abbott,et al.  Powder detection using THz imaging , 2002, CLEO 2002.

[12]  K. Kawase,et al.  Non-destructive terahertz imaging of illicit drugs using spectral fingerprints. , 2003, Optics express.

[13]  Kodo Kawase,et al.  Terahertz Imaging For Drug Detection And Large-Scale Integrated Circuit Inspection , 2004 .

[14]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[15]  P. Taday,et al.  Detection and identification of explosives using terahertz pulsed spectroscopic imaging , 2005 .

[16]  Chein-I Chang,et al.  Constrained band selection for hyperspectral imagery , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Adolfo Martínez Usó,et al.  Clustering-Based Hyperspectral Band Selection Using Information Measures , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Yuxuan Wang,et al.  A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network , 2007, 2007 IEEE International Symposium on Signal Processing and Information Technology.

[19]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[20]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[21]  Linda G. Shapiro,et al.  Unsupervised Template Learning for Fine-Grained Object Recognition , 2012, NIPS.

[22]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[23]  Yongchao Zhao,et al.  A Fast Volume-Gradient-Based Band Selection Method for Hyperspectral Image , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[24]  Noah Snavely,et al.  Material recognition in the wild with the Materials in Context Database , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Jonathan Krause,et al.  Fine-grained recognition without part annotations , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Ko Nishino,et al.  Automatically discovering local visual material attributes , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[28]  Lorenzo Bruzzone,et al.  Hyperspectral Band Selection Based on Rough Set , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[29]  Derek Hoiem,et al.  Geometry-Informed Material Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Qingquan Li,et al.  A Novel Ranking-Based Clustering Approach for Hyperspectral Band Selection , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Hang Zhang,et al.  Friction from Reflectance: Deep Reflectance Codes for Predicting Physical Surface Properties from One-Shot In-Field Reflectance , 2016, ECCV.

[32]  Stefanos Zafeiriou,et al.  Fine-Grained Material Classification Using Micro-geometry and Reflectance , 2016, ECCV.

[33]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Alexei A. Efros,et al.  A 4D Light-Field Dataset and CNN Architectures for Material Recognition , 2016, ECCV.

[35]  Kristin J. Dana,et al.  Deep TEN: Texture Encoding Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Balazs Kovacs,et al.  Shading Annotations in the Wild , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Asaf Tzadok,et al.  Fine-Grained Recognition of Thousands of Object Categories with Single-Example Training , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Chein-I Chang,et al.  Band Subset Selection for Anomaly Detection in Hyperspectral Imagery , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Hang Zhang,et al.  Differential Angular Imaging for Material Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Matthew P. Nelson,et al.  Real-time, short-wave, infrared hyperspectral conforming imaging sensor for the detection of threat materials , 2018, Commercial + Scientific Sensing and Imaging.

[41]  Zhengqi Li,et al.  Learning Intrinsic Image Decomposition from Watching the World , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[42]  Ling-Yu Duan,et al.  Group-Sensitive Triplet Embedding for Vehicle Reidentification , 2018, IEEE Transactions on Multimedia.

[43]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[44]  Kaiming He,et al.  Group Normalization , 2018, ECCV.

[45]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Ling-Yu Duan,et al.  Embedding Adversarial Learning for Vehicle Re-Identification , 2019, IEEE Transactions on Image Processing.

[47]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.