Calibrated Vehicle Paint Signatures for Simulating Hyperspectral Imagery

We investigate a procedure for rapidly adding calibrated vehicle visible-near infrared (VNIR) paint signatures to an existing hyperspectral simulator - The Digital Imaging and Remote Sensing Image Generation (DIRSIG) model - to create more diversity in simulated urban scenes. The DIRSIG model can produce synthetic hyperspectral imagery with user-specified geometry, atmospheric conditions, and ground target spectra. To render an object pixel’s spectral signature, DIRSIG uses a large database of reflectance curves for the corresponding object material and a bidirectional reflectance model to introduce s due to orientation and surface structure. However, this database contains only a few spectral curves for vehicle paints and generates new paint signatures by combining these curves internally. In this paper we demonstrate a method to rapidly generate multiple paint spectra, flying a drone carrying a pushbroom hyperspectral camera to image a university parking lot. We then process the images to convert them from the digital count space to spectral reflectance without the need of calibration panels in the scene, and port the paint signatures into DIRSIG for successful integration into the newly rendered sets of synthetic VNIR hyperspectral scenes.

[1]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[2]  Matthew J. Hoffman,et al.  Integrating Hyperspectral Likelihoods in a Multidimensional Assignment Algorithm for Aerial Vehicle Tracking , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[3]  Magnus Wrenninge,et al.  Synscapes: A Photorealistic Synthetic Dataset for Street Scene Parsing , 2018, ArXiv.

[4]  Alexander Berk,et al.  A 3-DIMENSIONAL RADIATIVE-TRANSFER HYPERSPECTRAL IMAGE SIMULATOR FOR ALGORITHM VALIDATION , 2001 .

[5]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[6]  John Kerekes,et al.  Efficient generation of image chips for training deep learning algorithms , 2017, Defense + Security.

[7]  Jonathan Piper,et al.  An End-to-End Hyperspectral Scene Simulator with Alternate Adjacency Effect Models and Its Comparison with CameoSim , 2019, Remote. Sens..

[8]  Christopher Kanan,et al.  AeroRIT: A New Scene for Hyperspectral Image Analysis , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[9]  John R. Schott,et al.  Incorporation of polarization into the DIRSIG synthetic image generation model , 2002, SPIE Optics + Photonics.

[10]  Matthew J. Hoffman,et al.  Tracking in Aerial Hyperspectral Videos Using Deep Kernelized Correlation Filters , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Faiz Ur Rahman,et al.  SIAMESE NETWORK WITH MULTI-LEVEL FEATURES FOR PATCH-BASED CHANGE DETECTION IN SATELLITE IMAGERY , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[12]  Harry N. Gross,et al.  An Advanced Synthetic Image Generation Model and its Application to Multi/Hyperspectral Algorithm Development , 1999 .

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[15]  Anil K. Jain,et al.  On-line signature verification, , 2002, Pattern Recognit..

[16]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[17]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[18]  Balvinder Kaur,et al.  Digital imaging and remote sensing image generator (DIRSIG) as applied to NVESD sensor performance modeling , 2016, SPIE Defense + Security.

[19]  Jitendra Malik,et al.  Habitat: A Platform for Embodied AI Research , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[20]  Scott D. Brown,et al.  Advances in wide-area hyperspectral image simulation , 2003, SPIE Defense + Commercial Sensing.

[21]  Ronald Kemker,et al.  Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[22]  Carl Salvaggio,et al.  MX-1: A New Multi-Modal Remote Sensing UAS Payload with High Accuracy GPS and IMU , 2019, 2019 IEEE Systems and Technologies for Remote Sensing Applications Through Unmanned Aerial Systems (STRATUS).

[23]  Marilyn A. Gilmore,et al.  CAMEO-SIM: a physics-based broadband scene simulation tool for assessment of camouflage, concealment, and deception methodologies , 2001 .

[24]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[25]  Scott D. Brown,et al.  DIRSIG5: Next-Generation Remote Sensing Data and Image Simulation Framework , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..