Shadow Detection and Restoration for Hyperspectral Images Based on Nonlinear Spectral Unmixing

Shadows are frequently observable in high-resolution images, raising challenges in image interpretation, such as classification and object detection. In this paper, we propose a novel framework for shadow detection and restoration of atmospherically corrected hyperspectral images based on nonlinear spectral unmixing. The mixture model is applied pixel-wise as a nonlinear combination of endmembers related to both pure sunlit and shadowed spectra, where the former are manually selected from scenes and the latter are derived from sunlit spectra following physical assumptions. Shadowed pixels are restored by simulating their exposure to sunlight through a combination of sunlit endmembers spectra, weighted by abundance values. The proposed framework is demonstrated on real airborne hyperspectral images. A comprehensive assessment of the restored images is carried out both visually and quantitatively. With respect to binary shadow masks, our framework can produce soft shadow detection results, keeping the natural transition of illumination conditions on shadow boundaries. Our results show that the framework can effectively detect shadows and restore information in shadowed regions.

[1]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Robert A. Leathers,et al.  A novel method for illumination suppression in hyperspectral images , 2008, SPIE Defense + Commercial Sensing.

[3]  P. Dare Shadow Analysis in High-Resolution Satellite Imagery of Urban Areas , 2005 .

[4]  José M. Bioucas-Dias,et al.  Nonlinear mixture model for hyperspectral unmixing , 2009, Remote Sensing.

[5]  Chengshan Han,et al.  Automatic Shadow Detection for Multispectral Satellite Remote Sensing Images in Invariant Color Spaces , 2020, Applied Sciences.

[6]  Chi-Keung Tang,et al.  Shadow Removal from Single RGB-D Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Yuhong He,et al.  Fully constrained linear spectral unmixing based global shadow compensation for high resolution satellite imagery of urban areas , 2015, Int. J. Appl. Earth Obs. Geoinformation.

[8]  Jocelyn Chanussot,et al.  Spectral Variability in Hyperspectral Data Unmixing: A comprehensive review , 2020, IEEE Geoscience and Remote Sensing Magazine.

[9]  Rynson W. H. Lau,et al.  DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  D. Cerra,et al.  Improving the Classification in Shadowed Areas using Nonlinear Spectral Unmixing , 2020, IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium.

[11]  Jörgen Ahlberg,et al.  Illumination and shadow compensation of hyperspectral images using a digital surface model and non-linear least squares estimation , 2011, Remote Sensing.

[12]  Hagit Hel-Or,et al.  Shadow Removal Using Intensity Surfaces and Texture Anchor Points , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Karl Kunisch,et al.  Total Generalized Variation , 2010, SIAM J. Imaging Sci..

[14]  Mohammed Bennamoun,et al.  Automatic Shadow Detection and Removal from a Single Image , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Qing Zhang,et al.  Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization , 2015, IEEE Transactions on Image Processing.

[16]  Bo Huang,et al.  Shadow Detection and Reconstruction in High-Resolution Satellite Images via Morphological Filtering and Example-Based Learning , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Antonio J. Plaza,et al.  A quantitative and comparative analysis of endmember extraction algorithms from hyperspectral data , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Yoshifumi Yasuoka,et al.  Simulated recovery of information in shadow areas on IKONOS image by combing ALS data , 2002 .

[19]  Farid Melgani,et al.  A Complete Processing Chain for Shadow Detection and Reconstruction in VHR Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[20]  Richard J. Murphy,et al.  A Physics-Based Deep Learning Approach to Shadow Invariant Representations of Hyperspectral Images , 2018, IEEE Transactions on Image Processing.

[21]  John F. Mustard,et al.  Spectral unmixing , 2002, IEEE Signal Process. Mag..

[22]  Zhangquan Shen,et al.  Review of shadow detection and de-shadowing methods in remote sensing , 2013, Chinese Geographical Science.

[23]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[24]  Rupert Müller,et al.  Towards the Spectral Restoration of Shadowed Areas in Hyperspectral Images Based on Nonlinear Unmixing , 2019, 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[25]  R. Richter,et al.  De‐shadowing of satellite/airborne imagery , 2005 .

[26]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[27]  Sabine Süsstrunk,et al.  Automatic and Accurate Shadow Detection Using Near-Infrared Information , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Guillaume Roussel,et al.  A sun/shadow approach for the classification of hyperspectral data , 2016, 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[29]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Claas Köhler,et al.  Airborne Imaging Spectrometer HySpex , 2016 .

[31]  Dimitris Samaras,et al.  Large-Scale Training of Shadow Detectors with Noisily-Annotated Shadow Examples , 2016, ECCV.

[32]  John R. Schott,et al.  Remote Sensing: The Image Chain Approach , 1996 .

[33]  Alfred O. Hero,et al.  Nonlinear Unmixing of Hyperspectral Images: Models and Algorithms , 2013, IEEE Signal Processing Magazine.

[34]  Michal Shimoni,et al.  A shadow detection method for remote sensing images using VHR hyperspectral and LIDAR data , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[35]  Daniel Schläpfer,et al.  An automatic atmospheric correction algorithm for visible/NIR imagery , 2006 .

[36]  Qiang Zhang,et al.  Detecting objects under shadows by fusion of hyperspectral and LiDAR DATA: A physical model approach , 2013, 2013 5th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[37]  Dimitris Samaras,et al.  Shadow Detection with Conditional Generative Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[38]  W. Shi,et al.  Quantitative Analysis of Shadow Effects in High-resolution Images of Urban Areas , 2005 .

[39]  Fatih Omruuzun,et al.  Shadow removal from VNIR hyperspectral remote sensing imagery with endmember signature analysis , 2015, Commercial + Scientific Sensing and Imaging.

[40]  Gerrit Polder,et al.  The hype in spectral imaging , 2020, Spectroscopy Europe.

[41]  Makoto Nagao,et al.  Region extraction and shape analysis in aerial photographs , 1979 .

[42]  Martin D. Levine,et al.  Removing shadows , 2005, Pattern Recognit. Lett..

[43]  Yong Liu,et al.  Recurrent Shadow Attention Model (RSAM) for shadow removal in high-resolution urban land-cover mapping , 2020 .

[44]  José M. Bioucas-Dias,et al.  Vertex component analysis: a fast algorithm to unmix hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[45]  Rob Heylen,et al.  A Multilinear Mixing Model for Nonlinear Spectral Unmixing , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[46]  John R. Miller,et al.  Comparative study between a new nonlinear model and common linear model for analysing laboratory simulated‐forest hyperspectral data , 2009 .

[47]  Pooya Sarabandi,et al.  Shadow detection and radiometric restoration in satellite high resolution images , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[48]  Cheng Shi,et al.  Towards Ghost-free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN , 2019, AAAI.

[49]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[50]  Richard J. Murphy,et al.  Unsupervised feature learning for illumination robustness , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[51]  James A. Gardner,et al.  Algorithm for de-shadowing spectral imagery , 2002, SPIE Optics + Photonics.

[52]  Victor J. D. Tsai,et al.  A comparative study on shadow compensation of color aerial images in invariant color models , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Deshuai Yuan,et al.  Urban Shadow Detection and Classification Using Hyperspectral Image , 2017, Journal of the Indian Society of Remote Sensing.

[54]  Paul D. Gader,et al.  A Review of Nonlinear Hyperspectral Unmixing Methods , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[55]  Lalit Kumar,et al.  Diffuse Skylight as a Surrogate for Shadow Detection in High-Resolution Imagery Acquired Under Clear Sky Conditions , 2018, Remote. Sens..

[56]  X. Briottet,et al.  Shadow detection in very high spatial resolution aerial images: A comparative study , 2013 .

[57]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[58]  Li Yan,et al.  Deshadowing of Urban Airborne Imagery Based on Object-Oriented Automatic Shadow Detection and Regional Matching Compensation , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[59]  Antonio J. Plaza,et al.  Automated Extraction of Image-Based Endmember Bundles for Improved Spectral Unmixing , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.