Large-Scale Semantic 3-D Reconstruction: Outcome of the 2019 IEEE GRSS Data Fusion Contest—Part A

In this article, we present the scientific outcomes of the 2019 Data Fusion Contest organized by the Image Analysis and Data Fusion Technical Committee of the IEEE Geoscience and Remote Sensing Society. The 2019 Contest addressed the problem of 3-D reconstruction and 3-D semantic understanding on a large scale. Several competitions were organized to assess specific issues, such as elevation estimation and semantic mapping from a single view, two views, or multiple views. In Part A, we report the results of the best-performing approaches for semantic 3-D reconstruction according to these various setups, whereas 3-D point cloud semantic mapping is discussed in Part B.

[1]  Zhengyang Wang,et al.  Smoothed dilated convolutions for improved dense prediction , 2018, Data Mining and Knowledge Discovery.

[2]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[3]  Gordon Christie,et al.  Learning Geocentric Object Pose in Oblique Monocular Images , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Pietro Perona,et al.  Fast Feature Pyramids for Object Detection , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[6]  Yong-Sheng Chen,et al.  Pyramid Stereo Matching Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Qian Du,et al.  Hyperspectral and LiDAR Data Fusion: Outcome of the 2013 GRSS Data Fusion Contest , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[8]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Alexandre Boulch,et al.  Processing of Extremely High-Resolution LiDAR and RGB Data: Outcome of the 2015 IEEE GRSS Data Fusion Contest–Part A: 2-D Contest , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Naoto Yokoya,et al.  Large-Scale Semantic 3-D Reconstruction: Outcome of the 2019 IEEE GRSS Data Fusion Contest—Part B , 2021, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[11]  S. K. McFeeters.  The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features , 1996 .

[12]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[13]  Mathias Schardt,et al.  Advanced DTM Generation from Very High Resolution Satellite Stereo Images , 2015 .

[14]  Qian Du,et al.  Multi-Modal Change Detection, Application to the Detection of Flooded Areas: Outcome of the 2009–2010 Data Fusion Contest , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[15]  Hongyan Zhang,et al.  Multi-Level Fusion of the Multi-Receptive Fields Contextual Networks and Disparity Network for Pairwise Semantic Stereo , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[16]  Yanfei Zhong,et al.  Pop-Net: Encoder-Dual Decoder for Semantic Segmentation and Single-View Height Estimation , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[17]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Wei Liu,et al.  Pairwise Stereo Image Disparity and Semantics Estimation with the Combination of U-Net and Pyramid Stereo Matching Network , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[19]  SSC Satimage,et al.  Precision Rectification of SPOT Imagery , 2007 .

[20]  Aleksandra Pizurica,et al.  Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[21]  Rowel Atienza,et al.  Fast Disparity Estimation Using Dense Networks , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Gregory D. Hager,et al.  Semantic Stereo for Incidental Satellite Images , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Saket Kunwar.  U-Net Ensemble for Semantic and Height Estimation Using Coarse-Map Initialization , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[24]  Naoto Yokoya,et al.  Advanced Multi-Sensor Optical Remote Sensing for Urban Land Use and Land Cover Classification: Outcome of the 2018 IEEE GRSS Data Fusion Contest , 2019, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[25]  Pablo d'Angelo,et al.  Dense multi-view stereo from satellite imagery , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[26]  Aloysius Wehr,et al.  Airborne laser scanning—an introduction and overview , 1999 .

[27]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[28]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Naoto Yokoya,et al.  Open Data for Global Multimodal Land Use Classification: Outcome of the 2017 IEEE GRSS Data Fusion Contest , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[30]  Jocelyn Chanussot,et al.  Decision Fusion for the Classification of Hyperspectral Data: Outcome of the 2008 GRS-S Data Fusion Contest , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Gabriele Moser,et al.  Processing of Extremely High Resolution LiDAR and RGB Data: Outcome of the 2015 IEEE GRSS Data Fusion Contest—Part B: 3-D Contest , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Nikos Paragios,et al.  Multitemporal Very High Resolution From Space: Outcome of the 2016 IEEE GRSS Data Fusion Contest , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[33]  Wei Liu,et al.  Semantic 3D Reconstruction Using Multi-View High-Resolution Satellite Images Based on U-Net and Image-Guided Depth Fusion , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[34]  Thierry Toutin,et al.  Review of developments in geometric modelling for high resolution satellite pushbroom sensors , 2012 .

[35]  Xiaojuan Qi,et al.  ICNet for Real-Time Semantic Segmentation on High-Resolution Images , 2017, ECCV.

[36]  Jocelyn Chanussot,et al.  Comparison of Pansharpening Algorithms: Outcome of the 2006 GRS-S Data-Fusion Contest , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Seyed Majid Azimi,et al.  3D Semantic Segmentation from Multi-View Optical Satellite Images , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[38]  William J. Emery,et al.  Urban Mapping Using Coarse SAR and Optical Data: Outcome of the 2007 GRSS Data Fusion Contest , 2008, IEEE Geoscience and Remote Sensing Letters.

[39]  Pablo d'Angelo.  Automatic Orientation of large multitemporal Satellite Image Blocks , 2013 .

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Marc Bosch,et al.  A multiple view stereo benchmark for satellite imagery , 2016, 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[43]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[44]  Xiangyu Zhang,et al.  Large Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Zhang Li,et al.  AUTOMATIC DSM GENERATION FROM LINEAR ARRAY IMAGERY DATA , 2004 .

[46]  Qian Du,et al.  Multi-Modal and Multi-Temporal Data Fusion: Outcome of the 2012 GRSS Data Fusion Contest , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Enric Meinhardt,et al.  Automatic 3D Reconstruction from Multi-date Satellite Images , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[48]  Foreword to the Special Issue on Optical Multiangular Data Exploitation and Outcome of the 2011 GRSS Data Fusion Contest , 2012 .

[49]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).