Automated near real-time earth observation level 2 product generation for semantic querying

Existing Earth observation (EO) content-based image retrieval (CBIR) systems support human-machine interaction through queries by metadata text information, image-wide summary statistics or either image, object or multi-object examples. No semantic CBIR (SCBIR) system in operating mode has ever been developed by the remote sensing (RS) community. At the same time, no EO dataderived Level 2 prototype product has ever been generated systematically at the ground segment, in contrast with the visionary goal of the Global Earth Observation System of Systems (GEOSS) implementation plan for years 2005-2015. Typical EO Level 2 products include: (i) a multi-spectral (MS) image corrected for atmospheric, adjacency and topographic effects, and (ii) a scene classification map (SCM), encompassing cloud and cloud-shadow quality layers. This work presents an original hybrid (combined deductive and inductive) feedback EO image understanding for semantic querying (EO-IU4SQ) system as a proof-of-concept where systematic multi-source EO big data transformation into Level 2 products is accomplished as a pre-condition for SCBIR, in agreement with the object-based image analysis (OBIA) paradigm and the Quality Assurance Framework for Earth Observation (QA4EO) guidelines. In the hybrid EO-IU4SQ system, statistical model-based/bottom-up/inductive/machine learning-from-data algorithms and physical model-based/top-down/deductive/human-to-machine knowledge transfer approaches are combined with feedback loops to take advantage of the complementary features of each and overcome their shortcomings.

[1]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[2]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[3]  Gilberto Câmara,et al.  An Algebra for Spatiotemporal Data: From Observations to Events , 2014, Trans. GIS.

[4]  Michael J. Mayo,et al.  Symbol Grounding and its Implications for Artificial Intelligence , 2003, ACSC.

[5]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[6]  Shiyong Cui,et al.  Information Content of Very-High-Resolution SAR Images: Semantics, Geospatial Context, and Ontologies , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[7]  Stefan Growe,et al.  KNOWLEDGE BASED INTERPRETATION OF MULTISENSOR AND MULTITEMPORAL REMOTE SENSING IMAGES , 1999 .

[8]  Iasonas Kokkinos,et al.  Deep Filter Banks for Texture Recognition, Description, and Segmentation , 2015, International Journal of Computer Vision.

[9]  Gabriele Moser,et al.  Processing of Extremely High Resolution LiDAR and RGB Data: Outcome of the 2015 IEEE GRSS Data Fusion Contest—Part B: 3-D Contest , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  L. Pessoa,et al.  Mach Bands: How Many Models are Possible? Recent Experimental Findings and Modeling Attempts , 1996, Vision Research.

[11]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[12]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Andrea Baraldi,et al.  Combined detection of intensity and chromatic contours in color images , 1996 .

[14]  Peter Baumann,et al.  Big Data Analytics for Earth Sciences: the EarthServer approach , 2016, Int. J. Digit. Earth.

[15]  Andrea Baraldi,et al.  Operational Two-Stage Stratified Topographic Correction of Spaceborne Multispectral Imagery Employing an Automatic Spectral-Rule-Based Decision-Tree Preliminary Classifier , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Takashi Matsuyama,et al.  SIGMA: A Knowledge-Based Aerial Image Understanding System , 1990 .

[17]  Derek Thompson,et al.  Fundamentals of spatial information systems , 1992, A.P.I.C. series.

[18]  Makoto Nagao,et al.  A Structural Analysis of Complex Aerial Photographs , 1980, Advanced Applications in Pattern Recognition.

[19]  David W. Jacobs,et al.  Segment-based simple-connectivity measure design and implementation , 2014 .

[20]  Andrea Baraldi,et al.  Operational Automatic Remote Sensing Image Understanding Systems: Beyond Geographic Object-Based and Object-Oriented Image Analysis (GEOBIA/GEOOIA). Part 2: Novel system Architecture, Information/Knowledge Representation, Algorithm Design and Implementation , 2012, Remote. Sens..