Oblique aerial images: a review focusing on georeferencing procedures

ABSTRACT The automatic processing of oblique aerial images is currently one of the most attractive topics in the fields of photogrammetry, remote sensing and computer vision. This kind of images has gained popularity throughout the last years, as it enables depiction of both vertical and horizontal structures of the urban environment, being ideal for various photogrammetric applications, apart from visualization and interpretation purposes. In most cases, the georeferencing of the images is indispensable for their metric exploitation, usually in terms of computation of their camera exterior orientation parameters. For other applications, georeferencing by means of image orthorectification is required, whereas a simple georeferencing in terms of mapping of the center point of the images or the outline of their footprint or image rectification via a 2D transformation also suffices in some cases. However, oblique airborne images pose several challenges in automated georeferencing procedures, due to complex image geometry and strongly varying appearance between images of the same scene taken from different viewing angles. In this paper, a thorough literature review is conducted in order to inform about the recent research in georeferencing of datasets including oblique aerial imagery. The review starts with a classification of oblique aerial imagery and a presentation of the most commonly used camera configurations for oblique imagery acquisition. A brief presentation of their history along with old applications is then made, followed by a categorization of the literature regarding state-of-the-art georeferencing procedures as well as other research trends and applications using oblique aerial images. Then, a more in-depth description of recent research on georeferencing of oblique image datasets is presented. Finally, the significant points, research results and conclusions derived from the literature are discussed, so that they can be utilized by future researchers in the development of georeferencing methods targeted to oblique airborne imagery.

[1]  Fabio Remondino,et al.  ISPRS benchmark for multi - platform photogrammetry , 2015 .

[2]  Jie Zhang Dense Point Cloud Extraction From Oblique Imagery , 2013 .

[3]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[4]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[6]  M.J.P.M. Lemmens Digital Oblique Aerial Cameras (1): A Survey of Features and Systems , 2014 .

[7]  K. Nurminen,et al.  Geometric processing workflow for vertical and oblique hyperspectral frame images collected using UAV , 2014 .

[8]  Paul R. Wolf,et al.  Elements of photogrammetry with air photo interpretation and remote sensing / Paul R. Wolf , 1974 .

[9]  Fabio Menna,et al.  MULTI-TEMPORAL ANALYSIS OF LANDSCAPES AND URBAN AREAS , 2012 .

[10]  Fabio Remondino,et al.  Oblique aerial imagery for NMA : some best practices , 2016 .

[11]  C. Ioannidis Metric Exploitation of a Single Low Oblique Aerial Image , 2015 .

[12]  Huang Yao,et al.  HIERARCHICAL REPETITION EXTRACTION FOR BUILDING FAÇADE RECONSTRUCTION FROM OBLIQUE AERIAL IMAGES , 2015 .

[13]  George Athanasiou,et al.  3D reconstruction of disaster scenes for urban search and rescue , 2018, Multimedia Tools and Applications.

[14]  J. Rau,et al.  A MODIFIED PROJECTIVE TRANSFORMATION SCHEME FOR MOSAICKING MULTI-CAMERA IMAGING SYSTEM EQUIPPED ON A LARGE PAYLOAD FIXED- WING UAS , 2015 .

[15]  S. Verykokou,et al.  Automatic Rough Georeferencing of Multiview Oblique and Vertical Aerial Image Datasets of Urban Scenes , 2016 .

[16]  M. Rothermel,et al.  Oblique Image Data Processing – Potential , Experiences and Recommendations , 2013 .

[17]  Fabio Remondino,et al.  OBLIQUE PHOTOGRAMMETRY SUPPORTING 3D URBAN RECONSTRUCTION OF COMPLEX SCENARIOS , 2017 .

[18]  F. Remondinob,et al.  ORIENTATION OF OBLIQUE AIRBORNE IMAGE SETS-EXPERIENCES FROM THE ISPRS / EUROSDR BENCHMARK ON MULTI-PLATFORM PHOTOGRAMMETRY , 2016 .

[19]  Zhanyi Hu,et al.  Geographic, geometrical and semantic reconstruction of urban scene from high resolution oblique aerial images , 2019, IEEE/CAA Journal of Automatica Sinica.

[20]  George Vosselman,et al.  Building extraction from oblique airborne imagery based on robust façade detection , 2012 .

[21]  GLACIER MONITORING SURVEY FLIGHTS BELOW CLOUDS IN ALASKA: OBLIQUE AERIAL PHOTOGRAPHY UTILISING DIGITAL MULTIPLE-IMAGE PHOTOGRAMMETRY TO COPE WITH ADVERSE WEATHER , 2006 .

[22]  C. Heipke,et al.  GRAPH MATCHING FOR THE REGISTRATION OF PERSISTENT SCATTERERS TO OPTICAL OBLIQUE IMAGERY , 2016 .

[23]  Thomas S. Huang,et al.  An installation for interactive transfer of information from oblique aerial photos to maps , 1977, Digital Image Processing.

[24]  Leif Kobbelt,et al.  Automatic Registration of Oblique Aerial Images with Cadastral Maps , 2010, ECCV Workshops.

[25]  F. Nex,et al.  OBLIQUE MULTI-CAMERA SYSTEMS - ORIENTATION AND DENSE MATCHING ISSUES , 2014 .

[26]  W. Ostrowski ACCURACY OF MEASUREMENTS IN OBLIQUE AERIAL IMAGES FOR URBAN ENVIRONMENT , 2016 .

[27]  Toby N. Tonkin,et al.  Reconstruction of former glacier surface topography from archive oblique aerial images , 2017 .

[28]  A. Vetrivel,et al.  Identification of damage in buildings based on gaps in 3D point clouds from very high resolution oblique airborne images , 2015 .

[29]  M. Smith A PHOTOGRAMMETRIC SYSTEM FOR ARCHAEOLOGICAL MAPPING USING OBLIQUE NON‐METRIC PHOTOGRAPHY , 2006 .

[30]  Frank Vermeulen,et al.  Mapping by matching: a computer vision-based approach to fast and accurate georeferencing of archaeological aerial photographs , 2012 .

[31]  F. Nex,et al.  Accurate Estimation Of Orientation Parameters Of Uav Images Through Image Registration With Aerial Oblique Imagery , 2017 .

[32]  Joachim Höhle Photogrammetric Measurements in Oblique Aerial Images , 2008 .

[33]  Fabio Remondino,et al.  Aerial multi-camera systems: Accuracy and block triangulation issues , 2015 .

[34]  M. Gerke Using horizontal and vertical building structure to constrain indirect sensor orientation , 2011 .

[35]  Ming Zhang Towards 3D Matching of Point Clouds Derived from Oblique and Nadir Airborne Imagery , 2014 .

[36]  Lei Yan,et al.  RBA: Reduced Bundle Adjustment for oblique aerial photogrammetry , 2016 .

[37]  K. Jacobsen Geometry of vertical and oblique image combinations , 2008 .

[38]  Fabio Remondino,et al.  AUTOMATIC ORIENTATION OF LARGE BLOCKS OF OBLIQUE IMAGES , 2013 .

[39]  E. A. Babcock Detection of Active Faulting Using Oblique Infrared Aerial Photography in the Imperial Valley, California , 1971 .

[40]  J. Moré,et al.  ORIENTATION STRATEGIES FOR AERIAL OBLIQUE IMAGES , 2012 .

[41]  Pierre Grussenmeyer,et al.  Oblique Aerial Photography Tool for Building Inspection and Damage Assessment , 2014 .

[42]  R. Chellappa,et al.  Automatic registration of oblique aerial images , 1994, Proceedings of 1st International Conference on Image Processing.

[43]  Mi Wang,et al.  Automatic Texture Acquisition for 3D Model Using Oblique Aerial Images , 2008, 2008 First International Conference on Intelligent Networks and Intelligent Systems.

[44]  M. Frioud,et al.  SAR AND OBLIQUE AERIAL OPTICAL IMAGE FUSION FOR URBAN AREA IMAGESEGMENTATION , 2017 .

[45]  D. Frommholz,et al.  EXTRACTING SEMANTICALLY ANNOTATED 3D BUILDING MODELS WITH TEXTURES FROM OBLIQUE AERIAL IMAGERY , 2015 .

[46]  Linfu Xie,et al.  An asymmetric re-weighting method for the precision combined bundle adjustment of aerial oblique images , 2016 .

[47]  Lu Wang,et al.  A robust approach for automatic registration of aerial images with untextured aerial LiDAR data , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Carl Salvaggio,et al.  Automatic georeferencing of imagery from high-resolution, low-altitude, low-cost aerial platforms , 2014, Defense + Security Symposium.

[49]  Shubi Zhang,et al.  Robust and Precise Registration of Oblique Images Based on Scale-Invariant Feature Transformation Algorithm , 2012, IEEE Geosci. Remote. Sens. Lett..

[50]  Ramakant Nevatia,et al.  3-D descriptions of buildings from an oblique view aerial image , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[51]  Jian Li,et al.  Robust and rapid matching of oblique UAV images of urban area , 2013, Other Conferences.

[52]  Mario A. Gomarasca,et al.  Elements of Photogrammetry , 2009 .

[53]  Zhenfeng Shao,et al.  Calibration of digital camera integration accuracy for low-cost oblique aerial photogrammetry , 2015, Geo spatial Inf. Sci..

[54]  V. Casella,et al.  MODELLING STEEP SURFACES BY VARIOUS CONFIGURATIONS OF NADIR AND OBLIQUE PHOTOGRAMMETRY , 2016 .

[55]  Carl Salvaggio,et al.  AUTOMATIC TIE-POINT GENERATION FOR OBLIQUE AERIAL IMAGERY: AN ALGORITHM , 2004 .

[56]  Juha Hyyppä,et al.  Determination of Exterior Orientation Using Linear Features from Vector Maps , 2006 .

[57]  A. Pedersen,et al.  Point clouds from oblique stereo-imagery: Two outcrop case studies across scales and accessibility , 2015 .

[58]  F. Prandia,et al.  LIDAR AND PICTOMETRY IMAGES INTEGRATED USE FOR 3 D MODEL GENERATION , 2008 .

[59]  Antonia Teresa Spano,et al.  UAV PHOTOGRAMMETRY WITH OBLIQUE IMAGES: FIRST ANALYSIS ON DATA ACQUISITION AND PROCESSING , 2016 .

[60]  A. Vetrivel,et al.  Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high resolution oblique aerial images, and multiple-kernel-learning , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[61]  A. Poznańska,et al.  Inlining 3D Reconstruction, Multi-Source Texture Mapping and Semantic Analysis using Oblique Aerial Imagery , 2016 .

[62]  Metric Exploitation of a Single Low Oblique Aerial Image , 2015 .

[63]  Jiann-Yeou Rau,et al.  Photo-realistic 3D mapping from aerial oblique imagery , 2010 .

[64]  Martin Kappas,et al.  Digital multi-image photogrammetry combined with oblique aerial photography enables glacier monitoring survey flights below clouds in Alaska , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[65]  J. Hannavy Encyclopedia of nineteenth-century photography , 2013 .

[66]  Fei Deng,et al.  AUTOMATIC TEXTURE RECONSTRUCTION OF 3D CITY MODEL FROM OBLIQUE IMAGES , 2016 .

[67]  Fabio Remondino,et al.  CHANGING THE PRODUCTION PIPELINE - USE OF OBLIQUE AERIAL CAMERAS FOR MAPPING PURPOSES , 2016 .

[68]  Massimiliano Molinari,et al.  Vertical Measurements in Oblique Aerial Imagery , 2014, ISPRS Int. J. Geo Inf..

[69]  Jefferey A. Shufelt,et al.  Performance Evaluation and Analysis of Monocular Building Extraction From Aerial Imagery , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  ABRAHAM THOMAS,et al.  Aerial Photography , 1919, Nature.

[71]  Li Zhang,et al.  Robust perspective invariant quasidense matching across large oblique images , 2014, J. Electronic Imaging.

[72]  Yunjun Yao,et al.  Use of UAV oblique imaging for the detection of individual trees in residential environments , 2015 .

[73]  Juha Hyyppä,et al.  ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences , 2016 .

[74]  J. Chandler,et al.  Minimising systematic error surfaces in digital elevation models using oblique convergent imagery , 2011 .

[75]  A. P. Nyaruhuma,et al.  Incorporating scene constraints into the triangulation of airborne oblique images , 2009 .

[76]  Norbert Pfeifer,et al.  ORIENTAL - AUTOMATIC GEO-REFERENCING AND ORTHO-RECTIFICATION OF ARCHAEOLOGICAL AERIAL PHOTOGRAPHS , 2013 .

[77]  Yang Huachao,et al.  Robust and Precise Registration of Oblique Images Based on Scale-Invariant Feature Transformation Algorithm , 2012, IEEE Geoscience and Remote Sensing Letters.

[78]  Martin Fera,et al.  Automated Archiving of Archaeological Aerial Images , 2016, Remote. Sens..

[79]  Ahmed Mohamed Hamruni The use of oblique and vertical images for 3D urban modelling , 2010 .

[80]  Frédéric Champagnat,et al.  Dense height map estimation from oblique aerial image sequences , 2008, Comput. Vis. Image Underst..

[81]  Jinghong Liu,et al.  Automatic orthorectification and mosaicking of oblique images from a zoom lens aerial camera , 2015 .

[82]  J. Höhle OBLIQUE AERIAL IMAGES AND THEIR USE IN CULTURAL HERITAGE DOCUMENTATION , 2013 .

[83]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[84]  L. Nicholson,et al.  Using structure‐from‐motion to create glacier DEMs and orthoimagery from historical terrestrial and oblique aerial imagery , 2017 .

[85]  Brent A. Archinal,et al.  PHOTOGRAMMETRIC PROCESSING OF APOLLO 15 METRIC CAMERA OBLIQUE IMAGES , 2016 .

[86]  Jing Xiao,et al.  AUTOMATIC BUILDING OUTLINING FROM MULTI-VIEW OBLIQUE IMAGES , 2012 .

[87]  Adib Tayeb Madani,et al.  DUBAI 3D TEXTUERD MESH USING HIGH QUALITY RESOLUTION VERTICAL/OBLIQUE AERIAL IMAGERY , 2016 .

[88]  Jiann-Yeou Rau,et al.  Facade detection in oblique aerial image using object based image anaylysis , 2012 .

[89]  S. Ji,et al.  ADAPTIVE HIERARCHICAL DENSE MATCHING OF MULTI-VIEW AIRBORNE OBLIQUE IMAGERY , 2015 .

[90]  U. Stilla,et al.  TEXTURE MAPPING OF 3 D BUILDING MODELS WITH OBLIQUE DIRECT GEO-REFERENCED AIRBORNE IR IMAGE SEQUENCES , 2009 .

[91]  M. Gerkeb SUB-CAMERA CALIBRATION OF A PENTA-CAMERA , 2016 .

[92]  Avideh Zakhor,et al.  Automatic registration of aerial imagery with untextured 3D LiDAR models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[93]  Fabio Remondino,et al.  Building Footprints Extraction from Oblique Imagery , 2013 .

[94]  Rog Palmer A computer method for transcribing information graphically from oblique aerial photographs to maps , 1977 .

[95]  Norman Kerle,et al.  Disasters : risk assessment, management, and post - disaster studies using remote sensing , 2015 .

[96]  C. Lemmen,et al.  Pictometry : potentials for land administration , 2007 .

[97]  Mathias Rothermel,et al.  Extracting 3D urban models from oblique aerial images , 2015, 2015 Joint Urban Remote Sensing Event (JURSE).

[98]  George Vosselman,et al.  VERIFICATION OF 3D BUILDING MODELS USING MUTUAL INFORMATION IN AIRBORNE OBLIQUE IMAGES , 2012 .

[99]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[100]  Jiann-Yeou Rau,et al.  Analysis of Oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[101]  Christian Früh,et al.  Automated texture mapping of 3D city models with oblique aerial imagery , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[102]  Jing Xiao,et al.  Oblique airborne photogrammetry : automatic building detection and verification , 2014 .

[103]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[104]  G. Bareth,et al.  Evaluating Dense 3d Reconstruction Software Packages for Oblique Monitoring of Crop Canopy Surface , 2016 .

[105]  A. P. Nyaruhuma Automatic verification of buildings using oblique airborne images , 2013 .

[106]  Jing Xiao,et al.  Automatic building detection using oblique imagery , 2013 .

[107]  H. Sui,et al.  Detecting building façade damage from oblique aerial images using local symmetry feature and the Gini Index , 2017 .

[108]  M. Doneus Precision mapping and interpretation of oblique aerial photographs , 2001 .

[109]  Markus Gerke,et al.  Supervised and unsupervised MRF based 3D scene classification in multiple view airborne oblique images , 2013 .

[110]  A. Vetrivel,et al.  POTENTIAL OF MULTI-TEMPORAL OBLIQUE AIRBORNE IMAGERY FOR STRUCTURAL DAMAGE ASSESSMENT , 2016 .

[111]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[112]  Frederick J. Doyle,et al.  Photographic systems for Apollo , 1970 .

[113]  M. Rothermel,et al.  Benchmarking High Density Image Matching for Oblique Airborne Imagery , 2014 .

[114]  S. Verykokou,et al.  EXTERIOR ORIENTATION ESTIMATION OF OBLIQUE AERIAL IMAGERY USING VANISHING POINTS , 2016 .

[115]  M. Gerke Dense matching in high resolution oblique airborne images , 2009 .

[116]  Parvaneh Saeedi,et al.  Two-Edge-Corner Image Features for Registration of Geospatial Images with Large View Variations , 2014 .

[117]  Uwe Stilla,et al.  QUALITY ASSESSMENT OF BUILDING TEXTURES EXTRACTED FROM OBLIQUE AIRBORNE THERMAL IMAGERY , 2016 .

[118]  Guler Yalcin,et al.  3D City Modelling with Oblique Photogrammetry Method , 2015 .

[119]  Qing Zhu,et al.  Stable least-squares matching for oblique images using bound constrained optimization and a robust loss function , 2016 .

[120]  Fabio Remondino,et al.  Oblique aerial imagery : a review , 2015 .

[121]  Z. Du,et al.  Reliable Spatial Relationship Constrained Feature Point Matching of Oblique Aerial Images , 2015 .

[122]  Yi Lin,et al.  Attempt of UAV oblique images and MLS point clouds for 4D modelling of roadside pole-like objects , 2014, Asia-Pacific Environmental Remote Sensing.

[123]  Eyal Ofek,et al.  Validation of vector data using oblique images , 2008, GIS '08.

[124]  A STUDY ON OBLIQUE CAMERA SYSTEMS CALIBRATION AND THE STABILITY OF THEIR PARAMETERS , 2016 .

[125]  K. Bakuła,et al.  TOWARDS EFFICIENCY OF OBLIQUE IMAGES ORIENTATION , 2016 .

[126]  Arko Lucieer,et al.  Assessing the Accuracy of Georeferenced Point Clouds Produced via Multi-View Stereopsis from Unmanned Aerial Vehicle (UAV) Imagery , 2012, Remote. Sens..

[127]  Dieter Fritsch,et al.  Towards All-in-one Photogrammetry , 2012 .

[128]  Maoteng Zheng,et al.  Camera Pose Determination and 3-D Measurement From Monocular Oblique Images With Horizontal Right Angle Constraints , 2014, IEEE Geoscience and Remote Sensing Letters.

[129]  An analysis of snow cover patterns as derived from oblique aerial photographs , 1991 .

[130]  Bing Zhang,et al.  Direct georeferencing of oblique and vertical imagery in different coordinate systems , 2014 .

[131]  Mauricio Galo,et al.  Generating Virtual Images from Oblique Frames , 2013, Remote. Sens..

[132]  Markus Gerke,et al.  FITTING OF PARAMETRIC BUILDING MODELS TO OBLIQUE AERIAL IMAGES , 2012 .

[133]  Jianqing Zhang,et al.  The feasibility study on the aerial triangulation over urban area constrained by vertical lines , 2005, International Symposium on Multispectral Image Processing and Pattern Recognition.

[134]  G. Kemper,et al.  CALIBRATION PROCEDURES ON OBLIQUE CAMERA SETUPS , 2016 .

[135]  M. Gerke,et al.  Automatic Structural Seismic Damage Assessment with Airborne Oblique Pictometry© Imagery , 2011 .

[136]  George Vosselman,et al.  AUTOMATIC DETECTION OF BUILDINGS WITH RECTANGULAR FLAT ROOFS FROM MULTI-VIEW OBLIQUE IMAGERY , 2010 .