3D reconstruction of disaster scenes for urban search and rescue

Natural and man-made disasters that may take place due to a catastrophic incident (e.g., earthquake, explosion, terrorist attack) often result in trapped humans under rubble piles. In such emergency response situations, Urban Search and Rescue (USaR) teams have to make quick decisions under stress in order to determine the location of possible trapped victims. Fast 3D modelling of fully or partially collapsed buildings using images from Unmanned Aerial Vehicles (UAVs) can considerably help USaR efforts, thus improving disaster response and increasing survival rates. The a-priori establishment of a proper workflow for fast and reliable image-based 3D modelling and the a priori determination of the parameters that have to be set in each step of the photogrammetric pipeline are critical aspects that ensure the readiness in an emergency response situation. This paper evaluates powerful commercial and open-source software solutions for the 3D reconstruction of disaster scenes for rapid response situations. The software packages are tested using UAV datasets of a real earthquake scene. A thorough analysis on the parameters of the various modelling steps that may lead to desired results for USaR tasks is made and indicative processing chains are proposed, taking into account the restriction of time. Furthermore, some weaknesses of the data acquisition process that have been detected by performing the experiments are outlined and some improvements and additions are proposed, including an initial preprocessing of the images using a graph-based approach.

[1]  Jan-Michael Frahm,et al.  Modeling and Recognition of Landmark Image Collections Using Iconic Scene Graphs , 2008, International Journal of Computer Vision.

[2]  Jun Zhong,et al.  Towards unsupervised physical activity recognition using smartphone accelerometers , 2016, Multimedia Tools and Applications.

[3]  Dietrich Paulus,et al.  Real-time 3D mapping of rough terrain: A field report from Disaster City , 2010, 2010 IEEE Safety Security and Rescue Robotics.

[4]  Li Liu,et al.  Recognizing Complex Activities by a Probabilistic Interval-Based Model , 2016, AAAI.

[5]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[6]  Metric Exploitation of a Single Low Oblique Aerial Image , 2015 .

[7]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[8]  Fumio Yamazaki,et al.  Construction of 3D models of buildings damaged by earthquakes using UAV aerial images , 2015 .

[9]  Erica Nocerino,et al.  Dense image matching: Comparisons and analyses , 2013, 2013 Digital Heritage International Congress (DigitalHeritage).

[10]  F. Nex,et al.  UAV for 3D mapping applications: a review , 2014 .

[11]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Daniel D. Morris,et al.  Factorization methods for structure from motion , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[13]  Angelos Amditis,et al.  UAV-based 3D modelling of disaster scenes for Urban Search and Rescue , 2016, 2016 IEEE International Conference on Imaging Systems and Techniques (IST).

[14]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[15]  A. Vetrivel,et al.  Identification of damage in buildings based on gaps in 3D point clouds from very high resolution oblique airborne images , 2015 .

[16]  Roberto Cipolla,et al.  Structure from motion , 2008 .

[17]  Luming Zhang,et al.  Action2Activity: Recognizing Complex Activities from Sensor Data , 2015, IJCAI.

[18]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[19]  Richard Szeliski,et al.  Skeletal graphs for efficient structure from motion , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Alexander Ferworn,et al.  3D Disaster Scene Reconstruction Using a Canine-Mounted RGB-D Sensor , 2013, 2013 International Conference on Computer and Robot Vision.

[21]  T. Fuse,et al.  DEVELOPMENT OF IMAGE SELECTION METHOD USING GRAPH CUTS , 2016 .

[22]  Hongbin Zha,et al.  Tracking Generic Human Motion via Fusion of Low- and High-Dimensional Approaches , 2013, IEEE Trans. Syst. Man Cybern. Syst..

[23]  S. Verykokou,et al.  EXTERIOR ORIENTATION ESTIMATION OF OBLIQUE AERIAL IMAGERY USING VANISHING POINTS , 2016 .

[24]  O. D. Faugeras,et al.  Camera Self-Calibration: Theory and Experiments , 1992, ECCV.

[25]  Filiberto Chiabrando,et al.  UAV Deployment Exercise for Mapping Purposes: Evaluation of Emergency Response Applications , 2015, Sensors.

[26]  G. Lewis,et al.  Evaluating the Use of a Low-Cost Unmanned Aerial Vehicle Platform in Acquiring Digital Imagery for Emergency Response , 2007 .

[27]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[28]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[29]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[31]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[32]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  George Athanasiou,et al.  Multi-scale 3D Modelling of Damaged Cultural Sites: Use Cases and Image-Based Workflows , 2016, EuroMed.

[34]  Wesley M. DeBusk.  Unmanned Aerial Vehicle Systems for Disaster Relief: Tornado Alley , 2009 .

[35]  Derek J. Walvoord,et al.  Assessing geoaccuracy of structure from motion point clouds from long-range image collections , 2014 .

[36]  Norman Kerle,et al.  UAV-based urban structural damage assessment using object-based image analysis and semantic reasoning , 2014 .

[37]  Bernhard Rinner,et al.  Fast aerial image acquisition and mosaicking for emergency response operations by collaborative UAVs , 2011, ISCRAM.

[38]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[39]  Alexander Ferworn,et al.  Disaster scene reconstruction: modeling and simulating urban building collapse rubble within a game engine , 2013, SummerSim.

[40]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  A. P. Nyaruhuma,et al.  Incorporating scene constraints into the triangulation of airborne oblique images , 2009 .

[42]  R. Fabio.  From point cloud to surface the modeling and visualization problem , 2003 .

[43]  Alexander Ferworn,et al.  Initial experiments on 3D modeling of complex disaster environments using unmanned aerial vehicles , 2011, 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics.

[44]  Stefan K. Gehrig,et al.  A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching , 2009, ICVS.

[45]  H. Hirschmüller.  Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[46]  S. Verykokou,et al.  Automatic Rough Georeferencing of Multiview Oblique and Vertical Aerial Image Datasets of Urban Scenes , 2016 .

[47]  Yongjun Zhang,et al.  Photogrammetry for First Response in Wenchuan Earthquake , 2009 .

[48]  Gauthier Lafruit,et al.  Cross-Based Local Stereo Matching Using Orthogonal Integral Images , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  George Athanasiou,et al.  INACHUS : Integrated wide area situation awarenesss and survivor localization in search and rescue operations , 2015 .

[50]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[51]  Carol J. Friedland,et al.  A SURVEY OF UNMANNED AERIAL VEHICLE ( UAV ) USAGE FOR IMAGERY , 2011 .

[52]  Hongbin Zha,et al.  Fusion of low-and high-dimensional approaches by trackers sampling for generic human motion tracking , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[53]  David S. Rosenblum,et al.  From action to activity: Sensor-based activity recognition , 2016, Neurocomputing.