Assessment of Structure from Motion for reconnaissance augmentation and bandwidth usage reduction

Modern militaries rely upon remote image sensors for real-time intelligence. A typical remote system consists of an unmanned aerial vehicle, or UAV, with an attached camera. A video stream is sent from the UAV, through a bandwidth-constrained satellite connection, to an intelligence processing unit. In this research, an upgrade to this remote-video-stream method of collection is proposed. A set of synthetic images of a scene captured by an UAV in a virtual environment is sent to a pipeline of computer vision algorithms, collectively known as Structure from Motion. The output of Structure from Motion, a three-dimensional (3D) model, is then assessed in a 3D virtual world as a possible replacement for the images from which it was created. This study shows Structure from Motion results from a modifiable spiral flight path and compares the geoaccuracy of each result. A flattening of height is observed, and an automated compensation for this flattening is proposed and performed. Each reconstruction is also compressed, and the size of the compression is compared with the compressed size of the images from which it was created. A reduction of 49–60% of required space, or bandwidth, is shown. A corresponding video demonstrating this technique is available online.

[1]  Paolo Cignoni,et al.  MeshLab: an Open-Source Mesh Processing Tool , 2008, Eurographics Italian Chapter Conference.

[2]  T. Franklin,et al.  An overview of the STEAMiE Educational game Engine , 2008, 2008 38th Annual Frontiers in Education Conference.

[3]  Tobias Höllerer,et al.  Theia: A Fast and Scalable Structure-from-Motion Library , 2015, ACM Multimedia.

[4]  Jean-Yves Guillemaut,et al.  Outdoor Dynamic 3D Scene Reconstruction , 2012 .

[5]  Karl C. Walli,et al.  Error characterization of flight trajectories reconstructed using Structure from Motion , 2013, 2013 IEEE Applied Imagery Pattern Recognition Workshop (AIPR).

[6]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[7]  Pierre Alliez,et al.  Eurographics Symposium on Geometry Processing (2007) Voronoi-based Variational Reconstruction of Unoriented Point Sets , 2022 .

[8]  Pascal Monasse,et al.  OpenMVG: Open Multiple View Geometry , 2016, RRPR@ICPR.

[9]  Mark A. Fonstad,et al.  Topographic structure from motion: a new development in photogrammetric measurement , 2013 .

[10]  Michael M. Kazhdan,et al.  Screened poisson surface reconstruction , 2013, TOGS.

[11]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[12]  Kevin W. Colson,et al.  Toward Automated Aerial Refueling: Relative Navigation with Structure from Motion , 2016 .

[13]  F. Nex,et al.  UAV for 3D mapping applications: a review , 2014 .

[14]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[15]  Josiah Manson,et al.  Streaming Surface Reconstruction Using Wavelets , 2008, Comput. Graph. Forum.

[16]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[17]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[19]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[20]  Rüdiger Westermann,et al.  Memory‐Efficient Interactive Online Reconstruction From Depth Image Streams , 2016, Comput. Graph. Forum.

[21]  Jean-Yves Guillemaut,et al.  Outdoor Dynamic 3-D Scene Reconstruction , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[23]  T. S. Cssi Kelso Analysis of the 2007 Chinese ASAT Test and the Impact of its Debris on the Space Environment , 2007 .

[24]  Michael Goesele,et al.  Let There Be Color! Large-Scale Texturing of 3D Reconstructions , 2014, ECCV.

[25]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[26]  Jan-Michael Frahm,et al.  Pixelwise View Selection for Unstructured Multi-View Stereo , 2016, ECCV.

[27]  Derek J. Walvoord,et al.  Assessing geoaccuracy of structure from motion point clouds from long-range image collections , 2014 .

[28]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[30]  George Pavlidis,et al.  Multi-image 3D reconstruction data evaluation , 2014 .

[31]  J. Liou Collision activities in the future orbital debris environment , 2004 .

[32]  Jared M Ekholm 3-D Scene Reconstruction from Aerial Imagery , 2012 .

[33]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[34]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[35]  J. Brasington,et al.  Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry , 2014 .

[36]  Arko Lucieer,et al.  An Automated Technique for Generating Georectified Mosaics from Ultra-High Resolution Unmanned Aerial Vehicle (UAV) Imagery, Based on Structure from Motion (SfM) Point Clouds , 2012, Remote. Sens..

[37]  Christopher Parsons,et al.  Real-Time Automated Aerial Refueling Using Stereo Vision , 2016, ISVC.

[38]  Carlos Hernandez,et al.  Multi-View Stereo: A Tutorial , 2015, Found. Trends Comput. Graph. Vis..