Autocalibrating vision guided navigation of unmanned air vehicles via tactical monocular cameras in GPS denied environments

This thesis presents a novel robotic navigation strategy by using a conventional tactical monocular camera, proving the feasibility of using a monocular camera as the sole proximity sensing, object avoidance, mapping, and path-planning mechanism to fly and navigate small to medium scale unmanned rotary-wing aircraft in an autonomous manner. The range measurement strategy is scalable, self-calibrating, indoor-outdoor capable, and has been biologically inspired by the key adaptive mechanisms for depth perception and pattern recognition found in humans and intelligent animals (particularly bats), designed to assume operations in previously unknown, GPS-denied environments. It proposes novel electronics, aircraft, aircraft systems, systems, and procedures and algorithms that come together to form airborne systems which measure absolute ranges from a monocular camera via passive photometry, mimicking that of a human-pilot like judgement. The research is intended to bridge the gap between practical GPS coverage and precision localization and mapping problem in a small aircraft. In the context of this study, several robotic platforms, airborne and ground alike, have been developed, some of which have been integrated in real-life field trials, for experimental validation. Albeit the emphasis on miniature robotic aircraft this research has been tested and found compatible with tactical vests and helmets, and it can be used to augment the reliability of many other types of proximity sensors.

[1]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Anup Basu,et al.  Active calibration of cameras: theory and implementation , 1995, IEEE Trans. Syst. Man Cybern..

[3]  Brigitte d'Andréa-Novel,et al.  VISION GUIDED BY VEHICLE DYNAMICS FOR ONBOARD ESTIMATION OF THE VISIBILITY RANGE , 2007 .

[4]  H Farid,et al.  Blind removal of lens distortion. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Eduardo Mario Nebot,et al.  Optimization of the simultaneous localization and map-building algorithm for real-time implementation , 2001, IEEE Trans. Robotics Autom..

[6]  Rüdiger Dillmann,et al.  Sequential 3D-SLAM for mobile action planning , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[7]  Kimon P. Valavanis,et al.  Advances in Unmanned Aerial Vehicles: State of the Art and the Road to Autonomy , 2007 .

[8]  Wolfram Burgard,et al.  Monte Carlo localization for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[9]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[10]  Michael Bosse,et al.  An Atlas framework for scalable mapping , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[11]  Nicholas Ayache,et al.  Artificial vision for mobile robots - stereo vision and multisensory perception , 1991 .

[12]  Stéphane Viollet,et al.  Bio-inspired optical flow circuits for the visual guidance of micro air vehicles , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[13]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[14]  Wolfram Burgard,et al.  Using the CONDENSATION algorithm for robust, vision-based mobile robot localization , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  Ashutosh Saxena,et al.  Depth Estimation Using Monocular and Stereo Cues , 2007, IJCAI.

[16]  K. Kinzler,et al.  Perceiving distance accurately by a directional process of integrating ground information , 2022 .

[17]  Bin Yang,et al.  Automatic extrinsic camera self-calibration based on homography and epipolar geometry , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[18]  Tom Drummond,et al.  Scalable Monocular SLAM , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[19]  Mark Crovella,et al.  Hyperbolic Embedding and Routing for Dynamic Graphs , 2009, IEEE INFOCOM 2009.

[20]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[21]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[22]  Michael S. Langer,et al.  Optical snow and the aperture problem , 2002, Object recognition supported by user interaction for service robots.

[23]  J. Ibanez-Guzman,et al.  Map aided SLAM in neighbourhood environments , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[24]  Joachim Hertzberg,et al.  Heuristic-Based Laser Scan Matching for Outdoor 6D SLAM , 2005, KI.

[25]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[26]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[27]  Sebastian Thrun,et al.  A Multi-Resolution Pyramid for Outdoor Robot Terrain Perception , 2004, AAAI.

[28]  George T Schmidt,et al.  INS/GPS Integration Architectures , 2010 .

[29]  Darius Burschka,et al.  V-GPS(SLAM): vision-based inertial system for mobile robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[30]  Takeo Kanade,et al.  A visual odometer for autonomous helicopter flight , 1999, Robotics Auton. Syst..

[31]  Evangelos E. Milios,et al.  Globally Consistent Range Scan Alignment for Environment Mapping , 1997, Auton. Robots.

[32]  Udo Frese,et al.  A Discussion of Simultaneous Localization and Mapping , 2006, Auton. Robots.

[33]  P. Newman,et al.  Using Naturally Salient Regions for SLAM with 3 D Laser Data ∗ , 2005 .

[34]  Salah Sukkarieh,et al.  Airborne simultaneous localisation and map building , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[35]  Philippe Gaussier,et al.  Navigation and Planning in an Unknown Environment Using Vision and a Cognitive Map , 2006, EUROS.

[36]  Jarmo Takala,et al.  Application of particle filters for indoor positioning using floor plans , 2010, 2010 Ubiquitous Positioning Indoor Navigation and Location Based Service.

[37]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[38]  Henrik I. Christensen,et al.  2D mapping of cluttered indoor environments by means of 3D perception , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[39]  Sebastian Thrun,et al.  Spontaneous, short-term interaction with mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[40]  Ferdinand van der Heijden,et al.  Better features to track by estimating the tracking convergence region , 2002, Object recognition supported by user interaction for service robots.

[41]  Dushyant Rao,et al.  Monocular Vision based Navigation in GPS-Denied Riverine Environments , 2011 .

[42]  Kazuo Ishii,et al.  Depth Perception Using a Monocular Vision System , 2008, ICONIP.

[43]  A Hughes,et al.  A quantitative analysis of the cat retinal ganglion cell topography , 1975, The Journal of comparative neurology.

[44]  Roland Siegwart,et al.  Orthogonal 3D-SLAM for Indoor Environments Using Right Angle Corners , 2007, EMCR.

[45]  Hetal M. Patel,et al.  Panoramic Image Mosaicing , 2013 .

[46]  E. Zrenner,et al.  The spectral sensitivity of dark- and light-adapted cat retinal ganglion cells , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[48]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Roland Siegwart,et al.  3D SLAM using planar segments , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[51]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[52]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..

[53]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[54]  Simon Lacroix,et al.  High resolution terrain mapping using low attitude aerial stereo imagery , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[55]  Johan Eklöf,et al.  Vision in echolocating bats , 2003 .

[56]  Andrew W. Fitzgibbon,et al.  Automatic Camera Recovery for Closed or Open Image Sequences , 1998, ECCV.

[57]  Sebastian Thrun,et al.  Online self-calibration for mobile robots , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[58]  Wolfram Burgard,et al.  Predictive Simulation of Autonomous Robots for Reliable Visualization over the Internet , 1998 .

[59]  Se-Young Oh,et al.  SLAM in Indoor Environments using Omni-directional Vertical and Horizontal Line Features , 2008, J. Intell. Robotic Syst..

[60]  H. Hatze,et al.  High-precision three-dimensional photogrammetric calibration and object space reconstruction using a modified DLT-approach. , 1988, Journal of biomechanics.

[61]  Eric Foxlin,et al.  Generalized architecture for simultaneous localization, auto-calibration, and map-building , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[62]  Walterio W. Mayol-Cuevas,et al.  Discovering Higher Level Structure in Visual SLAM , 2008, IEEE Transactions on Robotics.

[63]  Paul A. Beardsley,et al.  Active visual navigation using non-metric structure , 1995, Proceedings of IEEE International Conference on Computer Vision.

[64]  Michael Bosse,et al.  Vanishing points and 3D lines from omnidirectional video , 2002, Proceedings. International Conference on Image Processing.

[65]  Christopher G. Harris,et al.  3D positional integration from image sequences , 1988, Image Vis. Comput..

[66]  James J. Kuffner,et al.  Planning 3-D Path Networks in Unstructured Environments , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[67]  Ilkay Yavrucuk,et al.  Collaborative Target Tracking for Swarming MAVs Using Potential Fields and Panel Methods , 2008 .

[68]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[69]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[70]  Xinyu Cheng,et al.  The simple camera calibration approach based on a triangle and depth estimation from monocular vision , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[72]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.