A Robust Real-Time Embedded Vision System on an Unmanned Rotorcraft for Ground Target Following

In this paper, we present the systematic design and implementation of a robust real-time embedded vision system for an unmanned rotorcraft for ground target following. The hardware construction of the vision system is presented, and an onboard software system is developed based on a multithread technique capable of coordinating multiple tasks. To realize the autonomous ground target following, a sophisticated feature-based vision algorithm is proposed by using an onboard color camera and navigation sensors. The vision feedback is integrated with the flight control system to guide the unmanned rotorcraft to follow a ground target in flight. The overall vision system has been tested in actual flight missions, and the results obtained show that the overall system is very robust and efficient.

[1]  Nikolaos Papanikolopoulos,et al.  Robust target detection and tracking through integration of motion, color, and geometry , 2006, Comput. Vis. Image Underst..

[2]  Daniel P. Huttenlocher,et al.  Adaptive Bayesian recognition in tracking rigid objects , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  De Xu,et al.  Ceiling-Based Visual Positioning for an Indoor Mobile Robot With Monocular Vision , 2009, IEEE Transactions on Industrial Electronics.

[4]  Nelson R. Corby.  Machine Vision for Robotics , 1983, IEEE Transactions on Industrial Electronics.

[5]  Zdenek Hanzálek,et al.  Low-Cost Reconfigurable Control System for Small UAVs , 2011, IEEE Transactions on Industrial Electronics.

[6]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[7]  LI X.RONG,et al.  Survey of maneuvering target tracking. Part I. Dynamic models , 2003 .

[8]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[9]  S. Sukkarieh,et al.  SLAM aided GPS/INS Navigation in GPS Denied and Unknown Environments , 2004 .

[10]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Gaurav S. Sukhatme,et al.  Combined optic-flow and stereo-based navigation of urban canyons for a UAV , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Takeo Kanade,et al.  Vision-Based Autonomous Helicopter Research at Carnegie Mellon Robotics Institute 1991-1997 , 1998 .

[13]  Jake K. Aggarwal,et al.  Object tracking in an outdoor environment using fusion of features and cameras , 2006, Image Vis. Comput..

[14]  Fatih Murat Porikli,et al.  Achieving real-time object detection and tracking under extreme conditions , 2006, Journal of Real-Time Image Processing.

[15]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  François Chaumette,et al.  Visual servo control. II. Advanced approaches [Tutorial] , 2007, IEEE Robotics & Automation Magazine.

[17]  T. Hamel,et al.  A practical Visual Servo Control for a Unmanned Aerial Vehicle , 2008, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[18]  Sadayuki Tsugawa,et al.  Vision-based vehicles in Japan: machine vision systems and driving control systems , 1994, IEEE Trans. Ind. Electron..

[19]  Mark E. Campbell,et al.  Cooperative Tracking Using Vision Measurements on SeaScan UAVs , 2007, IEEE Transactions on Control Systems Technology.

[20]  Bahram Javidi,et al.  Physics of Automatic Target Recognition , 2007 .

[21]  Gene F. Franklin,et al.  Feedback Control of Dynamic Systems , 1986 .

[22]  Firooz A. Sadjadi,et al.  Theory of Invariant Algebra and Its Use in Automatic Target Recognition , 2007 .

[23]  Long Wang,et al.  Vision-Based Target Tracking and Collision Avoidance for Two Autonomous Robotic Fish , 2009, IEEE Transactions on Industrial Electronics.

[24]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[25]  Eric N. Johnson,et al.  Real-Time Vision-Based Relative Aircraft Navigation , 2007, J. Aerosp. Comput. Inf. Commun..

[26]  Fredrik Danielsson,et al.  Virtual Machine Vision in Computer Aided Robotics , 2009, 2009 IEEE Conference on Emerging Technologies & Factory Automation.

[27]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[28]  Gaurav S. Sukhatme,et al.  Visual servoing of an autonomous helicopter in urban areas using feature tracking , 2006, J. Field Robotics.

[29]  Eric Johnson,et al.  Augmenting UAV autonomy , 2006, IEEE Robotics & Automation Magazine.

[30]  Carlo Tomasi,et al.  Comparison of approaches to egomotion computation , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Sylvie Treuillet,et al.  A biprocessor-oriented vision-based target tracking system , 2002, IEEE Trans. Ind. Electron..

[32]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[33]  S. Hutchinson,et al.  Visual Servo Control Part II : Advanced Approaches , 2007 .

[34]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Aníbal Ollero,et al.  Reconfigurable Control Architecture for Distributed Systems in the HERO Autonomous Helicopter , 2011, IEEE Transactions on Industrial Electronics.

[36]  Seth Hutchinson,et al.  Visual Servo Control Part I: Basic Approaches , 2006 .

[37]  S. Sastry,et al.  Vision based terrain recovery for landing unmanned aerial vehicles , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[38]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Bruce Enderle.  Commercial Applications of UAV's in Japanese Agriculture , 2002 .

[40]  Yuan F. Zheng,et al.  Passive and active stereo vision for smooth surface detection of deformed plates , 1995, IEEE Trans. Ind. Electron..

[41]  Tong Heng Lee,et al.  Development of a vision-based ground target detection and tracking system for a small unmanned helicopter , 2009, Science in China Series F: Information Sciences.

[42]  Tong Heng Lee,et al.  Modeling and Control of the Yaw Channel of a UAV Helicopter , 2008, IEEE Transactions on Industrial Electronics.