Vision-based teleoperation of unmanned aerial and ground vehicles

We present a novel vision-based teleoperation control framework for a team of an unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV). Our control law allows a remote human user to teleoperate the team with some useful haptic feedback, while also ensuring the UAV-UGV coordination via the camera installed on the UAV (and seeing the UGV) and the velocity limitation of the UGV. For this, we first elucidate a geometric condition for the UAV and UGV velocities to ensure the UAV-UGV coordination by driving the image feature of the UGV to converge to a desired one on the image surface, while also guaranteeing that the UGV's velocity, which is often much slower than that of the UAV, is under a certain specified bound. The UAV is then tele-controlled to track the teleoperation command as close as possible, yet, only to the extent permissible by this geometric condition. Simulation is performed to illustrate the theory.

[1]  Andreas Zell,et al.  Automatic Take Off, Tracking and Landing of a Miniature UAV on a Moving Carrier Vehicle , 2011, J. Intell. Robotic Syst..

[2]  Frank L. Lewis,et al.  Control of a nonholomic mobile robot: Backstepping kinematics into dynamics , 1997 .

[3]  Dongjun Lee,et al.  Feedback r-passivity of Lagrangian systems for mobile robot teleoperation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Kolja Kühnlenz,et al.  A vision-guided autonomous quadrotor in an air-ground multi-robot system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[5]  Éric Marchand,et al.  Chasing a moving target from a flying UAV , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Vijay Kumar,et al.  Visual servoing of a UGV from a UAV using differential flatness , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[7]  Robert E. Mahony,et al.  Visual servoing of an under-actuated dynamic rigid-body system: an image-based approach , 2002, IEEE Trans. Robotics Autom..

[8]  Dongjun Lee,et al.  Backstepping Control of Quadrotor-Type UAVs and Its Application to Teleoperation over the Internet , 2012, IAS.

[9]  Dongjun Lee,et al.  Passive-Set-Position-Modulation Framework for Interactive Robotic Systems , 2010, IEEE Transactions on Robotics.

[10]  Robert E. Mahony,et al.  A Practical Visual Servo Control for an Unmanned Aerial Vehicle , 2008, IEEE Transactions on Robotics.

[11]  Antonio Franchi,et al.  Haptic teleoperation of multiple unmanned aerial vehicles over the internet , 2011, 2011 IEEE International Conference on Robotics and Automation.

[12]  Peter I. Corke,et al.  A novel approach to haptic tele-operation of aerial robot vehicles , 2010, 2010 IEEE International Conference on Robotics and Automation.

[13]  Robert E. Mahony,et al.  Image-Based Visual Servo Control of the Translation Kinematics of a Quadrotor Aerial Vehicle , 2009, IEEE Transactions on Robotics.

[14]  Frank L. Lewis,et al.  Control of a nonholonomic mobile robot: backstepping kinematics into dynamics , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[15]  Robert E. Mahony,et al.  Landing a VTOL Unmanned Aerial Vehicle on a Moving Platform Using Optical Flow , 2012, IEEE Transactions on Robotics.

[16]  Jonathan P. How,et al.  Cooperative Vision Based Estimation and Tracking Using Multiple UAVs , 2007 .

[17]  Camillo J. Taylor,et al.  Control of a Quadrotor Helicopter Using Dual Camera Visual Feedback , 2005, Int. J. Robotics Res..

[18]  Eric Johnson,et al.  Augmenting UAV autonomy , 2006, IEEE Robotics & Automation Magazine.

[19]  Alborz Geramifard,et al.  On the Design and Use of a Micro Air Vehicle to Track and Avoid Adversaries , 2010, Int. J. Robotics Res..

[20]  Tarek Hamel,et al.  A Control Approach for Thrust-Propelled Underactuated Vehicles and its Application to VTOL Drones , 2009, IEEE Transactions on Automatic Control.

[21]  François Chaumette,et al.  Dynamic visual servoing with image moments for a quadrotor using a virtual spring approach , 2011, 2011 IEEE International Conference on Robotics and Automation.

[22]  Rogelio Lozano,et al.  Real-Time Stabilization of an Eight-Rotor UAV Using Optical Flow , 2009, IEEE Transactions on Robotics.

[23]  Soon-Jo Chung,et al.  Monocular vision SLAM for indoor aerial vehicles , 2009, IROS.