A reinforcement learning approach for UAV target searching and tracking

Owing to the advantages of Unmanned Aerial Vehicle (UAV), such as the extendibility, maneuverability and stability, multiple UAVs are having more and more applications in security surveillance. The object searching and trajectory planning become the important issues of uninterrupted patrol. We propose an online distributed algorithm for tracking and searching, while considering the energy refueling at the same time. The quantum probability model which describes the partially observable target positions is proposed. Moreover, the upper confidence tree algorithm is derived to resolve the best route, with the assistance of teammate learning model which handles the nonstationary problems in distributed reinforcement learning. Experiments and the analysis of the different situations show that the proposed scheme performs favorably.

[1]  Xin Su,et al.  Power Allocation Scheme for Femto-to-Macro Downlink Interference Reduction for Smart Devices in Ambient Intelligence , 2016, Mob. Inf. Syst..

[2]  Richard L. Lewis,et al.  Optimal Rewards for Cooperative Agents , 2014, IEEE Transactions on Autonomous Mental Development.

[3]  Adel M. Alimi,et al.  Video stabilization with moving object detecting and tracking for aerial video surveillance , 2014, Multimedia Tools and Applications.

[4]  Arun Kumar Sangaiah,et al.  A short-term traffic prediction model in the vehicular cyber-physical systems , 2017, Future Gener. Comput. Syst..

[5]  Robin R. Murphy,et al.  Robot-Assisted Bridge Inspection , 2011, J. Intell. Robotic Syst..

[6]  Craig Boutilier,et al.  The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems , 1998, AAAI/IAAI.

[7]  Lei Liu,et al.  Latency estimation based on traffic density for video streaming in the internet of vehicles , 2017, Comput. Commun..

[8]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[9]  Abhijit Gosavi,et al.  Reinforcement Learning: A Tutorial Survey and Recent Advances , 2009, INFORMS J. Comput..

[10]  Randal W. Beard,et al.  Cooperative Path Planning for Target Tracking in Urban Environments Using Unmanned Air and Ground Vehicles , 2015, IEEE/ASME Transactions on Mechatronics.

[11]  Miguel A. Olivares-Méndez,et al.  Visual 3-D SLAM from UAVs , 2009, J. Intell. Robotic Syst..

[12]  Yumi Iwashita,et al.  Recognizing Humans in Motion: Trajectory-based Aerial Video Analysis , 2013, BMVC.

[13]  Kuo-Chu Chang,et al.  UAV Path Planning with Tangent-plus-Lyapunov Vector Field Guidance and Obstacle Avoidance , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[14]  Danwei Wang,et al.  Ground Target Tracking Using UAV with Input Constraints , 2013, J. Intell. Robotic Syst..

[15]  Edwin K. P. Chong,et al.  UAV Path Planning in a Dynamic Environment via Partially Observable Markov Decision Process , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[16]  Yongquan Yang,et al.  Collaborative strategy for visual object tracking , 2018, Multimedia Tools and Applications.

[17]  Gérard G. Medioni,et al.  Persistent Tracking for Wide Area Aerial Surveillance , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Simon Lacroix,et al.  Multi-robot target detection and tracking: taxonomy and survey , 2016, Auton. Robots.

[19]  Mei-Chen Yeh,et al.  Fast medium-scale multiperson identification in aerial videos , 2015, Multimedia Tools and Applications.

[20]  Mubarak Shah,et al.  Human identity recognition in aerial images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Zhihao Cai,et al.  Planning algorithm based on airborne sensor for UAV to track and intercept moving target in dynamic environment , 2014, Proceedings of 2014 IEEE Chinese Guidance, Navigation and Control Conference.

[22]  Naira Hovakimyan,et al.  Cooperative target tracking in balanced circular formation: Multiple UAVs tracking a ground vehicle , 2013, 2013 American Control Conference.

[23]  Arun Kumar Sangaiah,et al.  ESCAPE: Effective Scalable Clustering Approach for Parallel Execution of Continuous Position-Based Queries in Position Monitoring Applications , 2017, IEEE Transactions on Sustainable Computing.

[24]  Tao Zhuo,et al.  Multi-model cooperative task assignment and path planning of multiple UCAV formation , 2017, Multimedia Tools and Applications.

[25]  Xiaoguang Gao,et al.  UAV Path Planning Based on Bidirectional Sparse A* Search Algorithm , 2010, 2010 International Conference on Intelligent Computation Technology and Automation.

[26]  Xin Su,et al.  Interference cancellation for non-orthogonal multiple access used in future wireless mobile networks , 2016, EURASIP J. Wirel. Commun. Netw..

[27]  Eunji Lee,et al.  Probabilistic spatio-temporal inference for motion event understanding , 2013, Neurocomputing.

[28]  Peter Strobl,et al.  Monitoring of gas pipelines - a civil UAV application , 2005 .

[29]  Song-Chun Zhu,et al.  Joint inference of groups, events and human roles in aerial videos , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  A. Zajonc,et al.  The Quantum Challenge: Modern Research on the Foundations of Quantum Mechanics , 1997 .

[31]  João Pedro Hespanha,et al.  Robust UAV coordination for target tracking using output-feedback model predictive control with moving horizon estimation , 2015, 2015 American Control Conference (ACC).

[32]  Roland Siegwart,et al.  Aerial robotic contact-based inspection: planning and control , 2016, Auton. Robots.

[33]  Matthew A. Garratt,et al.  Monocular vision-based real-time target recognition and tracking for autonomously landing an UAV in a cluttered shipboard environment , 2017, Auton. Robots.

[34]  Arun Kumar Sangaiah,et al.  A Robust Time Synchronization Scheme for Industrial Internet of Things , 2018, IEEE Transactions on Industrial Informatics.