Autonomous target tracking of UAVs based on low-power neural network hardware

Detecting and identifying targets in unmanned aerial vehicle (UAV) images and videos have been challenging problems due to various types of image distortion. Moreover, the significantly high processing overhead of existing image/video processing techniques and the limited computing resources available on UAVs force most of the processing tasks to be performed by the ground control station (GCS) in an off-line manner. In order to achieve fast and autonomous target identification on UAVs, it is thus imperative to investigate novel processing paradigms that can fulfill the real-time processing requirements, while fitting the size, weight, and power (SWaP) constrained environment. In this paper, we present a new autonomous target identification approach on UAVs, leveraging the emerging neuromorphic hardware which is capable of massively parallel pattern recognition processing and demands only a limited level of power consumption. A proof-of-concept prototype was developed based on a micro-UAV platform (Parrot AR Drone) and the CogniMemTMneural network chip, for processing the video data acquired from a UAV camera on the y. The aim of this study was to demonstrate the feasibility and potential of incorporating emerging neuromorphic hardware into next-generation UAVs and their superior performance and power advantages towards the real-time, autonomous target tracking.

[1]  Shai Avidan,et al.  Ensemble Tracking , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Heinrich Klar,et al.  Digital Neurohardware: Principles and Perspectives , 1998 .

[4]  Gaurav Tewari,et al.  A hardware/software co-design model for face recognition using Cognimem Neural Network chip , 2011, 2011 International Conference on Image Information Processing.

[5]  B. V. K. Vijaya Kumar,et al.  Learning-based approaches to nonlinear multisensor fusion in target tracking , 2013, Proceedings of the 16th International Conference on Information Fusion.

[6]  Jamil Ahmad,et al.  Design and Implementation of a Neural Network for Real-Time Object Tracking , 2007 .

[7]  Ling Shao,et al.  Recent advances and trends in visual tracking: A review , 2011, Neurocomputing.

[8]  Ming-Hsuan Yang,et al.  Robust Object Tracking with Online Multiple Instance Learning , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Anamitra Makur,et al.  Online adaptive radial basis function networks for robust object tracking , 2010, Comput. Vis. Image Underst..

[10]  Yasser Baleghi,et al.  Improved Object Tracking Using Radial Basis Function Neural Networks , 2011, 2011 7th Iranian Conference on Machine Vision and Image Processing.

[11]  Indranil Saha,et al.  journal homepage: www.elsevier.com/locate/neucom , 2022 .

[12]  Morgan Bishop,et al.  Foundations of Neuromorphic Computing , 2013 .

[13]  Zhonghai Wang,et al.  Low frame rate video target localization and tracking testbed , 2013, Defense, Security, and Sensing.

[14]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[15]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[16]  Subhash Challa,et al.  Vision Based Target Tracking for Autonomous Land Vehicle Navigation: A Brief Survey , 2009 .

[17]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Clark S. Lindsey,et al.  An implementation of the Zer o Instruction Set Computer (ZISC036) on a PC/ISA-bus card , 1994 .

[19]  Ali Borji,et al.  Adaptive object tracking by learning background context , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Anamitra Makur,et al.  Robust Object Tracking with Radial Basis Function Networks , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[21]  Zhanpeng Jin,et al.  Autonomously Reconfigurable Artificial Neural Network on a Chip , 2010 .

[22]  Robinson E. Pino,et al.  Network Science and Cybersecurity , 2013, Advances in Information Security.

[23]  Darío Baptista,et al.  A survey of software and hardware use in artificial neural networks , 2013, Neural Computing and Applications.

[24]  Fernando Morgado Dias,et al.  Artificial neural networks: a review of commercial hardware , 2004, Eng. Appl. Artif. Intell..

[25]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[26]  Huchuan Lu,et al.  Object tracking based on local learning , 2012, 2012 19th IEEE International Conference on Image Processing.

[27]  Garrett S. Rose,et al.  Hardware-Based Computational Intelligence for Size, Weight, and Power Constrained Environments , 2014, Network Science and Cybersecurity.

[28]  Mikhail S. Tarkov,et al.  Real-Time Object Tracking by CUDA-accelerated Neural Network , 2013 .