Neural Networks Based Single Robot Arm Control for Visual Servoing

In this chapter, we investigate the kinematic control of a single robot arm with an eye-in-hand camera for visual servoing by using neural networks. The visual servoing problem is formulated as a constrained quadratic program, which is then solved via a recurrent neural network. By this approach, the visual servoing with respect to a static point object is achieved with the feature coordinate errors in the image space converging to zero. Besides, joint angle and velocity limits of the robot arm are satisfied, which thus enhances the safety of the robot arm during the visual servoing process. The performance of the approach is guaranteed via theoretical analysis and validated via a simulative example.

[1]  Bing Chen,et al.  Adaptive Neural Tracking Control for a Class of Nonstrict-Feedback Stochastic Nonlinear Systems With Unknown Backlash-Like Hysteresis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Ji Xiang,et al.  A Varied Weights Method for the Kinematic Control of Redundant Manipulators With Multiple Constraints , 2011, IEEE Transactions on Robotics.

[3]  Shuai Li,et al.  Simultaneous learning and control of parallel Stewart platforms with unknown parameters , 2017, Neurocomputing.

[4]  Shuzhi Sam Ge,et al.  Adaptive Control of Robotic Manipulators With Unified Motion Constraints , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Shuai Li,et al.  Time-Scale Expansion-Based Approximated Optimal Control for Underactuated Systems Using Projection Neural Networks , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[6]  Kao-Shing Hwang,et al.  A Virtual Torque-Based Approach to Kinematic Control of Redundant Manipulators , 2017, IEEE Transactions on Industrial Electronics.

[7]  Nadia Magnenat-Thalmann,et al.  Human-Like Behavior Generation Based on Head-Arms Model for Robot Tracking External Targets and Body Parts , 2015, IEEE Transactions on Cybernetics.

[8]  MengChu Zhou,et al.  Distributed Winner-Take-All in Dynamic Networks , 2017, IEEE Transactions on Automatic Control.

[9]  Yunong Zhang,et al.  QP-based refined manipulability-maximizing scheme for coordinated motion planning and control of physically constrained wheeled mobile redundant manipulators , 2016, Nonlinear Dynamics.

[10]  Dongsheng Guo,et al.  Theoretical analysis, numerical verification and geometrical representation of new three-step DTZD algorithm for time-varying nonlinear equations solving , 2016, Neurocomputing.

[11]  Shuzhi Sam Ge,et al.  Fault Diagnosis in Image-Based Visual Servoing With Eye-in-Hand Configurations Using Kalman Filter , 2016, IEEE Transactions on Industrial Informatics.

[12]  Shuai Li,et al.  Distributed Task Allocation of Multiple Robots: A Control Perspective , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Dongsheng Guo,et al.  Acceleration-Level Inequality-Based MAN Scheme for Obstacle Avoidance of Redundant Robot Manipulators , 2014, IEEE Transactions on Industrial Electronics.

[14]  Long Jin,et al.  G2-Type SRMPC Scheme for Synchronous Manipulation of Two Redundant Robot Arms , 2015, IEEE Transactions on Cybernetics.

[15]  Yunong Zhang,et al.  More illustrative investigation on window-shaped obstacle avoidance of robot manipulators using a simplified LVI-based primal-dual neural network , 2009, 2009 International Conference on Mechatronics and Automation.

[16]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[17]  Dongsheng Guo,et al.  Acceleration-level obstacle-avoidance scheme for motion planning of redundant robot manipulators , 2016, 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[18]  Anthony A. Maciejewski,et al.  Kinematic Design of Redundant Robotic Manipulators for Spatial Positioning that are Optimally Fault Tolerant , 2013, IEEE Transactions on Robotics.

[19]  Shuai Li,et al.  Noise-Tolerant ZNN Models for Solving Time-Varying Zero-Finding Problems: A Control-Theoretic Approach , 2017, IEEE Transactions on Automatic Control.

[20]  Shuai Li,et al.  Neural network based mobile phone localization using Bluetooth connectivity , 2013, Neural Computing and Applications.

[21]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[22]  Jian Li,et al.  Enhanced discrete-time Zhang neural network for time-variant matrix inversion in the presence of bias noises , 2016, Neurocomputing.

[23]  Shuai Li,et al.  Accelerating a Recurrent Neural Network to Finite-Time Convergence for Solving Time-Varying Sylvester Equation by Using a Sign-Bi-power Activation Function , 2012, Neural Processing Letters.

[24]  Shuai Li,et al.  Nonconvex function activated zeroing neural network models for dynamic quadratic programming subject to equality and inequality constraints , 2017, Neurocomputing.

[25]  Shuai Li,et al.  Adaptive Near-Optimal Control of Uncertain Systems With Application to Underactuated Surface Vessels , 2018, IEEE Transactions on Control Systems Technology.

[26]  Yajin Zhou,et al.  SP-NN: A novel neural network approach for path planning , 2007, 2007 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[27]  M. J. D. Hayes,et al.  Pareto Optimality and Multiobjective Trajectory Planning for a 7-DOF Redundant Manipulator , 2010, IEEE Transactions on Robotics.

[28]  Shuai Li,et al.  Dynamic Neural Networks for Kinematic Redundancy Resolution of Parallel Stewart Platforms , 2016, IEEE Transactions on Cybernetics.

[29]  Lin Xiao,et al.  A nonlinearly-activated neurodynamic model and its finite-time solution to equality-constrained quadratic optimization with nonstationary coefficients , 2016, Appl. Soft Comput..

[30]  M. Spong,et al.  Robot Modeling and Control , 2005 .

[31]  Hongliang Guo,et al.  A type of biased consensus-based distributed neural network for path planning , 2017 .

[32]  Shuzhi Sam Ge,et al.  A unified quadratic-programming-based dynamical system approach to joint torque optimization of physically constrained redundant manipulators , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[33]  Mei Liu,et al.  Performance analyses of recurrent neural network models exploited for online time-varying nonlinear optimization , 2016, Comput. Sci. Inf. Syst..

[34]  Dongsheng Guo,et al.  Novel Discrete-Time Zhang Neural Network for Time-Varying Matrix Inversion , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[35]  Lin Xiao,et al.  A finite-time convergent neural dynamics for online solution of time-varying linear complex matrix equation , 2015, Neurocomputing.

[36]  Shaocheng Tong,et al.  A Unified Approach to Adaptive Neural Control for Nonlinear Discrete-Time Systems With Nonlinear Dead-Zone Input , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[37]  Shuai Li,et al.  Predictive Suboptimal Consensus of Multiagent Systems With Nonlinear Dynamics , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[38]  Hao Zhang,et al.  Multirate parallel distributed compensation of a cluster in wireless sensor and actor networks , 2016, Int. J. Syst. Sci..

[39]  Shuai Li,et al.  A Dynamic Neural Network Approach for Efficient Control of Manipulators , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[40]  Xiaoping Liu,et al.  Direct adaptive neural control of nonlinear strict-feedback systems with unmodeled dynamics using small-gain approach , 2016 .

[41]  Shuai Li,et al.  Integration-Enhanced Zhang Neural Network for Real-Time-Varying Matrix Inversion in the Presence of Various Kinds of Noises , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Yunong Zhang,et al.  Dynamic design, numerical solution and effective verification of acceleration-level obstacle-avoidance scheme for robot manipulators , 2016, Int. J. Syst. Sci..

[43]  Yangming Li,et al.  A class of finite-time dual neural networks for solving quadratic programming problems and its k-winners-take-all application , 2013, Neural Networks.

[44]  Shuai Li,et al.  Cooperative Motion Generation in a Distributed Network of Redundant Robot Manipulators With Noises , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  S. Li,et al.  A recurrent neural network for inter-localization of mobile phones , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[46]  Long Jin,et al.  Taylor $O(h^{3})$ Discretization of ZNN Models for Dynamic Equality-Constrained Quadratic Programming With Application to Manipulators , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Pierre-Brice Wieber,et al.  Kinematic Control of Redundant Manipulators: Generalizing the Task-Priority Framework to Inequality Task , 2011, IEEE Transactions on Robotics.

[48]  Shuai Li,et al.  Kinematic Control of Redundant Manipulators Using Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Lin Xiao,et al.  A convergence-accelerated Zhang neural network and its solution application to Lyapunov equation , 2016, Neurocomputing.

[50]  Shuai Li,et al.  Modified ZNN for Time-Varying Quadratic Programming With Inherent Tolerance to Noises and Its Application to Kinematic Redundancy Resolution of Robot Manipulators , 2016, IEEE Transactions on Industrial Electronics.

[51]  Yunong Zhang,et al.  Different Complex ZFs Leading to Different Complex ZNN Models for Time-Varying Complex Generalized Inverse Matrices , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[52]  Yunong Zhang,et al.  Fault-tolerant motion planning and control of redundant manipulator , 2012 .

[53]  Éric Marchand,et al.  Virtual Visual Servoing: a framework for real‐time augmented reality , 2002, Comput. Graph. Forum.

[54]  Shuai Li,et al.  Manipulability Optimization of Redundant Manipulators Using Dynamic Neural Networks , 2017, IEEE Transactions on Industrial Electronics.

[55]  Hidenori Kimura,et al.  Visual servoing with hand-eye manipulator-optimal control approach , 1996, IEEE Trans. Robotics Autom..

[56]  KanounO.,et al.  Kinematic Control of Redundant Manipulators , 2011 .

[57]  Lin Xiao,et al.  A nonlinearly activated neural dynamics and its finite-time solution to time-varying nonlinear equation , 2016, Neurocomputing.

[58]  Shuai Li,et al.  Distributed Recurrent Neural Networks for Cooperative Control of Manipulators: A Game-Theoretic Perspective , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[59]  Yunong Zhang,et al.  State adjustment of redundant robot manipulator based on quadratic programming , 2012, Robotica.

[60]  Shuai Li,et al.  Neural-network based AUV path planning in estuary environments , 2012, Proceedings of the 10th World Congress on Intelligent Control and Automation.

[61]  Dongsheng Guo,et al.  A New Inequality-Based Obstacle-Avoidance MVN Scheme and Its Application to Redundant Robot Manipulators , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[62]  Shuai Li,et al.  A dynamic neural network approach for solving nonlinear inequalities defined on a graph and its application to distributed, routing-free, range-free localization of WSNs , 2013, Neurocomputing.