Neural Network Augmented Sensor Fusion for Pose Estimation of Tensegrity Manipulators

In this paper, we present a pose estimation strategy for the end effector of a tensegrity manipulator, based on the use of an extended Kalman filter and a deep feedforward neural network with three hidden layers. Our scheme is based on the fusion of sensor data obtained from an inertial measurement unit and ArUco fiducial markers. The method was implemented on a six bar tensegrity prism manipulator, tested using ground truth acquired from an external vision-based motion capture system, and compared with other estimation methods. The experimental results show the ability of our method to provide reliable pose estimates, also dealing with the problems caused by the tensegrity structure, including marker occlusions due to the presence of bars and strings.

[1]  M.F. Golnaraghi,et al.  A quaternion-based orientation estimation algorithm using an inertial measurement unit , 2004, PLANS 2004. Position Location and Navigation Symposium (IEEE Cat. No.04CH37556).

[2]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, 2008 IEEE Virtual Reality Conference.

[3]  Emanuele Menegatti,et al.  A robust and easy to implement method for IMU calibration without external equipments , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Jeffrey M. Friesen,et al.  State estimation for tensegrity robots , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Chia-Nan Ko,et al.  Short-term load forecasting using SVR (support vector regression)-based radial basis function neural network with dual extended Kalman filter , 2013 .

[6]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[7]  Sergey Levine,et al.  Deep reinforcement learning for tensegrity robot locomotion , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Robert E. Skelton,et al.  An introduction to the mechanics of tensegrity structures , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[9]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[10]  Niels Kjølstad Poulsen,et al.  Incorporation of time delayed measurements in a discrete-time Kalman filter , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[11]  Klaus Zimmermann,et al.  A multistable tensegrity structure with a gripper application , 2017 .

[12]  Guanghao Sun,et al.  Online state space filtering of biosignals using neural network-augmented Kalman filter , 2017, 2017 10th Biomedical Engineering International Conference (BMEiCON).

[13]  Norbert Link,et al.  Indoor 3D position estimation using low-cost inertial sensors and marker-based video-tracking , 2010, IEEE/ION Position, Location and Navigation Symposium.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  John Rieffel,et al.  Evolution of Locomotion on a Physical Tensegrity Robot , 2014 .

[16]  Rafael Muñoz-Salinas,et al.  Speeded up detection of squared fiducial markers , 2018, Image Vis. Comput..

[17]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[18]  Angelo M. Sabatini,et al.  Extended Kalman Filter-Based Methods for Pose Estimation Using Visual, Inertial and Magnetic Sensors: Comparative Analysis and Performance Evaluation , 2013, Sensors.

[19]  Robert E. Skelton,et al.  Tensegrity system dynamics with rigid bars and massive strings , 2019, Multibody System Dynamics.

[20]  Atil Iscen,et al.  Design and evolution of a modular tensegrity robot platform , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Nikhil Ketkar,et al.  Introduction to PyTorch , 2021, Deep Learning with Python.

[22]  H. Gurung,et al.  Self-sensing SMA Actuator Using Extended Kalman Filter and Artificial Neural Network , 2016 .

[23]  A. Tanju Erdem,et al.  Fusing Inertial Sensor Data in an Extended Kalman Filter for 3D Camera Tracking , 2015, IEEE Transactions on Image Processing.

[24]  Jian Zhou,et al.  A calibration method for enhancing robot accuracy through integration of an extended Kalman filter algorithm and an artificial neural network , 2015, Neurocomputing.

[25]  Huseyin Atakan Varol,et al.  Computer Vision-Based Pose Estimation of Tensegrity Robots Using Fiducial Markers , 2019, 2019 IEEE/SICE International Symposium on System Integration (SII).

[26]  Huseyin Atakan Varol,et al.  Generalized Dynamics of Stacked Tensegrity Manipulators , 2019, IEEE Access.