Julia for robotics: simulation and real-time control in a high-level programming language

Robotics applications often suffer from the ‘two-language problem’, requiring a low-level language for performance-sensitive components and a high-level language for interactivity and experimentation, which tends to increase software complexity. We demonstrate the use of the Julia programming language to solve this problem by being fast enough for online control of a humanoid robot and flexible enough for prototyping. We present several Julia packages developed by the authors, which together enable roughly 2× realtime simulation of the Boston Dynamics Atlas humanoid robot balancing on flat ground using a quadratic-programming-based controller. Benchmarks show a sufficiently low variation in control frequency to make deployment on the physical robot feasible. We also show that Julia’s naturally generic programming style results in versatile packages that are easy to compose and adapt to a wide variety of computational tasks in robotics.

[1]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[2]  Nicolas Mansard,et al.  Analytical Derivatives of Rigid Body Dynamics Algorithms , 2018, Robotics: Science and Systems.

[3]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[4]  David E. Orin,et al.  Centroidal Momentum Matrix of a humanoid robot: Structure and properties , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Endong Wang,et al.  Intel Math Kernel Library , 2014 .

[6]  Alan Edelman,et al.  Julia: A Fresh Approach to Numerical Computing , 2014, SIAM Rev..

[7]  Martin L. Felis.  RBDL: an efficient rigid-body dynamics library using recursive algorithms , 2017, Auton. Robots.

[8]  Amit Kumar,et al.  SymPy: Symbolic computing in Python , 2016, PeerJ Prepr..

[9]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[10]  Stephen P. Boyd,et al.  OSQP: an operator splitting solver for quadratic programs , 2017, 2018 UKACC 12th International Conference on Control (CONTROL).

[11]  A. Iserles,et al.  Lie-group methods , 2000, Acta Numerica.

[12]  Stephen P. Boyd,et al.  Disciplined Convex Programming , 2006 .

[13]  Ismael Ripoll,et al.  TLSF: a new dynamic memory allocator for real-time systems , 2004, Proceedings. 16th Euromicro Conference on Real-Time Systems, 2004. ECRTS 2004..

[14]  Darwin G. Caldwell,et al.  RobCoGen: a code generator for efficient kinematics and dynamics of articulated robots, based on Domain Specific Languages , 2016 .

[15]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[16]  Scott Kuindersma,et al.  Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot , 2015, Autonomous Robots.

[17]  Cliff Click,et al.  The java hotspot TM server compiler , 2001 .

[18]  Patrick Kofod Mogensen,et al.  Optim: A mathematical optimization package for Julia , 2018, J. Open Source Softw..

[19]  Iain Dunning,et al.  JuMP: A Modeling Language for Mathematical Optimization , 2015, SIAM Rev..

[20]  Edwin Olson,et al.  LCM: Lightweight Communications and Marshalling , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Stephen P. Boyd,et al.  CVXGEN: a code generator for embedded convex optimization , 2011, Optimization and Engineering.

[22]  Martin Törngren,et al.  Fundamentals of Implementing Real-Time Control Applications in Distributed Computer Systems , 1998, Real-Time Systems.

[23]  Miles Lubin,et al.  Forward-Mode Automatic Differentiation in Julia , 2016, ArXiv.

[24]  James H. Clark,et al.  Hierarchical geometric models for visible surface algorithms , 1976, CACM.

[25]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[26]  Jesper Smith,et al.  Real-Time control of Humanoid Robots using OpenJDK , 2014, JTRES '14.

[27]  Twan Koolen,et al.  Summary of Team IHMC's virtual robotics challenge entry , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[28]  Mose Giordano,et al.  Uncertainty propagation with functionally correlated quantities , 2016, 1610.08716.

[29]  David E. Orin,et al.  A compliant contact model with nonlinear damping for simulation of robotic systems , 1999, IEEE Trans. Syst. Man Cybern. Part A.

[30]  David E. Orin,et al.  Improved Computation of the Humanoid Centroidal Dynamics and Application for Whole-Body Control , 2016, Int. J. Humanoid Robotics.

[31]  V. T. Rajan,et al.  The Metronome: A Simpler Approach to Garbage Collection in Real-Time Systems , 2003, OTM Workshops.

[32]  Jerry E. Pratt,et al.  Comprehensive summary of the Institute for Human and Machine Cognition’s experience with LittleDog , 2011, Int. J. Robotics Res..

[33]  Marco Pavone,et al.  A Differentiable Augmented Lagrangian Method for Bilevel Nonlinear Optimization , 2019, Robotics: Science and Systems.

[34]  James Gosling,et al.  The Real-Time Specification for Java , 2000, Computer.

[35]  Zhang Yunquan,et al.  Model-driven Level 3 BLAS Performance Optimization on Loongson 3A Processor , 2012, ICPADS.

[36]  Christopher G. Atkeson,et al.  Optimization‐based Full Body Control for the DARPA Robotics Challenge , 2015, J. Field Robotics.

[37]  Qing Nie,et al.  DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia , 2017, Journal of Open Research Software.

[38]  J. Baumgarte.  Stabilization of constraints and integrals of motion in dynamical systems , 1972 .

[39]  Twan Koolen,et al.  LVIS: Learning from Value Function Intervals for Contact-Aware Robot Controllers , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[40]  et al.,et al.  Jupyter Notebooks - a publishing format for reproducible computational workflows , 2016, ELPUB.

[41]  Twan Koolen,et al.  Design of a Momentum-Based Control Framework and Application to the Humanoid Robot Atlas , 2016, Int. J. Humanoid Robotics.

[42]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.