Advances in humanoid control and perception

One day there will be humanoid robots among us doing our boring, time-consuming, or dangerous tasks. They might cook a delicious meal for us or do the groceries. For this to become reality, many advances need to be made to the artificial intelligence of humanoid robots. The ever-increasing available computational processing power opens new doors for such advances. In this thesis we develop novel algorithms for humanoid control and vision that harness this power. We apply these methods on an iCub humanoid upper-body with 41 degrees of freedom. For control, we develop Natural Gradient Inverse Kinematics (NGIK), a samplingbased optimiser that applies natural evolution strategies to perform inverse kinematics. The resulting algorithm makes very few assumptions and gives much more freedom in definable constraints than its Jacobian-based counterparts. A special graph-building procedure is introduced to build Task-Relevant Roadmaps (TRM) by iteratively applying NGIK and storing the results. TRMs form searchable graphs of kinematic configurations on which a wide range of task-relevant humanoid movements can be planned. Through coordinating several instances of NGIK, a fast parallelised version of the TRM building algorithm is developed. To contrast the offline TRM algorithms, we also develop Natural Gradient Control which directly uses the optimisation pass in NGIK as an online control signal. For vision, we develop dynamic vision algorithms that form cyclic information flows that affect their own processing. Deep Attention Selective Networks (dasNet) implement feedback in convolutional neural networks through a gatingmechanism that is steered by a policy. Through this feedback, dasNet can focus on different features in the image in light of previously gathered information and improve classification, with state-of-the-art results at the time of publication. Then, we develop PyraMiD-LSTM, which processes 3D volumetric data by employing a novel convolutional Long ShortTermMemory network (C-LSTM) to compute pyramidal contexts for every voxel, and combine them to perform segmentation. This resulted in state-of-the-art performance on a segmentation benchmark. The work on control and vision is integrated into an application on the iCub robot. A Fast-Weight PyraMiD-LSTM is developed that dynamically generates weights

[1]  Marijn F. Stollenga,et al.  Continual curiosity-driven skill acquisition from high-dimensional video inputs for humanoid robots , 2017, Artif. Intell..

[2]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[3]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ronan Boulic,et al.  An inverse kinematics architecture enforcing an arbitrary number of strict priority levels , 2004, The Visual Computer.

[5]  Adrien-Marie Legendre,et al.  Nouvelles méthodes pour la détermination des orbites des comètes , 1970 .

[6]  Konrad Zuse,et al.  Rechnender Raum , 1991, Physik und Informatik.

[7]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[8]  Howie Choset,et al.  Principles of Robot Motion: Theory, Algorithms, and Implementation ERRATA!!!! 1 , 2007 .

[9]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[10]  Kunihiko Fukushima,et al.  Artificial vision by multi-layered neural networks: Neocognitron and its advances , 2013, Neural Networks.

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  Marcus Liwicki,et al.  Texture Classification Using 2D LSTM Networks , 2014, 2014 22nd International Conference on Pattern Recognition.

[13]  A. Cardona,et al.  An Integrated Micro- and Macroarchitectural Analysis of the Drosophila Brain by Computer-Assisted Serial Section Electron Microscopy , 2010, PLoS biology.

[14]  Chris Hecker,et al.  Real-time motion retargeting to highly varied user-created morphologies , 2008, SIGGRAPH 2008.

[15]  Tom Schaul,et al.  High dimensions and heavy tails for natural evolution strategies , 2011, GECCO '11.

[16]  Jean-Claude Latombe,et al.  On the Probabilistic Foundations of Probabilistic Roadmap Planning , 2006, Int. J. Robotics Res..

[17]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[18]  Jürgen Schmidhuber,et al.  The Natural Gradient as a control signal for a humanoid robot , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[19]  Jürgen Schmidhuber,et al.  Deep Networks with Internal Selective Attention through Feedback Connections , 2014, NIPS.

[20]  Laurenz Wiskott,et al.  Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells , 2007, PLoS Comput. Biol..

[21]  L. L. Campbell,et al.  The relation between information theory and the differential geometry approach to statistics , 1985, Inf. Sci..

[22]  Tom Schaul,et al.  Exponential natural evolution strategies , 2010, GECCO '10.

[23]  Emil L. Post.  Finite combinatory processes—formulation , 1936, Journal of Symbolic Logic.

[24]  Dennis Gabor,et al.  Theory of communication , 1946 .

[25]  Nikolaos G. Tsagarakis,et al.  iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research , 2007, Adv. Robotics.

[26]  Rob Fergus,et al.  Stochastic Pooling for Regularization of Deep Convolutional Neural Networks , 2013, ICLR.

[27]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[28]  Jürgen Schmidhuber,et al.  Rapid Humanoid Motion Learning through Coordinated, Parallel Evolution , 2014, SAB.

[29]  Yaozong Gao,et al.  LINKS: Learning-based multi-source IntegratioN frameworK for Segmentation of infant brain images , 2015, NeuroImage.

[30]  Lise Getoor,et al.  Learning in Logic , 2010, Encyclopedia of Machine Learning.

[31]  Frank K. Soong,et al.  TTS synthesis with bidirectional LSTM based recurrent neural networks , 2014, INTERSPEECH.

[32]  Andrew W. Senior,et al.  Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition , 2014, ArXiv.

[33]  A. Morgan,et al.  Solving the Kinematics of the Most General Six- and Five-Degree-of-Freedom Manipulators by Continuation Methods , 1985 .

[34]  S. Dreyfus.  The computational solution of optimal control problems with time lag , 1973 .

[35]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[36]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[37]  Pietro Perona,et al.  Caltech-UCSD Birds 200 , 2010 .

[38]  B. Rosen,et al.  Functional mapping of the human visual cortex by magnetic resonance imaging. , 1991, Science.

[39]  Marcus Liwicki,et al.  Scene labeling with LSTM recurrent neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[41]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[42]  Maria da Graça Marcos,et al.  Trajectory planning of redundant manipulators using genetic algorithms , 2009 .

[43]  Eva Lundstr.  Adaptive Eigenvalue Computations Using Newton's Method on the Grassmann Manifold , 1999 .

[44]  C. Gross.  Genealogy of the “Grandmother Cell” , 2002, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[45]  Jürgen Schmidhuber,et al.  Optimal Curiosity-Driven Modular Incremental Slow Feature Analysis , 2016, Neural Computation.

[46]  Jürgen Schmidhuber,et al.  Explore to see, learn to perceive, get the actions for free: SKILLABILITY , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[47]  Jürgen Leitner,et al.  Task-relevant roadmaps: A framework for humanoid motion planning , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[48]  Yoshua Bengio,et al.  Maxout Networks , 2013, ICML.

[49]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[50]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  R. Paul,et al.  Kinematic control equations for simple manipulators , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[52]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[53]  Amir Alansary,et al.  MRBrainS Challenge: Online Evaluation Framework for Brain Image Segmentation in 3T MRI Scans , 2015, Comput. Intell. Neurosci..

[54]  Yann LeCun,et al.  Pedestrian Detection with Unsupervised Multi-stage Feature Learning , 2012, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  J. Angeles.  On the Numerical Solution of the Inverse Kinematic Problem , 1985 .

[56]  Paul J. Werbos,et al.  Applications of advances in nonlinear sensitivity analysis , 1982 .

[57]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[58]  Jürgen Leitner,et al.  The Modular Behavioral Environment for Humanoids and other Robots (MoBeE) , 2012, ICINCO.

[59]  Giorgio Metta,et al.  YARP: Yet Another Robot Platform , 2006 .

[60]  Tara N. Sainath,et al.  Improving deep neural networks for LVCSR using rectified linear units and dropout , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[61]  Pietro Perona,et al.  Visual Recognition with Humans in the Loop , 2010, ECCV.

[62]  Luca Maria Gambardella,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence Flexible, High Performance Convolutional Neural Networks for Image Classification , 2022 .

[63]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[64]  Ting Liu,et al.  A modular hierarchical approach to 3D electron microscopy image segmentation , 2014, Journal of Neuroscience Methods.

[65]  Max Suell Dutra,et al.  New technique for inverse kinematics problem using Simulated Annealing , 2008 .

[66]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[67]  Marijn F. Stollenga,et al.  The Neural Support Vector Machine , 2013 .

[68]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[69]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[70]  W. Wolovich,et al.  A computational technique for inverse kinematics , 1984, The 23rd IEEE Conference on Decision and Control.

[71]  Jürgen Schmidhuber,et al.  Autonomous learning of abstractions using Curiosity-Driven Modular Incremental Slow Feature Analysis , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[72]  Henry J. Kelley,et al.  Gradient Theory of Optimal Flight Paths , 1960 .

[73]  Jürgen Schmidhuber,et al.  Continually adding self-invented problems to the repertoire: First experiments with POWERPLAY , 2012, 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL).

[74]  Yann LeCun,et al.  A theoretical framework for back-propagation , 1988 .

[75]  Michael Kauschke,et al.  Closed form solutions applied to redundant serial link manipulators , 1996 .

[76]  Yann LeCun,et al.  What is the best multi-stage architecture for object recognition? , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[77]  T. Munich,et al.  Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks , 2008, NIPS.

[78]  Alekseĭ Grigorʹevich Ivakhnenko,et al.  CYBERNETIC PREDICTING DEVICES , 1966 .

[79]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[80]  John Tran,et al.  cuDNN: Efficient Primitives for Deep Learning , 2014, ArXiv.

[81]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[82]  Jürgen Schmidhuber,et al.  Multi-dimensional Recurrent Neural Networks , 2007, ICANN.

[83]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[84]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[85]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[86]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[87]  Beno Benhabib,et al.  A complete generalized solution to the inverse kinematics of robots , 1985, IEEE J. Robotics Autom..

[88]  A. Church.  An Unsolvable Problem of Elementary Number Theory , 1936 .

[89]  James Reinders,et al.  Intel threading building blocks - outfitting C++ for multi-core processor parallelism , 2007 .

[90]  Shun-ichi Amari,et al.  Differential geometrical theory of statistics , 1987 .

[91]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[92]  Luca Maria Gambardella,et al.  Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks , 2013, MICCAI.

[93]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[94]  Rajeev Motwani,et al.  Path Planning in Expansive Configuration Spaces , 1999, Int. J. Comput. Geom. Appl..

[95]  A. Hemami,et al.  A more general closed-form solution to the inverse kinematics of mechanical arms , 1987, Adv. Robotics.

[96]  M. A. Wiering,et al.  Using Guided Autoencoders on Face Recognition , 2011 .

[97]  John B. Moore,et al.  Essential Matrix Estimation Using Gauss-Newton Iterations on a Manifold , 2007, International Journal of Computer Vision.

[98]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[99]  M. A. Zohdy,et al.  Robust Control of Robotic Manipulators , 1989, 1989 American Control Conference.

[100]  Pyung Chang.  A closed-form solution for inverse kinematics of robot manipulators with redundancy , 1987, IEEE Journal on Robotics and Automation.

[101]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[102]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[103]  Nicolas Courty,et al.  Inverse Kinematics Using Sequential Monte Carlo Methods , 2008, AMDO.

[104]  Gregory S. Ammar,et al.  The geometry of matrix eigenvalue methods , 1986 .

[105]  Sepp Hochreiter,et al.  Untersuchungen zu dynamischen neuronalen Netzen , 1991 .

[106]  Giulio Sandini,et al.  The iCub humanoid robot: an open platform for research in embodied cognition , 2008, PerMIS.

[107]  Anuj Srivastava,et al.  A Gradient-Descent Method for Curve Fitting on Riemannian Manifolds , 2011, Foundations of Computational Mathematics.

[108]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[109]  A. E. Bryson,et al.  A Steepest-Ascent Method for Solving Optimum Programming Problems , 1962 .

[110]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[111]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[112]  K. Gödel.  Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme I , 1931 .

[113]  Jürgen Schmidhuber,et al.  Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation , 2015, NIPS.

[114]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[115]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[116]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[117]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[118]  Steven M. LaValle,et al.  Planning algorithms , 2006 .