Arm-Hand Behaviours Modelling: From Attention to Imitation

We present a new and original method for modelling arm-hand actions, learning and recognition. We use an incremental approach to separate the arm-hand action recognition problem into three levels. The lower level exploits bottom-up attention to select the region of interest, and attention is specifically tuned towards human motion. The middle level serves to classify action primitives exploiting motion features as descriptors. Each of the primitives is modelled by a Mixture of Gaussian, and it is recognised by a complete, real time and robust recognition system. The higher level systemcombines sequences of primitives using deterministic finite automata. The contribution of the paper is a compositional based model for arm-hand behaviours allowing a robot to learn new actions in a one time shot demonstration of the action execution.

[1]  Christel Baier,et al.  Probabilistic ω-automata , 2012, JACM.

[2]  Timo Kohlberger,et al.  Real-Time Optic Flow Computation with Variational Methods , 2003, CAIP.

[3]  David J. Heeger,et al.  Optical flow using spatiotemporal filters , 2004, International Journal of Computer Vision.

[4]  Pierre Dupont,et al.  Links between probabilistic automata and hidden Markov models: probability distributions, learning models and induction algorithms , 2005, Pattern Recognit..

[5]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[6]  A F Bobick,et al.  Movement, activity and action: the role of knowledge in the perception of motion. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[7]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[8]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[9]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[10]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[11]  Max Welling Donald,et al.  Products of Experts , 2007 .

[12]  R. Turner,et al.  Form and motion coherence activate independent, but not dorsal/ventral segregated, networks in the human brain , 2000, Current Biology.

[13]  J. Oncina,et al.  INFERRING REGULAR LANGUAGES IN POLYNOMIAL UPDATED TIME , 1992 .

[14]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[15]  Richard P. Wildes,et al.  Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation , 2000, ECCV.

[16]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[17]  A J Ahumada,et al.  Model of human visual-motion sensing. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[18]  Prashant Parikh.  A Theory of Communication , 2010 .

[19]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[20]  David A. Forsyth,et al.  Computational Studies of Human Motion: Part 1, Tracking and Motion Synthesis , 2005, Found. Trends Comput. Graph. Vis..

[21]  Larry Wasserman,et al.  All of Nonparametric Statistics (Springer Texts in Statistics) , 2006 .

[22]  Pedro García,et al.  IDENTIFYING REGULAR LANGUAGES IN POLYNOMIAL TIME , 1993 .

[23]  Christopher W. Geib,et al.  The meaning of action: a review on action recognition and mapping , 2007, Adv. Robotics.

[24]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[25]  Stergios B. Fotopoulos,et al.  All of Nonparametric Statistics , 2007, Technometrics.

[26]  Francisco Casacuberta,et al.  Probabilistic finite-state machines - part II , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  J. P. Jones,et al.  An evaluation of the two-dimensional Gabor filter model of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[28]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[29]  Edward H Adelson,et al.  The perception of coherent motion in two-dimensional patterns (abstract only) , 1984, COMG.

[30]  T. Flash,et al.  Neuronal encoding of human kinematic invariants during action observation. , 2010, Cerebral cortex.

[31]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[32]  Francisco Casacuberta,et al.  Probabilistic finite-state machines - part I , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Christopher M. Bishop,et al.  Non-linear Bayesian Image Modelling , 2000, ECCV.

[34]  Azaria Paz,et al.  Probabilistic automata , 2003 .

[35]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[36]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[37]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.