Understanding Mechanical Motion: From Images to Behaviors

Abstract We present an algorithm for producing behavior descriptions of planar fixed axes mechanical motions from image sequences using a formal behavior language. The language, which covers the most important class of mechanical motions, symbolically captures the qualitative aspects of objects that translate and rotate along axes that are fixed in space. The algorithm exploits the structure of these motions to robustly recover the objects behaviors. It starts by identifying the independently moving objects, their motion parameters, and their variation with respect to time using normal optical flow analysis, iterative motion segmentation, and motion parameter estimation. It then produces a formal description of their behavior by identifying individual uniform motion events and simultaneous motion changes, and parsing them with a motion grammar. We demonstrate the algorithm on three sets of image sequences: mechanisms, everyday situations, and a robot manipulation scenario.

[1]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[2]  Alessandro Verri,et al.  Against Quantitative Optical Flow , 1987 .

[3]  Xinhua Zhuang,et al.  Optic Flow Field Segmentation and Motion Estimation Using a Robust Genetic Partitioning Algorithm , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Azriel Rosenfeld,et al.  Recognition by Functional Parts , 1995, Comput. Vis. Image Underst..

[5]  Thomas F. Stahovich,et al.  Qualitative rigid-body mechanics , 1997, Artif. Intell..

[6]  Alfred M. Bruckstein,et al.  How to Catch a Crook , 1994, J. Vis. Commun. Image Represent..

[7]  E. R. Maki,et al.  The Creation of Mechanisms According to Kinematic Structure and Function , 1979 .

[8]  Leo Joskowicz,et al.  A representation language for mechanical behavior , 1996, Artif. Intell. Eng..

[9]  James W. Davis,et al.  An appearance-based representation of action , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[10]  Leo Joskowicz.  Mechanism comparison and classification for design , 1990 .

[11]  Larry S. Davis,et al.  Learned temporal models of image motion , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  Tamotsu Murakami,et al.  Mechanism Concept Retrieval Using Configuration Space. , 1994 .

[13]  Hilary Buxton,et al.  Watching behaviour: the role of context and learning , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[14]  Ehud Rivlin,et al.  Function From Motion , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Howard E. Shrobe.  Understanding Linkages , 1993, AAAI.

[16]  Jeffrey Mark Siskind,et al.  Naive physics, event perception, lexical semantics, and language acquisition , 1992 .

[17]  D. Subramanian,et al.  Kinematic synthesis with configuration spaces , 1995 .

[18]  David J. Heeger,et al.  Optical flow from spatialtemporal filters , 1987 .

[19]  Leo Joskowicz,et al.  Computational Kinematics , 1991, Artif. Intell..

[20]  Gilad Adiv,et al.  Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[22]  Narendra Ahuja,et al.  Motion and Structure From Two Perspective Views: Algorithms, Error Analysis, and Error Estimation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Lawrence Birnbaum,et al.  Sensible Scenes: Visual Understanding of Complex Structures through Causal Analysis , 1993, AAAI.

[24]  Kevin W. Bowyer,et al.  Indexing function-based categories for generic recognition , 1992, CVPR.

[25]  David C. Hogg,et al.  Learning the distribution of object trajectories for event recognition , 1996, Image Vis. Comput..

[26]  L. Stark,et al.  Dissertation Abstract , 1994, Journal of Cognitive Education and Psychology.

[27]  Kurt M. Marshek,et al.  An algebraic and predicate logic approach to representation and reasoning in machine design , 1990 .

[28]  Matthew Brand,et al.  Physics-Based Visual Understanding , 1997, Comput. Vis. Image Underst..

[29]  Tamotsu Murakami,et al.  Mechanism concept retrieval using configuration space , 1997 .

[30]  Leo Joskowicz,et al.  Automated modeling and kinematic simulation of mechanisms , 1993, Comput. Aided Des..

[31]  Shean Juinn Chiou,et al.  Design representation and computational synthesis of mechanical motions , 1992 .

[32]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[33]  Kevin W. Bowyer,et al.  Generic recognition through qualitative reasoning about 3-D shape and object function , 1991, CVPR.

[34]  Hans-Hellmut Nagel,et al.  A vision of vision and language' comprises action: an example from road traffic , 1994 .

[35]  Mubarak Shah,et al.  The trajectory primal sketch: a multi-scale scheme for representing motion characteristics , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[36]  Alfred M. Bruckstein,et al.  How to Track a Flying Saucer , 1996, J. Vis. Commun. Image Represent..

[37]  Jeffrey Mark Siskind,et al.  A Maximum-Likelihood Approach to Visual Event Classification , 1996, ECCV.

[38]  Alessandro Verri,et al.  Identifying multiple motions from optical flow , 1992, ECCV.

[39]  James W. Davis,et al.  Real-time closed-world tracking , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[41]  Hans-Hellmut Nagel,et al.  Algorithmic characterization of vehicle trajectories from image sequences by motion verbs , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Ramesh C. Jain,et al.  Determining Motion Parameters for Scenes with Translation and Rotation , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Boi Faltings,et al.  Qualitative Kinematics in Mechanisms , 1987, IJCAI.