The Computational Perception of Scene Dynamics

Understanding observations of interacting objects requires one to reason about qualitative scene dynamics. For example, on observing a hand lifting a can, we may infer that an “active” hand is applying an upwards force (by grasping) to lift a “passive” can. We present an implemented computational theory that derives such dynamic descriptions directly from camera input. Our approach is based on an analysis of the Newtonian mechanics of a simplified scene model. Interpretations are expressed in terms of assertions about the kinematic and dynamic properties of the scene. The feasibility of interpretations relative to Newtonian mechanics is determined by a reduction to linear programming. Finally, to select plausible interpretations, multiple feasible solutions are compared using a preference hierarchy. We provide computational examples to demonstrate that our model is sufficiently rich to describe a wide variety of image sequences.

[1]  Jeffrey Mark Siskind,et al.  Naive physics, event perception, lexical semantics, and language acquisition , 1992 .

[2]  Raymond Reiter,et al.  A Logical Framework for Depiction and Image Interpretation , 1989, Artif. Intell..

[3]  D. Proffitt,et al.  Understanding wheel dynamics , 1990, Cognitive Psychology.

[4]  Brian V. Funt,et al.  Problem-Solving with Diagrammatic Representations , 1980, Artif. Intell..

[5]  John K. Tsotsos,et al.  A framework for visual motion understanding , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[7]  Katsushi Ikeuchi,et al.  Toward an assembly plan from observation. I. Task recognition with polyhedral objects , 1994, IEEE Trans. Robotics Autom..

[8]  John McCarthy,et al.  Applications of Circumscription to Formalizing Common Sense Knowledge , 1987, NMR.

[9]  J. Freyd,et al.  Representing statics as forces in equilibrium. , 1988, Journal of experimental psychology. General.

[10]  Jeffrey Mark Siskind,et al.  Grounding language in perception , 1993, Other Conferences.

[11]  E. Shavit,et al.  Motion understanding from qualitative visual dynamics , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[12]  T. Chow,et al.  Classical Mechanics, Second Edition , 1995 .

[13]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, ECCV.

[14]  Manuel Blum,et al.  A Stability Test for Configurations of Blocks , 1970 .

[15]  W. Richards,et al.  Model structure and reliable inference , 1996 .

[16]  Allan D. Jepson,et al.  Priors, preferences and categorical percepts , 1996 .

[17]  David Baraff,et al.  Interactive simulation of solid rigid bodies , 1995, IEEE Computer Graphics and Applications.

[18]  A. Leslie,et al.  Do six-month-old infants perceive causality? , 1987, Cognition.

[19]  Allan D. Jepson,et al.  Computational Perception of Scene Dynamics , 1996, ECCV.

[20]  J. W. Humberston Classical mechanics , 1980, Nature.

[21]  Scott E. Fahlman,et al.  A Planning System for Robot Construction Tasks , 1973, Artif. Intell..

[22]  Michael R. M. Jenkin,et al.  Detecting Floor Anomalies , 1994, BMVC.

[23]  Roy Featherstone,et al.  Robot Dynamics Algorithms , 1987 .

[24]  Gary C. Borchardt,et al.  Event Calculus , 1985, IJCAI.

[25]  Norman I. Badler,et al.  Temporal scene analysis: conceptual descriptions of object movements. , 1975 .

[26]  Vladimir Lifschitz,et al.  Computing Circumscription , 1985, IJCAI.

[27]  D R Proffitt,et al.  Understanding collision dynamics. , 1989, Journal of experimental psychology. Human perception and performance.

[28]  Yasuo Kuniyoshi,et al.  Qualitative Recognition of Ongoing Human Action Sequences , 1993, IJCAI.

[29]  Leo Joskowicz,et al.  Computational Kinematics , 1991, Artif. Intell..

[30]  Lawrence Birnbaum,et al.  Sensible Scenes: Visual Understanding of Complex Structures through Causal Analysis , 1993, AAAI.

[31]  H. C. Corben,et al.  Classical Mechanics (2nd ed.) , 1961 .