From image sequences towards conceptual descriptions

Abstract Four approaches towards the extraction of conceptual descriptions from image sequences which all evolved from the same 1977 proposal are surveyed. After a critical evaluation of the insights gained from these experimental approaches, the necessity to strive for a complete conceptual model of a (necessarily limited) discourse world is emphasized. It is suggested to introduce a ‘generically describable situation’ as a new conceptual unit for the computer-internal representation of more complex discourse worlds. References to other approaches for the extraction of descriptions from images and image sequences are pointed out for comparison.

[1]  W. Enkelmann,et al.  Lernen durch Beobachtung von Szenen mit bewegten Objekten: Phasen einer Systementwicklung , 1983 .

[2]  D. McDermott.  A Temporal Logic for Reasoning About Processes and Plans , 1982, Cogn. Sci..

[3]  Heinrich Niemann,et al.  A Knowledge Based System for Analysis of Gated Blood Pool Studies , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  S Ullman,et al.  Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion , 1984, Perception.

[5]  J.F. Allen,et al.  Plans, goals, and language , 1986, Proceedings of the IEEE.

[6]  Dennis H. Klatt,et al.  Review of the ARPA speech understanding project , 1990 .

[7]  Minoru Asada,et al.  Representation of three-dimensional motion in dynamic scenes , 1983, Comput. Vis. Graph. Image Process..

[8]  Bernd Neumann,et al.  NOAS: Ein System zur natürlichsprachlichen Beschreibung zeitveränderlicher Szenen , 1986, Inform. Forsch. Entwickl..

[9]  Norman I. Badler,et al.  Temporal scene analysis: conceptual descriptions of object movements. , 1975 .

[10]  Minoru Asada,et al.  Analysis of three-dimensional motions in blocks world , 1984, Pattern Recognit..

[11]  Hans-Hellmut Nagel,et al.  Recent advances in motion interpretation based on image sequences , 1982, ICASSP '82. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[12]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Hans-Hellmut Nagel,et al.  New likelihood test methods for change detection in image sequences , 1984, Comput. Vis. Graph. Image Process..

[14]  Hsi-Jian Lee,et al.  Determination of 3D human body postures from a single view , 1985, Comput. Vis. Graph. Image Process..

[15]  Norihiro Abe,et al.  Toward a Learning of Object Models Using Analogical Objects and Verbal Instruction , 1982, AAAI.

[16]  Alex Pentland,et al.  Perceptual Organization and the Representation of Natural Form , 1986, Artif. Intell..

[17]  Badler,et al.  Techniques for Generating the Goal-Directed Motion of Articulated Structures , 1982, IEEE Computer Graphics and Applications.

[18]  Hans-Joachim Novak,et al.  Generating a Coherent Text Describing a Traffic Scene , 1986, COLING.

[19]  Ruzena Bajcsy,et al.  LandScan: A Natural Language and Computer Vision System for Analyzing Aerial Images , 1985, IJCAI.

[20]  Ingrid M. Walter.  Ein Datenmodell für die Extraktion von Episoden aus Bildfolgen , 1986, DAGM-Symposium.

[21]  James F. Allen.  Towards a General Theory of Action and Time , 1984, Artif. Intell..

[22]  Hans-Hellmut Nagel,et al.  Overview on Image Sequence Analysis , 1983 .

[23]  Wolfgang Wahlster,et al.  Natürlichsprachliche Argumentation in Dialogsystemen , 1981 .

[24]  Roger C. Schank,et al.  Computer Models of Thought and Language , 1974 .

[25]  Bernd Neumann,et al.  Natural Language Dialogue about Moving Objects in an Automatically Analyzed Traffic Scene , 1981, IJCAI.

[26]  Narendra Ahuja,et al.  3-D Motion Estimation, Understanding, and Prediction from Noisy Image Sequences , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  J. O'Rourke,et al.  Model-based image analysis of human motion using constraint propagation , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  H.-H. Nagel,et al.  Principles of (low-level) computer vision , 1987 .

[29]  Koichiro Akita,et al.  Image sequence analysis of real world human motion , 1984, Pattern Recognit..

[30]  Jane Wilhelms,et al.  Toward Automatic Motion Control , 1987, IEEE Computer Graphics and Applications.

[31]  Peter C. Lockemann,et al.  Database Support for Knowledge-Based Image Evaluation , 1987, VLDB.

[32]  W.A. Woods,et al.  Important issues in knowledge representation , 1986, Proceedings of the IEEE.

[33]  J. O'Rourke,et al.  A spherical representation of a human body for visualizing movement , 1979, Proceedings of the IEEE.

[34]  Gary A. Crocker.  Screen-Area Coherence for Interactive Scanline Display Algorithms , 1987, IEEE Computer Graphics and Applications.

[35]  Zeltzer,et al.  Motor Control Techniques for Figure Animation , 1982, IEEE Computer Graphics and Applications.

[36]  Yee-Hong Yang,et al.  Human body motion segmentation in a complex scene , 1987, Pattern Recognit..

[37]  A. Pentland.  Recognition by Parts , 1987 .

[38]  Hans-Hellmut Nagel,et al.  On the Estimation of Optical Flow: Relations between Different Approaches and Some New Results , 1987, Artif. Intell..

[39]  Hans-Hellmut Nagel,et al.  Formation of an object concept by analysis of systematic time variations in the optically perceptible environment , 1978 .

[40]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[41]  Norman I. Badler,et al.  Digital Representations of Human Movement , 1979, CSUR.

[42]  Yee-Hong Yang,et al.  A region based approach for human body motion analysis , 1987, Pattern Recognit..

[43]  R. Bajcsy,et al.  Three dimensional object representation revisited , 1987 .

[44]  H. Nagel,et al.  On the Selection of Critical Points and Local Curvature Extrema of Region Boundaries for Interframe Matching , 1983 .

[45]  Robert Lake,et al.  Near-Real-Time Control of Human Figure Models , 1987, IEEE Computer Graphics and Applications.

[46]  James F. Allen,et al.  A formal logic of plans in temporally rich domains , 1986, Proceedings of the IEEE.

[47]  Yoshiaki Shirai,et al.  Detection of the movements of persons from a sparse sequence of TV images , 1983, Pattern Recognition.

[48]  John K. Tsotsos,et al.  A framework for visual motion understanding , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Hans-Hellmut Nagel,et al.  Displacement vectors derived from second-order intensity variations in image sequences , 1983, Comput. Vis. Graph. Image Process..

[50]  F. Giunchiglia,et al.  Reasoning about scene descriptions , 1986, Proceedings of the IEEE.

[51]  Dana H. Ballard,et al.  Task frames: Primitives for sensory-motor coordination , 1985, Comput. Vis. Graph. Image Process..

[52]  Georg Zimmermann,et al.  Detektion und Verfolgung mehrerer Objekte in Bildfolgen , 1986, DAGM-Symposium.

[53]  D. Marr,et al.  Representation and recognition of the movements of shapes , 1982, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[54]  H J Woltring,et al.  Planar control in multi-camera calibration for 3-D gait studies. , 1980, Journal of biomechanics.

[55]  Chapman,et al.  Aspects of the Kinematic Simulation of Human Movement , 1982, IEEE Computer Graphics and Applications.

[56]  Hans-Hellmut Nagel,et al.  Toward the derivation of three-dimensional descriptions from image sequences for nonconvex moving objects , 1986, Comput. Vis. Graph. Image Process..

[57]  Norman I. Badler,et al.  Articulated Figure Positioning by Multiple Constraints , 1987, IEEE Computer Graphics and Applications.

[58]  Hans-Hellmut Nagel,et al.  Volumetric model and 3D trajectory of a moving car derived from monocular TV frame sequences of a street scene , 1981, Comput. Graph. Image Process..

[59]  Michael R. Lowry,et al.  Learning Physical Descriptions From Functional Definitions, Examples, and Precedents , 1983, AAAI.

[60]  Ronald J. Brachman,et al.  An overview of the KL-ONE Knowledge Representation System , 1985 .

[61]  Yoav Shoham,et al.  Temporal Logics in AI: Semantical and Ontological Considerations , 1987, Artif. Intell..

[62]  Roger C. Schank,et al.  SCRIPTS, PLANS, GOALS, AND UNDERSTANDING , 1988 .

[63]  Jane Wilhelms,et al.  Using Dynamic Analysis for Realistic Animation of Articulated Bodies , 1987, IEEE Computer Graphics and Applications.