Linguistic interpretation of human motion based on mental image directed semantic theory

The mental image directed semantic theory (MIDST) has proposed an intermediate knowledge representation scheme based on an omnisensual mental image model. This paper presents a formal language for describing multimedia contents, L/sub md/, whose syntax and semantics are based on MIDST and its application to linguistic interpretation of human motion data obtained through a motion capture system, which we think will serve for linguistic summarization of immense amount of time-sequenced non-linguistic data such as movies.

[1]  Larry S. Davis,et al.  Labeling of human face components from range data , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Ronald W. Langacker,et al.  Concept, Image, and Symbol , 1990 .

[3]  MikioAmano,et al.  Linguistic Interpretation of Human Motion Based on a Multimedia Description Language , 2004 .

[4]  John F. Sowa,et al.  Knowledge representation: logical, philosophical, and computational foundations , 2000 .

[5]  Gian Piero Zarri,et al.  NKRL, a knowledge representation tool for encoding the ‘meaning’ of complex narrative texts , 1997, Natural Language Engineering.

[6]  Stefan Carlsson,et al.  Recognizing and Tracking Human Action , 2002, ECCV.

[7]  Guangyou Xu,et al.  Human action recognition in smart classroom , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[8]  Michael J. Black,et al.  Implicit Probabilistic Models of Human Motion for Synthesis and Tracking , 2002, ECCV.

[9]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[10]  Patrick Olivier,et al.  A Computational View of the Cognitive Semantics of Spatial Prepostions , 1994, ACL.

[11]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[12]  Masao Yokota,et al.  Multimedia description language for more intelligent networking , 2004, Proceedings. 15th International Workshop on Database and Expert Systems Applications, 2004..

[13]  Fausto Giunchiglia,et al.  NALIG: A CAD system for interior design with high level interaction capabilities , 1993, Proceedings of 1993 IEEE Conference on Tools with Al (TAI-93).

[14]  D. Hironaka,et al.  Multimedia description language for more intelligent networking , 2004 .

[15]  Masao Yokota,et al.  Cross-media translation based on mental image directed semantic theory toward more comprehensible multimedia communication , 2004, 18th International Conference on Advanced Information Networking and Applications, 2004. AINA 2004..

[16]  Masao Yokota,et al.  Conceptual Analysis and Description of Words for Color and Lightness for Grounding them on Sensory Data , 2001 .

[17]  Shuji Doshita,et al.  Reconstructing Spatial Image from Natural Language Texts , 1992, COLING.