Interactive 3-D computer graphics driven through verbal instructions: Previous and current activities at ART

Abstract Interaction in 3-D computer graphics is usually accomplished through input devices such as the mouse, light-pens, touch-pads, and recently through data-gloves ‡ , among others. Since these input devices provide only low-level interaction (e.g., pointing, selecting, dragging, etc.), they are usually complemented with higher-level commands and macros available through selectable icons, keyboard input, pull-down or pop-up menus and, in the case of data-gloves, with hand gestures. This paper describes a higher level of interaction that allows verbal instructions to be combined with 3-D input devices for generating, manipulating or modifying 3-D shapes. The paper reviews previous work at ATR on this type of interaction and describes a current effort to index words to concepts, at the knowledge level[1, 2], and from concepts to a symbolic-level representation based on deformable superquadrics[3–5].

[1]  Ramanathan V. Guha,et al.  Cyc: toward programs with common sense , 1990, CACM.

[2]  Fumio Kishino,et al.  Real time hand shape recognition for man-machine interfaces , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[3]  Alexander G. Hauptmann,et al.  Speech and gestures for graphic image manipulation , 1989, CHI '89.

[4]  M. Wertheimer.  Laws of organization in perceptual forms. , 1938 .

[5]  Tsutomu Miyasato,et al.  Ontology-Based Approach for Interactive Virtual Object Generation , 1993 .

[6]  D'arcy W. Thompson.  On growth and form i , 1943 .

[7]  H. Keith Nishihara,et al.  Intensity, Visible-Surface, and Volumetric Representations , 1981, Artif. Intell..

[8]  G. Kelly.  The Psychology of Personal Constructs , 2020 .

[9]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[10]  Alex Pentland,et al.  Perceptual Organization and the Representation of Natural Form , 1986, Artif. Intell..

[11]  Stephen R. Ellis,et al.  Using virtual menus in a virtual environment , 1992, Electronic Imaging.

[12]  Dimitris N. Metaxas,et al.  Dynamic 3D models with local and global deformations: deformable superquadrics , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[13]  Fumio Kishino,et al.  Real time hand shape recognition using pipe-line image processor , 1992, [1992] Proceedings IEEE International Workshop on Robot and Human Communication.

[14]  Fumio Kishino,et al.  Real time hand gesture recognition using 3D prediction model , 1993, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[15]  Tomoichi Takahashi,et al.  Scene Description Using Spatial Relationships Derived From Visual Information , 1990, Other Conferences.

[16]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[17]  H. Kasahara,et al.  3-D shape indexing language , 1990, Ninth Annual International Phoenix Conference on Computers and Communications. 1990 Conference Proceedings.

[18]  Fumio Kishino,et al.  Object manipulation and layout in a 3D virtual space using a combination of natural language and hand pointing , 1992, Other Conferences.

[19]  P. Stevens.  Patterns in Nature , 1974 .

[20]  Riichiro Mizoguchi,et al.  MULTIS II: Enabling End-Users to Design Problem-Solving Engines via Two-Level Task Ontologies , 1993, EKAW.

[21]  E. Rosch.  ON THE INTERNAL STRUCTURE OF PERCEPTUAL AND SEMANTIC CATEGORIES1 , 1973 .