In this research, a dialog environment between human and virtual environment has been constructed. At preset, if a person wants to interact with virtual environment, special device such a data glove is required, but it makes difficult for general users to manipulate virtual objects. When we cannot manipulate objects directly, it is natural that we ask someone with a privilege to do the operation in place of us. In the case, it is convenient if methods used in daily life are allowed. That is, the method to be proposed here is based on the integration between verbal information through the utterances and non-verbal information by the gestures such as finger pointing. The experimental results have proved the effectiveness of this approach in terms of facilitating man-machine interaction and communication. The environment constructed in this research allows a user to communicate by talking and showing gestures to a personified agent in virtual environment. A user can use his/her finger to point at a virtual object and ask him to manipulate it.
[1]
Tsuji Saburo,et al.
A Consulting System Which Detects And Undoes Erroneous Operation By Novices
,
1987,
Other Conferences.
[2]
Jiang Yu Zheng,et al.
A training system using virtual machines for teaching assembling/disassembling operation to novices
,
1996,
1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).
[3]
Kazuaki Tanaka,et al.
Verbal/nonverbal communication between man and Avatar in virtual mechanical assembly training system
,
2003,
International Conference On Virtual Reality and Its Applications in Industry.