Object manipulation and layout in a 3D virtual space using a combination of natural language and hand pointing

We propose a 3D object manipulation and layout technique in a 3D virtual space using a combination of an abstract natural language and hand pointing to recognize the purpose of a manipulator. This technique quantitatively transforms verbal semantics into a spatial region that uses the probability functions corresponding to some indicator word. It also uses knowledge about each object, e.g. the back of the bookshelf must be attached to the wall. And an operation will be performed with the indication from a natural language and a hand movement which is sensed using a 3D position tracker and Data GloveTM to dissolve the ambiguous selection of a candidate in a cooperative work space.