A multimodal user interface system with force feedback and physical models

An experimental hardware and software platform aimed at enriching a graphic user interface (GUI) through the use of auditory and haptic information is described. MUIS (Multimodal User Interface System) allows for the design and creation of multimodal feedback interfaces, and includes a force-feedback device with three degrees of freedom: the Pantograph. Interface objects (windows, icons and pop-up menus) are represented in a physical structure that is well adapted to the real-time synthesis of coherent and redundant sounds and forces.