Flying Cities: building a 3D world from vocal input

Abstract Flying Cities is an artistic installation which generates imaginary cities from the speech of its visitors. Thanks to an original interactive process analysing people's vocal input to create 3D graphics, a tangible correspondence between speech and visuals opens new possibilities of interaction. This cross-modal interaction not only supports our artistic messages, but also aims at providing anyone with a pleasant and stimulating feedback from her/his speech activity. As the feedback we have received when presenting Flying Cities was very positive, our objective is now to cross the bridge between art and a potential application. We believe that it could become a new medium for creativity and a way to visually perceive a vocal performance in the context of the rehabilitation of people with reduced mobility or language impairments.

[1]  Sun-Hyung Park,et al.  Integrated speech training system for hearing impaired , 1994 .

[2]  Daniel Thalmann.  The virtual human as a multimodal interface , 2000, AVI '00.

[3]  Ryohei Nakatsu,et al.  “Interactive poem system” , 1998, MULTIMEDIA '98.

[4]  Agnes Sola.  Des nombres et des lettres , 1987 .

[5]  Mel Slater,et al.  Measuring Presence: A Response to the Witmer and Singer Presence Questionnaire , 1999, Presence.

[6]  Voula C. Georgopoulos.  An investigation of audio-visual speech recognition as applied to multimedia speech therapy applications , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[7]  Dirk Heylen,et al.  Inhabited interfaces: Attentive conversational agents that help , 2000 .

[8]  Marcel Locquin,et al.  Quelle langue parlaient nos ancêtres préhistoriques , 2002 .

[9]  Richard A. Bolt,et al.  “Put-that-there”: Voice and gesture at the graphics interface , 1980, SIGGRAPH '80.

[10]  Anthony Lewis-Brooks,,et al.  Creating aesthetically resonant environments for the handicapped, elderly and rehabilitation: Sweden , 2005 .

[11]  Roy S. Kalawsky,et al.  The Validity of Presence as a Reliable Human Performance Metric in Immersive Environments , 2000 .

[12]  K Vicsi,et al.  Distinctive training methods and evaluation of a multilingual, multimodal speech training system , 2002 .

[13]  Ernest A. Edmonds,et al.  Approaches to interactive art systems , 2004, GRAPHITE '04.

[14]  Abderrahmane Kheddar,et al.  Pseudo-haptic feedback: can isometric input devices simulate force feedback? , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[15]  Myron W. Krueger,et al.  VIDEOPLACE—an artificial reality , 1985, CHI '85.

[16]  Alain Désilets,et al.  VoiceCode: an innovative speech interface for programming-by-voice , 2006, CHI Extended Abstracts.

[17]  Alexander I. Rudnicky,et al.  Towards efficient human machine speech communication: The speech graffiti project , 2005, TSLP.

[18]  Tony Brooks.  Creating Aesthetic Resonant Environments for the Handicapped, Elderly and Rehabilitation: Sweden , 2004 .

[19]  Alexander I. Rudnicky,et al.  Universal speech interfaces , 2001, INTR.

[20]  S. E. Hutchins.  SAY & SEE articulation therapy software , 1992, Proceedings of the Johns Hopkins National Search for Computing Applications to Assist Persons with Disabilities.