Hacking Nintendo Wii to paint virtual graffiti

Any approach to the issues of virtual reality implies a general question about how to effectively plunge inside it. Despite a great variety of settings, the more feedbacks are the same as real ones, the more users feel involved on a likely situation. We can split such an assumption into the necessity of effective gestural recognition and likelihood of virtual tools to reality. In this paper we will present a system to draw virtual graffiti on the PC screen, as it were a wall. A virtual, but tangible aerosol bomb has been designed and built, by exploiting the Nintendo console gaming device and sensor system. Our object has the same shape and the same way of use as a common, real spray paint dispenser and it can be used on quite every available graphics application on a Desktop or notebook case. Besides the advantages due to the low costs and the good accuracy of gesture recognition, our system confirms the importance of similarity in simulated scenarios.

[1]  William T. Freeman,et al.  Television control by hand gestures , 1994 .

[2]  Olac Fuentes,et al.  The virtual tool approach to dextrous telemanipulation , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[3]  Steven D. Pieper,et al.  Hands-on interaction with virtual environments , 1989, UIST '89.

[4]  Danna Voth Evolutions in Gaming , 2007, IEEE Pervasive Computing.

[5]  Alan Wexelblat,et al.  An approach to natural gesture in virtual environments , 1995, TCHI.

[6]  Robert O. Ambrose,et al.  Telepresence control of the NASA/DARPA robonaut on a mobility platform , 2004, CHI.

[7]  D.J. Reinkensmeyer,et al.  A low cost parallel robot and trajectory optimization method for wrist and forearm rehabilitation using the Wii , 2008, 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[8]  R.S. Leder,et al.  Nintendo Wii remote for computer simulated arm and wrist therapy in stroke survivors with upper extremity hemipariesis , 2008, 2008 Virtual Rehabilitation.

[9]  Ivan E. Sutherland,et al.  Sketchpad: a man-machine graphical communication system , 1899, AFIPS '63 (Spring).

[10]  Johnny Chung Lee,et al.  Hacking the Nintendo Wii Remote , 2008, IEEE Pervasive Computing.

[11]  Dennis R. Wixon,et al.  CHI '02 Extended Abstracts on Human Factors in Computing Systems , 2002, CHI 2002.

[12]  Carla Diana (P)REVIEWA kiss is just a kiss; a sigh is just a deselection: a review of Designing Gestural Interfaces , 2009, INTR.

[13]  Francis K. H. Quek,et al.  Toward a vision-based hand gesture interface , 1994 .

[14]  Xilin Chen,et al.  Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks , 2003, ICMI '03.

[15]  Philip R. Cohen,et al.  QuickSet: multimodal interaction for distributed applications , 1997, MULTIMEDIA '97.

[16]  Maria Karam,et al.  A framework for research and design of gesture-based human-computer interactions , 2006 .

[17]  Andy Cockburn,et al.  Gesture navigation: an alternative 'back' for the future , 2002, CHI Extended Abstracts.