Non-visual natural user interfaces

Natural user interfaces (NUI) have recently become popular due to their characteristics that capitalize on a user's ability to utilize skills acquired through real world experiences. NUI's provide a method of device interaction that may be easier when compared to a traditional graphical user interfaces because users can interact by using gestures and motions which mimic motions that are used outside of device interaction. One example of NUI's is the recent trend in exergames - video games that require actions such as running in place, moving the arm in a bowling motion, or swinging the arm as if to swing a tennis racquet. Unfortunately, most of the cues given to the player to perform these natural motions are visual cues which makes it difficult for people who have visual impairments to participate in these activities. This dissertation investigates non-visual natural user interfaces. Three exergames were created to show techniques for using a combination of haptic and audio cues in order to promote physical activity for people who are visually impaired. A method, Real Time Sensory Substitution, was developed which allowed people who are visually impaired to participate in a commercially available exergame by introducing additional haptic cues. The exergames and Real Time Sensory Substitution were effective in promoting physical activity for temporal based challenges, however they lacked the information to assist users in complex spatial challenges. Techniques were developed and compared using proprioception (the body's ability to sense its own position) to assist users in finding targets in one, two, and three dimensions without using any visual cues. Proprioception was also used as a cell phone interface where the phone can convey information to the user without using the graphical display of the phone, but using the human body as the display mechanism. The techniques developed in these studies have set the stage for enhanced access to technology without the use of a video display.