Dimensions for the design and evaluation of multimodal videogames for the cognition of people who are blind

Multimodal serious video games are relevant tools to enhance the cognitive skills of people who are blind. For this purpose, it is necessary that designers and developers be able to create interfaces and interactions using the multimodal elements properly. Thus, there is a need to know the relevant elements for such applications, as well as to understand their roles and relationships. In this paper, we propose and discuss a 4-dimension classification: Interface, Interaction, Cognition, and Evaluation. Such classification was assembled from the features related to the design and evaluation of 21 multimodal video games and environments, identified via a bibliographic review based on the systematic review approach. Besides, we classify and discuss the 21 multimodal applications into the proposed classification.

[1]  Abdulmotaleb El-Saddik,et al.  Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications , 2012, Multimedia Tools and Applications.

[2]  Jaime Sánchez,et al.  Three-Dimensional Virtual Environments for Blind Children , 2006, Cyberpsychology Behav. Soc. Netw..

[3]  Jaime Sánchez,et al.  Designing Serious Videogames through Concept Maps , 2013, HCI.

[4]  David Mioduser,et al.  Construction of cognitive maps of unknown spaces using a multi-sensory virtual environment for people who are blind , 2008, Comput. Hum. Behav..

[5]  Rachel McCrindle,et al.  Audio space invaders , 2000 .

[6]  Jaime Sánchez,et al.  Action video game play and transfer of navigation and spatial cognition skills in adolescents who are blind , 2014, Front. Hum. Neurosci..

[7]  Jaime Sánchez,et al.  Modeling Videogames for Mental Mapping in People Who Are Blind , 2014, HCI.

[8]  Jaime Sánchez,et al.  3D sound interactive environments for blind children problem solving skills , 2006, Behav. Inf. Technol..

[9]  David W. Schloerb,et al.  BlindAid: A learning environment for enabling people who are blind to explore and navigate through unknown real spaces , 2008, 2008 Virtual Rehabilitation.

[10]  Iván Martínez-Ortiz,et al.  Implementing accessibility in educational videogames with , 2009, MTDL '09.

[11]  Jaime Sánchez,et al.  Enhancing orientation and mobility skills in learners who are blind through video gaming , 2013, Creativity & Cognition.

[12]  Baltasar Fernández-Manjón,et al.  Towards a low cost adaptation of educational games for people with disabilities , 2014, Comput. Sci. Inf. Syst..

[13]  V. Goel,et al.  Towards a Theory of Thinking , 2010 .

[14]  Petra Kaufmann,et al.  Experimental And Quasi Experimental Designs For Research , 2016 .

[15]  Brian Samways,et al.  Learning to Live in the Knowledge Society, IFIP 20th World Computer Congress, IFIP TC 3 ED-L2L Conference, September 7-10, 2008, Milano, Italy , 2008, Learning to Live in the Knowledge Society.

[16]  Clarisse Sieckenius de Souza,et al.  An exploratory study on non-visual mobile phone interfaces for games , 2008, IHC.

[17]  Jaime Sánchez,et al.  Audio haptic videogaming for developing wayfinding skills in learners who are blind , 2014, IUI.

[18]  Kai Petersen,et al.  Systematic Mapping Studies in Software Engineering , 2008, EASE.

[19]  Bei Yuan,et al.  Blind hero: enabling guitar hero for the visually impaired , 2008, Assets '08.

[20]  Alvaro Pascual-Leone,et al.  Blind children navigation through gaming and associated brain plasticity , 2009, 2009 Virtual Rehabilitation International Conference.

[21]  N. M. Morris,et al.  On Looking into the Black Box: Prospects and Limits in the Search for Mental Models , 1986 .

[22]  Thomas Westin,et al.  Game accessibility case study : Terraformers - a real-time 3D graphic game , 2004 .

[23]  Luis A. Guerrero,et al.  A Model to Develop Videogames for Orientation and Mobility , 2010, ICCHP.

[24]  Hong Z. Tan,et al.  HUMAN FACTORS FOR THE DESIGN OF FORCE-REFLECTING HAPTIC INTERFACES , 1994 .

[25]  Christian Freksa,et al.  Spatial and Temporal Structures in Cognitive Processes , 1997, Foundations of Computer Science: Potential - Theory - Cognition.

[26]  Jaime Sánchez.  A model to design interactive learning environments for children with visual disabilities , 2007, Education and Information Technologies.

[27]  Jaime Sánchez,et al.  Interactive 3D sound hyperstories for blind children , 1999, CHI '99.

[28]  Bei Yuan,et al.  Towards generalized accessibility of video games for the visually impaired , 2009 .

[29]  Jaime Sánchez,et al.  Development of Navigation Skills through Audio Haptic Videogaming in Learners who are Blind , 2013, J. Univers. Comput. Sci..

[30]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[31]  Jatinder Singh,et al.  Mendeley: A free research management tool for desktop and web , 2010, Journal of pharmacology & pharmacotherapeutics.

[32]  Mauricio Sáenz,et al.  Enhancing navigation skills through audio gaming , 2010, CHI EA '10.

[33]  David Mioduser,et al.  Haptic-feedback support for cognitive mapping of unknown spaces by people who are blind , 2008, Int. J. Hum. Comput. Stud..

[34]  Jaime Sánchez,et al.  Audiopolis, Navigation through a Virtual City Using Audio and Haptic Interfaces for People Who Are Blind , 2011, HCI.

[35]  Jaime Sánchez,et al.  Usability of a Multimodal Video Game to Improve Navigation Skills for Blind Children , 2010, TACC.

[36]  Jaime Sánchez,et al.  Virtual Mobile Science Learning for Blind People , 2008, Cyberpsychology Behav. Soc. Netw..

[37]  Jaime Sánchez,et al.  Audio and haptic based virtual environments for orientation and mobility in people who are blind , 2010, ASSETS '10.

[38]  Jaime Sánchez,et al.  AudioGene: Mobile Learning Genetics through Audio by Blind Learners , 2008, Learning to Live in the Knowledge Society.

[39]  Jaime Sánchez,et al.  Video Gaming for Blind Learners School Integration in Science Classes , 2009, INTERACT.

[40]  Cleiton Silva,et al.  Managing Literature Reviews Information through Visualization , 2018, ICEIS.

[41]  Mandayam A. Srinivasan,et al.  Virtual Environment System in Support of a Traditional Orientation and Mobility Rehabilitation Program for People Who Are Blind , 2013, PRESENCE: Teleoperators and Virtual Environments.

[42]  Vicki L. Hanson,et al.  PowerUp: an accessible virtual world , 2008, Assets '08.

[43]  Jaime Sánchez,et al.  Multimodal Videogames for the Cognition of People Who Are Blind: Trends and Issues , 2015, HCI.

[44]  Jaime Sánchez,et al.  Videogaming Interaction for Mental Model Construction in Learners Who Are Blind , 2014, HCI.

[45]  Jaime Sánchez,et al.  Metro navigation for the blind , 2010, Comput. Educ..

[46]  J. Guerrero,et al.  AINIDIU, CANDI, HELPMI: ICTs of a personal experience , 2012, 2012 Workshop on Engineering Applications.

[47]  Ernest A. Edmonds,et al.  AUXie: initial evaluation of a blind-accessible virtual museum tour , 2010, OZCHI '10.

[48]  Jaime Sánchez,et al.  Multimodal gaming for navigation skills in players who are blind , 2014, IHC.