Prototyping 3D haptic data visualizations

Haptic devices are becoming more widely used as hardware becomes available and the cost of both low and high fidelity haptic devices decreases. One of the application areas of haptics is haptic data visualization (HDV). HDV provides functionality by which users can feel and touch data. Blind and partially sighted users can benefit from HDV, as it helps them manipulate and understand information. However, developing any 3D haptic world is difficult, time-consuming and requires skilled programmers. Therefore, systems that enable haptic worlds to be rapidly developed in a simple environment could enable non-computer skilled users to create haptic 3D interactions. In this article we present HITPROTO: a system that enables users, such as mentors or support workers, to quickly create haptic interactions (with an emphasis on HDVs) through a visual programming interface. We describe HITPROTO and include details of the design and implementation. We present the results of a detailed study using postgraduate students as potential mentors, which provides evidence of the usability of HITPROTO. We also present a pilot study of HITPROTO with a blind user. It can be difficult to create prototyping tools and support 3D interactions, therefore we present a detailed list of 'lessons learnt' that provides a set of guidelines for developers of other 3D haptic prototyping tools.

[1]  Frederick P. Brooks,et al.  Project GROPEHaptic displays for scientific visualization , 1990, SIGGRAPH.

[2]  Abdulmotaleb El-Saddik,et al.  HAMLAT: A HAML-Based Authoring Tool for Haptic Application Development , 2008, EuroHaptics.

[3]  Linda Sheppard,et al.  Tactile graphics: A beginner's guide to graphics for visually impaired children , 2000 .

[4]  B. H. McCormick,et al.  Visualization in scientific computing , 1995 .

[5]  A. El Saddik,et al.  MPEG-7 Description of Haptic Applications Using HAML , 2006, 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006).

[6]  Jonathan C. Roberts Visualization equivalence for multisensory perception: learning from the visual , 2004, Computing in Science & Engineering.

[7]  Ricardo S. Avila,et al.  A haptic interaction method for volume visualization , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[8]  Jeffrey Heer,et al.  Software Design Patterns for Information Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[9]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[10]  Miriam Reiner,et al.  Seeing Through Touch: The Role of Haptic Information in Visualization , 2008 .

[11]  Virginia Ann Johnson,et al.  State-of-the-art Survey , 2022 .

[12]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[13]  Janick Bergeron High-Level Modeling , 2006 .

[14]  K E Barner,et al.  Design of a haptic data visualization system for people with visual impairments. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[15]  M. Petre,et al.  Usability Analysis of Visual Programming Environments , 1996 .

[16]  B. Thomas,et al.  Usability Evaluation In Industry , 1996 .

[17]  Jean Vanderdonckt,et al.  An open source workbench for prototyping multimodal interactions based on off-the-shelf heterogeneous components , 2009, EICS '09.

[18]  Meredith Ringel Morris,et al.  iStuff: a physical user interface toolkit for ubiquitous computing environments , 2003, CHI '03.

[19]  Jörn Diedrichsen,et al.  Integration of vision and haptics during tool use. , 2009, Journal of vision.

[20]  Anders Ynnerman,et al.  General proxy-based haptics for volume visualization , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[21]  Jonathan C. Roberts,et al.  Virtual haptic exploratory visualization of line graphs and charts , 2002, IS&T/SPIE Electronic Imaging.

[22]  Matthew O. Ward,et al.  XmdvTool: integrating multiple methods for visualizing multivariate data , 1994, Proceedings Visualization '94.

[23]  Jonathan C. Roberts Visualization equivalence for multisensory perception: learning from the visual , 2004 .

[24]  Jeffrey Rubin,et al.  Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests , 1994 .

[25]  Karin Coninx,et al.  High-Level Modeling of Multimodal Interaction Techniques Using NiMMiT , 2007, J. Virtual Real. Broadcast..

[26]  K. Tuer,et al.  A new design paradigm for the rapid development of haptic and telehaptic applications , 2005, Proceedings of 2005 IEEE Conference on Control Applications, 2005. CCA 2005..

[27]  James Floyd Kelly LEGO MINDSTORMS NXT-G Programming Guide , 2007 .

[28]  Jonathan C. Roberts,et al.  Haptic Interaction Techniques for Exploring Chart Data , 2009, HAID.

[29]  Abderrahmane Kheddar,et al.  Tactile interfaces: a state-of-the-art survey , 2004 .

[30]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[31]  Jonathan C. Roberts,et al.  HITPROTO: a tool for the rapid prototyping of haptic interactions for haptic data visualization , 2010, 2010 IEEE Haptics Symposium.

[32]  J.-O. Gurinder The Swedish library of talking books and Braille (TPB) , 1985 .

[33]  Jonathan C. Roberts,et al.  Using All Our Senses : the need for a Unified Theoretical Approach to Multi-sensory Information Visualization , 2010 .

[34]  John J. Bertin,et al.  The semiology of graphics , 1983 .

[35]  Baoxin Li,et al.  Fast and independent access to map directions for people who are blind , 2012, Interact. Comput..

[36]  Soledad Ballesteros,et al.  Touch and Blindness: Psychology and Neuroscience , 2005 .

[37]  Daniel C. Halbert,et al.  Programming by Example , 2010, Encyclopedia of Machine Learning.

[38]  Stephen A. Brewster,et al.  Multimodal virtual reality versus printed medium in visualization for blind people , 2002, Assets '02.

[39]  Jonathan C. Roberts,et al.  Where are we with Haptic Visualization? , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[40]  El Saddik,et al.  The Potential of Haptics Technologies , 2007, IEEE Instrumentation & Measurement Magazine.

[41]  Sebastian Boring,et al.  The HapticTouch toolkit: enabling exploration of haptic interactions , 2012, Tangible and Embedded Interaction.

[42]  Jonathan C. Roberts,et al.  Integration of haptics with Web3D using the SAI , 2009, Web3D '09.

[43]  Gitte Lindgaard,et al.  Usability testing: what have we overlooked? , 2007, CHI.

[44]  Thomas Hermann,et al.  TAXONOMY AND DEFINITIONS FOR SONIFICATION AND AUDITORY DISPLAY , 2008 .

[45]  Ben Shneiderman,et al.  Designing The User Interface , 2013 .

[46]  J. Millis,et al.  THE UNIVERSITY OF , 2000 .

[47]  T P Way,et al.  Automatic visual to tactile translation--Part I: Human factors, access methods, and image manipulation. , 1997, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[48]  Pierre Dragicevic,et al.  Support for input adaptability in the ICON toolkit , 2004, ICMI '04.

[49]  Terri L. Bonebright,et al.  Sonific ation Report: Status of the Field and Research Agenda , 2010 .

[50]  Jonathan C. Roberts,et al.  Review of Designs for Haptic Data Visualization , 2010, IEEE Transactions on Haptics.