Are We Ready to Build a System for Assisting Blind People in Tactile Exploration of Bas-Reliefs?

Nowadays, the creation of methodologies and tools for facilitating the 3D reproduction of artworks and, contextually, to make their exploration possible and more meaningful for blind users is becoming increasingly relevant in society. Accordingly, the creation of integrated systems including both tactile media (e.g., bas-reliefs) and interfaces capable of providing the users with an experience cognitively comparable to the one originally envisioned by the artist, may be considered the next step for enhancing artworks exploration. In light of this, the present work provides a description of a first-attempt system designed to aid blind people (BP) in the tactile exploration of bas-reliefs. In detail, consistent hardware layout, comprising a hand-tracking system based on Kinect® sensor and an audio device, together with a number of methodologies, algorithms and information related to physical design are proposed. Moreover, according to experimental test on the developed system related to the device position, some design alternatives are suggested so as to discuss pros and cons.

[1]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[2]  Nai Xia,et al.  Method of Registration , 2014 .

[3]  David J. Fleet,et al.  Model-based hand tracking with texture, shading and self-occlusions , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Antonis A. Argyros,et al.  Tracking the articulated motion of two strongly interacting hands , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Mircea Nicolescu,et al.  Vision-based hand pose estimation: A review , 2007, Comput. Vis. Image Underst..

[6]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Monica Carfagni,et al.  From 2D to 2.5D i.e. from painting to tactile model , 2014, Graph. Model..

[8]  Eleanor T. Loiacono-Mello,et al.  Factors that affect visually impaired users' acceptance of audio and music websites , 2013, Int. J. Hum. Comput. Stud..

[9]  Filiberto Chiabrando,et al.  Sensors for 3D Imaging: Metric Evaluation and Calibration of a CCD/CMOS Time-of-Flight Camera , 2009, Sensors.

[10]  G. N. Marichal,et al.  An Artificial Intelligence Approach for Gears Diagnostics in AUVs , 2016, Sensors.

[11]  Hsien-I Lin,et al.  Innovative Methodology for Multi-View Point Cloud Registration in Robotic 3D Object Scanning and Reconstruction , 2016 .

[12]  Jason M. O'Kane,et al.  Adaptive Robot Design with Hand and Face Tracking for Use in Autism Therapy , 2010, ICSR.

[13]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Antonis A. Argyros,et al.  Markerless and Efficient 26-DOF Hand Pose Recovery , 2010, ACCV.

[15]  Martyna Poreba,et al.  A Robust Linear Feature-Based Procedure for Automated Registration of Point Clouds , 2015, Sensors.

[16]  Luis Payá,et al.  Performance of Global-Appearance Descriptors in Map Building and Localization Using Omnidirectional Vision , 2014, Sensors.

[17]  Joost C. F. de Winter,et al.  Robust Hand Motion Tracking through Data Fusion of 5DT Data Glove and Nimble VR Kinect Camera Measurements , 2015, Sensors.

[18]  Stan Matwin,et al.  A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data , 2013, Artificial Intelligence Review.

[19]  Khairan D. Rajab,et al.  Point-Distance Computations: A Knowledge-Guided Approach , 2008 .

[20]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.

[21]  Jian Chang,et al.  A review of digital relief generation techniques , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[22]  Monica Carfagni,et al.  Tactile Representation of Paintings: An Early Assessment of Possible Computer Based Strategies , 2012, EuroMed.

[23]  Mario Cortese,et al.  Vision-Based Pose Estimation for Robot-Mediated Hand Telerehabilitation , 2016, Sensors.

[24]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[25]  Ian D. Reid,et al.  A Constant-Time Efficient Stereo SLAM System , 2009, BMVC.

[26]  Antonis A. Argyros,et al.  Full DOF tracking of a hand interacting with an object by modeling occlusions and physical constraints , 2011, 2011 International Conference on Computer Vision.

[27]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[28]  Lapo Governi,et al.  Making Blind People Autonomous in the Exploration of Tactile Models: A Feasibility Study , 2015, HCI.

[29]  Jason M. O'Kane,et al.  CHARLIE : An Adaptive Robot Design with Hand and Face Tracking for Use in Autism Therapy , 2011, Int. J. Soc. Robotics.

[30]  Christian Cipriani,et al.  Real-Time Single Camera Hand Gesture Recognition System for Remote Deaf-Blind Communication , 2014, AVR.

[31]  Aiguo Song,et al.  Design and Calibration of a New 6 DOF Haptic Device , 2015, Sensors.

[32]  Eibe Frank,et al.  An Empirical Comparison of Exact Nearest Neighbour Algorithms , 2007, PKDD.

[33]  Hans-Peter Seidel,et al.  Automatic Generation of Bas-reliefs from 3D Shapes , 2007, IEEE International Conference on Shape Modeling and Applications 2007 (SMI '07).

[34]  Wilfried Philips,et al.  Robust Arm and Hand Tracking by Unsupervised Context Learning , 2014, Sensors.

[35]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[36]  Shan Lu,et al.  Color-based hands tracking system for sign language recognition , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[37]  David Mioduser,et al.  Haptic-feedback support for cognitive mapping of unknown spaces by people who are blind , 2008, Int. J. Hum. Comput. Stud..