Creating Accessible Online Floor Plans for Visually Impaired Readers

We present a generic model for providing blind and severely vision-impaired readers with access to online information graphics. The model supports fully and semi-automatic transcription and allows the reader a choice of presentation mediums. We evaluate the model through a case study: online house floor plans. To do so, we conducted a formative user study with severely vision impaired users to determine what information they would like from an online floor plan and how to present the floor plan as a text-only description, tactile graphic, and on a touchscreen with audio feedback. We then built an automatic transcription tool using specialized graphics recognition algorithms. Finally, we measured the quality of system recognition as well as conducted a second user study to evaluate the usefulness of the accessible graphics produced by the tool for each of the three formats. The results generally support the design of the generic model and the usefulness of the tool we have produced. However, they also reveal the inability of current graphics recognition algorithms to handle unforeseen graphical conventions. This highlights the need for automatic transcription systems to return a level of confidence in the recognized components and to present this to the end-user so they can have an appropriate level of trust.

[1]  Oriol Ramos Terrades,et al.  CVC-FP and SGT: a new database for structural floor plan analysis and its groundtruthing tool , 2014, International Journal on Document Analysis and Recognition (IJDAR).

[2]  Anind K. Dey,et al.  Uncovering information needs for independent spatial learning for users who are visually impaired , 2013, ASSETS.

[3]  Ernest Valveny,et al.  Statistical segmentation and structural recognition for floor plan interpretation , 2013, International Journal on Document Analysis and Recognition (IJDAR).

[4]  Rainer Stiefelhagen,et al.  User Requirements Regarding Information Included in Audio-Tactile Maps for Individuals with Blindness , 2016, ICCHP.

[5]  Marcus Liwicki,et al.  Improved Automatic Analysis of Architectural Floor Plans , 2011, 2011 International Conference on Document Analysis and Recognition.

[6]  Marcus Liwicki,et al.  Automatic analysis and sketch-based retrieval of architectural floor plans , 2014, Pattern Recognit. Lett..

[7]  Rangachar Kasturi,et al.  A Robust Algorithm for Text String Separation from Mixed Text/Graphics Images , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  John Kenneth Salisbury,et al.  Phantom-Based Haptic Interaction with Virtual Objects , 1997, IEEE Computer Graphics and Applications.

[9]  Marion Hersh,et al.  Accessible Information: An Overview , 2008 .

[10]  LiwickiMarcus,et al.  Statistical segmentation and structural recognition for floor plan interpretation , 2014 .

[11]  Ingo Redeke,et al.  Image & Graphic Reader , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[12]  Simone Marinai,et al.  Object recognition in floor plans by graphs of white connected components , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[13]  Steve Pettifer,et al.  Evaluation of a non-visual molecule browser , 2003, Assets '04.

[14]  Baoxin Li,et al.  Automated description generation for indoor floor maps , 2012, ASSETS '12.

[15]  Ernest Valveny,et al.  Runlength Histogram Image Signature for Perceptual Retrieval of Architectural Floor Plans , 2013, GREC.

[16]  Timo Götzelmann,et al.  SmartTactMaps: a smartphone-based approach to support blind persons in exploring tactile maps , 2015, PETRA.

[17]  Masakazu Suzuki,et al.  Braille-Embedded Tactile Graphics Editor with Infty System , 2008, ICCHP.

[18]  Feng Hu,et al.  Automatic Pre-Journey Indoor Map Generation Using AutoCAD Floor Plan , 2016 .

[19]  Cagatay Goncu GraVVITAS: accessible graphics for visually impaired people , 2012 .

[20]  Marcus Liwicki,et al.  Improved Contour-Based Corner Detection for Architectural Floor Plans , 2013, GREC.

[21]  Baoxin Li,et al.  Towards building an automated system for describing indoor floor maps for individuals with visual impairment , 2015 .

[22]  Tong Lu,et al.  Automatic analysis and integration of architectural drawings , 2007, International Journal of Document Analysis and Recognition (IJDAR).

[23]  C. Hayes Electronics lend a helping hand to young and old [products for assisted living] , 2017 .

[24]  Fumihiko Ino,et al.  Training Strategies for CNN-based Models to Parse Complex Floor Plans , 2020, ICSCA.

[25]  Andrea R. Kennel Audiograf: a diagram-reader for the blind , 1996, Assets '96.

[26]  Eyal de Lara,et al.  Timbremap: enabling the visually-impaired to use maps on touch-enabled devices , 2010, Mobile HCI.

[27]  Gerhard Weber,et al.  Audio-Haptic Browser for a Geographical Information System , 2010, ICCHP.

[28]  Kim Marriott,et al.  Accessible Graphics: Graphics for Vision Impaired People , 2012, Diagrams.

[29]  Chin-Tau Lea,et al.  Automatic Floor Plan Analysis for Adaptive Indoor Wi-Fi Positioning System , 2016, 2016 International Conference on Computational Science and Computational Intelligence (CSCI).

[30]  Marcus Liwicki,et al.  Automatic Room Detection and Room Labeling from Architectural Floor Plans , 2012, 2012 10th IAPR International Workshop on Document Analysis Systems.

[31]  N. A. Bradley,et al.  Assistive Technology For Visually Impaired And Blind People , 2008 .

[32]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[33]  Daniel P. Siewiorek,et al.  Customizable 3D Printed Tactile Maps as Interactive Overlays , 2016, ASSETS.

[34]  Norihisa Miki,et al.  Electrotactile display composed of two-dimensionally and densely distributed microneedle electrodes , 2017 .

[35]  Volker Sorge,et al.  Chemical structure recognition: a rule-based approach , 2012, Electronic Imaging.

[36]  Sylvain Robert,et al.  Automatic reconstruction of 3D building models from scanned 2D floor plans , 2016 .

[37]  Ehud Reiter,et al.  NLG vs. Templates , 1995, ArXiv.

[38]  Hyoukryeol Choi,et al.  Development of Soft-Actuator-Based Wearable Tactile Display , 2008, IEEE Transactions on Robotics.

[39]  K.E. Barner,et al.  Visual to Tactile Conversion of Vector Graphics , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[40]  John A. Gardner,et al.  Scientific diagrams made easy with IVEO , 2006 .

[41]  Helen Petrie,et al.  Remote usability evaluations With disabled people , 2006, CHI.

[42]  Andrew G. Dempster,et al.  Indoor navigation for people who are blind or vision impaired: where are we and where are we going? , 2014, J. Locat. Based Serv..

[43]  Nizar Bouhlel,et al.  New tools for automating tactile geographic map translation , 2014, ASSETS.

[44]  Kim Marriott,et al.  GraVVITAS: Generic Multi-touch Presentation of Accessible Graphics , 2011, INTERACT.

[45]  Simone Marinai,et al.  Object Detection in Floor Plan Images , 2018, ANNPR.

[46]  Joe Marks,et al.  Semi-automatic delineation of regions in floor plans , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[47]  Kathleen F. McCoy,et al.  Interactive SIGHT: textual access to simple bar charts , 2010, New Rev. Hypermedia Multim..

[48]  Yasuhiro Aoki,et al.  A prototype system for interpreting hand-sketched floor plans , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[49]  Christian Ah-Soon,et al.  A complete system for the analysis of architectural drawings , 2000, International Journal on Document Analysis and Recognition.

[50]  Gitte Lindgaard,et al.  Improving accessibility to statistical graphs: the iGraph-Lite system , 2007, Assets '07.

[51]  Nicholas A. Giudice,et al.  Spatial Preposition Use in Indoor Scene Descriptions , 2016 .

[52]  Rob Miller,et al.  VizWiz: nearly real-time answers to visual questions , 2010, UIST.

[53]  Jochen Lang,et al.  Instrument for Haptic Image Exploration , 2014, IEEE Transactions on Instrumentation and Measurement.

[54]  Manuel Blum,et al.  Improving accessibility of the web with a computer game , 2006, CHI.

[55]  Helen Petrie,et al.  Providing interactive access to architectural floorplans for blind people , 2006 .

[56]  Gerhard Weber,et al.  ATMap: Annotated Tactile Maps for the Visually Impaired , 2011, COST 2102 Training School.

[57]  Dianne T. V. Pawluk,et al.  Automatic image conversion to tactile graphic , 2013, ASSETS.

[58]  Kim Marriott,et al.  Partitioning Open Plan Areas in Floor Plans , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[59]  Ernest Valveny,et al.  A system to detect rooms in architectural floor plan images , 2010, DAS '10.

[60]  Richard E. Ladner,et al.  Automated tactile graphics translation: in the field , 2007, Assets '07.

[61]  Kim Marriott,et al.  Accessible On-Line Floor Plans , 2015, WWW.

[62]  John A. Gardner,et al.  Scientific Diagrams Made Easy with IVEOTM , 2006, ICCHP.

[63]  T P Way,et al.  Automatic visual to tactile translation--Part II: Evaluation of the TACTile Image Creation System. , 1997, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[64]  Christoph Schlieder,et al.  TeDUB: A System for Presenting and Exploring Technical Drawings for Blind People , 2002, ICCHP.

[65]  Timo Götzelmann,et al.  Towards Automatically Generated Tactile Detail Maps by 3D Printers for Blind Persons , 2014, ICCHP.

[66]  Volker Sorge,et al.  End-to-end solution for accessible chemical diagrams , 2015, W4A.

[67]  Baoxin Li,et al.  Fast and independent access to map directions for people who are blind , 2012, Interact. Comput..

[68]  Konstantinos Papadopoulos,et al.  A University Indoors Audio-Tactile Mobility Aid for Individuals with Blindness , 2014, ICCHP.

[69]  Mireia Ribera Turró Web Content Accessibility Guidelines 1.0 , 2001 .

[70]  Juho Kannala,et al.  CubiCasa5K: A Dataset and an Improved Multi-Task Model for Floorplan Image Analysis , 2019, SCIA.

[71]  Ben Shneiderman,et al.  Data Sonification for Users with Visual Impairment: A Case Study with Georeferenced Data , 2008, TCHI.

[72]  Björn Stenger,et al.  Parsing floor plan images , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[73]  Kim Marriott,et al.  Creating eBooks with Accessible Graphics Content , 2015, DocEng.

[74]  Timothy C. Craven Some features of alt texts associated with images in Web pages , 2006, Inf. Res..

[75]  Emiel Krahmer,et al.  Squibs and Discussions: Real versus Template-Based Natural Language Generation: A False Opposition? , 2005, CL.