SignQUOTE: A Remote Testing Facility for Eliciting Signed Qualitative Feedback

Sign synthesis is still an evolving technology and improving it requires the elicitation of qualitative feedback from users. Current options for acquiring qualitative feedback are limited. Face-toface tests conducted in sign language are expensive. On the other hand, remote tests do not use the preferred language of the test participants. A new tool, SignQUOTE, (Signed Qualitative Usability Online Testing Environment) is a configurable, crossplatform remote testing system based entirely on sign language. It includes an innovative method for capturing qualitative feedback in sign language via webcam. In a comparison study, participants viewed animations of American Sign Language and gave suggestions for improvement. The authors conducted a study using SignQUOTE that presented stimuli identical to those used in a previously-conducted face-to-face study. When comparing the two approaches, the authors found results that are consistent with previous comparison studies of remote and face-to-face testing. SignQUOTE comes with a TestDesigner that allows researchers to customize tests. The software is available as open source.

[1]  Maria Ebling,et al.  On the contributions of different empirical data in usability testing , 2000, DIS '00.

[2]  Thomas S. Tullis,et al.  An Empirical Comparison of Lab and Remote Usability Testing of Web Sites , 2002 .

[3]  Ceil Lucas,et al.  Linguistics of American Sign Language: An Introduction , 1995 .

[4]  Charlotte Lee Baker-Shenk,et al.  American Sign Language : A Teacher's Resource Text on Grammar and Culture , 1991 .

[5]  Loraine J. DiPietro.  Registry of Interpreters for the Deaf. , 1970 .

[6]  Jean Scholtz,et al.  Adaptation of traditional usability testing methods for remote testing , 2001, Proceedings of the 34th Annual Hawaii International Conference on System Sciences.

[7]  Helen Petrie,et al.  Remote usability evaluations With disabled people , 2006, CHI.

[8]  Rosalee Wolfe,et al.  Synthetic Corpora: A Synergy of Linguistics and Computer Animation , 2010 .

[9]  H. Rex Hartson,et al.  Developing user interfaces: ensuring usability through product & process , 1993 .

[10]  Nanette M. Hogg,et al.  Computer-Mediated Communication and the Gallaudet University Community: A Preliminary Report , 2008, American annals of the deaf.

[11]  Anne R. Haake,et al.  Here, there, anywhere: remote usability testing that works , 2004, CITC5 '04.

[12]  Jerry Schnepp,et al.  Improving deaf accessibility in remote usability testing , 2011, ASSETS '11.

[13]  Alexis Héloir,et al.  Sign Language Avatars: Animation and Comprehensibility , 2011, IVA.

[14]  Jeffrey Heer,et al.  WebQuilt : A Proxy-based Approach to Remote Web Usability Testing , 2001 .

[15]  Carol Erting,et al.  Deafness & Literacy: Why Can’t Sam Read? , 2013 .

[16]  Rosalee Wolfe,et al.  Automatic verb agreement in computer synthesized depictions of american sign language , 2005 .

[17]  Sarah Waterson,et al.  In the lab and out in the wild: remote web usability testing for mobile devices , 2002, CHI Extended Abstracts.

[18]  Jan Stage,et al.  What happened to remote usability testing?: an empirical study of three methods , 2007, CHI.

[19]  Timothy G. Gregoire,et al.  Analysis of ordinal data to detect population differences. , 1987 .