The WOZ Recognizer: A Tool For Understanding User Perceptions of Sketch-Based Interfaces

Sketch recognition has the potential to be an important input method for computers in the coming years; however, designing and building an accurate and sophisticated sketch recognition system is a time consuming and daunting task. Since sketch recognition is still at a level where mistakes are common, it is important to understand how users perceive and tolerate recognition errors and other user interface elements with these imperfect systems. However, the problem in performing this type of research is that we cannot easily control aspects of recognition in order to rigorously study the systems. We performed a study examining user perceptions of three pen-based systems for creating logic gate diagrams: a sketch-based interface, a WIMP-based interface, and a hybrid interface that combined elements of sketching and WIMP. We found that users preferred the sketch-based interface and identified important criteria for pen-based application design. However, this work exposed the issue of studying recognition systems without fine-grained control over accuracy. In order to solve this problem, we developed a Wizard of Oz sketch recognition tool, the WOZ Recognizer, that supports controlled symbol and position accuracy and batch and real-time recognition modes for a variety of sketching domains. We present the design of the WOZ Recognizer, modeling recognition domains using graphs and symbol alphabets, and discuss the types of recognition errors we included in its design. Further, we discuss how sketches are altered, controlling the WOZ Recognizer, and how users interact with it. In addition, we present an evaluative user study of the WOZ Recognizer and the lessons we learned.