Extending end-user programming in a visual shell with programming by demonstration and graphical lan

This dissertation investigates the implementation and evaluation of Pursuit, a visual shell designed to enable non-programmers to construct file manipulation programs without developing expert programming knowledge. A "visual shell" (e.g., Apple Macintosh Finder) is a direct manipulation interface to a file system. Although considered easy to use, most visual shells do not support end-user programming. Pursuit explores ways to add end-user programming so that it is consistent with the direct manipulation paradigm. To enable users to construct programs, Pursuit contains a Programming by Demonstration (PBD) system. With PBD, users execute actions on real data and the system creates a general procedure. Such systems have well-known limitations: they can infer incorrectly, most contain no static representation of the inferred program, their feedback is often obscure or missing, and few provide editing facilities. This makes it difficult for users to know if the system has inferred correctly, to correct errors, and/or to change a program. To address these limitations, Pursuit explores an editable, graphical representation of the program while it is being constructed. By representing the program during the demonstration, users know immediately what the system has inferred (by observing the growing program representation) and can interactively and incrementally learn the syntax and semantics of the representation language. By allowing the program to be edited and saved, users can have an artifact to later examine, edit and share. In order to determine what effect program representation has on a user's ability to create programs, this thesis explores two representation languages. The first language employs a comic strip metaphor. Data, such as files and folders, is explicitly represented with familiar icons. Operations are implicitly represented by the changes they cause to data icons. Control constructs are represented by graphical objects (e.g., enclosing rectangles for loops, diverging lines for branches). A program is a series of operations and control constructs concatenated together. Essentially, programs are static representations of the dynamic changes to data objects. The second language has a conventional verb/argument structure. Data is represented with familiar icons. Operations consist of lines containing the operation's name followed by icons for the data objects. Control constructs are represented with keywords suggestive of their action (e.g., foreach, case) and indentation to define their scope. A program is a series of commands and control constructs listed one after another. Essentially, programs are a sequence of commands describing how data is manipulated. To evaluate the efficacy of Pursuit, we implemented a version for each representation language. PBD actions and program editing features were identical for both versions. We then performed a between-subject comparative evaluation study to (1) see if Pursuit met its goal of enabling non-programmers to construct and comprehend complex programs containing loops, variables and conditionals; and (2) see if the form of the representation language would alter the effectiveness of Pursuit. The study revealed that users in both groups could correctly construct and comprehend complex programs. Moreover, the study showed that the comic strip language doubled users' abilities to accurately generate programs. Trends indicated that the same language was also easier for users to comprehend. We conclude that combining PBD with an editable, graphical program representation enables users to construct and comprehend complex programs and that the form of the graphical representation can impact the user's ability to effectively use the PBD system.