Promoting Learning of Instructional Design via Overlay Design Tools

I begin by introducing Virtual Design Apprenticeship (VDA), a learning model — built on a solid foundation of education principles and theories — that promotes learning of design skills via overlay design tools. In VDA, when an individual needs to learn a new design skill or paradigm she is provided accessible, concrete examples that have been annotated with design rationale. These annotations make expert thinking visible and allow the novice to immediately use, and gradually understand, new best practices. By combining abstract rationale with concrete design instances, annotated artifacts become more useful than either could be alone. I describe the essential components of the VDA framework: annotated design artifacts, a repository of carefully chosen annotated examples, and a community of experts and learners. I walk the reader through an example of how VDA scaffolds learners as they move from a novice's understanding of a design space towards that of an expert. Within the context of this example, I present a set of design principles that guide the creation of VDA design tools — user interfaces built to mediate an individual's interactions with the three core VDA components. While VDA is applicable to most design fields, I narrow the scope of consideration to one particular domain of design by focusing in-depth on the instructional design difficulties that university-level faculty members face and how the VDA approach can address them. These instructors face precisely the type of design paradigm shift that VDA was developed to ease as they attempt to move away from traditional, lecture-based pedagogical methods and towards more modern, learner-centered techniques. I engaged with these instructors and a curriculum design research group in a six-year period of contextual inquiry. Findings from this study influenced my formulation of the VDA framework and the design of PACT, a design tool that leverages the learning principle of making thinking visible to assist novices as they transition from concrete to abstract reasoning about curriculum design. The central focus of PACT is the incorporation of annotated references to pedagogical design patterns — abstract representations of best practices in instructional design. I discuss the iterative design and implementation of PACT in detail, highlighting the ways in which it embodies the VDA design principles for promoting learning of instructional design via overlay design tools. Next, I study the challenges of converting abstract best practices and design patterns into concrete annotations that can be applied directly to content. My solution, the PACT Annotation Schema, is a formal mechanism for generating tags and pattern annotations from freeform pattern text. Formal representations of patterns are far more useful than generic references, both as scaffolds for learning and for structuring user interactions with design artifacts. Using this schema, I have generated the PACT Annotation Library, a collection of 56 tags and 74 pattern annotations based on the work of the Pedagogical Patterns Project. Visual representations of these formal annotations are the centerpiece of PACT's user interface. The PACT tool was evaluated in two distinct stages. First, I present a formative study conducted with early, prototype versions of the PACT tool. This study examines the utility of PACT for expert curriculum designers and curriculum research groups, using a sample annotation process — and reflection on the outcomes of that process — to demonstrate that my approach is feasible and useful for those groups. I then present a summative user study of the utility of PACT for novice learner-centered curriculum designers. I demonstrate PACT's significant impact on how novice designers learn from expert-generated examples, how they perceive the credibility of those examples, and the quality of curriculum designs those novices can produce. These findings show that the VDA approach to learning works and that the PACT overlay curriculum design tool is a successful realization of VDA's design principles. (Abstract shortened by UMI.)