Book Review : Test Theory for a New Generation of Tests

recent developments in cognitive psychology are discussed in Test Theory for a New Generation of Tests. The different chapters use cognitive models in very different ways, but all seek to reestablish the connection between test theory and substantive developments in psychology by explicitly modeling how examinees respond to assessment tasks. This approach is in sharp contrast with most existing psychometric theories, which employ general mathematical models as the basis for analysis and tend to ignore questions about the processes used by examinees in answering test items. The new approach calls for the development of models describing how examinees respond to test items, and seeks to use these theories in developing better tests and deeper interpretations of test scores. The new emphasis on substantive theories of performance brings the issue of how examinees respond to items back to center stage, with the mathematical models playing supporting roles as useful tools. Most of the proposals included in these chapters, especially those that depart most radically from current measurement practice, are a long way from operational use on any substantial scale. Some of the chapters give considerable attention to item response theory (IRT) models of various kinds. Others give relatively little attention to any current test theories. The possibility of scrapping current test theories altogether and starting over is discussed, but these suggestions for model-based assessment systems that integrate test design, analysis, and score interpretation are still quite sketchy. Nevertheless, the range of possibilities being offered is so wide, the potential for improving the usefulness of assessments (especially for instructional diagnosis) are so great, and the underlying rationale is so sensible, that it seems inevitable that the new approach will lead to profound changes in both measurement theory and assessment practices. In this book, 14 chapters are organized into four groups with comments after each group of chapters. The first three chapters, with comments by Nancy Cole, outline the foundations of the new test theory and set the stage for the rest of the book. Snow and Lohman (chapter 1) provide an overview of recent developments in cognitive theory and make the case that the new emphasis on structural and dynamic models for learning requires new psychometric models. Mislevy (chapter 2) provides a historical context and considers how current statistical models might be changed in order to reflect recent work on learning and cognition. Lohman and Ippel (chapter 3) tackle the difficult task of specifying the general shape that the new test theory might take, particularly in the area of cognitive diagnosis, and present some of the problems (e.g., shifts in examinee strategies during a test) that the new test theories must solve if they are to be widely applicable; their chapter provides a clear outline of the promise in the new test theory and of some of the issues that must be addressed if the promise is to become reality. The second section, with comments by Bert Green, addresses some extensions of current