Vanth Observation System Component Assessment

Since 1999, the VaNTH Observation System (VOS), a direct classroom observation system, has been used to collect data about classroom activities within bioengineering courses. Two components of the VOS, the Classroom Interaction Observation and the Global Ratings, specifically collect data about whether observed courses contain elements of the “How People Learn” (HPL) framework, as set forth in the National Research Council publication How People Learn: Mind, Brain, Experience, and School. 1 VOS observers use the Classroom Interaction Observation to collect information about the types of interactions that occur between faculty and students and among students within a course, and observers use the Global Ratings to evaluate summatively the elements of a course. Although several semesters of data have been collected at two of the universities, the validity of the VOS has not been assessed. To evaluate the validity of the VOS, five validity studies were conducted. Two content validity studies examined the extent to which eleven education content experts judged the elements of the HPL framework to be present within the Classroom Interaction Observation and Global Ratings components of the VOS, respectively. A convergent validity study noted the extent to which sampled Classroom Interaction Observation data collected in live classes correlated with full-class period Classroom Interaction Observation data collected in videotapes of those same classes. A second convergent validity study reported correlations between two different Classroom Interaction Observation assessment methods. Finally, a criterion validity study evaluated how well a newly-developed HPL Index classified Classroom Interaction Observation data within bioengineering courses that were designated as either traditional or nontraditional courses. This paper provides overviews of each validity study.