Validity: Meaning and Relevancy in Assessment for Engineering Education Research

Ten years ago marked the beginning of a transformative time for engineering education. The January 2005 special issue of this Journal, “The Art and Science of Engineering Education Research,” was characterized as a “milestone event” in the “process of reinventing” engineering education as a discipline (Streveler & Smith, 2006, p. 103). The issue ignited a conversation regarding how the engineering education community would identify itself and make greater contributions as a research discipline, along with calls for increasing the rigor of fundamental research in engineering education (e.g., Gabriele, 2005; Haghighi, 2005; Kerns, 2005; Fortenberry, 2006; Radcliffe, 2006; Streveler & Smith, 2006; Wormley, 2006). During the 2005 and 2006 time period of high momentum, researchers identified the development of assessment strategies and high-quality engineering education assessment instruments as integral parts of the community’s transformation (Lohmann, 2005; Olds, Moskal, & Miller, 2005; Radcliffe, 2006; Streveler & Smith, 2006). Perhaps Olds et al. (2005) stated the importance most clearly:

[1]  M. Haugan,et al.  Evaluation of Colorado Learning Attitudes about Science Survey. , 2014 .

[2]  Norman Fortenberry,et al.  An Extensive Agenda for Engineering Education Research , 2006 .

[3]  David N. Wormley.  A Year of Dialogue Focused on Engineering Education Research , 2006 .

[4]  Maria Araceli Ruiz-Primo,et al.  Assessment and science education: Our essential new priority? , 2012 .

[5]  Frank J. Landy,et al.  Stamp collecting versus science: Validation as hypothesis testing. , 1986 .

[6]  N. Schmitt.  Uses and abuses of coefficient alpha. , 1996 .

[7]  David Radcliffe.  Shaping the Discipline of Engineering Education , 2006 .

[8]  R. Mislevy.  Evidence and inference in educational assessment , 1994 .

[9]  Gary Gabriele.  Advancing Engineering Education in a Flattened World , 2005 .

[10]  S. Messick.  Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[11]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[12]  J. Pellegrino.  Assessment of Science Learning: Living in Interesting Times. , 2012 .

[13]  Jack R. Lohmann,et al.  Building a Community of Scholars: The Role of the Journal of Engineering Education as a Research Journal , 2005 .

[14]  GEORGE M. BODNER,et al.  The Purdue Visualization of Rotations Test , 1997 .

[15]  Sherra E. Kerns.  Keeping Us on the Same Page , 2005 .

[16]  So Yoon Yoon,et al.  Validation of the Teaching Engineering Self‐Efficacy Scale for K‐12 Teachers: A Structural Equation Modeling Approach , 2014 .

[17]  S. Messick.  Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[18]  Russell G. Almond,et al.  On the Structure of Educational Assessments, CSE Technical Report. , 2003 .

[19]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[20]  K. Smith,et al.  Conducting Rigorous Research in Engineering Education , 2006 .

[21]  Barbara M. Olds,et al.  Rigorous Methodology for Concept Inventory Development: Using the 'Assessment Triangle' to Develop and Test the Thermal and Transport Science Concept Inventory (TTCI)* , 2011 .

[22]  S. Messick.  The Interplay of Evidence and Consequences in the Validation of Performance Assessments , 1994 .

[23]  Kamyar Haghighi,et al.  Quiet No Longer: Birth of a New Discipline , 2005 .

[24]  L. Cronbach,et al.  Construct validity in psychological tests. , 1955, Psychological bulletin.

[25]  L. Weng.  Impact of the Number of Response Categories and Anchor Labels on Coefficient Alpha and Test-Retest Reliability , 2004 .

[26]  Frank S. Oreovicz,et al.  THE SCHOLARSHIP OF TEACHING AND LEARNING IN ENGINEERING , 2002 .

[27]  Monica F. Cox,et al.  Creating an Instrument to Measure Leadership, Change, and Synthesis in Engineering Undergraduates , 2014 .

[28]  Johannes Strobel,et al.  Hopes and Goals Survey for use in STEM elementary education , 2015 .

[29]  So Yoon Yoon,et al.  A Meta-Analysis on Gender Differences in Mental Rotation Ability Measured by the Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) , 2013 .

[30]  Barbara M. Moskal,et al.  Assessment in Engineering Education: Evolution, Approaches and Future Collaborations , 2005 .

[31]  Samuel Messick,et al.  VALIDITY OF TEST INTERPRETATION AND USE , 1990 .

[32]  R. Almond,et al.  Focus Article: On the Structure of Educational Assessments , 2003 .

[33]  P. K. Imbrie,et al.  Psychometric properties of the revised PSVT:R for measuring first year engineering students’ spatial ability , 2013 .