Validity and Reliability Evidence of the Engineering Professional Responsibility Assessment Tool

Background The development of social responsibility is important for educating holistic engineers, able to work across social and cultural boundaries to solve complex social issues. A way is needed to measure effective interventions for increasing social responsibility. Purpose/Hypothesis This article describes the Engineering Professional Responsibility Assessment (EPRA) instrument and provides evidence of its usability, validity, and reliability. The EPRA measures students' social responsibility attitudes and operationalizes the professional social responsibility development model, which describes the development of personal and professional social responsibility in engineers. The EPRA is intended to be used by educators to assess curricular interventions aimed at changing students' views of social responsibility. Design/Method The EPRA was developed in an iterative manner, using five different survey administrations with adjustments made to the instrument between each. Data from the final survey with 1,000 student responses were used to develop evidence of validity through expert feedback, structural equation modeling, multidimensional item response theory, and convergent evidence of validity and evidence of reliability using ordinal alpha. Results Evidence of validity and reliability indicates the appropriateness of the EPRA to measure social responsibility attitudes in engineering students. Conclusions The evidence of reliability and validity shows that the EPRA may be a useful tool to assess engineering student views of social responsibility, changes in those views over time, and the effectiveness of educational interventions intended to affect these attitudes.

[1]  Mary L. Tucker,et al.  Development and Construct Validity of Scores on the Community Service Attitudes Scale , 2000 .

[2]  A. Bielefeldt,et al.  GENDER DIFFERENCES IN THE SOCIAL RESPONSIBILITY ATTITUDES OF ENGINEERING STUDENTS AND HOW THEY CHANGE OVER TIME , 2015 .

[3]  Bruno D. Zumbo,et al.  Ordinal Versions of Coefficients Alpha and Theta for Likert Rating Scales , 2007 .

[4]  R. Cudeck,et al.  Applications of standard error estimates in unrestricted factor analysis: significance tests for factor loadings and correlations. , 1994, Psychological bulletin.

[5]  William C. Oakes,et al.  Designing in a Social Context: Situating Design in a Human-Centered, Social World , 2011 .

[6]  Kevin M. Passino,et al.  Educating the Humanitarian Engineer , 2009, Sci. Eng. Ethics.

[7]  K. Jöreskog,et al.  Factor Analysis of Ordinal Variables: A Comparison of Three Approaches , 2001, Multivariate behavioral research.

[8]  David Wallace,et al.  Mechanisms For Implementing Service Learning: Analysis Of Efforts In A Senior Product Design Class In Mechanical Engineering , 2005 .

[9]  C. Olney.  Validation of a Scale to Measure Development of Social Responsibility , 1995 .

[10]  Kaitlin Litchfield,et al.  Investigating Gains from EWB-USA Involvement , 2014 .

[11]  K. Jöreskog,et al.  Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models , 2010 .

[12]  Pui‐wa Lei,et al.  Introduction to Structural Equation Modeling: Issues and Practical Considerations , 2007 .

[13]  U Müller,et al.  Developmental sequences in class reasoning and propositional reasoning. , 1999, Journal of experimental child psychology.

[14]  S. Schwartz.  Normative Influences on Altruism , 1977 .

[15]  Philip Osteen.  An Introduction to Using Multidimensional Item Response Theory to Assess Latent Factor Structures , 2010, Journal of the Society for Social Work and Research.

[16]  Grinell Smith.  Does Gender Influence Online Survey Participation? A Record-Linkage Analysis of University Faculty Online Survey Response Behavior. , 2008 .

[17]  K. P. Singh,et al.  Covariate Dependent Markov Models for Analysis of Repeated Binary Outcomes , 2007 .

[18]  Cecilia I. Delve,et al.  Promoting values development through community service: A design , 1990 .

[19]  Wonsik Kim,et al.  An exploratory factor analysis of the Pittsburgh Freshman Engineering Attitudes Survey , 2008, 2008 38th Annual Frontiers in Education Conference.

[20]  Matthew W. Ohland,et al.  Design and Validation of a Web-Based System for Assigning Members to Teams Using Instructor-Specified Criteria , 2010 .

[21]  Klaas Sijtsma,et al.  On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha , 2008, Psychometrika.

[22]  Ryan D. Duffy,et al.  The Motivation to Serve Others: Exploring Relations to Career Development , 2010 .

[23]  Nancy J. Hayden,et al.  The Socially Responsible Engineer: Assessing Student Attitudes of Roles and Responsibilities , 2011 .

[24]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .

[25]  Adrianna Kezar,et al.  Assessing Community Service Learning , 2002 .

[26]  B. Zumbo,et al.  Estimating Ordinal Reliability for Likert-Type and Ordinal Item Response Data: A Conceptual, Empirical, and Practical Guide. , 2012 .

[27]  Nathan E. Canney,et al.  Assessing engineering students' understanding of personal and professional social responsibility , 2013 .

[28]  Angela R. Bielefeldt,et al.  A Framework for the Development of Social Responsibility in Engineers , 2015 .

[29]  William C. Oakes.  Creating Effective And Efficient Learning Experiences While Addressing The Needs Of The Poor: An Overview Of Service Learning In Engineering Education , 2009 .

[30]  Bernard Amadei,et al.  Model of Integrating Humanitarian Development into Engineering Education , 2010 .

[31]  Peter Guthrie,et al.  Widening engineering horizons: addressing the complexity of sustainable development , 2006 .