Development of the Exams Data Analysis Spreadsheet as a Tool To Help Instructors Conduct Customizable Analyses of Student ACS Exam Data

The American Chemical Society Examinations Institute (ACS-EI) has recently developed the Exams Data Analysis Spread (EDAS) as a tool to help instructors conduct customizable analyses of their student data from ACS exams. The EDAS calculations allow instructors to analyze their students’ performances both at the total score and individual item levels, while also providing national normative results that can be used for comparison. Additionally, instructors can analyze results based on subsets of items of their choosing or items based on the “big ideas” from the Anchoring Concepts Content Map (ACCM). In order to evaluate the utility and usability of the EDAS for instructors, the EDAS went through trial testing with 10 chemistry instructors from across the country. The instructor feedback confirmed that the EDAS has multiple implications for the classroom and departmental assessment, but some additional revisions were needed to increase its usability. This feedback was also used to make a video user-guide th...

[1]  K. L. Murphy,et al.  Updating the General Chemistry Anchoring Concepts Content Map , 2015 .

[2]  T. Holme,et al.  Using Structural Equation Modeling To Understand Chemistry Faculty Familiarity of Assessment Terminology: Results from a National Survey , 2013 .

[3]  M. Towns.  Developing Learning Objectives and Assessment Plans at a Variety of Institutions: Examples and Case Studies , 2010 .

[4]  P. Hutchings.  From Departmental to Disciplinary Assessment: Deepening Faculty Engagement , 2011 .

[5]  T. Holme,et al.  Methods for Addressing Missing Data with Applications from ACS Exams , 2015 .

[6]  N. J. Pienta.  Striking a Balance with Assessment , 2011 .

[7]  Stacey Lowery Bretz.  Navigating the Landscape of Assessment , 2012 .

[8]  Jeffrey R. Raker,et al.  Validating Chemistry Faculty Members’ Self-Reported Familiarity with Assessment Terminology , 2013 .

[9]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[10]  K. L. Murphy,et al.  Results from a National Needs Assessment Survey: A View of Assessment Efforts within Chemistry Departments , 2013 .

[11]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[12]  Jeffrey R. Raker,et al.  Building a Database for the Historical Analysis of the General Chemistry Curriculum Using ACS General Chemistry Exams as Artifacts , 2015 .

[13]  Trudy W. Banta,et al.  Closing the Assessment Loop , 2010 .

[14]  T. Holme,et al.  What Do Conceptual Holes in Assessment Say about the Topics We Teach in General Chemistry , 2015 .

[15]  Kristen L. Murphy,et al.  The ACS Exams Institute Undergraduate Chemistry Anchoring Concepts Content Map I: General Chemistry , 2017 .

[16]  R. Tibshirani,et al.  (37) Medications as Independent Risk Factors of Delirium in Patients With COVID-19: A Retrospective Study , 2018, Journal of the Academy of Consultation-Liaison Psychiatry.

[17]  Susan R. Singer,et al.  Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. , 2012 .

[18]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[19]  T. Holme,et al.  Investigating Faculty Familiarity with Assessment Terminology by Applying Cluster Analysis to Interpret Survey Data. , 2014 .

[20]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[21]  J. B. Brooke,et al.  SUS: a retrospective , 2013 .

[22]  Philip T. Kortum,et al.  Usability Ratings for Everyday Products Measured With the System Usability Scale , 2013, Int. J. Hum. Comput. Interact..

[23]  Stefano Federici,et al.  On the dimensionality of the System Usability Scale: a test of alternative measurement models , 2009, Cognitive Processing.