Assessing the reliability and validity of locally developed information literacy test items

Purpose – With a call for increased accountability for student learning across higher education, it is becoming more important for academic libraries to show their value to the greater university community with the use of quantitative data. This paper seeks to describe the development of an information literacy test at the University of Arizona to measure student learning in an online credit course. In order to measure the impact of an online course, a test that was statistically valid, and reliable was created by local librarians.Design/methodology/approach – The methodology involved administering test items to undergraduate students enrolled in an online information literacy course and applying both classical test theory and item response theory models to evaluate the validity and reliability of test items. This study included the longitudinal and cross‐sectional development of test items for pre and post‐testing across different student groups. Over the course of two semesters, 125 items were developed...

[1]  Hal Grossman,et al.  Assessing the assessment: how institutions administered, interpreted, and used SAILS , 2010 .

[2]  T. Kline.  Psychological Testing: A Practical Approach to Design and Evaluation , 2005 .

[3]  C. Leibiger,et al.  The South Dakota Information Literacy Exam : A Tool for Small and Medium-sized Universities to Document and Assess Information Literacy of Undergraduate and Graduate Students , 2007 .

[5]  Carolyn J. Radcliff,et al.  Applying Systems Design and Item Response Theory to the Problem of Measuring Information Literacy Skills , 2002 .

[6]  S. Wise,et al.  The Development and Validation of the Information Literacy Test , 2007 .

[7]  R. Hambleton,et al.  An NCME Instructional Module on Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. , 2005 .

[8]  Judith Emde,et al.  Assessing information literacy skills using the ACRL standards as a guide , 2007 .

[9]  F. Baker.  The basics of item response theory , 1985 .

[10]  Steven P. Reise,et al.  Item Response Theory , 2005 .

[11]  Megan Oakleaf.  Dangers and Opportunities: A Conceptual Map of Information Literacy Assessment Approaches , 2008 .

[12]  M. Miller,et al.  Measurement and Assessment in Teaching , 1994 .

[13]  Chong Ho Yu,et al.  A Non-technical Approach for Illustrating Item Response Theory , 2008 .

[14]  Valeda F. Dent,et al.  A longitudinal study of the development and evaluation of an information literacy test , 2005 .

[15]  Eduardo Backhoff Escudero,et al.  The Level of Difficulty and Discrimination Power of the Basic Knowledge and Skills Examination (EXHCOBA) , 2000 .

[16]  Zhenxiang Zhao,et al.  Food prices and fruit and vegetable consumption among young American adults. , 2009, Health & place.

[17]  Ann Manning Fiegen,et al.  Reflections on collaboration: learning outcomes and information literacy assessment in the business curriculum , 2002 .

[18]  Jane Essex,et al.  Trialling an Indicator of the Impact of Two Library Induction Models , 2011 .

[19]  S. Sim,et al.  Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. , 2006, Annals of the Academy of Medicine, Singapore.

[20]  Husein Abdul-Hamid,et al.  The Evolution of a Testing Tool for Measuring Undergraduate Information Literacy Skills in the Online Environment , 2010 .

[21]  Joanna M. Burkhardt.  Assessing Library Skills: A First Step to Information Literacy , 2007 .

[22]  Joanna M. Burkhardt,et al.  Information Literacy Successes Compared: Online vs. Face to Face , 2008 .

[23]  Patricia A Iannuzzi.  Information Literacy Competency Standards for Higher Education , 2000 .

[24]  Andrew Walsh,et al.  Information literacy assessment: Where do we start? , 2009, J. Libr. Inf. Sci..

[25]  M. Polonsky,et al.  Examining demographic and socio‐economic correlates of accurate knowledge about blood donation among African migrants in Australia , 2012, Transfusion medicine.

[26]  M. Polonsky,et al.  African culturally and linguistically diverse communities’ blood donation intentions in Australia: integrating knowledge into the theory of planned behavior , 2013, Transfusion.