Do images influence assessment in anatomy? Exploring the effect of images on item difficulty and item discrimination

Anatomists often use images in assessments and examinations. This study aims to investigate the influence of different types of images on item difficulty and item discrimination in written assessments. A total of 210 of 460 students volunteered for an extra assessment in a gross anatomy course. This assessment contained 39 test items grouped in seven themes. The answer format alternated per theme and was either a labeled image or an answer list, resulting in two versions containing both images and answer lists. Subjects were randomly assigned to one version. Answer formats were compared through item scores. Both examinations had similar overall difficulty and reliability. Two cross‐sectional images resulted in greater item difficulty and item discrimination, compared to an answer list. A schematic image of fetal circulation led to decreased item difficulty and item discrimination. Three images showed variable effects. These results show that effects on assessment scores are dependent on the type of image used. Results from the two cross‐sectional images suggest an extra ability is being tested. Data from a scheme of fetal circulation suggest a cueing effect. Variable effects from other images indicate that a context‐dependent interaction takes place with the content of questions. The conclusion is that item difficulty and item discrimination can be affected when images are used instead of answer lists; thus, the use of images as a response format has potential implications for the validity of test items. Anat Sci Educ © 2012 American Association of Anatomists.

[1]  Diana H. J. M. Dolmans,et al.  International handbook of research in medical education , 2002 .

[2]  M. Peters,et al.  Applications of mental rotation figures of the Shepard and Metzler type and description of a mental rotation stimulus library , 2008, Brain and Cognition.

[3]  E. Yetisir,et al.  Spatial abilities in an elective course of applied anatomy after a problem‐based learning curriculum , 2009, Anatomical sciences education.

[4]  J. Youdas,et al.  Use of an audience response system during peer teaching among physical therapy students in human gross anatomy: Perceptions of peer teachers and students , 2009, Anatomical sciences education.

[5]  Nirusha Lachman,et al.  Assessing the integration of audience response system technology in teaching of anatomical sciences , 2009, Anatomical sciences education.

[6]  C. Vleuten,et al.  A written test as an alternative to performance testing , 1989, Medical education.

[7]  Cees P M van der Vleuten,et al.  Different written assessment methods: what can be said about their strengths and weaknesses? , 2004, Medical education.

[8]  R. Trelease.  Diffusion of innovations: Smartphones and wireless anatomy learning resources , 2008, Anatomical sciences education.

[9]  J. Kooloos,et al.  Properties of publications on anatomy in medical education literature , 2011, Anatomical sciences education.

[10]  T. Greenhalgh,et al.  Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources , 2005, BMJ : British Medical Journal.

[11]  Matthew D B S Tam,et al.  Building virtual models by postprocessing radiology images: A guide for anatomy faculty , 2010, Anatomical sciences education.

[12]  Esther M. Bergman,et al.  Why don’t they know enough about anatomy? A narrative review , 2011, Medical teacher.

[13]  K. Rochford.  Spatial learning disabilities and underachievement among university anatomy students , 1985, Medical education.

[14]  C. Vleuten.  Programmatic assessment: From assessment of learning to assessment for learning , 2011 .

[15]  J. Brueckner,et al.  Using audience response technology for pretest reviews in an undergraduate nursing course. , 2006, The Journal of nursing education.

[16]  Carla A. Romney,et al.  Effect of visual–spatial ability on medical students' performance in a gross anatomy course , 2012, Anatomical sciences education.

[17]  Lutz Vollrath,et al.  Teaching and learning gross anatomy using modern electronic media based on the visible human project , 2003, Clinical anatomy.

[18]  J. Sweller.  Implications of Cognitive Load Theory for Multimedia Learning , 2005, The Cambridge Handbook of Multimedia Learning.

[19]  Janet M Beagle.  Pictures versus text: Modality effects across three levels of learning and study time , 2009 .

[20]  I K Lukić,et al.  Weekly quizzes in extended-matching format as a means of monitoring students' progress in gross anatomy. , 2001, Annals of anatomy = Anatomischer Anzeiger : official organ of the Anatomische Gesellschaft.

[21]  Piet Kommers,et al.  The role of stereopsis in virtual anatomical learning , 2008, Interact. Comput..

[22]  Byeong-Seok Shin,et al.  Visible Korean Human: Its techniques and applications , 2006, Clinical anatomy.

[23]  David B. Swanson,et al.  Extended‐matching items: A practical alternative to free‐response questions , 1993 .

[24]  F. Daly,et al.  Use of electronic anatomy practical examinations for remediating “at risk” students , 2009, Anatomical sciences education.

[25]  Wolfgang Schnotz,et al.  Commentary: Towards an Integrated View of Learning from Text and Visual Displays , 2002 .

[26]  M. Peters,et al.  A Redrawn Vandenberg and Kuse Mental Rotations Test - Different Versions and Factors That Affect Performance , 1995, Brain and Cognition.

[27]  Cees P. M. van der Vleuten,et al.  General overview of the theories used in assessment: AMEE Guide No. 57 , 2011 .

[28]  Guoyu Qian,et al.  Automatic testing and assessment of neuroanatomy using a digital brain atlas: Method and development of computer‐ and mobile‐based applications , 2009, Anatomical sciences education.

[29]  A. Scherpbier,et al.  An Inter- and Intra-University Comparison With Short Case-Based Testing , 1999, Advances in health sciences education : theory and practice.

[30]  L. Standing,et al.  Verbal-pictorial transformations in recognition memory. , 1975, Canadian journal of psychology.

[31]  Measurement Characteristics of Content-Parallel Single-Best-Answer and Extended-Matching Questions in Relation to Number and Source of Options , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[32]  S. Loughna,et al.  A survey of medical students on the impact of a new digital imaging library in the dissection room , 2009, Clinical anatomy.

[33]  M Verma,et al.  Assessment of clinical competence , 1994 .

[34]  Jeffrey D. Karpicke,et al.  The Critical Importance of Retrieval for Learning , 2008, Science.

[35]  Mark L. Howe,et al.  Stages-of-learning analysis of picture-word effects in associative memory. , 1981 .

[36]  S. Vandenberg,et al.  Mental Rotations, a Group Test of Three-Dimensional Spatial Visualization , 1978, Perceptual and motor skills.

[37]  Claire F. Smith,et al.  An investigation into medical students' approaches to anatomy learning in a systems‐based prosection course , 2007, Clinical anatomy.

[38]  Amy Hoyt,et al.  An audience response system may influence student performance on anatomy examination questions , 2010, Anatomical sciences education.

[39]  Zhuming Ai,et al.  Virtual Reality: New Method of Teaching Anorectal and Pelvic Floor Anatomy , 2003 .

[40]  D. Heylings,et al.  Evaluation of a computer program (‘disect’) to consolidate anatomy knowledge: A randomised-controlled trial , 2010, Medical teacher.

[41]  J. Dijkstra,et al.  A new framework for designing programmes of assessment , 2009, Advances in health sciences education : theory and practice.

[42]  D. Hunt.  Illustrated multiple choice examinations , 1978, Medical Education.

[43]  A. Gelman,et al.  Splitting a Predictor at the Upper Quarter or Third and the Lower Quarter or Third , 2007 .

[44]  Timothy D. Wilson,et al.  Computer visualizations: Factors that influence spatial anatomy comprehension , 2012, Anatomical sciences education.

[45]  Krishna Juluru,et al.  Developing a radiology-based teaching approach for gross anatomy in the digital era. , 2010, Academic radiology.

[46]  C. V. D. van der Vleuten,et al.  The assessment of professional competence: Developments, research and practical implications. , 1996, Advances in health sciences education : theory and practice.

[47]  Philip K. Wilson,et al.  Exploring the Human Interior: The Roles of Cadaver Dissection and Radiologic Imaging in Teaching Anatomy , 2005, Academic medicine : journal of the Association of American Medical Colleges.

[48]  P A Heng,et al.  Photorealistic virtual anatomy based on Chinese Visible Human data , 2006, Clinical anatomy.

[49]  C. V. D. van der Vleuten,et al.  Programmatic assessment: From assessment of learning to assessment for learning , 2011, Medical teacher.

[50]  M. Abu-Hijleh,et al.  Assessment of basic medical sciences in an integrated systems‐based curriculum , 1995, Clinical anatomy.

[51]  Parvati Dev,et al.  Collaborative learning using Internet2 and remote collections of stereo dissection images , 2006, Clinical anatomy.

[52]  Varna Taranikanti,et al.  Anatomy “steeplechase” online: Necessity sometimes is the catalyst for innovation , 2011, Anatomical sciences education.

[53]  C. V. D. Vleuten,et al.  The assessment of professional competence: Developments, research and practical implications , 1996 .

[54]  Shaaron Ainsworth,et al.  The functions of multiple representations , 1999, Comput. Educ..

[55]  A. Khurana,et al.  The anatomy of anatomy: A review for its modernization , 2010, Anatomical sciences education.

[56]  Fred Paas,et al.  Interactive and dynamic visualizations in teaching and learning of anatomy: a cognitive load perspective. , 2005, Anatomical record. Part B, New anatomist.

[57]  Arnold Zellner,et al.  Introduction to Measurement with Theory , 2009 .

[58]  G. Norman,et al.  How medical students learn spatial anatomy , 2001, The Lancet.

[59]  Zhuming Ai,et al.  Virtual Reality , 2003, Diseases of the colon and rectum.

[60]  Cees P M van der Vleuten,et al.  General overview of the theories used in assessment: AMEE Guide No. 57 , 2011, Medical teacher.

[61]  A. Darzi,et al.  Assessment of anatomical knowledge for clinical practice: perceptions of clinicians and students , 2011, Surgical and Radiologic Anatomy.

[62]  C. V. D. van der Vleuten,et al.  In-training assessment using direct observation of single-patient encounters: a literature review , 2010, Advances in health sciences education : theory and practice.

[63]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[64]  G. Miller.  The assessment of clinical skills/competence/performance , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[65]  Itiel E Dror,et al.  Spatial abilities of expert clinical anatomists: Comparison of abilities between novices, intermediates, and experts in anatomy , 2011, Anatomical sciences education.

[66]  C. V. D. van der Vleuten,et al.  Assessment in the Context of Uncertainty Using the Script Concordance Test: More Meaning for Scores , 2010, Teaching and learning in medicine.

[67]  Eliane Segers,et al.  The modality effect tested in children in a user-paced multimedia environment , 2010, J. Comput. Assist. Learn..

[68]  Cees P. M. van der Vleuten,et al.  Different written assessment methods: what can be said about their strengths and weaknesses? , 2004 .

[69]  Paul Ginns.  Meta-Analysis of the Modality Effect. , 2005 .