Building teacher competency for digital content evaluation

Abstract The purpose of the current study is to investigate teachers' digital content evaluation (DCE) competency as a combined set of teachers' intellectual ability and affect-motivation in evaluating digital content. To this end, this study employed an exploratory sequential mixed-methods design to examine 102 in-service teachers’ experiences in a one-year DCE professional development program. Qualitative thematic analysis followed by quantitative K-means cluster analysis identified and validated four levels of teacher competency in DCE: familiarization, utilization, integration, and reorientation. In addition, this study also proposed design guidelines to scaffold DCE competency development tailored to the needs of teachers at various competency levels.

[1]  Matthew J. Koehler,et al.  Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy and technology , 2007, Comput. Educ..

[2]  R. Mayer.  Rote Versus Meaningful Learning , 2002 .

[3]  Fethi A. Inan,et al.  Factors affecting technology integration in K-12 classrooms: a path model , 2010 .

[4]  Kent J. Crippen,et al.  The Interactive Effects of Personal Achievement Goals and Performance Feedback in an Undergraduate Science Class , 2013 .

[5]  M. S. Hoyert,et al.  A Brief Intervention to Aid Struggling Students: A Case of Too Much Motivation? , 2006 .

[6]  Peggy A. Ertmer.  Teacher pedagogical beliefs: The final frontier in our quest for technology integration? , 2005 .

[7]  Richard J. Shavelson,et al.  On the measurement of competency , 2010 .

[8]  J. Eccles,et al.  Ability Self-Perceptions and Subjective Task Values in Adolescents and Children , 2005 .

[9]  Richard Fliegel,et al.  Quantifying Learning in Critical Thinking , 2013 .

[10]  Jean Piaget,et al.  Part I: Cognitive development in children: Piaget development and learning , 1964 .

[11]  H. Mandl,et al.  Learning from Worked-Out Examples: The Effects of Example Variability and Elicited Self-Explanations , 1998, Contemporary educational psychology.

[12]  Michelene T. H. Chi,et al.  Laboratory Methods for Assessing Experts' and Novices' Knowledge , 2006 .

[13]  Ruth Wylie,et al.  The Self-Explanation Principle in Multimedia Learning , 2014 .

[14]  Min Kyu Kim.  Models of learning progress in solving complex problems: Expertise development in teaching and learning , 2015 .

[15]  Dennis Sparks.  Designing Powerful Professional Development for Teachers and Principals. , 2002 .

[16]  Johnny Saldaña,et al.  The Coding Manual for Qualitative Researchers , 2009 .

[17]  J. Hair.  Multivariate data analysis , 1972 .

[18]  Donald A. Redelmeier,et al.  Rainy weather and medical school admission interviews , 2009, Canadian Medical Association Journal.

[19]  Lars R. Bergman,et al.  Studying Individual Development in an Interindividual Context: A Person-Oriented Approach , 2002 .

[20]  Marko Sarstedt,et al.  A Concise Guide to Market Research: The Process, Data, and Methods Using IBM SPSS Statistics , 2011 .

[21]  Jennifer J. Vogel-Walcutt,et al.  Applying the modality principle to real-time feedback and the acquisition of higher-order cognitive skills , 2012 .

[22]  P. Salovey,et al.  The influence of teacher emotion on grading practices: a preliminary look at the evaluation of student writing , 2013 .

[23]  Thomas A. Brush,et al.  Integrating technology into K-12 teaching and learning: current knowledge gaps and recommendations for future research , 2007 .

[24]  D. Kuhn,et al.  Metacognition: A Bridge Between Cognitive Psychology and Educational Practice , 2004 .

[25]  M. Covington.  Goal theory, motivation, and school achievement: an integrative review. , 2000, Annual review of psychology.

[26]  Setlhomo Koloi-Keaikitse.  Assessment Training: A Precondition for Teachers’ Competencies and Use of Classroom Assessment Practices , 2016 .

[27]  Brad Hokanson,et al.  Integrating Technology in Classrooms: We Have Met the Enemy and He is Us , 2004 .

[28]  Anthony J. Guarino,et al.  Predicting Teacher Computer Use: A Path Analysis , 2000 .

[29]  Patrick Wachira,et al.  Technology Integration Barriers: Urban School Mathematics Teachers Perspectives , 2011 .

[30]  N. Rummel,et al.  Teacher Competencies for the Implementation of Collaborative Learning in the Classroom: a Framework and Research Review , 2015 .

[31]  J. Eccles,et al.  In the Mind of the Actor: The Structure of Adolescents' Achievement Task Values and Expectancy-Related Beliefs , 1995 .

[32]  A. Bandura.  Self-efficacy: toward a unifying theory of behavioral change. , 1977, Psychological review.

[33]  Matthew J. Koehler,et al.  Technological Pedagogical Content Knowledge (TPACK) , 2009 .

[34]  Chin-Chung Tsai,et al.  Facilitating Preservice Teachers' Development of Technological, Pedagogical, and Content Knowledge (TPACK) , 2010, J. Educ. Technol. Soc..

[35]  J. Eccles,et al.  Expectancy-Value Theory of Achievement Motivation. , 2000, Contemporary educational psychology.

[36]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[37]  Michele W. Spitulnik,et al.  Supporting Teachers’ Use of Technology in Science Instruction Through Professional Development: A Literature Review , 2008 .

[38]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[39]  Patricia A. Alexander,et al.  Modeling Domain Learning: Profiles From the Field of Special Education , 2004 .

[40]  Michael D. Mumford,et al.  Process analytic models of creative capacities , 1991 .

[41]  S. Iacus,et al.  Teachers’ evaluations and students’ achievement: a ‘deviation from the reference’ analysis , 2011 .

[42]  A. Lindmeier,et al.  Beyond Knowledge: Measuring Primary Teachers’ Subject-Specific Competences in and for Teaching Mathematics with Items Based on Video Vignettes , 2015 .

[43]  M. Valcke,et al.  Explaining different types of computer use among primary school teachers , 2004 .

[44]  Sheila M. Smith.  The Four Sources of Influence on Computer Self-Efficacy , 2001 .

[45]  Min Kyu Kim,et al.  Teacher professional development through digital content evaluation , 2017, Educational Technology Research and Development.

[46]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[47]  Robert S Siegler,et al.  Children's learning. , 2005, The American psychologist.

[48]  Hubert L. Dreyfus,et al.  From Socrates to expert systems , 1984 .

[49]  Theodore J. Kopcha,et al.  Teachers' perceptions of the barriers to technology integration and practices with technology under situated professional development , 2012, Comput. Educ..

[50]  E. Hundert,et al.  Defining and assessing professional competence. , 2002, JAMA.

[51]  Chia-Jung Lee,et al.  An implementation study of a TPACK-based instructional design model in a technology integration course , 2014, Educational Technology Research and Development.

[52]  Ulrich Trautwein,et al.  A person-centered approach to homework behavior: Students’ characteristics predict their homework learning type , 2017 .

[53]  Mable B. Kinzie,et al.  Computer technology training for prospective teachers: computer attitudes and perceived self-efficacy , 2000 .

[54]  Patricia A. Alexander,et al.  Profiling the Differences in Students' Knowledge, Interest, and Strategic Processing , 1998 .

[55]  Karen Reinhardt,et al.  Weaving Science Inquiry and Continuous Assessment: Using Formative Assessment to Improve Learning , 2003 .

[56]  Peggy A. Ertmer.  Addressing first- and second-order barriers to change: Strategies for technology integration , 1999 .

[57]  Susan M. Brookhart,et al.  Teachers' Grading: Practice and Theory , 1994 .

[58]  Nmc.  NMC Horizon Report: 2015 K-12 Edition , 2015 .

[59]  Hugh M. Cannon,et al.  Bloom Beyond Bloom: Using the Revised Taxonomy to Develop Experiential Learning Strategies , 2005 .

[60]  O. Dickhäuser,et al.  Assessing task difficulty for other people: when deeper evaluation means “it’s more about me!” , 2016 .

[61]  L. Kohlberg,et al.  Moral development: A review of the theory , 1977 .

[62]  Deborah Bybee,et al.  Possible selves and academic outcomes: How and when possible selves impel action. , 2006, Journal of personality and social psychology.

[63]  William G. Wraga.  The Continuing Arrogation of the Curriculum Field: A Rejoinder to Pinar , 1999 .

[64]  M. Chi.  Two Approaches to the Study of Experts' Characteristics , 2006 .

[65]  K. A. Ericsson,et al.  Recent advances in expertise research: a commentary on the contributions to the special issue , 2005 .

[66]  K. VanLehn,et al.  Cognitive skill acquisition. , 1996, Annual review of psychology.

[67]  Matthew J. Koehler,et al.  Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge , 2006, Teachers College Record: The Voice of Scholarship in Education.

[68]  Katariina Salmela-Aro,et al.  Task value profiles across subjects and aspirations to physical and IT-related sciences in the United States and Finland. , 2012, Developmental psychology.

[69]  Matthew J. Koehler,et al.  What Happens When Teachers Design Educational Technology? The Development of Technological Pedagogical Content Knowledge , 2005 .

[70]  Ronald B. Childress,et al.  An Evaluation of a School-Based, Job-Embedded Professional Development Program on Teachers’ Efficacy for Technology Integration: Findings from an Initial Study , 2013 .

[71]  D. Wiley,et al.  Open Educational Resources: A Review of the Literature , 2014 .

[72]  Curtis J. Bonk,et al.  The Tickit to Teacher Learning: Designing Professional Development According to Situative Principles , 2005 .

[73]  P. Pintrich.  A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students , 2004 .

[74]  Barbara S. Plake,et al.  Teachers' Ability to Estimate Item Difficulty: A Test of the Assumptions in the Angoff Standard Setting Method , 1998 .

[75]  Angela Jonen,et al.  The effects of expert scaffolding in elementary science professional development on teachers’ beliefs and motivations, instructional practices, and student achievement. , 2016 .

[76]  J. H. Sandholtz,et al.  Teachers, Not Technicians: Rethinking Technical Expectations for Teachers , 2004, Teachers College Record: The Voice of Scholarship in Education.

[77]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .

[78]  Peggy A. Ertmer,et al.  Removing obstacles to the pedagogical changes required by Jonassen's vision of authentic technology-enabled learning , 2013, Comput. Educ..

[79]  Wendy Martin,et al.  Connecting Instructional Technology Professional Development to Teacher and Student Outcomes , 2010 .

[80]  Joan E. Hughes,et al.  The Role of Teacher Knowledge and Learning Experiences in Forming Technology-Integrated Pedagogy , 2005 .

[81]  H. Becker.  Findings from the Teaching, Learning, and Computing Survey: Is Larry Cuban Right? , 2000 .

[82]  Hubert L. Dreyfus,et al.  From Socrates to Expert Systems: The Limits of Calculative Rationality , 1986 .

[83]  Chrystalla Mouza,et al.  The Impact of Professional Development on Teacher Learning, Practice and Leadership Skills: A Study on the Integration of Technology in the Teaching of Writing , 2007 .

[84]  Amy Staples,et al.  Rethinking the Technology Integration Challenge , 2005 .

[85]  Min Kyu Kim,et al.  Teacher Beliefs and Technology Integration. , 2013 .

[86]  Judy Kay,et al.  Over-confidence and confusion in using bloom for programming fundamentals assessment , 2012, SIGCSE '12.

[87]  J. Sinnott.  A model for solution of ill-structured problems: Implications for everyday and abstract problem solving. , 1989 .

[88]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[89]  Jonathan Brinkerhoff.  Effects of a Long-Duration, Professional Development Academy on Technology Skills, Computer Self-Efficacy, and Technology Integration Beliefs and Practices , 2006 .

[90]  Laura M. Desimone,et al.  What Makes Professional Development Effective? Results From a National Sample of Teachers , 2001 .

[91]  J. Gustafsson,et al.  Beyond dichotomies: Competence viewed as a continuum , 2015 .

[92]  Laura M. O’Dwyer,et al.  Measuring Teachers’ Technology Uses , 2004 .

[93]  Jason T. Abbitt.  Measuring Technological Pedagogical Content Knowledge in Preservice Teacher Education , 2011 .

[94]  G. Veletsianos,et al.  Using the Technological, Pedagogical, and Content Knowledge Framework to Design Online Learning Environments and Professional Development , 2009 .

[95]  Jacquelynne S. Eccles,et al.  Task values and ability beliefs as predictors of high school literacy choices: A developmental analysis. , 2006 .

[96]  Robert S. Siegler,et al.  The Logarithmic-To-Linear Shift: One Learning Sequence, Many Tasks, Many Time Scales , 2009 .

[97]  R. Sternberg.  Applying Psychological Theories to Educational Practice , 2008 .

[98]  Robert Thornberg,et al.  Grounded theory and theoritical coding , 2014 .

[99]  Randall S. Davies,et al.  Technology Integration in Schools , 2014 .

[100]  D. Kuhn.  A Developmental Model of Critical Thinking , 1999 .

[101]  Charles R. Graham,et al.  The Technological Pedagogical Content Knowledge Framework , 2014 .

[102]  Patricia A. Alexander,et al.  A model of domain learning: Reinterpreting expertise as a multidimensional, multistage process. , 2004 .

[103]  A. Renkl,et al.  How Effective are Instructional Explanations in Example-Based Learning? A Meta-Analytic Review , 2010 .

[104]  Mesut Duran,et al.  Impact of Research-Based Professional Development , 2011 .

[105]  Michelle Cosmah,et al.  Targeting Digital Technologies in Common Core Standards: A Framework for Professional Development , 2013 .

[106]  K. Thomas-Brown,et al.  Enhancing Teacher Competency through Co-Teaching and Embedded Professional Development. , 2015 .

[107]  Lloyd P. Rieber,et al.  Teaching with Technology , 2011 .

[108]  John H. Flavell,et al.  Cognitive development: Past, present, and future. , 1992 .

[109]  Peggy A. Ertmer,et al.  Teacher beliefs and technology integration practices: A critical relationship , 2012, Comput. Educ..