The influences of a two-tier test strategy on student learning: A lag sequential analysis approach

Recently, programming skills have become a core competence. Many teaching strategies were developed to improve programming skills. Among them, online tests were widely applied to enhance students learning. Nonetheless, they may not be able to engage students in deep thinking and reflections. Thus, a two-tier test strategy was proposed to address this issue. However, previous research mainly focused on investigating the effectiveness of the two-tier test strategy but there is a lack of studies that investigate why the two-tier test approach is effective. To this end, we developed an online test, where the two-tier test strategy was implemented. Additionally, an empirical study was conducted to explore the influences of the two-tier test approach on students' learning performance and behavior patterns. Pre-test and post-test scores were applied to assess students' learning performance while a lag sequential analysis was used to analyze behavior patterns. Regarding learning performance, the proposed two-tier test can improve students' programming skills. Regarding behavior patterns, the two-tier test approach facilitates students to develop a learning by reviewing strategy, which is useful to improve their programming skills. Two-tier tests can enhance students' programming skills.Two-tier tests change students' learning strategies.Two-tier tests do not change students' intention.A lag sequential analysis is a useful approach.

[1]  Dimitrios Kalles.  Students working for students on programming courses , 2008, Comput. Educ..

[2]  Henry Been-Lirn Duh,et al.  An investigation of learners' collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system , 2013, Comput. Educ..

[3]  Pinchas Tamir.  Multiple choice items: How to gain the most out of them , 1991 .

[4]  Ying Liu,et al.  Assessment of programming language learning based on peer code review model: Implementation and experience report , 2012, Comput. Educ..

[5]  Philip Machanick.  Teaching Java backwards , 2007, Comput. Educ..

[6]  A. Odom,et al.  Development and application of a two‐tier diagnostic test measuring college biology students' understanding of diffusion and osmosis after a course of instruction , 1995 .

[7]  Miguel A. Brito,et al.  Assessment frequency in introductory computer programming disciplines , 2014, Comput. Hum. Behav..

[8]  Huei-Tse Hou,et al.  Analyzing the Learning Process of an Online Role-Playing Discussion Activity , 2012, J. Educ. Technol. Soc..

[9]  Matthias Hauswirth,et al.  Teaching Java programming with the Informa clicker system , 2013, Sci. Comput. Program..

[10]  D. Saccuzzo,et al.  Psychological Testing: Principles, Applications, and Issues , 1982 .

[11]  Chin-Chung Tsai.  The interpretation construction design model for teaching science and its applications to Internet-based instruction in Taiwan , 2001 .

[12]  Sue Fitzgerald,et al.  Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers , 2008, Comput. Sci. Educ..

[13]  C. Chou,et al.  Diagnosing students' alternative conceptions in science , 2002, J. Comput. Assist. Learn..

[14]  Andrew Trotman,et al.  Programming contest strategy , 2008, Comput. Educ..

[15]  D. Treagust.  Development and use of diagnostic tests to evaluate students’ misconceptions in science , 1988 .

[16]  Haluk Özmen.  The influence of computer-assisted instruction on students' conceptual understanding of chemical bonding and attitude toward chemistry: A case for Turkey , 2008, Comput. Educ..

[17]  W. Michael Reed,et al.  Computer Experience and Interval-Based Hypermedia Navigation , 1997 .

[18]  Cees P. M. van der Vleuten,et al.  Different written assessment methods: what can be said about their strengths and weaknesses? , 2004 .

[19]  J. Ángel Velázquez-Iturbide,et al.  Student perception and usage of an automated programming assessment tool , 2014, Comput. Hum. Behav..

[20]  Ming Ming Chiu,et al.  Design and evaluation of instructor-based and peer-oriented attention guidance functionalities in an open source anchored discussion system , 2014, Comput. Educ..

[21]  Mark A. McDaniel,et al.  Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. , 2011 .

[22]  Gwo-Jen Hwang,et al.  A web-based programming learning environment to support cognitive development , 2008, Interact. Comput..

[23]  Zoltán Kátai,et al.  Technologically and Artistically Enhanced Multi-Sensory Computer-Programming Education. , 2010 .

[24]  Yao-Ting Sung,et al.  Effects of a Mobile Electronic Guidebook on Visitors' Attention and Visiting Behaviors , 2008, J. Educ. Technol. Soc..

[25]  Jeffrey D. Karpicke,et al.  The effect of type and timing of feedback on learning from multiple-choice tests. , 2007, Journal of experimental psychology. Applied.

[26]  Alan H. Schoenfeld,et al.  When Good Teaching Leads to Bad Results: The Disasters of 'Well-Taught' Mathematics Courses , 1988 .

[27]  Irene Govender,et al.  Pre-service and in-service teachers' experiences of learning to program in an object-oriented language , 2008, Comput. Educ..

[28]  Pei-Wei Tsai,et al.  Comparing the social knowledge construction behavioral patterns of problem-based online asynchronous discussion in e/m-learning environments , 2012, Comput. Educ..

[29]  Jeffrey D. Karpicke,et al.  Test-Enhanced Learning , 2006, Psychological science.

[30]  Chien Chou,et al.  Using a two-tier test to assess students' understanding and alternative conceptions of cyber copyright laws , 2007, Br. J. Educ. Technol..

[31]  Gwo-Jen Hwang,et al.  A group decision approach to developing concept-effect models for diagnosing student learning problems in mathematics , 2013, Br. J. Educ. Technol..

[32]  Gwo-Jen Hwang,et al.  Diagnosing student learning problems based on historical assessment records , 2008 .

[33]  Evangelia Gouli,et al.  Problem solving by 5-6 years old kindergarten children in a computer programming environment: A case study , 2013, Comput. Educ..

[34]  James Hiebert,et al.  On Having and Using Geometric Knowledge , 2013 .

[35]  Chin-Chung Tsai,et al.  Affordances of Augmented Reality in Science Learning: Suggestions for Future Research , 2012, Journal of Science Education and Technology.

[36]  Jeroen Keppens,et al.  Concept map assessment for teaching computer programming , 2008, Comput. Sci. Educ..

[37]  Yao-Ting Sung,et al.  Mobile guide system using problem-solving strategy for museum learning: a sequential learning behavioural pattern analysis , 2010, J. Comput. Assist. Learn..

[38]  Gwo-Jen Hwang,et al.  A two-tier test approach to developing location-aware mobile learning systems for natural science courses , 2010, Comput. Educ..

[39]  Yao-Ting Sung,et al.  An Analysis of Peer Assessment Online Discussions within a Course that uses Project-based Learning , 2007, Interact. Learn. Environ..

[40]  Luisa M. Regueras,et al.  A distributed system for learning programming on-line , 2012, Comput. Educ..

[41]  Henry H. Emurian,et al.  Managing programmed instruction and collaborative peer tutoring in the classroom: Applications in teaching Java™ , 2008, Comput. Hum. Behav..

[42]  Wu-Yuin Hwang,et al.  A pilot study of cooperative programming learning behavior and its relationship with students' learning performance , 2012, Comput. Educ..

[43]  Roger Bakeman,et al.  Observing Interaction: An Introduction to Sequential Analysis , 1986 .

[44]  Gwo-Jen Hwang,et al.  Development of a diagnostic and remedial learning system based on an enhanced concept–effect model , 2013 .

[45]  Eduardo Guzmán,et al.  A blended E-learning experience in a course of object oriented programming fundamentals , 2009, Knowl. Based Syst..

[46]  Kirsti Ala-Mutka,et al.  A Survey of Automated Assessment Approaches for Programming Assignments , 2005, Comput. Sci. Educ..

[47]  Maria Kordaki,et al.  A drawing and multi-representational computer environment for beginners' learning of programming using C: Design and pilot formative evaluation , 2010, Comput. Educ..