Shifting the Load: a Peer Dialogue Agent that Encourages its Human Collaborator to Contribute More to Problem Solving

Despite a growing need for educational tools that support students at the earliest phases of undergraduate Computer Science (CS) curricula, relatively few such tools exist–the majority being Intelligent Tutoring Systems. Since peer interactions more readily give rise to challenges and negotiations, another way in which students can become more interactive during problem solving, we created an artificial peer collaborator to determine its value for aiding CS students. Central to its development was the notion that it should monitor the student’s collaborative behavior and attempt to guide him/her towards more productive behavior. In prior work, we found that initiative shifts correlate with both Knowledge Co-Construction (KCC) and learning and are potentially easier to model as an indicator of productive collaboration in instructional software. In this paper, we describe a unique peer dialogue agent that we created to test the effects of tracking and reacting to initiative shifts. While our study did not find differences in learning gains when comparing agents that do and do not track and react to initiative shifts, we did find that students do learn when interacting with the agent and that attempting to influence initiative taking did make a difference. This suggests that by tracking initiative shifts, the agent was able to detect times when the student had been letting the agent do most of the “deep thinking” and that the agent’s tactics for encouraging the student to begin taking the initiative again were helpful.

[1]  Doug Baldwin,et al.  Guest Editors’ Introduction: Computer Science in the Liberal Arts , 2010, TOCE.

[2]  Jeannette M. Wing An introduction to computer science for non-majors using principles of computation , 2007, SIGCSE.

[3]  Baochuan Lu,et al.  Five focused strategies for increasing retention in Computer Science 1 , 2011 .

[4]  Claus Zinn,et al.  The Role of Initiative in Tutorial Dialogue , 2003, EACL.

[5]  Ursula Fuller,et al.  Developing a computer science-specific learning taxonomy , 2007, ACM SIGCSE Bull..

[6]  John R. Anderson Language, Memory, and Thought , 1976 .

[7]  John Mason,et al.  Why the high attrition rate for computer science students: some thoughts and observations , 2005, SGCS.

[8]  G. A. Miller,et al.  Finitary models of language users , 1963 .

[9]  J. Stasko,et al.  A Meta-Study of Algorithm Visualization Effectiveness , 2002, J. Vis. Lang. Comput..

[10]  Carol K. K. Chan Peer collaboration and discourse patterns in learning from incompatible information , 2001 .

[11]  Sandra Katz,et al.  Gender, achievement, and persistence in an undergraduate computer science program , 2006, DATB.

[12]  Csr Young,et al.  How to Do Things With Words , 2009 .

[13]  Tracy Camp,et al.  An ACM-W literature review on women in computing , 2002, SGCS.

[14]  Elliot Soloway,et al.  Studying the Novice Programmer , 1988 .

[15]  Davide Fossati,et al.  ChiQat-Tutor: An Integrated Environment for Learning Recursion , 2014 .

[16]  Brigid Barron When Smart Groups Fail , 2003 .

[17]  K. VanLehn,et al.  Coached program planning: dialogue-based support for novice program design , 2003, SIGCSE.

[18]  Curry I. Guinn,et al.  An Analysis of Initiative Selection in Collaborative Task-Oriented Discourse , 1998, User Modeling and User-Adapted Interaction.

[19]  Carolyn Penstein Rosé,et al.  Tools for Authoring a Dialogue Agent that Participates in Learning Studies , 2007, AIED.

[20]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[21]  John J. Sullivan,et al.  Intelligence and experience , 1963 .

[22]  Davide Fossati Data driven automatic feedback generation in the iList intelligent tutoring system , 2014 .

[23]  Leon E. Winslow,et al.  Programming pedagogy—a psychological overview , 1996, SGCS.

[24]  Lisa S. Blackwell,et al.  Implicit theories of intelligence predict achievement across an adolescent transition: a longitudinal study and an intervention. , 2007, Child development.

[25]  Kristy Elizabeth Boyer,et al.  A Tutorial Dialogue System for Real-Time Evaluation of Unsupervised Dialogue Act Classifiers: Exploring System Outcomes , 2015, AIED.

[26]  Margaret M. Recker,et al.  Learning Strategies and Transfer in the Domain of Programming , 1994 .

[27]  Sandra Katz,et al.  Is a Dialogue-Based Tutoring System that Emulates Helpful Co-constructed Relations During Human Tutoring Effective? , 2015, AIED.

[28]  Acm Ieee-Cs Joint Curriculum Task Force Computing Curricula 2001: Computer Science , 2002 .

[29]  David E. Meltzer,et al.  Relation between students’ problem-solving performance and representational format , 2005 .

[30]  S. Argamon,et al.  Hedged Responses and Expressions of Affect in Human/Human and Human/Computer Tutorial Interactions , 2004 .

[31]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[32]  Jun Xu,et al.  A problem solving oriented intelligent tutoring system to improve students' acquisition of basic computer skills , 2015, Comput. Educ..

[33]  L. Thomas,et al.  A cognitive approach to identifying measurable milestones for programming skill acquisition , 2006, ITiCSE-WGR '06.

[34]  Richard James,et al.  A cognitive approach to identifying measurable milestones for programming skill acquisition , 2006, ACM SIGCSE Bull..

[35]  Sandra Katz Gendered Attrition at the Undergraduate Level , 2006 .

[36]  Sandra Katz,et al.  Gender and race in predicting achievement in computer science , 2003, IEEE Technol. Soc. Mag..

[37]  Amy Soller,et al.  Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning , 2004, User Modeling and User-Adapted Interaction.

[38]  Barbara Di Eugenio,et al.  Knowledge Co-construction and Initiative in Peer Learning Interactions , 2009, AIED.

[39]  Shaochun Xu,et al.  Cognitive process during program debugging , 2004, Proceedings of the Third IEEE International Conference on Cognitive Informatics, 2004..

[40]  Barbara Di Eugenio,et al.  Modeling Knowledge Co-Construction for Peer Learning Interactions ? , 2008 .

[41]  Alex Lascarides,et al.  Indirect Speech Acts , 2001, Synthese.

[42]  田村 剛 アメリカにおける情報処理関連職種--Occupational Outlook Handbook 1990-1991 Editionを中心にして , 1992 .

[43]  Sylvia Beyer,et al.  Why are women underrepresented in Computer Science? Gender differences in stereotypes, self-efficacy, values, and interests and predictors of future CS course-taking and grades , 2014, Comput. Sci. Educ..

[44]  Barbara Di Eugenio,et al.  KSC-PaL: A Peer Learning Agent , 2010, Intelligent Tutoring Systems.

[45]  J. Flavell Metacognitive aspects of problem solving , 1976 .

[46]  Herbert J. Walberg,et al.  Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser. , 1991 .

[47]  Barbara Di Eugenio,et al.  Control and Initiative in Collaborative Problem Solving Dialogues , 1997 .

[48]  Hans Spada,et al.  Motivation and emotion in shaping knowledge co-construction , 2013 .

[49]  P. Pirolli,et al.  The role of learning from examples in the acquisition of recursive programming skills. , 1985 .

[50]  Oscar N. Garcia,et al.  Cognitive activities and support in debugging , 1998, Proceedings Fourth Annual Symposium on Human Interaction with Complex Systems.

[51]  Fred Karlsson,et al.  Constraints on multiple center-embedding of clauses , 2007 .

[52]  Fan Yang,et al.  Control in task-oriented dialogues , 2003, INTERSPEECH.

[53]  Helen Gartley,et al.  The Quality Assurance Agency for Higher Education , 2002 .

[54]  Heather H. Mitchell,et al.  AutoTutor: A tutor with dialogue in natural language , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[55]  David R. Traum,et al.  Initiative Taking in Negotiation , 2014, SIGDIAL Conference.

[56]  Terry Scott Bloom's taxonomy applied to testing in computer science classes , 2003 .

[57]  John Leaney,et al.  Introductory programming, criterion-referencing, and bloom , 2003, SIGCSE.

[58]  J. Norwood,et al.  Occupational outlook handbook , 1990 .

[59]  David McLean,et al.  An adaptation algorithm for an intelligent natural language tutoring system , 2014, Comput. Educ..

[60]  Davide Fossati,et al.  Towards explaining effective tutorial dialogues , 2009 .

[61]  Davide Fossati,et al.  Supporting Computer Science Curriculum: Exploring and Learning Linked Lists with iList , 2009, IEEE Transactions on Learning Technologies.

[62]  M. Chi,et al.  Learning from collaborative problem solving: An analysis of three hypothesized Mechanisms , 2004 .

[63]  Mordechai Ben-Ari,et al.  Constructivism in computer science education , 1998, SIGCSE '98.

[64]  R. Kozma The material features of multiple representations and their cognitive and social affordances for science understanding , 2003 .

[65]  Brenda Cantwell Wilson,et al.  A Study of Factors Promoting Success in Computer Science Including Gender Differences , 2002, Comput. Sci. Educ..

[66]  Fan Yang,et al.  Initiative conflicts in task-oriented dialogue , 2010, Comput. Speech Lang..

[67]  Mark Guzdial,et al.  Success in introductory programming: what works? , 2013, CACM.

[68]  Daniel D. Suthers,et al.  Coaching Web-based Collaborative Learning based on Problem Solution Differences and Participation , 2003, Int. J. Artif. Intell. Educ..

[69]  Lecia Jane Barker,et al.  Making Visible the Behaviors that Influence Learning Environment: A Qualitative Exploration of Computer Science Classrooms , 2004, Comput. Sci. Educ..

[70]  Marilyn A. Walker,et al.  Mixed Initiative in Dialogue: An Investigation into Discourse Segmentation , 1990, ACL.

[71]  Michelene T. H. Chi,et al.  Active-Constructive-Interactive: A Conceptual Framework for Differentiating Learning Activities , 2009, Top. Cogn. Sci..

[72]  Rachel Harsley Learning Together: Expanding the One-To-One ITS Model for Computer Science Education , 2015, ICER.

[73]  Arthur C. Graesser,et al.  Confusion and complex learning during interactions with computer learning environments , 2012, Internet High. Educ..

[74]  Joel A. Michael,et al.  Classifying Student Initiatives and Tutor Responses in Human Keyboard-to-Keyboard Tutoring Sessions , 2002 .

[75]  Pamela W. Jordan Topic Initiative in a Simulated Peer Dialogue Agent , 2007, AIED.

[76]  Jennifer Chu-Carroll,et al.  An Evidential Model for Tracking Initiative in Collaborative Dialogue Interactions , 1998, User Modeling and User-Adapted Interaction.

[77]  Anthony Scime,et al.  Globalized computing education: Europe and the United States , 2008, Comput. Sci. Educ..

[78]  Kristy Elizabeth Boyer,et al.  In-Context Evaluation of Unsupervised Dialogue Act Models for Tutorial Dialogue , 2013, SIGDIAL Conference.

[79]  J. Searle What is a Speech Act , 1996 .

[80]  Allan Fisher,et al.  Undergraduate women in computer science: experience, motivation and culture , 1997, SIGCSE '97.

[81]  Antonija Mitrovic,et al.  DB-Suite: Experiences with Three Intelligent, Web-Based Database Tutors. , 2004 .

[82]  Nickolas J. G. Falkner,et al.  A fast measure for identifying at-risk students in computer science , 2012, ICER '12.

[83]  Leen-Kiat Soh Incorporating an intelligent tutoring system into CS1 , 2006, SIGCSE '06.

[84]  V. Jovanovic Computer Science Curriculum 2013- An example , 2013, 2013 36th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[85]  Sue Fitzgerald,et al.  Debugging: a review of the literature from an educational perspective , 2008, Comput. Sci. Educ..

[86]  Cameron L. Fadjo,et al.  EngageCSEdu: engaging and retaining CS1 and CS2 students , 2015, Inroads.

[87]  Crina Damsa,et al.  The multi-layered nature of small-group learning: Productive interactions in object-oriented collaboration , 2014, Int. J. Comput. Support. Collab. Learn..

[88]  Eileen Kraemer,et al.  A spirit of camaraderie: The impact of pair programming on retention , 2013, 2013 26th International Conference on Software Engineering Education and Training (CSEE&T).

[89]  Kristy Elizabeth Boyer,et al.  Investigating the Relationship Between Dialogue Structure and Tutoring Effectiveness: A Hidden Markov Modeling Approach , 2011, Int. J. Artif. Intell. Educ..

[90]  Johanna D. Moore,et al.  BEETLE II: Deep Natural Language Understanding and Automatic Feedback Generation for Intelligent Tutoring in Basic Electricity and Electronics , 2014, International Journal of Artificial Intelligence in Education.

[91]  D. Janaki Ram,et al.  Identification of Cognitive Processes of Effective and Ineffective Students During Computer Programming , 2010, TOCE.