Using Dialogue Features to Predict Trouble During Collaborative Learning

A web-based, collaborative distance-learning system that will allow groups of students to interact with each other remotely and with an intelligent electronic agent that will aid them in their learning has the potential for improving on-line learning. The agent would follow the discussion and interact with the participants when it detects learning trouble of some sort, such as confusion about the problem they are working on or a participant who is dominating the discussion or not interacting with the other participants. In order to recognize problems in the dialogue, we investigated conversational elements that can be utilized as predictors for effective and ineffective interaction between human students. These elements can serve as the basis for student and group models. In this paper, we discuss group interaction during collaborative learning, our representation of participant dialogue, and the statistical models we are using to determine the role being played by a participant at any point in the dialogue and the effectiveness of the group. We also describe student and group models that can be built using conversational elements and discuss one set that we built to illustrate their potential value in collaborative learning.

[1]  Marilyn A. Walker,et al.  Using Natural Language Processing and discourse Features to Identify Understanding Errors , 2000, ICML.

[2]  Thomas G. Dietterich.  Machine Learning for Sequential Data: A Review , 2002, SSPR/SPR.

[3]  Paul Brna,et al.  Extending the scope of the student model , 1995, User Modeling and User-Adapted Interaction.

[4]  Susan E. Newman,et al.  Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics. Technical Report No. 403. , 1987 .

[5]  Sandra Katz,et al.  Extensions of intelligent tutoring paradigms to support collaborative learning , 1992 .

[6]  Alan M. Lesgold,et al.  What Makes Peer Interaction Effective? Modeling Effective Communication in an Intelligent CSCL , 1999 .

[7]  Alan M. Lesgold,et al.  Computational analysis of knowledge sharing in collaborative distance learning , 2002 .

[8]  Richard M. Felder,et al.  A Longitudinal Study of Engineering Student Performance and Retention. V. Comparisons with Traditionally‐Taught Students , 1998 .

[9]  Andreas Stolcke,et al.  Can Prosody Aid the Automatic Processing of Multi-Party Meetings? Evidence from Predicting Punctuation, Disfluencies, and Overlapping Speech , 2003 .

[10]  Mark K. Singley,et al.  Algebra jam: supporting teamwork and managing roles in a collaborative learning environment , 2000, CSCW '00.

[11]  Sandra Katz,et al.  Modeling pedagogical interactions with machine learning , 2000, Kognitionswissenschaft.

[12]  Amy Soller,et al.  Computational Modeling and Analysis of Knowledge Sharing in Collaborative Distance Learning , 2004, User Modeling and User-Adapted Interaction.

[13]  Mauri Kaipainen,et al.  Self-Organizing Dialogue Management , 2001 .

[14]  Elmar Nöth,et al.  Whence and Whither Prosody in Automatic Speech Understanding: A Case Study , 2002 .

[15]  Stephanie Jarboe,et al.  Procedures for Enhancing Group Decision Making , 1996 .

[16]  Helen Pain,et al.  Understandable Learner Models for a Sensorimotor Control Task , 2000, Intelligent Tutoring Systems.

[17]  J. Searle.  Expression and Meaning: A taxonomy of illocutionary acts , 1975 .

[18]  Diane J. Litman,et al.  Predicting Emotion in Spoken Dialogue from Multiple Knowledge Sources , 2004, NAACL.

[19]  Bradley A. Goodman,et al.  Towards Intelligent Agents for Collaborative Learning: Recognizing the Roles of Dialogue Participants , 2003, User Modeling.

[20]  Cindy E. Hmelo-Silver,et al.  Collaborative ways of knowing: issues in facilitation , 2002, CSCL.

[21]  Brad Hartfield,et al.  Computer systems and the design of organizational interaction , 1988, TOIS.

[22]  B. Bloom.  The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring , 1984 .

[23]  Julia Hirschberg,et al.  Generalizing prosodic prediction of speech recognition errors , 2000, INTERSPEECH.

[24]  Amy Soller,et al.  Supporting Social Interaction in an Intelligent Collaborative Learning System , 2001 .

[25]  Mari Ostendorf,et al.  Detection Of Agreement vs. Disagreement In Meetings: Training With Unlabeled Data , 2003, NAACL.

[26]  Sandra Katz,et al.  An Approach to Analyzing the Role and Structure of Reflective Dialogue , 1999 .

[27]  Mark K. Singley,et al.  Scaffolding group learning in a collaborative networked environment , 2002, CSCL.

[28]  Susan M. Land,et al.  Theoretical Foundations of Learning Environments. , 1999 .

[29]  Michael J. Baker,et al.  Modelling Dialogue and Beliefs as a Basis for Generating Guidance in a CSCL Environment , 1996, Intelligent Tutoring Systems.

[30]  Allan Collins,et al.  Cognitive Apprenticeship and Instructional Technology , 1988 .

[31]  C. Kneser,et al.  The Tutor's Role: An investigation of the power of Exchange Structure Analysis to identify different roles in CMC seminars , 2001 .

[32]  R. Stevens,et al.  Artificial neural network-based performance assessments , 1999 .

[33]  John E. Mathieu,et al.  Task and aggregation issues in the analysis and assessment of team performance. , 1997 .

[34]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[35]  Janyce Wiebe,et al.  A machine learning approach to assessing knowledge sharing during collaborative learning activities , 2002, CSCL.

[36]  M. Chi.  Constructing Self-Explanations and Scaffolded Explanations in Tutoring , 1996 .

[37]  A. Baskin,et al.  Studying with the prince: The computer as a learning companion , 1988 .

[38]  Jeremy H. Wright,et al.  Using Natural Language Processing and Discourse Features to Identify Understanding Errors in a Spoken Dialogue System , 2000 .

[39]  Elisabeth André,et al.  Exploiting emotions to disambiguate dialogue acts , 2004, IUI '04.

[40]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[41]  Amy Soller,et al.  Promoting Effective Peer Interaction in an Intelligent Collaborative Learning System , 1998, Intelligent Tutoring Systems.

[42]  Julia Hirschberg,et al.  Predicting Automatic Speech Recognition Performance Using Prosodic Cues , 2000, ANLP.

[43]  Ann L. Brown,et al.  Guided, Cooperative Learning and Individual Knowledge Acquisition , 2018, Knowing, Learning, and Instruction.

[44]  William E. Lorensen,et al.  Object-Oriented Modeling and Design , 1991, TOOLS.

[45]  Ken Samuel,et al.  Dialogue Act Tagging with Transformation-Based Learning , 1998, ACL.

[46]  Judy Kay,et al.  Bringing Scrutability to Adaptive Hypertext Teaching , 2000, Intelligent Tutoring Systems.

[47]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.

[48]  Valerie J. Shute,et al.  Rose garden promises of intelligent tutoring systems: Blossom or thorn , 1991 .

[49]  John R. Anderson.  The Adaptive Character of Thought , 1990 .

[50]  Andreas Stolcke,et al.  Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech? , 1998, Language and speech.

[51]  Robert M. Aiken,et al.  Monitoring computer-based collaborative problem solving , 1995 .

[52]  David W. Johnson.  Circles of learning: Cooperation in the classroom , 1986 .

[53]  Bradley A. Goodman,et al.  Student Modeling for an Intelligent Agent in a Collaborative Learning Environment , 2003, User Modeling.

[54]  Peter McBurney,et al.  Computer Decision-Support Systems for Public Argumentation: Criteria for Assessment , 2001 .

[55]  John R. Anderson,et al.  Cognitive Modeling and Intelligent Tutoring , 1990, Artif. Intell..