CO-WORKER: Toward Real-Time and Context-Aware Systems for Human Collaborative Knowledge Building

The information exchange occurring during human interactions, conveyed through verbal and nonverbal communication modes, builds up a new-shared knowledge among the interacting people. A current automatic meeting assistance system is just able to store such an exchange (for successive offline processing), while it would be valuable developing automatic tools that provide appropriate support as it takes place. Currently, the international scientific community is strongly committed toward the implementation of intelligent instruments able to recognize and process in real-time relevant interactional signals in order to provide timely support to the happening interaction. This work will argue on an even more comprehensive paradigm for collaborative computer support to human interaction, not adequately addressed in the literature so far, concerning the implementation of human–computer interaction (HCI) systems able to process in real-time multimodal signals, to infer contextual information, and support in a collaborative way human interaction in-group activities, such as learning, discussion, work cooperation, decision-making, and problem solving. Such systems should act as co-workers, actively cooperating and contributing to the group’s knowledge building and pretending to share with the group, significances and individual potentialities rather than act as passive data storing devices. In carrying out their functions, these HCI systems will be placed on a group cognitive level, where individual purposes, actions, and emotions are mediated by the group interaction, and meanings are mainly built through the group shared knowledge and experience.

[1]  P. Watzlawick,et al.  Pragmatics of Human Communication: A Study of Interactional Patterns, Pathologies and Paradoxes , 1964 .

[2]  H. Garfinkel Studies in Ethnomethodology , 1968 .

[3]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[4]  A. N. Leont’ev,et al.  Activity, consciousness, and personality , 1978 .

[5]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[6]  M. Hogg,et al.  Book Review: Social identifications: A social psychology of intergroup relations and group processes , 1991 .

[7]  T Z Strybel,et al.  Aurally Aided Visual Search in the Central Visual Field: Effects of Visual Load and Visual Enhancement of the Target , 1991, Human factors.

[8]  Clarence A. Ellis,et al.  Groupware: some issues and experiences , 1991, CACM.

[9]  P. Reuter-Lorenz,et al.  Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. , 1994, Journal of experimental psychology. Human perception and performance.

[10]  Dominique L. Scapin,et al.  Evaluating a user interface with ergonomic criteria , 1995, Int. J. Hum. Comput. Interact..

[11]  E. Hutchins Cognition in the wild , 1995 .

[12]  J. Bargh,et al.  Automaticity of social behavior: direct effects of trait construct and stereotype-activation on action. , 1996, Journal of personality and social psychology.

[13]  Rosalind W. Picard Affective computing: (526112012-054) , 1997 .

[14]  Michel Beaudouin-Lafon,et al.  Computer supported co-operative work , 1999 .

[15]  Karen K Kirst-ashman,et al.  Understanding Human Behavior and the Social Environment , 2000 .

[16]  Ana C. R. Paiva,et al.  Safira: Supporting affective interactions in real time applications , 2001 .

[17]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[18]  Michael Kipp,et al.  ANVIL - a generic annotation tool for multimodal dialogue , 2001, INTERSPEECH.

[19]  Brigitte Krenn,et al.  The NECA Project: Net Environments for Embodied Emotional Conversational Agents - Project Note , 2003, Künstliche Intell..

[20]  Jeffery A. Jones,et al.  Neural processes underlying perceptual enhancement by visual speech gestures , 2003, Neuroreport.

[21]  Jeffery A. Jones,et al.  Visual Prosody and Speech Intelligibility , 2004, Psychological science.

[22]  Mary Beth Rosson,et al.  Evaluating computer-supported cooperative work: models and frameworks , 2004, CSCW.

[23]  Kostas Karpouzis,et al.  Emotion Analysis in Man-Machine Interaction Systems , 2004, MLMI.

[24]  R. Sawyer The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[25]  Alexander H. Waibel CHIL - Computers in the Human Interaction Loop , 2005, MVA.

[26]  M. Mortillaro,et al.  The Potential of Affective Computing in E-Learning : MYSELF project experience , 2005 .

[27]  M. Scardamalia,et al.  Knowledge Building: Theory, Pedagogy, and Technology , 2006 .

[28]  Daniel D. Suthers,et al.  Computer-supported collaborative learning: An historical perspective , 2006 .

[29]  Gerry Stahl,et al.  Group Cognition: Computer Support for Building Collaborative Knowledge (Acting with Technology) , 2006 .

[30]  桐山 伸也 "The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind," Marvin Minsky, Simon & Schuster, 2006(私のすすめるこの一冊,コーヒーブレイク) , 2007 .

[31]  Thomas Hain,et al.  Recognition and understanding of meetings the AMI and AMIDA projects , 2007, 2007 IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU).

[32]  Peter Wittenburg,et al.  Annotation by Category: ELAN and ISO DCR , 2008, LREC.

[33]  Hervé Bourlard,et al.  Recognition and Understanding of Meetings Overview of the European AMI and AMIDA Projects , 2008 .

[34]  Francesco Piazza,et al.  Keyword spotting based system for conversation fostering in tabletop scenarios: Preliminary evaluation , 2009, 2009 2nd Conference on Human System Interactions.

[35]  Anna Esposito,et al.  The Perceptual and Cognitive Role of Visual and Auditory Channels in Conveying Emotional Information , 2009, Cognitive Computation.

[36]  Francesco Piazza,et al.  A real-time Speech-interfaced System for Group Conversation Modeling , 2009, WIRN.

[37]  Rainer Stiefelhagen,et al.  Computers in the Human Interaction Loop , 2009, Human-Computer Interaction Series.

[38]  Pierre Dillenbourg,et al.  The Evolution of Research on Computer-Supported Collaborative Learning , 2009 .

[39]  Gökhan Tür,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. 1 The CALO Meeting Assistant System , 2022 .

[40]  Allyson Hadwin,et al.  Measurement and assessment in computer-supported collaborative learning , 2010, Comput. Hum. Behav..

[41]  Gerry Stahl,et al.  The CSCL field matures , 2010, Int. J. Comput. Support. Collab. Learn..

[42]  Gerry Stahl,et al.  Studying Virtual Math Teams , 2010 .

[43]  K. Schmidt Cooperative Work and Coordinative Practices , 2011 .

[44]  Kjeld Schmidt,et al.  Cooperative Work and Coordinative Practices - Contributions to the Conceptual Foundations of Computer-Supported Cooperative Work (CSCW) , 2011, Computer Supported Cooperative Work.

[45]  R. Bales Interaction process analysis; a method for the study of small groups. , 2013 .