Personalized change awareness: Reducing information overload in loosely-coupled teamwork

Abstract Complex tasks such as treating patients with chronic conditions and developing software products are typically accomplished by teams that collaborate over an extended time duration. To remain coordinated, team members need to be aware of others' activities if those activities are likely to affect their own actions. However, identifying such interactions and sharing information appropriately is challenging, especially when the activities of team members are loosely-coupled. In practice, team members often either lack important information about others' activities, or are overwhelmed by the need to review too much information. This paper presents Personalized Change Awareness, a new approach for supporting team coordination which aims to automatically identify and share the subset of information about others' activities that is most relevant to each of the team members. The paper formally defines the computational problem of information sharing in loosely-coupled teamwork, which underlies the personalized change awareness approach. It defines a new representation, Mutual Influence Potential Networks (MIP-Nets) and an algorithm, MIP-DOI, that uses this representation to determine the information that is most relevant to each team member. In contrast to existing information sharing algorithms in multi-agent teams, MIP-DOI does not assume the availability of a priori knowledge of a team's possible plans, because human teams rarely explicitly define detailed long-term plans in advance. We demonstrate the ability of MIP-DOI to identify relevant information using simulations of collaborative activities. We further evaluated the contribution of personalized change awareness to team performance in a controlled user study. To this end, we developed a personalized change awareness mechanism for collaborative writing, which used MIP-DOI to determine which changes to share with each author. This evaluation demonstrates that the Personalized Change Awareness approach resulted in higher productivity and lower perceived workload without any change in final quality compared to the currently prevalent approach of sharing all change information. Our results also demonstrate that merely reducing the amount of information shared with co-authors is not enough: sharing a random subset of changes resulted in significantly lower quality of work than sharing a personalized subset of changes.

[1]  David Liben-Nowell,et al.  The link-prediction problem for social networks , 2007 .

[2]  Sarit Kraus,et al.  Collaborative Plans for Complex Group Action , 1996, Artif. Intell..

[3]  E. Hutchins Cognition in the wild , 1995 .

[4]  Anand S. Rao,et al.  Design of a Concurrent Agent-Oriented Language , 1995, ECAI Workshop on Agent Theories, Architectures, and Languages.

[5]  J. Shaffer Multiple Hypothesis Testing , 1995 .

[6]  Victor R. Lesser,et al.  Coordinating multi-agent reinforcement learning with limited communication , 2013, AAMAS.

[7]  Nikos A. Vlassis,et al.  Decentralized planning under uncertainty for teams of communicating agents , 2006, AAMAS '06.

[8]  Pasquale Lops,et al.  Content-based Recommender Systems: State of the Art and Trends , 2011, Recommender Systems Handbook.

[9]  Winfried Lamersdorf,et al.  Jadex: A BDI Reasoning Engine , 2005, Multi-Agent Programming.

[10]  Hector J. Levesque,et al.  Intention is Choice with Commitment , 1990, Artif. Intell..

[11]  Francisco S. Melo,et al.  QueryPOMDP: POMDP-Based Communication in Multiagent Systems , 2011, EUMAS.

[12]  Roni Stern,et al.  To Share or Not to Share? The Single Agent in a Team Decision Problem , 2014, AAAI.

[13]  Dhanvin Mehta,et al.  Optimizing multi-robot communication under bandwidth constraints , 2019, Autonomous Robots.

[14]  Claudia V. Goldman,et al.  Optimizing information exchange in cooperative multi-agent systems , 2003, AAMAS '03.

[15]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[16]  Sophie Chen,et al.  How Big is a Big Odds Ratio? Interpreting the Magnitudes of Odds Ratios in Epidemiological Studies , 2010, Commun. Stat. Simul. Comput..

[17]  Krzysztof Z. Gajos,et al.  Mutual Influence Potential Networks: Enabling Information Sharing in Loosely-Coupled Extended-Duration Teamwork , 2016, IJCAI.

[18]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[19]  Manuela M. Veloso,et al.  Reasoning about joint beliefs for execution-time communication decisions , 2005, AAMAS '05.

[20]  Paul Dourish,et al.  Awareness and coordination in shared workspaces , 1992, CSCW '92.

[21]  P. Warner Ordinal logistic regression , 2008, Journal of Family Planning and Reproductive Health Care.

[22]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[23]  Robert J. Walker,et al.  Customized awareness: recommending relevant external change events , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[24]  Gerhard Widmer,et al.  Learning in the presence of concept drift and hidden contexts , 2004, Machine Learning.

[25]  Grant T. Harris,et al.  Comparing Effect Sizes in Follow-Up Studies: ROC Area, Cohen's d, and r , 2005, Law and human behavior.

[26]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[27]  Sarit Kraus,et al.  Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination , 2010, AAAI.

[28]  G. W. Furnas,et al.  Generalized fisheye views , 1986, CHI '86.

[29]  Frans A. Oliehoek,et al.  Dec-POMDPs with delayed communication , 2007 .

[30]  Roni Stern,et al.  Collaborative health care plan support , 2013, AAMAS.

[31]  Carl Gutwin,et al.  A Design Framework for Awareness Cues in Distributed Multiplayer Games , 2018, CHI.

[32]  Ya'akov Gal,et al.  Plan Recognition and Visualization in Exploratory Learning Environments , 2013, TIIS.

[33]  Ryan K. Williams,et al.  Constrained-Action POMDPs for Multi-Agent Intelligent Knowledge Distribution , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Gil Tidhar,et al.  Planned Team Activity , 1992, MAAMAW.

[35]  Stephanie D. Teasley,et al.  Groupware in the wild: lessons learned from a year of virtual collocation , 1996, CSCW '96.

[36]  Carl Gutwin,et al.  A comparison of usage evaluation and inspection methods for assessing groupware usability , 2001, GROUP.

[37]  Saul Greenberg,et al.  A framework for asynchronous change awareness in collaborative documents and workspaces , 2006, Int. J. Hum. Comput. Stud..

[38]  Paul Scerri,et al.  Coordinating very large groups of wide area search munitions , 2004 .

[39]  P. Lowry,et al.  Building a Taxonomy and Nomenclature of Collaborative Writing to Improve Interdisciplinary Research and Practice , 2004 .

[40]  D. Sperber,et al.  Précis of Relevance: Communication and Cognition , 1987 .

[41]  David Kaufer,et al.  Flexible Diff-ing in a collaborative writing system , 1992, CSCW '92.

[42]  Milind Tambe,et al.  Monitoring Teams by Overhearing: A Multi-Agent Plan-Recognition Approach , 2002, J. Artif. Intell. Res..

[43]  Milind Tambe,et al.  The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models , 2011, J. Artif. Intell. Res..

[44]  Gal A. Kaminka,et al.  Flexible Teamwork in Behavior-Based Robots , 2005, AAAI.

[45]  Carl Gutwin,et al.  Loose Coupling and Healthcare Organizations: Deployment Strategies for Groupware , 2006, Computer Supported Cooperative Work (CSCW).

[46]  Milind Tambe,et al.  Robust Execution-time Coordination in DEC-POMDPs Under Model Uncertainty , 2011 .

[47]  Frank van Ham,et al.  “Search, Show Context, Expand on Demand”: Supporting Large Graph Exploration with Degree-of-Interest , 2009, IEEE Transactions on Visualization and Computer Graphics.

[48]  Feng Wu,et al.  Multi-Agent Online Planning with Communication , 2009, ICAPS.

[49]  Mary Czerwinski,et al.  WIPDash: Work Item and People Dashboard for Software Development Teams , 2009, INTERACT.

[50]  Manuela Veloso,et al.  What to Communicate? Execution-Time Decision in Multi-agent POMDPs , 2006, DARS.

[51]  Jeff G. Schneider,et al.  Game Theoretic Control for Robot Teams , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[52]  Elke Hinrichs,et al.  Anticipative Awareness in a Groupware System , 2008, COOP.

[53]  Carl Gutwin,et al.  Supporting Group Awareness in Distributed Software Development , 2004, EHCI/DS-VIS.

[54]  Marc Roper,et al.  Using Developer Activity Data to Enhance Awareness during Collaborative Software Development , 2009, Computer Supported Cooperative Work (CSCW).

[55]  Lada A. Adamic,et al.  Friends and neighbors on the Web , 2003, Soc. Networks.

[56]  Krzysztof Z. Gajos,et al.  From Care Plans to Care Coordination: Opportunities for Computer Support of Teamwork in Complex Healthcare , 2015, CHI.

[57]  Peter Stone,et al.  Cooperating with Unknown Teammates in Complex Domains: A Robot Soccer Case Study of Ad Hoc Teamwork , 2015, AAAI.

[58]  Victor R. Lesser,et al.  Communication decisions in multi-agent cooperation: model and experiments , 2001, AGENTS '01.

[59]  David F. Redmiles,et al.  Supporting collaborative software development through the visualization of socio-technical dependencies , 2007, GROUP.

[60]  Rohan Padhye,et al.  NeedFeed: taming change notifications by modeling code relevance , 2014, ASE.

[61]  Daniela E. Damian,et al.  The promises and perils of mining GitHub , 2009, MSR 2014.

[62]  Sebastian Gehrmann,et al.  Deploying AI Methods to Support Collaborative Writing: a Preliminary Investigation , 2015, CHI Extended Abstracts.

[63]  Julie A. Shah,et al.  ConTaCT: Deciding to Communicate during Time-Critical Collaborative Tasks in Unknown, Deterministic Domains , 2016, AAAI.

[64]  Paul Dourish,et al.  Unifying artifacts and activities in a visual tool for distributed software development teams , 2004, Proceedings. 26th International Conference on Software Engineering.

[65]  Feng Wu,et al.  Online planning for multi-agent systems with bounded communication , 2011, Artif. Intell..

[66]  Robert Fitch,et al.  Planning-Aware Communication for Decentralised Multi-Robot Coordination , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[67]  Paul Dourish,et al.  Seeking the source: software source code as a social and technical artifact , 2005, GROUP.

[68]  Milind Tambe,et al.  Agent Architectures for Flexible, Practical Teamwork , 1997, AAAI/IAAI.

[69]  Gal A. Kaminka,et al.  Integration of Coordination Mechanisms in the BITE Multi-Robot Architecture , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[70]  Larry V. Hedges,et al.  Converting Among Effect Sizes , 2009, Introduction to Meta‐Analysis.