Evaluation of human–agent user interfaces in multi-agent systems

Abstract A multi-agent system (MAS) is a computational system in which two or more intelligent agents work together to perform a set of tasks. MAS's have had a significant impact in a wide range of applications. However, little attention has been paid to the design of human–agent user interfaces. Our research was conducted primarily to further explore interaction between humans and intelligent agents, as well as attempt to solidify foundational concepts for intelligent multi-agent system interface design. Based on the foundation of the commercialized PC game Unreal Tournament 2003, a framework was developed that utilized fundamental interface design principles as well as newly designed guidelines. Three user interfaces were designed for an experiment conducted with 25 participants. Their performances were evaluated via effectiveness, efficiency, satisfaction, and workload metrics. Through the testing of scenario tasks, an in-depth study was conducted on the collaboration between the human user and intelligent agents within a system. Finally, the paper summarized a set of design principles and guidelines developed through the study, which could easily be applicable to the design of future user interfaces for multi-agent systems. Relevance to industry A deeper understanding of the manner by which the appropriate information is provided to the user interacting with multi-agent systems in the correct form at the right time should have a broad impact on the future development of intelligent multi-agent system interfaces.

[1]  Terry R. Payne,et al.  Task Characteristics and Intelligent Aiding , 2000 .

[2]  Gita Reese Sukthankar,et al.  Agent aided aircraft maintenance , 1999, AGENTS '99.

[3]  Jeffrey M. Bradshaw,et al.  Software agents , 1997 .

[4]  Jeffrey M. Bradshaw,et al.  An introduction to software agents , 1997 .

[5]  Lucy Suchman.  Plans and situated actions: the problem of human-machine communication , 1987 .

[6]  Milind Tambe,et al.  Intelligent Agents for Interactive Simulation Environments , 1995, AI Mag..

[7]  Jean C. Scholtz,et al.  Evaluation Methods for Human-System Performance of Intelligent Systems , 2002 .

[8]  Sascha Ossowski,et al.  ExperNet: An Intelligent Multiagent System for WAN Management , 2002, IEEE Intell. Syst..

[9]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[10]  Michal Pechoucek,et al.  CPlanT: Coalition Planning Multi-Agent System for Humanitarian Relief Operations , 2001, Multi-Agent-Systems and Applications.

[11]  Philippe R. Richard,et al.  AgentGeom: a multiagent system for pedagogical support in geometric proof problems , 2007, Int. J. Comput. Math. Learn..

[12]  Matthias Klusch,et al.  Agent-Based Integrated Services for Timber Production and Sales , 2002, IEEE Intell. Syst..

[13]  R. A. Brooks,et al.  Intelligence without Representation , 1991, Artif. Intell..

[14]  Jane T. Malin,et al.  Information for the user in design of intelligent systems , 1993 .

[15]  Arthur C. Graesser,et al.  Is it an Agent, or Just a Program?: A Taxonomy for Autonomous Agents , 1996, ATAL.

[16]  Pattie Maes,et al.  Agents that reduce work and information overload , 1994, CACM.

[17]  Victor R. Lesser,et al.  Cooperative Multiagent Systems: A Personal View of the State of the Art , 1999, IEEE Trans. Knowl. Data Eng..

[18]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[19]  Henry Hexmoor,et al.  M ODELING S OCIAL N ORMS I N M ULTIAGENT S YSTEMS , 2006 .

[20]  Rasika Amarasiri,et al.  A Hybrid Intelligent Multiagent System for E‐Business , 2004, Comput. Intell..

[21]  Jean Scholtz,et al.  Human-robot interaction: development of an evaluation methodology for the bystander role of interaction , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[22]  Nigel Shadbolt.  Working smarter, not harder , 2002, IEEE Intelligent Systems.