Using Agents in Virtual Environments to Assist Controllers to Manage Multiple Assets

Search and rescue operations often require complex coordination of a range of resources, including human and robotic resources. This paper discusses a proposed new framework that allows agent technology to be used in conjunction with a virtual environment to provide a human controller with an effective visualisation of the distribution of a collection of autonomous objects, in our case, Unmanned Aerial Vehicles (UAVs) so that they can be managed in a way that allows them to successfully complete the task in the minimum possible time. It is our contention that to do this effectively there needs to be two-way initiation of verbal conversations, but that it is not necessary for the system to completely understand the conversations required. An example scenario is presented that illustrates how such a system would be used in practice, illustrating how a single human can communicate with a swarm of semi-autonomous actors verbally and envisage their activities in a swarm based on the visual cues provided within the virtual environment. An agent-based solution is proposed that meets the requirements and provides a command station that can manage a search using a collection of UAVs effectively.

[1]  Peter Wallis,et al.  Dialogue Modelling for a Conversational Agent , 2001, Australian Joint Conference on Artificial Intelligence.

[2]  Frank Dignum Agents for games and simulations , 2011, Autonomous Agents and Multi-Agent Systems.

[3]  Stuart Cunningham,et al.  Research on Social Engagement with a Rabbitic User Interface , 2009 .

[4]  Peter Wallis,et al.  FROM DATA TO DESIGN , 2011, Appl. Artif. Intell..

[5]  Anand S. Rao,et al.  BDI Agents: From Theory to Practice , 1995, ICMAS.

[6]  Roger C. Schank,et al.  SCRIPTS, PLANS, GOALS, AND UNDERSTANDING , 1988 .

[7]  M. Tomasello Origins of human communication , 2008 .

[8]  Sabine Payr,et al.  Socially Situated Affective Systems , 2011 .

[9]  Emma Norling,et al.  Modelling human behaviour with BDI agents , 2009 .

[10]  Milica Gasic,et al.  The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management , 2010, Comput. Speech Lang..

[11]  Clinton Heinze,et al.  Thinking Quickly: Agents for Modeling Air Warfare , 1998, Australian Joint Conference on Artificial Intelligence.

[12]  Chung Hee Hwang,et al.  The TRAINS project: a case study in building a conversational planning agent , 1994, J. Exp. Theor. Artif. Intell..

[13]  Oliver Lemon,et al.  A Wizard-of-Oz Environment to Study Referring Expression Generation in a Situated Spoken Dialogue Task , 2009, ENLG.

[14]  Steve J. Young,et al.  Partially observable Markov decision processes for spoken dialog systems , 2007, Comput. Speech Lang..

[15]  Gregor Mehlmann,et al.  Multiple Agent Roles in an Adaptive Virtual Classroom Environment , 2010, IVA.

[16]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[17]  Emma Norling,et al.  What should the agent know?: the challenge of capturing human knowledge , 2008, AAMAS.

[18]  Cyril Brom,et al.  A Periphery of Pogamut: From Bots to Agents and Back Again , 2010, AGS.

[19]  Adrian R. Pearce,et al.  Plan Recognition in Military Simulation: Incorporating Machine Learning with Intelligent Agents , 1999 .

[20]  Lyuba Alboul,et al.  A Robot Swarm Assisting a Human Fire-Fighter , 2009, Adv. Robotics.

[21]  Lyuba Alboul,et al.  Multi-robot team formation control in the GUARDIANS project , 2010, Ind. Robot.

[22]  P. Wallis Believable Conversational Agents : Introducing the Intention Map , 2022 .

[23]  Markus Stumptner,et al.  AI 2001: Advances in Artificial Intelligence , 2002, Lecture Notes in Computer Science.

[24]  Alexiei Dingli,et al.  Demonstration of a Prototype for a Conversational Companion for Reminiscing about Images , 2010, ACL.

[25]  Georgios Chliveros,et al.  A system for reconstruction from point clouds in 3D: Simplification and mesh representation , 2010, 2010 11th International Conference on Control Automation Robotics & Vision.

[26]  Lyuba Alboul,et al.  Experiments in cooperative human multi-robot navigation , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  Oliver Lemon,et al.  Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation , 2011, Comput. Speech Lang..

[28]  C. Pelachaud,et al.  Emotion-Oriented Systems: The Humaine Handbook , 2011 .