Towards a methodology for describing the relationship between simulation and reality

For research that carries out experiments in simulation, an important question is how the results will translate into the real world. This paper proposes a method for comparing results obtained in simulated versus physical environments, based on interval relationships between metrics gathered in both settings. The approach is motivated by the fact that the relationship between absolute measures often does not tell much. For example, the amount of time taken to complete a task in simulation versus the same task in the physical world could always be shorter in simulation because of speed-up factors embedded in the simulator. Three different metrics are introduced that describe different interval relations, and these are demonstrated using two case studies.

[1]  Stéphane Doncieux,et al.  Crossing the reality gap in evolutionary robotics by promoting transferable controllers , 2010, GECCO '10.

[2]  Richard T. Vaughan,et al.  Use Your Illusion: Sensorimotor Self-simulation Allows Complex Agents to Plan with Incomplete Self-knowledge , 2006, SAB.

[3]  Rodney A. Brooks,et al.  Artificial Life and Real Robots , 1992 .

[4]  Richard T. Vaughan,et al.  Really Reusable Robot Code and the Player/Stage Project , 2006 .

[5]  Maja J. Matarić,et al.  A Distributed Model for Mobile Robot Environment-Learning and Navigation , 1990 .

[6]  Richard T. Vaughan,et al.  The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems , 2003 .

[7]  Susan L. Epstein,et al.  On Transfer from Multiagent to Multi-Robot Systems , 2012 .

[8]  Owen Holland,et al.  Architectures for functional imagination , 2009, Neurocomputing.

[9]  James F. Allen Maintaining knowledge about temporal intervals , 1983, CACM.

[10]  Elizabeth Sklar,et al.  An empirical evaluation of auction-based task allocation in multi-robot teams , 2014, AAMAS.

[11]  Elizabeth Sklar,et al.  A first step toward testing multiagent coordination mechanisms on multi-robot teams , 2013 .

[12]  Elizabeth Sklar,et al.  A Case for Argumentation to Enable Human-Robot Collaboration , 2013 .

[13]  Wolfram Burgard,et al.  The dynamic window approach to collision avoidance , 1997, IEEE Robotics Autom. Mag..

[14]  Patrick MacAlpine,et al.  Humanoid robots learning to walk faster: from the real world to simulation and back , 2013, AAMAS.

[15]  Elizabeth Sklar,et al.  Designing the HRTeam framework: lessons learned from a rough-and-ready human/multi-robot team , 2011, AAMAS'11.

[16]  Inman Harvey,et al.  Noise and the Reality Gap: The Use of Simulation in Evolutionary Robotics , 1995, ECAL.

[17]  James F. Allen An Interval-Based Representation of Temporal Knowledge , 1981, IJCAI.