Including a Model of Visual Processing With a Cognitive Architecture to Model a Simple Teleoperation Task

This article describes progress in providing user models with sufficient visual information and motor control to perform teleoperation with an unmodified, physically realized robot. User models that are built by extending cognitive models to interact directly with interfaces can provide a theoretical basis for predicting user behavior. These models can help in summarizing and explaining usability issues in domains for which conventional user testing is too time consuming, too demanding of other resources, or too dynamic for static models. The user model consists of an ACT-R cognitive model and the SegMan image-processing and interpretation system. ACT-R supports directing simple rover navigation and response-time predictions. SegMan supports interpreting many aspects of HCI interfaces and can now interpret simple aspects of video used in simple navigation tasks and can generate key presses and mouse actions directly. Processing limited amounts of an image as a human fovea helped make this system work in real time. A study in robot teleoperation provides evidence that the cognitive and perceptual-motor model approximates human behavior (based on comparison of task time, learning behavior, and mouse actions) in a simple navigation task. This work demonstrates how user modeling techniques are maturing to the extent that they can be used for assessing interfaces for dynamic tasks by predicting performance during teleoperation, a common human-robot interaction task.

[1]  P. Roelfsema.  Elemental operations in vision , 2005, Trends in Cognitive Sciences.

[2]  Michael E. Atwood,et al.  Project Ernestine: Validating a GOMS Analysis for Predicting and Explaining Real-World Task Performance , 1993, Hum. Comput. Interact..

[3]  Todd R. Johnson,et al.  An ACT-R Model of Human Object-Location Memory , 2003 .

[4]  John R. Anderson,et al.  Strategic Differences in the Coordination of Different Views of Space , 2019, Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society.

[5]  Robin R. Murphy,et al.  AAAI/RoboCup-2001 Urban Search and Rescue Events , 2002, AI Mag..

[6]  D. A. Grant,et al.  Testing the null hypothesis and the strategy and tactics of investigating theoretical models. , 1962, Psychological review.

[7]  J. Gregory Trafton,et al.  Children and robots learning to play hide and seek , 2006, HRI '06.

[8]  Wayne D. Gray,et al.  Argus: A suite of tools for research in complex cognition , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[9]  Frank E. Ritter,et al.  A User Modeling Design Tool Based on a Cognitive Architecture for Comparing Interfaces , 2002, CADUI.

[10]  Frank E. Ritter,et al.  Supporting cognitive models as users , 2000, TCHI.

[11]  Wayne D. Gray.  Proceedings of the Twenty-Fourth Annual Conference of the Cognitive Science Society , 2001 .

[12]  Robin R. Murphy,et al.  Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[13]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[14]  Michael D. Byrne,et al.  ACT-R/PM and menu selection: applying a cognitive architecture to HCI , 2001, Int. J. Hum. Comput. Stud..

[15]  David E. Kieras.  The why, when, and how of cognitive simulation: A tutorial , 1985 .

[16]  Mark O. Riedl,et al.  A perception/action substrate for cognitive modeling in HCI , 2001, Int. J. Hum. Comput. Stud..

[17]  Refractor.  Vision , 2000, The Lancet.

[18]  Frank E. Ritter,et al.  Model-based evaluation of expert cell phone menu interaction , 2007, TCHI.

[19]  Anthony J. Hornof,et al.  GLEAN: a computer-based tool for rapid GOMS model usability evaluation of user interface designs , 1995, UIST '95.

[20]  Dietrich Dörner.  Proceedings of the Fifth International Conference on Cognitive Modeling , 2001 .

[21]  Jean Scholtz,et al.  Evaluation of Operator Interventions in Autonomous Off-road Driving , 2003 .

[22]  Sang-Hwan Kim,et al.  Computational Cognitive Modeling of Operator Behavior in Telerover Navigation , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[23]  Allen Newell,et al.  Cumulating the science of HCI: from s-R compatibility to transcription typing , 1989, CHI '89.

[24]  Frank E. Ritter,et al.  The Rise of Cognitive Architectures , 2007, Integrated Models of Cognitive Systems.

[25]  Michael A. Goodrich,et al.  Towards predicting robot team performance , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[26]  David Chapman,et al.  Intermediate Vision: Architecture, Implementation, and Use , 1992, Cogn. Sci..

[27]  N. Schmajuk.  Cognitive maps , 1998 .

[28]  Donald D. Dudenhoeffer,et al.  Evaluation of supervisory vs. peer-peer interaction with human-robot teams , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[29]  Dario D. Salvucci.  Predicting the effects of in-car interface use on driver performance: an integrated model approach , 2001, Int. J. Hum. Comput. Stud..

[30]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[31]  Frank E. Ritter,et al.  Model-based evaluation of cell phone menu interaction , 2004, CHI '04.

[32]  Frank E. Ritter,et al.  Image Processing in Cognitive Models with SegMan , 2005 .

[33]  Urmila Kukreja,et al.  RUI: Recording user input from interfaces under Windows and Mac OS X , 2006, Behavior research methods.

[34]  N. Halelamien.  Visual Routines for Spatial Cognition on a Mobile Robot , 2004 .

[35]  Jakob Nielsen,et al.  Chapter 4 – The Usability Engineering Lifecycle , 1993 .

[36]  Michael D. Byrne,et al.  Using Computational Cognitive Modeling to Diagnose Possible Sources of Aviation Error , 2005 .

[37]  Jean Scholtz,et al.  Theory and evaluation of human robot interactions , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[38]  Raj Madhavan,et al.  Performance metrics for intelligent systems (PERMIS) workshop , 2008 .

[39]  Jean Scholtz,et al.  Beyond usability evaluation: analysis of human-robot interaction at a major robotics competition , 2004 .

[40]  Georgios Tsaparlis,et al.  PREFACE TO THE SPECIAL SECTION , 2004 .

[41]  Jean Scholtz,et al.  Evaluation of a human-robot interface: development of a situational awareness methodology , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[42]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[43]  Frank E. Ritter,et al.  Providing user models direct access to interfaces: an exploratory study of a simple interface with implications for HRI and HCI , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[44]  Richard Reviewer-Granger.  Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[45]  Kevin A. Gluck,et al.  Comparing Three Variants of a Computational Process Model of Basic Aircraft Maneuvering , 2003 .

[46]  Jean Scholtz,et al.  Beyond Usability Evaluation: Analysis of Human-Robot Interaction at a Major Robotics Competition , 2004, Hum. Comput. Interact..

[47]  Jean C. Scholtz,et al.  Evaluation Methods for Human-System Performance of Intelligent Systems , 2002 .

[48]  S. Ullman.  Aligning pictorial descriptions: An approach to object recognition , 1989, Cognition.

[49]  Wayne D. Gray.  Integrated Models of Cognitive Systems , 2007, Oxford series on cognitive models and architectures.

[50]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[51]  Michael D. Byrne,et al.  Returning Human Factors to an Engineering Discipline: Expanding the Science Base through a New Generation of Quantitative Methods - Preface to the Special Section , 2003, Hum. Factors.

[52]  David E. Kieras,et al.  Predictive engineering models using the EPIC architecture for a high-performance task , 1995, CHI '95.

[53]  Harold W. Thimbleby.  Reflections on symmetry , 2002, AVI '02.

[54]  Dan R. Olsen,et al.  Metrics for Evaluating Human-Robot Interactions , 2003 .

[55]  Michael Lewis,et al.  USARSim: Simulation for the Study of Human-Robot Interaction , 2007 .

[56]  Frank E. Ritter,et al.  Choosing and getting started with a cognitive architecture to test and use human-machine , 2004, MMI Interakt..

[57]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[58]  Rene J. de Pontbriand,et al.  Modeling Human Behavior with Integrated Cognitive Architectures: Comparison, Evaluation, and Validation Edited by Kevin A. Gluck & Richard A. Pew 2005, 440 pages, $55.00 Mahwah, NJ: Lawrence Erlbaum Associates ISBN 0-8058-5048-1 , 2007 .

[59]  Frank E. Ritter,et al.  Useful Mechanisms for Developing Simulations for Cognitive Models , 1995 .

[60]  David Chapman,et al.  Intermediate vision: Architecture, implementation, and use☆ , 1992 .

[61]  Leon Urbas,et al.  Model Based Analysis and Design of Human-Machine Dialogues through Displays , 2005, Künstliche Intell..

[62]  Jean Scholtz,et al.  Design guidelines for improved human-robot interaction , 2004, CHI EA '04.

[63]  David E. Kieras,et al.  Predictive engineering models based on the EPIC architecture for a multimodal high-performance human-computer interaction task , 1997, TCHI.