Towards generalized performance metrics for human-robot interaction

In order for cognitive robots to act adequately and safely in real world, they must be able to perceive and have abilities of reasoning up to a certain level. Toward this end, performance evaluation metrics are used as important measures to achieve these goals. This paper intends to be a further step towards identifying common metrics for task-oriented human-robot interaction. We believe that within the context of human-robot interaction systems, both human and robot independent actions and joint interactions can significantly affect the quality of the accomplished task, thus proposing a generic performance metric to assess the performance of the human-robot team. Toward the efficient modelling of such metric, we also propose a fuzzy temporal model to evaluate the human trust in automation while interacting with robots and machines to complete some tasks. Trust modelling is critical as it directly influences the interaction time that should be directly and indirectly dedicated toward interacting with the robot. Another fuzzy temporal-based model is also presented to evaluate the human reliability during interaction time, as many research studies state that a large percentage of system failures are due almost equally to humans and machines, and therefore, assessing this important factor in human-robot interaction systems is also crucial. The proposed framework is based on the most recent work in the area of cognitive human-machine interaction and performance evaluation.

[1]  Balbir S. Dhillon,et al.  Human Reliability and Error in Medical System , 2003, Series on Industrial and Systems Engineering.

[2]  Mingsheng Ying,et al.  A formal model of computing with words , 2002, IEEE Trans. Fuzzy Syst..

[3]  Birsen Donmez,et al.  Evaluation criteria for human-automation performance metrics , 2008, PerMIS.

[4]  P. Pina Identifying Generalizable Metric Classes to Evaluate Human-Robot Teams , 2008 .

[5]  N. Moray,et al.  Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. , 1996, Ergonomics.

[6]  Kevin W Williams,et al.  A Summary of Unmanned Aircraft Accident/Incident Data: Human Factors Implications , 2004 .

[7]  Michael A. Goodrich,et al.  Seven principles of efficient human robot interaction , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[8]  Dan R. Olsen,et al.  Metrics for Evaluating Human-Robot Interactions , 2003 .

[9]  Eugene S. Santos,et al.  Maximin Automata , 1968, Inf. Control..

[10]  Jean Scholtz,et al.  Common metrics for human-robot interaction , 2006, HRI '06.

[11]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[12]  Leonardo Maria Reyneri An Introduction to Fuzzy State Automata , 1997, IWANN.

[13]  K. Peeva Fuzzy automata and languages: theory and applications , 2004 .

[14]  Mansoor Doostfatemeh,et al.  New directions in fuzzy automata: a general and more usable formalism , 2006 .

[15]  Daniel Scherer,et al.  Proposing Strategies to Prevent the Human Error in Automated Industrial Environments , 2009, HCI.