Human-system interface based on speech recognition: application to a virtual nuclear power plant control desk

This work reports the development of a human-system interface based on automatic speech recognition, and its integration to a virtual nuclear power plant control desk under development. The later is aimed to reproduce a nuclear power plant's control desk using virtual reality technology, for operator training and ergonomic evaluation purposes. Initially, users were able to interact with this virtual control desk only through computer keyboard and mouse. Then, an automatic speech recognition system was developed to serve as a new user interface, so they can navigate and operate this virtual control desk in front of a computer monitor or a projection screen, free from keyboard and mouse, through spoken commands. The automatic speech recognition interface developed is based on a well-known signal processing technique named cepstral analysis, and on artificial neural networks. The speech recognition interface is described, along with its integration with the virtual control desk, and results are presented.

[1]  Alan V. Oppenheim,et al.  Discrete-Time Signal Pro-cessing , 1989 .

[2]  L. F. Hanes,et al.  Use of 2.5-D and 3-D technology to evaluate control room upgrades , 2006 .

[3]  S. Markidis,et al.  A virtual control room with an embedded, interactive nuclear reactor simulator , 2006 .

[4]  Terje Johnsen,et al.  Virtual Reality in Planning and Operations From Research Topic to Practical Issue , 2006 .

[5]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[6]  Lawrence R. Rabiner,et al.  An algorithm for determining the endpoints of isolated utterances , 1975, Bell Syst. Tech. J..

[7]  Asgeir Drøivoldsmo,et al.  New Tools and Technology for the Study of Human Performance in Simulator Experiments , 2003 .

[8]  Barry H. Kantowitz,et al.  Human Factors: Understanding People-System Relationships , 1983 .

[9]  Halit Eren,et al.  Instrument Engineers' Handbook: Process Software and Digital Networks, Fourth Edition , 2011 .

[10]  Angelia Sebok,et al.  Navigation in desktop virtual environments: an evaluation and recommendations for supporting usability , 2004, Virtual Reality.

[11]  L.P. Caloba,et al.  Using neural networks for automatic speaker recognition: a practical approach , 1995, 38th Midwest Symposium on Circuits and Systems. Proceedings.

[12]  Marcos R. S. Borges,et al.  Human factors approach for evaluation and redesign of human-system interfaces of a nuclear power plant simulator , 2008, Displays.

[13]  J. Wachtel,et al.  Methodological issues in the validation of complex human-machine systems , 1995 .

[14]  Isaac José Antonio Luquetti dos Santos,et al.  The use of a simulator to include human factors issues in the interface design of a nuclear power plant control room , 2008 .

[15]  Angelia Sebok,et al.  A Comparison of Two Presence Measures Based on Experimental Results , 2004 .

[16]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[17]  Hau-San Wong,et al.  Human Computer Interaction , 2006, Encyclopedia of Multimedia.

[18]  Biing-Hwang Juang,et al.  On the use of bandpass liftering in speech recognition , 1987, IEEE Trans. Acoust. Speech Signal Process..

[19]  R. N. Pikaar,et al.  Ergonomics in control room design , 1990 .

[20]  S. Furui,et al.  Cepstral analysis technique for automatic speaker verification , 1981 .

[21]  A Drøivoldsmo,et al.  Virtual reality tools for testing control room concepts , 2002 .

[22]  Cláudio Márcio do Nascimento Abreu Pereira,et al.  MAN MACHINE INTERFACE BASED ON SPEECH RECOGNITION , 2007 .

[23]  Antônio Carlos de Abreu Mól,et al.  Full Scope Simulator Of A Nuclear Power Plant Control Room Using 3d Stereo Virtual Reality Techniques For Operators Training , 2007 .

[24]  Espen Nystad,et al.  Using virtual reality technology to include field operators in simulation and training. , 2006 .

[25]  Antônio Carlos A. Mól,et al.  Virtual Control Desks for Nuclear Power Plant Simulation: Improving Operator Training , 2008, IEEE Computer Graphics and Applications.

[26]  Ronald W. Schafer,et al.  Digital Processing of Speech Signals , 1978 .

[27]  M. N. Louka,et al.  Using virtual reality to support multi-participant human-centered design processes for control room design , 2006 .