Automatic Speech Recognition Performance in a Simulation-Based Fast-Jet Cockpit Application (Automatische spraakherkenning toegepast voor controletaken in de cockpit van een jachtvliegtuig).

Abstract : A project on automatic speech recognition for control of systems in a fast-jet cockpit was conducted by the TNO Human Factors Research Institute (TNO-HFRI) and the National Aerospace Laboratory (NLR). The project comprised performance testing in an advanced fast jet simulator. In total the results of 17 sorties, performed by three experienced pilots, are presented. During each sortie the pilot had access to a control by voice of radio systems, displays and HOTAS functions. During the flight tests recordings were made of the speech signals and a video recording of the pilot actions. Analysis of all pilot actions including the voice control and debriefing was performed by the NLR and is reported separately. In this report the recognizer performance is analyzed. It was found that under these simulator flight conditions the performance (accuracy) drops from over 0.95 for read speech to 0.69 for the simulator spontaneous speech condition. Results obtained in four flight experiments performed in other laboratories showed similar results for read speech (three experiments) and for spontaneous speech (one experiment). From the original 281 word vocabulary only 65 words were used frequently by the pilots. These 65 words had a coverage of 90% of all words used during the tests. This means that the complexity of the recognition process can be reduced, which will lead to a better performance of the recognizer. From the speech material a calibrated data base was built with all the speech utterances annotated orthographically at command string level. A pilot study was performed with a modern phoneme/grammar based recognizer. With this speaker independent system a mean performance of 0.85 (accuracy) was obtained. It is expected that this performance will exceed the 0/95 if this type of recognizer is trained for the non-native English speaking pilots.