About the project

The Voice Crew Interaction (VOICI) project under Clean Sky 2 has aimed to develop an intelligent "natural crew assistant" in a cockpit environment.

The project ran 24 months, ending February 2020. Project partners were SINTEF (Norway, R&D institute, coordinator), sensiBel (Norway, SME), Multitel (Belgium, R&D institute), Acapela (Belgium, SME). Thales (France, Clean Sky 2 partner) has been Topic Manager.
The project is part of the "extended cockpit" work package under Clean Sky 2's "Systems" technology demonstration. By reducing crew workload VOICI contributes to optimization of operations; flight safety, crew awareness, better maintenance, reduced cost of operations and generally higher efficiency and lower stress.

The main goal has been to provide a proof-of-concept demonstrator at TRL 3 (*). This includes speech capture, speech recognition and interpretation/dialog, and speech synthesis for dialog with the crew. Important challenges  are robustness against a noisy environment, high speech recognition rate and robust interpretation of requests. 
(*) Technology Readiness Level 3, i.e. experimental proof of concept.

 



Results

By the project end the main goal has been achieved - providing proof of concept demonstration of the crew assistant for use cases selected by the Topic Manager. The project has contributed to innovation and job creation by improving component technologies, demonstrating the concept and establishing new cooperation within the consortium and with the Topic Manager.

blobid55.png

blobid56.png

VOICI has established an Audio Evaluation Environment, emulating a case selected by the Topic Manager: a Dassault Falcon 2000 cockpit. This includes a geometrical cockpit model and calibrated noise recording from a real Falcon 2000 flight. The latter was split into multiple channels so as to represent multiple noise sources. Hence, a realistic noise and reverberation environment has been created, including its directional properties. A digital version of the environment has been established by measuring impulse responses between multiple loudspeakers and multiple microphones.

 

Two methods of speech capture have been investigated: A pilot's headset using a close-talk microphone and an ambient microphone array. Captured speech has been processed by stand-alone speech/dialog processing functions, i.e. suitable for embedding in a cockpit without connection to external, e.g. cloud-based services. The processing includes speech enhancement, voice activity detection, context dependent speech recognition, dialog system and text-to-speech for the noisy environment.

 

Due to the high noise levels, the close-talk microphone has been found to be the preferred technology for speech capture. Still, arrays of SME partner sensiBel's very low noise optical microphones were shown to produce significant noise suppression when used with modern beamforming algorithms. VOICI has contributed to sensiBel's microphone technology development, representing a significant project exploitation potential both in aviation/transport and in a much wider market.

 

The speech processing engine has demonstrated 5% Word Error Rate for cruising flight conditions, relying on an context -dependent language model and deep learning approaches for training.  The dialog system was built to additionally include touch-screen input, together with external aviation information such as airport and weather data. It has been designed for easy inclusion of extended skills in terms of new stories, intent, entities and actions. Speech based feedback to the pilot was implemented by SME partner Acapela's text-to-speech function optimized for the noisy environment.  

The total crew assistant has been called SOLO.

blobid57.png  

               

Project Consortium

       

Relevant information

EU flag
This project has received funding from the CleanSky 2 Joint undertaking under the European Union Horizon 2020 research and innovation programme under grant agreement No. 785401
Clean sky 2 - Joint undertaking
Clean sky 2 - Joint undertaking