Speech recognition technology is pervasive everywhere, it seems, except in the commercial cockpit. Voice commands now field customer service queries, dictate text messages and assist drivers. Cockpit avionics, however, are commanded exclusively by human fingers and feet, which interface with the computers and controls by tapping pedals, turning knobs, pushing buttons, touching screens and manipulating sticks or yokes.
That is finally about to change, according to the staff at the Rockwell Collins advanced technology centre.
Building upon decades of research and experience, the Cedar Rapids, Iowa-based avionics specialist is poised to unveil at the NBAA convention a plan to integrate speech recognition into the commercial cockpit.
For commercial aircraft, adding an audible interface alternative to human hands and feet has had many false starts. Collins itself first considered introducing voice commands in Pro Line 21, the flightdeck system deployed in the late 1990s. As Pro Line Fusion was being developed a decade later, Collins’s marketers again touted voice recognition as a possible feature. The technology, alas, was still not quite ready.
Meanwhile, speech commands have flourished in the most advanced combat fighters. The Eurofighter Typhoon’s direct voice input system allows the pilot to, for example, audibly create a waypoint at a place of interest, one of more than 90 possible voice commands. Lockheed Martin has designed the F-35 cockpit with a system that allows a pilot to use voice commands for manually intensive “housekeeping chores”, such as changing a radio frequency.
F-35 cockpit is designed to allow voice command technology for simple tasks that are manually intensive like changing radio frequencies
Lockheed Martin
Armed with a decade of experience in combat aircraft, Collins appears poised to take the technology into the commercial market for the first time with the next generation of integrated avionics systems.
Perhaps aware of scepticism earned from a long history of false promises, Collins points to two fundamental breakthroughs that suggest this time may be different.
The first development is a new leap in the accuracy of speech recognition software. The second breakthrough is a voice command interface model that might satisfy the concerns of regulatory agencies.
“The reason voice isn’t there today is because the accuracy of voice over a wide range of pilots is not sufficient that it justifies that additional functionality,” says John Borghese, vice-president of Collins’s advanced technology centre.
Several years ago, the company set a goal for voice recognition accuracy. The software must correctly interpret well over 90% of words spoken in English, regardless of a pilot’s accent or diction, Borghese says. The software must work as well whether the command is spoken by a European, Asian, African or American, as well as whether it is in a wide range of regional accents within those groups.
Collins developed speech recognition software and spent several years in testing, canvassing the company’s wide network of pilot advisory groups. The pilots were invited to test the system by speaking commands in cockpit simulators, with any errors of interpretation analysed and corrected in the software algorithm. The process was repeated until the software proved it was accurate beyond the manufacturer’s stated threshold.
“We don’t see a difference in accuracy across accents, and that’s because we do a lot of homework,” says senior systems engineer Geoff Shapiro.
Refining the recognition software was only part of the challenge. To field the technology, the avionics company also needs to satisfy regulators that errors in interpretation or execution could never jeopardise safety.
Voice recognition systems can be used to bypass long navigation menus, speeding up data access
Rockwell Collins
On a recent visit to the advanced technology centre in Cedar Rapids, Shapiro demonstrated how the system works in a cockpit simulator.
“This button here we’re going to call our push control voice button,” he says, wearing a headset with his thumb over a red button on the control stick.
As long as Shapiro holds his thumb on the button, the system is “listening” to him speak. If he lifts his thumb off the button, the speech recognition system stops listening.
“I’m going to give it a command of a heading,” Shapiro says, toggling the red activation button.
“Turn left, heading 060,” he says.
The system audibly repeats the command and asks him to approve the heading change. Shapiro confirms by clicking the button on his stick again. In the simulated environment, the flight management system had been following a flight plan. Shapiro’s voice command and confirmation suspends the flight plan and switches the autopilot to a heading mode, then turns the aircraft left to the new heading.
Such an action is called a “macro command”, because it directs more than a single task. Shapiro next attempts to set a new altitude, but intentionally mumbles the command. The speech recognition system audibly requests him to “say again”. Shapiro then says, clearly, “set altitude to 10,000”. Again, nothing happens.
Shapiro explains it’s because the automation system recognises that is not a valid command. He needs to use a macro command that will perform three discrete tasks – switching from heading mode to a vertical speed mode, an ascent or a descent rate and a new altitude level.
“I can say: ‘climb and maintain flight level 310,’” he says. “It’s a macro command. It changed my mode to vertical speed, set my bug to 31,000ft and commanded a 1,500ft/min [7.62m/s] climb.”
So far, Shapiro has demonstrated a series of voice commands that appear to effectively control the simulated aircraft, but do not seem obviously superior to the conventional interface – the flight director mounted on top of the glare shield.
Shapiro concedes that some voice commands do not offer time or simplicity advantages over a manual system.
“Some things will, some things won’t,” he says. “If you’re flying along and ATC gives you a fix that you don’t know how to spell, how do you enter that into your [centre display unit]? Fifteen seconds later, you sheepishly call ATC, saying: ‘Hey, can you spell this fix for us?’ It happens quite a bit. With speech recognition, you don’t have that problem because everything is phonetic.”
Another possible advantage of a speech recognition system is helping pilots navigate the lengthy menus of features. Aircraft cockpit systems have become increasingly automated and sophisticated. Browsing through checklists and functions can take dozens of seconds. So Collins and other avionics companies have been trying to simplify the process of accessing the right information, and speech recognition is one of the tools.
“Let’s just say I want to bring up a…,” says Shapiro, then he remembers to toggle the voice command button.
“Bring up the engines,” he commands, then approves the confirmation request. A display showing the status of each engine flashes on the screen.
“I can control all the flightdeck content through speech as well. We have definitely seen serious time changes because of that,” Shapiro says. “I can get to the data quicker and bypassing several systems. That’s why this bypassing the menu makes sense here. Not everything will be better with voice but we believe enough will be better for this to be a benefit.”
The introduction of voice recognition technologies adds capability, but also raises a question: Is English always the best language to use in the cockpit? English is the universal language of air traffic control, but it is also a workload issue for pilots with a different primary language.
“When you fly, you speak English. It’s the standard,” Shapiro says. “Should they maintain English speaking? Certainly, there are arguments for both. If you’re Chinese, and you interface with your avionics using your natural language that’s one less barrier for you. It’s a valid paradigm to look at. I don’t think we’ve made up our mind what the solution is but the technology enables both.”
Source: FlightGlobal.com
