Merging data from several sources is the key to minimising pilot workload in the cockpit of tomorrow
Graham Warwick/WASHINGTON DC
Cockpits are entering the information age in a big way, forcing designers to find ways to pre-digest and present data in forms that will inform, and not overload, the crew.
The problem is at its most acute in the cockpit of a single-seat fighter, but sensor fusion - the art of merging different data sources to create a single coherent picture - is also increasingly coming to the aid of commercial aircraft crews.
Traditionally, the correlation of data from different sources has taken place inside the pilot's head. But with the explosion of information available to the combat pilot via datalink, and the increasing emphasis on safety in commercial operations, it has become clear that crews need help.
As a result, the rapid advances in computing technology that have transformed everyone's desk into an information nexus are being brought to bear on the pilot's "office".
Sensor fusion is a key element in design of the cockpits of the USA's Lockheed Martin/Boeing F-22 and Europe's Eurofighter Typhoon. The next step is already being taken in design of the cockpit for the US/UK Joint Strike Fighter (JSF). In the commercial arena, flightdeck advances are paving the way for creation of integrated hazard warning systems.
Such advances do not happen overnight, and on both sides of the Atlantic, flying testbeds are being used to prove the concepts and demonstrate the algorithms that lie at the heart of sensor fusion. They will have accumulated several hundred hours of experience by the time sensor fusion is tested for real in a fighter.
Collecting information
Sensor fusion is central to the F-22's operational concept. Low-observable air superiority requires the aircraft to collect information while minimising emissions. The F-22's integrated avionics, therefore, must create a coherent tactical picture by fusing information from sensors that are entirely passive or barely radiating.
When it becomes operational in December 2005, the F-22 will have several onboard sensors at its disposal, including:
• the Northrop Grumman APG-77 active electronically scanned array (AESA) radar;
• the Lockheed Martin Sanders ALR-94 electronic warfare (EW) system, which includes radar warner and missile launch detector;
• the TRW communication/navigation/identification (CNI) system, which includes Link 16 and intra-flight datalinks and identification friend or foe interrogator.
All sensor processing is performed by the F-22's two Raytheon common integrated processors (CIP). It is here that sensor data are correlated and fused to create a coherent picture to be used by the avionics system for situation assessment, for display to the pilot and for tasking the sensors to collect more data.
For the pilot, the result is a single display that integrates information which, in a previous generation fighter, would have been presented on separate radar, EW and datalink displays. Targets are identified and prioritised automatically. That information is then used to control the sensors automatically.
Sensor fusion capability is not achieved overnight. The F-22's 1.7 million lines of avionics software is being developed in blocks which gradually add capability. Each block is being tested thoroughly in ground laboratories and a flying testbed before being released for flight testing in the fighter.
The first avionics-test F-22 is not scheduled to fly until early next year, but already the initial block of software is flying in Boeing's 757-based avionics testbed. Block 1 allows integration testing of the active-array radar and common integrated processor. Later this year, the aircraft will be fitted with the F-22's CNI and EW systems, to allow Block 2 flight testing to begin.
Release of Block 2 software will allow initial sensor fusion testing to begin in the flying testbed (FTB) at least nine months earlier than in the F-22. Testing of the avionics, radar, CNI and EW systems in a dynamic environment is expected to uncover integration problems that would otherwise delay F-22 flight testing.
This has already happened during Block 1 testing, says Yogi Lowell, FTB test operations manager. Flights in the modified 757 uncovered an integration problem with the Litton inertial reference system (IRS) which had not occurred in ground laboratory testing and which would have slowed F-22 flight testing, he says.
"The FTB is an essential part of the F-22 programme," says Lowell. "It allows us to test the avionics before they get on to the F-22, to look at individual elements and systems and make sure they are working." The testbed will be used for verification of the integrated avionics in a "low dynamic" regime, he says.
Fully equipped
When fully operational, by the end of this year, the FTB will be equipped with a full shipset of F-22 avionics. The radar is installed in a production-standard F-22 forward fuselage mounted on the nose of the 757 and CNI and EW systems will be installed in a sensor wing mounted above the cockpit. This features production-standard F-22 apertures and edges, aligned to provide the same sensor fields of regard as in the fighter.
For Block 1 testing, the FTB is equipped with the radar, CIP, IRS, a simulated F-22 cockpit (left) and simulations of the fighter's vehicle and stores management systems. In addition to testing these systems, the aircraft has been used for demonstration flights intended to reassure the customer that development of the F-22's complete software is on schedule - likely to become an increasingly important role for this and similar flying testbeds.
Once the CNI and EW systems have been installed, testing will accelerate. In addition to flights against instrumented airborne targets in the north-west USA, the FTB is to be deployed to Edwards AFB, California, to operate with fight-test F-22s, and to Nellis AFB, Nevada, to observe Red Flag combat exercises. "Red Flag will allow us to look at high target densities in real time, and to see the system being stressed," says Lowell.
The original plan was to develop the F-22's software in three blocks, with warfighting capability not becoming available until the third block. This is now seen as risky. "What's unique about this programme is the sensor fusion in a dynamic fighter environment," says Tom Burbage, president of F-22 prime contractor Lockheed Martin Aeronautical Systems. "We don't want to wait to test it."
Development problems with the CNI system have delayed the availability of fully tested Block 3 software. As a result, the team has created an interim block, 3S, which will allow sensor-fused avionics testing to begin sooner. "We need to fly fused data early, so we have actually pulled technical capability forward," Burbage says. Block 3S will be released to the FTB within six months of Block 2, and well before Block 3 testing is scheduled to begin in the F-22.
Block 3S is a "critical release", Burbage says, as it will contain about 80% of the software under development for the F-22, compared with the 50% in Block 1. Full Block 3 software is scheduled to be released to the FTB in the latter part of next year. This will introduce sensor track fusion and the intra-flight datalink, allowing the 757 to co-operate with avionics-test F-22s. The final block, 3.1, is to be released to the FTB in the first part of 2001, still several months ahead of the F-22.
The flying testbed is to be used for avionics troubleshooting throughout the F-22 flight test programme. "Even in the later phases, the FTB is still in front of the F-22," says Lowell. "That means fixes can be put in place and tested before the F-22 ever sees the software."
Workbase concept
Boeing first used a flying testbed during the YF-22 demonstration/validation phase, in the late 1980s, to show that sensor fusion was a workable concept. Now it is repeating the exercise for the JSF programme, flying a 737-based Avionics Flying laboratory (AFL) as part of its concept demonstration effort (below).
The AFL will be used initially to test the Raytheon AESA radar planned for Boeing's JSF. Latter phases will add electro-optical/infrared (EO/IR) sensors and other systems. The aim is to complement flight testing of two X-32 concept demonstrators by proving the open systems avionics architecture proposed for the JSF.
Lockheed Martin plans a similar demonstration jointly with Northrop Grumman, which is modifying one of its BAC One-Elevens into the Co-operative Avionics Test Bed (CATB) - nicknamed the "Catbird". The aircraft is already fitted with a Northrop Grumman AESA radar, and will be equipped progressively with electro-optical/infrared sensors and other systems planned for Lockheed Martin's JSF.
CATB flight tests will complement the work of two X-35 concept demonstrator aircraft, as well as extensive ground-based simulations in support of Lockheed Martin's JSF bid.
Much of the avionics work under way during the current concept demonstration phase revolves around sensor fusion, which will be even more important in the JSF than it is in the F-22, as the pilot workload is inherently higher in a single-seat strike aircraft.
The sensor fusion task is made even more demanding in the JSF by the customer's desire to use the full spectrum of offboard data available, from threat information provided by surveillance aircraft to target imagery downloaded directly from satellites. With the additional onboard data that will be available from the JSF's advanced ASEA and EO/IR sensors, there is a danger of overloading the pilot.
As a result, piloted mission simulations being conducted by the JSF teams are focusing on sensor fusion. "There have never been simulations of this fidelity and maturity this early in a programme," says Carl Huncharek, deputy director for mission systems with Lockheed Martin's JSF team. "Simulation is a real engineering tool, not just a show and tell."
Avionics work is aimed towards improving the pilot's ability to find targets and avoid harm without increasing workload, he says. Mission simulations have demonstrated the capability of the core onboard sensors. "Now it is time to expand the aircraft's horizons," Huncharek says, by bringing in offboard sensors.
Offboard addition
The requirement calls for the JSF to be able to perform its mission using only onboard sensors, but the teams are keen to see whether offboard data will provide more capability at no additional cost. "There are specific functions in the radar and EO that dramatically improve the capability to find and identify targets," says Huncharek. "So we meet or exceed the requirements today, but can we improve the product without charging more money?"
Mission simulations allow offboard sensor features to be added incrementally and evaluated by pilots "to see which features buy their way on to the aircraft", he says. "We are focusing on the totality of sensor fusion - entity and image fusion at various levels. Some functions have high payoff, some do not."
Fusion of offboard and onboard information is "totally transparent" to the pilot, Huncharek says. "Where he is in the mission determines which information is fused. The system automatically filters the information and decides what is important."
Some potential export customers, lacking the diversity of offboard data sources available to US forces, have expressed concern that the JSF's onboard capability might be compromised. Not so, says Huncharek. "The core capability provides significantly better accuracy than available today. Any offboard information is an additive feature."
According to Capt Simon Henley, JSF project manager at the UK Ministry of Defence, the "plug-and-play" avionics architecture allows customers to "buy in" at different levels of affordability and capability. "There will always be a top-end 'dream', but customers will be able to buy and operate the aircraft at a level they can afford."
Incremental development
Sensor fusion is key to achieving a manageable pilot workload in Europe's multi-role Typhoon. The capability is being developed incrementally, beginning with integration of the radar. Later the infrared search and track (IRST) sensor, defensive aids subsystem (DASS) and Link 16 datalink will be incorporated.
The ECR90 radar, developed by a Marconi/-Fiar/Dasa/Indra team, is already flying in two Eurofighter development aircraft and a One-Eleven "hack" aircraft. Using the testbed has cut the cost of developing the radar, says John Roulston, technical director with Marconi Electronic Systems' Avionics group.
"We use the hack where aircraft dynamics are not an issue," he says. This includes demonstrating the radar's long-range detection and tracking performance.
"If aircraft speed and dynamics are an issue, then we use the Eurofighter," he says. This includes testing close-in combat modes, where aircraft motion is a major factor.
Marconi pioneered the flying testbed approach during development of its Blue Vixen radar for the British Aerospace Sea Harrier FA2. The ECR90 hack has proved "very satisfactory", says Roulston. "It's helped get the bugs out and given us good visibility." This includes flying VIPs in the hack at last year's Farnborough.
Flight testing of the Fiar/Pilkington IRST will begin in a Dassault Falcon hack aircraft this year, ahead of flights in the Eurofighter.
Sensor fusion flight testing will get under way in 2001, by which time the Marconi/Elettronica/ Enosa DASS will also be available, and the capability is to be ready by 2004, when the first Eurofighters with full multirole operational capability are to be delivered.
Source: Flight International
