Evidence-based training as a concept is difficult to grasp, because traditional instructional programmes are also based on evidence – so why is EBT hailed as a new system?
The evidence on which traditional recurrent training programmes are built consists of general accident and incident data – admittedly from a previous era of aviation – and a traditional understanding of which manoeuvres, phases of flight or situations were safety-critical. Based on this, national aviation authorities provide a list of mandated exercises and checks for pilots of public transport aircraft. So, traditional training is at least logic-based, however selective the logic might be.
Confused training departments also face the recently introduced premise of competency-based training. CBT is the philosophy on which the training programme and assessment system for the multicrew pilot licence (MPL) was based. The problem is that existing trainees understandably believe that when they were awarded their commercial pilot licence, they earned it by demonstrating competency.
For example, and according to the traditional model, everyone knows that an engine failure at V1 (take-off decision speed) can be catastrophic if the pilot handles it badly – this is also true of a go-around with an engine failure. As a result, existing rules require pilots to fly these exercises repeatedly during their recurrent training.
What is more, statistical evidence appears to support constant repetition of these exercises, as pilot failure to manage V1 power loss or single-engine go-arounds is never – nowadays – the cause of crashes. Surely that proves traditional training was always evidence-based?
Actually, it does not. Today’s reality is not only that engine failures are far more rare than they used to be, but also that modern aircraft have plenty of power to spare and are much easier to fly with asymmetric power than their predecessors – engine power loss is no longer difficult to manage. The difficulty that veteran pilots remember in “getting the aeroplane over the hedge” following an engine failure during take-off is not the problem it used to be in an early Boeing 747-100.
EBT reflects the demands on pilots imposed by the individual airline's operating environment, instead of a formula developed for 1950s aircraft
British Airways
In fact, the primary argument for a change to today’s EBT training philosophy is that the old system was a one-size-fits-all regime designed when flying aircraft was a very different game. Traditional training sessions were more about checking, and lacked analytical input. If a pilot failed a manoeuvre or exercise, the remedy was to require the pilot to repeat it until they got it right – rather than to determine why the pilot was getting it wrong in the first place.
Now, with the marriage of EBT and competency-based training, instructors are provided with an analytical tool to enable them to identify what is going wrong and to correct it.
This tool is a system of detailed descriptions of every basic manoeuvre. Thus, instructors know what competency looks like, and they have a standardised – rather than arbitrary – way of measuring it. A system that enables instructors to recognise the precise components of a performance shortfall also enables them to rise above just being pass/fail checkers, to become a coach instead.
Today’s EBT looks at what an airline’s pilots get wrong in reality – in their modern aircraft on line flights – and is designed to allow recurrent training to be tailored to deal with identified risks and observed shortfalls in performance where they occur, whether fleet-wide, on a particular type or even for individual pilots. Flight data analysis (FDA), line-oriented evaluation (LOE) or safety audits (LOSA) make it easy to identify what goes wrong – or fails to go right – both on a fleet-wide and individual pilot basis. Information is also gleaned from pilot and crew performance in the simulator.
EBT reflects the demands on pilots imposed by the individual airline’s real operating environment, rather than relying on a training formula developed for 1950s aircraft and aimed at all airlines, everywhere. For example, short-haul crews – unless FDA indicates otherwise – may not need practice at landings because they get plenty, but ultra long-haul pilots might fly only one or two real landings a month.
While regional or commuter crews may get plenty of visual circling approaches, main line jet pilots hardly ever face them, and would be nonplussed if they were to be faced with the need to perform one.
ICAO explains the rationale behind EBT in the introduction to its manual on the subject (ICAO Doc 9995): “The existing airline pilot training requirements in national regulations are largely based on the evidence of hull losses from early generation jets, and on a simple view that in order to mitigate a risk, simply repeating an event in a training programme was sufficient.
“Over time, many new events occurred and the subsequent addition of these events to the training requirements saturated recurrent training programmes and created an inventory or ‘tick box’ approach to training.”
ICAO recognises that switching to EBT demands a big change in mindset on the part of national aviation authorities, airline training departments, instructors and examiners. It points out: “The paradigm shift proposed under the EBT programme is not simply to replace a sometimes outdated set of critical events with a new set, but to use the events as a vehicle for developing and assessing crew performance across a range of necessary competencies.
“In addition, EBT refocuses the instructor population on to analysis of the root causes to correct inappropriate actions, rather than simply asking a flightcrew member to repeat a manoeuvre with no real understanding as to why it was not successfully flown in the first instance.”
The latter, in particular, represents a massive shift in the way assessment and training is intended to be carried out on flight simulation training devices (FSTD) under EBT. Old fashioned “trapping” is out – analytical guidance and instruction is in. This places a higher demand on instructors, but should make their job far more rewarding.
On EBT instructors, ICAO says: “It is important that only those individuals who possess a good understanding of the learning process and how to positively influence human behaviour are considered for instructor positions.”
That might rule out quite a few instructors working according to the traditional box-ticking system today, so preparing EBT instructors is a difficult task in its own right. The instructors have to learn to recognise behavioural indicators that lead to poor performance if they are to apply the appropriate instruction to correct it. Conversely, they should “recognise and highlight good performance”, says ICAO. Capt David Thomas, head of flight technical and training, flight operations at British Airways says, however, that the carrier’s instructors and examiners have, almost without exception, embraced the new system with enthusiasm.
Running an EBT programme, by definition, requires the airline to gather evidence of occurrences and how they were managed, or where standards fell below that the airline and its aviation authority aims to achieve. Then the training task becomes clear.
ICAO explains: “EBT seeks to redress the imbalance between training and checking. It recognises that an assessment of competence is necessary, but once completed, pilots learn more effectively when being trained by competent instructors to perform tasks and manage events measured according to a given set of behavioural indicators, while not under pure test conditions.”
In other words, the objective is that crews should finish their recurrent session as better, more skilled and knowledgeable pilots than they were when they arrived – not just having “passed” a test in which they took on the same exercises they faced the last time.
After all, in today’s reliable, highly automated aircraft, line flying does not provide on-the-job training the way it once did, so crew exposure to the unexpected can only be provided during recurrent training. As an Emirates training captain observed a few years ago – when the carrier was first implementing EBT – 1,500 hours of flying used to provide varied experience, but now – with few exceptions – it consists of exposure 1,500 times to precisely the same routine.
Another example of the move away from “one-size-fits-all” thinking is the fact that the ICAO manual provides a “comprehensive lists of threats and errors” categorised according to which generation of aircraft the crew is being trained on: a “classic” flightdeck, a transitional one with a flight management computer (early Boeing 757/767), or full digital cockpit with flight envelope protection. In turn these are divided into jets and turboprops. This type-related guidance, derived from global experience, is intended to help airlines identify operational areas that might need training attention specific to their own operations.
Because it is based on live data, EBT naturally becomes part of an airline’s safety management system, and the monitoring and reporting systems that feed into it. ICAO warns that although FDA is useful as an information feed, pilot occurrence reporting, LOSA and other feedback systems are still required. FDA, ICAO points out, can only highlight pre-determined types of incident, usually involving measured performance that goes outside defined parameters – like speed or acceleration. But since lateral and vertical navigation requirements vary with every flight, FDA cannot be set to recognise navigation errors or altitude busts.
ICAO says EBT delivery in a recurrent training session should be structured according to a standard format. This should consist of a preparation phase in which the instructor devises lesson plans and, if this is their first exposure to it, briefs the crew on the purpose and methodology of EBT. Then, following the session briefing, the crew go through an “evaluation phase” in the FSTD.
That is followed by the “manoeuvres training phase”, then a “scenario-based training phase”, and finally an assessment followed up by a facilitated self-debriefing session.
The airlines which already use EBT say the debriefing session is priceless – the crews almost inevitably talk comprehensively through their own performance, identifying errors and imperfections, and determine what the optimum course of action would have been with practically no need for instructor facilitation. What is more, the pilots enjoy it and find the whole experience refreshingly positive.
CASE STUDY: EMIRATES
Emirates was the first of the world’s big carriers to take its advanced training and qualification programme (ATQP) clearance straight through into the usage of EBT. Capt Martin Mahoney – now Emirates’ senior vice-president flight training – arrived from Thomas Cook Airways in late 2008, and by 2009 things were beginning to change. The ATQP analysis stage began that year, and the lessons that emerged from the data were applied in recurrent training before the official ATQP approval was conferred by the Gulf Civil Aviation Authority in 2011, when the practise of EBT could begin in earnest.
Also in 2009, Mahoney pushed up the standards required from its training output by one level, on the basis that pilots at a carrier with such an extensive and varied network could not afford to accept the licencing minima as a basis for any of its pilots. This, he points out, exposed flightcrew who were “bumping along the bottom”, and needed training to elevate them to the new levels. The training usually works, he says, but if it does not, the affected pilots lose their job.
FSTD time is assigned to pilots to hone manual flying skills
Emirates
Emirates was represented on the US Federal Aviation Administration-led flightdeck automation working group, which published its groundbreaking report in 2013 on the way pilots use automation, and the way automation affects traditional pilot skills. Divisional senior vice-president flight operations Capt Alan Stealey makes clear that while automation has been a net safety benefit for the industry, it creates problems of its own. Automation is so accurate and reliable that crews can grow to trust it too much, so in the event of a system failure or an incorrect data input the pilots can fail to pick up on the error, and/or be startled by unexpected results.
The holy grail for Emirates is a training system that produces crews who have a healthy suspicion of the flight management system’s output – so they would recognise any error before it mattered and be confident in coping with the aircraft if they lost some of the automation. Mahoney talks of the qualities he wants a crew to have: “Airmanship, awareness, suspicion and common sense.” That kind of crew has the “resilience” that all the airlines are trying to develop, because a resilient crew does not “freeze” in the face of the unexpected or the surprising.
In addition to the normal twice-yearly line proficiency and operational proficiency check sessions, Emirates gives all its pilots a manual flying training hour in the simulator every six months. Mahoney explains that the carrier does not want its pilots routinely to practice manual flying with paying passengers on board, so they give them simulator time to do it.
When Flightglobal visited Emirates’ training centre in early June, a Boeing 777 crew was being put through its paces in a -300ER simulator. The simulated location was Sanaa, Yemen – an airfield with an elevation of 7,200ft – and the task was to take off, carry out a visual right hand circuit at 1,500ft above airfield level and land, with no autopilot, autothrottle or flight director.
Another circuit involved severe windshear soon after take-off, still with no automatics, and a circuit to land. Then the simulator instructor put the “aircraft” at 35,000ft for the crew to experience the approach to a high altitude stall in a gentle turn, followed by a recovery. Finally, the crew was to fly a published let-down to runway 18 at Sanaa – still manually – involving a distance measuring equipment arc to intercept the instrument landing system.
Air traffic control gave them a go-around well above minimums during the final approach, and the instructor gave them a No 1 engine flame-out followed by a visual circuit to land. For this crew, at their next manual flying session, the location and routine will change based on evidence of what is needed, and the stall recovery practice will be carried out at low level.
Emirates is currently gathering the data it needs to make its cabin crew training evidence-based as well, and it runs training sessions in its full-motion cabin simulators involving the entire crew – pilots as well. Recent statistics show the problems crews are most likely to face these days are smoke and fumes, or disruptive passengers. The airline has a complete cabin fire simulator, in which crews can practice fighting real fires, varying from a laptop battery fire to a galley oven fire.
CASE STUDY: BRITISH AIRWAYS
Capt David Thomas, head of flight technical and training, flight operations at British Airways, says an intense concentration on human factors is not new to BA, and adopting evidence-based training has been an extension of that approach. “Behaviour is a part of everything now,” he says. “We develop non-technical skills – how people do business as opposed to simply what they do.”
Training needs at BA are identified using data or reporting. This is gathered from multiple sources: flight operations data analysis, line oriented evaluation and safety audits, the company’s advanced training and qualification programme and internal pilot surveys. External sources include input from centralised airline safety reporting databases, like the International Air Transport Association’s Safety Trend Evaluation, Analysis & Data Exchange System, and the wider industry.
Having said that, BA does not have complete freedom to base all its recurrent training on feedback. Of the two recurrent training periods pilots undergo each year, one is based on flightcrew licencing legal requirements, and thus is completely prescriptive, while the other is used for operations training based on evidence of fleet- or type-specific training.
BA has linked up with Thomson Airways to carry out detailed analytical checks on pilot performance. These are led by Dr Steve Jarvis, of human factors consultancy Jarvis Bagshaw. The checks include wiring pilots in the simulator up to electrocardiograph and electroencephalograph devices to measure the effects of workload. Also measured are physical reactions like eye-blinking, pupil dilation and eye-tracking – the latter to find out precisely what crews look at, whether as pilot flying or pilot monitoring.
For example, they have discovered that a pilot’s flight instrument scans are rather random when using the fully integrated digital primary flight display with the navigation display fully functional, but revert to a more traditional and disciplined scan when raw-data only is displayed using compass rose mode.
Knowing this information in detail not only helps prepare pilots to manage high workload situations, but it can become much clearer how the monitoring pilot can best improve their support to the pilot flying. The task of the monitoring pilot is a facet of total crew performance which – until recently – had hardly been studied, and knowledge about how best to carry out that role is still in development.
Source: Flight International
