A5 — The Institutional and Regulatory Response: How Aviation Rebuilt Pilot Training

August 24, 2026

Stream: The machinery by which aviation redesigned pilot training — AQP, MPL, EBT, CBTA, UPRT — and the data infrastructure without which none of it functions.

Verification key: [VERIFIED] = confirmed against the primary document itself in this research session. [SECONDARY] = confirmed against a reliable secondary source (regulator summary, established safety body, peer-reviewed literature) but not the primary text. [UNVERIFIED] = could not confirm; treat as a lead, not a fact.

1. The Core Move: Regulators Trading Prescription for Evidence

The single most important structural fact about aviation’s training reform is this: a regulator agreed to stop prescribing what training must contain, in exchange for the operator proving, with data, that its alternative worked. Everything else — MPL, EBT, CBTA, UPRT — is downstream of that bargain, first struck in the United States in 1990.

This matters for any transfer argument. The mechanism is not “aviation discovered better pedagogy.” It is “aviation built a regulatory instrument that made deviation from prescription conditional on evidence, and simultaneously built the data infrastructure that made evidence possible.” Those two things are inseparable. A transfer argument that takes the pedagogy and leaves the instrument and the data plumbing will not reproduce the result.

2. AQP — The FAA’s Advanced Qualification Program (1990– )

2.1 SFAR 58 and its statutory shape

The Advanced Qualification Program was created by Special Federal Aviation Regulation No. 58 (SFAR 58), published at 55 FR 40262, Federal Register Vol. 55, No. 191, Tuesday 2 October 1990, Docket No. 25-04, effective 2 October 1990, with a five-year sunset (2 October 1995). [VERIFIED] URL: https://www.faa.gov/sites/faa.gov/files/2022-11/FR_55_40262%20SFAR%2058.pdf

The regulatory trade is stated in Section 1(c): an AQP must provide “at least an equivalent means of compliance with current regulations.” [VERIFIED] Operators gained latitude over how they trained — curriculum structure, sequencing, media, timing, and crucially the abandonment of fixed programmed hours — and in return accepted a permanent obligation to collect and surrender performance data.

SFAR 58 permitted, against traditional Part 121/135 training:

Proficiency-based rather than hours-based qualification. Training continues “until the student can show that he or she has mastered the material,” rather than for a prescribed number of hours. [VERIFIED]
Data-driven curriculum development. Certificate holders collect de-identified performance data from trainees, instructors and evaluators throughout the programme’s life; the FAA uses it to validate that safety is maintained or exceeded. [VERIFIED]
Formal instructor and evaluator qualification. Structured evaluation criteria and recurrent training for evaluators themselves — a move beyond the traditional line-check-pilot model. [VERIFIED]
LOFT and integrated CRM. Line-Oriented Flight Training and CRM became required curriculum elements rather than optional additions. [VERIFIED]

2.2 Migration into 14 CFR Part 121 Subpart Y

SFAR 58 was made permanent by migration into 14 CFR Part 121, Subpart Y — Advanced Qualification Program. The eCFR source credit for the subpart is Docket FAA-2005-20750, 70 FR 54815, 16 September 2005. [VERIFIED] URL: https://www.ecfr.gov/current/title-14/chapter-I/subchapter-G/part-121/subpart-Y

The subpart runs §121.901 to §121.925: [VERIFIED]

Section	Title
§121.901	Purpose and eligibility
§121.903	General requirements for Advanced Qualification Programs
§121.905	Confidential commercial information
§121.907	Definitions
§121.909	Approval of Advanced Qualification Program
§121.911	Indoctrination curriculum
§121.913	Qualification curriculum
§121.915	Continuing qualification curriculum
§121.917	Other requirements
§121.919	Certification
§121.921	Training devices and simulators
§121.923	Approval of training, qualification, or evaluation by a person who provides training by arrangement
§121.925	Recordkeeping requirements

Note: the full text of §121.901 could not be retrieved in this session; only the section titles and source credit were confirmed. [UNVERIFIED — §121.901 substantive text]

2.3 The design machinery: AC 120-54A

The operative guidance is AC 120-54A, Advanced Qualification Program, dated 23 June 2006, Change 1 dated 31 January 2017, cancelling AC 120-54 (9 August 1991). [VERIFIED] URL: https://www.faa.gov/sites/faa.gov/files/2022-11/AC-120-54A,%20Chg.1,%20AQP.pdf

AQP approval proceeds through five sequential phases, each requiring FAA sign-off before the next. Phase II requires five documents: Job Task Analysis (JTA); Qualification Standards; ISD Methodology; Curriculum Outlines; Implementation and Operations Plan (I&O Plan). [VERIFIED]

The JTA is the foundation: “A task is a unit of work within a function having an identifiable beginning and ending point, which results in a measurable product.” [VERIFIED] From tasks come Terminal Proficiency Objectives (TPOs); from subtasks, Supporting Proficiency Objectives (SPOs); and Enabling Objectives (EOs) for knowledge, cognitive, motor and CRM factors. Objectives are rated by criticality and currency — critical tasks are trained and evaluated more often; currency tasks are those maintained by routine line performance and therefore need less formal periodic instruction. [VERIFIED]

Critically, the operator, not the regulator, writes the objectives, and they then become binding: “Each air carrier applicant, rather than the FAA, develops its own TPO’s on the basis of an instructional systems development (ISD) process.” [VERIFIED — Longridge, FAA]

The equivalence obligation is explicit: “The certificate holder must establish an initial justification and a continuing process to show how the AQP curriculum provides an equivalent level of safety for each requirement to be replaced,” and equivalence is judged systemically — “the entire training system must be examined as a comprehensive whole rather than considering any one component in isolation.” [VERIFIED]

Data obligations: operators must “develop data collection and analysis processes in order to obtain performance information on crewmembers, dispatchers, instructors, evaluators, and other operations personnel that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum.” [VERIFIED] Two databases are named: the Performance/Proficiency Database (PPDB) and the Program Audit Database (PADB), with an annual report. The FAA’s branch AFS-280 “uses de-identified data for program monitoring not to monitor individual crewmember, dispatcher or other operations personnel.” [VERIFIED] The de-identification is not incidental — it is the condition on which the data flows at all (see §8).

2.4 First Look — the ancestor of the EBT evaluation phase

AQP introduced a mechanism that later became the structural core of EBT. “First Look” evaluates selected TPOs before any briefing or practice: “Performance on selected TPO’s will be evaluated prior to each formal training session – that is, prior to any pre-briefing or practice on such items.” [VERIFIED — Longridge] Individually it diagnoses skill decay; in aggregate it tells the operator which parts of its curriculum are failing. The continuing qualification cycle — initially 26 months maximum — “may be subsequently extended by the FAA in three months increments to a maximum of 39 months, contingent upon the results of performance proficiency data.” [VERIFIED] The regulator literally priced the training interval against evidence.

2.5 Published evaluations of AQP’s effects

This is the first significant honesty problem in the stream. Rigorous published outcome evaluations of AQP are thin. Longridge’s own FAA overview does not provide metrics, benchmarks or empirical comparison methodology; it establishes the evidentiary framework rather than reporting results, and explicitly flags the unresolved measurement problem: AQP “generated a need to adapt psychometric methods to the development of effective performance measurement techniques, to establish appropriate data analysis strategies, and to enhance the reliability, sensitivity, and validity of measures of quality control.” [VERIFIED]

The most substantial published analysis of AQP performance data I located is not an FAA evaluation at all — it is Section 3.4.1 and Appendix 9 of the IATA EBT Data Report (2014), which used a mature US AQP operator’s grading data across a 24-month cycle to study skill mastery, retention and decay, comparing Pilot Non-proficient Grades (PNGs) across aircraft generations and training events. [VERIFIED] That is, AQP’s chief documented evidential yield was as an input to EBT, not as a self-evaluation.

Related contemporaneous literature exists (e.g. Hamman & Holt on Line Operational Evaluation, 1997, Proceedings of the Human Factors and Ergonomics Society Annual Meeting) but was not retrieved in full. [UNVERIFIED — content and findings]

Bottom line for the report: AQP’s defensible claim is procedural, not outcome-statistical. It demonstrably changed how curricula are built and what data exists. The claim that it measurably reduced accidents is not, on the evidence I could verify, established in the public literature.

3. MPL — The Multi-crew Pilot Licence (2006– )

3.1 Origin and legal basis

The MPL was introduced by Amendment 167 to ICAO Annex 1 — Personnel Licensing, adopted by the ICAO Council on 10 March 2006 and applicable 23 November 2006. [VERIFIED — ICAO Personnel Licensing FAQ] URL: https://www.icao.int/personnel-licensing-faq

Supporting procedures sit in Doc 9868, PANS-TRG, Part II, Section 1, Chapter 1 (“Competency-based training and licensing for the multi-crew pilot licence (MPL)”), with Appendices 1–3 (implementation guidelines; training scheme; competency units, elements and performance criteria) and Attachments A–D (programme design; sample training objectives; threat and error management; competencies of MPL examiners and inspectors). [VERIFIED — PANS-TRG 2nd ed. contents]

The ICAO standard specifies 240 hours minimum of actual and simulated flight time as pilot flying and pilot non-flying, with flexibility in the split, plus all PPL actual flying time and additional instrument, night and upset recovery flying. [VERIFIED]

3.2 What is structurally novel

Three things:

It is the first ICAO licence that is CBTA-compliant from the ground up. IATA states this plainly: “In 2006, ICAO supported a performance-based approach to training with the publication of standards for the Multi-crew pilot license (MPL), which is the first license that is CBTA compliant.” [VERIFIED — IATA CBTA White Paper, March 2024]
Threat and Error Management is embedded from day one, not bolted on after type rating — Attachment C to PANS-TRG Part II/1/Chapter 1 is dedicated to TEM. [VERIFIED]
Multi-crew from the outset. The trainee is trained for the job they will actually do (right seat of a multi-crew jet), rather than trained as a single-pilot commander and then converted.

The four-phase structure (core flying skills → basic → intermediate → advanced) is standard in secondary descriptions of the MPL scheme but I did not confirm the phase names against Appendix 2 to PANS-TRG Chapter 1 in this session. [UNVERIFIED — exact phase nomenclature]

3.3 Evidence on MPL graduate performance

Advocates. Flight Safety Foundation’s AeroSafety World reported an analysis of 586 trainees across 15 MPL programmes: all but one reporting ATO had a 100% pass rate (the other 98%); average 105 sectors flown before full qualification (range 70–281); approximately 21 execution-challenge incidents across the 586 datasets (~3.6%), mostly workload management, cruise monitoring and precision approaches; programmes averaging ~300 hours (range 240–330). Captains initially apprehensive reportedly came to prefer flying with MPL graduates over conventionally trained cadets. IATA data to end-2013: 2,400+ students enrolled, 800 graduates, ~30 airlines, with operators reporting graduate performance “as good as or better than students from traditional training.” [SECONDARY] URL: https://flightsafety.org/asw-article/bright-prospects/

Adoption. SKYbrary reports that as of December 2018 IATA tracked 3,433 MPL graduates and 39 approved training organisations, and that by 2014 ICAO data showed 26 States with MPL in their regulations — including Australia, Canada, China, Germany, Netherlands, Singapore, Spain, Sweden and the UK. The FAA does not issue MPLs; ALPA opposes US adoption. [SECONDARY] URL: https://skybrary.aero/articles/multi-crew-pilot-licence-mpl

Critics. The European Cockpit Association position paper of 19 December 2013 argues implementation compromised safety: MPL minima permit “a 50% reduction of real flight hours and 75% for PIC hours”; “there is no relevant Air Traffic Control (ATC) simulated environment available to date”; some approved syllabi contained only six landings instead of twelve, and some omitted “real Instrument Flight Rules (IFR) flight” or “asymmetric flight in real aircraft.” Its sharpest charge is conceptual: “the competency-based training concept was misunderstood or misinterpreted,” producing hour-based programmes wearing competency-based labels. ECA also noted “scarce/limited data feedback on the performance of MPL cadets and pilots” and that at that date “no MPL trainee has graduated to Captain yet.” [VERIFIED] URL: https://www.eurocockpit.eu/positions-publications/eca-position-paper-multi-crew-pilot-license

IFALPA, following a May 2013 workshop drawing on line captain reports, MPL advisory board data and ATO site visits, criticised training that “relies significantly more on simulator time, less on flight time; reduces solo flight experience; reduces actual landings” and “emphasizes flight management skills and automation to the detriment of basic flying skills, and provides limited training in airmanship, CRM.” [VERIFIED — text] The document number appears to be 15POS03 but this was inferred from a filename, not read off the document. [UNVERIFIED — document number]

Assessment for the report. The MPL evidence base is weak in a specific and instructive way: the positive data are training-system data (pass rates, sectors to line release) supplied by the organisations delivering the training, with no control group and no long-horizon career data. The critics’ central objection — that a competency-based licence was implemented as an hours-reduction exercise — is a claim about implementation fidelity, and it is exactly the failure mode any transfer of CBTA to another sector should expect.

4. EBT — Evidence-Based Training

4.1 What “evidence” actually means

This is the point the wider L&D world routinely gets wrong, and it must be stated precisely.

“Evidence-based” in EBT does not mean “there is evidence that this training works.” It means “the curriculum content and its frequency were derived from operational data rather than from tradition or regulation.” The evidence sits upstream of the curriculum, not downstream of the outcome.

The definitions confirm this. ICAO Doc 9995 defines EBT as “Training and assessment based on operational data that is characterized by developing and assessing the overall capability of a trainee across a range of core competencies rather than by measuring the performance in individual events or manoeuvres.” [VERIFIED] EASA’s definition in Regulation (EU) 2020/2036 is materially identical: “assessment and training based on operational data that is characterised by developing and assessing the overall capability of a pilot across a range of competencies.” [VERIFIED]

IATA restates it without ambiguity: “EBT is a CBTA program that uses specific training topics as vehicles to develop the pilot competencies. The training topics and their associated frequency were defined during the EBT design phase, through the analysis of both safety and training data from a worldwide perspective.” [VERIFIED — IATA CBTA White Paper, March 2024, §3.1]

The rationale ICAO gives is explicitly about the limits of scenario coverage: “It is impossible to foresee all accident scenarios… EBT addresses this by moving from pure scenario-based training to prioritizing the development and assessment of key competencies… Mastering a finite number of key competencies should allow a pilot to manage unforeseen situations in flight.” [VERIFIED — PANS-TRG 2nd ed., II-1-2-1, §2.2.2]

So the logic chain is: operational data → prioritised training topics by aircraft generation → topics used as vehicles → to develop competencies → which are the transferable countermeasures against unforeseen events. The evidence justifies the syllabus; the competencies are the claimed mechanism of transfer.

4.2 The documents, exactly

ICAO Doc 9995, AN/497, Manual of Evidence-based Training, First Edition, 2013. ISBN 978-92-9249-242-7. [VERIFIED] URL: https://skybrary.aero/sites/default/files/bookshelf/3177.pdf
IATA/ICAO/IFALPA, Evidence-Based Training Implementation Guide, 1st Edition, July 2013. ISBN 978-92-9252-191-2. [VERIFIED] URL: https://salient.aero/wp-content/uploads/2024/02/ebt-implementation-guide.pdf
IATA, Data Report for Evidence-Based Training, 1st Edition, August 2014. ISBN 978-92-9252-228-5. [VERIFIED] URL: https://www.iata.org/contentassets/c0f61fc821dc4f62bb6441d7abedb076/data-report-for-evidence-basted-training-ed20one.pdf

Correction to the common account: the Data Report is dated August 2014, not 2013. Many secondary sources (and the original brief for this stream) date it 2013. IATA’s own 2024 white paper confirms: “The publication of the EBT Data Report, 1st Edition, in 2014.” [VERIFIED] Doc 9995 (2013) therefore preceded the full publication of the data report that underpins it — the Manual came out while the evidence base was still being written up.

ICAO Doc 9868, PANS-TRG. First edition 2006; Second Edition 2015, superseding all previous editions on 10 November 2016; Third Edition 2020 (ICAO Store order code DOC-09868-003-01). [VERIFIED] Amendment 5 (2016) added “General provisions for competency-based training and assessment,” defining the role of pilot competencies within TEM. Amendment 7 (2020) “formalized the global expansion and applicability of CBTA principles to all licensing training (ICAO Annex 1) and operator training (ICAO Annex 6).” [VERIFIED — IATA CBTA White Paper §3.2] A version of PANS-TRG 3rd Edition incorporating Amendment 7 is published by Airservices Australia dated 27 November 2025; the file itself did not retrieve. [UNVERIFIED — that specific document's contents]

4.3 The actual evidence base — what was analysed

The IATA Data Report is built on 7 data streams and 18 specific data sources. [VERIFIED] The streams: cockpit observation reporting; flight data analysis studies; accident/incident analyses; training studies; airline pilot survey on training effectiveness; scientific reports; training criticality survey. [VERIFIED]

Dataset sizes, exactly: [VERIFIED]

Source	Scale
LOSA Archive	over 9,000 observations across 45 airlines worldwide
EBT Accident–Incident Study	3,045 accidents and incidents, 1962–2010, NTSB database as primary source; ~2,600 jet and ~350 turboprop events analysed; ~4% of targeted reports omitted as incomplete
EBT Flight Data Analysis	in excess of 1.7 million flights for Generation 3 and 4 aircraft across 9 models; a core cross-section of 5 types / 9 models over 3 years covering 1.6 million flights
Airline Pilot Survey on Training Effectiveness	966 respondents, distributed via IFALPA, open 110 days
Training Criticality Survey (TCS)	167 pilots, 161 three-part questions covering 40 threats and errors across all flight phases, spanning 51 aircraft types/variants; no Generation 1 jet respondents
Evidence Table	over 300 evidential statements
Training data	US AQP operator (mature programme, 24-month cycle) and European ATQP implementation data

Aircraft generations: six — four jet, two turboprop. [VERIFIED] Generation 4 jets are the fly-by-wire/glass generation (A320 family, A330/A340/A350/A380, B777, B787, Embraer E-Jets); Generation 3 covers A310/A300-600, B737-300 onwards, B757/B767, B747-400, CRJ, ERJ; Generation 2 the first-generation jets (B727, B737-100/200, DC-9, DC-10, early A300); Generation 1 the B707 and DC-8. [VERIFIED — Doc 9995 / Data Report figure 3.2.1.1]

Methodology: threats and errors were characterised on three dimensions — likelihood, severity, and training benefit — producing a criticality ranking of training topics per generation, then assigned priority A, B or C determining exposure frequency within a three-year rolling recurrent programme. [VERIFIED] EASA’s implementation maps these to frequency: A = every 6 months, B = every year, C = every 3 years. [SECONDARY — EASA EBT presentation notes]

The report is explicit that it is an analysis of analyses: “While the EBT Data Report is not a meta-analysis in a pure sense, it is derived from an analysis of analyses using a variety of sources and techniques to corroborate and challenge its own findings.” [VERIFIED]

4.4 What the evidence found

The findings are the substantive justification for everything EASA and ICAO subsequently built. Exact figures: [VERIFIED]

Manual flight path control

Manual aircraft control was a factor in 52% of all fatal accidents.
It was a factor in 84% of accidents and serious incidents having a high probability of mitigation through training.
LOSA error-management data: pilots detect only around 40% of aircraft handling errors. Commanders detect 39% of their first officers’ handling errors but only 9% of their own.
Runway excursions accounted for almost 30% of all fatal accidents 2000–2010, most with a manual control factor — a 12% increase on the prior decade (augmented CAST data).
A counter-intuitive nuance the report is careful about: skill-decay studies indicate manual skills as tested in manoeuvre validation are resistant to decay — but “that test is given a vacuum of realism, with no attendant distractions or environmental challenges.”

The unstable approach paradox — the report’s most striking finding

97% of unstable approaches result in a landing, of which 90% are uneventful (LOSA); FDA corroborates at ~98%.
Overall go-around rate is approximately 0.31% (FDA). Go-arounds occur on only 3% (LOSA) / 1.4% (FDA) of unstable approaches, despite SOPs mandating them.
A go-around from an unstable approach produces high-severity FDA events at 0.24 vs 0.13 per event — an 85% increase over go-arounds from stable approaches.
High-severity landing event rate is 140% higher after unstable approaches; the most dangerous landing events 179% higher.
Even in flight phases unrelated to approach and landing, flights with unstable approaches show ~19–20% higher overall event rates and 35% higher severe event rates.
Ratio of stabilised to unstable approaches is approximately 27:1.
LOSA records a “90% (SOP) violation factor” in terms of not executing a go-around from an unstable approach.
Conclusion drawn: the training failure is the go-around itself — practised only from published minima, one engine inoperative, at low weight, expected. “The flight crew clearly should be trained to confidently and effectively perform a go-around during the approach in almost any situation and condition.”

Non-technical competencies as measurable countermeasures LOSA ratings of Captain Leadership and Communication Environment against TEM outcomes, per flight: [VERIFIED]

Indicator	Outstanding leadership + outstanding communication	Poor leadership + poor communication
Threats	4.9	5.0
Mismanaged threats	0.3	1.1
Errors	2.3	7.0
Undesired aircraft states	0.4	1.8

The threat load is the same; the error rate triples. This is the single cleanest quantitative argument in the whole EBT corpus that non-technical competencies are causal countermeasures rather than soft correlates.

Automation, monitoring and compliance

28% of flights in the LOSA Archive have an automation error; two-thirds are usually undetected/unmanaged.
28% of flights have an SOP cross-verification error; one in ten of those goes unmanaged.
Only 26% of all errors logged by observers are detected and acted upon.
Deficiency in application of procedures according to published operating instructions was a factor in 49% of accidents with high training effect over the preceding 15 years.
Situation Awareness problems were present in over 41% of accidents rated highly preventable by training, with an increasing trend — and, counter-intuitively, Generations 3 and 4 (better displays) have a higher percentage of accidents where SA is a factor than earlier aircraft.
Self-reported non-compliance: 18% of surveyed pilots admit deviating from checklists frequently; 21% admit deviations from standard callouts on virtually every flight; 82% justified continuing an unstable approach on the judgement that the landing could be performed safely.

Pilots’ own verdict on the training they were receiving

54% of survey respondents had encountered an operational situation in the previous 6 months for which they felt insufficiently trained.
43.6% reported that the instructor in their last training session did not raise the level of their confidence.

Generational differences — the finding that justifies generation-specific syllabi

“Engine failure ranks as the fourth priority for Generation 2 Jets, and seventeenth for Generation 3 and 4 Jets.”
Ground and Landing are the two most significant accident phases for Generation 4 jets; Take-Off is particularly critical for Generation 3; Approach is most significant for Generation 2.
The proportion of accidents judged highly mitigable by training increases with generation — newer, more reliable aircraft leave training as a proportionally larger share of the remaining risk.

4.5 The ICAO pilot core competencies

The original set (Doc 9995 / EBT Implementation Guide, 2013) comprised eight competencies:

Application of Procedures
Communication
Aeroplane Flight Path Management, automation
Aeroplane Flight Path Management, manual control
Leadership and Teamwork
Problem Solving and Decision-Making
Situation Awareness
Workload Management

[VERIFIED — the count of eight is confirmed by the EBT Implementation Guide and by EASA's statement that its own set comprises "the 8 competencies of Doc 9995 plus 'Application of Knowledge'". The exact 2013 wording of items 1 and 7 is inferred from the revised set's numbering and EASA's abbreviation table (PRO, SAW), not read off Doc 9995 Appendix 1 directly.] [UNVERIFIED — verbatim 2013 wording of "Application of Procedures" and "Situation Awareness"]

The revised set (ICAO 2020, PANS-TRG Amendment 7; as published by IATA) comprises nine competencies. The three changes are: “Application of Knowledge” was added; “Application of Procedures” became “Application of Procedures and Compliance with Regulations”; “Situation Awareness” became “Situation Awareness and Management of Information.” Note carefully: this is an addition and two renamings/expansions — not a split. The internal numbering is decisive evidence: the new competency’s observable behaviours are numbered OB 0.1–0.7, prepended to the existing 1–8 series. [VERIFIED]

Source for the full list below: IATA, Competency Assessment and Evaluation for Pilots, and Instructors/Evaluators — Guidance Material, Fourth Edition, 2025, Appendix A. [VERIFIED] URL: https://www.iata.org/contentassets/c0f61fc821dc4f62bb6441d7abedb076/competency-assessment-and-evaluation-for-pilots-instructors-and-evaluators-gm.pdf

0. Application of Knowledge (KNO) — Demonstrates knowledge and understanding of relevant information, operating instructions, aircraft systems and the operating environment. OB 0.1 Demonstrates practical and applicable knowledge of limitations and systems and their interaction · OB 0.2 Demonstrates required knowledge of published operating instructions · OB 0.3 Demonstrates knowledge of the physical environment, the air traffic environment including routings, weather, airports and the operational infrastructure · OB 0.4 Demonstrates appropriate knowledge of applicable legislation · OB 0.5 Knows where to source required information · OB 0.6 Demonstrates a positive interest in acquiring knowledge · OB 0.7 Is able to apply knowledge effectively

1. Application of Procedures and Compliance with Regulations (PRO) — Identifies and applies appropriate procedures in accordance with published operating instructions and applicable regulations. OB 1.1 Identifies where to find procedures and regulations · OB 1.2 Applies relevant operating instructions, procedures and techniques in a timely manner · OB 1.3 Follows SOPs unless a higher degree of safety dictates an appropriate deviation · OB 1.4 Operates aeroplane systems and associated equipment correctly · OB 1.5 Monitors aircraft systems status · OB 1.6 Complies with applicable regulations · OB 1.7 Applies relevant procedural knowledge

2. Communication (COM) — Communicates through appropriate means in the operational environment, in both normal and non-normal situations. OB 2.1 Determines that the recipient is ready and able to receive information · OB 2.2 Selects appropriately what, when, how and with whom to communicate · OB 2.3 Conveys messages clearly, accurately and concisely · OB 2.4 Confirms that the recipient demonstrates understanding of important information · OB 2.5 Listens actively and demonstrates understanding when receiving information · OB 2.6 Asks relevant and effective questions · OB 2.7 Uses appropriate escalation in communication to resolve identified deviations · OB 2.8 Uses and interprets non-verbal communication in a manner appropriate to the organizational and social culture · OB 2.9 Adheres to standard radiotelephone phraseology and procedures · OB 2.10 Accurately reads, interprets, constructs and responds to datalink messages in English

3. Aeroplane Flight Path Management, automation (FPA) — Controls the flight path through automation. OB 3.1 Uses appropriate flight management, guidance systems and automation, as installed and applicable to the conditions · OB 3.2 Monitors and detects deviations from the intended flight path and takes appropriate action · OB 3.3 Manages the flight path safely to achieve optimum operational performance · OB 3.4 Maintains the intended flight path during flight using automation while managing other tasks and distractions · OB 3.5 Selects appropriate level and mode of automation in a timely manner considering phase of flight and workload · OB 3.6 Effectively monitors automation, including engagement and automatic mode transitions

4. Aeroplane Flight Path Management, manual control (FPM) — Controls the flight path through manual control. OB 4.1 Controls the aircraft manually with accuracy and smoothness as appropriate to the situation · OB 4.2 Monitors and detects deviations from the intended flight path and takes appropriate action · OB 4.3 Manually controls the aeroplane using the relationship between aeroplane attitude, speed and thrust, and navigation signals or visual information · OB 4.4 Manages the flight path safely to achieve optimum operational performance · OB 4.5 Maintains the intended flight path during manual flight while managing other tasks and distractions · OB 4.6 Uses appropriate flight management and guidance systems, as installed and applicable to the conditions · OB 4.7 Effectively monitors flight guidance systems including engagement and automatic mode transitions

5. Leadership and Teamwork (LTW) — Influences others to contribute to a shared purpose. Collaborates to accomplish the goals of the team. OB 5.1 Encourages team participation and open communication · OB 5.2 Demonstrates initiative and provides direction when required · OB 5.3 Engages others in planning · OB 5.4 Considers inputs from others · OB 5.5 Gives and receives feedback constructively · OB 5.6 Addresses and resolves conflicts and disagreements in a constructive manner · OB 5.7 Exercises decisive leadership when required · OB 5.8 Accepts responsibility for decisions and actions · OB 5.9 Carries out instructions when directed · OB 5.10 Applies effective intervention strategies to resolve identified deviations · OB 5.11 Manages cultural and language challenges, as applicable

6. Problem Solving and Decision-Making (PSD) — Identifies precursors, mitigates problems, and makes decisions. OB 6.1 Identifies, assesses and manages threats and errors in a timely manner · OB 6.2 Seeks accurate and adequate information from appropriate sources · OB 6.3 Identifies and verifies what and why things have gone wrong, if appropriate · OB 6.4 Perseveres in working through problems while prioritizing safety · OB 6.5 Identifies and considers appropriate options · OB 6.6 Applies appropriate and timely decision-making techniques · OB 6.7 Monitors, reviews and adapts decisions as required · OB 6.8 Adapts when faced with situations where no guidance or procedure exists · OB 6.9 Demonstrates resilience when encountering an unexpected event

7. Situation Awareness and Management of Information (SAW) — Perceives, comprehends and manages information and anticipates its effect on the operation. OB 7.1 Monitors and assesses the state of the aeroplane and its systems · OB 7.2 Monitors and assesses the aeroplane’s energy state, and its anticipated flight path · OB 7.3 Monitors and assesses the general environment as it may affect the operation · OB 7.4 Validates the accuracy of information and checks for gross errors · OB 7.5 Maintains awareness of the people involved in or affected by the operation and their capacity to perform as expected · OB 7.6 Develops effective contingency plans based upon potential risks associated with threats and errors · OB 7.7 Responds to indications of reduced situation awareness

8. Workload Management (WLM) — Maintains available workload capacity by prioritizing and distributing tasks using appropriate resources. OB 8.1 Exercises self-control in all situations · OB 8.2 Plans, prioritizes and schedules appropriate tasks effectively · OB 8.3 Manages time efficiently when carrying out tasks · OB 8.4 Offers and gives assistance · OB 8.5 Delegates tasks · OB 8.6 Seeks and accepts assistance, when appropriate · OB 8.7 Monitors, reviews and cross-checks actions conscientiously · OB 8.8 Verifies that tasks are completed to the expected outcome · OB 8.9 Manages and recovers from interruptions, distractions, variations and failures effectively while performing tasks

Definitional apparatus. PANS-TRG defines: Competency — “A combination of skills, knowledge and attitudes required to perform a task to the prescribed standard.” Competency-based training and assessment — “Training and assessment that are characterized by a performance orientation, emphasis on standards of performance and their measurement, and the development of training to the specified performance standards.” Competency element — “An action that constitutes a task that has a triggering event and a terminating event that clearly defines its limits, and an observable outcome.” Competency unit — “A discrete function consisting of a number of competency elements.” [VERIFIED — PANS-TRG 2nd ed., I-1-1]

Note the terminological shift: the 2015 edition uses competency unit / competency element / performance criteria (the older MPL vocabulary); the current framework uses competency / observable behaviour / performance criteria. [VERIFIED as to 2015 terms; the 3rd edition's own definition list was not retrieved] [UNVERIFIED — 3rd edition verbatim definitions]

Adaptation rules. IATA holds that the competency name, description and OBs are optional to adapt, but the final competency standard and conditions of demonstration must be adapted to purpose — e.g. for Workload Management, instructors should reach “effective” (grade 4) where line pilots need “adequate” (grade 3). [VERIFIED]

4.6 EASA’s EBT rules

Regulatory chain:

ED Decision 2015/027/R — introduced EBT into the European framework as “mixed EBT”: operators could voluntarily assess the nine competencies on a 1–5 scale alongside retained traditional LPC/OPC requirements. [VERIFIED — EASA]
Commission Implementing Regulation (EU) 2020/2036 of 9 December 2020, amending Regulation (EU) No 965/2012 “as regards the requirements for flight crew competence and training methods and postponing dates of application of certain measures in the context of the COVID-19 pandemic.” OJ L 416/24, 11 December 2020. Entry into force on the twentieth day following publication. [VERIFIED] URL: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32020R2036
ED Decision 2021/002/R — “Update of the AMC & GM to Subpart FC of Part-ORO (evidence-based training (EBT))”, issuing AMC & GM to Part-ORO Issue 2 Amendment 17, Part-ARO Issue 3 Amendment 11, Part-FCL Issue 1 Amendment 11, Part-ARA Issue 1 Amendment 11, and Annex I (Definitions) Issue 1 Amendment 12. [VERIFIED] The RMT is RMT.0599 “Evidence-based and competency-based training.” [VERIFIED] The precise date of ED Decision 2021/002/R was not confirmed. [UNVERIFIED — exact date]

Rule architecture:

ORO.FC.231 — Evidence-based training. Requires an approved EBT programme comprising at least six EBT modules distributed across a 3-year programme (two per year), each containing an evaluation phase and a training phase. Each pilot must complete “a minimum of two EBT modules within the validity period of the type rating, separated by a period of not less than 3 months,” plus line evaluations and ground training. [VERIFIED]
ORO.FC.146 — Personnel providing training, checking and assessment. EBT personnel must hold a Part-FCL instructor or examiner certificate and complete the operator’s EBT instructor standardisation programme; line evaluations are conducted by a suitably qualified commander nominated by the operator and standardised in EBT concepts. [VERIFIED]
ORO.FC.232 — referenced in ARO.OPS.226 as part of the compliance set; AMC2–7 ORO.FC.232 carry generation-specific training topics. [VERIFIED — existence and function] [UNVERIFIED — full text and title]
ARO.OPS.226 — Approval and oversight of evidence-based training programmes. Competent authorities must assess operator capability, verify compliance with at least ORO.FC.146, ORO.FC.231 and ORO.FC.232, resolve level 1 findings before full approval, verify record-keeping for flight crew qualifications, and maintain an oversight plan covering management supervision, operational risk identification, instructor standardisation and delivery inspections. The Agency must be notified when alternative means of compliance are evaluated. [VERIFIED]

Definitions inserted: evidence-based training; EBT module — “a combination of sessions in a qualified flight simulation training device as part of the 3-year period of recurrent assessment and training”; EBT programme. [VERIFIED]

Baseline vs Mixed EBT — the operative distinction:

	Mixed EBT (transitional)	Baseline EBT (full)
LPC/OPC	Retained as a safety net	Replaced — “replaces the licence proficiency check (LPC) and the operator proficiency check (OPC) with the competency assessment of the EBT system”
Sequence	Evaluation first, then scenario-based training	Same, fully implemented
Instructor training	Mandatory but no formal syllabus required	Mandatory, with recorded practical assessment
Instructor concordance	Not mandatory	Mandatory (ICAP)
Training feedback loop	Not required	Required
Malfunction clustering (equivalency of malfunctions)	No	Yes
Generation-specific topics and frequencies	Partial	Full compliance

[VERIFIED — EASA EBT presentation notes; the primary rule text distinguishing the two was not read verbatim] [SECONDARY as to the tabulated details]

Under baseline requirements, one complete EBT module replaces one operator proficiency check, and the line evaluation of competence substitutes the line check. [VERIFIED — EASA RMT.0599 safety material]

“Assessment and training” rather than pass/fail manoeuvres. ICAO states the logic: “EBT seeks to redress the balance between training and checking, recognizing that an assessment of competence is necessary, but once completed, pilots learn most effectively when not under pure test conditions.” And: EBT “refocuses instructors on the analysis of root causes of errors in order to correct inappropriate actions, rather than simply asking a pilot to repeat a manoeuvre without really understanding why it was not successfully flown at first.” [VERIFIED — PANS-TRG II-1-2-2, §3.3.4–3.3.5]

Grading is on a 1–5 scale per competency, with EASA’s baseline regulation mandating four levels of grading metric: [VERIFIED — IATA CBTA White Paper]

Level 0 (competent metrics) — whether the pilot(s) is/are competent or not
Level 1 (competency metrics) — numeric grade 1–5 per competency
Level 2 (observable behaviour metrics) — instructors record predetermined/required OBs
Level 3 (TEM metrics) — instructor records Threats, Errors or Reduction of Safety Margin

IATA’s remediation triggers: any competency graded 1; two successive grades of 2 in the same competency; or any grade 2 where the trainer judges the trainee will not reach adequate performance (grade 3) by the next session. [VERIFIED]

4.7 The three phases of an EBT module

Each module has three sequential phases: [VERIFIED — Doc 9995, EBT Implementation Guide, IATA 2024]

Evaluation phase. A line-oriented flight scenario (or scenarios) representative of the operator’s environment, in which the instructor observes without interrupting to assess crew competency across all competencies and identify individual training needs. It is not a comprehensive assessment of every competency in every module.
Manoeuvres training phase. Deliberate practice to defined proficiency of manoeuvres demanding significant crew capability — handling skills and associated procedures. The instructor is an active trainer.
Scenario-based training phase. The largest phase. Line-oriented scenarios developing competencies and addressing the individual training needs identified in phase 1, including critical system malfunctions and environmental threats. The instructor intervenes to enhance learning.

Why the evaluation phase is explicitly not a jeopardy check. The design intent is diagnostic: EASA states the evaluation “is done first and training is done after,” and its purpose is to identify “the training needs of the pilot/crew so that on day two a specific training adapted to that particular pilot/crew is provided.” [VERIFIED] The EBT Implementation Guide frames the shift: “EBT suggests a shift in balance between ‘checking’ and ‘training,’ recognizing that competence needs to be affirmed in a traditional sense according to present rules, but also recognizing the tremendous value and learning opportunities offered by modern FSTDs.” [VERIFIED]

The honest qualification: under baseline EBT the module does discharge the licence-revalidation and proficiency-check function, so jeopardy has not been abolished — it has been relocated from a discrete manoeuvre-by-manoeuvre check to a programme-level competency judgement, and decoupled from the diagnostic observation that opens the module. That decoupling is the actual innovation, and it is what makes a pilot willing to fly the evaluation scenario honestly rather than defensively.

4.8 The instructor: from checker to facilitator

“Facilitation is a key element in the instructor role in EBT.” [VERIFIED — EASA] The instructor’s job moves from scoring manoeuvre execution to (a) observing behaviour against a competency framework, (b) grading reliably, and (c) running a facilitated debrief in which the crew reconstructs its own reasoning.

The IATA instructor/evaluator competency framework formalises this with its own observable behaviours (IOB series) across Management of the learning environment, Instruction, and Interaction with the trainees. Diagnostic items include: IOB 3.4 “Applies instructional methods as appropriate (e.g., explanation, demonstration, facilitation, discover with assistance, discover without assistance)”; IOB 3.6 “Adapts the amount of instructor inputs to ensure that the training objectives are met”; IOB 3.9 “Encourages the trainee to self-assess”; IOB 3.10 “Allows trainee to self-correct in a timely manner”; IOB 3.11 “Applies trainee-centered feedback techniques (e.g., facilitation)”; IOB 2.3 “Intervenes appropriately, at the correct time and level.” [VERIFIED]

Instructor standardisation and calibration. Requirements under EASA baseline EBT: initial ground course plus practical competency assessment in the simulator; annual recurrent ground training; practical competency assessment every three years; and the Instructor Concordance Assurance Programme (ICAP) — uniform standardisation material assessed at least every 72 months, analysis of instructor grading data every 12 months, statistical methods including Cohen’s kappa, Fleiss’ kappa and intra-class correlation, and investigation of instructors showing “large deviation from standards, consistent positive/negative bias or poor improvement.” Minimum concordance standards are normally set by the operator, though the competent authority may impose criteria, especially where licence revalidation is performed under EBT. [VERIFIED — EASA EBT presentation notes] [SECONDARY]

The EBT Implementation Guide’s standardisation prescription is explicit about rater bias: groups must address “central tendency error, halo-effect error, and leniency error,” using video of real crews without coaching, immediate rating feedback, and continued sessions until group benchmarks are met; “under EBT assessment there should be on-going training and standardization to establish confidence in the crew performance data.” [VERIFIED]

4.9 Inter-rater reliability — the unresolved problem

This is the most serious open weakness in the entire competency-assessment edifice, and the literature says so.

Weber, D. E., Roth, W.-M., Mavin, T. J., & Dekker, S. W. A. (2013). “Should we pursue inter-rater reliability or diversity? An empirical study of pilot performance assessment.” Aviation in Focus, 4(2), 33–57. Six airline captains (mean 24.7 years’ experience) in three pairs assessed the same two videotaped ATR 72 simulator scenarios. In both scenarios one pair passed the captain while two pairs failed him. Scores on specific dimensions differed by up to 2.5 points on a 5-point scale. Only ~15% of assessment topics in Scenario 1 (6 of 41) and ~33% in Scenario 2 (7 of 21) overlapped across all three pairs. The authors’ central methodological warning: “high IRR-scores do not imply that assessors made the same observations” — assessors used identical criteria to reach different scores, and different criteria to reach identical scores. [VERIFIED — via full-text summary; no reliability coefficients were computed, findings are qualitative]
Roth, W.-M., Mavin, T., & Dekker, S. (2015). “Peer Assessment of Aviation Performance: Inconsistent for Good Reasons.” Cognitive Science. DOI: 10.1111/cogs.12152. [SECONDARY — citation confirmed via publisher listing; volume/pages and full findings not retrieved]
Weber, D. E. (2016). “Judging Airline Pilots’ Performance With and Without an Assessment Model: A Comparison Study of the Scoring of Raters From Two Different Airlines.” Journal of Aviation/Aerospace Education & Research, 25(2), 39–70. DOI: 10.15394/jaaer.2016.1645. Found systematic scoring differences between raters from two airlines. [VERIFIED — citation and abstract; no reliability coefficients reported in retrievable material]
Sun, H., Zhou, X., Zhang, P., Liu, X., Lu, Y., Huang, H., & Song, W. (2023). “Competency-based assessment of pilots’ manual flight performance during instrument flight training.” Cognition, Technology & Work, 25(4), 345–356. DOI: 10.1007/s10111-023-00737-3. 50 trainees (~87 flight hours), three experts (>10,000 hours). Derived observable behaviours from flight data using discrete Fréchet distance; achieved Spearman ρ = 0.947 (p < 0.01) against expert judgement. Notable as evidence that competency grades can be made reliable — by grounding them in objective flight data rather than unaided human observation. [VERIFIED]

Industry practice. Lufthansa presented its ICAP programme (developed with conavitra) at EATS, 6 November 2024, describing annual concordance analyses — but published no concordance figures. [VERIFIED — that no figures were disclosed]

The honest position for the report: aviation has mandated inter-rater reliability machinery (ICAP, kappa statistics, 12-month grading analysis) precisely because the underlying reliability problem is real and unsolved. The published empirical literature on unaided human competency grading in aviation shows poor agreement; the one study reporting high agreement achieved it by substituting an algorithm derived from flight data for the human rater. Any transfer argument that assumes competency frameworks produce reliable judgements as a matter of course is unsupported.

4.10 What is the actual published evidence that EBT is better?

Very little that is independent and peer-reviewed. This must be stated plainly.

EASA’s own safety material for RMT.0599 (2Q 2025) presents no quantified accident statistics, no safety-benefit estimate, no comparison of OPC outcomes to EBT module outcomes, and no baseline safety metrics. It is explicit that it “does not form part of the EASA regulatory system.” [VERIFIED] URL: https://www.easa.europa.eu/en/downloads/131390/en
The IATA CBTA White Paper (March 2024) argues the safety case deductively — competencies are countermeasures to threats and errors, therefore demonstrating more OBs should improve TEM and preserve safety margins — rather than reporting outcome data. Its own framing is conditional throughout (“should enhance,” “should lead to”). [VERIFIED]
The strongest quantitative claims in circulation come from vendor-conducted client reviews, not research. Salient’s “EBT Health Check” (Michael Varney, presented at EATS, November 2025) across 90 airlines, 8,000 pilots, 1,000 instructors reported: >90% of pilots say EBT improves confidence and competence; 88% of instructors believe it helps pilots improve; remedial training reductions up to 90%; one airline reduced check failures from 3.4% to 0.4%; another saw a 65–70% reduction in runway safety incidents within three years. No published research report, no control group, no peer review. [SECONDARY — and explicitly commercial]
Hwang, E. S., & Kim, J. K. “Success factors of evidence-based training for pilots,” Management Learning, DOI 10.1177/02666669231159100 — identified but not retrieved. [UNVERIFIED — content]

Note the reflexive irony, which is worth stating in the report: Evidence-Based Training is evidence-based in its curriculum derivation and largely faith-based in its outcome validation. That is not a scandal — it is the same position most safety interventions occupy, because the outcome (accidents) is too rare to power a trial. But it is precisely the confusion the L&D world imports when it borrows the phrase.

5. UPRT — Upset Prevention and Recovery Training

5.1 ICAO Doc 10011

ICAO Doc 10011, AN/506, Manual on Aeroplane Upset Prevention and Recovery Training, First Edition, 2014. [VERIFIED] URL: https://www.caa.gov.cz/wp-content/uploads/2019/07/ICAO-Doc-10011.pdf

An upset is “an aeroplane in flight unintentionally exceeding the parameters normally experienced in line operations or training,” with parameters: [VERIFIED]

pitch attitude greater than 25 degrees nose up; or
pitch attitude greater than 10 degrees nose down; or
bank angle greater than 45 degrees; or
within the above parameters, but flying at airspeeds inappropriate for the conditions.

Provenance: following a June 2009 RAeS conference on aeroplane upsets and loss of control, the Royal Aeronautical Society initiated a study undertaken by ICATEE — the International Committee on Aviation Training in Extended Envelopes. The LOCART (Loss of Control Avoidance and Recovery Training) initiative brought ICATEE and other groups into coordinated discussion under ICAO sponsorship in 2012. [VERIFIED]

Simulator fidelity at the edges — the core technical problem. Doc 10011 is unusually candid: FSTDs cannot replicate sustained g, motion cueing or emotional response; use of an FSTD “beyond the FSTD’s ability to provide accurate fidelity has the potential to introduce misleading concepts or inappropriate understanding of techniques which can result in a ‘negative’ training experience.” Aerodynamic stall training in FSTDs “should only be conducted as a carefully managed demonstration using only devices that have the highest levels of fidelity.” [VERIFIED]

Aeroplane vs simulator. Doc 10011 holds that current FSTDs “have limitations that render them incapable of providing the complete exposure to conditions synonymous with preventing or recovering from a LOC-I event,” and that on-aeroplane training should form part of initial UPRT at CPL(A) and MPL level, supplemented later by FSTD training — creating “a frame of reference that can be transferred to the FSTD environment.” The boundary is firm: “On-aeroplane UPRT is not intended to be delivered while operating transport category aeroplanes or aeroplanes requiring two or more crew members; for those operations, UPRT should not be permitted to be conducted outside the confines of a suitable FSTD.” [VERIFIED]

Instructor requirements: UPRT-qualified aeroplane flight instructors, “trained to proficiency and remain current to ensure competence in aeroplane manoeuvring as well as being able to consistently employ effective intervention skills”; FSTD instructors must be qualified to use UPRT-specific debriefing tools at the instructor operating station. [VERIFIED]

5.2 The FAA route: statute → rule → simulator standard

Statutory driver. The Airline Safety and FAA Extension Act of 2010, Public Law 111-216, §208 directed the FAA to ensure flightcrew members receive training in “recognizing and avoiding stalls, recovering from stalls, and recognizing and avoiding upset of an aircraft, as well as the proper techniques to recover from upset.” [VERIFIED] The proximate cause was the Colgan Air Flight 3407 accident, 12 February 2009, in which the NTSB found the captain responded to stick-shaker activation by pulling aft rather than pushing forward. [VERIFIED]

The training rule. Qualification, Service, and Use of Crewmembers and Aircraft Dispatchers, final rule, 78 FR 67800, 12 November 2013, Docket FAA-2008-0677, Amendment 121-366, effective 12 March 2014, compliance five years thereafter (12 March 2019). [VERIFIED] URL: https://www.federalregister.gov/documents/2013/11/12/2013-26845/qualification-service-and-use-of-crewmembers-and-aircraft-dispatchers

§121.423 Extended Envelope Training requires flight training in a full flight simulator on: [VERIFIED]

Upset recovery manoeuvres
Manually controlled slow flight
Manually controlled loss of reliable airspeed
Manually controlled instrument arrivals and departures
Recovery from stall and stick pusher activation, if aircraft equipped
Recovery from bounced landing

The decisive conceptual change is from “approach to stall” to “stall recovery.” The legacy standard rewarded recovery with minimum altitude loss at the first indication of the stall — which trained pilots to power out while holding attitude. The new standard requires recovery from an actual stall, where reducing angle of attack (and accepting altitude loss) is the correct action. Related changes: §121.424 (extended envelope ground training), §121.427 (adding 2 hours to qualification and 30 minutes to recurrent ground training), §121.409, and §121.414 (instructors and check airmen must be trained on the simulator’s motion and data limitations). [VERIFIED]

The simulator rule. Flight Simulation Training Device Qualification Standards for Extended Envelope and Adverse Weather Event Training Tasks, final rule, 81 FR 18178, 30 March 2016, Docket FAA-2014-0391, Amendment 60-4, effective 31 May 2016, with previously qualified FSTDs required to comply with FSTD Directive No. 2 by 12 March 2019. [VERIFIED] URL: https://www.federalregister.gov/documents/2016/03/30/2016-05860/flight-simulation-training-device-qualification-standards-for-extended-envelope-and-adverse-weather

Requirements: aerodynamic modelling of stalls to 10 degrees angle of attack beyond the stall identification point; a defined four-part stall identification criterion (no further pitch increase with aft stop held 2 seconds; unarrestable uncommanded nose-down pitch; deterrent buffeting; stick pusher); subject-matter-expert pilot subjective evaluation of stall model fidelity; UPRT validation against the FSTD’s defined validation envelope; ice accretion models; tuned gusting crosswind profiles; bounced landing modelling; and IOS feedback to instructors showing current AOA, stall identification AOA, operational limits and student control inputs. A new Level 7 FTD was introduced. [VERIFIED]

The fidelity admission is important and quotable: for stall training, “objectively validated ‘type specific’ behavior at an AOA beyond the aerodynamic stall may not be a reasonable goal.” The rule therefore permits “type representative” modelling built from engineering analysis and SME evaluation where flight-test data is absent, with Statements of Compliance documenting data sources and the extent of objective versus analytical validation. For envelope-protected aircraft, modelling in non-normal control laws (protections disabled) is required, since protections mask the very cues the training needs. [VERIFIED]

This is the regulator explicitly conceding the limits of the simulation and building a documentation regime around the concession rather than pretending fidelity exists. It is a strong, transferable example of how to handle a training medium that cannot fully represent the target task.

5.3 EASA’s UPRT

Commission Implementing Regulation (EU) 2018/1974 of 14 December 2018, amending Regulation (EU) No 1178/2011 (Aircrew), published 20 December 2018. [VERIFIED] ED Decision 2019/005/R, 27 February 2019, RMT.0581, issuing the AMC/GM. [VERIFIED] URL: https://www.easa.europa.eu/sites/default/files/dfu/Explanatory%20Note%20to%20%20ED%20Decicion%202019-005-R%20.pdf

Date of application conflict — flagged: EASA’s explanatory note states the UPRT requirements apply from 20 December 2019. EUR-Lex metadata for the same instrument shows a date of application of 31 January 2022. The likely explanation is that Article 2 sets a general application date with later dates for specified points, but I could not retrieve Article 2 verbatim (the legislation.gov.uk fetch was blocked). [UNVERIFIED — reconciliation of 20 December 2019 vs 31 January 2022]

Advanced UPRT — a separate on-aeroplane course at an approved ATO — is required for integrated ATP(A) and MPL courses, and prior to first multi-pilot aeroplane type rating and single-pilot high-performance complex aeroplane type ratings; basic UPRT elements sit within CPL(A) and ATPL(A) training. Minimum duration is set in FCL.745.A. The course covers aerodynamic principles, exposure to g-loads typically “from -1g up to 2,5/3g,” recovery from nose-high and nose-low upsets and stall events, and building resilience against the psychological and physiological effects of an upset. Instructors for advanced UPRT must complete a UPRT instructor course at an ATO under FCL.915(e), with an aerobatic rating or equivalent experience recommended; type-specific UPRT is addressed through amended TRI training under AMC1 FCL.930.TRI. [VERIFIED]

Air operations side: I could not confirm which instrument inserted UPRT obligations into Part-ORO (ORO.FC.220 operator conversion, ORO.FC.230 recurrent). Commission Implementing Regulation (EU) 2019/1387 is NOT the UPRT instrument — its subject is aeroplane landing performance calculations, runway surface condition assessment, aircraft safety equipment and operations without an extended range operational approval. [VERIFIED — negative finding] [UNVERIFIED — the correct Part-ORO UPRT amending regulation number]

6. Manual Flying Policy

The regulatory response to automation-induced skill degradation runs through advisory rather than binding instruments — which is itself a finding about how far a regulator can push against operating economics.

FAA SAFO 13002, “Manual Flight Operations,” 4 January 2013, Flight Standards Service. [VERIFIED] Issued after the FAA identified “an increase in manual handling errors” in flight operations data. Its core reasoning: autoflight systems improve safety and workload management, but “continuous use of those systems does not reinforce a pilot’s knowledge and skills in manual flight operations,” risking loss of “the ability to quickly recover the aircraft from an undesired state.” Recommends operators develop or review policies so pilots practise manual flying in non-RVSM airspace and during low workload conditions, clarify when to use automation versus manual control, and provide practice in initial and recurrent training. [VERIFIED] URL: https://www.faa.gov/sites/faa.gov/files/other_visit/aviation_industry/airline_operators/airline_safety/SAFO13002.pdf
FAA SAFO 17007, “Manual Flight Operations Proficiency,” 4 May 2017. [VERIFIED] Broader than 13002: defines manual flight as “managing the flight path through manual control of pitch, bank, yaw and/or thrust”; requires curricula to treat “manual flight is the foundation upon which other technical flying skills are built”; directs carriers to “permit and encourage manual flight operations” and to avoid restrictive blanket policies, leaving the decision to pilot judgement on weather, traffic, crew experience and threats. URL: https://www.faa.gov/sites/faa.gov/files/2022-11/SAFO17007.pdf
FAA AC 120-123, “Flightpath Management,” 21 November 2022, initiated by AFS-200. [VERIFIED — read directly] The most developed statement. Chapter 3 (Manual Flight Operations) states manual flight “is the foundation upon which other technical flying skills are built” and that “manual flight knowledge and skills are required in all situations, not only when all automated systems are off.” Skills are decomposed into cognitive, psychomotor and communication skills — explicitly rejecting the framing of manual flying as motor skill alone. Operator line policy “should permit and encourage MFO” including “encouragement to manually fly the aircraft including, at least periodically, the entire departure and arrival phases and potentially the entire flight,” and should require crews to apply TEM principles weighing weather, terrain, time of day, physiological factors, crew experience, traffic density, aircraft condition and ATC/procedural challenges. It requires practice across combinations of autoflight modes (FD on/AP off/AT on; FD on/AP off/AT off; FD off/AP off/AT off; FD on/AP on/AT off; FD off/AP off/AT on). And it explicitly proscribes blanket rules: policy “should avoid… overly general statements, such as ‘shall never manually fly at night’ or ‘shall always manually fly in day visual meteorological conditions’.” Operator policy must also address “how to regain proficiency of MFO skills after pilots have short and long absences.”
UK CAA Paper 2004/10, “Flight Crew Reliance on Automation,” December 2004, Simon Wood, Cranfield University, published by CAA Safety Regulation Group. [VERIFIED] Method: literature review; analysis of the MOR database 2002–2003; interviews with airline training departments, TRTOs, CAA and CRM/HF specialists. Found “much evidence to support the concern that crews were becoming dependent on flight deck automation”; cites research in which “pilots committed errors on 55% of occasions when the automation presented incorrect information”; a “natural tendency to follow the choice of least cognitive effort.” On skill loss: “very little research published,” but “consistently reported… discernible reduction in manual flying skills,” particularly long-haul. Its sharpest structural observation is the asymmetry of regulation: manual flying skills are rigorously tested, but “there are no similar licensing or type rating requirements” ensuring “procedural knowledge of how to control the flight path using automation.” URL: https://www.caa.co.uk/publication/download/12922

The tension, stated honestly. Every one of these instruments is advisory. Meanwhile the operational and economic pressures run the other way: RVSM and PBN airspace mandates require autopilot engagement; fuel-burn optimisation favours automation; on-time performance discourages hand-flying in high-workload environments; and the LOSA-derived finding that go-arounds are rarely well flown is itself partly a product of how seldom they are practised in realistic conditions. AC 120-123’s specific prohibition of blanket restrictions is a direct regulatory response to operators having written such policies. The regulator can encourage manual practice; it has not mandated it in line operations.

7. CBTA Beyond Pilots — the framework generalising

This section matters disproportionately for transfer arguments, because it shows the framework was not pilot-specific.

ICAO Doc 9868 PANS-TRG is structured by personnel group. In the Second Edition (2015): [VERIFIED — read directly from contents]

Part I — General Procedures. Chapter 2 “General provisions for competency-based training and assessment” (the generic CBTA method); Chapter 3 “Qualifications of course developers and instructors,” with an Attachment on their competencies.
Part II — Training and Assessment for Aircraft Operational Personnel. Section 1 Flight crew: Ch.1 MPL, Ch.2 Evidence-based training, Ch.3 UPRT, Ch.4 Other flight crew members [Reserved], Ch.5 Flight validation pilots [Reserved]. Section 2 Cabin crew [Reserved]. Section 3 Flight dispatchers/flight operations officers [Reserved].
Part III — Training and Assessment for Aircraft Maintenance Personnel. Ch.1 CBTA for aircraft maintenance personnel, with competency units, elements and performance criteria, and guidance on programme design.
Part IV — Training and Assessment for Air Traffic Management (ATM) Personnel. Ch.1 general provisions; Ch.2 CBTA for air traffic controllers; Ch.3 CBTA for air traffic safety electronic personnel (ATSEP).

The “[Reserved]” markers are themselves evidence: ICAO laid out the full architecture across every operational discipline and filled it in progressively.

Supporting manuals: [SECONDARY — titles and numbers confirmed via ICAO Store / catalogue listings; contents not read]

Doc 10056, Manual on Air Traffic Controller Competency-based Training and Assessment, Volume I — Air Traffic Control (ATC)
Doc 10098, Manual on Competency-based Training and Assessment for Aircraft Maintenance Personnel
Doc 10002, AN/502, Cabin Crew Safety Training Manual, First Edition, 2014 (with Corrigenda 1 and 2 dated 6 August 2021)

Amendment 7 to PANS-TRG (2020) “formalized the global expansion and applicability of CBTA principles to all licensing training (ICAO Annex 1) and operator training (ICAO Annex 6).” [VERIFIED — IATA CBTA White Paper §3.2]

IATA frames this as the Total Systems Approach (TSA): “the application of CBTA across all aviation disciplines in general, and to all modules and roles in a pilot’s entire career. Hence, the defined competencies for pilots, instructors and evaluators should consistently be applied throughout pilot aptitude testing, initial (ab-initio) training, type rating training and testing, command upgrade, recurrent and evidence-based training and instructor and examiner selection and training.” [VERIFIED] Source: IATA, Competency-Based Training and Assessment (CBTA) Expansion within the Aviation System, White Paper, March 2024. URL: https://www.iata.org/contentassets/c0f61fc821dc4f62bb6441d7abedb076/cbta-expansion-within-the-aviation-system.pdf

Other national adoption noted by IATA: Australian CASR introduced competency-based standards for all CASA flight crew qualifications in 2014; the UAE GCAA accepted EBT as an alternative means of compliance; EASA launched RMT.0194 to bring CBTA principles into the Aircrew regulation. [VERIFIED as IATA's claims] [UNVERIFIED — RMT.0194 status and outcome]

Note also that EBT has a strictly bounded scope. IATA is explicit: “Today, EBT is a CBTA program applicable to operator recurrent training only.” [VERIFIED] And ICAO: “EBT is optional. When choosing to implement EBT in their regulatory framework, States shall ensure that operators and training organizations apply the principles of Doc 9995.” [VERIFIED — PANS-TRG II-1-2-1, §2.1.3] EBT is not a global mandate; it is a permitted alternative that States may or may not enable.

8. How Data Feeds Training — and the Protections Without Which It Does Not

This is the precondition. It should be treated in the report as load-bearing, because the entire evidential apparatus of EBT rests on data that only exists because it was made legally and culturally safe to report.

8.1 The loop

Operational and safety data (the external feedback loop) — FDM/FOQA, ASAP and other voluntary reports, mandatory occurrence reports, LOSA observations — plus training data (the internal feedback loop) — grading, first-look results, remediation rates — feed training needs analysis, which sets curriculum content and frequency, whose effects are then measured back through the same channels. The IATA Data Report names these two loops explicitly: “internal training ‘feedback’ loop” and “external training ‘feedback’ loop.” [VERIFIED]

IATA’s maturity ladder for hazard identification places CBTA/EBT training metrics at the leading edge: [VERIFIED]

Reactive	Reactive/Proactive	Proactive	Proactive/Predictive
Analysis of accidents/incidents	Analysis of events including Undesired Aircraft States	Threat and Error Management analysis	Analysis of CBTA-EBT training metrics

8.2 The instruments

LOSA. ICAO Doc 9803, Line Operations Safety Audit (LOSA). [SECONDARY — document number confirmed; first edition 2002 not verified in this session] [UNVERIFIED — edition year] LOSA collects observational data on normal line operations using the Threat and Error Management framework. Its methodological weakness is acknowledged in the EBT Data Report itself: “Pilot behavior can be influenced by the presence of an observer,” mitigated by strict observer selection, training and protocols. [VERIFIED]

FOQA/FDM. FAA AC 120-82, “Flight Operational Quality Assurance,” 12 April 2004. [VERIFIED] URL: https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120-82.pdf Protections rest on 49 U.S.C. §40123, 14 CFR Part 193 and 14 CFR §13.401. Crew-identifying data is removed during initial processing and is electronically inaccessible; a gatekeeper, typically a line captain designated by the pilot association, holds secure time-limited access solely to enable crew follow-up. A FOQA Monitoring Team including pilots and airline representatives analyses the data. Aggregate data submitted to the FAA is de-identified before leaving airline premises. The training payoff is stated directly: training departments gain “more effective training that is focused on documented problem areas.” [VERIFIED]

ASAP. FAA AC 120-66C, “Aviation Safety Action Program,” 31 March 2020, cancelling AC 120-66B (15 November 2002). [VERIFIED] URL: https://www.faa.gov/documentLibrary/media/Advisory_Circular/AC_120-66C_(Edit).pdf Structure: a Memorandum of Understanding among the FAA, the certificate holder and the employee labour group, establishing that “safety issues are resolved through corrective action rather than through punishment or discipline”; an Event Review Committee (ERC) of three voting members (management, labour, FAA inspector) operating by consensus. Reports are accepted unless they involve reckless conduct (“gross disregard for, or deliberate indifference to, safety”), intentional conduct, or the “Big Five” — criminal activity, substance abuse, controlled substances, alcohol, intentional falsification — or repeated noncompliance for identical violations. For accepted reports, “The FAA takes no action against an employee who submits a report that is accepted.” Corrective actions may include training, mentoring or counselling, and official personnel files and training records do not reflect ASAP-related corrective actions, consistent with the Pilot Records Improvement Act of 1996. [VERIFIED]

SMS and the ICAO protection standard. ICAO Annex 19 — Safety Management, Second Edition, July 2016; Amendment 1 adopted 2 March 2016, effective 11 July 2016, applicable 7 November 2019. [VERIFIED] URL: https://www.bazl.admin.ch/dam/it/sd-web/oOXBOqNfEsWb/icao_annex_19_second-edition.pdf

5.1.2: “States shall establish a mandatory safety reporting system that includes the reporting of incidents.”
5.1.3: “States shall establish a voluntary safety reporting system to collect safety data and safety information not captured by mandatory safety reporting systems.”
5.3.1: States shall accord protection to safety data captured by, and safety information derived from, voluntary safety reporting systems and related sources, in accordance with Appendix 3.
Appendix 3, 2.1: “States shall ensure that safety data or safety information is not used for: (a) disciplinary, civil, administrative and criminal proceedings against employees, operational personnel or organizations; (b) disclosure to the public; or (c) any purposes other than maintaining or improving safety; unless a principle of exception applies.”
Appendix 3, Section 3 — three exceptions: conduct constituting gross negligence, wilful misconduct or criminal activity; release “necessary for the proper administration of justice” where benefits outweigh adverse impact; release “necessary for maintaining or improving safety” where benefits outweigh adverse impact. [VERIFIED]

SMS is mandated for approved training organisations exposed to safety risks, international commercial air transport operators, approved maintenance organisations, type-design and manufacturing organisations, ATS providers and certified aerodrome operators. [VERIFIED]

European occurrence reporting and just culture. Regulation (EU) No 376/2014 of the European Parliament and of the Council of 3 April 2014 on the reporting, analysis and follow-up of occurrences in civil aviation. [VERIFIED]

Article 2(12) defines just culture as “a culture in which front-line operators or other persons are not punished for actions, omissions or decisions taken by them that are commensurate with their experience and training, but in which gross negligence, wilful violations and destructive acts are not tolerated.” [VERIFIED]
Article 15 requires appropriate confidentiality of occurrence details and that information be used exclusively for safety purposes, never “to attribute blame or liability.”
Article 16 requires that employees “shall not be subject to any prejudice by their employer” on the basis of reporting — except for wilful misconduct or “manifest, severe and serious disregard of an obvious risk”; that organisations refrain from instituting proceedings for unintentional infringements coming to attention only through a report; and that each organisation “adopt internal rules describing how ‘just culture’ principles… are guaranteed and implemented.” [VERIFIED] URL: https://eur-lex.europa.eu/eli/reg/2014/376/oj/eng

8.3 The unresolved frontier: protecting training data

IATA has flagged a genuine gap. Training data is safety data, but it is used at the individual level in a way operational safety data is not — the whole EBT design requires individual pilot data to pass from the evaluation phase to the scenario-based training phase within a single module, and across six modules over three years. Four parties need individual access: the pilot (to receive tailored training), the instructor (to deliver it), the operator (to adapt sessions and run ICAP), and the State (for licence revalidation and oversight of the EBT programme including ICAP). [VERIFIED]

IATA’s recommendation: “CBTA training data should always be protected and used in a de-identified format for global safety management, while some protocols should permit the use of individual pilot data in the interest of ‘routine’ CBTA program operations,” and the reconstituted ICAO Personnel Training and Licensing Panel should propose new standards in Annex 1 and Annex 6 for CBTA training data protection, clarifying their relationship to Annex 13 and Annex 19. [VERIFIED]

This is the sharpest single lesson for transfer. Aviation is currently discovering that a competency-based, data-driven training system creates a new class of personal performance data that existing safety-data protections do not cleanly cover — and that if it is not protected, the honest self-assessment on which the whole diagnostic model depends will stop. Any organisation importing competency-based assessment inherits this problem on day one.

8.4 Why this is the precondition, stated plainly

Every number in §4.4 exists because someone reported something without being punished for it, or because a flight data recorder was read under a rule that de-identified the crew. The IATA Data Report acknowledges its own dependence: results “were inferred from confidential, de-identified data, the specifics of which are made known only to the EBT project group and then only on a ‘need-to-know’ basis.” [VERIFIED]

Remove any of — de-identification, the gatekeeper protocol, the ERC consensus mechanism, enforcement forbearance, the Annex 19 Appendix 3 prohibition on disciplinary use, the Article 16 non-prejudice rule — and the data thins immediately, at which point “evidence-based” training reverts to expert opinion with a database attached. The pedagogy is portable; the pedagogy without the protections is not.

9. Reconstructing a Modern Airline Training Programme — the design in brief

A reader should be able to build the outline from the above:

Choose the regulatory instrument. AQP (14 CFR Part 121 Subpart Y) in the US; baseline EBT (ORO.FC.231, approved under ARO.OPS.226) in Europe; or the traditional prescribed programme. AQP and EBT both require the operator to justify equivalence and to surrender data.
Derive the syllabus from data. Job Task Analysis (AQP) or the generation-specific training topics from the EBT Data Report and AMC ORO.FC.232 (EBT), contextualised with the operator’s own FDM, ASAP/voluntary reports and LOSA findings.
Set frequencies from criticality. Priority A / B / C mapped to 6-month / 1-year / 3-year exposure across a rolling three-year programme of six modules.
Structure each module in three phases — evaluation (diagnostic, uninterrupted, LOFT-style), manoeuvres training (to proficiency), scenario-based training (largest, addressing the diagnosed needs).
Assess against the nine competencies using observable behaviours, graded 1–5, capturing all four metric levels (competent / competency / OB / TEM), with defined remediation triggers.
Qualify and calibrate the instructors — initial course plus practical competency assessment, annual recurrent ground training, three-yearly practical reassessment, and an Instructor Concordance Assurance Programme with annual statistical analysis of grading data.
Cover extended envelope explicitly — §121.423 tasks in the US, UPRT per Doc 10011 / EASA Part-FCL and Part-ORO elsewhere — in an FSTD qualified for the task (Part 60 Amendment 60-4 / FSTD Directive No. 2), with instructors trained on the device’s limits.
Write a manual flying policy that permits and encourages hand flying, applies TEM to the decision, covers combinations of autoflight modes, addresses proficiency recovery after absence, and contains no blanket prohibitions.
Close the loop. Feed grading data, remediation rates and line evaluation results back into curriculum revision annually; feed operational data forward into scenario design; and protect all of it.

10. Summary of Gaps and Unverified Items

Listed for the record; see also the digest.

§121.901 substantive text not retrieved (only titles and source credit).
Verbatim 2013 wording of Doc 9995’s “Application of Procedures” and “Situation Awareness” not read off Appendix 1 directly; the eight-competency count and the three 2020 changes are confirmed, the exact 2013 strings are inferred.
PANS-TRG 3rd Edition (2020) full text not retrieved — its own definitions and competency table were not read directly; the framework is sourced from IATA GM 4th ed. 2025.
Exact date of EASA ED Decision 2021/002/R not confirmed.
Full text and title of ORO.FC.232 not confirmed.
The Commission Regulation that inserted UPRT into Part-ORO (ORO.FC.220 / ORO.FC.230) not identified. (EU) 2019/1387 is confirmed NOT to be it.
Article 2 of (EU) 2018/1974 not read; the 20 December 2019 vs 31 January 2022 application-date discrepancy is unresolved.
ICAO Doc 9803 (LOSA) edition and year not verified.
MPL four-phase nomenclature not confirmed against PANS-TRG Appendix 2.
IFALPA MPL position paper document number (15POS03) inferred from a filename.
EASA ATQP rule reference: EU-OPS 1.978 confirmed as the historical basis (cited in the IATA Data Report); the successor reference ORO.FC.A.245 was not verified in this session.
Roth et al. (2015) Cognitive Science volume/pages not confirmed.
Hwang & Kim, “Success factors of evidence-based training for pilots,” Management Learning — content not retrieved.
Hamman & Holt (1997) on Line Operational Evaluation — content not retrieved.
EASA RMT.0194 (CBTA in the Aircrew regulation) status and outcome not confirmed.
No independent, peer-reviewed outcome evaluation of EBT versus the OPC was located. This is a finding, not merely a gap.