← Lucian Cosinschi

A9 — Cross-Domain Export: What Happened When Other Industries Imported Aviation's Answer

Stream: Intellectual honesty / boundary conditions Date: 2026-08-24 Tagging convention: [VERIFIED] = primary metadata and/or abstract or full text retrieved directly from publisher, Europe PMC, or the issuing official body during this research. [SECONDARY] = reported reliably by a source I did retrieve, but the primary document itself was not read. [UNVERIFIED] = asserted here from background knowledge and not confirmed in this research; treat as a research task, not as a citable claim.

0. Executive summary

Aviation’s post-automation training settlement — Crew Resource Management, simulator-based recurrent checks, standardised callouts, non-punitive incident reporting, and the disciplined checklist — has been the most widely exported safety package of the last forty years. Healthcare imported it most enthusiastically and most completely. Maritime imported it by regulation. Nuclear power arguably co-invented parts of it. Road transport is importing it right now, badly, under commercial pressure.

The honest finding of this stream is uncomfortable and should be stated at the top of the report rather than buried:

  1. The single most-cited import — the surgical safety checklist — has a genuinely divided evidence base. A dramatic before/after study (Haynes 2009), a positive stepped-wedge randomised trial (Haugen 2015), and a null population-level natural experiment (Urbach 2014) all exist and are all methodologically defensible. The divergence is not noise; it is informative, and it points at implementation quality rather than at the artefact.

  2. Aviation cannot demonstrate that its own flagship training intervention reduces accidents. Two systematic reviews by Salas and colleagues, in Human Factors, twenty-five and twenty years into the CRM era, concluded explicitly that the safety effect of CRM “cannot be ascertained.” This is the most important single fact in this stream. Any claim that another industry should adopt aviation’s training model because aviation has proved it works is not supported by aviation’s own evaluation literature.

  3. The best-documented empirical analogue of the copilot/deskilling problem is not in aviation at all — it is in clinical AI. Povyakalo et al. (2013) found computer-aided detection in mammography helped the less discriminating readers and harmed the most discriminating ones. Budzyń et al. (2025) found that continuous exposure to AI in colonoscopy degraded endoscopists’ unaided detection rate by 6 percentage points. These are the cleanest measured instances anywhere of automation improving average performance while degrading the humans it supports.

  4. The boundary conditions matter more than the techniques. Aviation has a set of structural preconditions — a single global regulator, mandatory employer-funded recurrent training, standardised certified equipment, legally protected reporting, two operators cross-checking in real time, one countable catastrophic outcome — which most importing domains lack. Amalberti et al. (2005) argued in Annals of Internal Medicine that several of these are not merely helpful but are barriers which healthcare must cross before ultrasafe performance is even available to it.

The rest of this memo builds the sourced case.

1. Healthcare — the biggest and best-documented import

1.1 The founding transfer: Anesthesia Crisis Resource Management

The transfer is unusually well-documented because its originators said plainly what they were copying. Howard, Gaba, Fish, Yang and Sarnquist published Anesthesia Crisis Resource Management in 1992 in — significantly — an aviation journal.

Howard, S. K., Gaba, D. M., Fish, K. J., Yang, G. & Sarnquist, F. H. (1992). “Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents.” Aviation, Space, and Environmental Medicine 63(9), 763–770. PMID 1524531. [VERIFIED]

The abstract states the lineage without hedging: the authors “developed a course in Anesthesia Crisis Resource Management (ACRM) analogous to courses in Crew (Cockpit) Resource Management (CRM) conducted in commercial and military aviation.” The design is the aviation design: didactic instruction in dynamic decision-making and human performance, a realistic simulator session, and a videotaped debrief.

What is striking, and what is usually omitted when this paper is cited as a founding success, is the authors’ own candour about evidence. Two demonstration courses were run. Participants rated the course “intense, helpful… and highly enjoyable.” Knowledge test scores “showed a significant improvement following the first course (residents) but not the second course (experienced anesthesiologists).” And the authors write: “Although the ultimate utility of this training for anesthesiologists cannot easily be determined…” [VERIFIED — abstract]

So the founding document of healthcare CRM is a feasibility and acceptability study with a null result in experienced clinicians and an explicit disclaimer about outcome evidence. That is not a criticism of Gaba — it is a model of appropriate scientific modesty. It is a criticism of thirty years of citation practice that has treated it as proof of effect.

1.2 TeamSTEPPS and the institutionalisation of the analogy

TeamSTEPPS (Team Strategies and Tools to Enhance Performance and Patient Safety) is the US federal instantiation, developed jointly by the Agency for Healthcare Research and Quality and the Department of Defense. Its aviation ancestry is stated by AHRQ itself: the agency’s published evidence base for the programme includes a dedicated “Crew Resource Management Training” section, whose citations include Morey et al. (2003), “A transition of aviation crew resource management to hospital emergency departments: the MedTeams story,” presented at the 12th International Symposium on Aviation Psychology. [VERIFIED — AHRQ evidence-base page retrieved: https://www.ahrq.gov/teamstepps-program/evidence-base/crew-resource.html]

The AHRQ evidence-base page lists supporting studies including France et al. (2005), Gross et al. (2019), Haller et al. (2008), Morey et al. (2003), Paull et al. (2013), Pratt et al. (2007), Shea-Lewis (2009) and Sundar et al. (2007). [VERIFIED — as a list of what AHRQ cites]. I did not retrieve these individual studies; their designs and effect sizes are [UNVERIFIED] here.

The honest characterisation of the TeamSTEPPS evaluation literature, on what I was able to verify: it is large, it is dominated by single-site pre/post designs and by Kirkpatrick level 1–2 outcomes (reactions and learning), and the systematic review literature repeatedly reports heterogeneity and an absence of consensus on methodology. A representative retrieved review:

Welsch, L. A., Hoch, J., Poston, R. D., Parodi, V. A. & Akpinar-Elci, M. (2018). “Interprofessional education involving didactic TeamSTEPPS® and interactive healthcare simulation: A systematic review.” Journal of Interprofessional Care 32(6), 657–665. DOI 10.1080/13561820.2018.1472069. [VERIFIED] Of 44 articles reviewed, 11 met inclusion criteria. The authors report that “across the literature there are a wide variety of such programs described, but there is not a consensus on the most effective methodology.”

Gap flagged: I did not locate a large systematic review or meta-analysis of TeamSTEPPS against hard patient outcomes (mortality, adverse event rates). Whether one exists is an open research task. Do not assert that TeamSTEPPS is proven or unproven on hard outcomes without closing this.

1.3 The broader team-training evidence: positive, but on soft outcomes

The strongest quantitative support for the healthcare team-training family is a meta-analysis by Salas’s group:

Hughes, A. M., Gregory, M. E., Joseph, D. L., Sonesh, S. C., Marlow, S. L., Lacerenza, C. N., Benishek, L. E., King, H. B. & Salas, E. (2016). “Saving lives: A meta-analysis of team training in healthcare.” Journal of Applied Psychology 101(9), 1266–1304. DOI 10.1037/apl0000120. [VERIFIED] “Results demonstrate that healthcare team training improves each of Kirkpatrick’s criteria (reactions, learning, transfer, results; d = .37 to .89).” The authors also find team training “largely robust to trainee composition, training strategy, and characteristics of the work environment, with the only exception being the reduced effectiveness of team training programs that involve feedback.”

Two cautions must travel with this citation. First, the “results” tier in Kirkpatrick is heterogeneous and is not synonymous with patient mortality; the pooled d range covers reactions through results and should not be reported as a mortality effect. Second, the finding that programmes involving feedback were less effective is counter-intuitive, sits awkwardly with the debrief-centred CRM model, and deserves to be reported rather than quietly dropped.

A domain-specific meta-analysis shows the characteristic pattern — effects on some outcomes, null on others:

Brogaard, L., Glerup Lauridsen, K., Løfgren, B., Krogh, K., Paltved, C., Boie, S. & Hvidman, L. (2022). “The effects of obstetric emergency team training on patient outcome: A systematic review and meta-analysis.” Acta Obstetricia et Gynecologica Scandinavica 101(1), 25–36. DOI 10.1111/aogs.14263. PMC9564821. [VERIFIED] 21 studies (4 RCTs, 17 cohort). Brachial plexus injury: six cohort studies OR 0.47 (95% CI 0.33–0.68), but the single RCT OR 1.30 (95% CI 0.39–4.33) — low certainty. Apgar < 7 at 5 min: not significant (three cohort studies OR 0.77, 95% CI 0.51–1.19; two RCTs OR 0.87, 95% CI 0.72–1.05), moderate certainty. Effect “unclear” for hypoxic ischaemic encephalopathy, umbilical prolapse, decision-to-birth interval, and severe postpartum haemorrhage. Conclusion: “conflicting results call for controlled trials.”

Note the structure: the observational studies are positive, the randomised evidence is null or non-significant. This pattern recurs throughout this memo and is the single most important methodological signal in the whole cross-domain literature.

1.4 The WHO Surgical Safety Checklist — the divergence

This is the centrepiece of the section, and the divergence should be presented as three defensible studies rather than as one truth and two errors.

The original result.

Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A.-H. S., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C. M., Merry, A. F., Moorthy, K., Reznick, R. K., Taylor, B. & Gawande, A. A., for the Safe Surgery Saves Lives Study Group (2009). “A surgical safety checklist to reduce morbidity and mortality in a global population.” New England Journal of Medicine 360(5), 491–499. DOI 10.1056/NEJMsa0810119. [VERIFIED] Design: prospective pre/post, eight hospitals in eight cities (Toronto, New Delhi, Amman, Auckland, Manila, Ifakara, London, Seattle). 3,733 patients before, 3,955 after. 19-item checklist. Results: death 1.5% → 0.8% (P = 0.003); inpatient complications 11.0% → 7.0% (P < 0.001).

This is a before/after study with no concurrent control, conducted by a highly motivated team at sites that volunteered, several of which were resource-limited settings with high baseline event rates. Those are not fatal flaws but they are exactly the conditions under which regression to the mean, secular trend, and Hawthorne effects are strongest.

The population-level null.

Urbach, D. R., Govindarajan, A., Saskin, R., Wilton, A. S. & Baxter, N. N. (2014). “Introduction of surgical safety checklists in Ontario, Canada.” New England Journal of Medicine 370(11), 1029–1038. DOI 10.1056/NEJMsa1308261. [VERIFIED] Design: natural experiment exploiting a provincial policy mandating checklist adoption. 101 hospitals; 109,341 procedures in the 3 months before adoption and 106,370 in the 3 months after. Results: adjusted risk of death 0.71% (95% CI 0.66–0.76) before vs 0.65% (95% CI 0.60–0.70) after — odds ratio 0.91 (95% CI 0.80–1.03), P = 0.13. Adjusted risk of surgical complications 3.86% (95% CI 3.76–3.96) before vs 3.82% (95% CI 3.71–3.92) after — odds ratio 0.97 (95% CI 0.90–1.03), P = 0.29. Conclusion: “Implementation of surgical safety checklists in Ontario, Canada, was not associated with significant reductions in operative mortality or complications.”

The Ontario study is roughly thirty times larger than Haynes on procedure count and is a whole-population rollout rather than a volunteer sample. It is the single most important negative result in the cross-domain import literature.

The randomised positive.

Haugen, A. S., Søfteland, E., Almeland, S. K., Sevdalis, N., Vonen, B., Eide, G. E., Nortvedt, M. W. & Harthug, S. (2015). “Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial.” Annals of Surgery 261(5), 821–828. DOI 10.1097/SLA.0000000000000716. [VERIFIED] Design: stepped-wedge cluster RCT, two Norwegian hospitals, five clusters (cardiothoracic, neurosurgery, orthopaedic, general, urology), sequential randomised rollout, 2009–2010. 2,212 control procedures vs 2,263 checklist procedures. Results: complications 19.9% → 11.5% (P < 0.001), absolute risk reduction 8.4 (95% CI 6.3–10.5). Adjusted effect on complications remained significant, reported in the abstract as odds ratio 1.95 (95% CI 1.59–2.40) — note the abstract’s odds ratio is stated in a direction that reads as protective in context; the direction of reporting should be checked against the full paper before quoting. Mean length of stay decreased by 0.8 days (95% CI 0.11–1.43). In-hospital mortality fell significantly from 1.9% to 0.2% in one of the two hospitals, but the overall reduction across hospitals (1.6% → 1.0%) was not significant.

Note carefully: even the positive randomised trial did not demonstrate a significant overall mortality effect. It demonstrated a morbidity and length-of-stay effect. The mortality claim that circulates in the popular and managerial literature rests substantially on Haynes 2009’s uncontrolled design.

What the divergence means. The best available reading is not “checklists work” or “checklists don’t work” but “the checklist is not the active ingredient.” Clay-Williams and Colligan make this argument directly (§1.5). Three specific mechanisms are documented:

  • Fidelity collapse under mandate. Clay-Williams and Colligan report a US study in which “hospital documentation indicated 100% compliance with checklists, but observers found that on average only 4 of 13 checklist items were actually completed.” [VERIFIED — as reported in Clay-Williams & Colligan 2015 full text; the underlying study is their reference 18, which I did not retrieve — the primary is [UNVERIFIED] here]
  • Secular trend. When background rates are already falling, a mandated intervention rolled out into that trend cannot easily be shown to add anything (see §1.6, Matching Michigan).
  • The measurement becomes the target. Clay-Williams and Colligan: “high checklist compliance is no guarantee that the task is well-executed… some of the benefits that have been found to be associated with checklist usage, such as enhanced team building and nurses speaking up, are likely to be negated if compliance audits lead to sanctions.” [VERIFIED — full text]

1.5 The critical literature on the aviation analogy

This is the intellectual core of the stream. Four sources carry most of the weight.

(a) Clay-Williams & Colligan — healthcare imported the artefact without the system.

Clay-Williams, R. & Colligan, L. (2015). “Back to basics: checklists in aviation and healthcare.” BMJ Quality & Safety 24(7), 428–431. DOI 10.1136/bmjqs-2015-003957. PMC4484042. Open access (CC BY-NC). [VERIFIED — full text read]

Their argument, which is more technically precise than the usual “medicine is more complex” complaint, is that aviation uses several structurally different tools which healthcare collapsed into one object and called a checklist.

  • Aviation distinguishes normal checklists (action + verification steps, executed inside the workflow) from non-normal/emergency checklists, which are further divided into boldface items (executed from memory, then confirmed against the list), non-boldface items (decision support when time is not critical), and flowchart/decision-tree formats for multiple failures.
  • Aviation checklists “are designed for modern aircraft that are complicated, not complex; it is usually possible to define a single process path that offers optimum performance for each flight condition.” Those paths are flight-tested and published by the manufacturer.
  • Aviation keeps checklists and briefings separate, because “these processes are not mixed in the cockpit but remain distinct because they serve different purposes.” The WHO checklist fuses them: “The WHO checklist consists of a checklist (Sign In), a briefing (Time Out) and a checklist with a short briefing at the end (Sign Out).” Their warning: “If briefings are too closely coupled to checklist completion, teams may miss the cognitive shift required to move from linear or procedural work to complex or adaptive work.”
  • Roles are unspecified: “the roles of the team members in completing the checklist are not clear. Who will read the checklist? Who will verify that the actions have been completed?”
  • Workflow integration is the decisive difference: “aviation checklists are integral to the normal workflow. The aircraft does not stop while the checklist is completed… the checklist does not impose an additional burden or workload, but is actually perceived by aircrew as something that makes the flight easier. In contrast, the Time Out is performed before the case can begin, so essentially stands independently of the workflow… likely to be seen as something additional.”
  • Their diagnosis of the paradox: the Time Out “requires that everything stops and no one interrupts. In an emergency, or under extreme time pressure, it is difficult to get everyone on the team to stop… These are the times when mistakes are most likely to occur, yet paradoxically also when the Time Out portion of the checklist… is least likely to be performed as intended.”
  • Their conclusion contains the report’s best single warning sentence: “Introduction of a new tool without full consideration of its purpose, benefits and limitations may actually increase risk to patients, providers and the system as a whole.” And: “Overreliance on checklists as a safety net can lead to omission of other safety practices that may better support safety through reliability and resilience.”

They also make a specific and damaging point about the other famous checklist success, the central-line bundle: “successful reduction of CLABSIs was not due to the checklist alone: multiple interventions addressing ICU safety were implemented at the same time, and it remains unclear what role the checklist specifically played.” They list plausible non-checklist mechanisms: the Hawthorne effect, empowerment of nurses and levelling of the power gradient, and habit formation.

(b) Amalberti, Auroy, Berwick & Barach — five structural barriers. This is the strongest single citation for the boundary-conditions section.

Amalberti, R., Auroy, Y., Berwick, D. & Barach, P. (2005). “Five system barriers to achieving ultrasafe health care.” Annals of Internal Medicine 142(9), 756–764. DOI 10.7326/0003-4819-142-9-200505030-00012. [VERIFIED] “It is often assumed that copying and adapting the success stories of nonmedical industries, such as civil aviation and nuclear power, will make medicine as safe as these industries. However, the solution is not that simple.” The five barriers: “the need to limit the discretion of workers, the need to reduce worker autonomy, the need to make the transition from a craftsmanship mindset to that of equivalent actors, the need for system-level (senior leadership) arbitration to optimize safety strategies, and the need for simplification.” Plus three problems unique to healthcare: “a wide range of risk among medical specialties, difficulty in defining medical error, and various structural constraints (such as public demand, teaching role, and chronic shortage of staff).” And the crucial framing: “The most important difference among industries lies not so much in the pertinent safety toolkit, which is similar for most industries, but in an industry’s willingness to abandon historical and cultural precedents and beliefs that are linked to performance and autonomy.”

That last sentence is the thesis of this entire stream in one line: the toolkit transfers; the preconditions do not.

(c) Gaba — structural and organisational mismatch.

Gaba, D. M. (2000). “Structural and Organizational Issues in Patient Safety: A Comparison of Health Care to Other High-Hazard Industries.” California Management Review 43(1), 83–102. DOI 10.2307/41166067. [VERIFIED for author/year/title/journal/volume/issue; page range 83–102 is [SECONDARY] — the CMR page I retrieved did not print pagination] Abstract (verbatim from publisher page): “…this article discusses the ways in which the health care industry has failed to meet systematically the standards for achieving high reliability, based in part on two existing theories about the management of high-hazard environments — High Reliability Organization Theory (HROT) and Normal Accidents Theory (NAT).”

Gaba is important precisely because he is the man who brought CRM into medicine. That the founder of healthcare CRM spent 2000 writing about why healthcare is structurally unlike aviation is the strongest available evidence that the critical position is not anti-safety contrarianism.

(d) Kapur et al. — the comparative review, with reservations catalogued.

Kapur, N., Parand, A., Soukup, T., Reader, T. & Sevdalis, N. (2016). “Aviation and healthcare: a comparative review with implications for patient safety.” JRSM Open 7(1), 2054270415616548. DOI 10.1177/2054270415616548. PMC4710114. Open access. [VERIFIED — full text partially read] Domains compared: “Checklists, Training, Crew Resource Management, Sterile Cockpit, Investigation and Reporting of Incidents and Organisational Culture.” Their conclusion is deliberately hedged: “whilst healthcare has much to learn from aviation in certain key domains, the transfer of lessons from aviation to healthcare needs to be nuanced, with the specific characteristics and needs of healthcare borne in mind.” They explicitly note dissent: “Some authors have expressed reservations about the analogies between aviation and healthcare, and others have noted that industries such as mining and metal manufacture may provide just as valuable safety lessons as aviation.” They cite Reason’s concept of the “lethal convergence of benevolence” — that close personal contact in healthcare “may result in the bypassing of protocols, barriers and safeguards, often with patients’ best interests at heart.” [VERIFIED as reported by Kapur; the Reason primary is [SECONDARY]]

Kapur et al. also supply a scale comparison worth quoting carefully: worldwide flight hours roughly doubled from ~25 million (1993) to ~54 million (2013) while annual fatalities fell from ~450 to ~250; against an estimated 200,000 preventable medical deaths per year in the USA alone. [VERIFIED as reported by Kapur; the underlying figures are [SECONDARY] and the 200,000 estimate is contested in the wider literature — do not present it as settled]

(e) The recent revival of the debate.

Cristancho, S. M. (2024). “Why it’s time to reawaken our debates on the aviation analogy.” Perspectives on Medical Education 13(1), 332–335. DOI 10.5334/pme.1399. PMC11160406. [VERIFIED via publisher] Four claimed misalignments: (1) clinicians must consult patients, pilots do not consult passengers; (2) medical teams must balance safety and resilience, aviation prioritises safety; (3) clinicians legitimately bypass protocols in the patient’s interest, flight crews follow procedure; (4) aviation maintains rigid role separation, healthcare roles overlap. Empirically grounded claim: her interview research found frontline clinicians who succeeded during COVID-19 did so by abandoning checklists — “checklists become impractical when situations implode and priorities conflict.”

Also retrieved and available for the report:

Catchpole, K. (2013). “Spreading human factors expertise in healthcare: untangling the knots in people and systems.” BMJ Quality & Safety 22(10), 793–797. DOI 10.1136/bmjqs-2013-002036. [VERIFIED metadata only — Europe PMC holds no abstract and I did not retrieve the full text. Its argument is [UNVERIFIED] here.]

Catchpole, K., Cohen, T., Alfred, M., Lawton, S., Kanji, F., Shouhed, D., Nemeth, L. & Anger, J. (2024). “Human Factors Integration in Robotic Surgery.” Human Factors 66(3), 683–700. DOI 10.1177/00187208211068946. PMC11268371. [VERIFIED] — useful as a contemporary Catchpole statement that new operating-room technology “present[s] challenges for physical, procedural, team, and organizational integration” and that robotic surgery implementation “has focused on establishing the technical skills of the surgeon on the console, and has not systematically addressed the new skills required for other team members.” That is a precise restatement of aviation’s own automation lesson, arrived at independently.

1.6 The replication problem beyond checklists: Keystone vs Matching Michigan

This is the cleanest natural experiment in the whole import literature and belongs in the report.

Pronovost, P., Needham, D., Berenholtz, S., Sinopoli, D., Chu, H., Cosgrove, S., Sexton, B., Hyzy, R., Welsh, R., Roth, G., Bander, J., Kepros, J. & Goeschel, C. (2006). “An intervention to decrease catheter-related bloodstream infections in the ICU.” New England Journal of Medicine 355(26), 2725–2732. DOI 10.1056/NEJMoa061115. [VERIFIED] 103 reporting ICUs, predominantly Michigan; 1,981 ICU-months, 375,757 catheter-days. Median infection rate per 1,000 catheter-days fell from 2.7 at baseline to 0 at 3 months (P ≤ 0.002); mean fell from 7.7 to 1.4 at 16–18 months (P < 0.002). Incidence-rate ratio 0.34 (95% CI 0.23–0.50) at 16–18 months. Reported as “a large and sustained reduction (up to 66%).”

Then England tried to replicate it at national scale.

Bion, J., Richardson, A., Hibbert, P., Beer, J., Abrusci, T., McCutcheon, M., Cassidy, J., Eddleston, J., Gunning, K., Bellingan, G., Patten, M. & Harrison, D., for the Matching Michigan Collaboration & Writing Committee (2013). “‘Matching Michigan’: a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England.” BMJ Quality & Safety 22(2), 110–123. DOI 10.1136/bmjqs-2012-001325. PMC3585494. [VERIFIED] 215 of 223 English ICUs; 438,887 CVC-patient days; 1,092 CVC-BSIs over 20 months. Adult ICU mean rate fell 3.7 → 1.48 per 1,000 CVC-patient days (P < 0.0001); paediatric 5.65 → 2.89 (P = 0.625, not significant). The decisive sentence: “The trend for infection rate reduction did not accelerate following interventions training.” Conclusion: “The marked reduction in CVC-BSI rates in English ICUs found in this study is likely part of a wider secular trend for a system-wide improvement in healthcare-associated infections.”

Rates fell. The programme cannot be shown to have caused the fall. The companion ethnography explains why:

Dixon-Woods, M., Leslie, M., Tarrant, C. & Bion, J. (2013). “Explaining Matching Michigan: an ethnographic study of a patient safety program.” Implementation Science 8, 70. DOI 10.1186/1748-5908-8-70. PMC3704826. Open access. [VERIFIED] 98 staff interviews, non-participant observation on 19 ICUs, 29 further telephone interviews. Findings: “One Matching Michigan unit transformed its practices and culture in response to the program; five boosted existing efforts, and eleven made little change.” They name the two effects precisely: “The first is a failure to outperform the secular trend. The second is the decline effect, where an initially promising intervention appears not to deliver equally successful results when attempts are made to replicate it in new settings.” Cause: “it was not an exact replica of the original project. Outer and inner contexts strongly modified the program’s effects… national infection control policies that were perceived by some as top-down and punitive… undermined engagement.” And the memorable formulation: “The ‘phenotype’ of compliance may therefore arise through different ‘genotypes.’”

That last point is the deepest methodological insight available to the report. Observed compliance with an imported aviation-style practice tells you almost nothing about whether the underlying safety mechanism is present.

1.7 Automation bias in clinical AI — the closest empirical analogue to the copilot problem

If the report needs measured evidence that an automated aid can improve average performance while degrading the best operators, healthcare supplies it and aviation does not.

The foundational finding.

Povyakalo, A. A., Alberdi, E., Strigini, L. & Ayton, P. (2013). “How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography.” Medical Decision Making 33(1), 98–107. DOI 10.1177/0272989X12465490. [VERIFIED] Reanalysis of a published study: 50 professional readers, 180 mammograms, each read with and without computer-aided detection (CAD). Result: CAD associated with a +0.016 increase in sensitivity (95% CI 0.003–0.028) for the 44 least discriminating radiologists on 45 relatively easy, mostly CAD-detected cancers. But for the 6 most discriminating radiologists, sensitivity decreased by 0.145 (95% CI 0.034–0.257) on the 15 relatively difficult cancers. Their conclusion: “despite the original study detecting no significant average effect, CAD helped the less discriminating readers but hindered the more discriminating readers.”

This is the single most quotable result in the memo. The average effect was null. Underneath the null were two large opposite effects. An evaluation methodology that looks only at group means is structurally blind to exactly the harm that matters — the degradation of the expert who was the system’s last line of defence. That is precisely the shape of the cockpit-automation problem.

Deskilling, measured, in 2025.

Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H.-O., Blom, J., Buszkiewicz, M., Halvorsen, N., Hassan, C., Romańczyk, T., Holme, Ø., Jarus, K., Fielding, S., Kunar, M., Pellise, M., Pilonis, N., Kamiński, M. F., Kalager, M., Bretthauer, M. & Mori, Y. (2025). “Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study.” The Lancet Gastroenterology & Hepatology 10(10), 896–903. DOI 10.1016/S2468-1253(25)00133-5. [VERIFIED] Design: retrospective observational, four Polish endoscopy centres in the ACCEPT trial, comparing non-AI-assisted colonoscopies 3 months before vs 3 months after AI introduction. 1,443 patients (795 before, 648 after). Result: adenoma detection rate of standard, unaided colonoscopy fell from 28.4% (226/795) to 22.4% (145/648) — absolute difference −6.0% (95% CI −10.5 to −1.6; P = 0.0089). Multivariable OR for AI exposure 0.69 (95% CI 0.53–0.89). Interpretation: “Continuous exposure to AI might reduce the ADR of standard non-AI assisted colonoscopy, suggesting a negative effect on endoscopist behaviour.”

Caveats to report honestly: this is observational and retrospective, not randomised; a 3-month before/after window is short; and the study drew substantial correspondence in the same journal (four letters plus authors’ reply, Lancet Gastroenterol Hepatol 10(12), 1060–1062, DOI 10.1016/S2468-1253(25)00288-2, …00289-4, …00290-0, and authors’ reply …00324-3) [VERIFIED — the existence and citations of the correspondence; their content is [UNVERIFIED]]. It is nonetheless the first direct measurement of automation-induced skill decay in a clinical workforce, and it is the closest available analogue to the manual-flying-skills literature.

Experienced clinicians are not immune.

Dratsch, T., Chen, X., Rezazade Mehrizi, M., Kloeckner, R., Mähringer-Kunz, A., Püsken, M., Baeßler, B., Sauer, S., Maintz, D. & Pinto dos Santos, D. (2023). “Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance.” Radiology 307(4), e222176. DOI 10.1148/radiol.222176. [VERIFIED] 27 radiologists, 50 mammograms, a purported AI system that gave an incorrect BI-RADS category for 12 of 40 cases. Correct-rating rates when AI was right vs wrong: inexperienced 79.7% → 19.8% (P < .001); moderately experienced 81.3% → 24.8% (P < .001); very experienced 82.3% → 45.5% (P = .003). Conclusion: “inexperienced, moderately experienced, and very experienced radiologists reading mammograms are prone to automation bias.”

The very experienced readers were less affected than novices but still lost roughly 37 percentage points of accuracy when the machine was wrong. Expertise attenuates automation bias; it does not confer immunity.

Experience does not predict who benefits.

Yu, F., Moehring, A., Banerjee, O., Salz, T., Agarwal, N. & Rajpurkar, P. (2024). “Heterogeneity and predictors of the effects of AI assistance on radiologists.” Nature Medicine 30(3), 837–849. DOI 10.1038/s41591-024-02850-w. PMC10957478. [VERIFIED] 140 radiologists, 15 chest X-ray diagnostic tasks. “Conventional experience-based factors, such as years of experience, subspecialty and familiarity with AI tools, fail to reliably predict the impact of AI assistance. Additionally, lower-performing radiologists do not consistently benefit more from AI assistance, challenging prevailing assumptions.” “The occurrence of AI errors strongly influences treatment outcomes, with inaccurate AI predictions adversely affecting radiologist performance on the aggregate of all pathologies and on half of the individual pathologies investigated.”

This matters for training design: you cannot target automation-bias training by seniority, because seniority does not predict vulnerability.

The systematic-review baseline.

Goddard, K., Roudsari, A. & Wyatt, J. C. (2012). “Automation bias: a systematic review of frequency, effect mediators, and mitigators.” Journal of the American Medical Informatics Association 19(1), 121–127. DOI 10.1136/amiajnl-2011-000089. PMC3240751. [VERIFIED] 13,821 papers retrieved, 74 met inclusion criteria. Key framing: “although most research shows overall improved performance with use, there is often a failure to recognize the new errors that CDSS can introduce.” Mediators: cognitive style, task-specific experience, trust and confidence; environmental mediators of workload, task complexity and time constraint. Mitigators: “training and emphasizing user accountability, and DSS design factors such as the position of advice on the screen, updated confidence levels attached to DSS output, and the provision of information versus recommendation.”

That final distinction — information versus recommendation — is the most directly actionable design finding in this stream.

Goddard, K., Roudsari, A. & Wyatt, J. C. (2014). “Automation bias: empirical results assessing influencing factors.” International Journal of Medical Informatics 83(5), 368–375. DOI 10.1016/j.ijmedinf.2014.01.001. [VERIFIED] 26 UK NHS GPs, 20 hypothetical prescribing scenarios, advice incorrect in 6. Pre-advice accuracy 50.38% → post-advice 58.27%. Automation bias (correct → incorrect switching) occurred in 5.2% of all cases, against a net improvement of 8%. “Lower clinical experience was associated with more decision switching.”

Net positive, with a measurable minority of induced harms. That is the honest shape of the result, and it is the shape the report should expect in every domain.

2. Maritime

2.1 The lineage

Bridge Resource Management and Maritime Resource Management are direct CRM derivatives. The strongest verified anchor for the maritime CRM lineage in this research is Salas et al. (2006), which reviewed CRM training evaluations across “aviation, medicine, offshore oil production and maintenance, shipping/maritime, and nuclear power domains” and found “mixed results across and within domains” [VERIFIED — see §5.2].

[UNVERIFIED] — The STCW Manila amendments (2010), adding mandatory leadership, teamwork and resource-management competences to the STCW Convention, were not verified in this research. The claim is well known and almost certainly correct, but the report must not cite the specific regulation, table (A-II/1, A-II/2) or entry-into-force date (commonly given as 1 January 2012) on my authority. This is a discrete, easily closed gap: fetch the IMO STCW page and the Manila amendments text.

2.2 ECDIS-assisted groundings — the strongest structural parallel in the whole memo

This case deserves prominence because it reproduces the cockpit-automation pattern almost feature for feature: a screen-based system replacing a paper artefact, safety functions that must be configured to work, alarms that were not armed, a training regime that lagged the technology, and a repeated accident pattern.

Case 1 — CFL Performer, 2008.

Marine Accident Investigation Branch (2008). Report on the investigation of the grounding of CFL Performer, Haisborough Sand, North Sea, 12 May 2008. Report No. 21/2008. Southampton: MAIB. December 2008. Position 52° 54.123′ N, 001° 43.695′ E. [VERIFIED — official PDF retrieved: https://assets.publishing.service.gov.uk/media/547c7001e5274a428d000063/CFLPerformerReport.pdf]

Verbatim and near-verbatim findings:

  • ECDIS safety features “were not utilised and system warnings were not acted upon” during both planning and monitoring.
  • The route check page was never used. The passage plan’s title displayed in red, indicating it crossed the safety contour — and the plan was executed anyway.
  • The safety contour was set to 30 m, but “the safety contour alarm did not activate at any time during the voyage” because no watch vector had been defined. MAIB: “The setting of a watch vector is therefore an extremely important feature without which many of the chart alarms cannot operate.”
  • Display scale mattered: at 1:100,000, “the shallows over the bank are much harder to identify than when viewed at a scale of 1:50000.”
  • Training: none of the bridge officers had received ECDIS training beyond initial pre-service training; the chief and second officers on watch “had not been trained in the use of ECDIS.” MAIB: “the deck officers were ignorant of many of the system requirements and features and operated the system in a very basic and inherently dangerous manner.”
  • The automation-trust finding, which could have been written about a flight deck: the second officer “placed an undue reliance on the ECDIS”; and “there is a danger that many bridge watchkeepers will increasingly trust what is displayed without question. As this case demonstrates, such trust can be misplaced.”

Case 2 — Muros, 2016. The same sandbank, eight years later.

Marine Accident Investigation Branch (2017). Report on the grounding of the bulk carrier Muros, Haisborough Sand, North Sea, 3 December 2016. Accident Investigation Report 22/2017. Published 19 October 2017. [VERIFIED — gov.uk report page retrieved: https://gov.uk/maib-reports/grounding-of-bulk-carrier-muros]

Findings: an unsafe revised passage plan created by the officer of the watch; route verification performed at insufficient chart scale; the master directed route changes but did not review or approve the modified plan; ECDIS safety features “ignored, overlooked or disabled”; reduced alertness; and the summary judgement that ECDIS use aboard “was not as envisaged by regulators or equipment manufacturers.”

And the decisive institutional response, quoted verbatim from the report page: “No recommendations have been made but MAIB has started a safety study to characterise and understand the way ECDIS is currently being used on board ships.”

That sentence is the finding. After the same class of accident on the same sandbank eight years apart, the investigator concluded that it could not usefully recommend anything further about the equipment or the crew, and that the problem was systemic and needed characterising first. Compare this with aviation’s institutional capacity to mandate a training change fleet-wide through a single regulator.

The follow-on safety study.

Marine Accident Investigation Branch. Application and usability of ECDIS (safety study). Published at https://gov.uk/government/publications/application-and-usability-of-ecdis-safety-study [SECONDARY — the publication's existence and URL appeared in search results and is consistent with the Muros report's stated intention, but I did **not** retrieve the document. Its findings, date and report number are [UNVERIFIED].]

Case 3 — Ovit, 2013. The chemical tanker Ovit grounded on the Varne Bank in the Dover Strait in September 2013; MAIB attributed it to inadequate ECDIS use and passage planning. [SECONDARY — multiple maritime trade sources retrieved (gCaptain, Maritime Executive, Safety4Sea, Riviera) describe the case consistently. I was blocked from retrieving the official MAIB report page. The report number (commonly cited as 24/2014) and the exact grounding date are [UNVERIFIED] — retrieve the official report before citing.]

Why this parallel is strong. The ECDIS pattern contains, in one case series: mode/configuration confusion (safety contour and watch vector), alarm design failure (an alarm that silently cannot fire), scale-dependent display salience (an information-presentation failure), the substitution of trust for cross-checking, and — critically — a generic-versus-type-specific training gap directly analogous to aircraft type ratings. Every ECDIS manufacturer’s interface differs; there is no equivalent of a type rating enforced by a single global regulator with revocation power.

2.3 Costa Concordia and the authority gradient

[UNVERIFIED] — I did not research the Costa Concordia (13 January 2012) formal investigation or the academic BRM/authority-gradient analyses of it. This is a genuine gap. The case is frequently invoked as a maritime authority-gradient exemplar, and the Italian Ministry of Infrastructure and Transport marine casualty investigation report exists, but nothing in this memo should be asserted about its findings on my authority.

2.4 MASS and autonomous shipping

[UNVERIFIED] — The IMO’s Maritime Autonomous Surface Ships regulatory scoping exercise and the developing MASS Code were not researched. The training question it opens (what competence is required of a remote operator, and how it is assessed and revalidated) is exactly the question this report is about, and it should be researched properly rather than sketched.

3. Nuclear power

Nuclear is thinner in this memo than healthcare and maritime, by the coordinator’s instruction and by honest limitation. What follows is what I verified.

3.1 Three Mile Island, the Kemeny Commission and INPO

Institute of Nuclear Power Operations — official history. [VERIFIED — https://www.inpo.info/history retrieved] INPO “was formally established in December 1979, directly following the Three Mile Island accident.” The Carter administration created the Kemeny Commission to investigate and recommend operational improvements, “which led to INPO’s establishment.” INPO’s stated remit: “monitoring and evaluating appropriate safety standards — including those for management, quality assurance, and operating procedures and practices at nuclear facilities.” INPO later established the World Association of Nuclear Operators (WANO), whose purpose is to “maximize the safety and reliability of nuclear power plants worldwide by working together to assess, benchmark and improve performance.”

[UNVERIFIED] — The specific Kemeny Commission recommendations on operator training, and in particular the shift to plant-specific (rather than generic) simulator training, were not verified. This is important and the report should not paraphrase the Commission’s recommendations without retrieving the report (Report of the President’s Commission on the Accident at Three Mile Island, October 1979). Likewise, INPO’s National Academy for Nuclear Training and its accreditation of utility training programmes are well known but [UNVERIFIED] here — the INPO history page I retrieved did not cover them.

The structurally interesting point, which the report can make on the verified material alone, is that nuclear’s institutional response to an operator-performance accident was to create an industry-owned, non-regulatory peer body with the power to evaluate and accredit — a different solution from aviation’s (a public regulator plus a protected reporting system), arrived at for the same problem. That is a useful counter-example to the assumption that aviation’s institutional form is the only one that works.

3.2 Rasmussen and the intellectual provenance

Rasmussen, J. (1983). “Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models.” IEEE Transactions on Systems, Man, and Cybernetics SMC-13(3), 257–266. DOI 10.1109/TSMC.1983.6313160. [VERIFIED — citation confirmed via IEEE Xplore record 6313160 and dblp; the issue number (3) is [SECONDARY]]

The skill–rule–knowledge framework emerged from Risø National Laboratory’s process-control and nuclear work, not from aviation. It is the conceptual backbone of most subsequent thinking about automation-induced skill decay — including aviation’s — and the report should note that the direction of intellectual traffic between nuclear and aviation ran both ways.

Reason, J. (1990). Human Error. Cambridge: Cambridge University Press. [SECONDARY — not verified in this research; the Swiss cheese model's provenance and the "lethal convergence of benevolence" concept reach this memo via Kapur et al. (2016), which cites Reason directly.]

3.3 What nuclear does that aviation does not

Verified-adjacent observations that the report can make carefully:

  • Event rarity. Nuclear’s severe-accident base rate is far lower than aviation’s fatal-accident rate, which means nuclear cannot use accident counts as a training-effectiveness measure at all; it uses precursor and performance indicators instead. [UNVERIFIED as a sourced claim — the reasoning is sound but I did not retrieve INPO/IAEA indicator documentation.]
  • Time constants. Nuclear transients unfold over hours to days; cockpit upsets over seconds to minutes. This changes what training must produce: nuclear needs sustained diagnostic reasoning under shift handover, aviation needs immediate procedural retrieval under stress. Clay-Williams and Colligan’s boldface/non-boldface distinction (§1.5) is the right vocabulary for this difference and is [VERIFIED].
  • Site specificity. Nuclear plants are effectively one-offs; aviation has a small number of certified type variants. This is why nuclear went to plant-specific simulators while aviation could standardise. [SECONDARY — consistent with the INPO material retrieved, but the plant-specific simulator mandate itself is [UNVERIFIED] here.]

4. Rail and road

Thin, and flagged as thin rather than padded.

4.1 Rail

[UNVERIFIED] — ETCS/ERTMS driver-machine interface issues, automatic train protection and the vigilance problem, and formal investigations touching driver reliance on automation were not researched in this stream. Nothing in this memo should be cited on rail. This is the largest single gap and I recommend it be assigned rather than inferred. The obvious primary sources are the UK RAIB, the European Union Agency for Railways, and the ERA’s ERTMS documentation.

4.2 Road

Casner, S. M., Hutchins, E. L. & Norman, D. (2016). “The challenges of partially automated driving.” Communications of the ACM 59(5), 70–77. DOI 10.1145/2830565. [VERIFIED — citation confirmed via eScholarship record and ACM DL]

This citation matters for the report’s architecture because Casner is the same researcher associated with the manual-flying-skills work, carrying the aviation lesson explicitly into driving, with Hutchins (distributed cognition) and Norman (design). The paper is the correct anchor for the claim that the aviation automation lesson was consciously exported to road.

However: I retrieved only the abstract and metadata, not the argument. The published abstract is generic (“many obstacles remain… may take some time”). I could not verify any specific claim in the paper about monitoring, vigilance, skill degradation, mode confusion, handover, or pilot/driver differences. [UNVERIFIED] — do not attribute specific arguments to this paper without reading the full text. It is open access via eScholarship (https://escholarship.org/uc/item/197642mw) and this gap is cheap to close.

[UNVERIFIED] — NTSB investigations of Tesla Autopilot crashes (Williston, Florida, May 2016, NTSB/HAR-17/02; Mountain View, California, March 2018, NTSB/HAR-20/01) and their findings on driver monitoring and operational design domain were not researched. The report should not cite report numbers, findings or probable-cause language on my authority. The NTSB publishes these reports openly and the gap is straightforwardly closeable.

[UNVERIFIED] — SAE Level 3 handover and takeover-time research was not researched.

The one thing the report can say on verified material: road transport inverts nearly every aviation precondition simultaneously (see §5.3). Whatever the empirical literature turns out to show, the structural prediction is that aviation’s training answer cannot transfer to driving, because the mechanism that makes it work in aviation — mandatory, recurrent, employer-funded, regulator-enforced training on standardised equipment for a licensed professional workforce — has no counterpart for private drivers.

5. The boundary conditions — what aviation has that other domains do not

This is the most important section. The argument is not that aviation’s techniques are worthless elsewhere. It is that aviation’s techniques were embedded in a set of structural preconditions, and that importing the technique without the precondition reliably produces the observed pattern: strong early uncontrolled results, weak or null results at population scale, and compliance without mechanism.

Amalberti et al. (2005) state the general form of this argument better than anyone: “The most important difference among industries lies not so much in the pertinent safety toolkit, which is similar for most industries, but in an industry’s willingness to abandon historical and cultural precedents and beliefs that are linked to performance and autonomy.” [VERIFIED]

5.1 The nine preconditions

Below, each precondition, with its verification status and a judgement of necessary vs helpful. I distinguish these on a specific test: is there a documented case of the aviation technique working at scale in a domain lacking this feature? Where yes, the feature is helpful. Where the mechanism cannot logically operate without it, it is necessary.

(1) A single global regulator with real authority and harmonised standards (ICAO). [UNVERIFIED] as a sourced claim — I did not retrieve ICAO documentation. The characterisation is uncontroversial but should be cited to a primary ICAO source. Judgement: helpful, not necessary — but load-bearing for scale. Nuclear achieved comparable performance through INPO/WANO, an industry-owned peer body rather than a regulator [VERIFIED via INPO]. So a single regulator is not the only institutional form that works. What is necessary is some body with the authority to compel and to withdraw permission to operate. Healthcare’s failure mode in Ontario is instructive: a provincial mandate produced universal adoption and no measurable effect [VERIFIED — Urbach 2014], because mandating an artefact is not the same as holding authority over competence.

(2) Mandatory recurrent training and licence revalidation on a fixed cycle, employer-funded. [UNVERIFIED] as a sourced claim in this stream. Judgement: necessary. This is the strongest candidate for a genuinely necessary precondition, and the reasoning is available from verified material. The Budzyń finding — that unaided skill decayed measurably within three months of AI exposure [VERIFIED] — establishes that skill decay under automation operates on a timescale of months. Any intervention that does not re-test competence on a comparable cycle cannot detect or correct it. Healthcare, maritime and road have no equivalent of the six-monthly proficiency check. This is the precondition whose absence most directly defeats the technique.

(3) Standardised, certified equipment with a small number of type variants. Judgement: necessary for procedural standardisation; helpful otherwise. The ECDIS case is the proof [VERIFIED — MAIB 21/2008 and 22/2017]. When every manufacturer’s interface differs and safety functions must be configured by the user, a standard procedure cannot be written, a type rating cannot be defined, and the failure recurs on the same sandbank eight years apart. Clay-Williams and Colligan supply the aviation contrast: aviation checklists are written by the manufacturer against a flight-tested single optimum path for a specific aircraft type [VERIFIED]. Healthcare’s analogue — the Anaesthesia Apparatus Checkout Recommendations, “intended to be peer-reviewed, modified and updated for each specific type of manufactured anaesthesia equipment” — is, they note, the one place healthcare did reproduce the aviation model properly [VERIFIED].

(4) Legally protected, de-identified reporting. [UNVERIFIED] in specifics — I attempted to verify the NASA ASRS’s founding year, its limited-immunity provisions and the rationale for NASA rather than the FAA holding the data, and the official ASRS overview page I retrieved did not contain that information. ICAO Annex 19 and EU Regulation 376/2014 were not researched. Do not cite the 1976 founding date, the immunity provisions, or the EU regulation on my authority. Judgement: necessary — and the best-evidenced of all of them. Salas et al. (2006) name “access to data” as one of three critical needs without which CRM’s impact cannot be determined [VERIFIED]. Dixon-Woods et al. (2013) found that English infection-control policy “perceived by some as top-down and punitive” actively “undermined engagement in the program” [VERIFIED]. Clay-Williams and Colligan warn that compliance auditing with sanctions negates precisely the behaviours — nurses speaking up — that the intervention exists to produce [VERIFIED]. Kapur et al. identify a “blame-free culture in the case of reporting” as one of three cultural attributes distinguishing aviation from healthcare [VERIFIED]. Four independent verified sources converge: without protected reporting you get neither the evidence base nor the behaviour. This is the precondition the report should press hardest.

(5) Two operators doing the same job simultaneously, cross-checking, with an explicit challenge protocol. Judgement: necessary for CRM specifically; not necessary for safety generally. CRM is, mechanically, a protocol for managing a two-person authority gradient in a shared task with shared situational awareness. Cristancho (2024) identifies exactly this as a misalignment: aviation “maintains strict role separation, while healthcare professionals have overlapping responsibilities” [VERIFIED]. Clay-Williams and Colligan’s complaint that in the WHO checklist “the roles of the team members… are not clear. Who will read the checklist? Who will verify…?” is the same point [VERIFIED]. A surgical team is not two surgeons doing the same operation in parallel; it is a differentiated team. The technique was designed for a redundant dyad and applied to a differentiated group.

(6) One dominant, unambiguous, catastrophic and countable failure outcome. Judgement: necessary for evaluation; not for the intervention itself. Amalberti et al. name “difficulty in defining medical error” as one of three problems unique to healthcare [VERIFIED]. Without an agreed countable outcome you cannot build an evidence base, which is why Salas’s reviews found the CRM safety question unanswerable. The Brogaard meta-analysis shows what happens instead: significant on one outcome, null on another, unclear on four more [VERIFIED].

(7) Extreme capital intensity that makes simulators economically rational. Judgement: helpful, not necessary. Healthcare built high-fidelity simulation without aviation’s capital structure — that is exactly what Howard and Gaba did in 1992 [VERIFIED]. Simulation is affordable in many domains. What is not affordable elsewhere is aviation’s cadence of simulation, which returns to precondition (2).

(8) A small, elite, licensed, unionised, homogeneously trained workforce. Judgement: helpful, and probably necessary at scale. Amalberti et al.’s first three barriers — limiting worker discretion, reducing worker autonomy, and moving “from a craftsmanship mindset to that of equivalent actors” — are precisely a description of what a homogeneously trained workforce provides and what a craft profession resists [VERIFIED]. The phrase “equivalent actors” is the key: aviation can write one procedure because any qualified pilot is interchangeable with any other on that type. Medicine explicitly does not want this, and Cristancho argues it should not [VERIFIED].

(9) Extremely low base rates. Judgement: this is not an advantage — it is a trap, and it is aviation’s own methodological problem.

This deserves separate treatment because it inverts the usual telling.

5.2 Aviation cannot prove its own training works

The two decisive citations:

Salas, E., Burke, C. S., Bowers, C. A. & Wilson, K. A. (2001). “Team training in the skies: does crew resource management (CRM) training work?” Human Factors 43(4), 641–674. DOI 10.1518/001872001775870386. [VERIFIED] Reviewed 58 published accounts of CRM training using Kirkpatrick’s framework. “Results indicated that CRM training generally produced positive reactions, enhanced learning, and promoted desired behavioral changes. However, we cannot ascertain whether CRM has an effect on an organization’s bottom line (i.e., safety).”

Salas, E., Wilson, K. A., Burke, C. S. & Wightman, D. C. (2006). “Does crew resource management training work? An update, an extension, and some critical needs.” Human Factors 48(2), 392–412. DOI 10.1518/001872006777724444. [VERIFIED] Reviewed 28 further accounts across aviation, medicine, offshore oil, shipping/maritime and nuclear power. “Findings indicate that CRM training generally produced positive reactions from trainees; however, the impact of training on learning and behavioral changes suggest mixed results across and within domains. Furthermore, and as was found by Salas, Burke, et al. in 2001, we cannot ascertain whether CRM has had an impact on the organization’s bottom line (i.e., safety).” Their diagnosis of what is missing: “a mandate, access to data, and resources.” And the warning that is the thesis of this stream: “Because of the purported success of CRM training in aviation, other high-consequence domains have begun to implement CRM training for their workforces. However, the true impact of CRM training in aviation and these other domains has yet to be determined.”

Read that carefully. In 2006, twenty-seven years after United Airlines ran the first CRM course, the leading team-training research group in the world, reviewing the literature across five industries, could not establish that CRM reduces accidents — in aviation. And they identified the export itself as resting on a “purported” success.

This is not a fringe position. It is the standard finding of the field’s own systematic reviews, published in the field’s flagship journal, and it has never been overturned by anything I retrieved.

Why this happens, and why it is not evidence that CRM is useless. Aviation’s fatal accident base rate is so low that detecting a training effect on accidents requires statistical power no operator or regulator can assemble. You cannot randomise airlines to no-CRM. You cannot accumulate enough events. The very safety that makes aviation the exemplar is what makes its exemplary intervention unprovable. The honest statement is: CRM has strong evidence at Kirkpatrick levels 1–3 (reactions, learning, behaviour) and no established evidence at level 4 (organisational safety outcomes), in aviation or anywhere else.

The report should carry this as its central caution, because it reframes the entire cross-domain export. Healthcare was not importing a proven intervention and failing to replicate it. Healthcare was importing an unproven intervention and then, being a field with high base rates and a randomised-trial culture, actually testing it — which is why the null results appeared in medicine first. The Ontario study [VERIFIED — Urbach 2014] and Matching Michigan [VERIFIED — Bion 2013] are not evidence that healthcare implemented badly. They may be the first adequately powered tests the aviation safety package has ever received.

That is the most important sentence in this memo and it should survive into the report.

5.3 Which preconditions are necessary — the summary judgement

Necessary (the mechanism cannot operate without them):

  • Mandatory recurrent competence revalidation on a fixed cycle, employer-funded — because skill decay under automation runs on a months-long timescale (Budzyń 2025) [VERIFIED].
  • Legally protected, de-identified reporting — because without it you get neither behaviour nor evidence (Salas 2006; Dixon-Woods 2013; Clay-Williams 2015; Kapur 2016, four independent verified sources) [VERIFIED].
  • Standardised, certified equipment — necessary specifically for procedural standardisation and type-specific training (MAIB 21/2008, 22/2017; Clay-Williams 2015) [VERIFIED].
  • A countable, agreed failure outcome — necessary for evaluation, not for the intervention (Amalberti 2005) [VERIFIED].
  • Some authority able to compel competence and withdraw permission to operate — though not necessarily a single global public regulator; INPO/WANO is a working counter-form [VERIFIED].

Helpful but not necessary:

  • A single global regulator specifically (ICAO’s form is one solution, not the solution).
  • Capital intensity making simulators rational (simulation is affordable; the cadence is the constraint).
  • A small elite homogeneous workforce (helpful, and closely tied to Amalberti’s “equivalent actors” barrier, but a matter of degree).

Neither — actively misleading:

  • Extremely low base rates. This is not a precondition aviation possesses and others lack; it is a consequence that destroys aviation’s own ability to evaluate, and it is the reason the exported package arrived in other industries with far weaker evidence than its reputation suggested.

The paired-operator cross-check sits awkwardly: necessary for CRM as designed, but its absence does not preclude safety improvement — it precludes CRM specifically transferring unmodified. Domains with differentiated teams need a differently designed intervention, which is close to what Clay-Williams and Colligan and Cristancho both argue.

6. What the report should conclude from this stream

  1. State the divergence in the checklist literature honestly and completely, with all three studies and their actual numbers. Do not present Haynes 2009 as settled.
  2. State that aviation cannot demonstrate CRM’s safety effect. This is the most defensible and most surprising finding available, and it is sourced to two systematic reviews in Human Factors.
  3. Use Povyakalo et al. (2013) and Budzyń et al. (2025) as the empirical anchors for the deskilling and over-reliance argument, because they are measured, quantified and recent — where the aviation equivalents are largely inferential.
  4. Use the ECDIS case series as the cleanest cross-domain replication of the mode-confusion problem, and note that the investigator itself declined to make further recommendations and opened a systemic study instead.
  5. Frame the boundary conditions using Amalberti’s formulation — the toolkit transfers, the preconditions do not — and be specific about which preconditions are necessary.
  6. Resist the tempting narrative that other industries imported aviation’s answer and implemented it badly. The evidence better supports a different narrative: they imported an intervention whose effectiveness aviation had never established, and some of them then tested it properly.

7. Explicit gaps and unverified material

Not researched at all — do not cite anything on these from this memo:

  • Rail entirely: ETCS/ERTMS DMI, ATP and vigilance, RAIB/ERA investigations.
  • NTSB Tesla Autopilot investigations (Williston 2016; Mountain View 2018).
  • SAE Level 3 handover / takeover-time research.
  • Costa Concordia formal investigation and BRM/authority-gradient analyses.
  • IMO MASS regulatory scoping and the MASS Code.
  • ICAO Annex 19; EU Regulation 376/2014.
  • Kemeny Commission report text and its specific training recommendations.
  • INPO National Academy for Nuclear Training and training-programme accreditation.

Attempted and failed to verify:

  • NASA ASRS founding year (1976), limited-immunity provisions, de-identification process, and the rationale for NASA rather than FAA custody. The official ASRS overview page did not contain this. Retrieve the ASRS Program Briefing PDF.
  • MAIB Application and usability of ECDIS safety study — existence and URL seen, document not retrieved.
  • MAIB Ovit official report — blocked; report number 24/2014 is [UNVERIFIED].
  • Casner, Hutchins & Norman (2016) full text — citation verified, argument not. Open access at eScholarship; close this gap before using the paper for anything specific.
  • Catchpole (2013) BMJ Qual Saf — metadata verified, no abstract in Europe PMC, full text not retrieved. Its argument is [UNVERIFIED].
  • STCW Manila amendments (2010) — not verified.
  • A “Rogers, Lingard et al.” paper on the limits of the aviation analogy — not located. Cristancho (2024) and Clay-Williams & Colligan (2015) serve this role instead.
  • A “Bion & Rothen” paper on aviation and healthcare — not located. Bion appears in this memo via Matching Michigan, which is a different and stronger contribution. Verify whether the Bion & Rothen reference is correct before the report relies on it.
  • A systematic review of TeamSTEPPS against hard patient outcomes — not located; may not exist.
  • Alberdi et al. on CAD as a separate body of work — reached only via Povyakalo et al. (2013).

Verified but requiring a caution when quoted:

  • Haugen et al. (2015): the abstract reports the adjusted complication effect as “odds ratio 1.95 (95% CI 1.59–2.40)” in a context implying benefit. Check the direction of reporting against the full paper before quoting the OR.
  • Kapur et al. (2016): the “200,000 preventable medical deaths” figure is reported by them but is contested in the wider literature. Attribute, don’t assert.
  • Gaba (2000): page range 83–102 is [SECONDARY].
  • Rasmussen (1983): issue number (3) is [SECONDARY].
  • Clay-Williams & Colligan’s “4 of 13 checklist items” statistic: verified as their report of their reference 18; the primary study was not retrieved.