J. Korean Soc. Aviat. Aeronaut. 2020; 28(4):141-146

pISSN: 1225-9705, eISSN: 2466-1791

DOI: https://doi.org/10.12985/ksaa.2020.28.4.141

Technical Review

Study on EBT Implementation and Approval Process in Korea

Kyoung-Keun Han *
Author Information & Copyright ▼
*Professor, Department of Flight Operations, Hanseo University
Corresponding Author E-mail : kkhan@hanseo.ac.kr, Corresponding Author address : 236-49 Gomseom-ro, Nam-myeon, Taean-gun, Chungnam-do, Hanseo University, Republic of Korea

© Copyright 2020 The Korean Society for Aviation and Aeronautics. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Nov 30, 2020; Revised: Dec 09, 2020; Accepted: Dec 14, 2020

Published Online: Dec 31, 2020

ABSTRACT

The International Civil Aviation Organization (ICAO) developed Doc 9995 “Manual of Evidence-based Training,” with the intention of providing guidance by establishing a new methodology for the development and conduct of a recurrent training and assessment program. The airline pilot training regulations were largely based on the evidence of hull losses from early generation jets, and in order to mitigate a risk, simply repeating an event in a training program was sufficient. At the time, studies concluded that it was time to change the paradigm of training and assessment program for pilots. One airline alone implemented Evidence-Based Training (EBT) program in their flight crew training and assessment program while another airline partially implemented the EBT program. In the regulatory framework of MOLIT, specific EBT regulations have not been established yet. Therefore, it is recommended to develop rules and standards that comply with ICAO SARPs as soon as possible. In this study we review the key steps in the implementation of the baseline EBT, approval process of baseline EBT program, and policy options regarding the implementation of EBT. It will provide guidance to operators, Approved Training Organizations (ATO), and stakeholders.

Keywords: EBT(증거기반평가 및 훈련); EBT Implementation(EBT 이행); EBT Approval(EBT 승인); EBT Option(EBT 선택)

I. INTRODUCTION

The International Civil Aviation Organization (ICAO), International Air Transport Association (IATA), International Federation of Air Line Pilots’ Associations (IFALPA) and other stakeholders in the aviation industry have developed a new evidence-based recurrent assessment and training program for flight crew.

The global standards and regulations for airline pilot training were originally derived in response to accidents involving early generation jet aircrafts. These aircrafts were usually maneuver-based and pilot training standards have remained almost unchanged since their adoption. During the same period progressive changes in aircraft design, including developments in automation, system integration, reliability and radical changes in the operating environment have improved operational safety, but also revealed new operational pitfalls.

EBT means training and assessment based on operational data that is characterized by the development and assessment of the overall capability of a trainee across a range of core competencies rather than by measuring the performance during individual events or maneuvers.

This study illustrates inadequacies in the airline flight training regimes and identifies areas in which major changes are necessary. It strongly supports the implementation of such changes in both the decision making and development of airline pilot assessment and training programs and provides guidance to construct and implement suitable EBT programs for the airlines and the government.

Voluntary implementation of EBT by the operator is preferred for regulating the recurrent training and checking of flight crew. It provides an opportunity for the air operator certificate (AOC) holders to implement EBT for recurrent training and assessment of the flight crew.

II. EBT REGULATORY STATUS

2.1 ICAO Guidance for EBT

ICAO developed Doc 9995 ‘Manual of Evidence based Training’, which was intended to guide Civil Aviation Authorities (CAA), Operators (i.e., AOC holders) and Approved Training Organizations (ATO) in the recurrent assessment and training of pilots by establishing a new methodology for the development and conduct of a recurrent training and assessment program titled Evidence-Based Training (EBT).

ICAO has established The Manual of Evidence-Based Training (Doc 9995) to guide CAA, operators, and ATO on the regular evaluation and training of pilots referred to in Chapter 9.3 and 9.4.4 (pilot proficiency assessment) of Annex 6, Part I.

There are few or no national EBT management regulations, and most of the regulations currently used refer to ICAO Doc 9995 and IATA implementation guidelines. Therefore, for the time being, the above two documents are being considered as the final references to EBT.

2.2 FAA

There are no EBT-related regulations or references to FAA regulatory systems. The Advanced Qualification Program (AQP) was first developed in the 1990s and is an Alternative Means of Compliance (AMOC) that can replace the existing regulatory requirements of 14CFR on Pilot Training and Evaluation. Under the AQP, the FAA has the authority to approve an operator even if it deviates significantly from existing regulations if it can assure equal or better levels of safety.

2.3 EASA

In September 2015, EASA started RMT1).0696 “Implementation of evidence-based training within the European regulatory framework” and created an EBT Task Force to develop interim guidance material (GM) in order to promote a standardized and consistent method for the implementation of EBT within the existing framework. RMT.0696 followed an accelerated process within the regulatory system to benefit from the safety measures of EBT. RMT.0696 was planned to gain experience in implementing EBT and thus identify certain difficulties and inconsistencies that will be addressed through RMT.0599.

Presently, EASA Member States (and other states that have elected to adhere to the European aviation IRs) do not have a regulatory framework that lends itself to the full implementation of EBT. In fact, within the current regulatory framework, it is only possible to achieve mixed EBT2) implementation.

2.4 Other asian states

JCAB (Japan Civil Aviation Bureau) is known to have introduced new Competency-Based Training and Assessment (CBTA) requirements that provide operators more flexibility in trying out alternative training and screening systems such as EBT. Japan Airlines (JAL) has introduced EBT through the application of the CBTA program.

CAAS (Civil Aviation Administration of Singapore) has yet to introduce new requirements related to the implementation of EBT and ATQP. However, some airlines have expressed interest in introducing EBT in their regular training in accordance with existing regulations and will work closely with CAAS in this process.

2.5 MOLIT, Korea

It is known that specific EBT regulations do not yet fully exist in the regulatory framework of MOLIT. However, as an ICAO contracting state, it is required to develop rules and standards to conduct ICAO Standards and Recommended Practices (SARPs) as soon as possible.

Airline ‘A’ introduced baseline level EBT in their assessment and training programs in 2015. Therefore, the EBT program of Airline ‘A’ belongs to an enhanced level now. Meanwhile, Airline ‘B’ has acquired the EBT program as an alternative means of assessment and training program. Most LCCs in S. Korea plan to introduce EBT as regulations in the framework of MOLIT.

III. KEY STEPS FOR THE IMPLEMENTATION OF THE BASELINE EBT PROGRAM

3.1 Key steps for the implementation of the baseline EBT3), 4)

According to ICAO Doc 9995, ‘Manual of Evidence Based Training’, key steps for the implementation of the baseline EBT program can be summarized as below. Success in the implementation of EBT relies on an effective partnership between the Operator/ATO and MOLIT.

  1. Definition of an implementation and operations plan

    1A. Implementation strategy, consideration of options: Training and assessment according to EBT principles

    1B. Mixed implementation

  2. Instructor training and standardization

  3. Review of training effectiveness upon receipt of sufficient training system data

  4. Development of a competency framework, standards, and a grading system

  5. Malfunction clustering

  6. Approach type clustering

  7. Selection and adaptation of the scenarios and type of operation for the operator

    7A. Program design

  8. Adaptation of training program according to the training system feedback

  9. Instructor training and standardization

  10. Instructor competency assessment

  11. Information to pilots

  12. Implementation (a proof-of-concept trial phase should be considered necessary by MOLIT)

  13. Review of training effectiveness upon receipt of sufficient training system data

  14. Measurement of training system performance

Steps explained above should be conducted by MOLIT, operators, and ATOs with agreement and cooperation.

3.2 Approval process of baseline EBT program5)

  1. Operator and ATO contact MOLIT to arrange a meeting to initiate the EBT project and the intended scope. Demonstrate to MOLIT that the EBT proposal:

    1. maintains or improves safety through improved risk analysis;

    2. maintains or improves safety through more effective crew training to mitigate identified risks;

    3. meets the public interest as intended in the applicable regulations and their associated standards;

    4. meets the objectives of EBT instructor training;

    5. meets the objectives for informing all pilots of the new EBT program, principles, and philosophy.

    MOLIT arranges a meeting with MOLIT staff. Ensure personnel is experienced in EBT methodology and philosophy.

  2. Operator and ATO formalize a document to support the application for approval to:

    1. specify how the proposed change will continue to serve the public interest;

    2. identify the end-state objectives of the proposal;

    3. quantify the improvement being sought in the level of safety, efficiencies, or outcomes;

    4. determine the current regulatory impediments to achieving those desired improvements;

    5. identify the overriding hazards of the intended proposal and conduct a thorough risk profile;

    6. define the risk controlling measures in the form of a risk management plan that must be validated during the proof-of-concept trial;

    7. establish data collection and analysis procedures for the proof-of-concept trial.

  3. Operator and ATO submit a formal application for EBT approval.

    MOLIT acknowledges receipt and replies formally to start the project.

  4. Operator and ATO define and develop the EBT implementation and operations plan and present it to MOLIT. This will also include implementation risk assessment. The risk assessment is a live document and may be amended throughout the project.

    Include milestones of those parts of the implementation plan and initial hazard identification. Allocate personnel.

    MOLIT acknowledges receipt and allocates responsible team. Review implementation and operations plan.

  5. Operator and ATO devise detailed proof-of-concept plan including risk management process for implementation.

    MOLIT agrees on the scope and limitations. Consider the training and logistical difficulties of only ‘fleet-wide’ versus ‘operator-wide’ trials.

    AOC/ATO audit focus on:

    1. selection and training of staff;

    2. training program development, validation, and review;

    3. development and maintenance of courseware;

    4. administrative staff duties in support of program, the instructors and students;

    5. delivery of training;

    6. record-keeping;

    7. assessment and examination processes;

    8. feedback.

  6. Operator and ATO develop a set of competencies, observable behaviors, and performance standards.

    MOLIT should review.

  7. Operator and ATO develop an assessment and grading system or if already in use, adapt it as required for EBT. Prepare draft OM-Part D amendments.

    MOLIT should review.

  8. Operator and ATO submit EBT instructor training program including all EBT-related amendments made at the time of submission.

    MOLIT reviews training documentation, and conducts audit of instructor training program.

  9. Operator and ATO submit the EBT program.

    MOLIT reviews training documentation and amendments to Operations Manual.

  10. Operator and ATO implement proof-of-concept trial in cooperation with MOLIT. Complete and deliver a compliance checklist.

    MOLIT observation by MOLIT EBT team. Prepare for total EBT program audit.

  11. MOLIT conducts an audit on the program delivery. Verify compliance against all applicable criteria. Issue initial approval. Set-up process to verify Operator/ATO continued compliance.

  12. Baseline EBT program implemented.

MOLIT applies a continued audit oversight process. Review training effectiveness when sufficient training system data is generated and analyze training system performance.

IV. POLICY OPTIONS REGARDING THE IMPLEMENTATION OF EBT

4.1 Initial list of policy options

The analysis of the most controversial issues is followed by defining the policy options regarding the implementation of EBT for recurrent training and checking. The following options have been identified6)7):

  • 0: No policy change

  • 1: Voluntary EBT

    • 1.1: Implement EBT within the current license revalidation process

    • 1.2: Implement EBT with revalidation of license restricted to the AOC holder

    • 1.3: Implement EBT while separating the administrative action of revalidation of license from the technical assessment/check of the pilots

  • 2: Mandatory EBT

4.2 Detailed description of policy options

0: No policy change

Continuation of legacy training or ATQP for the conduct of recurrent training and checking for the flight crew.

1: Voluntary EBT

It provides an opportunity for the AOC holders to take a decision to implement the EBT system for the recurrent training and checking of their pilots and to shift from following the existing ‘prescriptive’ training or ATQP to EBT.

2: Mandatory EBT

This option envisages mandatory EBT for all operators and discontinuation of the legacy training. Implementation of mandatory use of competency-based training in all flight crew training performed by an AOC holder. Furthermore, for aircrafts subject to EBT, its implementation would be mandatory by removing the current prescriptive rules, thus making EBT the only alternative to ATQP.

V. CONCLUSION

EBT concept can be explained by four key principles; competency-based training, learning from positive performance, and driven by evidence. Core competencies and evidence are two important aspects that distinguish EBT from the check-focused training in the past. Another important aspect of EBT is the notion of resilience.

There are some strong reasons for why EBT should be the way of the future and more operators and ATOs should inculcate EBT into their own training and assessment programs. The way we trained in the past is no longer appropriate. EBT will not remain stagnant and will have to be modified periodically.

Few EBT implementation cases were reported in Korea. In the regulatory framework of MOLIT, specific EBT regulations have not been established yet.

It is recommended to develop rules and standards to comply with ICAO SARPs as soon as possible.

In this study, key steps for the implementation of the baseline EBT, the approval process of baseline EBT program, and policy options regarding the implementation of EBT were reviewed. It will help guide operators, ATO, and stakeholders.

Notes

1) Rulemaking Task.

2) Mixed EBT implementation means that only some portion of the recurrent assessment and training is dedicated to the application of EBT.

3) ICAO, “Manual of Evidence-based Training” 1st Edition, 2013.

4) IATA, “EBT Implementation Guide”, 1st Edition, 2013.

5) IATA, “EBT Implementation Guide”, 1st Edition, 2013.

6) EBT Implementation Guide, 1st Edition IATA, July 2013.

7) EASA, “Notice of Proposed Amendment 2018-07(A)”, 2018.

References

1.

ICAO, “Manual of Evidence-based Training” 1st Edition, 2013.

2.

IATA, “EBT Implementation Guide”, 1st Edition, 2013.

3.

ICAO, Annex 1 ‘Personnel Licensing’, 11th Edition, 2011.

4.

ICAO, Annex 6 ‘Operation of Aircraft’, 10th Edition, 2016.

5.

ICAO, Doc 9841 AN/456 ‘Manual on the Approval of Training Organizations’ 2nd. Edition, 2012.

6.

IATA, “Data Report for Evidence-Based Training”, 1st Edition, 2014.

7.

EASA, “Notice of Proposed Amendment 2018-07(A)”, 2018.

Journal of the Korean Society for Aviation and Aeronautics
#216, Science Building, 76, Hanggongdaehak-ro, Deogyang-gu, Goyang-si, Gyeonggi-do 10540, Korea
Tel : +82-2-300-0056 | Fax : +82-2-3158-2388 | ksaa_journal@ksaaonline.or.kr

Copyright © The Korean Society for Aviation and Aeronautics. 2024. Powered by Guhmok