Journal of the Korean Society for Aviation and Aeronautics
The Korean Society for Aviation and Aeronautics
Original Article

Developing Airport Safety Performance Indicators and Index:

In Kie Na*, Yu-Jin Choi**
*Senior Manager (Business Continuity Plan), Incheon International Airport Corporation
**Assistant Researcher, Incheon International Airport Corporation
Corresponding Author E-mail : cyj@airport.kr, Corresponding Author Address : 444, Je2terminaldaero, Jung-gu, Incheon, Korea

© Copyright 2023 The Korean Society for Aviation and Aeronautics. This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: Aug 08, 2023; Revised: Aug 18, 2023; Accepted: Sep 11, 2023

Published Online: Sep 30, 2023

ABSTRACT

An indicator system is an effective way to monitor ongoing safety status. Current aviation safety measurements account for many qualitative technical and lagging indicators. Conversely, quantitative and leading indicators have only a tiny proportion. This research added more quantitative leading indicators and reviewed them to harmonize lagging and leading indicators to measure airport safety and provide an index. The South Korean national gate, Incheon International Airport’s indicators, were applied as primary data to verify this research practically. Then, examples from International and national authorities were reviewed and extracted for use. Fifty-five safety specialists participated in the focus group discussion and three rounds of the Delphi survey. Finally, 51 sub-indicators were newly chosen. After this process, weights for each indicator could be assigned using the AHP (Analytical Hierarchy Process) to provide an integrated index. The result of the simulation with newly added indicators in the past five years (2020-2022) reliable trend showed in indicators and integrated index. Moreover, this allows monitoring the status of the details of indicators and holistic insight. This study considered that it is more suitable for a single company or service provider to use it according to the exact situation than in a macro- and general-purpose at the country or regional level.

Keywords: Airport Safety Performance(공항 안전성과); Index(지수); Leading Indicator(선행 지표); Lagging Indicator(후행 지표)

I. INTRODUCTION

Similar to other industries, aviation has tried to eliminate operational risk through the application of technology and academic achievements. As a result, the aviation industry has achieved a good record and is regarded as being safe, with a low number of accidents (Liou et al., 2007). As a result, many states and air service providers are attempting to develop safety management systems (SMS). SMS assurance uses safety performance indicators (SPIs) to process past data in order to ensure better results over the next period (next year, next phase). As a result of this concept, sustainability can be achieved from disasters, which is widely used in other industries. A crucial component of effective risk management is the detection of events at a site, such as accidents and near misses (Knegtering and Pasman, 2013). These events are organized into indicators, which managers can use to improve safety. Investment decisions can be made based on it. In many cases, however, more than such results-oriented data is required to fully explain potential risks. According to Liou et al. (2007), the aggregate accident rate can be used as a safety index, but it is necessary to complete the analysis. It is necessary to improve this aspect of the current airport safety performance monitoring system. Psychological indicators can therefore be used to provide a solution to this problem. In this study, we aim to improve the existing airport safety indicator system based on the results of an investigation of airport workers’ safety psychology.

II. AIRPORT SAFETY INDICATOR SYSTEM

2.1 International Aviation Organization

The International Civil Aviation Organization (ICAO) established ANNEX19. According to these documents, States and service providers including airports, airlines, etc. are required to comply with Standards And Recommended Practices (SARPS) (ICAO, 2016). This document emphasizes the importance of protecting safety data and information. Those are the fundamentals for the development of indicators and indices. Furthermore, it recommended that the states select quantifiable indicators that take into account their relevance to the service providers, including airports and airlines’ safety objectives. Member countries and service providers refer to these documents as the most influential guidelines. It is also presented in the DOC9859 (Safety Management Manual) examples of indicators that help recognize warnings from fields by setting triggers based on standard deviations (ICAO, 2018).

There has been a guide on Airport Performance Measures (APM) published by ACI (Airport Council International) since 2012 and this guide is available to airports worldwide. The guidebook provides six key performance areas, including core competency, safety, security, and service quality. There are several safety-related indicators, including runway accidents, excursions and incursions, collisions with birds, injuries, and lost work. Based on the type of airport, each indicator is classified according to its applicability (ACI, 2012). The indicator was defined with related parameters and listed the links between the driver and the indicator.

There is also an indicator system that is used by the National Union. The European Aviation Safety Agency (EASA) is composed of members from 31 countries throughout Europe, and it is responsible for certification, standardization, investigation, and monitoring. The European Plan for Aviation Safety (EPAS) includes a safety performance indicator. The major safety indicators include runway excursion, airborne collisions, controlled flight into terrain (CFIT), loss of control during flight, runway incursion, and fire (EASA, 2022).

2.2 Civil Aviation Authorities in States

The development of indicators was not only done by the international organizations described above but also according to the circumstances within the country. Airport Cooperative Research Program (ACRP), an institute specializing in airports supported by the US FAA, presented indicators for the airport sector, such as airside accidents, minor accidents, work-related casualties, runway invasions, and wild animal accidents in the Safety Management Systems guidebook for airports. Another report published, ‘Resource Guide to Airport Performance,’ included 29 key indicators, 132 key indicators, and 679 other indicators. Safety-related indicators include two core, ten major, and 31 other indicators (FAA ACRP, 2011).

The Belgian CAA has, like several other countries, established an aviation safety program in accordance with ICAO DOC9859 and EASA’s safety policy. According to the European Coordination Center for Accident and Incident Reporting Systems (BCAA, 2018), 36 detailed safety performance indicators were developed for accidents and incidents, aerodromes, navigation services, operations, technology, and other prominent fields.

ICAO GASP, Doc 9859, and EU EPAS have been implemented by the Finnish Transport and Communications Agency (TRAFICOM) through its Finnish National Aviation Safety Program (FASP). As stated in TRAFICOM (2018), the EU-level safety objectives must be considered in conjunction with the specific setting of acceptable levels of safety and scope at the national level. Airports, airlines, and ground handlers are provided with safety performance indicators through the FASP program. The safety performance indicators for airport operators are further subdivided into SMS performance, runway incursion/excursion, runway status defects, visibility, air traffic control flight, accidents, a lack of information about obstacles, a ground collision, foreign matter in moving areas, CFIT, as well as neglect of apron area supervision.

In 2008, the Singapore Civil Aviation Authority (CAAS) developed safety indicators for airport operators and airlines as part of the State safety program. Indicators related to airports include aircraft-related accidents, incidents, wildlife collisions, runway intrusions/deviations, and ground collision prevention system recommendations (CAAS, 2018).

In order to measure safety performance, South Korea set up a national aviation safety program through its civil aviation authority, MOLIT (Ministry of Land, Infrastructure and Transport). It is also through this CAA that the state’s aviation safety program was established. Under the theme of air service, each area is divided into sub-parts, such as airport facilities, navigation facilities, air traffic control and air operations, and its indicators are provided. As provided in ICAO Doc9859, the 3-year standard deviation was used as the alert method. There are several indicators related to the airport sector, including ground conflict/accidents, aircraft ground collisions, airport aerodrome facility malfunctions, and navigation safety facility failures. Incheon International Airport Corporation, the primary gate in South Korea, has established 25 safety indicators in support of national indicators since 2013 (IIAC, 2016; Table 1).

Table 1. Incheon Airport safety indicator list
Accidents Aircraft accidents
Safety hazards (Cont.) Incidents Incursions ·Aircraft
·Vehicle
·Personnel
Excursions
Aircraft incidents (excluding incursion and excursion)
Collision between ·Aircraft-Aircraft
·Aircraft-Vehicle
·Aircraft-Facility
·Aircraft-Personnel
·Vehicle-Vehicle
·Vehicle-Personnel
·Vehicle-Facility
Occurrences Near collision between ·Aircraft-Aircraft
·Aircraft-Vehicle
·Aircraft oilleakage
·Dangerous goods
·Bird strikes
·Pilot complaint report
Accident prevention Airside inspection ·FOD
Safety culture ·Hazard report
Safety activity ·SMS committee
·SMS Sub-committee
·Safety campaign
Safety simulation ·Gate and stands parking planning
Download Excel Table
2.3 Characteristics of Airport Safety Indicators

ICAO presented representative examples of these indicator systems, such as setting triggers for indicators related to the aviation industry of the state, but these examples were merely indicative. The reason may be the fact that international organizations are responsible for covering the entire aviation industry associated with all states. It is unnecessary to cover the details of service providers such as airports.

In contrast, the ACI did not present any criteria from its guidebook but rather included a number of customized indicators. The ACI and FAA ACRP’s guides present indicators based on their importance as opposed to ICAO’s list of indicators regardless of their relative weight. In addition to their differences, they share the following similarities:

?Various organizations and states have developed airport safety indicators that deal with the same or similar issues. This is due to the similarity between the airport infrastructure’s functional, topographical, and operational characteristics. When safety performance is measured using the same indicators, this provides a positive perspective on comparing safety performance.

?Risk levels, which represent severity and likelihood of occurrence, are commonly used in airport safety indicators. It helps to determine the type and timing of resource allocation according to the risk level. It should be noted, however, that the level of risk is only sometimes constant and reliable due to the ease with which this assessment can be altered. It is important to note that although the severity of a potential occurrence is relatively fixed, the likelihood can vary depending on the location, the time (day/night, weekday/holiday), the season, and the weather. Therefore, it is important to note that variability in likelihood can also distort risk levels and prevent appropriate decision-making.

?Most of the indicators related to airport safety consist of lagging indicators rather than leading indicators. Lagging and Leading indicators can be classified as measured symptoms in advance or events on past results (Lingard et al., 2017). The lagging indicator is outcome-oriented. On the other hand, the leading indicators are process-oriented (ICAO, 2018). Hallowell et al. (2020) studied leading indicators, including pre-task safety meetings, which are able to suppress accidents and have an inverse correlation with injury rates. Given the current indicators system in the airport industry, it cannot be considered the proactive way (Oster Jr et al., 2013). Therefore, leading indicators were extended in this study.

?The overall direction of indicators cannot be measured through the current system due to the mutually independent characteristics of each indicator. Although this fact is generated naturally, it may not be persuasive information for top management because it is difficult to make a comprehensive decision from several individual indicators. The relative weight among indicators is essential in prioritizing decision-making for limited resources. Moreover, there are only a few previous studies considering integrated perspectives.

2.4 Consideration of Psychological Indicator

Given the commonalities described above, this study must include a more innovative and integrated indicator system for sustainable safety management. In order to accomplish this, the study was intended to develop airport workers’ psychology as a component of the indicator. Safety accidents often occur as a result of workers’ minds manifesting in behaviors such as violations and negligence (Reason, 1990). This study attempts to confirm the effects of safety knowledge, attitude, consciousness, climate, and behavioral factors on safety-related behavior and to integrate them with existing indicators.

Safety attitudes have been studied early as an essential predictor of climate (Zohar, 1980). It is also defined as individuals’ belief and confidence about the environment and technology that enable them to continue working safely in their daily work (Wood and Bandura, 1989). Ajzen (2005) divided safety attitudes into favorable and unfavorable ones, saying that favorable attitudes are future-oriented responsibility and constructive beliefs, while unfavorable attitudes are destructive beliefs and characteristics they want to avoid. Fugas (2012) also differentiated between constructive and destructive beliefs regarding safety and argued that positive beliefs should be strengthened while negative beliefs should be reduced. According to Brown et al. (2000), the negative factor affecting worker safety behavior is a cavalier attitude, while the positive factor is safety efficacy. In this study, safety attitudes were divided into two categories: safety efficacy and cavalier attitude.

Safety knowledge shapes safety behavior by ensuring that workers are aware of safety operating procedures and have received appropriate training and instructions (Hofmann et al., 1995). According to Piers et al. (2009), safety knowledge involves understanding the risks involved in the operation and making sure it is distributed and utilized properly. Safety knowledge is divided into explicit and tacit knowledge. Explicit knowledge refers to knowledge that is easy for individuals to convey to organizations because it is official and can be documented. On the other hand, tacit knowledge is generally accumulated by individuals from experience and has characteristics that are variable and difficult to share depending on the situation (Nonaka and Takeuchi, 2007). Safety consciousness refers to “an individual’s awareness of safety issues” (Barling et al., 2002) and operates at both a cognitive and behavioral level (Koster et al., 2011). It is also a factor that significantly impacts safety reporting practices and the incidence of accidents (Barling et al., 2002; Kelloway et al., 2006).

The concept of safety climate is more collective than the concept of safety consciousness. Generally, it refers to the belief that safety should be at the center of organizational policies, procedures, and goals (Griffin and Neal, 2000; Hofmann and Stetzer, 1996; Neal and Griffin, 2002). It is important to note that safety behavior consists of actions taken to protect from danger based on safety knowledge and skills that are necessary for the performance of one’s job (Gressgard, 2014). Several previous studies have identified two types of safety behavior: compliance and participation.

jksaa-31-3-103-g1
Fig. 1. Overview of the research process
Download Original Figure

Hon et al. (2014) classified safety behavior into safety compliance and active participation in obligations. Jiang and Probst (2015) also said that safety compliance is a behavior required for safety (e.g., safety rules to wear protective gear before work). Neal and Griffin (2000) stated that it includes activities and training to actively participate in safety issues in advance and speak to managers. Borman and Motowidlo (1993) classified safety behavior into safety conformity behavior, a crucial activity to maintain safety, and safety expectation behavior that helps create the surrounding environment but does not affect safety directly.

III. CONCEPTUAL FRAMEWORK AND METHODOLOGY

The study scope was defined in the Incheon International Airport movement area. When recent statistical data were available, the range was 2020-2022. The research order chart and descriptions are:

As part of the first stage of the study, indicators from international organizations and states were introduced, and previous studies were reviewed and gathered. In the second stage, a Delphi survey was conducted on the collected candidate indicators. In order to increase reliability, more relevant or essential indicators should be selected. An Analytic Hierarchical Process (AHP) was used to calculate the weight reflecting relative importance among the indicators in the third stage. A final stage in the process involved calculating and comparing the indicators and index, i.e. the sum of the weights of each indicator over the past six years (2020-2022).

3.1 Previous Review

As part of ISO22301 (2019), an international standard for business continuity planning, safety indicators, and targets should be considered in order to meet the legal requirements of both internal and external stakeholders as well as the government. Indicators of this type apply to all other areas, including this study. Table 2 shows reviewed indicators from international organizations, national guidelines, and previous studies. Academic journals and reports find psychological indicators, including safety awareness and attitude, etc. 25 indicators from Incheon International Airport were also reviewed for candidates.

Table 2. Airport safety indicator pool
Organization # of relevant indicators Organization # of relevant indicators
ICAO, Doc 9859 5 FAA, ACRP Safety performance 22
ICAO, GASP 7 FAA, ACRP Airport performance 31
ACI 5 BCAA Safety Programme 22
EASA, ASCOS 6 Finland, FASP 9
Singapore, CAAS 7 Korea Aviation, Safety Program 10
Korea, Civil Aviation Act 18 Incheon airport SPI 25
literature (Zohar, 1980; Brown et al., 2000; Hofmann et al., 1995; Barling et al., 2002; Hon et al., 2014; Griffin and Neal, 2000) 10
Download Excel Table
3.2 Methodology

Based on the above statements, purposive sampling was used to select experienced experts from the airport, airlines, and ground handlers to participate in the survey. The following are details of each survey round. After repeated surveys of experts, the statistical average of the surveys is converged into a group opinion (Rowe and Wright, 2001), and the results are finally summarized after confirming consensus (Baker, Lovell, and Harris, 2006).

3.2.1 Delphi data processing

The Delphi method utilizes an iterative process to develop consensus among a diverse group of experts (Keeney et al., 2001). The importance of consensus among airport safety experts cannot be overstated since airport safety assessment is a field that has not been adequately addressed in existing literature (Yi et al., 2015). According to Keeney et al. (2001), consensus is defined as being reached among participants in a Delphi study when 75% or more of the respondents agree. When participants agree with the statement (i.e., 4 or higher on a 5-point Likert scale), consensus has been reached.

The Content Validity Ratio (CVR) index is a method of calculating internal validity to examine the appropriateness of item content, based on the percentage of Delphi panelists who rated the appropriateness of an item as significant (4 or higher) on a 5-point scale. Lawshe (1975) proposed the following formula for CVR calculation.

C V R = N ε N 2 N 2

CVR: Content validity ratio

N: Number of total panels

Nε: Number of panels that responded “Important” (Likert scale = 4 or 5)

To determine the degree of consensus in the expert panel’s opinion, we applied the method of Delbecq et al. (1975). The measure should be less than 0.5.

C G D = Q 3 Q 1 2

CGD: Convergence degree

Q3: first quartile, 25% of the total

Q1: third quartile, 75% of the total

3.2.2 Selection of the expert panel (BY Semi-structured interviews)

The first round consisted of experts selected from the same categories as those designated for the Delphi study. The Delphi research panel critically reviewed the findings to gain a deeper understanding of the consensus reached among the researchers and to create a framework for future research efforts in the field (Nicolaidou, et al., 2021).

The criteria for the expert panel were established by applying the Delphi experience criteria of Tran, Trung et al. (2020) in order to select experts who had at least five years of experience as airport safety instructors. The first round of the selection process was conducted on airport experts with an average of six years of experience. An open-ended question type was used to eliminate candidates who were not relevant. As a result of this process, indicators that were not relevant or overlapping were identified and removed.

The second round was conducted with 18 experts from the airport, airlines, and ground handlers with an average working experience of 17 years. A 5-level Likert questionnaire was devised to select from three perspectives: necessity, importance, and urgency. After evaluating the importance of each metric, we set up a comment field for experts to comment on modifying, deleting, or merging metrics.

During the third (last) round, 27 experts from airports, airlines, and ground operators were interviewed, with an average of 11 years of experience. When the survey was conducted, we were experiencing the height of the COVID-19 virus pandemic. Some of our experts became infected with the virus, which resulted in a low return rate. Several experts in airport safety were consulted to reach conclusions about the importance of airport safety indicators.

The 25 indicators were assessed using a questionnaire based on a Likert 5-level scale in order to determine their suitability and expert confidence. The final ten indicators could be selected after calculating the mean and standard deviation of relevance and expert confidence values (3.89 and 4.01, respectively). The result was the confirmation of the existing 7 indicators, as well as 18 new indicators (Fig. 2, Table 3-5).

jksaa-31-3-103-g2
Fig. 2. Flowchart of the Delphi process for selecting indicators
Download Original Figure
Table 3. Results of the 1st round Delphi study
No Item % of Consensus C V R C G D
1 Aircraft accidents 96.7% 0.89 0.00
2 Aircraft airspace intrusion 88.9% 0.89 0.50
3 Vehicle CAA invasion 86.7% 0.89 0.50
4 Aircraft abandonment 88.9% 1.00 0.50
5 Aircraft near miss 86.7% 0.78 0.50
6 Aircraft-aircraft collision 94.4% 0.89 0.00
7 Aircraft-vehicle collision 91.1% 0.89 0.50
8 Aircraft-facility collision 86.7% 0.67 0.50
9 Aircraft-person collision 81.1% 0.44 0.50
10 Vehicle-vehicle collision 73.3% 0.44 0.50
11 Vehicle-person collision 77.8% 0.44 0.00
12 Vehicle-facility collision 68.9% 0.44 0.50
13 Aircraft-aircraft close encounters 84.4% 0.44 0.50
14 Aircraft-vehicle close encounter 82.2% 0.44 0.50
15 Aircraft oil leak 66.7% 0.44 0.50
16 Dangerous goods incident 77.8% 0.22 0.50
17 Bird strike 78.9% 0.44 0.50
18 Reporting mooring control dissatisfaction 65.6% 0.22 0.50
19 FOD collection 75.6% 0.44 0.50
20 Hazardous find reporting 78.9% 0.11 0.50
21 SMS committee 74.4% 0.44 0.50
22 SMS working committee 75.6% 0.44 0.50
23 Safety campaign 63.3% 0.44 0.50
24 Airport operation certification 72.2% 0.22 1.00
25 Navigation safety oversight 71.1% 0.00 1.00
Download Excel Table
Table 4. Results of the 2nd round Delphi study
No Item % of Consensus C V R C G D
1 Injuries in transit areas 70.0% 0.110 0.5
2 Number of wake/propeller maintenance/engine aspiration incidents 76.7% 0.440 0.38
3 Wildlife collisions 65.6% 0.220 0.38
4 Wildlife sightings (reports) 95.6% 0.670 0.5
5 Number of safety reports collected 71.1% 0.110 0.5
6 Number of safety reporters 64.4% 0.000 0.5
7 Number of safety training programs 65.6% 0.220 0.38
8 Number of safety audits 63.3% 0.330 0
9 Number of measures implemented according to safety audit points (rate) 68.9% 0.110 0.5
10 Response rate after receiving risk reports 75.6% 0.330 0.5
11 Number of construction works in mobile areas 56.7% 0.560 0.5
12 Number of accidents caused by construction work in mobile areas 65.6% 0.110 0.5
13 Ratio of accidents to the number of construction activities 62.2% 0.330 0.00
14 Number of unauthorized works in airside areas 81.1% 0.440 0.00
15 Employee accidents and injuries 70.0% 0.220 0.50
16 Lost work time 55.6% 0.780 0.00
17 Rescue and firefighting 71.1% 0.220 0.50
18 Number of fatalities in mobile areas 86.7% 0.670 0.38
19 Number of driving violations in the movement zone 72.2% 0.110 0.50
20 Aircraft damage during the aircraft cycle 88.9% 0.780 0.50
21 Aircraft damage while in transit 90.0% 1.000 0.50
22 Equipment damage 65.6% 0.110 0.50
23 Workers’ compensation claims 75.2% 0.440 0.00
24 Airport medical emergency response 70.0% 0.000 0.50
25 Number of environmental violations 64.4% 0.110 0.88
26 Emission limits exceeded 77.8% 0.560 0.38
27 Number of crimes at the airport (violence, theft) 60.0% 0.440 0.00
28 Number of safety seminar participants 60.0% 0.670 0.00
29 Instrumental aircraft damage 81.1% 0.780 0.00
30 Number of injuries/accidents to passengers or other organizations 65.6% 0.000 0.50
31 Fire/smoke incidence rate 68.9% 0.000 0.50
32 Fires in transit areas 92.2% 0.440 0.38
33 Industrial Accident Management 74.4% 0.000 0.50
34 Disaster losses (manpower) 73.3% 0.000 0.50
35 User accidents (passengers, etc.) 64.4% 0.220 0.38
36 Safety complaints 86.7% 0.670 0.38
37 Disaster safety factors 67.8% 0.000 0.50
38 Internal safety culture 88.9% 0.440 0.38
39 External safety culture index 84.4% 0.440 0.38
40 Aviation safety education (SMS) elements 70.0% 0.110 0.50
41 Safety and health education factor 62.2% 0.110 0.50
42 Working days lost due to occupational illness or injury per airport employee 77.8% 0.440 0.50
43 Types of FODs 75.6% 0.440 0.38
44 Number of violations 71.1% 0.000 0.50
45 Failures/damages found 64.4% 0.110 0.50
46 Injuries per workforce size 60.0% 0.560 0.38
47 Safety awareness 83.3% 0.440 0.38
48 Safety attitude 84.4% 0.440 0.38
49 Safety knowledge 82.2% 0.440 0.38
50 Number of flights 68.9% 0.220 0.50
51 Equipment aging 77.4% 0.440 0.00
52 Number of vehicles/equipment 75.3% 0.440 0.38
53 Noise 62.2% 0.330 0.75
Download Excel Table
Table 5. Results of the 3rd round Delphi study
No Item % of Consensus C V R C G D
1 Safety awareness 85.4% 0.769 0.50
2 Number of fatalities in airside areas 80.0% 0.670 0.38
3 Number of safety seminar participants 80.8% 0.667 0.00
4 Aircraft damage while in transit 76.3% 0.615 0.50
5 Number of vehicles/equipment 76.2% 0.560 0.50
6 Aircraft damage during the aircraft cycle 75.4% 0.538 0.50
7 Customer safety complaints 85.4% 0.462 0.75
8 Fires in airsideareas 86.9% 0.462 0.88
9 Internal safety culture 76.2% 0.440 0.38
10 External safety culture index 76.5% 0.440 0.38
11 Number of safety reports collected 81.5% 0.385 0.50
12 Types of FOD 80.8% 0.385 0.50
13 Instrumental aircraft damage 75.6% 0.385 1.00
14 Safety knowledge 75.5% 0.385 1.00
15 Safety behavior 76.2% 0.385 1.00
16 Injuries per workforce size 85.4% 0.385 0.38
17 Number of wake/propeller maintenance/engine aspiration incidents 80.0% 0.385 0.38
18 Equipment aging 80.8% 0.385 0.00
19 Working days lost due to occupational illness or injury per airport employee 73.2% 0.110 0.88
20 Lost work time 71.1% 0.110 0.88
21 Number of construction works in airside areas 70.1% 0.110 0.88
22 Emission limits exceeded 65.4% 0.110 0.88
23 Number of unauthorized works in airside areas 60.4% 0.110 0.88
24 Workers’ compensation claims 55.4% 0.110 0.88
25 Number of crimes at the airport (violence, theft) 52.5% 0.110 0.88
Download Excel Table
3.3 Indicator Selection (Establishing Alert Levels)

Lastly, 51 indicators were reclassified and reorganized into four new categories (human factors, accidents/events, air traffic control/airside safety, and air navigation) by adding seven indicators related to air navigation facilities provided by the South Korean State Aviation Safety Program. The following is a classification of indicators from CAT I to CAT IV (Fig. 3).

jksaa-31-3-103-g3
Fig. 3. Overview of the research process
Download Original Figure

ICAO introduced the approach of applying a standard deviation to the average value of the past three years in selecting alert levels. This study also applied the alert level to set trigger systems. Moreover, the risk level was divided into four levels, and each risk level was expressed as good/moderate/caution/danger, respectively.

3.4 Index Creation

There is a need to monitor these indicators, however, the importance of each indicator cannot be rated equally. However, it is difficult to find previous studies that have examined the relative importance of the indicators and integrated them into a composite index. Consequently, an index that sets and integrates the weights of the indicators has been developed. There are two benefits to this approach.

Firstly, an index calculation provides an intuitive understanding of the trend resulting from each indicator incorporated. Using this method, the overall performance of the organization is identified and decisions can be made more effectively.

Secondly, it is possible to compare results over time and with other entities using the same indicators. Comparisons can easily lead to an overall assessment and the setting of short-term (e.g., annual) safety goals.

As a result, resources are used more efficiently in operations. An analytical hierarchy process (AHP) was applied in order to accomplish this. According to Soltanifar and Lofti (2011), AHP uses a weighted voting ranking approach in order to rank criteria and sub-criteria. To perform AHP, it was necessary to classify the 51 indicators for clustering. The safety function is responsible for completing this task in its own area. Category I (CAT I) was classified according to airside safety, air traffic control, and air navigation. It is the same as dividing areas in airport organizations. It is important to note that Category II consists of seven indicators covering areas that constitute Category I - human factors, accidents/events, facilities/ equipment, operational environment, air navigation functions, and communication. A total of 29 indicators are included in the CAT III group, and these indicators can be used to explain CAT II in greater detail. CAT IV follows CAT III and is further separated in the same way as above. As a result of the Delphi survey, 51 final sub-indicators were identified for each area of the CAT I. Some indicators, including the air navigation area, did not have CAT IV indicators in this process. The reason for this is that it has already been sufficiently subdivided and explained in detail.

The survey involved 16 experts from airlines, ground handlers, and the airport in order to determine weights based on AHP. A total of ten answer sheets were evaluated, 16 were used, and six with a low consistency ratio (CR) were removed from the analysis. Each weight of CAT IV indicators was calculated as CAT I weight × CAT II weight × CAT III weight × CAT IV average value (%). The final weighted score of each indicator was calculated as the CAT IV indicator weight × 100. There was no significant effect of some decimal points being corrected in the CAT IV weight results.

3.5 Simulation of Airport Indicators and Index (2020–2022)

Indicators with weights were simulated using data from 2020 to 2022 at Incheon International Airport. There are four warning levels based on the indicator’s 3-year average within the standard deviation range. The details are as follows:

·Green: Good, when the measurement year score is within 1st of the 3-year average

·Yellow: Moderate, when the measurement year score falls within 1sd ~ 2sd of the 3-year average.

·Orange: Caution, when the score in the measurement year falls within 2sd ~ 3sd of the 3-year average.

·Red: Danger, when the measurement year score exceeds 3sd of the 3-year average. Table 3 below shows the indicator, alert level, and index results from 2020 to 2022 (Table 6).

Table 6. Airport indicators, alert level, and index result
CAT I (Weight) CAT II (Weight) CAT III (Weight) CAT IV (Weight) Leading/Lagging 2020 2021 2022
ATC/Airside Safety (0.617) Human Factors (0.429) Safety Knowledge (0.136) Regulation Knowledge (0.024) Leading 85 85 84
Field (0.012) Leading 81 80 80
Safety Attitude (0.191) Safety efficacy (0.032) Leading 89 90 86
Cavalier attitude (0.018) Leading 70.9 74.81 73
Safety Behavior (0.280) Safety campaign (0.01) Leading 100% 100% 100%
Manager participation (0.016) Leading 100% 100% 94%
Worker participation (0.013) Leading 67% 92% 75%
Regulation violation (0.019) Lagging 166 210 175
Safety Report (0.016) Leading 194 491 593
Safety Culture (0.393) Individual Consciousness (0.052) Leading 81 82 83
Organizational Climate(0.052) Leading 81 83 83
Accidents/Events (0.291) Accident (0.276) Human Accident(0.02) Lagging 0 0 0
Aircraft Accident(0.029) Lagging 0 0 0
Runway Incursion (0.096) By other aircraft(0.012) Lagging 0 0 0
By vehicle (0.006) Lagging 0 0 0
Runway Excursion (0.086) By aircraft (0.015) Lagging 0 0 0
Incident (0.133) Human incident(0.024) Lagging 0 0 0
Collision (0.211) Aircraft-Aircraft (0.013) Lagging 0 0 0
Aircraft-Vehicle (0.008) Lagging 0 2 4
Aircraft-Facility (0.005) Lagging 0 0 0
Aircraft-Human (0.004) Lagging 0 0 0
Vehicle-Vehicle (0.002) Lagging 8 3 2
Vehicle-Human (0.003) Lagging 1 2 2
Vehicle-Facility (0.002) Lagging 0 0 2
Near-Miss (0.073) Aircraft-Aircraft (0.009) Lagging 0 0 0
Aircraft-Vehicle (0.004) Lagging 12 46 67
Aircraft fuel leakage (0.025) Aircraft fuel leakage(0.004) Lagging 7 19 30
Bird strike (0.034) Bird strike(0.006) Lagging 6 10 20
FOD (0.02) Maneuvering Area(0.001) Lagging 29 70 80
Ramp Area(0.001) Lagging 18 33 44
etc.(0.001) Lagging 16 20 37
Fire(0.047) Fire(0.008) Lagging 2 3 2
ATC/Airside Safety (0.617) Facilities/Equipment(0.193) Maneuvering Area Surface condition (0.355) Infrastructure Damage(0.021) Lagging 4 5 9
Airport Light Damage(0.021) Lagging 1 9 7
Ramp Area(0.394) Pavement Damage(0.023) Lagging 7 4 7
Airport Light Damage(0.024) Lagging 2 3 5
Movement Area vehicle(0.107) Aging(0.007) Lagging 410 370 335
Damage(0.006) Lagging 0 0 0
Movement Area Equipment (0.144) Aging(0.008) Lagging 3,156 3,051 3,480
Damage(0.009) Lagging 0 0 0
Operational Environment (0.088) Number of Vehicle/Equip-ment(0.151) Number of Vehicle/Equipment (0.008) Lagging 194 787 516
Adverse weather (0.381) Adverse weather(0.0207) Lagging 19 31 37
Bird(0.289) Bird(0.016) Lagging 141,179 143,063 140,024
Wildlife animal(0.179) Wildlife animals other than birds (0.009) Lagging 16 21 12
Air Navigation (0.383) Air Navigation safety Facilities (0.345) Instrument Landing System(0.066) Lagging 0 0.82 0
VOR/DMEVHF Omnidirectional Radio range/Distance Measuring Equipment(0.066) Lagging 0 0 0
Aviation Information & Communication(0.498) Air Navigation Radio Communications system A(0.095) Lagging 0 0 0
Automatic Terminal Information Service A(0.095) Lagging 0 0 0
Communication Factors(0.157) Aeronautical Radio Communications system B(0.02) Lagging 0 0 0
Aeronautical Fixed Telecommunication System(0.02) Lagging 0 0 0
Automatic Terminal Information Service B(0.02) Lagging 0 0 0
TOTAL 93.4 92.6 92.1
Download Excel Table

IV. RESULTS

During the past three years, the index has repeatedly risen and fallen. Every year, the index shows moderate performance (93.4 in 2020, 92.6 in 2021, and 92.1 in 2022). Below is the summary of 2022 found thanks to this study’s intentions and advantages.

· Human Factor Indicators (CAT-II).

Human indicators are the result of a survey of airside drivers. Almost all sub-indicators, including safety knowledge/attitude/consciousness, maintain stability, so the cultural aspect could be considered healthy. However, the score for participation in safety committees decreased (moderate) compared to the past. The reason was that face-to-face meetings were not possible due to COVID-19 and even online meetings were possible; there were cases where the evaluation score fell due to human resource management (leave, etc.) in the aviation industry.

· Accidents/Events (CAT-II)

The number of aircraft accidents, incidents, excursions, and incursions of this severity is very high, but the probability of occurrence is low. CAA and organizations use these indicators. In this study, these indicators have been very stable since the indicator index was established. There have been zero cases in five years at Incheon airport. Low-risk and high probability indicators, including ground safety accidents, near misses, fuel leakages, and bird strikes, occurred in 10 cases in 2022. They occurred frequently. Therefore, those indicators must be monitored and managed organizationally. Foreign Object Debris (FOD) collection has increased, so management needs to be followed in the future. With respect to FOD, it can be viewed as a leading indicator of the accident’s cause since it cannot be considered an accident’s result (but a possible cause as well). However, since FOD occurs due to worker neglect or abandonment, it can also be considered a lag indicator. This consideration can be applied to indicators that happened and were detected visually but have not yet caused any damage. This will depend on the entity’s decision to determine whether they are leading or lagging indicators.

· Facilities/Equipment (CAT-II)

Sub-indicators of this category help monitor damage or aging in the maneuvering/ramp/ movement area. This area can be controlled by prior inspection, maintenance? and repair. For example, in 2022, infrastructure (pavement surface) in the maneuvering area and airport lighting damage in the ramp area increased rapidly; these were marked as caution (orange) and danger (red), respectively, compared to the previous year. These findings improve those spots by strengthening prior inspections and expediting repairs.

· Operational Environment (CAT-II).

Operating environment refers to a set of indicators derived from background elements that do not directly affect safety. In contrast to Facilities/Equipment (CAT-II above), it is difficult to control through artificial means due to the fact that it is subject to changes in response to external factors, such as overall trends or natural forces. For example, the number of vehicles and equipment varies depending on the air traffic volume. The probability of accidents increases when the number of vehicles increases within the spatially limited airport airside. Adverse weather is also an indicator of natural disasters, and avoidance, such as suspension of operations, can serve as a safety strategy. Even if a curfew is in place, these category indicators are not related to safety operations. However, it must still be an indicator to be monitored at all times in a 24-hour airport. It is imperative to note that bird and wildlife influxes vary according to season. The number of wildlife species detected (excluding birds) surged in 2021 to danger (red) and returned to stability (green) in 2022.

· Air Navigation (CAT-I, including all CAT-II)

Air navigation (CAT-I, Air Navigation Safety Facilities, Aviation Information & Communication, Communication Factors in CAT-II) represents a very stable (green) indicator group with no reported cases, except for the instrument landing system in 2021. From this, it was found that Incheon Airport air navigation is operated safely with an indicator group of accidents/events. Table 7 summarizes the above conclusions.

Table 7. Summary of airport indicators alert level in 2022
Alert level Portion (%) Indicators (Number)
GOOD 41(80.4%) ° Human factors (10)
Regulation Knowledge, Field Knowledge, Safety efficacy, Cavalier attitude, Safety campaign, Worker participation, Regulation violation, Safety Report, Individual Consciousness, Organizational Consciousness
° Events (20)
Human Accident, Aircraft Accident, Runway Incursion by (other aircraft, vehicle),
Runway Excursion, Human incident, Collision (Aircraft-Aircraft, Aircraft-Facility, Aircraft-Human, Vehicle-Vehicle, Vehicle-Human, Vehicle-Facility, Near-Miss (Aircraft-Aircraft), Fire, Maneuvering Area Light Damage, Ramp Area Pavement Damage, Movement Area vehicle (Aging, Damage), Movement Area Equipment (Aging, Damage)
° Operational Environment (4)
- Number of Vehicle/Equipment, Adverse weather, Bird, Wildlife animals other than bird
° Air Navigation Safety Facilities (7)
- ILS, VOR/DME, Air Navigation Radio Communications System, Automatic Terminal Information Service A, Aeronautical Radio, Communications system B, Aeronautical Fixed Telecommunication System,
Automatic Terminal Information Service B
Moderate 7(13.7%) ° Human factors (1) - Manager participation,
° Events (6)
- Collision (Aircraft-Vehicle), Near-Miss (Aircraft-Vehicle), Aircraft fuel leakage, Bird strike, FOD(Maneuvering Area, Ramp Area)
Caution 1(1.9%) ° Facilities/Equipment (1) - Maneuvering Area Infrastructure Damage
Danger 2(3.9%) ° Events (1) - FOD (etc.),
° Facilities/Equipment (1) - Ramp Area Light Damage
Download Excel Table

V. CONCLUSION

The purpose of this study is to improve the safety of international airports. In order to investigate whether safety indicators at airports could be expressed in an easy-to-understand manner, including the overall index and its trend, a simulation was performed. Through 3 rounds of Delphi surveys, 51 indicators were extracted from seven factors (human factors, accidents/events, facilities/equipment, operational environment, aviation information and communication, and communication factors). Through the application of ICAO warning logic, an intuitive color system is used to express the status of each indicator over time. As a result of the AHP analysis, the relative importance of the indicators was determined, and the trend was created through a comprehensive measurement and comparison of airport safety indicators. Furthermore, vulnerable areas could be identified and managed.

In order to conduct this study, the scope of airport operational safety was set as an international standard. Nevertheless, due to the severity of the COVID-19 pandemic at the time of the survey, the response rate was not satisfactory. In light of this, it is recommended that the expert panel be carefully chosen in the future and that the scope of the study be expanded to conduct a more comprehensive study applicable to each airport.

Acknowledgment

This research was supported by a grant from the Incheon International Airport Corporation (Research on improving the evaluation of flight safety indicators and indexes at Incheon Airport).

References

1.

Liou, J. J., Tzeng, G. H., and Chang, H. C., “Airline safety measurement using a hybrid model”, Journal of Air Transport Management, 13(4), 2007, pp.243-249.

2.

Knegtering, B., and Pasman, H. “The safety barometer: How safe is my plant today? Is instantaneously measuring safety level utopia or realizable?”, Journal of Loss Prevention in the Process Industries, 26(4), 2013, pp.821-829.

3.

ICAO, “Annex 19, Safety Management”, Montreal, 2016, p.40.

4.

ICAO, “Doc9859, Safety Management Manual”, Montreal, 2018, pp.60-61.

5.

ACI, “Guide to Airport Performance Measures”, Montreal, 2012, pp.16-20.

6.

EASA, “The European plan for Aviation safety”, 2022, pp.77-87.

7.

FAA ACRP, “Report 19A: Resource Guide to Airport Performance Indicators”, 2011, pp. 213-227.

8.

Belgian Civil Aviation Authority, “ANNEX 9.2 Belgian Aviation Safety Programme”, 2015, pp.10-29.

9.

The Finnish Transport and Communications Agency, “Finnish Aviation Safety Objectives and Safety Performance Indicators and Targets”, 2018, pp.13-40.

10.

The Civil Aviation Authority of Singapore, “Singapore State Safety Program”, 2018, pp.25-36.

11.

Incheon International Airport Corporations, “Incheon Airport Safety Performance Indicators”, 2016, pp.3-5.

12.

Lingard, H., Hallowell, M., Salas, R., and Pirzadeh, P. “Leading or lagging? temporal analysis of safety indicators on a large infrastructure construction project”, Safety Science, 91, 2017, pp.206-220.

13.

Hallowell, M. R., Bhandari, S., and Alruqi, W., “Methods of safety prediction: Analysis and integration of risk assessment, leading indicators, precursor analysis, and safety climate”, Construction Management and Economics, 38(4), 2020, pp.308-321.

14.

Oster Jr, C. V., Strong, J. S., and Zorn, C. K.. “Analyzing aviation safety: Problems, challenges, opportunities”, Research in Transportation Economics, 43(1), 2013, pp.148-164.

15.

Reason, J. “Human Error”. Cambridge University Press, 1990.

16.

Zohar, D. “Safety climate in industrial organizations: Theoretical and applied implications”, Journal of Applied Psychology, 65(1), 1980, p.96.

17.

Wood, R., and Bandura, A. “Social cognitive theory of organizational management”, Academy of Management Review, 14(3), 1989, pp.361-384.

18.

Ajzen, I, “Attitudes, Personality, and Behavior”, McGraw-Hill Education, UK, 2005.

19.

Fugas, C. S., Silva, S. A., and Meliá, J. L., “Another look at safety climate and safety behavior: Deepening the cognitive and social mediator mechanisms”, Accident Analysis & Prevention, 45, 2012, pp.468-477.

20.

Brown, K. A., Willis, P. G., and Prussia, G. E., “Predicting safe employee behavior in the steel industry: Development and test of a sociotechnical model”, Journal of Operations Management, 18(4), 2000, pp.445-465.

21.

Hofmann, D. A., Jacobs R., and Landy, F., “High reliability process industries: Individual, micro, and macro organizational influences on safety performance.” Journal of Safety Research, 26(3), 1995, pp.131-149.

22.

Piers, M., Montijn, C., and Balk, A. “Safety Culture Framework for the ECAST SMS-WG”. European Commercial Aviation Safety Team (ECAST), 2009.

23.

Nonaka, I., and Takeuchi, H., “The knowledge-creating company”, Harvard Business Review, 85(7/8), 2007, p.162.

24.

Barling, J., Loughlin, C., and Kelloway, E. “Development and test of a model linking safety specific transformational leadership and occupational safety. Journal of Safety Specific Transformational Leadership and Occupational Safety”, Journal of Applied Psychology, 87, 2002, pp.488–496.

25.

de Koster., R.B. Stam, D., and Balk, B.M. “Accidents happen: The influence of safety-specific transformational leadership, safety consciousness, and hazard reducing systems on warehouse accidents”, Journal of Operations Management 29(7-8), 2011, pp. 753-765.

26.

Kelloway, E. K., Mullen, J., and Francis, L., “Divergent effects of transformational and passive leadership on employee safety”, Journal of Occupational Health Psychology 11, 2006, pp.76–86.

27.

Griffin, M. A., and Neal, A., “Perceptions of safety at work: A framework for linking safety climate to safety performance, knowledge, and motivation”, Journal of Occupational Health Psychology 5, 2000, pp.347–358.

28.

Hofmann, D. A., and Stetzer, A., “A cross-level investigation of factors influencing unsafe behaviors and accidents”, Personnel Psychology, 49, 1996, pp.307–338.

29.

Neal, A., and Griffin, M. A. “Safety climate and safety behavior”, Australian Journal of Management, Special Issue, 2002, pp.67–76.

30.

Gressgård, L. J., “Knowledge management and safety compliance in a high-risk distributed organizational system”, Safety and Health at Work, 2014, 5(2), pp.53-59.

31.

Hon C K., Chan A P., and Yam M C, “Relationships between safety climate and safety performance of building repair, maintenance, minor alteration, and addition (RMAA) works”, Safety Science, 65, 2014, pp.10-19.

32.

Jiang, L., and Probst, T. M., “Transformational and passive leadership as cross-level moderators of the relationships between safety knowledge, safety motivation, and safety participation”, Journal of Safety Research, 57, 2016, pp.27-32.

33.

Borman, W. C., and Motowidlo, S. M., “Expanding the Criterion Domain to Include Elements of Contextual Performance. Personnel Selection in Organizations”, San Francisco: Jossey-Bass, 1993, p.71.

34.

ISO 22301, “Security and Resilience-Business Continuity Management Systems-Requirements”, Switzerland, 2019, pp.1-30.

35.

In Kie Na, “A study on the relationship of airport safety management system and airside workers’behaviour”, Ph.D Thesis, Korea Aerospace University, Deogyang-gu, Goyang-si, Gyeonggi-do, Korea, 31 August 2019.

36.

Dalkey, N.C., 1968. “Experiments in Group Prediction (p.0014)”, Santa Monica, California: Rand Corporation.

37.

Gargon, E., Crew, R., Burnside, G., and Williamson, P. R., “Higher number of items associated with significantly lower response rates in COS Delphi surveys”, Journal of Clinical Epidemiology, 108, 2019, pp. 110-120.

38.

Delbecq, A. L., Van de Ven, A. H., and Gustafson, D. H., “Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes”, Scott, Foresman, 1975.1.

39.

Rowe, G., and Wright, G., “Expert Opinions in Forecasting: The Role of the Delphi Technique. Principles of Forecasting: A Handbook for Researchers and Practitioners”, 2001, pp.125-144.

40.

Baker, J., Lovell, K., and Harris, N., “How expert are the experts? An exploration of the concept of ‘expert’ within Delphi panel techniques”, Nurse Researcher, 2006, 14(1), pp.59-70.

41.

Keeney, S., Hasson, F., and McKenna, H. P., “A critical review of the Delphi technique as a research methodology for nursing”, Int. J. Nurs. Stud., 2001, 38, pp.195-200.

42.

Yi, Y. K., Yoo, K. E., and Park, S. S., “A study on the competitiveness evaluation index development of global aviation training organizations”, Journal of the Korean Society for Aviation and Aeronautics, 23(1), 2015, pp.74-84.

43.

Lawshe, C. H., “A quantitative approach to content validity”, Pers. Psychol, 28, 1975, pp.563–575.

44.

Nicolaidou, O., Dimopoulos, C., Varianou-Mikellidou, C., Mikellides, N., and Boustras, G., “Weak signals management in occupational safety and health: A Delphi study”, Safety Science, 105558, 2022, p.146.

45.

Tran, T., Trinh, T. P. T., Le, C. M., Hoang, L. K., and Pham, H. H., “Research as a base for sustainable development of universities: Using the Delphi method to explore factors affecting international publishing among Vietnamese academic staff”, Sustainability, 12(8), 2020, p.3449.

46.

Soltanifar, M., and Lotfi, F. H., “The voting analytic hierarchy process method for discriminating among efficient decision making units in data envelopment analysis”, Computers & Industrial Engineering, 60(4), 2011, pp.585-592.