Similar to other industries, aviation has tried to eliminate operational risk through the application of technology and academic achievements. As a result, the aviation industry has achieved a good record and is regarded as being safe, with a low number of accidents (Liou et al., 2007). As a result, many states and air service providers are attempting to develop safety management systems (SMS). SMS assurance uses safety performance indicators (SPIs) to process past data in order to ensure better results over the next period (next year, next phase). As a result of this concept, sustainability can be achieved from disasters, which is widely used in other industries. A crucial component of effective risk management is the detection of events at a site, such as accidents and near misses (Knegtering and Pasman, 2013). These events are organized into indicators, which managers can use to improve safety. Investment decisions can be made based on it. In many cases, however, more than such results-oriented data is required to fully explain potential risks. According to Liou et al. (2007), the aggregate accident rate can be used as a safety index, but it is necessary to complete the analysis. It is necessary to improve this aspect of the current airport safety performance monitoring system. Psychological indicators can therefore be used to provide a solution to this problem. In this study, we aim to improve the existing airport safety indicator system based on the results of an investigation of airport workers’ safety psychology.
II. AIRPORT SAFETY INDICATOR SYSTEM
The International Civil Aviation Organization (ICAO) established ANNEX19. According to these documents, States and service providers including airports, airlines, etc. are required to comply with Standards And Recommended Practices (SARPS) (ICAO, 2016). This document emphasizes the importance of protecting safety data and information. Those are the fundamentals for the development of indicators and indices. Furthermore, it recommended that the states select quantifiable indicators that take into account their relevance to the service providers, including airports and airlines’ safety objectives. Member countries and service providers refer to these documents as the most influential guidelines. It is also presented in the DOC9859 (Safety Management Manual) examples of indicators that help recognize warnings from fields by setting triggers based on standard deviations (ICAO, 2018).
There has been a guide on Airport Performance Measures (APM) published by ACI (Airport Council International) since 2012 and this guide is available to airports worldwide. The guidebook provides six key performance areas, including core competency, safety, security, and service quality. There are several safety-related indicators, including runway accidents, excursions and incursions, collisions with birds, injuries, and lost work. Based on the type of airport, each indicator is classified according to its applicability (ACI, 2012). The indicator was defined with related parameters and listed the links between the driver and the indicator.
There is also an indicator system that is used by the National Union. The European Aviation Safety Agency (EASA) is composed of members from 31 countries throughout Europe, and it is responsible for certification, standardization, investigation, and monitoring. The European Plan for Aviation Safety (EPAS) includes a safety performance indicator. The major safety indicators include runway excursion, airborne collisions, controlled flight into terrain (CFIT), loss of control during flight, runway incursion, and fire (EASA, 2022).
The development of indicators was not only done by the international organizations described above but also according to the circumstances within the country. Airport Cooperative Research Program (ACRP), an institute specializing in airports supported by the US FAA, presented indicators for the airport sector, such as airside accidents, minor accidents, work-related casualties, runway invasions, and wild animal accidents in the Safety Management Systems guidebook for airports. Another report published, ‘Resource Guide to Airport Performance,’ included 29 key indicators, 132 key indicators, and 679 other indicators. Safety-related indicators include two core, ten major, and 31 other indicators (FAA ACRP, 2011).
The Belgian CAA has, like several other countries, established an aviation safety program in accordance with ICAO DOC9859 and EASA’s safety policy. According to the European Coordination Center for Accident and Incident Reporting Systems (BCAA, 2018), 36 detailed safety performance indicators were developed for accidents and incidents, aerodromes, navigation services, operations, technology, and other prominent fields.
ICAO GASP, Doc 9859, and EU EPAS have been implemented by the Finnish Transport and Communications Agency (TRAFICOM) through its Finnish National Aviation Safety Program (FASP). As stated in TRAFICOM (2018), the EU-level safety objectives must be considered in conjunction with the specific setting of acceptable levels of safety and scope at the national level. Airports, airlines, and ground handlers are provided with safety performance indicators through the FASP program. The safety performance indicators for airport operators are further subdivided into SMS performance, runway incursion/excursion, runway status defects, visibility, air traffic control flight, accidents, a lack of information about obstacles, a ground collision, foreign matter in moving areas, CFIT, as well as neglect of apron area supervision.
In 2008, the Singapore Civil Aviation Authority (CAAS) developed safety indicators for airport operators and airlines as part of the State safety program. Indicators related to airports include aircraft-related accidents, incidents, wildlife collisions, runway intrusions/deviations, and ground collision prevention system recommendations (CAAS, 2018).
In order to measure safety performance, South Korea set up a national aviation safety program through its civil aviation authority, MOLIT (Ministry of Land, Infrastructure and Transport). It is also through this CAA that the state’s aviation safety program was established. Under the theme of air service, each area is divided into sub-parts, such as airport facilities, navigation facilities, air traffic control and air operations, and its indicators are provided. As provided in ICAO Doc9859, the 3-year standard deviation was used as the alert method. There are several indicators related to the airport sector, including ground conflict/accidents, aircraft ground collisions, airport aerodrome facility malfunctions, and navigation safety facility failures. Incheon International Airport Corporation, the primary gate in South Korea, has established 25 safety indicators in support of national indicators since 2013 (IIAC, 2016; Table 1).
ICAO presented representative examples of these indicator systems, such as setting triggers for indicators related to the aviation industry of the state, but these examples were merely indicative. The reason may be the fact that international organizations are responsible for covering the entire aviation industry associated with all states. It is unnecessary to cover the details of service providers such as airports.
In contrast, the ACI did not present any criteria from its guidebook but rather included a number of customized indicators. The ACI and FAA ACRP’s guides present indicators based on their importance as opposed to ICAO’s list of indicators regardless of their relative weight. In addition to their differences, they share the following similarities:
?Various organizations and states have developed airport safety indicators that deal with the same or similar issues. This is due to the similarity between the airport infrastructure’s functional, topographical, and operational characteristics. When safety performance is measured using the same indicators, this provides a positive perspective on comparing safety performance.
?Risk levels, which represent severity and likelihood of occurrence, are commonly used in airport safety indicators. It helps to determine the type and timing of resource allocation according to the risk level. It should be noted, however, that the level of risk is only sometimes constant and reliable due to the ease with which this assessment can be altered. It is important to note that although the severity of a potential occurrence is relatively fixed, the likelihood can vary depending on the location, the time (day/night, weekday/holiday), the season, and the weather. Therefore, it is important to note that variability in likelihood can also distort risk levels and prevent appropriate decision-making.
?Most of the indicators related to airport safety consist of lagging indicators rather than leading indicators. Lagging and Leading indicators can be classified as measured symptoms in advance or events on past results (Lingard et al., 2017). The lagging indicator is outcome-oriented. On the other hand, the leading indicators are process-oriented (ICAO, 2018). Hallowell et al. (2020) studied leading indicators, including pre-task safety meetings, which are able to suppress accidents and have an inverse correlation with injury rates. Given the current indicators system in the airport industry, it cannot be considered the proactive way (Oster Jr et al., 2013). Therefore, leading indicators were extended in this study.
?The overall direction of indicators cannot be measured through the current system due to the mutually independent characteristics of each indicator. Although this fact is generated naturally, it may not be persuasive information for top management because it is difficult to make a comprehensive decision from several individual indicators. The relative weight among indicators is essential in prioritizing decision-making for limited resources. Moreover, there are only a few previous studies considering integrated perspectives.
Given the commonalities described above, this study must include a more innovative and integrated indicator system for sustainable safety management. In order to accomplish this, the study was intended to develop airport workers’ psychology as a component of the indicator. Safety accidents often occur as a result of workers’ minds manifesting in behaviors such as violations and negligence (Reason, 1990). This study attempts to confirm the effects of safety knowledge, attitude, consciousness, climate, and behavioral factors on safety-related behavior and to integrate them with existing indicators.
Safety attitudes have been studied early as an essential predictor of climate (Zohar, 1980). It is also defined as individuals’ belief and confidence about the environment and technology that enable them to continue working safely in their daily work (Wood and Bandura, 1989). Ajzen (2005) divided safety attitudes into favorable and unfavorable ones, saying that favorable attitudes are future-oriented responsibility and constructive beliefs, while unfavorable attitudes are destructive beliefs and characteristics they want to avoid. Fugas (2012) also differentiated between constructive and destructive beliefs regarding safety and argued that positive beliefs should be strengthened while negative beliefs should be reduced. According to Brown et al. (2000), the negative factor affecting worker safety behavior is a cavalier attitude, while the positive factor is safety efficacy. In this study, safety attitudes were divided into two categories: safety efficacy and cavalier attitude.
Safety knowledge shapes safety behavior by ensuring that workers are aware of safety operating procedures and have received appropriate training and instructions (Hofmann et al., 1995). According to Piers et al. (2009), safety knowledge involves understanding the risks involved in the operation and making sure it is distributed and utilized properly. Safety knowledge is divided into explicit and tacit knowledge. Explicit knowledge refers to knowledge that is easy for individuals to convey to organizations because it is official and can be documented. On the other hand, tacit knowledge is generally accumulated by individuals from experience and has characteristics that are variable and difficult to share depending on the situation (Nonaka and Takeuchi, 2007). Safety consciousness refers to “an individual’s awareness of safety issues” (Barling et al., 2002) and operates at both a cognitive and behavioral level (Koster et al., 2011). It is also a factor that significantly impacts safety reporting practices and the incidence of accidents (Barling et al., 2002; Kelloway et al., 2006).
The concept of safety climate is more collective than the concept of safety consciousness. Generally, it refers to the belief that safety should be at the center of organizational policies, procedures, and goals (Griffin and Neal, 2000; Hofmann and Stetzer, 1996; Neal and Griffin, 2002). It is important to note that safety behavior consists of actions taken to protect from danger based on safety knowledge and skills that are necessary for the performance of one’s job (Gressgard, 2014). Several previous studies have identified two types of safety behavior: compliance and participation.
Hon et al. (2014) classified safety behavior into safety compliance and active participation in obligations. Jiang and Probst (2015) also said that safety compliance is a behavior required for safety (e.g., safety rules to wear protective gear before work). Neal and Griffin (2000) stated that it includes activities and training to actively participate in safety issues in advance and speak to managers. Borman and Motowidlo (1993) classified safety behavior into safety conformity behavior, a crucial activity to maintain safety, and safety expectation behavior that helps create the surrounding environment but does not affect safety directly.
III. CONCEPTUAL FRAMEWORK AND METHODOLOGY
The study scope was defined in the Incheon International Airport movement area. When recent statistical data were available, the range was 2020-2022. The research order chart and descriptions are:
As part of the first stage of the study, indicators from international organizations and states were introduced, and previous studies were reviewed and gathered. In the second stage, a Delphi survey was conducted on the collected candidate indicators. In order to increase reliability, more relevant or essential indicators should be selected. An Analytic Hierarchical Process (AHP) was used to calculate the weight reflecting relative importance among the indicators in the third stage. A final stage in the process involved calculating and comparing the indicators and index, i.e. the sum of the weights of each indicator over the past six years (2020-2022).
As part of ISO22301 (2019), an international standard for business continuity planning, safety indicators, and targets should be considered in order to meet the legal requirements of both internal and external stakeholders as well as the government. Indicators of this type apply to all other areas, including this study. Table 2 shows reviewed indicators from international organizations, national guidelines, and previous studies. Academic journals and reports find psychological indicators, including safety awareness and attitude, etc. 25 indicators from Incheon International Airport were also reviewed for candidates.
|Organization||# of relevant indicators||Organization||# of relevant indicators|
|ICAO, Doc 9859||5||FAA, ACRP Safety performance||22|
|ICAO, GASP||7||FAA, ACRP Airport performance||31|
|ACI||5||BCAA Safety Programme||22|
|EASA, ASCOS||6||Finland, FASP||9|
|Singapore, CAAS||7||Korea Aviation, Safety Program||10|
|Korea, Civil Aviation Act||18||Incheon airport SPI||25|
|literature (Zohar, 1980; Brown et al., 2000; Hofmann et al., 1995; Barling et al., 2002; Hon et al., 2014; Griffin and Neal, 2000)||10|
Based on the above statements, purposive sampling was used to select experienced experts from the airport, airlines, and ground handlers to participate in the survey. The following are details of each survey round. After repeated surveys of experts, the statistical average of the surveys is converged into a group opinion (Rowe and Wright, 2001), and the results are finally summarized after confirming consensus (Baker, Lovell, and Harris, 2006).
The Delphi method utilizes an iterative process to develop consensus among a diverse group of experts (Keeney et al., 2001). The importance of consensus among airport safety experts cannot be overstated since airport safety assessment is a field that has not been adequately addressed in existing literature (Yi et al., 2015). According to Keeney et al. (2001), consensus is defined as being reached among participants in a Delphi study when 75% or more of the respondents agree. When participants agree with the statement (i.e., 4 or higher on a 5-point Likert scale), consensus has been reached.
The Content Validity Ratio (CVR) index is a method of calculating internal validity to examine the appropriateness of item content, based on the percentage of Delphi panelists who rated the appropriateness of an item as significant (4 or higher) on a 5-point scale. Lawshe (1975) proposed the following formula for CVR calculation.
CVR: Content validity ratio
N: Number of total panels
Nε: Number of panels that responded “Important” (Likert scale = 4 or 5)
To determine the degree of consensus in the expert panel’s opinion, we applied the method of Delbecq et al. (1975). The measure should be less than 0.5.
CGD: Convergence degree
Q3: first quartile, 25% of the total
Q1: third quartile, 75% of the total
The first round consisted of experts selected from the same categories as those designated for the Delphi study. The Delphi research panel critically reviewed the findings to gain a deeper understanding of the consensus reached among the researchers and to create a framework for future research efforts in the field (Nicolaidou, et al., 2021).
The criteria for the expert panel were established by applying the Delphi experience criteria of Tran, Trung et al. (2020) in order to select experts who had at least five years of experience as airport safety instructors. The first round of the selection process was conducted on airport experts with an average of six years of experience. An open-ended question type was used to eliminate candidates who were not relevant. As a result of this process, indicators that were not relevant or overlapping were identified and removed.
The second round was conducted with 18 experts from the airport, airlines, and ground handlers with an average working experience of 17 years. A 5-level Likert questionnaire was devised to select from three perspectives: necessity, importance, and urgency. After evaluating the importance of each metric, we set up a comment field for experts to comment on modifying, deleting, or merging metrics.
During the third (last) round, 27 experts from airports, airlines, and ground operators were interviewed, with an average of 11 years of experience. When the survey was conducted, we were experiencing the height of the COVID-19 virus pandemic. Some of our experts became infected with the virus, which resulted in a low return rate. Several experts in airport safety were consulted to reach conclusions about the importance of airport safety indicators.
The 25 indicators were assessed using a questionnaire based on a Likert 5-level scale in order to determine their suitability and expert confidence. The final ten indicators could be selected after calculating the mean and standard deviation of relevance and expert confidence values (3.89 and 4.01, respectively). The result was the confirmation of the existing 7 indicators, as well as 18 new indicators (Fig. 2, Table 3-5).
Lastly, 51 indicators were reclassified and reorganized into four new categories (human factors, accidents/events, air traffic control/airside safety, and air navigation) by adding seven indicators related to air navigation facilities provided by the South Korean State Aviation Safety Program. The following is a classification of indicators from CAT I to CAT IV (Fig. 3).
ICAO introduced the approach of applying a standard deviation to the average value of the past three years in selecting alert levels. This study also applied the alert level to set trigger systems. Moreover, the risk level was divided into four levels, and each risk level was expressed as good/moderate/caution/danger, respectively.
There is a need to monitor these indicators, however, the importance of each indicator cannot be rated equally. However, it is difficult to find previous studies that have examined the relative importance of the indicators and integrated them into a composite index. Consequently, an index that sets and integrates the weights of the indicators has been developed. There are two benefits to this approach.
Firstly, an index calculation provides an intuitive understanding of the trend resulting from each indicator incorporated. Using this method, the overall performance of the organization is identified and decisions can be made more effectively.
Secondly, it is possible to compare results over time and with other entities using the same indicators. Comparisons can easily lead to an overall assessment and the setting of short-term (e.g., annual) safety goals.
As a result, resources are used more efficiently in operations. An analytical hierarchy process (AHP) was applied in order to accomplish this. According to Soltanifar and Lofti (2011), AHP uses a weighted voting ranking approach in order to rank criteria and sub-criteria. To perform AHP, it was necessary to classify the 51 indicators for clustering. The safety function is responsible for completing this task in its own area. Category I (CAT I) was classified according to airside safety, air traffic control, and air navigation. It is the same as dividing areas in airport organizations. It is important to note that Category II consists of seven indicators covering areas that constitute Category I - human factors, accidents/events, facilities/ equipment, operational environment, air navigation functions, and communication. A total of 29 indicators are included in the CAT III group, and these indicators can be used to explain CAT II in greater detail. CAT IV follows CAT III and is further separated in the same way as above. As a result of the Delphi survey, 51 final sub-indicators were identified for each area of the CAT I. Some indicators, including the air navigation area, did not have CAT IV indicators in this process. The reason for this is that it has already been sufficiently subdivided and explained in detail.
The survey involved 16 experts from airlines, ground handlers, and the airport in order to determine weights based on AHP. A total of ten answer sheets were evaluated, 16 were used, and six with a low consistency ratio (CR) were removed from the analysis. Each weight of CAT IV indicators was calculated as CAT I weight × CAT II weight × CAT III weight × CAT IV average value (%). The final weighted score of each indicator was calculated as the CAT IV indicator weight × 100. There was no significant effect of some decimal points being corrected in the CAT IV weight results.
Indicators with weights were simulated using data from 2020 to 2022 at Incheon International Airport. There are four warning levels based on the indicator’s 3-year average within the standard deviation range. The details are as follows:
·Green: Good, when the measurement year score is within 1st of the 3-year average
·Yellow: Moderate, when the measurement year score falls within 1sd ~ 2sd of the 3-year average.
·Orange: Caution, when the score in the measurement year falls within 2sd ~ 3sd of the 3-year average.
During the past three years, the index has repeatedly risen and fallen. Every year, the index shows moderate performance (93.4 in 2020, 92.6 in 2021, and 92.1 in 2022). Below is the summary of 2022 found thanks to this study’s intentions and advantages.
· Human Factor Indicators (CAT-II).
Human indicators are the result of a survey of airside drivers. Almost all sub-indicators, including safety knowledge/attitude/consciousness, maintain stability, so the cultural aspect could be considered healthy. However, the score for participation in safety committees decreased (moderate) compared to the past. The reason was that face-to-face meetings were not possible due to COVID-19 and even online meetings were possible; there were cases where the evaluation score fell due to human resource management (leave, etc.) in the aviation industry.
· Accidents/Events (CAT-II)
The number of aircraft accidents, incidents, excursions, and incursions of this severity is very high, but the probability of occurrence is low. CAA and organizations use these indicators. In this study, these indicators have been very stable since the indicator index was established. There have been zero cases in five years at Incheon airport. Low-risk and high probability indicators, including ground safety accidents, near misses, fuel leakages, and bird strikes, occurred in 10 cases in 2022. They occurred frequently. Therefore, those indicators must be monitored and managed organizationally. Foreign Object Debris (FOD) collection has increased, so management needs to be followed in the future. With respect to FOD, it can be viewed as a leading indicator of the accident’s cause since it cannot be considered an accident’s result (but a possible cause as well). However, since FOD occurs due to worker neglect or abandonment, it can also be considered a lag indicator. This consideration can be applied to indicators that happened and were detected visually but have not yet caused any damage. This will depend on the entity’s decision to determine whether they are leading or lagging indicators.
· Facilities/Equipment (CAT-II)
Sub-indicators of this category help monitor damage or aging in the maneuvering/ramp/ movement area. This area can be controlled by prior inspection, maintenance? and repair. For example, in 2022, infrastructure (pavement surface) in the maneuvering area and airport lighting damage in the ramp area increased rapidly; these were marked as caution (orange) and danger (red), respectively, compared to the previous year. These findings improve those spots by strengthening prior inspections and expediting repairs.
· Operational Environment (CAT-II).
Operating environment refers to a set of indicators derived from background elements that do not directly affect safety. In contrast to Facilities/Equipment (CAT-II above), it is difficult to control through artificial means due to the fact that it is subject to changes in response to external factors, such as overall trends or natural forces. For example, the number of vehicles and equipment varies depending on the air traffic volume. The probability of accidents increases when the number of vehicles increases within the spatially limited airport airside. Adverse weather is also an indicator of natural disasters, and avoidance, such as suspension of operations, can serve as a safety strategy. Even if a curfew is in place, these category indicators are not related to safety operations. However, it must still be an indicator to be monitored at all times in a 24-hour airport. It is imperative to note that bird and wildlife influxes vary according to season. The number of wildlife species detected (excluding birds) surged in 2021 to danger (red) and returned to stability (green) in 2022.
· Air Navigation (CAT-I, including all CAT-II)
Air navigation (CAT-I, Air Navigation Safety Facilities, Aviation Information & Communication, Communication Factors in CAT-II) represents a very stable (green) indicator group with no reported cases, except for the instrument landing system in 2021. From this, it was found that Incheon Airport air navigation is operated safely with an indicator group of accidents/events. Table 7 summarizes the above conclusions.
The purpose of this study is to improve the safety of international airports. In order to investigate whether safety indicators at airports could be expressed in an easy-to-understand manner, including the overall index and its trend, a simulation was performed. Through 3 rounds of Delphi surveys, 51 indicators were extracted from seven factors (human factors, accidents/events, facilities/equipment, operational environment, aviation information and communication, and communication factors). Through the application of ICAO warning logic, an intuitive color system is used to express the status of each indicator over time. As a result of the AHP analysis, the relative importance of the indicators was determined, and the trend was created through a comprehensive measurement and comparison of airport safety indicators. Furthermore, vulnerable areas could be identified and managed.
In order to conduct this study, the scope of airport operational safety was set as an international standard. Nevertheless, due to the severity of the COVID-19 pandemic at the time of the survey, the response rate was not satisfactory. In light of this, it is recommended that the expert panel be carefully chosen in the future and that the scope of the study be expanded to conduct a more comprehensive study applicable to each airport.