Soo-Hyun Kim and Hye-Yeong Chun*

Transcription

1 METEOROLOGICAL APPLICATIONS Meteorol. Appl. 23: (2016) Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: /met.1581 Aviation turbulence encounters detected from aircraft observations: spatiotemporal characteristics and application to Korean Aviation Turbulence Guidance Soo-Hyun Kim and Hye-Yeong Chun* Department of Atmospheric Sciences, Yonsei University, Seoul, South Korea ABSTRACT: Using observational data from Korean Air Lines (KAL) Boeing (B) , B and B flights from January to December 2012, the derived equivalent vertical gust velocity (DEVG) was calculated as a turbulence indicator. Based on 1 min flight segments using the calculated DEVG, the highest frequency of moderate-or-greater (MOG) turbulence occurred in the Northern Hemisphere winter, whereas the lowest frequency occurred in the Northern Hemisphere summer. Spatially, the KAL turbulence encounters (KAL-DEVG) covered five regions, Asia, Oceania, Western Europe, North America and South America, following major flight routes. The number of observed turbulence events is normalized by flight density and navigation times. As a result, 1 MOG turbulence is observed per flight and per 10 h of navigation. Over East Asia, the observed MOG KAL-DEVG mainly appeared to follow the jet stream and most turbulence events were related to shear instability and inertial instability. KAL-DEVG was used to evaluate the operational Korean Aviation Turbulence Guidance (KTG) system developed using a combination of the Regional Data Assimilation and Prediction System (RDAPS) of the Korea Meteorological Administration and pilot reports over East Asia. The forecasting performance evaluated by the skill score (defined as the area under the curve based on the probability of detection statistics) on the operational-ktg system against KAL-DEVG and RDAPS analysis data was found to be 0.815, with 95% confidence levels ranging from to Using the RDAPS 6 and 12 h forecast data, the skill score was slightly less than 0.8 in comparison to KAL-DEVG. KEY WORDS Aviation turbulence; Aircraft observation; Derived equivalent vertical gust velocity (DEVG); Korean aviation turbulence guidance (KTG) Received 17 August 2015; Revised 10 March 2016; Accepted 13 March Introduction Encountering unexpected aviation turbulence can lead to hazards for both passengers and aircraft. According to the 2010 annual report of the National Transportation Safety Board (NTSB, 2010), turbulence was the main cause of aviation accidents related to weather from 1997 to In South Korea, turbulence has accounted for about 24% of weather-related aviation accidents since 1957 (Kim and Chun, 2011). To reduce these damaging events, it is necessary to predict turbulence adequately based on an understanding of the generation mechanisms of turbulence through observational analysis and numerical modelling (Lane et al., 2003; Wolff and Sharman, 2008; Kim and Chun, 2010; Sharman et al., 2014). Sharman et al. (2006) developed the Graphical Turbulence Guidance (GTG) system using the output from the numerical weather prediction (NWP) model and the observed turbulence encounters from pilot reports (PIREPs). This methodology combines several turbulence diagnostics that agree closely with the observed turbulence to minimize forecasting errors caused by uncertainty in each turbulence diagnostic. Based on a methodology similar to the GTG system, Kim and Chun (2012a) developed the Korean Aviation Turbulence Guidance * Correspondence: H-.Y. Chun, Department of Atmospheric Sciences, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul 03722, South Korea. chunhy@yonsei.ac.kr (KTG) system in East Asia using PIREPs and the analysis data from the Regional Data Assimilation and Prediction System (RDAPS), which is the operational NWP model of the Korea Meteorological Administration with a 12 km horizontal resolution. More information about the RDAPS is available online ( as well as in previous studies by Kim and Chun (2012a) and Lee and Chun (2015a). Although PIREPs are used in various case studies and statistical analyses for turbulence encounters due to their relatively broad spatiotemporal distributions (e.g. Wolff and Sharman, 2008; Kim and Chun, 2010, 2011; Lee and Chun, 2015b), there are substantial errors in the intensities, timing and locations of these turbulence encounters (Schwartz, 1996; Cornman et al., 2004). As the use of objective and continuous data has been in demand, several studies using direct aircraft observations have been carried out recently (e.g. Kim and Chun, 2012b; Gill, 2014; Gill and Buchanan, 2014; Sharman et al., 2014). Gill and Buchanan (2014) evaluated ensemble forecasts from the Met Office Global and Regional Ensemble Prediction System with respect to the derived equivalent vertical gust velocity (DEVG), which is calculated using high-resolution automated aircraft observations. The DEVG can be obtained by considering the airspeed, vertical acceleration, altitude and weight of an aircraft in flight (Truscott, 2000; Gill and Stirling, 2013; Gill and Buchanan, 2014). In the present study, the DEVG is calculated as a turbulence indicator using aircraft-based observations provided by Korean Air Lines (KAL) and the spatiotemporal characteristics of each 2016 Royal Meteorological Society

2 Aviation turbulence encounters detected from aircraft observations 595 level of turbulence, as estimated by the DEVG data, are presented both globally and throughout East Asia. DEVGs above ft are considered exclusively. This lower limit of altitude was chosen based on Kim and Chun (2011), which showed that 5 year mean numbers of light-or-greater and moderate-or-greater (MOG) turbulence events from PIREPs exist mostly above ft. Similarly, over the contiguous United States, Wolff and Sharman (2008) considered upper-level turbulence encounters from PIREPs mostly above ft. In addition, the DEVG in East Asia was used to validate the KTG system, which operationally forecasts aviation turbulence above ft at 1000 ft intervals for 24 h (hereafter referred to as the operational-ktg system) at the Korea Aviation Meteorological Agency (KAMA). 2. KAL data and methodology 2.1. KAL data The KAL data used in the present study were recorded from January 2012 to December 2012 in the flight data recorders of Boeing (B) , B and B fleets. Among the various data in the flight data recorder, the latitude, longitude, altitude, airspeed, vertical acceleration, static air temperature and aircraft mass, which are recorded every second, were used to calculate the DEVG. Prior to calculating the DEVG, quality control (QC) procedures were conducted to remove errors from the KAL data, which are described briefly below. Data were considered to be erroneous if: 1. the deviation of the vertical acceleration per second exceeded ±2.5g (where g is the acceleration due to gravity). Although turbulence encounters are treated as MOG turbulence when the deviation of the vertical acceleration exceeds 0.5g (Truscott, 2000), the upper threshold was set to 2.5g in the present study in order to remove unusual errors safely; 2. the deviation of the static air temperature per 2 s was more than 20 C; 3. the deviation of the altitude per second had an unrealistically high value ( ft). On average, altitude variation per second in KAL data is about 27 ft (= 8.2 m); 4. the deviation of the aircraft mass was more than 4.5 tonnes per second. The average difference in the aircraft mass was 0.08 tonnes per second. Through these QC procedures, only data from 99 of the 170 flights were valid and these flight data were used in the present study. Note that nearly half of the original data were discarded from the aforementioned QC procedures, due to real data quality problems rather than to overly stringent QC procedures. Despite limited samples of flight data, KAL data contained the entire recorded time series from take-off to landing with high frequency. Therefore, analyses for each flight route are available. The tracks of all KAL flights used in the present study are shown in Figure 1 (solid lines). The KAL flights covered five major routes in Asia, Oceania, Western Europe, North America and South America. The number of flights in each of the five routes is shown in Table 1. The highest flight density (73) was in Asia, which includes Korea, China, Japan, Indonesia and the Philippines. Conversely, the South America route is the least dense, with only two flights to Brazil. The monthly distributions of the number of flights (Table 2) show that there were relatively large numbers of flights in August and September and relatively small numbers in January and November. Table 1. Number of flights on five major routes of KAL data from January 2012 to December Track of flights Number of flights Asia 73 Oceania 10 Western Europe 3 North America 11 South America 2 Total 99 Table 2. Number of flights by month of KAL data from January 2012 to December Month Number of flights January 1 February 6 March 6 April 11 May 10 June 6 July 10 August 16 September 14 October 8 November 2 December 9 Total DEVG calculation Following Truscott (2000), the DEVG is calculated using KAL data as: Am Δn DEVG = (1) V Here, m is the aircraft mass in tonnes, Δn is the fractional deviation of the vertical acceleration from 1g and V is the calibrated airspeed in knots. Considering aircraft types, A can be approximated as: ( )( ) m A = A + c 4 A c 5 m 1 (2) ( ) c2 A = c 1 + (3) c 3 + H Here, H is the altitude in thousands of feet, m is the reference mass of the aircraft in tonnes and c 1, c 2,, c 5 are empirical constants depending on the aircraft type. Table 3 shows the constants (c 1, c 2,, c 5 ) for B , B and B based on Truscott (2000). Following the previous studies of Truscott (2000) and Gill (2014), the standard levels of aviation turbulence were determined by the values of DEVG: null (NIL) turbulence when DEVG < 2, light (LGT) turbulence when 2 < DEVG < 4.5, moderate (MOD) turbulence when 4.5 < DEVG < 9 and severe (SEV) turbulence when DEVG > 9. Although KAL data were recorded at 1 s intervals and the DEVG was also calculated every second, the turbulence was counted at 1 min intervals using the maximum value of the DEVG within each segment. The 1 min segment length was chosen because the RDAPS analysis and forecast data in East Asia had a horizontal resolution of 12 km, which

3 596 S-.H. Kim and H-.Y. Chun Figure 1. Flight tracks (solid lines) and locations (dots) for the LGT, MOD and SEV turbulence from KAL-DEVG from January 2012 to December 2012, at altitudes above ft. The region of East Asia, which is equal to the domain of the NWP model, Regional Data Assimilation and Prediction System (RDAPS) operated by the Korea Meteorological Administration, is indicated by the grey box. Table 3. Empirical constants of the three flight types used in the calculation of the DEVG (Truscott, 2000). Flight type c 1 c 2 c 3 c 4 c 5 B B B corresponds to slightly more than 1 min of flight time at an average flight speed of 600 km h 1 at cruising level. Hereafter, these KAL turbulence encounters are denoted as KAL-DEVG. 3. Spatiotemporal characteristics and possible mechanisms of KAL-DEVG 3.1. Temporal and spatial characteristics of KAL-DEVG Figure 1 shows the locations of KAL-DEVG above ft, where the LGT, MOD and SEV turbulence encounters are indicated by dots, along with flight tracks shown as solid lines. The LGT, MOD and SEV turbulence encounters were observed in the height range ft with maximum frequencies at ft. Table 4 shows the frequency and relative ratio of each level of turbulence in KAL-DEVG obtained from 1 min segments globally and over East Asia. To examine the sensitivity to the segment period, additional results based on 30 min segments are also shown in Table 4. The East Asia region (grey box in Figure 1) is equal to the forecast region of the RDAPS and operational-ktg system. For the results of 1 min segment KAL-DEVG both globally and in East Asia, there were many more NIL turbulence encounters than other turbulence intensities. The NIL-level turbulence accounted for about 97%, whereas the MOG-level turbulence was only 0.3%. The proportion of MOG turbulence from KAL-DEVG was much less than that from PIREPs (5.44%) for the same period of KAL-DEVG (Table 5), as well as the data averaged over 11 years ( ) (4.20%), as shown by Lee and Chun (2015b). This is probably because the relatively short segment length (1 min) for KAL-DEVG led to more NIL turbulence (97.29%) compared to PIREPs (65.26%) (Table 5). The LGT turbulence from KAL-DEVG (2.33%) was also much less than that from PIREPs (29.30%). When 30 min segments were applied to the current KAL-DEVG calculation, the MOG turbulence in East Asia was similar (5.03%) to that from PIREPs (5.44%). Additionally, the LGT turbulence was less (18.49%) than that from PIREPs (29.30%) and the NIL turbulence was greater (76.48%) than that from PIREPs (65.26%) (Table 5). Considering the fact that NIL turbulence reports from pilots are less frequent, the proportions of each level of turbulence from the current KAL-DEVG calculation based on 1 min segments are likely to be more realistic than those obtained from PIREPs, at least in East Asia. Hereafter, 1 min KAL-DEVG are used. Table 6 indicates the counts for the LGT, MOD and SEV turbulence on each of the five major flight routes. It is interesting that the largest counts for LGT, MOD and SEV turbulence occurred on the North America route, followed by the Asia route. The results in Table 6 demonstrate that flights experienced more turbulence when they passed over the North Pacific and the South China Sea. When locations for turbulence events were examined more closely, most of them were spread uniformly along each route. Meanwhile, considering the fact that the number of flights and navigation time for each flight were different from each of the routes, normalized counts of the turbulence by the number of flights and flight time were required. This will be discussed later in Figure 3. Figure 2 shows the monthly and seasonal counts for each level of turbulence and the relative percentage of the MOG turbulence normalized by the monthly total counts globally (Figures 2(a) and (b)) and in East Asia (Figures 2(c) and (d)). Globally, the largest count of LGT turbulence (55) was observed in March and the smallest count (12) was observed in November. MOD turbulence was observed most frequently in October (10) whereas

4 Aviation turbulence encounters detected from aircraft observations 597 Table 4. Counts and percentages of NIL, LGT, MOD and SEV turbulence obtained from KAL-DEVG with 1 min segments (1-segment) and 30 min segments (30-segment) at altitudes above ft globally and for East Asia from January 2012 to December Turbulence intensity KAL-DEVG (Global) KAL-DEVG (East Asia) 1-segment 30-segment 1-segment 30-segment NIL (97.95%) 809 (76.76%) (97.29%) 335 (76.48%) LGT 400 (1.76%) 188 (17.84%) 281 (2.33%) 81 (18.49%) MOD 57 (0.25%) 49 (4.65%) 40 (0.33%) 20 (4.57%) SEV 9 (0.04%) 8 (0.76%) 5 (0.04%) 2 (0.46%) Total (100%) (100%) (100%) 438 (100%) The percentages are written in parentheses. The East Asia region is indicated by the grey box in Figure 1. Table 5. Counts and percentages of the NIL, LGT, MOD and SEV turbulence obtained from PIREPs at altitudes above ft in East Asia from January 2012 to December Turbulence intensity PIREPs (East Asia) NIL (65.26%) LGT (29.30%) MOD 904 (5.10%) SEV 61 (0.34%) Total (100%) The percentages are written in parentheses. Table 6. Numbers of the LGT, MOD and SEV turbulence obtained from KAL-DEVG on the five major flight routes. Turbulence intensity Asia Oceania Western Europe North America South America LGT MOD SEV only one MOD turbulence event was observed in April. With the exception of April, June, July, August and November, in which there was no SEV turbulence, one to two SEV turbulence encounters were observed per month. In East Asia, the largest count of LGT turbulence (40) was observed in March, with no LGT turbulence observed in January. MOD turbulence was observed most frequently in September and October (7), with no MOG turbulence in January. SEV turbulence was observed once per month in February, March, May, September and October; it was not observed in any of the other months. Due to variances in monthly air traffic, the MOG turbulence was normalized by the total number of turbulence encounters. Both globally and in East Asia, the maximum relative percentage of the MOG turbulence was in February (0.61% and 0.92%, respectively), whereas the minimum MOG turbulence percentage was in June (0.12% and 0.09%, respectively). In the global seasonal variances (Figure 2(b)), the LGT, MOD and SEV turbulence encounters were observed most frequently in the Northern Hemisphere spring months (MAM: March, April, May), autumn months (SON: September, October, November) and winter months (DJF: December, January, February), respectively, whereas they were observed least frequently in the Northern Hemisphere summer months (JJA: June, July, August). Note that the number of flights in JJA was the largest (32) among the four seasons, as shown in Table 2. Therefore, it is clear that the counts of the turbulence do not correspond to flight density. The normalized MOG turbulence was the greatest (0.48%) in DJF and the smallest (0.15%) in JJA. In East Asia (Figure 2(d)), the largest amounts of LGT and MOG turbulence were in MAM (100) and SON (18), respectively, and the smallest were in JJA for both the LGT (50) and MOG (8) cases. For the normalized MOG-level turbulence, the largest relative percentage was 0.52% in SON and the smallest was 0.21% in JJA. The seasonal preferences in the maximum and minimum relative percentages of the MOG-level turbulence encounters were consistent with those from PIREPs over East Asia in the same period of KAL-DEVG (not shown). Figure 3(a) shows the counts of the LGT, MOD and SEV turbulence according to flight density. LGT-level turbulence encounters per flight occurred 16.4, 7.0, 6.2, 5.0 and 1.7 times per year on the North America, Western Europe, Oceania, South America and Asia routes, respectively. In the same order as above, MOD-level turbulence encounters occurred 2.2, 1.3, 0.9, 1.0 and 0.2 times per flight, respectively. SEV-level turbulence encounters occurred 0.5, 0.3, 0.1, 0.5 and 0 times per flight, respectively. On average, six LGT- and one MOD-level turbulence encounters per flight were observed, except on the North America route (which had a much higher frequency) and the Asia route (which had a much lower frequency). However, SEV-level turbulence encounters occurred less than once per flight. Compared with the Asia route, relatively higher frequencies of LGT- and MOD-level turbulence encounters per flight were observed on the North America and Western Europe routes. For SEV-level turbulence encounters, there were relatively high frequencies on the North America and South America routes. This result was related to the longer flight times from Korea. In order to examine the dependency on flight time, the counts of LGT-, MOD- and SEV-level turbulence encounters were normalized by the flight time. Figure 3(b) shows the counts of LGT-, MOD- and SEV-level turbulence encounters normalized by a period of 10 h. The normalized LGT-, MOD- and SEV-level turbulence encounters are most frequently observed on the North America route followed by the Asia route. The normalized LGT (MOD and SEV) turbulence is least frequently observed on the South America (Oceania) route. On average, about six LGT-, one MOD- and less than one SEV-level turbulence encounters were experienced every 10 h. The results in Figure 3 demonstrate that more turbulence encounters occurred on the North America route passing over the North Pacific and on the Asia route passing over the South China Sea in Analysis of possible mechanisms for observed turbulence events Focusing on East Asia, the observed turbulence events were compared with their possible sources. Convection is one of the

5 598 S-.H. Kim and H-.Y. Chun Figure 2. Monthly (a), (c) and seasonal (b), (d) frequencies of turbulence from KAL-DEVG. Upper panels denote global data and lower panels denote data over the East Asia region (indicated by the grey box in Figure 1). The relative percentages of the MOG-level turbulence normalized by the monthly total numbers are depicted as solid lines. The Northern Hemisphere winter, spring, summer and autumn seasons are represented by DJF, MAM, JJA and SON, respectively. For better representation, the numbers of LGT, MOD and SEV turbulence events are multiplied by 10, 50 and 50, respectively. major sources of turbulence, which is referred to as convectively induced turbulence (CIT). This can be divided into out-of-cloud and in-cloud CIT (Lane et al., 2003; Kim and Chun, 2011). In the present study, in order to determine CIT, only strong convective activities were considered using the lightning flash data provided by the Korea Meteorological Administration. Turbulence events observed within 100 km horizontally and ±40 min temporally from the lightning flash data were regarded as CIT, following Kim and Chun (2011). Given that Federal Aviation Administration guidelines suggest that thunderstorms should be avoided by 32 km horizontally and by 1000 ft vertically (FAA, 2008), most of the observed turbulence events are expected to have resulted from out-of-cloud CIT. After determining CIT events, the remaining types of turbulence were considered to be clear air turbulence (CAT) events. It is noteworthy that the classification of CIT events based on lightning flash data is problematic even though it has been used widely in previous studies (e.g. Wolff and Sharman, 2008; Kim and Chun, 2011; Lee and Chun, 2015b). First, turbulence events related to convection without lightning cannot be captured as CIT events. Secondly, given that continental convective cloud produces more lightning than marine convective cloud (Christian et al., 2003), convection-related turbulence events over the ocean are more likely to be missed using this approach. Another possible mechanism generating turbulence is related to the jet stream and upper-level front. From the thermal wind relationship, the strong horizontal temperature gradient near the jet stream and upper-level front can be related to strong vertical wind shear (VWS), which produces a locally low Richardson number (Ri) and increases the likelihood of CAT occurrence caused by shear instability. Considering that there are strong horizontal deformation (DEF) and VWS near the jet stream, turbulence index 1 (TI1 = VWS DEF) was developed by Ellrod and Knapp (1992). They showed that the TI1 index can be used for the diagnosis of CAT events near the upper-level front and the jet stream core. To isolate CAT events related to the shear instability associated with the jet stream, the RDAPS analysis data at 0000, 0600, 1200 and 1800 UTC were used to calculate the TI1 index. If CAT events were located within 50 km horizontally, 3000 ft vertically and ±3 h temporally from any RDAPS grids at which the TI1 index was larger than the thresholds, then these CAT events were considered to be generated by the jet stream. The thresholds of the TI1 index for LGT, MOD and SEV turbulence were s 2, s 2 and s 2, respectively, following Kim and Chun (2012a). In accordance with Min et al. (2011), the shear instability associated with the jet stream is referred to in the present paper as the M1 generation mechanism. In anticyclonic flow, inertial instability related to anticyclonically sheared and curved flow and geostrophic adjustment can be considered to be a possible source of CAT. On this basis, the North Carolina State University turbulence index 2 (NCSU2 = ζ M, where ζ is the vertical vorticity and M is the Montgomery stream function on an isentropic surface) was proposed by Kaplan et al. (2005, 2006). This index was used to diagnose

6 Aviation turbulence encounters detected from aircraft observations 599 Figure 3. Counts of the LGT, MOD and SEV turbulence per (a) flight and (b) 10 h of navigation time along the Asia, Oceania, Western Europe, North America and South America flight routes. For better representation, the counts of the MOD and SEV turbulence events are multiplied by 2 and 3, respectively. CAT events under a strong ageostrophic flow. To classify CAT events related to the inertial instability, the NCSU2 index was used. If CAT events were located within the same spatial and temporal ranges applied to the TI1 index from any RDAPS grids at which the NCSU2 index was larger than the thresholds, then these CAT events were considered to be generated by the inertial instability. The thresholds of the NCSU2 index for LGT, MOD and SEV turbulence were s 3, s 3 and s 3, respectively, following Kim and Chun (2012a). The inertial instability mechanism is referred to as the M2 generation mechanism in the present study. Breaking of mountain waves can also generate CAT. According to Lindzen s linear wave saturation theory (Lindzen, 1981), the amplitude of mountain waves exponentially increases and leads to mountain-wave breaking at higher altitudes and changes in the basic-state flow due to the convergence of the momentum flux as the air density decreases exponentially. When a mountain wave reaches a critical level where the basic-state wind is equal to zero, the mountain wave can also break down. As was done by Kim and Chun (2011), to find CAT events related to mountain-wave breaking, vicinal areas of CAT events caused by mountain waves were determined by mountain-induced gravity wave drag parameterization. The parameterization considered in the present study was based on the work of Chun et al. (1996). A description of the parameterization is introduced briefly below. First, the surface-level stress of a mountain wave was calculated by τ sfc = kρ sfc N sfc U sfc h 2, where the subscript sfc stands for the surface level, k is a tunable co-efficient (= ), ρ is the air density, N is the Brünt Väisälä frequency, U is the basic-state wind which is projected onto the surface wind and h 2 is the variance in subgrid-scale topography height, which is obtained using digital elevation model data with a resolution of 30 s. Secondly, the level of mountain-wave breaking was identified using the minimum Ri (Ri m ) which includes wave effects: Ri m = Ri 1/2 (1 ε)/(1 + εri 1/2 ) 2, where ε is the inverse Froude number ε = (δh)n/u and δh is the displacement amplitude. Alternatively, if Ri m < 1/4, wave breaking occurred and the saturation wave stress was calculated as τ = ε 2 s kρu3 N, where ε s = Ri 1/2 (1 + 2Ri 1/2 )[2Ri 1/4 (1 + 2Ri 1/2 ) 1/2 1]. If Ri m 1/4, wave breaking does not occur and the wave stress is equal to that at the level below. At higher levels, these aforesaid procedures are repeated until the mountain-wave stress equals zero. When the basic-state wind equals zero, it can be seen that the mountain waves are absorbed at that level. From the vertical gradients of the mountain-wave stress, the mountain-wave drag could be obtained. CAT events near the nonzero gravity wave drag induced by a mountain within the same spatial and temporal ranges of the other two indices (TI1 and NCSU2) were considered to be generated by mountain waves. The generation by mountain waves is referred to as the M3 generation mechanism in the present study. Figure 4 shows Venn diagrams of the counts and the relative percentages of turbulence events (excluding NIL turbulence) for four generation mechanisms in East Asia: convection (CIT), shear instability (M1), inertial instability (M2), breaking of mountain wave (M3) and unknown sources from KAL-DEVG. Of the total LGT-level turbulence events, 8.2% (23) and 91.8% (258) were classified as CIT and CAT, respectively. In CAT events, the largest proportion (72.6%; 204) of LGT turbulence was from shear instability, whereas 57.7% (162), 33.8% (95) and 10.7% (30) were associated with inertial instability, mountain waves and unknown sources, respectively. Most LGT-level events occurred simultaneously with both shear instability and inertial instability. Considering the environment of East Asia where the jet stream is dominant, these two generation mechanisms were strongly correlated, as also shown by Kim and Chun (2011). For LGT-level CAT events, 51 cases (18.1%) were associated with three generation mechanisms. For MOD-level turbulence events, 2.5% (1) and 97.5% (39) were associated with CIT and CAT, respectively. For MOD-level CAT events, the largest portion (50%; 20) was related to inertial instability, whereas 42.5% (17), 30% (12) and 27.5% (11) were related to shear instability, mountain waves and unknown sources, respectively. Like LGT-level CAT, inertial instability and shear instability were the major sources of MOD-level CAT. Also, 15% (6) were associated simultaneously with three generation mechanisms. For SEV-level turbulence events, there were no cases of CIT. The five SEV-level CAT events were generated by shear instability (two), inertial instability (two), mountain-wave breaking (one) and unknown sources (two); one SEV-level CAT event was associated with both shear instability and inertial instability, and another with both inertial instability and mountain-wave breaking. 4. Evaluation of the KTG system using KAL-DEVG The operational-ktg system over East Asia was validated using the turbulence encounters from KAL-DEVG. The performance skill of the operational-ktg system was examined using probability of detection (POD) statistics for MOG and NIL turbulence. In computation of POD statistics, a POD of yes (PODY) for MOG turbulence could be assigned when the calculated integrated turbulence index of the operational-ktg system, KTG,

7 600 S-.H. Kim and H-.Y. Chun (a) (b) (c) Figure 4. Venn diagrams of the four generation mechanisms of observed (a) LGT, (b) MOD and (c) SEV turbulence events (excluding NIL turbulence) over East Asia from January 2012 to December The numbers in each category denote the occurrence frequency as a number and as a percentage. Note that CAT events include unknown sources. was larger than a certain threshold value in the grid point closest to the observed MOG turbulence. A POD of no (PODN) for the NIL turbulence was assigned when KTG was less than the threshold value. Note that the integrated turbulence index of the operational-ktg system is KTG, which is the same as the name of the system; in order to distinguish between the system and the index, italics have been used for the index. For the 40 given thresholds, determined by equally dividing the values from the minimum to the maximum of KTG, 40 PODY and PODN statistics were obtained. When the 40 PODY and PODN statistics were described in a PODY PODN curve, the performance of KTG could be measured by the area under the curve (AUC), which represents the skill score obtained from the PODY PODN curve. This method has been used in several other previous studies (e.g. Tebaldi et al., 2002; Frehlich and Sharman, 2004; Knox et al., 2008; Kim et al., 2011, 2015; Kim and Chun, 2012a). Figure 5 shows the frequency of the MOG turbulence by KTG, VWS and the TI1 index, as calculated from the RDAPS analysis data between and ft. These data are superimposed on the zonal wind at 200 mb and the observed MOG turbulence from KAL-DEVG and PIREPs from January 2012 to December PIREPs in South Korea were provided from KAMA and those in the east of the Korean Peninsula were provided from the National Center for Atmospheric Research. Nearly no PIREPs were available on the west of the Korean Peninsula. A higher availability of PIREPs would benefit the development of the KTG system and also aviation turbulence research in this region; if the number of flights were sufficient, flight data might be used to fill in gaps in those regions lacking in PIREPs. The maximum frequency of MOG turbulence by the operational-ktg system was 8.48% near the eastern periphery of the Tibetan Plateau. The occurrence frequencies more than 5% were in the latitudinal band between 30 N and 35 Nin which the major jet stream exists. The values for MOG turbulence between and ft in East Asia were 28 from KAL-DEVG and 350 from PIREPs. As shown in Tables 4 and 5, the frequency of MOG turbulence from KAL-DEVG (PIREPs) in the altitude band between and ft was 62.2% (36.3%) of the total MOG-level turbulence encounters. A large proportion of the MOG-level turbulence encounters from KAL-DEVG fell within the regions of high probability of MOG-level turbulence encounters from the operational-ktg system along the major jet stream. MOG-level turbulence encounters from PIREPs are more widely distributed than those from KAL-DEVG, but, similarly to KAL-DEVG, they exist in the localized region where a frequency occurrence of 6% is predicted from the operational-ktg system. These features can be seen in the results for the VWS and TI1 index that are highly correlated with the jet stream. Stronger VWS occurred at the entrance of the jet core than at the exit of the jet core, whereas the TI1 index was large at both entrance and exit, a result which is more similar to KTG. Considering the fact that the horizontal distribution of PIREPs followed the strong TI1 index at the exit of the jet stream, it is possible that the actual frequency of encountering MOG-level turbulence was much greater than that reported. This could be verified if sufficient PIREPs were to become available in this region. In Figure 5, PIREPs near the southeastern coast of the Kamchatka peninsula are not well captured by KTG, VWS and TI indices. Figure 6 indicates the measured performance of the operational-ktg system against the observational data. The NIL- and MOG-level turbulence encounters from KAL-DEVG and PIREPs above ft over East Asia within ±2 h from 0000, 0600, 1200 and 1800 UTC and the RDAPS analysis data were used in this validation. There were 7513 (7896) NIL- and 23 (554) MOG-level turbulence encounters against KAL-DEVG (PIREPs) in The AUC of the operational-ktg system against KAL-DEVG was (Figure 6(a)) and against PIREPs (Figure 6(b)). Although KAL-DEVG was not included in the construction of the operational KTG system, these AUCs are very close. To examine the statistical significance of the AUCs, additional experiments were conducted using 1000 subsets of randomly selected half-fraction samples from KAL-DEVG and PIREPs. From the AUCs of 1000 experiments, a 95% confidence boundary (Wilks, 1995) was calculated by AUC ± 1.96 ( s n 1 2), where AUC is the mean value of 1000 AUCs, s is the standard deviations of 1000 AUCs and n is the size of samples (= 1000). The 95% confidence boundary for AUCs against KAL-DEVG ranged from to (Figure 6(a)), whereas that against PIREPs ranged from to This observed difference in AUCs of the operational-ktg system between KAL-DEVG and PIREPs was probably due to the smaller number of turbulence encounters from KAL-DEVG, especially for MOG-level turbulence. To examine the sensitivity of the measured performance of the operational-ktg system to the amount of observational data, additional calculations of the AUC against PIREPs were performed that contained 1000 randomly selected subsets with the same number of NIL- and MOG-level turbulence encounters (7513 and 23, respectively) as KAL-DEVG (Figure 6(c)). In Figure 6(c), the median AUC is and the 95% confidence intervals range from to 0.813, which is similar to that against KAL-DEVG. In summary,

8 Aviation turbulence encounters detected from aircraft observations 601 Figure 5. Moderate or greater CAT frequencies derived from (a) the operational-ktg system, (b) the VWS ( s 1 ), and (c) the TI1 ( s 2 ) index between January 2012 and December 2012 at altitudes between and ft, superimposed on the mean wind speed (m s 1 ) at 200 mb (contour lines; every 5 m s 1 for wind speeds 20 m s 1 ) and the locations of the MOG-level turbulence encounters from KAL-DEVG and PIREPs. the results in Figure 6 demonstrate that the measured performance of the operational-ktg system against KAL-DEVG and PIREPs is sufficient for use in aviation turbulence forecasting with AUC values larger than 0.8, as suggested by Sharman et al. (2006) and Kim et al. (2011). Figure 7 shows the assessment results of the operational-ktg system with RDAPS using 6 and 12 h forecasts against KAL-DEVG and PIREPs. The same statistical examinations that were used in Figure 6 were conducted for this assessment. For KAL-DEVG, AUCs for 6 h (FCST06) and 12 h forecasts (FCST12) were and 0.772, respectively. The 95% confidence intervals ranged from to for FCST06 (Figure 7(a)) and from to for FCST12 (Figure 7(b)) from the 1000 subset statistical experiments. For PIREPs, AUCs for FCST06 and FCST12 were and 0.812, respectively. The 95% confidence intervals for FCST 06 and FCST 12 were from to and from and to 0.813, respectively, from the 1000 subset statistical experiments against PIREPs. As 2016 Royal Meteorological Society expected, the measured performance of the operational-ktg system decreased slightly with increasing forecast lead time, but it was not significantly different. Considering the strong dependency of the measured performance of the operational-ktg system on the number of turbulence encounters, as mentioned in Figure 6(c), an increased number of flight data-based turbulence encounters will be required for validation and/or development of the aviation forecast system. 5. Summary and discussion In the present study, aircraft-based turbulence encounters were derived from the Korean Air Lines (KAL) data between January 2012 and December 2012, after applying a series of quality control processes, by calculating the derived equivalent vertical gust velocity (DEVG) in 1 min segments from the original data recorded every second. The KAL data cover five major routes in Asia, Oceania, Western Europe, North America and Meteorol. Appl. 23: (2016)

9 602 S-.H. Kim and H-.Y. Chun (a) (b) (c) 95% upper bound (0.821) 95% lower bound (0.812) 95% upper bound (0.835) 95% lower bound (0.833) 95% upper bound (0.813) 95% lower bound (0.804) Figure 6. PODY PODN performance statistics of the operational-ktg system and the 95% confidence upper and lower boundaries from the 1000 experiments using subsets of randomly selected half-fraction samples, derived using the RDAPS analysis data over East Asia against (a) KAL-DEVG and (b) PIREPs. (c) The same analysis as (b) but against the selected PIREPs that have the same number of NIL and MOG turbulence events as KAL-DEVG. The AUCs are in parentheses in (a), (b) and (c). (a) (b) PIREPs (0.823) PIREPs: 95% upper bound (0.825) PIREPs: 95% lower bound (0.823) PIREPs (0.812) PIREPs: 95% upper bound (0.813) PIREPs: 95% lower bound (0.811) KAL-DEVG: 95% upper bound (0.782) KAL-DEVG: 95% lower bound (0.768) KAL-DEVG: 95% upper bound (0.776) KAL-DEVG: 95% lower bound (0.763) Figure 7. PODY PODN performance statistics of the operational-ktg system and the 95% confidence upper and lower boundaries from the 1000 experiments using subsets of randomly selected half-fraction samples, derived using the RDAPS (a) 6 h and (b) 12 h forecast data over East Asia and KAL-DEVG, along with the PODY PODN performance statistics against PIREPs. The AUC values are in parentheses in (a) and (b). South America. The relative percentages of all severity levels of turbulence were quite similar globally and over East Asia. Light (LGT) (moderate-or-greater (MOG)) turbulence was observed frequently in March (October); the relative percentage of MOG-level turbulence normalized by the total number of turbulence encounters was the highest in February and the lowest in July. In the seasonal distributions, globally and for East Asia, LGT-level turbulence encounters were most (least) frequently observed in the spring (summer), whereas MOG-level turbulence encounters were most (least) frequently observed in the autumn (globally summer; East Asia summer and winter). The normalized MOG-level turbulence encounters were minimum in the summer (both globally and in East Asia) and maximum in the winter (autumn) globally (East Asia). The autumn maximum of MOG-level turbulence encounters in East Asia from KAL-DEVG was consistent with that from pilot reports (PIREPs). Turbulence encounters were distributed spatially along the five major flight routes in Asia, North America, South America, Western Europe and Oceania, with different number of flights and navigation duration for each flight route. LGT and MOG turbulence were observed most frequently on the North America route. When normalized by the number of flights and navigation duration, about five LGT- and slightly more than one MOG-level turbulence encounters occurred per flight, whereas slightly more than four LGT- and one MOG-level turbulence encounters occurred per 10 h of navigation. Among four potential generation mechanisms of aviation turbulence (convection, shear instability, inertial instability and mountain-wave breaking), most of the LGT- and moderate (MOD)-level turbulence events of KAL-DEVG in East Asia were classified as clear air turbulence (CAT) events, which are not related to convective activities, and most CAT events were associated with shear instability and inertial instability. However, there were significant proportions of turbulence events that did not belong to any of the four mechanisms (10%, 27% and 40% of LGT, MOD and severe (SEV) turbulence events, respectively), a result which requires further investigation. Turbulence encounters obtained from KAL-DEVG were used in evaluation of the operational-ktg system in East Asia. Validation of an aviation turbulence forecasting system using flight data has also been conducted previously by Gill (2014), globally, and by Kim et al. (2015), over the contiguous United States. MOG-level turbulence encounters from KAL-DEVG and PIREPs were concentrated in regions where a frequency occurrence greater than 6% was predicted by the operational-ktg

10 Aviation turbulence encounters detected from aircraft observations 603 system between and ft. The skill score (area under the curve (AUC)) of the operational-ktg system based on the Regional Data Assimilation and Prediction System (RDAPS) analysis data against KAL-DEVG was with the 95% confidence level ranging from to in 1000 experiments with subsets of randomly selected half-fraction samples. This score was slightly less than that against PIREPs (0.834), which also had a narrow confidence interval ( ) in the 1000 subset experiments. The differences in the skill score are due to the smaller number of turbulence encounters from KAL-DEVG compared to PIREPs, especially in the number of the MOG-level turbulence encounters. When the operational-ktg system was evaluated using the RDAPS 6 and 12 h forecast data, the skill score decreased as the forecast lead time increased, with AUC values of (0.772) against KAL-DEVG and (0.812) against PIREPs at the 6 h (12 h) forecast. It is noteworthy that there were interannual variations in synoptic patterns, in particular the locations of the jet stream, and these can affect the results of evaluating forecast skill of the operational-ktg system. When an additional validation experiment was performed against 3 year (from July 2011 to May 2014) PIREPs, skill scores of the KTG system (not shown) indeed varied by year (0.811 for June 2011 to May 2012, for June 2012 to May 2013 and for June 2013 to May 2014). Surprisingly, there were no MOG-level turbulence encounters that matched between KAL-DEVG and PIREPs within a circle with a 150 km radius, ±4000 ft vertical range and ±50 min time intervals in East Asia during the whole of 2012, based on the matching methodology in Takacs et al. (2005). This implies that turbulence encounters from the two data sets occurred at different locations and times. Changing the above thresholds for matching the location and time did not make any difference to the results. With this in mind, evaluation of the operational-ktg system using the data set that combines KAL-DEVG and PIREPs was more meaningful. However, due to the much larger number of turbulence encounters from PIREPs than those from KAL-DEVG, the calculated AUC of the operational-ktg system against the combined observed data was close to that against PIREPs (not shown). In the present study, characteristics of turbulence encounters from KAL-DEVG were examined and these data were used to evaluate the forecasting performance of the operational-ktg system in East Asia. Although 1-year data are insufficient to construct seasonal/climatological characteristics of turbulence, considerable numbers of KAL-DEVG turbulence encounters derived from high frequency flight data proved useful in analysis of turbulence and validation of the KTG system. The skill score of the KTG system was >0.8, which is considered to be good for practical utility (Sharman et al., 2006; Kim et al., 2011), but was still less than that against PIREPs. There are two possible reasons for this result. First, the number of turbulence encounters from KAL-DEVG was small, which was confirmed by the additional performance test using PIREPs with the same number of KAL-DEVG turbulence encounters. Second, the operational-ktg system was made using PIREPs, despite the fact that these are nearly absent in China and Russia, as shown in Figure 5. If more turbulence encounters become available from aircraft data, these could be used not only to evaluate any existing aviation turbulence forecasting system, but also to construct new aviation turbulence forecasting systems. Given that there are no turbulence encounters that are matched between KAL-DEVG and PIREPs, even within relatively wide spatiotemporal boundaries, this highlights the need for aircraft-based turbulence observations to overcome the uncertainties in turbulence encounters from PIREPs that rely on the inevitably subjective judgement of pilots. Acknowledgements This work was funded by the Korea Meteorological Administration Research and Development Program under Grant KMIPA The authors would like to thank Dr Philip G. Gill for many helpful suggestions. References Christian HJ, Blakeslee RJ, Boccippio DJ, Boeck WL, Buechler DE, Driscoll KT, et al Global frequency and distribution of lightning as observed by the optical transient detector. J. Geophys. Res. 108(D1): Chun H-Y, Jung J-H, Oh J-H, Kim J-W Effects of mountain-induced gravity wave drag on atmospheric general circulation. J. Korean Meteorol. Soc. 32: Cornman LB, Meymaris G, Limber M An update on the FAA Aviation Weather Research Program s in situ turbulence measurement and reporting system. Preprints, 11th Conference on Aviation, Range, and Aerospace Meteorology, American Meteorological Society, Hyannis, MA. Ellrod G, Knapp D An objective clear-air turbulence forecasting technique: verification and operational use. Weather Forecast. 7: FAA Aeronautical Information Manual. Federal Aviation Adminstration: Washington, DC; 670. Frehlich R, Sharman RD Estimates of turbulence from numerical weather prediction model output with applications to turbulence diagnosis and data assimilation. Mon. Weather Rev. 132: Gill PG Objective verification of World Area Forecast Centre clear air turbulence forecasts. Meteorol. Appl. 21: Gill PG, Buchanan P An ensemble based turbulence forecasting system. Meteorol. Appl. 21: Gill PG, Stirling AJ Including convection in global turbulence forecasts. Meteorol. Appl. 20: Kaplan ML, Charney JJ, Waight KT, Lux KM, Cetola JD, Huffman AW, et al Characterizing the severe turbulence environments associated with commercial aviation accidents. A real-time turbulence model (RTTM) designed for the operational prediction of hazardous aviation turbulence environments. Meteorol. Atmos. Phys. 94: Kaplan ML, Huffman AW, Lux KM, Cetola JD, Charney JJ, Riordan AJ, et al Characterizing the severe turbulence environments associated with commercial aviation accidents. Part 2: Hydrostatic mesoscale numerical simulations of supergradient wind flow and streamwise ageostrophic frontogenesis. Meteorol. Atmos. Phys. 88: Kim J-H, Chan WN, Sridhar B, Sharman RD Combined winds and turbulence prediction system for automated air-traffic management applications. J. Appl. Meteorol. Climatol. 54: Kim J-H, Chun H-Y A numerical study of clear-air turbulence (CAT) encounters over South Korea on 2 April J. Appl. Meteorol. Climatol. 49: Kim J-H, Chun H-Y Statistics and possible sources of aviation turbulence over South Korea. J. Appl. Meteorol. Climatol. 50: Kim J-H, Chun H-Y. 2012a. Development of the Korean Aviation Turbulence Guidance (KTG) system using the operational Unified Model (UM) of the Korea Meteorological Administration (KMA) and pilot reports (PIREPs). J. Korean Soc. Aviat. Aeronaut. 20: Kim J-H, Chun H-Y. 2012b. A numerical simulation of convectively induced turbulence above deep convection. J. Appl. Meteorol. Climatol. 51: Kim J-H, Chun H-Y, Sharman RD, Keller TL Evaluations of upper-level turbulence diagnostics performance using the Graphical Turbulence Guidance (GTG) system and pilot reports (PIREPs) over East Asia. J. Appl. Meteorol. Climatol. 50: