NTDA Operational Demonstration. Turbulence PDT Task FY 2005 Year-End Progress Report

Transcription

1 NTDA Operational Demonstration Turbulence PDT Task FY 2005 Year-End Progress Report Deliverable E3 Submitted by The National Center for Atmospheric Research

2 Introduction In FY 2005, NCAR was given Technical Direction by the FAA s Aviation Weather Research Program Office to perform an operational demonstration of the NCAR Turbulence Detection Algorithm (NTDA). This work was performed under the Turbulence PDT Task , which states: Develop and verify NTDA data gridding, merging, and hazard region identification methods. Code NTDA in C for real-time operation. Leveraging software developed for Convective Weather and Oceanic Weather PDTs, implement real-time NEXRAD data ingest, processing, and display via website and text-based hazard map cockpit uplink. Purchase and configure hardware for required data processing and storage. Verify product performance via comparison with in-situ data and PIREPs, and solicit feedback. The implementation of this real-time system and collection of data over an entire summer should help facilitate the implementation of the NTDA in the NEXRAD Open Radar Products Generator (ORPG), and help develop compelling verification information. Both of these are prerequisites to having the NTDA approved by the NEXRAD Technical Advisory Committee and Software Recommendation and Evaluation Committee for installation on the NEXRAD system, an operational network of over 150 Doppler weather radars jointly developed and operated by the National Weather Service, the Department of Defense, and the FAA. The present report is provided in fulfillment of Turbulence PDT Deliverable E3: (Sep 05) Report, conference paper, or refereed journal article on results of the NTDA operational demonstration. The attached conference paper, entitled Real-time remote detection of convectively-induced turbulence, satisfies this requirement. It will be presented at the 32 nd AMS Conference on Radar Meteorology, October 2005, Albuquerque, NM.

3 P12R.1 REAL-TIME REMOTE DETECTION OF CONVECTIVELY-INDUCED TURBULENCE John K. Williams*, Larry Cornman, Jaimi Yee, Steven G. Carson, and Andrew Cotter National Center for Atmospheric Research, Boulder, Colorado 1. INTRODUCTION Although the connection between Doppler spectrum width and turbulence intensity has long been recognized, the spectrum width s sensitivity to low signal-to-noise ratios, overlaid echoes, and contamination by non-atmospheric scatterers has limited the utility of Doppler weather radars for automated turbulence detection. Under direction and funding from the FAA s Aviation Weather Research Program, these issues have been addressed through the development of the NCAR Turbulence Detection Algorithm (NTDA), a fuzzy-logic algorithm that uses radar reflectivity, radial velocity, and spectrum width to perform data quality control and compute estimates of eddy dissipation rate (EDR), an atmospheric turbulence metric, and associated confidence. A real-time demonstration of the NTDA using Level II data from sixteen NEXRADs in the upper Midwest is currently underway, producing a 3-D mosaic of turbulence intensity at five-minute intervals. This demonstration is the first step of a planned implementation that will offer turbulence detection over the conterminous United States, providing airline meteorologists, dispatchers, air traffic controllers, and pilots a new tool for identifying areas hazardous to aviation safety and helping to reduce delays and loss of capacity due to convective activity. Additionally, the NTDA data may be of use to forecasters, modelers, and those working to understand thunderstorm evolution. This paper presents an overview of the NTDA, describes the operational demonstration, and discusses verification of the NTDA s performance based on comparisons with in situ turbulence data provided by United Airlines aircraft via the FAA s automated EDR downlink system. control indices, or confidences, on the radar s polar grid. Every 5 minutes, the most recent NTDA output from all the radars were combined into a 3-D mosaic having a horizontal resolution of 0.02 in latitude and longitude (approximately 2 km) and vertical levels at multiples of 3,000 ft. up to 45,000 ft. Radar reflectivity and NTDA confidence values were mosaicked using the same technique. A web-based Java display similar to the visualization tool in Experimental ADDS ( allows users to view the NTDA EDR mosaic with confidence thresholding, EDR values without thresholding, the confidences values themselves, or the associated radar reflectivity mosaic. Several Experimental ADDS products may also be viewed, including Graphical Turbulence Guidance (GTG) turbulence forecasts, temperature, humidity, and wind speed. Wind barbs and pilot reports (PIREPS) may optionally be overlaid, as can real-time in situ turbulence data obtained via the FAA s automated turbulence reporting system installed on United Airlines Boeing 737s and 757s (Cornman et al., 1995 and 2004). For purposes of display, the EDR is scaled into a turbulence severity category of Smooth, Light, Moderate, Severe or Extreme based on its approximate hazard to a medium-sized (e.g., Boeing 757) aircraft flying at cruise speeds (see Figure 1). In general, smaller aircraft may be expected to experience more intense turbulence, while larger aircraft or slower airspeeds may lessen atmospheric turbulence effects. 2. NTDA OPERATIONAL DEMONSTRATION On 4 June 2005, NCAR began an operational demonstration of a turbulence remote sensing capability based on the NCAR Turbulence Detection Algorithm (NTDA). WSR-88D (NEXRAD) Level II data, comprising radar reflectivity, radial velocity, and spectrum width, were obtained from Integrated Radar Data Services (IRaDS) via Internet 2 and ingested using Unidata s Local Data Manager (LDM). Sixteen NEXRADs in the upper Midwest were used, covering a region from eastern Iowa and Missouri to western Pennsylvania and West Virginia and from southern Wisconsin and Michigan to northern Tennessee. Data from each radar sweep were processed using the NTDA to obtain eddy dissipation rate (EDR) estimates and associated quality * Corresponding author address: John K. Williams, National Center for Atmospheric Research, P.O. Box 3000, Boulder, CO 80307; jkwillia@ucar.edu. Figure 1: Interactive Java display for disseminating the NTDA operational demonstration data, shown for 0:40 UTC on 27 July Overlaid are in situ turbulence values reported by United Airlines B-757 aircraft.

4 Beginning on 29 July 2005, a demonstration system for producing custom text-based maps of turbulence ahead began generating messages every 5 minutes for all United Airlines aircraft penetrating the demonstration domain. Aircraft Situation Display to Industry (ASDI) data were used to determine the position and route of each aircraft and to predict its future path. NTDA mosaic data were then resampled onto an appropriate grid, approximately 115 nm ahead and 40 nm to either side of the predicted flight path, and turbulence hazard categories were represented by characters to produce the 2-D, text-based maps. The filed flight plan and relevant waypoints were also displayed. Messages containing significant turbulence were uplinked to the cockpit Aircraft Communications Addressing and Reporting System (ACARS) printers on select, preregistered flights. Although systematic verification of the uplink product has not yet been performed, initial pilot feedback has been generally positive. The turbulence messages and uplink demonstration will be described in detail in a future paper. 3. NCAR Turbulence Detection Algorithm (NTDA) The NCAR turbulence detection algorithm (NTDA) utilizes the first three moments of the Doppler spectrum the reflectivity, radial velocity, and spectrum width to perform data quality control and produce EDR estimates on the same polar grid as is used for the raw moment data (see Figure 2). A data preprocessing step combines reflectivity and Doppler moment data from split cut tilts (e.g., lower elevation angles where the long-prt reflectivity and short-prt Doppler sweeps are commonly separate). Data quality control is performed in two steps: data censoring and data confidence computations. Data censoring removes measurements contaminated by solar radiation and other sources. The confidence computations make use of a computed signal-to-noise ratio (SNR), overlaid power ratio, Radar Echo Classifier (Kessinger et al. 2003) clutter interest, the spectrum width field texture, radar reflectivity, range from the radar and height above ground, and local averaging volume coverage by usable data. An EDR estimate for each measurement point is computed by scaling the spectrum width according to a theoretical quantity dependent on range, and local confidenceweighted averages are computed to produce the final EDR and confidence. These steps are described in greater detail below. It is anticipated that a future version of the algorithm will also make use of radial velocity measurements. 3.1 Data censoring Values of VE and SW that are so contaminated as to be useless are removed before any calculations are performed. The first step of this process involves using image processing techniques to identify regions of bad data including ring artifacts in VE and SW fields like the one shown in Figure 3. These rings typically consist of 4 or more range gates containing repeated values in the VE and SW fields, with this pattern being repeated over several adjacent azimuths (though the repeated values themselves may be different for each beam). The fact that the artifact is duplicated in the VE and SW fields makes automated identification relatively straightforward, with little danger of removing good data. A second censoring step removes sun spikes, a form of contamination due to solar radiation that occurs when the radar is pointed near the sun. Sun spikes typically contain increased reflectivities, noisy radial velocities and elevated spectrum widths over one or more beams, causing contamination similar to the radial artifact in Figure 3. The NTDA removes sun spikes by computing the apparent position of the sun based on date and time and censoring beams within an azimuthal interval of ±2.0 and elevation within ±1.75. These intervals were determined empirically by examining over seven hundred incidents of sun spike contamination via plots like the one shown in Figure 4 and recording the displacements between the azimuth and elevation of the contaminated beams and the sun s computed position. The censoring intervals chosen are sufficient to remove nearly all of the sun spike contamination. 3.2 Data quality control Data quality control is accomplished by assigning a confidence to each spectrum width measurement. The confidence is assembled by combining interest values, or proto-confidences, based on a number of measured or computed quantities. The spectrum width confidence values are then used to give greater weight to higher-quality data in the final EDR computation, and they also provide input to the final EDR confidence calculation. A description of the relevant quantities and the functions that are used to map them to confidence values is presented below. WSR-88D Level-II Data DZ reflectivity VE radial velocity SW spectrum width Intermediate quantities SNR signal-to-noise ratio PR overlaid power ratio REC Radar Echo Classifier SWT Spectrum width texture Censored SW Censored VE Confidence computations VE confidence SW confidence Polar grid EDR and confidence Turbulence (EDR) EDR confidence Figure 2: Diagram of the NTDA, as implemented for the WSR-88D (NEXRAD) radar. The Level II reflectivity, radial velocity and spectrum width data are used to censor bad data and compute EDR and an associated confidence for each radar measurement point via a fuzzy-logic framework. A pre-processing step that isn t shown merges split-cut tilts (e.g., surveillance and Doppler sweeps) so that the reflectivity, radial velocity, and spectrum width data are on a common grid.

5 1 0.5 Figure 3: NEXRAD SW data showing a ring artifact and also two contaminated beams that appear similar to a sun spike (though this particular artifact is likely due to another cause). A ring exists in the same location in the VE data (not shown). If not properly censored, the elevated spectrum widths in these artifacts give rise to regions of spurious high EDR estimates. Figure 4: Example of sun spike contamination of NEXRAD reflectivity (top), radial velocity (middle) and spectrum width (bottom). The y-axis represents beam azimuths from 60.8 to 71.6 and the x-axis depicts range gates from 0 to 800 (0 to 200 km). This example is from a 0.5 elevation sweep from KAMA at 11:43 UTC on 16 May 2002, during the International H 2O Project (IHOP). Figure 5: Form of piecewise-linear interest map (or membership function ) used to convert certain measured or computed quantities into a confidence, or quality control, value. The choice of the parameters α 1, α 2, α 3, and α 4 depend on the input quantity. Several confidence values are combined to form the final spectrum width confidence value SNR computation The signal-to-noise ratio (SNR) in db is estimated for every Doppler measurement point via the formula SNR( r ) db ( ) 20log 230 r = Z r + 10 where dbz denotes the (nearest) measured reflectivity and r is the range from the radar in km. This expression does not produce the true SNR of the radar signal, but it does provide a quantity representative of the magnitude of the returned power that is useful for quality control. In general, radial velocity and spectrum width estimates obtained from signals with large values of SNR are subject to less estimation error. However, excessively high values of SNR may be indicative of contamination from a non-atmospheric source. The interest map used to convert SNR to a confidence interest value, C SNR, for spectrum width is the piecewise-linear function depicted in Figure 5. In the operational demonstration, the values of α 1, α 2, α 3, and α 4 were chosen as 10, 20, 70 and 80 db, respectively PR computation α 1 α 2 α 3 α 4 Input value The overlaid power ratio (PR) is the ratio of the echo power for a given measurement point to the sum of all overlaid echo powers, i.e., echo powers for ranges differing from the given point by multiples of the unambiguous Doppler range, R max. If SNR on a linear SNR /10 scale is denoted SNR = 10 and is taken to be lin zero wherever SNR is undefined, then PR may be expressed via the formula SNR lin( r) db if SNR lin( r krmax ) 0 PR( r) SNR lin( r krmax ) > = k 0 (2) k 0 otherwise (1)

6 Doppler moments (radial velocity and spectrum width) are censored by the NEXRAD Radar Data Acquisition (RDA) system whenever their signal power does not exceed the overlaid echo power by a specified margin (Doviak and Zrni 1993). However, in practice this margin is usually chosen to eliminate poor velocity estimates, whereas spectrum width estimates are more easily contaminated by overlaid echoes. Thus, PR may be used to further discriminate the quality of the measurements. The interest map used to convert PR to a spectrum width confidence interest value, C PR, has the form depicted in Figure 5 with α 1 and α 2 equal to 15 and 25 db, respectively (α 3, and α 4 are not used) REC clutter interest The NEXRAD RDA utilizes signal processing algorithms to attempt to remove the effects of ground or sea clutter identified by a pre-determined clutter mask or by the radar operator (Doviak and Zrni 1993). However, under certain meteorological conditions, an unusually high refractivity gradient may cause the radar beam from low elevation tilts to bend more than usual, illuminating surface objects and causing more widespread clutter. This anomalous propagation may not always be properly identified and corrected for, and may cause contamination of the radar measurements. The Radar Echo Classifier (REC; Kessinger et al. 2003) is a fuzzy-logic algorithm that uses local features of the reflectivity, radial velocity, and spectrum width fields to discriminate weather echoes from measurements contaminated by ground clutter. The NTDA utilizes the clutter interest map from the REC, a value between 0 and 1 that roughly represents the likelihood that a given measurement location is contaminated by clutter. The clutter map is converted to a spectrum width confidence interest value, C AP, via a piecewise linear map with α 1, α 2, and α 3 equal to 0.25, 0.75, and 1, respectively (α 4 is not used) Spectrum width texture If one assumes that the true wind field spectrum width (i.e., the weighted standard deviation of radial velocities over a radar illumination volume) changes only slowly in space, then a local standard deviation of the measured spectrum width may provide a good approximation to the spectrum width estimation error. Thus, a high local variance in spectrum width values reflects poor quality measurements. In the present implementation, the variance is computed over a rangeazimuth patch of radius 1.0 km, and the result is mapped into C SWV via the function in Figure 5 with α 2, α 3, and α 4 equal to 0, 4, and 16 m 2 s -2, respectively (α 1 is not used). Of course, the variance of the spectrum widths will also be large in regions where the turbulence intensity is changing rapidly. To ameliorate this problem, the variance of the residuals from a local linear fit of the spectrum width values is also computed. The residuals should be small if the variability in the spectrum width is due to a smooth change, but may be large if considerable nonlinearity or random noise is present. On the other hand, the linear regression inevitably fits some of the noise, so the residual variance is always smaller than the true variance. The linear fit is computed over a radius of 1.0 km, and the confidence value C SWLV is determined via Figure 5 with α 2, α 3, and α 4 equal to 0, 2, and 8 m 2 s -2, respectively (α 1 is not used). Finally, the linear fit residual value at the center point divided by the residuals standard deviation (the residual value s z-score ) may indicate whether that point is an outlier relative to neighboring spectrum width values. However, small standard deviations can lead to misleading results, so a standard deviation less than 0.5 m s -1 is replaced with 0.5 m s -1 before the z-score is computed. The modified z-score-based confidence, C SWLZ, is computed via Figure 5 with α 2, α 3, and α 4 equal to 0, 2, and 4, respectively (α 1 is not used) Radar reflectivity and height When looking at summertime data from the Midwest, very high values of spectrum widths are commonly observed in the boundary layer in lowreflectivity or clear air conditions. It is hypothesized that these spurious values result from swarms of insects and birds, which do not act as good tracers of the wind field. Hence, a confidence interest function was designed to provide low confidence to measurements near the ground having low reflectivity, with lower reflectivities being permitted as the height above ground increased. Here the height of a measurement point above the ground, HT, is determined using a fast but accurate approximation to the 4/3 earth model (Doviak and Zrni 1993) to account for beam-bending. The confidence value, C ZH, is determined using a twodimensional interest function that maps the quantity dbz HT using the function in Figure 5 with α 1 and α 2 equal to 15 and 25, respectively (α 3 and α 4 are not used). The resulting interest map is depicted in two dimensions in Figure 6. Figure 6: Two-dimensional interest map for reflectivityheight spectrum width confidence as a function of the measurement s height above ground (km) and radar reflectivity (dbz). This confidence calculation is useful in removing contamination due to insects and birds.

7 3.2.7 Measurement range from radar As the range from the radar increases, the quality of measured spectrum widths tends to decrease due to the weaker returned power, resulting in generally values of lower SNR and PR. These effects have already been accounted for. In addition, the beam broadens, so that measurements are derived from illumination volumes of increasing radius. Because ambient wind speed tends to change directions and increase with height, this broadening can create an increasingly large shear in the radial velocity that results in enhanced values of spectrum width. In principle, it may be possible to compute the vertical shear in the radial velocity field and adjust the spectrum widths to remove its effect. However, methods for doing so robustly have not yet been identified or implemented in the NTDA. As a substitute, the spectrum width confidence for values having a large measurement range from the radar is lowered. The value of the range-derived confidence, C RNG, is computed by mapping the measurement range according to Figure 5 with α 1, α 2, α 3, and α 4 equal to 0, 5, 140, and 275 km, respectively Final spectrum width confidence calculation In order to produce a single confidence value, C SW, for each spectrum width measurement, the values of C SNR, C PR, C REC, C SWV, C SWLV, C SWLZ, C ZH, and C RNG are combined using a method similar to a geometric average, but with different exponents. Thus, if any one of these confidences is zero, the final spectrum width confidence becomes zero. The formula with exponents used in the operational demonstration was C = C C C C C C C C (3) 2 / SW SNR PR REC SWV SWLV SWLZ ZH RNG Note that exponents of 0 appear for values that were not used in the summer 2005 demonstration. For split cuts (for most VCPs, the surveillance and Doppler sweeps at 0.5 and 1.5 ), the computation in (3) was performed in two steps. First, the product involving the confidences 2/3 derived from radar reflectivity (C SNR C PR C ZH) was averaged over a disc having radius 1.5 km. This averaging was done to accommodate the imperfect alignment between the reflectivity and Doppler azimuths, as well as the advection that may occur between the surveillance and Doppler sweeps. The averaged values were then combined with the product involving the other confidences Local coverage calculation Because turbulence is a statistical quantity, it can only be meaningfully computed by averaging a number of measurements having some degree of independence. If the number of measurements is small or they have significant dependence, the turbulence estimate may not be accurate. Furthermore, a region in which many spectrum width values are missing is typically suspect, since the remaining values may be contaminated by the same source that led to their neighbors being censored, though possibly to a smaller degree. Hence, another indicator of the final averaged EDR confidence is given by the local coverage by valid spectrum width measurements. More specifically, the ratio of the number of valid spectrum width measurements to the total number of points in a disc of radius 2.0 km is passed through a piecewise linear interest map to yield the coverage confidence, C COV, by taking α 2, α 3, and α 4 equal to 0, 0.3, and 0.6, respectively (α 1 is not used). 3.3 Raw EDR calculation For any spectrum width measurement that has not been removed by censoring or assigned zero confidence, a raw (as opposed to final) eddy dissipation rate (EDR) is estimated by multiplying the spectrum width by a scaling factor dependent on range and turbulence outer length scale: EDR raw( r ) = SW( r ) f ( r, L o ) (4) The scaling factor is computed via a theoretical formula derived under the assumptions that the radar illumination function is a 3-D Gaussian, that radar reflectivity is uniform within the illumination volume and that the turbulence has a von Kármán energy spectrum with a specified outer length scale, L o (Cornman and Goodrich 1996). Figure 7 shows the dependence of the scaling factor f on range for three choices of L 0. The version of the NTDA run in the operational demonstration used a fixed value L o = 500 m. 3.4 Final EDR and confidence calculation The NTDA produces final EDR estimates and associated confidences on a polar grid having the same azimuths as the Doppler moment data but decimated ranges. For the operational demonstration, values were produced at multiples of 1 km (every fourth Doppler range gate) up to the range at which the radar beam exceeded an MSL altitude of 55,000 ft. The final EDR value is computed by taking a local average, weighted by the associated spectrum width confidences, of the raw EDR values. If any value of C SW in a disc D around the output point r is nonzero, this may be written EDR( r ) i D SW i = i D C ( r )EDR ( r ) C SW raw ( r ) i 2 i ; (5) otherwise, EDR(r) is labeled as missing. Here the sum over D includes only those points for which EDR raw is defined. The final confidence is determined similarly as the average spectrum width confidence, multiplied by the coverage confidence: CSW ( r ) i D i C( r ) = CCOV ( r ) (6) 1 i D In the operational demonstration, an averaging disc of radius 2 km was used. Another possibility being considered for a future version of the NTDA is to use a confidence-weighted average confidence in place of the average confidence in (6).

8 A significant limitation of the NTDA is that it is only able to detect turbulence within in-cloud regions where radar reflectivity is sufficiently strong to produce reliable Doppler measurements, and where signal contamination due to ground clutter, second-trip echoes, or other factors is minimal. It is therefore typical for much of the TRS Viewer display to be shaded grey, representing No Data, as in Figure 1. It should be emphasized that No Data does not mean no turbulence ; rather, no information at all is provided by the NTDA in these regions. Figure 7: Plots of the SW to EDR scaling function f(r, L o) for the WSR-88D (NEXRAD) as a function of r for L o = 250, 500, and 1000 m, respectively. For the operational demonstration, L o = 500 m was used. 3.5 Mosaicking technique In the real-time demonstration, the NTDA was run on Level II data from sixteen NEXRADs in the upper Midwest, producing polar coordinate outputs of estimated EDRs and confidences for each radar tilt. Every five minutes, the latest data for each radar tilt was ingested into a mosaicking algorithm, which produced the 3-D gridded EDR and confidence data disseminated via the web-based Java display shown in Figure 1. Reflectivity data were mosaicked similarly and also made available via the Java display to supply users a meteorological context for the EDR estimates. The mosaicking algorithm works by first computing the latitude, longitude and MSL altitude of each relevant polar-coordinate radar data point, adjusting for beam bending using a standard model. It then loops through all the points on the Cartesian grid, computing a distance- and confidence-weighted average of nearby radar measurements at each location. Measurements having confidence less than 0.01 are ignored. The distance weighting function used in the operational demonstration is a 3-D Gaussian with a vertical standard deviation σ v = 1,500 ft. and a horizontal standard deviation σ h = 2 km. This computation may be represented by the following equation, where M is the mosaic value, ε(i) is a radar-derived measurement at (x(i), y(i), z(i)), and C(i) is its associated confidence: ( x( i ) x ) + ( y( i ) y ) ( z( i ) z ) 2 2 2σh 2σv ε( i) C( i)e { i: C( i ) > 0.01} M( x, y, z) = ( x( i ) x ) + ( y( i ) y ) ( z( i ) z ) 2 2 2σ h 2σv C( i)e { i: C( i ) > 0.01} The averaging volume is truncated at 3 σ v in the vertical and 4 σ h in the horizontal to speed the computation. An associated distance-weighted average confidence is computed for each grid point using a similar formula. In creating reflectivity mosaics, the confidence of every radar reflectivity measurement is taken as 1. (7) 4. NTDA VERIFICATION The availability of archived NEXRAD Level II data from the National Climate Data Center (NCDC) via a web-based interface has made it feasible to run the NTDA for any in-cloud turbulence case in which in situ turbulence data are available for comparison. For example, in the spring of 2002, a series of eleven flights were performed by the instrumented NASA Langley B- 757 aircraft as part of a successful test of an airborne radar turbulence detection algorithm developed by NCAR for the NASA Aviation Safety and Security Program s Turbulence Prediction and Warning Systems project (Cornman et al. 2003). The high-rate winds data recorded by the aircraft comprise a high-quality in situ dataset that is also ideal for evaluating the performance of the NTDA, run on archived Level II data from NEXRAD radars along the flight paths. The B-757 s 20 Hz vertical winds data were used to estimate EDR using a single parameter maximum likelihood -5/3 model that assumes a von Karman energy spectrum form (Cornman et al. 2004). In particular, a sliding window of width 256 points was used, with spectral frequency cutoffs set at 0.5 and 5 Hz. This temporal window corresponds to an alongpath distance of about 3 km at the aircraft s average cruising speed, but the windowing function applied before taking the Fourier transform reduces the effective path somewhat. Comparisons between the aircraft data and the results from the NTDA were performed by locating the nearest three NEXRAD radar sweeps in space and time to each aircraft location. A comprehensive series of overlay plots were generated to permit comparison between the aircraft EDR and the EDR computed by the NTDA for each sweep. Two sample plots are depicted in Figure 8 and Figure 9. Although precise collocation and quantitative matches were not achieved, both overlay plots show that the NTDA successfully detected hazardous turbulence of about the right intensity in the region of the aircraft encounter. Moreover, both encounters were in regions of relatively low reflectivity (< 30 dbz in Figure 8 and < 15 dbz in Figure 9) where commercial aircraft commonly fly. On the other hand, no radar moments data were available near the location of the moderate turbulence encounter northeast of the severe encounter in Figure 9 due to the absence of sufficiently dense cloud there.

9 Figure 8: In situ EDR values depicted along the aircraft track for NASA flight 230, 19:22:00-19:29:15 UTC, superimposed over the NTDA EDR values from the KLTX 2.4 elevation sweep beginning at 19:25:26 UTC. Both EDR values are a scale ranging from 0 (blue) to 0.7 m 2/3 s -1 (red). The labels on the range rings and the axes represent the distance from KLTX, in km. The aircraft is within about 1 km vertically of the sweep throughout this flight segment, and the radar reflectivity ranges from about 5-30 dbz within the turbulent region. Figure 9: Identical to Figure 8 for NASA flight 232, 18:54:51-19:01:23 UTC, and KFFC 2.4 elevation sweep beginning at 18:57:51 UTC. The aircraft is again within about 1 km of the sweep, and the radar reflectivity ranges between about 5-15 dbz in the region where the aircraft track intersects the radar-detected turbulence. Figure 10: Stacked track plot for NASA flight 230, 20:07:10-20:12:30 depicting the colorscaled timeseries of aircraft EDRs ( AC, bottom stripe) and the 2-km disc median NTDA EDRs from the three nearest sweeps of radars KAKQ, KCAE, KCLX, KFCX, KLTX, KMHX, and KRAX. Gray indicates that the radar was out of range, whereas white depicts times for which a radar sweep was within range but contained no usable data. The EDR color scale ranges from 0 to 1 m 2/3 s -1. A second level of processing was performed to extract the median NTDA EDR value from a disc of radius 2 km around each aircraft location as projected onto each nearby radar sweep (time within 3 minutes and vertical displacement less than 3 km). A timeseries plot was designed to visualize the EDR values from the three nearest tilts on nearby radars and compare them with the aircraft EDRs. An example of such a stacked track plot for NASA flight 230, 20:07:10-20:12:30 UTC is shown in Figure 10. Note that the four radars that provide coverage of this turbulence encounter generate similar EDR estimates and that these match well with the co-located in situ values, providing compelling evidence of the NTDA algorithm s skill. The set of timeseries, overlay, and stacked-track plots generated by the analysis described above were used to subjectively score the ability of the NTDA to detect moderate-or-greater turbulence encountered by the aircraft from 55 flight segment events over the eleven flights of the NASA flight test. A similar scoring exercise performed using the output of the airborne radar turbulence detection algorithm identified 34 correct detections, 8 misses, 4 nuisance alerts, and 9 correct nulls (Cornman et al. 2003), producing a probability of detection (PoD) of 81% with a nuisance alert rate of 11%. For the NTDA analysis, 15 events had no available archived NEXRAD data intersecting them. Of the remaining 40, preliminary scoring identified 32 correct detections, 2 misses, 6 nuisance

10 alerts, and no correct nulls, yielding a PoD of 94% and a NAR of 16%. This analysis suggests that the NTDA may have skill comparable to that of the airborne radar algorithm for detecting hazardous turbulence. However, it should also be noted that research flights aimed specifically at encountering turbulence may not provide a dataset representative of the conditions experienced by commercial aircraft in an operational environment, and so care must be taken in interpreting these results. 5. NTDA MOSAIC VERIFICATION While research aircraft data cases like those described above provide valuable information on the NTDA s ability to predict hazardous turbulence encounters, the available data are insufficient to draw statistically meaningful conclusions. On the other hand, the FAA s automated in situ turbulence reporting system (Cornman et al and 2004) is currently installed on about 200 United Airlines B-737 and B-757 aircraft, and efforts are underway to deploy it on additional aircraft types and airlines in the near future. The in situ turbulence algorithm downlinks median and peak EDR values measured at intervals of one minute or less, thereby supplying several hundred flight hours per day of objective turbulence measurements in locations and conditions where commercial aircraft commonly fly. The comparison with NTDA turbulence mosaic output was accomplished subjectively during the operational demonstration by viewing the in situ EDR data overlaid on the NTDA EDR grid, as depicted in Figure 1. An objective statistical comparison was also performed, as described below. 5.1 Comparison methodology The relatively high resolution of the NTDA mosaic product (approximately 2 km horizontally), coupled with the dynamic nature of convectively-induced turbulence, presents a challenge in performing verification. For example, an attempt to perform comparisons with pilot reports (PIREPs), the standard used for the evaluation of the FAA s Graphical Turbulence Guidance product (previously called ITFA, Sharman et al. 2002), failed to produce meaningful results due to the substantial uncertainties in the reports times and positions. This problem is significantly lessened for the automated in situ turbulence reports. Nevertheless, each in situ EDR report is based on data not only at the aircraft s reported position, but also on all positions through which it traveled since the previous report. Furthermore, since the NTDA grids are being generated only every five minutes, advection and evolution of the storm during that timespan may be significant. The comparison methodology used in this study handled the first issue by gathering statistics of gridded NTDA values lying within a volume surrounding the aircraft s presumed flight path a straight line segment between subsequent position reports. The second was addressed by selecting a volume size deemed large enough to compensate for advection, and also by using the first NTDA mosaic generated after an in situ EDR measurement for comparison. The volumes used were approximately rectangular solids along the major axis, they followed the great circle connecting the two flight path points, while extending perpendicular to the flight path in the horizontal and vertical. For each path segment, data were collected from both the NTDA EDR and reflectivity grids and associated with the in situ EDR measurements recorded at the end of the segment. All valid in situ reports above 1,500 ft. in the operational demonstration domain between 5 June and 5 October 2005 were compared with the archived NTDA mosaic data. The volumes used extended 3,000 ft. above and below the flight path, thereby encompassing the nearest two mosaic levels, and 10 km to either side. Various statistics were calculated from the grid data values falling within the volume, but only results for the median values are displayed below. 5.2 Results Figure 13 - Figure 14 show statistical analyses in the form of box plots for each in situ EDR level and Receiver Operating Characteristic (ROC) curves for NTDA detection of various levels of aircraft turbulence. Figure 13 shows comparisons between the in situ average EDR and the NTDA EDR mosaic median taken over the volume within 3,000 ft. vertically and 10 km horizontally of the relevant flight path segment. The box plots show the distribution of the median NTDA EDRs corresponding to different measured in situ EDR values, which are quantized at 0.05 m 2/3 s -1 intervals and are listed across the x-axis. These comparisons show good correlation but substantial spread in the NTDA EDR estimates and a positive bias of about 0.15 m 2/3 s -1. The spread may be due to time lags between the radar and aircraft measurements, the inherent difference between the two measurement systems, inhomogeneity over the mosaic comparison volumes, or inaccuracy in the in situ EDR reports. The bias may be due to inaccuracy in the in situ EDR reports, an incorrect choice of turbulence length scale (see Figure 7), or the fact that the NTDA does not yet account for spectrum width estimator bias or spectral broadening mechanisms other than turbulence. The Receiver Operating Characteristic (ROC) curves illustrate the tradeoff between the probability of detecting aircraftmeasured turbulence of various magnitudes vs. the probability of detecting the absence of turbulence (which is one minus the nuisance alarm rate). The fact that the curves approach the upper right corner of the plot indicates that the NTDA has skill in predicting aircraft turbulence encounters. In addition, the fact that the NTDA performs better for higher levels of turbulence is encouraging. Note, however, that the red line representing the ROC curve for severe or greater turbulence is not meaningful, since only two severe turbulence measurements were available for comparison. Figure 12 shows box plots and ROC curves for comparisons between the in situ peak EDR measurements and the NTDA EDR median values. The box plots show a clear correlation between the two measurements, though again with substantial spread and an apparent upward bias of the NTDA estimates for

11 small in situ EDR values. However, the NTDA appears to underestimate the turbulence severity for larger in situ peak EDR values. This may be explained by the fact that the in situ peak EDR measurements do not necessarily correlate well with the average turbulence in a volume surrounding the aircraft, which is the quantity represented by the radar measurement, mosaicking, and volumetric median. A similar mismatch phenomenon is illustrated using the aircraft EDR data. Figure 13 shows a comparison between in situ average and peak EDR values. The significant spread and increasingly large underestimate for large values is somewhat similar to the NTDA EDR vs. in situ peak EDR comparison, and reflects the fact that regions of high convectively-induced turbulence are often isolated. The lower plot in Figure 12 shows the ROC curves for the in situ peak EDR to NTDA EDR median value comparisons. The curves for light, moderate, and severe turbulence show somewhat less skill than those for the in situ average EDR, probably because the in situ peak EDR value is less representative of the turbulence in the volume being measured by the radar than the average value. Nevertheless, the NTDA again shows increasing skill for higher levels of aircraftmeasured turbulence, exhibiting the best skill for peak EDR values in the extreme turbulence category. In contrast, the comparison of median radar reflectivity mosaic values with in situ average and peak EDR measurements depicted in Figure 14 shows poor correlation between the reflectivity and in situ EDR values. In fact, a negative correlation for the first several in situ EDR levels may be observed, and only a slight positive correlation appears for the last few levels. Thus, the NTDA s skill in recognizing regions with elevated turbulence appears to be significantly greater than that of radar reflectivity. This is a noteworthy result, since reflectivity magnitude is often used by pilots, dispatchers and others to assess the likelihood of convectively-induced turbulence hazardous to aircraft. However, a bit of care must be taken in interpreting Figure 14. For one thing, roughly three times more in situ EDR to reflectivity comparisons were possible than in situ EDR to NTDA EDR comparisons. This is because the NTDA EDR values were stringently quality controlled, whereas uncensored reflectivity values were available over a larger region. The volumes over which the mosaic value medians were computed may also be expected to contain many more reflectivity measurements than NTDA EDRs. Because low SNR and high overlaid power ratios cause spectrum widths to be censored or assigned low confidence, the areas in which reflectivity is available but NTDA EDR estimates are not are typically those regions having low reflectivity. Furthermore, the mosaicking technique has a tendency to smear out even isolated measurements, which may magnify the effect of isolated pixels of very low and likely spurious reflectivity measurements. These effects likely result in the median values of reflectivity shown in Figure 14 being biased low. It should also be noted that commercial aircraft tend to avoid regions of enhanced reflectivity. Thus, these values do not comprise a sample of reflectivity or turbulence values representative of the atmosphere as a whole. Figure 11: (Top) Box plots indicating the distribution of NTDA EDR 3-D mosaic values (the median over the comparison volume described in the text) for different values of in situ average EDR measurements, shown along the x-axis. The horizontal red line in the middle of each box indicates the median value, while the upper and lower limits of the box represent the 25 th and 75 th percentiles, the whiskers on either end show the data range, and red dots indicate outliers. Notches in the boxes depict the uncertainty in the median value. To further aid interpretation, the number of data points used in creating each box plot are shown above the box. (Bottom) Receiver Operating Characteristic (ROC) curves for light or greater (in situ EDR > 0.10 m 2/3 s -1, green), moderate or greater (in situ EDR > 0.30 m 2/3 s -1, blue), and severe (in situ EDR > 0.50 m 2/3 s -1, red) turbulence. These plots show the tradeoff between the probability of detecting turbulence (y-axis) and the probability of detecting no turbulence (x-axis) as the NTDA EDR threshold is varied. For example, an 80% PoD of moderate turbulence would incur a 34% nuisance alert rate.

12 Figure 12: Identical to Figure 11, except using the in situ peak EDR measurements. The magenta line in the lower plot is the ROC curve for extreme turbulence (in situ EDR > 0.70 m 2/3 s -1 ). Figure 13: Box plots indicating the distribution of in situ average EDR measurements as a function of in situ peak measurements. Figure 14: Same as the upper plots in Figure 11 and Figure 12, except showing the distribution of median 3-D radar reflectivity mosaic values in place of NTDA EDR values. Because the median reflectivity mosaic values have no skill in predicting turbulence intensity, the ROC curves are not meaningful and so are not shown. 6. CONCLUSION A new Doppler radar turbulence detection algorithm, the NTDA, has been developed under direction and funding from the FAA s Aviation Weather Research Program (AWRP) to provide a real-time capability to detect potentially hazardous in-cloud turbulence using operational weather radars. The NTDA is a fuzzy-logic algorithm that uses radar reflectivity, radial velocity, and spectrum width to perform data quality control and compute atmospheric turbulence intensity estimates (EDR) and associated quality values, or confidences. The algorithm operates on each elevation tilt independently, with EDR estimates produced for each elevation tilt on a polar grid having roughly 1 azimuthal and 1 km radial spacing out to a range of 230 km. Data from multiple radars are collected and merged together to form a mosaic of incloud turbulence and associated confidence on a 3-D

13 grid with a horizontal spacing of approximately 2 km and vertical levels at multiples of 3,000 ft. up to 45,000 ft. A real-time demonstration of this NTDA product was performed in the summer of IRaDS Level II data from sixteen NEXRADs in the upper Midwest were obtained via Internet 2 and ingested using Unidata s Local Data Manager (LDM) into Linux servers at NCAR in real time. These data were processed using the NTDA, and mosaics were produced every 5 minutes. These mosaics were made available to users via an interactive, web-based Java display. In addition, custom text-based maps indicating turbulence ahead were generated for all United Airlines aircraft entering the demonstration domain, and cockpit uplinks were performed for select flights. Comparisons between NTDA EDR values and EDRs derived from research aircraft data provide evidence of the NTDA s skill but are insufficient for a statistically meaningful comparison. To provide an objective assessment, a statistical comparison between four months of mosaicked NTDA EDR data from the operational demonstration and in situ EDR data obtained from the FAA s automated turbulence reporting system was performed. These comparisons provided compelling evidence that the NTDA has significant skill in detecting moderate or greater turbulence. For example, the ROC curve in Figure 11 shows that, using the one-minute average in situ EDR value as truth, a probability of detection of 80% can be obtained with a nuisance alarm rate of 34%. This is an excellent result given the difficulties in comparing the radar and aircraft measurements. Furthermore, comparisons of in situ EDRs with reflectivity mosaic values suggest that the NTDA has significantly greater skill than radar reflectivity in diagnosing hazardous in-cloud turbulence. Efforts are currently underway to implement the NTDA in the NEXRAD Open Radar Products Generator (ORPG) so that EDR data can become part of the NEXRAD Level III data stream, available to all interested users for operational or scientific purposes. In particular, these data could be used to generate a nationwide real-time turbulence detection product similar to the 3-D mosaic produced during the summer 2005 operational demonstration. Such a product could provide both strategic and tactical turbulence hazard information to aviation meteorologists, airline dispatchers, pilots, general aviation users, and air traffic controllers. The in-cloud turbulence product would be potentially more useful than the radar reflectivity data currently available to these users in identifying regions of hazardous in-cloud turbulence, and would supplement the upper-level turbulence forecasts currently supplied by the Graphical Turbulence Guidance product on the National Weather Service Aviation Weather Center s Aviation Digital Data Service (ADDS). Eventually, it is intended that the NTDA output will be combined with satellite, in situ, and numerical weather prediction model data to identify and forecast regions of hazardous convectively-induced turbulence. The resulting turbulence nowcast capability could significantly improve aviation safety, passenger confidence, and air traffic flow during convective events. 7. ACKNOWLEDGEMENTS The authors wish to thank the NASA Aviation Safety and Security Program and Langley Research Center for providing aircraft data from the spring, 2002 TPAWS flight tests. This research is in response to requirements and funding by the Federal Aviation Administration (FAA). The views expressed are those of the authors and do not necessarily represent the official policy or position of the FAA. 8. REFERENCES Cornman, L. B., C. S. Morse, and G. Cunning, 1995: Real-time estimation of atmospheric turbulence severity from in-situ aircraft measurements, Journal of Aircraft, 32, Cornman, L. B., and R. K. Goodrich, 1996: The detection of atmospheric turbulence using Doppler radars. Preprints, Workshop on Wind Shear and Wind Shear Alert Systems. Oklahoma City, November. Amer. Meteor. Soc., Boston. Cornman, L. B., J. Williams, G. Meymaris, and B. Chorbajian, 2003; Verification of an Airborne Radar Turbulence Detection Algorithm. 6 th International Symposium on Tropospheric Profiling: Needs and Technologies, Cornman, L. B., G. Meymaris and M. Limber, 2004: An update on the FAA Aviation Weather Research Program s in situ turbulence measurement and reporting system. 11th AMS Conference on Aviation, Range, and Aerospace Meteorology. Doviak, R. J., and D. S. Zrni, 1993: Doppler Radar and Weather Observations. Academic Press, 562 pp. Kessinger, C., S. Ellis and J. Van Andel, 2003: The radar echo classifier: a fuzzy logic algorithm for the WSR-88D. Preprints-CD, 3rd AMS Conference on Artificial Intelligence Applications to Environmental Science, Long Beach, 9-13 Feb. Sharman, R., C. Tebaldi, J. Wolff and G. Wiener, 2002: Results from the NCAR Integrated Turbulence Forecasting Algorithm (ITFA) for predicting upper level clear-air turbulence. 10th AMS Conference on Aviation, Range, and Aerospace Meteorology,