PROBABILISTIC METHODS FOR AIR TRAFFIC DEMAND FORECASTING

Transcription

1 AIAA Guidance, Navigation, and Control Conference and Exhibit 5-8 August 2002, Monterey, California AIAA PROBABILISTIC METHODS FOR AIR TRAFFIC DEMAND FORECASTING Larry A. Meyn * NASA Ames Research Center Moffett Field, California lmeyn@mail.arc.nasa.gov ABSTRACT Efficient and effective air-traffic management decisions can be dependent on the accuracy of demand predictions for sector loading and airport arrival rates. The accuracy of deterministic demand-forecasting methods in current use can only be improved by increasing the accuracy of the data used to make demand predictions. These demand-forecasting methods are deterministic because they utilize location predictions of individual aircraft as precise values for estimating demand. However, demand is based on the total number of aircraft utilizing a resource, not on which particular aircraft are included in the demand count. So probabilistic methods, that model how the demand utilization uncertainties of all aircraft will interact, should provide better demand estimates. The development of simple and efficient probabilistic methods for air-traffic demandforecasts is presented. Monte Carlo simulations demonstrate that the methods produce count errors for Airport Arrival Rate estimates with 15 to 20% lower standard deviations than those produced by deterministic methods. A reduction that is equivalent to what could be achieved deterministically by reducing the uncertainty in airport arrival-time predictions by 25 to 35%. AAR CDF NOMENCLATURE Airport Arrival Rate Cumulative Distribution Function cond A logical expression CTAS Center-TRACON Automation System ETA Estimated Time of Arrival ETMS Enhanced Traffic Management System f F A probability density function A probability distribution function k Aircraft count value, (0 k N) MAP N PDF p P N [k] P N s t TFAS x s m Monitor Alert Parameter Total number of aircraft Probability Density Function Probability that an aircraft should be counted Probability that the count is k when N aircraft are considered, (0 k N) Count probability vector for N aircraft, P N = {P N [0],, P N [N]} Sum of aircraft demand probabilities Time variable Traffic Flow Automation System A real valued number Standard deviation Mean value or average value * Aerospace Engineer, Associate Fellow AIAA. Copyright 2002 by the American Institute of Aeronautics and Astronautics, Inc. No copyright is asserted in the United States under Title 17, U.S. Code. The U.S. Government has a royalty-free license to exercise all rights under the copyright claimed herein for Governmental purposes. All other rights are reserved by the copyright owner.

2 subscripts: arrive c det end enter exit i prob r so start u Predicted aircraft arrival at a waypoint, fix or boundary Cumulative Deterministic End of an airport arrival period Predicted aircraft entrance to a sector Predicted aircraft exit from a sector Associated with the ith aircraft Probabilistic Relative to a prediction Sector occupancy Start of an airport arrival period Uncertainty for arrival time predictions operators and functions:! Factorial, n! = n(n 1)(n 2) 2 1, 0! =1! = 1 Rounding, x is the closest integer value to x * Convolution, a * b is the convolution of vectors a and b or of continuous functions a and b Prob Probability, Prob(cond) is the probability that cond is true INTRODUCTION Effective and efficient air traffic management requires the ability to accurately predict future demand on available resources; i.e. controller workload, airspace capacity and runway capacity. The Federal Aviation Administration s (FAA) principle tool for making these forecasts is the Enhanced Traffic Management System (ETMS). 1 Several research programs have been initiated to improve demand prediction accuracy. A study by the Volpe National Transportation Systems Center shows that more accurate flight schedule data can produce significantly improved demand predictions. 2 The National Aeronautics and Space Administration (NASA) and the MITRE Corporation s Center for Advanced Aviation System Development (CAASD) are both pursuing the use of more accurate trajectory predictions and sector representations to improve demand estimates. NASA is developing the Traffic Flow Automation System (TFAS) for improved sector load predictions and CAASD is investigating the use of Collaborative Routing Coordination Tools (CRCT) for improved sector load predictions. The basic goal of all of these efforts is to estimate how many aircraft might require a system resource such as an airport arrival slot or occupancy of an airspace sector at a specific time or period in the future. Thus far, all of these demand-forecasting efforts rely on deterministic predictions and the only improvement in demand forecasts has been achieved by improving the accuracy of 4- dimensional aircraft trajectory predictions, sector representations and other data used to estimate when an aircraft will arrive at a sector or an airport. The purpose of this paper is to show that knowledge of the uncertainty in aircraft arrival estimates can be used to produce probabilistic demand forecasts that are more accurate than estimates produced using deterministic demand-forecasting methods. The accuracy of demand forecasts is fundamentally limited by the accuracy of aircraft location predictions. Demand forecasting methods that represent aircraft location as a precise function of time are considered deterministic. In reality, the data used to predict aircraft location have significant uncertainty and the actual location of an aircraft at some future time can vary significantly from the predicted location. This uncertainty can be represented as a probability distribution of the aircraft s location about the predicted location or as a probability distribution in time about a predicted boundary or way-point crossing time. Demand forecasts that utilize location predictions along

3 with estimates of location prediction uncertainty are considered probabilistic. The probabilistic demand-estimates are superior to deterministic demand-estimates whenever there is significant uncertainty in whether the aircraft involved should be included in the demand-estimate. If the probabilities of individual aircraft demand are all near zero or one, then the demand forecast can be adequately estimated deterministically. However, having aircraft with intermediate demand probabilities, especially those near 0.5, degrades the accuracy of deterministic demand predictions and increases the benefit of using probabilistic methods. To illustrate this, a simple, although somewhat contrived, example is presented. Suppose that 20 aircraft might possibly land at an airport during a specific arrival period. Furthermore, suppose that each aircraft individually has a probability of 0.6 of arriving during this period. From the deterministic viewpoint, an aircraft either arrives during the specific period or not. This determination is usually based on the predicted arrival-time, but, for simplicity, the deterministic count estimate in this example is calculated using Eqn. 1. N k det = Â p i 1) i=1 In this equation, the deterministic arrival prediction for each aircraft is determined by rounding its arrival probability value to either zero or one. If the rounded probability value is one for an aircraft, it is included in the deterministic demand count. In this case all aircraft have an arrival probability of 0.6, so the deterministic estimate is that 20 aircraft will arrive during the target period. In contrast, a very simple probabilistic approach is to base the demand estimate on the sum of the individual aircraft arrival probabilities rounded to the nearest integer value, which is represented by Eqn. 2. In this case, there are 20 aircraft with a probability of 0.6 of arriving during the specified period. Therefore, the probabilistic demand estimate is that 12 aircraft will arrive. k prob = s = N p i i=1 Â 2) In this example, the number of aircraft that might actually arrive during the specified period could range from 0 to 20. Since all of the aircraft have the same probability of arriving, ( p i = p = 0.6), the probability for each possible count follows a binomial distribution, which is given by Eqn. 3. 3,4 The probability calculated for each possible arrival count in the foregoing example is graphically displayed in Fig. 1. N! P N [ k] = k! ( N - k)! pk ( 1 - p) N -k 3) Use of Eqn. 3 shows that the simple probabilistic estimate of 12 aircraft is the most probable count, with a probability of 18%. Additionally, there is a greater than 50% chance that the count will be within ±1 count of the simple probabilistic estimate. In contrast, the deterministic estimate of 20 has a probability of less than 0.004% and there is only 1 chance in 20 that the count would even exceed 15. Admittedly, this is an extreme example of how the deterministic approach can produce unlikely demand-estimates. Use of a common arrival probability much closer to zero or one would have produced a deterministic estimate that was much closer to the most probable estimate. However, this paper demonstrates that probabilistic demand-estimates are statistically and significantly better than deterministic demand-estimates when the involved aircraft have a random range of demand probabilities. Furthermore, the additional information provided by the probability distributions of demand-estimates can be used to better interpret and manage excess-demand alerts. It will be shown that probabilistic demandprediction methods outperform deterministic demand-prediction methods for a given level of arrival-time prediction accuracy. The reason is that purely deterministic prediction methods are based on the best resource-utilization prediction for each individual aircraft, with no allowance for how uncertainties in individual predictions will interact when considered as a group. The

4 variety of methods that may be used to estimate the 4-dimensional trajectories and their uncertainties for individual aircraft will not be addressed in this paper. Likewise, the specific details of other uncertainty sources such as sector representations and flight schedules will not be addressed. Instead, the focus is on how the overall uncertainties in arrival time predictions for several aircraft interact when estimating airport or sector demand. The basic methods for probabilistic demand calculations are presented first, along with a method for efficiently calculating demand probability distributions. This is followed by presentation of simulation results that illustrate the performance of probabilistic and deterministic demandestimation methods for airport arrival-rate and sector loading. PROBABILISTIC DEMAND CALCULATIONS For the purposes of this paper, demand is assumed to be the number of aircraft utilizing a resource (airport or sector) at a specific time or during a specific period of time. Each aircraft being considered has some probability, from zero to one, that it should be included in a demand count for a specified time or period. The estimation of these individual count probabilities is based on the probability distributions of the predictions for when an aircraft passes a fix or boundary. Methods are presented for estimating individual aircraft count probabilities for Airport Arrival Rate (AAR) and sector loading. Once the individual aircraft count probabilities are determined, they can be used to estimate demand counts. Airport Arrival Probability The predicted time of arrival at a way-point or boundary, t arrive, has an uncertainty that can be represented by the Probability Density Function (PDF), 3,4 f u, as shown in Fig. 2a. This distribution can be thought of as the ensemble of actual arrival times that might occur given the predicted arrival time, t arrive,. The easiest way to estimate this distribution is from a historical ensemble of the differences between actual arrival times and predicted arrival times. As depicted, the mean actual arrival time (which in this case is the same as the most probable actual arrival time) does not align with the deterministically predicted arrival time. The difference between the mean of the actual arrival time distribution and the predicted arrival time is the bias. For illustration purposes, the arrival time PDF shown in Fig. 2a is biased so that aircraft will, on the average, arrive early or before the predicted arrival time. Arrival-time PDFs are usually defined using a time variable, t r that is defined relative to the predicted arrival time using Eqn. 4. t r = t - t arrive 4) The probability that the aircraft will go past the way-point during a specific period can be determined by integrating its arrival time PDF over that time period. For estimating AAR, the probability that an aircraft will arrive between the start of the arrival period, t start, and the end of the period, t end, is needed. Eqn. 5 can be used to determine arrival probability, via direct integration of f u. However, the estimation of arrival-time PDFs is not exact and it is likely that only a few predetermined PDFs, based on previous prediction performance data obtained during periods having representative environmental conditions, will be used in AAR estimation. In such a situation it is not practical to integrate f u for every arrival event. It is more efficient to utilize the Cumulative probability Distribution Function 3 (CDF), F c, shown in Fig. 2b, which is the integral of the f u determined using Eqn. 6. Use of the CDF eliminates the need for repeated integration calculations if the same f u function is used for several arrival events. An individual aircraft s arrival probability is determined by the difference between CDF probability values for the starting and ending times of the arrival period using Eqn. 7. t end p = Ú f u ( t - t arrive )dt 5) t start ( ) = f u t r F c t r Ú ( )dt r 6)

5 p = F c ( t end - t arrive ) - F c ( t start - t arrive ) 7) Sector Occupancy Probability To estimate AAR, it is necessary to determine the probability that an aircraft will pass an airport arrival fix location during a specific period. Sector loading, on the other hand, requires the probability that an aircraft will occupy a sector at a specific instant in time. For deterministic predictions, an aircraft is either in the sector or not in the sector, therefore the probability is zero before the predicted sector entry time, t enter, zero after the predicted sector exit time, t exit, and one in between these times. This probability distribution is represented in Fig. 3a as the deterministic sector occupancy function, F so,det. Since the predicted sector entry and exit times are subject to uncertainty, a PDF can be used to represent the predicted sector entry or exit time. An example of such a prediction uncertainty function, f u is depicted in Fig. 3a. For this function, the variable t r represents time relative to the predicted time of an event such as sector entry or exit. The probabilistic sector occupancy function, F so,prob, is the probability distribution function that results from the combination of F so,det and f u. If F so,det and f u are independent, the probability distribution function for their combination can be determined using convolution, 3,4 as shown in Eqn. 8. An alternative form of Eqn. 8, using the convolution operator, *, is presented in Eqn. 9. The resulting probabilistic sector occupancy function is depicted in Fig. 3b. F so,prob ( t) = Ú F so,det ( t - t r ) f u ( t r )dt r 8) - F so,prob ( t) = F so,det ( t)* f u ( t) 9) Two potential drawbacks to this method of determining the probabilistic sector occupancy are that the prediction uncertainty function, f u, may be a function of the prediction time and, that the convolution operation can be computationally expensive. The first drawback isn t a significant problem if f u is reasonably constant over the sector occupancy period, but the uncertainty in sector transit time may be large enough in some situations to make f u,exit significantly different from f u,enter. The second drawback, that convolution operations can be computationally expensive, is one that should always be considered, especially if large numbers of convolution operations might be required. A preferred way to address both of the aforementioned drawbacks is to apply an alternative method recommended here. The first step is to compute the CDFs, F c,enter and F c,exit, for the uncertainties in predicted sector entry and exit times as shown by Eqns. 10 and 11. If prediction uncertainty is significantly different for the sector entry and exit times, then both prediction uncertainty PDFs should be integrated. Otherwise, only one PDF needs to be integrated and F c,enter and F c,exit, can be determined from simple time offsets of the resulting CDF. F c,enter (t) gives the probability that an aircraft has entered the sector by time t and F c.exit (t) gives the probability that an aircraft has left the sector by time t. The difference between these two values is the probability of sector occupancy for time t, as shown by Eqn. 12. This method is shown graphically in Fig. 4. This procedure requires much less computation than the convolution method, since the integration of a single function is significantly easier to compute than the convolution of two functions. Furthermore, this method easily accounts for potential variations in prediction uncertainty over the sector occupancy period, which is a significant problem for the convolution method. F c,enter ( t) = Ú f u,enter ( t - t enter )dt 10) F c,exit ( t) = Ú f u,exit ( t - t exit )dt 11) F so,prob ( t) = F c,enter ( t) - F c,exit ( t) 12) Demand Count Probabilities Once the probabilities have been determined for each aircraft to be counted towards the total demand-count, it is then beneficial to calculate the probability distribution of all possible

6 demand-counts. The equation for binomial probability distributions, Eqn. 3, only works when all aircraft have equal probabilities, therefore a more general method is needed. For N aircraft, the possible counts range from 0 to N. The probability for each count value is designated as P N [n]. A vector containing all of the count probabilities for N aircraft is designated by P N. The first four count probability vector equations, for N = 0 to 3, are presented in Eqn. Set 13. P 0 = {1} P 1 = {( 1 - p 1 ), p 1 } P 2 = {( 1 - p 1 )( 1 - p 2 ),( 1 - p 1 )p 2 + p 1 ( 1 - p 2 ), p 1 p 2 } 13) P 3 = {( 1 - p 1 )( 1 - p 2 )( 1- p 3 ), p 1 ( 1 - p 2 )( 1- p 3 )+ ( 1- p 1 )p 2 ( 1 - p 3 )+ ( 1 - p 1 )( 1- p 2 )p 3, ( 1- p 1 )p 2 p 3 + p 1 ( 1- p 2 )p 3 + p 1 p 2 ( 1- p 3 ), p 1 p 2 p 3 } The first equation in the set simply states the obvious, that when zero aircraft are considered, the probability for a zero count, P 0 [0], is one. The equation for one aircraft is also very obvious, P 1 [0] is the complement of p 1, (1 p 1 ), and P 1 [1] is simply p 1. For N greater than 1, the count probabilities are determined as combinations of the individual aircraft probabilities and their compliments. For P 2, P 2 [0] is the joint probability that both aircraft are not counted, P 2 [1] is the sum of the two joint probabilities that only one aircraft is counted, and P 2 [2] is the joint probability that both aircraft are counted. As can be seen by examination of the equation for P 3, the number of terms needed to calculate count-probabilities increases rapidly with the number of aircraft. The number of terms that are required to calculate the probability vectors for N up to 5 is shown in Fig. 5. The number of terms required for each count probability can be represented by the number sequence known as Pascal s Triangle, 4 about which a number of properties are known. The most significant property for calculating probability vectors is that the total number of terms needed is N 2. Using this direct method for calculating count probabilities, the total number of basic floating-point operations needed to calculate the Nth probability vector is N2 N -1. This is a computationally significant effort for even moderate values of N. For example, for N = 25, nearly one billion floating point operations are needed. Fortunately, there is a much more efficient way to calculate count probability vectors. Examination of the probability vector for P 3 in Eqn. Set 13 reveals that each of the terms in each count probability expression differ from a term in an adjacent count probability expression by only one factor. This means that some products are calculated more than once when performing the calculations for determination of an entire probability vector. This leads to a significant number of redundant calculations as the number of aircraft considered increases. To eliminate these redundant calculations, an iterative procedure, defined by Eqn. Set 14, was developed to calculate probability vectors. P 0 [ 0] =1 For i = 1 to N : P i [ 0] = ( 1 - p i ) P i-1 [ 0] P i [ i]= p i P i-1 [ i - 1] For k = 1 to i -1 : P i [ k] = p i P i -1 [ k - 1]+ ( 1 - p i ) P i-1 [ k] 14) This procedure is expressed more concisely in Eqn. Set 15 using vector convolution, which states that probability vector, P i, is determined by the convolution of the previous probability vector, P i-1, with the probability complement pair, {(1 p i ), p i }, of the ith aircraft to be considered. This method is hereafter referred to as recursive convolution. The total number of basic floating-point operations needed to calculate the probability vector for N aircraft using recursive convolution is 3N( N + 1). 2 Computationally, this is much more efficient This was determined by manual examination of the equations. It was assumed that the probability complements, (1 p i ), were only calculated once. This was determined by manual examination of the equations.

7 than the direct method given by Eqn. Set 13. For example, for N = 25, the number of floating point operations needed is less than a thousand. Even for several hundred aircraft, the computation time is not significant for most applications. P 0 = { 1} P i = P i -1 * {( 1 - p i ), p i } AIR TRAFFIC DEMAND SIMULATIONS 15) A series of Monte Carlo simulations were carried out to assess the potential effectiveness of probabilistic and deterministic methods for the prediction of air traffic demand. A key feature of the simulations is that the count probabilities of the individual aircraft that may impact demand cover the entire range from zero to one. The difference in how aircraft with intermediate count probabilities are treated is the major difference between the deterministic and probabilistic methods. Most of the simulations consisted of predictions of airport arrival counts, with a few simulations of sector loading predictions included. AAR prediction simulations were first analyzed because they are simpler than sector loading predictions, and therefore a logical starting point for this research. However, the potential benefits of probabilistic forecasts for sector loading should be similar to what is shown for AAR predictions. The Monte Carlo simulation specifics are presented next, followed by simulation results showing deterministic and probabilistic count prediction accuracy for forecasts based on aircraft arrival-time predictions that have Gaussian error distributions. Since Gaussian error distributions might not be considered to be sufficiently realistic, similar results are then presented for aircraft arrival-time predictions having empirically determined error distributions. Next, results are shown for the accuracy of traffic alerts based on deterministic and probabilistic count predictions. Finally, an analysis of a sector loading simulation is presented. Monte Carlo Simulation Details For each simulation analyzed, the predicted arrival-times for 36 aircraft were randomly generated so that the arrival-times would be uniformly distributed over a period from 15 minutes before a 15-minute arrival-period to 15 minutes after the arrival-period. This results in an average arrival rate of 12 aircraft per 15- minute period. For each simulation, it was assumed that the difference between predicted arrival-times and the actual arrival-times were randomly distributed according a specified error distribution function. Deterministic arrival count predictions were calculated based on the predicted arrival-times. Probabilistic arrival count predictions were calculated based on the predicted arrival-times and the assumed error distribution function. Finally, actual arrivaltimes were determined based on randomly generated prediction errors that conformed to the assumed error distribution function. For each specified error distribution function, 100,000 simulations were run to compare deterministic and probabilistic arrival-count predictions with actual arrival-counts. The number of simulations was chosen to provide sufficient data for repeatable simulation statistics while keeping the run time for all the simulations runs needed for the 12 probability distributions used under one hour. Three methods were used for estimating probabilistic arrival-counts. The first method used the sum of the arrival probabilities for each aircraft as the arrival-count prediction. The second method rounded the probability sum to the nearest integer, and the third chose the mostprobable count value from a count-probability distribution calculated using Eqn. Set 15. The first method produces the smallest standard deviations in count error of the methods tried, but the predicted counts are not integer values. The second and third methods produced nearly identical integer count error distributions. Since non-integer count predictions may not be deemed acceptable, the second method was used for the results presented here.

8 Gaussian Arrival-time Prediction Errors The arrival-time prediction error was assumed to be normally distributed for the simulation results depicted in Figures 6 and 7. In these figures, arrival-count prediction error is quantified by the standard deviation of the difference between the predicted count and the actual count. Fig. 6 shows the error of deterministic and probabilistic count predictions for simulations where the standard deviation in arrival-time prediction error varied from 0.25 to 6 minutes. There are consistently lower errors for the probabilistic predictions when compared with the deterministic predictions. This advantage decreases as arrival-time prediction accuracy improves, but for most of the cases depicted the reduction in count error is equivalent to what could be achieved deterministically by a reduction in arrival-time prediction uncertainty of 25 to 35%. Fig. 7 compares the error of deterministic and probabilistic count predictions for simulations where the standard deviation in arrival-time prediction is 2 minutes, and the mean error varies from 0 to 4 minutes. Again the probabilistic predictions are consistently more accurate than the deterministic predictions. One significant difference is that the probabilistic predictions are insensitive to variations in mean arrival-time error, whereas the deterministic predictions show a strong sensitivity. Empirical Arrival-time Prediction Errors While Gaussian error distributions are convenient to work with, they may not be realistic descriptors of how arrival-time prediction errors are typically distributed. Fig. 8 depicts the prediction error distribution for ETA estimates from the Center-TRACON Automation System 5 (CTAS) and ETMS for a three-hour time period on one day. These ETA error distributions were originally acquired and used for preliminary estimation of TFAS capabilities. (Since these error distributions are Data obtained from an internal NASA document entitled Performance Analysis in support of TFAS development: Progress Report by Jim Murphy based on an extremely small sample set, they may not represent typical CTAS and ETMS prediction performance. However, these distributions do exhibit properties that are more realistic than Gaussian distributions and should be representative of the empirical distributions that might be used in an operational demand forecasting system.) As can be seen, the CTAS predictions are more precise (i.e. have a smaller standard deviation) than the ETMS predictions. It is also quite clear that both distributions are biased. As was shown previously in Fig. 7, bias in arrival-time predictions can degrade deterministic arrival-load predictions, if it is not accounted for. Fig. 9 shows the accuracy of deterministic and probabilistic count predictions for simulations using the CTAS and ETMS distributions of arrival-time prediction error. A third distribution, equivalent to the CTAS distribution with the bias removed, was also simulated. In all cases, the probabilistic predictions are more accurate. Probabilistic count predictions are not sensitive to bias errors, but the deterministic CTAS predictions are very sensitive to bias error. The probabilistic method accounts for known biases in the determination of the aircraft arrival-probabilities that are used in demand forecasts. However, the only way to account for a known bias with the deterministic method is to remove the bias from the arrivaltime predictions before they are used. Accuracy of Airport Arrival Rate Alerts The Monitor Alert Parameter 1 (MAP) is a numerical trigger value indicating that sector/airport efficiency may become degraded if the aircraft count meets or exceeds the MAP value. Assuming that an alert is triggered when the predicted aircraft count meets or exceeds the MAP value, it is useful to know the statistical accuracy of these alerts. Two types of errors can (currently with NASA Ames Research Center). The data is for a 3-hour time-period at the Dallas/Fort Worth International Airport on 14 March Reference 6 presents more comprehensive data and analysis on CTAS arrival-time prediction accuracy.

9 occur, missed-alerts and false-alerts. A missedalert is defined as an alert that should have been triggered but wasn t, and a false alert is defined as an alert that was triggered, but should not have been. The ratio of missed-alerts to total number of alerts that should have occurred is the missed-alert ratio. The false-alert ratio is the number of false-alerts divided by the total number of alerts, both true and false, that were indicated. For deterministic predictions, an alert is triggered whenever the predicted count meets or exceeds the MAP. For probabilistic predictions, the most probable count could be used to determine if an alert should be triggered. However, since the probability that the count will meet or exceed the MAP value can be easily determined, the alert trigger can be based on when this probability meets or exceeds a specified probability value. This latter method provides a means for tuning the alerts so that the balance of missed- and false-alerts can be adjusted, which might be desired if the effective costs of missed- and false-alerts are significantly different. Fig. 10 shows the missed- and false-alert ratios for both deterministic and probabilistic predictions. The arrival-time prediction error distribution had a standard deviation of 3 minutes, and a mean of zero. The MAP value for this simulation was 15. For probabilistic predictions, the probability at which an alert was triggered was varied from 0 to 100%. A requirement that the probability of the predicted alert must be 100%, essentially means that no alerts are triggered. Therefore, there are no false-alerts and every true alert event is missed. A requirement that the probability of the predicted alert be 0% or greater, means that alerts are triggered for every event. Therefore, there are no missed-alerts and the maximum possible number of false-alerts occurs. The missed- and false-alert ratios for the deterministic prediction method are both about For the probabilistic prediction method, equal missed- and false-alert ratios of 0.34 occur when the probability trigger-value is about 40%. This represents a 10% reduction in both missedand false-alert ratios when compared with the deterministic method. It is expected that alert accuracy should be strongly dependent on arrival-time prediction accuracy. This is shown in Fig. 11, which depicts missed- and false-alert ratios for several arrival-time prediction-error distributions that have standard deviations ranging from 1 minute to 6 minutes. The deterministic method provides missed- and false-alert ratios that are nearly equal in each simulation. Comparison of the deterministic results with points on the probabilistic curves that have equal missed- and false-alert ratios shows an average reduction in alert ratio values of about 10% for the probabilistic method. Alert accuracy is also dependent on the relative frequency of actual alert conditions. This is shown in Fig. 12, which depicts alert accuracy for MAP values that range from 13 to 18 for the situation where the arrival-time prediction error has a standard deviation of 3 minutes. The end points of the probabilistic curves along the bottom axis indicate the fraction of all simulated events that do not trigger an alert. Since the average arrival rate is 12, these fractions approach 1 as the MAP values get further from the average arrival rate. Overall, missed- and false-alert ratios increase with increasing MAP value. An explanation of this is that as events become more rare, they become harder to predict. Comparison of the deterministic results with points on the probabilistic curves that have equal missed- and false-alert ratios, shows that the probabilistic method missed- and false-alert ratios that are about 10% less than those produced by the deterministic method. Fig. 13 shows missed- and false-alert ratios for the empirical arrival-time error distributions depicted in Fig. 8 and using a MAP value of 15. The deterministic alert ratios are only reduced by 7.4% when going from the ETMS arrivaltime prediction error distribution to the CTAS distribution. However, use of an unbiased CTAS error distribution reduces alert errors by nearly 25%. This figure also shows that the probabilistic method produces alert error ratios

10 that are 18.5% lower than the deterministic method for the ETMS distribution, 30% lower for the CTAS distribution and 11.5% lower for the unbiased CTAS distribution. The benefit of going from deterministic, ETMS-based alert predictions to probabilistic, CTAS based alert predictions, is a 1/3 reduction in alert errors. Sector Load Simulation Sector loading is more complicated to predict than the AAR. The sector load/count depends on individual aircraft entrance and exit times, and alert conditions depend on the duration of the count meeting or exceeding the MAP value within the target time period. Detailed, statistical examinations of probabilistic sector-load predictions are not presented here. Instead, a single simulation of a sector-count prediction for a two-hour period is analyzed. The simulation conducted assumed that there were no aircraft in the sector at the beginning or the end of the simulation. To get two hours of a steady-state simulation, one that was unaffected by the starting and ending conditions, the simulation was started 30 minutes prior to the beginning of the forecast time and ran until a forecast time of 150 minutes (30 minutes past the two-hour period of interest.) The sector entrance times of 180 aircraft were randomly distributed over the total period of three hours using a uniform distribution, yielding an average of 60 aircraft entering the sector each hour. Each aircraft was assumed to have an average sector transit time of 10 minutes with a standard deviation of 2 minutes. Under these conditions, the average sector count is 10 aircraft. The uncertainty in predicting sector entrance times was assumed to be zero for the 30-minute period prior to start of the forecast time, since events occurring prior to the forecast time should be known without uncertainty. From the forecast time of zero and onward, the uncertainty in sector entrance time predictions was assumed to be monotonically increasing with forecast time. No uncertainty was assumed for the prediction of aircraft sector transit times in this simulation. This simulation was conducted for illustrative purposes only. Although the magnitude of the uncertainty in sector entry times is considered to be realistic, it is not rigorously based on actual prediction uncertainty data. Results of this simulation are presented in Fig. 14. The solid line represents the predicted count and the dashed line represents a possible actual count based on the assumed error distribution for sector entry-time predictions. The solid circles shown represent a non-integer, probable count, which is simply the sum of the occupancy probabilities for all aircraft at each half-minute interval. This value represents the mean or average of all possible sector-count values. Also shown are the 95% confidence limits for possible sector-count values. In the first few minutes of the forecast, the predicted and possible counts track the probable count very closely. However, as the forecast time extends further into the future, the predicted and possible counts diverge from each other and from the probable count. Realistically, the actual count will not track the probable count. Instead it will have similar excursions from the probable count, as do the predicted and possible counts that are shown. The 95% confidence limits give an indication of the magnitude of the excursions possible, but, by definition, actual sector counts may go outside these limits 5% of the time. Another way of viewing forecast sectorcount probabilities is presented in Fig. 15. The probability of possible sector counts is presented in two-minute intervals over the forecast time period. At a forecast time of 0 minutes, the probability is 1 for a sector count of 7. As the forecast time increases, the peak probability of any single count value steadily decreases, and the range of probable counts steadily increases. If the forecast were to be extended in time, the most probable count would slowly converge on the average simulation sector count of 10 and the spread of probable counts would increase to the point where the forecast would have little practical utility.

11 The uncertainty in sector entry and exit times is not the only uncertainty affecting sector load predictions. There is also the uncertainty in whether or not an aircraft will actually traverse a specific sector due to schedule changes or rerouting. This is impossible to account for deterministically, but relatively easy to account for using probabilistic methods. One method would be to have multiple flight plans for each aircraft that might be rerouted and assign probabilities to each flight plan according to the likelihood that the flight plan will be used. Then the probabilities for the aircraft s sector entry and exit can be multiplied by the probability that the flight plan traversing that sector will be used. CONCLUSIONS This paper presents an introduction to the use of probabilistic methods to forecast airtraffic demand for sector space or aircraft landing times. The methods presented are simple to implement and they are not computationally intensive. The simulations presented indicate that probabilistic methods can reduce the standard deviation in error for Airport Arrival Rate estimates by 15 to 20%, an amount that is equivalent to what could be achieved by reducing the uncertainty in airport arrival time prediction by 25 to 35%. Furthermore, probabilistic methods were shown to reduce the number of missed- and false-alerts by at least 10%, with even greater reductions possible if the arrival-time predictions have significant, known biases. Similar results appear possible in forecasting sector loads. If the error distributions of arrival-time predictions are not well known, the probabilistic methods presented should still be considered. Even the use of rough estimates for the arrival-time error distributions should yield probabilistic demand forecasts that have noticeable improvements over deterministic forecasts. The probabilistic forecasting methods presented here have not yet been applied to actual air traffic data. However, the principle difference between the probabilistic and deterministic demand forecasting methods is in how aircraft with probabilities other than zero or one (for airport arrival or sector occupancy) are dealt with. Realistically, such probabilities should cover the entire range from zero to one, so it is reasonable to expect that the probabilistic forecasting method should provide significant improvement over the deterministic method when in actual, operational use. The magnitude of actual forecast the improvements may not be the same as those presented, but it is likely that they will be very similar. Since probabilistic methods for demand forecasting presented are simple to implement, and they have the potential to provide significantly improved forecasts, it is recommended that they be seriously considered for incorporation into air traffic demand forecasting tools. REFERENCES 1 FAA, Order R Facility Operation and Administration, Chapter 17, Part 5, July 12, Gilbo, E. and McCabe, L., Relative Accuracy of CDM and ETMS in Predicting Airport Arrival Demand, DOT-VNTSC-FAA-99-12, Volpe National Transportation Systems Center, September Feldman, R. M. and Vladez-Flores, C., Applied Probability and Stochastic Processes, PWS Publishing Company, Boston, Brandt, S., Data Analysis: Statistical and Computational Methods for Scientists, 3 rd ed., Springer-Verlag, New York, Erzberger, H., CTAS: Computer Intelligence for Air Traffic Control in the Terminal Area, NASA TM , National Aeronautics and Space Administration, July Heere, K. and Zelenka, R., A Comparison of Center/TRACON Automation System and Airline Time of Arrival Predictions, NASA TM , National Aeronautics and Space Administration, February 2000.

12 Figure 1 Arrival count probability distribution for 20 aircraft, each having a 60% probability of arriving during a specified period. Figure 3 Probability of sector occupancy using convolution, a) functions for deterministic sector occupancy and prediction uncertainty, b) the convolution of prediction uncertainty and deterministic sector occupancy. Figure 4 Sector occupancy probability determined from sector entry and exit CDFs. Figure 2 Aircraft arrival-time probability distribution functions. Figure 5 Number of terms needed to express count probability distributions.

13 Figure 6 Variation of count error for deterministic and probabilistic airport arrival rate predictions with the standard deviation of arrival-time prediction error (mean error of zero minutes.) Figure 8 ETA prediction errors minutes from landing, based on 3 hours of data (440 aircraft) at ZFW in March Positive error values indicate actual arrivaltime is later than predicted. Figure 7 Variation of count error for deterministic and probabilistic airport arrival rate predictions with the mean of arrival-time prediction error (standard deviation of 2 minutes.) Figure 9 - Comparison of probabilistic and deterministic arrival rate prediction accuracy for some empirical arrival-time prediction error distributions.

14 Figure 10 Missed and false alert ratios for probabilistic and deterministic predictions. (s = 3 min, m = 0, MAP = 15) Figure 12 Variation of missed and false alert ratios with variation of MAP value. (s = 3 min, m = 0, MAP = 13-18) Figure 11 Variation of missed and false alert ratios with variation of arrival-time prediction accuracy. (s = 1-6 min, m = 0, MAP = 15) Figure 13 Missed and false alert ratios for probabilistic and deterministic predictions with ETMS and CTAS error distributions. (MAP = 15)

15 Figure 14 Simulated sector count forecast with the uncertainty in aircraft sector boundary crossing times increasing with time. Figure 15 Simulated sector count forecast probabilities with projected probability contours.