Algorithms for Air Traffic Flow Management under Stochastic Environments

Algorithms for Air Traffic Flow Management under Stochastic Environments Arnab Nilim and Laurent El Ghaoui Abstract A major ortion of the delay in the Air Traffic Management Systems (ATMS) in US arises from the stochastic disturbances such as convective weather. However, in the current ractice, the redicted storm zones are comletely avoided as if they are deterministic obstacles. As a result, the current strategy is too conservative and incurs a high delay. In this aer, we seek to reduce the system delay through exlicitly modelling the dynamic and stochastic nature of the storms and adding recourse in the routing and the flow management roblem. We address the multi-aircraft flow management roblem using a stochastic dynamic rogramming algorithm, where the evolution of the weather is modelled as a stationary Markov chain. Our solution rovides a dynamic routing strategy for Naircraft that minimizes the exected delay of the overall system while taking into consideration the constraints obtained by the sector caacities, as well as avoidance of conflicts among the aircraft. Our simulation suggests that a significant imrovement in delay can be obtained by using our methods over the existing methods. I. INTRODUCTION Air traffic delay due to convective weather has grown raidly over the last few years. According to the FAA 2002, flight delays have increased by more than 58 ercent since 1995, cancellations by 68 ercent. The airsace caacity reduces drastically with the resence of convective weather. The drastic reduction of airsace caacity interruts traffic flows and causes delays that rile through the system. Consequently, weather related delays, which are stochastic in nature, contribute to around 80% of the total delay in most of the years in US since 1995. There has been a major effort to address delay in the traffic flow management roblem in the deterministic setting [3], [1], [2], [4], where demand and caacities are considered deterministic. In these Research suorted in art by Eurocontrol-014692, and DARPA-F33615-01-C-3150. A. Nilim is with the deartment of Electrical Engineering and Comuter Sciences (EECS) at the University of California, Berkeley (UCB), USA. elghaoui@eecs.berkeley.edu L. El Ghaoui is with the deartment of EECS at UCB, USA. nilim@eecs.berkeley.edu works, various traffic flow management algorithms are roosed in order to reduce the system delay, given that the system caacity is exactly known. However, the major contributor of delay is weather, which is robabilistic in nature and cannot be addressed in this framework. In our revious work [5], we incororated recourse in the lanning rocess where we addressed the single aircraft roblem using Markov decision rocesses (where the weather rocesses is modelled as a stationary Markov chain) and a dynamic rogramming algorithm. Our aroach rovides a set of otimal decisions to a single aircraft that starts moving towards the destination along a certain ath, with the recourse otion of choosing a new ath whenever new information is obtained, such that the exected delay is minimized. As we addressed the roblem in the stochastic framework, we obtain the best olicy. In this aer, we extend our model for multile aircraft. The roblem of routing under convective weather becomes much more comlex in a congested airsace because both aircraft conflicts and traffic flow management issues must be resolved at the same time. In this work, we rovide a dynamic routing strategy for multile aircraft that minimizes the exected delay of the overall system while satisfying the consideration of the constraints obtained by the sector caacity, as well as avoidance of conflicts among the aircraft. Moreover, we have used a more general weather dynamic model where the redicted zones can have more than two different states. II. WEATHER UNCERTAINTY MODEL Various weather teams (Center for Civil Force Protection (CCFP), Integrated Terminal Weather System (ITWS) etc ) roduce redictions that some zones in the airsace may be unusable in certain time interval and their redictions are dynamically udated at every T = 15 minutes. The later an event is from the rediction time, the more unreliable it becomes. It is reasonable to assume that we have a deterministic knowledge of the weather in the time interval of 0 15 minutes in future. Hence, each aircraft has a erfect

knowledge about the weather in the regions that are 15 minutes (15 times the velocity of the aircraft rovides the distance) away from it. We discretize time as 1,2,...,n stages according to the weather udate. Stage 1 corresonds to 0 15 minutes from the current time, stage 2 corresonds to 15 30 minutes from the current time. We choose n that accommodates the worst case routing of the aircraft. Let there be m storms that are redicted to take lace at the region K 1,K 2,...K m. For each K i, there can be multile outcomes. Deending on the coverage area and the intensity of the rediction, we allow to have different realizations. Let l is the number of ossible outcomes of the rediction in each region. For an examle, K 11,K 12,...,K 1l, are the ossible outcome regions in K 1 (K 1i K 1 i < l and K il = K 1 ). 0 corresonds to the situation where there is no storm in the region K 1, 1 corresonds to the state where there is a storm, but only materialized in the region K 11, and similarly l corresonds to the situation where there is a storm, but only materialized in the region K 11. l corresonds to the worst ossible outcome when K 1l or the whole region K 1 has been affected by the storm. For each storm, there can be (l + 1) different outcomes. As there are m storms, the Markov chain is a (l +1) m state Markov chain. For l = 2, and m = 2: g relacements [0 0] T,[1 0] T,[2 0] T,[1 0] T,[1 1] T,[1 2] T,[2 0] T,[2 1] T, and [2 2] T 11 form the state sace. We define i j as the robability 21 of the storm state to be j in the next stage if 12 the current state is i. For m = 1, the Markov chain 22 can be reresented by the following figure, PSfrag relacements 23 32 l,l 1 l 1,l ll 1 2 l 00 01 10 12 11 0 1 l Fig. 1. 21 kl Markov Chain. lk ll We use a rectangular gridding system to reresent the airsace where we consider each grid oint as a way oint. There are N aircraft currently ositioned at O 1,O 2,...,O N and the destination oints of the aircraft are D 1,D 2,...,D N (Figure 3). O i = [O i (x),o i (y)] T R 2, where O i (x) and O i (y) are resectively the x and y coordinates of the origin of aircraft i. Similarly, D i = [D i (x),d i (y)] T R 2, where D i (x) and D i (y) are resectively the x and y coordinates of the destination of aircraft i. Without the resence of convective weather, aircraft will try to follow the straight line connecting the origin and the destination, if those aths don t result in conflicts. There is a rediction that there can be m storms located at K 1,...,K m laces such that those zones might be unusable at certain time. w W are the weather states and W = (l +1) m. The airsace that is considered here is confined in f sectors, and the caacities of the sectors are C 1,...,C f. The redictions are dynamically udated with time. In the current ractice, these stochastic convective zones are assumed to be comletely unusable, and solution roceeds as if they are deterministic constraints. As those zones were just redicted to be of unusable with a certain robability, it often turns out that the zones were erfectly usable. As the routing strategies do not use these resources, airsace resources are under-utilized, leading to congestion in the remaining airsace through rile effect. In our roosed model, 100 80 60 40 20 0 20 40 60 80 100 O 2 Origin O 1 O N weather (redicted) Destination D N D 2 D 1 0 20 40 60 80 100 120 140 160 180 200 III. PROBLEM FORMULATION We consider a two dimensional flight lan of multile aircraft whose nominal aths are obstructed by redicted convective weathers. All the aircraft considered here are in the TMA/En-route ortion of their flights. Hence, the velocities of all the aircraft considered are constant. Fig. 2. A 2-D view of the roblem. we will not exclude the zones which are redicted to be unusable (with some robability) at a certain time and we will take into consideration the fact that there will more udates with the course of flight and recourse will be alied accordingly. We take a less conservative route in avoiding the bad weather

zone where we take a risk in delay to attain a better exected delay instead of avoiding the bad weather zones deterministically. We consider the following two schemes, 1) All of the N aircraft have the same riority. 2) Every aircraft has a different riority. Without the loss of generality, we assume that (riority of aircraft 1)>(riority of aircraft 2)>... >(riority of aircraft N). Scheme 1: All of the N aircraft have equal riority. At each stage (15 minutes time san, before the next udate), the storm state is assumed to stay constant. X 1 R 2,X 2 R 2,...,X N R 2 reresent the locations of aircraft. If the state of the convective weather is w (as section II), we define a state of the system as s = (w,x 1,...,X N ) R 2N+1, which reresents the ositions of all the aircraft and the storm situation. Furthermore, we define S as the set of all ossible states s. At any stage t, we want to choose an action (the directions of all the N aircraft to follow) from the set of allowable actions in state s, A s. Actions are the directions of all the N aircraft to follow in each stage with different realizations of the weather. We can obtain the action A s, that is the directions of the aircraft to follow if we calculate the oints that can be reached by each aircraft in the next 15 minutes, given their current ositions. This can be aroximately calculated if we draw an annular region with 15 v AC ± ε as radii, with a redefined angle θ and checking which grid oints fall in the region. Let, A = s S A s. We assume that S and A do not vary with time. If we decide to choose an action (the directions of all the N aircraft to follow) a A s in state s at the stage t, we ay a cost c t (s,a), which is the sum of distances travelled by N aircraft with the action a for a time interval of 15 minutes. For notational simlicity, we assume that the velocities of all the aircraft are equal (v AC ). Hence, if we minimize the exected distance travelled, we minimize the exected delay. (Exactly the same otimization formulation holds even if the velocities of the aircraft are different from each other: where we need to multily the cost functions with aroriate multilying factors). Furthermore, we define an indicator function I k j (X k ), whose value is 1 if X k is in the sector j, and 0 otherwise. In our otimization roblem, we will like to minimize the exected sum of the distances travelled by N aircraft, while resolving all the otential conflicts and satisfying the sector caacity constraints. The otimization roblem can be written as, ( ) nt min E s c t (s,a) a A s.t. X i (t,w) X j i (t,w) 2 > r, i, j t w, I k j C j 0 j f, k where r is the minimum ermissible searation distance between two aircraft, and. 2 is the euclidian norm. Scheme 2: (riority of aircraft 1)>(riority of aircraft 2)>... >(riority of aircraft N) This is a sequential otimization roblem and the stes are as follows, Ste 1: In this ste, we assume that there is only one aircraft in the airsace and that is aircraft 1, which has the highest riority. The state of the otimization roblem is defined as s 1 = (w,x 1 ) S 1, where w is the storm state and X 1 is the osition of the aircraft 1. If we decide to choose an action a 1 A s1 in state s 1 at the stage t, we ay a cost ct 1 (s 1,a 1 ), which is the distance travelled by aircraft 1 with the action a 1. We otimize the following roblem, min a 1 A s1 E s1 ( nt ct 1 (s 1,a 1 ) If we solve this otimization roblem, we obtain the otimal olicy a o 1 which rovides us X ot(t,w), 1 the otimal osition of aircraft 1 for all time t and for all the storm state w. Ste 2: In this ste, we only consider aircraft 2, which has the second highest riority. The state of the otimization roblem s 2 = (w,x 2 ) S 2. At any stage t, we want to choose an action (the direction aircraft 2 to follow) from the set of allowable actions in state s 2, A s2. Let, A 2 = s2 S 2 A s2. For the aircraft 2, we solve the following otimization roblem, ( ) nt min E s2 a 2 A s2 ct 2 (s 2,a 2 ) X 2 (t,w) X 1 ot(t,w) 2 > r, t w, and obtain X 2 ot(t,w). Ste 3- Ste N: We kee following the same rocedure till we have solved for all N aircraft. For the Nth aircraft,which has the lowest riority, the otimization )

roblem is the following, min a N A sn E sn ( nt ct N (s N,a N ) s.t. X N (t,w) X 1 ot(t,w) 2 > r, t w, ) X N (t,w) X 2 ot(t,w) 2 > r, t w,... X N (t,w) X (N 1) ot (t,w) 2 > r, t w, I k j C j 0 j f, k where Xot,...,X 1 (N 1) ot are obtained from revious iterations. In both of the schemes, we look for the best olicy. Determining the best olicy is to decide where to go next given the currently available information. We consider the set of decisions facing all of the aircraft that start moving towards the destination along a certain ath, with the recourse otion of choosing a new ath whenever a new information is obtained. IV. MARKOV DECISION PROCESS Let a system has the finite state sace S, and the finite action set A. Moreover, we denote by P = (P a ) a A the collection of transition matrices, and by c t (i t,a t ) the cost corresonding to state i t and action a t at time t. If we are trying to solve the otimization roblem, which is to minimize the exected cost over a finite horizon: ) min a A E i ( NT c t (i t,a t ) the value function can be comuted via the Bellman recursion ( ) V t (i) = min c t (i,a) + P a (i, j)v t+1 ( j), a A j which rovides the otimal action at each stage [6]. V. SOLUTION OF SCHEME 1 We roose a Markov Decision Process algorithm to solve the traffic flow management where each of the N aircraft has same riority. The stes of the algorithm are as follows, Ste 1: Preliminary calculations The state of the Markov Decision Process (MDP) for this roblem is s = (w,x 1,X 2,...,X N ) ((2N + 1) tule vector) and s S (defined in the section III). If we discritize the airsace by D number of nodes, S = D 2N (l +1) m. There are n stages in this MDP (obtained in section II). In addition, we need to calculate the action set A of the MDP as described in the revious section. Once we have the set of all ossible controls, we readily obtain P a from the weather data. Ste 2: Assigning aroriate costs We assign costs in such a way that our algorithm rovides aths that include going through the zones in the absence of storms while avoid it if there is a storm. Furthermore, it should also make sure that there is no conflict among N aircraft. We define c(w,x 1,...,X N,(X 1i ),...,(X Ni )) as the sum of all 1 k N costs obtained by aircraft k to go to X Ki from X k. Provision 1: Avoid if storm, otherwise take a shortcut We introduce a function PROV 1 : R 4N+1 {0,1} in order to rovide us the rovision of avoiding a zone if there is a storm, otherwise taking a shortcut. Let the storm state w corresonds to the fact that K w 1 1 K 1,...,K w N N K N are the zones that will be affected by the convective weather. We further define K w = i K w i i. If there exists a k (1 k N) such that (X ki K w ) or ( there exists a 0 λ 1 such that λx ki + (1 λ)x k K w, which means that the line connecting the oints X k, X ki cut any of the redicted storm zone ) PROV 1 (w,x 1,...,X N,X 1i,...,X Ni ) = 1, PROV 1 (w,x 1,...,X N,X 1i,...,X Ni ) = 0 Provision 2: Avoid conflict among each other First, we introduce a function CF v1 v 2 : R 2 R 2 R 2 R 2 {0,1}, where it takes the origin and destination oints of two aircraft with velocities v 1 and v 2 and rovides 1 if they are in conflict and 0 otherwise. We will demonstrate how to obtain CF v1 v 2 (I 1,F 1,I 2,F 2 ), where I j is the initial oint and F j is the final oint of the aircraft j. At time t, the ositions of aircraft 1 and 2 are I 1 + F 1 I 1 F 1 I 1 2 v 1 t and I 2 + F 2 I 2 F 2 I 2 2 v 2 t resectively. The distance between them at time t, d(t) = I ( W)t 2, where U 1 = F 1 I 1 F 1 I 1 2, U 1 = F 2 I 2 F 2 I 2 2, I = I 2 I 1, and W = (v 1 U 1 v 2 U 2 ). argmin t d(t) = argmin t ( I t W) T ( I t W). In order to find the otimal time t at which two aircraft come to the closest oint, we set t ( I Wt)T ( I Wt) = 0. Solving this, we obtain t = W T I W T W. If t < 0, the two aircraft are diverging, hence CF v1 v 2 (I 1,F 1,I 2,F 2 ) = 0. Also, if I + t W 2 > r, CF v1 v 2 (I 1,F 1,I 2,F 2 ) = 0,

CF v1 v 2 (I 1,F 1,I 2,F 2 ) = 1. In this roblem, as we have assumed that all the aircraft are flying at the same seed, we can write CF(,.,.,.,.) instead of CF v1 v 2 (,.,.,.,.). In addition, we introduce a function PROV 2 : R 4N {0,1} that rovides us the rovision of conflict avoidance. If l,k l k, X li X ki 2 > r or CF(X l,x li,x k,x ki ) = 0, PROV 2 (X 1,...,X N,X 1i,...,X Ni ) = 0, PROV 2 (X 1,...,X N,X 1i,...,X Ni ) = 1 Finally, the cost function is defined as following, if PROV 1 (w,x 1,...,X N,X 1i,...,X Ni ) = 1 or PROV 2 (X 1,...,X N,X 1i,...,X Ni ) = 1, c(w,x 1,...,X N,X 1i,...,X Ni ) = A very high value, c(w,x 1,...,X N,X 1i,...,X Ni ) = N k=1 X ki X k 2 Ste 3: Assigning aroriate Value function We define V t (s) as the value function which is the exected minimum distance to go if the current state is s and the current stage is t. We need to add the following rovisions in the value function in order to obtain the correct solution. Provision 1: Reach the destination oints The value function should have boundary conditions such that we obtain a comlete ath (ath stating at the origin and ending at the destination) as a solution. For the destination oints D 1,...,D N, the conditions below would guarantee that the solution will rovide a comlete ath. For any state weather state w and the last stage n, if {X 1 = D 1 },...,{X N = D N } V n (w,x 1,...,X N ) = 0, V n (w,x 1,...,X N ) = a very high value Provision 2: Direct cost at the end of the flight We assign the boundary values to the value function for the states which corresonds to the aircraft locations that are less than 15v AC aart from the destination oints. Let the state corresonds to the aircraft location of X 1,...,X N and ( (X i D i 2 15v AC i) and the stage t > 1. If (w corresonds to the storm state such that no storm zone intersects the straight line {λx k +(1 λ)d k and 0 λ 1} ) V t (w,x 1,...,X N ) = k X k D k 2, for any t > 1, V t (w,x 1,...,X N ) = Provision 3: Sector Caacity In order to ensure that the total number of aircraft in a sector at any time does not exceed the sector caacity, we assign value function aroriately. For a state s = (w,x 1,...,X N ), if there exists at least one j such that k I k j > C j, then V t =. Ste 4: Imlementing the recursive equations The recursive equation that solves the roblem is as follows, V t (s) = min a A {c(s,a) + s P a (s,s )V t+1 (s )}. We use the backward dynamic rogramming technique to solve these equations. We start with the final stage and go back iteratively to the first stage and obtain solutions for every stage and for every state. At the first stage, the solution is readily obtained as we know the current state. The aircraft will continue flying according to the solution until a new udate is obtained. At the next stage, we will receive a new udate, which corresonds to a new state. As we have already calculated all the otimal control for all ossible states, we just check the vector V 2 (.) and obtain the control. The aircraft will roceed in this way till they reach the destination oints (checking the vector V n (.)). In this way, we comute a routing strategy that rovides the minimum exected delay. VI. SOLUTION OF SCHEME 2 In this scheme, we assume that (riority of aircraft 1)>(riority of aircraft 2)>... >(riority of aircraft N) (as described in the section III). Ste 1: Otimal route for aircraft 1 In this ste, we find the otimal route for aircraft 1, which has the highest riority. In a sense, we assume that there is no aircraft in the airsace. The MDP state s 1 = (w,x 1 ) R 3 and s 1 S 1 (defined in the section III). We discretize the airsace and time in a same way as described in section V and section II, which yields S 1 = D 2 (l + 1) m, and n stages in this MDP. Actions of this MDP are the directions of aircraft 1 to follow in each stage with different realizations of the weather. As described in section V, we obtain the action a 1, that is the directions of aircraft 1 to follow if we calculate the oints that can be reached by aircraft 1 in the next 15 minutes, given its current osition. We define c 1 (w,x 1,X 1i ) as the cost to go if the aircraft 1 goes from X 1 to X 1i in a stage. For assigning the aroriate value, we only need to add rovision 1 (calculation

of which is same as described in section V: ste 2: rovision 1). As there is only one aircraft, there is no need to add the rovision for conflict avoidance. The value function for the MDP is defined as Vt 1 (s 1 ), which is the exected minimum distance to go if the current state is s 1 and the current stage is t. In the value function, we add the the first two rovisions (the calculation is same as described in section V: ste 4: rovision 1and 2 ). As there is no other aircraft in the airsace, we do not need to add any rovision that require satisfying the sector caacities. Once we have assigned all the boundary values roerly, we can solve the following recursion, V 1 t (s 1 ) = min a 1 A 1 {c 1 (s 1,a 1 ) + s 1 P a (s 1,s 1)V 1 t+1(s 1)}, and we obtain the solution of the recursion which rovides us Xot(t,w). 1 Ste 2 N: Otimal route for aircraft 2 N We follow the same rocedure in the next stes, excet we add the rovisions that rohibit conflicts and satisfy the sector caacity constraints. In the the second ste, we add the conflict avoidance rovision in the cost function, where Xot(t,w) 1 t w are considered NO-GO zones. We use the same rocedure described in section V: ste 3: rovision 2, where we assign high cost for violating these constraints. In each iteration, we kee a record of I k j and assign a very a high value to the value function if the sector caacity constraints are violated. When we solve the aroriate recursion, these additional features will guarantee the conflict avoidance and satisfy the sector caacity constraints. For the Nth aircraft, which has the lowest riority, the rovision 2 for cost function will be as follows, if 1 k N 1, X Ni X ki 2 > r or CF(X N,X Ni,Xot,X k ot) ki = 0, PROV 2 N(X N,X Ni ) = 0, PROV 2 N(X N,X Ni ) = 1 The cost function will be defined in the same way (section V) using these two rovisions. Also, V N t (.) is very high for the states which violates the sector caacity constraints. With this rovisions in the cost and value functions, we can solve the recursion and obtain otimal strategy for all of the N aircraft that minimize the delay, given the above riority scheme. This scheme avoids combinatorial exlosion as S 1 = S 2 =... = S N = D 2 (l + 1) m. On the other hand, as this is a more constrained otimization roblem, it yields higher delays than the scheme 1. VII. SIMULATION In this section, we discuss the results of the imlementation of both algorithms in various scenarios involving dynamic routing and traffic flow management of multile aircraft under uncertainty. We have imlemented both algorithms in MATLAB and we ran our exeriment on a standard PC. In the first exeriment, there are two aircraft with origins at O 1 = [0,96] T, and O 2 = [0, 96] T, and destinations at D 1 = [312, 96] T and D 2 = [312,96] T (all the units in n.mi.). The velocities of the aircraft are 480 n.mi/hour. There is a rediction of a convective weather. The storm zone is a rectangle whose corner oints are [168,96] T, [168, 96] T, [192, 96] T, [192,96] T, which may obstruct the nominal flight ath of the aircraft. Moreover, there is a critical airsace within this zone which will definitely be affected if the the storm takes lace. The critical zone is assumed a rectangle with corner oints [168,60] T, [168, 60] T, [192, 60] T, [192,60] T (the shaded zone in figure 3). We assume that the weather information of the ortion of the airsace that can be reached in 15 minutes is deterministic and the robability of the storm roagates in a Markovian fashion with time. Also, the minimum searation distance between two aircraft, r = 5(n.mi) in this examle. The weather udate is received once every 15 minutes. We discritize the time in 15 minutes time intervals (stages). We define 0 as the state when there is no storm, 1 as the state when only the critical zone is affected by the storm, and 2 as the state when the whole redicted zone has been affected by the storm. The rediction matrix is a follows, ( ) 0.4 0.4 0.2 P = 0.33 0.33 0.33. 0.33 0.33 0.33 P(i + 1, j + 1) corresonds to the robability that the storm state will be j in the next stage, if the current storm state is i; i.e., P(2,1) = 0.3 means that the robability that the storm state will be 1 (no storm) in the next stage is 0.3 given the current storm state is 0 (only critical zone is affected by the storm). In this scenario, we imlemented both of our strategies (scheme 1 and 2) and comared their erformances with the traditional strategy (TS) where the convective zone is avoided as if it is a deterministic obstacle while avoiding conflicts. We define Delay Measure (DMi A ) as the extra flight ath

g relacements 200 150 100 50 0 50 100 150 200 Weather O 1 (Predicted) D 2 O 2 D 1 0 50 100 150 200 250 300 350 400 Fig. 3. Routing of two aircraft. required by aircraft i in a strategy A in excess of the nominal flight ath (which is the Euclidean distance between the origin and the destination; 366.34 (n.mi.) in this roblem); DMi A j = DMA i + DM A j. Furthermore, we introduce a erformance metric Imrovement Measure (IM A/B i ) of Strategy A over B, which is the ercentage of the maximum ossible imrovement gained for aircraft i by using strategy A instead of using strategy B ; IM A/B i = 100 DMi B DMA i, and IM A/B DMi B i j = 100 DMB i j DMA i j. Higher IM DMi B j corresonds to better delay. In TS, if we resolve the conflict, aircraft 1 and 2 follow aths with a length of 455.12n.mi). DM1 T S = DM2 T S = 455.12 366.34 = 88.78, and DM12 T S = DMT 1 S + DM2 T S = 177.56. Using the scheme 1, where both aircraft have equal riority, aircraft 1 and 2 will initially follow a ath with an angle of 30.96 0 and 30.96 0 resectively till they get the next udate. Both of thempsfrag will avoid relacements the storm zone when there is a storm and take a direct route if there is no storm and the solution of the strategy is conflict free. In this way, both the aircraft follow a flight ath that yield a exected delay of 398.67 (n.mi.). DM1 1 = DM1 2 IM 1/T S 12 = 100 88.78 32.33 = 32.33, IM1/T S 1 = IM 1/T S 2 = 74.78 = 63.58%. Similarly if we use scheme 2, where aircraft 1 has higher riority over the aircraft 2, aircraft 1 and 2 initially fly at an angle 36.86 0 and 53.13 0 till they get the next udate. Similar to the scheme 1, both of them will avoid the storm zone when there is a storm and take a direct route if there is no storm and the solution of the strategy is conflict free. The exected distance travelled by the aircraft are 382.08 (n.mi.) and 426.24(n.mi.) resectively; IM 2/T S 1 = 78.95%, IM 2/T S 2 = 40.18%, and IM 2/T S 12 = 56.76%. The summary of the result is resented in I. IM of Scheme 1 over TS IM of Scheme 2 over TS Aircraft 1 56.71% 78.95 % Aircraft 2 67.72% 40.18% System 63.03 % 56.76% TABLE I IMPROVEMENT COMPARISONS We observe that we obtain a better system delay in case of scheme 1. However, in real life, deending uon the aircraft tye, size, and hub and soke network, it might be more reasonable to rioritize the routing strategy. Moreover, the comutation time for scheme 2 is 8.31 seconds, which is much faster than the comutation time for scheme 1 (7.46 minutes). We can avoid a combinatorial exlosion in case of scheme 2 and can handle large number of aircraft. There is no significant comutation cost in adding an extra aircraft. In order to illustrate this oint, if we add one more aircraft in the system with O 3 = [0,0] T and D 3 = [360,0] T (figure 4), our algorithm gives the routing strategy for aircraft 3 with an additional 4.84 secs. Aircraft 3 should have an initial angle of 14.036 0, which yields IM 2/T S 3 = 34.62% (for aircraft 3) and IM 2/T S 123 = 51.23% (for the system). In addition, we ran 200 150 100 50 0 50 100 150 200 O 1 O 2 O 3 O 4 D 2 D1 0 50 100 150 200 250 300 350 400 Fig. 4. Routing of three aircraft. an exeriment for the same weather rediction where a latoon of aircraft are ositioned at O 1 = [48,48] T, O 2 = [24,24] T, and O 3 = [0,0] T. The destination oint for all of the them is [360,0 T ] and (riority of aircraft 1)>(riority of aircraft 2)>(riority of aircraft 3). Initial vector rovided by our algorithms for the aircraft

g relacements are 33.61 0, 36.65 0, and 38.65 0. The system level IM obtained by using scheme 2 over TS is 58.35% (figure 5). 200 150 100 50 0 50 100 150 200 Platoon Destination 0 50 100 150 200 250 300 350 400 Fig. 5. Routing of a latoon of aircraft. VIII. CONCLUSION We rovide a traffic flow management tool that can assist the Air Traffic Controllers and the Airline Disatchers in managing traffic flow dynamically and routing multile aircraft under weather uncertainty. Our roosed strategies deliver a less circuitous route for an aircraft whose nominal ath is otentially obstructed by weather. Moreover, they inhibit the overloading of aircraft in the neighboring sectors of the redicted storm zones, thus the rile effect of delay due to convective weather is restricted. As a result, our algorithms rovide a traffic flow management scheme that minimizes the exected delay of the system. The comlexity of the comutation deends on the number of the aircraft, the origin-destination air, size and location of the storms, tye of storm, level of discretization, and the rate of information udates. The comlexity of the algorithm, when all the aircraft have equal riority, does not scale well with the number of aircraft. However, when each aircraft has a different riority, the algorithm is scalable; i.e., the algorithm scales linearly with the number of aircraft. In real life, a mixed scheme is more aroriate; aircraft are rioritized in a number of sets such that aircraft in different sets have the different riorities, but the aircraft within each set have the same riority. Our algorithm can be readily alied to the mixed scheme. Currently, we are develoing a riority scheme based on the game theoretic model that will be fair to all the articiants in the system. In addition, we are working on roosing an aroximate dynamic rogramming model that can handle large scale roblems involving a large number of aircraft and convective zones. REFERENCES [1] D. Bertsimas and Stock Patterson. The air traffic flow roblem with enroute caacities. Oerations Research, 46:406 422, 1998. [2] B. Lucio. Multilevel aroach to atc roblems: On-line strategy control of flights. International Journal of Systems Science, 21:1515 1527, 1990. [3] B. Nicoletti M. Bielli, G. Calicchio and S. Riccardelli. The air traffic flow control roblem as an alication of network theory. Comutation and Oerations Research, 9:265 278, 1982. [4] H. Marcia. The otential of network flow models for managing air traffic. Technical reort, The MITRE CORPORATION MP 93W0000058, Virginia, 1995. [5] A. Nilim, L. El Ghaoui, V. Duong, and M. Hansen. Trajectorybased air traffic management (tb-atm) under weather uncertainty. In Proc. of the Fourth International Air Traffic Management R&D Seminar ATM, Santa Fe, New Mexico, 2001. [6] M. Putterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscince, New York, 1994.