On the Approximate Linear Programming Approach for Network Revenue Management Problems

Transcription

1 On the Approximate Linear Programming Approach for Network Revenue Management Problems Chaoxu Tong School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA Huseyin Topaloglu School of Operations Research and Information Engineering, Cornell University, Ithaca, New York 14853, USA June 7, 2012 Abstract One method to obtain high-quality bid prices for network revenue management problems involves using the approximate linear programming approach on the dynamic programming formulation of the problem. This approach ends up with a linear program whose number of constraints increases exponentially with the number of flight legs in the airline network. The linear program is solved by using constraint generation, where each constraint can be generated by solving a separate integer program. The necessity to solve integer programs and the slow convergence behavior of constraint generation are generally recognized as drawbacks of this approach. In this paper, we show how to effectively eliminate these drawbacks. In particular, we establish that constraint generation can actually be carried out by solving minimum-cost network flow problems with natural integer solutions. Furthermore, using the structure of minimum-cost network flow problems, we a priori reduce the number of constraints in the linear program from exponential in the number of flight legs to linear. It turns out that the reduced linear program can be solved without using separate problems to generate constraints. The reduced linear program also provides a practically appealing interpretation. Computational experiments indicate that our results can speed up the computation time for the approximate linear programming approach by a factor ranging between 13 and 135.

2 1 Introduction Bid prices form a powerful tool to obtain high-quality control policies for network revenue management problems. The fundamental idea is to associate a bid price with each flight leg in the airline network, characterizing the opportunity cost of a seat on the flight leg. In this case, an itinerary is opened for sale to customers if the revenue that would be obtained from the sale of the itinerary exceeds the total opportunity cost of the seats consumed by the itinerary; see Williamson (1992) and Talluri and van Ryzin (1998). It is known that bid price policies are not necessarily optimal, but their ease of implementation and intuitive appeal make them a popular choice in practice. Since the bid price characterizes the opportunity cost of a seat on a flight leg, it is natural to expect that the bid price should depend on how much time is left until the departure time. As the departure time of a flight leg approaches, there are fewer opportunities to utilize the capacities and one would intuitively expect that the bid price of a flight leg would decrease, all else being equal. Despite this intuitive expectation, numerous popular models in the literature provide static bid prices that do not depend on how much time is left until departure; see, for example, Talluri and van Ryzin (1998) and Talluri and van Ryzin (1999). As the sales take place and the departure time approaches, one has to periodically resolve these models to get bid prices that actually change as a function of the remaining time until departure. Recently, Adelman (2007) proposes a model that naturally provides bid prices that depend on time until departure. This model starts with the dynamic programming formulation of the network revenue management problem, which is difficult to solve since the state variable in this formulation is a high-dimensional vector. The idea is to approximate the value functions with affine functions and choose the slopes and intercepts of the affine approximations by plugging them into the linear programming representation of the dynamic program. The number of decision variables in this linear program is manageable, but the number of constraints grows exponentially with the number of flight legs. Therefore, constraint generation is a natural method for solving the linear program. The slopes of the affine value function approximations are ultimately used as bid prices. This model is quite appealing as it has theoretical footing in dynamic programming theory and generates bid prices that depend on time left until departure, but its implementation in practice can be difficult. To begin with, solving the linear program by using constraint generation requires iteratively generating constraints and each constraint can be generated by solving a separate integer program. This integer program is a drawback when applying the model to large airline networks. Furthermore, constraint generation methods are known to be slow in obtaining the optimal solution; see Ruszczynski (2006). Finally, implementing constraint generation requires a good deal of customized software development and this negatively affects the practical appeal of the model. In this paper, we consider the network revenue management model proposed by Adelman (2007) and eliminate essentially all of the drawbacks that hinder its practical implementation. First, we show that the integer program that needs to be solved for generating constraints can actually be formulated as a minimum-cost network flow problem whose continuous relaxation has no integrality gap. This result holds for an arbitrary airline network topology, allowing us to generate constraints very quickly even for large and complicated airline networks. The minimum-cost network flow formulation of the constraint 2

3 generation problem is of interest by itself, but we note that constraint generation may still be slow to ultimately obtain the optimal solution, although we can generate each constraint quickly. Second, the number of constraints in the linear program used by Adelman (2007) grows exponentially with the number of flight legs in the airline network. By using the minimum-cost network flow structure of the constraint generation problem, we a priori reduce this linear program to an equivalent linear program whose size grows only linearly with the numbers of flight legs and itineraries. The reduced linear program completely eliminates the need to solve separate problems to generate constraints. Third, we give an appealing intuitive interpretation for the reduced linear program that clearly shows its relationship to the network revenue management problem we want to solve. This interpretation is likely to enhance the appeal of the model to practitioners. Fourth, our computational experiments indicate that by using the results in this paper, we can solve the model proposed by Adelman (2007) up to 135 times faster. Over all of our test problems, the average speed up factor is about 52. There are a number of models in the literature that are used to compute bid prices. Simpson (1989) and Williamson (1992) compute static bid prices by using a deterministic approximation to the network revenue management problem that is formulated under the assumption that all itinerary requests take on their expected values. In the deterministic approximation, there is a capacity constraint for each flight leg, ensuring that the served itinerary requests do not violate the leg capacities. The optimal values of the dual variables associated with these capacity constraints are used as bid prices. Talluri and van Ryzin (1998) give an analysis of the bid prices obtained by this deterministic approximation. Talluri and van Ryzin (1999) introduce randomness into the deterministic approximation by using samples of the itinerary requests, rather than expected values. Farias and van Roy (2007) and Meissner and Strauss (2008) use piecewise linear approximations to the value functions in the dynamic programming formulation of the problem and the policies obtained from their approaches can be viewed as bid price policies. Zhang and Adelman (2009) extend the model proposed by Adelman (2007) to cover the case where each customer makes a choice among the itineraries that are open for sale. Topaloglu (2009) computes bid prices that depend on both how much time is left until departure and how much capacity is left on the flight legs, but his approach is more computationally intensive than that of Adelman (2007). Kunnumkal and Topaloglu (2010) use Lagrangian relaxation to relax the capacity constraints in the dynamic programming formulation. The relaxed dynamic program can be solved efficiently and ultimately yields bid prices that depend on time. The rest of the paper is organized as follows. In Section 2, we begin by formulating the network revenue management problem as a dynamic program. In Section 3, we give the linear programming representation of this dynamic program. To obtain tractable approximations to the value functions, we replace the value functions in the linear programming representation by affine approximations. In this case, we obtain a linear programming representation with manageable number of decision variables, but the number of constraints grows exponentially with the number of flight legs. In Section 4, we describe how to solve the linear programming representation by using constraint generation and show that each constraint can be generated by solving a minimum-cost network flow problem. In Section 5, we exploit the minimum-cost network flow structure to a priori reduce the linear programming representation into a linear program whose numbers of constraints and decision variables grow only 3

4 linearly with the numbers of flight legs and itineraries. In Section 6, we give a practical interpretation of this reduced linear program. In Section 7, we present computational experiments that demonstrate the computational benefits from the reduced linear program. In Section 8, we conclude. 2 Problem Formulation We have a set of flight legs over an airline network that can be used to serve the requests for itineraries that arrive randomly over time. At the beginning of each time period, we decide whether each itinerary is open for sale or closed. If there is a request for an itinerary that is open for sale, then we serve this itinerary request, generating a revenue and consuming capacities on the flight legs that are included in the requested itinerary. A request for a closed itinerary simply leaves the system. The objective is to find a policy to open the itineraries for sale or close them over time so as to maximize the total expected revenue from the served itinerary requests. We let L be the set of flight legs in the airline network and J be the set of itineraries. If we serve a request for itinerary j, then we generate a revenue of f j and consume a ij units of capacity on flight leg i. We assume that there are no group reservations so that each served itinerary request consumes at most one unit of capacity on a flight leg. In other words, we have a ij {0, 1} for all i L, j J. Thus, an itinerary j is characterized by the set of flight legs {i L : a ij = 1} that it uses and the revenue f j that it generates. In certain revenue management settings, an itinerary is referred to as a product. The available capacity on flight leg i is c i. The itinerary requests arrive one by one over the time periods T = {1,..., τ}. Arrivals of the itinerary requests at different time periods are independent. The probability that there is a request for itinerary j at time period t is p jt. We assume that j J p jt = 1 so that there is exactly one itinerary request at each time period. If there is a positive probability of having no itinerary request at time period t, then we can capture this situation by defining a dummy itinerary ϕ satisfying f ϕ = 0, a iϕ = 0 for all i L and p ϕt = 1 j J p jt. We use x it to denote the remaining capacity on flight leg i at the beginning of time period t so that the vector x t = {x it : i L} captures the state of the capacities on the flight legs at the beginning of this time period. We let u jt take value one if we open itinerary j for sale at time period t and take value zero otherwise. The vector u t = {u jt : j J } captures the decisions at time period t. The set of feasible decisions is given by U(x t ) = {u t {0, 1} J : a ij u jt x it i L, j J }, ensuring that if we want to open itinerary j for sale and itinerary j uses flight leg i, then there has to be capacity available on flight leg i. In this case, we can formulate the problem as a dynamic program as { { V t (x t ) = max p jt f j u jt + V t+1 (x t u jt a ij e i )} }, (1) u t U(x t ) j J where we use e i to denote the L -dimensional unit vector with a one in the element corresponding to flight leg i. The boundary condition of the optimality equation above is V τ+1 ( ) = 0 and the state space can be written as X = {0, 1,..., c i}. Assuming that we have access to the value functions {V t (x t ) : x t X, t T }, it is not difficult to compute the optimal decisions at each time period. If we have a ij x it for all i L and f j + V t+1 (x t a ij e i ) V t+1 (x t ), then it is optimal to open 4

5 itinerary j for sale at time period t when the remaining capacities on the flight legs are given by the vector x t. Otherwise, it is optimal to close itinerary j. The size of the state space X in the optimality equation in (1) grows exponentially with the number of flight legs, rendering the computation of the value functions intractable for essentially any practical airline network. In this paper, we focus on an approximate solution method proposed by Adelman (2007). We describe this solution method in the next section. 3 Approximate Linear Program It is a standard result in Markov decision processes that an optimality equation with finite sets of states and decisions can be formulated as a linear program; see Puterman (1994). In general, this linear program is not computationally useful by itself since its numbers of decision variables and constraints are proportional to the number of possible states, but the linear program can serve as a starting point for constructing approximations to the value functions. Using c to denote the vector of initial capacities {c i : i L}, the linear program corresponding to the optimality equation in (1) can be written as min ϑ 1 (c) (2a) subject to ϑ t (x t ) { } p jt f j u jt + ϑ t+1 (x t u jt a ij e i ) j J x t X, u t U(x t ), t T, (2b) where the decision variables are {ϑ t (x t ) : x t X, t T }. For notational uniformity, we follow the convention that the values of the decision variables {ϑ τ+1 (x τ+1 ) : x τ+1 X } in the problem above are set to zero. It is possible to show that the optimal objective value of problem (2) is equal to the optimal total expected revenue V 1 (c) that is obtained through the optimality equation in (1). Therefore, the objective function of problem (2) evaluated at any feasible solution provides an upper bound on the optimal total expected revenue. Problem (2) is not computationally useful by itself since the numbers of decision variables and constraints in this problem increase exponentially with the number of flight legs. To deal with this difficulty, Adelman (2007) proposes approximating the value function V t (x t ) with an affine function of the form Ṽt(x t ) = θ t + v it x it for all t T, where {θ t : t T } and {v it : i L, t T } are adjustable parameters. To choose a set of values for {θ t : t T } and {v it : i L, t T }, we plug the approximation Ṽt(x t ) = θ t + v it x it into problem (2) to obtain the linear program subject to min θ 1 + v i1 c i (3a) θ t + v it x it j J { p jt f j u jt + θ t+1 + } v i,t+1 [x it u jt a ij ] x t X, u t U(x t ), t T, (3b) where the decision variables are {θ t : t T } and {v it : i L, t T }. Similar to problem (2), we follow the convention that the values of the decision variables θ τ+1 and {v i,τ+1 : i L} in the problem above 5

6 are set to zero. The number of constraints in problem (3) still increases exponentially with the number of flight legs, but the number of decision variables is T + L T, which can be manageable for airline networks of practical significance. Therefore, it can be possible to solve problem (3) by using constraint generation for practical airline networks. There are two uses of problem (3). First, it is possible to show that the optimal objective value of this problem provides an upper bound on the optimal total expected revenue. In particular, letting {θ t : t T } and {v it : i L, t T } be an optimal solution to problem (3) and ˆϑ t (x t ) = θ t + v it x it for all x t X, t T, we observe that { ˆϑ t (x t ) : x t X, t T } is a feasible solution to problem (2). Since the objective function of problem (2) evaluated at a feasible solution provides an upper bound on the optimal total expected revenue, it immediately follows that ˆϑ 1 (c) = θ 1 + v i1 c i is an upper bound on the optimal total expected revenue. Such an upper bound on the optimal total expected revenue becomes useful when we try to assess the optimality gap of approximate or heuristic control policies. Second, we can use an optimal solution to problem (3) to construct an approximate control policy. Recalling that {θt : t T } and {vit : i L, t T } denote an optimal solution to problem (3), the idea is to use Ṽ t (x t ) = θt + v it x it as an approximation to V t (x t ). In this case, if we have a ij x it for all i L and f j + θt+1 + v i,t+1 [x it a ij ] θt+1 + v i,t+1 x it, then we open itinerary j for sale at time period t when the remaining capacities on the flight legs are given by the vector x t. To gain some insight into this policy, we write the last inequality as f j a ij v i,t+1. We interpret vit in the value function approximation Ṽ t (x t ) = θt + v it x it as the marginal value or the opportunity cost of a unit of capacity on flight leg i at time period t, in which case, the right side of the inequality above corresponds to the total value of the capacities used by itinerary j. Therefore, the control policy obtained from problem (3) compares the revenue from itinerary j with the total value of the capacities consumed by itinerary j and it opens itinerary j for sale if the revenue justifies the total value of the consumed capacities, as long as there are enough seats. In network revenue management, the marginal value or the opportunity cost of a seat is referred to as the bid price. Intuitively, the bid price of a flight leg, being the marginal value of a unit of capacity, should depend on how much time is left until the time of departure. An attractive feature of problem (3) is that it naturally provides bid prices that are indeed dependent on time. Throughout the rest of the paper, we focus on efficient solution methods for problem (3). 4 Constraint Generation Noting that the number of decision variables in problem (3) is manageable but the number of constraints grows exponentially with the number of flight legs, a possible solution method for this problem is to use constraint generation. The idea behind constraint generation is to iteratively solve a master problem that has the same decision variables as problem (3), but has only a subset of the constraints. After solving the master problem, we check whether any one of the constraints in (3b) is violated by the solution to the current master problem. If there is one such constraint, then we add this constraint 6

7 to the master problem and resolve the master problem. Otherwise, the solution to the current master problem is optimal to problem (3) and we stop. The key to efficient implementation of constraint generation is to be able to check whether the solution to the current master problem violates any one of the constraints in (3b). Using the fact that j J p jt = 1, we write constraints (3b) as θ t θ t+1 j J { p jt f j } a ij v i,t+1 u jt + [v i,t+1 v it ] x it for all x t X, u t U(x t ), t T. In this case, if we let { θ t : t T } and {ṽ it : i L, t T } be the solution to the current master problem, then we can check whether this solution violates any one of the constraints in (3b) by solving { max x t X, u t U(x t) j J { p jt f j } a ij ṽ i,t+1 u jt + } [ṽ i,t+1 ṽ it ] x it for all t T. If the optimal objective value of the problem above exceeds θ t θ t+1 for a particular time period t, then letting x t and ũ t be the optimal solution to problem (4), the constraint corresponding to x t X, ũ t U( x t ) and t T in problem (3) is violated by the solution to the current master problem. We add this constraint to the current master problem and solve the master problem with the added constraint. Therefore, efficient implementation of constraint generation is dependent on our ability to solve problem (4) quickly. Noting the definitions of the state space X and the set of feasible decisions U(x t ), the decision variables x t and u t in problem (4) have integrality constraints. Adelman (2007) notes that we can relax the integrality constraints on the decision variables x t without loss of optimality, but we still have the integrality constraints on the decision variables u t in problem (4). Due to such integrality constraints, problem (4) is recognized as a drawback when using constraint generation to solve problem (3) for large airline networks. In this section, we alleviate such concerns by showing that the continuous relaxation of problem (4) has no integrality gap. Therefore, we can generate constraints by solving the continuous relaxation of problem (4), which can be done efficiently for practical airline networks. To show that the continuous relaxation of problem (4) has no integrality gap, we exploit the fact that the dual of this continuous relaxation turns out to be a minimum-cost network flow problem with integer cost data. Using the definitions of the state space X and the set of feasible decisions U(x t ), we write the continuous relaxation of problem (4) as max j J { p jt f j } a ij ṽ i,t+1 u jt + [ṽ i,t+1 ṽ it ] x it (5a) subject to x it c i i L (5b) a ij u jt x it i L, j J (5c) u jt 1 j J (5d) x it is free, u jt 0 i L, j J. (5e) (4) 7

8 We do not explicitly impose nonnegativity constraints on the decision variables {x it : i L} in the problem above, since constraints (5c) along with the nonnegativity constraints on the decision variables {u jt : j J } ensure that the decision variables {x it : i L} are nonnegative. Our goal is to show that the dual of problem (5) is a minimum-cost network flow problem. To that end, we associate the dual variables {α it : i L}, {β ijt : i L, j J }, and {γ jt : j J } respectively with constraints (5b), (5c) and (5d) in problem (5). In the dual of problem (5), the constraints associated with the decision variables {u jt : j J } are given by a ij β ijt + γ jt p jt [f j a ij ṽ i,t+1 ] for all j J. Using the slack variables {µ jt : j J } for these constraints, the dual of problem (5) is given by min c i α it + γ jt j J subject to (6a) α it β ijt = ṽ i,t+1 ṽ it i L (6b) j J { a ij β ijt + γ jt µ jt = p jt f j } a ij ṽ i,t+1 j J (6c) α it 0, β ijt 0, γ jt 0, µ jt 0 i L, j J. (6d) The next proposition shows that problem (6) is a minimum-cost network flow problem. Proposition 1 Problem (6) is a minimum-cost network flow problem. Proof We establish the result by showing that all of the decision variables in problem (6) can be associated with the arcs in a directed network and all of the constraints in problem (6) correspond to the flow balance constraints of the nodes in this directed network. We consider a network with two sets of nodes N 1 = {i : i L} and N 2 = {j : j J } and an additional sink node. Constraints (6b) and (6c) are respectively the flow balance constraints for the nodes in N 1 and N 2. The flow balance constraint for the sink node is redundant and it is omitted in problem (6). The decision variable α it in problem (6) corresponds to an arc that connects the sink node to node i N 1. If a ij = 1, then the decision variable β ijt corresponds to an arc from node i N 1 to node j N 2. On the other hand, if a ij = 0, then the decision variable β ijt corresponds to an arc from node i N 1 to the sink node. Therefore, the decision variable β ijt does not appear in the flow balance constraint for node j N 2 when a ij = 0. The decision variable γ jt corresponds to an arc from the sink node to node j N 2. The decision variable µ jt corresponds to an arc from node j N 2 to the sink node. The demands of nodes i N 1 and j N 2 are respectively ṽ i,t+1 ṽ it and p jt [f j a ij ṽ i,t+1 ]. Figure 1 shows the structure of problem (6) for a small airline network. The upper portion of this figure shows the airline network. The flight legs are labeled by {1, 2} and represented in solid arcs, whereas the itineraries are labeled by {a, b, c} and represented in dotted arcs. The lower portion of the figure shows the minimum-cost network flow problem corresponding to problem (6). Each arc in Figure 1 corresponds to a decision variable in problem (6) and each node with the exception of the sink node corresponds to a constraint. Proposition 1 immediately implies that problem (5) has no integrality gap. In particular, problems (5) and (6) form a primal-dual pair so that an optimal dual solution to problem (6) is an optimal 8

9 solution to problem (5). Since problem (6) is a minimum-cost network flow problem with integer cost data, there exists an integer-valued optimal dual solution to problem (6) and this integer-valued optimal dual solution can be obtained by solving problem (6) by using the simplex algorithm; see Vanderbei (1997). Therefore, there exists an integer-valued optimal solution to problem (5), establishing that this problem has no integrality gap. We do not make any assumptions in Proposition 1 on the structure of the underlying airline network, implying that problem (5) has no integrality gap for any airline network structure. Furthermore, we can solve the dual of problem (5) by using minimum-cost network flow algorithms, which are quite efficient. These observations indicate that constraint generation is efficient to implement even for large airline networks with arbitrary topologies. It is also worthwhile to observe that it is necessary to have a ij {0, 1} for all i L, j J for Proposition 1 to hold. Otherwise, constraints (6c) cannot be interpreted as flow balance constraints. The fact that problem (5) has no integrality gap is of interest by itself, but we also build on this result in the next section to show that we can reduce problem (3) to a linear program whose size grows linearly with the numbers of flight legs and itineraries. 5 Reducing the Number of Constraints By the discussion in the previous section, if we use constraint generation to solve problem (3), then we can generate each constraint efficiently. However, although generating each constraint is efficient, we may still end up generating a large number of constraints to ultimately obtain the optimal solution by using constraint generation. In this section, we establish a complementary result, showing that we can a priori reduce problem (3) to a linear program whose numbers of constraints and decision variables grow only linearly with the numbers of flight legs and itineraries. This result significantly enhances the computational tractability of problem (3). To facilitate our discussion, we use v to denote the collection of decision variables {v it : i L, t T } in problem (3) and define Π t (v) as { { Π t (v) = max p jt f j } a ij v i,t+1 u jt + [v i,t+1 v it ] x it }. (7) x t X, u t U(x t ) j J The notation Π t (v) suggests that the quantity on the right side above depends on the whole collection v = {v it : i L, t T }, but Π t (v) actually depends only on v t and v t+1. Our choice of notation for Π t (v) is motivated by notational brevity. With this definition of Π t (v), constraints (3b) in problem (3) can succinctly be written as θ t θ t+1 Π t (v) for all t T and we can write problem (3) as min θ 1 + v i1 c i (8a) subject to θ t θ t+1 Π t (v) t T, (8b) where the decision variables are {θ t : t T } and {v it : i L, t T }. We make two useful observations in problem (8). First, the values of the decision variables {θ t : t T } and {v it : i L, t T } have to 9

10 satisfy θ t = θ t+1 + Π t (v) for all t T in an optimal solution to problem (8). Adding the last equality over all t T and noting the convention that the value of the decision variable θ τ+1 is set to zero, it follows that θ 1 = t T Π t(v) in an optimal solution to problem (8). Second, we observe that problems (4) and (7) have the same structure. Therefore, Proposition 1 implies that the continuous relaxation of problem (7) has no integrality gap. By using the same argument that we use just before Proposition 1, we write the dual of the continuous relaxation of problem (7) as Π t (v) = min c i α it + γ jt j J subject to (9a) α it β ijt = v i,t+1 v it i L (9b) j J { a ij β ijt + γ jt p jt f j } a ij v i,t+1 j J (9c) α it 0, β ijt 0, γ jt 0 i L, j J, (9d) in which case, the problem above and problem (7) share the same optimal objective value and this common optimal objective value is denoted by Π t (v). In the next proposition, we build on the two observations above to reduce problem (3) to a linear program whose numbers of constraints and decision variables grow linearly with the numbers of flight legs and itineraries. Proposition 2 Problem (3) is equivalent to the problem min c i α it + γ jt + c i v i1 t T t T j J subject to (10a) α it β ijt = v i,t+1 v it i L, t T (10b) j J { a ij β ijt + γ jt p jt f j } a ij v i,t+1 j J, t T (10c) α it 0, β ijt 0, γ jt 0, v it is free i L, j J, t T. (10d) In particular, problems (3) and (10) have the same optimal objective value and given an optimal solution to one problem, we can construct an optimal solution to the other. Proof Since problems (3) and (8) are equivalent to each other, we show that problem (8) is equivalent to problem (10). We use Z and ζ to respectively denote the optimal objective values of problems (8) and (10). First, we show that Z ζ. Since the optimal objective value of problem (9) gives Π t (v), problem (10) is equivalent to minimizing t T Π t(v) + c i v i1 over v, which implies that ζ = min v { t T Π t(v) + c i v i1 }. So, letting v be the optimal solution to the last optimization problem and defining θt as θt = τ s=t Π s(v ), the solution {θt : t T } and v = {vit : i L, t T } is feasible to problem (8) and we obtain Z t T Π t(v ) + c i vi1 = ζ. Second, we show that ζ Z. We let {θt : t T } and v = {vit : i L, t T } be an optimal solution to problem (8). To construct a feasible solution to problem (10), we solve problem 10

11 (9) for each t T after replacing the right side of constraints (9b) with {vi,t+1 v it : i L} and the right side of constraints (9c) with {p jt [f j a ij vi,t+1 ] : j J }. Letting {α it : i L}, {βijt : i L, j J } and {γ jt : j J } be an optimal solution to problem (9) for each t T, we have Π t (v ) = c i αit + j J γ jt for all t T. Also, noting constraints (9b) and (9c), the solution {αit : i L, t T }, {β ijt : i L, j J, t T }, {γ jt : j J, t T } and {v it : i L, t T } is feasible to problem (10), but not necessarily optimal. Therefore, we obtain ζ c i αit + γjt + c i vi1 = Π t (v ) + c i vi1 = θ1 + c i vi1 = Z, t T t T j J t T where the second equality uses the fact θ 1 = t T Π t(v ) is always satisfied by an optimal solution to problem (8). Thus, we obtain Z = ζ and we can use the construction above to get an optimal solution to one of the problems (3) and (10) by using an optimal solution to the other. The proof of Proposition 2 shows that if {αit : i L, t T }, {β ijt : i L, j J, t T }, {γjt : j J, t T } and v = {vit : i L, t T } is an optimal solution to problem (10), then letting θt = τ s=t Π s(v ), the solution {θt : t T } and {vit : i L, t T } is optimal to problem (3). Therefore, we can recover an optimal solution to problem (3) from problem (10). 6 Practical Interpretation At first glance, it is difficult to see an intuitive relationship between problem (10) and the network revenue management problem that we are interested in solving. In this section, we give a practical interpretation for problem (10) that clarifies this relationship. This practical interpretation also becomes useful to construct an efficient solution method for problem (10) later in this section. Associating the dual variables {w it : i L, t T } and {y jt : j J, t T } respectively with constraints (10b) and (10c) in problem (10), we write the dual of this problem as max t T j J p jt f j y jt (11a) subject to w i1 = c i i L (11b) w it = w i,t 1 j J p j,t 1 a ij y j,t 1 i L, t T \ {1} (11c) a ij y jt w it i L, j J, t T (11d) y jt 1 j J, t T (11e) w it is free, y jt 0 i L, j J, t T. (11f) Constraints (11b) and (11c) are associated with the decision variables {v it : i L, t T } in problem (10), whereas the constraints (11d) and (11e) are respectively associated with the decision variables {β ijt : i L, j J, t T } and {γ jt : j J, t T }. We observe that the constraints associated with the decision variables {α it : i L, t T } in the dual of problem (10) are given by w it c i for all i L, t T, but constraints (11b) and (11c) already imply that c i = w i1 w i2... w iτ for all i L. Therefore, the constraints associated with the decision variables {α it : i L, t T } 11

12 are redundant in problem (11) and they are omitted. We observe that the size of the problem above increases linearly with the numbers of flight legs and itineraries. We view the decision variable w it in problem (11) as the expected remaining capacity on flight leg i at the beginning of time period t and the decision variable y jt as the probability with which we open itinerary j for sale at time period t. At time period t, we open each itinerary j for sale with probability y jt, independent of the other itineraries. With probability 1 y jt, itinerary j is closed at time period t. To make a sale for itinerary j at time period t, we need to have itinerary j open and have a request for this itinerary, which happens with probability p jt y jt. Thus, the objective function of problem (11) accounts for the total expected revenue from the served itinerary requests. Constraints (11b) initialize the expected remaining capacities. Noting that j J p jt a ij y jt is the expected capacity consumption on flight leg i at time period t, constraints (11c) compute the expected remaining capacities at the next time period as a function of the expected remaining capacities and itinerary sales at the current time period. Constraints (11d) ensure that the expected capacity consumption on flight leg i at time period t conditional on the fact that there is a request for itinerary j does not exceed the expected remaining capacity on flight leg i. The number of constraints in problem (11) is substantially smaller than the number of constraints in problem (3), but problem (11) still has L T + L J T constraints, which may be too many to solve this problem directly by using a linear programming solver. Noting that constraints (11d) can be replaced by nonnegativity constraints on the decision variables {w it : i L, t T } whenever itinerary j does not use flight leg i, if we let L be the maximum number of flight legs that an itinerary uses, then we can reduce the number of constraints to L T + L J T, but this may also be too many. The key observation that allows us to solve problem (11) efficiently is that the expected remaining capacities {w it : i L, t T } are initialized to the total available capacities by constraints (11b). Since a ij {0, 1} and y jt is bounded by one, constraints (11d) are not active whenever w it exceeds one, which indicates that we do not expect constraints (11d) to be tight over a large portion of the selling horizon. This observation motivates solving problem (11) by using constraint generation, where we iteratively solve a master problem that has the same decision variables as problem (11) and includes all of constraints (11b), (11c) and (11e), but has only a subset of constraints (11d). After solving the master problem, we check whether any one of the constraints in (11d) are violated by the solution to the current master problem. If there is one, then we add this constraint to the master problem and resolve it. When compared with solving problem (3) by using constraint generation, applying constraint generation on problem (11) provides two potential advantages. First, we can simply enumerate over all i L, j J and t T to check whether any one of the constraints in (11d) are violated by the solution to the current master problem. We do not need to solve a separate problem to identify violated constraints. Second, we may be interested in solving problem (11) not only to obtain a control policy, but also to obtain the upper bound on the optimal total expected revenue provided by the optimal objective value of this problem. In practice, it may not be possible to solve problem (11) to optimality by using constraint generation. However, since problem (11) is a maximization problem and the master problem includes a subset of its constraints, the optimal objective value of the master problem is always an upper 12

13 bound on the optimal objective value of problem (11). Therefore, if we stop constraint generation at an intermediate iteration, then the optimal objective value of the master problem naturally provides an upper bound on the optimal total expected revenue. This observation does not immediately apply when we solve problem (3) by using constraint generation since this problem is a minimization problem, but as Proposition 3 in Adelman (2007) shows, we can use the slack variables of the constraints in problem (3) to obtain an upper bound on the optimal total expected revenue even when we stop constraint generation for problem (3) at an intermediate iteration. Problem (11) has a surprising connection to the earlier literature. Kunnumkal and Topaloglu (2010) propose problem (11) as a deterministic linear programming approximation to the network revenue management problem. The strongest result they can give for problem (11) is that the optimal objective value of this problem is greater than or equal to the optimal objective value of problem (3). Their result gives the impression that problem (3) is a stronger approximation for the network revenue management problem than problem (11) as it potentially yields a tighter upper bound, but our findings in this paper show that this is not the case and problems (3) and (11) are equivalent to each other. Another interesting observation is that Kunnumkal and Topaloglu (2010) obtain problem (11) by using Lagrangian relaxation to relax the capacity availability constraints in the dynamic programming formulation in (1). This observation naturally raises the question of whether using Lagrangian relaxation on the dynamic programming formulation of a general decision making problem, as is done in Kunnumkal and Topaloglu (2010), is equivalent to using affine approximations to the value functions, as is done in Adelman (2007). It is not too difficult to see that this is not true in general. To give a simple example, if the decisions made at one time period has no effect on the future decisions, then using a value function approximation of zero at each time period would still give the optimal policy since there is no need to use value functions to assess the impact of the current decisions on the future. So, we can obtain the optimal policy trivially by using affine value function approximations with zero slopes and zero intercepts. In contrast, relaxing certain constraints through Lagrangian relaxation at each time period may certainly result in suboptimal decisions, especially when there are integrality constraints on the decisions. Thus, using affine value function approximations is not equivalent to using Lagrangian relaxation in the dynamic programming formulation of a general decision making problem, but in the network revenue management setting, our results in this paper establish such an equivalence. There may be other ways to show the equivalence between problems (3) and (11). Subsequent to our work, Vossen and Zhang (2012) gave an alternative proof for the equivalence between problems (3) and (11) by aggregating the decision variables in the dual of problem (3). In their work, Vossen and Zhang (2012) take problems (3) and (11) as given and show how to obtain the optimal solution to one problem by using the optimal solution to the other. In contrast, we follow a constructive approach, showing the equivalence between problems (3) and (11) as we construct problem (11). 7 Computational Experiments In this section, we provide computational experiments that compare the solution times when we use constraint generation to solve problems (3) and (11). 13

14 7.1 Experimental Setup In our test problems, we work with two different types of airline networks. In the first type of network, we have a single hub serving N spokes. There is a flight leg from the hub to each spoke and another one from each spoke to the hub. Thus, the number of flight legs is 2N. It is possible to go from each origin to each destination in the airline network so that the number of possible origin-destination pairs is N(N + 1). Going from one spoke to another requires taking two flight legs through the hub. Going from a spoke to the hub or from the hub to a spoke is possible through a direct flight leg. Figure 2.a shows the structure of the first type of network with N = 8. In the second type of network, we have two hubs serving a total of N spokes. The first half of the spokes are connected to the first hub and the second half of the spokes are connected to the second hub. There is a flight leg from each spoke to a hub and another flight leg from the corresponding hub to each spoke. There are also two flight legs that connect the two hubs in two directions. Thus, the number of flight legs is 2N + 2. It is possible to go from each origin to each destination in the airline network, in which case, the number of possible origin-destination pairs is (N +1)(N +2). Going from one spoke to another requires taking two or three flight legs, whereas going from a spoke to a hub or from a hub to a spoke requires taking one or two flight legs. It is possible to go from one of the hubs to the other through a direct flight leg. Figure 2.b shows the structure of the second type of network with N = 8. We have a high-fare and a low-fare itinerary associated with each origin-destination pair in the airline network. The fare associated with a high-fare itinerary is four times the fare associated with the corresponding low-fare itinerary. The arrival probabilities are calibrated so that the probability of getting a request for a high-fare itinerary increases over time, whereas the probability of getting a request for a low-fare itinerary decreases. Since the total expected demand for the capacity on flight leg i is given by t T j J a ij p jt, we measure the tightness of the leg capacities by t T j J α = a ij p jt c. i In our test problems, we vary the number of time periods in the selling horizon over {600, 800, 1,000}, the number of spokes in the airline network over {8, 12} and the tightness of the leg capacities over {1.0, 1.3, 1.6}. This experimental setup yields 18 test problems for each type of airline network. 7.2 Implementation of Constraint Generation We experimented with a number of constraint generation strategies for solving problem (3). The following strategy consistently performed the best for our test problems. Letting { θ t : t T } and {ṽ it : i L, t T } be the solution to the current master problem, to check whether this solution violates any of constraints (3b) in problem (3), we solve problem (4) for each time period. If the optimal objective value of problem (4) exceeds θ t θ t+1 for a time period t, then using x t and ũ t to denote an optimal solution to problem (4), constraint (3b) corresponding to x t X, ũ t U( x t ) and t T is violated. We add this constraint to the master problem. Once we find five violated constraints, we stop looking for others. We solve the master problem with the added constraints and this concludes 14

15 one iteration of constraint generation. As the iterations progress, the constraints added in the earlier iterations may not be relevant any more and become loose. If the number of loose constraints exceeds 70% of the number of constraints in the master problem, then we remove all loose constraints from the master problem. Finally, Adelman (2007) shows that the decision variables {v it : i L, t T } satisfy v i1 v i2... v iτ for all i L in an optimal solution to problem (3). We add all of these constraints to the master problem before we even start constraint generation. Adding these constraints at the beginning significantly speeds up the overall performance of constraint generation. The same observation is also made by Adelman (2007). We solve problem (11) by using constraint generation as well. For our test problems, the best performing constraint generation strategy we found is as follows. We add all of constraints (11b), (11c) and (11e) to the master problem at the beginning. Therefore, we only look for violations of constraints (11d). After solving the master problem at the current iteration, we enumerate over all i L, j J and t T to find which of constraints (11d) are violated by the solution to the current master problem. We add all of the violated constraints to the master problem and solve the master problem with the added constraints. This concludes one iteration of constraint generation. We note that although we solve problem (11) by using constraint generation, generating constraints for this problem is trivial. 7.3 Computational Results Tables 1 and 2 summarize our main computational results, where the two tables respectively focus on the test problems with one and two hubs. In these tables, the first column shows the characteristics of the test problem by using the triplet (τ, N, α), where τ is the number of time periods in the selling horizon, N is the number of spokes and α characterizes the tightness of the leg capacities. Recalling that we have a high-fare and a low-fare itinerary for each origin-destination pair, if there are N spokes in the airline network, then the numbers of flight legs and itineraries are respectively 2N and 2N(N +1) for the airline networks with one hub, whereas the numbers of flight legs and itineraries are respectively 2N + 2 and 2(N + 1)(N + 2) for the airline networks with two hubs. The second to fifth columns in Tables 1 and 2 show the performance of constraint generation when applied to problem (3). In particular, the second column shows the CPU seconds required to solve problem (3) to optimality. The third column shows the CPU seconds required to solve problem (3) with 1% optimality gap. The fourth column shows what percentage of the CPU seconds is spent on generating constraints. The remaining percentage of the CPU seconds is spent on solving the master problem. The fifth column shows the total number of constraints generated to solve problem (3) to optimality. The interpretations of the sixth, seventh, eighth and ninth columns are similar to those of the previous four, but these columns focus on constraint generation when applied to problem (11). The last two columns in Tables 1 and 2 compare the solution times for problems (3) and (11). In particular, the tenth column shows the ratio between the CPU seconds required to solve problems (3) and (11) to optimality. The eleventh column shows the ratio between the CPU seconds required to solve problems (3) and (11) with 1% optimality gap. Our computational implementation is carried out by using Gurobi 4.5 as the linear programming solver. In our computational experiments, we focus on demonstrating the computational savings obtained by solving problem (11) by using constraint generation instead of solving problem 15

16 (3). We do not test the performance of the policies obtained from these problems. Adelman (2007) and Topaloglu (2009) compare the performance of the policies with a variety of benchmark strategies and report quite favorable results. The results in Tables 1 and 2 indicate that problem (11) provides significant computational savings over problem (3) in terms of CPU seconds. Over all of our test problems, the longest CPU seconds for problem (11) is 33 seconds, but problem (3) may take up to 1,931 seconds for some of the larger test problems with large number of time periods in the selling horizon and large number of spokes in the airline network. Similar observations apply if we are interested in obtaining a solution with 1% optimality gap. By using problem (11), we can obtain a solution with 1% optimality gap within 30 seconds in the worst case. For most of the test problems, we can obtain a solution with 1% optimality gap within three seconds. On the other hand, problem (3) may take up to 449 seconds to solve with 1% optimality gap. The ratio between the CPU seconds required to solve problems (3) and (11) can be as high as 135. The average of the ratios between the CPU seconds required to solve the two problems comes out to be 52. If we are interested in obtaining a solution with 1% optimality gap, then the average ratio between the CPU seconds is 71. To get a feel for the problem characteristics that affect the CPU seconds for problems (3) and (11), we observe that the CPU seconds for both problems increase as the number of time periods in the selling horizon and the number of spokes increase, which is not surprising, since these problem characteristics directly affect the numbers of decision variables and constraints in problems (3) and (11). On the other hand, we observe that the CPU seconds for both problems also increase as the leg capacities get tighter. Although the numbers of decision variables and constraints do not depend on the tightness of the leg capacities, we need to generate more constraints to obtain the optimal solution and this translates into longer CPU seconds. It is also worthwhile to note that the largest values for the ratios between the CPU seconds required to solve problems (3) and (11) correspond to the test problems with tight leg capacities. Therefore, although it takes more time to solve either of the two problems as the leg capacities get tighter, problem (11) is affected less. It is encouraging that problem (11) provides the largest computational savings when problem (3) is particularly difficult to solve. To sum up, our computational results indicate that using constraint generation to solve problem (11) is significantly more efficient than working with problem (3) directly. Problem (11) maintains its advantage across a variety of sizes of test problems and the gaps in the CPU seconds are most noticeable for the larger test problems with tighter leg capacities. Given these considerations, reducing problem (3) to problem (11) and solving the latter problem by using constraint generation is clearly a viable alternative to solving problem (3) by using constraint generation. The CPU seconds reported for problem (11) in Tables 1 and 2 correspond to the case where we solve this problem by using constraint generation as described in Section 7.2. A natural question is how much benefit we obtain from using constraint generation to solve problem (11). To answer this question, Table 3 compares the CPU seconds for problem (11) when we solve this problem by using constraint generation and when we solve this problem directly by using a linear programming solver. The left and right sides of Table 3 respectively focus on the test problems with one and two hubs. The first column 16