Dynamic Bid Prices in Revenue Management

Transcription

1 OPERATIONS RESEARCH Vol. 55, No. 4, July August 2007, pp ssn X essn nforms do /opre INFORMS Dynamc Bd Prces n Revenue Management Danel Adelman Graduate School of Busness, Unversty of Chcago, Chcago, Illnos 60637, dan.adelman@chcagogsb.edu We formally derve the standard determnstc lnear program LP) for bd-prce control by makng an affne functonal approxmaton to the optmal dynamc programmng value functon. Ths affne functonal approxmaton gves rse to a new LP that yelds tghter bounds than the standard LP. Whereas the standard LP computes statc bd prces, our LP computes a tme traectory of bd prces. We show that there exst dynamc bd prces, optmal for the LP, that are ndvdually monotone wth respect to tme. We provde a column generaton procedure for solvng the LP wthn a desred optmalty tolerance, and present numercal results on computatonal and economc performance. Subect classfcatons: revenue management, prcng: network; bd prces; dynamc programmng/optmal control: applcatons, approxmate. Area of revew: Manufacturng, Servce, and Supply Chan Operatons. Hstory: Receved July 2005; revson receved June 2006; accepted June Publshed onlne n Artcles n Advance July 20, Introducton The noton of bd-prce controls Tallur and van Ryzn 1998, Smpson 1989, Wllamson 1992) has been a powerful and nfluental soluton concept n revenue management for more than a decade. Maor arlnes, for nstance, have used bd-prce control polces for decdng when to open and close customer fare classes for sale. More generally, they can be used n revenue management settngs where the supply of resources s fxed and customer requests arrve over a fnte tme horzon to consume varous resource confguratons. The arrvng requests must ether be accepted or reected, wth the obectve of maxmzng expected proft over the tme horzon. The basc dea of bd-prce control s smple: Accept the request f the revenue earned exceeds the value of the resources consumed as measured by bd prces. Typcally, the bd prces are computed as optmal dual prces,.e., margnal resource values, of a smple determnstc lnear program. Whle the system under control s dynamc, to date there only exst models for computng statc bd prces, whch do not change as a functon of tme. The effect of dynamcally changng prces s created by re-solvng the statc bd-prce model through tme as the system evolves. One purpose of ths paper s to derve and explore a tractable model for computng a tme traectory of bd prces all at once wthn a sngle model. In mplementaton ths model may stll be re-solved over tme, and ths turns out to be a good strategy, but hopefully the prces wll be more accurate and effectve due to the fact that system dynamcs must somehow be taken nto account n computng them. The second purpose of ths paper s to further formalze the connecton between bd-prce control n revenue management and dynamc programmng, beyond Tallur and van Ryzn 1998). Gven the standard bd-prce lnear program, Tallur and van Ryzn 1998) heurstcally nterpret the embedded prcng mechansm as makng a lnear functonal approxmaton to the dynamc programmng value functon. However, the lnear program tself has never been derved drectly from the dynamc program. Rather, ts prces are heurstcally nterpreted ex post n terms of a value functon approxmaton. In ths paper, we start wth the dynamc program, and a pror make an affne functonal approxmaton to the value functon,.e., based on a lnear combnaton of affne bass functons. If r s the number of seats resources) stll avalable for flght resource type) n perod t, then we approxmate the value of state vector r by v t r t + V t r for parameters V t and t. From ths, we derve the standard bd-prce lnear program. We do ths by substtutng ths approxmaton nto a lnear programmng formulaton of the optmalty equatons, and then aggregatng constrants n the dual. We thereby show that the ntermedate lnear program, whch computes dynamc bd prces V t, provdes stronger bounds than the standard bd-prce lnear program. Furthermore, we may nterpret the standard, statc bd prces as approxmatng the dynamc ones. Ths analyss also leads to a new, alternatve proof that the standard lnear program provdes an upper bound on the optmal dynamc programmng value functon. Prevous proofs were based on sample-path arguments, for example, see the dscusson n Bertsmas and Popescu 2003). We provde numercal results showng that the standard bound can be as much as 50.4% larger than our new bound. Ths 647

2 648 Operatons Research 554), pp , 2007 INFORMS mproved bound can be used by researchers to numercally obtan mproved optmalty guarantees for new heurstcs. Ths s the frst paper to use the lnear programmng approach to approxmate dynamc programmng n the context of revenue management. Ths approach was frst consdered by Schwetzer and Sedmann 1985), and more recently by de Faras and Van Roy 2003). The dea s to approxmate the value functon wth a lnear combnaton of bass functons, and then substtute the approxmaton nto the lnear programmng formulaton of the optmalty equatons. To our knowledge, ths approach has not been consdered prevously n the fnte-horzon case, such as we have here. We show that ths approach has foundatonal sgnfcance n the context of bd-prce controls n revenue management. Indeed, bd prces need not be assocated wth a lnear functonal approxmaton, as they are usually thought of. In general, they may be vewed as prces on bass-weghted flowbalance constrants that appear n a prmal problem, for any arbtrary collecton of bass functons used to approxmate the value functon. The standard bd prces are assocated wth lnear bass functons. We show how to get ths approach to work effectvely n the affne bass case. Hopefully, our progress wll ad future researchers n explorng stronger functonal forms for bass functons, or n applyng these methods on other varatons of revenue management problems. For nstance, n the general approxmaton case, we establsh that obtanng feasblty s trval and that there s a smple optmalty guarantee. We demonstrate the mportance of structural propertes satsfed by optmal bd-prce traectores. We prove that the optmal bd prces are ndvdually nonncreasng over tme, and by addng ths as dual constrants, we obtan substantal computatonal speedup by as much as a factor of 85. It turns out that ths s key to mplementng the model on real-world ndustral-szed nstances. Ths structural result also permts us to nterpret the approxmate lnear program as fndng a tme threshold polcy that s optmal wth respect to an approxmate polcy evaluaton. We also report on an effectve column generaton algorthm for solvng the model wthn any desred optmalty tolerance. The economc performance of the resultng polcy, wth re-solvng, beats standard bd-prce control by as much as 21.4% n our numercal tests. Ths s substantal n the context of arlne revenues, for nstance. Prevous work has consdered varous other, but qute dfferent, approxmate dynamc programmng approaches to revenue management. As mentoned above, Tallur and van Ryzn 1998) ntepret varous revenue management models n terms of approxmatng the value functon. Bertsmas and Popescu 2003) consder usng the exact value functons of math programmng models, n partcular, the standard bd-prce lnear program, to compute opportunty costs. Our model could n fact be used n ther procedure, and therefore complements ther work. Bertsmas and de Boer 2005) use smulaton-based methods to estmate gradents useful n updatng bookng lmts. The latter two references, the book Tallur and van Ryzn 2004) and the revew artcle McGll and van Ryzn 1999), provde excellent, detaled revews of the lterature on other approaches to, and aspects of, the revenue management problem. Ths paper s organzed as follows. In 2, we provde some background, ncludng a standard dynamc programmng formulaton and the standard bd-prce control models. Then, n 3, we consder the general case of approxmatng the value functon wth a lnear combnaton of bass functons. Ths leads to a prmal model wth an nterestng structural form, as well as results regardng feasblty and optmalty guarantees. The heart of the paper s 4, whch makes the affne functonal approxmaton, derves the standard bd-prce lnear program from t, provdes structural propertes, and gves a useful column generaton algorthm. We provde numercal results n 5 on computatonal tmes, comparatve bound strength, and polcy performance. 2. Prelmnares In ths secton, we provde the basc formulatons and notaton we wll use throughout the paper. We frst present a known formulaton of the network revenue management problem as a Markov decson process. We then formulate the standard lnear program used for bd-prce control Markov Decson Process Formulaton Our formulaton essentally follows Cooper and Homemde-Mello 2003), and smlar models have appeared n the lterature McGll and van Ryzn 1999). The model s a fnte-horzon dscrete-tme Markov decson process wth tme unts t = 1, where s the last perod n whch sales can occur. The obectve s to maxmze the total expected revenue. There are m resources and n product or customer classes. There s an m-vector of resources c c, and an m n matrx A a. The entry c > 0 represents the nteger amount of resource avalable at the begnnng of the tme horzon t = 1), and the entry a represents the nteger amount of resource requred by a class customer. To ease notaton, we reserve the symbols,, and t for resources, classes, and tme, respectvely, and we omt wrtng the ndex sets. For example, the notaton means 1 m and means 1 m. In each perod, at most one customer arrves. A class customer arrves n perod t wth probablty p t, and wth probablty 1 p t no customer arrves. When a customer of class arrves, the controller must decde whether to accept or reect ths customer. If the controller chooses to accept, she receves f > 0 n revenue, and resources are consumed accordng to the th column of matrx A, denoted by A. Our analyss permts the revenue to depend on tme f we so wshed,.e., t could be gven by f t.) If there are not enough resources avalable to satsfy the

3 Operatons Research 554), pp , 2007 INFORMS 649 request, then the request must be reected and there s no reward. Even f enough resources are avalable, a request may be reected f t s more proftable to reserve resources for potental future customers. The state at the begnnng of any perod t s an m-vector of resources r that satsfes r R r m + r 0 1 c In Perod 1, we have r = c, and so for convenence we defne c f t = 1 R t = R f t = 2 Gven an ntal state c and the arrval probabltes p t, not all states are reachable, and so the sets R t are larger than needed. The controller chooses a vector u 0 1 n, where u = 1 f the controller would accept a class customer f one arrves ths perod, and u = 0 otherwse. Gven that the system s n state r, ths vector must satsfy u r = u 0 1 n r A u r R to ensure that there are enough resources avalable to satsfy a class request f u = 1. Gven r and u, wth probablty p t the reward for the current perod wll be f u and the next state wll be r A u. Wth probablty 1 p t, the reward n the current perod wll be zero and the next state wll be r. Let v t r denote the expected cost-to-go over perods t startng from state r at the begnnng of perod t. Defne the termnal value v +1 r 0 r, and assume that ths holds throughout. The optmalty equatons are { v t r = max p t f u + v t+1 r A u u r + 1 } p t )v t+1 r t r R t 1) It s easy to show that an optmal polcy chooses, n state r and perod t, 1 f r A f v t+1 r v t+1 r A u t r = 0 otherwse t r R t 2) where f v t+1 r v t+1 r A means that the revenue for a class customer meets or exceeds the opportunty cost of the resources that would be consumed. Note that the control polcy s nested: for any two classes 1 and 2 for the same tnerary A 1 = A 2, f f2 >f 1 then the polcy accepts 2 f t accepts 1. In prncple, the optmal value functon at the ntal state c can be computed by the lnear program D0) mnv 1 c v v t r p t f u +v t+1 r A u + 1 p t )v t+1 r t r R t u r 3) wth decson varables v t r t r. Ths can be shown from complementary slackness. It s apparently nontradtonal to consder lnear programmng for stochastc dynamc programs n the fnte-horzon case. However, we can vew ths dynamc program as a postve dynamc program Puterman 1994) wth the tme ndex ncluded n the state space, and therefore general results for that model, ncludng ones dealng wth lnear programmng, hold here. For notatonal convenence, we wrte the nequalty above as v t r p t f u + E v t+1 R t+1 r u where R t+1 R t+1 s the random vector representng the resource state at the begnnng of perod t + 1. Throughout the paper, the above condtonal expectaton reles only on the one-step transton probabltes, whch nvolve the demand probabltes p t as shown above. The followng result s elementary. It shows that any feasble soluton to D0) provdes an upper bound on the expected proft-to-go for an optmal polcy from every state and tme perod. Later, we wll consder feasble solutons that arse from specal functonal forms for v t. Proposton 1. Suppose that v t solves the optmalty Equatons 1), and ˆv t s any feasble soluton to D0). Then, ˆv t r v t r t r R t Proof. We show ths by nducton. For all r R, u r, dual feasblty 3) mples ˆv r Hence, ˆv r max p f u u r p f u = v r where the equalty s the optmalty Equaton 1) for perod t =. Now suppose the result s true for t + 1. Then, for all r R t, u r, by dual feasblty we have ˆv t r p t f u + E ˆv t+1 R t+1 r u p t f u + E v t+1 R t+1 r u

4 650 Operatons Research 554), pp , 2007 INFORMS In partcular, ˆv t r max u r p t f u + E v t+1 R t+1 r u = v t r where the last equalty s the optmalty equaton for perod t Standard LP for Bd-Prce Control In general, 1) and D0) are ntractable because of the hgh-dmensonal state space, known as Bellman s curse of dmensonalty. Consequently, the standard approach to revenue management solves a much smpler lnear program that gnores tme dynamcs. Let Y denote the expected number of unts of class we plan to satsfy over the fnte horzon. Then, the standard lnear program for bd-prce control s LP z LP = max f Y 4) Y a Y c 5) The dual of LP) s mn V 0 Y t V c + p t ) t a V + f V 0 p t 6) where V are dual prces on 5) and are dual prces on the rght-hand nequalty of 6). Let V and denote optmal dual prces. The dea of bd-prce control s to accept a class arrval f f a V, and reect t otherwse. In practce, LP) s re-solved frequently. Wllamson 1992) shows that ths polcy performs qute well, and n fact ths has been a wdely used approach to revenue management n ndustry. 3. Generalzed Functonal Approxmatons We now lay down a general framework for approxmately solvng D0). Consder a set of bass functons k r for all k n some set that ndexes them, and approxmate v t r ˆv t r = k V t k k r t r R t 7) where V t k s a parameter that weghts the kth bass functon at tme t. Note that f we specfy a separate bass functon for every possble state r, then we recover the exact model D0). The general dea s ths: By substtutng the approxmaton 7) nto D0), we restrct t nto an optmzaton problem over only the parameters V t k. If there are relatvely few parameters and there s enough structure n the k functons, then hopefully the restrcted optmzaton problem wll be consderably easer to solve than D0). From Proposton 1, the resultng approxmate value functon provdes an upper bound on the optmal value functon for every state. The approxmate lnear program n fact fnds the lowest upper bound on v 1 c of the form 7). Futhermore, we obtan a polcy whose performance can be compared wth the upper bound usng smulaton. It sets 1 f r A f u t r = V t+1 k k r k r A k 0 otherwse t r R t 8) Ths s computatonally easy to mplement provded the gven set of bass functons s not too large and the bass functons are easy to evaluate. Note that ths polcy does not ft the defnton of bdprce controls gven n Tallur and van Ryzn 1998), yet t s stll based on dual) prces. Indeed, substtutng 7) nto D0) yelds an optmzaton problem over the parameters V t k, D ) mn V 1 k k c 9) V k V t k k r V t+1 k E k R t+1 r u k Note that E k R t+1 r u = Its dual s P ) Z = max X 0 p t f u t r R t u r 10) p t k r A u + t 1 p t ) k r p t f u )X t r u 11) k r X t r u k c f t =1 = E k R t r u X t 1 r u r R t 1 u r t =2 k t 12) The constrants 12) are bass-weghted flow-balance constrants, whereby flow balance s mantaned around each

5 Operatons Research 554), pp , 2007 INFORMS 651 bass functon k separately. The V t k are dual prces on these constrants, and may be vewed as generalzed bd prces. The problem P ) has relatvely few constrants f not many bass functons are used but many varables. Thus, t may be well-suted for soluton va column generaton, provded the subproblems can be solved effcently. Denote the reduced proft of X t r u by t r u = k p t f u V t k k r V t+1 k E k R t+1 r u Gven an ntal set of prces V t k, to generate a new column for P ), or prove that none exst wth postve reduced proft, we solve max t t r u Later we consder a specal case n whch ths subproblem becomes a lnear nteger program for each t. Despte the enormous number of varables n P ), obtanng an ntal feasble soluton to begn a column generaton procedure s trval. There exsts one supported by only decson varables, correspondng wth the offer nothng polcy. Proposton 2. A feasble soluton to P ) s 1 f r = c u = 0 X t r u = t r R t u r 0 otherwse Proof. For all t, we have c R t and u = 0 c. For all t and k, the left-hand sde of 12) s k r X t r u = k c X t c 0 = k c Lkewse, for all t>1, the rght-hand sde s r R t 1 u r E k R t r u X t 1 r u = k c X t 1 c 0 = k c The next result gves an upper bound on the optmalty gap between an optmal soluton and a gven feasble soluton. Ths knd of result s relatvely standard, but we specalze t to our functonal approxmaton settng. It s useful because t provdes a stoppng crteron for a column generaton procedure. Ths result n fact holds for any basc feasble soluton X, but we present t n the form n whch we wll use t,.e., n terms of columns. Defne t = max t r u t Proposton 3. Suppose that k k s chosen so that 0 r = 1 r. Consder the restrcted verson of P ) contanng only decson varables X t r u, whose ndces are n a subset of all possble ndces. Let X V denote the correspondng optmal prmal-dual par for ths restrcted problem, and let t be t computed wth respect to V. Let Z denote ts optmal obectve value. Then, Z Z + t t=1 Proof. For k = 0, 12) becomes 1 f t = 1 X t r u = t = 2 whch mples r R t 1 u r X t 1 r u X t r u = 1 t 13) Consder any X feasble to P ), and any numbers V t k t k. Multply both sdes of 12) by V t k for each t, k, and add the resultng equatons together wth Z X = p t f u )X t r u We obtan Z X k t r u V 1 k k c = t r u X t r u t X t r u = t t r u t r u t r u X t r u ) = t t where the last equalty follows from 13), and t r u s the reduced proft of X t r u. Because ths relaton s true for all feasble solutons X, t s true n partcular for an optmal soluton X to P ), havng obectve value Z X = Z. Furthermore, from strong dualty appled to the restrcted problem, we have V 1 k k c = Z X = Z k Hence, we obtan Z Z + t t=1 to be the maxmum reduced proft for perod t under V. whch s the desred result.

6 652 Operatons Research 554), pp , 2007 INFORMS 4. Affne Functonal Approxmaton In ths secton, we study a specfc form of functonal approxmaton 7) wth affne bass functons. We gve the resultng prmal-dual formulatons, and then establsh ther relatonshp wth the standard LP for bd-prce control. We then gve some results on structural propertes satsfed by optmal solutons. Whle they are nterestng n ther own rght, we wll see later that these structural propertes also have a dramatc mpact on computatonal tmes for solvng larger-scale problem nstances. Lastly, we dscuss a column generaton algorthm Formulaton Consder the affne functonal approxmaton v t r t + V t r t r 14) where V t estmates the margnal value of a unt of resource n perod t, and t s a constant offset. We mplctly assume +1 = 0 and V +1 = 0. Ths approxmaton has the form 7), wth componentwse bass functons r = r and the constant bass functon 0 r = 1. Under 14), the control polcy 2) becomes 1 f r A f a V t+1 u t r = 0 otherwse t r R t 15) whch has the well-known bd-prce control structure Tallur and van Ryzn 1998). The lnear program D ), specalzed to ths form, becomes D1 mn V 1 + t t+1 + V 1 c 16) V t r V t+1 E R t+1 r u p t f u t r R t u r 17) By expandng the expectaton, we can rewrte 17) as t t+1 + p t f u The dual of D1) s P1 z P1 = max X V t r V t+1 r t r R t u r t p t a u )) p t f u )X t r u 18) r X t r u = c r R t 1 u r X t r u = X 0 f t = 1 r ) p t 1 a u t 19) X t 1 r u t = 2 1 f t =1 20) t =2 r R t 1 u r X t 1 r u Constrants 19) say that the expected mass of resources havng type flowng nto tme t must equal that flowng out. The frst part of 19) for t = 1 can be elmnated by notng R 1 = c and substtutng 20) for t = 1 nto the left-hand sde. Furthermore, 20) can be replaced smply by X t r u = 1 t 21) Ths allows us to nterpret the decson varables X t r u as state-acton probabltes. Note, however, that the X soluton does not, n general, yeld an mplementable randomzed control polcy. The support of X does not specfy an acton for every attanable state.) The obectve functon 18) maxmzes the approxmate total expected revenue over the fnte tme horzon Relatonshp wth the Standard LP for Bd-Prce Control To derve LP) from P1), defne Y t p t u X t r u 22) The obectve functon 4) follows mmedately from 18). Now fx, and sum 19) over tme t to obtan t r X t r u = c + t=2 r R t 1 u r r Rearrangng and cancelng terms yelds p t 1 a u )X t 1 r u c = a p t u X t r u t=1 1 + r X r u 23) r R u r

7 Operatons Research 554), pp , 2007 INFORMS 653 Next, we bound the second term on the rght-hand sde. We frst show that for all, t, r R t, u r, r a p t u By defnton of r, we have r a u. Multply both sdes by p t for any to obtan r p t a p t u Now sum over to obtan r r p t a p t u Hence, 23) mples c a p t u X t r u t = a Y whch yelds 5). To derve the nequaltes on the rght-hand sde of 6), start wth 22) to obtan Y = p t u X t r u t p t X t r u = p t 24) t t from 21). Together wth Proposton 1, ths proves the followng theorem. Theorem 1. Any feasble soluton to P1) yelds a feasble soluton to LP) havng the same obectve value. Hence, z LP z P1 v 1 c. It s well known that z LP s asymptotcally optmal,.e., converges to v 1 c, as the demand, capacty, and tme horzon scale lnearly e.g., see Cooper 2002). It follows, then, that z P1 s also asymptotcally optmal. Also, although the bound obtaned from P1) s stronger than that obtaned from LP), ths does not theoretcally guarantee that the assocated polcy 15) s better than standard bd-prce control based on LP). The followng example shows that we can have z LP >z P1. Example 1. Suppose that there s only one resource and one class,.e., m = n = 1. Hence, we drop the ndces and. Set = 2, c = 1, f = 1, a = 1, and p t = 0 5 for t = 1 2. Wthout loss of optmalty, we can elmnate X varables that set u = 0 when r = 1. Then, the lnear program P1) reduces to max 0 5 X X X = 1 X = X X X = X X 0 whch trvally has optmal obectve value equal to In contrast, LP) becomes max Y 0 Y 1 wth optmal obectve value equal to one. The dfference arses because, whereas LP) aggregates over all sample paths accordng to 22), P1) at least partally accounts for sample-path dynamcs. As an asde, note that n ths case the lnear program P1) s exact, meanng that t solves the orgnal optmalty Equatons 1). Comparng the obectve functon 16) wth that of the dual of LP) suggests that optmal dual prces from LP) are estmates of the optmal functonal approxmaton 14) computed by D1),.e., V V 1 ) p t 1 t Ths formalzes the heurstc nterpretaton of Tallur and van Ryzn 1998) that V 1 r V 1 r A V 1 a Note, however, that 15) ndcates the prces from tme perod 2, rather than 1, are relevant for the decson n tme perod 1. In the appendx, we show that the above analyss can be carred out wth a lower-fdelty, quas-statc value functon approxmaton. The resource prces do not depend on tme, as they do here, and so the upper bound s weaker Structural Propertes Gven a feasble soluton X to P1), defne the frst tme resource s used by t =argmn t 1 r R t u r wth X t r u >0 u =1 a >0 25) It s not necessarly the case that t = 1 n an optmal soluton X. For example, resource s consumpton may smultaneously requre, under the matrx A, the unproftable consumpton of other resources. Lemma 1. Fx. Assume that t exsts whch mples c > 0) and p t > 0 t. Then, 1) t > t, X t r u > 0 such that r <c, and 2) t t, X t r u > 0 only f r = c. Proof. We start by showng the frst part. Assume that t <. Let r and u be such that X t r u > 0 and there

8 654 Operatons Research 554), pp , 2007 INFORMS exsts an assocated as n the defnton of t. Observe that by defnton of t, we have the strct nequalty r p t a u <r c For perod t + 1, 19) therefore reads r X t +1 r u = r u r u r r + p t a u )X t r u p t a u r r u u c X t r u <c ) X t r u because of 20) and X t r u > 0. If r = c for all r, u such that X t +1 r u > 0, then by 20), the left-hand sde equals c, a contradcton. The desred result for all t > t follows by applyng the same argument recursvely. The second clam s true for t = 1 by defnton of R 1. Now consder any 1 <t t, and suppose that the statement s true for t 1. Then, 19) reads r X t r u = r X t 1 r u = c X t 1 r u = c r u r u r u because of 20). Suppose that r <c for some r, u such that X t r u > 0. Then, the left-hand sde becomes r X t r u = r u r r u u r X t r u + r X t r u <c r r u u X t r u + c X t r u = c whch s a contradcton. The desred result follows by nducton. Theorem 2. Assume that c > 0 and p t > 0 t. There exsts an optmal dual soluton V of D1) and a set of ndces t such that t t+1 t 26) V t = V t+1 t = 1 t 1 27) V t V t+1 t = t 28) V 0 29) Proof. Suppose that t, as defned n 25), exsts. Consder t = t. We later consder the case where t does not exst. We frst focus attenton on 27) and 28), whch we prove n three steps: 1) Vt V t+1 t > t. 2) Vt V t+1 t < t. 3) If Vt <V t+1 for some t t, then we can rase Vt to Vt+1 wthout loss of optmalty. Fnally, we prove the remanng clams. Step 1. Assume that t <. Consder an optmal prmaldual soluton X, V. From Lemma 1, there exsts an X t r u > 0 for whch r <c. Therefore, complementary slackness mples that 0 = t+1 t + p t f a V t+1 u V t V t+1 r 30) Consder a new r, equal to r except that r = r + 1 c. Note that u s stll feasble,.e., u r. By dual feasblty 17), 0 t+1 t + p t f a V t+1 u V t V t+1 r V t V t+1 r + 1 whch yelds the desred result,.e., 28) for t > t, when combned wth 30). Step 2. We now show that Vt V t+1 for all t < t. Equaton 20) mples that there exsts an X > 0 for t r u some r and u, and from Lemma 1 t has the property that r = c. By complementary slackness, we then have 0 = t+1 t + p t f a V t+1 u V t V t+1 r V t V t+1 c 31) Now consder a new r equal to r except r = c 1 0. Because, by defnton of t, u = 0 for all such that a > 0, we know that u s stll feasble,.e., u r. By dual feasblty 19), we have 0 t+1 t + p t f a V t+1 u V t V t+1 r V t V t+1 c 1 Combnng the last two dsplays yelds Vt V t+1. Step 3. Gven an optmal prmal-dual soluton X, V, consder the largest t t such that Vt <V t+1. Note that f V t Vt +1, then 28) s satsfed for t = t. We wll alter the optmal dual soluton to produce another soluton that acheves 27) or 28) f t = t ), yet s stll optmal because t preserves dual feasblty and complementary slackness. Denote the modfed soluton by V, and defne t to be the same as V except V t = V t+1 t = t + V t V t+1 c

9 Operatons Research 554), pp , 2007 INFORMS 655 We frst check the perod t nequaltes 17) for dual feasblty. Indeed, for any r and u, the reduced proft s t+1 t + p t f a V t+1 u Fgure Example dynamc bd-prce traectores. V t V t+1 r V t V t+1 r = t+1 t V t V t+1 c + p t f a V t+1 u V t V t+1 r 0 32) The nequalty follows because the second expresson s the reduced proft, under the orgnal dual feasble soluton V, for a potentally) modfed r havng r = c and u unchanged yet stll feasble). In partcular, consder any r, u, for whch X > 0. Because t t r u t, from the argument above r = c and 31) holds under V. Note that the expresson n 32) s the same as 31),.e., the reduced proft s the same under V as under V, and so complementary slackness s preserved. Next, we check dual feasblty and complementary slackness for the perod t 1 nequaltes 17), assumng that t>1. For any r, u, 17) reads t t 1 + p t 1 f a V t u V t 1 V t r V t 1 V t r 0 Under V, the reduced proft becomes t + V t V t+1 c t 1 + p t 1 f a V t a V t+1 u V t 1 V t r V t 1 V t+1 r Subtractng the frst reduced proft from the second yelds V t V t+1 c r + p t 1 a u 0 hence, dual feasblty s preserved. Furthermore, for any X > 0, we know that r t 1 r u = c and u = 0 for all such that a > 0. Therefore, the last nequalty changes to equalty,.e., complementary slackness s preserved. The modfed varables V t and t appear nowhere else besdes where consdered above, and so overall dual feasblty and complementary slackness s preserved. Now apply the same argument successvely for each smaller t. Because Vt V t+1 from Step 2 above), the procedure acheves 27). Bd-prce Resource 1 Resource 2 Resource 3 Resource Perod Fnal detals. If no t exsts, then Step 2 of the argument above shows that Vt V t+1 for all 1 t. The varable modfcaton procedure n Step 3 can stll be appled to acheve Vt+1 = V t for all t. Because V +1 = 0, t follows that Vt = 0 for all t, hence, 27) holds for t =. The nonnegatvty of Vt n 29) follows from 28) and V +1 = 0. The tme monotoncty of t, gven n 26), follows drectly from dual feasblty 17), because r = 0 and u = 0 s a feasble state-acton par. The nonnegatvty of n 29) follows from 26) and the fact that +1 = 0. The followng corollary follows drectly from the tme monotoncty of Vt and V +1 = 0. Corollary 1. There exsts an optmal dual soluton V of D1) and a set of tme thresholds such that for each class, f < f a V t+1 a V t+1 t = 1 1 t = The correspondng control polcy 15) s therefore 0 f t < u t r = t r R t 1 f t and r A Fgure 2. Algorthm Column generaton algorthm for solvng P1) to wthn an optmalty tolerance of. Column generaton Set = t c 0 t, solve the restrcted problem P1 )), and set t = for all t. whle t t > Z do for all t 1 compute t = max r u t r u select an r t u t arg max r u t r u update t r t u t solve P1 ))

10 656 Operatons Research 554), pp , 2007 INFORMS Table 1. Dmensons of test nstances. No. of nonhub locatons L) No. of legs m) No. of classes n) One mght then nterpret P1) as searchng over polces havng the above form, but evaluatng them only approxmately. In the sngle resource-type problem m = 1, t s well known that a tme threshold polcy s optmal. However, the tme thresholds depend on the current state, whereas here they only depend on the ntal state c. Bertsmas and Popescu 2003) argue that the tme threshold property does not hold n the network case. Ths renforces how P1) s stll an approxmaton. Note that LP) exhbts no noton of tme thresholds, as t s too coarse. Fgure 1 shows dynamc bd-prce traectores for each of four resources n a problem nstance wth 20 perods. These were obtaned by solvng D1) to optmalty. The traectores are monotone, wthout loss of optmalty as stated by Theorem 2. Intutvely, the margnal value of a resource decreases over tme because the opportuntes to use t are fewer. In fact, after the horzon, V +1 = 0 for all. Note that the curves exhbt nonlnearty toward the end of the horzon. In the begnnng of the horzon, the curves tend to be flat untl an nflecton pont s reached. Theorem 2 shows that sometme at or before that pont s reached, the resource s frst used.e., at tme t ) Column Generaton In passng from D0) to D1), we have reduced the number of decson varables from m R t = c + 1 t =1 whch s exponental n m, down to m + 1, a polynomal n m. However, there s stll an exponental number of constrants. Hence, the prmal program P1) s potentally suted for soluton va column generaton. To solve P1) usng column generaton, frst prce out an ntal prmal feasble soluton. Ths s suppled by Proposton 2. Denotng the resultng prces by V,, now solve max t r u = max p t f t t a V t+1 u V t V t+1 r t + t+1 whch maxmzes the reduced proft from 17). If the obectve value s greater than zero, then we add the column to the exstng set of columns for P1); otherwse, we have attaned optmalty. For fxed t>1, ths s equvalent to solvng the lnear nteger program max u r p t f a V t+1 u V t V t+1 r t + t+1 33) a u r 34) u ) r 0 c 36) Note that f a are ntegral, then wthout loss of optmalty, we can relax the ntegralty of r. Ths s because fxng u, n an extreme pont soluton ether r = max a u or r = c. Furthermore, fxng r, a u soluton s gven by 15). We beleve that because of ths structure, we fnd emprcally that CPLEX can effcently solve these subproblems to optmalty. We present the full algorthm n Fgure 2, where P1 )) denotes the restrcted verson of P1) wth columns comng only from, and Z denotes the correspondng optmal obectve value. Ths s a standard algorthm, except that we enter a batch of columns at once, potentally one column for each tme perod. We found ths to be much more effectve than addng one column at a tme. As a consequence of Proposton 3, to ensure that the obectve value of the current soluton X based on columns s wthn percent of an optmal soluton,.e., Table 2. Capacty and load statstcs for large test nstances. No. of nonhub locatons L) Capacty Load Capacty Load Capacty Load Capacty Load per leg factor per leg factor per leg factor per leg factor ,

11 Operatons Research 554), pp , 2007 INFORMS 657 Table 3. CPU seconds to solve P1) wth and wthout monotoncty constrants. No. of nonhub locatons L) Mono w/o Mono Mono w/o Mono Mono w/o Mono Mono w/o Mono , >84,000 Z /Z 1 +, t suffces to ensure that t t Z Ths s merely suffcent; for the soluton X, the rato Z /Z n fact may be closer to one than 1+. We employ ths as a stoppng crteron for the algorthm. We also consdered more complcated versons of ths basc algorthm. In one verson, whenever a t r u was found for whch X t r u has postve reduced proft, we added ths ndex to for all t for whch X t r u has postve reduced proft. Ths avoded havng to solve as many nteger programmng subproblems, but ths beneft was outweghed by havng to solve more lnear programs contanng more columns. It was better to add columns wth maxmum reduced proft. 5. Numercal Results We conducted numercal experments to study three questons: 1) How long does t take to solve nstances of practcal sze? 2) How much stronger s the upper bound gven by P1) as compared wth LP)? 3) How well does the approxmate polcy perform n smulatons, as compared wth the upper bound gven by P1) and standard bd-prce control? Table 4. CPU seconds to solve P1) wth monotoncty constrants for the same nstances except for half the load factor. No. of nonhub locatons L) , In our frst set of experments, we randomly generated 24 problem nstances of varyng complexty, havng tme horzons n the set We consdered an arlne servcng L locatons out of a sngle hub, where L Ths s an mportant, basc network structure of revenue management problems found n the arlne ndustry. Each locaton l 1 L s assocated wth two flght legs,.e., to and from the hub, each departng at the end of the tme horzon. Hence, there are m = 2L resources. There are m sngle-leg tnerares and L L 1 two-leg tnerares. For each tnerary, we desgnate a hgh-fare class and a low-fare class, so that the total number of classes s n = 2 m + L L 1. Table 1 gves the resultng dmensons of our test nstances. The dmensons of the largest nstances are of real-world sze. We generated the revenue for each low-fare class from a dscrete unform dstrbuton on the nterval 15 49, and set the hgh fare for the same tnerary equal to fve tmes that of the low fare. For smplcty, we consdered statonary demand wth the probablty 0.2 for havng no customer arrval n a perod, and the other probabltes generated randomly. For each tnerary, we splt the demand so that 25% was for the hgh-fare class, and 75% was for the lowfare class. For each nstance, we set the ntal capacty seats), c, to be the same for each leg. Table 2 dsplays the capacty per leg and load factor for each nstance. The load factor s defned as the expected demand for seat legs dvded by the supply,.e., load factor = / p t a cm Table 5. t Approxmate relatve dfference n upper bounds, LP / P1. No. of nonhub locatons L) ,

12 658 Operatons Research 554), pp , 2007 INFORMS Table 6. Bound and polcy results for statonary demand. Capacty Load P1) LP) BPC fxed DBPC fxed BPC DBPC L per leg factor Bound Bound Mean Std. err.) Mean Std. err.) Mean Std. err.) Mean Std. err.) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 3, ) 3, ) 3, ) , ) 6, ) 7, ) 7, ) , ) 18, ) 18, ) 19, ) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 3, ) 3, ) 3, ) , ) 7, ) 7, ) 8, ) , ) 20, ) 19, ) 21, ) In Table 3, we report the CPU seconds requred to solve P1) guaranteed to wthn = 5%, whch s a practcal tolerance gven the data uncertanty encountered n the real world. We ran these nstances on an Intel Xeon CPU runnng at 3.6 GHz wth 3 GB of memory, usng CPLEX 9.1. Addng monotoncty constrants to D1) had a dramatc mpact on the soluton tmes, and n fact makes soluton practcal on large-scale nstances. The mprovement was on average by a factor of at least) tmes, wth a range between 2.06 and tmes. For the largest nstance, wth L = 20 and = 1 000, the model solved n 3.25 hours wth monotoncty constrants. Wthout them the model dd not solve after one day, and so we termnated the algorthm. Table 4 shows CPU tmes, wth monotoncty constrants, when the load factor s cut n half by doublng the capacty c per leg. On average, the speedup was by a factor of 11.14, wth a range between 1.38 and There was a marked ncrease n speedup as the sze of the network ncreased. Ths speedup can be explaned by the fact that as the load factor decreases, the polcy that always accepts tends to be closer to beng optmal. The algorthm quckly generates the correspondng columns, and there s effectvely a complexty reducton because other combnatons of accept/reect decsons are less mportant. In Table 5, we report the approxmate) relatve dfference n upper bounds between the standard lnear program LP) and our P1),.e., z LP /z P1. The numbers reported n the table correspond wth P1) solved to an optmalty tolerance of = 5%, whch does not strctly gve an upper bound. However, when we solved P1) to wthn = 0 5%, the mprovement n the obectve value of P1) was on average only about 1%, and ths was acheved wth an ncrease n CPU tme by a factor of about fve, on average. Therefore, the dfferences reported n the table are representatve; they are also consstent wth smlar results that follow. The gap between the bounds can be qute substantal, up to 48.5% on these nstances. Two trends are notceable. As the number of locatons ncreases.e., the network becomes more complex the gap n the bounds ncreases. Also, as the tme horzon ncreases, the gap n the bounds decreases. On these nstances, the capacty and demand scale up lnearly as the tme horzon ncreases. As a consequence of the asymptotc result n Cooper 2002), LP) becomes asymptotcally optmal, and hence so does P1) due to Theorem 1. We next consdered polcy performance. It has been shown by Wllamson 1992) that statc bd-prce control, usng LP), performs qute well when the bd prces are reoptmzed frequently. Ths motvates the followng polces, where the acronym DBPC stands for dynamc bdprce control. DBPC fxed: Solve P1) D1) once. Gven a set of fxed dynamc bd prces, use the polcy gven by 15). DBPC: Solve P1) D1) multple tmes spread evenly over the tme horzon. Between soluton epochs, use the polcy gven by 15). Table 7. Bound and polcy results for h-lo demand. Capacty Load P1) LP) BPC fxed DBPC fxed BPC DBPC L per leg factor Bound Bound Mean Std. err.) Mean Std. err.) Mean Std. err.) Mean Std. err.) ) ) ) ) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 2, ) 2, ) 2, ) , ) 5, ) 5, ) 5, ) ) ) ) ) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 2, ) 2, ) 2, ) , ) 5, ) 5, ) 6, )

13 Operatons Research 554), pp , 2007 INFORMS 659 Table 8. Bound and polcy results for lo-h demand. Capacty Load P1) LP) BPC fxed DBPC fxed BPC DBPC L per leg factor Bound Bound Mean Std. err.) Mean Std. err.) Mean Std. err.) Mean Std. err.) ) ) ) ) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 2, ) 2, ) 2, ) , ) 5, ) 5, ) 6, ) ) ) ) ) ) ) ) ) , ) 1, ) 1, ) 1, ) , ) 2, ) 2, ) 2, ) , ) 6, ) 5, ) 6, ) BPC fxed: Solve LP) once. Gven a fxed set of statc bd prces, use the polcy gven n 2.2. BPC: Solve LP) multple tmes spread evenly over the tme horzon. Between soluton epochs, use the polcy gven n 2.2. Unless specfed otherwse, we re-solve P1) D1) and LP) fve tmes over the tme horzon. We notced a small mprovement n the DBPC polces when we used a smple sngle-perod lookahead nstead of 15). Let ˆv denote the affne approxmaton 14), and û denote polcy 15) under ˆv. Instead of usng approxmaton 14) n polcy 2), we nstead approxmate v t+1 r and v t+1 r A by lookng ahead to perod t + 1, applyng polcy û n that perod but evaluatng the subsequent state n perod t + 2 usng the affne approxmaton ˆv. Thus, we use the approxmaton ṽ t+1 r = p t+1 f +ˆv t+2 r A û t+1 r + 1 ) p t+1 ˆv t+2 r t 2 r Fgure 3. Relatve polcy dfference DBPC/BPC) Relatonshp between relatve bound mprovement and polcy mprovement. Polcy mprovement as a functon of bound mprovement H-lo demand Lo-h demand Statonary demand Relatve bound dfference LP)/P1) We also notced a small mprovement n the DBPC polces when we requred the revenue f to be strctly greater than the opportunty cost as measured by ṽ t+1 r ṽ t+1 r A. In ths set of experments, we randomly generated three sets of nstances havng the same characterstcs as the orgnal set, except L 3 5 and The frst set has statonary demand, whle the second and thrd sets have nonstatonary h-lo demand and lo-h demand, respectvely. Followng Bertsmas and Popescu 2003), under h-lo demand the arrval probabltes ncrease over tme t for hgh-fare classes accordng to p t = p / log a a + t, where p s the base statonary arrval probablty and a and are parameters. The arrval probabltes decrease over tme t for low-fare classes accordng to p t = p / log a a + t 1. Lo-h demand s the reverse, wth the arrval probabltes for hgh-fare classes decreasng over tme, and the arrval probabltes for low-fare classes ncreasng over tme. For our experments, we used a = 5 and = 1. We smulated each nstance 100 tmes for each polcy, usng the same sequence of customer demands across dfferent polces. We also solved P1) wth an optmalty tolerance of = 0 5%. The results are shown n Tables 6, 7, and 8 wth standard errors reported n parentheses. Fgure 3 summarzes these results by plottng the relatonshp between the dfference n the bounds and the dfference n polcy performance. The mprovement of the DBPC polcy over the BPC polcy ncreases as the qualty of the bound P1) relatve to LP) mproves. Furthermore, the dfferences are substantal: the LP) bound can be up to 50.4% worse than the Table 9. Relatve polcy mprovement from re-solvng every perod. L BPC DBPC

14 660 Operatons Research 554), pp , 2007 INFORMS Table 10. Bound and polcy results for h-lo demand wth decreasng load factor across nstances. Capacty Load P1) LP) BPC DBPC L per leg factor Bound Bound Mean Std. err.) Mean Std. err.) ) ) , ) 1, ) , ) 2, ) , ) 3, ) , ) 7, ) ) ) , ) 1, ) , ) 2, ) , ) 3, ) , ) 8, ) P1) bound, and the polcy DBPC can perform up to 21.4% better than BPC. There appears to be no obvous dependency on the demand characterstcs.e., statonary, h-lo, or lo-h although the polcy mprovements appear slghtly weaker for lo-h demand. Table 9 consders the effect of more frequent re-solvng when demand s statonary. It reports the mean polcy performance from re-solvng every perod, dvded by the mean polcy performance from Table 6 for re-solvng fve tmes over the tme horzon. Re-solvng leads to more mprovement for BPC than for DBPC. On the other hand, as shown n Tables 6, 7, and 8, re-solvng for BPC may actually degrade the polcy a well-known effect), whereas re-solvng consstently mproves DBPC. We suspect ths s because the model P1) s more accurate than LP), and therefore ts recommendatons are more on target wth outcomes. We also tested how the polces are affected by the load factor. We consdered the same nstances as for the statonary demand case n Table 6, except the statonary arrval probabltes are used as base probabltes n the models gven above for h-lo and lo-h demand. As shown n Tables 10 and 11, ths gves a load factor that decreases as ncreases. For h-lo demand the lo-h demand case s smlar), n Fgure 4 we plot how the polcy and bound dfferences change as a functon of load factor. As the load factor decreases, the advantage of P1) over LP) decreases because the problem smply becomes easer: Acceptng every customer becomes optmal. On the other hand, as demand becomes greater than supply,.e., the load factor ncreases, the problem becomes harder because now some demand must be turned away. Hence, usng P1) and the correspondng polcy DBPC yelds greater mprovements precsely when they are needed the most. As shown earler, ths ncreased performance comes wth a computatonal prce. In our experments, we also notced that ncreasng the rato of hgh fare to low fare mproves the performance of DBPC. Although unrealstc, n one nstance wth a hghto-low fare rato of 20 to 1, we found the smulated mean revenue gap between BPC and DBPC to be 43%. Appendx We can, n fact, derve the standard lnear program LP) usng the followng quasstatc affne functonal approxmaton to the value functon v t r t + V r t r R t A1) Ths s a smpler form than 14), whch has parameters V t dependng on the tme ndex t, whereas here the parameters V do not depend on t. We can therefore nterpret the optmal dual prces V of LP) as approxmatng the Table 11. Bound and polcy results for lo-h demand wth decreasng load factor across nstances. Capacty Load P1) LP) BPC DBPC L per leg factor Bound Bound Mean Std. err.) Mean Std. err.) ) ) , ) 1, ) , ) 2, ) , ) 3, ) , ) 7, ) ) ) , ) 1, ) , ) 2, ) , ) 3, ) , ) 8, )