The One-Dimensional Dynamic Dispatch Waves Problem

Transcription

1 The One-Dmensonal Dynamc Dspach Waves Problem Mahas Klapp Alan L. Erera Alejandro Torello H. Mlon Sewar School of Indusral and Sysems Engeerng Georga Insue of Technology Alana, Georga maklapp a gaech do edu, {aerera, aorello} a sye do gaech do edu June 30, 15 Absrac We sudy same-day delvery SDD sysems by formulang he Dynamc Dspach Waves Problem DDWP, whch models a depo where delvery requess arrve dynamcally hroughou a servce day. A any dspach epoch wave, he nformaon avalable o he decson maker s 1 a se of known, open requess whch reman unfulflled, and 2 a se of poenal requess ha may arrve laer n he servce day. A each wave, he decson maker decdes wheher or no o dspach a vehcle, and f so, whch subse of open requess o serve, wh he objecve of mnmzng expeced vehcle operang coss and penales for unserved requess. We consder he DDWP wh a sngle delvery vehcle and reques desnaons on a lne, where vehcle operang mes and coss depend only on he dsance beween pons. We propose an effcen dynamc programmng approach for he deermnsc varan, and leverage o desgn an opmal a pror polcy wh predeermned roues for he sochasc case. We hen show ha fully dynamc polces may perform arbrarly beer han a pror ones, and propose heurscs and dual bounds for hs case. Keywords: same-day delvery, dynamc dspach, approxmae dynamc programmng 1 Inroducon E-commerce and he home delvery channel connue o grow whn he consumer real ndusry secor. Accordng o Forreser [50], he onlne secor accouned for 9% of he $3.2 rllon U.S. real ndusry sales n 13 and s forecas o grow % per year hrough 18. Sales and densy growh can help reduce lasmle logscs coss. However, he onlne real segmen s exremely compeve and operaes wh very low margns, drvng a need for connued logscs opmzaon. Consder he case of Amazon. Is 13 annual repor shows an operaonal margn of jus 1% on annual revenue of $74.5 bllon; among expendures, cos of sales accouns for he larges fracon a 73.5%, bu fulfllmen coss 11.6% are also large [31]. 1

2 Table 1: Examples of same-day delvery plo programs n he US Servce Wha Charge/order US ces mplemened Amazon SDD ems from warehouses $8.99, $5.99 members $99/year ATL,BAL,BOS,CHI,DA-FW, IND,LA,NY,PHI,PHO,SJ-SFO, SEA,DC Google Express ems from assocaes free, bu $/monh $15 mn order SJ/SFO,NY,LA Insacar personal shoppers $3.99 or $99/year $35 mn order ATL,AUS,BOS,CHI,DC,DEN, HOU,LA,NY,PHI,SJ-SFO,SEA Walmar o Go ems from sores $ BAL,DEN,MIN,PHI,SJ-SFO One enhancemen o cusomer servce n hs secor s same-day delvery SDD. Several e-realers and logscs servce provders have nroduced programs n major US ces; see Table 1. We defne SDD as a dsrbuon servce where consumers place orders on he same day ha hey should be delvered. For companes ha mplemen SDD, s mperave o boh offer hgh levels of cusomer servce and keep logscs coss as low as possble. Provdng SDD servces requres wo core logscs processes: 1 order managemen a he sockng locaon, ncludng recevng, pckng, and packng orders; and 2 order dsrbuon from he sockng locaon o delvery locaons. To dae, wo classes of servce provders have deployed SDD servces: realers offerng ems prmarly from owned socks n dsrbuon ceners or real sores such as Amazon and Walmar, and logscs provders servng as nermedares ha pck up packages from sockng locaons and delver hem o cusomers, such as Google and Insacar as well as USPS, FedEx and UPS. Realers mus manage boh core logscs processes, whle logscs provders are ypcally concerned wh he second one. Ths research effor s a sudy of prmary radeoffs n SDD dsrbuon. We formulae he Dynamc Dspach Waves Problem DDWP as a Markov Decson Process MDP. The DDWP models he dynamcs of a sngle dspach facly depo where cusomer order requess arrve dynamcally hroughou an operang day. A any decson epoch, whch we call a wave, he logscs operaor manans a se of known open requess wh known delvery desnaons and a se of poenal requess ha may arrve before he end of he day. A each wave, he operaor decdes wheher or no o dspach vehcles loaded wh known orders, and he vehcle roues for dspached orders. The objecve s o mnmze expeced operaonal coss and expeced penales for unserved open requess a he end of he day. Such penales could represen he cos of drec dspach or revenue los due o unserved cusomers. Dynamc opmzaon problems lke he DDWP are hose where only a subse of all relevan nformaon 2

3 s known pror o he nal decson epoch, and he res s revealed over me durng he operang day. An opmal soluon o such problems s a dynamc polcy ha deermnes bes decsons gven he nformaon sae avalable a each decson epoch. In conras are smpler a pror polces ha specfy ceran decsons n advance, and may allow smple changes va recourse rules. A DDWP nsance s characerzed by s degree of dynamsm, a rao of he amoun of nformaon revealed dynamcally onlne o ha avalable offlne; see [44]. When hs rao s large, dynamc polces may subsanally ouperform a pror polces. We sudy he neracon beween wo mporan decsons n SDD dsrbuon sysems: dynamc dspach and vehcle roung. Dspach decsons refer o selecon of he mes a whch vehcles are dspached and he orders ha hey delver, whle vehcle roung decsons refer o he sequences of delveres for each dspached vehcle. Two fundamenal radeoffs exs. Frs, here s a radeoff beween wang and dspachng a vehcle o serve requess. When a vehcle s dspached, he queue of open requess s reduced bu an opporuny o serve fuure requess durng he roue s mssed. On he conrary, when an avalable vehcle s no dspached, we reduce he me remanng n he operang day and poenally ncrease he lkelhood ha fuure requess canno be served. Second, here s a radeoff beween dspachng longer, me consumng vehcle roues versus shorer ones. On one hand, a roue servng many requess uses more oal ravel me, and herefore keeps he vehcle away from he depo longer, bu requres less me per cusomer vsed. On he oher hand, a shorer roue uses more me per cusomer, bu reurns o he depo faser and enables he vehcle o be reused sooner for fuure orders. To smplfy he vehcle roung decsons, hs paper focuses on problem nsances where a sngle vehcle s avalable o make delveres o cusomer locaons on he posve real lne wh he depo as he orgn; ravel mes and vehcle operang coss are proporonal o dsances beween pons. We consder he followng o be our man conrbuons. 1. We formulae he DDWP o capure he basc aspecs of dynamc dspach, order selecon, and roung decsons for same-day delvery. 2. We develop an approach for deermnng opmal a pror soluons o he sochasc one-dmensonal varan by reducng hs problem o an equvalen deermnsc problem where all cusomer reques arrval mes are known n advance. 3. We show ha, alhough a pror polces work well n pracce, here exs problem nsances wh hgh degree of dynamsm for whch hese soluons are arbrarly worse han opmal dynamc polces. 3

4 Accordngly, we provde wo schemes o oban dynamc polces for he one-dmensonal problem. The frs s a rollou of he a pror polcy, and he second s an approxmae dynamc programmng approach ha uses an approxmae lnear program ALP o approxmae he cos-o-go funcon. We emprcally show he benefs of dynamc polces wh compuaonal expermens over wo ses of represenave nsances. The remander of he paper s organzed n he followng manner. Secon 2 conans a leraure revew, Secon 3 formulaes he model, and Secons 4 and 5 respecvely cover a pror and dynamc polces. Fnally, Secon 6 oulnes he resuls of a compuaonal sudy, and we conclude wh Secon 7. An Appendx conans all echncal proofs no ncluded n he body of hs documen. 2 Leraure Survey 2.1 Vehcle Roung and Dspach Problems The deermnsc Vehcle Roung Problem VRP and Travelng Salesman Problem TSP have been suded exensvely; see e.g., he exs [6, 30] for he TSP and [19, 28] for he VRP. Dynamc and sochasc VRPs are problem exensons where some parameers are unknown durng plannng and/or operaons. The smples sochasc VRP problems are a pror opmzaon models, where fxed operang recourse rules are used o modfy he soluon durng operaon; see [14, 19, 26] for recen surveys. Dynamc and onlne VRPs are problems where nformaon s revealed over me durng he operang perod, and roung and schedulng decsons are updaed n response; see [34, 38, 44, 46, 52]. Dfferen sochasc and dynamc VRPs focus on uncerany n dfferen ses of parameers. Some examples are he VRP wh sochasc demands [2, 11, 29, 43, 49], he VRP wh sochasc ravel mes [18, 35, 36, 39, 40, 41, 51, 53], and he VRP wh probablsc cusomers [8, 15, 24, 32, 33, 37, 54]. A relevan problem n he leraure ha ncorporaes he dspach dmenson s he Dynamc Mulperod Roung Problem DMPRP [4, 5, 55], whch consss of a dsrbuon cener dspachng orders wh a sngle vehcle and a plannng horzon dvded no me perods ypcally days. Cusomer orders arrve onlne a he begnnng of each perod wh a geographcal locaon, and each one has o be served whn wo perods. The decson maker mus choose whch orders o pospone and whch ones o serve mmedaely n he vehcle s roue defned by a TSP. A recen exenson called he Dynamc Mulperod Vehcle Roung Problem wh Probablsc Informaon DVRPP [3] covers he dynamc-sochasc case, where probabls- 4

5 c nformaon abou fuure requess arrval mes and servce me wndows s avalable a each decson epoch. In hs case, here s a flee of capacaed vehcles avalable a he depo, and each reques demands upon arrval a prevously known produc quany ha should be served whn s servce me wndow. The problem s solved heurscally usng a prze-collecng VRP model ha oupus whch orders o serve n each decson epoch. The przes of open requess are approxmaed as ncreasng funcons of he proxmy o he servce deadlne and decreasng funcons over he geographcal closeness o poenal fuure arrvals. Alhough relaed o he DDWP, hs model does no work for he same-day problem, snce assumes ha roung occurs beween consecuve perods of poenally unlmed me duraon. In he case of SDD, s fundamenal o consder roue duraon consrans and ncorporae how hese consrans affec he quany and duraon of fuure dspach perods, e.g., shorer roues allow more dspaches per day. Moreover, a model ha nends o serve all requess gnores he relave mporance beween dfferen requess. The closes dspach-relaed problem found n he leraure o he DDWP s perhaps he VRP wh release daes VRP-rd [7, 16] ha consders a deermnsc problem wh a depo dspachng orders wh prevously known release daes. A release dae specfes he earles me when he order can be pcked up a he depo before s ranspored o s cusomer s locaon. In [7] he auhors sudy smplfed versons of he problem, wh wo varans beng parcularly relevan o our work: one mnmzes he me requred o serve all orders by a sngle vehcle, and anoher mnmzes he oal ravel dsance subjec o a me budge. The auhors provde polynomal me algorhms for he cases n whch requess are placed over he lne and oher smplfed opologes. In [16], he auhors sudy an exenson of he VRP-rd n a general nework opology ha ncorporaes servce me wndows, capacy consrans and a homogeneous flee of capacaed vehcles. The auhors provde heursc soluons based on genec algorhms combned wh local search procedures. Our problem dffers from he VRP-rd by s dynamc-sochasc naure and s przecollecng feaures. Boh elemens are fundamenal for SDD, where he problem s conex s sochasc and where lmed me resources do no allow full coverage of requess mos of he me. We requre a model capable of selecng he leas expensve se of orders o be lef unaended or o be covered by more expensve ransporaon modes. In addon o same-day delvery wh smplfed roung coss, one-dmensonal models have applcaon n oher areas where a vehcle or machne s movemen s consraned along a sngle dmenson; see [22, 23, 56]. Also, our model s closely relaed o he Order Bachng Problem OBP [25, 42], whch deermnes opmal assgnmen of pck orders o baches and he pck our sequence for each bach n warehouse 5

6 operaons. The Dynamc Order Bachng Problem DOBP s an exenson wh orders arrvng dynamcally whle he decson maker processes prevous orders; see [13]. Recenly, [17] presen an analycal model o deermne he mng and he number of baches n an order fulfllmen sysem. To he bes of our knowledge, none of hese models allow a dealed selecon of he orders whn a bach, and solely conemplae hreshold rules such as consoldang he bach when a number o be deermned of orders have arrved. 2.2 MDP and Approxmae Dynamc Programmng Mos dynamc and sochasc VRPs can be modeled as MDPs; see [47]. Exac soluons o MDP models may no be possble due o he curse of dmensonaly, hrough s ofen racable o develop bounds on opmal coss; see [21] for a survey. An example s he a poseror bound [49] or Perfec Informaon Relaxaon PIR [12] ha dsregards he soluon s non-ancpave dynamcs and fnds he deermnsc opmal cos for each possble realzaon of he random parameers before compung he expeced cos. Anoher boundng echnque s he Approxmae Lnear Programmng ALP mehod [, 48] ha looks for subopmal soluons of he MDP s dual LP formulaon. The fundamenal dea s o elmnae he exponenal number of sae varables by enforcng a dependence on a prevously deermned low-dmensonal se of bass funcons. Moreover, s soluon can be used as a cos-o-go approxmaon n heursc polces. The ALP approach has been successfully appled n sochasc roung before, e.g. [1, 53]. In erms of approaches for he VRP and smlar dspach problems, he curse of dmensonaly necessaes approxmae dynamc programmng ADP soluon echnques, e.g., [27, 29, 43, 49]. One wdely used ADP mehod s o develop approxmaons for he opmal cos-o-go funcon and use o selec an approxmaely opmal acon a any encounered sae; see [9, 45]. Rollou algorhms [] have been wdely appled for sochasc roung models, e.g., [29, 49, 53]. 3 The One-Dmensonal Dynamc Dspach Waves Problem 3.1 Problem Defnon Consder a dynamc dspach and roung problem for a sngle vehcle operang over a fxed-duraon operang perod.e., a day. The vehcle s dspached from a depo, locaed a one end of a lne segmen, o serve a se of cusomer delvery requess. Afer compleng a roue, he vehcle reurns o he depo and may be dspached agan unl he end of he operang perod. A each decson epoch, he vehcle f avalable 6

7 may be dspached o serve any open cusomer requess, hose ha have arrved and are ready for dspach. In addon o nformaon abou open orders, probablsc nformaon descrbng unknown fuure order requess s also avalable. The objecve s o mnmze vehcle operang coss and penales for unserved requess. We consder a specfc class of problems of hs ype: 1. Le T := {1,...,T } be he se of waves decson epochs durng he operang perod, where waves are couned backwards so ha he waves number represens he waves-o-go before = 0, he deadlne for he vehcle o reurn o he depo. Le T 0 = T {0}. 2. Le N := {1,...,n} be he se of all poenal cusomer requess N, where each s characerzed by: 1 a known desnaon represened by a round-rp ravel me of d from he depo; 2 a penaly cos p > 0 ha mus be pad f he reques arrves bu s no served by = 0; and 3 a random arrval me τ drawn from a reques-dependen dsrbuon wh suppor T { 1}, where 1 ndcaes no arrval. Le N be ordered such ha d d j for < j. Noe ha our probablsc model enumeraes all possble reques arrvals. Anoher way of modelng hs problem s o defne a fxed se of locaons a whch orders appear wh poenally mulple arrvals per wave. Ths alernave probablsc model s mplcly capured n our seng by addng several requess wh an dencal cusomer locaon. A vehcle locaed a he depo a any wave T can be dspached o serve some subse S of he se of open revealed and unaended requess R { N : τ } a wave. Once a vehcle leaves he depo a wave, canno serve any reques arrvng a τ < unl reurns for reloadng; we assume ha once dspached, a vehcle mus fnsh s roue. Servng reques se S requres me, and we assume ha no addonal servce me s requred beyond vehcle ravel me. Gven reques locaons along he lne, he me requred by he vehcle o serve S s hen d S := max S d ; we assume vehcle operang cos for hs dspach s αd S. A vehcle dspached a reurns o he depo a d S. S s herefore consraned by d S, bu we assume no oher consrans on S, such as capacy, conssen wh movang SDD applcaons where me s he bndng resource. Toal sysem cos s measured by he sum of he vehcle operang coss over all dspaches plus he sum of he penales p for all R a he ermnal wave = 0. For purposes of analyss, we suppose n hs paper ha me beween consecuve waves s consan and equal o he me requred for he vehcle o complee a round rp wh dspach ravel me 1. Furhermore, suppose ha he d values are scaled such ha hey are all neger. 7

8 3.2 MDP formulaon of he DDWP We now formulae an MDP for he DDWP. A each wave T 0, he sysem sae s gven by,r,p S, where S s he sae space, represens he waves-o-go, R s he se of open requess, and P s he se of remanng poenal requess wh an unknown arrval me τ <. Requess no n R or P have been already served and so he par R,P belongs o he se Ξ := {R,P : R P N,R P = /0}. The maxmum number of waves and he hree possble saes for each requess open,poenal and served defne a bound on he cardnaly of he sae space gven by O3 n T. In any non-ermnal sae,r,p wh 1, we choose beween wang wh he vehcle a he depo, and dspachng he vehcle o serve a se of requess S R, whch s equvalen o selecng a roue of duraon d servng he se { R : d d}. Defne hen he acon space AR := {d R : d }, wh cardnaly On. Selecng an acon d n a gven sae, R, P ransforms he sae as follows. If a dspach of lengh d s seleced, R s paroned no he new se of unaended requess R d := { R : d > d} and he se of served requess R d = R \ R d. Tme moves forward o d and sae,r,p becomes d,r d Fd,P\F d where F d := { N : > τ d} s he se of newly arrvng requess. If no dspach occurs d = 0, he new sae s 1,R F1,P \ F 1. Le C R,P be a se funcon represenng he mnmum expeced cos-o-go a sae,r,p S. The opmal expeced cos C s defned recursvely over T 0 n 1, where ˆR s he se of known open requess a he sar of he horzon = T. Frs, a = 0 he cos-o-go s smply he sum of penales of unserved requess, and subsequenly, for each T he cos-o-go a sae,r,p s equal o he mnmum cos beween no dspach and a dspach o any dsance d AR : C 0 R,P = R p R,P Ξ 1a { [ C R,P = mn αd + E F d AR C max{1,d} Rd {0} d Fd,P \ d] } F, T,R,P Ξ 1b C [ = E ˆR CT ˆR,N \ ˆR ]. 1c Ths formulaon consders an unceran se ˆR, bu a useful specal case s when ˆR s known. The opmal acon d R,P A R {0} ha aans C R,P s hen defned as a se funcon for each sae,r,p. The vecor of opmal acons for each sae s called an opmal polcy. We can also express opmaly condons 8

9 usng a sandard LP dual reformulaon of 1, C = max E [ ˆR CT ˆR,N \ ˆR ] 2a {C 0} s.. C 0 R,P R p, R,P Ξ 2b [ C R,P E F C 1 1 R F 1,P \ F1], T,R,P Ξ 2c [ C R,P αd + E F C d Rd d Fd,P \ ] F d, T,R,P Ξ,d A R, 2d whch very clearly shows he dffculy n fndng an opmal polcy; formulaon 2 has exponenally many varables, exponenally many consrans and exponenally many erms n he expecaons. 4 A Pror Soluons for he Sochasc DDWP In hs secon we develop a pror polces for he DDWP defned n 1. We begn sudyng he deermnsc verson of he problem o undersand he srucure of opmal a pror polces. 4.1 The Deermnsc Case Suppose arrvals are known wh cerany a he begnnng of he horzon, and le he se of arrvng requess be N A := { N : τ > 0}. Requess sll arrve dynamcally over he operang perod, and hus remans nfeasble o serve a reques wh a vehcle dspach pror o s arrval me. y x dsance z T me dsance y T z me a Subopmal Operaon. b Operaon ha could be opmal. Fgure 1: Examples of vehcle operaons descrbed n he dsance versus me graph. Fgure 1a gves an nsance where arrval mes and desnaons for each reques N A are represened 9

10 by a coordnae τ,d n a dsance versus me graph. Also depced s an example vehcle dspach plan. The vehcle sars a he depo a wave T and was unl 1 when s dspached a dsance x. Then, reurns a 2 = 1 x and was unl 3 o execue a second dspach of dsance y, and so on. Requess covered by hs operaon are hose wh coordnaes nsde he shaded areas. We now sae and prove hree properes ha a leas one opmal vehcle dspach plan should sasfy: Propery 4.1 Decreasng consecuve dspaches. For all dspach pars sarng a wo waves > wh respecve dspach duraons d and d, we have d > d. Proof. If d d, by deleng he dspach a we reduce operaonal cos wh unalered coverage. Propery 4.2 No wa afer a dspach. The vehcle does no wa once he frs dspach has occurred. Proof. If a soluon was for w waves afer a dspach a, we can shf forward each vehcle dspach ha occurs pror o wave exacly w waves n me whou reducng he se of covered requess. Propery 4.3 Dspach duraon equals round-rp me o some reques. The duraon of each dspached roue equals d S = max S d, where S R s he se of requess served by he roue. Proof. If d > d S, by seng d = d S we reduce operaonal cos wh unalered reques coverage. Fgure 1b depcs an operaon ha sasfes all properes. A drec consequence of hese properes s ha we can formulae a deermnsc dynamc program wh a reduced sae space. Le he se of possble dspach duraons be D := {d N A : d τ }. We can fnd an opmal dspach plan va a dynamc program wh saes,x, where s he curren wave and x s he duraon of he prevous dspach compleed a wave x = 0 f no dspaches have occurred pror o. Fgure 2 s an example of he sysem a sae,x, where he las dspach was of duraon x a wave + x and covered all requess shaded n lgh gray. Requess shaded n medum gray wll never be served by an opmal dspach plan sasfyng he prevous hree properes and are hus los, and he requess shaded n dark gray could be covered by he nex dspach a wave. An acon gven sae,x s defned as he nex dspach duraon d A,x, where A,x = {d N A : d,d < x, τ < + x}, x D : + x T 3a A,0 = {0} {d N A : d, τ }. 3b

11 dsance unknown los requess covered requess x acon dependen fuure T + x 0 me Fgure 2: Example of sae and acon for he deermnsc DDWP. If no dspaches have occurred by x = 0, an opmal vehcle operaon may wa unl 1,.e., d = 0. Defne C x as he cos-o-go funcon n sae,x. Opmaly equaons solvable n On 2 T operaons are gven by 4, where C T 0 s he mnmum cos for he deermnsc DDWP: C 0 x = p, x D {0} 4a N A { } C 0 = mn αd d A,0 { C x = mn d A,x αd N A : d d, τ N A : d d, τ <+x p +C max{1,d} d p +C d d }, T 4b, T, x D : + x T. 4c In hs dynamc program, we nalze by assumng ha all arrved requess are no served by = 0. When we execue a dspach of duraon d, we ncur s operang cos whle also savng he penales of he requess served. 4.2 The sochasc case and a pror polces Consder agan he sochasc DDWP defned n 1. We nex develop he opmal sac a pror soluon n whch a schedule specfyng he waves a whch o dspach he vehcle and he duraon of each dspach s deermned only wh nformaon revealed a he sar of he horzon n wave T. The operang cos of such an a pror soluon s known, and he penales pad for unserved requess depend on he fuure arrvals. Ths observaon movaes an approach for deermnng an a pror soluon ha mnmzes expeced cos. Ths problem s equvalen o solvng a deermnsc DDWP nsance n whch each poenal reques N s coped T mes and assumed o arrve a every wave T for whch 11

12 s probably of arrval s posve, wh an adjused penaly for no servng he reques a wave equal o p Pτ = τ < T. Known requess τ T are no coped and arrve only a wave T wh probably one. Thus, he recursve equaons o fnd an opmal a pror polcy are a naural exenson of he deermnsc sysem 4, C AP 0 x = N:τ =T C AP 0 = mn d A,0 C AP x = mn d A,x p + Pτ > 0 τ < T p, N:τ <T αd N: τ =T, d d p Pτ τ < T p N: τ <T, d d αd P τ < + x τ < T p N: τ <T, d d +C max{1,d} AP d x D {0} 5a, T 5b + C dd AP, T,x D : + x T, 5c where he opmal expeced cos s gven by C AP 0. Noe ha hs polcy s found by solvng a deermnsc T DDWP, and so, sasfes Properes 4.1, 4.2 and 4.3. To esmae he expeced cos of mplemenng hs heursc polcy we mus see he orders arrved and wang for servce a = T. To do hs, we assume o know all orders N wh τ m = T for a gven se of M realzaons m {1,...,M} obaned va Mone Carlo samplng. For conssency n compuaonal resuls, we wll use he same M realzaons when comparng performance of lower bounds and dfferen soluons for a gven nsance. We can mprove he performance of an a pror polcy by allowng smple recourse acons durng he operaon. Le a polcy be represened by he ordered se of k dspaches, each wh dspach dsance d j and wave j : {d j, j } k j=1. The polcy sasfes j j+1 = d j and d j > d j+1 for j = 1,...,k 1. Consder he followng recourse acons: 1. Posponemen and cancellaon: Consder dspach j scheduled a wave j. Any open reques wh d d j+1 can be covered by dspach j + 1. So, f no reques has arrved snce he prevous dspach me j 1 wh d j+1 < d d j, hen pospone dspach j by modfyng s scheduled me j j 1 and s duraon d j d j 1 f d j 1 > d j+1, oherwse cancel dspach j. A rescheduled dspach s consdered agan for posponemen and cancellaon eravely. 2. Margnal prof adjusmen: Gven a dspach j ha has no been posponed or cancelled, adjus s 12

13 dsance d j o maxmze s margnal prof: Choose acual dspach dsance d equal o he locaon d r of an open reques r N wh d j+1 < d r d j, such ha maxmzes he followng margnal prof Vr mg := { N:d j+1 <d d r, τ } p d r. If V mg 0 for each possble reques r, hen agan pospone he dspach o j j 1 wh new dspach duraon d j 1 f d j 1 > d j+1, oherwse cancel dspach j. If he dspach s adjused such ha d < d j, we also pospone o me j+1 + d o poenally serve more cusomers wh no ncrease n cos. 5 Dynamc Polces for he Sochasc DDWP A pror polces, parcularly when adjused va recourse acons, may yeld reasonable soluons o many problems. However, here exs nsances for whch an opmal adjused a pror polcy s arbrarly worse han an opmal dynamc polcy. Pahologcal A Pror Insances Consder a famly of nsances wh 2 requess, T = 4, and a parameer z 0. Le locaons be d 1 = 1 and d 2 = 2, and penales p 1 = z + 1 and p 2 = z 2 + z + 2. Reques 1 arrves a τ 1 = 1, whle reques 2 arrves a τ 2 = 3 wh probably u = z z+1 and τ 2 = 2 wh probably v = 1 z+1. There are four possble a pror soluons See Fgure 3. Eher of he las wo opons c or d are opmal a pror polces wh smple recourse, and boh have expeced cos of 3 + z. ds. 2 u v ds. 2 u v ds. 2 u v ds. 2 u v a b c d Fgure 3: Feasble a pror dspach opons. The opmal dynamc polcy s dfferen. If reques 2 arrves a = 3, dspaches o d = 2 a = 3 and hen d = 1 a = 1 for oal cos of 3; oherwse dspaches o d = 2 a = 2 for cos of 3+z. The expeced cos of hs polcy s 3u zv = 3 + z z+1 < 4. As z, he opmal cos s bounded, whle any a pror polcy s cos s unbounded. 13

14 5.1 A Pror-Based Rollou Polcy One approach o buld a dynamc polcy s o roll ou he a pror polcy. A each wave T when he vehcle s avalable, we recompue an opmal a pror polcy gven updaed nformaon regardng requess open, poenal, and served; f he polcy dcaes a dspach d > 0 a, he decson s execued and a new a pror polcy s hen compued a d, oherwse a new a pror polcy s compued a 1. Compung such a rollou polcy requres On 2 T 3 operaons. 5.2 Approxmae Lnear Programmng for he DDWP Heursc dynamc polces can be generaed va he dual MDP reformulaon 2. Because hs formulaon has exponenally many varables and consrans, he ALP approach resrcs s feasble regon n such a way ha he resulng opmzaon model s racable and so yelds a lower bound for he opmal expeced cos-o-go ha can be used whn a rollou polcy. We can generae a lower bound C ALP R,P for he cos-o-go funcon of he DDWP a any feasble sae,r,p by represenng C R,P as a lnear funcon of a predeermned se of bass funcons, and hen solvng he resulng resrcon of 2. Le C R,P C ALP R,P, where C ALP R,P := R a + P b k=1 v k, 6 and where a represens he cos of reques f s open a wave, b represens he cos of poenal reques f hasn arrved by wave, and v k represens he ncremenal value of each wave k. Proposon 5.1. Applyng he resrcon 6 o 2 yelds a model equvalen o C ALP = max {a,b,v,s,u} N Pτ = T a T + Pτ < T b T T v =1 7a s.. a 0 = p,b 0 = 0, N 7b s a a 1, N, T 7c s b f a 1 f b 1, N, T 7d s v, T 7e N u d a 1 d >da d, N, T,d A N 7f 14

15 u d b g d a d ḡ d b d, N, T,d AN 7g N u d k= d+1 v k + αd T,d A N 7h u,s 0, 7 where f := Pτ = 1 τ < s he condonal probably ha poenal reques a wave arrves a he nex wave, g d := Pτ d τ < s he condonal probably ha poenal reques a wave arrves n one of he nex d waves, and also ḡ d := 1 gd and f := 1 f. Any se of values {a,b,v} used o compue C ALP R,P whch are feasble for 7b-7 yeld a lower bound of he cos-o-go funcon a any sae,r,p: C ALP R,P C R,P. In parcular, we have C ALP C. The proposon s proof s n he appendx. Model 7 has neresng properes whch gve economc nuon and accelerae compuaon mes; each of hese properes s proved n he appendx. Propery 5.2 Bounds. We may assume 0 a p and 0 b g p, N, T 0 whou loss of opmaly. Inuvely, Propery 5.2 mples ha he ndvdual cos per open reques a any wave s nonnegave and canno exceed he penaly for leavng he reques unaended, and ha he ndvdual cos for any poenal reques a any wave s nonnegave and canno exceed he penaly dscouned by he arrval probably. Propery 5.3 Los requess. Whou loss of opmaly, we may assume ha a = p for any N, T 0 : d >, and b = g p for any N, T 0 : d. Propery 5.3 says ha he cos of havng an open reques a me wh an mpossble dspach d > s equal o p. A smlar dea movaes he expresson for b. The followng heorem, also proved n he appendx, descrbes he performance of he ALP lower bound n he deermnsc case. Theorem 5.4 Srong dualy for he deermnsc case. Assume reques s arrval wave τ s deermnsc for each reques N. Then he bound gven by 7 s gh,.e., equal o he opmal cos of he deermnsc DDWP gven n 4. 15

16 The resul gves furher movaon o use ALP for he DDWP, snce he approxmaon s able o recover opmaly n he deermnsc case. Furhermore, relaes he ALP and he a pror soluon: f we ransform a sochasc nsance no a deermnsc one as descrbed n Secon 4, he ALP maches he a pror soluon, and boh can be used heurscally. However, he ALP can also be used whou he ransformaon, so can be vewed as a generalzaon of he a pror rollou polcy. We nex apply 7 o approxmae he opmal acon d R,P. Gven a feasble a,b,v o 7b-7, we have a closed lnear form lower bound for he expeced cos-o-go funcon measured afer a decson wh dspach dsance d AR has been aken va [ E F C d Rd d Fd,P \ d] F EF d [ = E F d = Rd a d [ C ALP d Rd Fd,P \ d] F Rd Fd a d + P g d a d ] + P\F d b d d k=1 v k + ḡ d b d d k=1 v k. 8 A smlar expresson can be obaned o underesmae he expeced cos-o-go measured afer he vehcle was for one wave a he depo. We use hese bounds o compue an approxmaely opmal acon d ALP R,P. Any feasble se of values {a, b, v} provdes an underesmae of he expeced cos-o-go n 8. In parcular, he ghes lower bound s acheved when maxmzng 8 subjec o 7b-7. Ths s a pos sae and decson re-opmzaon of he ALP n whch he values of {a,b,v} are recompued a each wave when he vehcle s ready a he depo, and for each poenal acon d AR m {0}. We compue he approxmae opmal acon by d ALP R, P = argmn d A R {0} max {a,b,v 7b 7} R a 1 αd + max {a,b,v 7b 7} Rd a d + P f a 1 + P g d a d + f b ḡ d b d k=1 v k, f d = 0 d k=1 v k, f d AR Ths nvolves solvng OnT lnear programs sharng he same feasble se of soluons. Is performance can be mproved by applyng LP warmsar and rulng ou subopmal dspach dsances see he appendx for deals. The procedure s repeaed for each realzaon m = 1,...,M of arrvals o esmae s expeced cos

17 6 Compuaonal Expermens We presen wo ses of compuaonal expermens usng dfferen famles of randomly generaed nsances. Our goal s o es he qualy of he varous heurscs and o oban qualave nsghs regardng soluons. The wo ses of expermens dffer n her models of he reques arrval process. In he frs se, we assume ha he condonal lkelhood of a reques arrval by he nex dspach a wave s consan over me bu may vary by reques. In he second se, we use an arrval dsrbuon ha assgns probables for he arrval me or he non-arrval even for each reques usng a mean arrval ha vares by reques. All heurscs were programmed n Java and compued usng a 2.1GHz Inel Core QM processor wh 8 GB RAM, usng CPLEX 12.4 when necessary as he LP solver. Table 2 summarzes he lower bounds and heursc polces coss ha we compued for he nsances n hs sudy. We do no nclude he ALP lower bound, as our prelmnary expermens revealed o be weaker han he PIR bound. Smlar behavor has been observed n oher sochasc roung conexs, e.g., [53]. For each parcular nsance, we smulaed M = 0 realzaons of he arrval me vecor τ, and use hs common se o esmae lower bounds and polces expeced coss va Mone Carlo samplng. Table 2: Lower bounds and heursc polces coss compued Type Lower bound A pror polces Dynamc polces Procedures perfec nformaon relaxaon PIR Sac a pror polcy AP & a pror polcy wh recourse acons APR dynamc a pror polcy rollou DAP & dynamc ALP polcy DALP 6.1 Desgn of Insance Se 1: Saonary Condonal Arrval Probably The frs se of expermenal nsances model arrvals usng a saonary condonal arrval dsrbuon for each reques. Therefore, for each N, he probably ha arrves a wave gven ha has no ye arrved s ndependen of,.e., Pτ = T = Pτ 1 τ < = θ and Pτ = 1 = 1 θ T 1. We consruc nsances wh dfferen sze, geography, and me flexbly as follows. Le n,l,r defne an nsance where n s he number of poenal requess over he horzon; l s he maxmum dsance beween a reques and he depo; and r := T /l s he rao beween he oal number of waves T and l. We consder all combnaons of n {5,,,40,60,80,0}, l {5,,}, and r = {1,2,3} and generae random nsances for each combnaon by varyng he vecors {θ }, {p }, and {d } as {θ }: probably parameer θ for each drawn..d. from a connuous unform dsrbuon, U 1 2T, 2 T ; 17

18 {p }: penaly parameer p drawn wh equal probably from he values {0.25l,0.5l,0.75l,l} {d }: dsance parameer d drawn wh equal probably from he values {1,...,l}. 6.2 Resuls for Insance Se 1 Fgure 4 repors he average dualy gap beween he PIR bound and he opmal expeced cos for small nsances n {5,} where he fully dynamc-sochasc problem s solvable o opmaly. % Gap = 0 bound op % Gap = 0 bound op r l r l a n = 5 b n = Fgure 4: Percenage gap beween PIR lower bound and opmal soluon values for Insance Se 1 Table 3: Overall performance of heurscs n Insance Se 1 Upper Bound %GAP vs OPT small nsances %GAP vs lower bound Tme per sample-nsance secs AP 11.53% 12.14% APR 5.27% 9.24% DAP 1.97% 6.59% DALP 1.65% 6.42% Table 3 presens average gap and soluon mes for each heursc. In case of he ALP-based polcy DALP, we employed a hybrd approach ha execues DAP unl he operaon reaches wave xl and, aferwards, execues an ALP-based polcy. The movaon s he wo polces complemenary behavor. The ALP ends o be oo conservave nally, when he remanng horzon ncludes many possbles has o under-approxmae, whle DAP smply assumes averages for he fuure; conversely, owards he end of he horzon he ALP can more accuraely assess possble fuure recourse acons, and hus can make beer decsons. Also, he lnear programs n he ALP end o have hghly degenerae polyopes for nsances wh hgh flexbly, makng hem dffcul o solve. Afer searchng over a grd of dfferen values n prelmnary expermens, we concluded ha x = 1.1 yelds he bes gap whle sll keepng compuaon mes low. Ths 18

19 conrass wh nave mplemenaons of ALP polces, whch can be compuaonally demandng. For small nsances wh n = 5 or n =, Fgure 5 shows he average relave gap o he opmal soluon. The dynamc a pror polcy rollou DAP and he dynamc ALP-based polcy DALP domnae he a pror soluons and acheve an average gap of 1.97% and 1.65%, respecvely. %gap = 0 bound op DALP DAP AP APR %gap = 0 bound op DALP DAP AP APR l r l r a n = 5 b n = Fgure 5: Average percenage gap beween heursc soluon coss and opmal coss for Insance Se 1 For larger nsances, he gap s compued wh respec o he PIR bound. Fgure 6 deals average gaps over all classes of nsances. %gap = 0 bound lb DALP DAP APR AP %gap = 0 bound lb DALP DAP APR AP %gap = 0 bound lb DALP DAP APR AP n l n l n l a r = 1 b r = 2 c r = 3 Fgure 6: Average percenage gap beween heursc soluon coss and lower bound for Insance Se 1 As expeced based on each heursc s recourse possbles, APR ouperforms AP and boh are ouperformed by he wo dynamc polces DAP and DALP. Also, he gap dfferences beween AP, APR and he dynamc polces decrease wh n. Ths suggess ha dynamc soluons produce a bgger gap mprovemen for nsances wh more reques arrval granulary,.e., where an early or lae arrval can sgnfcanly mpac coss unless correcve acons are aken. Conversely, for nsances wh more requess he margnal 19

20 value of dynamc soluons s smaller. Ths may be due o rsk poolng effecs beween requess, e.g., f one ou of 0 requess arrves early, anoher one wll lkely arrve lae and he relave dsurbance wll be mnor. Moreover, he relave gap of boh dynamc polces as a funcon of n reaches a maxmum and hen decreases as n grows. Ths confrms her effecveness for large n. Also, all four heurscs gaps ncrease as a funcon of r; he level of flexbly ranslaes no soluon complexy for our heurscs. Addonally, he gap ends o ncrease wh l; hs s lkely relaed o an ncrease n he problem s complexy. Fnally, he ALP-based polcy has an average gap smaller han DAP. For less flexble nsances r = 1 boh approaches average a relave gap of 3.4%, bu when he varably and recourse flexbly ncreases o r = 2 and r = 3 mproves over DAP, from 7.3% o 7.1% for r = 2 and from 9.0% o 8.8% for r = 3. Alhough small, hs mprovemen was conssenly observed across all nsances. 6.3 Desgn of Insance Se 2: Unform Arrvals The prevous arrval dsrbuons defned by a sngle parameer could be hdng neresng nerdependences beween mean, varance, probably of arrval, and degree of dynamsm. We defned a second se of expermens wh a fxed number of requess n =, waves T = 30 and maxmum locaon l =. The dsance vecor d and penaly vecor p are se as n he prevous expermens, bu arrvals have dsrbuons wh a probably w of arrval a he begnnng of he horzon.e., he degree of dynamsm, a probably q of no showng up, and a dscree unform probably 1 w q 2v+1 of arrvng durng he operaon a wave = max{1, µ v},...,mn{t 1, µ + v}, where µ s a reques-dependen parameer drawn..d. from a dscree unform dsrbuon U0,T 1 for each N. The parameer v represens he arrval varably half of he arrval range. We creaed nsances for each se of parameers v,q,w n he se {v,q,w : v = {0,1,2,4,8,30},q = {0,0.2,0.4},w = {0,0.2,0.4,0.6,0.8,1} : r + q 1}. 6.4 Resuls for Insance Se 2 Table 4: Overall performance of heurscs n Insance Se 2. Upper Bound %GAP Tme per sample-nsance secs AP 7.54% APR 5.62% DAP 4.46% DALP 4.24% Table 4 presens overall resuls for each heursc over he second se of expermens. We noce ha our

21 smple recourse rules n APR capure 58% = of he oal gap mprovemen ha he bes dynamc heursc capures over he sac soluon AP. Fgure 7 presens average relave gaps over nsances wh dfferen sengs of parameers q w or v. %GAP = 0 value lb DALP DAP APR AP q w %GAP = 0 value lb DALP DAP APR AP v Fgure 7: Average percenage gap beween heurscs cos and lower bound n Insance Se 2. From hese graphs we conclude ha he relave gaps of all four polces decrease as w ncreases; he more nformaon avalable a he nal wave, he closer we can ge o a deermnsc problem. There s zero gap n he exreme deermnsc cases w+q = 1. The value of dynamc soluons also decreases when w ncreases, whch s expeced, snce a smaller w mples a larger degree of dynamsm and more mporance s placed on recourse acons. Regardng he reques arrval probably, he gap ncreases wh q unless w + q = 1. Ths means ha s harder o opmze an nsance for whch here s a bgger probably of no arrval. The value of dynamc soluons also grows wh q. Wh respec o he varably of he nsance, he gap ncreases as v ncreases. Ths may be due boh o a decrease n he lower bound s qualy and o an ncrease n he opmal expeced cos. Fnally, he dynamc heurscs yeld larger margnal coss savngs when v ncreases. Ths means ha he more varably he sysem has, he more mporan s o mplemen a dynamc soluon. There s also a range of nermedae varably for whch DALP clearly domnaes DAP. In hs range, he addonal complexy of ALP yelds he mos benef. Table 5 provdes four examples of nsance famles whn hs rage; her average percen reducon n relave gap of DALP over DAP s 15.0%. 21

22 Table 5: Average gap percen reducon of DALP for cases wh nermedae varably n Insance Se 2. Famly q,w,v DALP %GAP DAP %GAP % reducon over DAP 0.4, 0.2, % 9.67% 14.9% 0.4, 0.2, 8.79% 12.41% 13.1% 0.4, 0.4, % 8.95% 16.6% 0.4, 0.4, %.31% 15.8% Aggregae 8.79%.34% 15.0% 7 Conclusons We have formulaed he dynamc dspach waves problem DDWP o capure he basc aspecs of dspach and roung decsons for same-day delvery. Ths papers naes work on he DDWP by sudyng he sngle-vehcle sochasc case where cusomer desnaons are placed over he lne. We develop a se of racable soluon polces ha dffer n her soluon dynamsm, from an a pror soluon o fully dynamc polces. Our compuaonal expermens ndcae ha he performance of an a pror polcy s good, especally when we nclude heursc mprovemens. In compuaonal ess over wo nsance ses hs polcy yelds an expeced cos whn 9.24% and 5.62% of he bes lower bound. Neverheless, we prove ha he benef of a fully dynamc polcy can be unbounded n he wors-case scenaro. Accordngly, we proposed and expermenally esed wo dynamc polces ha dffer by he naure of he approxmae cos-o-go funcon: he rollou of he a pror soluon and an ALP-based dynamc polcy. The rollou of he a pror polcy compues hs polcy a he sar of he horzon, bu only mplemens he frs acon, hen updaes all known nformaon and re-compues a new a pror soluon. In boh ses of nsances cus he a pror polcy s gap by 28.7% and.6%, respecvely. We have also found ha a dynamc polcy ha ncorporaes he ALP approach yelds he bes possble resuls. Is margnal mprovemen as gap reducon for boh ses of expermens s 2.6% and 4.9%, respecvely. In nsance famles wh nermedae varably, hs gap reducon grows o 15.0%. A fnal concluson of our sudy concerns he relave value of dynamc polces. Wh all oher hngs beng equal, he benef of a dynamc polcy over he opmal a pror soluon evenually decreases as n grows,.e., as he number of poenal orders ncreases. Ths s unsurprsng, snce for larger numbers of poenal orders one would expec an averagng effec. We found he maxmum benef n dynamc polces for order ses of around o 50; for smaller numbers, he exac opmal soluon s sll racable, whereas for larger numbers he a pror polcy s close o opmaly. Many same-day delvery applcaons, such as grocery home delvery, mgh expec maxmum daly order volume around hese numbers. Smlarly, 22

23 dynamc polces benefs decrease as orders become more lkely o appear a he sar of he horzon. In oher words, f many of he orders are no placed n he same day a all, bu raher are carred over from he prevous day, an a pror polcy performs que well. I s precsely n he mos unceran envronmens, where orders can appear a any momen, ha new models such as ours offer he mos benef. Fuure work on he DDWP needs o consder he soluon on a general nework opology, and hus become more applcable for SDD operaons n urban neworks. Ths problem s que challengng; n addon o dspach decsons, needs o deal wh dffcul vehcle roung problems. Gven hs addonal dffculy, one could deal wh hs problem by desgnng heurscs based on nsghs from he one-dmensonal case. I would also be neresng o exend hs model o mulple vehcles ha could pool he rsk assocaed wh leavng orders unaended and herefore reduce coss. Oher exensons could be ncorporang vehcle servce mes a each locaon or ncludng cusomer servce me wndows nsead of a deadlne a he end of he day. In general, same-day delvery offers many new challenges o he logscs research communy. References [1] D. Adelman, A Prce-Dreced Approach o Sochasc Invenory/Roung, Operaons Research 52 04, [2] A. Ak and A. Erera, A pared-vehcle recourse sraegy for he vehcle-roung problem wh sochasc demands, Transporaon scence 41 07, no. 2, [3] M. Albareda-Sambola, E. Fernández, and G. Lapore, The dynamc mulperod vehcle roung problem wh probablsc nformaon, Compuers & Operaons Research 48 14, no. 0, [4] E. Angelell, M. Savelsbergh, and M.G. Speranza, Compeve analyss of a dspach polcy for a dynamc mul-perod roung problem, Operaons Research Leers 35 07, no. 6, [5] E. Angelell, M.G. Speranza, and M.W.P. Savelsbergh, Compeve analyss for dynamc mulperod uncapacaed roung problems, Neworks 49 07, no. 4, [6] D.L. Applegae, R.E. Bxby, V. Chváal, and W.J. Cook, The ravelng salesman problem: A compuaonal sudy, Prnceon Unversy Press, Prnceon, New Jersey, 06. [7] C. Arche, D. Felle, and M.G. Speranza, Complexy of roung problems wh release daes, workng paper, Unversy of Bresca 15. [8] R. Ben and P. van Henenryck, Scenaro-based plannng for parally dynamc vehcle roung wh sochasc cusomers, Operaons Research 52 04, no. 6, [9] D. Bersekas, Dynamc programmng and opmal conrol, vol. 1, Ahena Scenfc Belmon, MA,

24 [] D. Bersekas and D. Casañón, Rollou algorhms for sochasc schedulng problems, Journal of Heurscs , no. 1, [11] D. Bersmas, A vehcle roung problem wh sochasc demand, Operaons Research , no. 3, [12] D. Brown, J. Smh, and P. Sun, Informaon relaxaons and dualy n sochasc dynamc programs, Operaons research 58, [13] Y. Bukchn, E. Khmelnsky, and P. Yakuel, Opmzng a dynamc order-pckng process, European Journal of Operaonal Research , no. 2, [14] A.M. Campbell and B. Thomas, Challenges and advances n a pror roung, The Vehcle Roung Problem: Laes Advances and New Challenges, Sprnger, 08, pp [15], Probablsc ravelng salesman problem wh deadlnes, Transporaon Scence 42 08, no. 1, [16] D. Caaruzza, N. Abs, and D Felle, The mul rp vehcle roung problem wh me wndows and release daes, Transporaon Scence, forhcomng 15. [17] E. Çeven and K.R. Gue, Opmal wave release mes for order fulfllmen sysems wh deadlnes, workng paper, Unversy of Lousvlle 15. [18] T. Cheong and C. Whe, Dynamc ravelng salesman problem: Value of real-me raffc nformaon, IEEE Transacons on Inellgen Transporaon Sysems 13 12, no. 2. [19] J.F. Cordeau, G. Lapore, M. Savelsbergh, and D. Vgo, Vehcle roung, Transporaon, handbooks n operaons research and managemen scence 14 06, [] D.P. de Faras and B. van Roy, The lnear programmng approach o approxmae dynamc programmng, Operaons Research 51 03, no. 6, [21] V. Desa, V. Faras, and C. Moallem, Bounds for Markov decson processes, Renforcemen Learnng and Approxmae Dynamc Programmng for Feedback Conrol 11, [22] S. Emde and N. Boysen, One-dmensonal vehcle schedulng wh a fron-end depo and non-crossng consrans, OR Specrum 36 14, no. 2, [23] G. Erdoğan, M. Baarra, and G. Lapore, Schedulng wn robos on a lne, Naval Research Logscs NRL 61 14, no. 2, [24] A. Erera, M. Savelsbergh, and E. Uyar, Fxed roues wh backup vehcles for sochasc vehcle roung problems wh me consrans, Neworks 54 09, no. 4, [25] N. Gademann and S. Velde, Order bachng o mnmze oal ravel me n a parallel-asle warehouse, IIE ransacons 37 05, no. 1, [26] M. Gendreau, G. Lapore, and R. Ségun, Sochasc vehcle roung, European Journal of Operaonal Research , no. 1,

25 [27] G. Godfrey and W. Powell, An adapve dynamc programmng algorhm for dynamc flee managemen, : Sngle perod ravel mes, Transporaon Scence 36 02, no. 1, [28] B.L. Golden, S. Raghavan, and E.A. Wasl eds., The Vehcle Roung Problem: Laes Advances and New Challenges, Sprnger, 08. [29] J. Goodson, J. Ohlmann, and B. Thomas, Rollou polces for dynamc soluons o he mulvehcle roung problem wh sochasc demand and duraon lms, Operaons Research 61 13, no. 1, [30] G. Gun and A.P. Punnen eds., The Travelng Salesman Problem and Is Varaons, Kluwer Academc Publshers, Dordrech, The Neherlands, 02. [31] Amazon.com Inc., Annual repor form -k, 13. [32] P. Jalle, Probablsc ravelng salesman problems, Ph.D. hess, Massachuses Insue of Technology, [33], A pror soluon of a ravelng salesman problem n whch a random subse of he cusomers are vsed, Operaons Research , no. 6, [34] P. Jalle and M. Wagner, Onlne vehcle roung problems: A survey, n Golden e al. [28], pp [35] A. Kenyon and D. Moron, Sochasc vehcle roung wh random ravel mes, Transporaon Scence 37 03, no. 1, [36] G. Lapore, F. Louveaux, and H. Mercure, The vehcle roung problem wh sochasc ravel mes, Transporaon scence , no. 3, [37], A pror opmzaon of he probablsc ravelng salesman problem, Operaons Research , no. 3, [38] A. Larsen, O.B.G. Madsen, and M.M. Solomon, Recen developmens n dynamc vehcle roung sysems, The Vehcle Roung Problem: Laes Advances and New Challenges, Sprnger, 08, pp [39] C. Lee, K. Lee, and S. Park, Robus vehcle roung problem wh deadlnes and ravel me/demand uncerany, Journal of he Operaonal Research Socey 63 11, no. 9, [40] T. Lepälä, On he soluons of sochasc ravelng salesman problems, European Journal of Operaonal Research , no. 4, [41] X. L, P. Tan, and S. Leung, Vehcle roung problems wh me wndows and sochasc ravel and servce mes: models and algorhm, Inernaonal Journal of Producon Economcs 125, no. 1, [42] T. Ma and P. Zhao, A revew of algorhms for order bachng problem n dsrbuon cener, Inernaonal Conference on Logscs Engneerng, Managemen and Compuer Scence LEMCS 14, Alans Press, 14. [43] C. Novoa and R. Sorer, An approxmae dynamc programmng approach for he vehcle roung problem wh sochasc demands, European Journal of Operaonal Research , no. 2,

26 [44] V. Pllac, M. Gendreau, C. Guére, and A. Medagla, A revew of dynamc vehcle roung problems, European Journal of Operaonal Research , no. 1, [45] W. Powell, Approxmae dynamc programmng: Solvng he curses of dmensonaly, John Wley & Sons, 07. [46] H. Psarafs, Dynamc vehcle roung: Saus and prospecs, Annals of Operaons Research , no. 1, [47] M. Puerman, Markov decson processes: dscree sochasc dynamc programmng, John Wley & Sons, 09. [48] P. Schwezer and A. Sedmann, Generalzed polynomal approxmaons n markovan decson processes, Journal of mahemacal analyss and applcaons , no. 2, [49] N. Secomand and F. Margo, Reopmzaon approaches for he vehcle-roung problem wh sochasc demands, Operaons Research 57 09, no. 1, [50] V. Sehgal, Onlne real forecas US, 13 o 18, Techncal repor, Forreser Research 14. [51] D. Taş, N. Dellaer, T. van Woensel, and T. de Kok, Vehcle roung problem wh sochasc ravel mes ncludng sof me wndows and servce coss, Compuers & Operaons Research 12. [52] B. Thomas, Dynamc vehcle roung, Wley Encyclopeda of Operaons Research and Managemen Scence. [53] A. Torello, W.B. Haskell, and M. Poremba, A Dynamc Travelng Salesman Problem wh Sochasc Arc Coss, Operaons Research 62 14, [54] S. Vocca, A.M. Campbell, and B. Thomas, The probablsc ravelng salesman problem wh me wndows, EURO Journal on Transporaon and Logscs 12, [55] M. Wen, J.-F. Cordeau, G. Lapore, and J. Larsen, The dynamc mul-perod vehcle roung problem, Compuers & Operaons Research 37, no. 9, [56] C. Zhang, Y. Wan, J. Lu, and R.J. Lnn, Dynamc crane deploymen n conaner sorage yards, Transporaon Research Par B: Mehodologcal 36 02, no. 6,