Robust Multi-Stage Decision Making

Transcription

1 Robust Multi-Stage Decision Making Erick Delage 1 Dan Iancu 2 1 CRC in Decision Making under Uncertainty HEC Montréal 2 Graduate School of Business Stanford University INFORMS Tutorial November 1st, / 53

2 Decisions Under Uncertainty Decision maker chooses x; u unknown minimize Cpx, uq x s.t. F px, uq ě 0 2 / 53

3 Decisions Under Uncertainty Decision maker chooses x; u unknown minimize Cpx, uq x s.t. F px, uq ě 0 Stochastic model: min x E u Cpx, uq s.t. Pr F`x, u ě 0 s ě 1 ɛ Distribution for u known Well-defined objective : average performance Good data, future like past,... min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U Uncertainty set U known Well-defined objective : worst-case performance Poor data, non-stationarity,... 2 / 53

4 Decisions Under Uncertainty Decision maker chooses x; u unknown minimize Cpx, uq x s.t. F px, uq ě 0 Stochastic model: min x E u Cpx, uq s.t. Pr F`x, u ě 0 s ě 1 ɛ Distribution for u known Well-defined objective : average performance Good data, future like past,... min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U Uncertainty set U known Well-defined objective : worst-case performance Poor data, non-stationarity,... Infinitely many constraints 2 / 53

5 Robust Optimization min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U For many classes of U, can reformulate as convex optimization 3 / 53

6 Robust Optimization min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U For many classes of U, can reformulate as convex optimization Soyster [1973], Falk [1976], Ben-Tal and Nemirovski [1998, 1999, 2000, 2002], Ben-Tal et al. [2002], El-Ghaoui and Lebret [1997], El-Ghaoui et al. [1998], Bertsimas and Sim [2003, 2004], Bertsimas et al. [2004],... 3 / 53

7 Robust Optimization min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U For many classes of U, can reformulate as convex optimization Soyster [1973], Falk [1976], Ben-Tal and Nemirovski [1998, 1999, 2000, 2002], Ben-Tal et al. [2002], El-Ghaoui and Lebret [1997], El-Ghaoui et al. [1998], Bertsimas and Sim [2003, 2004], Bertsimas et al. [2004],... Focus on tractability, degree of conservatism, probabilistic guarantees 3 / 53

8 Robust Optimization min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U For many classes of U, can reformulate as convex optimization Soyster [1973], Falk [1976], Ben-Tal and Nemirovski [1998, 1999, 2000, 2002], Ben-Tal et al. [2002], El-Ghaoui and Lebret [1997], El-Ghaoui et al. [1998], Bertsimas and Sim [2003, 2004], Bertsimas et al. [2004],... Focus on tractability, degree of conservatism, probabilistic guarantees Some decisions adjustable Ñ approach extended to dynamic settings Ben-Tal et al. [2004] 3 / 53

9 Robust Optimization min x Robust model: max Cpx, uq upu s.t. F px, uq ě u P U Applications... inventory management e.g., [Ben-Tal et al., 2005, Bertsimas and Thiele, 2006, Bienstock and Özbay, 2008,...] facility location and transportation [Baron et al., 2011,...] scheduling [Lin et al., 2004, Yamashita et al., 2007, Mittal et al., 2014,...] revenue management [Perakis and Roels, 2010, Adida and Perakis, 2006,...] project management [Wiesemann et al., 2012, Ben-Tal et al., 2009,...] energy generation and distribution [Zhao et al., 2013, Lorca and Sun, 2015,...] portfolio optimization [Goldfarb and Iyengar, 2003, Tütüncü and Koenig, 2004, Ceria and Stubbs, 2006, Pinar and Tütüncü, 2005, Bertsimas and Pachamanova, 2008,...] healthcare [Borfeld et al., 2008, Hanne et al., 2009, Chen et al., 2011,...] Book [Ben-Tal et al., 2009]; Review papers: [Bertsimas et al., 2011a, Gabrel et al., 2012] 4 / 53

10 Robust Optimization min x Our objectives with this tutorial... Robust model: max Cpx, uq upu s.t. F px, uq ě u P U Discuss static versus adjustable decisions Highlight some tractability issues Clarify the connection to robust dynamic programming Provide motivations for simple policies Illustrate time consistency issues 5 / 53

11 Outline 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 6 / 53

12 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 7 / 53

13 A Simple Inventory Problem Consider the following inventory management problem: ordering Tÿ hkikjcost hkkkkkikkkkkj holding cost hkkkkkkikkkkkkj backlog cost minimize c t x t ` h t py t`1 q` ` b t p y t`1 q` x,y where t 1 s.t. y t`1 y t ` x t d t, (Stock balance) 0 ď x t ď M t, (Min/max order size) y 1 a, (Initial stock level) x t is number of goods ordered at time t and received at t ` 1 y t is number of goods in stock at beginning of time t d t is demand between time t and t ` 1 a is the initial inventory 8 / 53

14 A Linear Programming Formulation This problem can be reformulated using the linear program minimize x,y,s`,s Tÿ `ct x t ` h t s`t ` b ts t t 1 s.t. s`t ě 0, s t ě t, s`t ě y t, s t ě y t, y t`1 y t ` x t d t, 0 ď x t ď M t, where s`t s t is amount of goods held in storage during stage t is amount of backlogged customer demands during stage t 9 / 53

15 A Linear Programming Formulation This problem can be reformulated using the linear program minimize x,y,s`,s Tÿ `ct x t ` h t s`t ` b ts t t 1 s.t. s`t ě 0, s t ě t, s`t ě y t, s t ě y t, y t`1 y t ` x t d t, 0 ď x t ď M t, How can we make this model robust to demand perturbations? 10 / 53

16 Naïve Robustification Scheme Given that the vector of demand d is assumed to lie in some uncertainty set U, let s consider the robust optimization model: minimize x,y,s`,s s.t. Tÿ `ct x t ` h t s`t ` b ts t t 1 s`t ě 0, s t ě t s`t ě y t s t ě y t y t`1 y t ` x t d d P t 0 ď x t ď M t 11 / 53

17 Naïve Robustification Scheme Given that the vector of demand d is assumed to lie in some uncertainty set U, let s consider the robust optimization model: minimize x,y,s`,s s.t. Tÿ `ct x t ` h t s`t ` b ts t t 1 s`t ě 0, s t ě t s`t ě y t s t ě y t y t`1 y t ` x t d d P t 0 ď x t ď M t Unfortunately, this makes the model infeasible even when U 2: # + y t`1 y t ` x t d p1q t y t`1 y t ` x t d p2q ñ d p1q t d p2q t t 11 / 53

18 A Less Naïve Robustification Scheme Robustify an alternate linear programming formulation: minimize x,s`,s Tÿ `ct x t ` h t s`t ` b ts t t 1 s.t. s`t ě 0, s t ě tÿ s`t ě y 1 ` x t 1 d t s t ě y 1 ` t 1 1 tÿ t ď x t ď M d t 1 x t where we simply replaced y t`1 : y 1 ` řt t 1 1 x t 1 d t1 in order to capture the fact that stock level evolves according to demand. 12 / 53

19 A Less Naïve Robustification Scheme Robustify an alternate linear programming formulation: minimize x,s`,s Tÿ `ct x t ` h t s`t ` b ts t t 1 s.t. s`t ě 0, s t ě tÿ s`t ě y 1 ` x t 1 d t d P t, s t ě y 1 ` t 1 1 tÿ t ď x t ď M t. d t 1 x t d P t, 13 / 53

20 A Less Naïve Robustification Scheme Robustify an alternate linear programming formulation: minimize x,s`,s Tÿ `ct x t ` h t s`t ` b ts t t 1 Still two issues remain: s.t. s`t ě 0, s t ě tÿ s`t ě y 1 ` x t 1 d t d P t, s t ě y 1 ` t 1 1 tÿ t ď x t ď M t. d t 1 x t d P t, 1 the orders should be adjustable w.r.t. the observed demand 2 ps`t, s t q should be fully adjustable (more subtle) 13 / 53

21 Why ps`t, s t q Should Be Fully Adjustable 14 / 53

22 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, 14 / 53

23 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, Less naïve robust model: min. 0.5x 1 ` s`1 ` s 1 x 1,s`,s 1 1 s.t. s`1 ě 0, s 1 ě 0 s`1 ě x 1 d d 1 P r0, 2s s 1 ě d 1 x d 1 P r0, 2s 0 ď x 1 ď / 53

24 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, Less naïve robust model: min. 0.5x 1 ` s`1 ` s 1 x 1,s`,s 1 1 s.t. s`1 ě 0, s 1 ě 0 s`1 ě x d 1 P r0, 2s s 1 ě 2 x d 1 P r0, 2s 0 ď x 1 ď / 53

25 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, Less naïve robust model: min. x 1 0.5x 1 ` x 1 lomon s` 1 s.t. 0 ď x 1 ď 2. ` 2 x 1 lomon s 1 16 / 53

26 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, Less naïve robust model: min. x 1 0.5x 1 ` x 1 lomon s` 1 s.t. 0 ď x 1 ď 2. ` 2 x 1 lomon s 1 Conclusions: Less naïve robust model states x 1 worst-case cost of 2 : 0, s` 1 : 0, and s 1 : 2 with 16 / 53

27 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: min. x 1 ordering hkikj hkkkkkikkkkkj holding 0.5x 1 ` px 1 d 1 q` ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, Less naïve robust model: min. x 1 0.5x 1 ` x 1 lomon s` 1 s.t. 0 ď x 1 ď 2. ` 2 x 1 lomon s 1 Conclusions: Less naïve robust model states x 1 worst-case cost of 2 : 0, s` 1 : 0, and s 1 : 2 with Alternatively, x 1 : 1 achieves a total cost lower than 1.5 for all d 1 P r0, 2s 16 / 53

28 Why ps`t, s t q Should Be Fully Adjustable Consider the two-stage problem: Deterministic model: Less naïve robust model: min. x 1 ordering hkikj holding+backlog hkkkikkkj 0.5x 1 ` x 1 d 1 ` pd 1 x 1 q` looooomooooon backlog s.t. 0 ď x 1 ď 2, min. x 1 0.5x 1 ` x 1 lomon s` 1 s.t. 0 ď x 1 ď 2. ` 2 x 1 lomon s 1 Conclusions: Less naïve robust model states x 1 worst-case cost of 2 : 0, s` 1 : 0, and s 1 : 2 with Alternatively, x 1 : 1 achieves a total cost lower than 1.5 for all d 1 P r0, 2s 17 / 53

29 An Accurate Two-stage Robust Inventory Model The robust two-stage problem actually takes the form: minimize x 1 sup d 1 Pr0,2s s.t. 0 ď x 1 ď 2, 0.5x 1 ` hpx 1, d 1 q 18 / 53

30 An Accurate Two-stage Robust Inventory Model The robust two-stage problem actually takes the form: where minimize x 1 sup d 1 Pr0,2s s.t. 0 ď x 1 ď 2, 0.5x 1 ` hpx 1, d 1 q hpx 1, d 1 q : min s`,s 1 1 s.t. s`1 s`1 ` s 1 ě 0, s 1 ě 0 s`1 ě x 1 d 1 s 1 ě x 1 ` d / 53

31 An Accurate Two-stage Robust Inventory Model The robust two-stage problem actually takes the form: where minimize x 1 sup d 1 Pr0,2s s.t. 0 ď x 1 ď 2, 0.5x 1 ` hpx 1, d 1 q hpx 1, d 1 q : min s`,s 1 1 s.t. s`1 s`1 ` s 1 ě 0, s 1 ě 0 s`1 ě x 1 d 1 s 1 ě x 1 ` d 1. Note how this form accounts for the fact that s`1 jointly depend on the realized value of d 1. and s 1 might 18 / 53

32 Alternate Representation of Less Naïve Robust Model Comparatively, the less naïve robust model was solving: where minimize x 1,s`,s 1 1 sup d 1 Pr0,2s s.t. s`1 ě 0, s 1 ě 0 0 ď x 1 ď 2, 0.5x 1 ` gpx 1, s`1, s 1, d 1q " gpx 1, s`1, s 1, d 1q : s`1 ` s 1 if s`1 ě x 1 d 1 and s 1 ě d 1 x 1 8 otherwise. 19 / 53

33 Alternate Representation of Less Naïve Robust Model Comparatively, the less naïve robust model was solving: where minimize x 1,s`,s 1 1 sup d 1 Pr0,2s s.t. s`1 ě 0, s 1 ě 0 0 ď x 1 ď 2, 0.5x 1 ` gpx 1, s`1, s 1, d 1q " gpx 1, s`1, s 1, d 1q : s`1 ` s 1 if s`1 ě x 1 d 1 and s 1 ě d 1 x 1 8 otherwise. Timing characterized by less naïve robust model s + 1:T, x 1 x T d [1] s - 1:T d [T] 19 / 53

34 Takeaway Message about Adjustable Decisions When robustifying decision models that either involve implementable decisions at different time periods (e.g. x t ) auxiliary decisions such as ps`t, s t q that are used to assess overall performance of implemented decisions one needs to carefully identify the chronology of decisions and observations and employ the adjustable robust counterpart framework introduced in (Ben-Tal et al., 2004) Accurate timing of decisions and observations in inventory problem x 1 d [1] x 2 d [2] d [T] s + 1:T, s - 1:T Implementable decisions Auxiliary decisions 20 / 53

35 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 21 / 53

36 Applying the ARO Framework to Inventory Management 22 / 53

37 Applying the ARO Framework to Inventory Management 1 Formulate a deterministic model where implementable and auxiliary decisions can be identified: Tÿ minimize `ct x t ` h t s`t ` b ts t x t,s`t,s t t 1 s.t. s`t ě 0, s t ě tÿ s`t ě y 1 ` x t 1 d t t, s t ě y 1 ` t 1 1 tÿ t ď x t ď M t. d t 1 x t t, 22 / 53

38 Applying the ARO Framework to Inventory Management 1 Formulate a deterministic model where implementable and auxiliary decisions can be identified: Tÿ minimize `ct x t ` h t s`t ` b ts t x t,s`t,s t t 1 s.t. s`t ě 0, s t ě tÿ s`t ě y 1 ` x t 1 d t t, s t ě y 1 ` 2 Identify an accurate chronology: t 1 1 tÿ t ď x t ď M t. d t 1 x t t, x 1 Ñ d r1s Ñ x 2 Ñ Ñ x T Ñ d rt s Ñ ps` 1:T, s 1:T q 22 / 53

39 Applying the ARO Framework to Inventory Management 3 Describe the adjustable robust counterpart min. x 1, tx t p qu T t 2, ts`t p q, s t p qut t 1 s.t. sup dpu Tÿ `ct x t pd rt 1s q ` h t s`t pdq ` b pdq ts t t 1 s`t pdq ě 0, s t pdq ě d P t tÿ s`t pdq ě y 1 ` x t 1pd rt1 1sq d t d P t s t pdq ě y 1 ` t 1 1 tÿ t 1 1 d t 1 x t 1pd rt1 d P t 0 ď x t pd rt 1s q ď M d P t. where Each x t : R t 1 Ñ R is adjusted to the observed demand d rt 1s Each s`t : RT Ñ R and s t : RT Ñ R are adjusted to entire d 23 / 53

40 Robust Inventory Management with Noisy Observations In some situations, the observations that are made are noisy. Noisy observations x 1 v [1] x 2 v [2] d [T] s + 1:T, x T s - 1:T where Implementable decisions Auxiliary decisions 24 / 53

41 Robust Inventory Management with Noisy Observations In some situations, the observations that are made are noisy. Noisy observations x 1 v [1] x 2 v [2] d [T] s + 1:T, x T s - 1:T where Implementable decisions Auxiliary decisions each s`t : RT Ñ R and s t : RT Ñ R are still adjusted to entire d 24 / 53

42 Robust Inventory Management with Noisy Observations In some situations, the observations that are made are noisy. Noisy observations x 1 v [1] x 2 v [2] d [T] s + 1:T, x T s - 1:T where Implementable decisions Auxiliary decisions each s`t : RT Ñ R and s t : RT Ñ R are still adjusted to entire d each x t : R t 1 Ñ R is adjusted to the observations v rts 24 / 53

43 Robust Inventory Management with Noisy Observations In some situations, the observations that are made are noisy. Noisy observations x 1 v [1] x 2 v [2] d [T] s + 1:T, x T s - 1:T where Implementable decisions Auxiliary decisions each s`t : RT Ñ R and s t : RT Ñ R are still adjusted to entire d each x t : R t 1 Ñ R is adjusted to the observations v rts each v t captures what is observed right before ordering x t e.g. v t d t ξ t with ξ t P r d t, d t s., with ř t ξ t ď Γ. 24 / 53

44 Robust Inventory Management with Noisy Observations In some situations, the observations that are made are noisy. Noisy observations x 1 v [1] x 2 v [2] d [T] s + 1:T, x T s - 1:T where Implementable decisions Auxiliary decisions each s`t : RT Ñ R and s t : RT Ñ R are still adjusted to entire d each x t : R t 1 Ñ R is adjusted to the observations v rts each v t captures what is observed right before ordering x t e.g. v t d t ξ t with ξ t P r d t, d t s., with ř t ξ t ď Γ. See (de Ruiter et al., 2014) for detailed methodology. 24 / 53

45 Robust Inventory Management with Noisy Observations min. x 1, tx t p qu T t 2, ts`t p q, s t p qut t 1 where # U 1 : s.t. sup pd,vqpu 1 c 1 x 1 ` ÿ `ct x t pv rts q ` h t s`t pdq ` b pdq ts t t s`t pdq ě 0, s t pdq ě d P t tÿ s`t pdq ě y 1 ` x t 1pv rt 1 sq d t pd, vq P U t s t pdq ě y 1 ` t 1 1 tÿ t 1 1 d t 1 x t 1pv rt 1 pd, vq P U t 0 ď x t pv rt 1s q ď M pd, vq P U t, pd, vq P U ˆ R T ˇˇˇˇˇ D ξ P R T`, v t d t ξ t, ξ t ď d t, ÿ t ξ t ď Γ + 25 / 53

46 NP-Hardness of Adjustable Robust Optimization Theorem (Ben-Tal et al., 2004): Solving a robust multi-stage linear programming model is NP-hard. 26 / 53

47 NP-Hardness of Adjustable Robust Optimization Theorem (Ben-Tal et al., 2004): Solving a robust multi-stage linear programming model is NP-hard. Sketch of proof: Consider the two-stage ARO model where x : R m Ñ R N : Nÿ minimize px i puq 1q xp q sup upr0,1s m i 1 " s.t. x i puq ě a J i,k u ` b u P r0, 1s i 1,..., k 1, 2, / 53

48 NP-Hardness of Adjustable Robust Optimization Theorem (Ben-Tal et al., 2004): Solving a robust multi-stage linear programming model is NP-hard. Sketch of proof: Consider the two-stage ARO model where x : R m Ñ R N : Nÿ minimize px i puq 1q xp q sup upr0,1s m i 1 " s.t. x i puq ě a J i,k u ` b u P r0, 1s i 1,..., k 1, 2, 3. Checking if optimal value is ě 0 can be as difficult as a 3-SAT problem with m variables and N clauses like v j1 _ v j2 _ v j3. E.g., choose pa, bq such that: x i pzq max k aj i,ku ` b i,k maxtu j1 ; u j2 ; 1 u j3 u 1 if u j1 1, u j2 1, or u j3 0, otherwise it s ă / 53

49 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 27 / 53

50 Connections With Robust Dynamic Programming Our dynamic decision problem can also be written: 28 / 53

51 Connections With Robust Dynamic Programming Our dynamic decision problem can also be written: «min c 1 x 1 ` max h 1 py 2 q` ` b 1 p y 2 q` 0ďx 1 ďm 1 d 1 PU 1 phq ` min c 2 x 2 ` max h 2 py 3 q` ` b 2 p y 3 q` `... 0ďx 2 ďm 2 d 2 PU 2 pd 1 q ı j ff ` min c T x T ` max rh T py T`1 q` ` b T p y T `1 q`s... 0ďx t ďm t d T PU T pd rt 1s q 28 / 53

52 Connections With Robust Dynamic Programming Our dynamic decision problem can also be written: «min c 1 x 1 ` max h 1 py 2 q` ` b 1 p y 2 q` 0ďx 1 ďm 1 d 1 PU 1 phq ` min c 2 x 2 ` max h 2 py 3 q` ` b 2 p y 3 q` `... 0ďx 2 ďm 2 d 2 PU 2 pd 1 q ı j ff ` min c T x T ` max rh T py T`1 q` ` b T p y T `1 q`s... 0ďx t ďm t d T PU T pd rt 1s q where: y t`1 : y t ` x t d t! ) U t pd rt 1s q : d P R : D ξ P R T t such that rd J rt 1s d ξj s J P U 28 / 53

53 Connections With Robust Dynamic Programming Our dynamic decision problem can also be written: «min c 1 x 1 ` max h 1 py 2 q` ` b 1 p y 2 q` 0ďx 1 ďm 1 d 1 PU 1 phq ` min c 2 x 2 ` max h 2 py 3 q` ` b 2 p y 3 q` `... 0ďx 2 ďm 2 d 2 PU 2 pd 1 q ı j ff ` min c T x T ` max rh T py T`1 q` ` b T p y T `1 q`s... 0ďx t ďm t d T PU T pd rt 1s q where: y t`1 : y t ` x t d t! ) U t pd rt 1s q : d P R : D ξ P R T t such that rd J rt 1s d ξj s J P U nested min-max problems proper updating rule: projection (analogous to conditioning ) 28 / 53

54 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t 29 / 53

55 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t Value function J t ps tq given by: J tps t q min c t x t` max ht py t`1 q``b t p y t`1 q``j t`1ps t`1 q ı 0ďx t ďm t d t PU t pd rt 1s q 29 / 53

56 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t Value function J t ps tq given by: J tps t q min c t x t` max ht py t`1 q``b t p y t`1 q``j t`1ps t`1 q ı 0ďx t ďm t d t PU t pd rt 1s q Observations: 1 General U Ñ high-dimensional S t Ñ curse of dimensionality 29 / 53

57 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t Value function J t ps tq given by: J tps t q min c t x t` max ht py t`1 q``b t p y t`1 q``j t`1ps t`1 q ı 0ďx t ďm t d t PU t pd rt 1s q Observations: 1 General U Ñ high-dimensional S t Ñ curse of dimensionality 2 When U has special structure, can reduce state space 29 / 53

58 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t Value function J t ps tq given by: J tps t q min c t x t` max ht py t`1 q``b t p y t`1 q``j t`1ps t`1 q ı 0ďx t ďm t d t PU t pd rt 1s q Observations: 1 General U Ñ high-dimensional S t Ñ curse of dimensionality 2 When U has special structure, can reduce state space U h-cube ˆTt 1rd t, d t s Ñ S t y t ) U budget!d : D z, }z} 8 ď 1, }z} 1 ď Γ, d t d t ` ˆd t z t Ñ S t t 1 ÿ y t, z τ T τ 1 29 / 53

59 Bellman Principle; Robust DP Recursions The state of the system at time t: S t : y t d J J J rt 1s yt d 1 d 2... d t 1 P R t Value function J t ps tq given by: J tps t q min c t x t` max ht py t`1 q``b t p y t`1 q``j t`1ps t`1 q ı 0ďx t ďm t d t PU t pd rt 1s q Observations: 1 General U Ñ high-dimensional S t Ñ curse of dimensionality 2 When U has special structure, can reduce state space Reduce computational burden Prove structural results, comparative statics x t pyq min`m t, maxp0, θ t yq (modified) base-stock policy 29 / 53

60 A More Subtle Point / 53

61 A More Subtle Point... Consider this problem: J max dprd, ds 0ďx min fpd, xq `y1 Ñ d 1 d Ñ y 2 ypdq Ñ x 2 x 30 / 53

62 A More Subtle Point... Consider this problem: J max dprd, ds 0ďx min fpd, xq DP yields Bellman-optimal policy x pdq max`0, θ ypdq 30 / 53

63 A More Subtle Point... Consider this problem: J max dprd, ds 0ďx min fpd, xq DP yields Bellman-optimal policy x pdq max`0, θ ypdq x pdq minimizes fpx, dq for any d 30 / 53

64 A More Subtle Point... Consider this problem: J max dprd, ds 0ďx min fpd, xq DP yields Bellman-optimal policy x pdq max`0, θ ypdq x pdq minimizes fpx, dq for any d x pdq sufficient for achieving J, i.e., J max dprd, ds f`d, x pdq 30 / 53

65 A More Subtle Point... Consider this problem: J max dprd, ds 0ďx min fpd, xq DP yields Bellman-optimal policy x pdq max`0, θ ypdq x pdq minimizes fpx, dq for any d x pdq sufficient for achieving J, i.e., J max dprd, ds f`d, x pdq Is x pdq necessary for achieving J? Is Bellman optimality necessary in robust dynamic problems? 30 / 53

66 Is Bellman Optimality Necessary? J max min dprd, ds 0ďx fpd, xq Is x pdq necessary for achieving J? 31 / 53

67 Is Bellman Optimality Necessary? J max min dprd, ds 0ďx fpd, xq Is x pdq necessary for achieving J? Define the set of worst-case optimal policies: X wc : x : rd, ds Ñ R` : f`d, xpdq ď d P U (. X wc non-empty : x P X wc 31 / 53

68 Is Bellman Optimality Necessary? J max min dprd, ds 0ďx fpd, xq Is x pdq necessary for achieving J? Define the set of worst-case optimal policies: X wc : x : rd, ds Ñ R` : f`d, xpdq ď d P U (. X wc non-empty : x P X wc X wc contains other policies: x aff pdq x pdq ` x p dq x pdq pd dq d d 31 / 53

69 Is Bellman Optimality Necessary? J max min dprd, ds 0ďx fpd, xq Is x pdq necessary for achieving J? Define the set of worst-case optimal policies: X wc : x : rd, ds Ñ R` : f`d, xpdq ď d P U (. X wc non-empty : x P X wc X wc contains other policies: x aff pdq x pdq ` x p dq x pdq pd dq d d In fact, X wc very degenerate: infinitely many policies (e.g., λ x ` p1 λqx aff, for any λ P r0, 1s ) 31 / 53

70 Worst-Case Optimality and Degeneracy This degeneracy is typical for robust multi-stage problems ( If adversary does not play optimally, you don t have to, either... ) 32 / 53

71 Worst-Case Optimality and Degeneracy This degeneracy is typical for robust multi-stage problems ( If adversary does not play optimally, you don t have to, either... ) Critically different from stochastic problems 32 / 53

72 Worst-Case Optimality and Degeneracy This degeneracy is typical for robust multi-stage problems ( If adversary does not play optimally, you don t have to, either... ) Critically different from stochastic problems A blessing: may allow finding policies with simple structure ( Bertsimas et al. [2010], Iancu et al. [2013] ) 32 / 53

73 Worst-Case Optimality and Degeneracy This degeneracy is typical for robust multi-stage problems ( If adversary does not play optimally, you don t have to, either... ) Critically different from stochastic problems A blessing: may allow finding policies with simple structure ( Bertsimas et al. [2010], Iancu et al. [2013] ) A curse: may yield inefficiencies in the decision process ( x Pareto-dominates x aff ) 32 / 53

74 Worst-Case Optimality and Degeneracy This degeneracy is typical for robust multi-stage problems ( If adversary does not play optimally, you don t have to, either... ) Critically different from stochastic problems A blessing: may allow finding policies with simple structure ( Bertsimas et al. [2010], Iancu et al. [2013] ) A curse: may yield inefficiencies in the decision process ( x Pareto-dominates x aff ) Worst-case optimal policies must be implemented with resolving 32 / 53

75 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 33 / 53

76 Static Policies Simplest possible policies : static (i.e., non-adjustable) Very tractable Optimal for LPs with row-wise uncertainty [Ben-Tal et al., 2009] min xpzq max cpz0 q J xpzq ` dpz 0 q z 0 PZ 0 subject to a j pz j q J xpzq ď b j pz j z j P Z j 1,..., J. Result extended recently [Bertsimas et al., 2015] Good performance under other uncertainty sets [Bertsimas and Goyal, 2010, Bertsimas et al., 2011b,...] 34 / 53

77 Affine Decision Rules Restrict attention to affine decision rules [Ben-Tal et al., 2004] 35 / 53

78 Affine Decision Rules Restrict attention to affine decision rules [Ben-Tal et al., 2004] Affinely Adjustable Robust Counterpart approximation scheme min X s.t. s`t max dpu Tÿ c t px 0 t ` X t dq ` h t ps`t ` S`t dq ` b tps t ` S t dq t 1 ` S`t d ě 0, s`t ` S`t d ě y 1 ` s t ` S t d ě y 1 s t ` S t d ě d P U tÿ px 0 τ ` X τ d d τ d P U, τ 1 tÿ px 0 τ ` X τ d d τ d P U, τ 1 0 ď x t ` X t d ď M d P U, Decision variables: coefficients X x 0 t, X t, s`t, S`t, s t, S t ( T t 1 Xt P R t 1 ˆ 0 T t`1 to ensure non-anticipativity 35 / 53

79 Affine Decision Rules Restrict attention to affine decision rules [Ben-Tal et al., 2004] Affinely Adjustable Robust Counterpart approximation scheme min X s.t. s`t max dpu Tÿ c t px 0 t ` X t dq ` h t ps`t ` S`t dq ` b tps t ` S t dq t 1 ` S`t d ě 0, s`t ` S`t d ě y 1 ` s t ` S t d ě y 1 s t ` S t d ě d P U tÿ px 0 τ ` X τ d d τ d P U, τ 1 tÿ px 0 τ ` X τ d d τ d P U, τ 1 0 ď x t ` X t d ď M d P U, Two layers of sub-optimality: policy and auxiliary variables 35 / 53

80 Affine Decision Rules Restrict attention to affine decision rules [Ben-Tal et al., 2004] Affinely Adjustable Robust Counterpart approximation scheme Tractable under fixed recourse [Ben-Tal et al., 2004] Excellent performance in a variety of applications [Ben-Tal et al., 2005, Mani et al., 2006, Adida and Perakis, 2006, Babonneau et al., 2010,...] Optimal for linear problems with simplex uncertainty [Ben-Tal et al., 2009] Optimal for our multi-period inventory model under hypercube uncertainty set U ˆTt 1 rd t, d t s [ Iancu et al., 2013, Bertsimas et al., 2010 ] 35 / 53

81 Piecewise Affine Rules One can also use piece-wise affine rules, e.g., t 1 ÿ x t pd rt 1s q : x 0 t ` X t d rt 1s ` θ` tk pd kq` ` θ tk p d kq` k 1 36 / 53

82 Piecewise Affine Rules One can also use piece-wise affine rules, e.g., t 1 ÿ x t pd rt 1s q : x 0 t ` X t d rt 1s ` θ` tk pd kq` ` θ tk p d kq` k 1 These policies are actually affine in ξ : rd, d`, d s It would suffice to have a tractable representation for: pd, ( ConvexHull d`, d q d P U, d` maxp0; dq, d maxp0; dq 36 / 53

83 Piecewise Affine Rules One can also use piece-wise affine rules, e.g., t 1 ÿ x t pd rt 1s q : x 0 t ` X t d rt 1s ` θ` tk pd kq` ` θ tk p d kq` k 1 These policies are actually affine in ξ : rd, d`, d s It would suffice to have a tractable representation for: pd, ( ConvexHull d`, d q d P U, d` maxp0; dq, d maxp0; dq Such representations known for U h-cube, U budget, and other sets based on absolutely symmetric convex functions [Ben-Tal et al., 2009] 36 / 53

84 Piecewise Affine Rules One can also use piece-wise affine rules, e.g., t 1 ÿ x t pd rt 1s q : x 0 t ` X t d rt 1s ` θ` tk pd kq` ` θ tk p d kq` k 1 These policies are actually affine in ξ : rd, d`, d s It would suffice to have a tractable representation for: pd, ( ConvexHull d`, d q d P U, d` maxp0; dq, d maxp0; dq Such representations known for U h-cube, U budget, and other sets based on absolutely symmetric convex functions [Ben-Tal et al., 2009] Large literature on other (nonlinear) rules: segregated rules [Chen and Zhang, 2009, Goh and Sim, 2010,?] piecewise constant [Bertsimas and Caramanis, 2010, Bertsimas et al., 2011b] kernelized [Chatterjee et al., 2011, Skaf and Boyd, 2009] polynomial [Ben-Tal et al., 2009, Bertsimas et al., 2011c] 36 / 53

85 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 37 / 53

86 What Is Time Consistency? Definition of time consistency: Axiomatic property that requires a decision maker s stated preferences over future courses of action to remain consistent with the actual preferred actions when planned-for contingencies arise. (Refer to Chapter 6 of (Shapiro et al., 2009) for background info.) 38 / 53

87 What Is Time Consistency? Definition of time consistency: Axiomatic property that requires a decision maker s stated preferences over future courses of action to remain consistent with the actual preferred actions when planned-for contingencies arise. (Refer to Chapter 6 of (Shapiro et al., 2009) for background info.) Intuitively, Relates to the way multi-stage decision models are employed to generate implemented decisions as the future unfolds 38 / 53

88 What Is Time Consistency? Definition of time consistency: Axiomatic property that requires a decision maker s stated preferences over future courses of action to remain consistent with the actual preferred actions when planned-for contingencies arise. (Refer to Chapter 6 of (Shapiro et al., 2009) for background info.) Intuitively, Relates to the way multi-stage decision models are employed to generate implemented decisions as the future unfolds Good news: All robust multi-stage decision models naturally express stage-wise preferences that are time consistent 38 / 53

89 What Is Time Consistency? Definition of time consistency: Axiomatic property that requires a decision maker s stated preferences over future courses of action to remain consistent with the actual preferred actions when planned-for contingencies arise. (Refer to Chapter 6 of (Shapiro et al., 2009) for background info.) Intuitively, Relates to the way multi-stage decision models are employed to generate implemented decisions as the future unfolds Good news: All robust multi-stage decision models naturally express stage-wise preferences that are time consistent Bad news: When decisions are implemented in a shrinking horizon fashion, the uncertainty set must be updated according to the proper update rule in order to give rise to this said consistency 38 / 53

90 Time Consistency in Robust Coffee-vendor Problem Robust coffee-vendor problem (Part I): A coffee stand is operated for one morning in the lobby of two hotels: hotel #1 during 7am-9am hotel #2 during 9am-11am possibility to replenish with fresh coffee at 9am 39 / 53

91 Time Consistency in Robust Coffee-vendor Problem Robust coffee-vendor problem (Part I): A coffee stand is operated for one morning in the lobby of two hotels: hotel #1 during 7am-9am possibility to replenish with fresh coffee at 9am hotel #2 during 9am-11am Given that each hotel has two (hundred) guests, the salesman estimates the following demand: one (hundred) cups in hotel #1 one (hundred) cups in hotel #2 very unlikely that more than three (hundred) cups of coffee would be needed overall 39 / 53

92 Time Consistency in Robust Coffee-vendor Problem Robust coffee-vendor problem (Part I): A coffee stand is operated for one morning in the lobby of two hotels: hotel #1 during 7am-9am possibility to replenish with fresh coffee at 9am hotel #2 during 9am-11am Given that each hotel has two (hundred) guests, the salesman estimates the following demand: one (hundred) cups in hotel #1 one (hundred) cups in hotel #2 very unlikely that more than three (hundred) cups of coffee would be needed overall This situation seems to motivate the following uncertainty set : U : td P r0, 2s 2 d 1 ` d 2 ď 3u 39 / 53

93 Time Consistency in Robust Coffee-vendor Problem Robust coffee-vendor problem (Part II): The coffee-vendor signed a contract with the two hotels to serve coffee to every thirsty guests. Here are some financial facts: contracts revenue = 600$ in total upfront production cost at 6am = 1$ per cup production cost at 9am = 4$ per cup salvage cost = 0$ per cup contracted penalty for unserved customers = 10$ per cup 40 / 53

94 Time Consistency in Robust Coffee-vendor Problem Robust coffee-vendor problem (Part II): The coffee-vendor signed a contract with the two hotels to serve coffee to every thirsty guests. Here are some financial facts: contracts revenue = 600$ in total upfront production cost at 6am = 1$ per cup production cost at 9am = 4$ per cup salvage cost = 0$ per cup contracted penalty for unserved customers = 10$ per cup This situation seems to motivate the following deterministic model: minimize x 1 ě0,x 2 ě0 x 1 ` 4x 2 ` 10pd 1 ` d 2 x 1 x 2 q` loooooooooooooooooooooomoooooooooooooooooooooon total cost (in hundreds of $) 40 / 53

95 Robust Two-stage Coffee-vendor Problem minimize x 1,x 2 p q sup dpu x 1 ` 4x 2 pd 1 q ` 10pd 1 ` d 2 x 1 x 2 pd 1 qq` s.t. x 1 ě 0, x 2 pd 1 q ě d P U, where U : td P r0, 2s 2 d 1 ` d 2 ď 3u. Conclusions that can be drawn based on optimal solution: Prepare 3 (hundred) cups at 6am Don t refresh the coffee at 9am under any circumstances The expected worst-case cost is 300$ under any circumstances This gives an anticipated worst-case profit of 600$ 300$ 300$ 41 / 53

96 Robust Two-stage Coffee-vendor Problem minimize x 1,x 2 p q sup dpu x 1 ` 4x 2 pd 1 q ` 10pd 1 ` d 2 x 1 x 2 pd 1 qq` s.t. x 1 ě 0, x 2 pd 1 q ě d P U, where U : td P r0, 2s 2 d 1 ` d 2 ď 3u. Conclusions that can be drawn based on optimal solution: Prepare 3 (hundred) cups at 6am Don t refresh the coffee at 9am under any circumstances The expected worst-case cost is 300$ under any circumstances This gives an anticipated worst-case profit of 600$ 300$ 300$ Is this truly the worst-case profit that will be achieved? 41 / 53

97 Scenario #1: Time Consistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel 42 / 53

98 Scenario #1: Time Consistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel On the way, he updates the uncertainty set using the proper update rule d 2 P td 2 P R p2, d 2 q P Uu td 2 P r0, 2s 2 ` d 2 ď 3u r0, 1s. 42 / 53

99 Scenario #1: Time Consistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel On the way, he updates the uncertainty set using the proper update rule d 2 P td 2 P R p2, d 2 q P Uu td 2 P r0, 2s 2 ` d 2 ď 3u r0, 1s. He solves the new robust optimization model minimize x 2 sup d 2 Pr0,1s s.t. x 2 ě 0, hkikj x 1 3 d 2 x hkkkkkkkkikkkkkkkkj 2 1 ď d 2 P r0, 1s 3 `4x 2 ` 10p 2 3 ` d 2 x 2 q` 42 / 53

100 Scenario #1: Time Consistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel On the way, he updates the uncertainty set using the proper update rule d 2 P td 2 P R p2, d 2 q P Uu td 2 P r0, 2s 2 ` d 2 ď 3u r0, 1s. He solves the new robust optimization model minimize x 2 sup d 2 Pr0,1s s.t. x 2 ě 0, hkikj x 1 3 d 2 x hkkkkkkkkikkkkkkkkj 2 1 ď d 2 P r0, 1s 3 `4x 2 ` 10p 2 3 ` d 2 x 2 q` This allows him to confirm that he should not refresh the coffee Worst-case cost stays 300$ Worst-case profit stays 600$ 300$ 300$ 42 / 53

101 Scenario #1: Time Consistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel On the way, he updates the uncertainty set using the proper update rule d 2 P td 2 P R p2, d 2 q P Uu td 2 P r0, 2s 2 ` d 2 ď 3u r0, 1s. He solves the new robust optimization model minimize x 2 sup d 2 Pr0,1s s.t. x 2 ě 0, hkikj x 1 3 d 2 x hkkkkkkkkikkkkkkkkj 2 1 ď d 2 P r0, 1s 3 `4x 2 ` 10p 2 3 ` d 2 x 2 q` This allows him to confirm that he should not refresh the coffee Worst-case cost stays 300$ Worst-case profit stays 600$ 300$ 300$ No surprises. 42 / 53

102 Scenario #2: Time Inconsistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel 43 / 53

103 Scenario #2: Time Inconsistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel Considering that the two hotels have an independent number of thirsty guests, he prefers to be safe by updating the uncertainty set to d 2 P r0, 2s instead of d 2 P r0, 1s 43 / 53

104 Scenario #2: Time Inconsistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel Considering that the two hotels have an independent number of thirsty guests, he prefers to be safe by updating the uncertainty set to d 2 P r0, 2s instead of d 2 P r0, 1s He now solves the new robust optimization model minimize x 2 sup d 2 Pr0,2s s.t. x 2 ě 0, 3 ` 4x 2 ` 10p d 2 x 2 hkkkkkkkkikkkkkkkkj 1 ď 1 x d 2 P r0, 2s 2 3 ` d 2 x 2 q` 43 / 53

105 Scenario #2: Time Inconsistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel Considering that the two hotels have an independent number of thirsty guests, he prefers to be safe by updating the uncertainty set to d 2 P r0, 2s instead of d 2 P r0, 1s He now solves the new robust optimization model minimize x 2 sup d 2 Pr0,2s s.t. x 2 ě 0, 3 ` 4x 2 ` 10p d 2 x 2 hkkkkkkkkikkkkkkkkj 1 ď 1 x d 2 P r0, 2s 2 3 ` d 2 x 2 q` This warns him that he should prepare an additional 1 (hundred) cups Worst-case cost is now 700$ instead of 300$ Worst-case profit is now 600$ 700$ 100$ 43 / 53

106 Scenario #2: Time Inconsistent Second-stage Preferences At 9am, let d 1 2 be observed, the coffee-vendor changes hotel Considering that the two hotels have an independent number of thirsty guests, he prefers to be safe by updating the uncertainty set to d 2 P r0, 2s instead of d 2 P r0, 1s He now solves the new robust optimization model minimize x 2 sup d 2 Pr0,2s s.t. x 2 ě 0, 3 ` 4x 2 ` 10p d 2 x 2 hkkkkkkkkikkkkkkkkj 1 ď 1 x d 2 P r0, 2s 2 3 ` d 2 x 2 q` This warns him that he should prepare an additional 1 (hundred) cups Worst-case cost is now 700$ instead of 300$ Worst-case profit is now 600$ 700$ 100$ This issue can arise any time a new confidence region is used to update the uncertainty set dynamically. 43 / 53

107 One Possible Fix Using Bi-level Modeling The following model accounts explicitly for time inconsistency minimize x 1,x 2 p q sup dpu s.t. x 1 ě 0 x 2 pd 1 q P arg min x 1 2 ě0 x 1 ` 4x 2 pd 1 q ` 10pd 1 ` d 2 x 1 x 2 pd 1 qq` max d 1 2 PŨ2pd 1 q c 2 x 1 2 ` bpd 1 ` d 1 2 x 1 x 1 d 1 P U 1, where U 1 : td 1 P R D d 2, pd 1, d 2 q P Uu Ũ 2 pd 1 q models exactly the type of uncertainty set that is used in stage #2 when d 1 is observed 44 / 53

108 One Possible Fix Using Bi-level Modeling The following model accounts explicitly for time inconsistency minimize x 1,x 2 p q sup dpu s.t. x 1 ě 0 x 2 pd 1 q P arg min x 1 2 ě0 x 1 ` 4x 2 pd 1 q ` 10pd 1 ` d 2 x 1 x 2 pd 1 qq` max d 1 2 PŨ2pd 1 q c 2 x 1 2 ` bpd 1 ` d 1 2 x 1 x 1 d 1 P U 1, where U 1 : td 1 P R D d 2, pd 1, d 2 q P Uu Ũ 2 pd 1 q models exactly the type of uncertainty set that is used in stage #2 when d 1 is observed This model suggests preparing 4 (hundred) cups at 6am, and nothing later. Under scenario #2, the worst-case profit is 200$!!! 44 / 53

109 One Possible Fix Using Bi-level Modeling The following model accounts explicitly for time inconsistency minimize x 1,x 2 p q sup dpu s.t. x 1 ě 0 x 2 pd 1 q P arg min x 1 2 ě0 x 1 ` 4x 2 pd 1 q ` 10pd 1 ` d 2 x 1 x 2 pd 1 qq` max d 1 2 PŨ2pd 1 q c 2 x 1 2 ` bpd 1 ` d 1 2 x 1 x 1 d 1 P U 1, where U 1 : td 1 P R D d 2, pd 1, d 2 q P Uu Ũ 2 pd 1 q models exactly the type of uncertainty set that is used in stage #2 when d 1 is observed This model suggests preparing 4 (hundred) cups at 6am, and nothing later. Under scenario #2, the worst-case profit is 200$!!! The question remains of how to efficiently resolve these types of models / 53

110 Takeaway Message About Time Consistency Time consistency can easily become an issue when implementing decisions obtained from robust multi-stage models 45 / 53

111 Takeaway Message About Time Consistency Time consistency can easily become an issue when implementing decisions obtained from robust multi-stage models Quality of solution of robust multi-stage decision model is contingent on the assumption that the proper update rule is followed: U t:t p d rt 1s q : tpd t,..., d T q p d 1,..., d t 1, d t,..., d T q P Uu. 45 / 53

112 Takeaway Message About Time Consistency Time consistency can easily become an issue when implementing decisions obtained from robust multi-stage models Quality of solution of robust multi-stage decision model is contingent on the assumption that the proper update rule is followed: U t:t p d rt 1s q : tpd t,..., d T q p d 1,..., d t 1, d t,..., d T q P Uu. Unclear how much can be loss when the uncertainty set is updated differently 45 / 53

113 Takeaway Message About Time Consistency Time consistency can easily become an issue when implementing decisions obtained from robust multi-stage models Quality of solution of robust multi-stage decision model is contingent on the assumption that the proper update rule is followed: U t:t p d rt 1s q : tpd t,..., d T q p d 1,..., d t 1, d t,..., d T q P Uu. Unclear how much can be loss when the uncertainty set is updated differently Should one account for inconsistency directly in the decision model? 45 / 53

114 Table of Contents 1 Introduction 2 When to Worry About Adjustable Decisions 3 The Adjustable Robust Counterpart Model 4 Connections With Robust Dynamic Programming (DP) 5 Simple Policies and Their Optimality 6 Should One Worry About Time Consistency? 7 Conclusions and Future Directions 46 / 53

115 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule 47 / 53

116 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule Correcting these issues requires skilful modeling and, in theory, significant computational resources 47 / 53

117 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule Correcting these issues requires skilful modeling and, in theory, significant computational resources From a practical point of view, 47 / 53

118 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule Correcting these issues requires skilful modeling and, in theory, significant computational resources From a practical point of view, 1 there is hope that simple policies might often be sufficient because they are worst-case optimal 47 / 53

119 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule Correcting these issues requires skilful modeling and, in theory, significant computational resources From a practical point of view, 1 there is hope that simple policies might often be sufficient because they are worst-case optimal 2 it is unclear to us whether time inconsistent updating should be banished or accounted for in the robust multi-stage model? 47 / 53

120 Key Insights About Robust Multi-stage Decision Making Two common pitfalls might mislead the decision process 1 treating adjustable decisions as static decisions 2 updating the uncertainty set differently from the proper update rule Correcting these issues requires skilful modeling and, in theory, significant computational resources From a practical point of view, 1 there is hope that simple policies might often be sufficient because they are worst-case optimal 2 it is unclear to us whether time inconsistent updating should be banished or accounted for in the robust multi-stage model? In any case, it is important to acknowledge these issues in reporting decisions that are prescribed by these models 47 / 53

121 Future Directions Full theoretical understanding of quality of simple policies Development of decomposition schemes for accelerating resolution of robust multi-stage problems Identify tractable conservative approximation methods for problems with integer or random recourse variables Advanced methods that provide the decision maker with better control of temporal correlation, level of conservatism, and time consistency 48 / 53

122 Thank you for attending! Questions? / 53

123 References I E. Adida and G. Perakis. A robust optimization approach to dynamic pricing and inventory control with no backorders. Mathematical Programming, 107:97 129, F. Babonneau, J.-P. Vial, and R. Apparigliato. Uncertainty and Environmental Decision Making, volume 138 of International Series in Operations Research & Management Science, chapter Robust Optimization for Environmental and Energy Planning, pages Springer US, O. Baron, J. Milner, and H. Naseraldin. Facility location: A robust optimization approach. Production and Operations Management, 20(5): , A. Ben-Tal and A. Nemirovski. Robust convex optimization. Mathematics of Operations Research, 23(4): , A. Ben-Tal and A. Nemirovski. Robust solutions of uncertain linear programs. Operations Research Letters, 25:1 13, A. Ben-Tal and A. Nemirovski. Robust solutions of linear programming problems contaminated with uncertain data. Mathematical Programming, Series A, 88(3): , September A. Ben-Tal and A. Nemirovski. Robust optimization - methodology and applications. Mathematical Programming, 92(3): , A. Ben-Tal, A. Nemirovski, and C. Roos. Robust solutions of uncertain quadratic and conic-quadratic problems. SIAM Journal on Optimization, 13(2): , A. Ben-Tal, A. Goryashko, E. Guslitzer, and A. Nemirovski. Adjustable robust solutions of uncertain linear programs. Mathematical Programming, 99(2): , A. Ben-Tal, B. Golany, A. Nemirovski, and J.-P. Vial. Retailer-supplier flexible commitments contracts: A robust optimization approach. Manufacturing & Service Operations Management, 7(3): , A. Ben-Tal, L. El-Ghaoui, and A. Nemirovski. Robust Optimization. Princeton Series in Applied Mathematics. Princeton University Press, D. Bertsimas and C. Caramanis. Finite adaptability in multistage linear optimization. Automatic Control, IEEE Transactions on, 55(12): , dec ISSN D. Bertsimas and V. Goyal. On the Power of Robust Solutions in Two-Stage Stochastic and Adaptive Optimization Problems. Mathematics of Operations Research, 35(2): , / 53

124 References II D. Bertsimas and D. Pachamanova. Robust multiperiod portfolio management in the presence of transaction costs. Computers and Operations Research, 35(1):3 17, D. Bertsimas and M. Sim. Robust discrete optimization and network flows. Mathematical Programming, 98(1-3):49 71, D. Bertsimas and M. Sim. The price of robustness. Operations Research, 52(1):35 53, D. Bertsimas and A. Thiele. A robust optimization approach to inventory theory. Operations Research, 54(1): , D. Bertsimas, D. Pachamanova, and M. Sim. Robust linear optimization under general norms. Operations Research Letters, 32 (6): , D. Bertsimas, D. Iancu, and P. Parrilo. Optimality of affine policies in multistage robust optimization. Mathematics of Operations Research, 35(2): , D. Bertsimas, C. Caramanis, and D. Brown. Theory and applications of robust optimization. SIAM Review, 53(3): , 2011a. D. Bertsimas, V. Goyal, and X. Sun. A geometric characterization of the power of finite adaptability in multistage stochastic and adaptive optimization. Mathematics of Operations Research, 36(1):24 54, 2011b. D. Bertsimas, D. Iancu, and P. Parrilo. A hierarchy of near-optimal policies for multistage adaptive optimization. IEEE Transactions on Automatic Control, 56(12): , 2011c. D. Bertsimas, V. Goyal, and B. Lu. A tight characterization of the performance of static solutions in two-stage adjustable robust linear optimization. Mathematical Programming, 150(2): , D. Bienstock and N. Özbay. Computing robust basestock levels. Discrete Optimization, 5(2): , S. Ceria and R. Stubbs. Incorporating estimation errors into portfolio selection: Robust portfolio construction. Journal of Asset Management, 7(2): , July D. Chatterjee, P. Hokayem, and J. Lygeros. Stochastic model predictive control with bounded control inputs: a vector space approach. IEEE Transactions on Automatic Control, 56(11): , X. Chen and Y. Zhang. Uncertain Linear Programs: Extended Affinely Adjustable Robust Counterparts. Operations Research, 57(6): , November/December / 53