Distributionally Robust Stochastic Optimization with Wasserstein Distance

Transcription

1 Distributionally Robust Stochastic Optimization with Wasserstein Distance Rui Gao DOS Seminar, Oct 2016 Joint work with Anton Kleywegt School of Industrial and Systems Engineering Georgia Tech

2 What is distributionally robust stochastic optimization? Stochastic optimization min E µ[ψ(x, ξ)] x X 2 / 22

3 What is distributionally robust stochastic optimization? Stochastic optimization min E µ[ψ(x, ξ)] x X Assumption: full knowledge of µ 2 / 22

4 What is distributionally robust stochastic optimization? Stochastic optimization min E µ[ψ(x, ξ)] x X Assumption: full knowledge of µ What if µ is not known exactly? Contaminated/noisy data Sample average approximation (SAA) 1 N N i=1 Ψ(x, ξ i ) Optimizer s curse in decision analysis Generalization error in statistical learning 2 / 22

5 What is distributionally robust stochastic optimization? Stochastic optimization min E µ[ψ(x, ξ)] x X Assumption: full knowledge of µ What if µ is not known exactly? Contaminated/noisy data Sample average approximation (SAA) 1 N N i=1 Ψ(x, ξ i ) Optimizer s curse in decision analysis Generalization error in statistical learning What if there is no true underlying distribution µ? Knightian uncertainty in economics Risk vs. ambiguity 2 / 22

6 What is distributionally robust stochastic optimization? Distributionally robust stochastic optimization (DRSO) min sup E µ [Ψ(x, ξ)] x X µ M Minimax approach Traced back at least to Žáčková [1966], Dupačová [1987] 3 / 22

7 What is distributionally robust stochastic optimization? Distributionally robust stochastic optimization (DRSO) min sup E µ [Ψ(x, ξ)] x X µ M Minimax approach Traced back at least to Žáčková [1966], Dupačová [1987] Q: How to specify M? 3 / 22

8 What is distributionally robust stochastic optimization? Distributionally robust stochastic optimization (DRSO) min sup E µ [Ψ(x, ξ)] x X µ M Minimax approach Traced back at least to Žáčková [1966], Dupačová [1987] Q: How to specify M? 1. Practical meaning 2. Tractability 3 / 22

9 Outline Literature review Moment-based ambiguity set Statistical-distance-based ambiguity set Main results Potential issues of φ-divergence Strong duality reformulation Structure of the worst-case distribution Applications Approximate DRSO by robust optimization On/off stochastic process control: when SAA fails Proof sketch 4 / 22

10 Moment-based ambiguity set Fixed mean and variance: Scarf [1958] Problem of moments: Dupa cová [1987], Prékopa [1995], Bertsimas & Popescu [2005] Second order moment uncertainty: Delage & Ye [2011] M = { µ P : (E ν [ξ] m 0 ) Σ 1 0 (E ν[ξ] m 0 ) γ 1 E ν [(ξ m 0 )(ξ m 0 ) ] γ 2 Σ 0 } 5 / 22

11 Moment-based ambiguity set Fixed mean and variance: Scarf [1958] Problem of moments: Dupa cová [1987], Prékopa [1995], Bertsimas & Popescu [2005] Second order moment uncertainty: Delage & Ye [2011] M = { µ P : (E ν [ξ] m 0 ) Σ 1 0 (E ν[ξ] m 0 ) γ 1 E ν [(ξ m 0 )(ξ m 0 ) ] γ 2 Σ 0 } Pros: Tractability: CQP/SDP reformulation 5 / 22

12 Moment-based ambiguity set Fixed mean and variance: Scarf [1958] Problem of moments: Dupa cová [1987], Prékopa [1995], Bertsimas & Popescu [2005] Second order moment uncertainty: Delage & Ye [2011] M = { µ P : (E ν [ξ] m 0 ) Σ 1 0 (E ν[ξ] m 0 ) γ 1 E ν [(ξ m 0 )(ξ m 0 ) ] γ 2 Σ 0 } Pros: Tractability: CQP/SDP reformulation Cons: Not fully use the data, sometimes yields overly conservative worst-case distributions 5 / 22

13 Statistical-distance-based ambiguity set Given a nominal distribution ν, e.g., empirical distribution, M = {µ P : StatDistance(µ, ν) θ} 6 / 22

14 Statistical-distance-based ambiguity set Given a nominal distribution ν, e.g., empirical distribution, M = {µ P : StatDistance(µ, ν) θ} φ-divergence: Ben-Tal et al. [2013], Jiang & Guan [2012] 6 / 22

15 Statistical-distance-based ambiguity set Given a nominal distribution ν, e.g., empirical distribution, M = {µ P : StatDistance(µ, ν) θ} φ-divergence: Ben-Tal et al. [2013], Jiang & Guan [2012] Prokhorov metric: Erdogan & Iyengar [2006] 6 / 22

16 Statistical-distance-based ambiguity set Given a nominal distribution ν, e.g., empirical distribution, M = {µ P : StatDistance(µ, ν) θ} φ-divergence: Ben-Tal et al. [2013], Jiang & Guan [2012] Prokhorov metric: Erdogan & Iyengar [2006] Wasserstein metric (of order 1): Esfahani & Kuhn [2015], Zhao & Guan [2015] 6 / 22

17 φ-divergence ambiguity set Ξ = { ξ 0, ξ 1,..., ξ B }, p, q B+1 I φ (p, q) := B q i φ i=0 Kullback-Leibler Burg entropy χ 2 -distance Hellinger Total Variation 1 φ(t) t log t log t (t t 1)2 ( t 1) 2 t 1 I φ pi log( p i q i ) qi log( q i p i ) (pi q i ) 2 p i ( pi q i) 2 pi q i ( pi q i ) 7 / 22

18 φ-divergence ambiguity set Ξ = { ξ 0, ξ 1,..., ξ B }, p, q B+1 I φ (p, q) := B q i φ i=0 Kullback-Leibler Burg entropy χ 2 -distance Hellinger Total Variation 1 φ(t) t log t log t (t t 1)2 ( t 1) 2 t 1 I φ pi log( p i q i ) qi log( q i p i ) (pi q i ) 2 p i ( pi q i) 2 pi q i Pros: ( pi Statistical meaningful: as a confidence region Tractability: LP/ CQP/ SDP reformulation q i ) 7 / 22

19 φ-divergence ambiguity set Ξ = { ξ 0, ξ 1,..., ξ B }, p, q B+1 I φ (p, q) := B q i φ i=0 Kullback-Leibler Burg entropy χ 2 -distance Hellinger Total Variation 1 φ(t) t log t log t (t t 1)2 ( t 1) 2 t 1 I φ pi log( p i q i ) qi log( q i p i ) (pi q i ) 2 p i ( pi q i) 2 pi q i Pros: ( pi Statistical meaningful: as a confidence region Tractability: LP/ CQP/ SDP reformulation Cons: Unrealistic worst-case distribution q i ) 7 / 22

20 φ-divergence yields unrealistic worst-case distribution Example I. Newsvendor problem min sup E µ [h(x ξ) + + b(ξ x) + ] x 0 µ M where Ξ = {0, 1,..., 100}, h = b = Empirical Empirical 0.1 Relative frequency Relative frequency Random demand Binomial Random demand Truncated Geometric 8 / 22

21 φ-divergence yields unrealistic worst-case distribution Example I. Newsvendor problem min sup E µ [h(x ξ) + + b(ξ x) + ] x 0 µ M where Ξ = {0, 1,..., 100}, h = b = Burg entropy Empirical 0.12 Burg entropy Empirical Relative frequency Relative frequency Random demand Binomial Random demand Truncated Geometric 8 / 22

22 φ-divergence yields unrealistic worst-case distribution Example I. Newsvendor problem min sup E µ [h(x ξ) + + b(ξ x) + ] x 0 µ M where Ξ = {0, 1,..., 100}, h = b = Wasserstein Burg entropy Empirical 0.12 Wasserstein Burg entropy Empirical Relative frequency Relative frequency Random demand Binomial Random demand Truncated Geometric 8 / 22

23 φ-divergence yields unrealistic worst-case distribution Example II. Image Retrieval 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale 8-bit Gray-scale 0 8-bit Gray-scale ν µ true µ pathol 9 / 22

24 φ-divergence yields unrealistic worst-case distribution Example II. Image Retrieval 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) 9 / 22

25 φ-divergence yields unrealistic worst-case distribution Example II. Image Retrieval 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) But I φkl (ν, µ true ) = 5.05 > I φkl (ν, µ pathol ) = / 22

26 φ-divergence yields unrealistic worst-case distribution Example II. Image Retrieval 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) But I φkl (ν, µ true ) = 5.05 > I φkl (ν, µ pathol ) = 2.33 I φ (µ, ν) = ( ) i q pi iφ q ignores the metric structure in Ξ i 9 / 22

27 Wasserstein distance Metric space (Ξ, d), p 1 { } Wp p (µ, ν) := min d p (ξ, ζ)γ(dξ, dζ) : γ has marginals µ, ν γ Ξ Ξ 10 / 22

28 Wasserstein distance Metric space (Ξ, d), p 1 { } Wp p (µ, ν) := min d p (ξ, ζ)γ(dξ, dζ) : γ has marginals µ, ν γ Ξ Ξ Example (Transportation problem) When ν = N i=1 q iδ ξi, µ = M j=1 p jδ ζj, Wp p (µ, ν) { = min γ ij d p ( ξ i, ζ j ) : γ ij 0 j i,j γ ij = q i, i, i } γ ij = p j, j. 10 / 22

29 Revisiting image retrieval example 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) 11 / 22

30 Revisiting image retrieval example 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) W 1 (ν, µ true ) = < W 1 (ν, µ pathol ) = / 22

31 Revisiting image retrieval example 10 4 Observation True 10 4 Pathological Frequence 1 Frequence Frequence bit Gray-scale ν 8-bit Gray-scale 0 8-bit Gray-scale µ true µ pathol Perceptually, StatDistance(ν, µ true ) < StatDistance(ν, µ pathol ) W 1 (ν, µ true ) = < W 1 (ν, µ pathol ) = sup E µ [Ψ(x, ξ)], where M := {µ P : W p (µ, ν) θ} µ M 11 / 22

32 Main results 12 / 22

33 Strong duality reformulation { sup Ψ(x, ξ)µ(dξ) = inf λθ p µ M Ξ λ 0 inf Ξ ξ Ξ [ λd p (ξ, ζ) Ψ(x, ξ) ] } ν(dζ) 13 / 22

34 Strong duality reformulation { sup Ψ(x, ξ)µ(dξ) = inf λθ p µ M Ξ λ 0 inf Ξ ξ Ξ [ λd p (ξ, ζ) Ψ(x, ξ) ] } ν(dζ) Moreau-Yosida regularization: inf ξ Ξ [ λd p (ξ, ζ) Ψ(x, ξ) ] 13 / 22

35 Strong duality reformulation { sup Ψ(x, ξ)µ(dξ) = inf λθ p µ M Ξ λ 0 inf Ξ ξ Ξ [ λd p (ξ, ζ) Ψ(x, ξ) ] } ν(dζ) Moreau-Yosida regularization: inf ξ Ξ [ λd p (ξ, ζ) Ψ(x, ξ) ] Esfahani & Kuhn Zhao & Guan Ours Application [2015] [2015] Ξ R n, convex + compact Polish space process control Ψ piecewise concave continuous u.s.c. ν empirical empirical any worst VaR W p p = 1 p = 1 p 1 existence Existence?? iff Proof Big hammer requires µ a.c. constructive robust TSP (Carlsson [2015]) 13 / 22

36 Structure of worst-case distribution Data-driven DRSO ν = 1 N N i=1 δ ξi µ = 1 N δ ξ i + p 0 N δ ξ i p 0 N δ ξ i 0 i i 0 where ξ i 0, ξ i 0 arg min{λ d p (ξ, ξ i 0) Ψ(ξ)}, ξ Ξ ξ i arg min{λ d p (ξ, ξ i ) Ψ(ξ)}, i i 0. ξ Ξ 0 Perturb: ξi ξ, i i i 0 Split: ξi ξ i0, w.p. p 0 ξ i 0, w.p. 1 p 0 14 / 22

37 Illustration: uncertainty quantification Ψ(x, ξ) = 1 C (ξ), where C is an open set in R n min µ(c) µ M arg min ξ Ξ {λ d p (ξ, ξ i ) Ψ(ξ)} = { ξ i } arg min ξ C d p (ξ, ξ i ) 15 / 22

38 Illustration: uncertainty quantification Ψ(x, ξ) = 1 C (ξ), where C is an open set in R n min µ(c) µ M arg min ξ Ξ {λ d p (ξ, ξ i ) Ψ(ξ)} = { ξ i } arg min ξ C d p (ξ, ξ i ) 15 / 22

39 Illustration: uncertainty quantification Ψ(x, ξ) = 1 C (ξ), where C is an open set in R n min µ(c) µ M arg min ξ Ξ {λ d p (ξ, ξ i ) Ψ(ξ)} = { ξ i } arg min ξ C d p (ξ, ξ i ) 15 / 22

40 Applications 16 / 22

41 Approximate DRSO by robust optimization Recall µ = 1 N Primal objective max ξ i,i i 0, ξ i 0,ξ i 0 i i δ 0 ξ i + p 0 N δ i ξ p 0 N δ ξ i 0 1 N i i 0 Ψ(x, ξ i ) + p 0 N Ψ(x, ξi 0) + 1 p 0 N Ψ(ξi0 ) 17 / 22

42 Approximate DRSO by robust optimization Recall µ = 1 N Primal objective max ξ i,i i 0, ξ i 0,ξ i 0 i i δ 0 ξ i + p 0 N δ i ξ p 0 N δ ξ i 0 1 N i i 0 Ψ(x, ξ i ) + p 0 N Ψ(x, ξi 0) + 1 p 0 N Ψ(ξi0 ) Q: i 0, p 0 unknown. Which to split? How much to split? 17 / 22

43 Approximate DRSO by robust optimization Recall µ = 1 N Primal objective max ξ i,i i 0, ξ i 0,ξ i 0 i i δ 0 ξ i + p 0 N δ i ξ p 0 N δ ξ i 0 1 N i i 0 Ψ(x, ξ i ) + p 0 N Ψ(x, ξi 0) + 1 p 0 N Ψ(ξi0 ) Q: i 0, p 0 unknown. Which to split? How much to split? A: Discretize! i: ξi ξ i1, w.p. 1 KN w.p. 1 KN ξ ik, w.p. 1 KN 17 / 22

44 Approximate DRSO by robust optimization v K (x) := sup (ξ ik ) i,k { 1 NK Ψ(x, ξ ik ) : i,k 1 NK i,k ξ, i i i 0 ξ ik = ξ i 0, i = i 0, 1 k Kp ξ i 0 i = i 0, Kp k N, v K (x) sup µ M E µ [Ψ(x, ξ)] v K (x) + O( 1 NK ) } d p (ξ ik, ξ i ) θ p, i, k 18 / 22

45 Approximate DRSO by robust optimization v K (x) := sup (ξ ik ) i,k { 1 NK Ψ(x, ξ ik ) : i,k 1 NK i,k ξ, i i i 0 ξ ik = ξ i 0, i = i 0, 1 k Kp ξ i 0 i = i 0, Kp k N, v K (x) sup µ M E µ [Ψ(x, ξ)] v K (x) + O( 1 NK ) Exact when Ψ is concave in ξ { 1 min max x X {ξ i } N N Ψ(x, ξ i ) : i=1 1 N } d p (ξ ik, ξ i ) θ p, i, k N } d p (ξ i, ξ i ) θ p i=1 18 / 22

46 Approximate DRSO by robust optimization v K (x) := sup (ξ ik ) i,k { 1 NK Ψ(x, ξ ik ) : i,k 1 NK i,k ξ, i i i 0 ξ ik = ξ i 0, i = i 0, 1 k Kp ξ i 0 i = i 0, Kp k N, v K (x) sup µ M E µ [Ψ(x, ξ)] v K (x) + O( 1 NK ) Exact when Ψ is concave in ξ { 1 min max x X {ξ i } N N Ψ(x, ξ i ) : i=1 1 N } d p (ξ ik, ξ i ) θ p, i, k N } d p (ξ i, ξ i ) θ p Two-stage DRSO has AARC (Affinely Adjustable Robust Counterpart) approximation i=1 18 / 22

47 On/off process control: when SAA and φ-divergence fail Two-state (on/off) system, $c cost when on, $0 when off Exogenous arrival process, $1 profit per arrival when on Maximize total revenue Sample paths t 19 / 22

48 On/off process control: when SAA and φ-divergence fail Two-state (on/off) system, $c cost when on, $0 when off Exogenous arrival process, $1 profit per arrival when on Maximize total revenue Sample paths t SAA and φ-divergence yields a degenerate optimal solution 19 / 22

49 On/off process control: when SAA and φ-divergence fail Two-state (on/off) system, $c cost when on, $0 when off Exogenous arrival process, $1 profit per arrival when on Maximize total revenue Sample paths t SAA and φ-divergence yields a degenerate optimal solution Optimal control: union of intervals 19 / 22

50 Proof sketch 20 / 22

51 Proof sketch of strong duality Dual objective h(λ) := λθ p Key observations Growth rate condition Ξ [ inf λd p (ξ, ζ) Ψ(x, ξ) ] ν(dζ) ξ Ξ Ψ(ξ) Ψ(ζ) κ := lim sup <. d(ξ,ζ) d p (ξ, ζ) One-dimensional convex minimization in λ { } h(λ) = conv θ p d p (ξ (ζ), ζ)ν(dζ) : ξ (ζ) arg min[λ d p (ξ, ζ) Ψ(ξ)] ξ Ξ Ξ 21 / 22

52 Proof sketch of strong duality Dual objective h(λ) := λθ p Ξ [ inf λd p (ξ, ζ) Ψ(x, ξ) ] ν(dζ) ξ Ξ Easy steps 1. Weak duality. sup Ψ(x, ξ)µ(dξ) inf h(λ) µ M Ξ λ 0 2. Existence of a dual minimizer λ. inf h(λ) = min h(λ). λ 0 κ λ M 21 / 22

53 Proof sketch of strong duality Dual objective h(λ) := λθ p Ξ [ inf λd p (ξ, ζ) Ψ(x, ξ) ] ν(dζ) ξ Ξ Less easy steps 3. First-order optimality (KKT) condition at λ. { } θ p conv d p (ξ (ζ), ζ)ν(dζ) : ξ (ζ) arg min[λ d p (ξ, ζ) Ψ(ξ)] ξ Ξ Ξ s.t. θ θ 0, (λ κ)(θ 0 θ ) = Construct an primal optimal solution µ. {γζ } ζ (conditional distribution of ξ µ given ζ = ζ) s.t. θ = d p (ξ, ζ)γζ (dξ)ν(dζ) Ξ Ξ 21 / 22

54 Proof sketch of strong duality Dual objective h(λ) := λθ p Non-trivial steps Ξ [ inf λd p (ξ, ζ) Ψ(x, ξ) ] ν(dζ) ξ Ξ Non-compactness of Ξ: arg min does not exist Borel measurability of γ ζ 21 / 22

55 Summary 1. Wasserstein distance captures the metric structure in Ξ, and yields a more reasonable worst-case distribution. 22 / 22

56 Summary 1. Wasserstein distance captures the metric structure in Ξ, and yields a more reasonable worst-case distribution. 2. Using a constructive approach, we prove strong duality in a general setting, which is then applied to process control problem, for which SAA fails to provide a meaningful solution. 22 / 22

57 Summary 1. Wasserstein distance captures the metric structure in Ξ, and yields a more reasonable worst-case distribution. 2. Using a constructive approach, we prove strong duality in a general setting, which is then applied to process control problem, for which SAA fails to provide a meaningful solution. 3. The worst-case distribution can be viewed as a perturbation of the nominal distribution, by which we establish a close connection between DRSO and robust optimization, which further results in computational benefits. 22 / 22