Distributionally Robust Convex Optimization Wolfram Wiesemann 1, Daniel Kuhn 1, and Melvyn Sim 2 1 Department of Computing, Imperial College London, United Kingdom 2 Department of Decision Sciences, National University of Singapore, Singapore
Expectation Constraints E Q [ (, )]
Stochastic Programs Expectation Constraints E Q [ (, )] inf E Q [ (, )] X inf X E Q [ (, [v(x, )] z) ] 0
Expectation Constraints Chance Constraints Stochastic Programs EQ [ (, )] inf EQ [ (, )] X Q [ (, ) ] inf X EEQQ [[v(x, ((,, )z ) )] ] 0
Expectation Constraints inf EQ [ (, )] X Q [ (, ) ] Q [ (, )] CVaR Constraints Chance Constraints Stochastic Programs EQ [ (, )] inf X EEQQ [[v(x, ((R,, )z ) )] ] + EQ [( (, ) 0 )+ ]
Expectation Constraints inf X Decision Criteria inf EQ [ (, )] X Q [ (, ) ] Q [ (, )] Utility Maximization CVaR Constraints Chance Constraints Stochastic Programs EQ [ (, )] Optimized Certainty Equivalents EEQQ [[v(x, ((R,, )z ) )] ] 0 Satisficing Measures + EQ [( (, ) )+ ] etc.
Distribtionally Robust E.C.s sup P P E P [ (, )]
Data Availability Distribtionally Robust E.C.s sup P P E P [ (, )] Exact distribution is unknown Information about support, symmetry properties, generalized moments etc. may be available
Data Availability Estimation Errors Distribtionally Robust E.C.s sup P P E P [ (, )] Exact distribution is unknown Information about Results overfitted to estimated nominal support, distribution symmetry properties, generalized Biased results with moments etc. may be poor out-of-sample available performance
Data Availability Estimation Errors Decision Theory Distribtionally Robust E.C.s sup P P E P [ (, )] Exact distribution is unknown Information about Results overfitted to estimated nominal Decision-makers are support, distribution symmetry ambiguity-averse properties, generalized Biased moments results with Worst-case etc. may be approach poor out-of-sample available has strong theoretical performance justification
Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I }
Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } u z
Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } u + = z (E P [ ], E P [ ])
Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } u + = z (E P [ ], E P [ ]) = { (, ) R R : + K }
Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } u + = z (E P [ ], E P [ ]) = { (, ) R R : + K }
Expressiveness of the Ambiguity Set Mean Value Information: P = P P (R ): E P [ ] K
Expressiveness of the Ambiguity Set Mean Value Information: P = P P (R ): E P [ ] K P = P P (R R ):E P [ ] =, P [ K ] =
Expressiveness of the Ambiguity Set Mean Value Information: P = P P (R ): E P [ ] K P = P P (R R ):E P [ ] =, P [ K ] = µ K µ
Expressiveness of the Ambiguity Set Mean Value Information: P = P P (R ): E P [ ] K P = P P (R R ):E P [ ] =, P [ K ] = µ Prop: P = P K µ
Expressiveness of the Ambiguity Set Variance/Covariance Information: P = P (R ):E P [ µ][ µ]
Expressiveness of the Ambiguity Set Variance/Covariance Information: P = P (R ):E P [ µ][ µ] P = P (R R ):E P =, P ( µ) ( µ) =
Expressiveness of the Ambiguity Set Variance/Covariance Information: P = P (R ):E P [ µ][ µ] P = P (R R ):E P =, P ( µ) ( µ) = Prop: P = P
Expressiveness of the Ambiguity Set Univariate Moments: P = P (R ) : EP ( [ µ]) Even Moments ( µ)
Expressiveness of the Ambiguity Set Univariate Moments: P = P P= P where (R R++ = log = log + (R ) : EP ( ) : EP [ ] =,, M = {,..., }, + +, P = = log, [ µ]) log + + {,..., } M + + ( µ), + =
Expressiveness of the Ambiguity Set Univariate Moments: P = P P= P where (R R++ = log = log + (R ) : EP ( ) : EP [ ] =,, M = {,..., }, + +, P = = log, Prop: P = [ µ]) log + + {,..., } M + + ( P µ), + =
Expressiveness of the Ambiguity Set Univariate Partial Moments: P = P (R ) : EP ( [ µ])+ Even & Odd Moments ( µ)
Expressiveness of the Ambiguity Set Univariate Partial Moments: P = P P= P where (R + + R+ = log = log (R ) : EP ( ) : EP [ ] =,, M = {,..., }, + +, P = = log, [ µ])+ log + + {,..., } M + + [ ( µ)]+, + =
Expressiveness of the Ambiguity Set Univariate Partial Moments: P = P P= P where (R + + R+ = log = log (R ) : EP ( ) : EP [ ] =,, M = {,..., }, + +, P = = log, Prop: P = [ µ])+ log + + {,..., } M + + [ ( P µ)]+, + =
Expressiveness of the Ambiguity Set Marginal Median: P = P (R ):P [ ], P [ ]
Expressiveness of the Ambiguity Set Marginal Median: P = P (R ):P [ ], P [ ] Mean-Absolute Deviation: P = P (R ):E P [ ]
Expressiveness of the Ambiguity Set Marginal Median: P = P (R ):P [ ], P [ ] Mean-Absolute Deviation: P = P (R ):E P [ ] P = P P (R R ):E P [ ] =, P [, ] =
Expressiveness of the Ambiguity Set Marginal Median: P = P (R ):P [ ], P [ ] Mean-Absolute Deviation: P = P (R ):E P [ ] P = P P (R R ):E P [ ] =, P [, ] = Prop: P = P
Expressiveness of the Ambiguity Set Huber Loss Function: P = P (R ):E P ( ) ( ) =,
Expressiveness of the Ambiguity Set Huber Loss Function: P = P (R ):E P ( ) P = P P (R R +): E P + +, + P, P [ ]= =
Expressiveness of the Ambiguity Set Huber Loss Function: P = P (R ):E P ( ) P = P P (R R +): E P + +, + P, P [ ]= = Prop: P P
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I }
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : =
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u z
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u z
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u u z z
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u u z z
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u u z z
Tractability Ambiguity Set P = { P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } Nesting Condition, I, = : = u u z z
Tractability sup P P E P [ (, )] (DRC) Theorem: If (N) holds and v(x,z) is convex in z, then (DRC) is equivalent to R,, R + : + I [ + ] + [ ] (, ) (, ), I. S( )
Proof of Tractability Theorem sup P P E P [ (, )]
Proof of Tractability Theorem sup P P E P [ (, )] sup (, )dµ(, ) µ M + (R R ) [ + ] dµ(, ) = [(, ) C ] dµ(, ) [(, ) C ] dµ(, ) I
Proof of Tractability Theorem sup P P E P [ (, )] inf + I R,, R + [ + ] + I [(, ) C ] [ ] (, ) (, )
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + I [(, ) C ] [ ] (, ) (, )
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + I [(, ) C ] [ ] (, ) (, ) Discontinuous in (z,u)
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + I [(, ) C ] [ ] (, ) (, ) Discontinuous in (z,u) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + I [(, ) C ] [ ] (, ) (, ) Discontinuous in (z,u) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + I [(, ) C ] [ ] (, ) (, ) Discontinuous in (z,u) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [(, ) C ] [ ] (, ) (, ) I Discontinuous in (z,u) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [(, ) C ] [ ] (, ) (, ) I I u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [ ] (, ) (, ), I S( ) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [ ] (, ) (, ), I S( ) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [ ] (, ) (, ) ( ), I S( ) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [ ] (, ) (, ), I S( ) u z
Proof of Tractability Theorem Semi-Infinite Constraint [ + ] + [ ] (, ) (, ), I S( ) q.e.d. u z
Constraint Functions sup P P E P [ (, )]
Constraint Functions sup P P E P [ (, )] Bi-Affine Functions (, ) = ( ) + ( ) ( ) ( ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Constraint Functions sup P P E P [ (, )] Quadratic-Affine Functions (, ) = ( ) + + ( ) + ( ) ( ) ( ) ( ) ( ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Constraint Functions sup P P E P [ (, )] Affine-Quadratic Functions (, ) = ( ) + + ( ) + ( ) ( ) ( ) ( ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Constraint Functions sup P P E P [ (, )] Bi-Quadratic Functions (, ) = ( ) ( ) + + ( ) + ( ) ( ) ( ) ( ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Constraint Functions sup P P E P [ (, )] Conic-Quadratic Functions (, ) = ( ) + ( ) ( ) ( ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Constraint Functions sup P P E P [ (, )] Maxima of Tractable Functions (, ) = max L (, ) (, ) z See A. Ben-Tal, A. Nemirovski (MOR, 1998). x
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I }
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, }
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } 0 1 0 1
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } 0 1 0 1
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } 0 1 P(C 1 )=1 0 1
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } P(C 2 )=0 0 1 P(C 1 )=1 0 1
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } P(C 1 \C 2 )=1 P(C 2 )=0 0 1 P(C 1 )=1 0 1
Intractability Ambiguity Set that Violates Condition (N): { P = P P (R R ) : E P [ + ] = P [(, ) ] [, ] I } P = P P (R ) : P [ ]= {, } P(C 1 \C 2 )=1 P(C 2 )=0 0 1 P(C 1 )=1 0 1 Theorem: Verifying if P is empty is strongly NP-hard even if.
Generic Ambiguity Sets I J I {} I J
Generic Ambiguity Sets I J I {} I J P = P P (R R ) : E P [ + ] = P [(, ) ], I, J
Generic Ambiguity Sets I J I {} I J P = P P (R R ) : E P [ + ] = P [(, ) ], I, J I = {, } I = {, } u z
Conservative Approximations Distributionally Robust Constraint: sup P P E P [ (, )] (DRC)
Conservative Approximations Distributionally Robust Constraint: sup P P E P [ (, )] (DRC) Naive Bound: min J sup P P E P [ (, )] (NB)
Conservative Approximations Distributionally Robust Constraint: sup P P E P [ (, )] (DRC) Naive Bound: min J sup P P E P [ (, )] (NB) Infimal Convolution Bound: J inf (, ) ( ) sup P P E P ( /, ) (ICB) where ( ) = (, ): =, =, > J J
Conservative Approximations Theorem: If (N ) holds and v(x,z) is convex in z, then = =
Conservative Approximations Theorem: If (N ) holds and v(x,z) is convex in z, then = = Theorem: If (N ) holds and v(x,z) is convex in z, then (ICB) is computationally tractable.
Conservative Approximations Theorem: If (N ) holds and v(x,z) is convex in z, then = = Theorem: If (N ) holds and v(x,z) is convex in z, then (ICB) is computationally tractable. Theorem: (NB) is strongly NP-hard even if (N ) holds and v(x,z) is convex in z.
Extensions Utility Maximization Theorem: If X is polyhedral and U is concave piecewise affine, then can be solved efficiently. inf E P [ ( (, ))] P P,
Extensions Optimized Certainty Equivalents Theorem: If X is polyhedral and U is concave piecewise affine, then sup inf R P P, + E P [ ( (, ) )] can be solved efficiently.
Extensions Chance Constraints Theorem: If v(x,z) is bi-affine in x and z and confidence set, then P includes only one sup P P P [ (, ) ] has a tractable conic representation for any [, ).
Extensions Satisficing Measures Theorem: If X is polyhedral and { } piecewise affine functions nonincreasing in, then can be solved efficiently., R inf E P [ ( (, ))], P P R is a family of concave
Bibliography [1] [2] [3] [4] [5] [6] [7] A. Ben-Tal, A. Nemirovski. Robust convex optimization. Mathematics of Operations Research 23(4), 769-805,1998 D. Bertsimas, I. Popescu. Optimal inequalities in probability theory: A convex optimization approach. SIAM Journal on Optimization 15(3), 780-804, 2004 D. Bertsimas, X. V. Doan, K. Natarajan, C.-P. Teo. Models for Minimax Stochastic Linear Optimization Problems with Risk Aversion. Mathematics of Operations Research 35(3), 580-602, 2010. E. Delage, Y. Ye. Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations Research 58(3), 596-612, 2010 J. Dupačová (as Žáčková). On minimax solutions of stochastic linear programming problems. Časopis pro pěstování matematiky 91(4), 423-430, 1966 A. Shapiro, A. Kleywegt. Minimax analysis of stochastic problems. Optimization Methods and Software 17, 523-542, 2002 H. Xu, S. Mannor. Distributionally Robust Markov Decision Processes. Mathematics of Operations Research 37(2), 288-300, 2012. Wolfram Wiesemann www.doc.ic.ac.uk/~wwiesema