Robust linear optimization under general norms

Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn Sim c a Sloan School of Management, Massachusetts Institute of Technology, Building E5-363, 50 Memorial Drive, Cambridge, MA 04-347, USA b Department of Mathematics and Sciences, Babson College, Forest Street, Babson Park, MA 0457, USA c NUS Business School, National University of Singapore, Business Link, Singapore 759, Singapore Received 8 August 003; received in revised form December 003; accepted 9 December 003 Abstract We explicitly characterize the robust counterpart of a linear programming problem with uncertainty set described by an arbitrary norm. Our approach encompasses several approaches from the literature and provides guarantees for constraint violation under probabilistic models that allow arbitrary dependencies in the distribution of the uncertain coecients. c 004 Elsevier B.V. All rights reserved. Keywords: Robust optimization; Stochastic programming; Linear programming; Norms; Dual norms; Constraint violation. Introduction Robust optimization is emerging as a leading methodology to address optimization problems under uncertainty. Applied to the linear programming problem max{c x Ãx 6 b; x P x } () with x R n, à R m n a matrix of uncertain coecients belonging to a known uncertainty set U, c R n and P x a given set representing the constraints involving certain coecients, the robust counterpart of Problem () is max{c x Ãx 6 b; x P x ; à U}: () Corresponding author. E-mail addresses: dbertsim@mit.edu (D. Bertsimas), dpachamanova@babson.edu (D. Pachamanova), dscsimm@nus.edu.sg (M. Sim). Research supported by the Singapore-MIT Alliance. An optimal solution x is robust with respect to any realization of the data, that is, it satises the constraints for any à U. The rst approach to modeling coecient uncertainty in the optimization literature was proposed by Soyster []. He considers problems of the kind max c x à j x j 6b; à j K j ;;:::;n;x 0 ; where x R n, c R n and b R m are data, and each column à j of the matrix à belongs to a convex set K j. Soyster shows that the problem is equivalent to a problem with linear constraints max c x A j x j 6 b; x 0 ; 067-6377/$ - see front matter c 004 Elsevier B.V. All rights reserved. doi:0.06/j.orl.003..007

D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 5 where the entries of the matrix A, a ij, satisfy a ij = sup A j K j (ã ij ). Some work followed Soyster s note (see, for example, [8]). However, the approaches suggested in this early literature solve a limited range of problems, are not easy to generalize, and are very conservative in the sense that they protect against the worst-case scenario. The interest in robust formulations in the optimization community was revived in the 990s. Ben-Tal and Nemirovski [ 4] introduced a number of important formulations and applications, and provided a detailed analysis of the robust optimization framework in linear programming and general convex programming. Independently, El Ghaoui et al. [9,0] derived similar results, and dealt with robust formulations of optimization problems by adapting robust control techniques. They showed that under the assumption that the coecient matrix data vary in ellipsoidal uncertainty sets, the robust counterparts of some important generic convex optimization problems (linear programming problems (LP), second-order cone problems (SOCP), semidenite programming problems (SDP)) are either exactly, or approximately tractable problems that are eciently solvable via interior point methods. However, under ellipsoidal uncertainty sets, the robust counterpart of an LP becomes an SOCP, of an SOCP becomes an SDP, while the robust counterpart of an SDP is NP-hard to solve. In other words, the diculty of the robust problem increases, as SDPs are numerically harder to solve than SOCPs, which in turn are harder to solve than LPs. Bertsimas and Sim [7] consider LPs such that each entry ã ij of à R m n is assumed to take values in the interval [ a ij ij ; a ij + ij ] and protect for the case that up to i of the uncertain coecients in constraint i, i= ;:::;m, can take their worst-case values at the same time. The parameter i controls the tradeo between robustness and optimality. Bertsimas and Sim show that the robust counterpart is still an LP. In this paper, we propose a framework for robust modeling of linear programming problems using uncertainty sets described by an arbitrary norm. We explicitly characterize the robust counterpart as a convex optimization problem that involves the dual norm of the given norm. Under a Euclidean norm we recover the second order cone formulation in [,,9,0], while under a particular D-norm we introduce we recover the linear programming formulation proposed in [7]. In this way, we shed some new light to the nature and structure of the robust counterpart of an LP. The paper is structured as follows. In Section, we review from [5] the robust counterpart of a linear programming problem when the deviations of the uncertain coecients lie in a convex set and characterize the robust counterpart of an LP when the uncertainty set is described by a general norm, as a convex optimization problem that involves the dual norm of the given norm. In Section 3, we show that by varying the norm used to dene the uncertainty set, we recover the second order cone formulation in [,,9,0], while under a particular D-norm we introduce we recover the linear programming formulation proposed in [7]. In Section 4, we provide guarantees for constraint violation under general probabilistic models that allow arbitrary dependencies in the distribution of the uncertain coecients. Notation. In this paper, lowercase boldface will be used to denote vectors, while uppercase boldface will denote matrices. Tilde (ã) will denote uncertain coef- cients, while check ( a) will be used for nominal values. à R m n will usually be the matrix of uncertain coecients in the linear programming problem, and vec(ã) R (m n) will denote the vector obtained by stacking its rows on top of one another.. Uncertainty sets dened by a norm In this section, we review from [5] the structure of the robust counterpart for uncertainty sets dened by general norms. These characterizations are used to develop the new characterizations in Section 3. Let S be a closed, bounded convex set and consider an uncertainty set in which the uncertain coecients are allowed to vary in such a way that the deviations from their nominal values fall in a convex set S U = {à (vec(ã) vec(a)) S}: The next theorem characterizes the robust counterpart. Theorem. Problem Ãx 6 b x P x à R m n such that (vec(ã) vec ( A)) S (3)

5 D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 can be formulated as a i x + max y S {y x} 6 b i ; x P x : i=;:::;m (4) Proof. Clearly since S is compact, for each constraint i, ã ix 6 b i for all vec(ã) vec( A) S if and only if max {ã ix} 6 b i : {vec(ã) vec( A) S} Since max {ã ix} = max {(vec(ã)) x i } {vec(ã) vec( A) S} {vec(ã) vec( A) S} the theorem follows. = (vec( A)) x i + max {y x}; {y S} We next consider uncertainty sets that arise from the requirement that the distance (as measured by an arbitrary norm) between uncertain coecients and their nominal values is bounded. Specically, we consider an uncertainty set U given by U = {à M(vec(Ã) vec( A)) 6 }; (5) where M is an invertible matrix, 0 and a general norm. Given a norm x for a real space of vectors x, its dual norm induced over the dual space of linear functionals s is dened as follows: Denition (Dual Norm). s : = max { x 6} s x: (6) The next result is well known (see, for example, []). Proposition. (a) The dual norm of the dual norm is the original norm. (b) The dual norm of the L p norm =p : x p = x j p (7) is the L q norm s q with q =+=(p ). The next theorem derives the form of the robust counterpart, when the uncertainty set is given by Eq. (5). Theorem. Problem Ãx 6 b x P x à U = {à M(vec(Ã) vec( A)) 6 }; (8) where M is an invertible matrix, can be formulated as a i x + M xi 6 b i ; i=;:::;m x P x ; (9) where x i R (m n) is a vector that contains x R n in entries (i ) n+ through i n, and 0 everywhere else. Proof. Introducing y =(M(vec(Ã) vec( A)))=, we can write U as U = {y y 6 } and obtain max {ã ix} {vec(ã) U} = max {(vec(ã)) x i } {vec(ã) U} = max {(vec( A)) x i + (M y) x} {y y 6} = a x + max {y y 6} {y (M x)}: By Denition, the second term in the last expression is M x. The theorem follows by applying Theorem. In applications, [,,9,0] work with uncertainty sets given by the Euclidean norm, i.e., U = {à M(vec(Ã) vec( A)) 6 }; where denotes the Euclidean norm. Since the Euclidean norm is self dual, it follows that the robust counterpart is a second order cone problem. If the uncertainty set is described by either or

D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 53 (these norms are dual to each other), then the resulting robust counterpart can be formulated as an LP. 3. The D-norm and its dual In this section, we show that the approach of [7] follows also from Theorem by simply considering a dierent norm, called the D-norm and its dual as opposed to the Euclidean norm considered in [,,9,0]. Furthermore, we show worst case bounds on the proximity of the D-norm to the Euclidean norm. Bertsimas and Sim [7] consider uncertainty sets given by &M(vec(Ã) vec( A)) & p 6 with p [;n] and for y R n = max {S {t} S N; S 6 p ;t N \S} y j +(p p ) y t : The fact that is indeed a norm, i.e., 0, &cy & p = c, =0 if and only y = 0, and &x + y & p 6 &x & p + follows easily. Specically [7] considers constraint-wise uncertainty sets, M a diagonal matrix containing the inverses of the ranges of coecient variation, and =. We next derive the dual norm. Proposition. The dual norm of the norm & & p is given by &s & p = max( s ; s =p): Proof. The norm can be written as = max u j y j u j 6 p; 0 6 u j 6 = min pr + t j r + t j y j ;t j 0; r 0; where the second equality follows by linear programming strong duality. Thus, 6 if and only if pr + t j 6 ;r+ t j y j ;t j 0; r 0 (0) is feasible. The dual norm &s & p is given by &s & p = & max s y: y & p 6 From Eq. (0) we have that &s & p = max s y pr + t j 6 ; y j t j r 6 0; y j t j r 6 0; r 0;t j 0; Applying LP duality again we obtain &s & p = min p u j Thus, &s & p = min u j v j 0; u j v j = s j ; 0;u j ;v j 0; j =;:::;n: v j 0; j =;:::;n: s j ; s j =p; and hence, &s & p = max( s ; s =p): Combining Theorem and Proposition leads to an LP formulation of the robust counterpart of the

54 D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 uncertainty set in [7]. We thus observe that Theorem provides a unied treatment of the approaches in [,,7,9,0]. 3.. Comparison with the Euclidean norm Since uncertainty sets in the literature have been described using the Euclidean and the D-norm it is of interest to understand the proximity between these two norms. Proposition 3. For every y R n { min ; } p & y & p 6 6 p +(p p ) n y { } min p ; & y n & { } p n 6 6 max y p ; : Proof. We nd a lower bound on = y by solving the following nonlinear optimization problem: max xj () j N &x & p =: Let S = {;:::; p }, t = p +. We can impose nonnegativity constraints on x and the constraints that x j x t ; j S and x j 6 x t ; j N \ S, without aecting the objective value of the Problem (). Observing that the objective can never decrease if we let x j = x t ; j N\S, we reduce () to the following problem: max xj +(n p )xt x j +(p p )x t =; x j x t j S; x t 0: () Since we are maximizing a convex function over a polytope, there exist an extreme point optimal solution to Problem (). There are S + inequality constraints. Out of those, S need to be tight to establish an extreme point solution. The S + extreme points can be found to be x k = e k k S; (3) x S + = e; (4) p where e is the vector of ones and e k is the unit vector with the kth element equals one, and the rest equal zero. Clearly, substituting all possible solutions, Problem () yields the optimum value of max{; n=p }. Taking the square root, the inequality y 6 max{; n=p} follows. Similarly, in order to derive an upper bound of = y, we solve the following problem: min xj j N (5) &x & p =: Using the same partition of the solution an before, and observing that the objective can never increase with x j =0; j N \ S \{t}, we reduce Problem (5) to the following problem: min xj + xt x j +(p p )x t =; x j x t j S; x t 0: (6) An optimal solution of Problem (6) can be found using the KKT conditions: p +(p p ) if j S; x j = p p p +(p p ) if j = t; 0 otherwise: Substituting, the optimal objective value of Problem (6) is ( p +(p p ) ). Hence, taking the square root, we establish the inequality 6 p +(p p ) y. Since 6 y 6 n; 6 y 6 ; y n y

D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 55 and { } &y & p = max y p ; y the bounds follow. 4. Probabilistic guarantees In this section, we derive probabilistic guarantees on the feasibility of an optimal robust solution when the uncertainty set U is described by a general norm with a dual norm. We assume that the matrix à has an arbitrary (and unknown) distribution, but with known expected value A R m n and known covariance matrix R (m n) (m n). Note that we allow arbitrary dependencies on Ã. We dene M = =, which is a symmetric matrix. Let x R n be an optimal solution to Problem (9). Recall that xi R (m n) denotes the vector containing x in entries (i ) n through i n, and 0 everywhere else. Theorem 3. (a) The probability that x satises constraint i for any realization of the uncertain matrix à is P(ã ix 6 b i ) =P((vec(Ã)) x i 6 b i ) + ( = xi = = xi ) : (7) (b) If the norm used in U is the D-norm & & p, then P(ã ix 6 b i ) { + min p ; n }: (8) (c) If the norm used in U is the dual D-norm & & p, then P(ã ix 6 b i ) }: (9) + min {; p n (d) If the norm used in U is the Euclidean norm, then P(ã ix 6 b i ) + : (0) Proof. Since an optimal robust solution satises (vec( A)) xi + = xi 6 b i ; we obtain that P((vec(Ã)) xi b i )6P((vec(Ã)) xi (vec( A)) xi + = xi ): () Bertsimas and Popescu [6] show that if S is a convex set, and X is a vector of random variables with mean X and covariance matrix, then P( X S) 6 +d ; () where d = inf ( X X) ( X X): X S We consider the convex set S i ={vec(ã) (vec(ã)) xi (vec( A)) xi + = xi }. In our case, d i = inf (vec(ã) vec( A)) (vec(ã) vec(ã) S i vec( A)): Applying the KKT conditions for this optimization problem we obtain an optimal solution ( ) vec( A) = xi + = xi xi with ( ) d = = xi = xi : Applying (), Eq. (7) follows. (b) If the norm used in the uncertainty set U is the D-norm, then by applying Proposition 3, we obtain Eq. (8). (c) If the norm used in the uncertainty set U is the dual D-norm, then by applying Proposition 3, we obtain Eq. (9). (d) If the norm used in the uncertainty set U is the Euclidean norm, then Eq. (0) follows immediately from Eq. (7). Under the stronger assumption that the data in each constraint are independently distributed according to a symmetric distribution [7] proves stronger bounds. In contrast the bounds in Theorem 3 are weaker, but

56 D. Bertsimas et al. / Operations Research Letters 3 (004) 50 56 have wider applicability as they include arbitrary dependencies. References [] A. Ben-Tal, A. Nemirovski, Robust convex optimization, Math. Oper. Res. 3 (4) (998) 769 805. [] A. Ben-Tal, A. Nemirovski, Robust solutions of uncertain linear programs, Oper. Res. Lett. 5 () (999) 3. [3] A. Ben-Tal, A. Nemirovski, Robust solutions of linear programming problems contaminated with uncertain data, Math. Program. A 88 (000) 4 44. [4] A. Ben-Tal, A. Nemirovski, On polyhedral approximations of the second-order Cone, Math. Oper. Res. 6 () (00) 93 8. [5] A. Ben-Tal and A. Nemirovski, Lectures on Modern Convex Optimization: Analysis, Algorithms, and Engineering Applications, MPR-SIAM Series on Optimization, SIAM, Philadelphia, 00. [6] D. Bertsimas, I. Popescu, Optimal inequalities in probability theory: a convex optimization approach, in SIAM J. Optim. 004, to appear. [7] D. Bertsimas, M. Sim, The price of robustness, Oper. Res. 5 () (004) 35 53. [8] J.E. Falk, Exact solutions of inexact linear programs, Oper. Res. 4 (976) 783 787. [9] L. El Ghaoui, H. Lebret, Robust solutions to least-squares problems with uncertain data, SIAM J Matrix Anal. Appl. 8 (4) (997) 035 064. [0] L. El Ghaoui, F. Oustry, H. Lebret, Robust solutions to uncertain semidenite programs, SIAM J. Optim. 9 () (998) 33 5. [] P.D. Lax, Linear Algebra, Wiley, New York, 997. [] A.L. Soyster, Convex programming with set-inclusive constraints and applications to inexact linear programming, Oper. Res (973) 54 57.