Convex optimization problems. Optimization problem in standard form

Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality constraints semidefinite programming vector optimization IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 1 Optimization problem in standard form minimize f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p x R n is the optimization variable f 0 : R n R is the objective or cost function f i : R n R, i = 1,..., m, are the inequality constraint functions h i : R n R are the equality constraint functions optimal value: p = inf{f 0 (x) f i (x) 0, i = 1,..., m, h i (x) = 0, i = 1,..., p} p = if problem is infeasible (no x satisfies the constraints) p = if problem is unbounded below IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 2

Optimal and locally optimal points x is feasible if x dom f 0 and it satisfies the constraints a feasible x is optimal if f 0 (x) = p ; X opt is the set of optimal points x is locally optimal if there is an R > 0 such that x is optimal for minimize (over z) f 0 (z) subject to f i (z) 0, i = 1,..., m, h i (z) = 0, i = 1,..., p z x 2 R examples (with n = 1, m = p = 0) f 0 (x) = 1/x, dom f 0 = R ++ : p = 0, no optimal point f 0 (x) = log x, dom f 0 = R ++ : p = f 0 (x) = x log x, dom f 0 = R ++ : p = 1/e, x = 1/e is optimal f 0 (x) = x 3 3x, p =, local optimum at x = 1 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 3 Implicit constraints The standard form optimization problem has an implicit constraint m p x D = dom f i dom h i, i=0 i=1 we call D the domain of the problem the constraints f i (x) 0, h i (x) = 0 are the explicit constraints a problem is unconstrained if it has no explicit constraints (m = p = 0) example: minimize f 0 (x) = k i=1 log(b i a T i x) is an unconstrained problem with implicit constraints a T i x < b i, i = 1,..., k IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 4

Feasibility problem find x subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p can be considered a special case of the general problem with f 0 (x) = 0: minimize 0 subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p p = 0 if constraints are feasible; any feasible x is optimal p = if constraints are infeasible IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 5 Convex optimization problem standard form convex optimization problem minimize f 0 (x) subject to f i (x) 0, i = 1,..., m ai T x = b i, i = 1,..., p f 0, f 1,..., f m are convex; equality constraints are affine problem is quasiconvex if f 0 is quasiconvex (and f 1,..., f m convex) often written as minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b important property: feasible set of a convex optimization problem is convex IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 6

example minimize f 0 (x) = x1 2 + x 2 2 subject to f 1 (x) = x 1 /(1 + x2 2) 0 h 1 (x) = (x 1 + x 2 ) 2 = 0 f 0 is convex; feasible set {(x 1, x 2 ) x 1 = x 2 0} is convex not a convex problem (according to our definition): f 1 is not convex, h 1 is not affine equivalent (but not identical) to the convex problem minimize x 2 1 + x 2 2 subject to x 1 0 x 1 + x 2 = 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 7 Local and global optima Any locally optimal point of a convex problem is (globally) optimal proof: suppose x is locally optimal and y is optimal with f 0 (y) < f 0 (x). x locally optimal means there is an R > 0 such that z feasible, z x 2 R = f 0 (z) f 0 (x). Consider z = θy + (1 θ)x with θ = R/(2 y x 2 ) y x 2 > R, so 0 < θ < 1/2 z is a convex combination of two feasible points, hence also feasible z x 2 = R/2 and f 0 (z) θf 0 (x) + (1 θ)f 0 (y) < f 0 (x), which contradicts our assumption that x is locally optimal IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 8

Optimality criterion for differentiable f 0 x is optimal if and only if it is feasible and f 0 (x) T (y x) 0 for all feasible y X x f 0 (x) if nonzero, f 0 (x) defines a supporting hyperplane to feasible set X at x IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 9 Optimality criterion: special cases unconstrained problem: x is optimal if and only if x dom f 0, f 0 (x) = 0 equality constrained problem minimize f 0 (x) subject to Ax = b x is optimal if and only if there exists a ν such that x dom f 0, Ax = b, f 0 (x) + A T ν = 0 minimization over nonnegative orthant x is optimal if and only if minimize f 0 (x) subject to x 0 x dom f 0, x 0, { f0 (x) i 0 x i = 0 f 0 (x) i = 0 x i > 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 10

Equivalent convex problems two problems are (informally) equivalent if the solution of one is readily obtained from the solution of the other, and vice-versa. Some common transformations that preserve convexity: eliminating equality constraints is equivalent to minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b minimize (over z) f 0 (Fz + x 0 ) subject to f i (Fz + x 0 ) 0, i = 1,..., m where F and x 0 are such that Ax = b x = Fz + x 0 for some z IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 11 introducing equality constraints is equivalent to minimize f 0 (A 0 x + b 0 ) subject to f i (A i x + b i ) 0, i = 1,..., m minimize (over x, y i ) f 0 (y 0 ) subject to f i (y i ) 0, i = 1,..., m y i = A i x + b i, i = 0, 1,..., m introducing slack variables for linear inequalities is equivalent to minimize f 0 (x) subject to ai T x b i, i = 1,..., m minimize (over x, s) f 0 (x) subject to ai T x + s i = b i, i = 1,..., m s i 0, i = 1,... m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 12

epigraph form: standard form convex problem is equivalent to minimize (over x, t) t subject to f 0 (x) t 0 f i (x) 0, i = 1,..., m Ax = b minimizing over some variables is equivalent to minimize f 0 (x 1, x 2 ) subject to f i (x 1 ) 0, i = 1,..., m minimize f0 (x 1 ) subject to f i (x 1 ) 0, i = 1,..., m where f 0 (x 1 ) = inf x2 f 0 (x 1, x 2 ) IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 13 Linear program (LP) minimize subject to c T x + d Gx h Ax = b convex problem with affine objective and constraint functions feasible set is a polyhedron c s P x IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 14

Examples diet problem: choose quantities x 1,..., x n of n foods one unit of food j costs c j, contains amount a ij of nutrient i healthy diet requires nutrient i in quantity at least b i to find cheapest healthy diet, minimize c T x subject to Ax b, x 0 piecewise-linear minimization minimize max i=1,...,m (ai T x + b i ) equivalent to an LP minimize t subject to ai T x + b i t, i = 1,..., m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 15 Chebyshev center of a polyhedron Chebyshev center of P = {x a T i x b i, i = 1,..., m} is center of largest inscribed ball x cheb B = {x c + u u 2 r} s a T i x b i for all x B if and only if sup{a T i (x c + u) u 2 r} = a T i x c + r a i 2 b i hence, x c, r can be determined by solving the LP maximize xc,r r subject to ai T x c + r a i 2 b i, i = 1,..., m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 16

Quadratic program (QP) minimize subject to (1/2)x T Px + q T x + r Gx h Ax = b P S n +, so objective is convex quadratic minimize a convex quadratic function over a polyhedron f 0 (x ) x P IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 17 Examples least-squares minimize Ax b 2 2 analytical solution x = A b (A is pseudo-inverse) can add linear constraints, e.g., l x u linear program with random cost minimize c T x + γx T Σx = E c T x + γ var(c T x) subject to Gx h, Ax = b c is random vector with mean c and covariance Σ hence, c T x is random variable with mean c T x and variance x T Σx γ > 0 is risk aversion parameter; controls the trade-off between expected cost and variance (risk) IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 18

Quadratically constrained quadratic program (QCQP) minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,..., m Ax = b P i S n +; objective and constraints are convex quadratic if P 1,..., P m S n ++, feasible region is intersection of m ellipsoids and an affine set IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 19 Second-order cone programming minimize f T x subject to A i x + b i 2 ci T x + d i, i = 1,..., m Fx = g (A i R n i n, F R p n ) inequalities are called second-order cone (SOC) constraints: (A i x + b i, c T i x + d i ) second-order cone in R n i +1 for n i = 0, reduces to an LP; if c i = 0, reduces to a QCQP more general than QCQP and LP IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 20

Robust linear programming the parameters in optimization problems are often uncertain, e.g., in an LP minimize c T x subject to ai T x b i, i = 1,..., m, there can be uncertainty in c, a i, b i two common approaches to handling uncertainty (in a i, for simplicity) deterministic model: constraints must hold for all a i E i minimize c T x subject to ai T x b i for all a i E i, i = 1,..., m, stochastic model: a i is random variable; constraints must hold with probability η minimize c T x subject to prob(ai T x b i ) η, i = 1,..., m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 21 Deterministic approach via SOCP choose an ellipsoid as E i : E i = {ā i + P i u u 2 1} (ā i R n, P i R n n ) center is ā i, semi-axes determined by singular values/vectors of P i robust LP minimize c T x subject to ai T x b i a i E i, i = 1,..., m is equivalent to the SOCP minimize c T x subject to āi T x + Pi T x 2 b i, i = 1,..., m (follows from sup u 2 1(ā i + P i u) T x = ā T i x + P T i x 2 ) IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 22

Stochastic approach via SOCP assume a i is Gaussian with mean ā i, covariance Σ i (a i N (ā i, Σ i )) ai T x is Gaussian r.v. with mean āi T x, variance x T Σ i x; hence ( ) prob(ai T b i āi T x x b i ) = Φ Σ 1/2 i x 2 where Φ(x) = (1/ 2π) x e t2 /2 dt is CDF of N (0, 1) robust LP minimize c T x subject to prob(ai T x b i ) η, i = 1,..., m, with η 1/2, is equivalent to the SOCP minimize c T x subject to āi T x + Φ 1 (η) Σ 1/2 i x 2 b i, i = 1,..., m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 23 Geometric programming monomial function f (x) = cx a 1 1 x a 2 2 x a n n, dom f = R n ++ with c > 0; exponent α i can be any real number posynomial function: sum of monomials f (x) = K k=1 geometric program (GP) c k x a 1k 1 x a 2k 2 x a nk n, dom f = R n ++ minimize f 0 (x) subject to f i (x) 1, i = 1,..., m h i (x) = 1, i = 1,..., p with f i posynomial, h i monomial IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 24

Geometric program in convex form change variables to y i = log x i, and take logarithm of cost, constraints monomial f (x) = cx a 1 1 x a n n transforms to log f (e y 1,..., e y n ) = a T y + b (b = log c) posynomial f (x) = K k=1 c kx a 1k 1 x a 2k 2 x a nk n ( K ) log f (e y 1,..., e y n ) = log e at k y+b k k=1 transforms to (b k = log c k ) geometric program transforms to convex problem ( K ) minimize log k=1 exp(at 0k y + b 0k) ( K ) subject to log k=1 exp(at ik y + b ik) 0, i = 1,..., m Gy + d = 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 25 Design of cantilever beam segment 4 segment 3 segment 2 segment 1 N segments with unit lengths, rectangular cross-sections of size w i h i given vertical force F applied at the right end design problem minimize subject to total weight upper & lower bounds on w i, h i upper bound & lower bounds on aspect ratios h i /w i upper bound on stress in each segment upper bound on vertical deflection at the end of the beam variables: w i, h i for i = 1,..., N IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 26 F

Objective and constraint functions total weight w 1 h 1 + + w N h N is posynomial aspect ratio h i /w i and inverse aspect ratio w i /h i are monomials maximum stress in segment i is given by 6iF /(w i h 2 i ), a monomial the vertical deflection y i and slope v i of central axis at the right end of segment i are defined recursively as v i = F 12(i 1/2) Ew i hi 3 + v i+1 y i = F 6(i 1/3) Ew i hi 3 + v i+1 + y i+1 for i = N, N 1,..., 1, with v N+1 = y N+1 = 0 (E is Young s modulus) v i and y i are posynomial functions of w, h IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 27 Formulation as a GP note minimize subject to w 1 h 1 + + w N h N w 1 maxw i 1, h 1 maxh i 1, w min w 1 i 1, i = 1,..., N h min h 1 i 1, i = 1,..., N S 1 maxw 1 i h i 1, S min w i h 1 i 1, i = 1,..., N 6iF σ 1 maxw 1 i y 1 maxy 1 1 h 2 i 1, i = 1,..., N we write w min w i w max and h min h i h max w min /w i 1, w i /w max 1, h min /h i 1, h i /h max 1 we write S min h i /w i S max as S min w i /h i 1, h i /(w i S max ) 1 The number of monomials appearing in y 1 grows approximately as N 2. IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 28

Minimizing spectral radius of nonnegative matrix Perron-Frobenius eigenvalue λ pf (A) Consider (elementwise) positive A R n n that is irreducible: (I + A) n 1 > 0. P-F Theorem: there is a real, positive eigenvalue of A, λ pf, equal to spectral radius max i λ i (A) determines asymptotic growth (decay) rate of A k : A k λ k pf as k alternative characterization: λ pf (A) = inf{λ Av λv for some v 0} minimizing spectral radius of matrix of posynomials minimize λ pf (A(x)), where the elements A(x) ij are posynomials of x equivalent geometric program: minimize subject to variables λ, v, x λ n j=1 A(x) ijv j /(λv i ) 1, i = 1,..., n IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 29 Quasiconvex optimization minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b with f 0 : R n R quasiconvex, f 1,..., f m convex can have locally optimal points that are not (globally) optimal (x, f 0 (x)) IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 30

convex representation of sublevel sets of f 0 if f 0 is quasiconvex, there exists a family of functions φ t such that: φ t (x) is convex in x for fixed t t-sublevel set of f 0 is 0-sublevel set of φ t, i.e., f 0 (x) t φ t (x) 0 example f 0 (x) = p(x) q(x) with p convex, q concave, and p(x) 0, q(x) > 0 on dom f 0 can take φ t (x) = p(x) tq(x): for t 0, φ t convex in x p(x)/q(x) t if and only if φ t (x) 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 31 quasiconvex optimization via convex feasibility problems φ t (x) 0, f i (x) 0, i = 1,..., m, Ax = b (1) for fixed t, a convex feasibility problem in x if feasible, we can conclude that t p ; if infeasible, t p Bisection method for quasiconvex optimization given l p, u p, tolerance ɛ > 0. repeat 1. t := (l + u)/2. 2. Solve the convex feasibility problem (1). 3. if (1) is feasible, u := t; else l := t. until u l ɛ. requires exactly log 2 ((u l)/ɛ) iterations (where u, l are initial values) IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 32

(Generalized) linear-fractional program linear-fractional program minimize subject to f 0 (x) Gx h Ax = b f 0 (x) = ct x + d e T x + f, dom f 0(x) = {x e T x + f > 0} a quasiconvex optimization problem; can be solved by bisection also equivalent to the LP (variables y, z) minimize subject to c T y + dz Gy hz Ay = bz e T y + fz = 1 z 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 33 Generalized linear-fractional program f 0 (x) = max i=1,...,r ci T x + d i ei T, dom f 0 (x) = {x ei T x+f i > 0, i = 1,..., r} x + f i a quasiconvex optimization problem; can be solved by bisection example: Von Neumann model of a growing economy maximize (over x, x + ) min i=1,...,n x + i /x i subject to x + 0, Bx + Ax x, x + R n : activity levels of n sectors, in current and next period (Ax) i, (Bx + ) i : produced, resp. consumed, amounts of good i x + i /x i : growth rate of sector i allocate activity to maximize growth rate of slowest growing sector IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 34

Generalized inequality constraints convex problem with generalized inequality constraints minimize f 0 (x) subject to f i (x) Ki 0, i = 1,..., m Ax = b f 0 : R n R convex; f i : R n R k i cone K i K i -convex w.r.t. proper same properties as standard convex problem (convex feasible set, local optimum is global, etc.) conic form problem: special case with affine objective and constraints minimize c T x subject to Fx + g K 0 Ax = b extends linear programming (K = R m +) to nonpolyhedral cones IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 35 Semidefinite program (SDP) with F i, G S k minimize c T x subject to x 1 F 1 + x 2 F 2 + + x n F n + G 0 Ax = b inequality constraint is called linear matrix inequality (LMI) includes problems with multiple LMI constraints: for example, x 1 ˆF 1 + + x n ˆF n + Ĝ 0, x 1 F 1 + + x n F n + G 0 is equivalent to single LMI x 1 [ ˆF 1 0 0 F 1 ] +x 2 [ ˆF 2 0 0 F 2 ] + +x n [ ˆF n 0 0 F n ] [ Ĝ 0 + 0 G ] 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 36

LP and SOCP as SDP LP and equivalent SDP LP: minimize c T x subject to Ax b SDP: minimize c T x subject to diag(ax b) 0 (note different interpretation of generalized inequality ) SOCP and equivalent SDP SOCP: minimize f T x subject to A i x + b i 2 c T i x + d i, i = 1,..., m SDP: minimize f [ T x (c T subject to i x + d i )I A i x + b i (A i x + b i ) T ci T x + d i ] 0, i = 1,..., m IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 37 Eigenvalue minimization minimize λ max (A(x)) where A(x) = A 0 + x 1 A 1 + + x n A n (with given A i S k ) equivalent SDP minimize subject to t A(x) ti variables x R n, t R follows from λ max (A) t A ti IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 38

Matrix norm minimization minimize A(x) 2 = ( λ max (A(x) T A(x)) ) 1/2 where A(x) = A 0 + x 1 A 1 + + x n A n (with given A i S p q ) equivalent SDP minimize subject to t [ ti A(x) T A(x) ti ] 0 variables x R n, t R constraint follows from A 2 t A T A t 2 I, t 0 [ ] ti A A T 0 ti IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 39 Convexity of vector-valued functions f : R n R m is K-convex if dom f is convex and f (θx + (1 θ)y) K θf (x) + (1 θ)f (y) for x, y dom f, 0 θ 1 example f : S m S m, f (X ) = X 2 is S m +-convex proof: for fixed z R m, z T X 2 z = Xz 2 2 is convex in X, i.e., z T (θx + (1 θ)y ) 2 z θz T X 2 z + (1 θ)z T Y 2 z for X, Y S m, 0 θ 1 therefore (θx + (1 θ)y ) 2 θx 2 + (1 θ)y 2 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 40

Vector optimization general vector optimization problem minimize (w.r.t. K) f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) 0, i = 1,..., p vector objective f 0 : R n R q, minimized w.r.t. proper cone K R q convex vector optimization problem minimize (w.r.t. K) f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b with f 0 K-convex, f 1,..., f m convex IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 41 Optimal and Pareto optimal points set of achievable objective values O = {f 0 (x) x feasible} feasible x is optimal if f 0 (x) is a minimum value of O feasible x is Pareto optimal if f 0 (x) is a minimal value of O nts O P f 0 (x po ) O f 0 (x ) x is optimal x po is Pareto optimal IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 42

Multicriterion optimization vector optimization problem with K = R q + f 0 (x) = (F 1 (x),..., F q (x)) q different objectives F i ; roughly speaking we want all F i s to be small feasible x is optimal if y feasible = f 0 (x ) f 0 (y) if there exists an optimal point, the objectives are noncompeting feasible x po is Pareto optimal if y feasible, f 0 (y) f 0 (x po ) = f 0 (x po ) = f 0 (y) if there are multiple Pareto optimal values, there is a trade-off between the objectives IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 43 Regularized least-squares multicriterion problem with two objectives F 1 (x) = Ax b 2 2, F 2 (x) = x 2 2 15 example with A R 100 10 shaded region is O heavy line is formed by Pareto optimal points F2(x) = x 2 2 10 5 0 0 5 10 15 F 1 (x) = Ax b 2 2 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 44

Risk return trade-off in portfolio optimization minimize (w.r.t. R 2 +) ( p T x, x T Σx) subject to 1 T x = 1, x 0 x R n is investment portfolio; x i is fraction invested in asset i p R n is vector of relative asset price changes; modeled as a random variable with mean p, covariance Σ p T x = E r is expected return; x T Σx = var r is return variance example mean return 15% 10% 5% allocation x 1 0.5 x(4) x(3) x(2) x(1) 0% 0% 10% 20% standard deviation of return 0 0% 10% 20% standard deviation of return IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 45 Scalarization to find Pareto optimal points: choose λ K 0 and solve scalar problem minimize λ T f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p if x is optimal for scalar problem, then it is Pareto-optimal for vector optimization problem s f 0 (x 1 ) O f λ 0 (x 3 ) 1 f 0 (x 2 ) λ 2 for convex vector optimization problems, can find (almost) all Pareto optimal points by varying λ K 0 IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 46

examples for multicriterion problem, find Pareto optimal points by minimizing positive weighted sum λ T f 0 (x) = λ 1 F 1 (x) + + λ q F q (x) regularized least-squares of page 111 (with λ = (1, γ)) minimize Ax b 2 2 + γ x 2 2 for fixed γ > 0, a least-squares problem risk-return trade-off of page 112 (with λ = (1, γ)) minimize p T x + γx T Σx subject to 1 T x = 1, x 0 for fixed γ > 0, a QP IOE 611: Nonlinear Programming, Winter 2006 4. Convex optimization problems Page 4 47