Convex Optimization M2
|
|
- Lillian Green
- 5 years ago
- Views:
Transcription
1 Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49
2 Duality A. d Aspremont. Convex Optimization M2. 2/49
3 DMs DM par A. d Aspremont. Convex Optimization M2. 3/49
4 Outline Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples theorems of alternatives generalized inequalities A. d Aspremont. Convex Optimization M2. 4/49
5 Lagrangian standard form problem (not necessarily convex) minimize f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p variable x R n, domain D, optimal value p Lagrangian: L : R n R m R p R, with dom L = D R m R p, L(x, λ, ν) = f 0 (x) + m λ i f i (x) + p ν i h i (x) weighted sum of objective and constraint functions λ i is Lagrange multiplier associated with f i (x) 0 ν i is Lagrange multiplier associated with h i (x) = 0 A. d Aspremont. Convex Optimization M2. 5/49
6 Lagrange dual function Lagrange dual function: g : R m R p R, g(λ, ν) = inf x D = inf x D L(x, λ, ν) ( f 0 (x) + g is concave, can be for some λ, ν m λ i f i (x) + ) p ν i h i (x) lower bound property: if λ 0, then g(λ, ν) p proof: if x is feasible and λ 0, then f 0 ( x) L( x, λ, ν) inf L(x, λ, ν) = g(λ, ν) x D minimizing over all feasible x gives p g(λ, ν) A. d Aspremont. Convex Optimization M2. 6/49
7 Least-norm solution of linear equations minimize x T x subject to Ax = b dual function Lagrangian is L(x, ν) = x T x + ν T (Ax b) to minimize L over x, set gradient equal to zero: x L(x, ν) = 2x + A T ν = 0 = x = (1/2)A T ν plug in in L to obtain g: a concave function of ν g(ν) = L(( 1/2)A T ν, ν) = 1 4 νt AA T ν b T ν lower bound property: p (1/4)ν T AA T ν b T ν for all ν A. d Aspremont. Convex Optimization M2. 7/49
8 Standard form LP minimize c T x subject to Ax = b, x 0 dual function Lagrangian is L(x, λ, ν) = c T x + ν T (Ax b) λ T x = b T ν + (c + A T ν λ) T x L is linear in x, hence g(λ, ν) = inf x L(x, λ, ν) = { b T ν A T ν λ + c = 0 otherwise g is linear on affine domain {(λ, ν) A T ν λ + c = 0}, hence concave lower bound property: p b T ν if A T ν + c 0 A. d Aspremont. Convex Optimization M2. 8/49
9 Equality constrained norm minimization dual function minimize subject to g(ν) = inf x ( x νt Ax + b T ν) = x Ax = b where v = sup u 1 u T v is dual norm of { b T ν A T ν 1 otherwise proof: follows from inf x ( x y T x) = 0 if y 1, otherwise if y 1, then x y T x 0 for all x, with equality if x = 0 if y > 1, choose x = tu where u 1, u T y = y > 1: x y T x = t( u y ) as t lower bound property: p b T ν if A T ν 1 A. d Aspremont. Convex Optimization M2. 9/49
10 Two-way partitioning minimize x T W x subject to x 2 i = 1, i = 1,..., n a nonconvex problem; feasible set contains 2 n discrete points interpretation: partition {1,..., n} in two sets; W ij is cost of assigning i, j to the same set; W ij is cost of assigning to different sets dual function g(ν) = inf x (xt W x + i ν i (x 2 i 1)) = inf x xt (W + diag(ν))x 1 T ν { 1 = T ν W + diag(ν) 0 otherwise lower bound property: p 1 T ν if W + diag(ν) 0 example: ν = λ min (W )1 gives bound p nλ min (W ) A. d Aspremont. Convex Optimization M2. 10/49
11 The dual problem Lagrange dual problem maximize g(λ, ν) subject to λ 0 finds best lower bound on p, obtained from Lagrange dual function a convex optimization problem; optimal value denoted d λ, ν are dual feasible if λ 0, (λ, ν) dom g often simplified by making implicit constraint (λ, ν) dom g explicit example: standard form LP and its dual (page 8) minimize subject to c T x Ax = b x 0 maximize b T ν subject to A T ν + c 0 A. d Aspremont. Convex Optimization M2. 11/49
12 Weak and strong duality weak duality: d p always holds (for convex and nonconvex problems) can be used to find nontrivial lower bounds for difficult problems for example, solving the SDP maximize 1 T ν subject to W + diag(ν) 0 gives a lower bound for the two-way partitioning problem on page 10 strong duality: d = p does not hold in general (usually) holds for convex problems conditions that guarantee strong duality in convex problems are called constraint qualifications A. d Aspremont. Convex Optimization M2. 12/49
13 Slater s constraint qualification strong duality holds for a convex problem if it is strictly feasible, i.e., minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b x int D : f i (x) < 0, i = 1,..., m, Ax = b also guarantees that the dual optimum is attained (if p > ) can be sharpened: e.g., can replace int D with relint D (interior relative to affine hull); linear inequalities do not need to hold with strict inequality,... there exist many other types of constraint qualifications A. d Aspremont. Convex Optimization M2. 13/49
14 Feasibility problems feasibility problem A in x R n. f i (x) < 0, i = 1,..., m, h i (x) = 0, i = 1,..., p feasibility problem B in λ R m, ν R p. λ 0, λ 0, g(λ, ν) 0 where g(λ, ν) = inf x ( m λ if i (x) + p ν ih i (x)) feasibility problem B is convex (g is concave), even if problem A is not A and B are always weak alternatives: at most one is feasible proof: assume x satisfies A, λ, ν satisfy B 0 g(λ, ν) m λ if i ( x) + p ν ih i ( x) < 0 A and B are strong alternatives if exactly one of the two is feasible (can prove infeasibility of A by producing solution of B and vice-versa). A. d Aspremont. Convex Optimization M2. 14/49
15 Inequality form LP primal problem minimize subject to c T x Ax b dual function g(λ) = inf x ( (c + A T λ) T x b T λ ) { b = T λ A T λ + c = 0 otherwise dual problem maximize b T λ subject to A T λ + c = 0, λ 0 from Slater s condition: p = d if A x b for some x in fact, p = d except when primal and dual are infeasible A. d Aspremont. Convex Optimization M2. 15/49
16 Quadratic program primal problem (assume P S n ++) minimize subject to x T P x Ax b dual function g(λ) = inf x ( x T P x + λ T (Ax b) ) = 1 4 λt AP 1 A T λ b T λ dual problem maximize (1/4)λ T AP 1 A T λ b T λ subject to λ 0 from Slater s condition: p = d if A x b for some x in fact, p = d always A. d Aspremont. Convex Optimization M2. 16/49
17 A nonconvex problem with strong duality minimize x T Ax + 2b T x subject to x T x 1 nonconvex if A 0 dual function: g(λ) = inf x (x T (A + λi)x + 2b T x λ) unbounded below if A + λi 0 or if A + λi 0 and b R(A + λi) minimized by x = (A + λi) b otherwise: g(λ) = b T (A + λi) b λ dual problem and equivalent SDP: maximize b T (A + λi) b λ subject to A + λi 0 b R(A + λi) maximize subject to t [ λ A + λi b b T t ] 0 strong duality although primal problem is not convex (not easy to show) A. d Aspremont. Convex Optimization M2. 17/49
18 Geometric interpretation For simplicity, consider problem with one constraint f 1 (x) 0 interpretation of dual function: g(λ) = inf (t + λu), where G = {(f 1(x), f 0 (x)) x D} (u,t) G t t G G λu + t = g(λ) p g(λ) u p d u λu + t = g(λ) is (non-vertical) supporting hyperplane to G hyperplane intersects t-axis at t = g(λ) A. d Aspremont. Convex Optimization M2. 18/49
19 epigraph variation: same interpretation if G is replaced with A = {(u, t) f 1 (x) u, f 0 (x) t for some x D} t A λu + t = g(λ) p g(λ) u strong duality holds if there is a non-vertical supporting hyperplane to A at (0, p ) for convex problem, A is convex, hence has supp. hyperplane at (0, p ) Slater s condition: if there exist (ũ, t) A with ũ < 0, then supporting hyperplanes at (0, p ) must be non-vertical A. d Aspremont. Convex Optimization M2. 19/49
20 Complementary slackness Assume strong duality holds, x is primal optimal, (λ, ν ) is dual optimal f 0 (x ) = g(λ, ν ) = inf x ( f 0 (x ) + f 0 (x ) f 0 (x) + m λ i f i (x) + m λ i f i (x ) + ) p νi h i (x) p νi h i (x ) hence, the two inequalities hold with equality x minimizes L(x, λ, ν ) λ i f i(x ) = 0 for i = 1,..., m (known as complementary slackness): λ i > 0 = f i (x ) = 0, f i (x ) < 0 = λ i = 0 A. d Aspremont. Convex Optimization M2. 20/49
21 Karush-Kuhn-Tucker (KKT) conditions the following four conditions are called KKT conditions (for a problem with differentiable f i, h i ): 1. Primal feasibility: f i (x) 0, i = 1,..., m, h i (x) = 0, i = 1,..., p 2. Dual feasibility: λ 0 3. Complementary slackness: λ i f i (x) = 0, i = 1,..., m 4. Gradient of Lagrangian with respect to x vanishes (first order condition): f 0 (x) + m λ i f i (x) + p ν i h i (x) = 0 If strong duality holds and x, λ, ν are optimal, then they must satisfy the KKT conditions A. d Aspremont. Convex Optimization M2. 21/49
22 KKT conditions for convex problem If x, λ, ν satisfy KKT for a convex problem, then they are optimal: from complementary slackness: f 0 ( x) = L( x, λ, ν) from 4th condition (and convexity): g( λ, ν) = L( x, λ, ν) hence, f 0 ( x) = g( λ, ν) If Slater s condition is satisfied, x is optimal if and only if there exist λ, ν that satisfy KKT conditions recall that Slater implies strong duality, and dual optimum is attained generalizes optimality condition f 0 (x) = 0 for unconstrained problem Summary: When strong duality holds, the KKT conditions are necessary conditions for optimality If the problem is convex, they are also sufficient A. d Aspremont. Convex Optimization M2. 22/49
23 example: water-filling (assume α i > 0) minimize n log(x i + α i ) subject to x 0, 1 T x = 1 x is optimal iff x 0, 1 T x = 1, and there exist λ R n, ν R such that λ 0, λ i x i = 0, 1 x i + α i + λ i = ν if ν < 1/α i : λ i = 0 and x i = 1/ν α i if ν 1/α i : λ i = ν 1/α i and x i = 0 determine ν from 1 T x = n max{0, 1/ν α i} = 1 interpretation n patches; level of patch i is at height α i flood area with unit amount of water 1/ν x i α i resulting level is 1/ν i A. d Aspremont. Convex Optimization M2. 23/49
24 Perturbation and sensitivity analysis (unperturbed) optimization problem and its dual minimize f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p maximize g(λ, ν) subject to λ 0 perturbed problem and its dual min. f 0 (x) s.t. f i (x) u i, i = 1,..., m h i (x) = v i, i = 1,..., p max. g(λ, ν) u T λ v T ν s.t. λ 0 x is primal variable; u, v are parameters p (u, v) is optimal value as a function of u, v we are interested in information about p (u, v) that we can obtain from the solution of the unperturbed problem and its dual A. d Aspremont. Convex Optimization M2. 24/49
25 Perturbation and sensitivity analysis global sensitivity result Strong duality holds for unperturbed problem and λ, ν are dual optimal for unperturbed problem. Apply weak duality to perturbed problem: p (u, v) g(λ, ν ) u T λ v T ν = p (0, 0) u T λ v T ν local sensitivity: if (in addition) p (u, v) is differentiable at (0, 0), then λ i = p (0, 0) u i, ν i = p (0, 0) v i A. d Aspremont. Convex Optimization M2. 25/49
26 Duality and problem reformulations equivalent formulations of a problem can lead to very different duals reformulating the primal problem can be useful when the dual is difficult to derive, or uninteresting common reformulations introduce new variables and equality constraints make explicit constraints implicit or vice-versa transform objective or constraint functions e.g., replace f 0 (x) by φ(f 0 (x)) with φ convex, increasing A. d Aspremont. Convex Optimization M2. 26/49
27 Introducing new variables and equality constraints minimize f 0 (Ax + b) dual function is constant: g = inf x L(x) = inf x f 0 (Ax + b) = p we have strong duality, but dual is quite useless reformulated problem and its dual minimize f 0 (y) subject to Ax + b y = 0 maximize b T ν f0 (ν) subject to A T ν = 0 dual function follows from g(ν) = inf (f 0(y) ν T y + ν T Ax + b T ν) x,y { f = 0 (ν) + b T ν A T ν = 0 otherwise A. d Aspremont. Convex Optimization M2. 27/49
28 norm approximation problem: minimize Ax b minimize subject to y y = Ax b can look up conjugate of, or derive dual directly g(ν) = inf x,y νt y ν T Ax + b T ν) { b = ν + inf y ( y + ν T y) A T ν = 0 otherwise { b = ν A T ν = 0, ν 1 otherwise (see page 7) dual of norm approximation problem maximize b T ν subject to A T ν = 0, ν 1 A. d Aspremont. Convex Optimization M2. 28/49
29 Implicit constraints LP with box constraints: primal and dual problem minimize subject to c T x Ax = b 1 x 1 maximize b T ν 1 T λ 1 1 T λ 2 subject to c + A T ν + λ 1 λ 2 = 0 λ 1 0, λ 2 0 reformulation with box constraints made implicit minimize f 0 (x) = subject to Ax = b { c T x 1 x 1 otherwise dual function g(ν) = inf 1 x 1 (ct x + ν T (Ax b)) = b T ν A T ν + c 1 dual problem: maximize b T ν A T ν + c 1 A. d Aspremont. Convex Optimization M2. 29/49
30 Problems with generalized inequalities Ki is generalized inequality on R k i definitions are parallel to scalar case: minimize f 0 (x) subject to f i (x) Ki 0, i = 1,..., m h i (x) = 0, i = 1,..., p Lagrange multiplier for f i (x) Ki 0 is vector λ i R k i Lagrangian L : R n R k 1 R k m R p R, is defined as L(x, λ 1,, λ m, ν) = f 0 (x) + m λ T i f i (x) + p ν i h i (x) dual function g : R k 1 R k m R p R, is defined as g(λ 1,..., λ m, ν) = inf x D L(x, λ 1,, λ m, ν) A. d Aspremont. Convex Optimization M2. 30/49
31 lower bound property: if λ i K i 0, then g(λ 1,..., λ m, ν) p proof: if x is feasible and λ K i 0, then f 0 ( x) f 0 ( x) + m λ T i f i ( x) + inf L(x, λ 1,..., λ m, ν) x D = g(λ 1,..., λ m, ν) p ν i h i ( x) minimizing over all feasible x gives p g(λ 1,..., λ m, ν) dual problem maximize g(λ 1,..., λ m, ν) subject to λ i K i 0, i = 1,..., m weak duality: p d always strong duality: p = d for convex problem with constraint qualification (for example, Slater s: primal problem is strictly feasible) A. d Aspremont. Convex Optimization M2. 31/49
32 Semidefinite program primal SDP (F i, G S k ) minimize subject to c T x x 1 F x n F n G Lagrange multiplier is matrix Z S k Lagrangian L(x, Z) = c T x + Tr (Z(x 1 F x n F n G)) dual function g(z) = inf x L(x, Z) = { Tr(GZ) Tr(Fi Z) + c i = 0, i = 1,..., n otherwise dual SDP maximize Tr(GZ) subject to Z 0, Tr(F i Z) + c i = 0, i = 1,..., n p = d if primal SDP is strictly feasible ( x with x 1 F x n F n G) A. d Aspremont. Convex Optimization M2. 32/49
33 Duality: SOCP example Let s consider the following Second Order Cone Program (SOCP): minimize f T x subject to A i x + b i 2 c T i x + d i, i = 1,..., m, with variable x R n. Let s show that the dual can be expressed as maximize subject to m (bt i u i + d i v i ) m (AT i u i + c i v i ) + f = 0 u i 2 v i, i = 1,..., m, with variables u i R n i, v i R, i = 1,..., m and problem data given by f R n, A i R n i n, b i R n i, c i R and d i R. A. d Aspremont. Convex Optimization M2. 33/49
34 Duality: SOCP We can derive the dual in the following two ways: 1. Introduce new variables y i R n i and t i R and equalities y i = A i x + b i, t i = c T i x + d i, and derive the Lagrange dual. 2. Start from the conic formulation of the SOCP and use the conic dual. Use the fact that the second-order cone is self-dual: t x tv + x T y 0, for all v, y such that v y The condition x T y tv is a simple Cauchy-Schwarz inequality A. d Aspremont. Convex Optimization M2. 34/49
35 Duality: SOCP We introduce new variables, and write the problem as minimize c T x subject to y i 2 t i, i = 1,..., m y i = A i x + b i, t i = c T i x + d i, i = 1,..., m The Lagrangian is L(x, y, t, λ, ν, µ) m m m = c T x + λ i ( y i 2 t i ) + νi T (y i A i x b i ) + µ i (t i c T i x d i ) m = (c A T i ν i n (b T i ν i + d i µ i ). m µ i c i ) T x + m (λ i y i 2 + νi T y i ) + m ( λ i + µ i )t i A. d Aspremont. Convex Optimization M2. 35/49
36 Duality: SOCP The minimum over x is bounded below if and only if m (A T i ν i + µ i c i ) = c. To minimize over y i, we note that inf y i (λ i y i 2 + ν T i y i ) = { 0 νi 2 λ i otherwise. The minimum over t i is bounded below if and only if λ i = µ i. A. d Aspremont. Convex Optimization M2. 36/49
37 Duality: SOCP The Lagrange dual function is g(λ, ν, µ) = n (bt i ν i + d i µ i ) if m (AT i ν i + µ i c i ) = c, ν i 2 λ i, µ = λ which leads to the dual problem otherwise maximize subject to n (b T i ν i + d i λ i ) m (A T i ν i + λ i c i ) = c ν i 2 λ i, i = 1,..., m. which is again an SOCP A. d Aspremont. Convex Optimization M2. 37/49
38 Duality: SOCP We can also express the SOCP as a conic form problem minimize c T x subject to (c T i x + d i, A i x + b i ) Ki 0, i = 1,..., m. The Lagrangian is given by: L(x, u i, v i ) = c T x i (A ix + b i ) T u i i (ct i x + d i)v i = ( c i (AT i u i + c i v i ) ) T x i (bt i u i + d i v i ) for (v i, u i ) K i 0 (which is also v i u i ) A. d Aspremont. Convex Optimization M2. 38/49
39 Duality: SOCP With L(x, u i, v i ) = ( c i the dual function is given by: g(λ, ν, µ) = The conic dual is then: ) T (A T i u i + c i v i ) x (b T i u i + d i v i ) n (bt i ν i + d i µ i ) if m (AT i ν i + µ i c i ) = c, otherwise maximize n subject to m (bt i u i + d i v i ) (AT i u i + v i c i ) = c (v i, u i ) K i 0, i = 1,..., m. i A. d Aspremont. Convex Optimization M2. 39/49
40 Proof Convex problem & constraint qualification Strong duality A. d Aspremont. Convex Optimization M2. 40/49
41 Slater s constraint qualification Convex problem minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b The problem satisfies Slater s condition if it is strictly feasible, i.e., x int D : f i (x) < 0, i = 1,..., m, Ax = b also guarantees that the dual optimum is attained (if p > ) there exist many other types of constraint qualifications A. d Aspremont. Convex Optimization M2. 41/49
42 KKT conditions for convex problem If x, λ, ν satisfy KKT for a convex problem, then they are optimal: from complementary slackness: f 0 ( x) = L( x, λ, ν) from 4th condition (and convexity): g( λ, ν) = L( x, λ, ν) hence, f 0 ( x) = g( λ, ν) with ( x, λ, ν) feasible. If Slater s condition is satisfied, x is optimal if and only if there exist λ, ν that satisfy KKT conditions Slater implies strong duality (more on this now), and dual optimum is attained generalizes optimality condition f 0 (x) = 0 for unconstrained problem Summary For a convex problem satisfying constraint qualification, the KKT conditions are necessary & sufficient conditions for optimality. A. d Aspremont. Convex Optimization M2. 42/49
43 Proof To simplify the analysis. We make two additional technical assumptions: The domain D has nonempty interior (hence, relint D = int D) We also assume that A has full rank, i.e. Rank A = p. A. d Aspremont. Convex Optimization M2. 43/49
44 Proof We define the set A as A = {(u, v, t) x D, f i (x) u i, i = 1,..., m, h i (x) = v i, i = 1,..., p, f 0 (x) t}, which is the set of values taken by the constraint and objective functions. If the problem is convex, A is defined by a list of convex constraints hence is convex. We define a second convex set B as B = {(0, 0, s) R m R p R s < p }. The sets A and B do not intersect (otherwise p could not be optimal value of the problem). First step: The hyperplane separating A and B defines a supporting hyperplane to A at (0, p ). A. d Aspremont. Convex Optimization M2. 44/49
45 Geometric proof t (ũ, t) A B u Illustration of strong duality proof, for a convex problem that satisfies Slater s constraint qualification. The two sets A and B are convex and do not intersect, so they can be separated by a hyperplane. Slater s constraint qualification guarantees that any separating hyperplane must be nonvertical. A. d Aspremont. Convex Optimization M2. 45/49
46 Proof By the separating hyperplane theorem there exists ( λ, ν, µ) 0 and α such that (u, v, t) A = λ T u + ν T v + µt α, (1) and (u, v, t) B = λ T u + ν T v + µt α. (2) From (1) we conclude that λ 0 and µ 0. (Otherwise λ T u + µt is unbounded below over A, contradicting (1).) The condition (2) simply means that µt α for all t < p, and hence, µp α. Together with (1) we conclude that for any x D, µp α µf 0 (x) + m λ i f i (x) + ν T (Ax b) (3) A. d Aspremont. Convex Optimization M2. 46/49
47 Proof Let us assume that µ > 0 (separating hyperplane is nonvertical) We can divide the previous equation by µ to get L(x, λ/µ, ν/µ) p for all x D Minimizing this inequality over x produces p g(λ, ν), where λ = λ/µ, ν = ν/µ. By weak duality we have g(λ, ν) p, so in fact g(λ, ν) = p. This shows that strong duality holds, and that the dual optimum is attained, whenever µ > 0. The normal vector has the form (λ, 1) and produces the Lagrange multipliers. A. d Aspremont. Convex Optimization M2. 47/49
48 Proof Second step: Slater s constraint qualification is used to establish that the hyperplane must be nonvertical, i.e. µ > 0. By contradiction, assume that µ = 0. From (3), we conclude that for all x D, m λ i f i (x) + ν T (Ax b) 0. (4) Applying this to the point x that satisfies the Slater condition, we have m λ i f i ( x) 0. Since f i ( x) < 0 and λ i 0, we conclude that λ = 0. A. d Aspremont. Convex Optimization M2. 48/49
49 Proof This is where we use the two technical assumptions. Then (4) implies that for all x D, ν T (Ax b) 0. But x satisfies ν T (A x b) = 0, and since x int D, there are points in D with ν T (Ax b) < 0 unless A T ν = 0. This contradicts our assumption that Rank A = p. This means that we cannot have µ = 0 and ends the proof. A. d Aspremont. Convex Optimization M2. 49/49
5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationOptimisation convexe: performance, complexité et applications.
Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationOn the Method of Lagrange Multipliers
On the Method of Lagrange Multipliers Reza Nasiri Mahalati November 6, 2016 Most of what is in this note is taken from the Convex Optimization book by Stephen Boyd and Lieven Vandenberghe. This should
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationCOM S 578X: Optimization for Machine Learning
COM S 578X: Optimization for Machine Learning Lecture Note 4: Duality Jia (Kevin) Liu Assistant Professor Department of Computer Science Iowa State University, Ames, Iowa, USA Fall 2018 JKL (CS@ISU) COM
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationHomework Set #6 - Solutions
EE 15 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre Tkacenko JPL Third Term 11-1 Homework Set #6 - Solutions 1 a The feasible set is the interval [ 4] The unique
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Subgradients
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationConvex Optimization in Communications and Signal Processing
Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,
More informationLagrangian Duality and Convex Optimization
Lagrangian Duality and Convex Optimization David Rosenberg New York University February 11, 2015 David Rosenberg (New York University) DS-GA 1003 February 11, 2015 1 / 24 Introduction Why Convex Optimization?
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationSolution to EE 617 Mid-Term Exam, Fall November 2, 2017
Solution to EE 67 Mid-erm Exam, Fall 207 November 2, 207 EE 67 Solution to Mid-erm Exam - Page 2 of 2 November 2, 207 (4 points) Convex sets (a) (2 points) Consider the set { } a R k p(0) =, p(t) for t
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationThe Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:
HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationA Tutorial on Convex Optimization II: Duality and Interior Point Methods
A Tutorial on Convex Optimization II: Duality and Interior Point Methods Haitham Hindi Palo Alto Research Center (PARC), Palo Alto, California 94304 email: hhindi@parc.com Abstract In recent years, convex
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 5 Fall 2009
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 5 Fall 2009 Reading: Boyd and Vandenberghe, Chapter 5 Solution 5.1 Note that
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 6 Fall 2009
UC Berkele Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 6 Fall 2009 Solution 6.1 (a) p = 1 (b) The Lagrangian is L(x,, λ) = e x + λx 2
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationSubgradients. subgradients and quasigradients. subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients and quasigradients subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE392o, Stanford University Basic inequality recall basic
More informationKarush-Kuhn-Tucker Conditions. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Karush-Kuhn-Tucker Conditions Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Given a minimization problem Last time: duality min x subject to f(x) h i (x) 0, i = 1,... m l j (x) = 0, j =
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationAgenda. 1 Duality for LP. 2 Theorem of alternatives. 3 Conic Duality. 4 Dual cones. 5 Geometric view of cone programs. 6 Conic duality theorem
Agenda 1 Duality for LP 2 Theorem of alternatives 3 Conic Duality 4 Dual cones 5 Geometric view of cone programs 6 Conic duality theorem 7 Examples Lower bounds on LPs By eliminating variables (if needed)
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationAdditional Homework Problems
Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationDuality Uses and Correspondences. Ryan Tibshirani Convex Optimization
Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions
More informationSubgradient. Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes. definition. subgradient calculus
1/41 Subgradient Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes definition subgradient calculus duality and optimality conditions directional derivative Basic inequality
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationConvex Optimization Lecture 6: KKT Conditions, and applications
Convex Optimization Lecture 6: KKT Conditions, and applications Dr. Michel Baes, IFOR / ETH Zürich Quick recall of last week s lecture Various aspects of convexity: The set of minimizers is convex. Convex
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationFinite Dimensional Optimization Part III: Convex Optimization 1
John Nachbar Washington University March 21, 2017 Finite Dimensional Optimization Part III: Convex Optimization 1 1 Saddle points and KKT. These notes cover another important approach to optimization,
More informationMachine Learning. Lecture 6: Support Vector Machine. Feng Li.
Machine Learning Lecture 6: Support Vector Machine Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 2018 Warm Up 2 / 80 Warm Up (Contd.)
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationCONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS
CONSTRAINT QUALIFICATIONS, LAGRANGIAN DUALITY & SADDLE POINT OPTIMALITY CONDITIONS A Dissertation Submitted For The Award of the Degree of Master of Philosophy in Mathematics Neelam Patel School of Mathematics
More informationSubgradients. subgradients. strong and weak subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients strong and weak subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE364b, Stanford University Basic inequality recall basic inequality
More informationConvex Optimization. Lecture 12 - Equality Constrained Optimization. Instructor: Yuanzhang Xiao. Fall University of Hawaii at Manoa
Convex Optimization Lecture 12 - Equality Constrained Optimization Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 19 Today s Lecture 1 Basic Concepts 2 for Equality Constrained
More informationLecture 5. The Dual Cone and Dual Problem
IE 8534 1 Lecture 5. The Dual Cone and Dual Problem IE 8534 2 For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The inner-product can be replaced by x T y if the coordinates of the
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More information