ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications
|
|
- Eustacia Carson
- 6 years ago
- Views:
Transcription
1 ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February 17, 2006
2 Lecture Outline Quadratic constrained quadratic programming (QCQP) Least-squares Second order cone programming (SOCP) Dual quadratic programming Geometric programming (GP) Dual geometric programming Wireless network power control Thanks: Stephen Boyd (some materials and graphs from Boyd and Vandenberghe)
3 Convex QCQP (Convex) QP (with linear constraints) in x: minimize (1/2)x T Px + q T x + r subject to Gx h Ax = b where P S n +, G Rm n, A R p n (Convex) QCQP in x: minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,2,..., m Ax = b where P S n +, i = 0,..., m
4 f 0 (x ) x P
5 Least-squares Minimize Ax b 2 2 = xt A T Ax 2b T Ax + b T b over x. Unconstrained QP, Regression analysis, Least-squares approximation Analytic solution: x = A b where, for A R m n, A = (A T A) 1 A T if rank of A is n, and A = A T (AA T ) 1 if rank of A is m. If not full rank, then by singular value decomposition. Constrained least-squares (no general analytic solution). For example: minimize Ax b 2 2 subject to l i x i u i, i = 1,..., n
6 LP with Random Cost minimize c T x subject to Gx h Ax = b Cost c R n is random, with mean c and covariance Ω Expected cost: c T x. Cost variance x T Ωx Minimize both expected cost and cost variance (with a weight γ): minimize subject to c T x + γx T Ωx Gx h Ax = b
7 SOCP Second Order Cone Programming: minimize subject to f T x A i x + b i 2 c T i x + d i, i = 1,..., m Fx = g Variables: x R n. And A i R n i n, F R p n If c i = 0, i, SOCP is equivalent to QCQP If A i = 0, i, SOCP is equivalent to LP
8 Robust LP Consider inequality constrained LP: minimize subject to c T x a T i x b i, i = 1,..., m Parameters a i are not accurate. They are only known to lie in given ellipsoids described by ā i and P i R n n : a i E i = {ā i + P i u u 2 1} Since sup{a T i x a i E} = ā T i x + P T i x 2, Robust LP (satisfy constraints for all possible a i ) formulated as SOCP: minimize subject to c T x ā T i x + P T i x 2 b i, i = 1,..., m
9 Dual QCQP Primal (convex) QCQP minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,2,..., m Ax = b Lagrangian: L(x, λ) = (1/2)x T P(λ)x + q(λ) T x + r(λ) where P(λ) = P 0 + mx λ i P i, q(λ) = q 0 + mx λ i q i, r(λ) = r 0 + mx λ i r i i=1 i=1 i=1 Since λ 0, we have P(λ) 0 if P 0 0 and Lagrange dual problem: g(λ) = inf x L(x, λ) = (1/2)q(λ)T P(λ) 1 q(λ) + r(λ) maximize subject to λ 0 (1/2)q(λ) T P(λ) 1 q(λ) + r(λ)
10 KKT Conditions for QP Primal (convex) QP with linear equality constraints: minimize (1/2)x T Px + q T x + r subject to Ax = b KKT conditions: Ax = b, Px + q + A T ν = 0 which can be written in matrix form: P AT A x ν 3 5 = 2 4 q b 3 5 Solving a system of linear equations is equivalent to solving equality constrained convex quadratic minimization
11 Monomials and Posynomials Monomial as a function f : R n + R: f(x) = dx a(1) 1 x a(2) 2... x a(n) n where the multiplicative constant d 0 and the exponential constants a (j) R, j = 1,2,..., n Sum of monomials is called a posynomial: f(x) = KX k=1 d k x a(1) k 1 x a(2) k 2... x a(n) k n. where d k 0, k = 1,2,..., K, and a (j) k R, j = 1, 2,..., n, k = 1,2,..., K Example: 2x 0.5 y π z is a monomial, x y is not a posynomial
12 GP GP standard form with variables x: minimize f 0 (x) subject to f i (x) 1, i = 1,2,..., m, h l (x) = 1, l = 1,2,..., M where f i, i = 0,1,..., m are posynomials and h l, l = 1,2,..., M are monomials Log transformation: y j = log x j, b ik = log d ik, b l = log d l GP convex form with variables y: minimize p 0 (y) = log P K 0 k=1 exp(at 0k y + b 0k) subject to p i (y) = log P K i k=1 exp(at ik y + b ik) 0, i = 1, 2,..., m, q l (y) = a T l y + b l = 0, l = 1, 2,..., M In convex form, GP with only monomials reduces to LP
13 Convexity of LogSumExp Log sum inequality (readily proved by the convexity of f(t) = t log t, t 0):! nx a i log a P nx n i i=1 a i log a i P b n i i=1 b i i=1 where a i, b i R +, i = 1,2,..., n i=1 Let ˆb i = log b i and P n i=1 a i = 1: log nx i=1 eˆb i! a Tˆb X n a i log a i i=1 So LogSumExp is the conjugate function of negative entropy Since all conjugate functions are convex, LogSumExp is convex
14 GP and Convexity Convexity is not invariant under nonlinear change of coordinates Function Function Function Function X Y B A Y X B A 2 3
15 Extensions of GP The following problem can be turned into an equivalent GP: maximize x/y subject to 2 x 3 x 2 + 3y/z y x/y = z 2 minimize x 1 y subject to 2x 1 1, (1/3)x 1 x 2 y 1/2 + 3y 1/2 z 1 1 xy 1 z 2 = 1 Let p, q be posynomials and r monomial minimize subject to p(x)/(r(x) q(x)) r(x) > q(x)
16 which is equivalent to minimize t subject to p(x) t(r(x) q(x)) (q(x)/r(x)) < 1 which is in turn equivalent to minimize t subject to (p(x)/t + q(x))/r(x) 1 (q(x)/r(x)) < 1 Generalized posynomials: composition of two posynomials. Generalized GP: minimize generalized posynomials over upper bound inequality constraints on other generalized posynomials Generalized GP can be turned into equivalent GP
17 Dual GP Primal problem: unconstrained GP in variables y minimize log Lagrange dual in variables ν: NX exp(a T i y + b i). i=1 maximize b T ν P N i=1 ν i log ν i subject to 1 T ν = 1, ν 0, A T ν = 0
18 Dual GP Primal problem: General GP in variables y minimize log P k 0 j=1 exp(at 0j y + b 0j) subject to Lagrange dual problem: log P k i j=1 exp(at ij y + b ij) 0, i = 1,..., m, maximize b T 0 ν 0 P k 0 j=1 ν 0j log ν 0j + P m i=1 b T i ν i P k i j=1 ν ij log ν ij 1 T ν i subject to ν i 0, i = 0,..., m, 1 T ν 0 = 1, P mi=0 A T i ν i = 0 where variables are ν i, i = 0,1,..., m A 0 is the matrix of the exponential constants in the objective function, and A i, i = 1,2,..., m are the matrices of the exponential constants in the constraint functions
19 Wireless Network Power Control Wireless CDMA cellular or multihop networks: Cell Phones Base Station Competing users all want: O1: High data rate O2: Low queuing delay O3: Low packet drop probability due to channel outage Optimize over transmit powers P such that: O1, O2 or O3 optimized for premium QoS class (or for maxmin fairness) Minimal QoS requirements on O1, O2 and O3 met for all users
20 A Sample of Power Control Problems 2 classes of traffic traverse a multihop wireless network: maximize Total System Throughput subject to Data Rate 1 Rate Requirement 1 Data Rate 2 Rate Requirement 2 Queuing Delay 1 Delay Requirement 1 Queuing Delay 2 Delay Requirement 2 Drop Prob 1 Drop Requirement 1 variables Drop Prob 2 Drop Requirement 2 Powers
21 Wireless Channel Models Signal Interference Ratio: SIR i (P) = P i G ii P N j i P jg ij + n i. Attainable data rate at high SIR: c i (P) = 1 T log 2 (KSIR i(p)). Outage probability on a wireless link: P o,i (P) = Prob{SIR i (P) SIR th } Average (Markovian) queuing delay with Poisson(Λ i ) arrival: D i (P) = 1 Γc i (P) Λ i
22 Conversion to GP Maximize the system throughput equivalent to minimize posynomial Q i ISR i Extension: maximize weighted sum of data rates: P i w ir i Outage probability bound: P o,i = Prob{SIR i SIR th } 1 = 1 Y k i 1 + SIR thg ik P k G ii P i Delay bound: 1 Γ T log 2 (SIR D i,max i) Λ i ISR i (P) 2 T Γ ( D 1 max +Λ i)
23 Cellular Case maximize subject to SIR i SIR k SIR k,min, k, Pj I k P j d γ j j α j < c k, k, P k1 G k1 = P k2 G k2, P k P k,max, k, P k 0, k. Recovers classical near-far problem in CDMA Max-min fairness objective function: maximize min k SIR k
24 GP Formulations Any combination is GP in high SIR Any combination without (C,D,c) is GP in any SIR Objective Function (A) Maximize R i (B) Maximize min i R i (C) Maximize P i R i (D) Maximize P i w ir i (E) Minimize P i P i Constraints (a) R i R i,min (b) P i1 G i1 = P i2 G i2 (c) P i R i R system,min (d) P o,i P o,i,max (e) 0 P i P i,max
25 Power Control by GP This suite of nonlinear nonconvex power control problems can be solved by GP (in standard form) Global optimality obtained efficiently For many combination of objectives and constraints Multi-rate, Multi-class, Multi-hop Feasibility Admission control Reduction in objective Admission pricing Key advantage: Allows nonlinear objectives and seemingly nonconvex constraints Key idea: These functions of SIR can be written as posynomials in P New results: low SIR regime and distributed algorithms
26 Numerical Example: Optimal Throughput-Delay Tradeoff 6 nodes, 8 links, 5 multi-hop flows, Direct Sequence CDMA max. power 1mW, target BER 10 3, path loss = distance 4 Maximized throughput of network increases as delay bound relaxes 290 Optimized throughput for different delay bounds B 2 C 280 A D Maximized system throughput (kbps) E 5 F Delay bound (s) Heuristics: Delay bound violation or system throughput reduction
27 Lecture Summary First type of nonlinearity: quadratic. Second type of nonlinearity: Posynomial or LogSumExp. Nonlinear problems that are, or can be converted into, convex optimization: QCQP (SOCP) and GP. Both cover LP as special cases. Significantly extend scope of optimization-theoretic modelling, analysis, and design. Readings: Section 4.4, 4.5, 5.5, 5.7 in Boyd and Vandenberghe M. Chiang, Geometric Programming for Communication Systems, Trends and Foundations in Information and Communications Theory, vol. 2, no. 1, pp , 2005.
ELE539A: Optimization of Communication Systems Lecture 16: Pareto Optimization and Nonconvex Optimization
ELE539A: Optimization of Communication Systems Lecture 16: Pareto Optimization and Nonconvex Optimization Professor M. Chiang Electrical Engineering Department, Princeton University March 16, 2007 Lecture
More informationGeometric Programming for Communication Systems
Geometric Programming for Communication Systems Mung Chiang Electrical Engineering Department, Princeton University Tutorial Presentation August 2005 Outline Part I: Geometric Programming What s GP? Why
More informationPOWER CONTROL BY GEOMETRIC PROGRAMMING
POWER CONTROL BY GEOMETRIC PROGRAMMING MUNG CHIANG, CHEE WEI TAN, DANIEL O NEILL, DAVID JULIAN Abstract. In wireless cellular or ad hoc networks that are interference-limited, a variety of power control
More information2640 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 6, NO. 7, JULY Power Control By Geometric Programming
2640 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 6, NO. 7, JULY 2007 Power Control By Geometric Programming Mung Chiang, Chee Wei Tan, Daniel P. Palomar, Daniel O Neill, and David Julian Abstract
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationEfficient Nonlinear Optimizations of Queuing Systems
Efficient Nonlinear Optimizations of Queuing Systems Mung Chiang, Arak Sutivong, and Stephen Boyd Electrical Engineering Department, Stanford University, CA 9435 Abstract We present a systematic treatment
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationDuality. Geoff Gordon & Ryan Tibshirani Optimization /
Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationDuality in Linear Programs. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Duality in Linear Programs Lecturer: Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: proximal gradient descent Consider the problem x g(x) + h(x) with g, h convex, g differentiable, and
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationLecture 4: Linear and quadratic problems
Lecture 4: Linear and quadratic problems linear programming examples and applications linear fractional programming quadratic optimization problems (quadratically constrained) quadratic programming second-order
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationUC Berkeley Department of Electrical Engineering and Computer Science. EECS 227A Nonlinear and Convex Optimization. Solutions 5 Fall 2009
UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 5 Fall 2009 Reading: Boyd and Vandenberghe, Chapter 5 Solution 5.1 Note that
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationLecture 20: November 1st
10-725: Optimization Fall 2012 Lecture 20: November 1st Lecturer: Geoff Gordon Scribes: Xiaolong Shen, Alex Beutel Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationDistributed Geometric-Programming-Based Power Control in Cellular Cognitive Radio Networks
Distributed Geometric-Programming-Based Power Control in Cellular Cognitive Radio Networks Qingqing Jin, Dongfeng Yuan, Zhangyu Guan Wireless Mobile Communication and Transmission WMCT Lab School of Information
More informationExercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem
Exercises Basic terminology and optimality conditions 4.1 Consider the optimization problem f 0(x 1, x 2) 2x 1 + x 2 1 x 1 + 3x 2 1 x 1 0, x 2 0. Make a sketch of the feasible set. For each of the following
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationConvex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014
Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,
More informationConvex Optimization. Lecture 12 - Equality Constrained Optimization. Instructor: Yuanzhang Xiao. Fall University of Hawaii at Manoa
Convex Optimization Lecture 12 - Equality Constrained Optimization Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 19 Today s Lecture 1 Basic Concepts 2 for Equality Constrained
More informationDual Decomposition.
1/34 Dual Decomposition http://bicmr.pku.edu.cn/~wenzw/opt-2017-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/34 1 Conjugate function 2 introduction:
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationl p -Norm Constrained Quadratic Programming: Conic Approximation Methods
OUTLINE l p -Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn OUTLINE OUTLINE
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More information11. Equality constrained minimization
Convex Optimization Boyd & Vandenberghe 11. Equality constrained minimization equality constrained minimization eliminating equality constraints Newton s method with equality constraints infeasible start
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationHomework Set #6 - Solutions
EE 15 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre Tkacenko JPL Third Term 11-1 Homework Set #6 - Solutions 1 a The feasible set is the interval [ 4] The unique
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationLecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
More informationIE 521 Convex Optimization
Lecture 14: and Applications 11th March 2019 Outline LP SOCP SDP LP SOCP SDP 1 / 21 Conic LP SOCP SDP Primal Conic Program: min c T x s.t. Ax K b (CP) : b T y s.t. A T y = c (CD) y K 0 Theorem. (Strong
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationEE364 Review Session 4
EE364 Review Session 4 EE364 Review Outline: convex optimization examples solving quasiconvex problems by bisection exercise 4.47 1 Convex optimization problems we have seen linear programming (LP) quadratic
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationECE Optimization for wireless networks Final. minimize f o (x) s.t. Ax = b,
ECE 788 - Optimization for wireless networks Final Please provide clear and complete answers. PART I: Questions - Q.. Discuss an iterative algorithm that converges to the solution of the problem minimize
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationConvex Optimization for Signal Processing and Communications: From Fundamentals to Applications
Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications Chong-Yung Chi Institute of Communications Engineering & Department of Electrical Engineering National Tsing
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationApplications of Robust Optimization in Signal Processing: Beamforming and Power Control Fall 2012
Applications of Robust Optimization in Signal Processing: Beamforg and Power Control Fall 2012 Instructor: Farid Alizadeh Scribe: Shunqiao Sun 12/09/2012 1 Overview In this presentation, we study the applications
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationSupport Vector Machines, Kernel SVM
Support Vector Machines, Kernel SVM Professor Ameet Talwalkar Professor Ameet Talwalkar CS260 Machine Learning Algorithms February 27, 2017 1 / 40 Outline 1 Administration 2 Review of last lecture 3 SVM
More informationDeterminant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University
Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More information1 Strict local optimality in unconstrained optimization
ORF 53 Lecture 14 Spring 016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, April 14, 016 When in doubt on the accuracy of these notes, please cross check with the instructor s
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationPattern Classification, and Quadratic Problems
Pattern Classification, and Quadratic Problems (Robert M. Freund) March 3, 24 c 24 Massachusetts Institute of Technology. 1 1 Overview Pattern Classification, Linear Classifiers, and Quadratic Optimization
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationMATH 4211/6211 Optimization Constrained Optimization
MATH 4211/6211 Optimization Constrained Optimization Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 Constrained optimization
More informationCOM Optimization for Communications 8. Semidefinite Programming
COM524500 Optimization for Communications 8. Semidefinite Programming Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 1 Semidefinite Programming () Inequality form: min c T x s.t.
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Subgradients
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationRegional Solution of Constrained LQ Optimal Control
Regional Solution of Constrained LQ Optimal Control José DeDoná September 2004 Outline 1 Recap on the Solution for N = 2 2 Regional Explicit Solution Comparison with the Maximal Output Admissible Set 3
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationThe Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System
The Q-parametrization (Youla) Lecture 3: Synthesis by Convex Optimization controlled variables z Plant distubances w Example: Spring-mass system measurements y Controller control inputs u Idea for lecture
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More information