ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February 17, 2006
Lecture Outline Quadratic constrained quadratic programming (QCQP) Least-squares Second order cone programming (SOCP) Dual quadratic programming Geometric programming (GP) Dual geometric programming Wireless network power control Thanks: Stephen Boyd (some materials and graphs from Boyd and Vandenberghe)
Convex QCQP (Convex) QP (with linear constraints) in x: minimize (1/2)x T Px + q T x + r subject to Gx h Ax = b where P S n +, G Rm n, A R p n (Convex) QCQP in x: minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,2,..., m Ax = b where P S n +, i = 0,..., m
f 0 (x ) x P
Least-squares Minimize Ax b 2 2 = xt A T Ax 2b T Ax + b T b over x. Unconstrained QP, Regression analysis, Least-squares approximation Analytic solution: x = A b where, for A R m n, A = (A T A) 1 A T if rank of A is n, and A = A T (AA T ) 1 if rank of A is m. If not full rank, then by singular value decomposition. Constrained least-squares (no general analytic solution). For example: minimize Ax b 2 2 subject to l i x i u i, i = 1,..., n
LP with Random Cost minimize c T x subject to Gx h Ax = b Cost c R n is random, with mean c and covariance Ω Expected cost: c T x. Cost variance x T Ωx Minimize both expected cost and cost variance (with a weight γ): minimize subject to c T x + γx T Ωx Gx h Ax = b
SOCP Second Order Cone Programming: minimize subject to f T x A i x + b i 2 c T i x + d i, i = 1,..., m Fx = g Variables: x R n. And A i R n i n, F R p n If c i = 0, i, SOCP is equivalent to QCQP If A i = 0, i, SOCP is equivalent to LP
Robust LP Consider inequality constrained LP: minimize subject to c T x a T i x b i, i = 1,..., m Parameters a i are not accurate. They are only known to lie in given ellipsoids described by ā i and P i R n n : a i E i = {ā i + P i u u 2 1} Since sup{a T i x a i E} = ā T i x + P T i x 2, Robust LP (satisfy constraints for all possible a i ) formulated as SOCP: minimize subject to c T x ā T i x + P T i x 2 b i, i = 1,..., m
Dual QCQP Primal (convex) QCQP minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,2,..., m Ax = b Lagrangian: L(x, λ) = (1/2)x T P(λ)x + q(λ) T x + r(λ) where P(λ) = P 0 + mx λ i P i, q(λ) = q 0 + mx λ i q i, r(λ) = r 0 + mx λ i r i i=1 i=1 i=1 Since λ 0, we have P(λ) 0 if P 0 0 and Lagrange dual problem: g(λ) = inf x L(x, λ) = (1/2)q(λ)T P(λ) 1 q(λ) + r(λ) maximize subject to λ 0 (1/2)q(λ) T P(λ) 1 q(λ) + r(λ)
KKT Conditions for QP Primal (convex) QP with linear equality constraints: minimize (1/2)x T Px + q T x + r subject to Ax = b KKT conditions: Ax = b, Px + q + A T ν = 0 which can be written in matrix form: 2 3 2 4 P AT A 0 5 4 x ν 3 5 = 2 4 q b 3 5 Solving a system of linear equations is equivalent to solving equality constrained convex quadratic minimization
Monomials and Posynomials Monomial as a function f : R n + R: f(x) = dx a(1) 1 x a(2) 2... x a(n) n where the multiplicative constant d 0 and the exponential constants a (j) R, j = 1,2,..., n Sum of monomials is called a posynomial: f(x) = KX k=1 d k x a(1) k 1 x a(2) k 2... x a(n) k n. where d k 0, k = 1,2,..., K, and a (j) k R, j = 1, 2,..., n, k = 1,2,..., K Example: 2x 0.5 y π z is a monomial, x y is not a posynomial
GP GP standard form with variables x: minimize f 0 (x) subject to f i (x) 1, i = 1,2,..., m, h l (x) = 1, l = 1,2,..., M where f i, i = 0,1,..., m are posynomials and h l, l = 1,2,..., M are monomials Log transformation: y j = log x j, b ik = log d ik, b l = log d l GP convex form with variables y: minimize p 0 (y) = log P K 0 k=1 exp(at 0k y + b 0k) subject to p i (y) = log P K i k=1 exp(at ik y + b ik) 0, i = 1, 2,..., m, q l (y) = a T l y + b l = 0, l = 1, 2,..., M In convex form, GP with only monomials reduces to LP
Convexity of LogSumExp Log sum inequality (readily proved by the convexity of f(t) = t log t, t 0):! nx a i log a P nx n i i=1 a i log a i P b n i i=1 b i i=1 where a i, b i R +, i = 1,2,..., n i=1 Let ˆb i = log b i and P n i=1 a i = 1: log nx i=1 eˆb i! a Tˆb X n a i log a i i=1 So LogSumExp is the conjugate function of negative entropy Since all conjugate functions are convex, LogSumExp is convex
GP and Convexity Convexity is not invariant under nonlinear change of coordinates 5 120 4.5 60 3.5 Function 100 80 60 Function 4 3.5 3 2.5 Function 40 20 0 Function 3 2.5 40 20 0 10 5 X 0 10 5 Y 0 2 1.5 1 0.5 4 2 B 0 0 2 A 4 20 10 Y 5 0 0 5 X 10 2 3 2 B 1 0 0 1 A 2 3
Extensions of GP The following problem can be turned into an equivalent GP: maximize x/y subject to 2 x 3 x 2 + 3y/z y x/y = z 2 minimize x 1 y subject to 2x 1 1, (1/3)x 1 x 2 y 1/2 + 3y 1/2 z 1 1 xy 1 z 2 = 1 Let p, q be posynomials and r monomial minimize subject to p(x)/(r(x) q(x)) r(x) > q(x)
which is equivalent to minimize t subject to p(x) t(r(x) q(x)) (q(x)/r(x)) < 1 which is in turn equivalent to minimize t subject to (p(x)/t + q(x))/r(x) 1 (q(x)/r(x)) < 1 Generalized posynomials: composition of two posynomials. Generalized GP: minimize generalized posynomials over upper bound inequality constraints on other generalized posynomials Generalized GP can be turned into equivalent GP
Dual GP Primal problem: unconstrained GP in variables y minimize log Lagrange dual in variables ν: NX exp(a T i y + b i). i=1 maximize b T ν P N i=1 ν i log ν i subject to 1 T ν = 1, ν 0, A T ν = 0
Dual GP Primal problem: General GP in variables y minimize log P k 0 j=1 exp(at 0j y + b 0j) subject to Lagrange dual problem: log P k i j=1 exp(at ij y + b ij) 0, i = 1,..., m, maximize b T 0 ν 0 P k 0 j=1 ν 0j log ν 0j + P m i=1 b T i ν i P k i j=1 ν ij log ν ij 1 T ν i subject to ν i 0, i = 0,..., m, 1 T ν 0 = 1, P mi=0 A T i ν i = 0 where variables are ν i, i = 0,1,..., m A 0 is the matrix of the exponential constants in the objective function, and A i, i = 1,2,..., m are the matrices of the exponential constants in the constraint functions
Wireless Network Power Control Wireless CDMA cellular or multihop networks: Cell Phones Base Station Competing users all want: O1: High data rate O2: Low queuing delay O3: Low packet drop probability due to channel outage Optimize over transmit powers P such that: O1, O2 or O3 optimized for premium QoS class (or for maxmin fairness) Minimal QoS requirements on O1, O2 and O3 met for all users
A Sample of Power Control Problems 2 classes of traffic traverse a multihop wireless network: maximize Total System Throughput subject to Data Rate 1 Rate Requirement 1 Data Rate 2 Rate Requirement 2 Queuing Delay 1 Delay Requirement 1 Queuing Delay 2 Delay Requirement 2 Drop Prob 1 Drop Requirement 1 variables Drop Prob 2 Drop Requirement 2 Powers
Wireless Channel Models Signal Interference Ratio: SIR i (P) = P i G ii P N j i P jg ij + n i. Attainable data rate at high SIR: c i (P) = 1 T log 2 (KSIR i(p)). Outage probability on a wireless link: P o,i (P) = Prob{SIR i (P) SIR th } Average (Markovian) queuing delay with Poisson(Λ i ) arrival: D i (P) = 1 Γc i (P) Λ i
Conversion to GP Maximize the system throughput equivalent to minimize posynomial Q i ISR i Extension: maximize weighted sum of data rates: P i w ir i Outage probability bound: P o,i = Prob{SIR i SIR th } 1 = 1 Y k i 1 + SIR thg ik P k G ii P i Delay bound: 1 Γ T log 2 (SIR D i,max i) Λ i ISR i (P) 2 T Γ ( D 1 max +Λ i)
Cellular Case maximize subject to SIR i SIR k SIR k,min, k, Pj I k P j d γ j j α j < c k, k, P k1 G k1 = P k2 G k2, P k P k,max, k, P k 0, k. Recovers classical near-far problem in CDMA Max-min fairness objective function: maximize min k SIR k
GP Formulations Any combination is GP in high SIR Any combination without (C,D,c) is GP in any SIR Objective Function (A) Maximize R i (B) Maximize min i R i (C) Maximize P i R i (D) Maximize P i w ir i (E) Minimize P i P i Constraints (a) R i R i,min (b) P i1 G i1 = P i2 G i2 (c) P i R i R system,min (d) P o,i P o,i,max (e) 0 P i P i,max
Power Control by GP This suite of nonlinear nonconvex power control problems can be solved by GP (in standard form) Global optimality obtained efficiently For many combination of objectives and constraints Multi-rate, Multi-class, Multi-hop Feasibility Admission control Reduction in objective Admission pricing Key advantage: Allows nonlinear objectives and seemingly nonconvex constraints Key idea: These functions of SIR can be written as posynomials in P New results: low SIR regime and distributed algorithms
Numerical Example: Optimal Throughput-Delay Tradeoff 6 nodes, 8 links, 5 multi-hop flows, Direct Sequence CDMA max. power 1mW, target BER 10 3, path loss = distance 4 Maximized throughput of network increases as delay bound relaxes 290 Optimized throughput for different delay bounds B 2 C 280 A 1 4 7 8 3 6 D Maximized system throughput (kbps) 270 260 250 240 E 5 F 230 220 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 Delay bound (s) Heuristics: Delay bound violation or system throughput reduction
Lecture Summary First type of nonlinearity: quadratic. Second type of nonlinearity: Posynomial or LogSumExp. Nonlinear problems that are, or can be converted into, convex optimization: QCQP (SOCP) and GP. Both cover LP as special cases. Significantly extend scope of optimization-theoretic modelling, analysis, and design. Readings: Section 4.4, 4.5, 5.5, 5.7 in Boyd and Vandenberghe M. Chiang, Geometric Programming for Communication Systems, Trends and Foundations in Information and Communications Theory, vol. 2, no. 1, pp. 1-156, 2005.