COM524500 Optimization for Communications 8. Semidefinite Programming Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 1 Semidefinite Programming () Inequality form: min c T x s.t. F(x) 0 where F(x) = F 0 + x 1 F 1 +... + x n F n, F i S p p. Standard form: min tr(cx) s.t. X 0 tr(a i X) = b i, i = 1,..., m where A i S n n, and C S n n. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 2
The inequality & standard forms can be shown to be equiv. F(x) 0 is commonly known as a linear matrix inequality (LMI). An with multiple LMIs min c T x s.t. F i (x) 0, i = 1,..., m can be reduced to an with one LMI since F i (x) 0, i = 1,..., m blkdiag(f 1 (x),..., F m (x)) 0 where blkdiag is the block diagonal operator. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 3 Example: Max. eigenvalue minimization Let λ max (X) denote the maximum eigenvalue of a matrix X. Max. eigenvalue minimization problem: min x λ max (A(x)) where A(x) = A 0 + x 1 A 1 +... + x n A n. We note that fixing x, λ max (A(x)) t A(x) ti 0 Hence, the problem is equiv. to min x,t t s.t. A(x) ti 0 Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 4
Standard LP: LP as min c T x s.t. x 0, a T i x = b i, i = 1,..., m Let C = diag(c), & A i = diag(a i ). The standard min tr(cx) s.t. X 0 tr(a i X) = b i, i = 1,..., m is equiv. to the LP since X 0 = diag(x) 0. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 5 Inequality form LP min c T x s.t. Ax b is equiv. to the min c T x s.t. diag(ax b) 0 because X 0 = X ii 0 for all i. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 6
Schur Complements Let X S n and partition X = A B T B C S = C B T A 1 B is called the Schur complement of A in X (provided A 0). Important facts: X 0 iff A 0 and S 0. If A 0, then X 0 iff S 0. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 7 Schur complements are useful in turning some nonlinear constraints into LMIs: Example: The convex quadratic inequality (Ax + b) T (Ax + b) c T x d 0 is equivalent to I (Ax + b) T Ax + b 0 c T x + d Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 8
QCQP as A convex QCQP can always be written as min A 0 x + b 0 2 2 ct 0 x d 0 s.t. A i x + b i 2 2 c T i x d i 0, i = 1,..., L By Schur complement, the QCQP is equiv. to min t [ ] s.t. I A 0 x + b 0 0 (A 0 x + b 0 ) T [ c T 0 x + d 0 + t ] I A i x + b i 0, i = 1,..., L (A i x + b i ) T c T i x + d i Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 9 Example: The second order cone inequality: Ax + b 2 f T x + d If the domain is such that f T x + d > 0, the inequality can be re-expressed as f T x + d 1 f T x + d (Ax + b)t (Ax + b) 0. By Schur complement, the inequality is equiv. to (ft x + d)i Ax + b 0 (Ax + b) T f T x + d This result indicates that SOCP can be turned to an. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 10
Application in Combinatorial Optimization Problem Statement: Consider the Boolean quadratic program (BQP): max x T Cx s.t. x i { 1, +1}, where C S n. The BQP is (very) nonconvex since the equality constraints x 2 i = 1 are nonconvex. In fact the BQP is NP-hard in general. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 11 Practical Example I: MAXCUT [GW95] Input: A graph G = (V, E) with weights w ij for (i, j) E. Assume w ij 0 and w ij = 0 if (i, j) / E. Goal: Divide nodes into two parts so as to maximize the weight of the edges whose nodes are in different parts. 1 w 13 3 w 14 4 2 w 25 5 Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 12
Suppose V = {1, 2,..., n}. The MAXCUT problem takes the form n n 1 x i x j max w ij 2 i=1 j=i+1 s.t. x i { 1, +1}, which can be rewritten as max x T Cx s.t. x i { 1, +1}, where C ij = C ji = 1 4 w ij for i j, & C ii = 1 2 n j=1 w ij. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 13 Practical Example II: ML MIMO detection [MDW + 02] H 11 H 1,n data symbols Multiplexer H m,1 Demultiplexer H m,n The tx & rx sides have n & m antennas, respectively. We consider a spatial multiplexing system, where each tx antenna transmits its own sequence of symbols. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 14
Assume frequency flat fading, & antipodal modulation. The received signal model may be expressed as y = Hs + v where y R m s { 1, +1} n H R m n v R n multi-receiver output vector; transmitted symbols; MIMO (or multi-antenna) channel; Gaussian noise with zero mean & covariance σ 2 I. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 15 Maximum-likelihood (ML) detection of s: min s {±1} n y Hs 2 2 = min s {±1} n st H T Hs 2s T H T y The ML problem is a nonhomogeneous BQP. But it can be reformulated as a homogeneous BQP. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 16
The ML problem can be homogenized as follows: min s {±1} n st H T Hs 2s T H T y = min s {±1} n c {±1} (c s) T H T H(c s) 2(c s) T H T y (s = c s) = min s {±1} n s T H T H s 2c s T H T y (c 2 = 1) c {±1} [ ] = min s T c HT H H T y ( s,c) {±1} n+1 y T H 0 c Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 17 BQP Approximation by : The BQP can be reformulated as max x,x tr(cx) s.t. X = xx T X ii = 1, Now, the objective function is linear for any C; the {±1} constraints on x becomes linear in X; but X = xx T is nonconvex. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 18
Using the fact that X = xx T = X 0 we approximate the BQP by solving the following : max tr(cx) s.t. X 0 X ii = 1, Once the is solved, its solution is used to approximate the BQP solution (e.g., by applying rank-1 approximation to the solution). Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 19 approximation looks heuristic, but it has been shown to provide good approx. accuracy. To see this, let f BQP = max x {±1} n xt Cx, f = max X 0,X ii =1 i trcx Goemans & Williamson [GW95] showed that for MAXCUT where C ij 0 for i j & C 0, 0.87856f f BQP f Nesterov [N98] showed that for C 0, 2 π f f BQP f Zhang [Z00] showed that if C ij 0 for i j, f BQP = f Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 20
Tx Downlink Beamforming for Broadcasting The problem is the same as that in the last lecture, except that the transmitter sends common information to all receivers. The received signal remains the same: y i = h T i x + v i, but the transmitted signal is now given by x = fs where f C m is the tx beamformer vector & s C is the information bearing signal. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 21 Problem: [SDL06] Minimize the tx power, subject to constraints that the received SNR is no less than some pre-specified threshold: h T i f 2 2 σ 2 i γ o, where γ o is the SNR threshold, & σ 2 i is the noise variance. Let Q i = 1 γ o h σi 2 i ht i. The problem can be written as min f 2 2 s.t. tr(ff H Q i ) 1, Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 22
Unfortunately the tx beamformer design problem here is nonconvex. Even worse, the problem is shown to be NP-hard [SDL06]. However, we can approximate the problem using. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 23 An equivalent form of the problem: min tr(f) f C m,f H m s.t. F = ff H tr(fq i ) 1, approximation: min tr(f) s.t. F 0 tr(fq i ) 1, Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 24
Another Problem: Maximize the weakest received SNR, subject to a constraint that the tx power is no greater than a threshold P o : max min i=1,...,n 1 σ 2 i s.t. f 2 2 P o This problem is also NP-hard. h i f 2 Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 25 The problem can be re-expressed as max t s.t. 1 σ 2 i h T i f 2 t, f 2 2 P o The associated approximation: max t s.t. 1 tr(h σi 2 i ht i F 0 F) t, tr(f) P o Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 26
References [MDW + 02] [GW95] [N98] [Z00] [SDL06] W.-K. Ma, T.N. Davidson, K.M. Wong, P.C. Ching, & Z.-Q. Luo, Quasi-ML multiuser detection using SDR with application to sync. CDMA, IEEE Trans. Signal Proc., 2002. M.X. Goemans & D.P. Williamson, Improved approx. alg. for max. cut & satisfiability problem using, J. ACM, 1995. Y.E. Nesterov, SDR and nonconvex quadratic opt., Opt. Meth. Software, 1998. S. Zhang, Quadratic max. and SDR, Math. Program., 2000. N.D. Sidiropoulos, T.N. Davidson, & Z.-Q. Luo, Transmit beamforming for physical layer multicasting, IEEE Trans. Signal Proc., 2006. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 27 Additional Reading on Applications [DLW00] [DTS01] T.N. Davidson, Z.-Q. Luo, & K.M. Wong, Design of orthogonal pulse shapes for communications via, IEEE Trans. Signal Proc., 2000. B. Dumitrescu, I. Tabus, & P. Stoica, On the parameterization of positive real sequences & MA parameter estimation, IEEE Trans. Signal Proc., 2001. Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 28