Semidefinite Relaxation and Its Applications in Signal Processing and Communications

Transcription

1 Semidefinite Relaxation and Its Applications in Signal Processing and Communications Wing-Kin (Ken) Ma Department of Electronic Engineering The Chinese University of Hong Kong Hong Kong MIIS 2012 Tutorial, Xi an, July 9-13, 2012 Acknowledgement: Anthony Man-Cho So, Dept. SEEM, CUHK, for co-creating this tutorial note.

2 BRAND X PICTURES A recent tutorial article: Z.-Q. Luo, W.-K. Ma, Anthony M.-C. So, Y. Ye, & S. Zhang, Semidefinite relaxation of quadratic optimization problems, in IEEE SP Magazine, Special Issue on Convex Optimization for Signal Process., May [ Zhi-Quan Luo, Wing-Kin Ma, Anthony Man-Cho So, Yinyu Ye, and Shuzhong Zhang ] [ VOLUME 27 NUMBER 3 MAY 2010 ] [ From its practical deployments and scope of applicability to key theoretical results] In recent years, the semidefinite relaxation (SDR) technique has been at the center of some of very exciting developments in the area of signal processing and communications, and it has shown great significance and relevance on a variety of applications. Roughly speaking, SDR is a powerful, computationally efficient approximation technique for a host of very difficult optimization problems. In particular, it can be applied to many nonconvex quadratically constrained quadratic programs (QCQPs) in an almost mechanical fashion, including the following problem: min x[r n x T Cx s.t. x T F i x$g i, i51,c, p, x T H i x5l i, i51,c, q, (1) where the given matrices C, F 1,c, F p, H 1,c, H q are assumed to be general real symmetric matrices, possibly indefinite. The class of nonconvex QCQPs (1) captures many problems that are of interest to the signal processing and communications community. For instance, consider the Boolean quadratic program (BQP) min x[r n x T Cx s.t. x i 2 5 1, i51,c, n. (2) The BQP is long known to be a computationally difficult problem. In particular, it belongs to the class of NP-hard problems. Nevertheless, being able to handle the BQP well has an enormous impact on multiple-input, multiple-output (MIMO) detection and multiuser detection. Another important yet NP-hard problem in the nonconvex QCQP class (1) is Digital Object Identifier /MSP IEEE SIGNAL PROCESSING MAGAZINE [20] MAY /10/$ IEEE Acknowledgment: Anthony So, Tom Luo, Yinyu Ye, & Shuzhong Zhang for co-authoring the magazine article; Qiang Li & Jiaxian Pan for helping prepare this tutorial note. Many thanks also go to everyone who has contributed to this powerful tool. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

3 Outline Part I: Basic concepts and overview of semidefinite relaxation (SDR) Part II: Theory, and implications in practice Part III: Applications A. MIMO detection B. Sensor network localization C. Transmit beamforming W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

4 Part I: Basic Concepts and Overview W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

5 A quick reminder of what convex quadratic functions & constraints are: A function f(x) = x T Cx = n i=1 n j=1 x ix j C ij is convex if and only if C 0 (C 0 means that C is positive semidefinite (PSD)) f(x) 10 5 f(x) x x x x (a) C 0. (b) C 0. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

6 A constraint set {x R n x T Fx 1} is convex if and only if F (a) F 0. (b) F 0. A constraint set {x R n x T Fx = 1} is nonconvex. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

7 Quadratically Constrained Quadratic Program Consider the class of real-valued quadratically constrained quadratic programs (QCQPs): x T Cx min x R n s.t. x T F i x g i, i = 1,...,p, x T H i x = l i, i = 1,...,q, where C,F 1,...,F p,h 1,...,H q S n ; S n is the set of all n n real symmetric matrices. We do not consider convex cases, and C,F i,h i may be arbitrary. Nonconvex QCQP is a very difficult problem in general. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

8 Nonconvex QCQP: How Hard Could it Be? Consider the Boolean quadratic program (BQP) min x R n s.t. x 2 i x T Cx = 1, i = 1,...,n, a long-known difficult problem falling in the nonconvex QCQP class. 2 You could solve it by evaluating all possible combinations; i.e., brute-force search. The complexity of a brute-force search is O(2 n ), not okay at all for large n! The BQP is NP-hard in general we still can t find an algorithm that can solve a general BQP in O(n p ) for any p > 0. x x 1 W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

9 Nonconvex QCQP: How Hard Could it Be? Consider the following problem min x R n x T Cx s.t. x T F i x 1, i = 1,...,m, where C,F 1,...,F m are all positive semidefinite, or C,F 1,...,F m Difficulty: feasible set is the intersection of the exteriors of ellipsoids. This problem is also NP-hard. x x 1 W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

10 Semidefinite Relaxation for QCQP Semidefinite relaxation (SDR) is a computationally efficient approximation approach to QCQP. Approximate QCQPs by a semidefinite program (SDP), a class of convex optimization problems where reliable, efficient algorithms are readily available. The idea can be found in an early paper of Lovász in 1979 [Lovász 79]. It is arguably the work by Goemans & Williamson [Goemans-Williamson 95] that sparked the significant interest in SDR. A key notion introduced by Goemans & Williamson is randomization; we will go through that. SDR has received much interest in the optimization field; now we have seen a number of theoretically elegant analysis results. (This may concern us more) In many applications, SDR works well empirically. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

11 Impacts of SDR in SP and Commun. The introduction of SDR in SP and commun. since the early 2000 s has reshaped the way we see many topics today. Applications identified include multiuser/mimo detection [Tan-Rasmussen 01], [Ma-Davidson-Wong- Luo-Ching 02] multiuser downlink tx beamforming: unicast [Bengtsson-Ottersten 01], multicast [Sidiropoulos-Davidson-Luo 06], &, more recently, multicell downlinks, relaying (incl. analog network coding), cognitive radio, secrecy... sensor network localization [Biswas-Liang-Wang-Ye 06] robust blind receive beamforming [Ma-Ching-Vo 04] code waveform design in radar [De Maio et al. 08] transmit B 1 shim in MRI [Chang-Luo-Wu et al. 08] fusion for distributed detection [Quan-Ma-Cui-Sayed 10] binary image restoration, phase unwrapping large-margin parameter estimation in speech recognition [Li-Jiang 07]... and the scope of applications is still expanding. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

12 The Concept of SDR For notational conciseness, we write the QCQP as min x R n x T Cx s.t. x T A i x i b i, i = 1,...,m. (QCQP) Here, i can represent either, =, or for each i; C,A 1,...,A m S n ; and b 1,...,b m R. A crucial first step of understanding SDR is to see that x T Cx = Tr(x T Cx) = Tr(Cxx T ), x T A i x = Tr(x T A i x) = Tr(A i xx T ), or, if we let X = xx T, x T Cx = Tr(CX), x T A i x = Tr(A i X). The objective and constraint functions are linear in X. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

13 The Concept of SDR The condition X = xx T is equivalent to X 0, rank(x) = 1, thus (QCQP) is the same as min X S n Tr(CX) s.t. Tr(A i X) i b i, i = 1,...,m X 0, rank(x) = 1. (QCQP) The constraints Tr(A i X) i b i are easy, but rank(x) = 1 is hard. Key Insight: Drop the rank-one constraint to obtain a relaxed QCQP min X S n Tr(CX) s.t. Tr(A i X) i b i, i = 1,...,m, X 0. (SDR) (SDR) is a convex problem. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

14 The SDR Some Merits We Can Immediately Say min X S n Tr(CX) (SDR) s.t. X 0, Tr(A i X) i b i, i = 1,...,m is a semidefinite program (SDP), whose globally optimal solution may be found by available numerical algorithms in polynomial time (often by interior-point methods, in O(max{m,n} 4 n 1/2 log(1/ǫ)), ǫ being soln. accuracy). For instance, using the software toolbox CVX, we can solve (SDR) in MATLAB with the following lines: (for simplicity we assume i = for all i here) cvx begin variable X(n,n) symmetric minimize(trace(c*x)); subject to for i=1:m trace(a(:,:,i)*x) >= b(i); end X == semidefinite(n) cvx end W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

15 Issues with the Use of SDR There is no free lunch in turning the NP-hard(QCQP) to the convex, polynomialtime solvable (SDR). The issue is how to convert an SDR solution to an approximate QCQP solution. If an SDR solution, say, denoted by X, is of rank one; or, equivalently, X = x x T, then x is feasible and in fact optimal to (QCQP). But the case of rank-one SDR solutions does not always hold (otherwise we would have solved an NP-hard problem in polynomial time!). There are many ways to produce an approximate QCQP solution from X, for instances where rank(x ) > 1. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

16 QCQP Solution Approximation in SDR: An Example Consider again the BQP min x T Cx s.t. x 2 (BQP) i = 1, i = 1,...,n. The SDR of (BQP) is min Tr(CX) s.t. X 0, X ii = 1, i = 1,...,n. (SDR) An intuitively reasonable idea (true even for engineers) is to apply a rank-1 approximation to the SDR solution X : 1) Carry out the eigen-decomposition X = r i=1 λ iq i q T i, where r = rank(x ), λ 1 λ 2... λ r > 0 are the eigenvalues and q 1,...,q r R n the respective eigenvectors. 2) Approximate the BQP by ˆx = sgn( λ 1 q 1 ). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

17 Application: MIMO Detection Scenario: A spatial multiplexing system with M t transmit & M r receive antennae. Symbols s C Spatial Multiplexer MIMO Detector Detected Symbols MIMO channel H C Objective: detect symbols from the received signals, given channel information. Received signal model: y C = H C s C +v C where H C C M r M t is the MIMO channel, s C C M t is the transmitted symbol vector, & v C C M r is complex circular Gaussian noise. Assume QPSK constellations, s C {±1±j} M t. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

18 Problem: maximum-likelihood (ML) detection (NP-hard) ŝ C,ML = arg min s C {±1±j} M t y C H C s C 2. The received signal model can be converted to a real form [ ] Re{yC } Im{y C } }{{} y = [ ] [ ] Re{HC } Im{H C } Re{sC } Im{H C } Re{H C } Im{s }{{} C } }{{} H s {±1} 2M t + [ ] Re{vC } Im{v C } }{{} v, and hence the ML problem can be rewritten (homogenized) as min s {±1} 2M t y Hs 2 = min ty Hs 2 s {±1} 2M t,t {±1} = min s {±1} 2M t,t {±1} [ s T t ][ H T H H T y y T H y 2 ][ ] s, t which is a BQP. Subsequently, SDR can be applied [Tan-Rasmussen 01], [Ma-Davidson-Wong-Luo-Ching 02]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

19 Bit Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF SDR, with randomization SDR, with rank 1 approx. performance lower bound SNR, in db Bit error rate performance under (M r,m t ) = (40,40). ZF zero forcing; MMSE-DF min. mean square error with decision feedback; LRA lattice reduction aided. Randomization will be explained shortly. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

20 Additional Remarks about the MIMO Detection Application The idea is not restricted to spatial multiplexing! It can also be used in multiuser CDMA, space-time/freq./time-freq. coding, multiuser MIMO, and even blind MIMO [Li-Bai-Ding 03], [Ma-Vo-Davidson-Ching 06],... Extensions that have been considered: MPSK constellations [Ma-Ching-Ding 04]; higher-order QAM constellations [Ma-Su-Jaldén-Chang-Chi 09] (and refs. therein); soft-in-soft-out MIMO detection (a.k.a. BICM-MIMO) [Steingrimsson-Luo- Wong 03]; fast implementations [Kisialiou-Luo-Luo 09], [Wai-Ma-So 11]. Performance analysis for SDR MIMO detection: diversity analysis [Jaldén-Ottersten 08] probabilistic approximation accuracy analysis [Kisialiou-Luo 10], [So 10]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

21 Alternative Interpretation of SDR: Solving QCQP in Expectation We return to the SDR solution approximation issue. Recall min x R n x T Cx s.t. x T A i x i b i, i = 1,...,m. (QCQP) Let ξ N(0,X) where X is the covariance. Consider a stochastic QCQP: min X S n, X 0 E ξ N(0,X) {ξ T Cξ} s.t. E ξ N(0,X) {ξ T A i ξ} i b i, i = 1,...,m, (E-QCQP) where we manipulate the statistics of ξ so that the objective function is minimized & constraints are satisfied in expectation. One can show that (E-QCQP) is the same as the SDR min Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. (SDR) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

22 The stochastic QCQP interpretation of SDR min X S n E ξ N(0,X) {ξ T Cξ} s.t. E ξ N(0,X) {ξ T A i ξ} i b i, i = 1,...,m (E-QCQP) essentially sheds lights into a different way of approximating QCQP. What we could do is the following: generate a random vector ξ N(0,X ) (X is an SDR soln.), and modify ξ so that it is QCQP-feasible. Such a randomized QCQP soln. approx. may be performed multiple times, to get a better approx. (Believe it or not) The stochastic QCQP interpretation is the intuition behind many important theoretical SDR approx. accuracy results, including the famous Goemans-Williamson result [Goemans-Williamson 95]. (to be covered in Part II) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

23 Example: Randomization in BQP or MIMO Detection A simple (and very important) example for illustrating randomizations is BQP: min x T Cx s.t. x 2 (BQP) i = 1, i = 1,...,n. Box 1. Gaussian Randomization Procedure for BQP given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l N(0,X ), and construct a feasible point x l = sgn(ξ l ). end determine l = arg min l=1,...,l xt l C x l. output ˆx = x l as an approximate solution to (BQP). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

24 Bit Error Probability L= 1 L= 5 L= 20 L= 50 L= 80 L= 120 L= SNR, in db Performance of various no. of randomizations in MIMO detection. M t = M r = 40. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

25 Complex-valued QCQP and SDR Consider a general complex-valued QCQP min x C n x H Cx s.t. x H A i x i b i, i = 1,...,m, (1) where C,A 1,...,A m H n ; H n denotes the set of n n Hermitian matrices. Using the same idea, SDR can be derived for complex-valued QCQP: min X H n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. The only difference is that the problem domain now is H n (change symmetric to hermitian in your CVX code). Note that while the ideas leading to real and complex SDRs are the same, their performance may be different (will elaborate upon later). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

26 Application: Multicast Transmit Beamforming Scenario: Common information broadcast in multiuser MISO downlink, assuming channel state information at the transmitter (CSIT). The transmit signal: x(t) = ws(t), User 1 where s(t) C is the tx. data stream, & w C N t is the tx. beamvector. Received signal for user i: y i (t) = h H i x(t)+v i (t), where h i C N t is the channel of user i, & v i (t) is noise with variance σ 2 i. Basestation User 2 W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

27 Consider a QoS-assured design: min w C N t w 2 s.t. SNR i γ i, i = 1,...,K, where each γ i is a prescribed SNR requirement for user i, and R i = { SNR i = E{ h H i ws(t) 2 }/σ 2 i = w H R i w/σ 2 i, h i h H i, h i is available (instant CSIT), E{h i h H i }, h i is random with known 2nd order stat. (stat. CSIT). The design problem can be rewritten as a complex-valued QCQP where A i = R i /γ i σ 2 i. min w 2 s.t. w H A i w 1, i = 1,...,K, This multicast problem is NP-hard in general, but can be approximated by SDR [Sidiropoulos-Davidson-Luo 06]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

28 A Randomization Example Relevant to Multicast Beamforming Consider the problem min x H Cx s.t. x H A i x 1, i = 1,...,m, ( ) where C,A 1,...,A m 0. Box 2. Gaussian Randomization Procedure for ( ) given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l CN(0,X ), and construct a feasible point x l = ξ l min i=1,...,m ξ H l A iξ l end determine l = arg min l=1,...,l xh l C x l. output ˆx = x l as an approximate solution to ( ). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

29 0.6 random samples ξ QCQP approx. samples x(ξ) globally opt. QCQP soln. x x x Illustration of randomizations in R 2, for Problem ( ). The gray area is the feasible set and colored lines the contour of the objective. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

30 H=randn(4,8)+j*randn(4,8) outcomes mean 1.6 v(wl)/vsdr Randomization index, l Approximation accuracy of Gaussian randomization in multicast beamforming. N t = 4, K = 8, v(w) = w 2 is the objective value, v SDR is the optimal value of SDR. Note that for any feasible w, v(w)/v SDR v QP /v SDR where v QP is the optimal value of QCQP. Courtesy to T.-H. Chang and Z.-Q. Luo. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

31 Extension to Complex-Valued Separable QCQP Consider a further extension, called complex-valued separable QCQP: min x 1,...,x k C n s.t. k i=1 xh i C ix i k l=1 xh l A i,lx l i b i, i = 1,...,m. By writing X i = x i x H i for all i, and then semidefinite-relaxing them, we obtain an SDR min k X 1,...,X k H n i=1 Tr(C ix i ) k s.t. l=1 Tr(A i,lx l ) i b i, i = 1,...,m, X 1 0,...,X k 0. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

32 Application: Unicast Transmit Downlink Beamforming Scenario: multiuser MISO downlink; each user receives an individual data stream. Transmit signal: User 1 x(t) = K w i s i (t), i=1 where s i (t) C is the data stream for user i, & w i C N t its tx. beamvector. Received signal of user i: Basestation User 2 y i (t) = h H i x(t)+v i (t) = h H i w i s i (t)+ h H i w l s l (t) +v i (t). l i } {{ } interference W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

33 The signal-to-interference-and-noise ratio (SINR) of user i: SINR i = w H i R iw i l i wh l R iw l +σi 2, where R i = h i h H i for instant. CSIT, and R i = E{h i h H i } for stat. CSIT. Consider the QoS-assured design: K min w 1,...,w K C N i=1 w i 2 t w H i s.t. R iw i l i wh l R iw l +σi 2 γ i, i = 1,...,K and its SDR K min W 1,...,W K H N i=1 Tr(W i) t s.t. Tr(R i W i ) γ i ( l i Tr(R iw l )+σi 2 ), i = 1,...,K, W 1,...,W K 0. ( ) ( ) ( )isshowntohavearank-onesolutionforr 1,...,R K 0, viauplink-downlink duality [Bengtsson-Ottersten 01]; SDR is optimal to ( ), so to speak! Part II will introduce an easy way to identify rank-one SDR instances. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

34 Additional Remarks about the Transmit Beamforming Application Transmit beamforming is now a key topic; see [Gershman-Sidiropoulos- Shahbazpanahi-Bengtsson-Ottersten 10], [Luo-Chang 10] for review. Apart from standard transmit beamforming, we have seen numerous extensions: one-way relay beamforming [Fazeli-Dehkordy-Shahbazpanahi-Gazor 09], [Chalise-Vandendorpe 09]; two-way relay beamforming (a.k.a. analog network coding) [Zhang-Liang- Chai-Cui 09]; cognitive radio beamforming [Zhang-Liang-Cui 10]; multicell coordinated beamforming [Bengtsson-Ottersten 01], [Dahrouj- Yu 10]; secrecy beamforming [Liao-Chang-Ma-Chi 10], [Li-Ma 11]. Interestingly, all these beamforming problems turn out to be, or be closely related to, nonconvex QCQPs. And, as it turns out, SDR plays a key role. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

35 SDR Versus Nonlinear Programming: They complement, not compete Since SDR is an approximation method, as an alternative one may choose to approximate (QCQP) by a nonlinear programming method (NPM) (like, SQP in the MATLAB Optimization Toolbox). So should we compare SDR and NPM? The interesting argument is that they complement each other, instead of competing: An NPM depends much on a good starting point, and that s usually the missing piece. To SDR, NPMs may serve as a local refinement of the solution. One may consider a two-stage approach where SDR is used as a starting point for NPMs. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

36 Application: Transmit B 1 Shim in MRI In MRI, a transmit RF coil array is used to generate a B 1 field. Load... RF Coils An undesirable effect is that the B 1 field exhibits strong inhomogeneity (spatial non-uniformity) across the load, due to complex interactions between the magnetic field and the loaded tissues. The goal is to design the transmit amplitudes and phases of the RF coils such that the resultant B 1 map is as uniform as possible. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

37 Load... RF Coils Let a i C n, i = 1,...,m, be the field response from the array to the ith pixel (MISO); i.e., the ith pixel receives a B 1 field of magnitude a T i x. The design problem is to minimize the worst-case field magnitude difference a T i x 2 b 2 min x C n max i=1,...,m s.t. x H Gx ρ. Here, x C n is the transmit vector of the RF coil array, m is the total no. of pixels, b > 0 is the desired pixel value (uniform over all pixels), x H Gx is an average specific absorption rate (SAR), and ρ is a pre-specified SAR limit. It can be approximated by SDR [Chang-Luo-Wu et al. 08]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

38 without optimization nonlinear prog. with random starting point, realization 1 nonlinear prog. with random starting point, realization 2 SDR with randomization, realization 1 SDR with randomization, realization 2 two-stage opt.: SDR w/ rand. +nonlinear prog., realization 1 two-stage opt.: SDR w/ rand. +nonlinear prog., realization 2 (a) (b) (c) (d) (e) (f) (g) obj. value= obj. value= obj. value= obj. value= obj. value= obj. value= obj. value= B 1 maps of various optimization methods. You can see that the two-stage, SDR+NPM method shows better solution fidelity. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

39 Part II: Theory W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

40 Provable Approximation Accuracies: Motivation So far we have introduced several procedures for generating an approximate QCQP solution from an SDR solution. A natural question arises: How good are these procedures? Of course, their performance can be observed empirically. However, can we prove something about their approximation accuracy? Such theoretical results can provide strong justification for the use of SDR in various problem settings. The SDR approximation accuracy analysis has been of great interest to optimization theorists, and it has enormous implications in practice. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

41 Provable Approximation Accuracies: Setup Let v(x) = x T Cx, and denote the optimal values of (QCQP) and (SDR) by v QP = min x T Cx s.t. x T A i x i b i, i = 1,...,m; v SDR = min Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. Moreover, let ˆx be an approximate solution to (QCQP), obtained using one of the solution generation procedures (e.g., randomization). Note that v(ˆx) v QP. The problem is to prove a constant γ 1 (called the approx. ratio) such that v(ˆx) γv QP either in expectation, or with high probability, or almost surely (since ˆx can be random). In general, the smaller γ, the better the solution generation procedure. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

42 Provable Approximation Accuracies: Remarks In the definition of approximation ratio, we are implicitly assuming that v QP,v SDR > 0. Given a solution generation procedure, we are usually interested in its performance on arbitrary instances of (QCQP). Thus, the approximation ratio γ should not depend on the problem data {A 1,...,A m,b,c}. However, it could depend on the problem dimensions m, n. For quadratic maximization problems, the notion of approximation ratio can be defined similarly. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

43 The Seminal Approx. Accuracy Result by Goemans & Williamson Consider v QP = max x T Cx x R n s.t. x 2 i = 1, i = 1,...,n with C 0, C ij 0 for all i j (the so-called MAXCUT in combinatorial optimization). In [Goemans-Williamson 95], it was shown that if the randomization procedure in Box 1 is used, then γv QP E{v(ˆx)} v QP, where γ In particular, the approximation ratio is independent of the problem dimension n. In the context of MAXCUT, this means that the approximation accuracy is independent of the number of vertices in the graph. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

44 Complex k-ary Quadratic Maximization Consider the problem v QP = max x H Cx x C n s.t. x i {1,ω,...,ω k 1 }, i = 1,...,n, (CQP-k) where C 0 and ω = exp(j2π/k) is the kth root of unity, for some given integer k 2. This is a generalization of the problem considered by Goemans and Williamson. Since x i 2 = 1 for all i, (CQP-k) can be handled by SDR. Specifically, v SDR = max Tr(CX) X H n s.t. X 0, X ii = 1, i = 1,...,n. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

45 Randomization Procedure for Complex k-ary Quad. Max. Again, a Gaussian randomization procedure can be used to generate a feasible solution to (CQP-k) from an SDR solution. Box 3. Gaussian Randomization Procedure for CQP-k given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l CN(0,X ), and construct the feasible point x l C n, where [ x l ] i = f([ξ l ] i ) and f(z) = 1, arg(z) [ π/k, π/k), ω,. arg(z) [π/k, 3π/k),. ω k 1, arg(z) [(2k 3)π/k,(2k 1)π/k). end determine l = arg max l=1,...,l xh l C x l. output ˆx = x l as the approximate solution to (CQP-k). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

46 Pictorial Illustration of the Randomization Procedure, for k = W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

47 Approx. Accuracy Result for Complex k-ary Quad. Max. In [So-Zhang-Ye 07], it is shown that if the randomization procedure in Box 3 is used, then γv QP E{ˆx H Cˆx} v QP, where γ = (ksin(π/k))2. 4π If we take k =, then the k-ary constraints in (CQP-k) become x i = 1, i = 1,...,n. ( ) In [So-Zhang-Ye 07] it is shown that by letting the function f in Box 3 to be f(z) = { z/ z, z > 0, 0, z = 0, the randomization procedure would yield γ = π/4 for the unit-modulus constraints ( ). It is interesting (and comforting) to note that lim k (ksin(π/k)) 2 4π = π 4. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

48 Applications of Complex k-ary Quadratic Maximization (CQP-k) has many applications in signal processing, e.g.: blind orthogonal space-time block code detection [Zhang-Ma 09] radar code waveform design [De Maio et al. 09] W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

49 Boolean QP max x R n Complex k-ary QP problem approx. accuracy γ; see (21)-(22) for def. references x T Cx s.t. x 2 i = 1, i = 1,..., n γ = , C 0, C ij 0 i j 2/π , C 0 1 (opt.), C ij 0, i j Goemans-Williamson [2], Nesterov [3], Zhang [6]. Relevant applications: [24] [26] max x C n x H Cx s.t. x i {1, ω,..., ω k 1 }, i = 1,..., n where ω = e j2π/k, and k > 1 is an integer. Complex constant-modulus QP max x C n s.t. x H Cx x i 2 = 1, i = 1,..., n For C 0, γ = (k sin(π/k))2. 4π e.g., γ = for k = 8, γ = for k = 16. For C 0, γ = π/4 = Remark: coincide with complex k-ary QP as k. Zhang-Huang [7], So-Zhang-Ye [8]. Relevant applications: [27], [37] Zhang-Huang [7], So-Zhang-Ye [8]. max x C n s.t. x H Cx ( x 1 2,..., x n 2 ) F where F R n is a closed convex set. max x R n s.t. x T Cx where A 1,..., A m 0. x T A i x 1, i = 1,..., m The same approx. ratio as in complex constant-modulus QP; i.e., γ = π/4 for C 0. If the problem is reduced to the real-valued case, then the approx. ratio results are the same as that in Boolean QP. For any C S n, γ = 1 2ln(2mµ) where µ = min{m, max i rank(a i )}. Ye [4], Zhang [6]. Nemirovski-Roos-Terlaky [5]. Extensions: Ye [72], Luo-Sidiropoulos- Tseng-Zhang [9] and So-Ye- Zhang [71]. Known approximation accuracies for quadratic maximization problems. The reference numbers refer to those in our Signal Processing Magazine article. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

50 Approx. Accuracy Result for Quadratic Minimization Consider now the problem v QP = min x T Cx x R n s.t. x T A i x 1, i = 1,...,m ( ) for C,A 1,...,A m 0, which arises in the study of multicast downlink beamforming. It was shown in [Luo-Sidiropoulos-Tseng-Zhang 07] that if the randomization procedure in Box 2 is used, then with high probability (instead of just in expectation), v QP v(ˆx) γv QP, where γ = 27m 2 /π. For the complex version of ( ), one has a better approximation ratio: γ = 8m. Notice that this ratio accommodates the worst possible problem instance {C,A 1,...,A m }. In practice, the approximation accuracies are usually much better a phenomenon that deserves further investigation. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

51 Interpretation in Multicast Transmit Beamforming Recall that in the context of multicast transmit beamforming, we encounter the following optimization problem: min w C N w 2 s.t. SNR i = 1 γ i σ 2 i w H R i w 1, i = 1,...,K, which is an instance of the quadratic minimization problem considered in [Luo- Sidiropoulos-Tseng-Zhang 07]. The aforementioned approximation accuracy result thus says that SDR together with the randomization procedure can produce a transmit beamforming vector that satisfies all the prescribed SNR requirements and whose power is at most 8m times the optimal. Again, this is just a worst-case guarantee. In practice, the performance is usually much better. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

52 min x C n s.t. x H Cx where A 1,..., A m 0. MIMO Detection problem approx. accuracy γ; see (18)-(19) for def. references x H A i x 1, i = 1,..., m min x R n y Hx 2 2 s.t. = 1, i = 1,..., n x 2 i where y = Hs+v; H C n n has i.i.d. standard complex Gaussian entries; s 2 i = 1 for i = 1,..., n; and v C n has i.i.d. complex mean zero Gaussian entries with variance σ 2. γ = 8m. If the problem is reduced to the real-valued case, then γ = 27m2 π. For σ 2 60n (which corresponds to the low signal-to-noise ratio (SNR) region), with probability at least 1 3 exp( n/6), γ For σ 2 = O(1) (which corresponds to the high SNR region), with probability at least 1 exp( O(n)), i.e. the SDR is tight. γ = 1, Luo-Sidiropoulos-Tseng-Zhang [9]; see also So-Ye-Zhang [71]. Relevant applications: [29] Kisialiou-Luo [67], So [69]. Extensions: So [68], [69]. Related: Jaldén-Ottersten [66]. Relevant applications: [17] [20], [22], [23] Known approximation accuracies for quadratic minimization problems. The reference numbers refer to those in our Signal Processing Magazine article. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

53 Rank Reduction in SDR Now you may notice that an SDR methodology basically has the following steps: 1) formulate a hard problem (nonconvex QCQP) as a rank-one-constrained SDP 2) remove the rank constraint to obtain an SDP 3) use some methods, such as randomizations, to produce an approximate solution to the original problem. Apparently, the lower the rank of the SDP solution, the better the approximation we would expect. Unfortunately, we cannot guarantee a low rank solution for the SDP in general. But we can identify special cases where the SDP solution rank is low, and, sometimes, even equal to one. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

54 Shapiro-Barvinok-Pataki (SBP) Result Consider the real-valued SDP (or SDR) min X S n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. (SDR) SBP Result [Pataki 98]: There exists an optimal solution X such that rank(x )(rank(x )+1) 2 m. In particular, SBP result implies that for m 2, a rank-1 X exists. Hence, For a real-valued QCQP with m 2, SDR is tight; i.e., solving the SDR is equivalent to solving the original QCQP. Note that a rank reduction algorithm may be required to turn an SDP solution to a rank-one solution [Ye-Zhang 03]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

55 Complex Extension of the Rank Reduction Result Let us consider the extension to the complex-valued SDP min X H n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. In this case, the SBP result can be generalized to [Huang-Zhang 07] and the direct consequence is that rank(x ) 2 m, For a complex-valued QCQP with m 3, SDR is tight; i.e., solving the SDR is equivalent to solving the original QCQP. A complex rank-1 decomposition algorithm for m 3 is available [Huang- Zhang 07]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

56 Application Revisited: Multicast Beamforming User 1 Recall the multicast beamforming problem: min w C N t w 2 s.t. SNR i = wr iw σ 2 i i = 1,...,K, γ i, K being the number of users. Basestation User 2 By the SBP result, SDR solves the multicast problem optimally for K 3. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

57 Further Extension of the Rank Reduction Result Recall the problem min k X 1,...,X k H n i=1 Tr(C ix i ) k s.t. l=1 Tr(A i,lx l ) i b i, i = 1,...,m, X 1 0,...,X k 0, which is an SDR of the so-called separable QCQP. A generalization of the SBP result [Huang-Palomar 09]: k i=1 rank(x i )2 m, and, as a subsequent result: Suppose that an SDR solution cannot have X i is tight for m k +2. = 0 for any i. Then SDR W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

58 Application Revisited: Unicast Beamforming User 1 Recall the design problem min w 1,...,w K C N t s.t. K i=1 w i 2 w H i R iw i l i wh l R iw l +σi 2 i = 1,...,K, γ i, ( ) Basestation User 2 which is a separable QCQP with K variables (beamvectors) and K constraints (SINR req.). By the SBP result, SDR solves ( ) optimally for any R 1,...,R K, regardless of R i 0 or not. And hey, it s still fine if you put two more quadratic constraints in ( )! W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

59 Further Results in SDR Rank Reduction: Motivation As we have seen before, it is not possible in general to find a low-rank solution to a QCQP in polynomial time, unless the number of constraints is small. In fact, most polynomial-time interior-point algorithms will return a solution that has the highest rank. In such situations, can we find a low-rank solution that has good approximation accuracy? The randomization procedure introduced before can generate a rank-one solution with good approximation accuracy. How about a rank-r solution, where r 2 is fixed and small? This has applications in sensor network localization, which will be covered later. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

60 Further Results in SDR Rank Reduction: Setup Consider the following (real-valued) rank-constrained semidefinite feasibility problem (RCSDF): find X 0, rank(x) r s.t. Tr(A i X) = b i, i = 1,...,m, (RCSDF) where A 1,...,A m 0, b 1,...,b m 0 and r 1 are given. What is the relationship between (RCSDF) and the QCQPs we studied before? For instance, consider the QCQP min x T Cx s.t. x T A i x 1, i = 1,...,m, where C,A 1,...,A m 0. Let x be an optimal solution. Then, X = x x T is feasible for the following RCSDF problem: find X 0, rank(x) 1 s.t. Tr(A i X) = Tr(A i X ), i = 1,...,m, Tr(CX) = Tr(CX ). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

61 Further Results in SDR Rank Reduction: Setup There are two difficulties in solving (RCSDF): There is no known polynomial-time algorithm for solving (RCSDF) in general. The problem may be infeasible. It is thus natural to study easier versions of (RCSDF). For instance, consider find X 0, rank(x) r s.t. α b i Tr(A i X) β b i, i = 1,...,m, (RCSDF-R) where α, β are some parameters. Note that when α = β = 1, we get back (RCSDF). Question: Under what conditions on α and β would (RCSDF-R) be feasible, and a feasible solution can be found efficiently? W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

62 SDR Rank Reduction: The So-Ye-Zhang (SYZ) Theorem Suppose there exists an X 0 such that Tr(A i X ) = b i for i = 1,...,m. [So-Ye-Zhang 08] One can find in randomized polynomial time a rank-r ˆX such that α b i Tr(A i ˆX) β bi for i = 1,...,m with high probability, where α = β = 1 e(2m) 2/r, 1 r 4ln(2m), { } 1 4ln(2m) max e(2m) 2/r, 1, r > 4ln(2m), r 1+ 12ln(4mq), 1 r 12ln(4mq), r 12ln(4mq) 1+, r > 12ln(4mq), r q = min{ 2m,n}. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

63 Some quick remarks: The larger the r, the closer α,β are to 1. Intuitively, r dictates how much information about X we can keep. From the definition of q, it is clear that the bounds can be made independent of n, the dimension of the matrices. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

64 Randomization Procedure for SDR Rank Reduction How to achieve the claimed bounds? (Surprise) Use Gaussian randomization! Box 4. Gaussian Randomization Procedure for Rank Reduction given a solution X that satisfies Tr(A i X ) = b i for all i and X 0, and an integer r 1. for l = 1,...,r generate ξ l N(0,X ); end output ˆX = 1 r ξ l ξ T l as the candidate solution. r l=1 Why this works? Intuitively, ˆX 0, rank( ˆX) r, E{ ˆX} = X. This extends our previous rank-1 Gaussian randomization procedures. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

65 Applications of the SYZ Theorem The SYZ theorem has applications in sensor network localization (new!) rank-2 multicast beamforming via Alamouti space-time coding [Wu- So-Ma 12] W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

66 Part III.A: MIMO Detection W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

67 Introduction MIMO detection is an important topic with a wide scope of applicability. The goal is to achieve good symbol error probability performance, preferably near-optimal, in a computationally efficient manner. Note that SDR is not the only efficient high-performance MIMO detection approach. The sphere decoding approach and the lattice reduction aided (LRA) approach are also powerful. Our focus: computational or implementation aspects of SDR; alternative interpretations of SDR; connections to other MIMO detectors; SDR for various types of constellations (we went thro {±1} so far); benchmarking SDR and representative MIMO detectors, through extensive simulation results. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

68 Problem Statement Consider a generic complex-valued M C N C MIMO model where y C = H C s C +v C, H C C M C N C the MIMO channel; s C S N C the tx symbol vector, with S C being the constellation set; v C C M C complex AWGN. We will focus on the ML detection problem Constellations: ŝ C,ML = arg min s C S N C y C H C s C 2. QPSK: S = { s = a+jb a,b {±1} } M-ary PSK (MPSK): S = { s = e j2πk/m k = 0,1,...,M 1 } 4 q -ary QAM: S = { s = a+jb a,b {±1,±3,...,±(2 q 1 1)} } W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

69 Scope of Applicability The simple MIMO model y C = H C s C +v C is popularly used in the point-topoint spatial multiplexing scenario. Symbols s C Spatial Multiplexer MIMO Detector Detected Symbols MIMO channel H C Actually, this MIMO model is general enough to cover a wide variety of digital communication scenarios. As such, MIMO detection methods developed for the generic MIMO model can be universally applied to many different scenarios. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

70 Example: CDMA Multiuser Detection User 1 Base station User 2 User Consider a multiuser CDMA scenario. rx signal model over one symbol interval: y = K c i α i s i +v, i=1 where y C N is the rx code vector; c i C N spreading code sequence vector of user i; s i tx symbol of user i; α i C rx amplitude/phase coefficient of user i. can be rewritten as y C = H C s C +v C (obviously). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

71 Example: Space-Time Block Coding Consider a point-to-point space-time block code (STBC) scenario: Y = H C C(s C )+V, where H C C M r M t the MIMO channel; C(s C ) C M t T is an STBC; Y C M r T is the rx space-time code block, T being the time length. Assume a linear dispersion STBC: C(s C ) = L A l Re{s C,l }+B l Im{s C,l }. l=1 The rx model can be converted to the generic MIMO form: vec(y ) = (I H C )X s+vec(v ), }{{} another H C where X = [ vec(a 1 ),...,vec(a L ),vec(b 1 ),...,vec(b L ) ] C M tt 2L, s = [ Re{s C } T,Im{s C } T ] T R 2L. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

72 Example: Space-Time Frequency Coding Scenario: point-to-point MIMO OFDM in the presence of frequency selective multipath channels. subcarrier subcarrier Tx Rx 1 Space- Frequency Block Code Encoder Tx Rx Space- Frequency Block Code Decoder Goal: precode across space and frequency, to harvest space and multipath diversity, esp., full space-multipath diversity. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

73 Let us have a case study on the algebraic space-frequency code (SFC) scheme [Su-Safar-Liu 05]. Subcarrier Transmit Antenna... Tx 1 Transmit Antenna Tx 2 Tx Operations: Subcarrier Subcarriers are partitioned into groups; In each group, symbols are precoded; Precoded symbols (x above) are appropriately interleaved in space and frequency. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

74 Assume one rx antenna, for ease of illustration. The rx signal model in each group can be represented by y = D H x+v, x = Θs, where x C ΓM t the precoded symbol vector; Θ C ΓM t ΓM t the precoder matrix; s S ΓM t the tx symbol vector; D H C ΓM t ΓM t a diagonal matrix whose diagonals contain channel freq. responses (dependent on the SFC interleaving pattern). A properly designed Θ can lead to full space-multipath diversity d = M t L, where M t is the no. of tx antennas & L is the no. of multipaths. To do so, one should choose Γ L. The rx model can again be written as the generic form y C = H C s C +v C. Note that the problem size in this example, ΓM t, may be large. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

75 Efficient High-Performance Approaches other than SDR Sphere decoders [Mow 92], [Viterbo-Biglieri 93], [Damen-El-Gamal- Caire 03]: An exact ML solver based on branch and bound, or tree search; Empirical experience with its runtime performance: very fast for high SNRs and small to moderate problem sizes N C ; can be (very) slow otherwise; Exponential complexity w.r.t. the problem size [Jaldén-Ottersten 05]. Lattice reduction aided (LRA) detectors [Yao-Wornell 02], [Wübben- Seethaler-Jaldén-Matz 11]: Use lattice reduction to improve the channel conditioning ; Interface well with linear and decision feedback detectors; Exhibit good diversity or diversity multiplexing tradeoff performance [Taherzadeh-Mobasher-Khandani 07], [Jaldén-Elia 10]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

76 Inhomogeneous QCQPs and SDR Consider a general inhomogeneous QCQP min x R n xt Cx+2c T x s.t. x T A i x+2a T i x i b i, i = 1,...,m. An inhomogeneous QCQP can be reformulated as a homogenous QCQP min x R n,t R [ x T t ][ ][ ] C c x c T 0 t s.t. t 2 = 1, and then handled by SDR. [ x T t ][ A i a i a T i 0][ x t ] i b i, i = 1,...,m W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

77 An Alternative Way to Derive SDR for Inhomogeneous QCQPs Recap of inhomogeneous QCQP: min x R n xt Cx+2c T x s.t. x T A i x+2a T i x i b i, i = 1,...,m. By letting X = xx T, and then by replacing it with we can derive an SDR X xx T, min X S n,x R n Tr(CX)+2cT x s.t. Tr(A i X)+2a T i x i b i, i = 1,...,m, X xx T. This inhomogeneous SDR is equivalent to the SDR from the homogenized [ ] QCQP X x formulation (last page), by Schur complement X xx T x T 0. 1 W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

78 SDR MIMO Detection for QPSK Constellations Let N = 2N C, M = 2M C, [ ] [ ] Re{yC } Re{sC } y =,s = Im{y C } Im{s C },v = [ ] Re{vC },H = Im{v C } [ ] Re{HC } Im{H C }. Im{H C } Re{H C } The complex-valued model y C = H C s C +v C can be turned to a real one y = Hs+v. where s {±1} N for QPSK constellations. ML detection problem: min y Hs 2 s RN s.t. s 2 i = 1, i = 1,...,N. SDR: min Tr(HT HS) 2s T H T y + y 2 S S N,s R N s.t. S ii = 1, i = 1,...,N, S ss T. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

79 Bit Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF SDR, with randomization Sphere decoding performance lower bound SNR, in db Bit error rate performance under (M C,N C ) = (10,10), QPSK constellations. The SNR is defined as E{ H C s C 2 } E{ v C 2. ZF zero forcing; MMSE-DF min. mean square error with decision } feedback; LRA lattice reduction aided; the Schnorr-Euchner sphere decoder is used. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

80 Bit Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF SDR, with randomization Sphere decoding performance lower bound SNR(dB) Bit error rate performance under (M C,N C ) = (20,20), QPSK constellations. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

81 Bit Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF SDR, with randomization SDR, with rank 1 approx. performance lower bound SNR, in db Bit error rate performance under (M C,N C ) = (40,40), QPSK constellations. It is too expensive to run sphere decoding in this example. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

82 Average Running Time, in Seconds 10 1 ZF LRA MMSE DF sphere decoding (Schnorr Euchner) 10 0 SDR, with randomization Problem Size N C Complexity comparison of various MIMO detectors. SNR= 12dB. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

83 Computational Efficiency of SDR MIMO Detection The bulk of complexity lies in solving the SDP. A common, arguably dominant, way to solve SDPs is to use the interior point methods (IPMs) their solution precision is good, & their complexities are provably polynomial-time in the problem size. For the SDP in QPSK SDR MIMO detection, an IPM can output a solution with a worst-case complexity of O((N +1) 3.5 log(ǫ 1 )) O(N 3.5 ), where ǫ > 0 is the desired solution accuracy. A few practical hints: You don t need a very small ǫ in MIMO detection, since you will round the solution anyway. While a general purpose software, such as CVX, can be used to solve the SDP conveniently, you d better off write your own IPM for maximizing the computational efficiency. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

84 Interior-Point Algorithm for SDR MIMO Detection The SDR problem in homogenous form: min Tr(CX) X S n [ H T H H T ] y y T H y 2, X = s.t. X 0, X ii = 1, i = 1,...,n, [ ] where S s C = s T. By exploiting its simple equality constraint 1 structure, a specialized (and fast) IPM can be derived [Helmberg-et al. 96]. given ǫ > 0, and strictly feasible X, y, and Z. repeat 1. update the barrier parameter µ := tr(zx)/2n. 2. compute y := [(Z 1 X)] 1 (µdiag(z 1 ) 1), Z := Diag( y) X := µz 1 X Z 1 ZX, X := ( X + X T )/2 3. find step-sizes α p (0,1] and α d (0,1] such that X+α p X 0 and Z+α d Z X := X + α p X, y := y + α d y, and Z := Z + α d Z. until tr(zx) ǫ. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

85 Cheap SDR by Row-by-Row Coordinate Descent While IPMs have good solution fidelity, they are generally not low complexity options (check out the IPM pseudo code last page). Low complexity SDR implementation has received much interest. A possible alternative is row-by-row (RBR) coordinate descent [Wen-Goldfarb- Ma-Scheinberg 09], [Wai-Ma-So 11]. Ready-to-use codes available at To describe RBR, consider a barrier SDR problem where σ > 0 is the barrier parameter. min Tr(CX) σlogdet(x) X S n s.t. X ii = 1, i = 1,...,n, (B-SDR) In (B-SDR), the log barrier function is used to enforce X 0 (more precisely, X 0), thereby avoiding to deal with the constraint X 0 explicitly. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

86 Let f(x) = Tr(CX) σlogdet(x), x i be the ith row of X, & X i be the collection of all elements of X except for x i. Idea of RBR: do a block coordinate descent on (B-SDR): given a starting point ˆX; repeat for i = 1,...,n ˆx i := arg min x i, X ii =1 f(x i, ˆX i ); end; until a stopping criterion is satisfied. The iterates are known to converge to the optimal solution of (B-SDR). Each per-row update is simple; e.g., the 1st row update can be equiv. written as min 1ξ 1 σlog(1 ξ T ˆX ξ 1 R n 12cT 1 2:n,2:nξ 1 ), ( ) where ξ 1 = [x 1 ] 2:n, c 1 = C 1,2:n. The soln. to ( ) is ξ 1 = κ ˆX 2:n,2:n c 1 for some κ, a simple closed form (matrix multiplication, no inverse)! W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

87 10 1 Bit Error Probability performance lower bound ZF LRA MMSE DFE SDR IPM SDR RBR (δ=1e 2) SNR, in db Bit error probability performance under (M C,N C ) = (40,40). Setting for RBR: σ = 10 2 /n; RBR terminates when f(k+1) f (k) δ, where f(k) is the objective value at iteration k. f (k) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

88 Average Running Time, in Seconds 10 1 ZF sphere decoding LRA MMSE DFE 10 0 SDR IPM SDR RBR (δ = 1e 2) SDR RBR (#Iterations =10) Problem Size N C Complexity of RBR. A tenfold runtime saving relative to IPM is observed. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

89 Other Relaxations for QPSK ML MIMO Detection Generally speaking, relaxation methods work by relaxing the original problem to a tractable problem. In that regard, relaxations other than SDR can be considered. Unconstrained relaxation (UR): The result is ZF. On-Sphere Relaxation (OSR): min y Hs 2. s R N min s 2 =N y Hs 2. The solution is ŝ OSR = (H T H +γi) 1 H T y for some γ; has an MMSE flavor. Box Relaxation (BR): min y Hs 2. s 2 i 1, i=1,...,n W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

90 Comparison of the Various Relaxations In order to compare, let f ML = min s {±1} N y Hs 2, f SDR = f BR = f OSR = min s 2 =N min Tr(H T HS) 2s T H T y + y 2, S ss T, S ii =1 i min y Hs 2, s 2 i 1, i=1,...,n y Hs 2, f UR = min s R N y Hs 2. It is shown that [Ma-Davidson-Wong-Luo-Ching 02], [Poljak-Rendl- Wolkowicz 95] max{f UR,f OSR,f BR} f SDR f ML. The result means that SDR provides a relaxation no worse than the other three methods. Or, the other methods may be seen as further relaxations of SDR. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

91 Regularization in LS Consider the least squares (LS) problem (for generic applications): min s R N y Hs 2. Sometimes, in order to make the problem better conditioned, we may turn to a regularized LS: min s R N y Hs 2 +s T Ts, for some regularizer T S N (common choice: T = ρi, ρ > 0). SDR can be interpreted as a regularized LS. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

92 A Regularized LS Perspective on SDR Consider a Lagrangian dual of ML, as an approx.: f ML g ML = max λ R N min s R N y Hs 2 + N λ i (s 2 i 1) i=1 = max λt 1+ min y Hs 2 +s T D(λ)s, (D) λ R N s R } N {{} regularized LS where g ML is the dual optimal value, D( ) is a diagonal operator. (D) intends to find a best regularization in a {±1} LS context. SDR is equivalent to (D): fsdr = gml. Also, the dual of SDR is (D) (the trick: strong duality of convex problems). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

93 Recap of SDR in dual form f SDR = max λ R N λt 1+ min s R N y Hs 2 +s T D(λ)s. Consider OSR and BR. By strong Lagrangian duality, they can be expressed as f OSR = max λ=γ1, γ R λt 1+ min s R N y Hs 2 +s T D(λ)s, f BR = max λ 0 λ T 1+ min s R N y Hs 2 +s T D(λ)s. Apart from showing a regularized LS interpretation of OSR and BR, the above eqs. reveal that the feasible set of λ in SDR subsumes that in OSR and BR. Hence, we can conclude the previous result that f SDR f OSR, f SDR f BR. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

94 SDR MIMO Detection for MPSK Constellations The ML problem in the MPSK case: min s C C N C y C H C s C 2 s.t. s C,i {1,e j2π/m,...,e j2π(m 1)/M }, i = 1,...,N C. Intuition: relax the constellations constraints to s C,i 2 = 1, & then apply SDR. Following this intuition, we can formulate a complex-valued SDR [Ma-Ding- Ching 04]: min S C H N C, s C C N C Tr(H C H H C S C ) 2Re{s C H H C H y C }+ y C 2 s.t. [S C ] ii = 1, i = 1,...,N C, S C s C s C H. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

95 10 0 Symbol Error Probability ZF MMSE DF SDR, with randomization performance lower bound SNR, in db Symbol error rate performance under (M C,N C ) = (20,20), 8-PSK constellations. Note that LRA methods are not applicable to MPSK constellations. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

96 SDR MIMO Detection for Higher-Order QAM Assume 16-QAM constellations, for ease of illustration. The ML problem (under the equivalent real-valued model): min y Hs 2. s {±1,±3} N A number of attempts have been made for SDR of 16-QAM ML detection [Wiesel-Eldar-Shamai 05], [Sidiropoulos-Luo 06], [Yang-Zhao- Zhou-Wu 07], [Mobasher-Taherzah-Sotirov-Khandani 07], [Mao-Wang- Wang 07]. We consider polynomial inspired SDR (PI-SDR) [Wiesel-Eldar-Shamai 05]; bound constrained SDR (BC-SDR) [Sidiropoulos-Luo 06]; virtually antipodal SDR (VA-SDR) [Mao-Wang-Wang 07]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

97 Bound Constrained SDR (BC-SDR) [Sidiropoulos-Luo 06]: The 16-QAM ML problem is equivalent to T T 2 min S S N,s R N Tr(HT HS) 2s H y + y s.t. S = ss T, S ii {1,9}, i = 1,...,N. ( s 2 i {1,9}) Relaxing S = ss T to S ss T is not enough to yield a convex relaxation. BC-SDR also relaxes {1,9} to [1,9], leading to min S S N,s R N Tr(HT HS) 2s T H T y + y 2 s.t. S ss T, 1 S ii 9, i = 1,...,N. (BC-SDR) BC-SDR is simple to implement, and a specialized IPM is available [Ma-Su- Jaldén-Chi 08]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

98 Polynomial Inspired SDR (PI-SDR) [Wiesel-Eldar-Shamai 05]: PI-SDR uses the fact that u {1,9} (u 1)(u 9) = 0 u 2 10u+9 = 0 to reformulate the ML problem as min S,s,U,u Tr(HT HS) 2s T H T y + y 2 s.t. S = ss T, U = uu T, d(s) = u, d(u) 10u+91 = 0. ( u 2 i 10u i+9 = 0, i) where d : R N N R N is the diagonal operator. PI-SDR is the SDR of the polynomial ML formulation: min Tr(H T HS) 2s T H T y + y 2 s.t. S ss T, U uu T, d(s) = u, d(u) 10u+91 = 0. (PI-SDR) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

99 Virtually Antipodal SDR (VA-SDR) [Mao-Wang-Wang 07]: VA-SDR uses the fact that s {±1,±3} s = b 1 +2b 2, b 1,b 2 {±1}. to rewrite the ML problem in a virtually antipodal form min y H(b 1+2b 2 ) 2 = min y HWb 2, b 1,b 2 {±1} N b {±1} 2N where W = [ I 2I ],b = [ b T 1 b T 2 ] T. By applying the same SDR as in QPSK constellations, VA-SDR is obtained: min Tr(W T H T HWB) 2b T W T H T y + y 2 s.t. B bb T, B ii = 1, i = 1,...,2N. (VA-SDR) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

100 Rather unexpectedly, the three SDRs are equivalent [Ma-Su-Jaldén-Chang- Chi 09]. Consider a unified SDR expression min (S,s) F Tr(HT HS) 2s T H T y + y 2, where F depends on the SDR employed: F BC SDR = { (S,s) S ss T,1 d(s) 91 }, F PI SDR = { (S,s) (U,u,S,s) W PI SDR }, W PI SDR = { (U,u,S,s) U uu T,S ss T,d(S) = u,d(u) 10u+91 = 0 }, F VA SDR = { (S,s) = (WBW T,Wb) B bb T,d(B) = 1 }. It is shown by analysis that F BC SDR = F PI SDR = F VA SDR. The same equivalence is also proven for 64-QAM PI-SDR, & for any 2 q -QAM VA-SDR. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

101 10 0 Symbol Error Probability BC SDR PI SDR VA SDR SNR, in db Symbol error rate performance of BC-SDR, PI-SDR, and VA-SDR under (M C,N C ) = (8,8), 16-QAM constellations. The three performance plots coincide. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

102 10 0 Symbol Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF BC SDR, with randomization Sphere decoding performance lower bound SNR, in db Symbol error rate performance under (M C,N C ) = (8,8), 16-QAM constellations. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

103 10 0 Symbol Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF BC SDR, with randomization Sphere decoding performance lower bound SNR, in db Symbol error rate performance under (M C,N C ) = (16,16), 16-QAM constellations. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

104 10 0 Symbol Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF BC SDR, with randomization performance lower bound SNR, in db Symbol error rate performance under (M C,N C ) = (40,40), 16-QAM constellations. Sphere decoding is too expensive to run in this case. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

105 Some Results in Performance Analysis Assume QPSK or BPSK constellations. SDR has a high probability of giving a rank-one solution, for high SNRs [Jaldén-Martin-Ottersten 03]. Assume BPSK constellations, & i.i.d. complex Gaussian H C. SDR is proven to achieve the full rx diversity [Jaldén-Ottersten 08]. Approximation accuracies: [So 09], [So 10] showed that in both the MPSK and 4 q -QAM scenarios, the SDR detector can produce a constant factor approximate solution to the ML detection problem with exponentially high probability if the SNR is sufficiently low. In other words, in the low SNR region, we have with very high probability, where y Hŝ 2 O(1) y Hs 2, ŝ is the solution produced by SDR (with a suitable randomization procedure), s is the optimal ML solution. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

106 Part III.B: Sensor Network Localization W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

107 Overview The sensor network localization(snl) problem is to determine the(x, y) coordinates of the sensors, given distance information between sensors. In ad-hoc sensor networks, the sensor locations may not be known. A sensor may acquire its location by equipping it with GPS, but this may be too expensive. We may have several anchor sensors that have self-localization capability, though. Since sensors can communicate with each other, each sensor pair can work out their distance (e.g., by measuring the time-of-arrival info., or by ping-pong). The inter-sensor distances, together with anchor locations, can be used to estimate all the sensor locations in a joint fashion. Anchors Sensors W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

108 Let{x 1,...,x n }, x i R 2, bethe collectionof all(unknown) sensor coordinates. Let {a 1,...,a m }, a i R 2, be the collection of all (known) anchor coordinates. The distance between sensor i and sensor j is d ij = (x i,1 x j,1 ) 2 +(x i,2 x j,2 ) 2 = x i x j. Likewise, the distance between sensor i and anchor j is d ij = x i a j. Theobtainedd ij & d ij areassumednoiseless(extensionfornoisycasesavailable). The SNL problem here is that of finding x 1,...,x n such that x i x j 2 = d 2 ij, (i,j) E ss, x i a j 2 = d 2 ij, (i,j) E sa, where E ss & E sa are the sensor-sensor & sensor-anchor edge sets, resp. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

109 Deriving an SDR of the SNL Problem: A First Attempt Let X = [ x 1,...,x n ] R 2 n. The SNL problem can be formulated as This follows since find X R 2 n s.t. x T i x i 2x T i x j +x T j x j = d 2 ij, (i,j) E ss, x T i x i 2x T i a j +a T j a j = d 2 ij, (i,j) E sa. x i x j 2 = (x i x j ) T (x i x j ) = x T i x i 2x T i x j +x T j x j and similarly for x i a j 2. By letting Y = X T X R n n, we can also formulate the SNL problem as follows: find X R 2 n,y R n n s.t. Y ii 2Y ij +Y jj = d 2 ij, (i,j) E ss, Y ii 2x T i a j +a T j a j = d 2 ij, (i,j) E (SNL) sa, Y = X T X. It is known [Saxe 79] that finding a solution to (SNL) is NP-hard. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

110 Observe that with X R 2 n, the constraint Y = X T X is equivalent to Y 0, rank(y ) 2. If we proceed as before and just drop the trouble-causing rank constraint, then we get the following SDR: find X R 2 n,y R n n s.t. Y ii 2Y ij +Y jj = d 2 ij, (i,j) E ss, Y ii 2x T i a j +a T j a j = d 2 ij, (i,j) E sa, Y 0. In this formulation, there is no connection between X and Y. In other words, the information in the original constraint Y = X T X is totally lost. The solution obtained could be quite awful. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

111 Deriving an SDR of the SNL Problem: Another Attempt To keep the connection between X and Y, instead of relaxing Y = X T X to Y 0, we relax it to Y X T X. This is an SDP constraint, since by the Schur complement, Y X T X Z = [ I X X T Y Then, we have the following SDR of the SNL problem: ] 0. find X R 2 n,y R n n s.t. Y ii 2Y ij +Y jj = d 2 ij, (i,j) E ss, Y ii 2x T i a j +a T j a j = d 2 ij, (i,j) E sa, Y X T X. (SNL-SDR) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

112 Theoretical Properties of the SDR Supposethatwehaveasolution(X,Y )to(snl-sdr).underwhatconditions will it be a solution to the original problem (SNL)? In [So-Ye 07], a complete characterization is obtained. [So-Ye 07] Suppose that the given SNL instance is connected. Then, the following statements are equivalent: The solution (X,Y ) to (SNL-SDR) is feasible for (SNL) (in particular, we have Y = X T X ). The max-rank solution to (SNL-SDR) has rank 2. The given SNL instance is uniquely localizable, i.e., it has a unique solution in all dimensions. Further theoretical results: [So-Ye 06], which uses tensegrity theory in discrete geometry. SDR rank reduction using the SYZ theorem [So-Ye-Zhang 08] (see the procedure in Box 4). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

113 Speeding Up the Computation When the number of sensors/edges is large, solving (SNL-SDR) could take a long time. The bottleneck comes not only from the large number of constraints, but also the large ((n+2) (n+2)) positive semidefinite (PSD) constraint Z = [ I X X T Y ] 0. Complexity-reduced implementations, at the cost of some SNL performance, have recently received attention in large-scale sensor network applications. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

114 Speeding Up the Computation: The Edge-Based SDR To circumvent the large PSD constraint, one approach is to first observe that each edge (i,j) E ss is responsible for the following constraints in (SNL): Y ii 2Y ij +Y jj = d 2 ij, Y ii = x T i x i, Y ij = x T i x j, Y jj = x T j x j. ( ) Now, we can treat the constraints in ( ) as a group and relax them using our previous technique, i.e., R 4 4 Z ij = I x i x j x T i Y ii Y ij x T j Y ij Y jj 0. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

115 This approach results in the following so-called edge-based SDR of the SNL problem, which was presented in [Wang-Zheng-Ye-Boyd 08]: find X R 2 n,y R n n s.t. Y ii 2Y ij +Y jj = d 2 ij, (i,j) E ss, Y ii 2x T i a j +a T j a j = d 2 ij, (i,j) E sa, I x i x j Z ij = x T i Y ii Y ij 0, (i,j) E ss. x T j Y ij Y jj (SNL-ESDR) Note that (SNL-ESDR) has E ss 4 4 PSD constraints, instead of one (n+ 2) (n+2) PSD constraint in (SNL-SDR). The smaller dimension (i.e., 4) of the PSD constraints in (SNL-ESDR) is computationally easier to handle, thus allowing a speedup in computation. However, it should be noted that (SNL-SDR) is a tighter relaxation than (SNL- ESDR). Indeed, each Z ij is a principal submatrix of Z, and every principal submatrix of a PSD matrix must also be PSD. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

116 Extensions of the Basic SDR So far we have only considered the noiseless version of the SNL problem. In general, the distance measurements {d ij } and { d ij } could be corrupted. A commonly used error model is d ij = d nom ij +g ij, nom d ij = d ij +ḡ ij, where{d nom ij }and{ d nom ij }arethenominalmeasurements,and{g ij }(resp.{ḡ ij }) are i.i.d. Gaussian random variables with mean 0 and variance σ 2 ij (resp. σ2 ij ). In [Biswas-Liang-Wang-Ye 06], the following maximum-likelihood (ML) SNL formulation is considered: min x 1,...,x n (i,j) E ss 1 σ 2 ij ( x i x j d ij ) 2 + (i,j) E sa 1 σ 2 ij ( xi a j d ij ) 2. (ML-SNL) W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

117 As shown in [Biswas-Liang-Wang-Ye 06], SDR can be employed to tackle the nonconvex problem (ML-SNL). The key lies in constructing suitable linearizations of the expressions ( x i x j d ij ) 2 and ( xi a j d ij ) 2. Let us focus on the former. The strategy is to proceed one level at a time. Let ǫ ij = ( x i x j d ij ) 2 = x i x j 2 2d ij x i x j +d 2 ij. ( ) Upon defining we see that ( ) is equivalent to u ij = x i x j, v ij = x i x j 2, v ij 2d ij u ij +d 2 ij = ǫ ij, v ij = u 2 ij, v ij = Y ii 2Y ij +Y jj, Y = X T X. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

118 Now, we can relax v ij = u 2 ij to v ij u 2 ij, and Y = XT X to Y X T X. Using the Schur complement, these are equivalent to the SDP constraints [ ] [ ] 1 uij I X U ij = 0, Z = u ij v ij X T 0. Y Hence, we obtain the following SDR of (ML-SNL): min X,Y,{U ij },{Ūij} 1 σ 2 ǫ ij + 1 (i,j) E ij σ 2 ǫ ij ss (i,j) E ij sa s.t. Y ii 2Y ij +Y jj = v 2 ij, (i,j) E ss, Y ii 2x T i a j +a T j a j = v 2 ij, (i,j) E sa, v ij 2d ij u ij +d 2 ij = ǫ ij, (i,j) E ss, v ij 2 d ij ū ij + d 2 ij = ǫ ij, (i,j) E sa, U ij 0, (i,j) E ss ; Ū ij 0, (i,j) E sa ; Z 0. (ML-SNL-SDR) Note that the solution to (ML-SNL-SDR) can be used as a starting iterate for a gradient descent procedure, which can further improve the solution quality. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

119 SDR (ML-SNL formulation), plus a 2nd-stage solution refinement by gradient descent. The distance measurements are noisy. : true sensor locations; : anchor locations; : SDR solution; : gradient descent trajectory (50 iterations). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

120 Gradient descent ML-SNL with a random starting point. : true sensor locations; : anchor locations; : gradient descent trajectory (50 iterations). W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

121 Extensions of the Basic SDR SDR can also be employed to handle ML-SNL formulations with uncertain anchor locations, and/or with uncertain propagation speed [Lui-Ma-So-Chan 09] (the latter happens in underground sensor networks). For uncertain anchor locations, one could adopt the following error model: a i = a nom i +z i, where a nom i R 2 is the nominal location of anchor i, and z i R 2 is a Gaussian random vector with mean zero and covariance matrix Φ i. Then, one has the following ML formulation (which is nonconvex): min x 1,...,xn a 1,...,am (i,j) E ss 1 σ 2 ij + i ( x i x j d ij ) 2 + (i,j) E sa 1 σ 2 ij (a i a nom i ) T Φ 1 i (a i a nom i ). ( xi a j d ij ) 2 This can be handled using the previously introduced SDR techniques. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

122 2 0 Mean square position error ( dbm 2 ) standard SDP 14 proposed SDP proposed ESDP CRLB σ d 2 ( dbm 2 ) Mean square position error performance versus noise power in the presence of anchor position uncertainty. Standard SDP SDR without anchor position uncertainty; Proposed SDP SDR with anchor position uncertainty; Proposed ESDP Edge-based SDR with anchor position uncertainty. Details available in [Lui-Ma-So-Chan 09]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

123 Part III.C: Transmit Beamforming W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

124 Overview Tx. beamforming(bf) has received much attention, with numerous formulations, application scenarios, solution approaches, their combinations,... Our focus: A QCQP-SDR perspective on various tx. BF problems; robust solutions under imperfect CSIT. What we will not go through: alternative solution approaches: second-order cone program(socp)(for unicast BF with instant. CSIT only) [Wiesel-Eldar-Shamai 06]; uplink-downlink duality (for unicast BF only) [Schubert-Boche 04]; alternative design formulations: max-min-fairness (can be handled by SDR); user admission; proportional fairness, sum rate max.,... W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

125 Multi-Group Multicast Beamforming A natural generalization of unicast and multicast BF. Scenario: multiuser MISO downlink with M groups of users, & with each group receiving the same info. [Karipidis-Sidiropoulos-Luo 08] {z} group 1 {z} group 2 {z} group 3 W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

126 Transmit signal: x(t) = M m=1 w m s m (t) where s m (t) C is the data stream for group m, & w m C N t its beamvector. Received signal of user k in the mth group: y m,k (t) = h H m,kx(t)+v m,k (t) = h H m,kw m s m (t)+ l mh H m,kw l s l (t) +v m,k (t), }{{} inter-group interference where k = 1,...,K m, & K m is the number of users in the mth group. SINR: w H SINR m,k = mr m,k w m l m wh l R m,kw l +σm,k 2 where R m,k = h m,k h H m,k for instant. CSIT, & R m,k = E{h m,k h H m,k} for stat. CSIT. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

127 Problem: a QoS-assured, total power minimizing, design min w 1,...,w M C N t M m=1 w m 2 s.t. SINR m,k = w H mr m,k w m l m wh l R m,kw l +σm,k 2 where γ m,k s are prescribed SINR requirements. γ m,k, k = 1,...,K m, m = 1,...,M, A separable QCQP with M variables, w 1,...,w M, and M m=1 K m constraints. By the SBP rank result, the rank-one optimality of SDR is assured when: K 1 3, K m = 1 m 1 (one group serving 3 users, the others 1 user); K 1 2, K 2 2, K m = 1 m 1,2 (two groups serving 2 users, the others 1 user). requires a more sophisticated randomization procedure [Karipidis-Sidiropoulos- Luo 08]. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

128 Multi-Cell Coordinated Beamforming Motivation: provide better interference management by coordinating the transmissions of base stations at different cells. Scenario: Unicast transmit beamforming in a multi-cell scale [Dahrouj-Yu 10], [Bengtsson-Ottersten 01] W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

129 Transmit signal of ith cell: x i (t) = K i j=1 w i,j s i,j (t), i = 1,...,N where s i,j (t) / w i,j is the tx. stream/ beamvector for user j in ith cell, resp.; K i is the no. of users in ith cell; N is the no. of cells. Received signal of user j in the ith cell: y i,j (t) = h H i,i,jx i (t)+ m ih H m,i,jx m (t)+v i,j (t), j = 1,...,K i where h m,i,j is the channel from mth cell to user j in the ith cell. Define CSIT R m,i,j in the same way as before. SINR: SINR i,j = w H i,j R i,i,jw i,j w H i,lr i,i,j w i,l + w H m,nr m,i,j w m,n l j m i n }{{}}{{} intra-cell interference inter-cell interference +σ 2 i,j W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

130 Design problem: min {w i,j } s.t. w i,j 2 i,j w H i,j R i,i,jw i,j l j wh i,l R i,i,jw i,l + m i,n wh m,nr m,i,j w m,n +σi,j 2 j = 1,...,K i, i = 1,...,N γ i,j, A QCQP with N i=1 K i variables & N i=1 K i constraints (despite its complicated appearance). SDR solves the multi-cell unicast problem optimally, by the SBP rank result. Practical issue: distributed optimization algorithms. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,

131 Cognitive Radio (CR) Beamforming Goal: access the channel owned by primary users (PUs) through spectrum sharing. Idea: the CR system avoids excessive interference to the PUs through tx. opt. Scenario: MISO downlink with the CR (or secondary) system, either unicast or multicast; K secondary users (SUs); L single-antenna PUs Primary Tx Primary Rx Secondary Tx Secondary Rx Assume known CSIT from the secondary transmitter to the PUs. W.-K. Ma, SDR and its Applications in SP and Commun., MIIS 2012 tutorial, Xi an, July 9-13,