Convex Optimization and Its Applications in Signal Processing

Transcription

1 Convex Optimization and Its Applications in Signal Processing Anthony Man-Cho So Department of Systems Engineering & Engineering Management The Chinese University of Hong Kong Hong Kong ChinaSIP 2014 Tutorial, July 9, 2014

2 BRAND X PICTURES The main references of this tutorial: Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, & S. Zhang, Semidefinite relaxation of quadratic optimization problems, IEEE Signal Process. Mag., May S. Cui, A. M.-C. So, & R. Zhang, Unconstrained & constrained optimization problems, Chapter 14 of Mathematical Foundations for Signal Processing, Communications, and Networking, CRC Press, [ Zhi-Quan Luo, Wing-Kin Ma, Anthony Man-Cho So, Yinyu Ye, and Shuzhong Zhang ] [ VOLUME 27 NUMBER 3 MAY 2010 ] [ From its practical deployments and scope of applicability to key theoretical results] In recent years, the semidefinite relaxation (SDR) technique has been at the center of some of very exciting developments in the area of signal processing and communications, and it has shown great significance and relevance on a variety of applications. Roughly speaking, SDR is a powerful, computationally efficient approximation technique for a host of very difficult optimization problems. In particular, it can be applied to many nonconvex quadratically constrained quadratic programs (QCQPs) in an almost mechanical fashion, including the following problem: min x[r n x T Cx s.t. x T F i x$g i, i51,c, p, x T H i x5l i, i51,c, q, (1) where the given matrices C, F 1,c, F p, H 1,c, H q are assumed to be general real symmetric matrices, possibly indefinite. The class of nonconvex QCQPs (1) captures many problems that are of interest to the signal processing and communications community. For instance, consider the Boolean quadratic program (BQP) min x[r n x T Cx s.t. x i 2 5 1, i51,c, n. (2) The BQP is long known to be a computationally difficult problem. In particular, it belongs to the class of NP-hard problems. Nevertheless, being able to handle the BQP well has an enormous impact on multiple-input, multiple-output (MIMO) detection and multiuser detection. Another important yet NP-hard problem in the nonconvex QCQP class (1) is Digital Object Identifier /MSP IEEE SIGNAL PROCESSING MAGAZINE [20] MAY /10/$ IEEE A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 1

3 Acknowledgment: Shuguang Cui, Tom Luo, Wing-Kin (Ken) Ma, Yinyu Ye, Rui Zhang, & Shuzhong Zhang for co-authoring the above articles; Qiang Li, Jiaxian Pan, Xiaoxiao Wu & Xiao Fu for helping prepare this slides. A good part of this tutorial is based the one given in ICASSP 2014 with Ken Ma. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 2

4 Outline Part I: Crash course on convex analysis and convex optimization Part II: Quadratically constrained quadratic programs and semidefinite programming Part III: Semidefinite relaxation: Theory, and implications in practice Part IV: Applications and latest advances A. transmit beamforming B. advanced topics in transmit beamforming C. sensor network localization Conclusion A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 3

5 Part I: Crash Course on Convex Analysis and Convex Optimization A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 4

6 Convex Functions: Definition and Examples Let S R n be given. A function f : S R {+ } is convex on S if for any x 1,x 2 S and α (0, 1), f(αx 1 + (1 α)x 2 ) αf(x 1 ) + (1 α)f(x 2 ). ( ) Geometrically, {αx 1 + (1 α)x 2 : α [0, 1]} is the line joining x 1 and x 2. Hence, the inequality ( ) simply says the chord joining (x 1,f(x 1 )) and (x 2,f(x 2 )) lies above the function. y f(x 1 ) f(x 2 ) x 1 x 2 x A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 5

7 Convex Functions: Definition and Examples Some examples of convex functions: Any affine function f : R n R; i.e., f(x) = a T x + b for some a R n and b R. Any norm on R n. This follows from the triangle inequality. Any non-negative combination of convex functions f 1,...,f m : S R; i.e., f(x) = m i=1 α if i (x) for any α 1,...,α m 0. Let S R n be a convex set; i.e., for any x 1,x 2 S and α (0,1), we have αx 1 + (1 α)x 2 S. Then, the indicator function 1 S : R n R {+ } of S, which is given by { 0 if x S, 1 S (x) = + otherwise, is convex on R n. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 6

8 Convex Functions: Definition and Examples Sometimes it is difficult to establish the convexity of a function directly from the definition. Is R n x x T Ax, where A S n and S n is the set of n n real symmetric matrices, convex on R n? Is R m n X σ 1 (X), where σ 1 (X) is the largest singular value of the matrix X, convex on the set of m n matrices R m n? Is S n ++ X log detx, where S n ++ S n is the set of n n positive definite matrices, convex on S n ++? It would be good to have some alternative, hopefully easier, ways to establish the convexity of a given function. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 7

9 Recognizing Convexity: Tricks of the Trade (Pointwise Supremum) Let I be an index set, and let {f i } i I be an arbitrary family of convex functions on S R n. Then, the function is convex on S. x sup i I {f i (x)} Example: Consider the function R m n X σ 1 (X). By the Courant- Fischer theorem, we have σ 1 (X) = sup {f u,v (X)}, (u,v) R m R n : u 2 = v 2 =1 where f u,v (X) = u T Xv. Note that X f u,v (X) is linear and hence convex. It follows that X σ 1 (X) is convex on R m n. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 8

10 Recognizing Convexity: Tricks of the Trade (Composition with Increasing Convex Functions) Let f : S T be a convex function on S, where S R n and T R. Let g : T R be an increasing convex function on T. Then, the function is convex on S. x g(f(x)) Example: Let be a norm on R n, and let p 1 be arbitrary. Since t t p is increasing and convex on R +, the function x x p is convex on R n. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 9

11 Recognizing Convexity: Tricks of the Trade (Gradient Inequality) Let S R n be an open convex set (such as S = R n or S = S n ++), and let f : S R be a differentiable function on S. The following statements are equivalent: f is convex on S. For any x 0,x S, we have f(x) f(x 0 ) + f(x 0 ) T (x x 0 ). ( ) Geometrically, the inequality ( ) says that a convex function has an affine minorant at any point x 0. y y = f(x) slope = f(x) f(x 0 ) x 0 y = f(x 0 ) + f(x 0 ) T (x x 0 ) x A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 10

12 Recognizing Convexity: Tricks of the Trade (Hessian Characterization) Let S R n be an open convex set, and let f : S R be a twice continuously differentiable function on S. The following statements are equivalent: f is convex on S. For any x S, the Hessian 2 f(x) is positive semidefinite (denoted by 2 f(x) 0). Example: Let f : R n R be a quadratic function; i.e., f(x) = 1 2 xt Ax + b T x + c for some A S n, b R n, and c R. A routine calculation shows f(x) = Ax + b, 2 f(x) = A. Hence, f is convex on R n if and only if A 0. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 11

13 Recognizing Convexity: Tricks of the Trade (Restriction on Lines) Let S R n be an open convex set, and let f : S R be a function on S. For any given x 0 S and h R n, define the set S x0,h and the function f x0,h : S x0,h R by S x0,h = {t R : x 0 + th S}, f x0,h(t) = f(x 0 + th). Then, S x0,h is convex. Moreover, the following statements are equivalent: f is convex on S. f x0,h is convex on S x0,h for any x 0 S and h R n. Thus, determining the convexity of a multivariate function f can be reduced to determining the convexity of (an infinite number of) univariate functions. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 12

14 Example: Consider the function f : S n ++ R given by f(x) = log detx. Let X 0 S n ++ and H S n be arbitrary. We compute f X0,H(t) = log det(x 0 + th) [ ( ) = log det X 1/2 0 I + tx 1/2 0 HX 1/2 0 n = log(1 + tλ i ) log detx 0, i=1 ] X 1/2 0 where λ 1,...,λ n are the eigenvalues of X 1/2 0 HX 1/2 0. Note that since X 0, H are fixed, so are λ 1,...,λ n. Since d 2 f X0,H(t) dt 2 = n i=1 λ 2 i (1 + tλ i ) 2 0 for any X 0 S n ++ and H S n, we conclude that f is convex on S n ++. For those who are familiar with matrix differential calculus, a more direct proof could be to note that f(x) = X 1 and 2 f(x) = X 1 X 1 0. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 13

15 Elements of Convex Optimization Consider the following optimization problem: min x R n f(x) s.t. g i (x) 0, i = 1,...,m 1, h j (x) = 0, j = 1,...,m 2. (P) We say that (P) is a convex optimization problem if f, g 1,...,g m1 are convex on R n, and h 1,...,h m2 are affine on R n. A very desirable property of convex optimization problems is that any locally optimal solution is also globally optimal. A fundamental question in optimization (convex or not) is how to characterize the set of optimal solutions. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 14

16 Optimality Conditions for Convex Optimization Suppose that (P) is a convex optimization problem, and that all functions involved are continuously differentiable. Karush-Kuhn-Tucker (KKT) Theorem Let x be an optimal solution to (P). Suppose that (P) satisfies some regularity conditions. Then, there exist multipliers v1,...,vm 1 R and w1,...,w m 2 R such that f(x ) + m 1 i=1 v i g i (x ) + m 2 j=1 w j h j (x ) = 0, v i 0, i = 1,...,m 1, (Dual Feasibility) v i g i (x ) = 0, i = 1,...,m 1. (Complementarity) Conversely, suppose that (x,v 1,...,v m 1, w 1,...,w m 2 ) satisfies the above system, and that x is feasible for (P). Then, x is optimal for (P). The set of multipliers {v 1,...,v m 1, w 1,...,w m 2 } serves as a certificate of optimality for x. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 15

17 Implications of the KKT Theorem The KKT theorem essentially reduces the problem of convex optimization to that of solving an equality/inequality system (the KKT system): Find x, v 1,...,v m1, w 1,...,w m2 such that f(x) + m 1 i=1 g i (x) 0, i = 1,...,m 1, h j (x) = 0, j = 1,...,m 2, v i g i (x) + m 2 j=1 w j h j (x) = 0, v i 0, i = 1,...,m 1, } (Primal Feasibility) (Dual Feasibility) v i g i (x) = 0, i = 1,...,m 1. (Complementarity) However, the necessity part of the KKT theorem requires some sort of regularity of (P). Such a requirement cannot be dropped in general. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 16

18 Failure of the KKT Theorem Consider the following convex optimization problem: min x R 2 f(x) = x 1 s.t. g 1 (x) = (x 1 1) 2 + (x 2 1) 2 1 0, g 2 (x) = (x 1 1) 2 + (x 2 + 1) x 2 (1, 1) x 1 (1, 1) Clearly, the optimal solution is x = (1, 0). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 17

19 Failure of the KKT Theorem Part of the KKT system at x = (1, 0) is given by which is clearly infeasible. [ ] [ ] [ ] v 0 1 +v 2 2 = 2 }{{}}{{}}{{} f(x ) g 1 (x ) g 2 (x ) [ 0 0 ], Thus, the KKT system is not a necessary condition for optimality, even for convex optimization problems! So what kind of regularity conditions are needed? A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 18

20 Regularity Conditions for the KKT Theorem Under either of the following conditions, the KKT system is necessary and sufficient for optimality: (Slater) There exists an x R n such that g i ( x) < 0, i = 1,...,m 1, h j ( x) = 0, j = 1,...,m 2. We call such an x a strictly feasible point of (P). (Linearly Constrained Problems) The functions g 1,...,g m1 and h 1,...,h m2 are all affine. There are other, more sophisticated, regularity conditions in the literature. However, we shall not discuss them here. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 19

21 Application: Power Allocation in Parallel AWGN Channels Scenario: Transmission over n parallel AWGN channels. For the ith channel: channel power gain: h i 0, additive Gaussian noise power: σ i > 0, allocated transmit power: p i 0, Maximum information rate that can be reliably transmitted over the ith channel: ( r i = log h ) ip i. σ i Problem: Allocate power p 1,...,p n to the channels so that the sum rate is maximized: n ( max log p R n h ) ip i σ i=1 i n s.t. p i P, i=1 p i 0, i = 1,...,n. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 20

22 Application: Power Allocation in Parallel AWGN Channels The problem is equivalent to the following standard form: min p R n s.t. n ln i=1 ( 1 + h ) ip i σ i n p i P, (v 0 ) i=1 p i 0, i = 1,...,n, (v i ) (PA) where v 0,v 1,...,v n are the multipliers. It is routine to verify that the function p n ln i=1 ( 1 + h ) ip i σ i is convex on R n +. Hence, (PA) is a linearly constrained convex optimization problem, and the KKT system is both necessary and sufficient for optimality. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 21

23 Application: Power Allocation in Parallel AWGN Channels The KKT system is given by v 0 v i = h i, i = 1,...,n, (a) h i p i + ( σ i n ) v 0 p i P = 0, (b) i=1 v i p i = 0, i = 1,...,n, (c) v i 0, i = 0, 1,...,n. (d) Let us try to extract insights from it. Suppose that h i > 0 for some i. Then, v 0 > 0 by (a), (d). Together with (b), this implies that n i=1 p i = P. Moreover, we must have v i < v 0 by (a). Thus, p i = 1 v 0 v i σ i h i. If p i > 0, then v i = 0 by (c). If p i = 0, then in order to satisfy (a) with some v i 0, we must have 1 σ i < 0. v 0 h i A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 22

24 Application: Power Allocation in Parallel AWGN Channels Hence, ( 1 p i = σ ) + i, i = 1,...,n, v 0 h i where (a) + = max{0, a}. Together with n i=1 p i = P, we arrive at n i=1 ( 1 σ ) + i = P. (*) v 0 h i We can then solve for v 0 using, e.g., the bisection method on (*). The optimal power allocation as given by (*) is the well-known waterfilling solution. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 23

25 How About Nonconvex Problems? Naturally, most problems of interest are nonconvex. In this case, the necessity part of the KKT theorem still holds (again, subject to regularity conditions). However, the KKT system may not be sufficient for optimality. Although nonconvex problems are generally difficult to handle, in the remainder of the tutorial, we show how various nonconvex problems can be efficiently and effectively approximated by convex optimization techniques. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 24

26 Part II: Quadratically Constrained Quadratic Programs and Semidefinite Programming A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 25

27 A Quick Review of Multivariate Quadratic Functions The (homogeneous) quadratic function x x T Cx = n i=1 n j=1 x ix j C ij is convex if and only if C f(x) 10 5 f(x) x x x x 1 (a) C 0. (b) C 0. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 26

28 The constraint set {x R n x T Fx 1} is convex if and only if F (a) F 0. (b) F 0. The constraint set {x R n x T Fx = 1} is nonconvex. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 27

29 Quadratically Constrained Quadratic Program Consider the class of real-valued quadratically constrained quadratic programs (QCQPs): x T Cx min x R n s.t. x T F i x g i, x T H i x = l i, where C,F 1,...,F p,h 1,...,H q S n. i = 1,...,p, i = 1,...,q, We do not assume convexity here. In particular, C, F i, H i can be arbitrary. Nonconvex QCQP is a very difficult problem in general. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 28

30 Nonconvex QCQP: How Hard Could it Be? Consider the Boolean quadratic program (BQP) min x R n x T Cx s.t. x 2 i = 1, i = 1,...,n, a long-known difficult problem falling in the nonconvex QCQP class. One could solve it by evaluating all possible combinations; i.e., brute-force search. The time complexity of a brute-force search is O(2 n ), not okay at all for large n! The BQP is NP-hard in general we still can t find an algorithm that can solve a general BQP in time O(n p ) for any fixed p > 0. x x 1 A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 29

31 Nonconvex QCQP: How Hard Could it Be? Consider another QCQP: min x R n x T Cx s.t. x T F i x 1, i = 1,...,m, where C, F 1,...,F m 0 (to make the dimension explicit, we will also use the notation C,F 1,...,F m S n +) Difficulty: feasible set is the intersection of the exteriors of ellipsoids. This problem is also NP-hard. x x 1 A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 30

32 Semidefinite Relaxation for QCQP Semidefinite relaxation (SDR) is a computationally efficient approximation approach to QCQP. Approximate QCQPs by a semidefinite program (SDP), a class of convex optimization problems where reliable, efficient algorithms are readily available. The idea can be found in an early paper of Lovász in 1979 [Lovász 79]. It is arguably the work by Goemans & Williamson [Goemans-Williamson 95] that sparked the significant interest in SDR. A key notion introduced by Goemans & Williamson is randomization; we will go through that. SDR has received much interest in the optimization field; now we have seen a number of theoretically elegant analysis results. (This may concern us more) In many applications, SDR works well empirically. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 31

33 Impacts of SDR in SP and Commun. The introduction of SDR in SP and commun. since the early 2000 s has reshaped the way we see many topics today. Existing applications include multiuser/mimo detection [Tan-Rasmussen 01], [Ma-Davidson-Wong- Luo-Ching 02] transmit beamforming: unicast beamforming [Bengtsson-Ottersten 01], multicast beamforming [Sidiropoulos-Davidson-Luo 06], & many others... source localization and sensor network localization [Cheung-Ma-So 04], [Biswas-Liang-Wang-Ye 06] code waveform design in radar [De Maio et al. 08] large-margin parameter estimation in speech recognition [Li-Jiang 07] optimal power flow in electrical grids [Low 14] (also [Bienstock 14]) (related) phase retrieval [Candès-Eldar-Strohmer-Voroninski 13] others: robust blind receive beamforming, transmit B 1 shim in MRI, distributed detection, phase unwrapping... We believe that more applications are on the way. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 32

34 The Concept of SDR For notational conciseness, we write the QCQP as min x R n x T Cx s.t. x T A i x i b i, i = 1,...,m. (QCQP) Here, i can represent either, =, or for each i; C,A 1,...,A m S n ; and b 1,...,b m R. A crucial first step of understanding SDR is to see that x T Cx = Tr(x T Cx) = Tr(Cxx T ), x T A i x = Tr(x T A i x) = Tr(A i xx T ), or, if we let X = xx T, x T Cx = Tr(CX), x T A i x = Tr(A i X) The objective and constraint functions are linear in X. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 33

35 The Concept of SDR The condition X = xx T is equivalent to X 0, rank(x) 1. Hence, (QCQP) can be reformulated as min X S n Tr(CX) s.t. Tr(A i X) i b i, i = 1,...,m, X 0, rank(x) 1. (QCQP) The constraints Tr(A i X) i b i are easy, but rank(x) 1 is hard. Key Insight: Drop the rank-one constraint to obtain a relaxed QCQP: min X S n Tr(CX) s.t. Tr(A i X) i b i, X 0. i = 1,...,m, (SDR) (SDR) is convex and is an instance of semidefinite program (SDP). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 34

36 Some Merits We Can Immediately Say A globally optimal solution to an SDP can be found by available numerical algorithms in polynomial time (often by interior-point methods, in O(max{m, n} 4 n 1/2 log(1/ǫ)), ǫ being soln. accuracy). For instance, using the software toolbox CVX, we can solve (SDR) in MATLAB with the following lines: (for simplicity we assume i = for all i here) cvx begin variable X(n,n) symmetric minimize(trace(c*x)); subject to for i=1:m trace(a(:,:,i)*x) >= b(i); end X == semidefinite(n) cvx end A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 35

37 The KKT Theorem for SDP Since SDP is a convex optimization problem, the KKT system is necessary (subject to regularity conditions) and sufficient for optimality. But what is the KKT system for, say, the following SDP? min X S n Tr(CX) s.t. Tr(A i X) b i, i = 1,...,m 1, Tr(B j X) = d j, j = 1,...,m 2, X 0. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 36

38 The KKT Theorem for SDP Introduce the proper multipliers: min X S n f(x) = Tr(CX) s.t. g i (X) = Tr(A i X) b i 0, i = 1,...,m 1, (v i ) h j (X) = Tr(B j X) d j = 0, j = 1,...,m 2, (w j ) X 0. (S) Then, the KKT system is given by }{{} C + f(x) m 1 i=1 v i }{{} A i + g i (X) Tr(A i X) b i, i = 1,...,m 1, Tr(B j X) = d j, j = 1,...,m 2, X 0, m 2 j=1 w j B j }{{} h j (X) + S }{{} ( Tr(XS)) = 0, v i 0, i = 1,...,m 1, S 0, v i (Tr(A i X) b i ) = 0, i = 1,...,m 1, Tr(XS) = 0. } (Primal Feasibility) (Dual Feasibility) (Complementarity) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 37

39 Issues with the Use of SDR There is no free lunch in turning the NP-hard (QCQP) to the convex, polynomialtime solvable (SDR). The issue is how to convert a solution to (SDR) into an approximate QCQP solution. If an SDR solution, say, denoted by X, is of rank one; or, equivalently, X = x x T, then x is feasible and in fact optimal to (QCQP). However, we cannot guarantee that X is always of rank-one. (Otherwise we would have solved an NP-hard problem in polynomial time!) There are many ways to produce an approximate QCQP solution from X when rank(x ) > 1. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 38

40 QCQP Solution Approximation in SDR: An Example Consider again the BQP The SDR of (BQP) is min X S n min x R n x T Cx s.t. x 2 i = 1, i = 1,...,n. (BQP) Tr(CX) (SDR) s.t. X 0, X ii = 1, i = 1,...,n. An intuitive (even for engineers) idea is to apply a rank-1 approximation to the SDR solution X : 1) Carry out the eigen-decomposition X = r λ i q i q T i, i=1 where r = rank(x ), λ 1 λ 2... λ r > 0 are the eigenvalues and q 1,...,q r R n the respective eigenvectors. 2) Approximate the BQP by ˆx = sgn( λ 1 q 1 ). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 39

41 Application: MIMO Detection Scenario: A spatial multiplexing system with M t transmit & M r receive antennae. Symbols s C Spatial Multiplexer MIMO Detector Detected Symbols MIMO channel H C Objective: Detect symbols from the received signals, given channel information. Received signal model: y C = H C s C + v C, where H C C M r M t is the MIMO channel, s C C M t is the transmitted symbol vector, & v C C M r is complex circular Gaussian noise. Assume QPSK constellations, s C {±1 ± j} M t. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 40

42 Problem: Maximum-likelihood (ML) detection (NP-hard) ŝ C,ML = arg min s C {±1±j} M t y C H C s C 2. The received signal model can be converted to a real form [ ] Re{yC } Im{y C } }{{} y = [ ] [ ] Re{HC } Im{H C } Re{sC } Im{H C } Re{H C } Im{s }{{} C } }{{} H s {±1} 2M t + [ ] Re{vC } Im{v C } }{{} v, and hence the ML problem can be rewritten (homogenized) as min s {±1} 2M t y Hs 2 = min ty Hs 2 s {±1} 2M t,t {±1} = min s {±1} 2M t,t {±1} [ s T t ] [ H T H H T y y T H y 2 ] [ ] s, t which is a BQP. Subsequently, SDR can be applied [Tan-Rasmussen 01], [Ma-Davidson-Wong-Luo-Ching 02]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 41

43 Bit Error Probability ZF MMSE DF LRA ZF DF LRA MMSE DF SDR, with randomization SDR, with rank 1 approx. performance lower bound SNR, in db Bit error rate performance under (M r, M t ) = (40, 40). ZF zero forcing; MMSE-DF min. mean square error with decision feedback; LRA lattice reduction aided. Randomization will be explained shortly. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 42

44 Average Running Time, in Seconds 10 1 ZF LRA MMSE DF sphere decoding (Schnorr Euchner) 10 0 SDR, with randomization Problem Size M t Complexity comparison of various MIMO detectors. SNR= 12dB. Sphere decoding is an exact ML method. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 43

45 Additional Remarks about the MIMO Detection Application The idea is not restricted to spatial multiplexing! It can also be used in multiuser CDMA, space-time/freq./time-freq. coding, multiuser MIMO, massive MIMO and even blind MIMO [Li-Bai-Ding 03], [Ma-Vo-Davidson-Ching 06],... Extensions that have been considered: MPSK constellations [Ma-Ching-Ding 04] higher-order QAM constellations [Ma-Su-Jaldén-Chang-Chi 09] (and refs. therein) soft-in-soft-out MIMO detection (a.k.a. BICM-MIMO) [Steingrimsson-Luo- Wong 03] fast implementations [Kisialiou-Luo-Luo 09], [Wai-Ma-So 11] Performance analysis for SDR MIMO detection: diversity analysis [Jaldén-Ottersten 08] probabilistic approximation accuracy analysis [Kisialiou-Luo 10], [So 10] A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 44

46 Alternative Interpretation of SDR: Solving QCQP in Expectation We return to the SDR solution approximation issue. Recall min x R n x T Cx s.t. x T A i x i b i, i = 1,...,m. (QCQP) Let ξ N(0, X), where X is the covariance. Consider a stochastic QCQP: min X S n, X 0 E ξ N(0,X) {ξ T Cξ} s.t. E ξ N(0,X) {ξ T A i ξ} i b i, i = 1,...,m, (E-QCQP) where we manipulate the statistics of ξ so that in expectation, the objective function is minimized & constraints are satisfied. One can show that (E-QCQP) is the same as the following SDR of (QCQP): min X S n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. (SDR) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 45

47 The stochastic QCQP interpretation of SDR min X S n, X 0 E ξ N(0,X) {ξ T Cξ} s.t. E ξ N(0,X) {ξ T A i ξ} i b i, i = 1,...,m (E-QCQP) motivates another approach to approximating QCQPs, namely generate a random vector ξ N(0, X ) (X is an SDR soln.), then modify ξ so that it is QCQP-feasible. Such a randomized QCQP soln. approx. may be performed multiple times, to get a better approx. The stochastic QCQP interpretation allows one to establish many important theoretical SDR approx. accuracy results. This will be explained in Part III. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 46

48 Example: Randomization in BQP or MIMO Detection A simple (and very important) example for illustrating randomizations is BQP: min x R n x T Cx s.t. x 2 i = 1, i = 1,...,n. (BQP) Box 1. Gaussian Randomization Procedure for BQP given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l N(0,X ), and construct a feasible point x l = sgn(ξ l ); end determine l = arg min l=1,...,l xt l C x l. output ˆx = x l as an approximate solution to (BQP). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 47

49 Bit Error Probability L= 1 L= 5 L= 20 L= 50 L= 80 L= 120 L= SNR, in db Performance of various no. of randomizations in MIMO detection. M t = M r = 40. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 48

50 Complex-valued QCQP and SDR Consider a general complex-valued QCQP min x C n x H Cx s.t. x H A i x i b i, i = 1,...,m, where C, A 1,...,A m H n ; H n denotes the set of n n Hermitian matrices. Using the same idea as before, one can derive an SDR for the complex-valued QCQP: Tr(CX) min X H n s.t. X 0, Tr(A i X) i b i, i = 1,...,m. The only difference is that the problem domain now is H n (change symmetric to hermitian in your CVX code). Note that while the ideas leading to real and complex SDRs are the same, their performance can be different. This will be explained in Part III. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 49

51 Application: Multicast Transmit Beamforming Scenario: Common information broadcast in multiuser MISO downlink, assuming channel state information at the transmitter (CSIT). The transmit signal: x(t) = ws(t), User 1 where s(t) C is the tx. data stream, & w C N t is the tx. beamvector. Received signal for user i: y i (t) = h H i x(t) + v i (t), where h i C N t is the channel of user i, & v i (t) is noise with variance σ 2 i. Basestation User 2 A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 50

52 Problem: Optimize w by a QoS-assured design: min w C N t w 2 s.t. SNR i γ, i = 1,...,K, where γ is a prescribed SNR requirement for all users, and R i = { SNR i = E{ h H i ws(t) 2 }/σ 2 i = w H R i w/σ 2 i, h i h H i, h i is available (instant CSIT), E{h i h H i }, h i is random with known 2nd order stat. (stat. CSIT). The design problem can be rewritten as a complex-valued QCQP min w C N w 2 s.t. w H A i w 1, i = 1,...,K, where A i = R i /γσ 2 i. This multicast problem is NP-hard in general, but can be approximated by SDR [Sidiropoulos-Davidson-Luo 06]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 51

53 A Randomization Example Relevant to Multicast Beamforming Consider the problem min x C n where C,A 1,...,A m 0. x H Cx s.t. x H A i x 1, i = 1,...,m, ( ) Box 2. Gaussian Randomization Procedure for ( ) given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l CN(0, X ), and construct a feasible point x l = ξ l min i=1,...,m ξ Hl A iξ l ; end determine l = arg min l=1,...,l xh l C x l. output ˆx = x l as an approximate solution to ( ). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 52

54 0.6 random samples ξ QCQP approx. samples x globally opt. QCQP soln. x x x Illustration of randomizations in R 2, for Problem ( ). The gray area is the feasible set and colored lines the contour of the objective. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 53

55 SDR lower bound BF via SDR successive opt. [Kim-Love-Park 11] Transmit Power (db) Number of Users, K Performance of SDR-based multicast beamforming with respect to the number of users. γ = 10dB. N t = 4. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 54

56 Extension to Complex-Valued Separable QCQP Consider a further extension, called complex-valued separable QCQP: min x 1,...,x k C n s.t. k x H i C i x i i=1 k x H l A i,l x l i b i, l=1 i = 1,...,m. By writing X i = x i x H i obtain an SDR for all i, and then semidefinite-relaxing them, we min X 1,...,X k H n s.t. k Tr(C i X i ) i=1 k Tr(A i,l X l ) i b i, l=1 X 1 0,...,X k 0. i = 1,...,m, A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 55

57 Application: Unicast Transmit Downlink Beamforming Scenario: Multiuser MISO downlink; each user receives an individual data stream. Transmit signal: User 1 x(t) = K w i s i (t), i=1 where s i (t) C is the data stream for user i, & w i C N t its tx. beamvector. Received signal of user i: Basestation User 2 y i (t) = h H i x(t) + v i (t) = h H i w i s i (t) + h H i w l s l (t) +v i (t). l i } {{ } interference A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 56

58 The signal-to-interference-and-noise ratio (SINR) of user i is given by where R i = h i h H i SINR i = w H i R iw i l i wh l R iw l + σi 2, for instant. CSIT, and R i = E{h i h H i Problem: Given users SINR requirements γ 1,...,γ K, solve min K w 1,...,w K C N i=1 w i 2 t s.t. SINR i γ i, i = 1,...,K. } for stat. CSIT. Write W i = w i w H i. The SDR of ( ) is K min W 1,...,W K H N i=1 Tr(W i) t s.t. Tr(R i W i ) γ i ( l i Tr(R iw l ) + σi 2 ), i = 1,...,K, W 1,...,W K 0. ( ) ( ) ( ) is shown to have a rank-one solution for R 1,...,R K 0, via uplink-downlink duality [Bengtsson-Ottersten 01]. Thus, the SDR is tight! In Part III, we will introduce an easier way to establish the tightness of SDRs. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 57

59 Total Transmit Power (db) BF via zero-forcing BF via SDR SINR requirement γ (db) Performance of unicast beamforming with respect to the SINR requirement. N t = K = 8, γ = γ 1 = = γ K. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 58

60 Additional Remarks about the Transmit Beamforming Application Transmit beamforming is now a key topic. For review articles, see [Gershman- Sidiropoulos-Shahbazpanahi-Bengtsson-Ottersten 10], [Luo-Chang 10]. From the original unicast and multicast beamforming problems, numerous extensions are emerging e.g., multicell coordinated beamforming, cognitive radio beamforming, relay beamforming, secrecy beamforming, and energy harvesting beamforming and they will be described in Part IV.A. All these beamforming problems turn out to be, or be closely related to, nonconvex QCQPs, and hence SDR plays a key role. In the transmit beamforming context, SDR is not just a direct application. There are new developments that were not previously seen even in optimization; they will be described in Part IV.B. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 59

61 Part III: Semidefinite Relaxation Theory A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 60

62 Provable Approximation Accuracies: Motivation So far we have introduced several procedures for generating an approximate QCQP solution from an SDR solution. A natural question arises: How good are these procedures? Of course, their performance can be observed empirically. However, can we prove something about their approximation accuracy? Such theoretical results can provide strong justification for the use of SDR in various problem settings. To measure the performance of a particular procedure, one intuitive approach is to quantify the gap between the objective value of the QCQP solution generated by the procedure and the optimal value of the QCQP. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 61

63 Provable Approximation Accuracies: Setup Let v(x) = x T Cx, and denote the optimal values of (QCQP) and (SDR) by v QP = min x T Cx s.t. x T A i x i b i, i = 1,...,m; v SDR = min Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. Moreover, let ˆx be an approximate solution to (QCQP), obtained using one of the solution generation procedures (e.g., randomization). Note that v(ˆx) v QP. We are interested to know if there exists a finite number γ 1 (called the approximation ratio) such that v(ˆx) γv QP either in expectation, or with high probability, or almost surely (since ˆx can be random). In general, the smaller γ, the better the solution generation procedure. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 62

64 Provable Approximation Accuracies: Remarks In the definition of approximation ratio, we are implicitly assuming that v QP,v SDR > 0. The notion of approximation ratio can be defined for problems where v QP 0. However, we shall not go through it in this tutorial. Given a solution generation procedure, we are usually interested in its performance on arbitrary instances of (QCQP). Thus, the approximation ratio γ should not depend on the problem data {A 1,...,A m, b, C}. However, it could depend on the problem dimensions m, n. For quadratic maximization problems, the notion of approximation ratio can be defined similarly. The problem of proving approximation accuracies has been of great interest to optimization theorists, and it has enormous implications in practice. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 63

65 The Seminal Approx. Accuracy Result by Goemans & Williamson Consider v QP = max x T Cx x R n s.t. x 2 i = 1, i = 1,...,n, 1 w 13 w with C 0, C ij 0 for all i j. Such a problem arises in the so-called MAXCUT in combinatorial optimization. 2 w 25 5 In [Goemans-Williamson 95], it was shown that if the randomization procedure in Box 1 is used, then γv QP E{v(ˆx)} v QP, where γ In particular, the approximation ratio is independent of the problem dimension n. In the context of MAXCUT, this means that the approximation accuracy is independent of the number of vertices in the graph. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 64

66 Complex k-ary Quadratic Maximization Consider the problem v QP = max x H Cx x C n s.t. x i {1, ω,...,ω k 1 }, i = 1,...,n, (CQP-k) where C 0 and ω = exp(j2π/k) is the kth root of unity, for some given integer k 2. This is a generalization of the problem considered by Goemans and Williamson. Since x i 2 = 1 for all i, (CQP-k) can be handled by SDR. Specifically, v SDR = max Tr(CX) X H n s.t. X 0, X ii = 1, i = 1,...,n. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 65

67 Randomization Procedure for Complex k-ary Quad. Max. Again, a Gaussian randomization procedure can be used to generate a feasible solution to (CQP-k) from an SDR solution. Box 3. Gaussian Randomization Procedure for CQP-k given an SDR solution X, and a number of randomizations L. for l = 1,...,L generate ξ l CN(0,X ), and construct the feasible point x l C n, where [ x l ] i = f([ξ l ] i ) and f(z) = 1, arg(z) [ π/k, π/k), ω,. arg(z) [π/k, 3π/k),. ω k 1, arg(z) [(2k 3)π/k, (2k 1)π/k); end determine l = arg max l=1,...,l xh l C x l. output ˆx = x l as the approximate solution to (CQP-k). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 66

68 Pictorial Illustration of the Randomization Procedure, for k = A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 67

69 Approx. Accuracy Result for Complex k-ary Quad. Max. In [So-Zhang-Ye 07], it is shown that if the randomization procedure in Box 3 is used, then γv QP E{ˆx H Cˆx} v QP, (k sin(π/k))2 where γ =. 4π If we take k =, then the k-ary constraints in (CQP-k) become x i = 1, i = 1,...,n. ( ) In [So-Zhang-Ye 07] it is shown that by letting the function f in Box 3 to be f(z) = { z/ z, z > 0, 0, z = 0, the randomization procedure would yield γ = π/4 for the unit-modulus constraints ( ). It is interesting (and comforting) to note that lim k (k sin(π/k)) 2 4π = π 4. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 68

70 Applications of Complex k-ary Quadratic Maximization (CQP-k) has many applications in signal processing, e.g.: blind orthogonal space-time block code detection [Zhang-Ma 09] radar code waveform design [De Maio et al. 09] distributed detection over multiple-access channels [Banavar-Smith- Tepedelenlioğlu-Spanias 12] A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 69

71 Boolean QP max x R n Complex k-ary QP problem approx. accuracy γ; see (21)-(22) for def. references x T Cx s.t. x 2 i = 1, i = 1,..., n γ = , C 0, C ij 0 i j 2/π , C 0 1 (opt.), C ij 0, i j Goemans-Williamson [2], Nesterov [3], Zhang [6]. Relevant applications: [24] [26] max x C n x H Cx s.t. x i {1, ω,..., ω k 1 }, i = 1,..., n where ω = e j2π/k, and k > 1 is an integer. Complex constant-modulus QP max x C n s.t. x H Cx x i 2 = 1, i = 1,..., n For C 0, γ = (k sin(π/k))2. 4π e.g., γ = for k = 8, γ = for k = 16. For C 0, γ = π/4 = Remark: coincide with complex k-ary QP as k. Zhang-Huang [7], So-Zhang-Ye [8]. Relevant applications: [27], [37] Zhang-Huang [7], So-Zhang-Ye [8]. max x C n s.t. x H Cx ( x 1 2,..., x n 2 ) F where F R n is a closed convex set. max x R n s.t. x T Cx where A 1,..., A m 0. x T A i x 1, i = 1,..., m The same approx. ratio as in complex constant-modulus QP; i.e., γ = π/4 for C 0. If the problem is reduced to the real-valued case, then the approx. ratio results are the same as that in Boolean QP. For any C S n, γ = 1 2ln(2mµ) where µ = min{m, max i rank(a i )}. Ye [4], Zhang [6]. Nemirovski-Roos-Terlaky [5]. Extensions: Ye [72], Luo-Sidiropoulos- Tseng-Zhang [9] and So-Ye- Zhang [71]. Known approximation accuracies for quadratic maximization problems. The reference numbers refer to those in our Signal Processing Magazine article. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 70

72 Approx. Accuracy Result for Quadratic Minimization Consider now the problem v QP = min x T Cx x R n s.t. x T A i x 1, i = 1,...,m, ( ) where C, A 1,...,A m 0. This arises in the study of multicast beamforming. It was shown in [Luo-Sidiropoulos-Tseng-Zhang 07] that if the randomization procedure in Box 2 is used, then with high probability (instead of just in expectation), v QP v(ˆx) γv QP, where γ = 27m 2 /π. For the complex version of ( ), one has a better approximation ratio: γ = 8m. Notice that this ratio accommodates the worst possible problem instance {C, A 1,...,A m }. In practice, the approximation accuracies are usually much better a phenomenon that deserves further investigation. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 71

73 Interpretation in Multicast Transmit Beamforming Recall that in the context of multicast transmit beamforming, we encounter the following optimization problem: min w C N s.t. w 2 1 γ i σi 2 w H R i w 1, i = 1,...,K. The aforementioned approximation accuracy result thus says that SDR together with the randomization procedure can produce a beamvector that satisfies all the prescribed SNR requirements and whose power is at most 8K times the optimal. Again, this is just a worst-case guarantee. In practice, the performance is usually much better. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 72

74 min x C n s.t. x H Cx where A 1,..., A m 0. MIMO Detection problem approx. accuracy γ; see (18)-(19) for def. references x H A i x 1, i = 1,..., m min x R n y Hx 2 2 s.t. = 1, i = 1,..., n x 2 i where y = Hs+v; H C n n has i.i.d. standard complex Gaussian entries; s 2 i = 1 for i = 1,..., n; and v C n has i.i.d. complex mean zero Gaussian entries with variance σ 2. γ = 8m. If the problem is reduced to the real-valued case, then γ = 27m2 π. For σ 2 60n (which corresponds to the low signal-to-noise ratio (SNR) region), with probability at least 1 3 exp( n/6), γ For σ 2 = O(1) (which corresponds to the high SNR region), with probability at least 1 exp( O(n)), i.e. the SDR is tight. γ = 1, Luo-Sidiropoulos-Tseng-Zhang [9]; see also So-Ye-Zhang [71]. Relevant applications: [29] Kisialiou-Luo [67], So [69]. Extensions: So [68], [69]. Related: Jaldén-Ottersten [66]. Relevant applications: [17] [20], [22], [23] Known approximation accuracies for quadratic minimization problems. The reference numbers refer to those in our Signal Processing Magazine article. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 73

75 Rank Reduction in SDR The SDR methodology introduced so far can be summarized as follows: 1) formulate a hard problem (nonconvex QCQP) as a rank-one-constrained SDP 2) remove the rank constraint to obtain an SDP 3) use some methods, such as randomizations, to produce an approximate solution to the original problem. It is natural to expect that the lower the rank of the SDP solution, the better the approximation. Unfortunately, we cannot guarantee a low rank solution for the SDP in general. However, we can identify special cases where the SDP solution rank is low or even equal to one. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 74

76 Shapiro-Barvinok-Pataki (SBP) Result Consider the real-valued SDP min X S n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. SBP Result [Pataki 98]: There exists an optimal solution X such that rank(x )(rank(x ) + 1) 2 m. In particular, SBP result implies that for m 2, a rank-1 X exists. Hence, For a real-valued QCQP with m 2, SDR is tight; i.e., solving the SDR is equivalent to solving the original QCQP. Note that a rank reduction algorithm may be required to turn an SDP solution to a rank-one solution [Ye-Zhang 03]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 75

77 Complex Extension of the Rank Reduction Result Let us now consider the complex-valued SDP min X H n Tr(CX) s.t. X 0, Tr(A i X) i b i, i = 1,...,m. In this case, the SBP result can be generalized to [Huang-Zhang 07] rank(x ) 2 m. As a direct corollary, we have For a complex-valued QCQP with m 3, SDR is tight. A complex rank-1 decomposition algorithm for m 3 is available [Huang- Zhang 07]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 76

78 Application Revisited: Multicast Beamforming User 1 Recall the multicast beamforming problem: min w C N t w 2 s.t. SNR i = wh R i w σ 2 i i = 1,...,K, γ i, K being the number of users. Basestation User 2 By the SBP result, SDR solves the multicast problem optimally for K 3. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 77

79 Further Extension of the Rank Reduction Result Recall the problem k min X 1,...,X k H n i=1 Tr(C ix i ) k s.t. l=1 Tr(A i,lx l ) i b i, X 1 0,...,X k 0, i = 1,...,m, which is an SDR of the so-called separable QCQP. We have the following generalization of the SBP result [Huang-Palomar 09]: k i=1 rank(x i )2 m. Consequently, Suppose that an SDR solution {X i } i satisfies X i SDR is tight for m k for all i. Then, the A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 78

80 Application Revisited: Unicast Beamforming User 1 Recall the design problem min w 1,...,w K C N t s.t. K i=1 w i 2 w H i R iw i l i wh l R iw l + σi 2 i = 1,...,K, γ i, ( ) Basestation User 2 which is a separable QCQP with K variables (beamvectors) and K constraints (SINR req.). By the aforementioned result, SDR solves ( ) optimally for any R 1,...,R K 0. And hey, it s still fine if you put two more quadratic constraints in ( )! A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 79

81 Part IV.A: Transmit Beamforming A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 80

82 SDR in Transmit Beamforming Many forefront advances of SDR we see recently lie in transmit beamforming (BF) optimization. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 81

83 Current Development of SDR in Transmit Beamforming Apart from unicast and multicast BF, we have seen numerous extensions: multigroup multicast [Karipidis-Sidiropoulos-Luo 08] cognitive radio BF [Phan-Vorobyov-Sidiropoulos-Tellambura 09] relay beamforming one-way relay beamforming [Fazeli-Dehkordy-Shahbazpanahi-Gazor 09], [Chalise-Vandendorpe 09] two-way relay beamforming (a.k.a. analog network coding) [Zhang-Liang- Chai-Cui 09] interference neutralization [Ho-Jorswieck 12] multicell coordinated beamforming [Bengtsson-Ottersten 01], [Dahrouj- Yu 10], [Shen-Chang-Wang-Qiu-Chi 12] secrecy beamforming [Liao-Chang-Ma-Chi 11] energy harvesting [Xu-Liu-Zhang 13], [Chalise-Ma-Zhang-Suraweera- Amin 13] A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 82

84 Overview Our focus: QCQP-SDR perspective on various transmit BF problems. A glimpse of some nice formulations. What we will not go through: alternative solution approaches and comparison second-order cone program (SOCP) (for unicast BF with instant. CSIT only) [Wiesel-Eldar-Shamai 06] uplink-downlink duality (for unicast BF only) [Schubert-Boche 04] A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 83

85 Multi-Group Multicast Beamforming A natural generalization of unicast and multicast BF. Scenario: Multiuser MISO downlink with M groups of users, & with each group receiving the same info. [Karipidis-Sidiropoulos-Luo 08]. {z} group 1 {z} group 2 {z} group 3 A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 84

86 Transmit signal: x(t) = M m=1 w m s m (t), where s m (t) C is the data stream for group m, & w m C N t its beamvector. Received signal of user k in the mth group: y m,k (t) = h H m,kx(t) + v m,k (t) = h H m,kw m s m (t) + l mh H m,kw l s l (t) +v m,k (t), }{{} inter-group interference where k = 1,...,K m, & K m is the number of users in the mth group. SINR: w H SINR m,k = mr m,k w m l m wh l R m,kw l + σm,k 2, where R m,k = h m,k h H m,k for instant. CSIT, & R m,k = E{h m,k h H m,k} for stat. CSIT. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 85

87 Problem: min w 1,...,w M C N t M m=1 w m 2 s.t. SINR m,k = w H mr m,k w m l m wh l R m,kw l + σm,k 2 where γ m,k s are prescribed SINR requirements. γ m,k, k = 1,...,K m, m = 1,...,M, A separable QCQP with M variables, w 1,...,w M, and M m=1 K m constraints. By the SBP rank reduction result in Part III, SDR has rank-1 solution (and solves the BF problem optimally) when K 1 3, K m = 1 m 1 (one group serving 3 users, the others 1 user); K 1 2, K 2 2, K m = 1 m 1,2 (two groups serving 2 users, the others 1 user). For non-rank-1 instances, solution approx. can be done by a (more sophisticated) Gaussian randomization procedure [Karipidis-Sidiropoulos-Luo 08]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 86

88 Cognitive Radio (CR) Beamforming Goal: Access the channel owned by primary users through spectrum sharing. Scenario: MISO downlink with the CR (or secondary) system. Primary Tx Primary Rx Secondary Tx Secondary Rx Idea: Avoid excessive interference to the primary users through tx. opt. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 87

89 CR spectrum-sharing model: K secondary users (SUs), L single-antenna primary users (PUs) tx. and rx. model for SUs: same as the previous multicast or unicast model interference to the lth PU given by g H l w 2, where g l is the channel from the secondary transmitter to the lth PU known CSIT from the secondary transmitter to the PUs Design for the multicast case [Phan-Vorobyov-Sidiropoulos-Tellambura 09]: min w w 2 s.t. SNR SU,i = w H R i w/σ 2 i γ, i = 1,...,K, w H G l w δ l, l = 1,...,L, (interference temperature (IT) constraints) where G l is the CSIT of lth PU (defined in the same way as R k ); δ l is the tolerable interference level to the lth PU; γ is SUs SNR requirement. By the SBP result, SDR is tight when K 2, L = 1 ( 2 SUs, 1 PU). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 88

90 Design for the unicast case (see, e.g., [Zhang-Liang-Cui 10]): min w 1,...,w K K w k 2 k=1 w H i s.t. SINR SU,i = R iw i l i wh l R kw l + σi 2 γ i, i = 1,...,K, K k=1 wh k G lw k δ l, l = 1,...,L. (IT constraints) A separable QCQP with K variables and K + L constraints. By the SBP result, SDR is tight if L 2 (two PUs or less). Remark: For instant. CSIT with SUs, SDR can be shown to be rank-1 optimal for any L. Alternatively, it can be reformulated as an SOCP. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 89

91 One-Way Relay Network Beamforming Scenario: One-way cooperative communication by a network of N single-antenna amplify-forward (AF) relays, K tx-rx pairs [Fazeli-Dehkordy-Shahbazpanahi- Gazor 09]. Phase I Source to Relay Phase II Relay to Destination Tx 1 f 1;1f1;2 f 1;N f K;1 f K;2 1 2 g K;1 g 1;1 g 1;2 g 1;N g K;2 g K;N Rx 1 f K;N Tx N Rx Goal: Design the AF weights so that the SINR requirements are met, and the total relay tx. power is minimized. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 90

92 System model: rx. signals for the source-to-relay link: K r(t) = f i s i (t) + n(t), i=1 where r(t) = [ r 1 (t),...,r N (t) ], r i (t) being the rx. signal of relay i; s i (t) is the data stream from source i to destination i; f i C N the channel from source i to the relays; n(t) is noise with covariance Σ n = Diag(σ 2 n,1,...,σ 2 n,n ). AF process: x(t) = Wr(t), where W = Diag(w 1,...,w N ); w i is the AF weight at relay i. rx. signals for the relay-to-destination link: y i (t) = g H i x(t) + v i (t), i = 1,...,K, where g i is the channel from the relays to destination i; v i (t) is noise with variance σ 2 v,i. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 91

93 Assuming instant. CSIT (for ease of illustration), we have SINR i = k i gh i Wf k 2 }{{} interference g H i Wf i 2 + g H i WΣ n W H g }{{} i +σv,i 2 noise amplification due to AF, i = 1,...,K. Problem: Let w = [ w 1,...,w N ] T C N. Solve min w E{ x(t) 2 } = w H Cw s.t. SINR i = w H A i w w H B i w + σ 2 v,i γ i, i = 1,...,K, where A i = (f i g i)(f i g i) H, B i = k i (f k g i)(f k g i) H + Diag( g i,1 2 σ 2 n,1,..., g i,n 2 σ 2 n,n ), C = Diag( f 1 2 +σ 2 n,1,..., f N 2 +σ 2 n,n ). The problem is a QCQP with K constraints; SDR is tight for K 3. Remark: While this relay application has some unicast flavor i.e., one data stream for one user it does not imply that the same SDR tightness result in standard unicast BF holds for the relay BF problem. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 92

94 One-Way MIMO Relay Beamforming Scenario: One-way relaying by an MIMO AF relay, K tx-rx pairs [Chalise- Vandendorpe 09]. Phase I Source to Relay Phase II Relay to Destination Tx 1 f 1 g 1 Rx 1 f K Relay g K Tx Rx Everything is the same as that in the last relay example, except that a matrix AF process is considered: x(t) = Wr(t), where W C N N is a general N N matrix (instead of being diagonal). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 93

95 Let w = vec(w) C N2. The design problem (after some careful derivations): min w E{ x(t) 2 } = w H Cw s.t. SINR i = w H A i w w H B i w + σ 2 v,i γ i, i = 1,...,K, where A i = (f i g i)(f i g i) H, B i = k i (f k g i)(f k g i) H +Σ T n (g i g H i ), & C = ( K i=1 f i ft i + ΣT n) I. Again, SDR is tight for K 3. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 94

96 Two-Way Relay Beamforming Scenario: Two-way communication between two users, using an MIMO AF relay [Zhang-Liang-Chai-Cui 09] h 1 h 2 Relay User 1 User 2 Phase I: Two users transmit r(t) = h 1 s 1 (t)+h 2 s 2 (t)+n(t). Phase II: Matrix AF relaying h 1 h 2 Relay User 1 User 2 x(t) = Wr(t). In addition, the users can self-cancel their previously tx. data. y 1 (t) = h H 1 x(t) + v 1 (t) = h H 1 Wh 1 s 1 (t) }{{} self interference, cancelled y 2 (t) = h H 2 x(t) + v 2 (t) = h H 2 Wh 1 s 1 (t) + h H 2 Wh 2 s 2 (t) }{{} self interference, cancelled +h H 1 Wh 2 s 2 (t) + h H 1 Wn(t) +v }{{} 1 (t), noise amp. +h H 1 Wn(t) +v }{{} 2 (t). noise amp. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 95

97 The design problem min W E{ x(t) 2 } s.t. SNR 1 = SNR 2 = h H 1 Wh 2 2 h H 1 WΣ nw H h 1 + σ 2 v,1 h H 2 Wh 1 2 h H 2 WΣ nw H h 2 + σ 2 v,2 γ 1, γ 2 can be converted to a 2-constraint QCQP, by applying w = vec(w) C N2 (the same way as in the last example). Hence, SDR solves the two-relaying BF problem optimally. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 96

98 Physical-Layer Security Scenario: One intended user (Bob) and multiple illegitimate users (Eves). Goal: Block eavesdropping by degrading illegitimate users QoS. Transmit beam for Bob Transmitter (Alice) Artificial noise Legitimate receiver (Bob) Eavesdropper (Eve) Idea: Use spatially selective artificial noise (AN) to jam illegitimate users [Swindlehurst 09], [Liao-Chang-Ma-Chi 11]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 97

99 tx. signal: x(t) = ws(t) + z(t), where z(t) CN(0,Σ) is AN. SINRs: SINR Bob = h H w 2 Tr(Σhh H ) + σn 2, SINR Eve,i = g H i w 2 Tr(Σg i g H i ) +, i = 1,...,L, σ2 v,i where h and g i are the channels of Bob and Eve i, resp. Problem: Given an SINR specification (γ,β), solve min w,σ 0 w 2 + Tr(Σ) s.t. SINR Bob γ, SINR Eve,i β, i = 1,...,L. ( ) ( ) provides a secrecy rate guarantee log(1 + γ) log(1 + β). ( ) can be handled by SDR, by replacing W = ww H with W 0. By the SBP result, you can (immediately!) declare that SDR is tight for L 2. By exploiting specific problem structures of ( ), it is proven that SDR always gives rank-1 solution with W for any L [Liao-Chang-Ma-Chi 11]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 98

100 Isotropic AN design No AN design Proposed AN design Total Transmit Power (db) /σ 2 v, in db Transmit power performance of various secret BF designs. N t = 4; L = 3; σ 2 n = 0dB; γ = 10dB; β = 0dB. No-AN design refers to ( ) without AN. Isotropic AN design refers to a closed-form design in which w = αp max h/ h, Σ = (1 α)p max (I hh H / h 2 ), with P max being the total tx power and 0 < α 1 being a power allocation factor. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 99

101 Energy Harvesting Scenario: Unicast multiuser MISO downlink, with energy harvesting (EH) receivers that can harvest energy from radio signals. Goal: Simultaneous information transmission and wireless power transfer via BF [Xu-Liu-Zhang 13], [Chalise-Ma-Zhang-Suraweera-Amin 13]. EH Receiver ID User A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 100

102 System model: K information decoding (ID) users, L EH receivers. tx. signal model: K M x(t) = w i s i (t) + v i u i (t), i=1 where s i (t) & w i are ID user i s data steam & beamvector, resp.; every u i (t) is an energy-carrying (& no-info.) signal for transfering energy to EH receivers; v i is the corresponding energy beamvector. Power harvested by EH receiver i: P EH i = ζ i E{ y EH i (t) 2 } = ζ i i=1 ( M v H i G i v i + i=1 ) K w H i G i w i, where y EH i (t) is EH receiver i s rx. signal; ζ i the EH efficiency; G i the CSIT. ID users SINRs: Every ID user is assumed to know u i (t) and can cancel them from the rx. signal. By letting R i to be the CSIT of ID user i, the SINRs are i=1 SINR ID i = w H i R iw i l i wh l R iw l + σi 2, i = 1,...,K. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 101

103 Problem: Given ID users SINR requirements and EH receivers requirements, denoted by γ 1,...,γ K and β 1,...,β L, resp., solve min {w i },{v i } K w i 2 + i=1 s.t. SINR ID i P EH i M v i 2 i=1 γ i, i = 1,...,K, β i, i = 1,...,L. At first sight, one may do the following SDR formulation: Let W i = w i w H i,i = 1,...,K, V i = v i v H i, i = 1,...,L, and then SDR them. Specific problem structures can be exploited to formulate a simpler SDR. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 102

104 Recall the design problem min {w i },{v i } K i=1 w i 2 + Tr(( M i=1 v iv H i )) s.t. SINR ID i = P EH i = ζ i w H i R iw i l i wh l R iw l + σi 2 ( Tr ( G i ( M i=1 v iv H i ) ) + Tr γ i, i = 1,...,K, ( G i ( K i=1 w iw H i ) )) β i, i = 1,...,L. Let W i = w i w H i, i = 1,...,K, V = M i=1 v iv H i, where one should note that V 0, rank(v ) M. Then, SDR {W i }, V. The resulting SDR has less design variables than the previously mentioned SDR. Also, by the SBP result, SDR has rank-1 solution w.r.t. {W i } for L 2 (two EH receivers or less); SDR has rank r 1 solution w.r.t. V for L 2; i.e., when there are two EH receivers or less, it suffices to use M = 1, or one energy beam. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 103

105 EH beamforming without energy beams EH beamforming with energy beams Total Transmit Power (Watt) EH Receivers Power Requirement β (Milliwatt) Transmit power performance of EH beamforming with respect to the EH receivers power requirement. EH beamforming with energy beams refers to the EH formulation in the previous slide, while EH beamforming without energy beams refers to the same formulation without v i s. N t = 8, K = 6, L = 2, γ 1 = = γ K = 5dB, β = β 1 = β 2, ζ 1 = ζ 2 = 0.5. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 104

106 Multicell Coordinated Beamforming Motivation: Provide better interference management by coordinating the transmissions of base stations at different cells. Scenario: Unicast MISO downlink in a multicell scale [Dahrouj-Yu 10], [Bengtsson-Ottersten 01]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 105

107 tx. signal of ith cell: x i (t) = K i j=1 w i,j s i,j (t), i = 1,...,N, where s i,j (t) and w i,j are the tx. stream and beamvector for user j in the ith cell, resp.; K i is the no. of users in the ith cell; N is the no. of cells. rx. signal of user j in the ith cell: y i,j (t) = h H i,i,jx i (t) + m ih H m,i,jx m (t) + v i,j (t), j = 1,...,K i, where h m,i,j is the channel from mth cell to user j in the ith cell. Define CSIT R m,i,j in the same way as before. SINR: SINR i,j = w H i,j R i,i,jw i,j w H i,lr i,i,j w i,l + w H m,nr m,i,j w m,n l j m i n }{{}}{{} intra-cell interference inter-cell interference +σ 2 i,j. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 106

108 Problem: N min {w i,j } s.t. i=1 K i j=1 w i,j 2 w H i,j R i,i,jw i,j l j wh i,l R i,i,jw i,l + m i j = 1,...,K i, i = 1,...,N. n wh m,nr m,i,j w m,n + σ 2 i,j γ i,j, ( ) While ( ) looks complicated, one can observe that ( ) is a QCQP with N i=1 K i variables & N i=1 K i constraints; by the SBP result, ( ) can be optimally solved by SDR. A recent direction: Distributed multicell coordinated BF practically desirable, free from centralized opt. (requires a central station) in the SDR context, the challenge is the same as solving SDP distributively can be achieved by application of distributed opt. methods, e.g., alternating direction method of multipliers (ADMM) [Shen-Chang-Wang-Qiu-Chi 12] A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 107

109 In ADMM, the idea is to reformulate the SDR of ( ) in a consensus opt. form: min Tr(W i,j ) {W i,j } j N j min Tr(R i,i,j W i,j ) {t i,j,u m,i,j }, s.t. {u public i=1 l j m,i,j } Tr(R i,i,jw i,l ) + t i,j + σi,j 2 γ i,j, i,j, W i,1,...,w i,ki 0, u i,m,j = n Tr(R i,m,jw i,n ), m i,j s.t. u m,i,j = u public m,i,j, m, i,j, m i, t i,j = m i upublic m,i,j, i,j, where t i,j is the sum intercell interference (ICI) from other cells to user j in cell i; u m,i,j (resp. u public m,i,j ) is local (resp. public) ICI from cell m to user j in cell i. Then, one can apply ADMM to solve the above problem distributively (details skipped). In layman terms, at each iteration ADMM does the following: Each cell solves a single-cell problem, but with ICI awareness. Cells exchange their local ICI info. and update public ICI info. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 108

110 Other Design Formulations For tutorial purposes, we so far considered only the QoS-constrained power minimization design formulation. SDR can also handle design formulations such as max-min fairness; rate profile problems; joint BF and user selection [Matskani-Sidiropoulos-Luo-Tassiulas 08], [Wai-Ma 12]. Can SDR be employed to tackle even more challenging design formulations, particularly, the (NP-hard) sum rate maximization (SRM) problem? Tricky, if not impossible... Doable when we consider a harder version of SRM, namely, discrete SRM (DSRM) [Wai-Li-Ma 13]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 109

111 Discrete Sum Rate Maximization Scenario: Unicast multiuser MISO downlink. Problem: Maximize the discretized sum rate under the total power constraint: where SINR i = max w 1,...,w K K i=1 ϕ (log 2(1 + SINR i )) w H P i R iw i l i wh l R iw l +σ i 2 ϕ(r) = s.t. K i=1 w 2 P, with R i = h i h H i R M. if R M r, R 1 if R 1 r < R 2, 0 if 0 r < R 1, (instant. CSIT); (DSRM) with 0 < R 1 < < R M being the supported rate values; P is the power limit. DSRM is motivated by finite rate constraints in practical modulation and coding schemes [Cheng-Philipp-Pesavento 12]. DSRM subsumes joint BF and user selection, wherein M = 1. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 110

112 Consider the simple case of M = 1, where ϕ(r) = 0 if 0 r < R 1 & ϕ(r) = R 1 if R 1 r. Since R 1 w H log 2 (1 + SINR i ) 2 R1 i 1 + R iw i l i wh l R iw l + σi 2 f i ({w l }) w H 1 l R i w l 2 R1 1 wh i R i w i + σi 2 0, l i we can write ϕ(log 2 (1 + SINR i )) = R 1 R 1 ψ(f i ({w l })), where ψ(x) is the unit step function. Idea: Apply the approx. (commonly seen in compressive sensing) ϕ(log 2 (1 + SINR i )) R 1 R 1 max{0, f i ({w l })}. Such an idea can be extended to M > 1. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 111

113 Using the aforementioned step-to-hinge approx. and then SDR, (DSRM) can be approximated by max {W l } K i=1 M (R i 1 R i )max m=1 0, Ri + Ri σ 2 i Tr R i l i W l 1 2 W Ri 1 i s.t. W 1,...,W K 0, where R 0 = 0. (DSRM-SDR) is convex. K i=1 Tr(W i) P, (DSRM-SDR) Technical remarks: - (DSRM-SDR) guarantees rank-one optimal solution with {W l } if we add a small regularization term ǫ K i=1 Tr(W i), ǫ > 0, in the objective. - However, (DSRM-SDR) is not tight with its discrete rate approx. An iterative refinement procedure is employed to generate an approx. solution; see [Wai- Li-Ma 13] for details. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 112

114 Sum rate (bits/channel use) WMMSE (Unquantized) WMMSE (Quantized) Alg. 1 in [Cheng-Philipp-Pesavento 12] DSRM-SDR (with iterative refinement) Power budget P (db) Sum rate performance with respect to the total power limit P. N t = 6, K = 8, & the discrete rate set follows that in 3GPP LTE standard (M = 15, from R 1 = 0.15bits to R M = 5.55bits). WMMSE refers to the SRM algorithm in [Shi-Razaviyayn-Luo-He 11], which does not take into account finite rate constraints. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 113

115 Part IV.B: Advanced Topics in Transmit Beamforming Topic 1. Rank-Two Beamforming A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 114

116 Motivation Recall that the rationale of SDR is to use a rank-unconstrained SDP to find a rank-1 solution W, which can be physically realized by BF. A natural question arises: Can we do rank-2 SDR? Suppose that we want to do this. Then, there are two challenges to tackle: From a communication viewpoint, we never said BF is the only way to go. The question is how to design an alternative transmit scheme. From an optimization viewpoint, how to proceed with solution generation, and what is its theoretical performance? Solution: A combination of BF and the Alamouti space-time block code. The SDP rank reduction theory in [So-Ye-Zhang 08]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 115

117 The Alamouti Space-Time Block Code Let s = [ s 1,s 2 ] T. The Alamouti code is C(s) = [ ] s1 s 2 s 2 s, 1 which is arguably the most famous among space-time codes. Features: orthogonal: C(s)C H (s) = s 2 I easy to detect s simple performance characterization ideal choice for isotropic transmission in 2 1 MISO channels Extensions: Beamformed Alamouti coding, often for point-to-point MIMO [Jöngren-Skoglund-Ottersten 02], [Pascual-Iserte-Palomar-et al. 06],... Recent development: Beamformed Alamouti for multicasting [Wu-Ma-So 13] (also [Wu-So-Ma 12]), [Wen-Law-Alabed-Pesavento 12]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 116

118 BF Alamouti System Model Scenario: Multicast MISO downlink with instant. CSIT. Parse x(t) into blocks via X(n) = [ x(2n) x(2n + 1) ]. In block n, we transmit s(n) = [ s(2n) s(2n + 1) ] T by X(n) = BC(s(n)), where B C N 2 is a transmit beamforming matrix, and [ ] s1 s C(s) = 2 s 2 s 1 is the Alamouti space-time code. By utilizing the special structure of the Alamouti code, rx signals can be equivalently turned to SISO models with SNRs SNR i = hh i BB H h i σi 2. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 117

119 Rank-2 Beamforming Problem Consider the power minimization design v QP2 = min B C N 2 Tr(BBH ) s.t. h H i BB H h i /σ 2 i γ, i = 1,...,K. (PM-ALAM) Observe that W = BB H W 0 and rank(w) 2. Hence, upon letting A i = h i h H i /(σ2 i γ), (PM-ALAM) can be reformulated as max Tr(W) W H N s.t. Tr(A i W) 1, i = 1,...,K, W 0, rank(w) 2. Let us do the same trick dropping the rank constraint. Then, we get max Tr(W) W H N s.t. Tr(A i W) 1, i = 1,...,K, W 0, which is the same SDR as that for the previous beamforming problem! A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 118

120 Let W be a solution to (SDR). Gaussian Randomization If W satisfies W = ˆB ˆB H, or rank(w ) 2, then we are done ˆB is optimal to (PM-ALAM). Now, consider instances for which rank(w ) > 2. randomization for rank-2 W? Can we do Gaussian The answer is yes. Box 4. Gaussian Randomization Procedure for (PM-ALAM) given an SDR solution W, and a number of randomizations L. for j = 1,...,L generate ξ 1,j,ξ 2,j CN(0, W ), and define B j = [ ξ 1,j ξ 2,j ]; let ˆB j = B q j min i=1,...,k Tr( B j BH j A i) end output ˆB = ˆB j, where j = arg min j=1,...,l Tr( ˆB j ˆBH j ). ; A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 119

121 SDR lower bound BF via SDR BF Alamouti via rank-2 SDR Transmit Power (db) Number of Users, K Performance of the rank-two SDR beamformed Alamouti scheme with respect to the number of users. γ = 10dB. N t = 4. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 120

122 Theoretical Performance of SDR-based BF Alamouti By the Shapiro-Barvinok-Pataki (SBP) rank reduction result (see Part III), we have When K 8, there exists a solution W of (SDR) whose rank satisfies rank(w ) 2. Implication: SDR exactly solves the rank-two beamforming problem for eight users or less (recall that for BF, we have three users or less). Let v(b) = Tr(BB H ). For K 8, the following result is established in [Wu-So-Ma 12], [Wu-Ma-So 13]: With high probability, the solution ˆB generated by the rank-2 Gaussian randomization procedure in Box 4 satisfies v QP2 v( ˆB) Kv QP2. This provable worst-case performance gain is better than that of BF, which is 8K. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 121

123 Remarks The ideas used in the rank-two SDR beamformed Alamouti scheme can also be applied to other scenarios, such as multicast relaying [Schad-Law-Pesavento 12]; multi-group multicast [Ji-Wu-So-Ma 13], [Law-Wen-Pesavento 13]; energy harvesting [Chalise-Ma-Zhang-Suraweera-Amin 13]. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 122

124 Extension: Multi-Group Multicast CR Networks Scenario: Multi-group multicast in a CR network. Problem: Max-min-fair design formulation under the rank-two beamformed Alamouti scheme: v MMF = max B 1,...,B M C N 2 s.t. min k=1,...,km m=1,...,m M m=1 M m=1 Tr(R m,k B m B H m) l m Tr(R m,kb l B H l ) + σ2 m,k Tr(B m B H m) P, Tr(G l B m B H m) δ l, l = 1,...,L, (IT constraints) (MMF) where P is the transmit power limit. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 123

125 Theoretical Results for Multi-Group Multicast CR Networks Let W m = B m B H m. Using the Huang-Palomar extension of the SBP rank reduction result, we obtain the following: When M m=1 K m M + 7, there exists an SDR solution {W m} with rank(w m) 2 for m = 1,...,M. Suppose now that M m=1 K m > M + 7. Let v({b m }) = min k=1,...,km m=1,...,m Tr(R m,k B m B H m) l m Tr(R m,kb l B H l ) +. σ2 m,k In [Ji-Wu-So-Ma 13], the following result is established: A Gaussian randomization procedure can generate a feasible solution { ˆB m } to (MMF) with rank( ˆB m ) 2 for all m. Moreover, with high probability, v MMF v({ ˆB m }) v MMF M. 8 m=1 K m(3 log 8(L + 1)) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 124

126 Theoretical Results for Multi-Group Multicast CR Networks The above results generalize those in [Chang-Luo-Chi 08], which concern rank-1 beamforming in the multi-group multicast scenario with no primary user. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 125

127 Further Discussion: Can We Do Rank-r Beamforming, r 3? From an optimization viewpoint, yes. For example, we can consider rank-3 SDR, wherein an effective power loss of O(K 1/3 ) can be proven. From a realizable physical layer viewpoint, not straightforward. A generalization of the Alamouti code is the class of orthogonal space-time block codes (OSTBCs). Full-rate OSTBCs do not exist for r > 2 [Liang-Xia 03]. For example, for r = 3, the maximal rate is 3/4, and the code is C(s) = s 1 s 2 s 3 0 s 2 s 1 0 s 3. s 3 0 s 1 s 2 The question of rank-r beamforming has led to new studies that are no longer about SDR e.g., stochastic beamforming [Wu-Ma-So 13], which can perform virtual rank-r beamforming for any r 1. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 126

128 Part IV.B: Advanced Topics in Transmit Beamforming Topic 2. Worst-Case Robust Beamforming A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 127

129 Motivation Scenario: Unicast multiuser MISO downlink, with instant. CSIT h 1,...,h K. User 1 Previously, we have formulated the following QoS-constrained power minimization problem: Basestation User 2 min w 1,...,w K C N K i=1 w i 2 s.t. w H i h i 2 l i wh l h i 2 + σi 2 γ i, i = 1,...,K. As seen before, this can be handled by SDR. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 128

130 Motivation Issue: CSIT is generally imperfectly known in practice. Presumed User 1 Suppose that the presumed CSIT, {h i }, is inaccurate. Basestation Actual User 1 Actual User 2 If we directly substitute the presumed CSIT into the standard power minimization design min w 1,...,w K C N K i=1 w i 2 s.t. w H i h i 2 l i wh l h i 2 + σi 2 γ i, i = 1,...,K Presumed User 2 and run it, then the resultant design may have severe SINR outage. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 129

131 Non robust Counts over different channel realizations Actual SINR satisfaction probability (Mean = ) Histogram of the actual SINR satisfaction probabilities of the non-robust QoS-constrained power minimization design. N t = K = 3; i.i.d. complex Gaussian CSI errors with zero mean and variance 0.002; γ = 11dB. The design has more than 50% outage most of the time. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 130

132 Worst-Case Robust Beamforming Problem Design goal: Guarantee the SINR requirements even in the worst-case scenario. Let W i = w i w H i, i = 1,...,K, and define SINR i ({W j }, h i ) = Tr(W i h i h H i ) l i Tr(W lh i h H i ) +. σ2 i We adopt the CSIT model h i = h i + e i, where h i is the presumed channel; e i is a deterministic unknown with e i r i (r i is known). Consider the following worst-case robust BF design problem: min K W 1,...,W i=1 Tr(W i) K s.t. SINR i ({W j }, h i + e i ) γ i, e i r i, i = 1,...,K, rank(w i ) 1, W i 0, i = 1,...,K. (RPM) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 131

133 By noting that SINR i ({W j },h i ) γ i h H i 1 W i W l h i σi 2, γ i l i (RPM) can be rewritten as min K W 1,...,W i=1 Tr(W i) K ( s.t. ( h i + e i ) H 1 γ i W i l i W l ( h i + e i ) σi 2, e i r i, i, rank(w i ) 1, W i 0, i = 1,...,K. (RPM) Again, the idea is to drop the rank constraints in (RPM). (RPM) without rank constraints, or SDR, is convex. However, it does not mean that the problem is easy there are infinitely many constraints w.r.t. the e i s. Question: Is the SDR of (RPM) efficiently solvable? The answer is yes, using the so-called S-lemma [Zheng-Wang-Ng 08]. ) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 132

134 S-Lemma S-Lemma: Let f 0 (x) = x H A 0 x + 2Re{b H 0 x} + c 0, f 1 (x) = x H A 1 x + 2Re{b H 1 x} + c 1, and suppose that there exists ˆx such that f 1 (ˆx) < 0. Then, f 0 (x) 0 for all x satisfying f 1 (x) 0 [ there exists ] λ [ 0 such ] that A0 b 0 A1 b b H + λ 1 0 c 0 b H 0. 1 c 1 ( ) In other words, the infinitely many constraints on the LHS of ( ) is equivalent to the linear matrix inequality on the RHS. The S-lemma is widely used in optimization, signal processing, communications and many other areas. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 133

135 Worst-Case Robust BF via SDR and S-Lemma By applying the S-lemma to (RPM) and dropping the rank constraints, we obtain the following SDR: min λ 1,...,λ K, W 1,...,W K K Tr(W i ) i=1 [ I s.t. h H i ] 1 γ i W i l i W l [ I hi ] + [ λi I 0 0 σ 2 i λ ir 2 i ] 0, i, λ 1,...,λ K 0, W 1,...,W K 0. (RPM-SDR) (RPM-SDR) is an SDP, with a finite number of constraints. A mysterious finding in simulations: Rank-one SDR solution is obtained in almost all the problem instances! A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 134

136 Total Transmit Power (db) Robust SOCP [Shenouda-Davidson 07] Robust MMSE [Vu cić-boche 09] SDR + S-Lemma SINR requirement γ (db) Performance of worst-case robust SDR and other solutions. N t = K = 4, γ = γ 1 = = γ K, σ 2 1 = = σ2 K = 0.1, r 1 = = r K = 0.1. SDR yielded rank-1 solution in all the trials run. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 135

137 Rank-One Optimality for Worst-Case Robust BF? Challenge: Can we prove (or disprove) the rank-one optimality of SDR for the worst-case robust unicast BF problem K min W 1,...,W K H N i=1 Tr(W i) s.t. SINR i ({W j }, h i + e i ) γ i, e i r i, i = 1,...,K, W i 0, i = 1,...,K. Note: The SBP rank reduction result does not work here! Some sufficient conditions are currently available: [Song-Shi-Sanjabi-Sun-Luo 12]: An SDR solution {W i } must be of rank one if r i s are small enough in a problem instance-dependent manner. [Chang-Ma-Chi 11]: An SDR solution {W i } must be of rank one if another related problem (what?) has a unique solution. Proving or disproving SDR rank-one optimality in unicast robust BF remains an intriguing open problem. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 136

138 Recall the SDR problem The Result in [Chang-Ma-Chi 11] min W 1,...,W K 0 s.t. K i=1 Tr(W i) min SINR i ({W j }, h i + e i ) γ i, e i r i i = 1,...,K. ( ) Consider a different problem max e i r i, i=1,...,k min W 1,...,W K 0 K i=1 Tr(W i) s.t. SINR i ({W j }, h i + e i ) γ i, i = 1,...,K. ( ) [Chang-Ma-Chi 11] reveals the duality between ( ) & ( ). Specifically, the SDR of ( ) (w.r.t. e i s) yields the same optimal value as ( ); any solution of ( ) is a solution of the SDR of ( ); if the SDR of ( ) has a unique inner solution {Ŵ i}, then a solution of ( ) must be of rank one. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 137

139 Remarks The combination of SDR and S-lemma for robust BF design may also be applied to other scenarios. A Gaussian randomization procedure can be developed for the robust BF design case. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 138

140 Part IV.B: Advanced Topics in Transmit Beamforming Topic 3. Outage-based Robust Beamforming A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 139

141 Motivation Recall that to study the effect of imperfect CSIT, we have adopted the model h i = h i + e i, where h i is the presumed channel and e i is the channel error. Previously, we assumed that the error e i lies in a ball centered at the origin with known radius; i.e., e i r i, where r i is known. This gives rise to a worst-case robust BF design problem. Alternatively, one can consider the following Gaussian channel error model: e i CN(0, C i ), where C i 0 is a known covariance matrix. In particular, we have h i CN( h i, C i ). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 140

142 Motivation Under this probabilistic error model, a meaningful, but very difficult, problem is the following outage-based BF design problem: min W 1,...,W K K i=1 Tr(W i) s.t. Prob hi CN( h i,c i ) {SINR i ({W j }, h i ) γ i } 1 ρ i, i = 1,...,K, rank(w i ) 1, W i 0, i = 1,...,K. (OPM) Here, ρ i (0, 1) is user i s maximum tolerable outage probability. Recall that SINR i ({W j }, h i ) = Tr(W i h i h H i ) l i Tr(W lh i h H i ) +. σ2 i The above outage-based design problem is an instance of the so-called chanceconstrained or probabilistically-constrained optimization problem. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 141

143 Tackling the Probabilistic Constraints To get a more tractable problem, a natural move is to apply SDR and remove the rank constraints. However, the SDRed (OPM) remains hard. Indeed, although the outage-based SINR constraints Prob hi CN( h i,c i ) {SINR i ({W j },h i ) γ i } 1 ρ i can be rewritten as Prob ei CN(0,C i ) ( h i + e i ) H 1 W i W l ( h i + e i ) σi 2 γ i 1 ρ i, l i the probability on the LHS has no simple closed form expression. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 142

144 Tackling the Probabilistic Constraint: Monte Carlo? In principle, we can handle the SDRed probabilistic constraint by Monte Carlo methods. Specifically, let e 1 i,...,el i be i.i.d. according to CN(0, C i ). Here, L 1 is the number of independent samples of e i. Consider the SDP constraints ( h i + e l i) H 1 W i W l ( h i + e l γ i) σi 2, i l i W 1,...,W K 0. l = 1,...,L, ( ) It can be shown [Calafiore-Campi 05] that for sufficiently large L (which depends on the outage tolerance ρ i ), any solution to ( ) will satisfy the corresponding SDRed probabilistic constraint with high confidence (but not necessarily always this could be problematic in some applications). In addition, this method is extremely time consuming in practice. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 143

145 Tackling the Probabilistic Constraint: Convex Restriction To circumvent the aforementioned difficulties, let us consider an alternative approach. Let V i ({W j }) = Prob ei CN(0,C i ) ( h i + e i ) H 1 W i W l ( h i + e i ) < σi 2 γ i l i be the violation probability. Recall that we want V i ({W j }) ρ i. By applying some simple transformations, we can express V i as V i ({W j }) = Prob e CN(0,I) { e H Qe + 2Re{e H r} + s < 0 } for some Q, r and s that depend on C i, W 1,...,W K, and the index i. (Here and in the sequel, we drop the index i for notational simplicity.) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 144

146 Thus, the crux of the outage-based design problem is how to process the probabilistic constraint Prob e CN(0,I) { e H Qe + 2Re{e H r} + s < 0 } ρ. (PC) Here is an idea: Suppose that we can find an efficiently computable convex function f(q,r,s) such that Prob e CN(0,I) { e H Qe + 2Re{e H r} + s < 0 } f(q, r, s). Then, by construction, the convex constraint f(q,r,s) ρ (CR-PC) serves as a sufficient condition for (PC) to hold. We call (CR-PC) a convex restriction of (PC). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 145

147 Finding a Convex Restriction Can we find such an f? Does it even exist? As it turns out, the answer is Yes! (And there are many such functions.) The constructions are based on large deviation bounds on the tail probability { Prob e CN(0,I) e H Qe + 2Re{e H r} + s < 0 }. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 146

148 Finding a Convex Restriction: Sphere Bounding The first construction is motivated by ideas from robust optimization. Consider a set B such that Prob e CN(0,I) {e B} 1 ρ. Then, we have the following implication: δ H Qδ + 2Re{δ H r} + s 0 for all δ B = Prob e CN(0,I) {e H Qe + 2Re{e H r} + s < 0} ρ. Hence, where Prob e CN(0,I) {e H Qe + 2Re{e H r} + s < 0} f(q,r,s), f(q, r, s) = { ρ if δ H Qδ + 2Re{δ H r} + s 0 δ B, + otherwise. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 147

149 Finding a Convex Restriction: Sphere Bounding Note that f(q,r,s) ρ if and only if δ H Qδ + 2Re{δ H r} + s 0 δ B. (SB) Thus, (SB) is a sufficient condition for (PC) to hold. Is (SB) an efficiently computable constraint? That depends on what B is. Suppose that we choose B = {δ : δ d}, d = Φ 1 (1 ρ) χ 2 2n, 2 where Φ 1 χ 2 m( ) is the inverse cumulative distribution function of the (central) Chisquare random variable with m degrees of freedom [Wang-Chang-Ma-Chi 10]. Then, we have Prob e CN(0,I) {e B} = 1 ρ. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 148

150 Finding a Convex Restriction: Sphere Bounding Moreover, by the S-lemma, the constraint δ H Qδ + 2Re{δ H r} + s 0 δ B is equivalent to [ Q + ti ] r r H s td 2 which is an SDP in the variables (Q,r,s,t). Thus, (SB-CR) is a convex restriction of (PC). 0, t 0, (SB-CR) A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 149

151 Outage-Based BF Design via Sphere Bounding Applying the sphere bounding technique to the outage-based SINR constraint ( Prob ei CN(0,C i ) {( h i + e i ) H 1 γ i W i ) } l i W l ( h i + e i ) σi 2 1 ρ i, we obtain the following convex restriction: [ I h H i ] ( 1 γ i W i ) [ ] [I l i W ] ti I 0 l hi + 0 σi 2 λ id 2 i where d 2 i = Φ 1 (1 ρ χ 2 i )/2. 2n 0, t i 0, ( ) It is worth noting that the constraint ( ) has exactly the same form as that in the worst-case robust BF design problem (RPM-SDR). The only difference lies in how the parameter d i is determined. For worst-case robust design, d i is a pre-specified parameter that determines the radius of the ball in which the error vector e i lies; i.e., e i d i. For outage-based design, d i is determined by the maximum tolerable outage probability ρ i. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 150

152 Finding a Convex Restriction: Bernstein-Type Inequality Another construction is based on the so-called Bernstein-type inequality [Bechar2009], [Wang-Chang-Ma-So-Chi 11], which states that Prob e CN(0,I) {e H Qe + 2Re{e H r} + s < 0} f(q, r, s) = e T 1 (s), where T(η) = Tr(Q) 2η Q 2 F + r 2 η max{λ max ( Q),0}. Is the constraint f(q, r, s) = e T 1 (s) ρ efficiently computable? Yes! It is equivalent to the SDP Tr(Q) 2 ln(ρ) t 1 + ln(ρ) t 2 + s 0, Q 2 F + 2 r 2 t 1, t 2 I + Q 0, t 2 0 (BI-CR) in the variables (Q,r,s, t 1,t 2 ). Thus, (BI-CR) is a convex restriction of (PC). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 151

153 Outage-Based BF Design via Bernstein-Type Inequality Applying the Bernstein-type inequality to the outage-based SINR constraints, we obtain another convex restriction of the outage-based BF design problem [Wang-Chang-Ma-So-Chi 11]. Yet another mysterious finding in simulations: For the Bernstein-type inequality approach, rank-one SDR solution is obtained in almost all the problem instances the same phenomenon as that observed in worst-case robust design (or the sphere bounding approach). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 152

154 Comparing the Convex Restrictions The above development raises the following natural question: What is the approximation quality (w.r.t. the original probabilistic constraint) of the convex restrictions obtained by the sphere bounding and Bernstein-type inequality approaches? Unfortunately, this remains an intriguing open question. This leads to the next natural question: Which of the two convex restrictions has better approximation quality? In [Wang-So-Chang-Ma-Chi 14], the following result is shown: Under some mild assumptions, the convex restriction based on the Bernstein-type inequality approach yields a better approximation of the original probabilistic constraint than that based on the sphere bounding approach. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 153

155 Counts over different channel realizations SDR + Sphere Bounding Actual SINR satisfaction probability (Mean = ) Histogram of the actual SINR satisfaction probabilities of the SDR+sphere bounding method. N t = K = 3; i.i.d. complex Gaussian CSI errors with zero mean and variance 0.002; γ = 11dB; ρ = 0.1 (90% SINR satisfaction). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 154

156 Counts over different channel realizations SDR + Bernstein-type Inequality Actual SINR satisfaction probability (Mean = ) Histogram of the actual SINR satisfaction probabilities of the SDR+Bernstein method. N t = K = 3; i.i.d. complex Gaussian CSI errors with zero mean and variance 0.002; γ = 11dB; ρ = 0.1 (90% SINR satisfaction). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 155

157 Feasbility Rate (%) SDR + Sphere Bounding SDR + Bernstein Probabilistic SOCP SINR requirement γ (db) Feasibility performance of the SDR methods and the probabilistic SOCP method [Shenouda- Davidson 08]. N t = K = 3; σ 2 e = 0.002; γ = 11dB; ρ = 0.1 (90% SINR satisfaction). A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 156

158 Total Transmit Power (db) Non-robust SDR + Sphere Bounding SDR + Bernstein Probabilistic SOCP SINR requirement γ (db) Transmit power performance of the SDR methods and the probabilistic SOCP method. N t = K = 3; σ 2 e = 0.002; ρ = 0.1 (90% SINR satisfaction). Non-robust refers to the perfect CSIT-based design, which is not robust against CSIT errors. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 157

159 Part IV.C: Sensor Network Localization A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 158

160 Overview The sensor network localization (SNL) problem is to determine the (x, y) coordinates of the sensors, given distance measurements between sensors. In ad-hoc sensor networks, sensors locations are important but may not be known. Though one can equip every sensor with GPS, it is too expensive to do so. Thus, we may only have several sensors, called anchors, that have self-localization capability. Anchors Sensors A pair of sensors that are within communication range of each other can measure the distance between themselves (e.g., via TOA, RSS, etc.). The inter-sensor distance measurements, together with anchor locations, can be used to jointly estimate the sensors locations. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 159

161 Problem Formulation Let {x 1,...,x n }, x i R 2, be the collection of all (unknown) sensor coordinates. Let {a 1,...,a m }, a i R 2, be the collection of all (known) anchor coordinates. The distance between sensor i and sensor j (resp. sensor i and anchor j) is x i x j (resp. x i a j ). Let E ss and E sa denote the set of sensor-sensor and sensor-anchor edges, resp. Problem: Assuming the distance measurements {d ij } (i,j) Ess and { d ij } (i,j) Esa are noiseless (extensions for noisy cases will be discussed later), we need to find x 1,...,x n R 2 such that x i x j 2 = d 2 ij, (i, j) E ss, x i a j 2 = d 2 ij, (i, j) E sa. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 160

162 Deriving an SDR of the SNL Problem: A First Attempt Let X = [ x 1,...,x n ] R 2 n. The SNL problem can be formulated as find X R 2 n s.t. x T i x i 2x T i x j + x T j x j = d 2 ij, x T i x i 2x T i a j + a T j a j = d 2 ij, (i, j) E ss, (i, j) E sa. This follows since x i x j 2 = (x i x j ) T (x i x j ) = x T i x i 2x T i x j + x T j x j, and similarly for x i a j 2. By letting Y = X T X R n n, we can also formulate the SNL problem as find X R 2 n, Y R n n s.t. Y ii 2Y ij + Y jj = d 2 ij, (i, j) E ss, Y ii 2x T i a j + a T j a j = d 2 ij, (i, j) E sa, Y = X T X. (SNL) It is known [Saxe 79] that finding a solution to (SNL) is NP-hard. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 161

163 Observe that with X R 2 n, the constraint Y = X T X is equivalent to Y 0, rank(y ) 2. If we proceed as before and just drop the trouble-causing rank constraint, then we get the following SDR: find X R 2 n, Y R n n s.t. Y ii 2Y ij + Y jj = d 2 ij, (i, j) E ss, Y ii 2x T i a j + a T j a j = d 2 ij, (i, j) E sa, Y 0. In this formulation, there is no connection between X and Y. In other words, the information in the original constraint Y = X T X is totally lost. The solution obtained could be quite awful. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 162

164 Deriving an SDR of the SNL Problem: Another Attempt To keep the connection between X and Y, instead of relaxing Y = X T X to Y 0, we relax it to Y X T X. This is an SDP constraint, since by the Schur complement, [ ] Y X T I X X Z = X T 0. Y Then, we have the following SDR of the SNL problem: find X R 2 n,y R n n s.t. Y ii 2Y ij + Y jj = d 2 ij, (i, j) E ss, Y ii 2x T i a j + a T j a j = d 2 ij, (i, j) E sa, Y X T X. (SNL-SDR) Note that rank(z) 2 Y = X T X rank(y ) 2. Remark: Although we focus on 2-D localization, our techniques can be easily extended to handle r-d localization for any r 2. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 163

165 Theoretical Properties of the SDR Suppose that we have a solution (X,Y ) to (SNL-SDR). Under what conditions will it be a solution to the original problem (SNL)? The following complete characterization is obtained in [So-Ye 07]: Suppose that the given SNL instance is connected. Then, the following statements are equivalent: The solution (X, Y ) to (SNL-SDR) is feasible for (SNL) (in particular, we have Y = X T X ). The max-rank solution to (SNL-SDR) has rank at most 2. The given SNL instance is uniquely localizable; i.e., it has a unique solution in all dimensions. Since most polynomial-time interior-point algorithms for solving SDPs will return a solution that has the highest rank, we can localize uniquely localizable instances in polynomial time. The above result fits the theme of compressed sensing and low-rank optimization, which are two currently very active research areas. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 164

166 Rank of SDR Solution and Dimension Reduction The following was also established in [So-Ye 07]: Every rank d 2 solution to (SNL-SDR) corresponds to a set of feasible (w.r.t. the distance constraints) d-dimensional coordinates for the sensors. Question: While it is NP-hard to find a rank-2 solution to (SNL-SDR), is it possible to find a low rank solution (and hence achieve dimension reduction)? A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 165

167 Dimension Reduction via Unfolding One heuristic is to stretch apart pairs of non-adjacent nodes. This will tend to flatten the configuration of nodes. Mathematically, this corresponds to adding an objective function to (SNL): max x i x j 2 x 1,...,x n R 2 (i,j) N ss s.t. x i x j 2 = d 2 ij, x i a j 2 = d 2 ij, (i, j) E ss, (i, j) E sa, (SNL-OBJ) where N ss {(i, j) : (i,j) E ss } is a subset of the non-adjacent pairs. Again, we can apply SDR to (SNL-OBJ). Interestingly, the solution to the resulting SDR often has low rank. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 166

168 In [So-Ye 06], some theoretical justification is given to explain this phenomenon. It is related to the so-called tensegrity theory in discrete geometry. A. M.-C. So, Convex Optimization and Its Applications, ChinaSIP 2014 Tutorial 167