Handout 6: Some Applications of Conic Linear Programming

Transcription

1 G00E04: Convex Optimization and Its Applications in Signal Processing Handout 6: Some Applications of Conic Linear Programming Instructor: Anthony Man Cho So June 3, 05 Introduction Conic linear programming CLP, and in particular, semidefinite programming SDP, has received a lot of attention in recent years. Such a popularity can partly be attributed to the wide applicability of CLP, as well as recent advances in the design of provably efficient interior point algorithms. In this lecture we will consider some applications that arise in engineering and statistics and show how CLP techniques can be used to model them. Logarithmic Chebyshev Approximation Recall that in Handout we considered a data fitting problem in which we are given m data pairs a i, b i, where a i R n and b i R for i =,..., m, and our goal is to find an x R n that best fits the data in the l sense: minimize max b i a i x. i m As we showed in Handout, the above problem can be formulated as an LP. Now, suppose that we are interested in the following optimization problem: minimize max logbi log a i x. i m Such a problem arises when, for instance, the data b,..., b m have the dimension of a power or intensity. Here, we assume that b,..., b m > 0, and that log a i x = if a i x 0. We now show that can be formulated as an SDP problem. o see this, we first observe that whenever a i x > 0, we have logbi log a i x a = log max i x b i, b i a i x, where i =,..., m. It follows that is equivalent to minimize subject to t t a i x t for i =,..., m, b i which in turn is equivalent to the following SDP: minimize subject to t t a i x/b i a i x/b i 0 t 0.

2 3 Robust Linear Programming o motivate the problem considered in this section, consider the following LP: minimize subject to c x Âx ˆb, where  Rm n, ˆb R m, and c R n are given. As it often occurs in practice, the data of the above LP, namely  and ˆb, are uncertain. In an attempt to tackle such uncertainties, Ben al and Nemirovski [3] introduced the idea of robust optimization. In the robust optimization setting, we assume that the uncertain data lie in some given uncertainty set U. Our goal is then to find a solution that is feasible for all possible realizations of the uncertain data and optimizes some given objective. o derive the robust counterpart of, let us first rewrite it as minimize c z subject to Az 0, z n =, 3 where A = [ ˆb] R m n, c = c, 0 R n, and z R n. Now, suppose that each row a i R n of the matrix A lies in an ellipsoidal region U i whose center u i R n is given here, i =,..., m. Specifically, we impose the constraint that a i U i = x R n : x = u i B i v, v } for i =,..., m, where a i is the i th row of A, u i R n is the center of the ellipsoid U i, and B i is some n n positive semidefinite matrix. hen, the robust counterpart of 3 is the following optimization problem: minimize c z subject to Az 0 for all a i U i, i =,..., m, z n =. Note that in general there are uncountably many constraints in 4. However, we claim that 4 is equivalent to an SOCP problem. o see this, observe that a i z 0 for all a i U i iff 0 max v R n : v ui B i v z } = u i z B i z, where i =,..., m. Hence, we conclude that 4 is equivalent to minimize c z subject to B i z u i z for i =,..., m, z n =, which is an SOCP problem. For further readings on robust optimization, we refer the readers to [7,, 4, ]. 4

3 4 Chance Constrained Linear Programming In the previous section, we showed how one can handle data uncertainties in a linear optimization problem using the robust optimization approach. Such an approach is particularly suitable if there is little information about the nature of the data uncertainty, and/or if the constraints are not to be violated under any circumstance. However, it can produce very conservative solutions. In situations where the distribution of the data uncertainty is partially known, one can often obtain a better solution by allowing the constraints to be violated with small probability. o illustrate this possibility, let us consider the following simple LP: minimize c x subject to a x b, x P, 5 where a, c R n are given vectors, b R is a given scalar, and P R n is a given polyhedron. For the sake of simplicity, suppose that P is deterministic, but the data a, b are randomly affinely perturbed; i.e., l l a = a 0 ɛ i a i, b = b 0 ɛ i b i, i= where a 0, a,..., a l R n and b 0, b,..., b l R are given, and ɛ,..., ɛ l are i.i.d. mean zero random variables supported on [, ]. hen, for any given tolerance parameter δ 0,, we can formulate the following chance constrained counterpart of 5: i= minimize c x subject to Pra x > b δ, x P. 6 In other words, a solution x P is feasible for 6 if it only violates the constraint a x b with probability at most δ. he constraint is known as a chance constraint. Note that when δ = 0, 6 reduces to a robust linear optimization problem. Moreover, if x R n is feasible for 6 at some tolerance level δ 0, then it is also feasible for 6 at any δ δ. hus, by varying δ, we will be able to adjust the conservatism of the solution. Despite the attractiveness of the chance constraint formulation, it has one major drawback; namely, it is generally computationally intractable. Indeed, even when the distributions of ɛ,..., ɛ l are very simple, the feasible set defined by the chance constraint can be non convex. One way to tackle this problem is to replace the chance constraint by its safe tractable approximation; i.e., a system of deterministic constraints H such that i x R n is feasible for whenever it is feasible for H safe approximation, and ii the constraints in H are efficiently computable tractability. o develop a safe tractable approximation of, we first observe that is equivalent to the following system of constraints: l Pr y 0 ɛ i y i > 0 δ, 7 i= y i = a i x b i for i = 0,,..., l. 8 3

4 Since ɛ,..., ɛ l are i.i.d. mean zero random variables supported on [, ], by Hoeffding s inequality [4], we have l t Pr ɛ i y i > t exp i= l i= y i for any t > 0. It follows that when y 0 ln l yi δ, 9 we have l y0 Pr y 0 ɛ i y i > 0 exp i= δ. l i= y i In other words, 9 is a sufficient condition for 7 to hold. he upshot of 9 is that it is a second order cone constraint. Hence, we conclude that constraints 8 and 9 together serve as a safe tractable approximation of the chance constraint. So far we have only discussed how to handle a single scalar chance constraint. Of course, the case of joint scalar chance constraints; i.e., chance constraints of the form i= Pr a i x > b i for i =,,..., m δ are also of great interest. However, they are not as easy to handle as a single scale chance constraint. For some recent results in this direction, see [, 6,, 8]. It is instructive to examine the relationship between the robust optimization approach and the safe tractable approximation approach. Upon following the derivations in the previous section, we see that constraint 9 is equivalent to the following robust constraint: d y 0 for all d U, where U is the ellipsoidal uncertainty set given by } U = x R l : x = e Bv, v, and B R l l is given by B = ln [ 0 0 δ 0 I In other words, we are using the following robust optimization problem minimize subject to c x ]. l d i a i x b i 0 for all d U, i=0 x P as a safe tractable approximation of the chance constrained optimization problem 6. For further elaboration on the connection between robust optimization and chance constrained optimization, see [7, 6]. We remark that chance constrained optimization has been employed in many recent applications; see, e.g., [, 0, 6, 5, 4]. For a more comprehensive treatment and some recent advances of chance constrained optimization, we refer the reader to [9, Chapter 4] and []. 4

5 5 Quadratically Constrained Quadratic Optimization A class of optimization problems that has frequently arisen in applications is that of quadratically constrained quadratic optimization problems QCQPs; i.e., problems of the form minimize x Qx subject to x A i x b i for i =,..., m, 0 where Q, A,..., A m S n are given. In general, due to the non convexity of the objective function and constraints, problem 0 is intractable. Nevertheless, it can be tackled by the so called semidefinite relaxation technique. o introduce this technique, we first observe that for any Q S n, x Qx = trx Qx = trqxx. Hence, problem 0 is equivalent to minimize trqxx subject to tra i xx b i for i =,..., m. Now, using the spectral theorem for symmetric matrices see Section 3. of Handout B, one can verify that X = xx X 0, rankx. It follows that problem 0 is equivalent to the following rank cosntrained SDP problem: minimize trqx subject to tra i X b i for i =,..., m, X 0, rankx. he advantage of the formulation in over that in 0 is that it reveals where the difficulty of the problem lies; namely, in the non convex constraint rankx. By dropping this constraint, we obtain the following semidefinite relaxation of problem 0: minimize trqx subject to tra i X b i for i =,..., m, X 0. Note that problem is an SDP, and hence can be efficiently solved. Of course, an optimal solution X to problem may not even be feasible for problem 0, since we need not have rankx. Moreover, it is generally impossible to convert X into an optimal solution x to problem 0. However, as it turns out, it is often possible to extract a near optimal solution to problem 0 from an optimal solution to problem. In the next section, we will illustrate how this can be done for a well known combinatorial problem the Maximum Cut Problem Max Cut. For a survey of the theory and the many applications of the semidefinite relaxation technique, see [8]. 5

6 5. An Approximation Algorithm for Maximum Cut in Graphs Suppose that we are given a simple undirected graph G = V, E and a function w : E R that assigns to each edge e E a non negative weight w e. he Maximum Cut Problem Max Cut is that of finding a set S V of vertices such that the total weight of the edges in the cut S, V \S; i.e., sum of the weights of the edges with one endpoint in S and the other in V \S, is maximized. By setting w ij = 0 if i, j E, we may denote the weight of a cut S, V \S by ws, V \S = w ij, 3 i S,j V \S and our goal is to choose a set S V such that the quantity in 3 is maximized. he Max Cut problem is one of the fundamental computational problems on graphs and has been extensively studied by many researchers. It has been shown that the Max Cut problem is unlikely to have a polynomial time algorithm see, e.g, []. On the other hand, in a seminal work, Goemans and Williamson [3] showed how SDP can be used to design a approximation algorithm for the Max Cut problem; i.e., given an instance G, w of the Max Cut problem, the algorithm will find a cut S, V \S whose value ws, V \S is at least times the optimal value. In this section, we will describe the algorithm of Goemans and Williamson and prove its approximation guarantee. o begin, let G, w be a given instance of the Max Cut problem, with n = V. hen, we can formulate the Max Cut problem as an integer quadratic program, viz. v = maximize w ij x i x j i,j E 4 subject to x i = for i =,..., n. Here, the variable x i indicates which side of the cut vertex i belongs to. Specifically, the cut S, V \S is given by S = i,..., n} : x i = }. Note that if vertices i and j belong to the same side of a cut, then x i = x j, and hence its contribution to the objective function in 4 is zero. Otherwise, we have x i x j, and its contribution to the objective function is w ij / = w ij. In general, problem 4 is hard to solve. hus, we consider relaxations of 4. One approach is to observe that both the objective function and the constraints in 4 are linear in x i x j, where i, j n. In particular, if we let X = xx R n n, then problem 4 can be written as v = maximize i,j E subject to diagx = e, X = xx. w ij X ij Since X = xx iff X 0 and rankx, from our earlier discussion, we can drop the non convex rank constraint and arrive at the following relaxation of 5: v sdp = maximize i,j E subject to diagx = e, 6 X 0. w ij X ij 5 6

7 Note that 6 is an SDP. Moreover, since 6 is a relaxation of 5, we have v sdp v. Now, suppose that we have an optimal solution X to 6. In general, the matrix X need not be in the form xx, and hence it does not immediately yield a feasible solution to 5. However, we can extract from X a solution x, } n to 5 via the following randomized rounding procedure:. Compute the Cholesky factorization X = U U of X, where U R n n. Let u i R n be the i th column of U. Note that u i = for i =,..., n.. Let r R n be a vector uniformly distributed on the unit sphere S n = x R n : x = }. 3. Set x i = sgn u i r for i =,..., n, where if z 0, sgnz = otherwise. In other words, we choose a random hyperplane through the origin with r as its normal and partition the vertices according to whether their corresponding vectors lie above or below the hyperplane. Since the solution x, } n is produced via a randomized procedure, we are interested in its expected objective value; i.e., E i,j E w ij x i x j = i,j E w ij E [ x ix j ] = w ij Pr [ sgn u i r sgn u j r ]. i,j E 7 he following theorem provides a lower bound on the expected objective value 7 and allows us to compare it with the optimal value vsdp of 5. heorem Let u, v S n, and let r be a vector uniformly distributed on S n. hen, we have Furthermore, for any z [, ], we have where Pr [ sgn u r sgn v r ] = π arccos u v. 8 π arccosz α z > z, 9 θ α = min 0 θ π π cos θ. Proof o establish 8, observe that by symmetry, we have Pr [ sgn u r sgn v r ] = Pr u r 0, v r < 0. Now, by projecting r onto the plane containing u and v, we see that u r 0 and v r < 0 iff the projection lies in the wedge formed by u and v. Since r is chosen from a spherically symmetric distribution, its projection will be a random direction on the plane containing u and v. Hence, we have Pr u r 0, v r < 0 = arccos u v, π 7

8 as desired. o establish the first inequality in 9, consider the change of variable z = cos θ. Since z [, ], we have θ [0, π]. hus, it follows that π arccosz = θ π = θ π cos θ cos θ α z, as desired. he second inequality in 9 can be established using calculus and we leave the proof to the readers. Corollary Given an instance G, w of the Max Cut problem and an optimal solution to 6, the randomized rounding procedure above will produce a cut S, V \S whose expected objective value satisfies ws, V \S 0.878v. Proof Let x be the solution obtained from the randomized rounding procedure, and let S be the corresponding cut. By 7 and heorem, we have E [ ws, V \S ] = π i,j E = 0.878v sdp 0.878v, w ij arccos u i u j i,j E w ij u i u j as desired. 6 Sparse Principal Component Analysis Principal Component Analysis PCA see, e.g, [3] is a very important tool in data analysis. It provides a way to reduce the dimension of a given data set, thus revealing the sometimes hidden underlying structure of and facilitating further analysis on the data set. o motivate the problem of finding principal components, consider the following scenario. Suppose that we are interested in some attributes X,..., X n of a population. In order to estimate the values of these attributes, one may sample from the population. Specifically, let X ij be the value of the j th attribute of the i th individual, where i =,..., m and j =,..., n. For j, k =,..., n, define X j = m m X ij, i= σ jk = m m X ij X j X ik X k i= to be the sample mean of X j and the sample covariance between X j and X k, respectively. Define Σ = [σ jk ] j,k to be the sample covariance matrix. he goal is then to find the principal components u,..., u n such that the linear combination n j= u jx j has maximum sample variance. In other words, we would like to solve the following problem: max u u Σu, 0 8

9 where u = u,..., u n. he model given by 0 is based on the belief that principal components with a large associated variance indicate interesting features in the data. However, one of the shortcomings of 0 is that the linear combination may use a large number of the original variables; i.e., most of the factors u,..., u n are non zero, thus posing a difficulty in interpreting which variables are the most influential. his motivates us to consider the problem of finding sparse principal components; i.e., a set u,..., u n } of principal components that maximizes the variance while at the same time having only a few non zero u i s. here are many ways to formulate such a problem. For instance, one can take the following penalty function approach see [9]: vρ = max u u } Σu ρ u 0, where u 0 = i,..., n} : u i 0} is the number of non zero elements in the vector u R n, and ρ R is a parameter controlling the level of sparsity. Note that the objective function in is non convex and the associated optimization problem is hard to solve in general. In this section, we will show how to formulate a tractable relaxation of using SDP. o begin, let us observe that Σ 0, and that we may assume without loss that Σ Σ Σ nn 0. Let Σ / 0 be such that Σ = Σ / Σ /. he following proposition shows that we only need to consider the case where ρ Σ. Proposition If ρ Σ, then u = 0 is optimal for. Proof Note that for any u R n, we have u Σu n Σ ij u i u j i,j= Σ n u i u j i,j= = Σ u, where follows from the fact that for any n n positive semidefinite matrix A, we have A ij max i n A ii for i, j n. Furthermore, by the Cauchy Schwarz inequality, for any u R n, we have u u u 0. It follows that vρ = max u } Σu ρ u 0 Σ ρ u 0 0. u In particular, we see that u = 0 is an optimal solution to. his completes the proof. By Proposition, we may assume that ρ Σ, in which case is equivalent to the problem vρ = max u = u } Σu ρ u 0. 3 Now, for a given vector u R n, let wu 0, } n be the indicator vector of the sparsity pattern of u; i.e., wu i = iff u i 0 for i =,..., n. Observe that u Σu = ū diagwu Σ diagwu ū 9

10 for any ū R n such that ū i = u i whenever u i 0, where i =,..., n. On the other hand, by the Courant Fischer characterization see, e.g., [5, heorem 4..], for a given sparsity pattern vector w 0, } n, we have max u = u diagw Σ diagw u = λ max diagw Σ diagw, where λ max M is the largest eigenvalue of the matrix M. hus, upon letting σ i R n to be the i th column of Σ / where i =,..., n, we may write 3 as follows: } vρ = max λ max diagw Σ diagw ρe w w 0,} n = max λ max diagw Σ / Σ / diagw w 0,} n = max λ max Σ / diagw Σ / } ρe w w 0,} n = max max u = w 0,} n = max max u = w 0,} n i= = max u = } ρe w u Σ / diagw Σ / } u ρe w n [ σ w i i u ] ρ n [ σ i u ] ρ i= 4 5, 6 where 4 follows from the fact that λ max M M = λ max MM for any matrix M and diagw = diagw for any w 0, } n, and 5 follows from the fact that u Σ / diagw Σ / u = diagw Σ / u = n σ i u n = w i σ i u. wi i= i= Note that the objective function in 6 is still non convex. However, the upshot of 6 is that the objective function is linear in u i u j, where i, j n. In particular, let U = uu R n n. hen, it is easy to verify that σ i u = σ i Uσ i for i =,..., n. Moreover, for a symmetric matrix U, we have U = uu and u = iff tru =, ranku = and U 0. hus, we see that 6 is equivalent to the following problem: n vρ = maximize σ i Uσ i ρ i= subject to tru =, U 0, ranku =. 7 Note that the function U n i= σ i Uσ i ρ is convex. Unfortunately, problem 7 is still hard to solve in general, since we are maximizing a convex function, and the constraint ranku = 0

11 is not convex. However, not all is lost, as we may proceed as follows. First, let us introduce a notation. Given an n n symmetric matrix M, let λ,..., λ n be its eigenvalues. Define trm = n maxλ i, 0}. i= Now, observe that if U = uu and u =, then U / = U = uu. Moreover, for any α R, the matrix αuu has one eigenvalue equal to α and n eigenvalues equal to 0, which implies that tr αuu = α. It follows that σ i Uσ i ρ σ = tr i uu σ i ρ uu = tr u u σ i σi u ρ u = tr U / σ i σi U / ρu. he point of the above manipulation is that the function U tr U / σ i σi U / ρu is concave, as the following theorem shows: heorem Let M be an n n symmetric matrix. hen, the function X tr X / MX / is concave on S n. Proof he key step of the proof is to show that tr X / MX / = max trp M = min X P 0 try X. 8 Y M, Y 0 his would imply the desired result, since the last expression is a pointwise minimum of affine functions of X and hence is concave. o prove the first equality in 8, observe that since X / MX / is symmetric, by the spectral theorem for symmetric matrices see, e.g., [5, heorem 4..5], we may write X / MX / = UΓU for some orthogonal matrix U and diagonal matrix Γ. hen, we have tr X / MX / n = maxγ ii, 0} i= = max tr diagw Γ diagw w 0,} n [ = max tr X / U diagw U X / M w 0,} n Since X X / U diagw U X / 0 for all w 0, } n, we conclude that tr X / MX / max trp M. X P 0 Conversely, let P be such that X P 0. For any ɛ > 0, let X ɛ = X ɛi. Note that X ɛ 0 for all ɛ > 0. Let Xɛ / 0 be such that X ɛ = Xɛ / Xɛ /. In particular, Xɛ / is invertible. hus, there exists a symmetric matrix D ɛ such that P = X / ɛ UD ɛ U X / ɛ. ].

12 Since the region Y S n : X Y 0} is compact, upon taking ɛ 0 and considering a subsequence if necessary, we conclude that P = X / UDU X / for some symmetric matrix D. Since P 0, we see that D ɛ 0 for all ɛ > 0. Furthermore, since X ɛ X P, we have X ɛ P = Xɛ / UI D ɛ U Xɛ / 0 for all ɛ > 0. Hence, we conclude that I D ɛ 0 for all ɛ > 0, which in turn implies that I D 0. In particular, all the eigenvalues of D lie in [0, ]. Note, however, that D need not be diagonal. Now, in order to complete the proof of the first equality in 8, it suffices to prove the following: trp M = tr DU X / MX / U = trdγ tr X / MX /. owards that end, we need the following result see, e.g., [5, heorem..]: heorem 3 Fan s Inequality Let M, M S n. Consider the function λ : S n R n defined by λm = λ M,..., λ n M, where λ M λ M λ n M are the eigenvalues of M. hen, we have trm M λm λm. Armed with heorem 3, we proceed as follows. Let Γ be the diagonal matrix whose i th diagonal entry is maxγ ii, 0}, where i =,..., n. hen, we have trp M = trdγ λd λγ trγ = tr X / MX /, as desired. Now, since the second equality in 8 follows from the CLP Strong Duality heorem, the proof of heorem is completed. Upon applying heorem and dropping the problematic rank constraint ranku = in 7, we obtain the following convex relaxation: n v ρ = maximize tr U / σ i σi U / ρu i= 9 subject to tru =, U 0. Upon letting M i ρ = σ i σi ρi for i =,..., n and using 8, we see that 9 is equivalent to n v ρ = maximize trp i M i ρ i= subject to tru =, U P i 0 for i =,..., n, U 0, which is an SDP. We remark that since 9 is a relaxation of 7, we have v ρ vρ. Recently, there has been some interest in analyzing the quality of various convex relaxations of the sparse principal component analysis problem. For an analysis of the above SDP relaxation, we refer the reader to [0].

13 References [] F. Alizadeh and D. Goldfarb. Second Order Cone Programming. Mathematical Programming, Series B, 95:3 5, 003. [] A. Ben-al, L. El Ghaoui, and A. Nemirovski. Robust Optimization. Princeton Series in Applied Mathematics. Princeton University Press, Princeton, NJ, 009. [3] A. Ben-al and A. Nemirovski. Robust Convex Optimization. Mathematics of Operations Research, 34: , 998. [4] A. Ben-al and A. Nemirovski. Selected opics in Robust Convex Optimization. Mathematical Programming, Series B, :5 58, 008. [5] J. M. Borwein and A. S. Lewis. Convex Analysis and Nonlinear Optimization: heory and Examples, volume 3 of CMS Books in Mathematics. Springer ScienceBusiness Media, Inc., New York, second edition, 006. [6] W. Chen, M. Sim, J. Sun, and C.-P. eo. From CVaR to Uncertainty Set: Implications in Joint Chance Constrained Optimization. Operations Research, 58: , 00. [7] X. Chen, M. Sim, and P. Sun. A Robust Optimization Perspective on Stochastic Programming. Operations Research, 556:058 07, 007. [8] S.-S. Cheung, A. M.-C. So, and K. Wang. Linear Matrix Inequalities with Stochastically Dependent Perturbations and Applications to Chance Constrained Semidefinite Optimization. SIAM Journal on Optimization, 4: , 0. [9] A. d Aspremont, F. Bach, and L. El Ghaoui. Optimal Solutions for Sparse Principal Component Analysis. Journal of Machine Learning Research, 9:69 94, 008. [0] A. d Aspremont, F. Bach, and L. El Ghaoui. Approximation Bounds for Sparse Principal Component Analysis. Preprint, available at 0. [] L. El Ghaoui, M. Oks, and F. Oustry. Worst Case Value at Risk and Robust Portfolio Optimization: A Conic Programming Approach. Operations Research, 54: , 003. [] M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the heory of NP Completeness. W. H. Freeman and Company, New York, 979. [3] M. X. Goemans and D. P. Williamson. Improved Approximation Algorithms for Maximum Cut and Satisfiability Problems Using Semidefinite Programming. Journal of the ACM, 46:5 45, 995. [4] W. Hoeffding. Probability Inequalities for Sums of Bounded Random Variables. Journal of the American Statistical Association, 5830:3 30, 963. [5] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, Cambridge,

14 [6] W. W.-L. Li, Y. J. Zhang, A. M.-C. So, and M. Z. Win. Slow Adaptive OFDMA Systems through Chance Constrained Programming. IEEE ransactions on Signal Processing, 587: , 00. [7] M. S. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret. Applications of Second Order Cone Programming. Linear Algebra and Its Applications, 84:93 8, 998. [8] Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, and S. Zhang. Semidefinite Relaxation of Quadratic Optimization Problems. IEEE Signal Processing Magazine, 73:0 34, 00. [9] A. Shapiro, D. Dentcheva, and A. Ruszczyński. Lectures on Stochastic Programming: Modeling and heory, volume 9 of MPS SIAM Series on Optimization. Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, 009. [0] M. B. Shenouda and. N. Davidson. Outage Based Designs for Multi User ransceivers. In Proceedings of the 009 IEEE International Conference on Acoustics, Speech, and Signal Processing ICASSP 009, pages , 009. [] A. M.-C. So. Moment Inequalities for Sums of Random Matrices and heir Applications in Optimization. Mathematical Programming, Series A, 30:5 5, 0. [] A. M.-C. So. Lecture Notes on Chance Constrained Optimization. Available at se.cuhk.edu.hk/ manchoso/papers/ccopt.pdf, 03. [3] M. Stuart. A Geometric Approach to Principal Component Analysis. he American Statistician, 364: , 98. [4] K.-Y. Wang, A. M.-C. So,.-H. Chang, W.-K. Ma, and C.-Y. Chi. Outage Constrained Robust ransmit Optimization for Multiuser MISO Downlinks: ractable Approximations by Conic Optimization. IEEE ransactions on Signal Processing, 6: , 04. [5] Y. J. Zhang and A. M.-C. So. Optimal Spectrum Sharing in MIMO Cognitive Radio Networks via Semidefinite Programming. IEEE Journal on Selected Areas in Communications, 9:36 373, 0. 4