Low-Rank Matrix Recovery

Transcription

1 ELE 538B: Mathematics of High-Dimensional Data Low-Rank Matrix Recovery Yuxin Chen Princeton University, Fall 2018

2 Outline Motivation Problem setup Nuclear norm minimization RIP and low-rank matrix recovery Phase retrieval / solving random quadratic systems of equations Matrix completion Matrix recovery 11-2

3 Motivation Matrix recovery 11-3

4 Motivation 1: recommendation systems??????????????? Netflix challenge: Netflix provides highly incomplete ratings from 0.5 million users for & 17,770 movies How to predict unseen user ratings for movies? Matrix recovery 11-4

5 In general, we cannot infer missing ratings????????????????????????????? Underdetermined system (more unknowns than observations) Matrix recovery 11-5

6 ... unless rating matrix has other structure??????????????? A few factors explain most of the data Matrix recovery 11-6

7 ... unless rating matrix has other structure??????????????? A few factors explain most of the data low-rank approximation How to exploit (approx.) low-rank structure in prediction? Matrix recovery 11-6

8 Motivation 2: sensor localization n sensors / points x j R 3, j = 1,, n Observe partial information about pairwise distances D i,j = x i x j 2 2 = x i x j 2 2 2x i x j Goal: infer distance between every pair of nodes Matrix recovery 11-7

9 Motivation 2: sensor localization Introduce x 1 x X = 2. Rn 3 x n then distance matrix D = [D i,j ] 1 i,j n can be written as x [ ] D = x 1 2 2,, x n 2 2 2XX }{{ } x n 2 }{{} rank 3 2 }{{} rank 1 } rank 1 {{ } low rank rank(d) n low-rank matrix completion Matrix recovery 11-8

10 Motivation 3: structure from motion Given multiple images and a few correspondences between image features, how to estimate the locations of 3D points? Snavely, Seitz, & Szeliski Structure from motion: reconstruct 3D scene geometry and }{{} structure camera poses from multiple images }{{} motion Matrix recovery 11-9

11 Motivation 3: structure from motion Tomasi and Kanade s factorization: Consider n 3D points {p j R 3 } 1 j n in m different 2D frames x i,j R 2 1 : locations of the j th point in the i th frame x i,j = M i }{{} projection matrix R 2 3 p j }{{} 3D position R 3 Matrix recovery 11-10

12 Motivation 3: structure from motion Tomasi and Kanade s factorization: Matrix of all 2D locations x 1,1 x 1,n X =..... x m,1 x m,n = M 1. [ p 1 ] p n R 2m n M m }{{} low-rank factorization Goal: fill in missing entries of X given a small number of entries Matrix recovery 11-10

13 Motivation 4: missing phase problem Detectors record intensities of diffracted rays electric field x(t 1, t 2 ) Fourier transform ˆx(f 1, f 2 ) Fig credit: Stanford SLAC intensity of electrical field: ˆx(f1, f 2 ) 2 = x(t 1, t 2 )e i2π(f1t1+f2t2) dt 1 dt 2 2 Phase retrieval: recover signal x(t 1, t 2 ) from intensity ˆx(f 1, f 2 ) 2 Matrix recovery 11-11

14 A discrete-time model: solving quadratic systems A x Ax y = Ax Solve for x R n in m quadratic equations y k = a k x 2, k = 1,..., m or y = Ax 2 where z 2 := { z 1 2,, z m 2 } Matrix recovery 11-12

15 An equivalent view: low-rank factorization Lifting: introduce X = xx to linearize constraints y k = a kx 2 = a k(xx )a k = y k = a kxa k = a k a k, X (11.1) find X 0 s.t. y k = a k a k, X, k = 1,, m rank(x) = 1 Matrix recovery 11-13

16 Problem setup Matrix recovery 11-14

17 Setup Consider M R n n rank(m) = r n Singular value decomposition (SVD) of M: where Σ = M = UΣV }{{ } = (2n r)r degrees of freedom σ 1... σ r r σ i u i vi i=1 contains all singular values {σ i }; U := [u 1,, u r ], V := [v 1,, v r ] consist of singular vectors Matrix recovery 11-15

18 Low-rank matrix completion Observed entries M i,j, (i, j) }{{} Ω sampling set Completion via rank minimization minimize X rank(x) s.t. X i,j = M i,j, (i, j) Ω Matrix recovery 11-16

19 Low-rank matrix completion Observed entries M i,j, (i, j) }{{} Ω sampling set An operator P Ω : orthogonal projection onto the subspace of matrices supported on Ω Completion via rank minimization minimize X rank(x) s.t. P Ω (X) = P Ω (M) Matrix recovery 11-16

20 More general: low-rank matrix recovery Linear measurements y i = A i, M = Tr(A i M), i = 1,... m An operator form y = A(M) := A 1, M. A m, M R m Recovery via rank minimization minimize X rank(x) s.t. y = A(X) Matrix recovery 11-17

21 Nuclear norm minimization Matrix recovery 11-18

22 Convex relaxation minimize X R n n s.t. rank(x) }{{} nonconvex P Ω (X) = P Ω (M) minimize X R n n s.t. rank(x) }{{} nonconvex A(X) = A(M) Question: what is the convex surrogate for rank( )? Matrix recovery 11-19

23 Nuclear norm Definition 11.1 The nuclear norm of X is X := n σ i (X) }{{} i=1 i th largest singular value Nuclear norm is a counterpart of l 1 norm for rank function Relations among different norms X X F X r X F r X (Tightness) {X : X 1} is the convex hull of rank-1 matrices obeying uv 1 (Fazel 02) Matrix recovery 11-20

24 Additivity of nuclear norm Fact 11.2 Let A and B be matrices of the same dimensions. If AB = 0 and A B = 0, then A + B = A + B. If row (resp. column) spaces of A and B are orthogonal, then A + B = A + B Similar to l 1 norm: when x and y have disjoint support, x + y 1 = x 1 + y 1 (a key to study l 1 -min under RIP) Matrix recovery 11-21

25 Proof of Fact 11.2 Suppose A = U A Σ A V A and B = U BΣ B VB, which gives AB = 0 A B = 0 V A V B = 0 U A U B = 0 Thus, one can write A = [U A, U B, U C ] B = [U A, U B, U C ] Σ A Σ B 0 [V A, V B, V C ] [V A, V B, V C ] and hence A + B = [U A, U B ] [ ΣA Σ B ] [V A, V B ] = A + B Matrix recovery 11-22

26 Dual norm Definition 11.3 (Dual norm) For a given norm A, the dual norm is defined as X A := max{ X, Y : Y A 1} l 1 norm nuclear norm l 2 norm Frobenius norm dual dual dual dual l norm spectral norm l 2 norm Frobenius norm Matrix recovery 11-23

27 Representing nuclear norm via SDP Since the spectral norm is the dual norm of the nuclear norm, X = max{ X, Y : Y 1} The constraint is equivalent to Y 1 Y Y I Schur complement [ I Y Y I ] 0 Fact 11.4 X = max Y { X, Y [ ] } I Y Y 0 I Matrix recovery 11-24

28 Representing nuclear norm via SDP Since the spectral norm is the dual norm of the nuclear norm, X = max{ X, Y : Y 1} The constraint is equivalent to Y 1 Y Y I Schur complement [ I Y Y I ] 0 Fact 11.5 (Dual characterization) X = min W 1,W 2 { 1 2 Tr(W 1) Tr(W 2) [ ] } W1 X X 0 W 2 Optimal point: W 1 = UΣU, W 2 = V ΣV (where X = UΣV ) Matrix recovery 11-24

29 Aside: dual of semidefinite program (primal) minimize X C, X s.t. A i, X = b i, 1 i m X 0 (dual) maximize y b y s.t. m y i A i + S = C Exercise: use this to verify Fact 11.5 i=1 S 0 Matrix recovery 11-25

30 Nuclear norm minimization via SDP Convex relaxation of rank minimization ˆM = argmin X X s.t. y = A(X) This is solvable via SDP 1 minimize X,W1,W 2 2 Tr(W 1) Tr(W 2) [ ] W1 X s.t. y = A(X), X 0 W 2 Matrix recovery 11-26

31 RIP and low-rank matrix recovery Matrix recovery 11-27

32 RIP for low-rank matrices Almost parallel results to compressed sensing... 1 Definition 11.6 The r-restricted isometry constants δr ub (A) and δr lb (A) are the smallest quantities s.t. (1 δ lb r ) X F A(X) F (1 + δ ub r ) X F, X : rank(x) r 1 One can also define RIP w.r.t. 2 F rather than F. Matrix recovery 11-28

33 RIP and low-rank matrix recovery Theorem 11.7 (Recht, Fazel, Parrilo 10, Candes, Plan 11) Suppose rank(m) = r. For any fixed integer K > 0, if 1+δ ub Kr 1 δ lb (2+K)r < K 2, then nuclear norm minimization is exact It allows δkr ub to be larger than 1 Can be easily extended to account for noisy case and approximately low-rank matrices Matrix recovery 11-29

34 Geometry of nuclear norm ball Level set of nuclear norm ball: [ x y y z ] 1 Fig. credit: Candes 14 Matrix recovery 11-30

35 Some notation Recall M = UΣV Let T be the span of matrices of the form (called tangent space) T = {UX + Y V : X, Y R n r } Let P T be the orthogonal projection onto T : P T (X) = UU X + XV V UU XV V Its complement P T = I P T : P T (X) = (I UU )X(I V V ) MP T (X) = 0 and M P T (X) = 0 Matrix recovery 11-31

36 Proof of Theorem 11.7 Suppose X = M + H is feasible and obeys M + H M. The goal is to show that H = 0 under RIP. The key is to decompose H into H 0 + H 1 + H }{{} H c H 0 = P T (H) (rank 2r) H c = P T (H) (obeying MH c = 0 and M H c = 0) H 1 : the best rank-(kr) approximation of H c H 2 : the best rank-(kr) approximation of H c H 1... (K is const) Matrix recovery 11-32

37 Proof of Theorem 11.7 Informally, the proof proceeds by showing that 1. H 0 dominates i 2 H i (by objective function) see Step 1 2. (converse) i 2 H i dominates H 0 + H 1 (by RIP + feasibility) see Step 2 These cannot happen simultaneously unless H = 0 Matrix recovery 11-33

38 Proof of Theorem 11.7 Step 1 (which does not rely on RIP). Show that H j F H 0 / Kr. (11.2) j 2 This follows immediately by combining the following 2 observations: (i) Since M + H is assumed to be a better estimate: M M + H M + H c H 0 (11.3) M + H c H 0 }{{} Fact 11.2 (MHc =0 and M H c=0) = H c H 0 (11.4) (ii) Since nonzero singular values of H j 1 dominate those of H j (j 2): H j F Kr H j Kr [ H j 1 /(Kr) ] H j 1 / Kr = H j F 1 H j 1 1 H c (11.5) j 2 Kr j 2 Kr

39 Proof of Theorem 11.7 Step 2 (using feasibility + RIP). Show that ρ < K/2 s.t. If this claim holds, then H 0 + H 1 F ρ j 2 H j F (11.6) H 0 + H 1 F ρ H (11.2) j F ρ 1 H 0 j 2 Kr ρ 1 ( ) 2 2r H0 F = ρ Kr K H 0 F 2 ρ K H 0 + H 1 F (11.7) where the last line holds since, by construction, H 0 and H 1 lie in orthogonal subspaces. This bound (11.7) cannot hold with ρ < K/2 unless H 0 + H 1 = 0 }{{} equivalently, H 0=H 1=0 Matrix recovery 11-35

40 Proof of Theorem 11.7 We now prove (11.6). To connect H 0 + H 1 with j 2 H j, we use feasibility: A(H) = 0 A(H 0 + H 1 ) = j 2 A(H j), which taken collectively with RIP yields (1 δ(2+k)r lb ) H 0 + H 1 F A(H 0 + H 1 ) F = A(H j) j 2 A(Hj ) j 2 F (1 + δub Kr) H j F j 2 F This establishes (11.6) as long as ρ := 1+δub Kr 1 δ lb (2+K)r < K 2. Matrix recovery 11-36

41 Gaussian sampling operators satisfy RIP If the entries of {A i } m i=1 are i.i.d. N (0, 1/m), then δ 5r (A) < with high prob., provided that m nr (near-optimal sample size) This satisfies the assumption of Theorem 11.7 with K = 3 Matrix recovery 11-37

42 Precise phase transition Using the statistical dimension machienry, we can locate precise phase transition (Amelunxen, Lotz, McCoy & Tropp 13) nuclear norm min { works if m > stat-dim ( D (, X) ) fails if m < stat-dim ( D (, X) ) where stat-dim ( D (, X) ) ( r n 2 ψ n) and ψ (ρ) = inf τ 0 { ρ + (1 ρ) [ ρ(1 + τ 2 ) + (1 ρ) 2 τ ]} (u τ) 2 4 u 2 du π Matrix recovery 11-38

43 Aside: subgradient of nuclear norm Subdifferential (set of subgradients) of at M is M = { } UV + W : P T (W ) = 0, W 1 Does not depend on the singular values of M Z M iff P T (Z) = UV, P T (Z) 1. Matrix recovery 11-39

44 Derivation of the statistical dimension [ Ir WLOG, suppose X =, then X 0 = [ ] G11 G 12 Let G = be i.i.d. standard Gaussian. G 21 G 22 ] {[ Ir W ] } W 1. From the convex geometry lecture, we know that [ stat-dim ( D(, X) ) inf E inf G τz 2 F τ 0 Z X [ [ ] [ ] ] = inf E inf G11 G 12 Ir 2 τ 0 τ G 21 G 22 W W : W 1 ] F Matrix recovery 11-40

45 Derivation of statistical dimension Observe that [ E inf W : W 1 [ [ ] G11 G 12 τ G 21 G 22 [ Ir W ] ] 2 = E G 11 τi r 2 F + G21 2 F + G12 2 F + inf G 22 τw 2 F W 1 = r ( 2n r + τ 2) [ n r ] + E (σ i (G 22) τ) 2 +. i=1 F ] empirical distributions of {σ i(g 22)/ n r} Matrix recovery 11-41

46 Derivation of statistical dimension Observe that [ E inf W : W 1 [ [ ] G11 G 12 τ G 21 G 22 [ Ir W ] ] 2 = E G 11 τi r 2 F + G21 2 F + G12 2 F + inf G 22 τw 2 F W 1 = r ( 2n r + τ 2) [ n r ] + E (σ i (G 22) τ) 2 +. i=1 F ] Recall from random matrix theory (Marchenko-Pastur law) [ n r ] 1 n r E ( ( ) ) 2 2 σi G22 τ (u τ) 2 4 u du, π i=1 where G 22 N ( 1 0, I). Taking ρ = r/n and minimizing over τ lead to n r closed-form expression for phase transition boundary. 0 Matrix recovery 11-41

47 Numerical phase transition (n = 30) Figure credit: Amelunxen, Lotz, McCoy, & Tropp 13 Matrix recovery 11-42

48 Sampling operators that do NOT satisfy RIP Unfortunately, many sampling operators fail to satisfy RIP (e.g. none of the 4 motivating examples in this lecture satisfies RIP) Two important examples: Phase retrieval / solving random quadratic systems of equations Matrix completion Matrix recovery 11-43

49 Phase retrieval / solving random quadratic systems of equations Matrix recovery 11-44

50 Rank-one measurements Measurements: see (11.1) y i = a i xx }{{ } a i = a i a i, M, 1 i m }{{} :=M :=A i A (X) = A 1, X A 2, X. A m, X = a 1 a 1, X a 2 a 2, X. a m a m, X Matrix recovery 11-45

51 Rank-one measurements Suppose a i ind. N (0, I n ) If x is independent of {a i }, then ai a i, xx = a i x 2 x 2 2 A ( xx ) F m xx F Consider A i = a i a i : with high prob., ai a i, A i = ai 4 2 n a i a i F = A(A i ) F a i a i, A i n Ai F Matrix recovery 11-46

52 Rank-one measurements Suppose a i ind. N (0, I n ) If the sample size m n (information limit) and K 1, then max X: rank(x)=1 A(X) F X F min X: rank(x)=1 A(X) F n n m X F = 1 + δub K 1 δ2+k lb max X: rank(x)=1 A(X) F X F min X: rank(x)=1 A(X) F X F n K Violate RIP condition in Theorem 11.7 unless K is exceeding large Matrix recovery 11-46

53 Why do we lose RIP? Problems: Some low-rank matrices X (e.g. a i a i ) might be too aligned with some (rank-1) measurement matrices loss of incoherence in some measurements Some measurements A i, X might have too high of a leverage on A(X) when measured in F Solution: replace F by other norms! Matrix recovery 11-47

54 Mixed-norm RIP Solution: modify RIP appropriately... Definition 11.8 (RIP-l 2 /l 1 ) Let ξr ub (A) and ξr lb (A) be the smallest quantities s.t. (1 ξ lb r ) X F A(X) 1 (1 + ξ ub r ) X F, X : rank(x) r More generally, it only requires A to satisfy sup X:rank(X) r A(X) 1 X F inf X:rank(X) r A(X) 1 X F 1 + ξub r 1 ξr lb (11.8) Matrix recovery 11-48

55 Analyzing phase retrieval via RIP-l 2 /l 1 Theorem 11.9 (Chen, Chi, Goldsmith 15) Theorem 11.7 continues to hold if we replace δr ub and ξr lb (defined in (11.8)), respectively and δ lb r with ξ ub r Follows the same proof as for Theorem 11.7, except that F (highlighted in red) is replaced by 1 in Slide Matrix recovery 11-49

56 Analyzing phase retrieval via RIP-l 2 /l 1 Theorem 11.9 (Chen, Chi, Goldsmith 15) Theorem 11.7 continues to hold if we replace δr ub and ξr lb (defined in (11.8)), respectively and δ lb r with ξ ub r Back to the example in Slide 11-46: If x is independent of {a i }, then ai a i, xx = a i x 2 x 2 2 A ( xx ) 1 m xx F A(A i ) 1 = a i a i, A i + j:j i a i a i, A j (n+m) Ai F For both cases, A(X) 1 X F are of the same order if m n Matrix recovery 11-49

57 Analyzing phase retrieval via RIP-l 2 /l 1 Informally, a debiased operator satisfies RIP condition of Theorem 11.9 when m nr (Chen, Chi, Goldsmith 15) B(X) := A 1 A 2, X A 3 A 4, X. R m/2 Debiasing is crucial when r 1 A consequence of the Hanson-Wright inequality for quadratic form (Hanson & Wright 71, Rudelson & Vershynin 03) Matrix recovery 11-50

58 Theoretical guarantee for phase retrieval (PhaseLift) minimize X R n n tr X }{{} for PSD matrices s.t. y i = a i Xa i, 1 i m X 0 (since X = xx ) Theorem (Candès, Strohmer, Voroninski 13, Candès, Li 14) Suppose a i ind. N (0, I). With high prob., PhaseLift recovers xx exactly as soon as m n Matrix recovery 11-51

59 Extension of phase retrieval (PhaseLift) minimize X R n n tr X }{{} for PSD matrices s.t. a i Xa i = a i Ma i, 1 i m X 0 Theorem (Chen, Chi, Goldsmith 15, Cai, Zhang 15) Suppose M 0, rank(m) = r, and a i ind. N (0, I). With high prob., PhaseLift recovers M exactly as soon as m nr Matrix recovery 11-52

60 Matrix completion Matrix recovery 11-53

61 Sampling operators for matrix completion Observation operator (projection onto matrices supported on Ω) Y = P Ω (M) where (i, j) Ω with prob. p (random sampling) P Ω does NOT satisfy RIP when p 1! For example, } {{ } M P Ω (M) F = 0, or equivalently,???????????? } {{ } Ω 1+δ ub K 1 δ lb 2+K = Matrix recovery 11-54

62 Which sampling pattern? Consider the following sampling pattern????? If some rows / columns are not sampled, recovery is impossible Matrix recovery 11-55

63 Which low-rank matrices can we recover? Compare the following rank-1 matrices: ? 0? 0 0? 0?...? 0? 0 if we miss the top-left entry, then we cannot hope to recover the matrix Matrix recovery 11-56

64 Which low-rank matrices can we recover? Compare the following rank-1 matrices: ? 1? 1 1? 1?...? 1? 1 it is possible to fill in all missing entries by exploiting the rank-1 structure Matrix recovery 11-57

65 Which low-rank matrices can we recover? Compare the following rank-1 matrices: } 0 0 {{ 0 } hard vs } 1 1 {{ 1 } easy Column / row spaces cannot be aligned with canonical basis vectors Matrix recovery 11-58

66 Coherence Definition Coherence parameter µ of M = UΣV is the smallest quantity s.t. X h P U (e i ) h e i max i U e i 2 2 µr n i U and max i V e i 2 2 µr n µ 1 (since n i=1 U e i 2 2 = U 2 F = r) µ = 1 if 1 n 1 = U = V (most incoherent) µ = n r if e i U (most coherent) Matrix recovery 11-59

67 Performance guarantee Theorem (Candes & Recht 09, Candes & Tao 10, Gross 11,...) Nuclear norm minimization is exact and unique with high probability, provided that m µnr log 2 n This result is optimal up to a logarithmic factor Established via a RIPless theory Matrix recovery 11-60

68 Numerical performance of nuclear-norm minimization n = 50 Fig. credit: Candes, Recht 09 Matrix recovery 11-61

69 KKT condition Lagrangian: L (X, Λ) = X + Λ, P Ω (X) P Ω (M) = X + P Ω (Λ), X M When M is the minimizer, the KKT condition reads 0 X L(X, Λ) X=M Λ s.t. P Ω (Λ) M W s.t. UV + W is supported on Ω, P T (W ) = 0, and W 1 Matrix recovery 11-62

70 Optimality condition via dual certificate Slightly stronger condition than KKT guarantees uniqueness: Lemma M is the unique minimizer of nuclear norm minimization if the sampling operator P Ω restricted to T is injective, i.e. P Ω (H) 0, nonzero H T W s.t. UV + W is supported on Ω, P T (W ) = 0, and W < 1 Matrix recovery 11-63

71 Proof of Lemma For any W 0 obeying W 0 1 and P T (W 0 ) = 0, one has M + H M + UV + W 0, H = M + UV + W, H + W 0 W, H = M + P Ω ( UV + W ), H + P T (W 0 W ), H = M + UV + W, P Ω (H) + W 0 W, P T (H) if we take W 0 s.t. W 0, P T (H) = P T (H) }{{} exercise: how to find such an W 0 M + P T (H) W P T (H) = M + (1 W ) P T (H) > M unless P T (H) = 0. But if P T (H) = 0, then H = 0 by injectivity. Thus, M + H > M for any H 0. This concludes the proof. Matrix recovery 11-64

72 Constructing dual certificates Use the golfing scheme to produce an approximate dual certificate (Gross 11) Think of it as an iterative algorithm (with sample splitting) to find a solution satisfying the KKT condition Matrix recovery 11-65

73 (Optional) Proximal algorithm In the presence of noise, one needs to solve minimize X 1 2 y A(X) 2 F + λ X which can be solved via proximal methods Proximal operator: prox λ (X) = arg min Z = UT λ (Σ)V { } 1 Z X 2F + λ Z 2 where SVD of X is X = UΣV with Σ = diag({σ i }), and T λ (Σ) = diag({(σ i λ) + }) Matrix recovery 11-66

74 (Optional) Proximal algorithm Algorithm 11.1 Proximal gradient methods for t = 0, 1, : ) X t+1 = T µt (X t µ t A A(X t ) where µ t : step size / learning rate Matrix recovery 11-67

75 Reference [1] Lecture notes, Advanced topics in signal processing (ECE 8201), Y. Chi, [2] Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, B. Recht, M. Fazel, P. Parrilo, SIAM Review, [3] Exact matrix completion via convex optimization, E. Candes, and B. Recht, Foundations of Computational Mathematics, 2009 [4] Matrix completion from a few entries, R. Keshavan, A. Montanari, S. Oh, IEEE Transactions on Information Theory, [5] Matrix rank minimization with applications, M. Fazel, Ph.D. Thesis, [6] Modeling the world from internet photo collections, N. Snavely, S. Seitz, and R. Szeliski, International Journal of Computer Vision, Matrix recovery 11-68

76 Reference [7] Computer vision: algorithms and applications, R. Szeliski, Springer, [8] Shape and motion from image streams under orthography: a factorization method, C. Tomasi and T. Kanade, International Journal of Computer Vision, [9] Topics in random matrix theory, T. Tao, American mathematical society, [10] A singular value thresholding algorithm for matrix completion, J. Cai, E. Candes, Z. Shen, SIAM Journal on Optimization, [11] Recovering low-rank matrices from few coefficients in any basis, D. Gross, IEEE Transactions on Information Theory, [12] Incoherence-optimal matrix completion, Y. Chen, IEEE Transactions on Information Theory, Matrix recovery 11-69

77 Reference [13] The power of convex relaxation: Near-optimal matrix completion, E. Candes, and T. Tao, [14] Tight oracle inequalities for low-rank matrix recovery from a minimal number of noisy random measurements, E. Candes, and Y. Plan, IEEE Transactions on Information Theory, [15] PhaseLift: Exact and stable signal recovery from magnitude measurements via convex programming, E. Candes, T. Strohmer, and V. Voroninski, Communications on Pure and Applied Mathematics, [16] Solving quadratic equations via PhaseLift when there are about as many equations as unknowns, E. Candes, X. Li, Foundations of Computational Mathematics, [17] A bound on tail probabilities for quadratic forms in independent random variables, D. Hanson, F. Wright, Annals of Mathematical Statistics, Matrix recovery 11-70

78 Reference [18] Hanson-Wright inequality and sub-gaussian concentration, M. Rudelson, R. Vershynin, Electronic Communications in Probability, [19] Exact and stable covariance estimation from quadratic sampling via convex programming, Y. Chen, Y. Chi, and A. Goldsmith, IEEE Transactions on Information Theory, [20] ROP: Matrix recovery via rank-one projections, T. Cai, and A. Zhang, Annals of Statistics, [21] Low rank matrix recovery from rank one measurements, R. Kueng, H. Rauhut, and U. Terstiege, Applied and Computational Harmonic Analysis, [22] Mathematics of sparsity (and a few other things), E. Candes, International Congress of Mathematicians, Matrix recovery 11-71