Subspace Projection Matrix Completion on Grassmann Manifold

Size: px

Start display at page:

Download "Subspace Projection Matrix Completion on Grassmann Manifold"

Victor Williamson
5 years ago
Views:

1 Subspace Projection Matrix Completion on Grassmann Manifold Xinyue Shen and Yuantao Gu Dept. EE, Tsinghua University, Beijing, China ICASSP 2015, Brisbane

2 Contents 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

3 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

4 Background Subspace projection matrix {P X := XX X R N s, rank(x) = s} {S R N dim(s) = s} One to one correspondence with subspaces rank(p X ) = s Low rank matrix = Low dimensional subspace s N Degree of freedom = s(n s) Low degree of freedom = High dimensional subspace More structures: symmetric, non-zero eigenvalue 1,...

5 Background Matrix completion Uniformly random positions Ω i, i = 1,, M... Low rank matrix M O(N 5/4 s log N) [Candès and Recht, 2008, Caltech] M O(Ns log 2 N max{µ 2 1, µ 0 }) [Recht, 2009, UW-Madison]...

6 Background Grassmann manifold of s dimensional subspaces in R N Gr N,s := O(N) O(s)O(N s) (1) s = 1, RP N 1 = {[x] R N x tx, t 0} dim(gr N,s ) = s(n s) Tangent space at X Gr N,s T X Gr N,s = {ξ R N s : ξ T X = 0} (2) Inner product η, ξ T X Gr N,s η, ξ X := trace ( (X T X) 1 η T ξ ) (3)

7 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

8 Motivation Exploration: fewer samples? Structures: sparsity, low rank, low-dim manifolds... M O(s(N s))? Subspace identification A demonstration: Multi-bandpass filter x(0) x(1) x(n-1) y(0) y(1) y(n-1)

9 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

10 Formulation Setup X 0 R N s, dim(span(x 0 ))=s A : R N N R N N [A(P X0 )] Ωi : = [P X0 ] Ωi, i = 1, 2, M, [A(P X0 )] i,j : = 0, (i, j) / Ω Ω {1, 2,, N} {1, 2,, N} uniformly randomly chosen, Ω = M < N 2 Y := A(P X0 )

11 Formulation Recovery P X = XX = X X T, X Gr N,s ˆX = argmin X GrN,s A(XX T ) Y 2 F (4) Cost function well-defined on Gr N,s Non-convex optimization Recovery error: projection distance d p (span( ˆX), span(x 0 )) = 1 2 X 0X T 0 ˆX ˆX T 2 F (5)

12 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

13 Algorithm: GGDLS Grassmann Manifold Gradient Descent Line Search Algorithm Require: a smooth scalar field f on Gr N,s ; a smooth mapping R : T Gr N,s Gr N,s ; scalars for the Armijo step size α > 0, β, σ (0, 1); Input: Initial iterate X 0 Gr N,s ; Output: Sequence of iterates {X k }. For k = 0, 1, 2, do: 1.Compute the Euclidean gradient f Xk at X k ; 2.Project f Xk onto the tangent space T Xk Gr N,s to obtain η k ; 3.X k+1 = R Xk (t k η k ), t k is the Armijo step size; Until: Stopping criterion satisfied;

14 Algorithm: GGDLS [ f X ] p,q = j:(p,j) Ω [P X Y ] p,j X j,q + i:(i,p) Ω [P X Y ] i,p X i,q Definition gradf(x) defined as the unique element in T X M such that gradf(x), ξ X = f(x)[ξ], ξ T X M (6) gradf(x k ) = η k = (I N X k Xk T) f X k Definition R defined by R X : T X M M such that R X (0) = X, R X (0) = id TX M (7) R X : Q factor of the QR decomposition of X

15 1 Background 2 Motivation 3 Formulation 4 Algorithm: GGDLS 5 Numerical Experiment

16 Numerical Experiment Subspace projection matrix completion: noiseless N = 100, s = 10, α = 1, β = 0.8, σ = 0.01, initial value from Y Left figure: X 0 X T 0 ˆX ˆX T F / s < 10 2 for successful recovery, 100 trials per point Right figure: M s(n s) = 3.5 Probability of Recovery IALM LMaFit 0.2 GGDLS SRF FPCA M/(s(N s)) Time elapsed per recovery (sec) GGDLS LMaFit FPCA IALM SRF

17 Numerical Experiment Subspace projection matrix completion: noisy N = 100, s = 10, α = 1, β = 0.8, σ = 0.01, initial value from Y M s(n s) = trials per point RSNR(dB) GGDLS SRF 10 LMaFit FPCA MSNR(dB)

18 Numerical Experiment Subspace projection matrix completion: noiseless N = 20, α = 100, β = 0.8, σ = 0.01, initial value from Y X 0 X0 T ˆX ˆX T F / s < 10 2 for successful recovery 200 trials per point Probability 100% 60 90% number of samples M s(n s) 80% 70% 60% 50% 40% 30% % % rank s

19 Theoretical Support: Subspace Projection Matrix Sensing Theorem For fixed 0 < ε < 1 and β > 0, assume that N 3. Let A : R N N R M be a random orthoprojector with M ( 2 + β ε 2 ε 3 /3 ) O ( s(n s) log N ε ). (8) If M < N 2, then with probability exceeding 1 e c1βs(n s), in which c 1 is a universal constant, the following property holds for every pair of P X, P Y P N,s, P X P Y (1 ε) M N A(P X P Y ) 2 M (1 + ε) P X P Y F N. (9) Restricted Isometry Property of Subspace Projection Matrix Under Random Compression, Xinyue Shen and Yuantao Gu, 2015, IEEE Signal Processing Letters, 22(9),

20 Conclusion Subspace projection matrices Structural data: on manifold Low rank: more structures, less number of samples High rank: low intrinsic dimension GGLDS algorithm Optimize on a non-convex manifold Exploit the unique structure of subspace projection matrices

Subspace Projection Matrix Recovery from Incomplete Information

Subspace Projection Matrix Recovery from Incomplete Information Xinyue Shen and Yuantao Gu Abstract Structural signal retrieval from highly incomplete information is a concerning problem in the field of