Mismatch and Resolution in CS

Transcription

1 Mismatch and Resolution in CS Albert Fannjiang, UC Davis Coworker: Wenjing Liao, UC Davis SPIE Optics + Photonics, San Diego, August 2-25, 20

2 Mismatch: gridding error Band exclusion Local optimization Numerical results Comparison Conclusion Outline

3 Example: spectral estimation Noisy signal: y(t) = s j= cje i2πω jt + n(t) where ωj are the frequencies, cj are the amplitudes and n(t) is the external noise. Main problem: the frequencies. Vectorization: Φx + e = y Set y = (y(t k )) C N to be the data vector where t k, k =,..., N are the sample times in the unit interval [0, ]. = We can only hope to recover ωj are separated by at least (resolution)

4 Approximate ωj by the closest subset of cardinality s of a regular grid G = {p,..., p M }, M s,. Write x = (xj) C M where xj = cj whenever the grid points are the nearest grid points to the frequencies and zero otherwise. The measurement matrix Φ = [ a... a M ] C N M with aj = (e i2πt kpj N ) C N, j =,..., M. Errors: e = n + d, n = external noise, d = gridding error.

5 0.25 relative gridding error versus the refinement fractor relative gridding error averaged in 00 trials refinement factor F Gridding error is inversely proportional to refinement factor F G = Z/F.

6 pairwise coherence pattern coherence versus the radius of excluded band average band excluded coherence in 00 trials 00*4000 matrix with F = 20 & coherence = radius of excluded band (unit:rayleigh Length) Coherence pattern Φ Φ for matrix with F = 20 (left).

7 Coherence band Let η > 0. Define the η-coherence band of the index k to be the set Bη(k) = {i µ(i, k) > η}, and the η-coherence band of the index set S to be the set Bη(S) = k S Bη(k). Due to the symmetry µ(i, k) = µ(k, i), i Bη(k) if and only if k Bη(i). Denote B η (2) (k) Bη(Bη(k)) = j B η(k) B η(j) B η (2) (S) Bη(Bη(S)) = k S B η (2) (k).

8 Band-excluded OMP We make the following change to the matching step imax = arg min i r n, ai, i / B η (2) (S n ) meaning that the double η-band of the estimated support in the previous iteration is avoided in the current search. This is natural if the sparsity pattern of the object is such that Bη(j), j supp(x) are pairwise disjoint. Algorithm. Band-Excluded Orthogonal Matching Pursuit (BOMP) Input: Φ, y, η > 0 Initialization: x 0 = 0, r 0 = y and S 0 = Iteration: For n =,..., s ) imax = arg mini r n, ai, i / B η (2) (S n ) 2) S n = S n {imax} 3) x n = arg minz Φz y 2 s.t. supp(z) S n 4) r n = y Φx n Output: x s.

9 Two-dimensional case

10 Performance guarantee Theorem Let x be s-sparse. Let η > 0 be fixed. Suppose that Bη(i) B η (2) (j) =, i, j supp(x) and that η(5s 4) x max + 5 e 2 < x min 2x min where xmax = max k x k, x min = min x k. k Let ˆx be the BOMP reconstruction. Then supp(ˆx) Bη(supp(x)) and moreover every nonzero component of ˆx is in the η-coherence band of a unique nonzero component of x. BOMP can resolve 3 RLs. Numerical experiments indicates resolution close to RL when the dynamic range is close to RL.

11 Local optimization Algorithm 2. Local Optimization (LO) Input:Φ, y, η > 0, S 0 = {i,..., i k }. Iteration: For n =, 2,..., k. ) x n = arg minz Φz y 2, supp(z) = (S n \{in}) {jn}, for some jn Bη({in}). 2) S n = supp(x n ). Output: S k. Algorithm 3. BLOOMP Input: Φ, y, η > 0 Initialization: x 0 = 0, r 0 = y and S 0 = Iteration: For n =,..., s ) imax = arg mini r n, ai, i / B η (2) (S n ) 2) S n = LO(S n {imax}) where LO is the output of Algorithm 2. 3) x n = arg minz Φz y 2 s.t. supp(z) S n 4) r n = y Φx n Output: x s.

12 BLOOMP: performance guarantee Theorem 2 Let η > 0 and let x be a s-sparse well-separated vector. Let S 0 and S k be the input and output, respectively, of the LO algorithm. If x min > (ε + 2(s )η) ( η + ( η) 2 + η 2 ) and each element of S 0 is in the η-coherence band of a unique nonzero component of x, then each element of S k remains in the η-coherence band of a unique nonzero component of x.

13 Band-Excluded Thresholding (BET) Two variants: Band-excluded Matched Thresholding (BMT) and Band-excluded Locally Optimized (BLOT). Input: Φ, y, η > 0. Initialization: S 0 =. Iteration: For k =,..., s, Algorithm 4. BMT ) i k = arg maxj y, aj, j / B η (2) (S k ). 2) S k = S k {i k } Output ˆx = arg minz Φz y 2 s.t. supp(z) S s Algorithm 5. BLOT Input: x = (x,..., x M ), Φ, y, η > 0. Initialization: S 0 =. Iteration: For n =, 2,..., s. ) in = arg minj xj, j B η (2) (S n ). 2) S n = S n {in}. Output: ˆx = arg min Φz y 2, supp(z) LO(S s ).

14 BLO-based algorithms BLO Subspace Pursuit (BLOSP) BLO Iterative Hard Thresholding (BLOIHT) Algorithm 6. BLOSP Input: Φ, y, η > 0. Initialization: x 0 = 0, r 0 = y Iteration: For n =, 2,..., ) S n = supp(x n ) supp(bmt(r n )) 2) x n = arg min Φz y 2 s.t. supp(z) S n. 3) S n = supp(blot( x n )) 4) r n = minz Φz y 2, supp(z) S n. 5) If r n 2 ɛ or r n 2 r n 2, then quit and set S = S n ; otherwise continue iteration. Output: ˆx = arg minz Φz y 2 s.t. supp(z) S.

15 Algorithm 7. BLOIHT Input: Φ, y, η > 0. Initialization: ˆx 0 = 0, r 0 = y. Iteration: For n =, 2,..., ) x n = BLOT(x n + Φ r n ). 2) If r n 2 ɛ or r n 2 r n 2, then quit and set S = S n ; otherwise continue iteration. Output: ˆx. In addition, the technique BLOT can be used to enhance the recovery capability with unresolved grids of the L -minimization principles, Basis Pursuit (BP) and the Lasso min z z, subject to y = Φz. min z 2 y Φz λσ z, where σ is the standard deviation of the each noise component and λ is the regularization parameter.

16 Numerical results For two subsets A and B in R d of the same cardinality, the Bottleneck distance d B (A, B) is defined as d B (A, B) = min max f M a A a f(a) where M is the collection of all one-to-one mappings from A to B. success probability versus noise level when dynamic range = 5 success probability versus the number of measurements when dynamic range = 5 and noise = success rate in 00 trials OMP BOMP BSP BCoSaMP BNIHT relative noise percentage success rate in 00 trials OMP BOMP BSP BCoSaMP BNIHT number of measurements For dynamic range greater than 3, BOMP has the best performance.

17 success probability versus dynamic range when noise = 0 success probability verus dynamic range when noise = % success rate in 00 trials OMP BOMP BSP BCoSaMP BNIHT dynamic range success rate in 00 trials LOOMP BLOOMP BLOSP BLOCoSaMP BLOIHT Lasso BLOT(0.5) Lasso BLOT(sqrt2) dynamic range LO dramatically improves the performance w.r.t. dynamic range

18 Spectral CS Duarte-Baraniuk 200: Spectral Iterated Hard Thresholding (SIHT) y = Φx + e = ΦΨα + e where Φ is i.i.d. Gaussian matrix and Ψ is an oversampled, redundant DFT frame. Assumption: α is widely separated. Performance metric: Ψ(α ˆα) Ψα

19 coherence versus the radius of excluded band coherence versus the radius of excluded band radius of excluded band (unit:rayleigh Length) radius of excluded band (unit:rayleigh Length) Coherence bands of the DFT frame Ψ (left) and Φ = ΦΨ (right).

20 SIHT(µ=0.) Lasso BSP BOMP OMP s OMP 2s OMP 5s BLOSP Lasso BLOT(0.5) BLOOMP relative error of the signal versus noise level relative error of the signal versus the number of measurements SIHT(µ=0.) BSP BOMP OMP s OMP 2s OMP 5s BP BLOSP BP BLOT BLOOMP average relative error of the signal in 00 trials average relative error of the signal in 00 trials relative noise percentage number of measurements Relative errors versus relative noise (left, dynamic range=) and number of measurements (right, dynamic range=0)

21 Frame-adapted BP: synthesis approach Candes et al 200: min z Ψ z, Φz y 2 ε Assumption: Ψ z is sparse..4 analysis coefficient Analysis coefficients Ψ z reorganized according to magnitudes.

22 Conclusion