Three Generalizations of Compressed Sensing

Transcription

1 Thomas Blumensath School of Mathematics The University of Southampton June, 2010 home prev next page

2 Compressed Sensing and beyond y = Φx + e x R N or x C N x K is K-sparse and x x K 2 is small y R M or y C M Φ is linear with nontrivial null-space i.e. M < N Φ satisfies RIP, nullspace property, incoherence property or alternative g = T (f) + e f H where H is a Hilbert space f A lies in a closed (non-convex) subset A H and f f A is small g B where B is a Banach space T is linear or non-linear with nontrivial null-set or ill-conditioned T is bi-lipschitz on A (or has Restricted Strict Convexity Property) Recovery by convex optimization, CoSaMP, IHT, OMP and co. Recovery by Projected Landweber Algorithm home prev next page 1 of 16

3 The general framework Let T be a bounded continuous map (possibly non-linear) between a Hilbert space H and a Banach space B. Given noisy measurements g = T (f) + e, where g, e B, f H, we want to find a good estimate of f. We are interested in problems in which T is ill-conditioned or non-invertible. To solve the problem, we make the assumption that f lies in or close to a known subset A H. Main question: Under which conditions can we efficiently recover good estimates of f? home prev next page 2 of 16

4 The bi-lipschitz Condition In order to be able to recover all elements f A, we need T to be one to one as a map from A to (a subset of) B. However, to deal with errors, we also require the inversion to be stable. This can only be achieved if the inverse not only exists, but is also Lipschitz. In particular, we require that there exist constants 0 < α and β < such that α f 1 f 2 2 T (f 1 f 2 ) 2 B β f 1 f 2 2, for all f 1, f 2 A. We let the bi-lipschitz constants of T on A be the smallest constants α and β for which the above condition holds for all f 1, f 2 A. home prev next page 3 of 16

5 Projecting onto the constraint set We can define the set-valued mapping and an ɛ-projection, such that p ɛ A(f) = { f : f f 2 inf f ˆf 2 + ɛ} ˆf A P ɛ A(f) p ɛ A(f). Importantly, f A satisfies f f A 2 inf f ˆf 2 + ɛ ˆf A NOTE: If for all f H, there exist f A A such that f f A 2 = inf ˆf A f ˆf 2, then we call A proximal and set ɛ = 0, else we have to choose ɛ > 0. home prev next page 4 of 16

6 The naive solution The naive solution to the problem would be to search through the set A H until we find an element f opt such that g T (f opt ) B inf g T ( ˆf) B + δ, ˆf A for some small δ. For linear T and B a Hilbert space, if T is bi-lipschitz on A, then we have f f opt 2 α T (f f A ) + e B + f f A + δ α + ɛ, where f A = P ɛ A (f). home prev next page 5 of 16

7 Projected Landweber Algorithm Given g and T, let f 0 = 0 and let ɛ n > 0 (or ɛ n 0 if A is proximal). The Projected Landweber Algorithm is the iterative procedure defined by the recursion f n+1 = P ɛn A (f n µ 2 f n ), where f n is a subgradient of g T f 2 B evaluated at f n. A subgradient at f 1 is any f1 H, such that Re f1, f 2 + g T (f 1 ) 2 B g T (f 1 + f 2 ) 2 B holds for all f 2. home prev next page 6 of 16

8 Projected Landweber Algorithm with linear Hilbert space observations If B is a Hilbert space and if T is linear, then the Projected Landweber algorithm can be written as the iterative procedure f n+1 = P ɛn A (f n + µt (g T f n )), which in Euclidean space with sparsity constraint simplifies to the Iterative Hard Thresholding algorithm x n+1 = H K (x n + µφ (y Φx n )). home prev next page 7 of 16

9 Recovery guarantee for linear observations in Hilbert space Theorem 1. Let T be a linear map between Hilbert spaces H and B, such that T is bi-lipschitz from A to B with constants β 1 µ < 1.5α. Let f A = PA ɛ (f), then, for any f H, after n = ẽ + ɛ 2 ln(δ 2µ f A ) ln(2/(µα) 2) iterations, the Projected Landweber Algorithm calculates a solution f n satisfying f f n ( c + δ) ( ẽ + where c 4 3α 2µ and ẽ = T (f f A) + e. ) ɛ 2µ + f A f. home prev next page 8 of 16

10 A note on the error bound Note that the result is in terms of the error ẽ = g T f A = T (f f A ) + e. This is in contrast to standard CS results, where due to the special structure of A, the bi-lipschitz property also gives a bound on T (f f A ) in terms of f f A. In order to guarantee stability to this error in the general setting, T has to be bounded. home prev next page 9 of 16

11 EXAMPLE: Low rank matrix recovery Let A R m n be the set of matrices X with rank no more than K and let T be a linear map from R m n to R M. Vectorising X as x R N, we can write the observations in the form y = Φx + e for some matrix Φ. In it was also shown that Theorem 2. If Φ is a random nearly isometrically distributed linear map, then with probability 1 e c 1M, (1 δ) X 1 X 2 F T (X 1 X 2 ) (1 + δ) X 1 X 2 F for all rank K matrices X 1, X 2 R m n, whenever N c 0 K(m + n)log(mn), where c 1 and c 0 are constants depending on δ only. B. Recht, M. Fazel, and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, arxiv,, no. arxiv: v1, home prev next page 10 of 16

12 EXAMPLE: Analog Compressed Sensing Theorem 3. Let H be the set of real valued square integrable functions whose Fourier transform is supported on the interval [ B N /2, B N /2]. Let A be the subset of H of functions, whose Fourier transform is supported in S [ B N /2, B N /2], where S is the union of finitely many intervals of finite width, so that S < B N. Let B be the set of square integrable functions whose Fourier transform has a support in [ B M /2, B M /2]. There exist bi-lipschitz embeddings from A to B, whenever where c is some constant. B M c S ln ( ) BN, S home prev next page 11 of 16

13 EXAMPLE: Inverse problems Let H be the Hilbert space of bounded differentiable functions defined on 0 x π with f(0) = 0 = f(π) and let T be the operator defined by the pde y(x, t) t = κ 2 y(x, t) x 2, y(0, t) = 0 = y(π, t) such that T y(x, 0) = y(x, 1). Theorem 4. T is bi-lipschitz on A H if A is the one dimensional manifold f(τ; x) = n=1 C n(τ) sin(nx), 0 τ 1, where the C n (τ) satisfy: 1. For all 0 τ 1, τ 2 1, n=1 (C n(τ 1 ) C n (τ 2 )) 2 < 2. For all 0 τ 1 and n, C n (τ) is a twice differentiable function with first derivative C n(τ) γ n for some γ n > 0, where γ n depends only on n. 3. For all 0 τ 1 and n, n=1 C 2 n (τ) Γ for some Γ R. home prev next page 12 of 16

14 Nonlinear Compressed Sensing Let B be a Banach space, let T be non-linear and let A be some non-convex subset of H. We say that T and B have the Restricted Strict Convexity Property (RSCP) on A, if for some 0 < α and β < α g T (f 1) 2 B g T (f 2) 2 B Re (f 2), (f 1 f 2 ) f 1 f 2 2 β, (1) holds for all f 1, f 2 H for which f 1 f 2 A + A. The RSCP constants α and β are assumed to be the best possible. home prev next page 13 of 16

15 Nonlinear Compressed Sensing Result Theorem 5. Let A be a proximal union of subspaces. Given f = T (f) + e where f H, assume T and 2 B Property satisfy the Restricted Strict Convexity α g T (f 1) 2 B g T (f 2) 2 B Re (f 2), (f 1 f 2 ) f 1 f 2 2 β, (2) for all f 1, f 2 H for which f 1 f 2 A + A + A with constants β 1/µ 4/3α, then, after n = 2 ln ( δ ẽ B f A ) ln (1 µα) iterations, the Projected Landweber Algorithm calculates a solution f n satisfying f n f (2 µ + δ) ẽ B + f f A. (3) where ẽ = g T (f A ). home prev next page 14 of 16

16 Conclusions Compressed Sensing results hold in much more general settings. Sparsity can be replaced by quite general non-convex sets A. The finite dimensional Euclidean setting can be relaxed to more general Hilbert spaces. Non-linear mappings can be considered. The error in the observation domain can be measured with more general norms. This general setting unifies many recent results and offers a general approach to solve new problems. home prev next page 15 of 16

17 home prev next page 16 of 16