Interpolation via weighted l 1 -minimization

Transcription

1 Interpolation via weighted l 1 -minimization Holger Rauhut RWTH Aachen University Lehrstuhl C für Mathematik (Analysis) Mathematical Analysis and Applications Workshop in honor of Rupert Lasser Helmholtz Zentrum München September 20, 2013 Joint work with Rachel Ward (University of Texas at Austin)

2 Function interpolation Aim Given a function f : D C on a domain D reconstruct or interpolate f from sample values f (t 1 ),..., f (t m ). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 2

3 Function interpolation Aim Given a function f : D C on a domain D reconstruct or interpolate f from sample values f (t 1 ),..., f (t m ). Approaches (Linear) polynomial interpolation assumes (classical) smoothness in order to achieve error rates works with special interpolation points (e.g. Chebyshev points). Compressive sensing reconstruction nonlinear assumes sparsity (or compressibility) of a series expansion in terms of a certain basis (e.g. trigonometric bases) fewer (random!) sampling points than degrees of freedom Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 2

4 Function interpolation Aim Given a function f : D C on a domain D reconstruct or interpolate f from sample values f (t 1 ),..., f (t m ). Approaches (Linear) polynomial interpolation assumes (classical) smoothness in order to achieve error rates works with special interpolation points (e.g. Chebyshev points). Compressive sensing reconstruction nonlinear assumes sparsity (or compressibility) of a series expansion in terms of a certain basis (e.g. trigonometric bases) fewer (random!) sampling points than degrees of freedom This talk: Combine sparsity and smoothness! Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 2

5 A classical interpolation result C r ([0, 1] d ): r-times continuously differentiable periodic functions Existence of set of sampling points t 1,..., t m and linear reconstruction operator R : C m C r ([0, 1] d ) such that for every f C r ([0, 1] d ) the approximation f = R(f (t 1 ),..., f (t m )) satisfies the optimal error bound f f Cm r/d f C r. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 3

6 A classical interpolation result C r ([0, 1] d ): r-times continuously differentiable periodic functions Existence of set of sampling points t 1,..., t m and linear reconstruction operator R : C m C r ([0, 1] d ) such that for every f C r ([0, 1] d ) the approximation f = R(f (t 1 ),..., f (t m )) satisfies the optimal error bound f f Cm r/d f C r. Curse of dimension: Need about m C f ε d/r samples for achieving error ε < 1. Exponential scaling in d cannot be avoided using only smoothness (DeVore, Howard, Micchelli 1989 Novak, Wozniakowski 2009). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 3

7 Sparse representation of functions D: domain endowed with a probability measure ν ψ j : D C, j Γ (finite or infinite) {ψ j } j Γ orthonormal system: ψ j (t)ψ k (t)dν(t) = δ j,k, j, k Γ D We consider functions of the form f (t) = j Γ x j ψ j (t) f is called s-sparse if x 0 := {l : x l 0} s and compressible if the error of best s-term approximation error is small. σ s (f ) q := σ s (x) q := inf x z q (0 < q ) z: z 0 s Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 4

8 Fourier Algebra and Compressibility Fourier algebra A p = {f C[0, 1] : f p < }, 0 < p 1, f p := x p = ( x j p ) 1/p, f (t) = x j ψ j (t). j Z j Z Motivating example: D = [0, 1], ν Lebesgue measure, ψ j (t) = e 2πijt, t [0, 1] d, j Z d. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 5

9 Fourier Algebra and Compressibility Fourier algebra A p = {f C[0, 1] : f p < }, 0 < p 1, f p := x p = ( x j p ) 1/p, f (t) = x j ψ j (t). j Z j Z Motivating example: D = [0, 1], ν Lebesgue measure, ψ j (t) = e 2πijt, t [0, 1] d, j Z d. Compressibility via Stechkin estimate σ s (f ) q = σ s (x) q s 1/q 1/p x p = s 1/q 1/p f p. Since f := sup x [0,1] f (t) f 1, the best s-term approximation f 0 = j S x jψ j, S = s, satisfies f f 0 s 1 1/p f p. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 5

10 Sampling Task: Reconstruct sparse or compressible f (t) = j Γ x j ψ j (t) from samples f (t 1 ),..., f (t m ) with given sampling points t 1,..., t m D. With y l = f (t l ), l = 1,..., m and sampling matrix A l,j = ψ j (t l ), l = 1,..., m; j Γ we can write y = Ax Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 6

11 Sampling Task: Reconstruct sparse or compressible f (t) = j Γ x j ψ j (t) from samples f (t 1 ),..., f (t m ) with given sampling points t 1,..., t m D. With y l = f (t l ), l = 1,..., m and sampling matrix A l,j = ψ j (t l ), l = 1,..., m; j Γ we can write y = Ax Aim: Minimal number m of required samples, m N = Γ. Leads to underdetermined linear system. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 6

12 Detour: Compressive sensing Reconstruction of x C N from y = Ax with A C m N, m N. Ingredients: Sparsity / Compressibility Efficient reconstruction algorithms Randomness / Random matrices Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 7

13 Detour: Compressive sensing Reconstruction of x C N from y = Ax with A C m N, m N. Ingredients: Sparsity / Compressibility Efficient reconstruction algorithms Randomness / Random matrices Applications in Signal / Image Processing: Radar, Magnetic Resonance Imaging, Optics, Statistics, Astronomy,... Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 7

14 Reconstruction via l 1 -minimization l 0 -minimization min z 0 subject to Az = y is NP-hard. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 8

15 Reconstruction via l 1 -minimization l 0 -minimization min z 0 subject to Az = y is NP-hard. Convex relaxation: l 1 -minimization min z 1 subject to Az = y Version for noisy data: min z 1 subject to Az y 2 η. Alternatives: Orthogonal Matching Pursuit, Iterative Hard Thresholding, CoSaMP, Iteratively Reweighted Least Squares,... Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 8

16 Restricted Isometry Property (RIP) Recovery guarantees, Error Estimates? Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 9

17 Restricted Isometry Property (RIP) Recovery guarantees, Error Estimates? Definition The restricted isometry constant δ s of a matrix A C m N is defined as the smallest δ s such that for all s-sparse x C N. (1 δ s ) x 2 2 Ax 2 2 (1 + δ s ) x 2 2 Requires that all s-column submatrices of A are well-conditioned. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 9

18 RIP implies recovery by l 1 -minimization Theorem (Candès, Romberg, Tao 04 Candès 08 Foucart, Lai 09 Foucart 09/ 12 Li, Mo 11 Andersson, Strömberg 12) Assume that the restricted isometry constant of A C m N satisfies δ 2s < 4/ Then l 1 -minimization reconstructs every s-sparse vector x C N from y = Ax. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 10

19 Stability Theorem (Candès, Romberg, Tao 04 Candès 08 Foucart, Lai 09 Foucart 09/ 12 Li, Mo 11 Andersson, Strömberg 12) Let A C m N with δ 2s < 4/ Let x C N, and assume that noisy data are observed, y = Ax + η with η 2 σ. Let x # by a solution of min z z 1 such that Az y 2 σ. Then x x # 2 C σ s(x) 1 s + Dσ and x x # 1 Cσ s (x) 1 + D sσ for constants C, D > 0, that depend only on δ 2s. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 11

20 Matrices satisfying the RIP Open problem: Give explicit matrices A C m N with small δ 2s 0.62 for large s. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 12

21 Matrices satisfying the RIP Open problem: Give explicit matrices A C m N with small δ 2s 0.62 for large s. Goal: δ s δ, if for constants C δ and α. m C δ sln α (N), Deterministic matrices known, for which m C δ,k s 2 suffices if N m k. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 12

22 Matrices satisfying the RIP Open problem: Give explicit matrices A C m N with small δ 2s 0.62 for large s. Goal: δ s δ, if for constants C δ and α. m C δ sln α (N), Deterministic matrices known, for which m C δ,k s 2 suffices if N m k. Way out: consider random matrices. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 12

23 RIP for Gaussian and Bernoulli matrices Gaussian: entries of A independent N (0, 1) random variables Bernoulli : entries of A independent Bernoulli ±1 distributed rv Theorem Let A R m N be a Gaussian or Bernoulli random matrix and assume m Cδ 2 (s ln(en/s) + ln(2ε 1 )) for a universal constant C > 0. Then with probability at least 1 ε the restricted isometry constant of 1 m A satisfies δ s δ. Consequence: Recovery via l 1 -minimization with probability exceeding 1 e cm provided m Cs ln(en/s). Bound is optimal as follows from lower bound for Gelfand widths of l p -balls, 0 < p 1. (Gluskin, Garnaev 1984 Foucart, Pajor, R, Ullrich 2010) Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 13

24 Back to Sampling (ψ j ) j Γ bounded orthonormal system: max j Γ ψ j K Sampling points t 1,..., t m are chosen i.i.d. according to orthogonalization measure ν. Sampling matrix A with entries A l,j = ψ j (t l ) is random matrix. Theorem (Candès, Tao 06 Rudelson, Vershynin 06 R 08/ 10) If m CK 2 δ 2 s max{ln 3 (s) ln(n), ln(ε 1 )}, then the restricted 1 isometry constant of m A satisfies δ s δ with probability at least 1 ε. Implies stable recovery of all s-sparse f from m C K s ln 4 (N) random samples via l 1 -minimization with high probability. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 14

25 Trigonometric Polynomials D = [0, 1] d, ν: Lebesgue measure ψ j (t) = e 2πij t, j Z d, t [0, 1] d Boundedness constant K = 1 Exact recovery of s-sparse trigonometric polynomials from m Cs ln 3 (s) ln(n) i.i.d. samples uniformly distributed on [0, 1] d via l 1 -minimization. Error estimate for general f (Fourier coefficients supported on Γ) f f 2 Cσ s (f ) 1 / s Cs 1/2 1/p f p, f f C s 1 1/p f p, ( ) f f m 1 1/p C ln(m) 3 f p. ln(n) Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 15

26 Chebyshev polynomials Chebyshev-polynomials C j, j = 0, 1, 2, dx C j (x)c k (x) π 1 x = δ j,k, j, k N 0, 2 C 0 = 1 and C j = 2. Stable recovery of polynomials that are s-sparse in the Chebyshev system from m Cs ln 3 (s) ln(n) samples drawn i.i.d. from the Chebyshev measure dx π 1 x 2. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 16

27 Legendre polynomials L j, j = 0, 1, 2, Legendre polynomials L j (x)l k (x)dx = δ j,k, L j j + 1, j, k N 0. K = max j=0,...,n 1 L j = N. Leads to bound m CK 2 s ln 3 (s) ln(n) = CNs ln 3 (s) ln(n). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 17

28 Legendre polynomials L j, j = 0, 1, 2, Legendre polynomials L j (x)l k (x)dx = δ j,k, L j j + 1, j, k N 0. K = max j=0,...,n 1 L j = N. Leads to bound m CK 2 s ln 3 (s) ln(n) = CNs ln 3 (s) ln(n). Preconditioned system Q j (x) = v(x)l j (x) with v(x) = (π/2) 1/2 (1 x 2 ) 1/4 satisfies 1 1 dx Q j (x)q k (x) π 1 x = δ j,k, Q j 3, j, k N 0. 2 Stable recovery of polynomials that are s-sparse in the Chebyshev system from m Cs ln 3 (s) ln(n) samples drawn i.i.d. from the Chebyshev measure dx π 1 x 2. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 17

29 Spherical harmonics Y k l, k l k, k N 0 : orthonormal system in L 2 (S 2 ) 1 4π 2π π 0 0 Yl k k (φ, θ)yl (θ, φ) sin(θ)dφdθ = δ l,l δ k,k (φ, θ) [0, 2π) [0, π): spherical coordinates (x = cos(θ) sin(φ), y = sin(θ) sin(φ), z = cos(φ)) S 2 With ultraspherical polynomials p α n, Yl k (φ, θ) = eikφ (sin θ) k p k l k (cos θ), (φ, θ) [0, 2π) [0, π) Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 18

30 Restricted Isometry Property for Spherical Harmonics L -bound: Y k l k 1/2 Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 19

31 Restricted Isometry Property for Spherical Harmonics L -bound: Y k l k 1/2 Preconditioning I (Krasikov 08) With w(θ, φ) = sin(θ) 1/2 wy k l Ck 1/4. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 19

32 Restricted Isometry Property for Spherical Harmonics L -bound: Y k l k 1/2 Preconditioning I (Krasikov 08) With w(θ, φ) = sin(θ) 1/2 wy k l Ck 1/4. Preconditioning II (Burq, Dyatkov, Ward, Zworski 12) With v(θ, φ) = sin 2 (θ) cos(θ) 1/6, vy k l Ck 1/6. RIP for associated preconditioned random sampling matrix 1 m A C m N with sampling points drawn according to ν(dθ, dφ) = v 2 (θ, φ) sin(θ)dθdφ = tan(θ) 1/3 dθdφ with high probability if m CsN 1/6 log 4 (N). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 19

33 Questions Can we take into account if ψ j is growing with j, i.e., not uniformly small? How can we combine sparsity with smoothness? Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 20

34 Questions Can we take into account if ψ j is growing with j, i.e., not uniformly small? How can we combine sparsity with smoothness? Use weights! Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 20

35 Trigonometric system: smoothness and weights ψ j (t) = e 2πijt, j Z, t [0, 1] Derivatives satisfy ψ j = 2π j, j Z. For f (t) = j x jψ j (t) we have f + f = j x j ψ j + j x j ψ j x j ( ψ j + ψ j ) j Z = x j (1 + 2π j ) =: x ω,1. j Z Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 21

36 Trigonometric system: smoothness and weights ψ j (t) = e 2πijt, j Z, t [0, 1] Derivatives satisfy ψ j = 2π j, j Z. For f (t) = j x jψ j (t) we have f + f = j x j ψ j + j x j ψ j x j ( ψ j + ψ j ) j Z = x j (1 + 2π j ) =: x ω,1. j Z Weights model smoothness! Combine with sparsity (compressibility) weighted l p -spaces with 0 < p 1 Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 21

37 Weighted norms and weighted sparsity For a weight ω = (ω j ) j Γ with ω j 1, introduce x ω,p := ( j Γ x j p ω 2 p j ) 1/p, 0 < p 2. Special cases: x ω,1 = j Γ x j ω j, x ω,2 = x 2 Weighted sparsity x ω,0 := j:x j 0 x is called weighted s-sparse if x ω,0 s. ω 2 j Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 22

38 Weighted best s-term approximation error σ s (x) ω,p := Weighted best approximation inf z: z ω,0 s x z ω,p Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 23

39 Weighted best s-term approximation error σ s (x) ω,p := Weighted best approximation inf z: z ω,0 s x z ω,p Weighted quasi-best s-term approximation error of x Consider nonincreasing rearrangement of (x j ω 1 j ) j Γ, i.e., permutation π such that x π(1) ω 1 π(1) x π(2) ω 1 π(2) Choose largest k such that k l=1 ω2 π(l) s, set S = {π(1),..., π(k)} and σ s (x) ω,p := x x S ω,p, where (x S ) j = x j for j S and (x S ) j = 0 for j / S. (Note that x S ω,0 s by construction.) Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 23

40 Weighted Stechkin estimate Theorem For a weight ω, a vector x, 0 < p < q 2 and s > ω 2, σ s (x) ω,q σ s (x) ω,q (s ω 2 ) 1/q 1/p x ω,p. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 24

41 Weighted Stechkin estimate Theorem For a weight ω, a vector x, 0 < p < q 2 and s > ω 2, σ s (x) ω,q σ s (x) ω,q (s ω 2 ) 1/q 1/p x ω,p. If s 2 ω 2, say, then σ s (x) ω,q σ s (x) ω,q C p,q s 1/q 1/p x ω,p, C p,q = 2 1/p 1/q. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 24

42 Weighted Stechkin estimate Theorem For a weight ω, a vector x, 0 < p < q 2 and s > ω 2, σ s (x) ω,q σ s (x) ω,q (s ω 2 ) 1/q 1/p x ω,p. If s 2 ω 2, say, then σ s (x) ω,q σ s (x) ω,q C p,q s 1/q 1/p x ω,p, C p,q = 2 1/p 1/q. Lower bound on s natural because otherwise the single element set S = {j} with ω j = ω not allowed as support set. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 24

43 Weighted Stechkin estimate Theorem For a weight ω, a vector x, 0 < p < q 2 and s > ω 2, σ s (x) ω,q σ s (x) ω,q (s ω 2 ) 1/q 1/p x ω,p. If s 2 ω 2, say, then σ s (x) ω,q σ s (x) ω,q C p,q s 1/q 1/p x ω,p, C p,q = 2 1/p 1/q. Lower bound on s natural because otherwise the single element set S = {j} with ω j = ω not allowed as support set. Lemma If s w 2, then σ 3s (x) ω,p σ s (x) ω,p. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 24

44 (Weighted) Compressive Sensing Recover a weighted s-sparse (or weighted-compressible) vector x from measurements y = Ax, where A C m N with m < N. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 25

45 (Weighted) Compressive Sensing Recover a weighted s-sparse (or weighted-compressible) vector x from measurements y = Ax, where A C m N with m < N. Weighted l 1 -minimization min z C N z ω,1 subject to Az = y Noisy version min z C N z ω,1 subject to Az y 2 η Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 25

46 Weighted restricted isometry property (WRIP) Definition The weighted restricted isometry constant δ ω,s of a matrix A C m N is defined to be the smallest constant such that (1 δ ω,s ) x 2 2 Ax 2 2 (1 + δ ω,s ) x 2 2 for all x C N with x ω,0 = l:x l 0 ω2 j s. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 26

47 Weighted restricted isometry property (WRIP) Definition The weighted restricted isometry constant δ ω,s of a matrix A C m N is defined to be the smallest constant such that (1 δ ω,s ) x 2 2 Ax 2 2 (1 + δ ω,s ) x 2 2 for all x C N with x ω,0 = l:x l 0 ω2 j s. Since ω j 1 by assumption, the classical RIP implies the WRIP, δ ω,s δ 1,s = δ s. Alternative name: Weighted Uniform Uncertainty Principle (WUUP) Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 26

48 Recovery via weighted l 1 -minimization Theorem Let A C m N and s 2 ω 2 such that δ ω,3s < 1/3. For x C N and y = Ax + e with e 2 η let x be a minimizer of Then min z ω,1 subject to Az y 2 η. x x ω,1 C 1 σ s (x) ω,1 + D 1 sη, x x 2 C 2 σ s (x) ω,1 s + D 2 η. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 27

49 Function interpolation {ψ j } j Γ, finite ONS with respect to prob. measure ν. Given samples y 1 = f (t 1 ),..., y m = f (t m ) of f (t) = j Γ x jψ j (t) reconstruction amounts to solving y = Ax with sampling matrix A C m N, N = Γ, given by A lk = ψ k (t l ). Use weighted l 1 -minimization to recover weighted-sparse or weighted-compressible x when m < Γ. Choose t 1,..., t m i.i.d. at random according to ν in order to analyze the WRIP of the sampling matrix. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 28

50 Weighted RIP of random sampling matrix ψ j : D C, j Γ, N = Γ <, ONS w.r.t. prob. measure ν. Weight ω with ω j ψ j. Sampling points t 1,..., t m taken i.i.d. at random according to ν. Random sampling matrix A C m N with entries A lj = ψ j (t l ). Theorem If m Cδ 2 s ln 3 (s) ln(n) then the weighted restricted isometry constant of 1 m A satisfies δ ω,s δ with probability at least 1 N ln3 N. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 29

51 Weighted RIP of random sampling matrix ψ j : D C, j Γ, N = Γ <, ONS w.r.t. prob. measure ν. Weight ω with ω j ψ j. Sampling points t 1,..., t m taken i.i.d. at random according to ν. Random sampling matrix A C m N with entries A lj = ψ j (t l ). Theorem If m Cδ 2 s ln 3 (s) ln(n) then the weighted restricted isometry constant of 1 m A satisfies δ ω,s δ with probability at least 1 N ln3 N. Generalizes previous results (Candès, Tao Rudelson, Vershynin Rauhut) for systems with ψ j K for all j Γ, where the sufficient condition is m Cδ 2 K 2 s log 4 N. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 29

52 Abstract weighted function spaces A ω,p = {f : f (t) = j Γ x j ψ j (t), f ω,p := x ω,p < } If ω j ψ j then f f ω,1. If ω j ψ j + ψ j (when D R) then f + f f ω,1, and so on... Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 30

53 Interpolation via weighted l 1 -minimization Theorem Assume N = Γ <, ω j ψ j and 0 < p < 1. Choose t 1,..., t m i.i.d. at random according to ν where m Cs log 4 (N) for s 2 ω 2. Then with probability at least 1 N ln3 N the following holds for each f A ω,p. Let x be the solution of min z C N z ω,1 subject to j Γ z j ψ j (t l ) = f (t l ), l = 1,..., m and set f (t) = j Γ x j ψ j(t). Then f f f f ω,1 C 1 s 1 1/p f ω,p, f f L 2 ν C 2 s 1/2 1/p f ω,p. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 31

54 Quasi-interpolation in infinite-dimensional spaces Γ =, lim j ω j = and ω j ψ j. Theorem Let f A ω,p for some 0 < p < 1, and set Γ s = {j Γ : ω 2 j s/2} for some s. Choose t 1,..., t m i.i.d. at random according to ν where m Cs log 4 ( Γ s ). With η = f ω,p / s let x be the solution to min z ω,1 subject to (f (t l ) z j ψ j (t l )) m z C Γs l=1 2 η m. j Γ s and put f (t) = j Γ s x j ψ j(t). Then with prob. 1 N ln3 (N) f f f f ω,1 C 1 s 1 1/p f ω,p, f f L 2 ν C 2 s 1/2 1/p f ω,p. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 32

55 Numerical example I for the trigonometric system Original function Runge s example f (x) = x 2 Weights: w j = 1 + j 20 Interpolation points chosen uniformly at random from [ 1, 1]. 1 Least squares solution 1 Unweighted l1 minimizer 1 Weighted l1 minimizer Residual error 0.5 Residual error 0.5 Residual error Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 33

56 Numerical example II for the trigonometric system Original function f (x) = x Weights: w j = 1 + j. 20 Interpolation points chosen uniformly at random from [ 1, 1]. 1 Least squares solution 1 Unweighted l1 minimizer 1 Weighted l1 minimizer Residual error 0.5 Residual error 0.5 Residual error Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 34

57 Numerical example for Chebyshev polynomials Original function f (x) = x 2 Weights: w j = 1 + j. 20 Interpolation points chosen i.i.d. at random according to Chebyshev measure dν(x) = dx π 1 x 2. 1 Least squares solution 1 Unweighted l1 minimizer 1 Weighted l1 minimizer Residual error 0.5 Residual error 0.5 Residual error Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 35

58 Numerical example for Legendre polynomials Original function f (x) = x 2 Weights: w j = 1 + j. 20 Interpolation points chosen i.i.d. at random according to Chebyshev measure. 1 Least squares solution 1 Unweighted l1 minimizer 1 Weighted l1 minimizer Residual error 0.5 Residual error 0.5 Residual error Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 36

59 Back to spherical harmonics Y k l, k l k, k N 0 : spherical harmonics. Recall: L -bound: Yl k k 1/2 Preconditioned L -bound for v(θ, φ) = sin 2 (θ) cos(θ) 1/6 : vy k l Ck 1/6. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 37

60 Back to spherical harmonics Y k l, k l k, k N 0 : spherical harmonics. Recall: L -bound: Yl k k 1/2 Preconditioned L -bound for v(θ, φ) = sin 2 (θ) cos(θ) 1/6 : vy k l Ck 1/6. Weighted RIP: With weights ω k,l k 1/6 the preconditioned random sampling matrix 1 m A C m N satisfies δ ω,s δ with high probability if m Cδ 2 s log 4 (N). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 37

61 Comparison of error bounds Error bound for reconstruction of f A ω,p from m Cs ln 3 (s) ln(n) samples drawn i.i.d. at random from the measure ν(dθ, dφ) = tan(θ) 1/3 via weighted l 1 -minimization: f f f f ω,1 Cs 1 1/p f ω,p, 0 < p < 1. Compare to error estimate for unweighted l 1 -minimization: If m CN 1/6 s ln 3 (s) ln(n) then f f 1 Cs 1 1/p f p, 0 < p < 1. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 38

62 Numerical Experiments for Sparse Spherical Harmonic Recovery Original function unweighted l 1 ω k,l = k 1/6 ω k,l = k 1/ Original function f (θ, φ) = 1 θ 2 +1/10 Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 39

63 High dimensional function interpolation Tensorized Chebyshev polynomials on D = [ 1, 1] d C k (t) = C k1 (t 1 )C k2 (t 2 ) C kd (t d ), k N d 0 with C k the L 2 -normalized Chebyshev polynomials on [ 1, 1]. Then 1 2 d C k (t)c j (t)dt = δ k,j, j, k N d 0. [ 1,1] d Expansions f (t) = k N d 0 x kc k (t) with x p < for 0 < p < 1 and large d (even d = ) appear in parametric PDE s (Cohen, DeVore, Schwab 2011,...). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 40

64 (Weighted) sparse recovery for tensorized Chebyshev polynomials L -bound: C k = 2 k 0/2. Curse of dimension: Classical RIP bound requires m C2 d s ln 3 (s) ln(n). Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 41

65 (Weighted) sparse recovery for tensorized Chebyshev polynomials L -bound: C k = 2 k 0/2. Curse of dimension: Classical RIP bound requires m C2 d s ln 3 (s) ln(n). Weights: ω j = 2 k 0/2. Weighted RIP bound: m Cs ln 3 (s) ln(n) Approximate recovery requires x l ω,p! Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 41

66 Comparison Classical Interpolation vs. Weighted l 1 -minimization Classical bound f f Cm r/d f C r Interpolation via l 1 -minimization ( ) f f m 1 1/p C ln 4 f ω,p, 0 < p < 1. (m) Better rate if 1/p 1 > r/d, i.e., p < 1 r/d + 1. For instance, when r = d, then p < 1/2 sufficient. Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 42

67 Avertisement S. Foucart, H. Rauhut, A Mathematical Introduction to Compressive Sensing Applied and Numerical Harmonic Analysis, Birkhäuser, Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 43

68 Rupert Lasser, All the best for your retirement! Holger Rauhut, RWTH Aachen University Interpolation via Weighted l 1 -minimization 44