On Fourier and Wavelets: Representation, Approximation and Compression

Transcription

1 On Fourier and Wavelets: Representation, Approximation and Compression Martin Vetterli EPFL & UC Berkeley MTNS 06, Kyoto, July 25, 2006 Audiovisual Communications Laboratory

2 Acknowledgements Collaborations: Sponsors: NSF Switzerland T.Blu, EPFL M.Do, UIUC P.L.Dragotti, Imperial College P.Marziliano, NIT Singapore I.Maravic, EPFL R.Shukla, EPFL C.Weidmann, TRC Vienna Discussions and Interactions: A.Cohen, Paris VI I. Daubechies, Princeton R.DeVore, Carolina D. Donoho, Stanford M.Gastpar, Berkeley V.Goyal, MIT J. Kovacevic, CMU S. Mallat, Polytech. & NYU M.Unser, EPFL MTNS06-2

3 Outline 1. Introduction through History 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-3

4 Outline 1. Introduction through History From Rainbows to Spectras Signal Representations Approximations Compression 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-4

5 From Rainbows to Spectras Von Freiberg, 1304: Primary and secondary rainbow Newton and Goethe MTNS06-5

6 Signal Representations (1/2) 1807: Fourier upsets the French Academy... Fourier Series: Harmonic series, frequency changes, f 0, 2f 0, 3f 0,... But : Gibbs paper 1899: Gibbs correction Orthogonality, convergence, complexity MTNS06-6

7 Signal Representations (2/2) 1910: Alfred Haar discovers the Haar wavelet dual to the Fourier construction Haar series: Scale changes S 0, 2S 0, 4S 0, 8S 0... orthogonality MTNS06-7

8 Theorem 1 (Shannon-48, Whittaker-35, Nyquist-28, Gabor-46) If a function f(t) contains no frequencies higher than W cps, it is completely determined by giving its ordinates at a series of points spaced 1/(2W) seconds apart. [if approx. T long, W wide, 2TW numbers specify the function] It is a representation theorem:, is an orthogonal basis for BL f(t) in BL can be written as slow! Note: Shannon-BW, BL sufficient, not necessary. many variations, non-uniform etc Kotelnikov-33! MTNS06-8

9 Representations, Bases and Frames Ingredients: as set of vectors, or atoms, an inner product, e.g. a series expansion Many possibilities: orthonormal bases (e.g. Fourier series, wavelet series) biorthogonal bases overcomplete systems or frames Note: no transforms, uncountable MTNS06-9

10 Approximations, aproximation The linear approximation method Given an orthonormal basis for a space S and a signal the best linear approximation is given by the projection onto a fixed sub-space of size M (independent of f!) The error (MSE) is thus Ex: Truncated Fourier series project onto first M vectors corresponding to largest expected inner products, typically LP MTNS06-10

11 The Karhunen-Loeve Transform: The Linear View (1/2) Best Linear Approximation in an MSE sense: Vector processes., i.i.d.: Consider linear approximation in a basis Then: Karhunen-Loeve transform (KLT): For 0<M<N, the expected squared error is minimized for the basis {g n } where g m are the eigenvectors of R x ordered in order of decreasing eigenvalues. Proof: eigenvector argument inductively. Note: Karhunen-47, Loeve-48, Hotelling-33, PCA, KramerM-56, TC MTNS06-11

12 The Karhunen-Loeve Transform: The Linear View (2/2) Geometric intuition: Principal axes of distribution: Shapes: ellipsoids To first approximation, keep all coefficients above a threshold: This can be used in many settings, classification, denoising and compression (inverse waterfilling thm) MTNS06-12

13 Compression: How many bits for Mona Lisa? MTNS06-13

14 A few numbers D. Gabor, September 1959 (Editorial IRE) "... the 20 bits per second which, the psychologists assure us, the human eye is capable of taking in,... Index all pictures ever taken in the history of mankind Huffman code Mona Lisa index A few bits (Lena Y/N?, Mona Lisa ), what about R(D). Search the Web! billion images online, or bits JPEG 186K There is plenty of room at the bottom! JPEG2000 takes a few less, thanks to wavelets Note: 2 (256x256x8) possible images (D.Field) Homework in Cover-Thomas, Kolmogorov, MDL, Occam etc (from a contemporary: 0 bits, I don t care for this modern stuff) MTNS06-14

15 Source Coding: some background Exchanging description complexity for distortion: rate-distortion theory [Shannon:58, Berger:71] known in few cases...like i.i.d. Gaussians (but tight: no better way!) or -6dB/bit typically: difficult, simple models, high complexity (e.g. VQ) high rate results, low rate often unknown MTNS06-15

16 Limitations of the Standard Models Splendeurs et misères de la fonction débit-distortion (after Balsac) Precise results beautiful (maybe too much for its own good) upper and lower bounds constructive Problems complexity: exponential in code length code construction: finding good codes is hard Paradox: Best codes used in practice are suboptimal (Effros) transform codes dominate the scene of real compression So: unlike in lossless compression, lossy compression uses IT in a limited way;) Audio/Image/Video: distortion measures? MTNS06-16

17 New image coding standard JPEG 2000 Old versus new JPEG: D(R) on log scale Main points: improvement by a few db s lot more functionalities (e.g. progressive download on internet) at high rate ~ -6db per bit: KLT behavior low rate behavior: much steeper: NL approximation effect? is this the limit? MTNS06-17

18 New image coding standard JPEG 2000 From the comparison, JPEG fails above 40:1 compression JPEG2000 survives Note: images courtesy of MTNS06-18

19 The Swiss Army Knife Formula of Transform Coding [Goyal00] Model: iid vector process of size N,, R x, MSE, high rate vector quantizer, entropy code transform and scalar quantizers, entropy code Trace min: ortho; diff. entropy min: independence Gaussian case: coincide! But in general not MTNS06-19

20 Representation, Approximation and Compression: Why does it matter anyway? Parsimonious or sparse representation of visual information is key in storage and transmission indexing, searching, classification, watermarking denoising, enhancing, resolution change But: it is also a fundamental question in information theory signal/image processing approximation theory vision research Successes of wavelets in image processing: compression (JPEG2000) denoising enhancement classification Thesis: Wavelet models play an important role Antithesis: Wavelets are just another fad! MTNS06-20

21 Interaction of topics AT: deterministic setting, large classes of fcts HA: function classes, existence, embeddings IT: boundings, converses, stochastic setting SP: bases, algorithms, complexity The interaction is the fun! MTNS06-21

22 Outline 1. Introduction through History 2. Fourier and Wavelet Representations Fourier and Local Fourier Transforms Wavelet Transforms Piecewise Smooth Signal Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-22

23 Fourier and Wavelet Representations: Spaces Norms: Hilbert spaces: Inner product: Orthogonality: Banach spaces: C P spaces: p-times diff. with bounded derivatives -> Taylor expansions Holder/Lipschitz : locally smooth (non-integer) MTNS06-23

24 Example consider and p < 1: quasi norm, p -> 0: sparsity measure MTNS06-24

25 More Spaces Sobolev Spaces W S (R) If then f is n-times continuously differentiable Equivalently decays at Besov Spaces with respect to a basis (typically wavelets) or wavelet expansion has finite norm MTNS06-25

26 A Tale of Two Representations: Fourier versus Wavelets Orthonormal Series Expansion Time-Frequency Analysis and Uncertainty Principle Then Not arbitrarily sharp in time and frequency! MTNS06-26

27 Local Fourier Basis? The Gabor or Short-time Fourier Transform Time-frequency atoms localized at When small enough Example: Spectogram MTNS06-27

28 The Bad News Balian-Low Theorem then either is a short-time Fourier frame with critical sampling or: there is no good local orthogonal Fourier basis! Example of a basis: block based Fourier series Note: consequence of BL Thm on OFDM, RIAA MTNS06-28

29 Example of Gabor frames Pretty bad MTNS06-29

30 The Good News! There exist good local cosine bases. Replace complex modulation by appropriate cosine modulation where w(t) is a power complementary window Result: MP3! Many generalisations MTNS06-30

31 Example of time-frequency tiling, state of the art audio coder In this example, it switches from 1024 channels down to 128, makes for pretty crisp attacks! It also makes the RIAA nervous. MTNS06-31

32 Another Good News! Replace (shift, modulation) by (shift, scale) or then there exist good localized orthonormal bases, or wavelet bases MTNS06-32

33 Examples of bases Haar Daubechies, D 2 MTNS06-33

34 Wavelets and representation of piecewise smooth functions Goal: efficient representation of signals like where: Wavelet act as singularity detectors Scaling functions catch smooth parts Noise is circularly symmetric Note: Fourier gets all Gibbs-ed up! MTNS06-34

35 Key characteristics of wavelets and scaling functions (1/3) Daubechies-88, Wavelets derived from filter banks, ortho-lp with N zeros at, Scaling function: Orthonormal wavelet family: Scaling function and approximations Scaling function spans polynomials up to degree N-1 Strang-Fix theorem: if has N zeros at multiples of (but the origin), then spans polynomials up to degree N-1 Two scale equation: Smoothness: follows from N, MTNS06-35

36 Key characteristics of wavelets and scaling functions (2/3) Lowpass filters and scaling functions reproduce polynomials Iterate of Daubechies L=4 lowpass filter reproduces linear ramp scaling function linear ramp Scaling functions catch trends in signals MTNS06-36

37 Key characteristics of wavelets and scaling functions (3/3) Wavelet approximations wavelet has N zeros moments, kills polynomials up to degree N-1 wavelet of length L=2N-1, or 2N-1 coeffs influenced by singularity at each scale, wavelet are singularity detectors, wavelet coefficients of smooth functions decays fast, e.g. f in c P, m << 0 Note: all this is in 1 dimension only, 2D is another story MTNS06-37

38 How about singularities? If we have a singularity of order n at the origin (0: Dirac, 1: Heaviside, ), the CWT transform behaves as In the orthogonal wavelet series: same behavior, but only L=2N-1 coefficients influenced at each scale! E.g. Dirac/Heaviside: behavior as and Wavelets catch and characterize singularities! MTNS06-38

39 Thus: a piecewise smooth signal expands as: phase changes randomize signs, but not decay a singularity influence only L wavelets at each scale wavelet coefficients decay fast MTNS06-39

40 Outline 1. Introduction through History 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory Non-linear approximation Fourier versus wavelet, LA versus NLA 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-40

41 From linear to non-linear approximation theory The non-linear approximation method Given an orthonormal basis for a space S and a signal the best nonlinear approximation is given by the projection onto an adapted subspace of size M (dependent on f!) set of M largest The error (MSE) is thus and Difference: take the first M coeffs (linear) or take the largest M coeffs (non-linear) MTNS06-41

42 From linear to non-linear approximation theory Nonlinear approximation This is a simple but nonlinear scheme Clearly, if is the NL approximation scheme: This could be called adaptive subspace fitting From a compression point of view, you pay for the adaptivity in general, this will cost bits These cannot be spent on coefficient representation anymore LA: pick a subspace a priori NLA: pick after seeing the data MTNS06-42

43 Non-Linear Approximation Example Nonlinear approximation power depends on basis Example: Two different bases for Fourier series Wavelet series: Haar wavelets Linear approximation in Fourier or wavelet bases Nonlinear approximation in a Fourier basis Nonlinear approximation in a wavelet basis MTNS06-43

44 Non-linear Approximation Example Fourier basis: N=1024, M=64, linear versus nonlinear D= 2.7 D= 2.4 Nonlinear approximation is not necessarily much better! MTNS06-44

45 Non-linear Approximation Example Wavelet basis: N=1024, M=64, J=6, linear versus nonlinear D= 3.5 D= 0.01 Nonlinear approximation is vastly superior! MTNS06-45

46 Nonlinear approximation theory and wavelets Approximation results for piecewise smooth fcts between discontinuities, behavior by Sobolev or Besov regularity k derivatives coeffs when Besov spaces can be defined with wavelets bases. If then [DeVoreJL92]: MTNS06-46

47 Approximation in Sobolev and Besov Spaces Linear Approximation, W S [0,N] Sobolev-s: uniformly smooth Fourier: Wavelets: Non-Linear Approximation Besov-s: smooth between a finite # of discontinuities Fourier: does not work, Wavelets: approximation power given by the smoothness! Key: effect of discontinuities limited, because wavelets are concentrated around discontinuities f(t) in W S (0,N) between finite # of discontinuities, then f(t) in B p (0,N) (wavelet of compact support) Then: Result can be refined to get MTNS06-47

48 Smooth versus piecewise smooth functions: It depends on the basis and on the approximation method s=2, N=2^16, D_3, 6 levels MTNS06-48

49 Limit of Besov Spaces ;) Randomly change sign of wavelet coeffs (conserves Besov norm) seems to work in 1D, but not in 2D! MTNS06-49

50 Outline 1. Introduction through History 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression A small but instructive example Piecewise polynomials and D(R) Piecewise smooth and D(R) Improved wavelet schemes 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-50

51 Wavelets and Compression Compression is just one bit trickier than approximation A small but instructive example: Assume, signal is of length N, k is U[0, N-1] and is This is a Gaussian RV at location k Note: R x = l! Linear approximation: Non-linear approximation, M > 0: MTNS06-51

52 Given budget R for block of size N: 1. Linear approximation and KLT: equal distribution of R/N bits This is the optimal linear approximation and compression! 2. Rate-distortion analysis [Weidmann:99] High rate cases: Obvious scheme: pointer + quantizer This is the R(D) behavior for R >> Log N Much better than linear approximation Low rate case: Hamming case solved, inc. multiple spikes: - there is a linear decay at low rates L2 case: upper bounds that beat linear approx. MTNS06-52

53 Given budget R for block of size N: Example 1: Binary, Hamming, 1 and k spikes Example 2: Bernoulli-Gaussian MTNS06-53

54 But is NLA always better than LA for compression? Counter example: R bits for a jointly gaussian vector source of size N i.i.d. N(0,1) Optimal scheme is LA performance is NLA incurs penalty need log 2 (N K) bits to index significant coeffs, which is O(N) when K is O(N/2) or worst case performance is but c 1, c 2 depends on coding, density, etc Thus, we could loose several db s by going to NLA but this is not the operating range where NLA is interesting! MTNS06-54

55 Piecewise smooth functions: pieces are Lipschitz- The following D(R) behavior is reachable [CohenDGO:02]: There are 2 modes: corresponding to the Lipschitz- pieces corresponding to the discontinuities MTNS06-55

56 Lipschitz- pieces: Linear Approximation The wavelet transform at scale j decays as (j << 0) Keep coefficients up to scale J, or choose a stepsize for a quantizer Therefore, M ~ 2 J coefficients Squared error: Rate: Number of coefficients Thus Just as good as Fourier, but local! MTNS06-56

57 Rate-distortion bounds for piecewise polynomial functions D(R) behavior of nonlinear approximation with wavelets Consider the simplest case: Haar! Recall that and consider describing the significant coefficients Thus Choose a stepwise for a quantizer. Therefore - number of scales J before coefs set to zero - number of bits per coefficient, thus Distortion: number of scales times MTNS06-57

58 Rate-distortion behavior using an oracle An oracle decides to optimally code a piecewise polynomial by allocating bits where needed : Consider the simplest case Two approximations errors : quantization of step location : quantization of amplitude Rate allocation: versus Result: MTNS06-58

59 Piecewise polynomial, with max degree N A. Nonlinear approximation with wavelets having zero moments B. Oracle-based method Thus wavelets are a generic but suboptimal scheme oracle method asymptotically superior but dependent on the model Conclusion on compression of piecewise smooth functions: D(R) behavior has two modes: 1/polynomial decay: cannot be (substantially) improved exponential mode: can be improved, important at low rates MTNS06-59

60 Can we improve wavelet compression? Key: Remove depencies across scales: dynamic programming: Viterbi-like algorithm tree based algorithms: pruning and joining wavelet footprints: wavelet vector quantization Theorem [DragottiV:03]: Consider a piecewise smooth signal f(t), where pieces are Lipschitz-. There exists a piecewise polynomial p(t) with pieces of maximum degree such that the residual is uniformly Lipschitz-. This is a generic split into piecewise polynomial and smooth residual MTNS06-60

61 Footprint Basis and Frames Suboptimality of wavelets for piecewise polynomials is due to independent coding of dependent wavelet coefficients Compression with wavelet footprints Theorem: [DragottiV:03] Given a bounded piecewise polynomial of deg D with K discontinuities. Then, a footprint based coder achieves This is a computational effective method to get oracle performance What is more, the generic split piecewise smooth into uniformly smooth + piecewise polynomial allows to fix wavelet scenarios, to obtain This can be used for denoising and superresolution MTNS06-61

62 Denoising (use coherence across scale) This is a vector thresholding method adapted to wavelet singularities MTNS06-62

63 Outline 1. Introduction through History 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions The need for truly two-dimensional constructions Tree based methods Non-separable bases and frames 6. Beyond Shift Invariant Subspaces 7. Conclusions and Outlook MTNS06-63

64 Going to Two Dimensions: Non-Separable Constructions Going to two dimensions requires non-separable bases Objects in two dimensions we are interested in textures: per pixel smooth surfaces: per object! MTNS06-64

65 Models of the world: Gauss-Markov Piecewise polynomial the usual suspect Many proposed models: mathematical difficulties one size fits all reality check Lena is not PC, but is she BV? MTNS06-65

66 Current approaches to two dimensions. Mostly separable, direct or tensor products DWT Fourier and wavelets are both direct product constructions Wavelets: good for point singularities but what is needed are sparse coding of edge singularities! 1D: singularity 0-dimensional (e.g. spike, discontinuity) 2D: singularity 1-dimensional (e.g. smooth curve) MTNS06-66

67 What the natural world and perception tells us Olshausen BA, Field DJ, Emergence of Simple-Cell Receptive Field Properties by Learning a Sparse Code for Natural Images, Nature, method: statistical analysis of natural scenes to learn sparse codes sparse coding involves directional analysis and multiple resolutions edges are key elements Note: out of the 2 (256x256x8) possible images, very, very few are actually natural images! MTNS06-67

68 Recent work on geometric image processing Long history: compression, vision, filter banks Current affairs: Signal adapted schemes Bandelets [LePennec & Mallat]: wavelet expansions centered at discontinuity as well as along smooth edges Non-linear tilings [Cohen, Mattei]: adaptive segmentation Tree structured approaches [Shukla et al, Baraniuk et al] Bases and Frames Wedgelets [Donoho]: Basic element is a wedge Ridgelets [Candes, Donoho]: Basic element is a ridge Curvelets [Candes, Donoho] Scaling law: width ~length 2 L(R 2 ) set up Multidirectional pyramids and contourlets [Do et al] Discrete-space set-up, l(z 2 ) Tight frame with small redundancy Computational framework MTNS06-68

69 Nonseparable schemes and approximation Approximation properties: wavelets good for point singularities ridgelets good for ridges curvelets good for curves Consider c 2 boundary between two csts Rate of approximation, M-term NLA in bases, c 2 boundary Fourier: O(M -1/2 ) Wavelets: O(M -1 ) Curvelets: O(M -2 ) Note: adaptive schemes, Bandelets: O(M - ) MTNS06-69

70 Compression of non-separable objects Object we know how to compress. Basis element Approximation Wavelets Ridgelets Rate/distortion Oracle Wavelets.poor Ridgelets.suboptimal adaptive schemes: close to oracle fixed basis: under investigation MTNS06-70

71 Tree Based Geometric Compression [ShuklaDDV[ ShuklaDDV:03] Idea tree and quadtree algorithms popular, many pruning algorithms optimality proofs for wedgelets [Donoho:99] new pruning and joining algorithm Intuition: full tree dyadic tree pruned & joined tree Results: Rate-distortion optimal for piecewise polynomials that is, like an oracle method (up to constants) MTNS06-71

72 Extension to Quadtree: Example Results: consider a piecewise polynomial 2D signal, with polynomial boundaries, the following rate-distortion behavior is achieved this is like an oracle method, and >> than prune algorithms which have a penalty complexity: polynomial MTNS06-72

73 The prune-join quadtree algorithm polynomial fit to surface and to boundary on a quadtree rate-distortion optimal tree pruning and joining quadtree with R(D) pruning R(D) Joining of similar leaves Note: careful R(D) optimization! MTNS06-73

74 Geometric Compression versus JPEG2000 at 0.11 bits/pixel, PSNR: Pruned-joined quadtree JPEG2000 MTNS06-74

75 Upperbounds on R-D Behaviors A. Boundary is piecewise polynomial: Oracle: The prune quadtree algorithm (independent coding): The prune-join quadtree algorithm with joint coding: B. Boundary is piecewise smooth: The prune quadtree algorithm (code blocks independently) achieves the oracle performance (up to log factor): C. Computational complexity of quadtree algorithms: MTNS06-75

76 Behavior of tree algorithms on piecewise smooth fcts ppf: piecewise polynomial functions psf: piecewise smooth functions Signal Class Oracle Coder Wavelet Coder Prune tree Coder Prune-join tree Coder 1-D PPF 2-D PPF 1-D PSF 2-D PSF At most log penalty with polynomial complexity (and a bit more work gets rid of logs ) MTNS06-76

77 Multiresolution directional filterbanks and contourlets [M.Do] Idea: find a direct discrete-space construction that has good approximation properties for smooth functions with smooth boundaries directional analysis as in a Radon transform multiresolution as in wavelets and pyramids computationally easy bases or low redundancy frame Background: curvelets [Candes-Donoho] indicate that good fixed bases do exist for approximation of piecewise smooth 2D functions a frequency-direction relationship indicates a scaling law an effective compression algorithm requires - close to a basis (e.g. tight frame with low redundancy) - discrete-space set up and computationally efficient Question: can we go from l(z 2 ) to L(R 2 ), just like filter banks lead to wavelets? Answer: contourlets! MTNS06-77

78 Directional Filter Banks [BambergerS[ BambergerS:92, DoV:02] divide 2-D spectrum into slices with iterated tree-structured f-banks fan filters quincunx sampling shearing MTNS06-78

79 Iterated directional filter banks: efficient directional analysis Example: MTNS06-79

80 Example of basis functions 6 levels of iteration, or 64 channels elementary filters are Haar filters orthonormal directional basis 64 equivalent filters, below the 32 mostly horizontal ones are shown This ressembles a local Radon transform, or radonlets! changes of sign (for orthonormality) approximate lines (discretizations) MTNS06-80

81 Adding multiresolution: : use a pyramid! Result: tight pyramid and orthogonal directional channels => tight frame low redundancy < 4/3, computationally efficient MTNS06-81

82 Frequency split flexible tilings: d ~ j 1/2 or width ~ length 2 usual dyadic tiling It just becomes a MD filter design problem Equivalent basis functions: Contourlets MTNS06-82

83 A directional multiresolution analysis Theorem [Do:01] For a finite number of directions, this generates a tight frame for L 2 (R 2 ) with frame bound equal 1 Method: Define embedded lowpass directional spaces V j,k and directional bandpass spaces W j,k This defines contourlets: how do they compare to wavelets? MTNS06-83

84 Basis functions: Wavelets versus contourlets Wavelets Contourlets MTNS06-84

85 Pyramidal directional filter bank expansion: Example Pepper image and its expansion Compression, denoising, inverse problems: if it is sparse, it is going to work! MTNS06-85

86 Approximation properties Wavelets Contourlets MTNS06-86

87 An approximation theorem Curvelets lead to optimal approximation, what about contourlets? Result [M.Do:03] Simple B/W image model with c 2 boundary Contourlet with scaling w ~ l 2 and 1 directional vanishing moment Then the M-term NLA satisfies Proof (very sketchy...): - Amplitude of contourlets ~ 2-3j/4 and coeffs ~2-3j/4 l 3 jkn - Three types of coefficients (significant which match direction insignificant that overlap, and zero) - levels 3J and J, respectively, leading to M ~ 2 3J/2 - squared error can be shown to be ~ 2-3J MTNS06-87

88 Example: denoising with contourlets original noisy wavelet 13.8 db countourlets 15.4 db MTNS06-88

89 Outline 1. Introduction through History 2. Fourier and Wavelet Representations 3. Wavelets and Approximation Theory 4. Wavelets and Compression 5. Going to Two Dimensions: Non-Separable Constructions 6. Beyond Shift Invariant Subspaces: Finite Rate of Innovation Shift-Invariance and Multiresolution Analysis A Variation on a Theme by Shannon A Representation Theorem 7. Conclusions and Outlook MTNS06-89

90 Shift-Invariance and Multiresolution Analysis Most sampling results require shift-invariant subspaces Wavelet constructions rely in addition on scale-invariance Multiresolution analysis (Mallat, Meyer) gives a powerful framework. Yet it requires a subspace structure. Example: uniform or B-splines Question: can sampling be generalized beyond subspaces? Note: Shannon BW sufficient, not necessary MTNS06-90

91 A Variation on a Theme by Shannon Shannon, BL case: or 1/T degrees of freedom per unit of time But: a single discontinuity, and no more sampling theorem... Are there other signals with finite number of degrees of freedom per unit of time that allow exact sampling results? => Finite rate of innovation Usual setup: x(t): signal, h(t): sampling kernel, y(t): filtering of x(t) and y n : samples MTNS06-91

92 A Toy Example K Diracs on the interval: 2K degrees of freedom. Periodic case: Key: The Fourier series is a weighted sum of K exponentials Result: Taking 2k+1 samples from a lowpass version of BW-(2K+1) allows to perfectly recover x(t) Method: Yule-Walker system, annihilating filter, Vandermonde system Note: Relation to spectral estimation and ECC (Berlekamp-Massey) MTNS06-92

93 A Representation Theorem [VMB:02] For the class of periodic FRI signals which includes sequences of Diracs non-uniform or free knot splines piecewise polynomials there exist sampling schemes with a sampling rate of the order of the rate of innovation which allow perfect reconstruction at polynomial complexity Variations: finite length, 2D, local kernels etc MTNS06-93

94 And the noisy case. Use subspace methods (I.Maravic) Application example: UWB MTNS06-94

95 A local algorithm for FRI sampling The return of Strang-Fix! local, polynomial complexity reconstruction, for diracs and piecewise polynomials MTNS06-95

96 Conclusions Wavelets and the French revolution too early to say? from smooth to piecewise smooth functions Sparsity and the Art of Motorcycle Maintenance sparsity as a key feature with many applications denoising, inverse problems, compression LA versus NLA: approximation rates can be vastly different! To first order, operational, high rate, D(R) improvements still possible low rate analysis difficult Two-dimensions: really harder! and none used in JPEG approximation starts to be understood, compression mostly open Beyond subspaces: FRI results on sampling, many open questions! MTNS06-96

97 Outlook Do we understand image representation/compression better? high rate, high resolution: there is promise low rate: room at the bottom? New images plenoptic functions (set of all possible images) non BL images (FRI?) manifolds, structure of natural images Distributed images interactive approximation/compression SW, WZ, DKLT... MTNS06-97

98 Why Image Representation Remains a Fascinating Topic MTNS06-98

99 Publications For overviews: D.Donoho, M.Vetterli, R.DeVore and I.Daubechies, Data Compression and Harmonic Analysis, IEEE Tr. on IT, Oct M. Vetterli, Wavelets, approximation and compression, IEEE Signal Processing Magazine, Sept Coming up: M.Vetterli, J.Kovacevic and V.Goyal, Fourier and Wavelets: Theory, Algorithms and Applications, 200X ;) For more details, Theses C.Weidmann, Oligoquantization in low-rate lossy source coding, PhD Thesis, EPFL, M. N. Do, Directional Multiresolution Image Representations, Ph.D. Thesis, EPFL, P. L. Dragotti, Wavelet Footprints and Frames for Signal Processing and Communications, PhD Thesis, EPFL, R.Shukla, Rate-distortion optimized geometrical image processing, PhD Thesis, EPFL, MTNS06-99

100 Papers: A.Cohen, I.Daubechies, O.Gulieruz and M.Orchard, On the importance of combining wavelet-based non-linear approximation with coding strategies, IEEE Tr. on IT, 2002 P. L. Dragotti, M. Vetterli. Wavelets footprints: theory, algorithms and applications, IEEE Transactions on Signal Processing, May R. Shukla, P. L. Dragotti, M. N. Do and M. Vetterli, Rate-distortion optimized tree structured compression algorithms for piecewise smooth images, IEEE Transactions Image Processing, M. N. Do and M. Vetterli, Framing pyramids. IEEE Transactions on Signal Processing, Sept M. N. Do and M. Vetterli, Contourlets. in Beyond Wavelets, J. Stoeckler and G. V. Welland eds., Academic Press, M.N.Do and M.Vetterli, Contourlets: A computational framework for directional multiresolution image representation, submitted, C.Weidmann, M.Vetterli, Rate-distortion behavior of sparse sources, IEEE Tr. on IT, under revision. M. Vetterli, P. Marziliano and T. Blu, Sampling signals with finite rate of innovation, IEEE Transactions on SP, June I. Maravic and M. Vetterli, Sampling and Reconstruction of Signals with Finite Rate of Innovation in the Presence of Noise, IEEE Transactions on SP, 2004, submitted. MTNS06-100