Computational Harmonic Analysis (Wavelet Tutorial) Part II

Transcription

1 Computational Harmonic Analysis (Wavelet Tutorial) Part II Understanding Many Particle Systems with Machine Learning Tutorials Matthew Hirn Michigan State University Department of Computational Mathematics, Science & Engineering Department of Mathematics

2 Wavelet Transform

3 Wavelets Wavelet 2 L 2 (R) satisfies: Zero average: R = Normalized: k k 2 =1 Centered around t = Localized in time and frequency Can be either real or complex valued

4 Wavelet Transform Wavelet dictionary obtained by scaling and translating : D = { u,s } u2r,s2r +, u,s (t) = 1 p s t u s Wavelet transform: Wf(u, s) =hf, u,s i where Z +1 = f(t)s 1/2 (s 1 (t u)) dt 1 = f s(u) s(t) =s 1/2 (s 1 t) log 2 (s) f(t) Fig A Wavelet Tour of Signal Processing, 3 rd ed. Real wavelet transform Wf(u, s) computed with a Mexican hat wavelet The vertical axis represents log 2 s. Black, grey and white points correspond respectively to positive, zero and negative wavelet coe cients. t u Note: b s(!) = p s ˆ(s!) Thus, since: \ f s(!) = ˆf(!) b s(!) the wavelet transform Wf(u, s) captures.5 the frequency information of f organized by the frequency bands of s Fig A Wavelet Tour of Signal Processing, 3 rd ed. Scaled Fourier transforms ˆ(2 j!) 2, for 1 6 j 6 5 and! 2 [, ].

5 Real Wavelet Reconstruction Theorem (Calderón, Grossman and Morlet): Let 2 L 2 (R) be a real function such that f(t) = 1 C kfk 2 2 = 1 C C = Z +1 ˆ(!) 2! Then, for any f 2 L 2 (R): Z +1 Z +1 Z +1 Z +1 d! < +1 s 2 1 Wf(u, s)s 1/2 (s 1 (t u)) du ds 1 Wf(u, s) 2 du ds s 2. C < 1 is called the wavelet admissibility condition. C < +1 ) ˆ() =. su cient. This is almost If additionally, ˆ 2 C 1, then C < +1. Can insure this with su cient time decay: (t) apple K 1+ t 2+

6 Scaling Function Numerically the wavelet transform is only computed up to scales s<s, which loses the low frequency information of f captures this infor- The scaling function mation. Defined by: Denote: ˆ(!) 2 = s(t) = 1 p s t s Z +1 1 ˆ(s!) 2 ds s and s(t) = s ( t) A Wavelet Tour of Signal Processing, 3 rd ed. Mexican hat wavelet for = 1 and its Fourier Fig transform. A Wavelet Tour of Signal Processing, 3 rd ed. Scaling function associated to a Mexican hat wavelet and its Fourier transform. ˆ ˆ The low frequency approximation of f at scale s is: Af(u, s) =hf, Reconstruction still holds: f(t) = 1 C Z s u,si = f s(u) Wf(,s) s(t) ds s 2+ 1 C s Af(,s ) s (t)

7 Analytic Wavelets Complex valued, analytic wavelets admit a time-frequency analysis, like the windowed Fourier transform. The wavelet is analytic if: 8! <, ˆ(!) = The wavelet transform Wf(u, s) of an analytic wavelet satisfies very similar reconstruction and energy preservation formulas as the real wavelet transform.

8 Analytic Wavelet Construction Let g be a real, symmetric window. ^ ψ(ω) Define a wavelet as: (t) =g(t)e i t ) ˆ(!) =ĝ(! ) Thus if ĝ(!) =for! >, then ˆ(!) = for! <, and is analytic. ^ ω g( ) ω η Fig A Wavelet Tour of Signal Processing, 3 rd ed. Fourier transform ˆ(!) of a wavelet (t) =g(t) exp(i t). is centered in time at t = and in frequency at! =. Gabor wavelets use a Gaussian window, and so are not strictly analytic and do not have precisely zero average. However ˆ(!) for! apple. Morlet wavelets also use a Gaussian window, but subtract a constant in order to have zero average: (t) =g(t)(e i t C)

9 Analytic Wavelet Heisenberg Boxes ω ψ ^ (ω) u,s Suppose is centered at t = with central frequency! =. The time variance 2 t and frequency variance! 2 of are: η s s σ t σ ω s 2 t = Z +1 2! = t2 (t) 2 dt Z +1 (! ) 2 ˆ(!) 2 d! η s ψ ^ u,s(ω) ψ u,s ψu,s s σ t u u Fig A Wavelet Tour of Signal Processing, 3 rd ed. Heisenberg boxes of two wavelets. Smaller scales decrease the time spread but increase the frequency support, which is shifted towards higher frequencies. σ ω s t Scalogram: P W f(u, /s) = Wf(u, s) 2

10 Time-Frequency Plane: Wavelets vs. Windowed Fourier Comparison of time-frequency tilings: Windowed Fourier Transform Wavelet Transform

11 Hyperbolic Chirp Revisited 1 1 f(t) ξ / 2π ξ / 2π t u f(t) =a 1 cos 1 1 t + a2 cos 2 2 t Spectrogram P S f(u, ) of windowed Fourier transform u u 1 Wavelet Tour of Signal Processing, 3 rd ed. Sum of two hyperbolic chirps. (a): Spectrogram P S f(u, ). (b): Ridge support calculated from the spectrogram Scalogram P W f(u, /s) of analytic wavelet transform

12 Hyperbolic Chirp Revisited 1 1 f(t) ξ / 2π ξ / 2π t u u f(t) =a 1 cos 1 1 t + a2 cos 2 2 t Local maxima of spectrogram P S f(u, ) Local maxima of scalogram P W f(u, /s) Wavelet Tour of Signal Processing, 3 rd ed. Sum of two hyperbolic chirps. (a): Spectrogram P S f(u, ). (b): Ridge support calculated from the spectrogram

13 Parallel Linear Chirps f(t) ξ / 2π t f(t) =a 1 cos(bt 2 + ct)+a 2 cos(bt 2 ) ξ / 2π u u ξ / 2π 5 ξ / 2π velet Tour of Signal Processing, 3 rd ed. Sum of two parallel linear chirps. (a): Spectrogram P S f(u, ) = Sf(u, ) 2. (b): Ridge support calculated from the spectrogram. Spectrogram: P S f(u, ) u Fig A Wavelet Tour of Signal Processing, 3 rd ed. (a): Normalized scalogram 1 P W f(u, ) of two parallel linear chirps. (b): Wavelet ridges. Scalogram: P W f(u, /s) u

14 Sparsity and Time- Frequency Resolution Lesson: Best transform depends on the signal f time-frequency properties. A transform that is adapted to the signal time-frequency property has fewer local maxima, and is thus sparser. Transforms that are not adapted to the signal di use the signal s energy over many atoms, leading to more local maxima and a less sparse representation. Thus sparsity is a natural criterion to guide the construction of time-frequency transforms.

15 Wavelet Zoom f(t) log 2 (s) 6 t 4 2 u Fig A Wavelet Tour of Signal Processing, 3 rd ed. Real wavelet transform Wf(u, s) computed with a Mexican hat wavelet The vertical axis represents log 2 s. Black, grey and white points correspond respectively to positive, zero and negative wavelet coe cients.

16 Taylor s Theorem We now turn to measuring the local regularity of f at a point v. Suppose f is m times di erentiable in [v h, v + h]. Let p v be the Taylor polynomial of f in the neighborhood of v: p v (t) = m 1 X k= f (k) (v) (t v) k k! Taylor s Theorem: The residual " v (t) = f(t) p v (t) satisfies8 t 2 [v h, v + h]: " v (t) apple t v m m! sup u2[v h,v+h] f (m) (u)

17 Lipschitz Regularity Lipschitz Regularity: A function f is point wise Lipschitz (Hölder) at v, if there exists K> and a polynomial p v of degree m = b c such that 8 t 2 R, f(t) p v (t) apple K t v f is uniformly Lipschitz over [a, b] if it satisfies the above for all v 2 [a, b] witha K independent of v. Global Lipschitz regularity and the Fourier transform: A function f is bounded and uniformly Lipschitz over R if: Z +1 1 ˆf(!) (1 +! ) d! < +1

18 Wavelet Vanishing Moments A wavelet has n vanishing moments if: 8 apple k<n, Z +1 1 tk (t) dt = Wavelet transform kills polynomials p with deg(p) apple n 1: Wp(u, s) =. Let f be Lipschitz <nat v, so that: f(t) =p v (t)+" v (t) with " v (t) apple K t v Then: Wf(u, s) =W " v (u, s) We are going to measure from Wf(u, s), with u close to v.

19 Multiscale Differential Operator 2 1 f(t) t.2.4 s Fig A Wavelet Tour of Signal Processing, 3 rd ed. Wavelet transform Wf(u, s) calculated with = where is a Gaussian, for the signal f shown above. The position parameter u and the scale s vary respectively along the horizontal and vertical axes. Black, grey and white points correspond respectively to positive, zero and negative wavelet coe cients. Singularities create large amplitude coe cients in their cone of influence. u Wavelet transform Wf(u, s) withwavelet with one vanishing moment Black: positive White: negative Grey: zero Singularities create large amplitude wavelet coe cients Notice that the coe cients give information regarding the derivative of f - this is not an accident!

20 Multiscale Differential Operator Theorem: A wavelet with a fast decay has n vanishing moments if and only if there exists with a fast decay such that: (t) =( As a consequence: 1) ndn (t) dt n with Wf(u, s) =s n dn du n(f s )(u), s (t) =s 1/2 ( t/s) = =

21 Wavelet Zoom on an Interval Let 2 C n (R) have n vanishing moments and derivatives that have fast decay. Theorem: If f 2 L 2 (R) is uniformly Lipschitz apple n over [a, b], then there exists A> such that 8 (u, s) 2 [a, b] R +, Wf(u, s) apple As +1/2 Conversely, suppose f is bounded and Wf(u, s) apple As +1/2 8 (u, s) 2 [a, b] R + for an <n, /2 Z. Then f is uniformly Lipschitz on [a +,b ] for any >.

22 Wavelet Zoom at a Point Let 2 C n (R) have n vanishing moments and derivatives that have fast decay. Theorem (Ja ard): If f 2 L 2 (R) is Lipschitz apple n at v, then there exists A> such that 8 (u, s) 2 R R +, Wf(u, s) apple As +1/2 1+ u v s Conversely, if <n, /2 Z and there exists A> and < such that 8(u, s) 2 R R +, Wf(u, s) apple As +1/2 1+ u v s then f is Lipschitz at v.!

23 Wavelet Modulus Maxima Previous two theorems show that the local Lipschitz regularity of f at v depends on the decay of Wf(u, s) as s!. In fact, we only need to look at the local maxima of Wf(u, s) to detect and characterize singularities of f. Wavelet modulus maximum is a point (u,s ) such that Wf(u, s ) is locally maximum at u = u.

24 Maxima Propagation Wavelet modulus maxima Theorem (Hwang, Mallat): f is singular at a point v only if there is a sequence of wavelet modulus maxima (u p,s p ) that converges to v at fine scales: lim (u p,s p )=(v, ) p!+1 Theorem (Hummel, Poggio, Yuille): If = ( 1) (n) for a Gaussian, then the wavelet modulus maxima belong to connected curves that are not interrupted as s!. { The maximum slope of log 2 Wf(u, s) as a function of log 2 s along the maximum line converging to v is +1/2. Full line: Decay of log 2 Wf(u, s) along maxima line converging to t =.5. Dashed line: t =.42. Maxima line converging to

25 Dyadic Wavelet Transform and Maxima Dyadic wavelet transform: Wf(u, 2 j )=f 2j(u) Wavelet maxima (keeping the sign)

26 Wavelet Maxima Approximation in 1D Analysis f(t) Synthesis Approximation of f(t) with 1% wavelet maxima Approximation of f(t) with 5% wavelet maxima

27 Wavelet Transform and Modulus Maxima in 2D Increasing Scale (a) Wavelet transform in horizontal direction (b) Wavelet transform in vertical direction (c) Wavelet transform modulus (d) Angles (e) Wavelet modulus maxima

28 Wavelet Transform and Modulus Maxima in 2D Increasing Scale (a) Wavelet transform in horizontal direction (b) Wavelet transform in vertical direction (c) Wavelet transform modulus (d) Angles (e) Wavelet modulus maxima (e) Wavelet modulus maxima above a threshold

29 Wavelet Maxima Approximation in 2D (a) Original Image (b) Approximation from 1% wavelet maxima (e) (c) Approximation from thresholded wavelet maxima (f)

30 Dyadic Wavelet Frames

31 Translation Invariant Frames Recall translation invariant dictionary: D = { u, } 2,u2R, u, (t) = (t u) and the (frame) operator: f(u, )=hf, u, i = f (u), (t) = ( t) A translation invariant dictionary is a frame for L 2 (R) if there exists B A> such that for all f 2 L 2 (R), Akfk 2 2 apple X k f(, )k 2 2 apple Bkfk2 2 where k f(, )k 2 2 = Z +1 1 f(u, ) 2 du = Z +1 1 f (u) 2 du When A = B the frame is tight. Frames are redundant.

32 Translation Invariant Frames Theorem: If there exists B that for almost every! 2 R, A> such A apple X ˆ (!) 2 apple B, then D = { u, } 2,u2R, u, (t) = (t u) is a frame for L 2 (R). Define the generators {' } frame via: b' (!) = P ˆ (!) ˆ (!) 2 of the dual We then have the following reconstruction formula: f(t) = X f(, ) ' (t) = X f ' (t)

33 Dyadic Wavelet Frame A translation invariant dyadic wavelet dictionary is defined as: Fig A Wavelet Tour of Signal Processing, 3 rd ed. Scaled Fourier transforms ˆ(2 j!) 2, for 1 6 j 6 5 and! 2 [, ]. D = n u,2 j (t) =2 j (2 j (t u)) o u2r,j2z Dyadic wavelet transform: Wf(u, 2 j )=f 2j(u), 2j(t) =2 j ( 2 j t) 2 supp( b j, ) Corollary: If there exists B that for all! 2 R \{}, A> such 1 A apple +1 X j= 1 ˆ(2 j!) 2 apple B then the dyadic wavelet dictionary is a frame. If A = B = 1, then reconstruction is particularly simple: j j +1 X f(t) = j= 1 f 2j 2 j(t) real( j, ) imag( j, )

34 1D Wavelet Transform at Different Scales Wf(u, 2 j )=f 2j(u) captures the details of f at the scale 2 j.

35 2D Wavelet Transform at Different Scales Rotations j, (u) Scales j (u)