2D Wavelets Hints on advanced Concepts 1
Advanced concepts Wavelet packets Laplacian pyramid Overcomplete bases Discrete wavelet frames (DWF) Algorithme à trous Discrete dyadic wavelet frames (DDWF) Overview on edge sensitive wavelets Contourlets 2
Wavelet packets 3
Wavelet packets Both the approximation and the detail subbands are further decomposed 4
Packet tree 5
Wavelet Packets frequency η η /2 increasing the scale η /4 time 6
Laplacian Pyramid residuals prediction Low-pass 2 2 Interpolation - residual coarser version 7
Overcomplete bases 8
Translation Covariance Translation covariance { DWT{ f ( x, y) } DWT{ T{ f ( x, y) } T = If translation covariance does not hold: T { DWT{ f ( x, y) } DWT{ T{ f ( x, y) } Signal Translation DWT Wavelet coefficients Signal DWT Translation Wavelet coefficients NOT good for signal analysis 9
Rationale In pattern recognition it is important to build representations that are translation invariant. This means that when the pattern is translated the descriptors should be translated but not modified in the value. CWTs and windowed FT provide translation covariance, while sampling the translation parameter might destroy translation covariance unless some conditions are met. Intuition: either the sampling step is very small compared to the translation or the translation is a multiple of the sampling step. Adaptive sampling could also be a solution (tracking local maxima) 10
Translation invariant representations 11
12 Translation covariance The signal descriptors should be covariant with translations Continuous WT and windowed FT are translation covariant. Wavelet frames (DWF) are constructed by sampling continuous transforms over uniform time grids. In general, the sampling grid removes the translation covariance because the translation factor τ is a priori not equal to the translation interval ), ( 1 ) ( ), ( ) ( 1 ) ( ), ( ) ( ) ( s u Wf dt s u t s t f s u Wf u f dt s u t s t f s u Wf t f t f s τ ψ τ ψ ψ τ τ τ = = = = =
Sampling and translation covariance τ If the translation does not correspond to a multiple of the sampling step, a different set of samples will be obtained by keeping the same sampling grid when the signal shifts 13
Sampling and translation covariance τ 14
Translation invariant representation au f ψ () t a If the sampling interval 0 is small enough than the samples of are approximately translated when f is shifted. τ = ku a Translation covariance holds if namely it is a multiple of the sampling 0 interval Wf(u,a ) u Wf tau (u,a ) τ au0 u 15
Translation invariant representation Translation invariant representations can be obtained by sampling the scale parameter s but not the translation parameter u Uniformly sampling the translation parameter destroys covariance unless the translation is very small 16
Dyadic Wavelet Transform Sampling scheme Dyadic scales Integer translations Wf ( u,2 ψ 2 ) = ( t) = ψ 2 f ( t) 1 2 ( t) = t u ψ dt = f ψ 2 1 2 t ψ 2 If the frequency axis is completely covered by dilated dyadic wavelets, then it defines a complete and stable representation The normalized dyadic wavelet transform operator has the same properties of a frame operator, thus both an analysis and a reconstruction wavelets can be identified 2 ( u) Special case: algorithme à trous 17
Algorithme à trous Similiar to a fast biorthogonal WT without subsampling Fast dyadic transform The samples of the discrete signal a 0 [n] are considered as averages of some function weighted by some scaling kernels φ(t-n) a 0 [ n] = f ( t), ϕ( t n) For any 0 we denote a d 2 [ n] = [ n] = f ( t), ϕ f ( t), ψ 2 2 ( t n) The dyadic wavelet coefficients are computed for ( t n) = Wf ( n,2 ) > 0 over the integer grid For any filter x[n], we denote by x [n] the filters obtained by inserting 2-1 zeros between each sample of x[n] create holes (trous, in French) x [ n] = x [ n] 18
19 Algorithme à trous Proposition The dyadic wavelet representation of a 0 is defined as the set of wavelet coefficients up to the scale 2 J plus the remaining low-pass frequency information a J Fast filterbank implementation ( ) ] [ ~ ] [ ~ 2 1 ] [ ] [ ] [ ] [ ] [ 0 1 1 1 1 n g d n h a n a n g a n d n h a n a + = = = + + + + any For,{ } 1 J J a d
20 Analysis ~ h ~ h +1 ~ g ~ g +1 a a +1 d +1 a +2 d +2 2-1 trous No subsampling!! 2 1 0 2 2 1 0 1 2 1 0 0 0 0 0 0 0 ] [ ~ 0 0 ] [ ~ ] [ ~ h h h n h h h h n h h h h n h = = =
Synthesis a +1 a a +2 h +1 + h + d +2 g +1 d +1 g Overcomplete wavelet representation: [a J, {d } 1 J ] 21
Algorithme a trous d 1 f g(z) d 2 f s(z) h(z) a 1 f g(z 2 ) a 2 f g(z 4 ) d 3 f h(z 2 ) a 3 f h(z 4 ) 22
DWT vs DWF DWT Non-redundant Signal il subsampled Not translation invariant Total number of coefficients: N x N y DWF Redundant (in general) Signal is not subsapled Filters are upsampled Translation invariant Total number of coefficients: (3J+1)N x N y Compression Feature extraction 23
Discrete WT vs Dyadic WT DWT Original Dyadic WT LL HL LH HH 24
Example 1 25
Example 2 26
Rotation covariance Oriented wavelets k {ψ ( x, y )} 1 k K In 2D, a dyadic WT is computed with several wavelets which have different spatial orientations We denote ψ k 2 (x, y) = 1 2 ψ k The WT in the direction k is defined as! x 2, y $ # & " 2 % W k f (u, v, 2 ) = f (x, y),ψ k 2 (x u, y v) = f ψ k 2 (u, v) One can prove that this is a complete and stable representation if there exist A>0 and B>0 such that K + { } ( ) 2 A ˆ 2 ( ωx, ωy) R 0,0, ψ 2 ωx,2 ωy k= 1 = B 27
Oriented wavelets Then, there exists a reconstruction wavelet family such that Gabor wavelets f (x, y) = + = K 1 W k f (w, y,2 ) 2 k=1 { } ψ k ( x, y) = g(x, y)exp iη(xcosα + ysinα ) k k ψ k 2 (x, y) g(x, y) = 1 ' 2π exp )! x2 + y 2 $ + ) ( # " 2 &, Gaussian Gabor wavelets *) %-) ˆψ k (ω 2 x,ω y ) = 2 ĝ(2 ω x ηcosα k,2 ω y ηsinα k ) ω y increases ω x In the Fourier plane the energy of the Gabor wavelet is mostly concentrated in ( 2 ηcosα,2 ηsinα ) in a neighborhood proportional to 2 k k - 28
Gabor wavelets dyadic scales ˆψ k 2 (ω x,ω y ) = 2 ĝ(2 ω x ηcosα k,2 ω y ηsinα k ) ( x, ωy) g ω ω y η 2 1 α k ω x 29
Gabor wavelets { } ψ k ( x, y) = g(x, y)exp iη(xcosα + ysinα ) k k g(x, y) = 1 ' 2π exp )! x2 + y 2 $ + ) ( # " 2 &, Gaussian Gabor wavelets *) %-) ˆψ k (ω 2 x,ω y ) = 2 ĝ(2 ω x ηcosα k,2 ω y ηsinα k ) ( x, ωy) g ω ω y η 2 α k g ( ω x η cos α, ω y η sin α ) k k ω x 30
Gabor wavelets dyadic scales 2 - ω y η 2 2 η ρ = ( cosαk) + ( sinα k) = 2 2 1 θ = tg k ( α ) k ρ α k ω x Other directional wavelet families Dyadic Frames of Directional Wavelets [Vandergheynst 2000] Curvelets [Donoho&Candes 1995] Steerable pyramids [Simoncelli-95] Contourlets [Do&Vetterli 2002] 31
Dyadic Frames of Directional Wavelets Pierre Vandergheynst (LTS-EPFL) http://people.epfl.ch/pierre.vandergheynst 32
Dyadic Directional WF 33
Dyadic Directional Wavelet Frames Directional selectivity at any desired angle at any scale Not only horizontal, vertical and diagonal as for DWT and DWF Rotation covariance for multiples of 2π/K Recipe Build a family of isotropic wavelets such that the Fourier transform of the mother wavelet expressed in polar coordinates is separable Ψ^ ( ω,φ) = Γ( ω)θ( φ) ( ) ω = ω x 2 +ω y 2 Split each isotropic wavelet in a set of oriented wavelets by an angular window Express the angular part Θ(φ) as a sum of window functions centered at θ k ω y ω ω x 34
Partitions of the F-domain θ( ϕ) 2π ϕ 35
Dyadic Directional Wavelet Frames 4 orientations (K=4) 36
Dyadic Directional Wavelet Frames Scaling function Orientation Scale Properties Overcomplete Translation covariance Rotation covariance for given angles DDWF Feature Images or Neural images 37
Contourlets Brief overview Minh Do, CM University Martin Vetterli, LCAV-EPFL 38
Contourlets Goal Design an efficient linear expansion for 2D signals, which are smooth away from discontinuities across smooth curves Efficiency means Sparseness [Do&Vetterli] Piecewise smooth images with smooth contours Inspired to curvelets [Donoho&Candes] 39
Curvelets Basic idea Curvelets can be interpreted as a grouping of nearby wavelet basis functions into linear structures so that they can capture the smooth discontinuity curve more efficiently 2-2- wavelets c2-/2 2- curvelets More efficient in capturing the geometry -> more concise (sparse) representation 40
Parabolic scaling M. Do and M. Vetterli, The Contourlet Transform: An Efficient Directional Multiresolution Image Representation, IEEE-TIP 41
Curvelets Embedded grids of approximations in spatial domain. Upper line represents the coarser scale and the lower line the finer scale. Two directions (almost horizontal and almost vertical) are considered. Each subspace is spanned by a shift of a curvelet prototype function. The sampling interval matches with the support of the prototype function, for example width w and length l, so that the shifts would tile the R 2 plan. The functions are designed to obey the key anisotropy scaling relation: width length 2 Close resemblance with complex cells (orientation selective RF)! 42
Better approximation properties for geometries: for a given rate, a better representation of edges is reached Application: coding 43
Summary of useful relations If f is real ˆ( ) ˆ ω f ω = f( e ) fˆ ˆ fˆ fˆ e fˆ ( ω+ π) ( ω+ π) = f( e ) ω * ( ω) = ( ) = ( ω) ˆ ˆ * ( ω+ π) n tˆ( ω) = f ( ω+ π) = f( e ) t[ n] = ( 1) f[ n] ω ˆ* ω ˆ ( ω+ π) 1 n gˆ( ω) = e f ( ω+ π) = e f( e ) g[ n] = ( 1) f[1 n] 44
Conclusions Multiresolution representations are the fixed point of vision sciences and signal processing Different types of wavelet families are suitable to model different image features Smooth functions -> isotropic wavelets Contours and geometry -> Curvelets Adaptive basis More flexible tool for image representation Could be related to the RF of highly specialized neurons 45
References A Wavelet tour of Signal Processing, S. Mallat, Academic Press Papers A theory for multiresolution signal decomposition, the wavelet representation, S. Mallat, IEEE Trans. on PAMI, 1989 Dyadic Directional Wavelet Transforms: Design and Algorithms, P. Vandergheynst and J.F. Gobbers, IEEE Trans. on IP, 2002 Contourlets, M. Do and M. Vetterli (Chapter) 46