Wavelets and differential operators: from fractals to Marrs primal sketch Michael Unser Biomedical Imaging Group EPFL, Lausanne, Switzerland Joint work with Pouya Tafti and Dimitri Van De Ville Plenary talk, SMAI 2009, La Colle sur Loup, 25-29 Mai, 2009 The quest for invariance Invariance to coordinate transformations Primary transformations (X): translation (T), scaling (S), rotation (R), affine (similarity) (A=S+R) A continuous-domain operator L is X-invariant iff. it commutes with X; i.e, f L 2 (R d ), XLf = C X LXf C X : normalization constant All classical physical laws are TSR-invariant Classical signal/image processing operators are invariant (to various extents) Filters (linear or non-linear): T-invariant Differentiators, wavelet transform: TS-invariant Contour/ridge detectors (Gradient, Laplacian, Hessian): TSR-invariant Steerable filters: TR-invariant 2
Invariant signals Natural signals/images often exhibit some degree of invariance (at least locally, if not globally) Stationarity, texture: T-invariance Isotropy (no preferred orientation): R-invariance Self-similarity, fractality: S-invariance (Pentland 1984; Mumford, 2001) 3 OUTLINE Splines and T-invariant operators Green functions as elementary building blocks Existence of a local B-spline basis Link with stochastic processes Imposing affine (TSR) invariance Fractional Laplace operator & polyharmonic splines Fractal processes Laplacian-like, quasi-isotropic wavelets Polyharmonic spline wavelet bases Analysis of fractal processes The Marr wavelet Complex Laplace/gradient operator Steerable complex wavelets Wavelet primal sketch Directional wavelet analysis 4
General concept of an L-spline L{ }: differential operator (translation-invariant) δ(x) = d i=1 δ(x i): multidimensional Dirac distribution Definition The continuous-domain function s(x) is a cardinal L-spline iff. L{s}(x) = k Z d a[k]δ(x k) Cardinality: the knots (or spline singularities) are on the (multi-)integers Generalization: includes polynomial splines as particular case (L = dn dx N ) 5 Example: piecewise-constant splines Spline-defining operators Continuous-domain derivative: D = d dx jω Discrete derivative: + { } 1 e jω Piecewise-constant or D-spline s(x) = k Z s[k]β 0 +(x k) D{s}(x) = k Z + s(k) {}}{ a[k] δ(x k) B-spline function β 0 +(x) = + D 1 {δ}(x) 1 e jω jω 6
Splines and Greens functions Definition ρ(x) is a Green function of the shift-invariant operator L iff L{ρ} = δ ρ(x) L{ } δ(x) δ(x) L 1 { } ρ(x) (+ null-space component?) Cardinal L-spline: L{s}(x) = k Z d a[k]δ(x k) Formal integration a[k]δ(x k) k Z d L 1 { } s(x) = k Z d a[k]ρ(x k) V L = span {ρ(x k)} k Z d 7 Example of spline synthesis δ(x) L= d dx =D L 1 : integrator L 1 { } ρ(x) δ(x x 0 ) Translation invariance L 1 { } Green function = Impulse response ρ(x x 0 ) a[k]δ(x k) k Z Linearity L 1 { } s(x) = k Z a[k]ρ(x k) 8
Existence of a local, shift-invariant basis? Space of cardinal L-splines V L = s(x) : L{s}(x) = a[k]δ(x k) L 2(R d ) k Z d Generalized B-spline representation A localized function ϕ(x) V L is called generalized B-spline if it generates a Riesz basis of V L ; i.e., iff. there exists (A > 0, B < ) s.t. A c l2 (Z d ) k Z c[k]ϕ(x k) B c d L2 (R d l 2 (Z ) d ) V L = s(x) = c[k]ϕ(x k) : x R d, c l 2 (Z d ) k Z d continuous-domain signal discrete signal (B-spline coefficients) 9 Link with stochastic processes Splines are in direct correspondence with stochastic processes (stationary or fractals) that are solution of the same partial differential equation, but with a random driving term. Defining operator equation: L{s( )}(x) = r(x) Specific driving terms r(x) =δ(x) s(x) =L 1 {δ}(x) : Green function r(x) = k Z d a[k]δ(x k) s(x) : Cardinal L-spline r(x): white Gaussian noise s(x): generalized stochastic process non-empty null space of L, boundary conditions References: stationary proc. (U.-Blu, IEEE-SP 2006), fractals (Blu-U., IEEE-SP 2007) 10
Example: Brownian motion vs. spline synthesis L= d dx white noise or stream of Diracs L 1 : integrator L 1 { } Brownian motion Cardinal spline (Schoenberg, 1946) 11 IMPOSING SCALE INVARIANCE Affine-invariant operators Polyharmonic splines Associated fractal random fields: fbms 12
Scale- and rotation-invariant operators Definition: An operator L is affine-invariant (or SR-invariant) iff. s(x), L{s( )}(Rθ x/a) = Ca L{s(Rθ /a)}(x) where Rθ is an arbitrary d d unitary matrix and Ca a constant Invariance theorem The complete family of real, scale- and rotation-invariant convolution operators is given by the fractional Laplacians F γ ( ) 2 $ω$γ Invariant Green functions (a.k.a. RBF) ρ(x) = (Duchon, 1979) xγ d log x, if γ d is even xγ d, otherwise 14
Polyharmonic splines Spline functions associated with fractional Laplace operator ( ) γ/2 Distributional definition [Madych-Nelson, 1990] s(x) is a cardinal polyharmonic spline of order γ iff. ( ) γ/2 s(x) = k Z 2 d[k]δ(x k) Explicit Shannon-like characterization { V 0 = s(x) = } s[k]φ γ (x k) k Z 2 φ γ (x): Unique polyharmonic spline interpolator s.t. φ γ (k) = δ k F ˆφγ (ω) = 1 1 + k Z d \{0} ( ) γ ω ω+2πk 15 Construction of polyharmonic B-splines Laplacian operator: Discrete Laplacian: d F ω 2 F d 4 sin 2 (ω i /2) = 2 sin(ω/2) 2 i=1 0-1 0-1 4-1 Polyharmonic B-splines (Rabut, 1992) 0-1 0 Discrete operator: localization filter Q(e jω ) 2 sin(ω/2) γ ω γ 1 F β γ (x) Continuous-domain operator: ˆL(ω) 16
Polyharmonic B-splines properties Stable representation of polyharmonic splines (Riesz basis) V Condition: γ > d γ = ( 2 ) 2 s(x) = c[k]β γ (x k) :c[k] l 2 (Z d ) k Z d Two-scale relation: β γ (x/2) = k Z d h γ [k]βγ(x k) Order of approximation γ (possibly fractional) Reproduction of polynomials The polyharmonic B-splines {ϕ γ (x k)} k Z d reproduce the polynomials of degree n = γ 1. In particular, ϕ γ (x k) = 1 (partition of unity) k Z d (Rabut, 1992; Van De Ville, 2005) 17 Associated random field: multi-d fbm Formalism: Gelfands theory of generalized stochastic processes White noise fractional Brownian field ( ) γ 2 fractional integrator (appropriate boundary conditions) ( ) γ 2 Whitening (fractional Laplacian) Hurst exponent: H = γ d 2 (Tafti et al., IEEE-IP 2009) 18
LAPLACIAN-LIKE WAVELET BASES Operator-like wavelet design Fractional Laplacian-like wavelet basis Improving shift-invariance and isotropy Wavelet analysis of fractal processes (multidimensional generalization of pioneering work of Flandrin and Abry) 19 Multiresolution analysis of L2(R d ) s V (0) ( ) Multiresolution basis functions: ϕ i,k (x) = 2 id/2 x 2 ϕ i k 2 i Subspace at resolution i: V (i) = span {ϕ i,k } k Z d s 1 V (1) s 2 V (2) s 3 V (3) Two-scale relation V (i) V (j), for i j Partition of unity i Z V (i) = L 2 (R d ) 20
General operator-like wavelet design Search for a single wavelet that generates a basis of L 2 (R d ) and that is a multi-scale version of the operator L; i.e., ψ = L φ where φ is a suitable smoothing kernel General operator-based construction Basic space V 0 generated by the integer shifts of the Green function ρ of L: V 0 = span{ρ(x k)} k Z d with Lρ = δ Orthogonality between V 0 and W 0 = span{ψ(x 1 2 k)} k Z d \2Z d ψ( x 0 ), ρ( k) = φ, Lρ( k + x 0 ) = φ, δ( k + x 0 ) = φ(k x 0 ) = 0 (can be enforced via a judicious choice of φ (interpolator) and x 0 ) Works in arbitrary dimensions and for any dilation matrix D 21 Fractional Laplacian-like wavelet basis ψ γ (x) = ( ) γ 2 φ2γ (Dx) φ 2γ (x): polyharmonic spline interpolator of order 2γ > 1 D: admissible dilation matrix ( Wavelet basis functions: ψ (i,k) (x) det(d) i/2 ψ γ D i x D 1 k ) {ψ (i,k) } (i Z, k Z2 \DZ 2 ) forms a semi-orthogonal basis of L 2 (R 2 ) f L 2 (R 2 ), f = f, ψ (i,k) ψ (i,k) = f, ψ (i,k) ψ (i,k) i Z k Z 2 \DZ 2 i Z k Z 2 \DZ 2 where { ψ (i,k) } is the dual wavelet basis of {ψ (i,k) } The wavelets ψ (i,k) and ψ (i,k) have γ vanishing moments The wavelet analysis implements a multiscale version of the Laplace operator and is perfectly reversible (one-to-one transform) The wavelet transform has a fast filterbank algorithm (based on FFT) [Van De Ville, IEEE-IP, 2005] 22
Laplacian-like wavelet decomposition Nonredundant transform f(x) d i [k] = f,ψ (i,k) dyadic sampling pattern D = [ ] 2 0 0 2 first decomposition level (one-to-one) β γ (D 1 x k) scaling functions (dilated by 2) ψ γ (D 1 x k) wavelets (dilated by 2) 23 Improving shift-invariance and isotropy Wavelet subspace at resolution i Wavelet sampling patterns W i = span {ψ i,k } (k Z 2 \DZ 2 ) Non-redundant (basis) Augmented wavelet subspace at resolution i W + i = span { ψ (i,k) }(k Z 2 ) = span{ψ iso,(i,k)} (k Z2 ) Mildly redundant (frame) Admissible polyharmonic spline wavelets Operator-like generator: ψ γ (x) = ( ) γ/2 φ 2γ (Dx) More isotropic wavelet: ψ iso (x) = ( ) γ/2 β 2γ (Dx) Quasi-isotropic polyharmonic B-spline [Van De Ville, 2005] β 2γ (x) C γ exp( x 2 /(γ/6)) 24
Building Mexican-Hat-like wavelets ψ γ (x) = ( ) γ/2 φ 2γ (2x) ψ iso (x) = ( ) γ/2 β 2γ (2x) Gaussian-like smoothing kernel: β 2γ (2x) 25 Mexican-Hat multiresolution analysis Pyramid decomposition: redundancy 4/3 ψ iso (x) = ( ) γ/2 β 2γ (Dx) Dyadic sampling pattern First decomposition level: ϕ(d 1 x k) Scaling functions (U.-Van De Ville, IEEE-IP 2008) ψ(d 1 (x k)) Wavelets (redundant by 4/3) 26
Wavelet analysis of fbm: whitening revisited Operator-like behavior of wavelet Analysis wavelet: ψ γ =( ) γ 2 φ(x) =( ) H 2 + d 4 ψ γ (x) Reduced-order wavelet: ψ γ γ (x) = ( ) 2 φ(x) with γ = γ (H + d 2 ) > 0 Stationarizing effect of wavelet analysis Analysis of fractional Brownian field with exponent H: ( B H, ψ x0 ) H ( γ a ( ) 2 + d 4 B H, ψ γ x0 ) ( a = W, ψ x0 ) γ a Equivalent spectral noise shaping: S wave (e jω )= n Z d ˆψ γ(ω +2πn) 2 Extent of wavelet-domain whitening depends on flatness of S wave (e jω ) Whitening effect is the same at all scales up to a proportionality factor fractal exponent can be deduced from the log-log plot of the variance (Tafti et al., IEEE-IP 2009) 27 Wavelet analysis of fractional Brownian fields Theoretical scaling law : Var{w a [k]} = σ 2 0 a (2H+d) log-log plot of variance 40 H est =0.31 log 2 (Var{w log 2 w2 n[k] a}) 38 36 34 32 30 28 26 24 22 1 2 2 2 3 2 4 4 5 2 Scale scale [n] a 6 8 8 7 2 16 8 28
Fractals in bioimaging: fibrous tissue DDSM: University of Florida (Digital Database for Screening Mammography) (Laine, 1993; Li et al., 1997) 29 Wavelet analysis of mammograms log-log plot of variance H est =0.44 35 log log 2 (Var{w 2 w2 n[k] a}) 30 25 20 15 1 2 2 2 3 2 4 4 5 2 6 8 8 7 2 16 8 Scale scale [n] a Fractal dimension: D = 1 + d H = 2.56 with d = 2 (topological dimension) 30
Brain as a biofractal (Bullmore, 1994) 1mm Courtesy R. Mueller ETHZ 31 Wavelet analysis of fmri data log 2 w2 n[k] H est =0.35 38 36 34 32 30 28 26 24 81 2 2 2 8 3 4 2 4 scale [n] 5 4 2 2 6 8 2 7 2 2 Brain: courtesy of Jan Kybic Fractal dimension: D = 1 + d H = 2.65 with d = 2 (topological dimension) 32
...and some non-biomedical images... He s t = 0.53 40 w 2 [k]" log log n a }) 2 2 (Var{w 35 30 25 21 22 232 4 45 2 scalea[n] Scale 68 87 2 8 16 33 THE MARR WAVELET Laplace/gradient operator Steerable Marr wavelets Wavelet primal sketch Directional wavelet analysis 34
Complex TRS-invariant operators in 2D Invariance theorem The complete family of complex, translation-, scale- and rotationinvariant 2D operators is given by the fractional complex Laplacegradient operators L γ,n =( ) γ N 2 with N N and γ N R ( + j ) N x 1 x 2 F ω γ N (jω 1 ω 2 ) N Key property: steerability L γ,n {δ}(r θ x)=e jnθ L γ,n {δ}(x) 35 Simplifying the maths: Unitary Riesz mapping Complex Laplace-gradient operator ( L γ,n =( ) γ N 2 + j ) N =( ) γ 2 R N x 1 x 2 F where R =L1 2,1 ( jω1 ω 2 ω ) Property of Riesz operator R R is shift- and scale-invariant R is rotation covariant (a.k.a. steerable) R is unitary in particular, R will map a Laplace-like wavelet basis into a complex Marr-like wavelet basis 36
Complex Laplace-gradient wavelet basis ψ γ(x) =( ) γ 1 2 ( + j ) φ 2γ (Dx) =Rψ γ (x) x 1 x 2 φ 2γ (x): polyharmonic spline interpolator of order 2γ > 1 D: admissible dilation matrix Wavelet basis functions: ψ (i,k) (x) =Rψ ( (i,k)(x) = det(d) i/2 ψ γ D i x D 1 k ) {ψ (i,k) } (i Z, k Z 2 \DZ 2 ) forms a complex semi-orthogonal basis of L 2 (R 2 ) f L 2 (R 2 ), f = f,ψ(i,k) ψ (i,k) = f, ψ (i,k) ψ (i,k) i Z k Z 2 \DZ 2 i Z k Z 2 \DZ 2 where { ψ (i,k) } is the dual wavelet basis of {ψ (i,k) } The wavelet analysis implements a multiscale version of the Gradient-Laplace (or Marr) operator and is perfectly reversible (one-to-one transform) The wavelet transform has a fast filterbank algorithm [Van De Ville-U., IEEE-IP, 2008] 37 Wavelet frequency responses Laplacian-like / Mexican hat Marr pyramid (steerable) ˆψ iso,i (ω) ˆψRe,i (ω) ˆψIm,i (ω) Unitary mapping (Riesz transform) γ = 4 38
Marr wavelet pyramid Steerable pyramid-like decomposition: redundancy 2 4 3 ψ Re (x) = x β 2γ(2x) ψ Im (x) = y β 2γ(2x) Basic dyadic sampling cell ψ 1 ψ 0 ψ 1 ψ 2 ψ 3 ψ 2 ψ 3 ϕ( /2) ϕ( /2) Wavelet basis Overcomplete by 1/3 39 Processing in early vision - primal sketch Wavelet primal sketch [Van De Ville-U., IEEE-IP, 2008] blurring smoothing kernel φ Laplacian filtering zero-crossings and orientation segment detection and grouping Canny edge detection scheme image Marr-like wavelet pyramid Canny edge detector wavelet primal sketch 40
Edge detection in wavelet domain Edge map (using Canny s edge detector) Key visual information (Marr s theory of vision) Similar to Mallats representation from wavelet modulus maxima [Mallat-Zhong, 1992]... but much less redundant 41 Iterative reconstruction Reconstruction from information on edge map only Better than 30dB PSNR 31.4dB [Van De Ville-U., IEEE-IP, 2008] 42
Directional wavelet analysis: Fingerprint Modulus Wavenumber Modulus Orientation Orientation Wavelet-domain structure tensor γ = 2, σ = 2 43 Marr wavelet pyramid - discussion Comparison against state-of-the-art Steerable Complex Marr wavelet pyramid dual-tree pyramid Translation invariance ++ ++ ++ Steerability ++ + ++ Number of orientations 2K 6 2 Vanishing moments no yes, 1D γ Implementation filterbank/fft filterbank FFT Decomposition type tight frame frame complex frame Redundancy 8K/3 + 1 4 8/3 Localization slow decay filterbank design fast decay Analytical formulas no no yes Primal sketch - - yes Gradient/structure tensor - - yes [Simoncelli, Freeman, 1995] [Kingsbury, 2001] [Selesnick et al, 2005] 44
CONCLUSION Unifying operator-based paradigm Operator identification based on invariance principles (TSR) Specification of corresponding spline and wavelet families Characterization of stochastic processes (fractals) Isotropic and steerable wavelet transforms Riesz basis, analytical formulaes Mildly redundant frame extension for improved TR invariance Fractal and/or directional analyses Fast filterbank algorithm (fully reversible) Marr wavelet pyramid Multiresolution Marr-type analysis; wavelet primal sketch Reconstruction from multiscale edge map Implementation will be available very soon (Matlab) 45 References D. Van De Ville, T. Blu, M. Unser, Isotropic Polyharmonic B-Splines: Scaling Functions and Wavelets, IEEE Trans. Image Processing, vol. 14, no. 11, pp. 1798-1813, November 2005. M. Unser, D. Van De Ville, The pairing of a wavelet basis with a mildly redundant analysis with subband regression, IEEE Trans. Image Processing, Vol. 17, no. 11, pp. 2040-2052, November 2008. D. Van De Ville, M. Unser, Complex wavelet bases, steerability, and the Marr-like pyramid, IEEE Trans. Image Processing, Vol. 17, no. 11, pp. 2063-2080, November 2008. P.D. Tafti, D. Van De Ville, M. Unser, Invariances, Laplacian-Like Wavelet Bases, and the Whitening of Fractal Processes, IEEE Trans. Image Processing, vol. 18, no. 4, pp. 689-702, April 2009. EPFLs Biomedical Imaging Group Preprints and demos: http://bigwww.epfl.ch/ 46