Multiscale Frame-based Kernels for Image Registration

Multiscale Frame-based Kernels for Image Registration Ming Zhen, Tan National University of Singapore 22 July, 16 Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 1 / 31

Outline 1 Motivation 2 Frame Theory, Reproducing Kernels and Frames 3 Eg. Spline-based Reproducing Kernels Conditions and Interpretations Relationship across different scales 4 Simple Simulations Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 2 / 31

Kernels and Deformation Fields LDDMM formulation: Vector field, v t, is constrained to an RKHS, (V,, V ), such that it has the form: v t( ) = N K(g t(x n), )α n,t (1) n=1 1 K : Ω Ω R d, a matrix-valued function, is the reproducing kernel associated with the RKHS V. (*Here, K : Ω Ω R is a scalar-valued function multiplied with a d d identity matrix.) 2 K determines the properties of v t. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 3 / 31

... from Multiscale Gaussian Kernels 1 Deformable objects deformed with the LDDMM framework are commonly used with Gaussian kernels of chosen bandwidths. 2 multiscale Gaussian: weighted sum of Gaussian kernels of varying bandwidths such that both smooth and sharp variations can be captured within a single deformation map. [Risser et al. ] Figure: Taken from Risser et al. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 4 / 31

to Multiscale frame-based Kernels 1 Propose to use a generic multiscale reproducing kernel that is frame-based 1 Exploit known relationships between b-splines, Riesz bases, frames to build RKHS from spline-based frames. 2 Multiscale structure 3 Compact 4 Example 1: compact b-splines (denoted as φ) are dilated iteratively using pre-defined refinement masks M : ˆφ(2 ) ˆφ( ) and combined together like the sum of Gaussians. Example 2: wavelet masks W : ˆφ(2 ) ˆψ( ) that generate compactly supported spline-based functions (wavelet frames) 1 0.9 0.8 0.7 0.6 0. 0.4 0.3 0.2 0.1 0 1 0 1 2 3 4 6 7 (a) Refinement Mask 0. 0.4 0.3 0.2 0.1 0 0.1 0.2 0.3 1 0 1 2 3 4 6 7 (b) Wavelet Mask Figure: Refinement masks and wavelet masks; B-spline of order 2 Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Literature Review Reproducing kernels from Frames Construction of reproducing kernels, and consequently, RKHS, is well-studied in the machine learning literature, within the regularisation theory framework for function approximations. 1 In Rakotomamonjy(0), general conditions for the construction of arbitrary reproducing kernel Hilbert spaces from frame and dual frame elements were given. 2 Wavelet frames with multiresolution properties were used to design kernels for classification or regression problems utilising support vector machines (Zhang, 04; Opfer, 06; Ling, 06; Shang, 08; Zhang, ) 3 In this work, we consider tight frames constructed from b-splines and spline framelets (Ron, 199; Ron, 1997; Daubechies, 03) with multiscale structures inherited from wavelet frames. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 6 / 31

Preliminaries - Frame Theory - Operators 1 ϕ := {ϕ i } i I - a family of functions, where I is a finite or infinitely countable index set. 2 H - a Hilbert space with inner product, H. Let ϕ i H, f H and c := {c i } i I l 2. We define a synthesis operator T ϕ and an adjoint decomposition operator Tϕ such that T ϕ : l 2 (ϕ) H : {c i } i I c i ϕ i (2) i I T ϕ : H l 2 (ϕ) : f { f,ϕ i H } i I (3) Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 7 / 31

Preliminaries - Frame Theory - Definition of Frames Definition of Frames ϕ is a frame of (H,, H ) if and only if the frame condition C 1 f 2 H f,ϕ i H 2 C 2 f 2 H (4) i I holds for some C 1,C 2 such that 0 < C 1 C 2 <. 1 Tϕ is injective. 2 T ϕ is surjective. 3 When C 1 = C 2, ϕ is known as a tight frame. If C 1 = C 2 = 1, ϕ is supertight/parseval ( ϕ i H = c). Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 8 / 31

Preliminaries - Frame Theory - Frame Operator Frame Operator T ϕtϕ : H H : f f,ϕ i H ϕ i () i I 1 It is continuously invertible. 2 We denote its inverse as R := (T ϕt ϕ ) 1, R = R (self-adjoint). 3 T Rϕ is the right inverse of Tϕ and TϕT Rϕ (f ) = i I f,rϕ i H ϕ i = f. 4 Rϕ is known as the dual frame of ϕ and commonly denoted as ϕ. Rϕ helps to reconstruct any f H with the frames ϕ by first decomposing f into a set of specific coefficients { f,rϕ i } i I, known as the canonical frame coefficients. 6 The canonical frame coefficients { f, ϕ i } i I l 2 has the smallest sum of squares amongst all possible coefficients e l 2, where e := {e i } i I,T ϕ(e) = f, i.e. { f, ϕ i } i I 2 l 2 e 2 l 2. 7 Every frame has a dual frame and every dual frame is the frame of some dual frame. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 9 / 31

Reproducing kernels and Frames (Rakotomamonjy, 0) Let ϕ be a known set of frames for Hilbert space (H,, H ). Then H is an RKHS with the reproducing kernel K(x,y) = ϕ i (y)ϕ i (x) (6) i I 1 Every finite set of vectors {ϕ i } i I in a vector space equipped with an inner product is a frame for the Hilbert space H := span{ϕ i } i I (Christensen, 08) and every frame has a dual frame. It is possible to build an RKHS with a reproducing kernel of the form (6) (but it s feasible only if the dual frame has an analytical form). 2 Given a (super)tight frame ϕ, i.e. ϕ = ϕ of a Hilbert space (H,, H ), then the corresponding reproducing kernel is of the form K(x,y) = ϕ i (y)ϕ i (x) (7) i I Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Sum of Reproducing Kernels (Berlinet, 11) Let K 1 and K 2 be reproducing kernels of spaces H 1 and H 2 of functions f 1,f 2 R Ω. Then K = K 1 + K 2 is the reproducing kernel of the space H = H 1 H 2 = {f f = f 1 + f 2,f 1 H 1,f 2 H 2 } with the norm H defined by ) f 2 H = min ( f 1 2 H 1 + f 2 2 H 2 : f = f 1 + f 2,f 1 H 1,f 2 H 2 1 Let {ϕ i } i I and {ϕ i } i I be two sets of basis vectors and H 1 := span{ϕ i } i I, H 2 := span{ϕ i } i I. Then K(x,y) := i (I I ) ϕ i (x)ϕ i (y) is the reproducing kernel of H := H 1 H 2 = span{ϕ i } i (I I ) with the norm f 2 H = min( f 1 2 H 1 + f 2 2 H 2 : f = f 1 + f 2,f 1 H 1,f 2 H 2 ). Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 11 / 31

Reproducing kernels and tight frames Let ϕ be a known set of frames for Hilbert space (H,, H ). Then H is an RKHS with the reproducing kernel K(x,y) = ϕ i (y)ϕ i (x) (8) i I if and only if ϕ = ϕ. 1 The inner product, H is equivalent to, l2, since f,f H = f,ϕ i 2 H = ci 2 = c i,c i l2 (9) i I i I Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 12 / 31

Reproducing kernels and tight frames (Continued) 1 We have f ( ) = N α nk(x n, ) = c i ϕ i ( ) n=1 where c := {c i },c i = N n=1 αnϕ i (x n). 2 The frame coefficients live in the orthogonal complement of the null space of T ϕ. 3 c are the canonical frame coefficients of ϕ. 4 The reproducing kernel guarantees the l 2 -minimisation of the frame coefficients. i I Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 13 / 31

Weighted Sum of Reproducing Kernels Consider a more general form of RK that admits a positive weight parameter λ i > 0 for ϕ i, i I λ i < : K λ (x,y) = λ i ϕ i (y)ϕ i (x) () i I 1 Tunable weights allow us to further control the characteristics or properties of a reproducing kernel constructed via scaled summations. 2 Not unlike the sum of weighted Gaussians, it makes sense to put stronger emphasis on the larger bandwidths to build a smoother function space. 3 Suitable conditions on the weights can produce functions that live in a Sobolev space of arbitrary smoothness. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 14 / 31

Weighted Sum of Reproducing Kernels (Continued) Let i I λ i ϕ i (x) 2 <. K λ is admissible and corresponds to (is the RK of) some Hilbert space, i.e. the function space spanned by { λ i ϕ i } i I. Then { λ i ϕ i } i I is a supertight/parseval frame of (H λ,, Hλ ): { } c H λ = c i ϕ i : i 2 < (11) λ i i I i I f,g Hλ : = P (c),p (e) l2,λ (12) { } c l 2,λ : = c = {c i } i I : i 2 < (13) λ i i I c c,e l2,λ = i e i (14) λ i i I P is the projection into the orthogonal complement of the null space of the operator T ϕ,λ, s.t. T ϕ,λ (c) = f, T ϕ,λ (e) = g. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Eg. Spline-based Reproducing Kernels 1 Certain combinations of b-splines gives us tight frames. 2 B-splines are refinable and compact. 3 Their refinability gives a natural multiscale structure that incorporates a relationship across the different scales. 4 B-splines can be shown to converge asymptotically to Gaussians. B-splines are scaling functions of multiresolution analysis (MRA) and can be used to build spline wavelets with natural multiresolution structure. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 16 / 31

Example 1 - Multiscale b-splines (Rakotomamonjy, 0, Opfer, 06, Han, 09) Let φ m j,k := 2jd/2 φ m (2 j k), where φ m is an univariate b-spline of order m, λ j > 0, j Z λ j <. Then { λ j φ m j,k } j,k is a supertight frame of the Hilbert space (B,, B ): B = f : f = c j,k φ m (2 j c j,k 2 k), λ j 2 jd < () j Z k Z d j Z k Z d f 2 c j,k 2 B : = min λ j 2 jd = f 2 V j λ j 2 jd : f j V j := span{φ m (2 j k)} (16) j Z k Z d j Z with the reproducing kernel K S (x,y) = λ j φ m j,k(x)φ m j,k(y) = λ j 2 jd K S,j (x,y), (17) j Z,k Z d j Z where K S,j (x,y) = φ m (2 j x k)φ m (2 j y k). (18) k Z d Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 17 / 31

Example 1(Multiscale Interpretation) Using the kernel K S, v t B takes the form N v t = λ j 2 jd α n,tφ m (2 j x n k)φ m (2 j k) = λ j 2 jd v t,j j Z k Z d n=1 j Z The solution can be interpreted as a weighted sum of functions v t,j, where each v t,j belongs to span{φ m (2 j k)}, and can be constructed with the canonical coefficients c j,k = N α n,tφ m (2 j x n k) n=1 Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 18 / 31

Example 2 - with Spline Framelets Let φ m be the centred b-spline of order m for d = 1. Then φ is refinable with the refinement mask ˆτ 0 (ξ) := e iξm 2 cos m ( ξ 2 ) and it forms an MRA, {V j} j Z of L 2 (R). Then the set of m functions Ψ m = {ψl m : l = 1,...,m} defined (in the Fourier domain) as ˆψ l m (ξ) := i l e iξm 2 ( m l ) cos m l (ξ/4)sin m+l (ξ/4) (ξ/4) m (19) is a set of generators for a wavelet system X(Ψ m ) := {ψ m j,k,l := 2j/2 ψ m l (2j k) : 1 l m;j,k Z}. 1 X(Ψ m ) L 2 (R) is a tight frame in L 2 (R) that is constructed from MRA (Ron, 1997). 2 We refer to the generators Ψ m as spline framelets. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 19 / 31

Example 2 (continued) The reproducing kernel for (L 2 (R),, ) is defined as K F (x,y) = ψj,k,l(x)ψ m j,k,l(y) m () j Z k Z l 1,...,m f,g : = f,ψj,k,l g,ψ m j,k,l m (21) j Z k Z l 1,...,m Since ψ m was created using wavelet masks from the scaling function φ m, the reproducing kernel given in () can be truncated from below: K F (x,y) = φ m j min,k(x)φ m j min,k(y) + ψj,k,l(x)ψ m j,k,l(y) m (22) k Z j Z j min k Z l 1,...,m Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Example 2 (continued) Extension from L 2 (R) to L 2 (R d ) For simplicity, we consider the case d = 2. Using the tensor products of spline framelets in L 2 (R), the spline framelets for L 2 (R 2 ) are defined as ψ m l,l (x) := ψm l (x 1 )ψ m l (x 2 ), x = (x 1,x 2 ) R 2, 0 l,l m, ψ0 m = φm. Let Ψ m l,l framelets for 0 l,l m, (l,l ) (0,0). Then Ψ m l,l L 2 (R d ) with the reproducing kernel K F,2 (x,y) = j Z k Z 2 (l,l ) L where L := {(l,l ) : 0 l,l m, (l,l ) (0,0)}. be the set of newly defined spline is a tight wavelet frame for ψj,k,(l,l m ) (x)ψm j,k,(l,l )(y) (23) Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 21 / 31

Spline-based Kernel Computation - Worked Example 1 0.8 0.6 0.4 0.2 0 0.2 Plot of Kernel Splines τ 0 φ τ 1 φ τ 2 φ τ φ 3 τ 4 φ A worked example to Examples 1 and 2 where m = 4. τ 0 = 16 1 [1,4,6,4,1] τ 1 = 16 1 [1, 4,6, 4,1] τ 2 = 1 8 [1, 2,0,2, 1] τ 3 = 6 16 [1,0, 2,0,1] τ 4 = 1 8 [1,2,0, 2, 1] (24) 0.4 0 0. 1 1. 2 2. 3 3. 4 Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 22 / 31

Asymptotic properties of b-splines and spline framelets 1 Normalised b-splines are point-wise close to some Gaussian function as m goes to infinity. Specifically, it has been shown that mφ m ( mx) converges uniformly to 6 π e 6x2 as m (Chen, 04). 2 The spline framelets ψl m, 1 l m converges to the lth derivative of some Gaussian function G m,l (Shen, 13) (m G m,l := 1 ) ( ) 6 4 l l π(m l/2) exp 6x 2 m l/2 () Specifically, let G m l dl := dx G l m,l (x m /2). Then we have the inequality max max ψl m (x) Gm l 1 l m x R (x) (lnm)/2 m 3/2 Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 23 / 31

Weights and Frames of Sobolev Space Under specific conditions on λ j, { λ j φ m j,k } j,k is also a frame of a Sobolev space of arbitrary smoothness. For s R, we define the Sobolev space H s (R d ) as a space consisting of all tempered distributions f such that f 2 H s (R d ) := 1 (2π) d ˆf 2 (1 + ξ 2 2) s dξ < (26) R d 1 The wavelet system {2 j(d/2 s) φ m (2 j k) : j Z,k Z d } is a wavelet frame in H s (R d ) for all 0 < s < m 1/2 (Han, 09). 2 The upper bound of s is the regularity of the m order b-spline. 3 { λ j φ m j,k } j,k is a frame of H s (R d ) when λ j := 2 js. 4 By varying λ j, a level of smoothness can be imposed on the vector fields. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 24 / 31

Relationship Across Scales Let d = 1. Consider the masks {h l } l {0,1,...,m} such that { φ m (x) = k Z h 0,kφ m (2x k) ψ m l (x) = k Z h l,kφ m (2x k) (27) (Relationship between coefficients) Let e j,k,l := N n=1 α(xn)ψm l (2j x n k) and e j,k,0 := N n=1 α(xn)φm (2 j x n k), 1 l m. Then for 0 l m, e j,k,l = h l,k 2ke j+1,k,0 k Z Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Relationship Across Scales (Relationship between coefficients of derivatives) Using the same masks {h l } l {0,1,...,m}, and the notation D x f = df /dx, we use the following identities to compute D x v. D x ψl m (x) = k Z 2h l,kd x φ m (2x k) e j,k,l = k Z 2h l,k 2ke j+1,k,0 = N n=1 α(xn)dxn ψl m (2j x n k) where, 0 l m, ψ m 0 := φm. (28) Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 26 / 31

3 40 4 0 3 40 4 0 3 40 4 0 Test Results (a) (b) 0 0 3 40 4 0 0 0 3 40 4 0 (c) (d) (e) Figure: Visualisation of kernels of different scale. The kernel used here is K S, with a b-spline of order 4. (a) Template Image (b) Target Image (c) g 1 (x) for j min = j max = 2 (d) g 1 (x) for j min = j max = 3 (e) g 1 (x) for j min = j max = 4. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 27 / 31

Test Results (a) (b) (c) (d) Figure: Examples of K S in LDDMM: Panels (a) Template Image, (b) Target Image, (c) Deformed grid points using K S, j min = 3, j max = 0 (d) Deformed template using K S. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 28 / 31

Test Results 0 0 0 0 4 4 4 4 40 40 40 40 3 3 3 3 40 0 40 0 0 40 0 40 0 (a) (b) (c) (d) 0 0 0 0 4 4 4 4 40 40 40 40 3 3 3 3 0 40 0 40 0 0 0 40 40 0 (e) (f) (g) (h) Figure: A simulation of a deformable diffeomorphic image matching algorithm. Panels (a) Template Image, (b) Target Image, (c,e,g) Deformed grid points using Gaussian kernels, (d,f,h) Deformed template using Gaussian kernels. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 29 / 31

Test Results 0 0 0 0 4 4 4 4 40 40 40 40 3 3 3 3 0 0 40 0 40 0 0 40 0 40 0 (a) (b) (c) (d) Figure: A simulation of a deformable diffeomorphic image matching algorithm using proposed kernels. Panels (i,j) and (k,l) shows the deformed grid points and deformed template using K S and K F respectively. For K S and K F, s = 3, j min = 3, j max = 3. Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 / 31

Thank you for your attention Ming Zhen, Tan (National University of Singapore) Multiscale Frame-based Kernels for Image Registration 22 July, 16 31 / 31