Random walks for deformable image registration Dana Cobzas

Random walks for deformable image registration Dana Cobzas with Abhishek Sen, Martin Jagersand, Neil Birkbeck, Karteek Popuri Computing Science, University of Alberta, Edmonton

Deformable image registration?t I target J source Outline: Continuous formulation of deformable registration Discrete formulation and related work Diffusion and regularization from continuous to discrete Random walker for deformable registration Examples and discussion

Deformable image registration?t transformation I target I,J :R, R d T : R d, T =Idu J source Eu= E D I, J T u E R u Data similarity regularization u=argmin u E u u=(u x,u y ) displacement field

Variational image registration u=argmin u x I x, J xu x d x Data similarity (point-wise score) x J x, u x d x Adaptable regularization + EXAMPLE: SSD Linear isotropic regularization I x, J xux=i x J xu x 2 J x, u x=w J x 2 u x 2 u y 2 scalar valued diffusivity

Euler-Lagrange equations min u E D I, J T x w. u x 2 u y 2 d x E D u x w. u x =0 E D u y w. u y =0 Linear isotropic diffusion Continuous solution to deformable registration: Nonlinear optimization (gradient descent, Gauss-Newton...) Semi-implicit scheme to solve the EL PDEs Regularization can be decoupled from the data term update demon's algorithm Main limitation local minima > need for better frameworks

Discrete deformable registration u i D I I i, i=1..n J i, i=1..n J Image = Graph G = (N, E) d k Quantized deformations D = {d 1, d 2,...,d K } Labels L = {u 1, u 2,...,u K } Note: D can be very large Ex: deformations [0-20pix] 41 3 = 68921 labels

Discrete deformable registration u i D I I i, i=1..n J i, i=1..n J Image = Graph G = (N, E) Registration as labeling : MRF Quantized deformations D = {d 1, d 2,...,d k } Labels L = {u 1, u 2,...,u k } Note: D can be very large Ex: deformations [0-20pix] 41 3 = 68921 labels Eu= i N i u i i, j E ij u i, u j data similarity Regularization Interaction

Related work Eu= i N i u i i, j E ij u i, u j Global optimum ij metric graph cut optimization [T.W. Tang et al. MICCAI 2007][L. Tang et al. MICCAI 2010] More complex interaction terms linear programming method based on primal-dual principle [Glocker et al. MIA 2008] We propose : Gaussian MRF for deformable registration solved using random walker algorithm [Grady PAMI 2006, CVPR 2005] Fast algorithm Can easily incorporate a large number of displacements Robustness to noise Unique global minimum (has to be relaxed)

Related work Eu= i N i u i i, j E ij u i, u j ij metric graph cut optimization [T.W. Tang et al. MICCAI 2007][L. Tang et al. MICCAI 2010] More complex interaction terms linear programming method based on primal-dual principle [Glocker et al. MIA 2008] Here: Gaussian MRF for deformable registration solved using random walker algorithm [Grady PAMI 2006, CVPR 2005] Fast algorithm Can easily incorporate a large number of displacements Robustness to noise Unique global minimum (has to be relaxed) Regularization E R u= u 2 dxdy diffusion u=0 NEXT: - diffusion and regularization - diffusion on graphs - RW for image registration - Experiments and discussion

Diffusion u t Uniform =div u=u u t Isotropic =div w u u0=u 0 u0=u 0 u(0) w s 2 = 1 s 2 s= u 0 u(t) u(t)... Scalar-valued diffusivity Changes the metric of the space Slows down/blocks diffusion at edges Steady state

Diffusion u t Uniform =div u=u u t Isotropic =div w u u0=u 0 u0=u 0 u(0) w s 2 = 1 s 2 s= u 0 u(t) u(t)... Steady state Scalar-valued diffusivity Changes the metric of the space Slows down/blocks diffusion at edges Numerical solution: Parabolic PDEs fully implicit time discretization > elliptic PDEs solve with implicit or semi-implicit FD schemes

Regularization and diffusion f u? Energy functional : Euler Lagrange Equation : E u u= 1 2 f u 2 u 2 dxdy u=argmin u E u u u f =u data regularization

Regularization and diffusion f u? Energy functional : Euler Lagrange Equation : data E u u= 1 2 f u 2 u 2 dxdy u=argmin u E u u u f regularization Uniform diffusion u =u t =u Fully implicit time discretization of the uniform diffusion with initial value f and time step α

Regularization Uniform Isotropic w E R u= u 2 dxdy E R u= w f 2 u 2 dxdy Euler Lagrange u f =u u f = w. u

Regularization Uniform Isotropic w E R u= u 2 dxdy E R u= w f 2 u 2 dxdy Euler Lagrange u f =u u f = w. u Deformable registration min u E D I, J T x w. u x 2 u y 2 d x E D u x w. u x =0 E D u y w. u y =0

u: u=0 ux=f B, x K B Discrete diffusion Finite differences + implicit scheme 1 1-4 1 1 Boundary conditions u i-1,j-1 u i-1,j u i-1,j+1 u i,j-1 u i,j u i,j+1 u i+1,j-1 u i+1,j u i+1,j+1 Pixel location Lu=0 [ L u B B][ u B] u B L f =0 L u u u = B T f B Generalize L isotropic diffusion Combinatorial Laplacian L ij = d i = k w ik w ij if i= j if i, j E 0 otherwise Positive semi-definite (PSD)

u=0 ux=f B, x K B Discrete diffusion Boundary conditions Combinatorial Dirichlet integral u: Finite differences + implicit scheme 1 u i-1,j-1 u i-1,j u i-1,j+1 1-4 1 u i,j-1 u i,j u i,j+1 1 u i+1,j-1 u i+1,j u i+1,j+1 Pixel location D [u]= u 2 dx D[u ]=u t A T A u=u t L u A (ij)k incidence matrix (combinatorial gradient) Critical points = minima [L-PSD] Lx=0 Lu=0 [ L u B B][ u B] u B L f =0 L u u u = B T f B Generalize L isotropic diffusion Combinatorial Laplacian L ij = d i = k w ik w ij if i= j if i, j E 0 otherwise Positive semi-definite (PSD)

Random walker algorithm Lu=0 [ L u B B][ u B] u B L f =0 L u u u = B T f B [Grady PAMI 2006] Combinatorial Laplacian L ij = d i = k w ik if i= j if i, j E 0 otherwise w ij w ij =exp I i I j 2 f B seeds ; u i prob that a random walker from node i first reaches a seed.

RW for deformable registration u i D Quantized deformations D = {d 1, d 2,...,d K } Labels L = {u 1, u 2,...,u K } I i, i=1..n Registration as labeling : MRF J i, i=1..n E u= i N i u i i, j E ij u i,u j data similarity potential Gaussian MRF : Regularization Interaction E k u k = i N l=1: K,l k i l u i k 2 i k 1 u i k 2 ij E w ij u i k u j k 2 Modifications : 1. u i k = prob of node i having label u k 2. Gaussian MRF with interaction u i k,u j k =w ij u i k u j k 2 3. 'priors' for probability of a displacement i k =exp I i J id k 2

Gaussian MRF for deformable registration Displacement probability i k =exp I i J id k 2 1 if displacement d k fits perfectly the similarity score Small - otherwise Data potential i u i k = l =1: K,l k i l u i k 2 i k 1 u i k 2 Displacement l k has high prob. > low prob. for label k Displacement k has high prob. > high prob. for label k

Gaussian MRF for deformable registration Displacement probability i k =exp I i J id k 2 1 if displacement d k fits perfectly the similarity score Small - otherwise Data potential Regularization Interaction i u i k = l =1: K,l k i l u i k 2 i k 1 u i k 2 Displacement l k has high prob. > low prob. for label k u i k,u j k =w ij u i k u j k 2 w ij =exp J i J j 2 Displacement k has high prob. > high prob. for label k Recall : regularization term (Dirichlet integral) Encourages two neighbors to have similar labels if the image intensity is similar. D[u]= w J 2 u 2 dx D[ u]=u t Lu

RW with priors [following Grady CVPR 2005] E k u k = i N l=1 : K, l k i l u i k 2 i k 1 u i k 2 ij E w ij u i k u j k 2 Combinatorial Laplacian L L ij = d i = k w ik if i= j if i, j E 0 otherwise w ij k =diag k

RW with priors [following Grady CVPR 2005] E k u k = i N l=1: K,l k i l u i k 2 i k 1 u i k 2 ij E w ij u i k u j k 2 Combinatorial Laplacian L L ij = d i = k w ik if i= j if i, j E 0 otherwise w ij k =diag k E k u k = l=1 : K,l k u kt l u k 1 u k T k 1 u k u kt Lu k Minimum when K L l=1 l u k = k Note: The combined matrix still positive semi-definite One equation system per label!!

Augmented graph K L l=1 l u k = k Combinatorial Laplace equation for an augmented graph w ij =exp J i J j 2 Large (1) if neighbors have similar color (allows diffusion)

Augmented graph K L l=1 l u k = k Combinatorial Laplace equation for an augmented graph u 1...... u 1l u K i l =exp I i J id l 2 w ij =exp J i J j 2 Large (1) if Matching color (SSD) Large (1) if neighbors have similar color (allows diffusion)

Implementation Computational cost Need to solve for each displacement label (actually K-1) Size of K L l=1 l u k = k Multi-resolution pyramid Loose global optimality displacements # range ¼ 60 [15,15] ½ 40 [-10,10] 1 30 [-7,7]

Synthetic image + Experiments: Quality of deformations known deformation field Demon's alg. RW 256x256 image 3 levels resol. 160sec (MATLAB) Ang. err. (deg) 1.58 ± 1.20 0.45 ± 0.97 Mag. Err. (deg) 1.12 ± 1.54 1.65 ± 2.89 SSD [0-255] 10.88 5.01

Brain MRI with WM/GM/CSF segmentation source target Reg. Demons SSD=8.82 Reg. RW SSD=3.38 Data from Brain segmentation Repository Dice 0.82 0.79 0.83 0.84 0.81 0.84

Dice 0.62 0.79 Abdominal CT with muscle segmentation source target Data from CCI, Edmonton Reg. Demons SSD=15.46 Reg. RW SSD=9.64

Discussion CONTRIBUTION: New discrete formulation of deformable image registration based on the RW method Image dependent regularization term Global solution with respect to discretization LIMITATIONS AND EXTENSIONS : Computational cost : Solve a large equation system for each label FFD Requirement of a point-wise similarity score Eu= i N i u i i, j E ij u i, u j Possible solution Lower dim models : Compute displacements only at control points Approximate non-local scores in a neighborhood defined by the control points FFD