Segmentation using deformable models

Transcription

1 Segmentation using deformable models Isabelle Bloch bloch Téécom ParisTech - CNRS UMR 5141 LTCI Paris - France Modèles déformables p.1/43

2 Introduction Deformable models = evolution of curves / surfaces to minimize an energy function for finding the best image partition into homogeneous regions finding the contours of an object Criteria on contours region homogeneity regularity other types of constraints Important distinctions: parametric / geometric (implicit) contour-based / region-based Modèles déformables p.2/43

3 Parametric active contours First work: Kass, Witkins and Terzopoulos in 1988 Principle: evolution of a curve under internal and external forces v(s) = [x(s),y(s)] t s [0,1] Energy: E total = 1 0 [E int (v(s)) + E image (v(s)) + E ext (v(s))]ds Internal energy: ( dv E int = α(s) ds ) 2 ( d 2 ) 2 v + β(s) ds 2 control of tension (length of the curve) and of curvature (T = dv ds, dt ds = kn) Energy from image information (gradient): E image = f External energy: many possibilities Modèles déformables p.3/43

4 Influence of parameters: example Deformation of an active contour with fixed end-points for α/β equal to 2, 3, 4 (without image energy). Modèles déformables p.4/43

5 Solving the minimization problem Variational formulation: Euler-Lagrange equation (αv ) (s)+(βv ) (s)+ P(v) = 0 + limit conditions Discretization using finite differences P(v) = E image (v)+e ext (v) F(v) = P(v) V t = [v t 0, v t 1, v t 2,..., v t n 1] t β h 2v i+2 ( α h +4 β h 2)v i+1 +( 2α h +46β h 2)v i ( α h +4 β h 2)v i 1 + β h 2v i 2 = F(v i ) (A: penta-diagonal matrix) AV = F Modèles déformables p.5/43

6 Curve evolution γ v t αv +βv = F(v) v(t+1) = (τa+i) 1 (τf(v(t))+v(t)) initialization (crucial for convergence) choix de τ (inertia) matrix inversion constant discretization step for t (set to have a displacement of 1-2 pixels) stopping criterion Modèles déformables p.6/43

7 Types of active contours actifs and corresponding matrices Closed active contour 2α+6β α 4β β 0 0 β... α 4β... α 4β 2α+6β α 4β β 0... β α 4β 2α+6β α 4β β 0 β α 4β 2α+6β α 4β β α 4β 2α+6β α 4β β β α 4β 2α+6β Modèles déformables p.7/43

8 Types of active contours actifs and corresponding matrices Free end-points β β -2 β α 2β 2α+5β α 4β β 0... β α 4β 2α+6β α 4β β 0 β α 4β 2α+6β α 4β β α 4β 2α+6β β α 4β β 2α+5β -2 β α 2β... β (with v (0) = v (1) = v (0) = v (1) = 0) Modèles déformables p.7/43

9 Types of active contours actifs and corresponding matrices Fixed end-points 2α+6β α 4β β α 4β 2α+6β α 4β β 0... β α 4β 2α+6β α 4β β 0 β α 4β 2α+6β α 4β 0 0 β α 4β 2α+6β β -2 β... β Modèles déformables p.7/43

10 Example Modèles déformables p.8/43

11 Example Modèles déformables p.8/43

12 Evolution example Evolution Modèles déformables p.9/43

13 Optimization using dynamic programming Amini, P Q M N positions à l instant t positions possibles à t+1 snake à t snake à t+1 M N P Q Modèles déformables p.10/43

14 Balloon force (Cohen, 1991) Problem: bad initialization no attraction no forces curve collapses Solution using a pression force (balloon): n(s) unit normal vector at s F(v(s)) = k 1 n(s) k P P Initialization: inside or outside the object (not necessarily very close to the searched object) Remark: the force does not derive anymore from a potential Modèles déformables p.11/43

15 Distance constraint Distance map D(x, y) potential P dist (x,y) = we D(x,y) F ext = P dist Modèles déformables p.12/43

16 Gradient vector flow (Xu and Prince, 1997) Objectives: get rid of the constraints on the initialization convergence towards concave regions Diffusion of gradients in the whole image For v (x,y) = (u(x,y),v(x,y)), minimization of: E = µ(u 2 x +u2 y +v2 x +v2 y )+ f(x,y) 2 v f(x,y) 2 dxdy f = contour map Modèles déformables p.13/43

17 GVF: solving and generalizing µ 2 u (u f x )(f 2 x +f 2 y) = 0 More general formulation: Examples for functions g and h: g(r) = µ, h(r) = r 2 g(r) = exp( r 2 k µ 2 v (v f y )(f 2 x +f 2 y) = 0 v t = g( f ) 2 v h( f )(v f) ), h(r) = 1 g(r) v(x,y,0) = f(x,y) Modèles déformables p.14/43

18 Classical active contour: illustration Final result, iter = Modèles déformables p.15/43

19 Balloon force Modèles déformables p.16/43

20 Balloon force Modèles déformables p.16/43

21 GVF Modèles déformables p.17/43

22 GVF Modèles déformables p.17/43

23 GVF: evolution example Evolution Modèles déformables p.18/43

24 GVF: example in medical imaging Modèles déformables p.19/43

25 GVF: example in medical imaging Modèles déformables p.19/43

26 Extension to 3 dimensions Segmentation with regularization: data fidelity smooth surface Minimization of energy: E(v) = Ω w 10 v r 2 +w 01 v s 2 +w 20 2 v r 2 2 +w 02 2 v s w 11 2 v r s 2 drds + P(v)drds Ω first order: elastic membrane (curvature) second order: thin plate (torsion) P : attraction potential Modèles déformables p.20/43

27 Iterative solving scheme (I +τa)v t = (v t 1 +τf(v t 1 )) v t = unknowns at iteration t : vertices (finite differences) freedom degrees (finite elements) τf(v t 1 ): force inversion of (I +τa): regularization Difficulties (as in 2D): choice of time step τ (evolution speed) choice of potentials initialization minimization (local minima) Modèles déformables p.21/43

28 Examples of potentials Gradient: P(v) = I(v) 2 Attraction towards given contours: convolution distance Expansion: evolution of curves/surfaces Temporal regularization etc. F(v) = k P P (v) F(v) = k 1 n k P P (v) Modèles déformables p.22/43

29 Non parametric contours using level sets Principle: Let Γ(t) be a closed hypersurface (dimension N 1) Let ψ (dimension N) be a function taking values in R with Γ(t) = { X R N /ψ( X,t) = 0} propagation of Γ propagation of ψ Example: distance function Evolution equation of ψ ψ t = F ψ Modèles déformables p.23/43

30 Additional dimension: distance Modèles déformables p.24/43

31 Level sets and change of topology Example Modèles déformables p.25/43

32 Example: segmentation of the skull in 3D MRI Modèles déformables p.26/43

33 Example: segmentation of the skull in 3D MRI Modèles déformables p.26/43

34 Example: segmentation of the skull in 3D MRI Modèles déformables p.26/43

35 Example: segmentation of the skull in 3D MRI Modèles déformables p.26/43

36 Example: segmentation of the skull in 3D MRI Modèles déformables p.26/43

37 Example: segmentation of the skull in 3D MRI Evolution Result Modèles déformables p.26/43

38 Geodesic active contours (Caselles, 1997) Principle J 1 (v) = b a v (s) 2 ds+λ b a g( i(v(s)) ) 2 ds J 2 (v) = 2 λα b a v (s) g( i(v(s)) )ds to be minimized computation of the geodesics of a new metric, induced by the image Evolution equation v t = g(i)κ n ( g. n) n Using level sets ψ t = g( (I) ) ψ κ+ g. ψ Modèles déformables p.27/43

39 Geodesic active contours: example Evolution Modèles déformables p.28/43

40 Region-based approach: Mumford and Shah Image f Approximation by smooth functions gi (x,y) on regions R i limited by contours Γ j : Γ R Γ R 4 R 1 Γ 1 R 3 Γ 4 Functional to be minimized: U(Γ,g,f) = µ 2 I (f(x,y) g i (x,y)) 2 dxdy +λ g i (x,y) 2 dxdy +ν R i Γ i dl Modèles déformables p.29/43

41 Mumford and Shah: some particular cases gi = ±1: Ising model gi = constant on each R i : U 0 (Γ,f) = 1 µ 2U(Γ,f,g) = i R i (f g i ) 2 dxdy +ν Γ i dl g i = 1 s i R i f(x,y)dxdy where s i represents the surface of R i U (Γ) = [ ( ) ] f 2 ν dl n Γ where ν = constant and f n = component of the gradient of f normal to Γ = geodesic problem: Γ i of minimal length and maximal transverse gradient Modèles déformables p.30/43

42 Formulation using level sets (1) Contour Γ Two regions R1 = int(γ) and R 2 = ext(γ) with constant values g 1 and g 2 U(Γ,g,f) = µ 2 (f g 1 ) 2 dxdy +µ 2 (f g 2 ) 2 dxdy +ν R 1 R 2 Γ dl Level set: φ(x, y) = 0 on Γ > 0 in R 1 = int(γ) < 0 in R 2 = ext(γ) Γ(t) = {φ(t) = 0} Evolution (mean curvature motion): φ t = φ ( φ φ ) φ(0) = φ 0 Modèles déformables p.31/43

43 Formulation with level sets (2) H(z) = { 1 if z 0 0 if z < 0 δ(z) = H (z) Γ dl = I H(φ) dxdy = I δ(φ) φ dxdy U(Γ,g,f) = µ 2 (f g 1 ) 2 H(φ)dxdy+µ 2 I I (f g 2 ) 2 (1 H(φ))dxdy+ν I δ(φ) φ dxdy g = g 1 H(φ)+g 2 (1 H(φ)) Modèles déformables p.32/43

44 Solving via Euler-Lagrange equations Minimization of U: g 1 = I fh(φ)dxdy I H(φ)dxdy g 2 = I f(1 H(φ))dxdy I (1 H(φ))dxdy φ t = δ(ϕ)[ν ( φ φ ) µ2 (f g 1 ) 2 +µ 2 (f g 2 ) 2 ] initialization / reinitialization solving using narrow band or fast marching approaches Modèles déformables p.33/43

45 Multi-phase models T. Chan and L. Vese (2002) Segmentation of a MRI image with two level sets Modèles déformables p.34/43

46 Multi-phase models T. Chan and L. Vese (2002) Example with junctions Modèles déformables p.34/43

47 Texture segmentation Modèles déformables p.35/43

48 Texture segmentation Modèles déformables p.35/43

49 Introduction of spatial relations Modèles déformables p.36/43

50 Examples of spatial representations of relations close to below Modèles déformables p.37/43

51 Deformable model simplex mesh evolution using both contour and region information: Internal force: F int = α 2 X β 2 ( 2 X) γ X t = F int(x)+f ext (X) External force: F ext = λv +νf R v: Gradient Vector Flow (GVF) FR : force derived from spatial relations v t = g( f ) 2 v h( f )(v f) Modèles déformables p.38/43

52 Force derived from spatial relations: example Modèles déformables p.39/43

53 Results Modèles déformables p.40/43

54 Results (a few slices) Modèles déformables p.41/43

55 Importance of spatial relations Modèles déformables p.42/43

56 Homogeneity from intensity distributions example in 3D US fetal imaging Modèles déformables p.43/43

57 Homogeneity from intensity distributions example in 3D US fetal imaging Modèles déformables p.43/43

58 Homogeneity from intensity distributions example in 3D US fetal imaging Modèles déformables p.43/43

59 Homogeneity from intensity distributions example in 3D US fetal imaging Modèles déformables p.43/43

60 Homogeneity from intensity distributions example in 3D US fetal imaging Modèles déformables p.43/43