A Weighted Multivariate Gaussian Markov Model For Brain Lesion Segmentation

Size: px

Start display at page:

Download "A Weighted Multivariate Gaussian Markov Model For Brain Lesion Segmentation"

Pauline Alexander
5 years ago
Views:

1 A Weighted Multivariate Gaussian Markov Model For Brain Lesion Segmentation Senan Doyle, Florence Forbes, Michel Dojat July 5, 2010

2 Table of contents Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

3 Introduction & Background Outline Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

4 Introduction & Background Goal Delineation & Quantification of Lesions Establish Patient Prognosis Chart Development over Time

5 Introduction & Background Goal Delineation & Quantification of Lesions Establish Patient Prognosis Chart Development over Time Automatically!

6 Introduction & Background Goal Delineation & Quantification of Lesions Establish Patient Prognosis Chart Development over Time Automatically!

7 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...)

8 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

9 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

10 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

11 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

12 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

13 Introduction & Background MRI Imaging 3D Volume of Voxels Different MRI Sequences (T1, T2-w, FLAIR, Diffusion Weight...) Cerebro Spinal Fluid (CSF), White Matter (WM), Grey Matter(GM)

14 Introduction & Background Challenges Noise Intensity Inhomogeneity

15 Introduction & Background Challenges Noise Intensity Inhomogeneity Partial Volume Effects

16 Introduction & Background Challenges Noise Intensity Inhomogeneity Partial Volume Effects Other Artefacts

17 Introduction & Background Challenges Noise Intensity Inhomogeneity Partial Volume Effects Other Artefacts Poor Contrast between Tissues

18 Introduction & Background Challenges Noise Intensity Inhomogeneity Partial Volume Effects Other Artefacts Poor Contrast between Tissues Poor Representation of Lesion

19 Introduction & Background Challenges Noise Intensity Inhomogeneity Partial Volume Effects Other Artefacts Poor Contrast between Tissues Poor Representation of Lesion

20 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance

21 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size

22 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size Our approach

23 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size Our approach Adopt a Model so that Lesion Voxels become Inliers

24 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size Our approach Adopt a Model so that Lesion Voxels become Inliers Introduce Weight Field

25 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size Our approach Adopt a Model so that Lesion Voxels become Inliers Introduce Weight Field Bayesian Estimation of Weights

26 Introduction & Background Inliers Versus Outliers Lesions usually modelled as outliers. Widely Varying & Inhomogeneous Appearance Small in Size Our approach Adopt a Model so that Lesion Voxels become Inliers Introduce Weight Field Bayesian Estimation of Weights

27 A Weighted Multi-Sequence Markov Model Outline Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

28 A Weighted Multi-Sequence Markov Model Multi-Sequence PD T1 T2

29 A Weighted Multi-Sequence Markov Model Multi-Sequence Figure: Feature Space, Lesion in Red

30 A Weighted Multi-Sequence Markov Model Multi-Sequence Figure: Feature Space, Lesion in Red

31 A Weighted Multi-Sequence Markov Model Additional Information Spatial Dependency Prior Probabilistic Tissue Atlas

32 A Weighted Multi-Sequence Markov Model Additional Information Spatial Dependency Prior Probabilistic Tissue Atlas

33 A Weighted Multi-Sequence Markov Model Additional Information Spatial Dependency Prior Probabilistic Tissue Atlas

34 A Weighted Multi-Sequence Markov Model Additional Information Spatial Dependency Prior Probabilistic Tissue Atlas

35 A Weighted Multi-Sequence Markov Model Additional Information Spatial Dependency Prior Probabilistic Tissue Atlas

36 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im }

37 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z.

38 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z. Nonnegative Weights ω = {ω i, i V }, ω W, ω i = {ω i1,..., ω im }.

39 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z. Nonnegative Weights ω = {ω i, i V }, ω W, ω i = {ω i1,..., ω im }. Joint distribution p(y, z, ω; ψ), ψ = {β, φ} with H(y, z, ω; ψ) = H Z (z; β) + H W (ω) + i V log g(y i z i, ω i ; φ) Missing Data term

40 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z. Nonnegative Weights ω = {ω i, i V }, ω W, ω i = {ω i1,..., ω im }. Joint distribution p(y, z, ω; ψ), ψ = {β, φ} with H(y, z, ω; ψ) = H Z (z; β) + H W (ω) + i V log g(y i z i, ω i ; φ) Missing Data term Parameter Prior term

41 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z. Nonnegative Weights ω = {ω i, i V }, ω W, ω i = {ω i1,..., ω im }. Joint distribution p(y, z, ω; ψ), ψ = {β, φ} with H(y, z, ω; ψ) = H Z (z; β) + H W (ω) + i V log g(y i z i, ω i ; φ) Missing Data term Parameter Prior term Data term

42 A Weighted Multi-Sequence Markov Model Definitions Regular 3D Grid, i = 1,..., N, i V Observations y i = {y i1,..., y im } Labelling z = {z 1,..., z N }, z i {1... K}, z Z. Nonnegative Weights ω = {ω i, i V }, ω W, ω i = {ω i1,..., ω im }. Joint distribution p(y, z, ω; ψ), ψ = {β, φ} with H(y, z, ω; ψ) = H Z (z; β) + H W (ω) + i V log g(y i z i, ω i ; φ) Missing Data term Parameter Prior term Data term

43 A Weighted Multi-Sequence Markov Model Data term M-Dimensional Gaussians, Diagonal Covariance g(y i z i, ω i ; φ) = M m=1 G(y im ; µ zi m, s z i m ω im )

44 A Weighted Multi-Sequence Markov Model Data term M-Dimensional Gaussians, Diagonal Covariance g(y i z i, ω i ; φ) = M m=1 G(y im ; µ zi m, s z i m ω im ) Related to Logarithmic Pooling ( ) M G(y im ; µ zi m, s zi m) ω im. m=1

45 A Weighted Multi-Sequence Markov Model Data term M-Dimensional Gaussians, Diagonal Covariance g(y i z i, ω i ; φ) = M m=1 G(y im ; µ zi m, s z i m ω im ) Related to Logarithmic Pooling ( ) M G(y im ; µ zi m, s zi m) ω im. m=1

46 A Weighted Multi-Sequence Markov Model Missing Data term Potts Model, external field ξ, interaction parameter η H Z (z; β) = i V (ξ izi + η z i, z j ) j N (i)

47 A Weighted Multi-Sequence Markov Model Missing Data term Potts Model, external field ξ, interaction parameter η H Z (z; β) = i V (ξ izi + η z i, z j ) j N (i) β = {ξ, η} with ξ = { t (ξ i1... ξ ik ), i V } being a set of real-valued K-dimensional vectors and η a real positive value.

48 A Weighted Multi-Sequence Markov Model Missing Data term Potts Model, external field ξ, interaction parameter η H Z (z; β) = i V (ξ izi + η z i, z j ) j N (i) β = {ξ, η} with ξ = { t (ξ i1... ξ ik ), i V } being a set of real-valued K-dimensional vectors and η a real positive value.

49 A Weighted Multi-Sequence Markov Model Parameter Prior term Weights independent of ψ, across Sequences M Define Conjugate Prior p(ω) = p(ω im ) m=1 i V

50 A Weighted Multi-Sequence Markov Model Parameter Prior term Weights independent of ψ, across Sequences M Define Conjugate Prior p(ω) = p(ω im ) m=1 i V p(ω im ) is a Gamma distribution with hyperparameters α im (shape) and γ im (inverse scale) H W (ω) = M ((α im 1) log ω im γ im ω im ). m=1 i V

51 A Weighted Multi-Sequence Markov Model Parameter Prior term Weights independent of ψ, across Sequences M Define Conjugate Prior p(ω) = p(ω im ) m=1 i V p(ω im ) is a Gamma distribution with hyperparameters α im (shape) and γ im (inverse scale) H W (ω) = M ((α im 1) log ω im γ im ω im ). m=1 i V Modes fixed to expert weights {ω exp im, m = 1... M, i V } by setting α im = γ im ω exp im + 1

52 A Weighted Multi-Sequence Markov Model Parameter Prior term Weights independent of ψ, across Sequences M Define Conjugate Prior p(ω) = p(ω im ) m=1 i V p(ω im ) is a Gamma distribution with hyperparameters α im (shape) and γ im (inverse scale) H W (ω) = M ((α im 1) log ω im γ im ω im ). m=1 i V Modes fixed to expert weights {ω exp im, m = 1... M, i V } by setting α im = γ im ω exp im + 1

53 Variational Bayesian EM Solution Outline Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

54 Variational Bayesian EM Solution Alternating Maximization Weights viewed as Missing Variables Let D be the set of all probability distributions on Z W.

55 Variational Bayesian EM Solution Alternating Maximization Weights viewed as Missing Variables Let D be the set of all probability distributions on Z W. Alternating maximization of a function F s.t. for any q D, F (q, ψ) = E q [log p(y, Z, W ; ψ)] E q [log q(z, W )]

56 Variational Bayesian EM Solution Alternating Maximization Weights viewed as Missing Variables Let D be the set of all probability distributions on Z W. Alternating maximization of a function F s.t. for any q D, F (q, ψ) = E q [log p(y, Z, W ; ψ)] E q [log q(z, W )] E-step: q (r) = arg max q D F (q, ψ(r) ) (1) M-step: ψ (r+1) = arg max ψ Ψ F (q(r), ψ)

57 Variational Bayesian EM Solution Alternating Maximization Weights viewed as Missing Variables Let D be the set of all probability distributions on Z W. Alternating maximization of a function F s.t. for any q D, F (q, ψ) = E q [log p(y, Z, W ; ψ)] E q [log q(z, W )] E-step: q (r) = arg max q D F (q, ψ(r) ) (1) M-step: ψ (r+1) = arg max ψ Ψ F (q(r), ψ)

58 Variational Bayesian EM Solution Alternating Maximization Weights viewed as Missing Variables Let D be the set of all probability distributions on Z W. Alternating maximization of a function F s.t. for any q D, F (q, ψ) = E q [log p(y, Z, W ; ψ)] E q [log q(z, W )] E-step: q (r) = arg max q D F (q, ψ(r) ) (1) M-step: ψ (r+1) = arg max ψ Ψ F (q(r), ψ)

59 Variational Bayesian EM Solution Alternating Maximization E-step solved over restricted class of probability distributions, D, s.t. q(z, ω) = q Z (z) q W (ω) where q Z D Z and q W D W E-Z-step: q (r) Z E-W-step: q (r) W = arg max F (q (r 1) q Z D W q Z ; ψ (r) ) Z = arg max q W D W F (q W q (r) Z ; ψ(r) ).

60 Variational Bayesian EM Solution Alternating Maximization E-step solved over restricted class of probability distributions, D, s.t. q(z, ω) = q Z (z) q W (ω) where q Z D Z and q W D W E-Z-step: q (r) Z E-W-step: q (r) W = arg max F (q (r 1) q Z D W q Z ; ψ (r) ) Z = arg max q W D W F (q W q (r) Z ; ψ(r) ).

61 Variational Bayesian EM Solution Alternating Maximization Use K-L divergence properties to avoid taking derivatives ( E-Z: q (r) Z exp E q (r 1) W ) [log p(z y, W; ψ (r) ] ) E-W: q (r) W (E exp (r) q [log p(ω y, Z; ψ (r) )] M: ψ (r+1) = arg max Z E ψ Ψ q (r) Z q(r) W (2) (3) [log p(y, Z, W ; ψ)]. (4)

62 Variational Bayesian EM Solution Alternating Maximization Use K-L divergence properties to avoid taking derivatives ( E-Z: q (r) Z exp E q (r 1) W ) [log p(z y, W; ψ (r) ] ) E-W: q (r) W (E exp (r) q [log p(ω y, Z; ψ (r) )] M: ψ (r+1) = arg max Z E ψ Ψ q (r) Z q(r) W (2) (3) [log p(y, Z, W ; ψ)]. (4)

63 Variational Bayesian EM Solution E-Z Note p(z y, ω; ψ) is Markovian with energy H(z y, ω; ψ) H Z (z; β) + i V Therefore (2) becomes log g(y i z i, ω i ; φ), (linear in ω) (5) E-Z: q (r) Z p(z y, E (r 1) q [W ]; ψ (r) ). (6) W

64 Variational Bayesian EM Solution E-Z Note p(z y, ω; ψ) is Markovian with energy H(z y, ω; ψ) H Z (z; β) + i V Therefore (2) becomes log g(y i z i, ω i ; φ), (linear in ω) (5) E-Z: q (r) Z p(z y, E (r 1) q [W ]; ψ (r) ). (6) W Mean-Field like algorithm provides MRF approximation and q (r) Z (z) i V M G(y im ; µ (r) z i m, m=1 s (r) z i m ω (r 1) im )p(z i z (r) N (i) ; β(r) ), (7)

65 Variational Bayesian EM Solution E-Z Note p(z y, ω; ψ) is Markovian with energy H(z y, ω; ψ) H Z (z; β) + i V Therefore (2) becomes log g(y i z i, ω i ; φ), (linear in ω) (5) E-Z: q (r) Z p(z y, E (r 1) q [W ]; ψ (r) ). (6) W Mean-Field like algorithm provides MRF approximation and q (r) Z (z) i V M G(y im ; µ (r) z i m, m=1 s (r) z i m ω (r 1) im )p(z i z (r) N (i) ; β(r) ), (7)

66 Variational Bayesian EM Solution E-W Similarly p(ω y, z; ψ) is Markovian with energy H(ω y, z; ψ) H W (ω) + i V log g(y i z i, ω i ; φ) (8) E (r) q [W im ] denoted by ω (r) im becomes W ω (r) im = α im γ im + 1 K 2 k=1 δ(y im, µ (r) km, s(r) km ) q(r) Z i (k) (9) where δ(y, µ, s) = (y µ) 2 /s is the squared Mahalanobis distance.

67 Variational Bayesian EM Solution E-W Similarly p(ω y, z; ψ) is Markovian with energy H(ω y, z; ψ) H W (ω) + i V log g(y i z i, ω i ; φ) (8) E (r) q [W im ] denoted by ω (r) im becomes W ω (r) im = α im γ im + 1 K 2 k=1 δ(y im, µ (r) km, s(r) km ) q(r) Z i (k) (9) where δ(y, µ, s) = (y µ) 2 /s is the squared Mahalanobis distance.

68 Variational Bayesian EM Solution M β solved using mean field approximation φ updated using µ (r+1) km = s (r+1) km = N i=1 q(r) Z i N i=1 q(r) Z i N i=1 q(r) Z i (k) ω (r) im y im (k) ω (r) im (k) ω (r) im (y im µ (r+1) N i=1 q(r) Z i (k), km )2,

69 Variational Bayesian EM Solution M β solved using mean field approximation φ updated using µ (r+1) km = s (r+1) km = N i=1 q(r) Z i N i=1 q(r) Z i N i=1 q(r) Z i (k) ω (r) im y im (k) ω (r) im (k) ω (r) im (y im µ (r+1) N i=1 q(r) Z i (k), km )2,

70 Lesion Segmentation Procedure Outline Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

71 Lesion Segmentation Procedure Expert Weighting Region L defines candidate lesions L weighted with ω L > 1 (ω \L = 1)

72 Lesion Segmentation Procedure Expert Weighting Region L defines candidate lesions L weighted with ω L > 1 (ω \L = 1) K = 3, ω exp im = 1 and γ im = 1 setting

73 Lesion Segmentation Procedure Expert Weighting Region L defines candidate lesions L weighted with ω L > 1 (ω \L = 1) K = 3, ω exp im = 1 and γ im = 1 setting Threshold Most Informative Weight Map (chi-square percentile)

$\L = 1) K = 3, ω exp im = 1 and γ im = 1 setting Threshold Most Informative Weight Map (chi-square$

74 Lesion Segmentation Procedure Expert Weighting Region L defines candidate lesions L weighted with ω L > 1 (ω \L = 1) K = 3, ω exp im = 1 and γ im = 1 setting Threshold Most Informative Weight Map (chi-square percentile)

$\L = 1) K = 3, ω exp im = 1 and γ im = 1 setting Threshold Most Informative Weight Map (chi-square$

75 Lesion Segmentation Procedure Expert Weighting Region L defines candidate lesions L weighted with ω L > 1 (ω \L = 1) K = 3, ω exp im = 1 and γ im = 1 setting Threshold Most Informative Weight Map (chi-square percentile)

76 Lesion Segmentation Procedure Inlier Setting K = 4, initialized with K = 3 result (+ thresholded lesions) γ im = γ L i L (γ L = 10)

77 Lesion Segmentation Procedure Inlier Setting K = 4, initialized with K = 3 result (+ thresholded lesions) γ im = γ L i L (γ L = 10) γ im = γ L i L. (γ L = 1000)

78 Lesion Segmentation Procedure Inlier Setting K = 4, initialized with K = 3 result (+ thresholded lesions) γ im = γ L i L (γ L = 10) γ im = γ L i L. (γ L = 1000) ω L depends on number of voxels in L

79 Lesion Segmentation Procedure Inlier Setting K = 4, initialized with K = 3 result (+ thresholded lesions) γ im = γ L i L (γ L = 10) γ im = γ L i L. (γ L = 1000) ω L depends on number of voxels in L

80 Lesion Segmentation Procedure Inlier Setting

81 Lesion Segmentation Procedure Inlier Setting

82 Results Outline Introduction & Background A Weighted Multi-Sequence Markov Model Variational Bayesian EM Solution Lesion Segmentation Procedure Results

83 Results Simulated Data 0% IIH Method 3% 5% 7% 9% Mild lesions (0.02% of the voxels) AWEM 68 (+1) 49 (-21) 36 (+2) 12 (+8) [1] [3] [2] 52 NA NA NA Moderate lesions (0.18% of the voxels) AWEM 86 (+7) 80 (-1) 73 (+14) 64 (+27) [1] [3] [2] 63 NA NA NA Severe lesions (0.52% of the voxels) AWEM 92 (+7) 86 (-2) 78 (+6) 68 (+27) [1] [3] [2] 82 NA NA NA

84 Results Simulated Data 40% IIH Method 3% 5% 7% 9% Mild lesions (0.02% of the voxels) AWEM 0 (-75) 0 (-65) 0 (-20) 0 (-30) [1] [3] Moderate lesions (0.18% of the voxels) AWEM 52 (-24) 51 (-25) 52 (-15) 10 (-38) [1] [3] Severe lesions (0.52% of the voxels) AWEM 87 (+1) 84 (+1) 77 (+3) 66 (+8) [1] [3]

85 Results Real Data (Rennes MS) LL EMS AWEM Patient (+20) Patient (+2) Patient (-2) Patient (+7) Patient (-2) Average 55 +/ /-16

86 Results Results (a) (b) (c) Figure: Real MS data, patient 3. (a): Flair image. (b): identified lesions with our approach (DSC 45%). (c): ground truth.

87 Results Results

88 Results Results

89 Results Results (a) (b) (c) Figure: Real stroke data. (a): DW image. (b): identified lesions with our approach (DSC 63%). (c): ground truth.

90 Results Discussion & Future Work Extension to full covariance matrices: temporal multi-sequence data, eg. patient follow-up Exploration of Markov Prior Other expert weighting schemes, possibly lesion specific Extension to handle intensity inhomogeneities Evaluation in a semi-supervised context

91 Results D. Garcia-Lorenzo, L. Lecoeur, D.L. Arnold, D. L. Collins, and C. Barillot. Multiple Sclerosis lesion segmentation using an automatic multimodal graph cuts. In MICCAI, pages , F. Rousseau, F. Blanc, J. de Seze, L. Rumbac, and J.P. Armspach. An a contrario approach for outliers segmentation: application to multiple sclerosis in MRI. In IEEE ISBI, pages 9 12, K. Van Leemput, F. Maes, D. Vandermeulen, A. Colchester, and P. Suetens. Automated segmentation of multiple sclerosis lesions by model outlier detection. IEEE trans. Med. Ima., 20(8): , 2001.

Brain Lesion Segmentation: A Bayesian Weighted EM Approach

Brain Lesion Segmentation: A Bayesian Weighted EM Approach Senan Doyle, Florence Forbes, Michel Dojat November 19, 2009 Table of contents Introduction & Background A Weighted Multi-Sequence Markov Model