COMPUTATIONAL METHODS IN MRI: MATHEMATICS

Transcription

1 COMPUTATIONAL METHODS IN MATHEMATICS Imaging Sciences-KCL November 20, 2008

2 OUTLINE 1 MATRICES AND LINEAR TRANSFORMS: FORWARD

3 OUTLINE 1 MATRICES AND LINEAR TRANSFORMS: FORWARD 2 LINEAR SYSTEMS: INVERSE PROBLEMS

4 OUTLINE 1 MATRICES AND LINEAR TRANSFORMS: FORWARD 2 LINEAR SYSTEMS: INVERSE PROBLEMS 3 JACOBIANS

5 MATRICES Matrix: table of numbers; array; used to store data.

6 MATRICES Matrix: table of numbers; array; used to store data. Matrix size is M (rows) times N columns

7 MATRICES Matrix: table of numbers; array; used to store data. Matrix size is M (rows) times N columns If M = 1 or N = 1: vector; used to store coordinates.

8 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis [ 2.3 ]

9 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis [ 2.3 ]

10 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis

11 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis 2 [ 2.3 ]

12 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis 2 [ 2.3 ]

13 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis [ ] b11 b B = 12 b 21 b 22 A

14 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis [ ] b11 b B = 12 b 21 b 22 A

15 MATRIX OPERATIONS: SUM AND SCALAR PRODUCTS geometric basis [ ] a11 +b A+B = 11 a 12 +b 12 a 21 +b 21 a 22 +b 22

16 LINEAR OPERATIONS: MATRIX PRODUCTS Operation f is linear iff f(αa+βb) = αf(a)+βf(b)

17 LINEAR OPERATIONS: MATRIX PRODUCTS Operation f is linear iff f(αa+βb) = αf(a)+βf(b)

18 LINEAR OPERATIONS: MATRIX PRODUCTS Operation f is linear iff f(αa+βb) = αf(a)+βf(b)

19 LINEAR OPERATIONS: MATRIX PRODUCTS Operation f is linear iff f(αa+βb) = αf(a)+βf(b)

20 LINEAR OPERATIONS: MATRIX PRODUCTS [ ] cos(θ) sin(θ) A = sin(θ) cos(θ) Operation f is linear iff f(αa+βb) = αf(a)+βf(b) Transformation of coordinates by a matrix represents linear transforms: write Ax

21 MATRIX PRODUCTS =

22 MATRIX PRODUCTS P N N P M M =

23 MATRIX PRODUCTS A(m,:) B(:,n) = (AB)(m,n)

24 MATRIX PRODUCTS A mk B kn k A mk B kn =

25 MATRIX PRODUCTS =

26 MATRIX PRODUCTS =

27 MATRIX PRODUCTS = + +

28 LINEAR OPERATIONS: DIMENSIONS :

29 LINEAR OPERATIONS: DIMENSIONS :

30 LINEAR OPERATIONS: DIMENSIONS :

31 LINEAR OPERATIONS: DIMENSIONS?! :

32 LINEAR OPERATIONS: DIMENSIONS

33 LINEAR OPERATIONS: DIMENSIONS Entire line maps to a point. 2 1 [ ]

34 EXAMPLE I: FOURIER TRANSFORM S(k) S(k j ) = n e 2πix nk j s(x n )

35 EXAMPLE I: FOURIER TRANSFORM S(k) S(k j ) = n e 2πix nk j s(x n )

36 EXAMPLE I: FOURIER TRANSFORM S(k) S(k j ) = n F jn s(x n )

37 EXAMPLE II: CONVOLUTION c(x m ) Convolution is another example of linear transform (s c)(x m ) = n c(x m x n )s(x n )

38 EXAMPLE II: CONVOLUTION c(x m ) Convolution is another example of linear transform (s c)(x m ) = n c(x m x n )s(x n )

39 EXAMPLE II: CONVOLUTION c(x m x n ) Convolution is another example of linear transform (s c)(x m ) = n c(x m x n )s(x n ) Matrix is C nm :=c(x m x n ). (Circulant matrix)

40 COMPUTATIONAL ASPECTS Matlab commands: *, kron,.*, +, - BLAS: Level 1 (vec-vec): x αx+y Level 2 (mat-vec): x αax+βy Level 3 (mat-mat): A αab+βc (available as: netlib, GSL, ATLAS, Goto, CUDA, Sunperf, Intel MKL, AMD ACML) Example:

41 BLAS DGEMV SUBROUTINE DGEMV(TRANS,M,N,ALPHA,A,LDA,X,INCX,BETA,Y,INCY) x αax+βy * On entry, TRANS specifies the operation to be performed as * follows: * TRANS = N or n y := alpha*a*x + beta*y. * TRANS = T or t y := alpha*a *x + beta*y. * TRANS = C or c y := alpha*a *x + beta*y. * Unchanged on exit.

42 BLAS DGEMV SUBROUTINE DGEMV(TRANS,M,N,ALPHA,A,LDA,X,INCX,BETA,Y,INCY) x αax+βy * On entry, M specifies the number of rows of the matrix A. * M must be at least zero. * Unchanged on exit. * * N - INTEGER. * On entry, N specifies the number of columns of the matrix A. * N must be at least zero. * Unchanged on exit.

43 INVERSE PROBLEMS problem: compute y = Ax given A, x. Physically: image acquisition (or simulation).

44 INVERSE PROBLEMS problem: compute y = Ax given A, x. Physically: image acquisition (or simulation). problem: compute x such that y = Ax: image reconstruction.

45 INVERSE PROBLEMS problem: compute y = Ax given A, x. Physically: image acquisition (or simulation). problem: compute x such that y = Ax: image reconstruction.

46 INVERSE PROBLEMS problem: compute y = Ax given A, x. Physically: image acquisition (or simulation). problem: compute x such that y = Ax: image reconstruction. Now list of some possible strategies. Possible only if M N.

47 I: DIRECT INVERSION Find B such that BA =I. Then we know that BAx =By. B :=A 1. Example: θ (x +iy)e iθ x +iy [ ] cosθ sinθ R = sinθ cosθ yxr

48 I: DIRECT INVERSION Find B such that BA =I. Then we know that BAx =By. B :=A 1. Example: θ (x +iy)e iθ x +iy [ ] cosθ sinθ R = sinθ cosθ Here D = 2. High dimensional example: F F =I for Fourier. yxr

49 I: DIRECT INVERSION Find B such that BA =I. Then we know that BAx =By. B :=A 1. Example: θ (x +iy)e iθ x +iy [ ] cosθ sinθ R = sinθ cosθ yxr Here D = 2. High dimensional example: F F =I for Fourier. Cost: in general high (mem and speed), unless B =A t (rotation) etc Matlab: inv

50 II: DIAGONALISATION Find U,D =diag(λ) such that U 1 AU =D,A =UDU 1. NB: A 1 =Udiag(1./λ)U 1 ex = exp(-(x.*x + Y.*Y)/128; ; Cost: in general very high, unless we can find U 1 =U t Matlab: diag.

51 II: DIAGONALISATION Find U,D =diag(λ) such that U 1 AU =D,A =UDU 1. NB: A 1 =Udiag(1./λ)U 1 ex = exp(-(x.*x + Y.*Y)/128; cr = ifft2(fft2(r).*fft2(ex)); ; Cost: in general very high, unless we can find U 1 =U t Matlab: diag.

52 II: DIAGONALISATION Find U,D =diag(λ) such that U 1 AU =D,A =UDU 1. NB: A 1 =Udiag(1./λ)U 1 ex = exp(-(x.*x + Y.*Y)/128; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex)); Cost: in general very high, unless we can find U 1 =U t Matlab: diag.

53 III: TRIANGULAR DECOMPOSITION: LU Find L,U such that A =LU Solve Ax =y by solving Lz =y and Ux =z 0 0 x = y

54 III: TRIANGULAR DECOMPOSITION: LU Find L,U such that A =LU Solve Ax =y by solving Lz =y and Ux =z 0 0 x = y Matlab default \ (lu) Cost moderate. (Default for M such that M

55 IV: ITERATIVE METHODS Generate a sequence of x k which converge to x.

56 IV: ITERATIVE METHODS Gradient descent: Ellipses are level lines (iso-contours) for x t Ax x t y

57 IV: ITERATIVE METHODS Gradient descent: Ellipses are level lines (iso-contours) for x t Ax x t y

58 IV: ITERATIVE METHODS Conjugate gradient: correct for distortion by A

59 IV: ITERATIVE METHODS Conjugate gradient: correct for distortion by A

60 IV: ITERATIVE METHODS Conjugate gradient: correct for distortion by A

61 IV: ITERATIVE METHODS Conjugate gradient: correct for distortion by A low

62 BENCHMARK Test your own machinebench in Matlab. NB: LINPACK test used forhttp://

63 BENCHMARK Test your own machinebench in Matlab. NB: LINPACK test used 80 0 forhttp:// for k=100:100:10000; A=randn(k,k); b = randn(k,1); tic; A\b; t(k) = toc; end

64 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/128; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

65 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/128; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

66 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/064; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

67 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/064; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

68 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/032; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

69 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/032; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

70 NON-REGULAR PROBLEMS Deconvolution: Operator C diagonalised by FT: F CF =diag(c) But this usually implies division by zero (or very small number) (coil sensitivities are zero in some places) Makes the problem ill-posed. ex = exp(-(x.*x + Y.*Y)/032; cr = ifft2(fft2(r).*fft2(ex)); dcr = ifft2(fft2(cr)./fft2(ex));

71 SVD Problem with deconvolution: division by zero! SVD generalises this statement to all matrices! A =USV t where U,V are orthogonal, S rectangular-diagonal. Condition number is ratio of non-zero diagonal elements. 1 is good, quality of numerical solutions degrades with condition going up. 0

72 NUMBER OF EQS. NUMBER OF UNKNOWNS If matrices are not square, they can t be inverted. If M >N (more eqs than unknown). In general no exact solution. It makes sense to pick the closest: ˆx == argmin (Ax) n y n 2 =: argmin Ax y 2 2. If M <N, infinity of solutions. We need a way to pick one. Usually: pick x which minimises some condition.

73 V: PSEUDO-INVERSE (M >N) Equivalent to min Ax y normal equation A t Ax =A t y Pseudo-inverse (A t A) 1 A t y y

74 V: PSEUDO-INVERSE (M >N) Equivalent to min Ax y normal equation A t Ax =A t y Pseudo-inverse (A t A) 1 A t y y

75 V: PSEUDO-INVERSE (M >N) A\y Equivalent to min Ax y normal equation A t Ax =A t y Pseudo-inverse (A t A) 1 A t y y

76 EXAMPLE: SENSE y = subsample(fx)

77 EXAMPLE: SENSE x 1 :Ax 1 =y y =Ax

78 EXAMPLE: SENSE x 1 :Ax 1 =y y =Ax x 2 :Ax 2 =y

79 EXAMPLE: SENSE x 1 :Ax 1 =y y =Ax x 2 :Ax 2 =y x 3 :Ax 3 =y

80 EXAMPLE: SENSE x 1 :Ax 1 =y y =Ax x 2 :Ax 2 =y x 3 :Ax 3 =y x 4 :Ax 4 =y

81 EXAMPLE: SENSE x 1 :Ax 1 =y y =Ax x 2 :Ax 2 =y A(tx 0 +(1 t)x) x 3 :Ax 3 =y x 4 :Ax 4 =y

82 UNDERSAMPLING Undersampling in k-space means aliasing. For example, factor 2: s(y) = 1 (s(x)+s(x +F/2)). 2 Write x 1 :=x x 2 :=x+f/2.

83 UNDERSAMPLING Undersampling in k-space means aliasing. For example, factor 2: s(y) = 1 (s(x)+s(x +F/2)). 2 Write x 1 :=x x 2 :=x+f/2. In other words, we have 2 unknowns, but only 1 equation.

84 Parallel MR: multiple coils measure weighted signals: s i (x) =c i (x)s(x).

85 SENSE Combined with undersampling: s 1 = 1 2 ( s(x 1) + s(x 2 )) s 2 = 1 2 ( s(x 1) + s(x 2 ))

86 SENSE Combined with undersampling: s 1 = 1 2 (c 1(x 1 )s(x 1 ) +c 1 (x 2 )s(x 2 )) s 2 = 1 2 (c 2(x 1 )s(x 1 ) +c 2 (x 2 )s(x 2 )) We can solve for s(x 1 ),s(x 2 )

87 SENSE Combined with undersampling: s 1 = 1 2 (c 1(x 1 )s(x 1 ) +c 1 (x 2 )s(x 2 )) s 2 = 1 2 (c 2(x 1 )s(x 1 ) +c 2 (x 2 )s(x 2 )) We can solve for s(x 1 ),s(x 2 )if the matrix with c nm =c n (x m ) is well conditi oned

88 NORMS x 2 Euclidean norm, L 2, SOS, relevant in physical problems (energy), etc x 1 sum of absolute values, useful for robust estimation in statistics x p := ( n x n p ) 1/p x 0 number of nonzeros x =max x n

89 NORMS x 2 Euclidean norm, L 2, SOS, relevant in physical problems (energy), etc x 1 sum of absolute values, useful for robust estimation in statistics x p := ( n x n p ) 1/p x 0 number of nonzeros x =max x n P-norms unit circles for p=0.5,1,1.5,2,2.5 (from inside to outside)

90 REGULARISATION Problem: small (or even 0!) singular values. Regularisation:= do something about small singular values-choose a solution among the many possibles Tikhonov: min Ax y +α x

91 REGULARISATION Problem: small (or even 0!) singular values. Regularisation:= do something about small singular values-choose a solution among the many possibles Tikhonov: min Ax y +α x Minimum norm min x given Ax =y (see below).

92 REGULARISATION Problem: small (or even 0!) singular values. Regularisation:= do something about small singular values-choose a solution among the many possibles Tikhonov: min Ax y +α x Minimum norm min x given Ax =y (see below). SVD truncation: pinv(a) =Vdiag(1./σ i )U t.

93 REGULARISATION Problem: small (or even 0!) singular values. Regularisation:= do something about small singular values-choose a solution among the many possibles Tikhonov: min Ax y +α x Minimum norm min x given Ax =y (see below). SVD truncation: pinv(a) =Vdiag(1./σ i )U t. Prior information: e.g. min Ax y +α Bx. B 1 : object autocorrelation. Tikhonov: Vdiag(σ i./(σ 2 i + α 2 ))U t

94 EXAMPLE: DECONVOLUTION Blurry - Noisy deconvwnr: using SNR with prior info (A t A+αB t B) 1 A t y

95 NORMAL EQUATION-ITERATIVE METHODS For M >N: min Ax y 2 implies A Ax =A y the normal equation CGNE: Apply CG to A Ax =A y: LSQR Matlab: lsqr

96 LAPACK operations: BLAS library, inverse (and others): LAPACK Example SUBROUTINE SGGLSE( M, N, P, A, LDA, B, LDB, C, D, X, WORK, LWORK,INFO ) * Purpose * ======= * SGGLSE solves the linear equality-constrained least squares (LSE) * problem: * minimize c - A*x 2 subject to B*x = d * where A is an M-by-N matrix, B is a P-by-N matrix, c is a given * M-vector, and d is a given P-vector. It is assumed that * P <= N <= M+P, and rank(b) = P and rank( (A) ) = N.

97 COMPRESSED SENSING Total undersampling: M << N L 2. Solution A t (AA t ) 1 y. min x 2 such that Ax =y

98 COMPRESSED SENSING Total undersampling: M << N L 2. Solution A t (AA t ) 1 y. min x 2 such that Ax =y

99 COMPRESSED SENSING Total undersampling: M << N L 0. Sparsest solution: A\y. min x 0 such that Ax =y

100 COMPRESSED SENSING Total undersampling: M << N L 0. Sparsest solution: A\y. min Ex 0 such that Ax =y

101 COMPRESSED SENSING Total undersampling: M << N min x 1 such that Ax =y L 1. Under some conditions, equivalent to L 0. Acquisition-reconstruction is Compressed Sensing OMP algorithms: remnove strongest alias(es), re-project, iterate.

102 SUMMARY SENSE: deconvolution with undersampling compensated by overdetermination. Deconvolution: regularised. Undersampling: prior assumption.

103 DETERMINANT Determinant of a matrix: product of eigenvalues

104 DETERMINANT Determinant of a matrix: product of eigenvalues Area /Volume of shape generated by columns

105 DETERMINANT Determinant of a matrix: product of eigenvalues Area /Volume of shape generated by columns det=0 iff matrix is singular

106 CHANGE OF COORDINATES Non-rigid transform: Φ(x) (e.e. in registration)

107 CHANGE OF COORDINATES Non-rigid transform: Φ(x) (e.e. in registration) Multivariate derivative at x: best linear (affine) transform approximating Φ at x. Denoted DΦ(x) dφ(x)/dx. Thus in dimension D it is a D D matrix.

108 CHANGE OF COORDINATES Non-rigid transform: Φ(x) (e.e. in registration) Multivariate derivative at x: best linear (affine) transform approximating Φ at x. Denoted DΦ(x) dφ(x)/dx. Thus in dimension D it is a D D matrix. Imaging: integration or sums. Change of coordinates implies need change of are/volume element.

109 CHANGE OF COORDINATES Non-rigid transform: Φ(x) (e.e. in registration) Multivariate derivative at x: best linear (affine) transform approximating Φ at x. Denoted DΦ(x) dφ(x)/dx. Thus in dimension D it is a D D matrix. Imaging: integration or sums. Change of coordinates implies need change of are/volume element. Jacobian determinant: determinant of this matrix

110 INTEGRATION R s(x)dx = R s(y)(dx/dy)dy valid in any dimension. Thus it involves the Jacobian determinant. Can be used to derive formulas for behaviour of Fourier Transform f Ff Translation R s(x+u)e 2πik tx dx = R s(y)e 2πikt (y u) dx =e 2πikt u F s(k)

111 INTEGRATION R s(x)dx = R s(y)(dx/dy)dy valid in any dimension. Thus it involves the Jacobian determinant. Can be used to derive formulas for behaviour of Fourier Transform f Ff Translation R s(x+u)e 2πik tx dx = R s(y)e 2πikt (y u) dx =e 2πikt u F s(k) : R s(ax)e 2πiktx dx = R s(y)e 2πikt (A 1 y) dx/dydy = R s(y)e 2πiA t k t (y) 1 det(a) dy = 1 F det(a) s(a t (k))

112 INTEGRATION R s(x)dx = R s(y)(dx/dy)dy valid in any dimension. Thus it involves the Jacobian determinant. Can be used to derive formulas for behaviour of Fourier Transform f Ff Translation R s(x+u)e 2πik tx dx = R s(y)e 2πikt (y u) dx =e 2πikt u F s(k) : R s(ax)e 2πiktx dx = R s(y)e 2πikt (A 1 y) dx/dydy = R s(y)e 2πiA t k t (y) 1 det(a) dy = 1 F det(a) s(a t (k)) Rotation: special case where det(a) = 1

113 INTEGRATION R s(x)dx = R s(y)(dx/dy)dy valid in any dimension. Thus it involves the Jacobian determinant. Can be used to derive formulas for behaviour of Fourier Transform f Ff Translation R s(x+u)e 2πik tx dx = R s(y)e 2πikt (y u) dx =e 2πikt u F s(k) : R s(ax)e 2πiktx dx = R s(y)e 2πikt (A 1 y) dx/dydy = R s(y)e 2πiA t k t (y) 1 det(a) dy = 1 F det(a) s(a t (k)) Rotation: special case where det(a) = 1 Affine: linear with translation.