Medical Visualization - Tensor Visualization. J.-Prof. Dr. Kai Lawonn

Transcription

1 Medical Visualization - Tensor Visualization J.-Prof. Dr. Kai Lawonn

2 Lecture is partially based on the lecture by Prof. Thomas Schultz 2

3 What is a Tensor? A tensor is a multilinear transformation that maps vectors to a scalar: T is a tensor of rank r 3

4 What is a linear map? Let T be a linear map Then, the following two conditions are satisfied: Multilinear means linearity is fulfilled for every argument 4

5 Examples Let be the rank r=1 and n=2 5

6 Examples Let be the rank r=1 and n=2 6

7 Examples Let be the rank r=1 and n=2 WHY??? 7

8 Examples Let be the rank r=1 and n=2 Why is the first one a tensor? 8

9 Examples Let be the rank r=1 and n=2 Why is the first one a tensor? Linear! 9

10 Examples Let be the rank r=1 and n=2 The second map is not linear and therefore it is not a tensor! 10

11 Tensor Let be the rank r=1 The tensor can be written as: 11

12 Examples Let be the rank r=1 and n=3 12

13 Examples Let be the rank r=2 and n=2 13

14 Examples Let be the rank r=2 and n=2 14

15 Examples Let be the rank r=2 and n=2 WHY??? 15

16 Examples Let be the rank r=2 and n=2 16

17 Examples Let be the rank r=2 and n=2 c = 0 17

18 Examples Let be the rank r=2 and n=2 c = 0 NOT LINEAR!!! 18

19 Tensor Let be the rank r=2 The tensor can be written as: 19

20 Tensor Let be the rank r=2 The tensor can be written as: 20

21 Geometric Intuition of Linear Maps A linear (non-singular) transform A always takes (hyper-)spheres to (hyper-)ellipses. A A (From: Thomas Schultz) 21

22 Geometric Intuition of Linear Maps Thus, one good way to understand what A does is to find which vectors are mapped to the main axes of the ellipsoid. A A (From: Thomas Schultz) 22

23 Geometric Intuition of Linear Maps If A is symmetric: A = V S V T V orthogonal rows and columns are orthogonal unit vectors Transpose equals inverse: V T =V -1 The eigenvectors of A are the axes of the ellipse 1 1 A s 2 s 1 (From: Thomas Schultz) 23

24 Symmetric matrix: Eigendecomposition In this case A is just a scaling transformation. The eigendecomposition of A tells us which orthogonal axes it scales and by how much: Av s v i i i 1 1 A s 2 s 1 24

25 General Linear Transformations: SVD (Singular Value Decomposition) In general, A will also contain rotations, not just scales: A U V T 1 1 A s 2 s 1 25

26 General Linear Transformations: SVD orthonormal orthonormal AV U A v s u, s 0 i i i i 1 1 s 2 s 1 A 26

27 SVD more formally SVD exists for any matrix Formal definition: For square matrices A R n n, there exist orthogonal matrices U, V R n n and a diagonal matrix S, such that all the diagonal values s i of S are nonnegative and A USV T = A 27 U S V T

28 SVD more formally The diagonal values of S (s 1,, s n ) are called the singular values. They are usually sorted: s 1 s 2 s n The columns of U (u 1,, u n ) are called the left singular vectors. They are the axes of the ellipsoid. The columns of V (v 1,, v n ) are called the right singular vectors. They are the pre-images of the axes of the ellipsoid. A USV T = A U S V T 28

29 Reduced SVD For rectangular matrices, we have two forms of SVD. The reduced SVD looks like this: The columns of U are orthonormal Cheaper form for computation and storage = A U S T 29 V

30 Full SVD We can complete U to a full orthogonal matrix and pad S by zeros accordingly = T A U S V 30

31 Spectra and Diagonalization If A is symmetric, the eigenvectors are orthogonal (and there s always an eigenbasis). A A = U U T Au i = i u i 31

32 Properties of Tensors (Square) tensors can be symmetric or unsymmetric represented by symmetric / unsymmetric matrix definite or indefinite Positive semi-definite: v T Mv 0 for all v Easily seen from non-negative eigenvalues of M Strictly positive definite: v T Mv > 0 for all v second-order (matrix representation) or higher-order (multi-way array, outside our scope) 32

33 Summary: Introduction to Tensors Tensors are linear maps between vector spaces Can be represented as matrices Decomposition in terms of eigenvalues and eigenvectors (if symmetric) or SVD (in general) Visualization by their effect on the unit sphere (tensor ellipsoid) Applications include geometry, flow fields, material science, diffusion tensor imaging 33

34 Diffusion Tensor MRI 34

35 Introduction to Diffusion MRI Goal: Investigate the microstructure of biological tissue using Magnetic Resonance Imaging (MRI) Challenge: Voxel size is far too large to resolve the structures of interest 2 mm 1 mm 1 mm Voxel size ø μm Axon (nerve fiber) size 35

36 Images from Gordon Kindlmann Introduction to Diffusion MRI Approach: Use water molecules as a contrast agent Exploits their spontaneous heat motion at the desired spatial scale Analogy: Observe diffusion of ink on paper Kleenex Newspaper 36

37 Introduction to Diffusion MRI Water molecules at any temperature above absolute zero undergo Brownian motion or molecular diffusion In free water, this motion is completely random, and water molecules move with equal probability in all directions (isotropic diffusion) In the presence of constraining structures, such as the axons connecting neurons together, water molecules move more often in the same direction than they do across these structures (anisotropic diffusion) 37

38 Introduction to Diffusion Weighting Standard MRI Diffusion Weighted MRI

39 Stejskal-Tanner Equation:

40 Diffusion Tensor Imaging When at least six directions are acquired d becomes a 3 3 symmetric diffusion tensor D Shown tensor field as a matrix of scalar fields Shows all information Rarely permits an effective interpretation 40

41 Vertical Gradient Horizontal Gradient

42 Vertical Gradient

43 Diffusion Tensor Imaging D = 3x3 symmetric matrix Diffusion Tensor

44 Estimating Diffusion Tensors Stejskal-Tanner Equation: Taking the logarithm and solving for D produces a system of linear equations (one per measurement i): D = symmetric 3x3 matrix six free variables Six measurements exact solution More measurements least squares solution 44

45 Eigenvector Decomposition Diffusion tensor D decomposes into: 3 eigenvalues λ 1 λ 2 λ 3 If D is positive definite: All λ>0 Note: Diffusivity is a non-negative physical quantity. Due to measurement noise, we might obtain S 0 <S(d), which can lead to D with negative λ s 3 orthogonal eigenvectors e 1 /e 2 /e 3 e 1 indicates single main fiber orientation Axes of tensor ellipsoid are aligned with eigenvectors and scaled by eigenvalues

46 Summary: Diffusion Tensor MRI Diffusion Tensor MRI takes spatially resolved measurements of molecular heat motion Diffusion leads to MR signal attenuation Often assumed to follow an exponential law, parameterized by an apparent diffusion coefficient Diffusion Tensor captures directionally dependent (anisotropic) diffusivity Often applied to the (human) brain Free diffusion is symmetric (v and v equally likely); diffusion MRI currently cannot reliably detect non-symmetric diffusion 46

47 Tensor Field Visualization (Tensor Glyphs) 47

48 Overview of Tensor Field Visualization Tensor Field Visualization Glyph-Based Derived Scalar Fields Derived Vector Fields Slices DVR Isosurfaces Ellipsoid Glyphs Westin s Glyph Superquadric Glyphs Streamlines LIC Topology 48

49 Overview of Tensor Field Visualization Tensor Field Visualization Reduce information content Can be used to derive dense visualizations Slices Derived Scalar Fields DVR Isosurfaces Streamlines Derived Vector Fields LIC Topology 49

50 Overview of Tensor Field Visualization Tensor Field Visualization Ellipsoid Glyphs Glyph-Based Westin s Glyph Superquadric Glyphs Convey the full information in a tensor But only at discrete points in space (i.e., sparse visualization) 50

51 Visualizing Individual Coefficients Simplest idea: Show tensor field as a matrix of scalar fields Shows all information Rarely permits an effective interpretation 51

52 Tensor Ellipsoid A glyph is a geometric object whose shape, size, orientation, and color conveys the data Common example for tensor data: Ellipsoid Axes aligned with eigenvectors scaled with eigenvalues Implicit Equation: x T D 2 x 1

53 Examples 53

54 A Classic Use of Glyphs Glyphs are useful to Quickly assess data quality Identify artifacts Inspect the data immediately, without complex visualization algorithms Images courtesy of Gordon Kindlmann Flipped z Coordinate Corrected Sign

55 Principles for Tensor Glyph Design Faithful and expressive visualization requires: Preservation of Symmetry: Glyph should have same symmetries as the tensor Continuity: Disambiguity: 55

56 Westin Measures Three rotationally invariant metrics allow us to differentiate the three possible types of anisotropy (Westin et al., 1997) c l = λ 1 λ 2 λ 1 + λ 2 + λ 3 c p = 2 λ 2 λ 3 λ 1 + λ 2 + λ 3 c s = 3λ 3 λ 1 + λ 2 + λ 3 57

57 Westin Measures c l = λ 1 λ 2 λ 1 + λ 2 + λ 3 c p = 2 λ 2 λ 3 λ 1 + λ 2 + λ 3 c s = These metrics add up to one and can thus be used to define a barycentric space of diffusion tensor shapes c s =1 3λ 3 λ 1 + λ 2 + λ 3 c l =1 c p =1 58

58 Box and Cylinder Glyphs Box Glyphs Cylinder Glyphs 59

59 Superquadric Tensor Glyphs Ellipsoids suffer from visual ambiguities: Superquadric Glyphs greatly reduce them: Images taken from Kindlmann [2004]

60 Superquadric Shape Space Superquadrics are given as where exponentiation is defined to be sign preserving cos α θ sign cos θ cos θ α 61

61 The Idea Behind Superquadric Glyphs Ellipsoids are transformations of the sphere Superquadrics smoothly interpolate between sphere, cylinder, and box Ellipsoids Superquadrics

62 Superquadric Parameters from Tensor Shape If c l < c p, we obtain the basic superquadric tensor glyph shape as follows: α = 1 c l γ β = 1 c p γ 63

63 Superquadric Glyphs: Sharpness Superquadric glyphs combine advantages of boxes, cylinders and ellipsoids Edges give strong visual cues for orientation Round shapes are used when orientation is not clearly defined Sharpness parameter decides how quickly sharp features develop = 1.5 = 6 64

64 Superquadric Parameters when c l c p If c l c p, we use an alternative family of superquadrics that is defined to be symmetric around the x axis: α = 1 c p γ β = 1 c l γ 65

65 Superquadric Glyphs: Final Equation To obtain final glyph, superquadric geometry is scaled by eigenvalues and rotated so that principal axes align with eigenvectors: scaling rotation normalized eigenvalues basis shape 66

66 Coloring For Indefinite Tensors We color each point x on the glyph by Satisfies Preservation of Symmetry Continuity Disambiguity sign(x T Dx) + red - blue 67

67 The Lune of Tensor Shape 68

68 69

69 70

70 71

71 Image taken from Kindlmann et al. [2006] Glyphs on Regular Grid Compared to glyphs on a regular grid 76

72 Image taken from Kindlmann et al. [2006] Glyph Packing Packing glyphs distribute the glyphs regularly on the screen 77

73 Glyphs: Benefits and Drawbacks Unlike scalar- or vector-based visualizations, glyphs convey the full information present in the tensor glyphs only show the field at discrete points in space Glyphs work well for 2D slices, but easily create an illegible mass in 3D (due to occlusion) In 3D, culling of tensors is necessary, e.g., based on an anisotropy threshold (in the example: c l + c p 0.5) 78

74 Visualizing Derived Scalar Fields 79

75 Scalar Invariants Invariant = scalar quantity that is a function of the tensor does not change under rotation / changes of the frame of reference Eigenvalues / singular values are invariant All measures that can be expressed as functions of eigenvalues / singular values are invariant Many tensor visualization methods focus on symmetric tensors Eigenvalues parameterize the shape of symmetric tensors 80

76 Frobenius Norm Frobenius Norm Defined like l 2 vector norm: T = i,j T ij 2 For symmetric T, same as l 2 norm of eigenvalues: λ λ λ

77 Matrix Trace / Mean Diffusivity Matrix Trace Sum of diagonal elements: tr(t) = T xx +T yy +T zz For symmetric T, same as sum of eigenvalues: λ 1 +λ 2 +λ 3 Mean Diffusivity (MD) Average eigenvalue: tr(t)/3 Interpretation in DT-MRI: Average diffusivity (over all directions) 82

78 Fractional Anisotropy (FA) Fractional Anisotropy Quantifies the degree of anisotropy Correlates with fiber density / integrity Also correlates with orientation dispersion Based on variance of eigenvalues Normalized to [0,1] (if positive definite) 83

79 Images from Gordon Kindlmann Eigenvalue Space Illustration: Isosurfaces of Frobenius norm, MD, and FA in 3D space spanned by eigenvalues Frobenius Norm MD FA 84

80 Image from Gordon Kindlmann FA vs. Tensor Mode FA and mode are not independent Maximum FA=1 only achieved with mode=1 86

81 Westin Measures Westin s c l, c p, c s quantify the extent to which the tensor ellipsoid is linear / planar / spherical Used in design of superquadric tensor glyphs c l = λ 1 λ 2 λ 1 + λ 2 + λ 3 c s =1 c p = 2 λ 2 λ 3 λ 1 + λ 2 + λ 3 c s = 3λ 3 λ 1 + λ 2 + λ 3 c l =1 c p =1 87

82 Volume Rendering Tensor Fields Define color / opacity based on scalar measures such as Westin s shape Shading based on gradient of opacity Approximation: Pre-compute derived scalar field on a grid 88

83 Isosurfaces in Tensor Fields Isosurface of c l Shows outline of white matter core Segmented to highlight different fiber bundles [Schultz et al. 2007] 89

84 Summary: Derived Scalar Measures Scalar Invariants are independent of the chosen frame of reference Define the shape of the tensor Can be expressed in terms of eigenvalues Measure overall norm, amount of anisotropy (e.g., FA), type of anisotropy (e.g., mode or Westin s measures) Can be used to apply standard volume visualization to tensors (e.g., volume rendering, isosurfaces) 90

85 Visualizing Derived Vector Fields 91

86 Visualizing Eigenvector Fields Most methods for vector field visualization can be applied to eigenvector fields Keep in mind: Unlike proper vectors, eigenvectors (i.e., v such that Mv=λv) do not have an intrinsic norm or orientation any scaled version αv (α 0) is also an eigenvector in particular, -v is also an eigenvector Keep in mind: When two (or more) eigenvalues are equal, v can be rotated freely in their two- (or higher) dimensional eigenspace 92

87 Color Coded Eigenvector Maps Standard Color Coding: XYZ->RGB Usually applied to principal eigenvector: R v 1x, G v 1y, B v 1z Usually modulated by FA or c l Note: FA can be large even for planar anisotropy! 93

88

89 Seed Points on Mid-Sagittal Plane Deterministic Tractography

90 Fiber Tracking / Tractography In the context of diffusion MRI, streamline integration in the principal eigenvector field is called fiber tracking or tractography Need to orient eigenvectors Flip sign if dot product with previous vector is <0 Need for additional stopping criteria Anisotropy: Stop when FA or c l are low (better to use c l ) Curvature: Stop when dot product with previous vector is small White matter mask: Stop when leaving the white matter (e.g., found from traditional MRI) 96

91 Fiber Tracking as Solution of an ODE Deterministic Tractography can be viewed as solving the ordinary differential equation xሶ t = v x t Vector field v derived from tensor field Example: Major eigenvector of diffusion tensor Choose sign of v so that we are tracking forwards Basser et al. [2000]: Euler integration with stepsize s: x i+1 = x i + s v(x i ) More exact: Higher-order schemes (Runge-Kutta) Stop on high curvature or low anisotropy Use interpolation to obtain continuous tensor field At each step, interpolate tensor and compute its principal eigenvector

92 FACT Fiber Assignment by Continuous Tracking (FACT) [Mori et al. 1999] Follow principal eigenvector until voxel is left Stop when direction would change abruptly Advantage: No interpolation needed Drawback: Follows the streamline less precisely Image taken from Mori et al. [1999]

93 Streamtubes Stream tubes [Zhang et al. 2003] also encode second and third eigenvector Elliptical cross-section reflects second/third eval Fix maximum radius, preserve aspect ratio Color indicates c l (large c l red) 100

94 Superquadric Streamtubes Superquadric streamtubes [Wiens et al. 2014] Superquadric instead of elliptical cross-section Shape index σ = λ 3 λ 2 Spherical for λ 2 = λ 3 Clear edges for λ 2 λ 3 γ 101

95 Superquadric Streamtubes: Connectivity Given square cross-sections, it s important to connect corners to corners: 102

96 Superquadric Streamtubes: Example 103

97 Warning: Inconsistent Terminology! Terminology is inconsistent in the literature Sometimes, streamlines in eigenvector fields are called tensor lines, even if they are not created by the tensor lines algorithm by Weinstein et al. Sometimes, streamlines in eigenvector fields are called hyperstreamlines ; other authors use that term to denote stream tubes [Delmarcelle/Hesselink 1992] 104

98 Fiber Bundles in the Brain 105

99 Interpreting DT-MRI Fibers Reminder: Individual axons are much smaller than voxel size DT-MRI streamlines are often called fibers, but do not correspond to individual axons axons often run in parallel ( fiber bundle ), making it possible to detect them at voxel level Streamlines follow trajectory of fiber bundles ø μm l 1-3mm 106

100 Animated Tractography 112

101 Summary: Fiber Tractography Vector Field Visualization can be applied to eigenvector fields Account for lack of orientation! Fiber tracking traces lines that are tangential to inferred fiber direction in diffusion MRI Similar to streamline integration Tensor lines can track through isotropic regions Stream tubes encode additional eigenvectors 113

102 Questions??? 118