COMPUTATIONAL IMAGING. Berthold K.P. Horn

Transcription

1 COMPUTATIONAL IMAGING Berthold K.P. Horn

2 What is Computational Imaging? Computation inherent in image formation

3 What is Computational Imaging? Computation inherent in image formation (1) Computing is getting faster and cheaper precision physical apparatus is not

4 What is Computational Imaging? Computation inherent in image formation (1) Computing is getting faster and cheaper precision physical apparatus is not (2) Can t refract or reflect some radiation

5 What is Computational Imaging? Computation inherent in image formation (1) Computing is getting faster and cheaper precision physical apparatus is not (2) Can t refract or reflect some radiation (3) Detection is at times inherently coded

6 Computational Imaging System

7 Examples of Computational Imaging: (1) Synthetic Aperture Imaging (2) Coded Aperture Imaging (3) Exact Cone Beam Reconstruction (4) Diaphanography DiffuseTomography

8 (1) SYNTHETIC APERTURE IMAGING Traditional approach: Coupling of resolution, DOF, FOV to NA Precision imaging flat illumination with: Michael Mermelstein, Jekwan Ryu, Stanley Hong, and Dennis Freeman

9 Objective Lens Parameter Coupling

10 Synthetic Aperture Imaging Traditional approach: Coupling of resolution, DOF, FOV to NA Precision imaging flat illumination New approach: Precision illumination Simple imaging Multiple images Textured illumination

11

12

13

14

15 Synthetic Aperture Imaging Precision illumination Simple imaging Multiple images Textured illumination Image detail in response to textures Non-uniform samples in FT space

16 SAM M6

17 Creating Interference Pattern

18 Reflective Optics M6

19 Creating Interference Pattern

20 Fourier Transform of Texture Pattern

21 Interference Pattern Texture

22 Synthetic Aperture Microscopy Interference of many Coherent Beams Amplitude and Phase Control of Beams

23 Amplitude and Phase Control

24 Synthetic Aperture Microscopy Interference of many Coherent Beams Amplitude and Phase Control of Beams On the fly calibration Non-uniform inverse FT Least Squares

25 Wavenumber Calibration using FT

26 Hough Transform Calibration

27 Least Squares Match in FT

28 Polystyrene Micro Beads (1µm)

29

30 (2) CODED APERTURE IMAGING Can t refract or reflect gamma rays Pinhole tradeoff resolution and SNR with: Richard Lanza, Roberto Accorsi, Klaus Ziock, and Lorenzo Fabris.

31 Coded Aperture Imaging Can t refract or reflect gamma rays Pinhole tradeoff resolution and SNR Multiple pinholes Complex masks can cast shadows

32 Coded Aperture Principle

33 Decoding Method Rationale

34 Coded Aperture Imaging Can t refract or reflect gamma rays Pinhole tradeoff resolution and SNR Complex masks can cast shadows Decoding by Correlation Special Masks with Flat Power Spectrum

35 Mask Design Inverse Systems

36 Masks XRT Coarse

37 Mask Design 1D Definition: q is a quadratic residue (mod p) if n s.t. n 2 q(mod p) Legendre symbol ( a ) p = 1 { 1 if a is quadratic residue otherwise Correlation with zero shift (p 1)/2 Correlation with non-zero shift (p 1)/4

38 Mask Design Auto Correlation a(i) = (p 1) 4 (1 + δ(i)) Power Spectrum A(j) = (p 1) (δ(j) + 1) 4

39 Coded Aperture Extensions Imaging Nearby Objects Mask / Countermask Combination * Dynamic Reconstruction

40 Coded Aperture Application Detection of Fissile Material Large Area Detector Myth Signal and Background Amplified

41 Large Area Alone Doesn t Help

42 Imaging and Large Area Do!

43 Coded Aperture Example Imaging 1/R instead of 1/R 2

44 Coded Aperture Detector Array

45 Computational Imaging System

46

47

48 Dynamic Reconstruction Three weak, distant radioactive sources Reconstruction Animation

49 Coded Aperture Applications Detection of Fissile Material Imaging 1/R instead of 1/R 2 Increasing Gamma Camera Resolution Replacing Rats with Mice.

50 (4) EXACT CONE BEAM ALGORITHM Faster Scanning Fewer Motion Artifacts Lower Exposure Uniform Resolution with: Xiaochun Yang

51 Exact Cone Beam Reconstruction Faster Scanning Fewer Motion Artifacts Lower Exposure Uniform Resolution Parallel Beam Fan Beam Planar Fan Cone Beam

52 Parallel Beam to Fan Beam Coordinate Transform in 2D Radon Space

53 Cone Beam Geometry 3D

54 Radon s Formula In 2D: ~ derivatives of line integrals In 3D: derivatives of plane integrals Can t get plane integrals from projections ( ) f (r, θ)dr dθ 1 f (x, y) dx dy r

55 Radon s Formula in 3D f(x) = 1 8π 2 2 R f (l, β) S 2 l 2 l=x β dβ where R f (l, β) = f(x) δ(x β l)dv

56 Grangeat s Trick z f (x, y, z) dx dy = f (r, φ, θ) dr dφ θ

57 Exact Cone Beam Reconstruction Data Sufficiency Condition Good Orbit for Radiation Source

58 Radon Space 2D

59 Circular Orbit is Inadequate (3D)

60 Data Insufficiency

61 Good Source Orbit

62 Exact Cone Beam Reconstruction Data Sufficiency Condition Good Orbit for Radiation Source Practical Issue: Spiral CT Scanners Practical Issue: Long Body Problem.

63 (3) DIAPHANOGRAPHY (Diffuse Optical Imaging) Highly Scattering Low Absorption Many Sources Many Detectors with: Xiaochun Yang, Richard Lanza, David Boas and Anna Custo.

64

65

66

67 Diaphanography Randomization of Direction Scalar Flux Density

68 30 Head coronal slice scalp mm skull CSF gray mm white

69 Total fluence [CW] Log 10 Fluence source detector separation [mm]

70 scalp skull Normalized PPF brain source detector separation [mm]

71 mm 20 Total fluence [TD] Log 10 TPSF mm mm time steps [ns]

72 Diaphanography Approximation: Diffusion Equation v(x,y) + ρ(x,y)c(x,y) = 0 v(x,y) flux density ρ(x, y) scattering coefficient c(x,y) absorption coefficient Forward: given c(x,y) find v(x,y)

73

74 3 3 node grid example Here we have three input nodes (1, 2, 3), three output nodes (7, 8, 9), and three interior nodes (4, 5, 6). In three experiments we apply currents to each of the input nodes in turn, each time reading out all of the output nodes, yielding a total of nine measurements. We try and recover the nine leakage conductances to ground from each of the nine nodes. It is natural to partition the conductance matrix as follows given that I 4, I 5, I 6, I 7, I 8, and I 9 are always zero, and that we do not measure V 1, V 2, V 3, V 4, V 5, and V 6.. G... V 11 G12.. G13 3. G... V 21 G G23 V 5 = V. G G32.. G33 V 7 V 8 V 9 V 1 V 2 I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9

75 . g 1 g 12 g.. 13 g g 12 g 2 g g g 13 g 23 g g g g 4 g 45 g.. 46 g g g 45 g 5 g g g.. 36 g46 g 56 g g g g 7 g 78 g g g 78 g 8 g g 69. g 79 g 89 g 9 V 1 V 2 V 3 V 4 V 5 = V 6 V 7 V 8 V 9 I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9

76 We can write the inverse as follows:. C C12.. C13. C C22.. C23. C C32.. C33 I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9 V 1 V 2 V 3 V 4 = V 5 V 6 V 7 V 8 V 9

77

78 c(k)(r) FORWARD SOLUTION I(r 1,r 2 ) I (r 1,r 2 ) UPDATE RULE - "ASSIGNING BLAME"

79 Diaphanography Invert Diffusion Equation Regions of Influence.

80 COMPUTATIONAL IMAGING (1) Synthetic Aperture Imaging (2) Coded Aperture Imaging (3) Exact Cone Beam Reconstruction (4) Diaphanography DiffuseTomography

81 COMPUTATIONAL IMAGING

82 Synthetic Aperture Lithography Create pattern controlled interference Example: Two Dots Example: Straight Line Destructive interference safe zone Example: Bessel Ring.

83 SAM M4

84 Amplitude and Phase Control

85

86 Interference Pattern Texture

87 Fourier Transform of Texture Pattern

88 Uneven Fourier Sampling

89 Resolution Enhancement Reflective Optics Illumination Vaccum UV Short Wavelength

90 Resolution Enhancement Reflective Optics Illumination Vaccum UV Short Wavelength Fluorescence Mode Resolution Determined by Illumination

91 Masks Fresnel Camera

92 Masks Legri URA

93 Masks XRT Fine

94 Masks Hexagonal

95 Maximizing SNR n n min w 2 i subject to w i = 1 i=1 i=1 yields w i = 1 n

96 Spatially Varying Background

97 Coded Aperture Extensions Artifacts due to Finite Distance Mask / Countermask Combination Multiple Detector Array Positions Synthetic Aperture Radiography

98 Dynamic Reconstruction

99 Diaphanography Approximation: Diffusion Equation Leaky Resistive Sheet Analog (2D)

100

101

102 mm mm 40mm 0.8 Normalized PPF brain scalp skull time steps [ns] (a) Deviation of Sensitivity to Scalp Skull (PPF PPF o )/PPF o 0 20 mm mm mm time steps [ns] (b) (PPF PPF o )/PPF o mm mm Deviation of Sensitivity to Brain mm time steps [ns] (c)

103 Sample Resistive Grid Inversion Berthold K.P. Horn 1996 February 14th Here we consider about the simplest possible case of the two-d resistive sheet to get some insight into the more general problem. (draw your own picture here :) There are four nodes, arranged in N = 2 rows and M = 2 columns. On the left are the two nodes used for input ((1) and (2)) and on the right are the two output nodes ((3) and (4)). Four horizontal resistors with conductance g 12, g 13, g 24 and g 34 connect these four nodes. These resistors represent scattering, and are assumed to be of known value. There is also a vertical leakage path from each of the four nodes to ground with conductances g 1, g 2, g 3 and g 4. These represent absorption, and are the unknowns. We are to recover the values of the unknown leakage resistors. We perform two experiments: First we inject current at node (1) and measure the potentials on nodes (3) and (4). Call the trans-impedance (ratio of output potential to injected current) observed this way R 3,1 and R 4,1. Then we inject instead current at node (2) and again measure the potential on nodes (3) and (4). Call the trans-impedance observed this way R 3,2 and R 4,2. If the grid was N M instead of 2 2 then we would have performed N experiments, each time injecting current on one of the N input nodes and reading out the potential on each of the N output nodes. We then try to recover the N M unknown leakage conductances to ground. Clearly there is not enough constraint if M>Nsince there are then more unknowns than measurements. Conversely if M<N, we have redundant information and may want to use a least squares approach to obtain the best possible answer. Here we deal with the simple case where M = N = 2. The node equations in this case are: I 1 = g 1 V 1 + g 13 (V 1 V 3 )+g 12 (V 1 V 2 ) I 2 = g 2 V 2 + g 12 (V 2 V 1 )+g 24 (V 2 V 4 ) I 3 = g 3 V 3 + g 13 (V 3 V 1 )+g 34 (V 3 V 4 ) I 4 = g 4 V 4 + g 24 (V 4 V 2 )+g 34 (V 4 V 3 ) or. (g 1 + g 13 + g 12 ) g.. 12 g13 0 g 12 (g 2 + g 12 + g 24 ). 0 g 24 g (g 3 + g 13 + g 34 ) g 34 0 g g34 (g 4 + g 24 + g 34 ) V 1 V 2 = V 3 V 4 Note that all off diagonal elements are negative, and that the unknown leakage conductances all appear on the diagonal. Also, the matrix becomes singular if all leakages conductances are set to zero, since then each row adds up to zero. Making use of the partitioning indicated above, we can write V 1 G 11. G 12 V 2 = V G 21. G 3 22 V 4 This partitioning is convenient, since in the experiments we always have I 3 = 0 and I 4 = 0, and since V 1 and V 2 are not known. If we invert this set of equations we get: C 11. C 12 C 21. C 22 1 I 1 I 2 I 3 I 4 I 1 I 2 I 3 I 4 V 1 V = 2 V 3 V 4 I 1 I 2 I 3 I 4

104 where Upon multiplying out we obtain C 11 =(G 11 G 12 G 1 22 G 21) 1 C 21 = G 1 22 G 21C 11 C 22 =(G 22 G 21 G 1 11 G 12) 1 C 12 = G 1 11 G 12C 22 ( ) I1 C 11 = I 2 where the term involving C 12 drops out because I 3 = 0 and I 4 = 0. This equation is of no interest since we don t know V 1 and V 2. But we also obtain ( ) I1 C 21 = I 2 where the term involving C 22 drops out because I 3 = 0 and I 4 = 0. The four unkowns (g 1,g 2,g 3,g 4 ) occur in the matrix C 21, but we obviously can t solve for them using a single set of measurements. We can however, combine two sets of measurements and obtain: ( ) ( ) I1,1 I C 1,2 V3,1 V 21 = 3,2 I 2,1 I 2,2 V 4,1 V 4,2 where typically we would choose I 1,1 =1,I 2,1 = 0 for the first experiment and I 1,2 =0,I 2,2 = 1 for the second. We then obtain ( ) R3,1 R C 21 = 3,2 R 4,1 R 4,2 where, provided I 2,1 = 0 and I 1,2 =0,R 3,1 = V 3,1 /I 1,1, R 4,1 = V 4,1 /I 1,1, and R 3,2 = V 3,2 /I 1,2, R 4,2 = V 4,2 /I 1,2. So from image measurements we can recover the matrix C 21, and we know that or then, since we obtain ( V1 V 2 ( V3 V 4 ) ) C 21 = G 1 22 G 21C 11 C 21 C 1 11 = G 1 22 G 21 C 11 =(G 11 G 12 G 1 22 G 21) 1 C 21 (G 11 G 12 G 1 22 G 21) = G 1 22 G 21 We need to manipulate this some more to try and isolate the two matrices G 11 and G 22, which contain the unknowns g 1, g 2, g 3, and g 4. We see that or or finally C 21 G 11 = C 21 G 12 G 1 22 G 21 G 1 22 G 21 C 21 G 11 G 1 21 G 22 = C 21 G 12 I G 11 G 1 21 G 22 = G 12 C21 1 Here C 1 21 is obtained from experimental measurements, while G 11 and G 22 contain the unknown leakage conductances. In our simple example, G 12 and G 21 are diagonal. 2

105 Numerical Example Suppose that the horizontal resistors have conductance as follows: g 13 =1,g 12 =2,g 24 = 1, and g 34 =2. Next assum that the leakage or vertical resistors have conductance g 1 =1,g 2 =2,g 3 = 2, and g 4 =1. Then the conductance matrix is and so and so while so and So in the inverse we have and Finally G 1 11 = 1 16 C 11 = C 21 = ( ) and G 12 G 1 22 G 21 = G 1 22 = 1 16 G 21 G 1 11 G 12 = G 1 11 = 1 16 G 11 G 12 G 1 22 G 21 = 1 16 G 22 G 21 G 1 11 G 12 = 1 16 ( 75 ) ( 23 ) G 1 = C = G 1 22 = 1 16 ( ) ( ) ( ) ( 60 ) ( 75 ) and C 22 = and C 12 = ( 60 ) ( 23 ) In the first experiment we have I 1 = 1 and the other node currents are zero so V 1 75 V 2 = V V 4 20 In the second experiment we have I 2 = 1 and the other node currents are zero so V 1 34 V 2 = V V

106 Note that we can only measure V 3 and V 4 in each case. This is the end of the forward problem (finding trans-impedance given leakage conductances). The inverse task is to recover the unknown leakage conductances. Extracting the relevant parts from the above experimental data we see that C 21 = ( ) so so ( ) C = ( ) G 12 C = and G 11 = ( g ) 2 g 2 +3 and G 22 = ( ) g g 4 +3 So ( )( ) G 11 G 1 21 G g g =. 2 g g 4 +3 So that we get the following equations in the unknown leakage conductances: (g 1 + 3)(g 3 +3)+4=24 2(g 1 +3)+2(g 4 +3)=16 2(g 3 +3)+2(g 2 +3)=20 (g 2 + 3)(g 4 +3)+4=24 or ḡ 1 ḡ 3 =20 ḡ 1 +ḡ 4 =8 ḡ 2 +ḡ 3 =10 ḡ 2 ḡ 4 =20 where ḡ 1 = g 1 +3, ḡ 2 = g 2 +3, ḡ 3 = g 3 + 3, and ḡ 4 = g These equations have only one solution: ḡ 1 =4, ḡ 2 =5,ḡ 3 = 5, and ḡ 4 = 4, that is g 1 =1,g 2 =2,g 3 = 2, and g 4 =1. Summary While this shows a solution method for a 2 2 grid, some of the points noted here also apply to the more general case, although an explicit solution cannot be expected then. In general, the matrix would have to be partitioned into a 3 3 arrangement corresponding to the fact that in addition to input nodes and output nodes there are then also interior nodes. 4

107 We can write the inverse as follows:. C C12.. C13. C C22.. C23. C C32.. C33 I 1 I 2 I 3 I 4 I 5 I 6 I 7 I 8 I 9 V 1 V 2 V 3 V 4 = V 5 V 6 V 7 V 8 V 9 We can use the formula for the inverse of matrix partitioned into four parts twice on this matrix partitioned into nine parts. But it may be a bit much to expect to easily obtain explicit formulae the way we did for the 2 2 case... Note that we are only really interested in the bottom left corner (C 31 ) of the inverse, given that I 4, I 5, I 6, I 7, I 8, and I 9 are always zero, and that we do not measure V 1, V 2, V 3, V 4, V 5, and V 6. Each experiment yields three measurements and thus three equations of the form C 31 I 1 I 2 = V 7 V 8. I 3 V 9 By performing three experiments we can find all nine elements of the matrix C 31. Each of these is a polynomial in the unknown leakage conductances g 1, g 2, g 3, g 4, g 5, g 6, g 7, g 8, and g 9 (or rather, we can cross-multiply to obtain nine such polynomials). The part of the inverse of this conductance matrix that we need is the lower left corner, C 31. Using the decomposition rule for partitioned 2 2 matrices twice, we get C 31 = G 1 33 G 32(G 22 G 23 G 1 33 G 21) 1 G 21 ( G11 G 12 (G 22 G 23 G 1 33 G 21) 1 G 21 ) 1 Note that the term (G 22 G 23 G 1 33 G 21) 1 G 21 appears twice. This can be exploited to save on computation. 6