A Study of Numerical Algorithms for Regularized Poisson ML Image Reconstruction

Transcription

1 A Study of Numerical Algorithms for Regularized Poisson ML Image Reconstruction Yao Xie Project Report for EE 391 Stanford University, Summer September 1, 2007 Abstract In this report we solved a regularized Poisson maximum likelihood (ML) image reconstruction problem, using various numerical methods. Rather than the commonly assumed Gaussian ML formulation, we considered a Poisson ML formulation, which is more accurate in some applications such as low dose computed tomography (CT), and also avoids the problematic log conversion in the Gaussian formulation. The speed of these methods are compared using a 64 by 64 pixel size example. We found that in our case the diagonally preconditioned conjugate gradient (PCG) method has the best performance. 1 Introduction Deterministic Filtered backprojection (FBP) and Regularized Maximum-Likelihood(ML) estimation are two major approaches to transmission tomography image reconstruction, such as CT. Though the latter method typically yields higher image quality, its practical application is yet limited by its high computational cost (0.02 second per image for FBP versus several hours per image for ML-based method). Recently, Preconditioning Conjugate Gradient (PCG) has attracted a lot of interests due to its efficiency in solving ML problems. However, many of the works using PCG are mainly devoted to emission tomography image reconstruction problems. Here rises a question: can we apply PCG to transmission tomography? The comparison between emission tomography and transmission tomography shows that their mathematical formulation are similar if we use Gaussian model to describe the Poisson noise, which is a strategy adopted by most existing methods. However, the Gaussian model has two drawbacks limiting its use in real systems. On one hand, the reduction of radiation dose in modern CT systems due to fast scanning calls for more accurate description based 1

2 on Poisson model; on the other hand, the measured data produced by the Gaussian model can be negative due to scatter, which causes difficulty in log conversion. This study compares the efficiency of various methods (including PCG for ET) in solving the Poisson model based ML problem for TT. There seems to be little work on comparison between different approaches except for a recent one [DBT + 05]. We found that the diagonally preconditioned conjugate gradient (CG) method may be the fastest for our problem. Also, no log conversion was needed in the algorithms based on Poisson model. 2 Regularized ML reconstruction problem The goal of statistical tomography reconstruction is to estimate the image from noisy measurements, with some regularization to prevent overfitting the noisy data. For the monoenergetic transmission tomography under the Poisson noise model, the problem can be formulated as where minimize f(x) = y T Lx + b T exp( Lx) + βφ(cx) (1) x = [x 1,, x n ] R n 1 is the unknown pixel vector y = [y 1,, y m ] R m 1 is the measured data vector contaminated by Poisson noise b = [b 1,, b m ] T the object R m 1 represents the intensities of X-rays before passing through L = [l 1,, l m ] T R m n is the forward projection matrix that models the physical data acquisition process C R P n is the differentiation operator on the neighboring pixels Φ p (x) is the potential function that penalizes the roughness of the image β is the weight on prior term We use the generalized Geman prior function for Φ p (x) [De 01], which is convex and twice differentiable. By this choice, the regularized ML problem (1) is convex. Note that in the parallel beam CT scan geometry L is circulant, and approximately circulant in the fan-beam scan geometry [FB99]. As a comparison, if we use a Gaussian noise model, the cost function for regularized ML is given by f Gaussian (x) = log ((diag b) 1 y) Lx W + βφ(cx), (2) where W is the (fixed) estimate of the noise covariance matrix, and Φ(x) is the prior function. The first term is a quadratic function in x rather than a exponential function as in (1). 2

3 3 Optimality condition To solve the minimization problem, we find the gradient and Hessian matrix of (1) as and g = L T ( y b T exp( Lx) ) + βc T Φ (x), (3) H = L T DL + βc T MC > 0, (4) The diagonal matrices are given by D = diag{b 1 exp( l T 1 x),, b m exp( l T mx)}, and M = diag{φ (x 1 ),, Φ (x n )}, where Φ ( ) and Φ ( ) are the first and second order derivatives of the potential function. Since f(x) is differentiable and convex, the necessary and sufficient condition for a point x to be optimal is g(x ) = 0, which has no analytical solution. Thus it is solved by iterative algorithm x k+1 = x k + s k x k, (5) where x k is the search direction, and s k is the step size at iteration k. We use backtracking line search [BV04] to find the optimal s k. The stopping criterion for these iterative algorithms is g 2 ɛ with small ɛ > 0. It is still not clear so far which stopping criterion is the best for image reconstruction problems. 4 Algorithms 4.1 Gradient descent method The simplest method is to choose x = g, which has approximately linear convergence rate [BV04]. 4.2 Exact Newton method The exact Newton method directly solves the Newton system: H x = g for search direction. We used the backslash operator in MATLAB to solve this equation, which actually forms the (dense) Cholesky factorization of H. Newton method has quadratic convergence when x is near x. However, for realistic image size, the computational cost exact inversion of the Hessian matrix can be prohibitive. 4.3 Truncated Newton method The truncated Newton method solve Newton system using Conjugate gradient (CG) or Preconditioned CG (PCG) methods, by terminating the iteration before convergence [Boy07]. It is less reliable than Newton s method, but with good pre-conditioners, it can handle very large problems. Two commonly preconditioner for emission tomography (PET imaging) are: 3

4 diagonal preconditioner: M D = diag(h), combination of circulant and diagonal preconditioner [CPC + 93][FB99]: M CDC = Γ 1 F T Ω 1 F Γ 1, where F is the Fourier transform matrix, and the diagonal matrix Ω is approximately the spectrum of the following matrix K(β/c) = L T L + C T C F T ΩF, { } where ck(β/c) H with c = tr(lt DL), and Γ = diag i L2 ij D ii. mtr(l T L) i L2 ij The idea of M CDC is to diagonalize the approximate circulant matrix L by the a Fourier transform matrix; M CDC is said to be the best preconditioner for emission tomography problems [FB99]. 4.4 Approximate Newton methods The idea for approximate Newton method is to approximate H by a surrogate inversion is easy to compute. We considered using: Ĥ, whose Diagonal H: Ĥ = diag{h}. Precomputed fixed H We note from (4) the special structure of H and use a fixed Hessian matrix to calculate the Newton step: Ĥ = L T ˆDL + C T ˆMC, where D and M are calculated from the measured data y and the image reconstructed by FBP method x F BP. By doing so we fix the Hessian matrix and only need to invert it once. From our later numerical examples, we found that this proposed method approximate the exact Newton method well. One heuristic explanation is that the approximation of the Hessian by the fixed matrix is good when x is close to x, where the quadratic convergence of Newton method actually happens. So given a good initial guess, after sufficient number of iterations, this method has similar convergence rate to the exact Newton method. 4

5 4.5 WLSTR The WLSTR (weighted least square transmission reconstruction) method [DB05a] is a commonly used statistical image reconstruction method. It uses a Gaussian noise model. Each iteration of WLSTR is fast, but it takes many steps to converge to a suboptimal solution. The search direction of WLSTR is given by[db05a]: x = LT diag{y}(y Lx) L T diag{y}l1 (6) 4.6 Quadratic approximation of the likelihood function We can use quadratic approximation to the likelihood function to find an approximate search direction. A similar idea was presented in [EF99a]. However, we have an interesting observation that, if we denote the Radon transform of the search direction as ρ = L(x x k ) = L x, the quadratic approximation of the likelihood function can be written as f(x) f(x k ) + ( y I T exp( Lx) ) T ρ ρt Dρ, which is minimized by ρ = D 1 ( y I T exp( Lx) ). The inversion of matrix D is simple since it is diagonal. Then the searching direction can be found by solving the equation: L x = ρ. The proposed method is promising in that it may potentially convert the optimal sequence from the image space to the measured data (sinogram) space, so that the expensive forward and backward projection computations are avoided. Also, the Hessian matrix inversion is replaced by a diagonal matrix inversion. We are still not sure how to deal with the prior term, so in the implementation we use the search direction calculated from other methods for the second term. 4.7 Decomposition methods The decomposition approach groups variables or measurements into blocks and each time only solves a sub-problem. In doing so each iteration is less expensive, but more iterations may be needed (even no convergence) to a sub-optimal solution. For example, two existing approaches are: ICD (iterative coordinate method) [SB93], which divides the pixels into groups and in each iteration only updates one group; OS, (ordered subset method) [EF99b][EF99b], which divides the measured data into blocks. In each iteration only one group of data is used and these groups are used sequentially. It can be used with any of the methods presented above. It is proven that OS converges [De 01] to an sub-optimal solution. 5

6 5 Implementation To make our code more efficient, we use sparse matrix data type for L and C, and precompute Ax, A x, Cx, and C x outside the backtracking linear search loop. For CG and PCG algorithms, we do not have to actually form or invert the Hessian matrix but rather pass a function to MATLAB. In practice, the L and C matrices can still be too large to store. Note that we only need Lx, L T y, Cx in computing the search direction. So we can use efficient online implementations (such as Fourier transform based). The largest problem we can solve so far is the pixel image. The time it takes to solve this image with 200 rays and 360 measurements, using PCG with maximum 100 iterations per Newton step, is 5128 seconds (85 min). Forward backward projectors using more efficient codes (such as C mex) are need for larger image. From our numerical example results, we found that the reconstructed pixel values are mostly positive and the negative values are quite small. So herein no nonnegative constraints are used. With nonnegative constraint, we can use interior point to solve. We also tried to solve this problem using CVX (give reference). However, it seems that the exponential cost function takes forever to solve. 6 Numerical Examples We implemented a parallel beam CT geometry, with 100 detectors, and 180 uniform angular sampling. The rays are spread out wide enough to cover the entire image. The image has pixels (m = and n = 4096). We use I j = I 0 = 10 5 electrons/photon, lower than the common CT system (hence noisier data), where we expect the Poisson model to have some advantages than the Guassian noise model. We g 2 < 10 8 as a stopping criterion. The problem is solved on a IBM Laptop with Intel dual core CPU at 2.0 GHz and 1GB RAM. The actual computational time is listed in Table 1. Fig. 1 shows the difference between the reconstructed image and true image, using the deterministic methods and regularized ML methods with β = 10, β = 100, and β = 1000, respectively. We choose β = 100 for the following examples as a good tradeoff between noise level and resolution. Fig. 2 shows the likelihood function value as the number of iterations, for, for various methods. Fig. 3(a) shows g 2 versus the number of iterations, for the exact and approximate Newton method, using CG methods with the maximum number of CG steps N MAX = 10, 30, and 100, respectively. From the actual computational time listed in Table 1 N MAX = 30 wins. Fig. 3(b) shows g 2 versus the number of iterations for other methods. The exact Newton method converges with in 10 iterations, Newton method with diagonal approximate Hessian matrix uses less than 100 iterations, and Newton method using ICD converges within 1000 iterations. Newton method with quadratic approximate likelihood function, the Newton 6

7 method with OS, and WLSTR stagnate near g = Fig. 4 shows the g 2 as the number of CG iterations. In our case M D works best. For the pixel image case, the diagonally preconditioned CG speed up the exact Newton method by a ratio of 185:1059, and the CG without preconditioning by 185:206. The other preconditioner M CDC, uses more CG and Newton iterations and longer time to converge. We indeed has a case, pixel size image, where M CDC uses less number of iterations than the diagonal iteration, but each iteration takes longer time (due to the relatively complicated preconditioner). So overall it takes longer time. Finally, the peak errors in the reconstructed image, versus the number of iterations, is shown in Fig. 5. All the methods stagnate at certain peak error level due to the noise floor. Comparing the results in Fig. 5 with that in Fig. 3(b), we could argue that a stopping criterion g 2 < 10 2 would be good enough for this noise level. 7 Future work Given more time, we may explore the following aspects: Implement larger real instances, such as 3D imaging, with voxels (about 8 millions of variables).this requires efficient forward and backward projection functions to compute Lx and L T y. Compress sensing in CT imaging. The problem of reconstructed image from under sampling data can be formulated as L1 regularized ML problem minimize log(likelihood) + β x 1. Or using L1 regularization in other transform domain that has sparse structure: minimize log(likelihood) + β Φx 1, where Φ is some sparse transform such as DCT transform, or any other over complete basis. Similar approaches have been used in Magnetic resonance imaging (MRI) [LDPss], but has only recently become interested in the CT community. 7

8 Figure 1: The difference image of the reconstructed images in Fig.?? with the true image; (a): the deterministic method; and regularized ML with (b): β = 10, (c): β = 100, and (d): β = liks 1.6 x Newton, exact cg 100 gradient quadratic fixed H diagonal H WLSTR Newton, ICD Newton, OS iter Figure 2: Likelihood function value vs. number of outer loop iterations, for various methods. 8

9 cg 10 cg 30 cg 100 Newton, exact f iter f Newton, exact gradient quadratic fixed H diagonal H WLSTR Newton, ICD Newton, OS iter Figure 3: Norm of the gradient g 2 versus the number of iterations, using (a): exact and truncated Newton (CG) with N max = 10, 30, and 100, respectively; (b): other methods. f cg 10 cg 30 cg 100 pcg diag 10 pcg diag 30 pcg diag 100 pcg cdc 10 pcg cdc 30 pcg cdc pcgiter Figure 4: Norm of the gradient g 2 versus cumulative CG iterations, using CG, diagonally preconditioned PCG and M CDC conditioner PCG. 9

10 peak err Newton, exact gradient quadratic fixed H diagonal H WLSTR Newton, ICD Newton, OS iter Figure 5: Peak error x k iterations. x true in the reconstructed image versus number of TABLE 1: ACTUAL CONVERGING TIME (IN SECONDS) (listed only the time for methods converge within 1000 iterations) method pixels pixels Exact Newton cg cg cg pcg-diag pcg-diag pcg-diag pcg-cdc pcg-cdc pcg-cdc fixed H os icd References [Boy07] S. Boyd. EE 364b Notes. Winter [BV04] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press,

11 [CPC + 93] N. H. Clinthorne, T.-S. Pan, P.-C. Chiao, W. L. Rogers, and J. A. Stamos. Preconditioning methods for improved convergence rates in iterative reconstructions. IEEE Trans. on Med. Imaging, 12(1), March [DB05a] [DB05b] B. De Man and S. Basu. Efficient maximum likelihood reconstruction for transmission tomography. 14th International Conference of Medical Physics, Nuremberg, Germany, Sept B. De Man and S. Basu. Generalized Geman prior for iterative rconstruction. 14th International Conference of Medical Physics, Sept [DBT + 05] B. De Man, S. Basu, J.-B. Thibault, J. Hsieh, J. Fessler, C. Bouman, and K. Sauer. A study of four minimization approaches for iterative reconstruction in X-ray ct IEEE Nuclear Science Symposium Conference Record, pages , [De 01] [EF99a] [EF99b] [FB99] Bruno De Man. Iterative reconstruction for reduction of metal artifacts in computed tomography. PhD thesis, University of Leuven, Leuven-Heverlee, Belgium, May H. Erdogan and J. A. Fessler. Monotonic algorithms for transmission tomography. IEEE Trans on Med. Imaging, Nov H. Erdogan and J. A. Fessler. Ordered subsets algorithms for transmission tomography. Phys. Med. Biol., Nov J. A. Fessler and S. D. Booth. Conjugate-gradient preconditioning methods for shift-variant PET image reconstruction. IEEE Tran. Med. Imaging, 8(5): , Sept [LDPss] M. Lustig, D. L. Donoho, and J. M. Pauly. Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson Med, in press. [SB93] K. Sauer and C. Bouman. A local update strategy for iterative reconstruction from projections. IEEE Trans. Signal Proc., 41(2): ,