ANALYSIS OF p-norm REGULARIZED SUBPROBLEM MINIMIZATION FOR SPARSE PHOTON-LIMITED IMAGE RECOVERY

Transcription

1 ANALYSIS OF p-norm REGULARIZED SUBPROBLEM MINIMIZATION FOR SPARSE PHOTON-LIMITED IMAGE RECOVERY Aramayis Orkusyan, Lasith Adhikari, Joanna Valenzuela, and Roummel F. Marcia Department o Mathematics, Caliornia State University, Fresno, Fresno, CA 9374, USA Applied Mathematics, University o Caliornia, Merced, Merced, CA 95343, USA ABSTRACT Critical to accurate reconstruction o sparse signals rom low-dimensional low-photon count observations is the solution o nonlinear optimization problems that promote sparse solutions. In this paper, we explore recovering high-resolution sparse signals rom low-resolution measurements corrupted by Poisson noise using a gradientbased optimization approach with non-convex regularization. In particular, we analyze zero-inding methods or solving the p-norm regularized minimization subproblems arising rom a sequential quadratic approach. Numerical results rom luorescence molecular tomography are presented. Index Terms Photon-limited imaging, nonconvex optimization, sparse reconstruction, l p -norm, luorescence molecular tomography. INTRODUCTION Photon-limited data observations generally ollow a Poisson distribution with a certain mean detector photon intensity, i.e., y Poisson(A ), where y Z m + is a vector o observed photon counts, R n + is the vector o true signal intensity, and A R+ m n is the system matrix that linearly projects the true signal to the detector photon intensity []. The Poisson reconstruction problem has the ollowing constrained optimization orm: minimize Φ() F () + τ pen() R n subject to, () where F () is the negative Poisson log-likelihood unction F () = T A m i= y i log(e T i A + β), where is the m-vector o ones, e i is the i-th column o the m m identity matrix, β > (typically β ), pen : R n R is a sparsity-promoting penalty unctional, and τ > is a regularization parameter. This work was supported by NSF Grants CMMI and DMS Various convex penalty techniques have previously been used as regularization terms in (). For example, when the solution is sparse in the canonical basis, an l norm is simple to implement [, 3]. A total variation seminorm with a split Bregman approach can also be used [4, 5]. Other related methods include [6, 7, 8, 9]. In this work, we consider the non-convex penalty unction pen() = p p = n i= i p ( p < ) in () as a bridge between the convex l norm and the l counting seminorm [,, ]. The solution to this nonconvex problem can be ound by minimizing a sequence o quadratic models to the unction F () approximated by second-order Taylor series expansion where the Hessian replaced by a scaled identity matrix α k I with α k > [3, 4, 5]. Simpliying the second-order approximation yields a sequence o subproblems o the orm k+ = arg min R n sk + τ α k p p subject to, () where s k = k α k F ( k ). Note that the subproblem () can be separated into scalar minimization problems o the orm s = arg min Ω s () = R ( s) + λ p, subject to. (3) where and s denote elements o the vectors and s k respectively and λ = τ/α k [6]. Given a regularization parameter λ > and p-norm or Ω s () in (3), there exists a threshold value γ p (λ) (that explicitly depends on p and λ) such that i s γ p (λ), the global minimum o (3) is s = ; otherwise, the global minimum will be a non-zero value (see Fig. ). When s = γ p (λ), there exists γ such that Ω γ ( γ ) = Ω γ () and Ω γ( γ ) =. (4) By solving (4) simultaneously, we can explicitly ind the threshold value γ p (λ) or given p and λ values. Speciically, γ p (λ) is given by γ p (λ) = (λ( p)) p +λp(λ(

2 .8 Ω s ().3 Ω γ () In order to simpliy the computation o this iteration and avoid computing two dierent roots n p and n p, we multiply the numerator and denominator by p n : (a).. Ω γ ( )=Ω γ ( γ ) γ.5 (b) n+ = p sn p + λp(p ) n. (7) n + λp(p ) The perormance o ixed-point iteration and Newton s method very much depend on the choice o the initial point, which we discuss next. Fig.. The plot o the scalar quadratic unction Ω s (), where p =.5 and λ =.. (a) When s is less than the speciic threshold value γ p (λ), then s = is the unique global minimum. (b) When s = γ p (λ), there are global minima at = and γ. I s > γ p (λ), then the global minimum is uniquely at some s >. p)) p p (see [7] or details). For any s > γp (λ), the unique minimum s o Ω s () is greater than and is obtained by setting Ω s to : Ω s( s ) = s s + λp( s ) p =. (5) We now describe zero-inding algorithms to compute the root s. To our knowledge, this work is the irst careul analysis o these minimization techniques or solving the non-convex quadratic subproblem given by (3).. ZERO-FINDING METHODS.. Fixed-Point Iteration Method A point is said to be a ixed point o a unction G() i G( ) =. Setting Ω s() equal to zero, we have s λp( ) p =. The ixed-point iteration method is an iterative method or inding ixed points o a unction. In particular, it deines a sequence o points { n } given by n+ = G( n ). Previous methods or inding the root o Ω s() use the ixed point iteration (see, e.g., [6, 7]):.. Newton s Method n+ = g( n ) = s λp p n. (6) Note that there are various ways o deining ixed point iterations. One particular ixed-point ormulation is Newton s method, which is given by the iterations n+ = G( n ) = n Ω s( n ) Ω s ( n ). In our case, the iterations or Newton s method are given by n+ = n n s + λpn p + λp(p )n p = s + λp(p ) p n + λp(p ) p n..3. Initialization When s = γ p (λ), the solution γ such that Ω ( γ ) = γ γ p (λ) + λp( γ ) p = is given explicitly by γ = (λ( p)) p. Then i s = γ p (λ) + ε or some ε >, we now analyze how to estimate s to initialize the zero-inding methods described previously. First-order Taylor series approximation. To deine the initial point, we can linearize Ω s() around γ and ind the zero o the linearization. More speciically, Ω s( γ + δ) Ω s( γ ) + δω s ( γ ) = γ (γ p (λ) + ε) + λp( γ ) p + δ( + λp(p )( γ ) p ) = ε + δ( + λp(p )( γ ) p ). Setting this equal to zero and solving or δ suggests the use o the initialization s = γ + δ, where δ = ε + λp(p )( γ ) p. Second-order Taylor series approximation. Similarly, we can use a second-order Taylor approximation to Ω s around γ : Ω s( γ + δ) Ω s( γ ) + δω s ( γ ) + δ Ω s ( γ ), which yields the ollowing approximation: s = γ + δ, where δ = b + b 4ac, a where a = λp(p )(p ) ( γ ) p 3, b = +λp(p )( γ ) p, and c = ε. The linearization and second-order Taylor approximation, however, diverge quickly rom the true solution as s becomes large (see Fig. ). We now discuss bounds on s that allow us to make more eective initial approximations to s. We irst prove a lemma, which will be useul in showing bounds on s as well as other results.

3 5 4 3 Quadratic Approximation Linear Approximation γ l()= s Ω γ() Fig.. Approximations to Ω γ() centered at γ. As increases, both the linear and quadratic Taylor approximation diverge rom Ω γ(). In contrast, the approximation l() = s, which are the irst two terms in Ω γ(), is more accurate or large values o. Lemma. Let λ > and p <. Then or s γ p (λ), λp( p)( s ) p p. Proo. Recall that or s = γ p (λ), there exists an γ > such that (4) hold. From Ω γ ( γ ) = Ω γ (), we can obtain and rom Ω γ( γ ) =, we have γ + λ( γ ) p = γ p (λ), (8) γ + λp( γ ) p = γ p (λ). (9) Setting (8) equal to (9) and with some algebraic manipulation, we have λp( p)( γ ) p = p. For s > γ p(λ), the unique minimizer s > γ. Thus, λp( p)( s ) p < λp( p)( γ ) p = p, which completes the proo. This result allows us to prove the ollowing theorem, which bounds the minimizer, s, o Ω s (): Theorem. For λ > and p <, the minimizer, s, o Ω s is bounded by s s. I p, then the minimizer is urther bounded by 3 s s s. Proo. Recall that the minimizer o Ω s solves Ω s( s ) =. Solving or s, we have s = s + λp(s. () ) p Rewriting the main result o Lemma, λp(s ) p p ( p). Observe that i p, λp( s ) p p ( p) we obtain and +λp( s ) p 3. Using these bounds in () yields the desired results. Note that Theorem implies that as s increases, so does s. Moreover, as s, ( s ) p, and thereore, by (5), s s. Thus, a sensible initial estimate or s is s. Fixed-point initialized Newton s Method. We can improve the initial guess rom s by inding a point between s and s. The mean-value theorem guarantees the existence o ξ (s, s) such that Ω s (ξ) = Ω s (s) Ω s ( s ) s. s Rearranging, we ind that s = s Ω s(s) Ω s( s ) Ω s (ξ) = s λps p λp( p)ξ p. By Lemma, p λp( p)ξ p, and thus, s s λps p (, s). We note that this is precisely the irst ixed point iteration initialized at s..4. Convergence Guarantee o convergence. Let e n = n and e n+ = n+ represent the errors on the n-th and n+-th iterations respectively. For ixed point iteration, we have e n+ = n+ = G( n ) = G( + e n ), = G( ) + e n G ( ) + e ng (ξ) = + e n G ( ) + e ng (ξ) = e n G ( ) + e ng (ξ). For small e n, e n+ e n G ( ). In our context, G() = s λp p and G () = λp( p) p. By Lemma, G () <. Thereore, the error is decreasing and the ixed point iteration method is guaranteed to converge. To show Newton s method is guaranteed to converge, let c be a critical point o Ω s() i.e. Ω s ( c ) =. In particular, c = (λp( p)) p and or any > c, Ω s () = + λp(p ) p > i.e. Ω s() is increasing in the interval ( c, ). Then, Ω s () = λp(p )(p ) p 3 > or all (, ), which implies Ω s() is convex. Finally, we note that c < (λp(p )) p = γ, i.e Ω s() has a root in ( c, ). Thereore, Ω s() is increasing, convex, and has a zero in ( c, ), and Newton s method is guaranteed to converge rom any starting point in the interval ( c, ) (see Theorem pg. 86 in [8]). Rate o convergence. Let ε be some set tolerance such that on the n-th iteration i e n = n s ε then we

4 will consider the algorithm to have converged to the root. For ixed point iteration, we have convergence when ε e n = C e n = C n e () where C = G ( s ) = λp( p)( s ) p. Solving or n, the number o iterations required to converge, we have n Fixed Point ln ε ln e ln C. () For Newton s method, we have convergence when ε e n = C e n = C n e n (3) where C = λp( p)( p)(s ) p 3 λp( p)(s ) p. Solving or n in (3) yields n Newton ( ) ln ln ln C + ln ε. (4) ln C + ln e Fig. 3 shows the theoretical number o iterations or ixed-point iterations and Newton s method to converge. Note that when s is near γ p (λ), ixed-point iterations take many more iterations than Newton s method. However, or large s, ixed-point iterations only require our iterations. Although this is still twice as many as the iterations or Newton s method, the number o loating point operations or ixed-point iterations is much smaller than that or Newton s method (compare (6) and (7)). Since s can take on any real value, we expect the average perormance o ixed-point iteration and Newton s method will be comparable, which we see in the next section. is set to 7, inside the two rods and outside. We chose a total o excitation source positions and, 57 detector positions on the top surace o the cube, which gives us,57 =,4 measurements. About one-tenth o all the measurements were used (i.e., measurements). We assumed that the excitation wavelength is 65 nm and the emission wavelength is 7 nm in the construction o the system matrix A. The tissue optical properties were µ a =. mm, µ s =.4 mm at both 65 nm and at 7 nm. For this experiment, the simulated measurement vector y is corrupted by Poisson noise with signal-to-noise ratio (SNR) o 3 db ( 57% noise). In our method, we used p =.74 and A T y as the initial guess. Fig. 4 shows the true signal ( ) and our reconstruction. Time (sec) Iterations Fixed-point iteration.89,8,974 Newton s method.8 476,585 Table. Time and iteration average over trials or ixed-point iteration and Newton s method to reconstruct the luorescence molecular tomography data. (a) x (b) x Fig. 4. (a) Horizontal slices o a simulated luorescence capillary rod targets. (b) Reconstruction using p-norm regularized subproblem minimization. 4. CONCLUDING REMARKS Fig. 3. Theoretical number o iterations required to converge as a unction o s. Here p =.5, λ =, ε = 8, e = s, and γ p (λ) s. 3. NUMERCAL EXPERIMENTS We simulated a 3D cubic phantom with two embedded luorescence capillary rod targets (see e.g., [9, ]). For the inite element mesh, there are a total o 8,69 nodes inside the 3D cube while only 36 nodes are located inside the two rods. The luorophore concentration o the nodes In this paper, we analyzed methods or solving the p-norm regularized subproblems arising rom minimizing the Poisson-log likelihood or reconstructing sparse signals rom photon-limited measurements. These nonconvex subproblems do not have closed orm solutions, and as such, they require numerical approaches or computing the minimizers. While Newton s method in theory should converge to the solution aster than ixedpoint iterations, the number o loating-point operations needed to perorm each iteration osets the computational advantage o using derivative inormation. Acknowledgments. We thank Dr. Dianwen Zhu and Pro. Changqing Li or providing the numerical codes or generating the luorescence molecular tomography data.

5 5. REFERENCES [] D. L. Snyder and M. I. Miller, Random point processes in space and time, Springer-Verlag, New York, NY, 99. [] D. Lingenelter, J. Fessler, and Z. He, Sparsity regularization or image reconstruction with Poisson data, in Proc. SPIE Computational Imaging VII, 9, vol [3] Z. T. Harmany, R. F. Marcia, and R. M. Willett, Sparse Poisson intensity reconstruction algorithms, in Proceedings o IEEE Statistical Signal Processing Workshop, Cardi, Wales, UK, September 9. [4] M. Figueiredo and J. Bioucas-Dias, Deconvolution o Poissonian images using variable splitting and augmented Lagrangian optimization, in Proc. IEEE Workshop on Statistical Signal Processing, 9. [5] S. Setzer, G. Steidl, and T. Teuber, Deblurring Poissonian images by split Bregman techniques, J. Visual Commun. Image Represent., 9, In Press. [6] J. A Fessler and A. O. Hero, Penalized maximum-likelihood image reconstruction using space-alternating generalized em algorithms, IEEE Trans. Image Processing, vol. 4, no., pp , 995. [7] S. Ahn and J. Fessler, Globally convergent image reconstruction or emission tomography using relaxed ordered subsets algorithms, IEEE Trans. Med. Imaging, vol., no. 5, pp , May 3. [8] J. A. Fessler and H. Erdoğan, A paraboloidal surrogates algorithm or convergent penalized-likelihood emission image reconstruction, in Proc. IEEE Nuc. Sci. Symp. Med. Im. Con., 998. [9] M. Raginsky, R. M. Willett, Z. T. Harmany, and R. F. Marcia, Compressed sensing perormance bounds under Poisson noise, IEEE Trans. Signal Process., vol. 58, no. 8, pp ,. [] R. Chartrand and W. Yin, Iteratively reweighted algorithms or compressive sensing, in Proc. IEEE International Conerence on Acoustics, Speech, and Signal Processing, 8, pp [3] J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., vol. 8, no., pp. 4 48, 988. [4] S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Processing, vol. 57, no. 7, pp , 9. [5] Z. T. Harmany, R. F. Marcia, and R. M. Willett, This is SPIRAL-TAP: Sparse Poisson intensity reconstruction algorithms - theory and practice, IEEE Trans. Image Processing, vol., no. 3, pp ,. [6] L. Adhikari and R. F. Marcia, Nonconvex relaxation or Poisson intensity reconstruction, in Proc. IEEE International Conerence on Acoustics, Speech, and Signal Processing, Brisbane Australia, April 5. [7] W. Zuo, D. Meng, L. Zhang, X. Feng, and D. Zhang, A generalized iterated shrinkage algorithm or nonconvex sparse coding, in 3 IEEE International Conerence on Computer Vision (ICCV), Dec 3, pp [8] D. R. Kincaid and E. W. Cheney, Numerical Analysis: Mathematics o Scientiic Computing, American Mathematical Society, 3rd edition,. [9] C. Li, G. S. Mitchell, J. Dutta, S. Ahn, R. M. Leahy, and S. R. Cherry, A three-dimensional multispectral luorescence optical tomography imaging system or small animals based on a conical mirror design, Opt. Express, vol. 7, no. 9, pp , Apr 9. [] F. Fedele, J. P. Laible, and M. J. Eppstein, Coupled complex adjoint sensitivities or requency-domain luorescence tomography: theory and vectorized implementation, Journal o Computational Physics, vol. 87, no., pp , 3. [] R. Chartrand, Exact reconstruction o sparse signals via nonconvex minimization, Signal Processing Letters, IEEE, vol. 4, no., pp. 77 7, Oct 7. [] R. Chartrand and V. Staneva, Restricted isometry properties and nonconvex compressive sensing, Inverse Problems, vol. 4, no. 3, pp. 35, 8.