A Comparison of Some State of the Art Image Denoising Methods

Transcription

1 A Comparson of Some State of the Art Image Denosng Methods Hae Jong Seo, Pryam Chatterjee, Hroyuk Takeda, and Peyman Mlanfar Department of Electrcal Engneerng, Unversty of Calforna at Santa Cruz Abstract We brefly descrbe and compare some recent advances n mage denosng. In partcular, we dscuss three leadng denosng algorthms, and descrbe ther smlartes and dfferences n terms of both structure and performance. Followng a summary of each of these methods, several examples wth varous mages corrupted wth smulated and real nose of dfferent strengths are presented. Wth the help of these experments, we are able to dentfy the strengths and weaknesses of these state of the art methods, as well as seek the way ahead towards a defntve soluton to the long-standng problem of mage denosng. I. INTRODUCTION Denosng has been an mportant and long-standng problem n mage processng for many decades. In the last few years, however, several strong contenders have emerged whch produce stunnng results across a wde range of mage types, and for vared nose dstrbutons, and strengths. The emergence of multple very successful methods n a relatvely short perod of tme s n tself nterestng, n part because t ponts to the possblty that we may be approachng the lmts of performance for ths problem. At the same tme, t s nterestng to note that these methods share an underlyng lkeness n terms of ther structure, whch s based on nonlnear weghted averages of pxels, where the weghts are computed from metrc smlarty of pxels, or neghborhoods of pxels. The sad weghts are computed by gvng hgher relevance to nearby pxels whch are more spatally, and tonally smlar to a gven reference patch of nterest. In ths sense, as we wll see below, they are all based on the dea of usng a kernel functon whch controls the level of nfluence of smlar and/or nearby pxels. Overall, two mportant problems present themselves. Frst, what are the fundamental performance bounds n mage denosng, and how close are we to them? And second, what makes these kernel-based methods so successful, can they be mproved upon, and how? Whle we do not ntend to address ether of these questons n ths paper, we do take a modest step n exposng the smlartes, strengths, and weaknesses of these competng methods, pavng the way for the resoluton of the more fundamental questons n future work. Ths work was supported n part by AFOSR Grant F II. NONPARAMETRIC KERNEL-BASED METHODS In ths secton, we gve descrptons of three algorthms. We dscuss Data-adaptve Kernel Regresson of Takeda et al. [1], Non-local Means of Buades et al. [2], and Optmal Spatal Adaptaton of Kervrann, et al. [3]. A. Data-Adaptve Kernel Regresson The kernel regresson framework defnes ts data model n 2-D as y = z(x ) + ε, = 1,, P, x = [x 1, x 2 ] T, (1) where y s a nosy sample at x, z( ) s the (htherto unspecfed) regresson functon to be estmated, ε s an..d zero mean nose, and P s the total number of samples n a neghborhood (wndow) of nterest. Whle the specfc form of z( ) may reman unspecfed, we can rely on a generc local expanson of the functon about a samplng pont x. Specfcally, f x s near the sample at x, we have the N-th order Taylor seres z(x ) z(x) + { z(x)} T (x x) (x x) T {Hz(x)}(x x) + (2) = β 0 +β T 1 (x x)+β T 2 vech{ (x x)(x x) T} +,(3) where and H are the gradent (2 1) and Hessan (2 2) operators, respectvely, and vech( ) s the half-vectorzaton operator whch lexcographcally orders the lower trangular porton of a symmetrc matrx. Furthermore, β 0 s z(x), whch s the pxel value of nterest. Snce ths approach s based on local approxmatons and we wsh to preserve mage detal as much as possble, a logcal step to take s to estmate the parameters {β n } N n=0 from all the samples {y } P =1 whle gvng the nearby samples hgher weghts than samples farther away n spatal and radometrc terms. A formulaton of the fttng problem capturng ths dea s to solve the followng optmzaton problem, mn {β n } N n=0 P y β 0 β T 1 (x x) =1 β T 2 vech { (x x)(x x) T} q K adapt (x x, y y) (4)

2 where q s the error norm parameter (q = 2 or 1 typcally), N s the regresson order (N = 2 typcally), and K adapt (x x, y y) s the data-adaptve kernel functon. Takeda et al. ntroduced steerng kernel functons n [1]. Ths data-adaptve kernel s defned as K steer (x x, y y) = K H (x x), (5) where H s the (2 2) steerng matrx, whch contans four parameters. One s a global smoothng parameter whch controls the smoothness of an entre resultng mage. The other three are the scalng, elongaton, and orentaton angle parameters whch capture local mage structures. We estmate those three parameters by applyng sngular value decomposton (SVD) to a collecton of estmated gradent vectors n a neghborhood around every samplng poston of nterest. Wth the steerng matrx, the kernel contour s able to elongate along the local mage orentaton. In order to further enhance the performance of ths methods, we apply orentaton estmaton followed by steerng regresson repeatedly to the outcome of the prevous step. We call the overall process teratve steerng kernel regresson (ISKR). Returnng to the optmzaton problem (4), the mnmzaton eventually provdes a pont-wse estmator of the regresson functon. For nstance, for the zeroth regresson order (N = 0) and q = 2, we have the estmator n the general form of: P =1 ẑ(x) = K adapt(x x, y y) y P =1 K adapt(x x, y y). (6) B. Non-Local Means The Non-Local Means (NLM) method of denosng was ntroduced by Buades et al. [2] where the authors use a weghted averagng scheme to perform mage denosng. They make use of the fact that n natural mages a lot of structural smlartes are present n dfferent parts of the mage. The authors argue that n the presence of uncorrelated zero mean Gaussan nose, these repettve structures can be used to perform mage restoraton. The estmator of the non-local means method s expressed as =j ẑ NL (x j ) = K (y hs,hr w j y w ) y =j K (y hs,hr w j y w ), (7) where y w s a column-stacked vector that contans the gven data n a patch w (the center of w beng at x ): y w = [ y l ] T, y l w. (8) The kernel functon s defned as the weghted Gaussan kernel: { } K hs,hr (y wj y w ) = exp y w j y w 2 W hs h 2, (9) where the weght matrx W hs s gven by W hs = dag {, K hs (x j 1 x j ), K hs (0), K hs (x j+1 x j ), }, (10) h s and h r are the parameters whch control the degree of flterng by regulatng the senstvty to the neghborhood dssmlartes 1, and K hs s defned as the Gaussan kernel functon. Ths essentally mples that the restored sgnal at poston x j s a lnear combnaton (weghted mean) of all those gven data whch exhbt a largely smlar (Gaussanweghted) neghborhood. The method, as presented n theory, results n an extremely slow mplementaton due to the fact that a neghborhood around a pxel s compared to every other pxel neghborhood n the mage n order to calculate the contrbutng weghts. Thus for an mage of sze M M, the algorthm runs n O(M 4 ). Such drawbacks have been addressed n some recent publcatons mprovng on the executon speed of the nonlocal means method [4], [5], whle modestly compromsng on the qualty of the output. In summary, the mplementaton of the work bols down to the pseudocode descrbed n algorthm 1. Algorthm 1 Non-Local Means algorthm y Nosy Image z Output Image h r, h s Flterng parameters for every pxel y j y do w j patch wth y j at the center W j search wndow for w j for every w { W j and j } do K() exp yw ywy 2 W hs h 2 r ẑ(x j ) ẑ(x j ) + K() y end for z(x j ) ẑ(x j )/ K() end for C. Optmal Spatal Adaptaton Whle the related NLM method s controlled by smoothng parameters h r, h s calbrated by hand, the method of Kervrann et al. [3] called optmal spatal adaptaton (OSA) mproves upon Non-Local Means method by adaptvely choosng a local wndow sze. The key dea behnd ths method s to teratvely grow the sze of a local search wndow W startng wth a small sze at each pxel and to stop the teraton at an optmal wndow sze. The dmensons of the search wndow grow as (2 l +1) (2 l +1) where l s the number of teratons. To be more specfc, suppose that ẑ (0) (x ) and ˆv (0) are the ntal estmate of the pxel value and the local nose varance at x, whch are ntalzed as ẑ (0) (x ) = y, ˆv (0) = ˆσ 2, (11) 1 A large value of h r results n a smoother mage whereas too small a value results n nadequate denosng. The choce of ths parameter s largely heurstc n nature.

3 ISKR NLM OSA σ = 50 σ = 25 σ = 15 Nosy mages Fg. 1. Examples of whte Gaussan nose reducton: The columns left through rght show the nosy mage and the restored mages by ISKR [1], NLM [2], OSA [3]. The rows from top to down are showng the experments wth dfferent standard devatons (σ = 15, 25, 50). The correspondng PSNR values are shown n Table I. where σ s an estmated standard devaton. In each teraton, the estmaton of each pxel s updated based on the prevous teraton as follows. P K (z w z w ) yj (ℓ+1) j xj W H z (x ) = P (12) K (z w z w ) j xj W H where z w s a column stack vector that contans the pxels 1 2, and h n an mage patch w, H r s the j = hr (Vj ) smoothng parameter. The matrx V j contans the harmonc means of estmated local nose varances: " # (v 1 )2 (v j 1 )2 (v )2 (v j )2 1 V j = dag, 2,, (13) 2 (v 1 ) + (v j 1 )2 (v )2 + (v j )2 and K s defned as the Gaussan kernel functon: ( ) 1 (z w z wj )T (V (z w z wj ) j ) KH (z w z wj ) = exp. h2r (14) A patch sze p s consdered to be able to take care of the local geometry and texture n the mage and s fxed (e.g 9 9 or 7 7) whle the sze of a local search wndow W s grows teratvely, determned by a pont-wse statstcallybased stoppng rule. The optmal wndow sze s determned by mnmzaton of the local mean square error (MSE) estmate at each pxel wth respect to the search wndow sze. In the absence of ground truth, ths s approxmated as the upper bound of the MSE obtaned by estmatng the bas and the varance separately. Ths estmaton process s presented n detal n [3]. III. E XPERIMENTS In ths secton, we wll compare the denosng performance of the methods ntroduced n the prevous secton by usng synthetc and real nosy mages. For all the experments, we chose q = 2 and N = 2 for ISKR. The frst denosng experment s shown n Fg. 1. For ths experment, usng the Lena mage, we added whte

4 The fsh mage ISKR NLM OSA Fg. 2. Fsh denosng examples: The mages n the frst row from left to rght llustrate the nosy mage, the estmated mages by ISKR, NLM, and OSA method, respectvely, and the second row llustrate absolute resdual mages n the lumnance channel. TABLE I THE PSNR VALUES OF THE EXAMPLES OF WHITE GAUSSIAN NOISE REDUCTION (FIG. 1). STD (σ) Nosy SKR NLM OSA Gaussan nose wth three dfferent standard devatons (σ = 15, 25, 50). The synthetc nosy mages are n the frst column of Fg. 1, and the denosed mages by ISKR, NLM, and OSA are shown n the second, thrd, and fourth columns, respectvely. The correspondng PSNR 2 values are shown n Table I. For ISKR and NLM, we chose the parameters to produce the best PSNR values. The OSA method automatcally chose ts smoothng parameter. Next, we appled the three method to some real nosy mages: Fsh and JFK mages. The nose statstcs are unknown for all the mages. Applyng ISKR, NLM, and OSA n Y C b C r channels ndvdually, the restored mages are llustrated n the frst rows of Fgs. 2 and 3, respectvely. To compare the performances of the denosng methods, we take the absolute resduals n the lumnance channel, whch are shown below the correspondng denosng results of each method. ( ) Peak Sgnal to Nose Rato = 10log 2 10 [db] Mean Square Error IV. CONCLUSION Whle the present study s modest n ts scope, several nterestng but prelmnary conclusons do emerge. Frst, we consder the relatve performance of the consdered methods. Whle very popular recently, the NLM method s performance, measured both qualtatvely and quanttatvely, s nferor to the other two methods. Ths s a bt surprsng gven the relatvely recent surge of actvty n ths drecton. The computatonal complexty of the NLM method s also very hgh, but as we mentoned earler, ths s a problem that has recently been addressed [4], [5]. The other two methods (ISKR and OSA) are very close n performance, wth OSA havng a slght edge n terms of PSNR. However, as the authors have also stated n ther paper [3], ths method tends to do less well when there s excessve texture present n the mage. The ISKR algorthm suffers from a smlar, but somewhat mlder verson of the same problem. A good comparson of these effects can be seen n Fg. 2. The OSA method s performance depends strongly on the ntal estmate of the nose varance, whch can be badly based f the assumptons of Gaussan nose statstcs are volated. Indeed, f the estmated varance s much hgher than the correct nose varance, ths method can perform rather poorly. As such, t s worth pontng out that n the real experments reported n ths paper (Fgs. 2 and 3) the automatcally estmated nose varance led to rather poor results for OSA. Therefore, we adjusted ths value by hand untl the most vsually appealng result was obtaned. To be far, we followed the same lne of thnkng and chose the

5 The JFK mage ISKR NLM OSA Fg. 3. JFK denosng examples: The mages n the frst row from left to rght llustrate the nosy mage, the estmated mages by ISKR, NLM, and Kevrann s method, respectvely, and the second row llustrate absolute resdual mages n the lumnance channel. parameters for ISKR and NLM as well to yeld the best vsual results. Whle the ISKR does not depend on an explct estmate or knowledge of the underlyng nose varance (or dstrbuton), several parameters such as wndow sze, and the number of teratons, must be set by hand. Regardng the latter, f the teratons are contnued, the mage becomes ncreasngly blurry and MSE rses. Also, the ISKR s computatonally very ntensve, and efforts must be made n order to mprove ths aspect of the algorthm. In terms of possble mprovements, for all consdered methods, there s room for growth and further nnovaton. In terms of both NLM, and OSA, t s worth notng that the weghts produced by these methods for local pxel processng are always restrcted to be non-negatve numbers. Ths s an nherent lmtaton whch can be overcome, and should lead to mproved performance. For the ISKR, the choce of novel teraton methods; a proper stoppng rule (lmtng the number of teratons) based on the analyss of resduals of the estmaton process; and reducton of computatonal complexty are all mportant ssues for future research. [3] C. Kervrann and J. Bourlanger, Optmal spatal adapaton for patchbased mage denosng, IEEE Transactons on Image Processng, vol. 15, no. 10, October [4] M. Mahmoud and G. Sapro, Fast mage and vdeo denosng va nonlocal means of smlar neghborhoods, Sgnal Processng Letters, IEEE, vol. 12, no. 12, pp , [5] M. V. Radu Cpran Blcu, Fast nonlocal means for mage de-nosng, n Proceedngs of IS&T/SPIE Symposum on Electronc Imagng, Dgtal Photography III Conference, vol. 6502, San Jose, Calforna USA, January-February REFERENCES [1] H. Takeda, S. Farsu, and P. Mlanfar, Kernel regresson for mage processng and reconstructon, IEEE Transactons on Image Processng, vol. 16, no. 2, pp , February [2] A. Buades, B. Coll, and J. M. Morel, A revew of mage denosng algorthms, wth a new one, Multscale Modelng and Smulaton (SIAM nterdscplnary journal), vol. 4, no. 2, pp , 2005.