FREQUENCY DOMAIN BLIND DECONVOLUTION IN MULTIFRAME IMAGING USING ANISOTROPIC SPATIALLY-ADAPTIVE DENOISING

Transcription

1 FREQUENCY DOMAIN BLIND DECONVOLUTION IN MULTIFRAME IMAGING USING ANISOTROIC SATIALLY-ADATIVE DENOISING Vladimir Katkvik, Dmitriy aliy, Kare Egiazaria, ad Jaakk Astla Istitute Sigal rcessig, Tampere Uiversity Techlgy, OBx 553, FIN-330, Tampere, Filad ABSTRACT I this paper we preset a vel methd r multirame blid deblurrig isy images It is based miimizati the eergy criteri prduced i the requecy dmai usig a recursive gradiet-precti algrithm Fr lterig ad regularizati we use the lcal plymial apprximati (LA bth the image ad blur peratrs, ad paradigm the itersecti c dece itervals (ICI applied r selecti adaptively varyig scales (widw sizes LA The LA-ICI algrithm is liear ad spatially-adaptive with respect t the smthess ad irregularities the image ad blur peratrs Simulati experimets demstrate e ciecy ad gd perrmace the prpsed decvluti techique INTRODUCTION Image prcessig based multiple bservatis e scee aims t ehace cmprehesive restrati quality, te whe kwledge abut image rmati is icmplete Classical elds applicati are the astrmy, remte sesig, medical imagig, etc Multisesr data di eret spatial, tempral, ad spectral reslutis are explited r image sharpeig, imprvemet registrati accuracy, eature ehacemet, ad imprved classi cati Other examples ca be see i digital micrscpy, where the same specime may be recrded at several di eret cus settigs r i multispectral radar imagig thrugh a scatterig medium which has di eret traser uctis at di eret requecies Image restrati is a iverse prblem which assumes havig a prir irmati abut the rmati mdel This mdel icludes all srts distrtis related t the image degradati Fr istace, the atmspheric turbulece, the relative mti betwee a bect ad the camera, the ut--cus camera, the variatis i ptical ad electric imagig cmpets, etc Cvetially, the image acquisiti is mdelled by the cvluti with the pit-spread ucti (SF ad ise The SF itrduces lw-pass distrtis it a image which are called te as blur Whe the blur is ukw, the image restrati becmes a blid iverse prblem r blid decvluti Fr multiple bservatis e scee, it is a multirame, r multichael, blid iverse prblem A theretical breakthrugh the blid ad -blid decvluti techiques has bee de i wrks perect blur ad image recstructi With the blur uctis satisyig certai c-primeess requiremets the existece ad uiqueess the sluti is guarateed uder quite urestrictive cditis, ie bth the blur ad the rigial image ca be determied exactly i the absece ise, ad stably estimated i its presece [, 2, 3] A umber wrks have bee de t deal with isy data I particular, the blid decvluti based the Bussgag lters is prpsed i [4] The iverse lter is build as a liear apprximati the ptimal Wieer decvluti lter Blid ise-resistat decvluti algrithms based the least square methd has bee prpsed i [5] The criteri icludes the stadard quadratic delity term as well as a quadratic term the crss-chael balace Overall, the criteri is quadratic as the ttal variati (TV ad Mumrd-Shah eergy uctials are used as the regularizatrs These quadratic terms, r pealty uctis, the criteri result i a liear edge-preservig lterig [7, 8, 9] It is shw i [5] that the prpsed algrithm usig this srt regularizati perrms quite well The vel apprach btaied as a urther develpmet [5] was prpsed i the recet paper [6] The mai emphasize this wrk is de multichael deblurrig spatially misaliged images The prpsed algrithm des t require the accurate size supprts blur uctis, ad the bserved images are t suppsed t be perectly spatially aliged The techique prpsed i this paper is based the requecy dmai represetati the bservati mdel Oe the bee ts this apprach ccers the ability t wrk with large images ad with large supprts SFs The recursive prcedure cmpleted by the spatially-adaptive LA- ICI lters wrks as a spatially-adaptive regularizatr r the blur-peratr iversi Fr -blid image decvluti this spatially-adaptive LA-ICI iverse has bee preseted i [0] Simulati experimets shw the e ciecy the restrati algrithm which demstrates gd cvergece ad high quality image restrati The algrithm is quite rbust with respect t the supprt sizes used i the SF estimati 2 OBSERVATION MODEL Csider a 2D sigle-iput multiple-utput (SIMO liear spatially ivariat imagig system Such a system is apprpriate r the mdel multiple cameras, multiple cuses a sigle camera, r acquisiti images rm a sigle camera thrugh a chagig medium The iput t this system is a ukw image y(x x 2 where = x x 2 : x = 2 ::: x 2 = 2 ::: 2g the size 2 This image is distrted by ukw ite impulse respse uctis mdelled by the SFs v (x, = ::: L It is assumed that v (x are discrete spatially ivariat The discrete cvlutis the iput y(x ad the SFs v (x are degraded by the additive white Gaussia ise t prduce the bserved utput images: z (x = (y ~ v (x + (x = ::: L ( It is assumed that the ise i each chael is ucrrelated with the ise rm ther chaels ad (x have the Gaussia distributi N (0 The parameters are the stadard deviatis the ise i the chaels The prblem is t recstruct bth the image y ad the SFs v rm the bservatis z (x : x 2, = ::: Lg

2 3 GRADIENT-ROJECTION ALGORITHM Usig the arseval therem, x y2 (x = Y (2 =( 2, we itrduce the llwig basic quadratic criteri lss-ucti: where J = L 2 = L i= Z Y V Y 2 + (2 L d i Z iv Z V i V 2, d i = 2 i 2 V2 + 2 = Vi2 : (3 Here, Z ( Y ( ad V ( are the Furier trasrms (FTs the sigals z (x y(x, ad v (x respectively Fr the sake simplicity, we d t shw i the rmulas the requecy argumet The symbl Fg is used r the Furier trasrm The ecessary ucstraied miimum cditis r z ca be writte Y J = V J = 0 = ::: L r ay requecy Csiderig d i as a cstat parameter, we d ater elemetary maipulatis Y J V J = L 2 = (Z V Y V + 2Y, (4 (Z V Y Y + (5 2 L i= i6= L d i(z iv Z V izi + 3 V = where the star ( stays r the cmplex-cugate variable The estimates the sigal ad the SFs are slutis the llwig prblem: (^y ^v = arg mi J (6 y2q yv 2Q v where the admissible cvex sets Q y r y ad Q v r v are de ed as Q y = y : 0 y g Q v = v : xv(x = v(x 0v(x =0 i x> x2> : The sets Q v impse the psitivity ad rmalized mea value assumptis SFs v : The parameter > 0 de es the size the supprt v (x The recursive precti gradiet algrithm is used r sluti (6 Firstly, the values Y (k ad are calculated: Y (k = Y (k Y J(Y (k V (k (7 = V (k V J(Y (k V (k (8 where k = ::: k > 0 ad k > 0 are step-size parameters The crrespdig gradiet cmpets are give i (4-(5 Secdly, Y (k, are prected t the sets Q y Q v : Qy yg = max 0 mi( yg (9 Qv v g = v = v (x v 0, (0 x v (x = 0 i x > x 2 > The iitializati Y (0 V (0 is assumed i (7-(8 The rmalizati the SFs ca be de i the requecy dmai by replacig x v(k (x, where v (k = (x = F (0, as (0 = ( The precti Q y requires the iverse FT y (k (x = F Y (k (g with the precti y (k (x calculated accrdig t (9 The ill-cditiig the csidered iverse prblem meas that the criteri J has di eret scale behavir r di eret requecies I rder t eable stable iteratis r all requecies the step-sizes k ad k shuld be small ad, as result, the partial cvergece rates Y (k ad ca be very slw Fr the csidered requecy dmai calculatis, a behavir the algrithm the variables Y ad V is de ed maily by the secd rder derivative H Y Y Y J r Y ad the Hessia matrix H V T = V r V The cvergece rate the algrithm (7-(8 ca be essetially imprved usig the diagal terms the Hessia matrix H V V T ad HY Y as scalig actrs the step sizes: Y (k = Qy Y (k Y J(Y (k ( H Y Y ( (k = Qv V V H V J(Y (k (2 V where H Y Y = V ii6= dizi2 + 3 J i ad H V V = Y Substituti H Y Y ad H V V it (-(2 gives the llwig al rmulas r the iteratis: ( (k Y (k = Qy ( k Y (k ZV = 2 + k (3 (k V 2 = 2 +2 (k = Qv ( k V + (4 Z Y (k = 2 (k + Z ii6= d(k i V i Zi k Y (k 2 = 2 + ii6= d(k i Z i (k (k where d i are calculated i (3 r V = V Sme the restrictis de ig Q y ad Q v are t pricipal ad impsed ly i rder t imprve the cvergece ad the accuracy the algrithm I particular it ccers the requiremet 0 y 4 LA-ICI ADATIVE DENOISING The adaptive LA-ICI lterig algrithm is described i a umber publicatis [0, ] It rms a bak the directial liear lters with kerels g h btaied by LA A rtated directial symmetric kerel g h is used with the agle which de es the directiality the lter, ad h as a legth the kerel supprt (r a scale parameter the kerel i this directi The directiality the kerel is de ed by the -symmetric widw-ucti used i the LA The directial estimates are calculated usig the cvlutis by h (x = (g h ~ z (x r, i the requecy dmai, as the prducts the crrespdig FTs: G h ( Z ( where G h = Fg h g The -liearity the adaptive lterig is icrprated it the ICI rule This ICI is the algrithm r selecti the adaptive scale parameter h r every estimati pixel x The estimates by h (x are calculated r a grid

3 Figure : Observatis the isy blurred "Testpat" images Figure 2: Iitial guess, calculated as the mea the three bserved images, the estimate ad estimati errrs h 2 H = h h 2 ::: h Jg, where h < h 2 < ::: < h J The adaptive scale is de ed as the largest h + thse scales i H which estimate des t di er sigi catly rm the estimates crrespdig t the smaller widw sizes This cmm idea is implemeted as llws We csider a sequece c dece itervals D s = byhs (x by ^yhs h s(x + ^yhs s = :: J where > 0 is a parameter ad ^yhs is stadard deviati the estimate by hs cmputed as ^yhs =q x g2 h (x: s The ICI rule is stated as llws: csider the itersecti the c dece itervals I s = T s i= Di ad let s+ be the largest the idices s r which I s is -empty The the ptimal scale h + is de ed as h + = h s + ad, as result, the ptimal scale estimate is by h + (x The parameter is a key elemet the algrithm as it says whe a di erece betwee estimate deviatis is large r small T large value this parameter leads t sigal versmthig ad t small value leads t udersmthig I this paper we treat as a xed desig parameter Optimizati h r each the sectr estimates yields the adaptive scales h + ( r each directi The ui the supprts g h + ( is csidered as a apprximati the best lcal viciity x i which the estimati mdel ts the data The al estimate is calculated as a liear cmbiati the btaied adaptive directial estimates by h + (x : It is cveiet t treat this cmplex LA-ICI multidirectial algrithm as a adaptive lter with tw iputs z ad ad the e utput ^y The iput-utput equati ca be writte as ^y = LI z g by detig the calculatis imbedded i this algrithm as a LI peratr 5 BLIND DECONVOLUTION ALGORITHM Nw we are i a psiti t describe the develped blid multichael decvluti algrithm Iitializati: We use the Gaussia desity r v (0 ad the mea the bserved images y (0 = L = z(x=l as the iitial estimates 2 Image estimati: Calculate Y (k accrdig t (3 withut precti Figure 3: The estimates ad true SFs the three chael imagig system 3 Image lterig: Filter Y (k by the LA-ICI algrithm as llwig 3a Calculate the iverse FT y (k = F Y (k 3b Calculate the estimate the stadard deviati y (k the ise i y (k usig the media estimate the sigal s di ereces (eg [0, ] 3c Filter y (k accrdig t the algrithm: y (k, LI y (k y (k 4 Image precti: 4a rect y (k t the segmet [0 ], y (k, Qy y (k accrdig t (9 4b Calculate Y (k = F y (k 5 SF estimati: Calculate accrdig t (4 withut lterig ad precti Repeat these calculatis K it times This iteral iteratis imbedded i the mai recursive algrithm are used t accelerate the cvergece rate the algrithm 6 SF precti: 6a Calculate v (k = F 6b rect the SFs estimates v (k, Qv v (k accrdig t (0 7 SF lterig: Filter v (k by the LA-ICI algrithm: 7a Calculate the stadard deviati the ise i v (k similarly t (3c 7b Filter v (k accrdig t the algrithm: v (k, LI v (k (k v, = ::: L 8 Icrease k ad repeat steps (2 (8 K times Nte that the LA-ICI lterig-regularizati is embedded i the recursive algrithm itrduced rigially i the rm (3-(4 This LA-ICI lterig is prduced i the spatial dmai ad requires the backward ad rward FT the requecy dmai estimates, Y (k Operatis i the requecy dmai (3-(4 d t impse restrictis the supprt size SFs v : Hwever, the precti Qv v (k meas that the maximal size SFs v (k des t exceed a xed value, which is 55 i simulati results

4 Camerama Lea Testpat Text Bats BSNR SNR MAE SNR MAE SNR MAE SNR MAE SNR MAE Table : SNR ad MAE criteria values r the deblurred grayscale estimates give i the ext secti The reduced size SF reduces the amut ecessary cmputatis 6 SIMULATION EERIMENTS 6 Restrati grayscale images We csider three chael bservatis with the llwig di eret SFs: Bx-car 9 9 uirm Bx-car 7 7 uirm rtated by 45 0 "Iverse-quadratic" v (x x 2 = ( + x 2 + x 2 2, x x 2 = 7 : : : 7 (Fig3 The level ise i the bservatis z, = 2 3 is such that the blurred sigal-t-ise rati (BSNR BSNR= 0 lg 0 (y ~ v (x 2 ~x (y ~ v(x 2 is equal 2 2 t 20, 30, 40 r 50 db The arrw directial supprts the LA kerels g h are de ed by the tw-dimesial scale h = (h h 2 with h ad h 2 de ig the legth ad the width the kerels respectively Fr image lterig these supprts are lie-wise give by the set H = ( (2 (3 (5 (7 ( g All these kerels have the width h 2 = Fr the SF we use quadrat supprt kerels with equal legths ad widths H = ( (2 2 (3 3 (5 5 (7 7 ( g The zer rder LA with uirm widw uctis is used r g h : Thus, all the estimates are calculated as the sample mea bservatis icluded i the kerel supprts Fr the image the estimates ad the adaptive scales h + (x are calculated r eight directis (i = (i =4 i = ::: 8, with the parameter = 0:9 Fr the SFs the estimates ad the adaptive scales h + (x are calculated r ur directis (i = (i =2 i = ::: 4, with the parameter = :5 These ICI adaptive directial estimates are aggregated i the al e usig the weighted mea the directial estimates with the weights equal t the iverse variaces these estimates [0],[] Observatis the image Testpat image crrupted by a additive zer-mea Gaussia ise (BSNR= 40 db are shw i Fig The iitial guess as well as the estimate ad the estimati errrs are shw i Fig2 We may te that the recstructi is early perect, i particular i the di cult cetral part at the image The develped requecy dmai techique ca be used withut restrictis the size the supprts the SFs Hwever, eve quite apprximate irmati abut the maximal sizes the SFs imprves the cvergece rate as well as the quality image ad SFs restrati We assume that the supprts SFs v d t exceed size 55: This step is imprtat als rm the viewpit reducig cmputatial cst the iterative scheme The parameter balacig the delity ad the chael equalizati terms is a desig parameter the algrithm ad essetially a ects the accuracy It is xed t be equal t 2 i the scheme It llws rm the experimets that the i uece the regularizati parameters 2 ad 3 is isigi cat ad we x them t be equal t 0 7 It has bee ud r varius scearis that gd results are btaied r k = 0:6 ad k = 0:9 The ttal umber iteratis i 2 the algrithm K is xed t be 20 The umber iteral iteratis K it is xed t be 7 The true three SFs ad their estimates btaied r Testpat image are shw i Fig3 It is clearly see that they are well-restred despite sme mir arteacts The blurrig e ects give by the used SFs are sigi cat, as it ca be see i Fig Nevertheless, r the well-kw test images used i ur experimets the restrati is very gd The umerical evaluati ca be see i Table r a variety stadard test grayscale images ad ise settigs All the images the sizes except Bats, which is The rst clum the table shws BSNR values r bservatis z I this table llwig umerical errr values are preseted: peak sigal-t-ise rati (SNR i db, SNR= 20 lg 0 (max x y(x=rmse mea abslute errr (MAE, MAE= x y(x ^y(x =2: As it is see the quality restrati is gd ad prves that the prpsed techique is e ciet r isy data 62 Restrati clr images As a test image r blid decvluti clr images we used RGB Fruits image (Fig4c We assume that blurrig peratr v r a sigle bservati z = (R G B is the same r all clr R (red G (gree ad B (blue chaels The SFs v used are the same as r grayscale images experimets prvided i the previus secti The level ise is set t be 40 db r each chael The bservatis z btaied are illustrated i Fig4d- The atural apprach t recstructi is the use (3- (4 directly t these crrupted clr chaels separately The parameter r the ICI rule is xed t be 0:9 r all clr chaels: The restred clr image is shw (Fig4a The SNR ad MAE values are (34:7 33:3 32:5 ad (3:2 3:83 4:49 r R G ad B clr chaels, respectively Hwever, usually atural clr images are highly crrelated We use the ppet clr space trasrmati i rder t decrrelate these clr sigals [2]: " I # " =3 =3 =3 I 2 = =2 0 =2 I 3 =4 =2 =4 # " R G B where I is e achrmatic chael, ad I 2 I 3 are tw ppet clr chaels As a result clr space trasrmati, I has higher SNR which makes prblem decvluti easier, ad I 2 I 3 have lwer SNR but image details, like edges ad smth areas, are emphasized mre Therere, it is reasable t use lwer r I i rder t avid versmthig ad due t lwer level ise ( = 0:7 i ur experimets, ad higher r I 2 I 3 t suppress ise as much as pssible ( = 0:9 ad :0 The results image restrati are illustrated i Fig4b The SNR ad MAE values are equal t (35:5 34:9 33: ad (3:0 3:25 4:24, respectively r the chaels R, G ad B the image i the RGB clr space These SNR values are abut db higher the thse r straightrward restrati i RGB clr space It is wrth t stress that the di erece i visual quality evaluati is sigi cat als Cmparis Fig4a ad #

5 a b c d e Figure 4: Blid recstructi i RGB (a ad Oppet clr space (b F ruits image (c rm 3 blurred isy bservatis: d blurred with bxcar 9 9 SF e blurred with rtated by 45 0 bxcar 7 7 SF blurred with iverse-quadratic 7 7 SF Fig4b clearly shws that 4b is mre atural ad tiy details are preserved very well The MATLAB implemetati the develped algrithms is available at t acilitate reprducti results 7 CONCLUSIONS I this paper we prpse a iterative multichael blid decvluti algrithm isy images The icrprated deisig ad regularizati based the spatially adaptive LA-ICI techique Simulatis prduced r bth grayscale ad clr images shw high quality restrati i terms bective umerical criteria ad subective visual evaluati 8 ACKNOWLEDGMENTS This wrk was supprted by the Academy Filad, prect N (Fiish Cetre Excellece prgram ( I part, the wrk Dr Vladimir Katkvik is supprted by Visitig Fellw grat rm Nkia Fudati REFERENCES [] G Harikumar ad Y Bresler, erect blid restrati images blurred by multiple lters: Thery ad e - ciet algrithm, IEEE Tras Image rcessig, vl 8, pp , Feb 999 [2] G Harikumar ad Y Bresler, Exact image decvluti rm multiple FIR blurs, IEEE Tras Image rcessig, vl 8, 6, pp , 999 [3] G B Giaakis ad R W Heath Jr, Blid ideti- cati multichael FIR blurs ad perect image restrati, IEEE Tras Image rcessig, vl 9, pp , 2000 [4] G aci, Campisi, S Clese, ad G Scara, Multichael blid image decvluti usig the bussgag algrithm: Spatial ad multiresluti appraches, IEEE Tras Image rcess, vl 2,, pp , 2003 [5] F Srubek ad J Flusser, Multichael blid iterative image restrati, IEEE Tras Image rcess, vl 2, 9, pp , 2003 [6] F Srubek ad J Flusser, Multichael blid decvluti spatially misaliged images, IEEE Tras Image rcess, vl 4, 7, pp , 2005 [7] L Rudi, S Osher, ad E Fatemi, Nliear ttal variati based ise remval algrithms, hys D, vl 60, pp , 992 [8] D Mumrd ad J Shah, Optimal apprximati by piecewise smth uctis ad assciated variatial prblems, Cmmu ure ApplMath, vl 42, pp , 989 [9] T Cha, S Osher, J She, The digital TV lter ad liear deisig, IEEE Tras Image rcessig, vl 0, 0, pp 23-24, 200 [0] V Katkvik, K Egiazaria ad J Astla, A spatially adaptive parametric image deblurrig, IEEE Trasactis Image rcessig, vl 4, 0, pp , 2005 [] V Katkvik, K Egiazaria, ad J Astla, Adaptive widw size image de-isig based itersecti c dece itervals (ICI rule, Jural Mathematical Imagig ad Visi, vl 6, pp , 2002 [2] K N lataitis, A N Veetsapuls, Clr Image rcessig ad Applicatis, Spriger-Verlag Berli Heidelberg 2000