Diffrnt Focus Points Imags Fusion Basd on Strabl Filtrs Lin zhng School of Elctronics & information nginring Xi an Jiaotong Univrsity Xi an,shaanxi,china LiinZhng@sina.com Chongzhao an School of Elctronics & information nginring Xi an Jiaotong Univrsity Xi an,shaanxi,china czhan@xtu.du.cn Dongguang Zuo School of Elctronics & information nginring Xi an Jiaotong Univrsity Xi an,shaanxi,china zdg_ll_zly@6.nt Abstract-An imag fusion algorithm basd on strabl filtrs is prsntd in this papr. Two spatially rgistrd imags with diffrnt focus points ar fusd by mans of dciding clar obcts. Firstl two-lvl Laplacian pyramid dcomposition is takn on th original imags. Thn transform with quadratur pair of strabl filtrs is applid on high-pass lvl of ach pyramid to comput local orintd nrgy. Finall clar rgions in ach original imag ar dcidd by comparing th local orintd nrgy at vry corrsponding spatial point of th two imags. Th xprimntal rsult shows that this fusion mthodology prforms wll in prsrving dg information. Kywords: Imag fusion, Strabl filtrs, quadratur pair, local orintd nrgy. Introduction Imag fusion combins diffrnt aspcts of information from th sam imaging modality or from two distinct imag modalitis []. Fusd imag provids for robust oprational prformanc, i.. incrasd confidnc, rducd ambiguit improvd rliability or a dpr insight about th natur of obsrvd data. As a powrful tool, imag fusion is widly usd in th digital imag procssing and undrstanding, such as imag nhancing, automatic obct dtction, D rconstruction, and th imag sharpning, which will b discussd in this papr Rcnt yars, a numbr of imag fusion mthods hav bn proposd. Th simplst on is accomplishd by avraging th two original imags. Faturs from ach original imag ar prsntd in a fusd imag, but significantly rducd. or Sophisticatd tchniqus includ Principl componnt analysis (PCA) mthod [], Laplacian pyramid mthod [], Wavlt transform mthod [], tc. PCA mthod is basd on a dimnsionality rducing procss in which principl componnt of an imag is obtaind. Laplacian pyramid mthod and Wavlt transform mthod ar basd on multirsolution dcomposition and rconstruction, in which transform cofficints ar fusd rathr than spatial imag pixls. In this papr, a nw tchniqu basd on strabl filtrs is prsntd, and thn usd to mrg two rgistrd imags with diffrnt focus points. Sinc th two imags hav diffrnt focus points, w can s that obcts in-focus in th first imag must b out-focus in th scond imag, and vic vrsa. So th two original imags ar complmntary ach othr. If th clar in-focus obcts ar found out in ach original imag with som algorithms, w can mrg thm into rsult imag that hav thoroughly good quality. Sinc a clar obct consists of distinct faturs, w can find it out by dciding if th corrsponding faturs ar distinct. Thr ar many mthods to dtct faturs,.g. gradint oprator, LO oprator, Canny oprator, tc. In this papr, w apply local nrgy modl basd on strabl filtrs to dg dtction. orron postulats that faturs ar prcivd at points of maximum phas congruncy in an imag [5]. Vnkatsh and Own showd that phas congruncy is proportional to local nrg dfind as sum of th squard rsponss of a quadratur pair of symmtric and anti-symmtric filtrs. Thrfor maximum in phas congruncy at faturs corrspond to th maximum of local nrgy at faturs [6]. Th local nrgy mthod usd for fatur dtction has th advantag that it can accuratly locat composit dgs which ar mor common in ral world imags than idal dg, whras normal gradint basd mthods xhibit localization rrors [7]. In ordr to xtract th local nrg many kinds of quadratur pair hav bn proposd [8]. On simpl and suitabl quadratur pair usd in this papr is composd of a strabl filtr and it ilbrt transform, which not only dtct th local nrg calld local orintd nrgy hr, but also th local dominant dirction along th fatur dirction [9]. If two-lvl Laplacian pyramid dcomposition is mad on ach original imag, it is obvious that th high-pass lvl contains mor prominnt information on faturs than low-pass lvl of th pyramid or th original imag. So by using th strabl filtrs, th local orintd nrgy is obtaind on th high-pass lvl rathr than original imag. And thn w can dcid clar obcts in vry original
imag by comparing th local orintd nrgy at vry corrsponding spatial point of th two imags. This papr is organizd as follows. In sction, transform with strabl filtrs is prsntd. W will dsign a quadratur pair of strabl filtrs in this sction, computation of local orintd nrgy is thn prsntd and w apply it in dciding clar imag. Th nw fusion mthod of diffrnt focus points imags is introducd in sction. In sction, an xprimnt and th rsult ar shown. And th sction 5 is dvotd for a brif summary. Transform with strabl filtrs Strabl filtrs ar filtrs whos arbitrary orintation can b synthsizd as a linar combination of a st of basis filtrs [9]. Thy ar usd in many vision and imag procssing tasks, such as imag nhancmnt, imag data comprssion and motion analysis. Whn thy ar usd as quaduatur pairs, thy hav a good prformanc in dg dtction.. Dsign of quadratur pair of strabl filtr Thr ar many typs of basis filtrs for a strabl filtr. Th usful on is a st of rotation vrsions about strabl filtr. Thn, if a filtr f can str, its arbitrary orintation f is xprssd as whr f f = k ( ) f = is arbitrary rotation angl from x-axis., =,, L th rotation vrsion along angl k ( ), =,, L form th basis filtrs, which ar, =,, L ar intrpolation functions., and It has bn shown that not all filtrs could b usd for strabl filtrs in th form of quation, but thos satisfy th following two constraints [9]: Constraint : In polar coordinats, f could b xpandd in a Fourir sris, which has finit trms: whr N inφ f ( r, φ ) = a ( r) r + n n= N = x y and = arg φ. Constraint : intrpolation functions k ( ),,L, = ar th solution of following quation: i in i = in It has bn provd that i in L k i L k in L k, n =,,L ( ) ( ) ( ) nth drivativ of a aussian in th x dirction is strabl [][]. In this papr, th scond drivativ of th aussian function,, is slctd as a strabl filtr, which has th following form: x = = (x ) As th numbr of basis filtrs changs, th form of basis filtrs is also chang. According to [9], if th numbr of nonzro cofficints a n (r) is T in xprssion, thn th minimum numbr of basis filtrs isn t lss than T. Th mor basis filtrs ar usd, th mor stabl strabl filtr is. But for simplifying, th last numbr of basis filtrs ar usd. Writ in th form of xprssion r i i r ( r, φ ) = r ( + ) + (r ), which has thr nonzro cofficints. So w slct a basis with thr filtrs. In practic, for rasons of symmtry and robustnss against nois, w choos th thr basis filtrs spacd qually in angl btwn and : ; = ; = = Thn, with constraint, Thr intrpolation functions could b calculatd: k( ) = k( ) = k( ) = ( + cos ) [ + cos( ( )] [ + cos( ( )] And arbitrary orintation of is dscribd as: = k k ( ) ( ) whr thr basis filtrs ar + + k =.9( x =.9( y =.8xy, which hav bn normalizd. ( ) ) ) For fatur dtction, quadratur pair of filtrs is ndd. Th quadratur filtr of is th ilbrt transform of it, which is unabl to str. So w us a third -ordr polynomial tims aussian to approximat it
= (.7867x +.x ) Using abov mthod, th corrsponding normalizd basis filtrs of ) ar y = (.5x +.978x = (.75 y +.978x ) = (.75x +.978xy = (.5 y +.978 y ) And th intrpolation functions ar obtaind: l ( ) = cos ( ) ; l ( ) = cos ( )sin( ) l ( ) = cos( )sin ( ) ; l ( ) = sin ( ) ) So, arbitrary orintation of ) is als o obtaind by = l ( ) + l ( ) y + l + l ( ) ( ). Local orintd nrgy and its application In sction, w hav discussd th local orintd nrgy modl usd to dtct imag faturs. A fatur is composd of points whr th local orintd nrg obtaind along local dominant orintation, is maximum. Local orintd nrgy is calculatd by string th quadratur pair of strabl filtrs. r, firstly w giv th orintd nrgy proposd in [9] E ) = I * + I * whr, I is a tst imag, is th string angl. As strabl filtrs turn around, orintd nrgy also changs. Whn th dirction of strabl filtrs fits th local dominant orintation, orintd nrgy also arrivs to its maximum, ust th local orintd nrgy. If a clar imag I is blurrd, th blurrry imag can b xprss as th original imag convoluting to a gaussian function with varianc σ [], I = I( x, * σ Sinc σ is a low-pass filtr along vry spatial dirction, som high frquncy information of original imag ar lost, includ thos on faturs. Using th quadratur par of strabl filtrs on I and I rspctivl th corrspondnc on I I y along dominant orintation. must b lss than on ) According to th dfinition of local orintd nrg w can dduc that th local orintd nrgy is lss on I. So local orintd nrgy is a good masur usd to dcid blurry imag, and thn th clar imag. Imag fusion In ordr to fus two original imags, i =, with diffrnt focus points, local orintd nrgy on ach imag is calculatd by using quandratur pair of strabl filtrs. As mntiond abov, maximum local orintd nrgy is corrsponding to th fatur point. And if a clar obct in imag is blurrd, its fatur points will hav dcrasd maximum local orintd nrgy. So, if w compar local orintd nrgy at th corrsponding fatur points on two rgistrd original imags, w can dcid th blurrd out-focus obcts and th clar in-focus obcts. In this papr, local orintd nrgy is not calculatd dirctly on original imags, but on th high-passd vrsion of it. Two-lvl Laplacian pyramid dcomposition is carrid out for ach imag rspctivly []. It is obvious that faturs ar mor significant in th high-pass lvl L i, i, than in th low-pass lvl L i =, i =, or original imag. So w dtct th faturs on high-pass imags instad of on original imag for highr dcision accuracy. As a rsult, th procdur of our algorithm is, first quadratur pair of strabl filtrs is constructd with quations,. Thn original imags ar dcomposd into two-lvl Laplacian pyramids, and w calculat th local orintd nrgy on high-pass lvl of ach Laplacian pyramid using quation i d E ) = L * + i whr, d I i i d L *, i =, is th rotation angl of dominant orintation at d pixl (, x. By Comparing E ) and d E ), th obcts in original imags, which hav d gratr local orintd nrg ar dcidd as clar obcts and slctd to th fusd imag. Exprimnt and Rsult W applid th abov mthodology to mrg two original imags, showd in figur (a), (b). In figur (a), front clock is blurr and back clock is clar. In figur (b), front clock is clar, and back on is blurry. Th prfct fusion imag showd in figur (c) is obtaind by manually cut and past. Figur (d), (), (f), (g) ar th fusd imags by using PCA mthod, laplacian pyramid mthod, wavlt mthod and our algorithm. And figur (h), (i), (), (k) ar th diffrnt imags of abov four mthods from prfct fusion imag.
(a) Clock imag () Fusd imag obtaind by Laplacian pyramid mthod (b) Clock imag (f) Fusion imag obtaind by wavlt mthod (c) Prfct fusion imag obtaind by manually cut and past (g) Fusd imag obtaind by our mthod (d) Fusd imag obtaind by PCA mthod (h) Diffrnt from th prfct imag (PCA mthod)
comput th diffrnc, w us th man-squar rror SE = N N i= = ( g ( i, ) g( i, )) N is th siz of imag. ( i, ) ( i, whr g is th fusion imag, and g ) is th prfct fusion imag. Th bttr th bhavior of th fusion imag is, th small SE is. S tabl. (i) Diffrnt from th prfct imag (Laplacian pyramid mthod) thod Tabl : Th SE of four mthods PCA Wavlt thod thod Laplacian Pyramid thod Our thod SE 55.8.59 6.9 9.7 It sms that SE of PCA mthod is th largst. And SE of Laplacian pyramid mthod is small. Although th rsult imag is vry clar, th rror with our mthod is somwhat larg. Th main rason for this mayb is, in som smooth rgions, whr mainly contain low-frquncy information, our mthod maks som mistaks for lacking nough high-frquncy information to analysis. () Diffrnt from th prfct imag (Wavlt mthod) If w add nois to original imags, our fusion algorithm can dtct th in-focus obcts corrctly. But th nois couldn t b rasd in rsult imag, bcaus th mthod dtcts th nois dots as fatur points. So how to distinguish nois dots is a work for futur. 5 Conclusion Strabl filtrs dsignd in quadratur pair ar usd to combin two diffrnt focus points imags in this papr. Local maximum orintd nrgy computd by strabl filtrs is a dscription to faturs, and applid to dcid clar obcts, which will b mrgd into rsult imag. (k) Diffrnt from th prfct imag (Our mthod) Figur : Fusion rsults by diffrnt fusion mthodology It sms that th fusion imags obtaind with DWT mthod, laplacian mthod and our mthod prsrv dg information wll and ar clar thoroughly. But th fusd imag using PCA mthod is somwhat blurr which couldn t prsrv clar dg information. As in [], W can quantify th bhavior of PCA mthod, laplacian pyramid mthod, Wavlt transform mthod and our mthod in comparison with th prfct fusion imag. To Th transform with strabl filtrs is shift-invariant, and th local orintd nrgy can dtct faturs robustly without introducing location rror []. Both abov proprtis ar highly dsirabl for imag fusion. Th xprinc rsult shows that our mthod prsrv th dg information to high dgr. Thr ar also many works to do in th futur. On is how to distinguish nois dots in original imag. And anothr work is to improv this algorithm, which will not only has a good ability to prsrv faturs, but also arriv at a lowr SE. Rfrncs [] J. K. Aggarwal, ultisnsor Fusion for Computr
Vision, Brlin idlbrg: Springr-Vrlag, 99. [] Chn, C.., Xiaohui Zhang, On th rols of PCA and ICA in data fusion. oscinc and Rmot Snsing Symposiun,. Procdings. IARSS. IEEE Intrnational, Volum: 6,. Pag(s): 6-6 vol.6 [] A.Akrman III, Pyramidal Tchniqus for ultisnsor Fusion. SPIE Vol.88 Snsor Fusion V(99), pp.-. [] Xuan Yang, Wanhai Yang, Jihong Pi, Diffrnt Focus Point Imag Fusion Basd On Wavlt Dcomposition. RD Intrnational Confrnc On Information Fusion. [5]. C. orron and D. C. Burr, Fatur dtction in human vision: a phas-dpndnt nrgy modl, Proc. Royal Soc. London, vol. 5, pp.9-5, 988. [6] S.Vnkatsh and R. Owns. On th classification of imag faturs. Pattrn Rcognition Lttrs. :9-9, 99. [7] P. Prona and J. alik, Dtcting and localizing dgs composd of stps, paks and roofs. In Proc. rd Int. Conf. Comput Vision, Osaka, Japan, Dc. 99, pp.5-57. [8]ichal John Robin. Local nrgy fatur tracing in digital imags and volums. Ph.D thsis. pp7-8. 999 [9] W. T. Frman and E.. Adlson, Th dsign and us of strabl filtrs, IEEE Trans, Pattrn Anal.achin Intll., vol.pai-, pp. 89-96,99. [] P. Danilsson and O. sgr, Rotation invarianc in gradint and highr ordr drivativ dtctors. Comp. Vision raphics Imag Procssing, vol. 9, pp.98-, 99. [] W.T. Frman and E.. Adlson, Strabl filtrs, in Topical tg. Imag Undrstanding achin Vision. Opt. Soc. Amr., Tch. Digst Sris, vol., Jun 989. [] Ptr J. Burt, Edward. Adlson. Th Laplacian Pyramid as a Compact Imag Cod. IEEE Transaction on Communications, VOL. CO -, NO., APRIL, 98 [].Li, B.S. anunath and Sanit K. itra. ulti-snsor imag fusion using th wavlt transform. IEEE Int. Conf. Imag Procssing, Austin, TX, Nov. 99, vol, pp5-55 [] Iztak Korn, Andrw Lain, and Frd Taylor. Imag fusion using strabl dyadic wavlt transform. Imag Procssing, 995, Procdings., Intrnational Confrnc on, Volum, 995 pp-5, vol.