Ssors & Trasducrs 03 by IFSA http://www.ssorsportal.com A Novl Approach to Rcovrig Dpth from Dfocus H Zhipa Liu Zhzhog Wu Qiufg ad Fu Lifag Collg of Egirig Northast Agricultural Uivrsity 50030 Harbi Chia Collg of Scic Northast Agricultural Uivrsity 50030 Harbi Chia Tl.: 086+88455779 E-mail: hzhipa988003@63.com Rcivd: 8 Sptmbr 03 /Accptd: Novmbr 03 /Publishd: 30 Dcmbr 03 Abstract: This papr proposs a ovl approach to rcovrig dpth from dfocus which is a dtrmiistic approach i spatial domai. Two dfocusd gray imags from th sam sc ar obtaid by chagig two paramtrs (imag distac ad focal lgth of camra) othr tha oly paramtr (imag distac). Th ida of this approach is to covrt th gray imags ito th gradit imags by Cay oprator othr tha Sobl oprator th calculat th ratio of th ara of rgio with larg gradit valu to that of th whol imag rgio i ach block for ach dfocusd imag by momt-prsrvig mthod ad rcovr dpth from sc accordig to th ratio of th ratio of o gradit imag to that of th othr gradit imag. Th xprimtal rsults show that th proposd approach is mor accurat ad fficit tha th traditioal approach. Copyright 03 IFSA. Kywords: Dpth from dfocus Cay oprator Gradit imags Th ratio Momt-prsrvig mthod.. Itroductio Computr visio is to us computr to raliz popl s visual fuctio that popl ca fid 3D structur from D kow imags to idtify 3D world. I computr visio 3D rcostructio of sc is a importat cott whil th ky of 3D rcostructio is to calculat th distac btw ach poit i sc ad camra which is also calld dpth rcovry. Thr ar may approachs to stimatig dpth of th sc icludig Structur from Motio (SFM) [] Dpth from Stro (DFS) [] ad Dpth from Focus (DFF) [3] tc. But i DFS corrspodc problm d b solvd wll; i SFM ad DFF may imags ar dd what s mor rquirmt for masurmt accuracy is highr th masurmt procss is mor complx. Thrfor this papr focuss o dpth from dfocus (DFD) [4-6] which ca avoid corrspodc problm [7] high rquirmt ad complxity. DFD is to rcovr dpth of th sc from two dfocusd gray imags of th sam sc at a sigl viwpoit. I 987 DFD was firstly proposd by Ptlad [4]. Aftr that may approachs wr proposd to solv DFD. Nowadays ths approachs ar dividd ito two classs: statistical approachs [8-9] ad dtrmiistic approachs [6]. Dtrmiistic approachs ca b dividd ito frqucy domai approachs [0-] ad spatial domai approachs [6] agai. But statistical approachs hav high computatioal cost. Furthrmor dtrmiistic approachs i frqucy domai ca b iaccurat owig to widow ffct dg ad ois tc. By cotrast dtrmiistic approachs i spatial domai ar both simpl ad ral-tim. Thrfor this papr focuss o dtrmiistic approach i spatial domai. May dtrmiistic approachs i spatial domai ar to dduc dtrmiistic rlatioships btw dpth ad som paramtrs (.g. th sprad paramtr [3] dformatio paramtr [4] tc.). As log as ths paramtrs ar stimatd th dpth ca 36 Articl umbr P_574
b obtaid. This papr calculats th ratio of th ara of rgio with larg gradit valu to that of th whol imag rgio i ach block for ach dfocusd imag by momt-prsrvig mthod to rcovr dpth from sc [5]. I [5-6] covrtig th gray imags ito th gradit imags lads to wid dgs by Sobl oprator. Howvr this papr uss Cay oprator to avoid th abov shortcomigs. This ca obtai mor accurat gradit imags which ca improv th dpth rcovry ffctivly. Furthrmor i som paprs th rlatioships btw dpth ad th ratios ar obtaid by chagig oly paramtr (imag distac [5]); howvr this papr gralizs th abov formulas ad ot oly ca chag o paramtr but also ca chag mor paramtrs. This papr is discussd by chagig two paramtrs (imag distac ad focal lgth of camra). This papr proposs a ovl dtrmiistic approach to rcovrig dpth from dfocus accordig to th abov improvmts. I Sctio th dtrmiistic dpth from dfocus approach is proposd. Sctio. laborats th rlatioships btw dpth ad th ratio paramtr. Sctio. prsts Cay oprator ad momt-prsrvig mthod which ar usd to calculat th ratio paramtr. Th stps of dpth stimatio ar show i Sctio.3. Exprimtal rsults with sythtic dfocusd imags ar carrid out i Sctio 3.. Mthod.. Dtrmiistic Rlatioship Btw Dpth ad Ratio Paramtr β I Fig. accordig to th basic imagig pricipl wh th camra is focusd objct distac D focal lgth F l ad imag distac v thr ar followig gomtric rlatioships: Whil wh th camra is dfocusd th poit o th objct bcoms a fuzzy disk radius of th disk (dfocusd radius) is r b. Dfocusd radius r b radius of th ls r 0 ad actual imag distac v 0 hav th followig gomtric rlatioships: rb v00 r Fl D v0 So th formula of dpth from sc is: vrf D 0 0 l vr 0 0 Fr rf l 0 b l () (3) Two dfocusd gray imags from th sam sc ar obtaid by chagig two paramtrs (imag distac ad focal lgth of camra) ad imag distac ad objct distac of th first dfocusd imag rspctivly ar v 0 ad F l ad that of th scod o rspctivly ar v 0 ad F l. Accordig to (3) dpth is rprstd as th followig: v r F v r F (4) 0 0 l 0 0 l D v 0 r 0 F l r 0 rf l v 0 r 0 F l r 0 r F l whr r is th dfocusd radius of th first dfocusd imag r is that of th scod o. Accordig to (4) th xprssio of r 0 is: ( v0r v0r ) Fl Fl r0 (5) v v ( F F ) ( v v ) F F 0 0 l l 0 0 l l Plug th xprssio of r 0 i (4) this papr gts dtrmiistic rlatioship btw dpth ad : ( v0 v0 ) Fl Fl D ( v F ) F ( v F ) F 0 l l 0 l l (6) r whr. r.. Cay Oprator ad Momt- Prsrvig Mthod Fig.. Schmatic diagram of ls imagig () D v F l By quatio (6) D ca b kow aftr r calculatig ad but solvig r ad r r dirctly is vry difficult this papr uss momtprsrvig mthod to solv. This papr gts two gray imag f xy f xy with diffrt dfocusd radiuss by chagig v 0 ad F l. Covrtig th gray 37
imags ito th gradit imags lads to wid dgs by Sobl oprator. So this papr uss Cay oprator to obtai mor accurat gradit imags which ca improv th dpth rcovry ffctivly. Th gradit f xy f xy rspctivly ar: imags of imags whr g xy g g (7) x y g xy g g (8) x y g f ( xi y j) w ( i j) mx m x j i g f ( xi y j) w ( i j) y y j i m ;. Lvl Cay oprator (9) wx ( i j) i j (0) Vrtical Cay oprator wy ( i j) i j. () Th gradit imag is dividd ito svral blocks this papr assums that multipl pixls hav th sam dpth valus i th sam ara ad i ach block th bright ara rprsts th dgs th dark ara rprsts th o-dgs. Each block of th sc ara th gray lvl of th dgs of imag with largr gradit valus chags drastically th gray lvl of th o-dgs of imag with smallr gradit valus chags ot drastically. Assumig that th gray valus of bright ara ad dark ara i th gradit imag g( x y) rspctivly ar z z b; th ratio of th siz of bright ara (dark ara) to that of th whol imag ara is q ( q ). For focusd imag th valu b of q is smallr for dfocusd imag th valu of q is largr. As th imag from focusd to dfocusd th valu of q chags from small to big so w ca thik that th dfocusd radius of th imag is proportioal to q that is r k q () I () k is th costat. Dfi th first thr g xy ar: momts of th imag mj g x y j [ j ( )] 3 (3) ( xy ) N( xy ) whr N( x y ) as th imag ara ad as th pixl umbrs of imag rgio usig momt-prsrvig mthod it ca gt th followig four quatios: solutio for (4) whr q qb qz qz b b m (4) qz qz b b m 3 3 qz qz b b m3 z c c 4c z c c 4c z mb q z zb b b b b b m mm 3 c m m m3 mm cb m m (5) I imag g ( ) xy Th ratio of th siz of bright ara (dark ara) to that of th whol imag ara is w( w b). As a rsult r k q q (6) r k w w.3. Dpth Estimatio I this papr th stps of dpth from sc ca b simply summarizd as follows: Stp: Two dfocusd gray imags from th sam sc ar obtaid by chagig two paramtrs (imag distac ad focal lgth of camra). Shorthad for rspctivly fxy fxy. Stp: Accordig to (7) (8) (9) (0) () th gray imags ca b covrtd to gradit imags g x y g x y. shorthad for rspctivly Stp3: Th gradit imags g x y g x y ar dividd ito may blocks with sam siz amog thm vry block of g x y adg x y is o-too accordig to () (3) (4) (5) (6) it uss momt-prsrvig mthod to calculat th valus (... ) of vry block. Stp4: Plug th valus (... ) of vry block i (6) it ca calculat th dpth valus D ( D D... D ) of vry block of th sc. 38
3. Exprimtal Rsults 3.. Dpth rcovry Rsults Two imags with two stps ar adoptd i th xprimt th siz ar both 400 400 th distac from th frot of th sc to ls is 400 mm th distac from th back of th sc to ls is 000 mm. Wh th zoom ls focus to.3 m focal distac of camra is 34 mm ad th zoom ls focus to.6 m focal distac of camra is 35 mm rspctivly two origial dfocusd gray imags ar gott as show i Fig.. Th two gray imags ar covrtd to two gradit imags. Last th two gradit imags ar dividd ito svral sub-imags with th sam siz th siz of sub-imag is0 0. Accordig to o-to-o sub-imags of th two imags w ca calculat dpth from sc. Amog thm Fig. 3 ar th gradit imags of th two gray imags aftr usig Cay oprator ad Fig. 4 ar th rcovrd dpth map ad dpth surfac by th approach of this papr (chag th camra two paramtrs Cay oprator); Wh th zoom ls focus to.3 m ad.6 m rspctivly w ca gt th othr two origial dfocusd gray imags as show i Fig. 5. Fig. 6 ar th gradit imags of th two gray imags aftr usig Sobl oprator ad Fig. 7 ar th rcovrd dpth map ad dpth surfac by th traditioal approach (chag th camra oly o paramtr Sobl oprator). Fig. 3. Th two gradit imags: gradit imag of hadlig Fig. a by Cay oprator gradit imag of hadlig Fig. b by Cay oprator. Fig.. Two origial dfocusd gray imags: th origial dfocusd gray imag wh focal distac of camra is 34 mm ad camra focuss o.3 m th origial dfocusd gray imag wh focal distac of camra is 35 mm ad camra focuss o.6 m. Fig. 4. Th rcovrd dpth map ad dpth surfac by th approach of this papr (chag th camra two paramtrs Cay oprator): rcovrd dpth map by th approach of this papr rcovrd dpth surfac by th approach of this papr. 39
Fig. 5. Th othr two origial dfocusd gray imags: th origial dfocusd gray imag wh camra focuss o.3m th origial dfocusd gray imag wh camra focuss o.6 m. Fig. 6. Th othr two gradit imags: gradit imag of hadlig Fig. 5a by Sobl oprator gradit imag of hadlig Fig. 5b by Sobl oprator. Fig. 7. Th rcovrd dpth map ad dpth surfac by th traditioal approach (chag th camra oly o paramtr Sobl oprator): rcovrd dpth map by th traditioal approach rcovrd dpth surfac by th traditioal approach. 3.. Error Aalysis Error ca b masurd i thr idicators: th avrag (Mas) stadard dviatio (Std) ad ma squar rror (RMS). Thir formulas ar as follows: Mas Dˆ Std RMS k k ( Dk D) k ˆ ( Dk D) k ˆ (7) whr ˆ k D is th masurd valu D is th masurd avrag valu D is th ral valu is th pixl umbrs i a ara. Each rror valu of masurig dpth of usig th approach of this papr (chag th camra two paramtrs at th sam tim solv gradit imags by Cay oprator) ad usig th traditioal approach (chag th camra oly o paramtr solv gradit imags usig Sobl oprator) as show i Tabl ad Tabl. From Tabl ad Tabl it ca b s that th dpth masurmt rsults by th approach of this papr ar bttr tha th dpth masurmt rsults by th traditioal approach th avrag valu is clos to th tru valu ad its rrors ar smallr. 40
Tabl. Th rrors of th approach of this papr (chag th camra two paramtrs Cay oprator). Dpth Th valus of thr idicators (mm) Mas Std RMS 400 398.89 0.9.3 000 06.40 0.66 6.43 Tabl. Th rrors of th traditioal approach (chag th camra oly o paramtr Sobl oprator). Dpth (mm) Th valus of thr idicators Mas Std RMS 400 384. 0.64 5.79 000 74. 0.00 58.89 4. Coclusios This papr proposs a ovl approach to rcovrig dpth from dfocus th approach is spatial domai dtrmiistic approach two dfocusd gray imags from th sam sc ar obtaid by chagig two paramtrs (imag distac ad focal lgth of camra) ad covrts th gray imags ito th gradit imags by Cay oprator ad uss momt-prsrvig mthod to rcovr th dpth from th sc. Udr th prmis of guarat of accuracy it is simpl. Th xprimtal rsults show that th approach is accurat ad fficit ad it also ca b s that th approach is also fasibl for th complx sc. Th approach rcovrs th dpth from th sc basd o th valu q of gradit imag it is hardly affctd by illumiatio chag but it rqusts that targt sc must hav vidt clar dgs or txturs. Ackowldgmts This work was supportd i part by th Natur Scic Foudatio of Northast Agricultural Uivrsity udr cotract No.0RCA0 ad Scic ad Tchology Foudatio of Educatio Dpartmt i Hilogjiag provic udr th cotract No. 55037. Rfrcs []. J. Civra A. J. Daviso J. M. Martiz Motil Structur from Motio usig th Extdd Kalma Filtr Sprigr Tracts i Advacd Robotics Vol. 75 0. []. M. Blyr Ch. Rhma ad C. Rothr Extractig 3D Sc-Cosistt Objct Proposals ad Dpth from Stro Imags Lctur Nots i Computr Scic Vol. 7576 0 pp. 467-48. [3]. K. B. Gyug ad G. T. Tia A ovl dpth-from focus-basd masurmt systm for th rcostructio of surfac morphology with dpth discotiuity Itratioal Joural of Advacd Maufacturig Tchology Fbruary 009 Vol. 40 Issu - pp. 58-65. [4]. A. P. Ptlad A w ss for dpth of fild IEEE Tras o Pattr Aalysis ad Machi Itlligc 9 4 987 pp. 53-53. [5]. M. Subbarao Paralll dpth rcovry by chagig camra paramtrs i Procdigs of th IEEE Itratioal Cofrc o Computr Visio Florida 988 pp. 43-55. [6]. M. Suarao G. Surya Dpth from dfocus: A spatial domai approach Itratioal Joural of Computr Visio 3 3 994 pp. 7-94. [7]. Y. Y. Schchr N. Kiryati Dpth from dfocus vs. stro: how diffrt rally ar thy It. J. Comput. Vis Vol. 39 000 pp. 4-6. [8]. A. N. Rajagopala S. Chaudhuri A MRF modlbasd approach to simultaous rcovry of dpth ad rstoratio from dfocusd imags IEEE Tras. Pattr Aal. Mach. Itll. Vol. 999 pp. 577-589. [9]. A. N. Rajagopala S. Chaudhuri Optimal rcovry of dpth from dfocusd imags usig a MRF modl i Procdigs of IEEE Cofrc o Computr Visio Istitut of Elctrical ad Elctroics Egirs Nw York 998 pp. 047-05. [0]. M. Gokstorp Computig dpth from out-of-focus blur usig a local frqucy rprstatio i Procdigs of IEEE Cofrc o Pattr Rcogitio Istitut of Elctrical ad Elctroics Egirs Nw York 994 pp. 53-58. []. M. Wataab S. K. Nayar Ratioal filtrs for passiv dpth from dfocus It.J. Comput. Vis Vol.7 998 pp. 03-5. []. A. N. Josph Raj R. C. Stauto Ratioal filtr dsig for dpth from dfocus Pattr Rcogit. 45 0 pp. 98-07. [3]. G. Surya M. Subbarao Dpth from dfocus by chagig camra aprtur: a spatial domai approach i Procdigs of th IEEE Computr Socity Cofrc o Computr Visio ad Pattr Rcogitio (CVPR '93) 993 pp. 6-67. [4]. Ch. Hog Z. Quabig G. Yaya A Nw Dpth Rcovry Algorithm Basd o Dfocus Imag Computr Applicatios ad Softwar Vol. 7 No. Fb. 00 pp. 7-73. [5]. T. Tia J. Pa Dpth Estimatio from Dfocus Basd o Momt-Prsrvig Joural of Shaghai Jiaotog Uivrsity Vol. 34 No. 7 Jul. 000 pp. 97-90. [6]. T. Du-Mig L. Chi-Tu A momt-prsrvig approach for dpth from dfocus Pattr Rcogitio 3 5 998 pp. 55-560. 03 Copyright Itratioal Frqucy Ssor Associatio (IFSA). All rights rsrvd. (http://www.ssorsportal.com) 4