Krnl Intgral Imags: A Framwork for Fast Non-Uniform Filtring Mohamd Hussin Dpt. of Computr Scinc Univrsity of Maryland mhussin@cs.umd.du Fatih Porikli Mitsubishi Elctric Rsarch Labs Cambridg, MA 0239 fatih@mrl.com Larry Davis Dpt. of Computr Scinc Univrsity of Maryland lsd@cs.umd.du Abstract Intgral imags ar commonly usd in computr vision and computr graphics applications. Evaluation of box filtrs via intgral imags can b prformd in constant tim, rgardlss of th filtr siz. Although Hckbrt [6] xtndd th intgral imag approach for mor complx filtrs, its usag has bn vry limitd, in practic. In this papr, w prsnt an xtnsion to intgral imags that allows for application of a wid class of non-uniform filtrs. Our approach is suprior to Hckbrt s in trms of prcision rquirmnts and suitability for paralllization. W xplain th thortical basis of th approach and instantiat two concrt xampls: filtring with bilinar intrpolation, and filtring with approximatd Gaussian wighting. Our xprimnts show th significant spdups w achiv, and th highr accuracy of our approach compard to Hckbrt s.. Introduction Filtring is a fundamntal imag procssing opration. Th computational complxity of imag filtring dpnds on th complxity and siz of th filtr. For sparabl filtrs, for xampl, fficint computation is possibl by applying two conscutiv on-dimnsional filtrs instad of th original two-dimnsional filtr. Howvr, vn whn taking advantag of th filtr s sparability, th computational tim incrass with th filtr s siz, which is unfavorabl for larg filtrs. In som cass, w do not vn know th filtr siz in advanc,.g. whn th filtr siz is dtrmind dynamically basd on fatur valus. In such cass, th sparability of th filtr dos not hlp. For box filtrs, which ar usd to comput avrags and summations ovr rctangular imag rgions, thr is an lgant tchniqu that can ovrcom ths difficultis. Givn an intgral of imag faturs (Figur ), filtring with a box filtr at any point can b prformd in constant tim rgardlss of th filtr siz. Unfortunatly, using pr-computd intgrals is limitd, in practic, to box filtrs. In this papr, w prsnt a novl xtnsion that maks pr-computd intgrals usabl for mor complx filtrs. Th ida of using pr-computd intgrals was first introducd, with th nam summd-ara tabls, by Crow [3] to b usd for txtur mapping in computr graphics. Rcntly, it was popularizd in th fild of computr vision, with th nam intgral imags, by Viola and Johns [], who usd it for fast computation of Haar wavlt faturs. Latr on, intgral imags wr gnralizd by Porikli [9] to intgral histograms, which allow for fast construction of fatur histograms. Mor rcntly, intgral imags and intgral histograms wr usd to spd construction of Histograms of Orintd Gradint dscriptors by Zhu t al. [3], Rgion Covarianc dscriptors by Tuzl t al. [0], and th SURF dscriptors by Bay t al. []. To th bst of our knowldg, usag of intgral imags in computr vision applications has bn limitd to th spcial cas of box filtring although som of ths applications can prform bttr whn using non-uniform filtrs. For xampl, Dalal and Triggs [4] us bilinar intrpolation btwn nighboring clls and Gaussian wighting of pixls within a block of pixls in constructing thir histograms of orintd gradints faturs for human dtction. Thy show how ths wighting schms nhanc th dtctor s accuracy. To dvlop a fast vrsion of Dalal and Triggs dtctor, Zhu t al. [3] sacrific th bnfits of ths wighting schms to nabl usag of intgral imags. Anothr xampl is in th work of Bay t al. [], whr Gaussian drivativ filtrs ar approximatd by box filtrs so that intgral imags can b usd. Prhaps, a bttr approximation would b possibl if intgral imags wr abl to handl non-uniform wighting filtrs. A third xampl is in building apparanc modls for tracking, whr pixls closr to th cntr of th trackd rgion ar givn highr wights than pixls closr to th bordrs,.g. Elgammal t al. [5]. Considr a particl filtr trackr,.g. Zhou t al. [2], whr apparanc modls for hundrds of ovrlapping rgions nd to b constructd, possibly for many trackd targts, on vry fram. Applying non-uniform wighting of pixls in such a situation without th aid of a fast tchniqu similar to intgral imags can b impractical for ral-tim application.
Hckbrt [6] introducd th thortical foundation of th summd-ara tabls (intgral imags) tchniqu and xtndd th thory to allow for mor complx filtrs. Howvr, his xtnsion rquird a vry high prcssion numrical rprsntation vn for modrat imag sizs [7]. Similar to Hckbrt, w prsnt an approach to xtnd th intgral imags tchniqu to allow for non-uniform filtrs. Howvr, our approach has lowr prcision rquirmnt than Hckbrt s and is mor suitabl for paralll implmntation. W call our approach krnl intgral imags. A krnl intgral imag is a group of intgral imags such that a linar combination of box filtrs applid to thm is quivalnt to applying a mor complx filtr. W instantiat two xampls of applying our approach that ar rlvant to computr vision applications: fatur filtring with bilinar intrpolation, and approximation of filtring with Gaussian wighting. Our xprimntal analysis shows th significant spdups w achiv, and th supriority of our approach to Hckbrt s in trms of accuracy. Th rst of th papr is organizd as follows: sction 2 introducs notation and xplains intgral imags in an abstract form. Sction 3 mploys filtring with bilinar intrpolation as an xampl to introduc our xtnsion, which is aftrwards formalizd in sction 4. Thn, th xampl of filtring with approximat Gaussian wighting is dscribd in sction 5. In sction 6, w compar our approach to Hckbrt s. Empirical analysis of spdups and numrical rrors ar prsntd in sction 7, followd by conclusions in sction 8. For clarity of prsntation, w focus on on and two dimnsional signals. Th xtnsion to highr dimnsions is straight forward. 2. Fast Filtring via Intgral Imags 2.. Prliminaris Lt f : x R b a function that maps a point x = (x, x 2 ) to a ral valu, whr 0 x i N i, N i > 0, i =, 2. Thrfor, th domain of f, D f, is a rctangl boundd by th lins x i = 0 and x i = N i, i =, 2. A rctangular rgion (rfrrd to as a rgion from now on) R D f is dfind by a pair of points x b and x such that x b,x D f, and x b i < x i, i =, 2. Th two points x b and x rprsnt th two xtrm points of th rgion R. W rfr to th ordrd pair r = (x b,x ) as th rgion dfinition. Figur illustrats som of ths dfinitions. In practic, th function f rprsnts th raw intnsity valu or som othr fatur at ach point in an imag. Its domain, D f, is th st of all pixl coordinats in th imag. N N 2 is th imag siz. A filtring of th valus of f ovr a rgion R can b dfind as a function A f : R R that maps th rgion to a ral valu. Th form of th filtring function w considr Figur. An intgral of imag faturs. Th valu of th intgral at a point is th sum of th valus of imag faturs in th rctangular ara from th origin to th point. Th sum of fatur valus ovr any axis-alignd rctangular rgion (.g. th small whit rctangl) is dtrmind by th valu of th intgral at th four cornrs of th rgion. can b xprssd as A f (R) = x Ra r f(x), () whr th contribution function a r f (x) dfins th contribution of th point x to th filtring of th function f ovr th rgion R. In gnral, as th suprscript of a r f indicats, th contribution of a pointx dpnds not only on th point coordinats and th function f, but also on th dfinition of th rgion, i.. its two xtrm points. In this sction w first considr th simplr cas, whr th contribution of a point is indpndnt of th rgion s dfinition. W handl th gnral cas in sctions 3 and 4. Thus, for now, w dnot th contribution function by a f instad of a r f. Thrfor, th filtring function is rdfind as A f (R) = x Ra f (x). (2) W call such a filtring function and its associatd contribution function rgion-indpndnt functions. In its simplst form, th contribution function can b qual to th function f. That is a f (x) = f(x). (3) But, in fact, w can us any function that can b valuatd indpndntly from th filtring rgion s dfinition. For xampl, w can dfin th contribution function as a f (x) = x f 2 (x). (4) Thrfor, filtring with rgion-indpndnt contributions is much mor gnral than just summing fatur valus ovr a rctangular rgion.
2.2. Intgral Imags Whn filtring is computd ovr many rgions that ovrlap, using quation 2 is not fficint. This is bcaus th computations prformd in aras that ar shard among mor than on ovrlapping rgion will b rpatd for ach rgion. Luckily, th filtring quation has a sub structur that allows for a dynamic programming solution. This dynamic programming solution is what w rfr to as intgral imags. Dfin th intgral imag of a function f, I f, as a function with th sam domain and codomain as f, and of th form I f (x) = a f (y). (5) y D f,y i x i,i=,2 Th valu of th intgral imag of a function f at a point x is th sum of th contributions of all points in th rgion dfind by (o,x), whr o is th origin or th coordinat systm. Givn this formulation of intgral imags, it bcoms much simplr to valuat th filtring function ovr any rgion R. A filtring function can b writtn in trms of an intgral imag as A f (R) = I f (x, x 2 ) I f(x b, x 2 ) I f(x, xb 2 )+I f(x b, xb 2 ), (6) whr (x b,x ) dfins th filtring rgion R (Figur ). In gnral, having th intgral imag, filtring ovr a rgion R is rducd to O() computations compard to O((x xb ) (x 2 xb 2 )) computations using th original filtring function formulation, quation 2. Howvr, th cost of constructing th intgral imag itslf is O(N N 2 ). Thrfor, th utility of using intgral imags is ralizd only whn w filtr ovr many ovrlapping rgions. In th cas of xhaustivly filtring ovr th ntir domain of rgions, th spdups obtaind whn using intgral imags wr rportd in [9] to b svral ordrs of magnitud for a broad rang of paramtr choics. 3. Extnding Intgral Imags for Filtring with Rgion-Dpndnt Contributions Bfor discussing th formal tratmnt of th gnral cas, whr th contribution functions ar dpndnt on th filtring rgion s dfinition, w start with a concrt xampl. Considr filtring with bilinar intrpolation. A practical xampl is constructing th SIFT dscriptor [8], whr filtring is prformd ovr adjacnt rgions in a 4 4 grid of clls of pixls, such that ach pixl contributs to mor than on cll via bilinar intrpolation. W want to dfin th contribution function in this cas. A rgionris dfind byr = (x b,x ), whrx b = (x b, x b 2) and x = (x, x 2 ). Th cntr of th rgion is xc = (x c, xc 2 ) = (xb + x )/2, half th width of th rgion is hw = (x xb )/2, and half th hight of th rgion is hh = (x 2 x b 2)/2. Th contribution function at a point x = (x, x 2 ) R is dfind as a r f (x) = ( hw x x c hw ) ( hh x2 x c 2 ) f(x). hh (7) Apparntly, th contribution of a point is rgion-dpndnt. Hnc, th simpl intgral imag approach prsntd in sction 2 is not dirctly applicabl hr. For simplicity of prsntation, w considr only th cas whn x x c and x 2 x c 2. Th othr cass can b handld similarly. By manipulating quation 7, w obtain ( hw a r x + x c ) f(x) = hw ( hh x2 + x c ) 2 f(x) (8) hh ( ) ( ) x = x x 2 x 2 f(x) (9) hw hh ( x = x ) 2 f(x) hw hh ( ) x (x 2 f(x)) hw hh ( ) x 2 (x f(x)) + hw hh ( ) (x x 2 f(x)) (0) hw hh = g r h f(x) + g r 2 h 2f(x) + g r 3h 3f (x) + g r 4h 4f (x), () whr h f = f(x), h 2f = x 2 f(x), h 3f = x f(x), h 4f = x x 2 f(x), and g r through gr 4 ar th corrsponding cofficints from xprssion 0. Now, w hav xprssd th original contribution function as a linar combination of simplr functions, h f throughh 4f, with wighting cofficints g r through g4. r Th intrsting obsrvation hr is that all th h functions ar rgion-indpndnt, and non of th g cofficints dpnds on th point x or th function f, thy only dpnd on th rgion s dfinition. W call functions such as th g cofficints point-indpndnt. Substituting quation into th filtring function, quation, yilds A f (r) = g r h f (x) + g2 r h 2f (x) + x R g3 r x R x R h 3f (x) + g4 r h 4f (x). (2) x R Equation 2 xprsss th original filtring function as a linar combination of othr filtring functions. Morovr,
all of th componnt filtring functions in this linar combination ar rgion-indpndnt. In fact, th linar combination obtaind for th filtring function is xactly th sam as th linar combination for th contribution function itslf. Sinc ach of th componnt filtring functions in quation 2 is rgion-indpndnt, ach can b computd fficintly using an intgral imag for its own contribution function. Thn, by substituting th rsulting valus in quation 2, w obtain th dsird filtring. In summary, to us intgral imags in this xampl w xprss th dsird rgion-dpndnt contribution function as a linar combination of svral rgion-indpndnt contribution functions. Thn, th dsird rgion-dpndnt filtring is asily computd as a linar combination of th corrsponding rgion-indpndnt filtring functions, which can b fficintly computd via intgral imags. 4. Krnl Intgral Imags In this sction, w trat th cas of rgion-dpndnt filtring functions in a mor formal way. Rcall from th xampl of bilinar intrpolation that th mchanism usd to nabl usag of intgral imags is xprssing th filtring function as a linar combination of othr rgionindpndnt filtring functions. To undrstand why this works, w rwrit th final form of th contribution function, quation, in a mor compact form as a r f(x) = < g r,h f (x) >, (3) whr g r = and h f (x) = g r g r 2 g r 3 g r 4, (4) h f (x) h 2f (x) h 3f (x) h 4f (x), (5) In othr words, w can xprss th contribution function as a dot product of two vctor functions: on of thm is rgion-indpndnt and th othr is point-indpndnt. This is actually a ncssary and sufficint condition to xprss th filtring function as a linar combination of rgionindpndnt filtring functions. W outlin th proof of this fact rathr informally hr. Th sufficincy dirction is straight forward following th sam argumnt as in th bilinar intrpolation xampl. Basically, by distributing th summation of th filtring function ovr trms of th dot product, as w did to obtain quation 2, sufficincy immdiatly follows. Th ncssity dirction is drivd as follows. Starting from th linar combination of filtring functions, as in quation 2, w can xprss th linar combination as a dot product. Thn, by pulling th summation out, w obtain an xprssion of th contribution function that is a dot product of two parts, on of thm is rgionindpndnt, and th othr on is point-indpndnt. Th dot product immdiatly rminds us of th krnl trick that is frquntly usd in machin larning, whr fatur vctors ar implicitly transformd into a typically highr dimnsional spac by rplacing a dot product by a krnl function that is quivalnt to a dot product in th transformd spac [2]. Sinc applying any transformation to th vctorsg r and h f (x), in quation 3, will not chang thir rgion-indpndnc or point-indpndnc naturs, th condition w statd abov still holds on th transformd vctors. Thrfor, w can gnraliz th form of th contribution functions w considr to a r f (x) = H(gr,h f (x)), (6) whr H is a krnl function, i.. a function that computs a dot product btwn its two argumnts possibly aftr mapping thm to anothr dimnsional spac. W call this gnralization of intgral imags krnl intgral imags. In our cas, vn if th krnl prforms a dot product implicitly, to comput our filtring function w hav to prform it xplicitly. Somtims, th krnl computs th dot product in an infinit dimnsional spac. In ths cass, approximation of th dot product with a small numbr of trms may b sufficint for th application in hand. This point will b clarifid whn w us it in an xampl in sction 5. 5. Filtring with Gaussian Wighting In many applications of imag fatur filtring in computr vision, highr wights ar givn to pixls closr to th cntr of th filtring rgion and lowr wights to pixls closr to th bordrs of th filtring rgion. That is applid, for xampl, in objct tracking,.g. [5], whr highr wights ar givn to pixls that mor likly blong to th objct than th background. Th sam ida was shown to improv human dtction prformanc in [4]. In both cass, th wighting function usd is a Gaussian wighting function. To simplify th mathmatical tratmnt, w considr th on dimnsional cas. Considr a rgion R dfind by th two limiting points x b and x. Th cntr of R is dfind as x c = (x b + x )/2. Dnot th standard dviation of th Gaussian wighting function by σ r. Th contribution function in this cas can b dfind as a r x xc f (x) = ( σ r )2 f(x). (7) Clarly, th contribution function is rgion-dpndnt. Con-
0 2 3 4 5 0.95 0.9 0.85 0.8 ( a r f (x) = x x c) 2 σ r 0.95 0.9 0.85 0.8 a r f (x) = (x xc ) 2 anothr so that pixls closr to th cntr gt mor importanc, th diffrnc btwn th two functions in cas th slctd valu of σ r maks a diffrnc is not xpctd to b important. In gnral, whthr th approximation is accurat nough or not, and whthr it is worth using mor trms of th xpansion to achiv highr accuracy or not, dpnds on th valu of σ r and on th application itslf. 0.75 0.75 0 2 3 4 5 Figur 2. Comparison of th Gaussian wighting function and its approximation, quations 7 and 9, whn th filtring rgion is btwn 0 and 5 and σ r is 5. 6. Krnl Intgral Imags vs. Rpatd Intgration 0.9 0.8 0.7 0.6 0.5 0.4 0.3 ( a r f (x) = x x c) 2 σ r 0.9 0.8 0.7 0.6 0.5 0.4 0.3 a r f (x) = (x xc ) 2 Hckbrt [6] prsntd an lgant mthod, calld filtring by rpatd intgration, to xtnd usag of prcomputd intgrals to mor complx filtrs. For compltnss of prsntation, w brifly compar our mthod to his mthod. For dtails, plas rfr to [6]. 0.2 0. 0 0 2 3 4 5 0.2 0. 0 0 2 3 4 5 Figur 3. Comparison of th Gaussian wighting function and its approximation, quations 7 and 9, whn th filtring rgion is btwn 0 and 5 and σ r is 2.5. sidr th Eulr xpansion of quation 7 a r f(x) = ( ) i (x x c ) 2i f(x). (8) i i! i=0 Equation 8 can b viwd as a dot product in an infinit dimnsional spac btwn two vctor functions on of thm is rgion-indpndnt and th othr on is pointindpndnt. (To s this, considr xpanding th xprssion (x x c ) 2i in ach trm of th powr sris.) Hnc, th krnl intgral imag mthod applis. But, it rquirs computation of an infinit numbr of intgrals. Howvr, w can approximat th contribution function by taking a fw of th initial trms of th xpansion. For xampl, taking th first two trms only, w obtain th contribution function a r f (x) = ( (x xc ) 2 ) f(x). (9) This approximation is valid, i.. dos not giv ngativ wights, as long as σ r is slctd so that (x xc ) 2 σ. Figurs 2 and 3 show plots of th original Gaussian wighting r2 function, quation 7, and its approximation, quation 9, whn x b = 0, x = 5, and σ r = 5 and 2.5, rspctivly. In th cas of σ r = 5 plots ar vry similar. Howvr, for th cas of σ r = 2.5, th diffrnc is quit larg. For applications that nd wighting of pixls with rspct to on Hckbrt s approach is basd on th fact that mor complx filtrs can b constructd by convolving a box-filtr with itslf. For xampl, if w convolv a box filtr with itslf onc, w obtain a triangular filtr, which is vry similar to filtring with bilinar intrpolation in two dimnsions. If w convolv a box filtr with itslf twic, w obtain a quadratic filtr, which is similar to th approximation w us for Gaussian filtrs. In fact, convolution of a box filtr with itslf an infinit numbr of tims producs th Gaussian filtr. Suppos that w want to us a filtr that is gnratd by convolving a box filtr with itslf n tims. Hckbrt s approach is basd on th fact that convolution with such a filtr is quivalnt to intgrating th imag n tims and thn convolving th n th intgral with th n th drivativ of th filtr. Th n th drivativ of such a filtr turns out to b a simpl spars filtr, which is vry fficint to convolv with. Th main drawback of th rpatd intgration approach is intgrating th imag svral tims. Th rquird prcision to rprsnt th intgration valus grow linarly with th numbr of intgrations [7]. In our approach, w comput intgrals of svral functions. But, ach is intgratd only onc. For xampl, in approximating a Gaussian filtr by a quadratic filtr, th rpatd intgration mthod rquirs intgrating th imag thr tims conscutivly, whil krnl intgral imags rquirs computing nin indpndnt intgrals. Exprimntally, krnl intgral imags in this cas producs smallr numrical rrors using th standard doubl-prcision floating point numbr rprsntation, as w show in sction 7.3. Anothr advantag of our approach is that th intgrals computd ar indpndnt of on anothr. That allows for paralll computation of th intgrals.
7. Exprimntal Rsults 7.. Implmntation Dtails W valuatd our approach in trms of spdup by comparing to th convntional filtring approach (quation ). W implmntd filtring with bilinar intrpolation, and filtring with approximat Gaussian wighting. Both ar implmntd in two dimnsions. For bilinar intrpolation, quation in sction 3 considrs only th cas whr x x c and x 2 x c 2. If w considr th origin at th lowr lft cornr of th filtring domain, thn quation considrs only th cas of th top right quadrant of th filtring rgion. Figur 4 lists cofficints of diffrnt trms for th four quadrants. In ordr to prform fast filtring in this cas, w comput four diffrnt intgral imags, on for ach of th contribution functions. Th intgration itslf is conductd in four stps, sinc ach rgion s quadrant has a diffrnt cofficint for ach of th intgrals, as shown in figur 4. For th cas of approximating Gaussian wighting in two dimnsions, by xpanding quation 9 and xtnding th notation to two dimnsions, w obtain a r f (x) = [( xc 2 [( xc 2 2 ) + 2xc x x2 ) + 2xc 2 σ x r2 2 x2 2 ] ] f(x). (20) Hnc, to prform fast filtring, w comput nin intgral imags. Ths ar intgral imags for th contribution functions: f(x), x f(x), x 2 f(x), x x 2 f(x), x 2 f(x), x 2 2 f(x), x x 2 2 f(x), x2 x 2f(x), and x 2 x2 2f(x). Th cofficint of ach rgion-indpndnt filtring function can asily b obtaind from quation 20. Unlik th cas of bilinar intrpolation, thr is no nd to handl ach rgion quadrant sparatly sinc thy all hav th sam cofficints. 7.2. Running Tim Analysis In th two filtring xampls, th function filtrd on, f(x), is th intnsity at point x. Sinc intnsity valus do not affct th computation tim, w gnrat imags with a constant intnsity valu. Gnratd imags ar squars that diffr in th numbr of pixls, i.. ara. Gnratd imag aras rang from 0000 to 200000 pixls, with an incrmnt of 0000 pixls. Each imag is scannd with sampld rgion sizs and locations. Th minimum rgion sid lngth was st to 5 pixls, with sid lngth incrmnt of 5 pixls. Imags ar scannd with ach rgion siz in all possibl locations with incrmnts of 5 pixls in both dirctions. For ach rgion, th two filtring typs ar computd using intgral imags 60 55 50 45 40 35 30 25 20 5 Slow Downs in Stup Tim Filtring with Bilinar Intrp Filtring with Aprox Guass Wts 0 0 0.2 0.4 0.6 0.8.2.4.6.8 2 Imag Siz (pixls) x 0 5 Figur 5. Th slow down in stting up intgrals vs th naiv st up of convntional approachs. 250 200 50 00 50 Spdup in Filtring Tim Filtring with Bilinar Intrp Filtring with Aprox Guass Wts 0 0 0.2 0.4 0.6 0.8.2.4.6.8 2 Imag Siz (pixls) x 0 5 Figur 6. Spdups of using intgral imags compard to convntional mthod. Ths plots considr spdups in filtring tim only. 250 200 50 00 50 Ovrall Spdup in Stup Plus Filtring Tims Filtring with Bilinar Intrp Filtring with Aprox Guass Wts 0 0 0.2 0.4 0.6 0.8.2.4.6.8 2 Imag Siz (pixls) x 0 5 Figur 7. Spdups of using intgral imags compard to convntional mthod. Ths plots considr spdups whn adding construction tim to filtring tim. and using convntional filtring. For ach imag, two tim priods ar masurd: ) th tim to st up ncssary structurs, that is intgral imags or just typ convrsion whn
f(x) x 2 f(x) x f(x) x x 2 f(x) Top Right Quadrant x x 2 x x 2 Top Lft Quadrant x b x 2 x b x 2 Lowr Right Quadrant x xb 2 x x b 2 Lowr Lft Quadrant x b x b 2 x b x b 2 Figur 4. Cofficints of diffrnt contribution functions in th cas of bilinar intrpolation, quation 7, for th four rgion quadrants. All cofficints in th tabl hav to b normalizd by dividing by hw hh th convntional filtring is usd, 2) and th tim to scan th imag and comput filtring ovr all scannd rgions. Th plots in figur 5 show th slow-downs in th stup tim. In th cas of bilinar intrpolation, th slow down is around 20x, and in th cas of approximat Gaussian wighting, it is around 45x. On th othr hand, figur 6 shows th spdups obtaind whn considring only th tim to scan th imag and valuat th filtring function at all probd rgions. Th spdups ar monotonically incrasing with th imag siz. For an imag siz of 200000 pixls, w achiv a spdup of around 90x in th cas of bilinar intrpolation, and 220x in th cas of approximat Gaussian wighting. This shows th significant bnfit of using our approach, spcially in th cas of Gaussian wighting. Thrfor, dspit th complxity of computing mor intgral imags during stup, filtring with Gaussian wighting bnfits mor from using intgral imags. Finally, figur 7 shows spdups whn adding th stup and filtring tims togthr. Th curvs in this figur look vry similar to th curvs in figur 7, which considr spdups on filtring tim only. This shows that in th two wighting schms valuatd, th stup tim is almost ngligibl with rspct to th filtring tim. 7.3. Rlativ Error Analysis In this st of xprimnts, w valuat th two fast filtring mthods, krnl intgral imags and rpatd intgration, in trms of thir rlativ rror. Th rror w masur hr is th diffrnc btwn th valu computd by a fast filtring mthod and th valu computd by convntional filtring (quation ). Th rlativ rror is th ratio btwn this diffrnc and th valu computd by convntional filtring. W gnrat 0 random imags of siz 024 024. W valuat th filtring function on a rgion of siz 3 3 at all possibl locations in th imag. For ach location w comput th rlativ rror and plot rlativ rror valus against th distanc from th rgion s top-lft cornr to th imag s top-lft cornr. Th distanc masur w us is th ara of th rctangl boundd by ths two cornrs. This distanc masur is quivalnt to th numbr of fatur points that ar addd to produc th intgral valu(s) associatd with th rgion s top lft cornr. Th rror is xpctd to incras with this distanc masur. In th cas of bilinar intrpolation, rlativ rrors ar always zros, but not so for approximat Gaussian wighting. Th problm with th approximat Gaussian wighting is th intgration of highr ordr contribution functions, such as x 2 x2 2f(x). Ths contribution functions rquir highr prcision to rprsnt. Thir intgrals rquir vn highr prcision that is outsid th rang th doublprcision floating point rprsntation. Figur 8 shows a third-dgr polynomial fit of th rlativ rrors in th cas of approximat Gaussian wighting using krnl intgral imags. Th figur compars two mthods of computing intgrals in trms of th rror thy produc. Th on-pass mthod scans th imag onc and computs th valu of th intgral at a pixl as a function of its thr prcding pixls. Th two-pass approach scans th imag twic: onc intgrating horizontally and onc vrtically. Th rror gnrally incrass with th distanc from th origin. Th twopass mthod producs around an ordr of magnitud lowr rror than th on-pass mthod. That is xpctd sinc in th on-pass mthod, numbrs grow mor rapidly allowing for largr rrors whn adding two numbrs that diffr by many ordrs of magnitud. Figur 8 also shows th rlativ rrors, using two-pass intgration, of th rpatd intgration mthod whn usd to approximat Gaussian filtrs with a quadratic filtr. Th rror of our approach, vn whn using on-pass intgration, is lowr than th rror of th rpatd intgration mthod. Similar to our approach, th rpatd intgration mthod producs no rrors whn applid to bilinar intrpolation filtring. In ths xprimnts w us non-ngativ numbrs to rprsnt intnsity and pixl coordinat valus. Ths valus can b linarly mappd to allow for both ngativ and positiv numbrs. In this way, th ffctiv prcision usd can b incrasd by utilizing th sign bit in th binary rprsntation, and thrfor th accuracy can b nhancd, as shown in [7]. 8. Conclusion W prsntd an xtnsion to th intgral imag framwork that allows for fast filtring undr non-unform rgiondpndnt wighting of fatur valus. W rfr to th xtndd framwork as krnl intgral imags. To show th utility of th xtnsion, w providd two xampls of
0 4 0 5 0 6 0 7 0 8 0 9 0 0 0 On Pass KII Two Passs KII Two Passs RI Rlativ Error 0 2 0 2 3 4 5 6 7 8 9 0 Distanc (Ara) From Origin x 0 5 Figur 8. Rlativ rrors of computing Gaussian wightd filtring as a function of distanc (ara) to th origin. KII stands for Krnl Intgral Imags. RI stands for Rpatd Intgration. widly usd non-uniform filtring: on that can b implmntd xactly via our framwork, that is filtring with bilinar intrpolation, and on that can b approximatd, which is filtring with Gaussian wighting. Our xprimnts show that using our approach, significant spdups can b achivd. Th prsntd tchniqu provids a highr prcision and mor suitability for paralll implmntation than th rpatd intgration approach [6], which also xtndd th intgral imags framwork for complx filtrs. [7] J. Hnsly, T. Schurmann, G. Coomb, M. Singh, and A. Lastra. Fast summd-ara tabl gnration and its applications. EUROGRAPHICS, 24(3):547 555, 2005. [8] D. Low. Distinctiv imag faturs from scal-invariant kypoints. ntrnational Journal of Computr Vision, 60:9 0, 2004. [9] F. Porikli. Intgral histogram: A fast way to xtract histogram faturs. In IEEE Computr Socity Confrnc on Computr Vision and Pattrn Rcognition, 2005. [0] O. Tuzl, F. Porikli, and P. Mr. Rgion covarianc: A fast dscriptor for dtction and classification. In Europan Confrnc on Computr Vision (ECCV), 2006. [] P. Viola and M. Jons. Rapid objct dtction using a boostd cascad of simpl faturs. In IEEE Computr Socity Confrnc on Computr Vision and Pattrn Rcognition, 200. [2] S. K. Zhou, R. Chllappa, and B. Moghaddam. Visual tracking and rcognition using apparanc-adaptiv modl in particl filtrs. IEEE Transactions on Imag Procssing, 3():49 506, Novmbr 2004. [3] Q. Zhu, S. Avidan, M.-C. Yh, and K.-T. Chng. Fast human dtction using a cascad of histograms of orintd gradints. In IEEE Computr Socity Confrnc on Computr Vision and Pattrn Rcognition, Nw York, Jun 2006. Acknowldgmnt This work was fundd, in part, by Army Rsarch Laboratory s Robotics Collaborativ Tchnology Allianc program; contract numbr DAAD 9-02-002 ARL-CTA- DJH. Rfrncs [] H. Bay, T. Tuytlaars, and L. V. Gool. Surf: Spdd up robust faturs. In Europan Confrnc on Computr Vision (ECCV), 2006. [2] V. Chrkassky and F. Mulir. Larning from Data: Concpts, Thory, and Mthods. Wily, 998. [3] F. Crow. Summd-ara tabls for txtur mapping. Computr Graphics (ACM SIGGRAPH), 8(3):207 22, 984. [4] N. Dalal and B. Triggs. Histograms of orintd gradints for human dtction. In IEEE Computr Socity Confrnc on Computr Vision and Pattrn Rcognition, pags 886 893, 2005. [5] A. Elgammal, R. Duraiswami, and L. Davis. Efficint computation of krnl dnsity stimation using fast gauss transform with applications for sgmntation and tracking. In Scond Intrnational Workshop on Statistical and Computational Thoris of Vision, 200. [6] P. Hckbrt. Filtring by rpatd intgration. ACM SIG- GRAPH, 20(4):35 32, 986.