do: nfo:do/0.063/.322393
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 A COLOR FEATURES-BASED METHOD FOR OBJECT TRACKNG EMPLOYNG A PARTCLE FLTER ALGORTHM Bud Sugand, Hyoungseop Km, Joo Koo Tan, Sej shkawa Graduae School of Engneerng, Kyushu nsue of Technology bud@kmlab.cnl.kyuech.ac.jp Absrac We proposed a mehod for objec rackng employng a parcle fler based on color feaure mehod. A hsogram-based framework s used o descrbe he feaures. Hsograms are useful because hey have propery ha hey allow changes n he objec appearance whle he hsograms reman he same. Parcle flerng s used because s very robus for non-lnear and non-gaussan dynamc sae esmaon problems and performs well when cluer and occlusons are presen on he mage. Bhaacharyya dsance s used o wegh he samples n he parcle fler by comparng each sample's hsogram wh a specfed arge model and makes he measuremen machng and sample s wegh updang more reasonable. The mehod s capable o rack successfully he movng objec n dfferen oudoor envronmen wh and whou nal posons nformaon, and also, capable o rack he movng objec n he presence of occluson usng an appearance condon. n hs paper, we propose a color feauresbased mehod for objec rackng based on he parcle flers. The expermenal resuls and daa show he feasbly and he effecveness of our mehod. Keywords: objec rackng, parcle fler, color hsogram. nroducon Objec rackng s an mporan sep n many applcaons such as human-compuer neracon, medcal magng, vdeo compresson, vdeo survellance and gesure recognon. The objec rackng s a challengng problem due o he presence of nose, occluson, cluer and dynamc changes n he scene oher han he moon of objecs of neres. A varey of rackng algorhms has been proposed and mplemened o overcome hese dffcules. They can be roughly classfed no wo caegores: deermnsc mehods and sochasc mehods. Deermnsc mehods rack he objec by performng an erave search for a smlary beween he emplae and he curren mage. The robus smlary measures have been appled and he mean-shf algorhm or oher opmzaon echnques have been ulzed o fnd he opmal soluon [-2]. Model-based rackng algorhms ncorporae a pror nformaon abou he objecs o develop represenaons such as skn complexon, body blobs, knemacs skeleon, slhouees or layer nformaon [3-5]. Appearance-based approaches apply recognon algorhms o learn he objecs eher n some bass such as he egenspace formed from observaons or n kernel space [6]. On he oher hand, he sochasc mehods use he sae space o model he underlyng dynamcs of he rackng sysem. n a lnear-gaussan model wh lnear measuremen, here s always only one mode n he poseror probably densy funcon (PDF). The Kalman fler can be used o propagae and updae he mean and covarance of he dsrbuon of he model [7]. For nonlnear or non-gaussan problems, s mpossble o evaluae he dsrbuons analycally and many algorhms have been proposed o approxmae hem. The parcle fler, also known as sequenal Mone Carlo [7], s mos popular approach whch recursvely consrucs he poseror PDF of he sae space usng Mone Carlo negraon. has been developed n he compuer vson communy and appled o rackng problem and s also known as he Condensaon algorhm [8]. The parcle fler based rackng algorhms usually use conours, appearance models, or color feaures [8-0]. The conour-based mehods are nvaran agans he llumnaon varaon bu compuaonally expensve whch resrcs he number of samples (parcles) [8]. Unforunaely when he dmensonaly of he sae space ncreases, he number of samples requred for he samplng ncreases exponenally. On he oher hand, he color hsogram s robus agans nose, occluson, roaon and scale nvaran, and s also easy o mplemen [9-0]. n hs paper, we employ a parcle fler o rack a movng objec based on he color feaures. A arge model s racked usng he parcle fler by comparng s hsogram wh he hsogram of every sample usng he Bhaacharyya dsance whch makes he measuremen machng and wegh updang more reasonable. 2. Parcle fler The parcle fler s a Bayesan sequenal mporance samplng echnque, whch recursvely approxmaes he poseror dsrbuon usng a fne se of weghed samples (parcles). consss of essenally wo seps: predcon and updae. Gven all avalable observaons z : = {z,, z } up o me, he predcon sage uses he probablsc sysem ranson model p(x x ) o predc he poseror a me as, ( z : ) p( x x ) p( x z: ) p x = dx. ()
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 A me, he observaon z s avalable, he sae can be updaed usng Bayes rule, p ( ) ( z x ) p( x z: ) p x z: = (2) p( z z: ) where p(z x ) s descrbed by he observaon equaon. n he parcle fler, he poseror p(x z : ) s approxmaed by a fne se of N samples x ( =,..., ) wh he weghs w. The { } N canddae of samples ~ x are drawn from an mporance dsrbuon q(x x :,z : ) and he wegh of he samples are, ( ) p z ~ x p ~ x x w = w. (3) q ~ x x:, z: n our case, q(x x :, z : ) = p(x x ) and he weghs become he observaon lkelhood p(z x ). The samples are resampled o generae new unweghed parcles se accordng o her mporance weghs. 3. Observaon model The observaon model s used o measure he observaon lkelhood of he samples and s an mporan ssue for objec rackng. Many observaon models have been bul for parcle flerng rackng. One of hem s a conour based appearance emplae. The racker based on a conour emplae gves an accurae descrpon of he arges bu performs poorly n cluer and s generally me-consumng. The nalzaon of he sysem s relavely dffcul and edous. n conras, color-based rackers are faser and more robus, where he color hsogram s ypcally used o model he arges o comba he paral occluson, and non-rgdy. 3.. Color feaures Ths secon descrbes how he color feaures s modeled n a recangular regon R, where R can be a regon surroundng he objec o be racked or regon surroundng one of he hypohecal regons. A color hsogram s commonly used for objec rackng because hey are robus o paral occluson and are roaon and scale nvaran. They are also flexble n he ypes of objec ha hey can be used o rack, ncludng rgd and non-rgd objec. The color dsrbuon s expressed by an m-bns hsogram, whose componens are normalzed so ha s sum of all bns equals one. For a regon R n an mage, gven a se of n samples n R, denoed by X = {x, =, 2,, n} R, he m-bns color hsogram H(R) = {h j }, (j =, 2,...,m) can be obaned by assgnng each pxel x o a bn, by he followng equaon: h j = δ j[ b( x )]. (4) n ( ) X x Here b(x ) s he bn ndex where he color componen a x falls no, and δ s he Kronecker dela funcon. n our expermen, 8 8 8 bns hsogram s consruced for each regon R n RGB color space. 3.2. Weghed hsograms To ncrease he relably of he arge model, smaller wegh are assgned o he pxels ha are furher away from regon cener by employng a weghng funcon 2 r r < g( r) =. (5) 0 oherwse Here, r s he dsance from he cener of he regon. Usng hs wegh, he probably of he quanzed hsogram n he objec model a locaon y s gven by, j p f g [ h( j ) u] u y x j = a ( ) y = δ x. (6) Where s he number of pxels n he regon, x j s he poson of pxels n he regon, δ s he Kronecker dela funcon, a s he normalzaon facor, and f s he scalng facor defned as, f =. (7) y x = g a m ( u) o ensures ha = p = u y. 3.3. Dsance Measure A model hsogram s he weghed color hsogram of he objec o be racked. The model hsogram s consruced durng he nalzaon of he sysem. Fg.. shows an example of arge hsogram a me sep o. n subsequen frames, a every me, here are N parcles ha represen N hypohecal saes need o be evaluaed. The observaon lkelhood model s used o assgn a wegh assocaed o a specfc parcle (new observaon) dependng on how smlar he objec hsogram q and he hsogram p(x ) of he regon descrbed by he h parcle x are.
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 hsogram a x(0) 0.2 0. 0.08 0.06 0.04 0.02 0 0 20 30 40 50 bn number (a) hsogram of a arge a me 0 (b) arge objec a me 0 Fg.: Color hsogram of a arge model durng nalzaon To evaluae he smlary beween he model hsogram q and he parcle s hsogram ( p x ), where x s he h parcle a me, we employ he Bhaacharyya coeffcen ρ, m ρ [ p(x ),q] = pu ( x ) qu (8) u= where u s he hsogram bn ndex. The larger ρ s, he more smlar he wo dsrbuons are. For wo dencal normalzed hsograms we oban ρ =, ndcang a perfec mach. To quanfy he dsance beween wo dsrbuons, he dsance d s defned as, [ p q] d = ρ, (9) whch s known as he Bhaacharyya dsance. The observaon lkelhood funcon uses hs dsance o assgn a wegh o each parcle. A sample wh small Bhaacharyya dsance corresponds o a large wegh; smlarly, a sample wh large Bhaacharyya dsance corresponds o a small wegh. The wegh w () of h parcle of x s calculaed as, 2 ( ) d w = exp 2πσ 2σ 2 (0) ( [ ( ) ]) ρ p x, = exp q 2πσ 2σ 2 3.4. nalzaon The nalzaon sraegy s o pu he samples around he regon where he arge s lkely o appear. The rackng mode s begun when he samples sasfy some specal appearance condons. There are many praccal applcaons usng nalzaon sraegy. For nsance, can redscover a racked arge whch s occluded by oher objecs for a long perod. f he arge appears, he Bhaacharyya dsances of samples around he objec poson should be remarkable smaller han he average of sample se. Therefore, he mean value µ b and he sandard devaon σ b of he Bhaacharyya dsances of all nal samples are frsly calculaed as, µ b = ρ[ px, q] () = 2 2 σ b = ( ρ[ px, q] µ ) (2) = and hen an appearance condon s defned as, d = ρ[ px, q] > µ + 2σ (3) ndcang a 95[%] confdence ha a sample belongs o he objec. An appearng and dsappearng hreshold T s defned o ndcae he quany of he samples fulfllng he appearance condon durng nalzaon. More han T means he arge beng found and sarng rackng, conrary suaon means he racker wll ener he nalzaon mode. An example of nalzaon sraegy s shown n Fg. 2. The samples are nally placed a posons where he objec s mos lkely o appear, lke mage borders and edge of he occluded objec. 4. Parcle fler rackng n hs paper, he parcle fler racker consss of an nalzaon of he arge model and a parcle fler mplemenaon of a sochasc rackng sysem. n each eraon, he parcle fler rackng algorhm consss of wo seps: predcon and updae. The sae of he parcle fler s defned as x = ( x, y, x&, y&, hx, hy, h& x, h& y ) where x, y ndcae he locaon of he arge, x &, y& he moon, h x,h y he lengh of half axes, and h & x, h & y he scales n he x and y drecons, respecvely. n he predcon sage, he samples n he sae space are propagaed hrough a dynamc model. n hs paper, we use an auoregressve process model whch s descrbed by followng formulaon. x = Ax + v (4) Here A defnes he deermnsc componen of he model and v - s a mulvarae Gaussan random varable, respecvely.
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 # #5 Fg.2: An example of nalzaon sraegy n our applcaon we currenly use a frs order model for A descrbng a regon movng wh consan velocy x &, y& and scale change h & x, h & y. The updae sage apples he observaon models o esmae he observaon lkelhood for each sample. The algorhm of parcle fler rackng can be descrbed n followng 5 seps and furher programmng deals could refer o [7]:. nalzaon of he samples: Gven he color hsogram of he arge model from Eq.6: (u) q = q ( u =,...,m) { } Drawng he N parcles randomly. 2. Predcon sage : For each parcle do he followng: Propagae each sample accordng o sysem model of Eq. 4. Calculae he color hsogram p u x () from Eq. 6. 3. Updaed sage : Applyng observaon model o esmae he observaon lkelhood Calculae he Bhaacharya dsance from Eq.9. Calculae he wegh from Eq. 0. 4. Esmaed poson of x accordng o mean esmae : N ( ) ( ) E( x ) = = w x 5. Resamplng : Generae a new se of samples ( ) ( ) { } x, w ( =,..., N) Calculae he normalzed cumulave probably '( ) c k : ( ) (0) ( ) ( ) ( ) '( ) c c = 0, c = c + w, c = ( N ) c Generae unformly dsrbued number r [0,]. Fnd, by bnary search, he smalles j for whch '( c j ) ( ) ( j) r and se x = x #0 #20 Fg.3: Objec rackng wh unknown nal poson varance of samples 50 45 40 35 30 25 20 5 0 5 0 0 20 40 60 80 frame number Fg.4: Varance of samples poson 5. Expermens The algorhm has been mplemened and esed usng 2.54 [GHz] Penum V PC wh 52 [MB] memory. The expermen resuls show ha our algorhm works successfully. The sze of he npu frames s 320 240 pxels. n each expermen, we use 00 parcles. 5.. Objec rackng wh unknown nal poson n hs expermen, we rack he movng objec wh unknown nal poson and se o be unform dsrbuon x o U(,320) and y o U(,240). The expermenal resuls are shown n Fg.3. The whe dos represen he samples dsrbuon and he red do represens he mean sae of he samples poson. As presened n he fgure, nally, he samples are dsrbued unformly around he scene. The objec moves from lef o rgh and begn o appear n frame #5, however, he objec s sared o rack n frame #0. n hs expermen, he varance of he samples poson dsrbuon s ulzed o judge wheher he objec has been racked or no. As shown n Fg. 4, we consder he objec has been racked when he varance s below 0.
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 # #5 #30 #50 Fg.5: Objec rackng wh known nal poson 80 True rajecory 70 esmaon rajecory 60 50 objec (ree) as shown n frame #. The objec s sared o appear a frame #6, bu as he appearance condon s no fulflled, he objec s sll judged no o be racked. When he appearance condon s fulfll (frame #9), he objec s judge o be racked unl occluded by a ree. Fg.8 shows he successful objec rackng n he presence of occluson. As shown n ha fgure, he objec s racked unl dsappears behnd a ree (frame #45). The fnal mean sae esmae s calculaed, bu as he appearance condon s no fulflled, he objec s judged o be dsappeared (frame #50). Alhough he objec s dsappeared, he esmaon of he samples se sll connue and he dynamc model and he prevous sae propagaons evolve furher. The objec s racked agan when appears behnd he ree (frame #54), due o he appearance condon and connue o be racked unl he end of he frame. Number of samples ha fulfll he appearance condon s shown n Fg.9. As presened n ha fgure, he objec s deeced and sared o be racked when he hreshold T s abou 0. Tha means he objec s deeced and racked when he number of samples fulfllng he appearance condon s more han 0 samples. Y 40 30 20 0 00 0 50 00 50 200 250 300 X Fg.6: True vs. esmae poson # #6 5.2. Objec rackng wh known nal poson n hs expermen, he nal poson of he movng objec s known and se o x o N(22,3) and y o N(45,3). As presened n Fg.5, he objec s racked successfully. The red do represens he mean sae of he samples poson. A he begnnng, he samples are dsrbued around nal poson of he objec. The objec moves from lef o rgh and s racked from he begnnng of frame (frame #). Fg.6 shows he comparson beween rue and esmae poson of he arge objec. From ha fgure, we oban he RMSE of he esmaed poson s abou 2.06 for X poson and 2.8 for Y poson. #9 #20 Fg.7: Objec rackng wh appearance condon 5.3. Objec rackng wh appearance condon and occluson n hs expermen, we apply he nalzaon sraegy o deec and rack he objec when s occluded by anoher objec such as ree. The expermenal resuls are shown n Fg.7 and Fg.8. The red do represens he esmaed poson of he objec and he whe dos represen samples poson a each me. As presened n Fg.7, a he begnnng, he nal samples posons are placed a he poson where he objec s mos lkely o appear, such as mage borders and edge of he occluded #45 #50 #54 #60 Fg.8: Objec rackng wh occluson
nernaonal Conference on Power Conrol and Opmzaon, Bal, ndonesa, -3, June 2009 From ha fgure, we can undersand ha he objec s racked a frame #9 unl s occluded by ree a frame #50. A hs me, he number of samples a appearance condon s dropped o zero. A frame #54, he number of samples a appearance condon s 23 and objec s begun o be racked agan. 6. Conclusons n hs paper, we presened an effcen and robus color feaure-based mehod for objec rackng employng a parcle fler algorhm. The ulzaon of color feaurebased hsogram as arge model makes a parcle fler rackng approach more robus. The parcle fler manans mulple hypoheses abou he sae of he racked objecs by represenng he sae space by a se of weghed samples. n addon, an nalzaon sraegy s used as racked arge may dsappear and reappear. The expermenal resuls llusrae ha he mehod can effecvely rack he arge wh boh known and unknown nal posons, and also, when he objec s occluded by anoher objec usng he appearance condon. Furher mprovemens could be performed o speed up he compuaonal me and o oban he beer resul. n addon, performng he proposed mehod o realze mul-arge rackng s also neresng for fuure researches. 7. References [] S. Brchfeld, Ellpcal Head Trackng Usng nensy Gradens And Color Hsograms, n Proc. EEE Conf. Comp. Vson Paern Recognon, 998, pp 232 237. [2] D. Comancu, V. Ramesh and P. Meer, Kernel- Based Objec Trackng, EEE Trans. Paern Anal. Mach. nell., May 2003, Vol. 25(5), pp. 564 577. [3] G. Cheung, S. Baker and T. Kanade, Shape-From- Slhouee of Arculaed Objecs and s Use for Human Body Knemacs Esmaon and Moon Capure, n Proc. EEE Conf. Comp. Vson Paern Recognon, 2003, pp. 77 84. [4] C. R. Wren, A. Azarbayejan, T. Darrell and A. Penland, Pfnder: Real-Tme Trackng of he Human Body, EEE Trans. Paern Anal. Mach. nell., 997, Vol. 9(7), pp. :780 785. [5] H. Tao, H. S. Sawhney and R. Kumar, Objec Trackng wh Bayesan Esmaon of Dynamc Layer Represenaons, EEE Trans. Paern Anal. Mach. nell., Jan. 2002, Vol. 24(), pp. 75 89. [6] M. J. Black and A. D. Jepson, Egen Trackng: Robus Machng and Trackng of Arculaed Objecs Usng a Vew-Based Represenaon, n l Journal of Compuer Vson, 998, Vol. 26(), pp. 63 84. [7] S. Arulampalam, S. Maskell, N. Gordon and T. Clapp, A uoral on Parcle Flers for On-Lne Non-lnear/Non-Gaussan Bayesan Trackng, EEE Transacons of Sgnal Processng, 2002, pp. 74 88 [8] M. sard and A. Blake, Condensaon Condonal Densy Propagaon for Vsual Trackng, n l Journal of Compuer Vson, Aug. 998, Vol. 29(), pp. 5 28. [9] K. Nummaro, E. Koller-Meer and L. Van Gool, Objec Trackng wh an Adapve Color-Based Parcle Fler, Proceedngs of he Symposum for Paern Recognon of he DAGM, Sepember 2002, pp. 353-360. [0] P. Perez, C. Hue, J. Vermaak and M. Gangne, Color-based probablsc rackng, Proceedngs of he 7 h European Conference on Compuer Vson- Par, 2002, pp. 66 67. num ber of appearance 00 80 60 40 20 0 0 20 40 60 80 00 20 frame number Fg.9: Number of samples fulfllng he appearance condon