Object Tracking Based on Visual Attention Model and Particle Filter

Transcription

1 Inernaonal Journal of Informaon Technology Vol. No Objec Trackng Based on Vsual Aenon Model and Parcle Fler Long-Fe Zhang, Yuan-Da Cao 2, Mng-Je Zhang 3, Y-Zhuo Wang 4 School of Compuer Scence and Engneerng, Bejng Insue of Technology, Bejng, Chna. 8 {longfezhang, ydcao 2, frankwyz 4 }@b.edu.cn, zhangmngje@yeah.ne 3 Absrac An objec rackng mehod based on vsual aenon model and parcle fler s presened. An mproved vsual aenon model s employed o measure he smlary beween racked objecs and canddae objecs. Gaussan weghed color, nensy, orenaon and moon salency map are calculaed wh sraegy o compose he aenon value, whch can be used o measure he smlary of he objecs. Ths smlary measuremen s more accurae han ohers used n objec rackng algorhms. Expermenal resuls show ha boh sngle objec and mulple-objecs could be racked effcenly. Keyword: vsual aenon model, parcle fler, objec rackng. I. Inroducon In recen years, more and more researchers use parcle flerng o rack movng objecs n mage sequences [-5]. As even he objec moon model and he observaon model are nonlnear and he nose s non-gaussan, he objec can be racked well. Parcle flerng s a echnque for mplemenng a recursve Bayesan fler by sequenal Mone Carlo smulaons [6]. The key dea s o represen he requred poseror densy funcon by a se of random samples wh assocaed weghs and o compue esmaes based on hese samples and weghs. The weghs of parcles are relaed o he observaon of he objec n he curren frame. Almos all he feaures such as color[5] and conour[4] are exploed o presen he racked objecs. Bu low level feaures can no provde enough nformaon for objec rackng. In oher words, objec rackng s a knd of vsual rackng. Vsual rackng echnology offers an nmae and mmedae way of nerpreng users behavors o help a compuer search he movng objecs wh he same feaure. The gaze behavor of parcpans s compared wh daa obaned hrough a model of Vsual Aenon (VA) [7] o deec dfferences n behavor arsng from varyng vdeo conen. Ths paper explos vsual aenon model o measure he smlary among hese objecs, usng parcle flerng o rack he objec n mage sequence. Secon II presens vsual aenon model and he smlary measuremen of objecs. Secon III brefly nroduces he parcle flerng. Secon IV descrbes he objec rackng algorhm. Secon V presens he expermenal resuls n deal; The las secon s he concluson. 9

2 Longfe Zhang, Yuanda Cao, Mngje Zhang, Yzhuo Wang Objec Trackng Based on Vsual Aenon Model and Parcle Fler II. Vsual aenon model Aenon s a neurobologcal concepon. I could mply he concenraon of menal power upon an objec by observng or lsenng [9]. Compuaonal aenon makes us represen he mulmeda nformaon so close o he sense of human beng. I e.al [7,8] revewed he recen works on compuaonal models of vsual aenon, and presen an useful spaal salence based vsual aenon model. Sun[9] rases anoher objec based vsual aenon model o smulae he gaze of human eyes. Ma[] uses aenon model o summarze he vdeo clps. In hs paper, an mproved spaal salence based vsual aenon model s presened o esablsh he objec feaures for objec rackng. A. Vsual Aenon Model for Objec Defnon: The vsual aenon model for Objec s defned as a se of aenon objecs: AO { ROI,AV, MPS}, <<N () ROI recangle 2 B (2) + k < k< 2 N AV { AVC,AVI,AVO,AVM } (3) Where: ROI MPS AV B k AVC AVI AVO AVM Regon-Of-Ineres of AO Mnmal Percepble Sze of AO Aenon Value of AO Blocks by separae he AO wh he sze of MPS Aenon Value of AO caused by color feaure Aenon Value of AO caused by color nensy feaure Aenon Value of AO caused by orenaon feaure Aenon Value of AO caused by moon feaure We chose a pyramd srucure o represen he orgnal mage for a scalable compung. In he pyramd srucure, he orgnal mage s decomposed no ses of blocks. These blocks can be lowpass and bandpass componens va Gaussan and Laplacan pyramds[], respecvely. The Gaussan pyramd consss of lowpass flered (LPF) versons of he npu mage, wh each sage of he pyramd compued by lowpass flerng and subsamplng of he prevous sage mage. Bu n hs paper, we decompose he frame(mage) no N scale, 2 2(N +) blocks B k wh he sze of MPS. I s more accurae o f he aenon dsrbuon by s soropy characersc. B. Color and Inensy Salence HSV color space s more easly perceved by he human eye. Thus, we explo he mean normalzed HSV color o presen he prmary color feaure of he objec. We separae he frame F o blocks B k. We could calculae he color feaure of each blocks o form he pyramd mage, whch nroduced n secon A. So ha we could ge he color dsrbuon of he frame and could measure he prmary aenon of he frame. The mehods o exrac he color and nensy feaures s as follows: Sep. Calculae mean color of each blocks as :

3 Inernaonal Journal of Informaon Technology Vol. No ( H, S, V ) ( ( ) ( )) (4) C B mean h B, mean s B, mean v B k k k k Bk Bk Bk Sep 2. Compue he chess-board dsance beween B k and he ROI ( regarded as B j ) by (5): ( ) ( B, B ) B B MAX x B x B, y B y B (5) d k j k j k j k j where:(x(b),y(b)) s he coordnae of he cener of block B; Sep 3. Calculae color conras beween block B k and he ROI (regarded as B j) wh color dsance equaon(6): c( Bk, Bj) SBV cos cos sn sn 5 j B H j B S j BV k B H k B S k BV j B H j B S j BV k B H k B V k B V j B k + + (6) Sep 4. The color feaure aenon value AVC s: AVC k gauss gauss W ( ( ROI, B )) C B k d k k W ( ( ROI, B )) d k (7) Sep 5. Calculae he color varance beween blocks n prevous frame F - and hs frame F a he same locaon by equaon (6) and record as m( Bk) c( Bk, B k ) (8) AVM( B ) s calculae by (9): AVM k gauss gauss W ( ( ROI, B )) W B k d k k m W ( ( ROI, B )) d k m (9) Sep 6. Then, we ge he color dsrbuon and he color value of blocks n he objecs. The nensy of aenon value of he objec can be calculaed by (): AVI k W ( ( ROI, B )) ( B, B ) gauss d k c k j k W ( ( ROI, B )) gauss d k () Where W m s he wegh of moon, W gauss (.) s he wegh of s poson defned wh normalzed Gaussan conversaon kernel cenered n he objec. The wegh selecon s shown n fgure.

4 Longfe Zhang, Yuanda Cao, Mngje Zhang, Yzhuo Wang Objec Trackng Based on Vsual Aenon Model and Parcle Fler Fg.. Aenon nensy dsrbuon. The convoluon emplae valued by normalzed Gaussan (W gauss (.)) fs N scales pyramd mages (k5,2 2(N+) 64*64 blocks). C. Orenaon Salence The objec s local orenaon nformaon s obaned from nensy usng orened Gabor pyramds OP( ζ,θ),where ζ represens he scale and θ s he orenaon. (Gabor flers, whch are he produc of a cosne grang and a 2D Gaussan envelope, approxmae he recepve feld sensvy profle (mpulse response) of orenaon-selecve neurons n prmary vsual corex [7,9].) We defne θ P, P as he orenaon dfference beween pxels p 2 and p 2. Le he orenaon vecors of p and p 2 n he curren orenaon pyramd respecvely. V p θ and V p 2 φ be Noe ha V,θ, and φ hemselves all conss of mulple componens. For example, f we have four preferred orenaons, hen: V ( θ ) [ V (), V ( π 2), V ( π), V (3π 4)] () p p p p p We defne he orenaon salence ( p, p ) as: o 2 o 2 gauss d 2 (, 2) p, p W p, p sn θ (2) p p Snusod funcon s a nonlnear and monooncally ncreasng funcon from o over he range [,π/2] and symmerc n [,π], hus we choose snusod as he orenaon salence facor. Then θ p, p2 can be gven by equaon (3): θ p, p2 π π φ Vp ( θ) V (mod ) p θ φ π dθ dφ + 2 V ( θ ) V (( θ + φ)mod π) dθdφ π p p2 (3) The dscree form s: θ p, p2 ζ ζ jϕ V ( ϕ) V (( ϕ + jϕ)mod π) j ζ ζ j p p V ( ϕ ) V (( ϕ + jϕ)mod π) p p2 (4) 2

5 Inernaonal Journal of Informaon Technology Vol. No Where and j s he pyramd level, ζ s defned as he number of orenaon pyramds or preferred orenaons. φ s jϕ and θ s ϕ, where ϕ πζ ; Thus, we could ge he orenaon aenon AVO : AVO n( ζ ) ζ k o ( d ( cener( ROI), pk) ) ( cener( ROI), pk) Wgauss d ( cener( ROI), pk) n( ζ ) ζ k where n s he neghborhood pxel of a pxel, and n s 8; 8( ζ ) pxels of objec. D. Vsual Aenon Smlary W gauss By he equaon (3), we could ge he AV of AO : (5) s he number of neghborhood AV WC AVC + WI AVI + Wm AVM + WO AVO (6) where W c,w,w m and W o are he weghng coeffcens of color aenon, nensy aenon, moon aenon and orenaon aenon, respecvely. In hs paper, we se hem o /4. Because he racked objecs wll no change much, such as ROI and color, we could use aenon model o measure he smlary beween he arge and canddae objecs. We use ROI - of he arge AO n F - (he frame n -) nsead of ROI n (7) and (9~) o calculae he AVC j and AVM j of canddae objecs AO j, and we could ge AV j by (6). Fnally, we can calculae he smlary beween he wo objecs by (7): Sm( AO, AO ) AV AV (7) j j III. Parcle Flerng Parcle flerng s a echnque for mplemenng a recursve Bayesan fler by sequenal Mone Carlo smulaons [6]. The key dea s o represen he requred poseror densy funcon by a se of random samples wh assocaed weghs and o compue esmaes based on hese samples and weghs. In recen years, parcle flerng has been used o rack objecs n a cluer, n whch he poseror densy p(x /Z ) and he observaon densy p(z /X ) are ofen non-gaussan, and he sysem model s nonlnear. Where X denoes he sae vecor of he racked objec a me, and Z denoes all he observaon vecor of he racked objec {z,..., z } up o me. Parcle flerng uses a weghed sample (parcle) se S o approxmae he probably dsrbuon of he objec sae p(x /Z ), where N s he number of parcles n he parcle se and s he observaon me. Each sample consss of an elemen s whch represens he hypohecal sae of he objec and a correspondng dscree samplng probably w. where {(, n ),, N S s w n K N }, and w. n 3

6 Longfe Zhang, Yuanda Cao, Mngje Zhang, Yzhuo Wang Objec Trackng Based on Vsual Aenon Model and Parcle Fler Every parcle n he sample se evolves accordng o a sysem model, producng he sample se S+ ( s, ),, + w + n N w { } K (8) K ( + + ) + + n n ( +, + ) p z s p s s q s s z (9) N Where: K s he normalzaon facor o ensure: w n +, and q ( ) s he mporance densy operaor. Parcle flerng models uncerany, and consders he mulple sae hypoheses smulaneously, so provdes a robus rackng framework. IV. Objec rackng mehod Suppose he regon (we use O sands for he regon ) conanng he racked objec n mage sequence s known. The Objec rackng mehod based on parcle flerng s: Inpu: he racked objec O; Oupu:he sequence of he esmaed sae of O;, Inalzaon: Generae he nal parcle se: {(, ),, } S s w n K N (2) s s he sample drawn from Gaussan dsrbuon n n where w / N ; g x ; X, Σ, whose mean value s X, s he nal poson of he racked objec, s he covarance marx. for,2,,f (where F s he frame number) Ge he curren sae of he racked objec X, usng he algorhm (Vsual aenon model). Calculae he velocy ( v ) of he racked objec: v X X (2) Updae parcles n S and s wegh Where ω s he process nose. w s s + v + ω (22) exp j, j ( X s ) j 2 N X 2σ (23) N X s dmenson number of he sae vecor. 4

7 Inernaonal Journal of Informaon Technology Vol. No Resamplng: (a) calculae he normalzed cumulave probables c c c + w, 2,...,N. (b) generae a unformly dsrbued random number u U[,/ N] se. (c) For j,...,n u u + ( j / j ) N; whle u j >c + ; end whle; ( j) s+ s% + ; end for; Calculae he sum of weghs: N Sum w n. (24) Normalze he weghs: ( n w w ) / Sum (25) Esmae he sae of he objec: N s X w s n end for Usng he objec rackng algorhm lsed above, he seleced objec n he mage sequence can be effcenly racked. V. Expermen The mage sequences used n our expermens are capured by a PC camera a 5 frames/sec wh a resoluon of pxels. They conss of 68 frames of a able enns ball movng n a ypcal laboraory envronmen. Every parcle n he parcle se s a 4D vecor, whch represens he regon whch conans he canddae objec. The number of parcles s 2. (26) (a)frame #73 (b)frame #74 (c)frame #26 Fg. 2. Sngle objec rackng 5

8 Longfe Zhang, Yuanda Cao, Mngje Zhang, Yzhuo Wang Objec Trackng Based on Vsual Aenon Model and Parcle Fler Fgure 2 (a) shows he nal poson of he racked objec, he yellow recangle s he regon whch conans he seleced objec, he yellow pon s he cener of he recangle. Fgure 3 shows he spaal salency map of vsual aenon of frame #73, whch s showed n fgure 2(a). Fgure 2 (b) and (c) show he rajecory of he racked objec. The yellow recangle s he observed poson of he regon whch conans he racked objec, he pons s he cener of he recangle. The yellow pons are he observed cener of he recangles, and he red pons are he esmaed cener of he recangles. Fg. 3. Salency map of Spaal vsual aenon n he frame #73. The whe par on he mddle of he lef s he arge objec. The color bar scales he vsual aenon value. (a) (b) (c) Fg. 4. The esmaed and observed rackng curve on (x-),(y-),and (x-y) axs Fgure.4 shows he esmaon curve and he observaon curve of he objec from frame 73 o 26. (a) and (b) shows he error curve of he esmaed able enns ceners n blue, along wh he rue (observed) able enns ceners shown as lnes wh dos n red. (c) shows he esmaed and observed poson of objec cener s poson. The average error of esmaed poson on he x axs s.235 and on he y axs s.37. The resuls showed a good performance for objec rackng. (a)frame #299 (b) frame #35 (c)frame #37 Fg. 5. Mul-objec rackng 6

9 Inernaonal Journal of Informaon Technology Vol. No Fgure.5 shows he second expermen: able enns balls rolled from one sde of he desk o he oher. (a) shows he seleced objecs o be racked n dfferen recangle. The pons n dfferen color n (b) and (c) shows he rajecory of dfferen objecs. The wo expermen show ha he proposed objec rackng mehod n hs paper can rack he seleced objecs effcenly. VI. Concluson An objec rackng mehod based on a compuaonal vsual aenon model and parcle fler s presened. An mproved vsual aenon model s proposed o presen he racked objec sensvely and measure he smlary of objecs accuraely. Insead of RGB color space, HSV, whch s more concden o human s percevng, s used n vsual aenon model. Gaussan weghed color space salence map wh color nensy and orenaon are used o esmae he parameers of parcle fler more accuraely han color hsogram whch s wdely used n objec rackng. We employed a parcle fler mehod for usng he vsual aenon-based objec rackng. Boh sngle and mulple objecs can be racked well. Expermenal resuls show ha he proposed mehod can yeld a good resul. References [] Zaver, M.A.; Desa, U.B.; Merchan, S.N.; Auomaed model selecon based rackng of mulple arges usng parcle flerng. TENCON 23. Conference on Convergen Technologes for Asa-Pacfc Regon, Vol.2 (23) [2] Boers, Y.; Dressen, J.N. Mularge parcle fler rack before deec applcaon[c] Radar, Sonar and Navgaon, IEE Proceedngs, Vol.5, Iss.6 (24) [3] T.Hong; S.K.Wang; Z.Q.Wang; Fronal moon rackng based on mage feaures analyss and parcle fler.[c] Machne Learnng and Cybernecs, 24. Proceedngs of 24 Inernaonal Conference on, Vol.7 (24) [4] S.h.k.Zhou, Chellappa, R.; Moghaddam, B Vsual rackng and recognon usng appearance-adapve models n parcle flers. IEEE Transacons on Image Processng, Vol.3, Iss. (24 ) [5] Yoshnor Saoh, Takayuk Okaan and Kochro Deguch. A Color-based Trackng by Kalman Parcle Fler. Proceedngs of he 7h Inernaonal Conference on Paern Recognon (ICPR 4) /4 (24) [6] A. Douce, N. D. Freas, and N. Gordon, Sequenal Mone Carlo Mehods n Pracce. New York: Sprnger-Verlag (2). [7] I L, C. Koch, and E.Nebur. A Model of Salency-Based Vsual Aenon for Rapd Scene Analyss. IEEE Transacons on Paern Analyss and Machne Ingellgence, Vol.2, No. (998) [8] I L, Koch C. Compuaonal Modelng of Vsual Aenon. Naure Revews Neuroscence 2(3).(2) [9] Y.R. Sun, Rober Fsher. Objec-based vsual aenon for compuer vson. Arfcal Inellgence Vol.46, Iss. (23) [] Y.F. Ma, X.S. Hua, L. Lu, H.J. Zhang. User Aenon Model based Vdeo Summarzaon. IEEE Transacons on Mulmeda (24) [] Greenspan, H.; Belonge, S.; Goodman, R. ec. Overcomplee seerable pyramd flers and roaon nvarance. Proceedngs CVPR '94.(994)

10 Longfe Zhang, Yuanda Cao, Mngje Zhang, Yzhuo Wang Objec Trackng Based on Vsual Aenon Model and Parcle Fler Longfe Zhang receved he BS. degree from he deparmen of Compuer Scence and Technology, Henan Unversy, Chna, n 2. He receved he MS. and Ph.D. degrees from he Deparmen of Compuer Scence and Engneerng a he Bejng Insue of Technology(BIT) n 25. He has been wh he School of Compuer Scence and Engneerng, BIT. Hs research neress nclude Compuer Vson, Machne Learnng, and Paern Recognon. Yuanda Cao receved he Dploma n elecrcal engneerng form he Bejng Insue of Technology n 969. He has been wh school of Compuer Scence and Engneerng, Bejng Insue of Technology. He s a fellow of Chna Compuer Federaon, drecor of nellgen nformaon nework specal commsson, Chnese Assocaon for Arfcal Inellgence. He s professor and Ph.D. advsor. Hs major research neress nclude Arcle Inellgen, Paern recognon and Nework Secury. Mngje Zhang receved he BS. degree from he deparmen of Compuer Scence and Technology, Zhengzhou Unversy, Chna, n 2. He receved he MS. and Ph.D. degrees from he Deparmen of Compuer Scence and Engneerng a he Bejng Insue of Technology n 25. Hs research neress nclude Compuer Vson, Image Processng, and Paern Recognon. Yzhuo Wang receved he BS. degree from he deparmen of Compuer Scence and Technology, Henan Unversy, Chna, n 2. He receved he MS. and Ph.D. degrees from he Deparmen of Compuer Scence and Engneerng a he Bejng Insue of Technology n 25. He has been wh he School of Compuer Scence and Engneerng, Bejng Insue of Technology. Hs research neress nclude Image Processng, Daa Compressng. 8