Visual tracking via saliency weighted sparse coding appearance model

Transcription

1 nd Internatonal Conference on Pattern Recognton Vual trackng va alency weghted pare codng appearance model Wany L, Peng Wang Reearch Center of Precon Senng and Control Inttute of Automaton, Chnee Academy of Scence Beng, Chna {Wany.l, AbtractSpare codng ha been ued for target appearance modelng and appled uccefully n vual trackng. However, noe may be nevtably ntroduced nto e repreentaton due to background clutter. To cope w problem, we propoe a alency weghted pare codng appearance model for vual trackng. Frtly, a pectral flterng baed vual attenton computatonal model, whch combne bo bottom-up and topdown vual attenton, propoed to calculate alency map. Secondly, poolng operaton n pare codng weghted by calculated alency map to help target repreentaton focu on dtnctve feature and uppre background clutter. Extenve experment on a recently propoed trackng benchmark demontrate at e propoed algorm outperform tate-ofe-art meod n trackng obect under background clutter. Keywordvual trackng; alency; vual attenton; pare codng. I. ITRODUCTIO Vual trackng an mportant computer von tak and wdely appled to robotc, urvellance, and human computer nteracton, etc. Depte extenve reearch on topc [1-3], robut trackng tll reman a huge challenge due to dfferent factor, uch a occluon, llumnaton change and background clutter. Recently, appearance modelng baed on pare codng (AMSC) [4-8] ha been uccefully appled n vual trackng and appealng expermental reult are reported. The framework of AMSC contan ree man layer [9]: mage layer, codng layer and poolng layer. A et of local mage patche are ampled n mage layer. Each patch parely coded by a learned dctonary n codng layer. To decrbe e appearance of e nput mage, produced code n codng layer are furer pooled n e poolng layer and form e fnal feature vector. Real-world vdeo frame almot alway contan background clutter. A poolng operaton elect feature wout noton of foreground, feature extracted from background may have tronger repone to e learned dctonary. A a reult, e poolng operaton alone may nevtably ntroduce noe nto e repreentaton. Thu correpondng trackng algorm may fal when background clutter occur. Human vual ytem (HVS) ha robut vual trackng Hong Qao State Key Laboratory of Management and Control for Complex Sytem Inttute of Automaton, Chnee Academy of Scence Beng, Chna capablty. In trackng proce of HVS, vual attenton play a crtcal role, whch drect proceng reource to potentally mot relevant vual data uch a foreground regon, epecally drect our gaze rapdly toward obect of nteret. A a reult, human can ealy acheve robut trackng. In e computer von communty, many computatonal model have been propoed to mulate [10]. The output of ee vual attenton computatonal model alency map of whch alency a good ndcator of foreground. A a reult, f we ue vual alency to gude poolng operaton n pare codng, better pare repreentaton focung on foreground regon can be obtaned. Motvated by above-mentoned dcuon, paper propoe a alency weghted pare codng appearance model for trackng obect under background clutter. The contrbuton of work are ummarzed a follow. Frt, a pectral flterng baed vual attenton computatonal model, whch combne bo top-down and bottom-up vual attenton, propoed to calculate alency map. Second, we propoe a alency weghted poolng functon and lead to a alency weghted pare codng appearance model for vual trackng. Extenve experment on challengng equence demontrate e effectvene of e propoed meod. II. PROPOSED METHOD A. Spectral Flterng baed Vual Attenton Computatonal Model (a). Spectral Flterng The convoluton of e mage ampltude pectrum w a low-pa Gauan kernel of an approprate cale equvalent to an mage alency detector [11]. The key dea nclude two part: 1) Spke n e ampltude pectrum correpond to repeated pattern. 2) Repeated pattern can be uppreed by pectral flterng, u e alency pop out from e ret of e mage. The alency map can be obtaned by recontructng e 2-D gnal ung e orgnal phae and e fltered ampltude pectrum, hown a Eq. (1)-(2). A(,) uv Ff(, xy) g, (1) S S (2) 1 F A (, ) P( u, v ) S u v e, Th work wa upported by e SF (atonal atural Scence Foundaton) of Chna under e grant , , and /14 $ IEEE DOI /ICPR

2 1 where Ff and F f ndcate e Fourer tranform and nvere Fourer tranform of e mage f( xy, ) repectvely, g a low-pa Gauan kernel, f g ndcate e convoluton of f and g, x repreent e ampltude of complex number x, P(u, v) e phae of Fourer tranform of mage f( xy, ). (b). Extracton of Early Vual Feature Gven an nput mage, let r, g, and b denote e red, green, and blue channel repectvely, en e ntenty mage I can be computed a Irg b /3, and e nvere ntenty mage can be computed a I off =255 I. Four broadly-tuned color channel are created for red, green, blue and yellow repectvely,.e., R rg b /2, G grb /2, Bbrg /2, and Yrg /2 rg /2 b n whch negatve value are et to zero. Accordngly, map RG created to multaneouly account for red/green and green/red double opponency and BY for blue/yellow and yellow/blue double opponency, whch are calculated by Eq. (3) and Eq. (4). RG R G, (3) BY B Y. (4) (c). Bottom-up Salency Map After applyng pectral flterng to feature channel RG, BY and I, reulted feature map F, {RG,BY,I} are lnearly ummed and normalzed to yeld e bottom-up alency map, hown a Eq. (5). 1 Sbu F, {RG,BY,I}. (5) 3 (d). Top-down Salency Map The top-down alency map calculated by Eq. (6)-(8). When new frame f( t ) come, e prevouly learned weght vector w (ee e paragraph at e end of ecton for detaled learnng procedure) ued to weght e feature map {F }, whch are determned by pectral flterng. Dependng on e value w, e map are ued to compute e exctaton map E or e nhbton map I. E e weghted um of all feature map S at are mportant for e learned regon,.e., w > 1: E(f( t)) w F (f( t)), w 1, { R, G, B, Y, I, Ioff} (6) The nhbton map I conder e feature more preent n e background an n e target regon,.e., w 1: 1 I(f( t)) ( ) F (f( t)), w 1, { R, G, B, Y, I, Ioff} (7) w Map w w 1 are completely unmportant for e target and are gnored. The top-down alency map S td reult from e dfference of E and I and a clppng of negatve value: S td (f( t)) max(e(f( t)) I(f( t)), 0 ). (8) T The weght vector w ( wr, wg, wb, wy, wi, wi ) off repreentng alent feature of e target obect relate to t urroundng computed at e frt frame f(0) a Eq. (9). The value w for feature map F computed a e rato between e mean alency of target regon and background: mean(f (T(0))) w, { R, G, B, Y, I, Ioff}, (9) mean(f (f(0) \ T(0))) where T(0) denote target regon and f(0) \ T(0) denote background regon. (e). Fnal Salency Map Combned Bottom-up and Topdown Vual Attenton The fnal alency map computed a e lnear combnaton of e bottom-up alency map and e top-down alency map, hown a Eq. (10). SS (1 )S, [0,1], (10) td where e top-down coeffcent, whch ued to tune e relatve mportance of top-down vual attenton and bottom-up vual attenton. B. Salency Weghted Spare Codng Apperance Model (a). Salency Poolng Poolng operaton an eental tep n e general framework of appearance modelng baed on pare codng (AMSC) for vual trackng. A tradtonal poolng operaton elect feature wout noton of foreground, feature extracted from background may have tronger repone to e learned dctonary. A a reult, e poolng operaton alone may nevtably ntroduce noe nto e repreentaton. To cope w e problem, we propoe a alency weghted poolng functon to help poolng operaton focu on regon where foreground may appear. The tradtonal poolng operaton hown n Eq. (11) and e alency weghted poolng functon hown n Eq. (12). Thee two poolng operaton reult n fnal pooled feature vector f and f. bu f poolng( B ), (11) f wpoolng( B) poolng( B ), (12) where B [ b1, b2,, b ] repreent e pare code of one canddate. b, {1,2,,} e pare code of e local mage patch n e canddate. B [ 1b1, 2b2,, b] e alency weghted pare code. mean(s(y )), {1,2,, } e mean alency of e local mage patch y. 4093

3 (b). Salency Weghted Spare Codng To demontrate e advantage of e above propoed alency poolng functon, we ntroduce e propoed alency poolng functon to e adaptve tructural local pare appearance model [8] (ASLA) a a cae tudy. Gven target template T [ T1,, Tn ], a et of overlapped local mage patche are ampled nde e target regon. To encode e local patche nde e poble canddate regon, mage patche ampled from target template are ued a e ( ) dctonary,.e., [ 1, 2, ( )] R d n D d d d n, where n e number of target template, e number of local patche ampled from e target regon and d e dmenon of e mage patch vector. By ung l 2 normalzaton on e vectorzed mage patche ampled from T, each column of D obtaned. For a target canddate, local mage patche are extracted n e ame way, and denoted a [ 1, 2,, ] R d Y y y y. Under a pare contrant, pare codng repreent e local patch y wn e target regon a a lnear combnaton of only a few ba element of dctonary D. The pare ( ) 1 code R n b of y can be olved by 2 t 2 1 b b arg mn y Db b.. b 0. (13) The pare code of one canddate are denoted a B [ b, b,, b ]. 1 2 The pare code are en pooled by e algnment poolng operator, whch compute e feature element for each patch a e um of e code correpondng to e ba element at e ame poton w e patch and defned a follow. The ( ) 1 pare code R n b of y dvded nto n egment, (1) (2) ( ).e., b T [ T, T,, n T b b b ], where ( kt ) 1 b R, k {1,2,,n} e k egment of e coeffcent vector b correpondng to template T k. Thee egmented coeffcent are lnear ummed to obtan v for e patch, n 1 v b, 1,2,,, (14) C k 1 ( k ) where v correpond to e local patch and C a normalzaton contant. All e vector v of local patche n a canddate regon form a quare matrx V, and e fnal pooled feature obtaned by takng e dagonal element of e quare matrx V, f poolng( B) Dag( V ). (15) Ung e propoed alency weghted poolng functon,.e., Eq. (12) to Eq. (15), e pooled feature w alency property calculated a Eq. (16). f wpoolng( B) poolng( B ) Dag( V ), (16) where B [ 1b1, 2b2,, b], V [ 1v1, 2v2,, v], mean(s( y)), {1,2,, } e mean alency of e local mage patch y. C. Trackng Algrm The propoed appearance model embedded wn e Bayean trackng framework. Baed on e et of all avalable meaurement z 1: t { z 1,, zt}, e target tate xt can be etmated a x arg max p( x z ), (17) t t 1: t xt where x e tate of e ample. W e Bayean t eorem, e poteror probablty p( xt z 1: t) can be nferred recurvely a Eq. (18). p( x z ) p( z x ) p( x x ) p( x z ) dx, (18) t 1: t t t t t1 t1 1: t1 t1 where p( zt x t) ndcate e obervaton model and p( xt x t1) repreent e dynamc model. The target moton between two conecutve frame modeled by affne tranformaton w x parameter. The tate tranton formulated a p( xt xt 1) ( xt; xt 1, ), where a dagonal covarance matrx. The obervaton model contructed by Eq. (19). where (k) ( t t), k 1 p z x f (19) (k) f denote e mlarty between e canddate k 1 and e target baed on e pooled feature vector f calculated ung Eq. (16), and (k) f e k component of e pooled feature vector f. For template update, we ue e update cheme of ASLA [8], whch explot bo party and ubpace learnng. III. EXPERIMETS Our tracker mplemented n MATLAB whch run at 1.5 fp on a Intel(R) 2.93 GHz Dual Core PC w 2GB memory. For each equence, e locaton of e target obect ntalzed a e ground tru poton n e frt frame. The top-down coeffcent n Eq. (10) et to 0.5. The oer parameter are ame a e default of ASLA [8]. Snce e propoed alency poolng functon appled to adaptve tructural local pare appearance model [8] (ASLA) a a cae tudy, our meod referred to a SWASLA (mean Salency Weghted ASLA). To evaluate our approach (SWASLA), we compare t w oer meod on e trackng benchmark propoed ut recently n [1] ncludng 50 vdeo and 29 meod. For qualtatve comparon, we compare e propoed meod w e baelne meod,.e., ASLA. For quanttatve evaluaton, we compare our meod w 29 meod of e benchmark. 4094

4 A. Qualtatve Comparon Fg.1 Fg.4 how e comparatve trackng reult of ASLA and SWASLA on four vdeo w erou background clutter. Fg.1 how e trackng reult of Baketball equence. ASLA drft away from e target nce e 460 frame. Fg.2 how e trackng reult of Snger2, from e 20 frame, ASLA loe e target. Fg.3 how e trackng reult of Soccer, ALSA dtracted by a mlar mage regon at e 66 frame. Fg.4 how e trackng reult of Subway, ALSA drft at e 42 frame and dtracted by anoer man. In ee equence, e propoed meod SWASLA can alway lock onto e obect whle ASLA loe target to ome extent. B. Quanttatve Evaluaton The ucce plot metrc [1] are ued for a quanttatve evaluaton. The ucce plot how e rato of ucceful frame at e rehold of overlap core vared from 0 to 1. Gven tracked boundng box S t and ground tru boundng GT box S t, e overlap core defned a GT GT SC S S S S, where and repreent e t t t t t nterecton and unon of two regon repectvely, and. denote e number of pxel n e regon. A frame w overlap core larger an a gven rehold wll be counted a a ucceful tracked frame. Area under curve (AUC) core are ued to ummarze and rank e tracker. For robutne evaluaton, we run tracker n two way, one-pa evaluaton (OPE) and patal robutne evaluaton (SRE). OPE e conventonal way to evaluate tracker, whch to run tracker roughout a tet equence w ntalzaton from e ground tru poton n e frt frame and report e ucce rate. SRE to ntalzaton by perturbng e ntalzaton patally (.e., tart by dfferent boundng boxe). Fg.5 ummarze e overall ucce plot of OPE and SRE. For SWASLA, AUC of OPE on e left e rd whle AUC of SRE on e rght e econd of all meod, outperformng e ASLA. Fg.6 how e ucce plot of OPE and SRE on background clutter ubet, SWASLA outperform e oer meod. Fg.1. Comparatve trackng reult on Baketball equence. Frame o: 80, 200, 460, 480. (a) Salency map calculated by propoed vual attenton model. (b) Trackng reult. Red rectangle repreent for e trackng reult of our meod, SWASLA, whle green rectangle denote ASLA. Fg.2. Comparatve trackng reult on Snger2 equence. Frame o: 12, 20, 50, 350. (a) Salency map calculated by propoed vual attenton model. (b) Trackng reult. Red rectangle repreent for e trackng reult of our meod, SWASLA, whle green rectangle denote ASLA. 4095

5 Fg.3. Comparatve trackng reult on Soccer equence. Frame o: 16, 66, 80, 150. (a) Salency map calculated by propoed vual attenton model. (b) Trackng reult. Red rectangle repreent for e trackng reult of our meod, SWASLA, whle green rectangle denote ASLA. Fg.4. Comparatve trackng reult on Subway equence. Frame o: 20, 42, 80, 120. (a) Salency map calculated by propoed vual attenton model. (b) Trackng reult. Red rectangle repreent for e trackng reult of our meod, SWASLA, whle green rectangle denote ASLA. Fg.5. Succe plot of OPE and SRE. The performance core for each tracker hown n e legend. For each fgure, e top 10 tracker are preented for clarty (bet vewed on hgh-reoluton dplay) 4096

6 Fg.6. Succe plot of OPE and SRE of background clutter ubet. The value appear n e ttle e number of equence n ubet,.e., ere are 21 equence n ubet. The performance core for each tracker hown n e legend. For each fgure, e top 10 tracker are preented for clarty (bet vewed on hgh-reoluton dplay) IV. COCLUSIO Th paper propoe a alency weghted pare codng appearance model for vual trackng. A novel pectral flterng baed vual attenton computatonal model calculate alency map and calculated alency map ued to weght e poolng operaton n pare codng. Expermental reult on a recently propoed trackng benchmark how e effectvene of e propoed meod. Frtly, e preented meod outperform baelne tracker n overall performance. Secondly, e propoed meod can cope w background clutter robutly and outperform tate-of-e-art meod on background clutter ubet. [9] S. Zhang, H. Yao, X. Sun, and X. Lu, "Spare codng baed vual trackng: Revew and expermental comparon," Pattern Recognton, vol. 46, pp , [10] A. Bor and L. Itt, "State-of-e-Art n Vual Attenton Modelng," Pattern Analy and Machne Intellgence, IEEE Tranacton on, vol. 35, pp , [11] J. L, M. D. Levne, X. An, X. Xu, and H. He, "Vual Salency Baed on Scale-Space Analy n e Frequency Doman," Pattern Analy and Machne Intellgence, IEEE Tranacton on, vol. 35, pp , REFERECES [1] Y. Wu, J. Lm, and M.-H. Yang, "Onlne Obect Trackng: A Benchmark," preented at e IEEE Conference on Computer Von and Pattern Recognton (CVPR), [2] A. Ylmaz, O. Javed, and M. Shah, "Obect trackng: A urvey," ACM Comput. Surv., vol. 38, p. 13, [3] H. Yang, L. Shao, F. Zheng, L. Wang, and Z. Song, "Recent advance and trend n vual trackng: A revew," eurocomputng, vol. 74, pp , [4] S. P. Zhang, H. X. Yao, and S. H. Lu, "Robut vual trackng ung feature-baed vual attenton," n 2010 IEEE Internatonal Conference on Acoutc, Speech, and Sgnal Proceng, ed ew York: IEEE, 2010, pp [5] Q. Wang, F. Chen, J. Yang, W. Xu, and M.-H. Yang, "Tranferrng vual pror for onlne obect trackng," Image Proceng, IEEE Tranacton on, vol. 21, pp , [6] B. Lu, J. Huang, L. Yang, and K. C., "Robut trackng ung local pare appearance model and K-electon," n Computer Von and Pattern Recognton (CVPR), 2011 IEEE Conference on, 2011, pp [7] Q. Wang, F. Chen, W. Xu, and M.-H. Yang, "Onlne dcrmnatve obect trackng w local pare repreentaton," n Applcaton of Computer Von (WACV), 2012 IEEE Workhop on, 2012, pp [8] J. Xu, L. Huchuan, and Y. Mng-Huan, "Vual trackng va adaptve tructural local pare appearance model," n Computer Von and Pattern Recognton (CVPR), 2012 IEEE Conference on, 2012, pp