A novel kernel-pls method for object tracking

Transcription

1 valable onlne Journal of Chemcal and Pharmaceucal Research, 204, 6(7): Research rcle ISSN : CODEN(US) : JCPRC5 novel kernel-pls mehod for obec rackng Y Ouyang*, Yun Lng and Byan Wu School of Comuer and Informaon Engneerng, Zheang Gongshang Unversy, Hangzhou, Chna BSRC In hs aer, we roose a On-lne kernel-pls aroach o mrovng boh he robusness and accuracy of obec rackng whch s arorae for real-me vdeo survellance. ycal rackng wh color hsogram machng rovdes robusness bu has nsuffcen accuracy, because does no nvolve saal nformaon. On he oher hand, rackng wh xel-wse machng acheves accurae erformance bu s no robus agans deformaon of a arge obec. o ackle hese roblems, hs aer resens a rackng mehod ha combne hsogram-wse machng and xel-wse emlae machng va leans a robus obec reresenaon by Kernel-PLS analyss and adas o aearance change of he arge. In hs aer, we roose a novel On-lne Kernel-PLS analyss, for generang a low-dmensonal dscrmnave feaure subsace. s obec aearance s emorally correlaed and lkely o reea over me, we learn and ada mulle aearance models wh On-lne Kernel-PLS analyss for robus rackng. Key words: Image rocessng; PLS; Obec rackng; Classfcaon; Feaure saces INRODUCION arge rackng has been an moran oc n comuer vson for several years. Recen years have been sgnfcan rogress n rackng. o dsngush arges wh backgrounds and wh each oher, mos vsual rackng mehods focus on rackng arge aearance searaely; hey usually ry o fnd roer aearance models ha dsngush obec wh all oher arges or backgrounds, and ado meanshf or arcle flerng lke aroach o onlne adus arge aearance models, and use udaed models o connuously rack arges. he mos rackng mehods reored o handle hs dffculy, hus far, s o adavely udae he arge aearance model a each frame: learn a new aearance model wh me nvaran characerscs exraced from hsorc observed arge samles, and ado he model o he curren frame. Such as, IV [] algorhm uses a subsace model by addng adavely modfy he aearance of he model. rackng-learnng-deecon (LD) [2] mehod o rack ask s decomosed no hree sub-rocesses: rackng, learnng and learnng, each sub- ask as a searae ask, each sub-ask can be erformed s rackng wll fal. (2 ) he arge dsaeared, deecon oeraor connues. (3 ) less real-me rackng and oher ssues. In order o solve he above roblems, Kalal [3] analyze a varey of nformaon n vdeo mages and roosed a new learnng framework called P-N Learnng for ranng a bnary classfer from labeled and unlabeled examles. he learnng rocess s guded by osve exers and negave exers consrans whch resrc he labelng of he unlabeled se. P-N learnng evaluaes he classfer on he unlabeled daa, denfes examles ha have been classfed n conradcon wh srucural consrans and augmens he ranng se wh he correced samles n an erave rocess. Learnng rocesses for any errors exs, he exer oeraor P-N due o muual comensaon of he error robably s lmed whn a ceran range n order o acheve sably. Based on P-N learnng mehod, Sago [4] roosed PLS mehod o oban a beer arge rackng. he Fragmen-based racker [] ams o solve aral occluson wh a reresenaon based on hsograms of local aches. he rackng ask s carred ou by combng voes of machng local aches usng a emlae. here are some sarse marxes combned wh arcle fler for arge rackng alcaons L [5]. Kernel-based aern recognon mehods [4] such as Suor Vecor Machnes (SVMs) [5], Kernel-PC 659

2 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): (KPC)[6] and Kernel-Paral Leas Squares analyss (KPLS) [7,8] have revously been aled n a mulude of conexs for exloraory analyss and classfcaon, ncludng bologcal alcaons [9]. arge rackng roblem s challengng as rackng rocessng needs o deal wh aearance varaons caused by many facors such as llumnaon, ose deformaon, occluson, background cluer, and camera moon. hs aer focuses on he ssue of nonlnear K PLS analyss for arge rackng. o ackle hese challenges, we resen a rackng mehod ha learns a robus obec reresenaon by Kernel aral leas squares analyss and adas o aearance change of he arge. Frsly, we exend hese earler works[4] by embeddng nonlnear kernel analyss for PLS rackng. o mrove he exsng work, we erform he color hsogram robably densy funcon for he obec color consran s modeled as a smooh funcon ha ndcaes how well he canddae se mages and arge s me. o mrove sably for sandard belef roagaon, he color consran funcon node s conneced o four graden varable nodes. lhough deecon resuls can be mroved by ulzng overlang blocks for low-level feaure exracon whn he deecon wndow, he dmensonaly of he feaure vecor becomes exremely hgh. s a resul, he seed of he human deecor decreases sgnfcanly due o he me needed o exrac feaures and roec hem. o overcome hs roblem, we emloy a On-lne learnng aroach. In a fas frs sage, based on a small number of feaures, he maory of deecon wndows (hose wh low robably of conanng humans) are dscarded. he remanng wndows are evaluaed durng a second sage where he comlee se of feaures allows challengng samles o be correcly classfed. ON-LINE KERNEL-PLS NLYSIS(KPLS) N K X R Y R he PLS mehod emloys he descror marx, where N denoes he number of samles and K he number of varables n X, o redc he resonse marx N M, where M denoes he number of varables n Y. he unque roery of he PLS mehod comared o oher lnear regresson mehods s s ably o searae he modelng of covaraon from srucured nose, defned as sysemac Y-orhogonal varaon, whle smulaneously maxm he covarance beween X and Y. he K-PLS algorhm follows he rncles of he lnear PLS algorhm whle s wren n dual form. hs mles ha all exressons have been rewren so ha he nu marx X s exressed as he ouer roduc XX n all nsances. he ouer roduc XX s subsequenly relaced by he kernel marx K n he K-PLS algorhm. K s deflaed for he Y-orhogonal comonens. In smle erms, n he algorhm for model esmaon, K consss of wo nsances of he ransformed daa marx. One of hese nsances reresens he redcve weghs W and should hus be reaned hroughou he calculaons, whereas he oher should be deflaed accordngly by he Y-orhogonal varaon. Subsequenly, XX, (, ) s subsued for he kernel Gram marx K wh enres K = k x x x x, where and corresonds o he -h k( x, x ) and -h row vecor n he descror marx X, resecvely, and reresens he kernel funcon. Hence, one can avod exlcly mang X o hgher-dmensonal saces as well as comung do roducs n he feaure sace, whch s comuaonally benefcal. he ransformaon o hgher dmensonal saces s erformed mlcly by he kernel funcon k( x, x ) ; where common kernel funcons use Gaussan funcons as follow: k x y ex x y σ 2 2 (, ) = ( /2 ) (0.) he kernel funcons n Equaons (.) deend on he arameers σ, whch nfluences he redcve ably of he kernel-based mehod. he radonal aroach o kernel arameer selecon s o erform an exhausve grd search over he enre arameer sace based on redefne arameer. he generalzaon roeres of he model are evaluaed usng e.g. cross-valdaon [9] o denfy he arameer seng yeldng he lowes ossble generalzaon error, whch can resul n a large number of calculaon and run mes. he roecon drecon n he feaure sace a each 660

3 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): sage s gven by he vecor u of as a u u, whch s n he rmal sace whle we mus work n dual sace,and exress a mulle = X β (0.2) u whch s clearly conssen wh he dervaon of n he rmal PLS algorhm. For he dual PLS algorhm we mus mlemen he deflaon of Y. hs redundan se for he rmal wll be needed o ge he requred dual reresenaons. We use Y o denoe he -h deflaon. β = Y Y X X β = Y Y K β (0.3) β β = wh he normalzaon β,hen comue a rescaled and c. where = a X u = X X β = K β c Y Y X u c = = = a u X X u a (0.4) c denoes he same wegh vecors c n PLS algorhm. can be consder as a rescaled dual reresenaon of he ouu vecor c. Le u, v, σ X be he frs sngular vecor of Y,he deflaon of X as X = ( I + ) X (0.5) wh an equvalen deflaon of he kernel marx K + = X + X + = ( I ) K ( I ) (0.6) In hs aer, we formulae obec rackng as a classfcaon roblem wh On-lne Kernel PLS analyss o learn a low-dmensonal and dscrmnave feaure subsace. S.Wold e al. [2,3]were he frs o exend he lnear PLS model o s nonlnear form. hey have done hs by relacng he lnear nner relaon beween he score vecors and u by a nonlnear model. u = k( ) + e = k( X, w) + e (0.7) where k reresens a connuous nonlnear funcon, e denoes a vecor of resduals. he relaon beween each ar of laen varables s modeled searaely. We use radal bass funcon o model k. I can be observed ha he vecor of weghs of w, comued n he frs se of he NIPLS algorhm, reresens he samle covarance beween he ouu sace score vecor u and he nu sace daa marx X. However, he use of a nonlnear model o relae he score vecors n he nner relaon affecs he comuaon of w. lhough w reresens he assocaon among varables of X and u also n nonlnear PLS, hs assocaon wll be closely relaed o covarance values only f he nonlnear mang beween laen varables s monoonc and slghly nonlnear. In our model, a kernel funcon modelng he nner relaon s used o udae an nal KPLS esmae of he wegh vecor w. S.Wold [2,3] roosed o udae w by means of a Newon-Rahson lnearzaon of g. he rocedure hus consss of a frs-order aylor seres exanson of g, followed by he calculaon of he correcon erm w whch s 66

4 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): used o udae w. So, consder he nonlnear nner relaon, where k( ) = k( X, w) wh resec o w. he second-order aylor exanson of has he form s connuous and dfferenable k uˆ u w = + 0 w (0.8) k u where 0 = k( ) s he value of g a he known value of. Smlarly, w sands for he aral dervaves of g numercally evaluaed a he same known value of. he second erm of 2 can be wren elemen-wse as k = w N = k w w (0.9) hs on several dfferen mehods o comue he correcon consder he marx form of he lnear aroxmaon û w were roosed. o smlfy furher noaon where û = Zv (0.0) k Z = [ u0 ] w v = [ w] and. he followng varans o comue w were suggesed: D k( x, y) = α φ( x) φ( y) = Φ( x) Φ( y) = =< Φ( x), Φ ( y) > (0.) where { φ } s a sequence of lnearly ndeenden funcons, { α } are osve numbers and D dmenson of he sace H. Followng hs relaon he feaure ma Φ can be wren as N s he Φ : X F (0.2) x Φ ( x) = ( a φ ( x), a φ ( x),..., a φ ( x)) 2 2 N N (0.3) hus, f we are only neresed n he comuaon of do roducs n F, does no maer how F was consruced and smly all do roducs can be relaced by a unque kernel funcon assocaed wh F. hs s moran o noe because dfferen feaure saces assocaed wh he same kernel funcon can be consruced. In leraure, hs relacemen of a do roduc wh he kernel funcon value s known as he kernel rck mehod. modfed verson of he NIPLS algorhm where ses and 3 are merged and he score vecors and u are scaled o un norm nsead of scalng he wegh vecors w and c. he obaned kernel form of he NIPLS algorhm s as follows Se: = ΦΦ u = Ku Se2: Se3: c = Y Se4: u = Yc Se5: u lhough se 2 guaranee orhogonaly of he score vecors can be rescaled o follow he sandard lnear NIPLS algorhm wh he un norm wegh vecors vecor w. Se 3 and 4 can be furher merged whch may become useful n 662

5 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): alcaons where an analogous kernel mang Φ of he Y-sace daa s consdered; ha s, he Gram marx K = ΦΦ y of he cross do roducs beween all maed ouu daa s consruced. hen, he kernel NIPLS algorhm consss of he followng four ses. Se 3 and 4 can be furher merged whch may become useful n alcaons where an analogous kernel mang Φ of he Y-sace daa s consdered; ha s he Gram marx K = ΦΦ y of he cross do roducs beween all maed ouu daa s consruced. hen, he kernel NIPLS algorhm consss of he followng four ses Se: = Ku Se2: u = K y Se3: Se4: u he On-lne KPLS algorhm learns a hreshold n lnear funcon n a kernel-defned feaure sace. h( x) = sgn < w, φ( x) > (0.4) If he wegh vecor afer udaes s denoed by ( x, y ) s msclassfed s gven by w = w + + yφ ( x ) (0.5) w hen he udae rule for he (+) udae when an samle Hence, he corresondng dual udae rule s smle a = a + (0.6) f we assume ha he wegh vecor s exressed as N = = w a y φ( x ) (0.7) W = [ w,..., w k ] Once he wegh marx ˆ = s comued, he nal aearance model can be denoed by { x, x, W} x m, where s he mean of he osve samles. es samle, x R, can be roeced ono he learned laen feaure sace secfed by o ge a laen feaure vecor z = W ( x xˆ ). Usng Z wh lower dmensonaly, a arge obec can be more easly dscrmnaed from he background han n he orgnal feaure sace X. 663

6 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): Fgure.Pseudo-code for he On-lne KPLS algorhm RGE PPERNCE MODEL Snce he aearance change of an obec durng a long erod of me may be que nonlnear and comlex, one lnear aearance model s no lkely o suffce. However, aearance of a arge obec may be emorally correlaed and may reea over me. We herefore learn mulle aearance models for more effecve obec reresenaon. Vdeo sequence of a arge over a long erod can be dvded no mulle ses. Whn he -h se, he obec aearance dosed no change much and we use KPLS analyss o learn a dscrmnave aearance model = { x, x, W}. herefore, he aearance of a arge obec can be reresened by mulle aearance models = k {,..., }, where k s he number of aearance models. he roosed reresenaon scheme s more effecve han exsng mehods base on sngle lnear aearance model. In hs aer, he dsance beween a es samle x m R and he learned aearance model se s defned as where and 2 k Ds = W ( x x) W ( x x). = 2 2 (0.8) x s he mean of he osve samles used n ranng s Eucldean norm., x s he mean of all he samles n ranng he arge and background aearances may change due o facors such as llumnaon, ose, occluson, camera moon, and so on. o deal wh hs roblem, we roose an adave obec reresenaon mehod. Le he curren se of aearance models be = { =,..., k, k K}., When he rackng resul a me s obaned, we use he corresondng arge observaon x o udae. Snce we x o all he aearance models n for deermnng he ds and he aearance model l have comued he dsances from he arge observaon rackng resul, we selec he aearance model s wh he smalles dsance ' wh he larges dsance d d. If s s less han a redefned hreshold, x s ulzed o udae s. 664

7 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): he udae rocess ncludes hree comonens: he mean of he osve samle samles x, and he wegh marx W. x,he mean of all he ranng x can be udaed by usng a random udae robably. Boh ˆx and W can be udaed by KPLS mehod wh he osve and negave samles. If s l < K, a new aearance model k + s added o. If s l s larger han he redefned hreshold and k s larger han he redefned hreshold and k = K, a new aearance model s nalzed o relace l n. he roosed adave aearance model s summarzed as follow: Fgure 2.dave earance Model Based on KPLS analyss ON-LINE KPLS RCKING MEHOD VI MP x = [ x,... x ] Gven he observaon se of he arge : Maxmum Poseror (MP) esmaon, u o me, he rackng resul s can be deermned by he s = argmax ( s x ) : (0.9) ( s x ) where : s nferred by he Bayes heorem recursvely wh follow equaon. ( s x ) ( x s ) ( s x ) : : (0.20) ( s x: ) = ( s s ) ( s x: ) ds (0.2) 665

8 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): hs nference s governed by he dynamc model : ( s s ) whch descrbes he emoral correlaon of he ( x s ) rackng resuls n consecuve frames, and he lkelhood funcon (.e., observaon model) whch s denoes he lkelhood of observng x. he man ssue for any adave aearance model s ha s lkely o use nosy or ms-algned observaons for udae, hereby causng rackng drf gradually. For onlne rackng, he only ground ruh a our dsosal s he labeled arge obec n he frs frame. X = { L( x, y), s} Le X as random varable of a vdeo sequence, defnng he sae vecor cener locaon, L(, ),where x y s he arge s s he scale facor, ( y x) denoes he robably of clques belongs o arge. Moon model ( x x ) generae a redc x, whch denoes he correlaon of arge's emoral srucure n he vdeo. We assume ha he moon model obeys Gaussan dsrbuon: ( x x ) = N( m( x, x ), Λ) (0.22) where Λ s he dagonal covarance marx, flerng mehod o rack arge. Predcon sage: Bel( x) ( x x ) Bel( x ) dx = Udae sage: Bel( x ) a ( y x ) Bel( x ) m( x, x denoes he means of wo random varable. We use Bayes Observaon Markov assumons: Observaon deends only on he sae of he curren observaons,.e. ( y x ) = ( y x ) ( z x ) : :, he flerng rocess s manly deermned by he dynamc model ( x x ), whch descrbes he sae emoral correlaon of he arge beween frames. EXPERIMENL NLYSIS o evaluae On-lne KPLS, we comle a se of 5 challengng rackng sequences (e.g. car4, woman) ha are ublcly avalable onlne. Due o sace consrans, we wll only show resuls on 4 of hese sequences. hese vdeos are recorded n ndoor and oudoor envronmens and nclude challengng aearance varaons due o changes n ose, llumnaon, scale, and he resence of occluson. he cener dsance error resuls of roosed rackng mehod are comared wh ha of curren sae-of-he-ar mehods IV [], LD [2], LO [7] and RO [8]. Exermenal vdeo sequences snger and baskeball from VD[9], lemmng and lquor from PROS[0], woman from Fragrack[] and grl_mov from SP [6]. Imlemened n MLB on an Inel Core 2.26 GHz comuer wh 4GB RM and no code omzaon. able. he mean of Cener D denoe average errors of cener oson Seq\Mehod IV LD SP LO RO OUR Baskeball Grl_mov Bol Lquor Woman

9 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): woman grl_mov baskeball lquor Fgure 3. rackng resuls comarson durng rackng rocess Fgure 4. CDE resuls comarson durng rackng rocess Fg 3 woman and lquor subsequence occurred wh he camera movemen, IV rackng wll fal. he key reason s no full use of he arge and background aearance model, and makng he rackng accuracy s lmed. On-lne KPLS 667

10 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): rackng mehod consders local and envronmenal feaures surroundng, and adavely adusng he arge aearance model, whch can more accuraely dsngush arge and mrove rackng accuracy. In Fgure3 baskeball sequence, because he arge of he block s no serous, he roosed mehod and SP almos unanmously, n grl_mov, when raced o 5. In lquor obec aears n a smlar suaon, he roosed mehod wh SP smlar. he resence of he arge under occluson, he roosed mehod s sueror o oher mehods. In able-and Fgure 4, he aer uses Cener Dsance Error (CDE)o analyss he rackng erformance, where '-' ndcaes aral frame deecon falure rackng rocess.cde reresens he cener oson coordnaes errors beween he rackng resuls and he reference sandard (ground ruh), where$. $ s he Euler dsance, '-' ndcaes some frames rackng falure n he rackng rocess. he arge obecs are arally occluded n he woman sequences, whch has a arally occluson henomenon. Usng On-lne KPLS mehod can elmnae he arge because of changes n background aearance model for he mac, and he adave aearance learnng aroach hrough dynamc udae aearance models, arge rackng accuracy of beer han LD, IV, LO and RO mehods o deec and deal wh serous obscured arges. he resuls (Fgure 4) show ha s normal condon n he cener and he ground ruh deecon frame error s bascally he same, when here s heavy occluson, our mehod s sgnfcanly beer han oher rackng algorhms. CONCLUSION hs aer resens a novel On-lne Kernel PLS analyss mehod for obec rackng, and ulzes he osve samles and negave samles for adavely udaed nformaon on he arge - background aearance model. o solvng he hese roblems, such as occluson, llumnaon changng, and he shae changng durng he rocess of he arge rackng. We emloy he mage KPLS wegh scores as confdence resonse ma o deermne he arge locaon nformaon. lhough radonal rackng mehod rackng deecon resuls can be mroved by ulzng overlang blocks for low-level feaure exracon whn he deecon wndow, he dmensonaly of he feaure vecor becomes exremely hgh. s a resul, he seed of he human deecor decreases sgnfcanly due o he me needed o exrac feaures and roec hem. o overcome hs roblem, we emloy a On-lne learnng aroach. In a fas frs sage, based on a small number of feaures, he maory of deecon wndows (hose wh low robably of conanng humans) are dscarded. he remanng wndows are evaluaed durng a second sage where he comlee se of feaures allows challengng samles o be correcly classfed. cknowledgmen he auhors would lke o hanks he anonymous revewers for her consrucve and useful commens, whch heled n mrovng he resenaon of our work. REFERENCES [] Ross D,Lm J, Ln R S, e al. Inernaonal Journal of Comuer Vson, 2008, 77(-3): [2] Kalal Z, Mkolaczyk K and Maas J. IEEE ransacons on. 202, 34(7): [3] Kalal Z,Maas J and Mkolaczyk K. Pn learnng: Boosrang bnary classfers by srucural consrans, Comuer Vson and Paern Recognon (CVPR), 200 IEEE Conference on, 200: [4] Sago H, Kr mer N and suda K. Paral leas squares regresson for grah mnng. Proceedngs of he 4h CM SIGKDD nernaonal conference on Knowledge dscovery and daa mnng, 2008: [5] Wang Yu-Xang, Xu Han and Ma ng-hua. Journal of Elecroncs & Informaon echnology,202,v34(): [6] Wang S,Lu H and Yang F, e al. Suerxel rackng[c].comuer Vson (ICCV), 20 IEEE Inernaonal Conference on, 20: [7] Oron S,Bar-Hllel and Lev D, e al. Locally orderless rackng. Comuer Vson and Paern Recognon (CVPR), 202 IEEE Conference on, 202: [8] Nng J, Zhang L, Zhang D and Wu C. Robus obec rackng usng on color-exure hsogram. Inernaonal Journal of Paern Recognon and rfcal Inellgence. 2009, 23(07): [9] Kwon J and Lee K M. Vsual rackng decomoson. Comuer Vson and Paern Recognon (CVPR), 200 IEEE Conference on, 200: [0] Sanner J,Lesner C and Saffar, e al. Pros: Parallel robus onlne smle rackng. Comuer Vson and Paern Recognon (CVPR), 200 IEEE Conference on, 200:

11 Y Ouyang e al J. Chem. Pharm. Res., 204, 6(7): [] dam,rvln E and Shmshon I. Robus Fragmens-based rackng usng he Inegral Hsogram.Comuer Vson and Paern Recognon 2006 IEEE Comuer Socey Conference on, 2006: [2] rygg J, Wold S. Journal Chemomercs,2002, 6:9-28. [3] rygg J, Wold S. Journal Chemomercs,2003, 7:53-64 [4] Shawe-aylor J, Crsann N: Kernel mehods for aern analyss, Cambrdge Unversy Press; 2004:462. [5] Schölkof B, Smola : Learnng wh Kernels: Suor Vecor Machnes, Regularzaon, Omzaon, and Beyond.Cambrdge, MI Press; 200. [6] Rosal R, Grolam M,reo LJ, Cchock : Neural Comu l 200, 0(3): [7] Lndgren F, Gelad P, Wold S: Journal Chemomercs,993, 7(): [8] Rosal R, reo LJ: J Mach Learn Res,2002,2(2): [9] nderson DC, L W, Payan DG, Noble WS: J Proeome Res 2003,2(2):