Information Sciences

Transcription

1 Ifrmati Scieces 292 (2015) Ctets lists available at ScieceDirect Ifrmati Scieces jural hmepage: Kerel sparse represetati fr time series classificati Zhihua Che a, Wagmeg Zu b,, Qighua Hu c, Liag Li d a Schl f Sftware, Harbi Istitute f Techlgy, Harbi, Chia b Schl f Cmputer Sciece ad Techlgy, Harbi Istitute f Techlgy, Harbi, Chia c Schl f Cmputer Sciece ad Techlgy, Tiaji Uiversity, Tiaji, Chia d Schl f Advaced Cmputig, Su Yat-Se Uiversity, Guagzhu, Chia article if abstract Article histry: Received 12 February 2014 Received i revised frm 13 August 2014 Accepted 29 August 2014 Available lie 8 September 2014 Keywrds: Sparse represetati Dictiary learig Time series classificati Kerel methd I recet years there has bee grwig iterests i miig time series data. T vercme the adverse ifluece f time shift, a umber f effective elastic matchig appraches such as dyamic time warp (DTW), edit distace with real pealty (ERP), ad time warp edit distace (TWED) have bee develped based the earest eighbr classificati (NNC) framewrk, where the distace d(x, C i ) betwee a test sample x ad e specific class C i is simply defied as the miimum distace betwee x ad the traiig samples i this class. I may applicatis, the sparse represetati classifier (SRC) was applied by defiig d(x, C i ) as the distace f x t a liear cmbiati f the samples i class C i, ad it usually utperfrmed NNC i terms f classificati accuracy. Hwever, due t time shift, a liear cmbiati f several time series is geerally meaigless ad may result i pr classificati perfrmace. I this paper, a family f Gaussia elastic matchig kerels was itrduced t deal with the prblems f time shift ad liear represetati. I this way, a liear cmbiati f time series ca be cducted i the implicit kerel space. The a kerel sparse represetati learig framewrk fr time series classificati was prpsed. T imprve cmputatial efficiecy ad classificati perfrmace, bth usupervised ad supervised dictiary learig techiques were develped by extedig KSVD ad label csistet KSVD algrithms. Experimetal results shwed that the prpsed methds geerally utperfrmed state-f-the-arts methds i terms f classificati accuracy. Ó 2014 Elsevier Ic. All rights reserved. 1. Itrducti A time series is a sequece f umerical values, typically measured at successive time istats spaced at uifrm time itervals. It ca be used t describe the states f bjects ad reflect their variatis alg time tags. Fr example, a electrcardigram, which is widely used t aalyze abrmal heart rhythms, is preseted as a lie graph, where the x-axis is time ad the y-axis stads fr the average vltage measured by the electrdes. Time series are used i varius applicatis ad ca be easily acquired by the existig techiques. Clusterig, classificati, ad miig f time series [9,24,30,38] have bee extesively studied i may applicatis, such as sig laguage recgiti [27], trajectry-based activity recgiti [2], electrcardigraphy (ECG) based medical diagsis [37], stck market time series categrizati, ad predicti [13,39]. Crrespdig authr. Tel.: ; fax: address: wmzu@hit.edu.c (W. Zu) /Ó 2014 Elsevier Ic. All rights reserved.

2 16 Z. Che et al. / Ifrmati Scieces 292 (2015) Similarity matchig betwee time series is essetial fr time series classificati [41]. There are tw key prblems i cmputig the distace r similarity betwee time series: time warpig ad high dimesiality. Time warpig is a geeral pheme i time series, which creates huge challeges fr autmatic time series classificati. Fr example, althugh time series ca be viewed as pits i vectr space, cvetial Euclidea distace is very sesitive t time warpig ad may fail i measurig similarity betwee time series. Dyamic time warpig (DTW) was itrduced t vercme the limitati f Euclidea distace [3,48]. Hwever, DTW is time csumig. T imprve the efficiecy f DTW, Kegh ad Rataamahataa [21] prpsed a lwer budig measure which sigificatly speeds up the calculati f DTW. T avid the excessive distrti f DTW, Vlachs et al. [40] suggested a cstrait f the lgest cmm subsequece (LCSS) by assigig weights t differet pits. Che et al. [7] itrduced a edit distace real sequece (EDR), which is rbust agaist the factrs f ise, time warpig, ad scalig. Ufrtuately, DTW, LCSS, ad EDR are t distace metrics as they d t satisfy the triagle iequality. Recetly, Che et al. [6] itrduced a methd called edit distace with real pealty (ERP) ad Marteau [34] prpsed a aligmet-based distace metric, called time warp edit distace (TWED). Bth f them satisfy the triagle iequality ad are effective i measurig the dissimilarity betwee time series. Usig the existig distace measures, the earest eighbr classificati (NNC) framewrk is usually used fr time series classificati. Let the vectr y be the test sample ad the sample matrix X =[X 1, X 2,..., X K ] be the traiig data matrix, where X i =[x i,1, x i,1,..., x i,i ] is the sample matrix f class C i, ad x i,j detes the jth traiig sample f class C i. Give a distace measure d(x, y), NNC [11] defies the distace f y t class C i, d(y, C i ), as the miimum distace f {d(y, x ij ) j j =1,..., i }, ad the y belgs t the class crrespdig t the miimum f {d(y, C i )j i =1,..., K}, makig the distace measure critical fr NNbased classificati ad clusterig [11,21,28]. Despite its ppularity, the high dimesiality f samples ad the limited size f traiig sets degrade the perfrmace f NNC fr time series classificati. I recet years, a class f sparse represetati based classifiers (SRC) [42,51] ad cllabrative represetati based classifiers (CRC) [52] have bee develped. SRC ad CRC ca achieve prmisig perfrmace i may applicatis, such as face recgiti [42,52], image classificati [14], ad traffic sig recgiti [29]. SRC ad CRC first assig a cefficiet.a ij fr each traiig sample x ij, ad the defie the distace f y t the ith class C i as d(y, C i Þ¼ y P j x ija ij, where kkis sme vectr rm. Whe the size f the traiig set is limited, SRC ad CRC ca jitly utilize all the traiig samples frm class C i t cmpute d(y, C i ), ad usually achieve higher classificati accuracy tha cvetial NNC methds. Zhag et al. [52] prvided sme gemetric explaatis f the wrkig mechaism f SRC/CRC. T further ehace the discrimiative ability ad cmputatial efficiecy, researchers studied the dictiary learig prblem by learig a set f atms frm the traiig set i bth the usupervised [1,46] ad the supervised [14,32,31,54,45,19] maer. I the field f time series classificati, hwever, little atteti has bee give t the SRC/CRC based appraches. Oe pssible reas may be that SRC/CRC perate i liear space while d(y, C i ) is cmputed based the Euclidea distace. But fr time series classificati, Euclidea distace is sesitive t time warpig ad may achieve pr classificati perfrmace. Mrever, dictiary learig f time series is ather challegig task. Recetly, kerel SRC [15,53,47] ad kerel dictiary learig [35,36] appraches have bee prpsed, which makes it pssible t vercme the difficulties f applyig SRC t time series classificati. I this paper, the applicati f sparse represetati based classifiers t time series classificati was ivestigated. First, by itrducig a family f Gaussia elastic matchig kerels, time series were embedded it a implicit reprducig kerel Hilbert space, which allwed the use f kerel SRC fr time series classificati. Secd, based Gaussia elastic matchig kerels, the kerel KSVD algrithm fr usupervised dictiary learig f time series was used, makig kerel SRC cmputatially efficiet ad scalable. Third, by icrpratig class label ifrmati, a kerel versi f the label csistet KSVD methd (Kerel LC-KSVD) was prpsed t further imprve the discrimiative capability f the leared dictiary. Fially, experimetal results shwed that, cmpared with NNC, sigificat imprvemet ca be btaied by usig the prpsed kerel sparse represetati based appraches. Mrever, the prpsed kerel LC-KSVD methd is mre efficiet because f the ehacemet f the cmputatial efficiecy. The remaider f the paper is rgaized as fllws. I Secti 2, a brief survey f the related wrk such as SRC ad KSVD is prvided. I Secti 3, several Gaussia elastic matchig kerels are itrduced, ad the kerel SRC is used fr time series classificati. I Secti 4, the kerel KSVD algrithm is used fr usupervised dictiary learig, ad the a kerel LC- KSVD methd fr supervised dictiary learig is prpsed. I Secti 4, the experimetal results f the prpsed methds are preseted. Fially, Secti 5 gives sme ccludig remarks. 2. Related wrk 2.1. Sparse represetati-based classificati Dete y as the test sample ad X =[X 1, X 2,..., X K ] as the traiig data matrix, where X i =[x i,1, x i,2,..., x i,i ] is the sample matrix f class C i, x i,j is the jth traiig sample frm class C i, ad i stads fr the ttal umber f traiig samples f class C i. Mtivated by the cmpressed sesig thery [4,5,10], Wright et al. [42] prpsed a sparse represetati based classifier (SRC), where the test sample y is apprximated by a liear cmbiati f traiig samples frm all classes with the l 1 -rm sparsity regularizati:

3 ð^aþ ¼arg mi a ky Xak 2 2 þ kka k 1 ; ð1þ where a =[a 1 ;..., a i ;...; a K ] ad a i is the cdig vectr assciated with X i. With the reslved ^a, SRC assigs the class label f y by LabelðyÞ ¼arg mi i dðy; C i Þ¼ky X i^a i k 2 : ð2þ Wright et al. used secd-rder ce prgrammig r LASSO t slve this l 1 -miimizati prblem [42]. Recetly, Yag et al. [44] cducted a cmparative study several l 1 -miimizati slvers: the augmeted Lagragia methd (ALM), iterir-pit methd, fast gradiet based methds, ad hmtpy. I additi t the l 1 -rm regularizer, ther frms f sparsity regularizati [52,18,43,49] such as l 2 -rm [52] ad grup sparsity [18] were studied fr sparse represetati based classificati. T capture the liear similarity betwee samples, the kerel sparse represetati methd was develped [15,53,47]. Give the liear mappig fucti U(x), the ier prduct i the implicit reprducig kerel Hilbert space (RKHS) ca be defied by Kðx; yþ ¼hUðxÞ; UðyÞi: ð3þ Let U i =[U(x i,1 ), U(x i,2 ),..., U(x i,i )] ad U =[U 1, U 2,..., U K ]. Kerel SRC was prpsed t slve the fllwig l 1 -miimizati prblem: ð^aþ ¼arg mi a kuðyþ Uak 2 2 þ kkak 1 : ð4þ By itrducig the kerel matrix K =U T U ad kerel vectr k =U T U(y), the prblem i Eq. (4) becmes ; ð5þ ð^aþ ¼arg mi a a T Ka þ 2k T a þ kkak 1 which ca the be slved by the l 1 -miimizati slvers. Z. Che et al. / Ifrmati Scieces 292 (2015) Dictiary learig SRC ca be viewed as a sparse cdig prblem, where the dictiary is defied as the etire set f traiig samples such as D = X. Hwever, if the umber f the traiig samples is large, the time f sparse cdig rapidly icreases. T imprve the scalability ad discrimiative ability f SRC, dictiary learig was ivestigated t seek a apprpriate ad ccise dictiary fr classificati. Dictiary learig appraches ca be gruped it tw categries: usupervised ad supervised dictiary learig. I usupervised dictiary learig, the target is t fid a cmpact dictiary, where each sample ca be sparsely cded by the dictiary. FOCUSS [25], MOD [12], ad KSVD [1] are several represetative usupervised methds, where the dictiary D is btaied by slvig a ptimizati prblem such as: ðd; AÞ ¼ arg mikx DAk 2 F s:t 8i; k a ik 0 6 T 0 ; ð6þ D;A where a i detes the ith clum f A, kk 0 detes the l 0 -rm which cuts the umber f -zer elemets f a vectr, ad kk F detes the Frbeius rm. Dictiary learig is a -cvex ptimizati prblem, ad mst algrithms lear the dictiary D by iteratig betwee updatig A ad updatig D. Supervised dictiary learig aims t lear a discrimiative dictiary frm the traiig set X by icrpratig the class label ifrmati. Several supervised dictiary learig appraches such as discrimiative KSVD [54], task-drive dictiary learig [31], Fisher discrimiati dictiary learig [45], ad label-csistet KSVD (LC-KSVD) [19,20] have bee prpsed. The lss fucti f supervised dictiary learig geerally icludes bth a recstructi term ad a discrimiati term. Fr example, the lss fucti f LC-KSVD is defied as ðd; W; B; AÞ ¼ arg mi D;W;B;A kx DA k 2 F þ a k Q BA k2 F þ b k H WA k2 F s:t 8i; k a ik 0 6 T 0 ; ð7þ where W is the classificati parameters, A is the discrimiative sparse cdes f the iput samples X, a ad b ctrl the relative ctributi f the tw discrimiati terms, ad the matrices Q ad H are defied based the class label ifrmati. The mdel i Eq. (7) ca be refrmulated as ðd; W; B; AÞ ¼ arg mi D;W;B;A X D B p ffiffiffi C p ffiffiffi B a Q a B AA p ffiffiffi b H b W 2 F s:t 8i; ka i k 0 6 T 0 ; ð8þ ad ca be efficietly slved usig the existig K-SVD slver. Please refer t [19,20] fr mre details LC-KSVD.

4 18 Z. Che et al. / Ifrmati Scieces 292 (2015) SRC with the Gaussia elastic matchig kerel Althugh time series ca be simply represeted as a 1D vectr, the cvetial SRC i Eq. (1) usually cat achieve satisfactry classificati perfrmace due t the prperty f time warpig ad high dimesiality f the data. Mtivated by the success f elastic matchig methds, the Gaussia RBF kerel ca be geeralized it a class f Gaussia elastic matchig kerel. The, a SRC with Gaussia elastic matchig kerel methd fr time series classificati is suggested Gaussia elastic matchig kerel Give tw samples x ad y, the Gaussia RBF kerel is defied as! Kðx; yþ ¼exp kx yk2 ; ð9þ r 2 where r is the hyper-parameter f the Gaussia RBF kerel, kx yk 2 is the square f the Euclidea distace betwee x ad y. Mtivated by the success f elastic matchig methds, a class f Gaussia elastic matchig kerels was prduced by substitutig the Euclidea distace with the elastic distace measures such as DTW, ERP, ad TWED Gaussia DTW kerel Dyamic time warpig (DTW) [3,21] is widely applied i time series classificati ad clusterig. Give tw time series x =[x 1, x 2,..., x m ] ad y =[y 1, y 2,..., y ], where x i (y i ) detes the ith elemet f the time series x(y), the DTW distace betwee x ad y is recursively defied as d dtw x m 1 ; y 1 8 >< ¼ j xm y jþ mi >: d dtw d dtw d dtw x m 1 1 ; y 1 x m 1 1 ; y 1 1 x m 1 ; y 1 1 ; ð10þ where x q p ¼ðx p ; x pþ1 ;...; x q Þ detes the subsequeces f x. By replacig the Euclidea distace i the Gaussia RBF kerel with the DTW distace, the Gaussia DTW kerel is defied as! K dtw ðx; yþ ¼exp d dtwðx; yþ 2 : ð11þ r 2 The Gaussia DTW kerel had bee studied i the supprt vectr machie framewrk, but icsistet classificati perfrmace was reprted, partially because the Gaussia DTW kerel is t a psitive semi-defiite (PSD) kerel [38,16,17] Gaussia ERP kerel The edit distace with real pealty (ERP) [6] is a cmbiati f the l 1 rm ad the edit distace, which is a distace metric rbust agaist time shifts. Give tw time series x =[x 1, x 2,..., x m ] ad y =[y 1, y 2,..., y ], the ERP distace is recursively defied as, 8 P m i¼1 jx i gj; if ¼ 0 8 d erp x m 1 1 ; y 1 þjxm gj d erp x m 1 ; >< >< y 1 ¼ mi d erp x m 1 1 ; y 1 1 þjxm y j ; therwise; >: d erp x m 1 ; y 1 1 þjy gj >: P i¼1 jy i gj; if m ¼ 0 ð12þ where g stads fr the cstat with the default value 0 [6]. Similarly, the Gaussia ERP kerel [50] is defied as! K erp ðx; yþ ¼exp d erpðx; yþ 2 : ð13þ r Gaussia TWED kerel Marteau prpsed the time warp edit distace (TWED) [34] by csiderig the time stamp factrs f time series ad applyig the pit patter matchig prcedure [33] (PPM) t address time warpig. TWED is als a distace metric. By takig it accut time stamps, the time series are represeted by x =[(x 1, t x1 ), (x 2, t x2 ),...,(x m, t xm )] ad y =[(y 1, t y1 ), (y 2, t y2 ),...,(y, t y )], where t xi (t yi ) stads fr the time stamp f elemet x i (y i ). Fr ay time series with t xi < t xj ad "i < j, the TWED distace betwee x ad y is recursively defied as

5 Z. Che et al. / Ifrmati Scieces 292 (2015) d twed x1 m 1 ; y 1 þ j xm x m 1 jþ ct xm t xm 1 d twed x m 1 ; >< þ k y 1 ¼ mi d twed x1 m 1 ; y 1 1 þ j xm y jþ ct xm t y þ j xm 1 y 1 jþ ct xm 1 t y 1 ; ð14þ >: d twed x m 1 ; y 1 1 þ j y y 1 jþct y t y 1 þ k where c ad k are tw -egative cstats. By substitutig the Euclidea distace i the Gaussia RBF kerel with the TWED distace, the Gaussia TWED kerel [50] is defied as! K twed ðx; yþ ¼exp d twedðx; yþ 2 : ð15þ r Kerel SRC based time series classificati Dete y as the test sample ad X =[X 1, X 2,..., X K ] as the traiig data matrix, where X i =[x i,1, x i,1,..., x i,i ], i =1,2,..., K,is the sample matrix f class C i, i stads fr the umber f traiig samples i class C i, ad x i,j detes the jth traiig time series f class C i. The kerel fucti is defied as K(x, y)=hu(x), U(y)i, where U(x) stads fr the crrespdig liear mappig fucti f x. I kerel SRC, the dictiary is defied as U =[U 1, U 2,..., U K ], where U i =[U(x i,1 ), U(x i,2 ),..., U(x i,i )]. Give a test sample y, the fllwig kerel sparse represetati mdel fr time series classificati was used: ð^a Þ ¼ arg mi k a UðyÞ Uak 2 2 ; s:t: kak 0 6 T 0 ; ð16þ where ^a ¼ ½^a 1 ; ^a 2 ;...; ^a K Š T is the ptimal sluti. After btaiig ^a, SRC assigs the class label f y by usig the fllwig rule LabelðyÞ ¼arg mi i kuðyþ U i^a i k 2 : ð17þ By itrducig the kerel matrix K = U T U ad kerel vectr k = U T U(y), the kerel rthgal matchig pursuit (OMP) algrithm [36] was mdified t slve the kerel sparse represetati mdel i Eq. (16). Let ^a s be the curret estimate f ^a, ad I s be the set f idices f selected atms. The residue r s is defied as r s ¼ UðyÞ U^a s : ð18þ The first step f kerel OMP is the prjecti f the residual t each f the remaiig atms, s i ¼ hr s ; Uðx i Þi ¼ Kðy; x i Þ X Kðx j ; x i Þa j ; i R I s : ð19þ j2i s Let i max ¼ arg max js i j: ð20þ The kerel OMP simply updates the set f idices I s+1 = I s [ i max, ad updates ^a sþ1 by ^a sþ1 ¼ K 1 sþ1 k sþ1; where K s+1 is the sub-matrix f K based the idex set I s+1, ad k s+1 is the sub-vectr f k based the idex set I s+1. Fially, the kerel OMP algrithm fr kerel sparse represetati is summarized i Algrithm 1. Give e test sample fr the first class i the Face (Fur) dataset, by applyig the Gaussia ERP kerel, Fig. 1 shws the cdig cefficiets f kerel SRC 24 traiig samples ad the distace t each class. It ca be see that the cdig cefficiets crrespdig t the first class are relatively sigificat, ad the distace t the first class is , which is much ð21þ Fig. 1. Kerel sparse represetati f time series.

6 20 Z. Che et al. / Ifrmati Scieces 292 (2015) smaller tha thse t the ther classes. Frm Fig. 1, mst cefficiets f the ther classes are zers (cmpared t the first class, cefficiets frm ther classes are much smaller), which idicates the effectiveess f kerel SRC. Algrithm 1. Kerel OMP fr kerel sparse represetati Iput: time series y, traiig set X, T 0 Output: ^a 1. Iitialize s =0,I 0 = ;, ^a 0 ¼ 0 2. While s < T 0 3. s i ¼ Kðy; x i Þ P Kðx j2is j; x i Þa j ; i R I s 4. i max = arg maxjs i j 5. Update the idex set: I s+1 = I s [ i max 6. ^a sþ1 ¼ K 1 sþ1 k sþ1 7. s = s Ed while 9. ^aði s ðjþþ ¼ ^a s ðjþ fr "j 2 I s, ad therwise zer Discussi Because f time distrti, cvetial sparse represetati perfrmed i the Euclidea space cat wrk well fr time series classificati. Csiderig the success f elastic matchig methds such as DTW, ERP, ad TWED, a class f Gaussia elastic matchig kerel was itrduced. Based the Gaussia elastic matchig kerel, a kerel SRC mdel tgether with a kerel OMP algrithm was prpsed. By substitutig the Euclidea distace with the elastic distace measures, the Gaussia elastic matchig kerel utilized kerel SRC t suppress the adverse ifluece f time drift. It shuld be ted that the Gaussia elastic matchig kerel cat be guarateed t be a psitive defiite symmetric (PDS) kerel. Empirical studies by Lei ad Su shwed that the Gaussia DTW kerel is t PDS acceptable t supprt vectr machie [26]. Experimetal results shwed that i sme cases SVM with the Gaussia DTW kerel eve perfrmed prer tha SVM with the Gaussia RBF kerel r NNC with DTW. Frtuately, as shw i ur previus studies [50], the Gaussia elastic matchig kerel based elastic metric (ERP r TWED) geerally satisfied the PDS prperty. Mrever, eve the Gaussia elastic matchig kerel is t PDS, the prpsed mdificati methds [8] ca make the -PDS kerel acceptable t kerel SRC. I practice, first whether K r K s are PDS is checked. If K r K s is t PDS, the -PDS K r K s is replaced with the prper PDS matrices by usig the spectrum clip methd [8]. 4. Usupervised ad supervised dictiary learig f time series Dictiary learig makes the SRC mdel i Eq. (16) applicable t a large scale traiig set, ad ehaces the discrimiati f the dictiary. I this secti, first a kerel KSVD with Gaussia elastic matchig kerel fr usupervised dictiary learig is itrduced, ad the a kerel LC-KSVD methd fr supervised dictiary learig is prpsed Kerel KSVD fr usupervised dictiary learig I the sparse represetati mdel i Eq. (16), the dictiary is simply the whle set f traiig samples. I this subsecti, the kerel KSVD algrithm [35] is used t lear a mre represetative dictiary. Give the traiig set X f samples, the gal f kerel KSVD is t lear a dictiary U(D) fm atms by slvig the fllwig ptimal prblem, ðd; AÞ ¼ arg mi D;A kuðxþ UðDÞA ; s:t: ka i k 0 6 T 0 ; ð22þ k 2 F where a i is the ith clum f matrix A. T make the mdel easy t slve, it is assumed that the dictiary atm is represeted by a liear cmbiati f the traiig samples such as U(D) =U(X)B, where B is the m atm represetati dictiary. Thus, the kerel KSVD mdel is refrmulated as ðb; AÞ ¼ arg mi D;A kuðxþ UðXÞBA ; s:t: ka i k 0 6 T 0 : ð23þ k 2 F The prpsed kerel dictiary learig methd [35] is used t lear the dictiary by iteratig betwee the updatig f the cdig cefficiets A ad the updatig f the dictiary B Updatig A via kerel OMP Give the atm represetati dictiary B,A is updated by slvig idepedet sparse cdig prblems, ða i Þ ¼ arg mi D;A kuðx i Þ UðXÞBa i k 2 2 ; s:t: ka i k 0 6 T 0 : ð24þ

7 Z. Che et al. / Ifrmati Scieces 292 (2015) Let ^a s be the curret estimate f ^a ad I s be the set f idices f selected atms. The residue r s is defied as r s ¼ UðyÞ UB^a s : The first step f KOMP is the prjecti f the residual t each f the remaiig atms, ð25þ Let s i ¼ hr s ; UðXÞb i i ¼ k T b i a T S BT Kb i ; i R I s : ð26þ i max ¼ arg max js i j: ð27þ The kerel OMP simply updates the set f idices I s+1 = I s [ i max, ad cstructs the sub-matrix K s+1.ifk s+1 is t semi-psitive defiite, a semi-psitive defiite apprximati is used by slvig the fllwig prblem, ek sþ1 ¼ arg mi K0 kk K sþ1 k 2 F : ð28þ Accrdig t Che et al. [8], e K sþ1 ca be easily btaied usig the spectrum clippig peratr. Based e K sþ1 ; ^a sþ1 is updated by, ^a sþ1 ¼ e K 1 sþ1 k sþ1; where k s+1 is the sub-vectr f k based the idex set I s+1. This prcedure is repeated util T 0 atms are selected. ð29þ Dictiary updatig I the dictiary update phase, first the spectrum clippig peratr K is used t btai a PDS apprximati e K. Detig b k by the kth clum f B ad a j the jth rw f A, the errr matrix E k is cmputed as E k ¼ I X b j a j!: ð30þ j k Let x k be the grup f examples that use the kth atm ad X k be a jx k j matrix with X k (x k (i), i) = 1 ad zer elsewhere. The clum reduced errr matrix is the btaied by E R k ¼ E kx k. By applyig SVD t ðe R k ÞT KE e R T E R e k KE R k ¼ VKV T ; a k is updated by a k ¼ ðk 1 Þ 1=2 E R kv 1 ; ð32þ where k 1 is the first eigevalue ad v 1 is the first eigevectr. This prcedure is repeated util all the m atms are updated Kerel label csistet K-SVD fr supervised dictiary learig T further efrce the cmpactess ad discrimiati f the dictiary, a kerel label csistet KSVD (LC-KSVD) algrithm is prpsed. Usig the Gaussia elastic matchig kerel, the lss fucti f kerel LC-KSVD is defied as < D; W; U; A >¼ arg mi kuðxþ UðDÞA D;W;U;A k 2 2 þ a k Q UA k2 2 k, ð31þ þ b k H WA k2 2 ; k a ik 0 6 T 0 ; ð33þ The first term is the stadard dictiary learig mdel. I the secd term, Q =[q 1, q 2,..., q ], is the umber f traiig samples i X, ad q i is a discrimiative sparse cde crrespdig t the ith traiig sample. If the traiig sample x i shares the same label with dictiary atm U(d j ), the jth elemet f q i will be e ad therwise zer. I the third item, H =[h 1, h 2,..., h ], where h i is a label vectr crrespdig t x i.ifx i belgs t the kth class, the kth elemet f h i will be e ad therwise zer. Als, a ad b are tw -egative parameters, ad U ad W are tw trasfrmati matrices t be leared. The kerel LC-KSVD mdel ca be equivaletly frmulated as UðXÞ UðDÞ B C B C < D; W; U; A >¼ arg a Q a U AA D;W;U;A p ffiffiffi ; k a i k 0 6 T 0 : ð34þ b H b W By itrducig a implicit mappig x i Uðx i Þ B C B C W@ a qi A a qi A; ð35þ b hi b hi

8 22 Z. Che et al. / Ifrmati Scieces 292 (2015) the crrespdig kerel fucti is defied as x i x j * x K 0 B C i x j + B a qi a qj CC B C AA ¼ W@ a qi A; W a qj A ¼ Kðx i ; x j Þþa q i ; q j þ b hi ; h j : ð36þ b hi b hj b hi b hj Let z i ¼½x T i ; a q T T b h i ŠT. The kerel LC-KSVD mdel ca the be frmulated as, i ; < B; A >¼ arg miku 0 ðzþ U 0 ðzþbak 2 F s:t: ka i k 0 6 T; 8i; ð37þ B;A which ca be slved by usig the kerel KSVD algrithm itrduced i Secti 4.1. Oce B is btaied, U ad W ca be acquired via U = QB,W = HB, respectively. I the classificati stage, give the test sample y, first the sparse represetati prblem is slved by ð^a Þ ¼ arg mi a kuðyþ UðXÞBak 2 2 ; s:t: kak 0 6 T 0 ; ð38þ ad the y is classified based the fllwig rule: j ¼ arg max j fl ¼ W ^a g; ð39þ where l is the K 1 class label vectr. 5. Experimetal results I this secti, a series f experimets were cducted t assess the prpsed methds usig the UCR time series datasets [22,23] frm tw aspects: classificati errr rate ad cmputatial cst. Sixtee data sets were used i the experimets, fur f which were tw-class tasks ad the rest were multi-class prblems. Each dataset csists f a traiig subset ad a test subset. Table 1 prvides a brief summary f the datasets Classificati results ad aalysis Usig the classificati errr rate as the perfrmace idicatr, the prpsed methds were cmpared with the state-fthe-art algrithms based the earest eighbr classifier, icludig NNC with Euclidea (1NN-ED), NNC with DTW (1NN- DTW), NNC with ERP (1NN-ERP), ad NNC with TWED (1NN-TWED). The prpsed methds were gruped it three categries. The first e was based the kerel SRC mdel where the dictiary is the etire set f the traiig samples, ad three kerel SRC methds, SRC with Gaussia DTW kerel (SRC-DTW), SRC with Gaussia ERP kerel (SRC-ERP), ad SRC with TWED kerel (SRC-TWED) were evaluated. The secd categry was based kerel KSVD, where the dictiary is leared usig the kerel KSVD algrithm i the usupervised way, ad three KSVD appraches, kerel KSVD with Gaussia DTW kerel (KSVD-DTW), kerel KSVD with Gaussia ERP kerel (KSVD-ERP), ad kerel KSVD with Gaussia TWED kerel (KSVD-TWED) were evaluated. The third categry was based the kerel label csistet KSVD algrithm, where the dictiary is leared i the supervised maer, ad three kerel LC-KSVD appraches, LC-KSVD with Gaussia DTW kerel (LC- KSVD-DTW), LC-KSVD with Gaussia ERP kerel (LC-KSVD-ERP), ad LC-KSVD with Gaussia TWED kerel (LC-KSVD-TWED) Table 1 Attributes f the UCR time series datasets. Datasets Class Legth Istaces Traiig Test Adiac Beef CBF Cffee ECG Face(All) Face(Fur) FISH Gu-pit Lightig Lightig Olive il Swedish leaf Sythetic ctrl Trace Tw patters

9 Z. Che et al. / Ifrmati Scieces 292 (2015) Table 2 Classificati errr rates btaied by the cvetial 1NN classifier with differet distace measures ad the sparse represetati methds with differet Gaussia elastic kerels. 1NN- ED 1NN- DTW 1NN- ERP 1NN- TWED SRC- DTW SRC- ERP SRC- TWED KSVD- DTW KSVD- ERP KSVD- TWED LC-KSVD -DTW LC- KSVD - ERP Adiac Beef CBF Cffee ECG Face(All) Face(Fur) FISH Gu-pit Lightig Lightig Olive il Swedish leaf Sythetic_ctrl Trace Tw patters LC- KSVD- TWED Fig. 2. Cmparis betwee the kerel KSVD mdel ad the kerel SRC mdel. I each sub-graph, the x axis stads fr the errr rates geerated by the kerel KSVD mdel with a certai Gaussia kerel (DTW, ERP, TWED respectively), ad the y axis represets the errr rates frmed by the kerel SRC mdel with the same kerel shw the xaxis. The straight lie has a slpe f 1.0, s a dt the lie meas the idetical errr rate calculated by the tw methds the same data set, a dt abve (r belw) the lie meas the KSVD mdel perfrms better (r weaker) tha the SRC mdel that data set. Fig. 3. Cmparis betwee the kerel LC-KSVD mdel ad the kerel SRC mdel. I each sub-graph, the x axis stads fr the errr rates geerated by the kerel LC-KSVD mdel with a certai Gaussia kerel (DTW, ERP, TWED respectively), the y axis represets the errr rates frmed by the kerel SRC mdel with the same kerel shw the xaxis. The straight lie has a slpe f 1.0, s a dt the lie meas the idetical errr rate calculated by the tw methds the same data set, a dt abve (r belw) the lie meas the LC-KSVD mdel perfrms better (r weaker) tha the SRC mdel that data set. were evaluated. The cdes f the prpsed methds are available at: Represetati/tree/master/KerelKSVD%20fr%20time%20series. Table 2 lists the classificati errr rates f all the methds these 16 data sets. Geerally, the prpsed sparse represetati based classificati methds were superir t the state-f-the-art earest eighbr classifiers. Fr example, SRC-ERP ca achieve lwer errr rates tha 1NN-ERP 14 data sets. Oly the CBF dataset, 1NN-ERP was lwer tha SRC-ERP. Fr all the 16 data sets, KSVD-TWED always btaied a lwer r equal errr rate tha 1NN-TWED. It is

10 24 Z. Che et al. / Ifrmati Scieces 292 (2015) Fig. 4. Ruig time f kerel SRC, kerel KSVD, ad kerel LC-KSVD the 16 data sets (the Gaussia TWED kerel is used as a example). iterestig t pit ut that cmpared with ERP ad TWED, the imprvemet f SRC-DTW agaist 1NN-DTW is relatively weak, which may be explaied i that ERP ad TWED are distace metrics while DTW is t. I additi, based the experimetal results, kerel LC-KSVD ad kerel KSVD ca t sme extet achieve better perfrmace tha kerel SRC. Fr example, LC-KSVD-ERP ad KSVD-ERP achieved lwer classificati accuracy tha SRC-ERP ly 3 ad 4 data sets, respectively. KSVD-TWED shwed weakess ly Trace by 0.01 lwer tha SRC-TWED, but the results geerated by LC-KSVD-TWED ad SRC-TWED were rughly the same. I rder t give a clearer cmparis, the perfrmaces f these 3 categries f methds were aalyzed. Frm Fig. 2, it ca be see that KSVD-DTW perfrms a little better tha SRC-DTW. KSVD-ERP is mre effectively tha SRC- ERP. The KSVD-TWED utperfrms SRC-TWED because there is hardly ay dt belw the lie. Frm Fig. 3, fr the Gaussia DTW kerel ad the Gaussia ERP kerel, the kerel LC-KSVD mdel is superir t the kerel SRC mdel. Hwever, LC-KSVD- TWED ad SRC-TWED (the right sub-graph) preset similar perfrmace sice mst f the dts i this sub-graph are clse t the straight lie Ruig time I this secti, usig the Gaussia TWED kerel, the ruig time f kerel SRC, kerel KSVD, ad kerel LC-KSVD were cmpared. Fig. 4 shws the ruig time f these three methds the 16 data sets. I the classificati stage, the prcedures f kerel SRC ad kerel KSVD are similar. If the size f the dictiary f kerel SRC is the same as that f kerel KSVD, the cmputatial cst f these tw methds wuld be rughly the same. I the traiig stage, the umber f atms was set be the same as the umber f traiig samples. Frm Fig. 4, it ca be see that the differece f ruig time f kerel SRC ad kerel KSVD is isigificat. I kerel LC-KSVD, the umber f atms was set much lwer tha the size f the traiig set, ad the classificati rule was simpler tha thse f kerel SRC ad kerel KSVD. Thus, the ruig time f kerel LC- KSVD was much less tha the ther methds. Takig bth the classificati accuracy ad ruig time it accut, kerel LC-KSVD is a suitable chice fr time series classificati.

11 Z. Che et al. / Ifrmati Scieces 292 (2015) Cclusi I this paper, the applicatis f kerel sparse represetati based classifiers fr time series classificati were studied. The itrducti f a class f Gaussia elastic matchig kerels, the Gaussia DTW kerel, the Gaussia ERP kerel, ad the Gaussia TWED kerel, makes it pssible t utilize SRC while suppressig the ifluece f time drift. Bth the kerel sparse represetati ad dictiary learig methds were ivestigated, ad three kerel sparse represetati based classifiers, icludig kerel SRC, kerel KSVD, ad kerel LC-KSVD were prpsed. Experimetal results the UCR time series datasets shwed that the prpsed methds ca achieve much lwer errr rates tha the state-f-the-art earest eighbr classifiers, 1NN-DTW, 1NN-ERP, ad 1NN-TWED. Mrever, the ruig time f kerel LC-KSVD was much less tha kerel SRC ad kerel KSVD. I the future, we will study the cstructi f elastic PDS kerels ad will develp mre apprpriate discrimiative dictiary learig algrithms fr time series classificati. Ackwledgemet This wrk was supprted by Natial Prgram Key Basic Research Prject (973 Prgram) uder Grat 2013CB329304, Natial Natural Sciece Fudati f Chia (NSFC) uder Grats , , , , ad New Cetury Excellet Talets i Uiversity uder Grat NCET ad NCET Thaks t Dr. Edward C. Migt, Shadg Uiversity, fr liguistic advice. Refereces [1] M. Ahar, M. Elad, A. Bruckstei, K-SVD: a algrithm fr desigig f vercmplete dictiaries fr sparse represetati, IEEE Tras. Sigal Prcess. 54 (11) (2006) [2] F.I. Bashir, A.A. Khkhar, D. Schfeld, Object trajectry-based activity classificati ad recgiti usig hidde Markv mdels, IEEE Tras. Image Prcess. 16 (7) (2007) [3] D.J. Berdt, J. Cliffrd, Usig dyamic time warpig t fid patters i time series, AAAI-94 Wrkshp Kwledge Discvery i Databases, 1994, pp [4] E. Cadès, J. Rmberg, T. Ta, Stable sigal recvery frm icmplete ad iaccurate measuremets, Cmmu. Pure Appl. Math. 59 (8) (2006) [5] E. Cadès, T. Ta, Decdig by liear prgrammig, IEEE Tras. Ifrm. Thery 51 (12) (2005) [6] L. Che, R. Ng, O the marriage f Lp-rms ad edit distace, i: Prceedigs f the Thirtieth Iteratial Cferece Very Large Data Bases (VLDB 04), 2004, pp [7] L. Che, M.T. Özsu,V. Oria, Rbust ad fast similarity search fr mvig bject trajectries, i: Prceedigs f the 2005 ACM SIGMOD Iteratial Cferece Maagemet f Data (SIGMOD 05), 2005, pp [8] Y.H. Che, E.K. Garcia, M.R. Gupta, A. Rahimi, L. Cazzati, Similarity-based classificati: ccepts ad algrithms, J. Machie Lear. Res. 10 (2009) [9] H. Deg, G. Ruger, E. Tuv, M. Vladimir, A time series frest fr classificati ad feature extracti, Ifrm. Sci. 239 (2013) [10] D.L. Dh, Cmpressed sesig, IEEE Tras. Ifrm. Thery 52 (4) (2006) [11] R.O. Duta, P.E. Hart, D.G. Strk, Patter Classificati, secd ed., Jh Wiley & Ss, New Yrk, [12] K. Ega, S.O. Aase, J.H. Husøy, Multi-frame cmpressi: thery ad desig, EURASIP Sigal Prcess. 80 (10) (2000) [13] T.C. Fu, C.W. Law, K.K. Cha, F.L. Chug, C.M. Ng, Stck time series categrizati ad clusterig via SB-tree ptimizati, i: Prceedigs f the Third Iteratial Cferece Fuzzy Systems ad Kwledge Discvery (FSKD 06), 2006, pp [14] S. Ga, I.W.-H. Tsag, Y. Ma, Learig categry-specific dictiary ad shared dictiary fr fie-graied image categrizati, IEEE Tras. Image Prcess. 23 (2) (2014) [15] S. Ga, I.W. Tsag, L.T. Chia, Sparse represetati with kerels, IEEE Tras. Image Prcess. 22 (2) (2013) [16] S. Gudmudss, T.P. Ruarss, S. Sigurdss, Supprt vectr machies ad dyamic time warpig fr time series, i: IEEE Iteratial Jit Cferece Neural Netwrks (IJCNN 08), 2008, pp [17] A. Jalalia, S.K. Chalup, GDTW-P-SVMs: variable-legth time series aalysis usig supprt vectr machies, Neurcmputig 99 (1) (2013) [18] K. Jia, T.H. Cha, Y. Ma, Rbust ad practical face recgiti via structured sparsity, i: Prceedigs f the 12th Eurpea Cferece Cmputer Visi (ECCV 12), 2012, pp [19] Z.L. Jiag, Z. Li, L.S. Davis, Learig a discrimiative dictiary fr sparse cdig via label csistet K-SVD, i: 2011 IEEE Cferece Cmputer Visi ad Patter Recgiti (CVPR 11), 2011, pp [20] Z. Jiag, Z. Li, L.S. Davis, Label csistet K-SVD: learig adiscrimiative dictiary fr recgiti, IEEE Tras. Patter Aal. Machie Itell. (2013). [21] E. Kegh, C.A. Rataamahataa, Exact idexig f dyamic time warpig, Kwl. Ifrm. Syst. 7 (3) (2005) [22] E. Kegh, S. Kasetty, O the eed fr time series data miig bechmarks: a survey ad empirical demstrati, Data Miig Kwl. Discvery 7 (4) (2003) [23] E. Kegh, Q. Zhu, B. Hu, Y. Ha, X. Xi, L. Wei, C.A. Rataamahataa, The UCR Time Series Classificati/Clusterig, < time_series_data/>. [24] M. Krawczak, G. Szkatuła, A apprach t dimesiality reducti i time series, Ifrm. Sci. 260 (2014) [25] K. Kreutz-Delgad, J.F. Murray, B.D. Ra, K. Ega, T.W. Lee, T.J. Sejwski, Dictiary learig algrithms fr sparse represetati, Neural Cmput. 15 (2) (2003) [26] H.S. Lei, B.Y. Su, A study the dyamic time warpig i kerel machies, i: Third Iteratial IEEE Cferece Sigal-Image Techlgies ad Iteret-Based System (SITIS 07), 2007, pp [27] J.F. Lichteauer, E.A. Hedriks, M.J.T. Reiders, Sig laguage recgiti by cmbiig statistical DTW ad idepedet classificati, IEEE Tras. Patter Aal. Machie Itell. 30 (11) (2008) [28] L. Li, X. Liu, S.-C. Zhu, Layered graph matchig with cmpsite cluster samplig, IEEE Tras. Patter Aal. Machie Itell. 32 (8) (2010) [29] H. Liu, Y. Liu, F. Su, Traffic sig recgiti usig grup sparse cdig, Ifrm. Sci. 266 (2014) [30] E.A. Maharaj, P. D Urs, Fuzzy clusterig f time series i the frequecy dmai, Ifrm. Sci. 181 (7) (2011) [31] J. Mairal, F. Bach, J. Pce, Task-drive dictiary learig, IEEE Tras. Patter Aal. Machie Itell. 34 (4) (2012) [32] J. Mairal, F. Bach, J. Pce, G. Sapir, A. Zisserma, Supervised dictiary learig, Adv. Neural Ifrm. Prcess. Syst. (2008). [33] V. Mäkie, Usig edit distace i pit-patter matchig, i: Prc. 8th Iteratial Sympsium Strig Prcessig ad Ifrmati Retrieval (SPIRE 01), 2001, pp

12 26 Z. Che et al. / Ifrmati Scieces 292 (2015) [34] P.F. Marteau, Time warp edit distace with stiffess adjustmet fr time series matchig, IEEE Tras. Patter Aal. Machie Itell. 31 (2) (2009) [35] H.V. Nguye, V.M. Patel, N.M. Nasrabadi, R. Chellappa, Kerel dictiary learig, i: 2012 IEEE Iteratial Cferece Acustics, Speech ad Sigal Prcessig (ICASSP 12), 2012, pp [36] H. Va Nguye, V.M. Patel, N.M. Nasrabadi, R. Chellappa, Desig f -liear kerel dictiaries fr bject recgiti, IEEE Tras. Image Prcess. 22 (12) (2013) [37] K. Npe, J. Krtelaie, T. Seppäe, Ivariat trajectry classificati f dyamical systems with a case study ECG, Patter Recg. 42 (9) (2009) [38] H. Pree, B. Herwig, T. Gruber, B. Sick, K. David, P. Lukwicz, O geeral purpse time series similarity measures ad their use as kerel fuctis i supprt vectr machies, Ifrm. Sci. 281 (2014) [39] M. Pulid, P. Meli, O. Castill, Particle swarm ptimizati f esemble eural etwrks with fuzzy aggregati fr time series predicti f the Mexica Stck Exchage, Ifrm. Sci. 280 (2014) [40] M. Vlachs, M. Hadjieleftheriu, D. Gupuls, E. Kegh, Idexig multi-dimesial time-series with supprt fr multiple distace measures, i: Prceedigs f the Nith ACM SIGKDD Iteratial Cferece Kwledge Discvery ad Data Miig (KDD 03), 2003, pp [41] X.Y. Wag, A. Muee, H. Dig, G. Trajcevski, P. Scheuerma, E. Kegh, Experimetal cmparis f represetati methds ad distace measures fr time series data, Data Miig Kwl. Discvery 26 (2013) [42] J. Wright, A. Yag, A. Gaesh, S. Sastry, Y. Ma, Rbust face recgiti via sparse represetati, IEEE Tras. Patter Aal. Machie Itell. 31 (2) (2009) [43] Y. Xu, Q. Zhu, Z. Fa, D. Zhag, J. Mi, Z. Lai, Usig the idea f the sparse represetati t perfrm carse-t-fie face recgiti, Ifrm. Sci. 238 (2013) [44] A.Y. Yag, Z.H. Zhu, A.G. Balasubramaia, S. Sastry, Y. Ma, Fast l1-miimizati algrithms fr rbust face recgiti, IEEE Tras. Image Prcess. 22 (8) (2013) [45] M. Yag, L. Zhag, X. Feg, D. Zhag, Fisher discrimiati dictiary learig fr sparse represetati, i: 2011 IEEE Iteratial Cferece Cmputer Visi (ICCV 11), 2011, pp [46] S. Yag, Y. Lv, Y. Re, L. Yag, L. Jia, Usupervised images segmetati via icremetal dictiary learig based sparse represetati, Ifrm. Sci. 269 (2014) [47] J. Yi, Z.H. Liu, Z. Ji, W.K. Yag, Kerel sparse represetati based classificati, Neurcmputig 77 (1) (2012) [48] D. Yu, X. Yu, Q. Hu, J. Liu, A. Wu, Dyamic time warpig cstrait learig fr large margi earest eighbr, Ifrm. Sci. 181 (13) (2011) [49] B. Zhag, F. Karray, Q. Li, L. Zhag, Sparse represetati classifier fr micraeurysm detecti ad retial bld vessel extracti, Ifrm. Sci. 200 (2012) [50] D. Zhag, W.M. Zu, D. Zhag, H.Z. Zhag, Time series classificati usig supprt vectr machie with Gaussia elastic metric kerel, i: Prceedigs f 20th Iteratial Cferece Patter Recgiti (ICPR 10), 2010, pp [51] H.C. Zhag, N.M. Nasrabadi, Y.N. Zhag, T.S. Huag, Jit dyamic sparse represetati fr multi-view face recgiti, Patter Recg. 45 (4) (2012) [52] L. Zhag, M. Yag, X. Feg, Sparse represetati r cllabrative represetati: which helps face recgiti, i: 2011 IEEE Iteratial Cferece Cmputer Visi (ICCV 11), 2011, pp [53] L. Zhag, W.D. Zhu, P.C. Chag, J. Liu, Z. Ya, T. Wag, F.Z. Li, Kerel sparse represetati-based classifier, IEEE Tras. Sigal Prcess. 60 (4) (2012) [54] Q. Zhag, B.X. Li, Discrimiative K-SVD fr dictiary learig i face recgiti, i: 2010 IEEE Cferece Cmputer Visi ad Patter Recgiti (CVPR 10), 2010, pp