Gene Expression Data Classification with Kernel independent Component Analysis

Transcription

1 Research Joural of Matheatcal ad Statstcal Sceces ISSN Gee Expresso Data Classfcato wth Kerel depedet Copoet Aalyss Abstract Abdallah Bashr Musa College of Matheatcs ad Coputer Scece, Hebe Uversty, Baodg 07002, CHINA Avalable ole at: Receved 22 d March 204, revsed 5 th Aprl 204, accepted 0 th May 204 he challege of classfyg the characterstcs of gee expresso data s that the sze of the trag data s sgfcatly lower tha the uber of features. Logstc regresso (LR) s stadard statstcal ethod that broadly used edcal, epdeology ad boforatcs coutes for classfcato task; however, such stuato of gee expresso data, LR does ot work effcetly due to ult- collearly ad over- fttg probles, therefore, odfyg of LR to aalyss the croarray data s requred. For solvg those probles, reducto deso s usually used. Recetly, kerel approaches have prove to be good for classfcato such type of data. Kerel depedet copoet aalyss (KICA) s the olear for of depedet copoet aalyss (ICA). I ths paper, LR s appled to classfy the features that selected by KICA. o evaluate the classfcato perforace of ths techque, ths ethod has copared to kerel prcple copoet aalyss (KPCA) ad depedet copoet aalyss (ICA). Nuerous perforace etrcs such as accuracy, sestvty, specfcty, precso, F-score, the area uder recever operatg characterstc curve (AUC) ad the recever operatg characterstc (ROC) aalyss are used. Keywords: Gee expresso data. Kerel prcpal copoet aalyss (KICA). Logstc regresso (LR). Kerel prcpal copoet aalyss (KPCA).. Itroducto Logstc Regresso (LR) -4 s cosdered as a stadard ultvarate statstcal classfcato techque that has bee used broadly a varety of applcatos cludg docuet classfcato 5 ad boforatcs 6-8. he advatage of usg LR s that t yelds a-posteror probabltes, I other words, besdes predctg class labels LR provdes a probablstc terpretato about ths labelg. LR has a addtoal advatage that the exteso to the ult-class case s well descred. he fast creasg of croarray techology has geerated a rche ad huge of gee expresso data sets. Usually, gee expresso data sets clude large uber of features wth sall uber of data sze ad also t cludes a hgh level of oses. Statstcally, t s kow that applyg of all types of regresso cludg logstc regresso requres the data sze to be relatvely bgger tha the the uber of features; ths akes the classcal LR classfcato ethod vald to aalyze these types of data. herefore, costructg a logstc regresso odel usg gee expresso data s very dffcult ad cosdered as a serous techcal challege, whch has attracted researchers atteto. Although gee expresso data has bg uber of features, t has bee observed that a few of the ca be explag ost of data varato, so these features ca be extracted to buld relable classfer. Accordgly, for aalyzg gee expresso data, tally we eed to select or extract oly these few relevat gees (features) fro the orgal data. Istead of classfyg the whole expresso data, feature selecto ethods a to select the ost essetal features wth low desos fro the orgal data ad the the classfcato ethod ca be appled o these selected features. he prevous studes show that the choce of gee selecto ethod has uch effect o the perforace of the classfcato ethods, ad thus the classfcato ethods should be cosdered together wth the gee selecto crtera 9. Feature selecto represets a essetal part classfcato tasks. Geerally, the a a of feature selecto ethods s to decrease the coputatoal coplexty ad produces a few eagful features that clude the axu forato the data, whch ca be used as a put features to a classfer to ga axu accuracy that ca be obtaed. he portat advatage of features selecto ethods that t selects few ucorrelated or depedet features (gee) cosequetly solves the probles of over fttg ad co-learty sultaeously, whch result provg the accuracy of classfcato whe applyg logstc regresso to those selected features. Several ache learg techques have bee used to classfy the gee expresso data, such as support vector ache (SVM); LORIS NANNI 0 who appled support vector ache (SVM) wth dfferet feature reducto ethods, ISABELLE GUYON who appled the SVM ethod of recursve feature elato (RFE)to gee selecto, Mchael P. S. Brow 2 who used SVMs to classfy gees based o gee expresso, Iteratoal Scece Cogress Assocato

2 errece S 3 who developed a ethod for aalyzg croarray data usg SVM, Hao Hele Zhag 4 who appled regularzato ethod wth support vector aches (SVMs) to classfy cacer data, NIR FRIEDMAN 5 who aalyss gee expresso data based o Bayesa etworks, Perre Bald 6 who develop a Bayesa probablstc fraework for croarray data aalyss ad Kyoughwa Bae 7 who proposed a Gee selecto ethod by usg a two-level herarchcal Bayesa odel, Kyeog Eu Lee 8 who used Bayesa xture pror to perfor the gee expresso data varable selecto.although SVM usually acheves low test error despte sall saple szes ad t gves good perforace regardg classfyg of gee expresso data 9, t cosdered to be a black box predctor, t ether akes t predcto plct or gves cte the rule goverg ts predcto whch s ot the case LR 20. Regard to logstc regresso, t has bee successfully exteded to classfy croarray data; J.G. Lao (2007) 2 develops a paraetrc bootstrap ethod for buldg a logstc regresso odel for dsease classfcato usg croarray data, Fort ad Labert-Lacrox 22 eployed partal least squares ad logstc regresso for classfyg croarray data, She L. a EC 23 they used pealzed logstc regresso for cacer classfcato usg croarray expresso data, JI ZHU 24 proposed pealzed logstc regresso (PLR) as a for the croarray cacer dagoss proble, Mauree A 25 who preseted a logstc regresso approach for detfyg erched bologcal groups gee expresso data ad Dahv. Nguye 26 appled PCA partal least squares (PLS) wth LR for tuor classfcato usg croarray gee expresso data. Fro features selecto/extracto pot of vew, popular features secto ethods such as prcple copoets aalyss (PCA) ad depedet copoet aalyss have bee used for classfcato gee expresso data; K.Y.Yeug ad W.L.Ruzzo 27 who used PCA for clusterg gee expresso data ad De- Shuag Huag 28 appled Idepedet copoet aalyss-based pealzed dscrat ethod for tuor classfcato usg gee expresso data. Because applyg of lear features selecto ethods ay gve accurate results for soe real- world data sets, recetly, kerel features selecto ethods have bee developed ad used, such as kerel prcpal copoet aalyss; Zhequ Lu, Dechag Che ad Hala Besal 29, they used kerel prcpal copoet aalyss(kpca) for deso reducto ad appled logstc regresso for classfcato gee expresso data ad recetly, Qgsog Gao 30 shows that KPCA wth LR s a vald ad powerful gee- or rego-based ethod for the aalyss of geoe-wde assocato studes( GWAS) data set. hs paper, proposes the applcato of kerel depedet copoet aalyss (KICA) 3 wth logstc regresso for classfyg gee expresso data. Coprehesve coparso experet betwee KICA wth LR, ICA ad KPCA, wth LR s perfored. Logstc regresso: Logstc Regresso (LR) -4, 32 s a ultvarate statstcal classfcato ethod cooly used for odelg dchotoous (bary) data. Let x R deote a vector of explaatory or feature varables, ad let y {, + } deote the assocated bary class label or outcoe. he logstc odel s descrbed as: exp(y( β x + α)) pr ( y / x) = = () + exp(-y( β x + α)) + exp(y( β x + α)) Where Pr(y/x) s the codtoal probablty of y gve x R. he logstc odel has paraeters α R represet the tercept te ad β represet the weght vector. β x + α = 0 R defes a hyperplae the feature space, o whch P(y/x)=0.5. he codtoal probablty Pr(y x) s larger tha 0.5 f otherwse. β x + α has the sae sg as y, ad less tha 0.5 Suppose we are gve a set of observed or trag data {, } =, where x R deote the -th saple ad x y y {, + } deote the correspodg class label. hese saples are assued to be depedet saples. Accordg to the logstc odel, the vector of the codtoal probabltes assocated of these saples s: exp y ( β x + α ) pr ( α, β ) = p ( y/ x ) = =,..., (2) + exp y ( β x + α ) he lkelhood fucto assocated wth the saples s = = pr ( α, β ), ad the log lkelhood fucto s: log pr ( α, β ) = f ( β a + α y ) Where a = (3) = x y R ad f s the logstc loss fucto that s: f ( z) = log( + exp( z) (4) Usg (4),(3) ca be wrtte as = log pr ( α, β ) = log( + exp ( β a + α y ))) = he egatve of the log lkelhood fucto s called the (eprcal) logstc loss, ad dvdg by we obta the average logstc loss, lavg ( α, β ) = log( + exp ( β a + α y ))) (6) = (5) Iteratoal Scece Cogress Assocato 2

3 he odel paraeters β ad α ca be detered by axu lkelhood estato fro the observed exaples, by solvg the covex optzato proble. ze l ( α, β (7) avg ) he proble (7) s called the logstc regresso proble (LRP). hs LRP s a sooth covex optzato proble, ad ca be solved by a wde varety of ethods, such as gradet descet, steepest descet, Newto, quas-newto, or cojugate-gradets (CG) ethods. I ths paper the Newto ethod s used. Oce we fd axu lkelhood values of α ad β, that s, a soluto of (7), we ca predct the probablty of the two possble outcoes. Gve a ew features vector x R, by usg the assocated logstc regresso odel, the logstc regresso classfer s fored as: φ ( x) = sg( β x + α ) Where + z > 0 sg( z ) = (8) z 0 Idepedet Copoet Aalyss (ICA): ICA s a relatvely ew statstcal ad coputatoal techque for data aalyss. ICA orgated fro the sgal-processg couty, where t was developed as a powerful procedure for bld source separato 33. he task of ICA s to fd represetato of o- Gaussa data so those copoets are statstcally depedet or as depedet as possble 34. he basc ICA odel for feature trasforato ca be descrbed as: s = u (9) t x t Where x t s p atrx represets the observed feature vectors, s t s p atrx represet the ew depedet estated vectors for classfcato purpose, u s called the de- xg atrx ad s used to fd a etrely ew coordate syste of statstcally depedet o-gaussa drectos, wth the frst IC drecto beg the ost o- Gaussa. he algorth works teratvely ad deteres the ost o-gaussa drecto frst. Based o ths drecto t fds the ext ost o-gaussa drecto whch s depedet fro the frst, etc. For p desoal data vectors t deteres up to p desoal depedet vectors, so t projects the feature vectors represetg the orgal data to depedet copoets, u ust be estated fro the data. Statstcal depedece has bee defed as the jo probablty desty fucto of the copoet s t s equal to the product of argal destes fuctos of the dvdual copoets. May algorths have bee developed for perforg ICA, the fxedpot algorth s popular aog the. Fxed pot fast ICA preseted by Hyvare ad Oja 36 s used ths paper. I fast ICA, prcple copoet aalyss (PCA) s used to perfor the whteg before estatg the depedet copoets vectors; the orgal put vectors wll be trasfored to a set of ew ucorrelated vectors wth zero eas ad uty varace. Usg PCA to get the whteg reduced the deso of x t ad cosequetly, reduced the uber of s t that wll be coputed. After the process of data whteg s fshed, the fxed potalgorth s perfored to estate the trasforato atrx ad depedet copoets. Mutual forato s descrbed as a easure of the depedece betwee rado varables. Mzg the utual forato betwee the copoets s equvalet to axzg ther egetropy. he egetropy the fast ICA ca be approxately expressed as follows: 2 J ( s ) [ E{ s ) E{ V (0) G t ) ( )}] t( ) Where G s practcally ay o-quadratc fucto, V s a Gaussa varable wth zero ea ad ut varace ad µ s - u desoal vector, coprsg oe of the rows of the atrx.here are ay fuctos ca be used as G 33. Substtutg equato (0) obtag the followg optzato proble: Maxze J G ( µ V 2 µ ) = [ E{ x ) E{ )}] () Subject to µ 2 (2) E{( x ) } = =,2,..., Oe ew depedet copoet ca be estated by solvg ths optzato proble through the Fast ICA algorth ad based o ths the whole reduced depedet copoets t atrx ca be estated. I ths paper, we used Fast ICA wth skew to obta the depedet copoets. Kerel Idepedet Copoet Aalyss (KICA): Kerel prcple copoet aalyss (KICA) 37 s the kerel verso the ICA. For a gve trag data x, suppose ths trag data s trasforg to ew feature space F through soe olear appg. Where, s a Mercer s kerel that allows the Φ ( x), F = Φ( x) k x, x ) = Φ ( x ) Φ ( x ) calculato of ( j j the dot product ths space wthout explctly kowg the olear appg. I ths olear space the ceterg ad whteg that have bee etoed the prevous secto of ICA s obtaed as follow: For the ceterg task the data where Φ x ) =, 2,..,k, should be trasfored to * Φ ( x ) = Φ( x ) E( Φ( x ) (3) * Where: For the whteg E ( Φ ( x )) = 0 ths space, the task here s to fd a trasforato atrx Q satsfy that the ( s * Iteratoal Scece Cogress Assocato 3

4 covarace atrx of the data s ut atrx * ( Φ ( x ) = Q( Φ ( ) x For arbtrary vector the KICA trasforato ca be obtaed as: where W * deotes the orthogoal trasforato atrx that ca Z x be obtaed as descrbed for ICA, whle Q s the atrx * * obtaed fro kerel ceterg ad whteg. Z = W QΦ( Z ) Dfferet kerel fuctos ca be used, but choosg a sutable kerel fucto for a certa applcato s pertet. Gaussa kerel s selected for ths paper. Methodology he data sets: he gee expresso data sets used ths study are the ost two faous gee expresso data; Colo tuor ad Leukea cacer data sets. able- gve a uercal suary of the data sets. able- Suary of the gee expresso data sets Data set Data sze Nuber of features Colo tuor Leukea cacer Experetal set up: Kerel Idepedet copoet aalyss (KICA) wth Gaussa kerel, kerel prcples copoets aalyss (KPCA) wth the Gaussa (RBF) kerel ad depedet copoet aalyss (ICA) are used as feature selecto ethods for desoalty reducto logstc regresso. he kerel ca algorth wth Gaussa kerel s used for applyg KICA; for a gve trag data x t returs a atrx W where the kerel depedet copoet s: s=w*x. he perforace of KICA s copared to the perforace of KPCA ad ICA. For all the ethods, the orgal deso s reduced to expla 85 percet for Colo tuor ad 75 percet leukea cacer for the total varato of the data. he Gaussa (RBF) s used for the applcato of KPCA. For applyg ICA, the reducto deso ad whteg s obtaed by usg the prcple copoet aalyss (PCA) ethod. he for obtag the depedet copoets fro these reduced whteg data, fast ICA algorth wth skew s used. Oce the feature KICA, KPCA ad ICA trasfored the orgal features spaces to ew lower deso space Logstc regresso wth Newto ethod wth 0 fold cross-valdato ethods s appled to classfy the features ths ew space. Results ad Dscusso For the applcato of KICA the kerel ca verso.2 3 avalable at s used. For the applcato of KPCA, the Statstcal Patter Recogto oolbox for MALAB (stprtool) verso For the applcato of ICA, the fast-ica software package, s used. he l_logreg A large logstc regresso proble package verso avalable at ( wth Newto ethod, uder atlab ( R2009a) terface s used for estato the logstc regresso odel paraeters. he ROCs are obtaed by usg spss.6.0 (SPSS Ic, Chcago, IL, USA). he classfcato results of LR applcato the features that selected by the kerel depedet copoet aalyss (KICA), kerel prcple copoet aalyss (KPCA) ad depedet copoet aalyss (ICA) are preseted table-2, table-3 ad table-4 respectvely. Each value these tables represets the average assocated wth correspodg etrc. he uber of copoets for the ew deso selected by those ethods for the two data sets s show table 2. he ROCs of the Colo tuor ad Leukea cacer for three ethods are show fgure- ad fgure-2 respectvely. he perforaces easures of three ethods for Colo tuor ad Leukea cacer s depcted fgure-3. Dscusso: Fro table- 2, table-3 ad table-4, t ca be observed that perforace easures values of KICA are greater tha correspodg easures values for KPCA ad ICA for the two data sets, also t s vvd that ICA perfors better tha KPCA o Leukea cacer data set. hs ca be cofred fro fgure-3, where the fgure shows that KICA has hgher bar for all perforace easures for the two data sets coparso to KPCA ad ICA. he ROCs curve for the two data sets also support ths pot. able-2 he results of the perforace easures for K ICA Data set he perforace easures Nuber of Accuracy Sestvty Specfcty F-score Precso AUC copoets Colo tuor Leukea cacer Iteratoal Scece Cogress Assocato 4

5 able-3 he results of the perforace easures for KPCA Data set he perforace easures Accuracy Sestvty Specfcty F-score Precso AUC Colo tuor Leukea cacer able-4 he results of the perforace easures for ICA Data set he perforace easures Accuracy Sestvty Specfcty F-score Precso AUC Colo tuor Leukea cacer Fgure- he ROCs for Colo Fgure-2 he ROCs for Leukea cacer Colo tuor Leukea cacer KICA KICA KPCA Fgure-3 he perforace easures of KICA, KPCA ad ICA for Colo ad Leukea cacer data sets Iteratoal Scece Cogress Assocato 5

6 Cocluso I ths paper the kerel depedet copoet aalyss (KICA) s used as a features selecto ethod for reducto deso of gee expresso data; Colo tuor ad Leukea cacer data sets. LR for classfcato s appled for the features that selected by KICA. KPCA ad ICA wth LR are used to assess the perforace of KICA wth LR. he coparso shows that KICA outperfored ICA ad KPCA. hat s why KICA ca be used effectvely as a features selecto ethod wth LR for classfyg the gee expresso data. Fro the sae classfcato pot of vew, ad fro the fact that aïve Bayse classfcato ethod requred the trag features to be depedet, future work ay apply KICA techque wth aïve Bayse. Refereces. Hoser D.W. ad Leeshow S., Appled logstc regresso, 2d ed. Wley seres probablty ad statstcs, Wley, Ic, New York, (2000) 2. Meard S., Appled logstc regresso aalyss, 2 d ed. Sage publcatos Ic, (2002) 3. Neter J., Kuter M.H., Nachtshe C.J. ad Wassera W., Appled lear statstcal odels, 4th ed. Irw, Chcago, (996) 4. Rya.P., Moder regresso ethods, 2d ed. Wley, New York, (2008) 5. Brzezsk J.R. ad Kafl G.J., Logstc regresso odelg for cotext-based classfcato. I: Proceedgs teth teratoal workshop o database ad expert systes applcatos, (999) do:0.09/dexa Lao J.G. ad Ch K.V., Logstc regresso for dsease classfcato usg croarray data: odel selecto a large p ad sall case, Boforatcs, 23(5), (2007) 7. Sartor M.A., Lekauf G.D., Medvedovc Lrpath M, A logstc regresso approach for detfyg erched bologcal groups gee expresso data, Boforatcs, 25(2), 2 27 (2008) 8. Asgary M.P., Jahaddeh S., Abdolalek P. ad Kazeejad A., Aalyss ad detfcato of b-tur types usg ultoal logstc regresso ad artfcal eural etwork, Boforatcs, 23(23), (2007) 9. Lee Jae Wo, et al., A extesve coparso of recet classfcato tools appled to croarray data, Coputatoal Statstcs ad Data Aalyss, 48(4), (2005) 0. Na Lors, Sheryl Braha, ad Alessadra Lu, Cobg ultple approaches for gee croarray classfcato, Boforatcs, 28(8), 5-57 (202). Guyo Isabelle, et al., Gee selecto for cacer classfcato usg support vector aches, Mache learg, 46(-3), (2002) 2. Brow Mchael P.S., et al., Kowledge-based aalyss of croarray gee expresso data by usg support vector aches, Proceedgs of the Natoal Acadey of Sceces, 97(), (2000) 3. Furey, errece S., et al., Support vector ache classfcato ad valdato of cacer tssue saples usg croarray expresso data, Boforatcs, 6(0), (2000) 4. Zhag, Hao Hele, et al., Gee selecto usg support vector aches wth o-covex pealty, Boforatcs, 22(), (2006) 5. Freda, Nr, et al., Usg Bayesa etworks to aalyze expresso data, Joural of coputatoal bology, 7(3-4), (2000) 6. Bald, Perre ad Athoy D. Log, A Bayesa fraework for the aalyss of croarray expresso data: regularzed t-test ad statstcal fereces of gee chages, Boforatcs, 7(6), (200) 7. Bae, Kyoughwa ad Ba K. Mallck, Gee selecto usg a two-level herarchcal Bayesa odel, Boforatcs 20(8), (2004) 8. Lee, Kyeog Eu, et al., Gee selecto: a Bayesa varable selecto approach, Boforatcs, 9(), (2003) 9. Bae, Kyoughwa ad Ba K. Mallck, Gee selecto usg a two-level herarchcal Bayesa odel, Boforatcs, 20(8), (2004) 20. Musa A.B., Coparatve study o classfcato perforace betwee support vector ache ad logstc regresso, It J Mach Lear Cyber, 4(), 3-24 (203) 2. Lao J.G. ad Khew-Voo Ch, Logstc regresso for dsease classfcato usg croarray data: odel selecto a large p ad sall case, Boforatcs, 23(5), (2007) 22. Fort, Gersede, ad Sophe Labert- Lacrox,Classfcato usg partal least squares wth pealzed logstc regresso, Boforatcs, 2(7), 04- (2005) 23. She, L, ad Eg Chog a, Deso reducto-based pealzed logstc regresso for cacer classfcato usg croarray data, IEEE/ACM rasactos o Coputatoal Bology ad Boforatcs (CBB), 2(2), (2005) 24. Zhu, J, ad revor Haste, Classfcato of gee croarrays by pealzed logstc regresso, Bostatstcs, 5(3), (2004) Iteratoal Scece Cogress Assocato 6

7 25. Sartor, Mauree A., George D. Lekauf ad Maro Medvedovc, LRpath: a logstc regresso approach for detfyg erched bologcal groups gee expresso data, Boforatcs, 25(2), 2-27 (2009) 26. Nguye, Dah V. ad Davd M. Rocke, O partal least squares deso reducto for croarray-based classfcato: a sulato study, Coputatoal statstcs ad data aalyss, 46(3), (2004) 27. Yeug, Ka Yee ad Walter L. Ruzzo, Prcpal copoet aalyss for clusterg gee expresso data, Boforatcs 7(9), (200) 28. Huag, De-Shuag ad Chu-Hou Zheg, Idepedet copoet aalyss-based pealzed dscrat ethod for tuor classfcato usg gee expresso data, Boforatcs, 22(5), (2006) 29. Lu, Zhequ, Dechag Che ad Hala Besal,Gee expresso data classfcato wth kerel prcpal copoet aalyss, BoMed Research Iteratoal, 2, (2005) 30. Gao, Qgsog, et al, Gee-or rego-based assocato study va kerel prcpal copoet aalyss, BMC geetcs, 2(), 75 (20) 3. Bach Fracs R. ad Mchael I. Jorda, Kerel depedet copoet aalyss, he Joural of Mache Learg Research, 3, -48(2003) 32. Musa, Abdallah Bashr, Logstc Regresso Classfcato for Ucerta Data, Research Joural of Matheatcal ad Statstcal Sceces -ISSN 2320: 6047 (204) 33. Hyva re A, Karhue J, Oja E, Idepedet copoet aalyss, Wley, New York, (200) 34. Hyvare A, Oja E, Idepedet copoet aalyss: algorths ad applcatos, Neural Network, 3, (2000) 35. Musa, Abdallah Bashr, A coparso of l-regularzo, PCA, KPCA ad ICA for desoalty reducto logstc regresso, Iteratoal Joural of Mache Learg ad Cyberetcs, -3 (203) 36. Hyvare A. ad Oja E., A fast fxed-pot algorth for depedet copoet aalyss, Neural Coput, 9(7), (997) 37. J, X, et al, Kerel depedet copoet aalyss for gee expresso data clusterg, Idepedet Copoet Aalyss ad Bld Sgal Separato, Sprger Berl Hedelberg, (2006) 38. va der Maate L, Statstcal patter recogto toolbox for Matlab (stprtool) verso 2., verso 0.7.2b, (200) 39. Gavert H., Hurr J., Sarela J. ad Hyvare A., Fast ICA formatlab 7.x ad 6.x, Verso 2.5, (2005) 40. Koh K., K S.J. ad Boyd S., l_logreg: A large-scale solver for l-regularzed logstc regresso probles Avalable at /l_logreg/, (2009) Iteratoal Scece Cogress Assocato 7