An Efficient Classification System for Medical Diagnosis using SVM

Transcription

1 A Efficit Classificatio Systm for Mical Diaosis usig SVM G. Ravi Kumar 1, Dr. G.A Ramachadra 2 ad K.Nagamai 3 1. Rsarch Scholar, Dpartmt of Computr Scic & Tchology, Sri Krishadvaraya Uivrsity, Aapur , Adhra Pradsh, Idia grkodaravi@gmail.com 2. Associat Profssor, Dpartmt of Computr Scic & Tchology, Sri Krishadvaraya Uivrsity, Aapur , Adhra Pradsh, Idia 3. Rsarch Scholar, Dpartmt of Computr Scic, Rayalasma Uivrsity, Kurool , Adhra Pradsh, Idia kachrlamai@gmail.com ABSTRACT Classificatio is a miig ad machi larig task aim at buildig a classifir usig som traiig istacs for prictig classs for w istacs.buildig ffctiv classificatio systms is o of th ctral tasks of miig. Support Vctor Machis (SVMs) ar amog th most popular ad succssful classificatio algorithms Th SVM approach to machi larig is kow to hav both thortical ad practical advatags. Th accuracy of a SVM modl is largly dpt o th slctio of th modl paramtrs such as C, Gamma ad P. Thr ar a umbr of paramtrs such as C, Dgr ad Gamma that apply to th SVM modl ad th slct krl fuctio. Krl tchiqus hav log b us i SVM to hadl liarly isparabl problms by trasformig to a high dimsioal spac. Slctig th optimal valus ca siificatly impact th accuracy of th modl. This papr aims to stablish a accurat SVM classificatio modl for Mical prictio, i ordr to mak full us of th ivaluabl iformatio i cliical, spcially which is usually ior by most of th xistig mthods wh thy aim for high prictio accuracis. This papr prsts a compariso amog th diffrt SVM krls with diffrt paramtrs o th thr mical sts. Th mpirical rsults dmostrat th ability to us mor graliz krl fuctios ad it gos to prov that th polyomial ad RBF krl s prformac is cosisttly improv with suitabl paramtrs lik dgr ad gamma. Exprimtal rsults show that krl slctio gratly improvs th quality of classificatio. Kywords: Data miig, SVM, krls, classificatio, Polyomial ad RBF I. INTRODUCTION Classificatio is o of th most importat tasks i miig[5]. Classificatio is a miig ad machi larig task aim at buildig a classifir usig som traiig istacs for prictig classs for w istacs. Classificatio is th procss of larig a fuctio or a modl from a st (traiig ) so that th fuctio ca b us to prict th classificatio of a ovl istac, whos classificatio is ukow. Classificatio modls ar frqutly rprst as ruls of this form: P c whr P is a pattr i th traiig (P forms th st of prictig attribut(s)) ad c is th class labl or targt attribut. Th classificatio of mical has bcom a icrasigly challgig problm, du to rct advacs i mical miig tchology. Classificatio of this trmous amout of is tim cosumig ad utilizs xcssiv computatioal ffort, which may ot b appropriat for may applicatios. Th Support Vctor Machi (SVM) is a statistical larig algorithm that classifis th sampls usig a subst of traiig sampls call support vctors[3]. It is actually bas o larig with krls som of which form th support vctors. A grat advatag of this tchiqu is that it ca us larg iput ad fatur sts. Thus, it is asily to tst th ifluc of th umbr of faturs o classificatio accuracy. Svral rct studis hav rport that SVM grally ar capabl of dlivrig highr prformac i trms of classificatio accuracy tha th othr classificatio algorithms. It has b show that SVM is cosisttly suprior to othr suprvis larig. SVMs hav b mploy i a wid rag of ral world problms such as txt pattr rcoitio, tst classificatio ad bioiformatics catgorizatio, had-writt digit rcoitio, to rcoitio, imag classificatio ad objct dtctio, micro-array g xprssio aalysis, classificatio. II.SUPPORT VECTOR MACHINE Th Support Vctor Machi (SVM) is a suprvis larig mthod for Data aalysis, Pattr rcoitio, Classificatio Pag 78

2 ad Rgrssio aalysis. It is a classificatio tchiqu bas o Th algorithm costructs a maximum margi hyprpla which statistical larig thory [2,3]. Th SVM is a promisig w mthod sparats a st of positiv xampls from a st of gativ xampls. for th classificatio of both liar ad oliar. SVMs follow I th cas of xampls ot liarly sparabl, SVM uss a krl th Structural Risk Miimizatio (SRM) pricipl that rsults i a fuctios to map th xampls from iput spac ito high classifir with th last xpct risk o th tst st ad hc good dimsioal fatur spac. Usig a krl fuctio ca solv th oliar problm.thr ar two typs of SVMs, (1) Liar SVM, which gralisatio. A SVM prforms classificatio by costructig a N- dimsioal hyprpla that optimally sparats th ito two sparats th poits usig a liar dcisio boudary ad (2) catgoris. Th goal of a SVM is to sparat istacs ito two No-liar SVM, which sparats th poits usig a o-liar classs usig xampls of ach from th traiig to dfi th dcisio boudary. sparatig hyprpla. Th SVM mthod [7,8 ] provids a optimally Liar SVM prforms wll o sts that ca b asily sparatig hyprpla i th ss that th margi btw two sparat by a hyprpla ito two parts. But somtims sts ar groups is maximiz. This is show i Figur 1 complx ad ar difficult to classify usig a liar krl. No-liar Th subst of istacs that actually dfi th SVM classifirs ca b us for such complx sts. Th cocpt hyprpla ar call th support vctors, ad th margi is dfi bhid o-liar SVM classifir is to trasform th st ito a as th distac btw th hyprpla ad th arst support high dimsioal spac whr th ca b sparat usig a liar vctor. By maximizig this sparatio, it is bliv that th SVM dcisio boudary. I th origial fatur spac th dcisio bttr gralizs to us istacs, whil also mitigatig th boudary is ot liar. Th mai problm with trasformig th ffcts of oisy or ovr-traiig. Error is miimiz by st to highr dimsio is th icras i complxity of th maximizig th margi, ad th hyprpla is dfi as th ctr classifir. Also th xact mappig fuctio that ca sparat li of th sparatig spac, cratig quivalt margis for ach liarly i highr dimsioal spac is ot kow. I ordr to class. Prformac is most commoly valuat as classificatio ovrcom this, a cocpt call krl trick is us to trasform th accuracy ad/or margi width. Giv two SVMs with idtical to highr dimsioal spac. classificatio accuracy, o would prfr to choos th SVM with a Support vctor machis (SVMs) hav b us for largr margi width, ad vic vrsa. This trad-off is usually various applicatios as a powrful tool for pattr classificatio. Th icorporat ito th traiig of a SVM. succss of SVMs is bas o (1) mappig th iput spac to a highdimsioal fatur spac, ad (2) th maximizatio of th margi btw two classs i th fatur spac. O of th advatags of SVMs is that w ca improv gralizatio ability by propr X2 slctio of krls. I most cass polyomial krls ad radial basis fuctio twork (RBF) krls ar us. Optimal Hypr Pla 1. SVM Classificatio Cosidr th biary classificatio problm T = {(x 1, y 1 },.,{x l, y l }), x i ε R, y i ε {-1,1} (1) O th o had, w should mak th margi btw th two Maximum Marigi classs poits as larg as possibl, o th othr had, w should mak th classificatio rror as small as possibl. Figur 1: A liar sparabl support vctor machi X1 Th optimal classificatio problm is trasform ito followig covx quadratic programmig mi w 2 + c (2) Pag 79

3 s.t. y i ((w. x i ) + b) 1-0, i = 1, 2,, l, i = 1, 2,.., l Th solutio to abov optimizatio problm (2) is trasform ito its dual problm(3) by th saddl poit of th Lagrag fuctio mi α mi y i y j α i α j K(x i, x j ) - (3) s.t. y i α i = 0 0 α i c, i = 1,2,...,l By rsolvig its dual problm, w ca gt its solutio α * = (α 1 *,α 2 *,.. α l * ) T, ad th classificatio dcisio fuctio is f(x) = y i α i * K(x i, x) + b (4) Whr krl fuctio K(xi, x) = ((ɸ(x i ). ɸ(x)) is a symmtric fuctio satisfyig Mrcr's coditio, wh th giv sampl sts ar ot sparabl i th primal spac, w could map th with mappig ɸ ito a high dimsioal fatur spac whr liar classificatio is wll prform. III. Krl slctio Krl slctio plays a importat rol i SVM traiig ad classificatio. A proprly dsi krl fuctio ca miimiz gralizatio rror, acclrat covrgc sp, ad icras prictio accuracy. Thr ar two commo optimizatio mthods, addig paramtrs ad krl alimt. Addig paramtrs is a mthod for puttig additioal paramtrs i th krl ad optimizig thos paramtrs so as to improv th prformac. Thr ar four krl mthods ar availabl: 1) Liar: k(x, y) = x T y + c, which dos ot hav ay paramtrs to optimiz. 2) Polyomial: k(x, y) = (x T y + 1) d, whr w optimiz th dgr d. 3) Radial Basis Fuctio (RBF): k(x, y) = xp(- 2 ), whr w optimiz th γ valu. To assss th ffctivss of th mthod, thr diffrt mical sts wr us 1.SVM paramtrs Th quality of SVM modls strogly dps o a propr sttig of paramtrs ad SVM approximatio prformac is ssitiv to paramtrs. For Gaussia krl, paramtrs to b rgulat iclud hypr paramtrs C,ε ad krl paramtr γ. Th valus of C, γ ad ε ar rlat to th actual fuctio modl ad thr ar ot fix for diffrt st. So th problm of paramtr slctio is complicat. Th valus of paramtr C, γ ad ε affct modl complxity i a diffrt way. Th paramtr C dtrmis th tradoff btw modl complxity ad th tolrac dgr of dviatios largr tha ε. Th paramtr ε cotrols th width of th ε-itsiv zo ad ca affct th umbr of SV i optimizatio problm. Th krl paramtr σ dtrmis th krl width ad rlats to th iput rag of th traiig st. IV.EXPERIMENTAL RESULTS Th prformac mtrics of two krls ar compar to fid a optimal ad fficit krl ad it is carri out usig WEKA softwar. A comprhsiv prformac study has b coduct to valuat krl slctio usig ral-lif sts obtai from th UCI Machi Larig Rpository[6] to tst its prformac agaist two krals with diffrt paramtrs as show i Tabl 1. W implmt SVM classificatio for two typs of krls: polyomial ad Radial basis fuctio(rbf). Tabl 1 provids th attribut iformatio of thr sts Datasts Faturs Istacs Class Brast cacr Pima Diabtis Mammographic Mass Th prformac of a chos classifir is validat bas o accuracy, rror rat ad computatio tim. Th classificatio accuracy is prict i trms of Ssitivity ad Spcificity. Th computatio tim is ot for ach classifir is tak i to accout. Th valuatio paramtrs ar th spcificity, ssitivity, ad ovrall accuracy. 4) Sigmoid(MLP): k(x, y) = tah(αx T y+c), whr w optimiz th α slop ad c itrcpt costat. O of th bst-kow mthods is th support vctor machis (SVM), a krl-bas mthod which has foud applicatios i may pattr rcoitio problms [2], [3]. Two diffrt krls wr aalys i this work: Th Gaussia krl ad th polyomial o. Th accuracy of a SVM modl is largly dpt o th slctio of th modl paramtrs such as C, Gamma ad P. Thr ar a umbr of paramtrs such as C, Dgr ad Gamma that apply to th SVM modl ad th slct krl fuctio. Slctig th optimal valus ca siificatly impact th accuracy of th modl. For classificatio problms, th optimal valu of C typically is i th Pag 80

4 rag of 1 to 100. W radomly choos a traiig st(70%) ad a tstig st(30%) of thr sts. 1. Th cofusio matrix of ach SVM krl Classificatio mthod is prst i tabl-2 to tabl-7. Th valus to masur th prformac of th mthods (i.. accuracy, ssitivity, spcificity, rror rat ad tim) ar driv from th cofusio matrix ad show i figur -2 to figur-13. It has b suggst that accuracy obtai by th SVM dps largly o th krl slct ad th paramtrs. Th study focus o polyomial ad radial basis krl (RBF). Th polyomial ad RBF krls hav paramtr C that corrspods to th palty for misclassificatio. Th highr th valu of C is, th mor th palty, ladig th classificatio modl to b ovr-fittig. Covrsly, smallr valu of C lads to a mor graliz modl that may ot b abl to classify th ukow accuratly. I this papr, C was vari from 1 to 100 for liar ad 1 to 100 for th RBF krl ad thus th optimal valu of C was dtrmi alog with calculatig th prictio rat. Bas o th avrag prictio rat obtai by varyig th mtio paramtrs thy propos optimal valus for C as 1 for liar krl ad 10 for RBF. SVM algorithm was valuat usig two krls for thr sts. For SVM usig polyomial krl, th valu of p was chag from 1 to 2.5. Oly 1, 1.5 ad 2.5 wr chos i this valuatio. For SVM usig RBF krl, th valu of γ was chag from 1 to 2.5. Oly 1, 1.5 ad 2.5 wr chos i this valuatio. But i all thr sts th optimal paramtrs of.5 ad γ=1.5 shows th highst accuracy ad low umbr of rrors ar obtai our rsult as show i figur -2 to figur-13. Tabl 2: Cofusio Matrix of Brast cacr Traiig ad Tstig usig Polyomial kral Kr Traiig Data (499) Tstig Data (200) l param Dsir d trs B i Mali Bi Malig Bi Big Malia Malig 5 58 t Bi Big Malia t Malig 8 55 Bi Big Malia Malig t Bi Big Malia Malig t Bi Big Malia Malig t Bi Big Malia Malig t Bi Big Pag 81.5 c=100, Malia Malig 7 56 t c=100, Bi Big Malia Malig 9 54 t c=100, Bi Big Malia t Malig 7 56 Tabl 3: Cofusio Matrix of Brast cacr Traiig ad Tstig usig RBF krl Kr Traiig Data (499) Tstig Data (200) l para Dsir mt B M Rsu B Mali rs ig ali at lt i Bi 31 6 Bi γ=1.0 5 Malia 4 17 Mali 3 60 γ=1.5 γ =2.5 γ =1.0 γ =1.5 γ =2.5 t 4 Bi 31 8 Bi 3 Malia 4 17 Mali t 4 Bi Bi 1 Malia 4 17 Mali t 4 Bi Bi 0 Malia 7 17 Mali t 1 Bi Bi 1 Malia 8 17 Mali t 0 Bi Bi 0 Malia 9 16 Mali t

5 c=10 0, γ =1.0 Bi Bi Malia Mali 5 58 t 7 c=10 Bi 31 9 Bi , γ =1.5 Malia Mali 5 58 t 7 c=10 Bi Bi , γ =2.5 Malia t Mali 2 61 Tabl 4: Cofusio Matrix of Pima Diabtis Traiig ad Tstig usig Polyomial kral Kr l Traiig Data (538) Tstig Data (230) param trs Dsir Rsu lt Ng ativ Posi tiv Ng ativ Positiv Ngativ Nga tiv Positi Positiv v Nga Ngativ tiv Positi Positiv v Nga Ngativ tiv Positi Positiv v Nga Ngativ tiv Positi Positiv v Nga Ngativ tiv Positi Positiv t v Nga Ngativ tiv Positi Positiv v Nga Ngativ c=100, tiv Positi Positiv v Nga Ngativ c=100, tiv.5 Positi Positiv v Nga Ngativ c=100, tiv Positi Positiv v Tabl 5: Cofusio Matrix of Pima Diabtis Traiig ad Tstig usig RBF krl Krl param trs Traiig Data (538) Tstig Data (230) Dsir Dsir d Pag 82 Rsu lt Nga tiv Positi v γ=1.0 Nga tiv Ngati v Positi Positiv v γ=1 Nga Ngati.5 tiv v Positi Positiv v γ=2 Nga Ngati.5 tiv v Positi Positiv v γ Nga Ngati =1.0 tiv v Positi Positiv v γ Nga Ngati =1.5 tiv v Positi Positiv v γ Nga Ngati =2.5 tiv v Positi Positiv v c=100, Nga Ngati γ =1.0 tiv v Positi Positiv v c=100, Nga Ngati γ =1.5 tiv v Positi Positiv v c=100, Nga Ngati γ =2.5 tiv v Positi Positiv v Tabl 6: Polyomial krl Prformac of Mammographic Mass Traiig ad Tstig Krl param trs.5 Ng ativ Positiv Traiig Data (672) Tstig Data (289) Dsir B ig Mali Rsu lt B i Mali at Bi Bi Mali Mali at Bi Bi Mali Mali at Bi Bi Mali Mali at Bi Bi Mali Mali

6 at Bi Bi Mali Mali at Bi Bi Mali Mali at c=100,p Bi Bi =1 7 Mali Mali at c=100,p Bi Bi =1.5 5 Mali Mali at c=100,p Bi Bi =2.5 3 Mali at Mali Tabl 7: Cofusio Matrix of Mammographic Mass Traiig ad Tstig usig RBF krl Malia t Mali Figur-2: Polyomial krl accuracis with diffrt dgrs for brast cacr traiig Kr l param trs γ=1.0 γ=1.5 γ= 2.5 γ =1.0 γ =1.5 γ =2.5 c=100, γ =1.0 c=100, γ =1.5 c=100, γ =2.5 Traiig Data (672) Tstig Data (289) Dsir Bi Malig Rsu lt Bi Mali Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi Malia Mali t Bi Bi igur-3: Polyomial krl accuracis with diffrt dgrs for brast cacr tstig Figur -4: RBF krl accuracis with diffrt dgrs for brast cacr traiig igur -5: RBF krl accuracis with diffrt Gammas for brast cacr tstig Pag 83 F F

7 igur-6: Polyomial krl accuracis with diffrt dgrs for diabtis traiig F Figur -10: Polyomial krl accuracis with diffrt dgrs for mammographic traiig -7: Polyomial krl accuracis with diffrt for diabtis traiig Figur dgrs igur -11: Polyomial krl accuracis with diffrt dgrs for mammographic tstig F -8: RBF krl accuracis with diffrt Gammas for diabtis traiig Figur Figur-12: RBF krl accuracis mammographic Figur-9: RBF krl accuracis with diffrt Gammas for diabtis tstig r -13: RBF krl accuracis with diffrt Gammas for mammographic tstig Figu V.CONCLUSION Th SVM approach to machi larig is kow to hav both thortical ad practical advatags. Exprimtal rsults show that krl slctio gratly improvs th quality of classificatio. Our xprimtal rsults show that polyomial dgr from 1 to 2.5 ad RBF gamma paramtr 1 to 2.5 is optimal krl paramtrs for liar ad oliar classificatio o thr UCI sts. Th slctio of multipl krl paramtrs is addrss to achiv Pag 84

8 accuracy, rrors ad tim. Th highst accuracy is achiv polyomial krl for liar ad RBF krl for oliar classificatio. [4]. C.-W. Hsu ad C.-J. Li, A compariso of mthods for multiclass support vctor machis, IEEE Tras. Nural Ntw., vol. 13, o. 2, pp , Mar REFERENCES [1]. Brhard Schölkopf ad Alx Smola, Larig with krls. MIT Prss, Cambridg, MA, [2]. Bosr, B., Guyo, I., Vapik, V.: A traiig algorithm for optimal margi classifirs. I: ACM Cof. o Computatioal Larig Thory. (1992) [3]. C.-N. J. Yu ad T. Joachims, Traiig structural svms with krls usig sampl cuts, i Th 14th ACM SIGKDD Cofrc o Kowlg Discovry ad Data Miig (KDD), 2008, pp [5]. Ha J ad Kambr M, Data Miig Cocpts ad Tchiqus. Morga Kaufma, 2000 [6]. T. Joachims, Traiig liar svms i liar tim, i Th 12th ACM SIGKDD Itratioal Cofrc o Kowlg Discovry ad Data Miig (KDD), 2006, pp [6]. UCI machi larig rpository. [7].Vapik, V.N. Statistical Larig Thory. Joh Wily ad Sos, Nw York, USA, [8]. Vapik, V.N. Th Natural of Statistical Larig thory. Sprigr Vrlg,NwYork,USA Pag 85