A Robust Fuzzy Support Vector Machine for Two-class Pattern Classification

Transcription

1 76 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton G. H. L, J. S. aur, and C.W. ao Abstract hs papr proposs a systmatc mthod to classfy data wth outlrs. h ssntal tchnqus consst of th outlr dtcton and th fuzzy support vctor machn (FSVM). In ths approach, th man body st for ach class s frst dtrmnd by th outlr dtcton algorthm (ODA) that stmats th outlrs basd on th total smlarty obctv functon. hn, ncorporatd wth th total smlarty masur of th ODA, a fuzzy mmbrshp dgr s assgnd to ach tranng sampl. Exprmnts show that th proposd mthod can gratly rduc th ffcts of outlrs n th tranng procss and th fnal dcson surfac of th FSVM s nsnstv to outlrs. Kywords: Outlr dtcton, Support vctor machns, Fuzzy SVMs.. Introducton h thory of support vctor machns (SVMs) frst dvlopd by Vapnk and hs rsarch group s a powrful mthodology for solvng pattrn classfcaton and rgrsson stmaton problms [], [], [3], [4]. hos tchnqus ar basd on th thortcal larnng thory that mbods th structural rsk mnmzaton (SRM) prncpl []. SVMs hav bn shown to provd hgh gnralzaton prformanc on a wd rang of applcatons. h SVM tchnqu can b consdrd as an altrnatv tranng mthod for polynomal-functon, radal-bass-functon, and multlayr-prcptron classfrs by slctng propr krnl functons. In rcnt yars, SVMs hav bn appld broadly and succssfully to varous flds such as pattrn rcognton [3], mag classfcaton [5], tm prdcton [6], and rgrsson [7], [8]. In th thory of SVM, on of th man assumptons s that all data n th tranng st ar tratd qually. Corrspondng Author: C. W. ao s wth th Dpartmnt of Elctrcal Engnrng, atonal ILan Unvrsty. E-mal: cwtao@nu.du.tw Manuscrpt rcvd 8 Aug. 003; rvsd Dc. 003; accptd Jun Howvr, nosy data or outlrs ar usually nvtabl n practcal applcatons [9]. hs may mak th dcson surfac dvat svrly from th optmal hyprplan du to th unawarnss of outlrs n th tranng procss and lad to th dgradaton of gnralzaton prformanc n th tst stag. Rcntly, som algorthms hav bn proposd to tackl th outlr problm. In [0], th wghtd last squar support vctor machn (LS-SVM) s proposd to rduc th ffcts of outlrs. vrthlss, th paramtrs n thos tchnqus nd to b chosn carfully. In [], an adaptv margn SVM s proposd basd on th utlzaton of adaptv margns for ach tranng pattrn. Howvr, thr s no gnral way to us th class cntr n th margn of ach tranng data to supprss th ffcts of nos and outlrs. Usng of th dstanc btwn ach data pont and th cntr of th rspctv class, a robust SVM [5] s proposd to calculat th adaptv margn whch maks th SVM lss snstv to outlrs. In [], a robust SVM basd on an acclratd dcomposton algorthm s proposd to solv th ovr-fttng problm that rsults from outlrs n th tranng data st. hs approach and th tchnqu n [5] both dpnd on th dstanc btwn th tranng data and class cntrs n th fatur spac. In addton, som fuzzy support vctor machns (FSVMs) hav bn proposd to tackl th outlr problm [3], [4]. Huang t al., [3] adopt a fuzzy c-mans algorthm cascadd wth an unsuprvsd nural ntwork to dtct outlrs n a tranng data st. hn, a mmbrshp modl s dvlopd to assgn mmbrshp valus to man body tranng sampls and outlrs accordng to thr rlatv mportanc n th tranng data st. hrfor, FSVMs can rduc th ovr-fttng ffcts and outprform SVMs n classfcaton problms wth outlrs. In ths papr, a systmatc mthod usng th outlr dtcton algorthm (ODA) and th FSVM s proposd to handl th outlr problm. As shown n Fgur, th proposd approach frst adopts th ODA to obtan th man body from all th tranng sampls n ach class. Incorporatd wth th total smlarty masur from th ODA, ach sampl can b proprly assgnd a mmbrshp dgr by a pr-slctd sgmod functon. hus th proposd approach nhancs th nsnstvty of th FSVM to outlrs. h rst of ths papr s organzd as follows. Scton gvs a brf rvw of SVMs and FSVMs. In Scton

2 G. H. L, t al.: A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton 77 3, th proposd algorthm s dvlopd. h xprmntal rsults and dscussons ar prsntd n Scton 4. Fnally th concluson s gvn n Scton 5. Fgur. h proposd classfcaton systm.. Rlatd Background In ths scton, w wll brfly rvw th algorthms of SVMs and FSVMs. Mor dtald dscrptons can b found n [], [], [3], and [4]. A. Support Vctor Machns h support vctor machn s a classfr basd on th structural rsk mnmzaton to fnd th hyprplan that maxmzs th margn btwn classs. Wthout loss of gnralty, th thory of SVMs s ntroducd through a two-class classfcaton problm. Assum that th sampls from class on and class two ar assocatd wth a class labl y = and y = rspctvly. Gvn a tranng data st S of data ponts S = { x, } y = () th n n whch th nput sampl x blongs to on of th two classs labld by y {, }. h tranng goal of th SVM s to fnd an optmal hyprplan w ϕ ( x) b = 0 that maxmally sparats th two classs of tranng sampls, whr ϕ() s a nonlnar functon whch maps th nput spac nto a hghr dmnsonal spac, w s a wght vctor, and b s a bas of th hyprplan. hn th sampl pont x can b assgnd ts corrspondng class labl and th classfr can b xprssd as = ( ϕ b) g( x ) sgn w ( x ) () whr sgn( ) stands for th bpolar sgn functon. For a sparabl cas, thr xsts a wght vctor w and a bas b such that ach sampl pont satsfs th followng condtons: w ϕ( x) b>, for y = w ϕ( x) b<, for y = whch ar quvalnt to (3) y ( w ϕ ( x ) b), =,,...,. (4) In ths sparabl cas, th optmal hyprplan that maxmzs th margn of sparaton can b found. Howvr, n th non-sparabl cas, th sparatng hyprplan n th hghr dmnsonal spac dos not xst. In ordr to handl such cass, a st of nonngatv slack varabls { ξ } = s ntroducd such that th followng condtons ar satsfd y ( w ϕ( x ) b) ξ, =,,..., (5) ξ 0, =,,...,. (6) hs approach allows tranng sampls that volat Eq. (4). Accordng to th structural rsk mnmzaton, th optmal dcson can b found by solvng th followng quadratc programmng (QP) problm: mn L ( w, ξ) = w w Cξ (7) = subct to Eqs. (5) and (6), whr C s a prdfnd postv constant. A smallr C mposs a lss pnalty on mprcal rrors. Instad of solvng th QP optmzaton n th prmal spac, a st of Lagrang multplrs s ntroducd for Eqs. (5) and (6), and th followng dual problm can b obtand max Q( α ) = α αα y y ϕ( ) ϕ( ) α x x (8) = = = subct to 0 α C, =,,..., (9) = αy = 0, =,,...,. (0) Accordng to th Kuhn-uckr condtons, th soluton { α } = to Eqs. (8), (9), and (0) has to satsfy th followng condtons ( 0 0 ) α y ( w ϕ( x ) b ) ξ = 0, =,,..., () ( C α) ξ 0,,,..., = =. () hos ponts wth α > 0 ar calld support vctors whch can b dvdd nto two typs. If 0 < α < C, th corrspondng tranng ponts ust l on on of th margns. If α = C, ths typ of support vctors ar rgardd as msclassfd data.

3 78 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 B. Fuzzy Support Vctor Machns h SVM has bn ntroducd as a powrful tool for solvng classfcaton problms. Howvr, thr ar stll som dffcults n applyng th thory to practcal problms. On of th maor dffcults s that th SVM algorthm s snstv to outlrs. Although th nflunc of outlrs can b rducd by choosng a propr paramtr C, t s not asy to fnd a sutabl C. In th formulaton n Eq. (7), th paramtr C s a usr-dfnd paramtr to pnalz th tranng data wth a postv ξ. A largr C mposs a havr pnalty on th msclassfd tranng data and thus rsults n fwr support vctors and a narrowr sparaton rgon, whl a smallr C sts a smallr pnalty for th rror and thus lads to a wdr margn [6]. It s clarly that all tranng ponts n th class ar tratd qually n th thory of th SVM. hs may mak th SVM vry snstv to nos and outlrs [9]. In many applcatons of pattrn classfcaton, som tranng ponts ar mor mportant than th othrs. hrfor, t s vry mportant to dstngush th manngful tranng ponts from outlrs or nosy sampls. hs can b achvd by assgnng a mmbrshp valu u to ach tranng pont x. Assum that a tranng data st S Fuzzy of data ponts wth corrspondng mmbrshp valus s gvn by S {,, } Fuzzy = x y u = (3) n th whr x s th nput data sampl, y {, } s ts labl, and 0 u s ts mmbrshp valu. In contrast wth SVMs, th trm uξ s usd as a wghtd masur of th rror n FSVMs. Bcaus a propr mmbrshp valu s assgnd to ach tranng sampl, FSVMs should b mor robust n th classfcaton problms. In ths formulaton, th optmal sparatng hyprplan s rgardd as th soluton to mn L ( w, ξ) = w w Cu ξ (4) = subct to y ( w ϕ( x) b) ξ, =,,..., (5) ξ 0, =,,..., (6) whr C s a postv paramtr to b dfnd by th usr. h largr (smallr) th valu u s, th mor (lss) nflunc th paramtr ξ has, and th mor (lss) mportant th tranng pont x s. ow th Lagrangan functon can b constructd as Q( w, b, ξ, α, v, u ) C uξ = w w = ( ( ( ) ) ) α y w ϕ x b ξ vξ = = (7) whr { α, } v = ar Lagrang multplrs. h soluton s gvn by th saddl pont of th Lagrangan functon Q( w, b, ξ, α, v, u). hus, th followng dual optmzaton problm (functon of α only) can b obtand max Q( α ) = α αα y y ϕ( ) ϕ( ) α x x (8) = = = subct to 0 α u C, =,,..., (9) = αy = 0, =,,..., (0) and th Kuhn-uckr condtons ar ( 0 0 ) α y ( w ϕ( x ) b ) ξ = 0, =,,..., () ( α) ξ 0,,,..., uc = =. () Whn compard wth SVMs, t s obvous that th uppr bounds of FSVMs n Eq. (9) ar dffrnt from thos of SVMs. h uppr bounds of FSVMs ar functon of mmbrshp valus u such that thy can adust th wghtng for manngful ponts and outlrs n th tranng procss, whl th uppr bounds of SVMs ar constants. Smlarly, thr ar also two knds of support vctors. ranng ponts wth 0 < α < uc wll l on th margn of th hyprplan. ranng ponts wth α = uc ar msclassfd. Onc th valus of α hav bn found, th soluton of can b dtrmnd by SV = w 0 w0 = αyϕ( x ) (3) SV whr s th numbr of support vctors, and th valu of thrshold b 0 can b obtand from Eq. (). In ths way, th dcson functon can b obtand n th nw fatur spac,.., SV g( x) = sgn αyϕ( x) ϕ( x) b0. (4) = h calculaton of Eqs. (8), (8), and (4) rqurs th computaton of th nnr product ϕ( x) ϕ( x ) or ϕ( x ) ϕ( x ) n a hgh dmnsonal fatur spac. By usng a sutabl Krnl functon K, ( K( x, x )

4 G. H. L, t al.: A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton 79 = ϕ( x ) ϕ( x )), that obys th Mrcr condton [], [5], Eqs. (8) and (8) can b computd n th nput spac through ths krnl tchnqu. Smlarly, th dscrmnant functon can b wrttn as SV g( x) sgn yk(, x ) b. (5) = α x 0 = h maor advantag of usng a krnl functon s that th xplct computaton of ϕ( x ) can b avodd. Instad of calculatng th nnr product of ϕ( x ) ϕ( x ) n th fatur spac, th krnl functon K( x, x) can b obtand n th prmal nput spac. 3. h Proposd chnqus In ths scton, a systmatc mthod s dvlopd to classfy data sts contanng outlrs. It ncluds an ODA for dtctng outlrs, th FSVM machn wth th fuzzy mmbrshp functon, and th krnl paramtr stmaton. A. Outlrs Dtcton In ths work, w ntroduc a smpl but ffctv mthod basd on th smlarty masur whch ams to dtct th outlrs n a tranng st. It s assumd n ths algorthm that outlrs n a class hav two mportant charactrstcs [7], [8]. h frst on s that th numbr of outlrs should b much smallr than th numbr of pattrns n th man body. h scond on s th outlrs should at last somwhat sparat from th man body. Mor prcsly, lt S = { x, x,..., x m } b a st of tranng sampls from a class dstrbutd n an unknown probablty dnsty functon n th nput spac. W us a smlarty masur functon (SMF) dnotd as D( x, S) to masur whthr th data x s locatd nsd, nar or far from th man body of S. h SMF D( x, S) s dfnd as m x x D ( x, S) = xp = β, λ > 0 (6) whr β =, = S m x x z z x S m x, (7) and dnots th Eucldan dstanc. In Eq. (6), λ s an adustabl paramtr that s usd to rplac th ffct of th paramtr β. hrfor, th paramtr β can b assgnd a fxd valu, for xampl, th sampl varanc of th data. Eq. (6) s also usd n th λ mountan clustrng mthod proposd by Yagr and Flv [9], [0]. Wth a propr paramtr λ, t can b rgardd as th stmat of th dnsty shap of th data ponts n th nghborhood of x. h data pont x wth a smallr D( x, S) wll b locatd farthr away from th man body of S. Whn thr ar mor data ponts around x, th valu D( x, S) wll bcom largr. hus th st { D( x, )} m S = can b tratd as an ndx to dtct th outlrs. In ordr to analyz th ffct of th paramtr λ, th smlarty masur for a spral-shapd data st S wth 3 outlrs s calculatd usng Eq. (6). Plots (b), (c), and (d) n Fgur show th rsults of th SMF wth λ =, 0, and 0, rspctvly, and β s pr-slctd as th sampl varanc. From Fgur (b), t s clar that th SMF wth λ = can not solat th outlrs from th man body of S. Howvr, aftr ncrasng λ to 0 or 0, th paks of th SMF can clarly dstngush th outlrs from th man body. It should b notd that th pak valus of th man body ar much largr than thos of th outlrs. Although Fgurs (b), (c), and (d) can provd th vsual mprsson for slctng th valu of th paramtr λ, a mthod for slctng a propr λ must b takn nto account for a systmatc mthod. In an applcaton, th prcntag of th outlrs n th data st S s unknown. In ordr to dtct all th outlrs, a largr ntal valu δ % s adoptd to rprsnt th canddats for outlrs. Accordng to Fgur and Eq. (6), ncrasng λ s quvalnt to dcrasng th nghborhood radus of th data pont x. For a data pont, Eq. (6) can b rwrttn as x m x x D ( x, S) = xp, λ > 0 (8) = β whr th scond trm n th rght sd of th qualty rprsnts th contrbuton from th othr lmnts n th data st S to D( x, S). If th data pont x s an outlr and λ has bn proprly dtrmnd, th valu of th scond trm should b vry small. hus, w frst slct a propr λ such that D( x, S) < θ for th data ponts x wth th δ % lowst valu of th D( x, S) n th data st. hn th numbr of outlrs can b stmatd. In our xprmnts, th ntal thrshold s st to b θ = 0.0 ( m ) (9) and ts maxmal valu s θ = 4. h ODA for a gvn class can b summarzd as followng stps : λ

5 80 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 (a) (b) (c) (d) Fgur. (a) A data st wth spral shap. (b), (c), and (d) ar 3D plots usng Eq. (6) wth λ=, 0, and 0, rspctvly. Stp ) St λ =, Δ λ =0., δ %=5%, β as a fxd valu, γ = nt ( m δ %), and outlr_loop = tru. Stp ) Calculat { D( x, )} m S = and rarrang th corrspondng valus n ascndng ordr dnotd as { g } m =. Stp 3) If g γ θ, thn GOO Stp 4 Els λ = λδ λ GOO Stp End Stp 4) If (outlr_loop), thn P dff = { g g} γ = P = P max max dff = γ P P P man dff ( { max} ) End If ( P 3 P ) > 0., thn max γ outlr man γ = arg max P = dff Els γ = arg g ( g 0.5( g g )) outlr End γ = γ outlr θ = g γ outlr_loop = fals λ = GOO Stp Els λ = λ β τ b = λ Stop th ODA { γ } It should b notd that th pr-dtrmnd valu β

6 G. H. L, t al.: A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton 8 must b larg nough such that g γ > θ for th ntal valu λ =. γ outlr s th stmatd numbr of outlrs and λ s th stmat of λ. Fnally, th man body of th data st S and th paramtr τ b = β/ λ for th man body can b obtand. For th convnnc of dscrptons, th suprscrpts and ar usd to dnot varabls for Class and Class, rspctvly. hus, { } m S = x = ( { } m S = x = ) dnots th data st wth an output labl y = ( y = ) n rst of th papr, whr m ( m ) s th numbr of sampls n S ( S ). In ordr to valuat th ODA, a data st wth two classs s shown n Fgur 3 (a), whr data ponts wth th " " (" ") sgn rprsnt S ( S ) and ach class contans 6 outlrs. Fgur 3 (b) shows th valu of th SMF functon for ach data pont wth δ %=5%. It s obvous that th ampltuds of th outlrs ar much lowr than thos of th man body. Aftr applyng th ODA, th man body of S ( S ) wth th corrspondng λ = 5.0 ( λ = 4.7 ) s shown n Fgur 3 (c), th outlrs for ach class ar dntfd corrctly, and th bandwdth for th man body of S ( S ) s τ =.04 b ( τ b =.3 ). Fgur 3 (d) shows that th obtand λ s for th SMF can rprsnt th dnsty shap of th two man body sts. B. Mmbrshp Functons for FSVMs In gnral, th crtron to choos th mmbrshp valu for ach sampl dpnds on th rlatv mportanc of th data pont n ts class. As dscrbd prvously, th ODA s capabl of fttng th dnsty shap of a gvn data st, and th stmatd τ b = β/ λ can rprsnt th sutabl ntrpolaton nghborhood radus of a data pont n th gvn data st. hus, ncorporatd wth th smlarty masur obtand from th ODA, a mmbrshp valu can b assgnd to vry tranng sampl by th followng fuzzy modl to form th tranng fuzzy sts, cf. Eq. (3). h mmbrshp valu st for S and S s dfnd as (a) (b) (c) Fgur 3. (a) A data st of two classs wth outlrs. (b) h ampltuds of th SMF functon for ach data pont. (c) h corrspondng man body st aftr applyng th ODA tchnqu. (d) h dnsty shap of th man body sts wth λ =5.0, and 4.7, rspctvly. (d)

7 8 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 u = {,...,, u u u,..., u m m } whr u = mn{ u, u } and u mn{ u, u }. (30) = u s an ndx of th mportanc of th data pont data st u wth S,.., ( η ( x )) S η x n th = (3) xp D (, ) θ m (, S ) = xp x x D x, λ (3) = β whr λ s obtand from th ODA, β s a pr-dtrmnd valu, and η controls th chang rat of th sgmod functon. D( x, S ) masurs th smlarty of x n S, θ s dtrmnd by th ODA algorthm, and η s a wghtd factor. In contrast to, masurs th ovrlappng dgr btwn th u u data pont x and all lmnts n th data st S and s dfnd as u = (33) xp D(, ) θ wth λ ( η ( x )) 3 S η 4 λ m S = x x λ = β D( x, ) xp, (34) whr β and θ ar pr-dtrmnd valu for S, η 3 control th chang rat of th sgmod functon, η 4 s a wghtd factor, and D( x, S ) masurs th smlarty btwn th data pont S n and th data st. Smlarly, th mmbrshp valu of a data pont S can b obtand. x u C. Paramtr Slcton In ths approach, th FSVM wth th RBF krnl s adoptd to classfy data wth outlrs. In th FSVM, th rgulaton paramtr C controls th trad-off btwn th margn maxmzaton and th amount of msclassfcaton, and th krnl paramtr of th RBF, σ, controls th capablty of th classfr. Snc th outlrs ar assocatd wth low mmbrshp valus, th paramtr C can b st to a suffcnt larg valu such that th FSVM can obtan a smallr msclassfcaton rat for th man body sts. h slcton of paramtr σ for th RBF krnl s also non-trval. Rcntly, svral mthods wr proposd to nvstgat th slcton of Gaussan krnl paramtr [], []. In addton, Yng t al., [3] proposd an optmzaton approach to tran SVMs wth hybrd krnls such that a supror gnralzaton prformanc ovr tst data could b obtand, whr th paramtrs of th hybrd krnls ar dtrmnd by mnmzng th uppr bound of th VC dmnson. Accordng to [3], snc only th RBF krnl s adoptd n ths papr, th obctv functon bcoms mn q( ) σ σ = R w (35) x x (36) subct to b0 = y yαk (, ) SV SV, = SV and Eq.(8), n whch R s th radus of th smallst sphr contanng all of th transformd data ponts. hs optmzaton problm s to mnmz th uppr bound of th VC dmnson through th paramtr adustmnt of th krnl. Mor dtald dscrptons can b found n [3]. In ths papr, th optmzaton crtron s usd to choos th krnl paramtr σ for th man body sts of th tranng data. On th othr hand, t should b notd that th ODA tchnqu can approxmatly rprsnt th actual dnsty shap of th data sts. hus, th krnl paramtr σ can also b drctly stmatd from th rsults of outlr dtcton of S and S, and t s dfnd as σ = τ τ (37) ( b b ) whr τ b = β / λ and τ b = β / λ. In th followng scton, ths two mthods for slctng th krnl paramtr σ wll b adoptd and compard. 4. Exprmntal Rsults o llustrat th FSVM of ths papr, artfcal data and bnchmark data ar conductd to valuat th prformanc. Data sts wth dffrnt dstrbuton shaps and outlrs ar frst xprmntd to valuat th proposd mthod. h data sts ar shown n Fgur 4, whr Plots (c) and (d) ar of th sam shap but wth dffrnt dstancs btwn classs. ranng sampls n data sts S and S ar ndcatd by " " and " " symbols, rspctvly. abl lsts th numbrs of sampls and outlrs n ach cas. In th xprmnts, th FSVM wth th RBF krnl s adoptd to classfy th data. h paramtrs η, η, η 3, η 4, and C ar st

8 G. H. L, t al.: A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton 83 to b 0,, 0,, and 500, rspctvly, S θ s st to b θ for, and th krnl paramtr ar stmatd by Eq. (37) and by mnmzng th uppr bound of th VC dmnson (.., Eq.(35)). Fgurs 5 (a), (d), (g), and () show th classfcaton rsults of th SVM wth krnl paramtrs stmatd by Eq. (37), whr th black sold curvs rprsnt th sparatng boundars. h support vctors for ach class ar markd wth rd crcl, and th color dot curvs ndcat th qual output lvls of th SVM classfr btwn - and wth an ntrval of 0.. From th classfcaton rsults of th SVM classfrs, t s clar that th dcson surfacs dvat svrly from th optmal ons du to th unawarnss of outlrs. Whn th proposd mthod s adoptd to classfy th tst xampls, th corrspondng rsults wth th krnl paramtr stmatd by Eq. (37) ar shown n Fgurs 5 (b), (), (h), and (k). abl gvs th stmatd paramtr λ from th ODA, and th numbr of th dtctd outlrs. Although th ovr-fttng problm du to th outlrs occurs n Fgur 4 (a), th sgmod fuzzy functon can dtrmn th rlatv mportanc of data ponts n th corrspondng class, and th obtand dcson surfacs ar lss snstv to th outlrs. In addton, th proposd FSVM mthod wth th krnl paramtr dtrmnd by mnmzng th uppr bound of th VC dmnson s usd to classfy th sam data sts. h rsults ar shown n Fgurs 5 (c), (f), (), and (l). abl 3 lsts th valus of krnl paramtrs for th xampls stmatd by ths two mthods. It s obvous that th proposd FSVM mthod can largly rduc th ffct of outlrs usng th krnl paramtr σ dtrmnd by Eq. (37) or Eq. (35). Howvr, th computatonal complxty usng Eq. (35) s much hghr. o furthr valuat th classfcaton and gnralzaton prformanc of th FSVM, w apply th approach to th banana data, twonorm data and thyrod data from UCI lstd n abl 4, whr ach data st s splt nto 00 sampl sts of tranng and tst st. h paramtrs η, η, η 3, and η 4 ar st to b, 0.5,, and., rspctvly. θ s obtand from th ODA for S. hn th prformanc btwn th SVM and th FSVM s masurd by thr avrag rror ovr on hundrd parttons of th datast nto tranng and tst sts. For our comparson, th krnl paramtr for ach tranng and tst st s stmatd by Eq. (37) drctly, and th robustnss tst s conductd wth C =, 0, 50, 00, 500, 000, and δ %=0%, 5%, 0%. abl 5 lsts th avrag tst rror rats for th SVM and th proposd FSVM whl varyng C and δ %. It s obvous that th FSVM has bttr prformanc n most cass. For comparson, th sam data sts ar usd to valuat th SVM, th FSVM usng stratgy of krnl-targt algnmnt (K) [4], and th FSVM usng stratgy of k- (k-) [4] n whch th paramtrs ar th sam as statd n [4]. abl 6 lsts th tst rror rats whr th rsults of th proposd FSVM ar th bst prformanc n abl 5. For thyrod data st, th FSVM usng stratgy of k- can not mprov th prformanc of SVMs [4]. hus, w lav blank n abl 6. h smulaton rsults show that th proposd FSVM s vry comparabl to th FSVM usng stratgy of krnl-targt algnmnt (K) and outprforms ts countr part - th convntonal SVM. abl. umbr of data ponts and outlrs for tst xampls Exampl # of data ponts S # of outlrs # of data ponts S # of outlrs Fgur 4 (a) Fgur 4 (b) Fgur 4 (c) Fgur 4 (d) abl. Paramtrs for th proposd FSVM classfr S S Exampl λ # of stmatd λ # of stmatd outlrs outlrs Fgur 4 (a) Fgur 4 (b) Fgur 4 (c).5.6 Fgur 4 (d).5.6 abl 3. Krnl paramtr for th proposd FSVM classfr Krnl paramtr Fgur 4 (a) Fgur 4 (b) Fgur 4 (c) Fgur 4 (d) Eq. (37) Eq. (35) (a) (c) (b) (d) Fgur 4. Four tst xampls.

9 84 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 (a) (b) (c) (d) () (f) (g) (h) () () (k) (l) Fgur 5. (a), (d), (g) and () ar th classfcaton rsults of data ponts wth outlrs usng th SVM machn. (b), (), (h) and (k) ar th classfcaton rsults usng th proposd FSVM machn wth krnl paramtr stmatd by Eq. (37). (c), (f), () and (l) ar th classfcaton rsults usng th proposd FSVM machn wth krnl paramtr dtrmnd by mnmzng th uppr bound of th VC dmnson.

10 G. H. L, t al.: A Robust Fuzzy Support Vctor Machn for wo-class Pattrn Classfcaton 85 abl 4 Fatur of bnchmark data Data # of tranng pattrns # of tst pattrns nputs classs Banana wonorm hyrod abl 5 Comparson of th avrag tst rror rat for th SVM and th proposd FSVM Data Banana (%) wonorm (%) hyrod (%) δ % C SVM FSVM SVM FSVM SVM FSVM abl 6 Comparson of th avrag tst rror rat for th SVM, th FSVM usng stratgy of krnl-targt algnmnt (K), th FSVM usng stratgy of k- (k-), and th proposd FSVM Data SVM K k- th proposd FSVM Banana wonorm hyrod Concluson A systmatc mthod for th two-class classfcaton of data wth outlrs has bn dvlopd n ths papr. h ssntal tchnqus consst of th outlr dtcton algorthm (ODA) and th FSVM. In ths approach, th man body st for ach class s frst dtrmnd by th ODA. hn, a mmbrshp valu s assgnd to ach tranng sampl by th sgmod fuzzy modl. Aftr that, th FSVM wth th stmatd krnl paramtr s adoptd to classfy th data. Basd on th xprmntal rsults, th proposd mthod s shown to b robust aganst outlrs. 6. Rfrncs [] C. Corts and V.. Vapnk, Support vctor ntworks, Machn Larnng, vol. 0, pp , 995. [] V.. Vapnk, h atur of Statstcal Larnng hory, w York: Sprngr-Vrlag, 995. [3] C. Burgs, A tutoral on support vctor machns for pattrn rcognton, Data Mnng and Knowldg Dscovry, vol., no., 998. [4] B. Schölkopf, C. Burgs, and A. Smola, Advancs n Krnl Mthods: Support Vctor Larnng, Cambrdg, MA: MI Prss, 999. [5] Q. Song, W. J. Hu, and W. F. X, Robust support vctor machn wth bullt hol mag classfcaton, IEEE rans. Syst., Man, Cybrn. C, vol. 3, pp , ov. 00. [6] S. Mukhr, E. Osuna, and F. Gros, onlnar prdcton of chaotc tm srs usng a support vctor machn, n Proc. SP, 997, pp [7] H. Druckr t al., Support vctor rgrsson machns, n ural Informaton Procssng Systms. Cambrdg, MA: MI Prss, vol. 9, 997. [8] V. Vapnk, S. Golowch, and A. J. Smola, Support vctor mthod for functon approxmaton, rgrsson stmaton, and sgnal procssng, n ural Informaton Procssng Systms. Cambrdg, MA: MI Prss, vol. 9, 997. [9] C. Chuang, S. Su, J. Jng, and C. Hsao, Robust support vctor rgrsson ntworks for functon approxmaton wth outlrs, IEEE rans. ural twroks, vol. 3, pp , ov. 00. [0] J. A. K. Suykns, J. D Brabantr, L. Lukas, and J. Vandwall, Wghtd last squars support vctor machns: robustnss and spars approxmaton, urocomput., vol. 48, no., pp , 00. [] R. Hrbrch and J. Wston, Adaptv margn support vctor machns for classfcaton, n Proc. 9th ICA, vol., pp , Spt [] W. J. Hu, and Q. Song, An acclratd dcomposton algorthm for robust support vctor machns. IEEE rans. Crcut and Systm, vol. 5, pp , May 004. [3] H. P. Huang and Y. H. Lu, Fuzzy support vctor machns for pattrn rcognton and data mnng, Intrnatonal Journal on Fuzzy Systms, vol. 4, no. 3, pp , Spt. 00. [4] C. F. Ln and S. D. Wang, Fuzzy support vctor machns, IEEE rans. ural tworks, vol. 3,

11 86 Intrnatonal Journal of Fuzzy Systms, Vol. 8, o., Jun 006 ssu., pp , Mar. 00. [5] Smon Haykn, ural tworks: A Comprhnsv Foundaton, Scond Edton, w Jrsy: Prntc-Hall, 999. [6] M. Pontl and A. Vrr, Massachustts Inst. chnol., AI Mmo no. 6. Proprts of support vctor machns, 997. [7] V. Barnt and. Lws, Outlrs n Statstcal Data, Y: Wly, 994. [8] X. Lu, G. Chng, and John X. Wu, Analyzng outlr cautously, IEEE ransacton on Knowldg and Data Engnrng, vol. 4, no., pp , 00. [9] R. R. Yagr and D. P. Flv, Approxmat clustrng va th mountan mthod, IEEE rans. Systms, Man and Cybrntcs, vol. 4, pp , 994. [0] M. S. ang, and K. L. Wu, A smlarty-basd robust clustrng mthod, IEEE rans.pattrrn Analyss and Machn Intllgnc, vol. 6, no. 4, pp , Aprl [] M. Crstann, J. Shaw-aylor, and C. Campbll, Dynamcally adaptng krnls n support vctor machns, IPS-98 or urocol chncal Rport Srs C-R , Dpt. of Engnrng Mathmatcs, Unv. of Brstol, U.K., 998. []. Crstann and J. Shaw-aylor, An Introducton to Support Vctor Machns, Cambrdg Unv. Prss, [3] Y. an, and J. Wang, A support vctor machn wth a hybrd krnl and mnmal Vapnk-Chrvonnks dmnson, IEEE rans. Knowldg and Data Engnrng, vol. 6, no. 4, pp , Aprl [4] C. F. Ln and S. D. Wang, ranng algorthm fuzzy for fuzzy support vctor machns wth nosy data,, IEEE XIII Workshop on ural tworks for Sgnal Procssng, pp , 003. Jnshuh aur rcvd th B.S. and M.S. dgrs n Elctrcal Engnrng from atonal awan Unvrsty, ap, awan, R.O.C., n 987 and 989, rspctvly, and th Ph.D. dgr n Elctrcal Engnrng from Prncton Unvrsty, n 993. H was a Mmbr of chncal Staff n Smns Corporat Rsarch, Inc. H s currntly a Profssor at th atonal Chung Hsng Unvrsty, awan, R.O.C. Hs rsarch ntrsts nclud nural ntworks, pattrn rcognton, computr vson, and fuzzy logc systms. Dr. aur rcvd 996 IEEE Sgnal Procssng Socty s Bst Papr Award. C. W. ao rcvd th B.S. dgr n lctrcal ngnrng from atonal sng Hua Unvrsty, Hsnchu, awan, R.O.C., n 984, and th M.S. and Ph.D. dgrs n lctrcal ngnrng from w Mxco Stat Unvrsty, Las Crucs, n 989 and 99, rspctvly. H s currntly a Profssor wth th Dpartmnt of Elctrcal Engnrng, atonal I-Lan Unvrsty, I-Lan, awan. Hs rsarch ntrsts ar on th fuzzy nural systms ncludng fuzzy control systms and fuzzy nural mag procssng. Dr. ao s an Assocat Edtor of th IEEE RASACIOS O SYSEMS, MA, AD CYBEREICS. H s lstd n Who s Who n th World. Gwo-Hr L rcvd th B.S. and M.S. dgrs n Elctrcal Engnrng from Chung-Chng Insttut of chnology, awan, R.O.C., n 989 and 99, rspctvly. H s currntly pursung th Ph.D. dgr n th Dpartmnt of Elctrcal Engnrng at atonal Chung-Hsng Unvrsty, awan. Snc 99, h has bn an ngnr wth Aronautcal Rsarch Laboratory, Chung-Shan Insttut of Scnc and chnology. Hs rsarch ntrsts nclud flght smulaton, 3D graphcs, nural ntworks, fuzzy logc systms, and machn larnng.