A Comparison of Neural Network, Rough Sets and Support Vector Machine on Remote Sensing Image Classification

Transcription

1 A Comparso of Neural Network, Rough Sets ad Support Vector Mache o Remote Sesg Image Classfcato Hag XIAO 1, Xub ZHANG 1, Yume DU 1: School of Electroc, Iformato ad Electrcal Egeerg Shagha Jaotog Uversty : Cotug Educato School, Shagha Jaotog Uversty No. 1954, Huasha Road, Shagha P.R. Cha xaohag@stu.edu.c, Abstract: - Ths paper frst revewed the relevat theores of eural etwork, rough sets ad support vector mache (SVM. All of them have great advatages o dealg wth varous mprecse ad comeplete data. However, there exsts essetal dfferece amog them. Except for eural etwork, rough sets ad support vector mache are seldom used the feld of remote sesg mage classfcato. How to combe the theores wth the applcato of remote sesg s a mportat tedecy the later research. I the paper, eural etwork, rough sets ad support vector mache are appled to the area of remote sesg mage classfcato. Dfferet etworks, thresholds ad kerel fuctos are used three methods respectvely for the purpose of comparg the expermetal results. The paper provdes us a ew vewpot o remote sesg mage classfcato the future work. Key-Words: - eural etwork, varable precso rough sets model, support vector mache, remote sesg mage classfcato 1 Itroducto Remote sesg was proposed 1960 s as a sythetcal techque whch has bee used a wde rage of applcatos such as geoscece, agrculture, evromet ad so o. Remote sesg mage classfcato s oe of the most mportat techologes the feld of remote sesg mage processg. How to crease precso of classfcato s a key ssue remote sesg mage processg. Curretly, the maor methods remote sesg mage classfcato are statstc classfcato method, structural classfcato method ad fuzzy classfcato method. I these methods, statstc such as mea, varace, stadard devato ad dscrete degree are take as crtera to dstgush dfferet categores. All of them eed umerous statstc calculato but a low classfcato precso. Neural etwork has bee used remote sesg mage classfcato these years ad gotte a satsfyg result. There have plety of lteratures the feld [1][][7]. However, there also have several weakess eural etwork such as slow learg rate, dffcult covergece, complex etwork structure ad umbguous meag of etwork. Wth the developmet of research, there appear may ew theores formato processg ad kowledge dscovery sce 1980 s. I these theores, rough sets theory ad support vector mache(svm are the most attractve. However, remote sesg mage classfcato, these two theores are stll ther begg stage. Few lteratures o rough sets the feld are foud the research [3]. There s eve fewer researches o SVM feld of remote sesg mage classfcato [4][5]. Advatages have bee show rough sets theory dealg wth varous mprecse, complete formato ad eural etworks. However, there exsts essetal dfferece betwee them. Rough sets smulate abstract logc md of our huma beg whle eural etworks smulate tuto md. Rough sets theory express logc rules based o dscerblty relato ad kowledge reducto whle eural etworks state relato betwee put ad output by usg olear mappg. I geeral, eural etworks ca ot reduce dmesos of puts. More complex structures ad trag cost requred eural etworks of a hgher put dmesos. Rough sets theory ca be used to decrease redudace amog put formato through fdg ther relatos, but rough sets theory s very sestve to oses. Therefore, the good results derved from sample data may ot appear good whe they are appled the set of test data. That s to say, rough sets have a weak error tolerace ad geeralzato performace. Whereas, eural etworks have a better capablty of at-ose, self-orgazato, ad ISBN: ISSN:

2 geeralzato [6]. Fortuately, varable precso rough sets (VPRS model proposed by Zarko provde us a very useful tool to solve the problem [8]. Varable precso rough sets model allow for some degree of msclassfcato, whch ca avod the hgh sestvty of computatoal results. It s ecessary to crease the system redudace. Ths paper maly troduce applcato of the varable precso rough sets o remote sesg mage classfcato. I the result, we ca fd that the performace of system acheve much mprovemet by usg the varable precso rough sets. SVM s a ew mache learg theory proposed by Vapk et al. md 1990s. It s a uversal method to solve multdmesoal fucto. It has bee appled some areas such as fucto smulato, patter recogto ad data classfcato ad obtaed a perfect result. There exst some defects eural etwork such as determato of etwork structure, local mma problems, uder learg ad over learg. All of them restrct the applcato of eural etwork. SVM has advatages solvg the problems of o- lear, patter selected, hgh dmeso, small specme, whch s good complemetary wth eural etwork. Ths paper s orgazed as follows. I secto II, basc theores of three methods are brefly revewed. I secto III, we apply three methods to the classfcato of remote sesg mage respectvely ad gve the comparatve results. Secto IV s the aalyss of the three methods whch advatage ad dsadvatage of three methods are aalyzed ad some suggesto for future research are also preseted. The last secto s cocluso. Bref Revew of Neural etwork, Rough Sets ad Support Vector Mache.1 Neural etwork Sce eural etwork researches revved 1980s, substatal progress has bee acheved applcato as well as theory. Neural etworks have bee wdely appled patter recogto, cotrol optmzato, predctg maagemet ad so o. I the feld of artfcal tellgece, eural etworks have bee combed wth geetc algorthm, fuzzy sets [9]. Classfcato s a very mportat task area of formato processg ad kowledge dscovery. Classfcato of eural etwork s a supervsed trag algorthm. It has a hgh tolerace capablty ad self-orgazato performace. Lots of work have bee doe ad large umbers of lteratures have bee troduced the feld of eural etworks. Presetly, most methods of eural etwork remote sesg mage classfcato use BP learg algorthm for supervsed learg classfcato. BP etwork s a feedforward etwork whch s fact a olear crtero fucto.. Varable Precso Rough Sets Model Rough sets theory s a ew mathematcal tool to deal wth vagueess ad ucertaty data, whch ca aalyze ad deal wth varous mprecse ad complete formato. However, tradtoal rough sets are very sestve to eve small msclassfcato errors whch restrct ts applcato greatly. Hece, t s ecessary to crease the system redudace. Here, we maly troduce the VPRS model. I covetoal rough sets, uverse U s kow ad cocluso s oly sutable for obects belogg tou. It s very dffcult to satsfy the costras practce. To solve the problem, a method must be foud to geeralze coclusos obtaed from sample data to a more wde area. W.Zarko proposed VPRS to solve the problem. Let X, Y be o-empty sets fte feld. If there exst x Y for all x X, we call that X Y. It s obvously that o msclassfcato errors are allowed for the codto. A ew dea s preseted VPRS whch gve a ew measuremet method o cluso relato as follows. 1 card ( X Y / card ( X f card ( X > 0 c( X, Y = (1 0 f card ( X = 0 where card ( deote cardal umber of sets. c( X, Y deotes degree of msclassfcato set X to Y. That s to say, there are c( X, Y *100% elemets msclassfed. Obvously, X Y whe c ( X, Y = 0. Therefore, we ca gve a admssble msclassfcato error ( Accordg to the defto, there s: Y X f ad oly f c( X, Y ( the Zarko proposed deftos as follows, Suppose that U s uverse, R s dscerblty relato o U. R = {E 1, E,, E } are parttos of equvalet classes o U. -lower approxmato ( -postve rego of set X, ISBN: ISSN:

3 { E R : c( X } R X = (3 -upper approxmato( -egatve rego of set X, { E R : c( X < 1 } R X = (4 -boudary rego, { E R : < c( X < 1 } BNR X = (5 -egatve rego, { E R : c( X 1 } NEGR X = (6 Zarko gves a very mportat defto VPRS, amely qualty of classfcato. γ ( P, = card( POS( P, / card( U (7 whch POS( P, s a -postve rego o partto Q. Attrbute reducto ad optmal set of attrbute are the most mportat cocepto rough sets model. VPRS provde us two mportat crtera [8], 1. γ ( P, = γ ( RED( P,,. o attrbute ca be elmated from RED( P, wthout affectg the requremet1. There have bee may algorthms for attrbute reducto. Optmal reducto ca be derved from combed mmum cost crtero aturally f t s possble to assg a cost fucto to attrbutes. I the absece of attrbute cost fucto, two basc approaches were preseted by Zarko whch optmal reducto ca be determed accordg to the umber of attrbutes ad rules [8]..3 Support Vector Mache Support vector mache s a ew mache learg theory proposed md 1990s whch s arsed from the statstc learg theory fouded by the Vapk research groups 1960s. It has bee successfully appled to fucto smulato, patter recogto ad data classfcato..3.1 Learly Separable SVM Suppose that there are samples vectors ( x 1, y 1, ( x, y,, ( x, y belogg to two separate classes where y {+1, 1}. Our target s to buld a crtero fucto to separate the two classes. If there exsts a hyperplae w x + b = 0 whch makes, ( w x + b 1, y = 1 equvalet to ( w x + b 1, y = 1 [ ( w x + b] 1 0 (8 y We ca obta the optmzg problem as follows, 1 m w (9 The Lagrage fucto ca be defed as below, 1 L( w, b, α = ( w w (10 α { y[( w x + b] 1} = 1 where are the Lagrage multplers. (10 ca be α trasformed to ts dual problem order to mmze the equato. Accordg to Küh-Tucker codto, we ca obta the optmal classfcato fucto, f ( x = sg{( w x (11 = sg{ α y ( x x = 1 sg s the symbolc fucto..3. Learly No-separable SVM Separable crtero fucto s bult o the Eucldea dstace, that s K ( x, x = x x. For o-lear problem, sample x ca be mapped oto a hgher dmesoal space ad use the lear classfer o t. That s, we make the trasformato o x, Φ : R H. x Φ(x = ( φ1( x, φ ( x, (1 the the crtero fucto s chaged to: f ( x = sg{ w Φ( x (13 = sg{ α y Φ( x Φ( x = 1 Therefore, (11 s oly related to er product of trag samples: ( x x. Thus, oly the er product calculatos are eeded the hgher dmesoal space whch ca be realzed usg the fucto tal space. Itroducto of kerel fucto make us solve the problem o the put space wthout calculatg the o-lear mapped patters Φ explctly. Therefore, whe we costruct the classfcato fucto, we make the comparso o the put space ad make the o-lear trasformato to results. Thus eormous work wll be fshed o put space stead of o hgher dmesoal space. Thus ts classfcato fucto s obtaed, f ( x = sg{ α y K ( x x (14 = 1 Ths s called support vector mache. ISBN: ISSN:

4 Complexty of costructg SVM reles o the umber of support vector stead of dmeso of egespace. Dfferet algorthms are formed usg dfferet kerel fuctos. The kerel fuctos commo use are Polyomal fucto, Radal Bass Fucto (RBF, Mult-layer Perceptro ad so o that are show as follows, (1 Polymoal kerel: d x = [( x, x + 1] ( RBF kerel: x { } = exp γ x - x (3 Sgmod kerel: T x = tah( γ x x Θ (4 Sple kerel : 1 x = 1 + ( x x + ( x x ( x x 1 3 ( x x 6 where ( x x = m( x, x (5 Fourer kerel: N + 1 s ( x x x = x x s Image Classfcato Expermetal data are obtaed from the Ladsat mage of someplace Cha whch clude 1,,3,4,5,7 sx bads (Fg.1(a~(f. Illustrato of bads s show table 1. The mage s Obects the mage are classfed by artfcal method to fve categores: rver, house, forest, grass ad road. Gray scale s used as features for classfcato [10] mage blocks are extracted from dfferet bad mages the same place. Calculate the mea gray value μ of the block of each bads respectvely ad get the mea vector of gray value ( μ 1, μ, μ 3, μ 4, μ5, μ7 as the sample features. 10 samples are selected from each class whch 80 s for trag ad the other 40 for testg. There are total 600 data selected for the expermet. Samples are traed ad tested by usg eural etwork, rough sets ad SVM respectvely. For eural etworks, we use the ordary BP etwork (BPNN ad mprovemet BP etwork traed by Leveberg-Marquart algorthm (BPLM. For VPRS, we select the 0, 0.3, 0.45 respectvely as msclassfcato error. For support vector mache, 3 A Emprca l Compar so of Neural Network, Rough Sets ad Support Vector Mache o Remote Sesg Fg.1(a Fg.1(b Fg.1(a~(f are obtaed from bad 1,,3,4,5 ad 7 respectvely Fg.1(c Fg.1(d Fg.1(e Fg.1(f Table 1. Bads of remote sesg mage Bads Blue Gree Red Near Ifrared Ifrared Far Ifrared wavelegth(μm ISBN: ISSN:

5 RBF, sple fucto, bsple, aovasple ad sgmod are choosed as the kerel fucto of support vector mache. Because SVM maly solve the two classes problem, we smply adopt the multlevel classfcato strategy the paper [4]. The expermet s worked o the Petum IV 1.5G CPU, 51M memory PC. The results are show table ad table 3. From talbe ad table3, tme cosumed s the logest BPLM, although precso s the hghest t. chagg from 0 to 0.45, the capacty of recogto ca ot be mproved essetally because of the defect rough sets. From the sythetc performace of the three methods, we fd the SVM get the best results. I SVM, except for the sgmod kerel, all the kerel fuctos obta a better result. It spet shorter tme SVM tha rough sets. The precso s hgh ad t s eve hgher tha eural etworks whe usg sple kerel. All of them show a brght future of SVM o remote sesg Table. Result of classfcato by dfferet methods Method Rver House Forest Grass Road Neural BPNN ot coverget Network BPLM 79.1% 88.3% 88.4% 64.3% 88.% =0 55.1% 50.0% 70.7% 1.% 60.0% Rough Logc = % 61.3% 71.3% 17.1% 69.% = % 66.1% 75.3% 6.1% 77.7% RBF 95.1% 60.3% 65.5% 95.0% 51.7% sple 95.7% 80.7% 70.0% 95.3% 71.8% SVM bsple 94.7% 65.5% 65.8% 68.8% 75.1% aovasple 96.1% 80.3% 55.9% 95.1% 55.4% sgmod 60.1% 0.% 35.4% 65.3% 39.3% Method Table 3. Precso of classfcato Trag Tme(s Trag Sample Precso Test Sample Neural BPNN ot covergece Network BPLM % 8.1% = % 47.4% Rough =0.3 Logc % 54.8% = % 61.0% RBF % 73.6% sple % 8.7% SVM bsple % 74.0% aovasple % 76.6% sgmod % 44.% Note: for the rough logc, trag tme s tme of costructg the rough set rules BPNN s ot coverget the expermet. The results show defects of eural etwork remote sesg mage classfcato. Hgher dmesos of put formato wll lead to complex etwork structure ad log tme trag. I VPRS, short tme are cosumed o costructg the rough set rules. Good results are obtaed the trag samples. However, the precso of classfcato to test data are ot deal whch also accout for the lmtato of rough sets. We fd that although performace of system s mproved to some extet through mage classfcato. 4 Aalyss I the paper, we apply eural etwork, rough sets ad support vector mache to remote sesg mage classfcato respectvely. Advatages ad dsadvatages of the three methods are aalyzed detal as below. ISBN: ISSN:

6 Precso of classfcato s hgh usg eural etwork. However, the trag tme s log ad t probably exsts local mma. For mult-put system lke remote sesg mage, put vectors ca be lesse ad etwork structures be smplfed by usg rough sets. Whereas, eural etworks structures wll become very complex whe creasg the put dmesos. The meag of rough sets etwork s very clear oce the rough logcs are determed, whle the umber of hdde layer ad euros eural etworks are determed by experece. I VPRS, some ecessary redudace are remaed the process of formato reducto whch crease the at-ose performace of system ad decrease the loss of the useful formato. Furthermore, t provdes us a varable parameter for us accordg to our requremet. Although VPRS obta more mprovemet tha tradtoal rough sets remote sesg mage classfcato, t has also some dsadvatages. Frst, the msclassfcato error s ot cotuous, that s to say, the precso of VPRS s ot cotuous. Secod, selecto of optmal attrbutes stll reles o the specfc costrats. I the paper, we oly smply select them accordg to [8]. I the specfc codto, t does ot mea that s larger ad structure of etwork s more complex. Ths s related to the umber of attrbute ad logc selected. We ca ot try to mprove the redudace through smply creasg. Furthermore, besdes remote sesg mage classfcato, the umber of equvalet classes also restrct the system performace. The umber s more, the precso s hgher, but the complexty also become hgher ad trag tme become loger. The problem also eeds to be dscussed the future research. I the expermet, we fd that hgh precso ad short trag tme ca be obtaed usg SVM. Comparg wth eural etwork, SVM s more sutable for processg the complex ad hgh dmesoal data. However, there are stll may problems to be solved SVM. The performace of SVM largely depeds o the kerel. Selecto of kerel fucto lmts the applcato of SVM greatly. From the expermet secto III, the result s ot deal whe Sgmod kerel are used ad trag tme s loger whe use RBF kerel although other results are very good. Now, the research of kerel s stll at ts begg stage [11]. The classfcato usg SVM stll focus o the two classes. How to process the problem of multclasses classfcato eeds to be studed the future 5 Cocluso Neural etwork, rough sets ad support vector mache are the effectve tools o dealg wth varous mprecse ad comeplete data. There exsts essetal dfferece amog them. Except for eural etwork, rough sets ad support vector mache are seldom used the research of remote sesg mage classfcato. I the paper, Detal aalyss are descrbed whch provdes us a ew vewpot o remote sesg mage classfcato. From the expermet, we fd etwork, rough sets ad SVM all have ther advatages ad dsadvatages remote sesg mage classfcato. Therefore, how to combe the three theores ad apply them to remote sesg mage classfcato better s a mportat tedecy the later research. Ths prmary study ca be foud the lterature [1]. Refereces: [1] Heerma P D., Khazee N, Classfcato of multspectral remote sesg data usg a back propagato eural etwork, IEEE Tras o Geoscece ad Remote Sesg, Vol.30, No.1, 199, pp [] Atkso P M, Tatall A R L, Neural etworks remote sesg, It J Remote Sesg,Vol.8, No.4, 1997, pp [3] Pal S K, Mtra P, Multspectral mage segmetato usg the rough set talzed EM algorthm, IEEE Trasactos o Geoscece ad Remote Sesg, Vol.40, No.11, 00, pp [4] Azm-Sadad M R., Zekavat S A, Cloud classfcato usg support vector maches, IEEE 000 Iteratoal Proceedgs o Geoscece ad Remote Sesg Symposum, 000(, pp [5] Melga F, Bruzzoe L, Support vector maches for classfcato of hyperspectral remote-sesg mages, IEEE Iteratoal o Geoscece ad Remote Sesg Symposum, 00(1, pp [6] Mak B, Muakata T, Rule extracto from expert heurstcs: A comparatve study of rough sets wth eural etworks ad ID3, Europea Joural of Operatoal Research, Vol.136, No.1, 00, pp [7] Solares. Crsta, Saz. Aa Mara, Bayesa etwork classfers. A applcato to remote sesg mage classfcato, WSEAS ISBN: ISSN:

7 Trasactos o Systems, Vol. 4, No. 4, Aprl, 005, pp [8] Zarko W, Varable Precso Rough Set Model, Joural of Computer ad System Sceces, 1993(46, pp [9] Wag W Y, L Y H, Evolutoary Learg of BMF Fuzzy-Neural Networks Usg a Reduced-Form Geetc Algorthm, IEEE Trasactos o Systems, Ma ad Cyberetcs - Part B: Cyberetcs, 003, pp [10] Fsher P F, Vsualzato of the Re Lablty classfed Remotely Sesed Images, Photogrammetrc Egeerg ad Remote Sesg, 1994(7, pp [11] Burges C J C, Buldg Locally Ivarat Kerels, I: Proceedgs of the 1997 NIPS Workshop o Support Vector Maches, [1] Wu Zhaocog, Research o remote sesg mage classfcato usg eural etwork based o rough sets, 001 Iteratoal Coferece o Ifo-Tech ad Ifo-Net - ICII 001, 001(1, pp ISBN: ISSN: