Improved Classification Based on Predictive Association Rules

Transcription

1 Proceedngs of he 009 IEEE Inernaonal Conference on Sysems, Man, and Cybernecs San Anono, TX, USA - Ocober 009 Improved Classfcaon Based on Predcve Assocaon Rules Zhxn Hao, Xuan Wang, Ln Yao, Yaoyun Zhang Inellgence Compung Research Cener, Shenzhen Graduae School Harbn Insue of Technology Shenzhen, Chna hzx984@gmal.com, wangxuan@nsun.h.edu.cn, yaoln@h.edu.cn, xaon5@gmal.com Absrac Classfcaon based on predcve assocaon rules (CPAR) s a knd of assocaon classfcaon mehods whch combnes he advanages of boh assocave classfcaon and radonal rule-based classfcaon. For rule generaon, CPAR s more effcen han radonal rule-based classfcaon because much repeaed calculaon s avoded and mulple lerals can be seleced o generae mulple rules smulaneously. Despe hese advanages above n rule generaon, he predcon processes have he weaknesses of class rule dsrbuon mbalance and nerrupon of ncorrec class rules. Furher, s useless o nsances sasfyng no rules. To ackle hese problems, hs paper presens Class Weghng Adjusmen, Cener Vecor-based Preclassfcaon and Pos-processng wh Suppor Vecor Machne. Expermens on Chnese ex classfcaon corpus TanCorp show ha our algorhm acheves an average mprovemen of 5.9% on F score compared wh CPAR. Keywords CPAR, Class Weghng Adjusmen, Vecor-based Pre-classfcaon, Suppor Vecor Machne Cener I. INTRODUCTION Classfcaon mehods based on assocaon rules have grasped much more aenon of researchers nowadays. Accordng o several repors, hgher classfcaon accuracy has been acheved han radonal classfcaon approaches e.g. C4.5, FOIL and RIPPER []. I can make predcon from example wh unknown labels by mnng assocaon rules n ranng corpus. In general, hree seps are conaned n assocaon rules classfcaon []: ().Rule Generaon: exrac canddae rules n he ranng corpus whch sasfy mnmum suppor predefned wh some daa mnng algorhm. (). Rule Selecon: evaluae all canddae rules, only he rules sasfyng confdence predefned can be reaned. (3). Classfcaon: choose he bes rule from classfer for predcon. Mos radonal assocaon classfcaon algorhms only have concerned abou selecng lerals hrough frequenly ems. The arbues of a rule only depends on he mnmum suppor predefned and her classfcaon ables are gnored. The hreshold of mnmum suppor plays a very mporan role. However, hese approaches are me-consumng and s dffcul o selec hgh qualy rules. CPAR has combned he advanages of boh assocave classfcaon and radonal rule-based classfcaon. I employs Predcve Rule Mnng algorhm o generae rules drecly from ranng corpus. In hs way, CPAR generaes much smaller se of hgh-qualy predcve rules and generaes all he rules accordng o he rules generaed before o avod redundan rule generaon. For selecng lerals, nsead of selecng only he bes leral for he rule, CPAR selecs all he lerals close o he bes one hus generaes more useful rules [3]. The rule generaon of CPAR makes a more effcen assocave classfcaon algorhm. Bu, here are manly hree flaws n he processes of rule evaluaon and classfcaon. For each of hem a soluon s proposed as follows: () The number of rules of each class has mbalanced dsrbuon. I may range from several dozen o several housand. Therefore, an example s predced more easly o he class wh more rules han he fewer ones. So Class Weghng Adjusmen s proposed whch wll balance he nensy of classfcaon rules by adjusng wegh facor eravely, he effec of srong classes and weak ones wll be more balanced. () In he sage of classfcaon, each class s reaed unformly for examples whch wll ncrease he probably of ms-classfcaon. Before classfcaon, smlary compuaon beween example and he cener vecor of each class s proposed. We only load he rules of a class whose smlary of cener vecor o he example s greaer han average smlary. Smlary compuaon s based on vecor space model. (3) CPAR s useless for he nsances sasfyng no rules. Expermenal daa of TanCorp shows ha here are abou 4% esng nsances sasfy no rules. Posprocessng wh Suppor Vecor Machne (SVM) s proposed o predc hese examples [4]. The paper s organzed as follows. Secon descrbes he general deas of classfcaon based on predcve assocaon rules. In Secon 3, we dscuss our Improved CPAR (ICPAR) algorhm n deal. Secon 4 presens he expermenal resul of ICPAR and make comparsons wh CPAR, KNN, and SVM. The concludng remarks are presened n Secon 5. II. CLASSIFICATION BASED ON PREDICTIVE ASSOCIATION RULES CPAR s an assocaon classfcaon algorhm whch combnes he advanages of boh assocave classfcaon and radonal rule-based classfcaon. A greedy algorhm Predcve Rule Mnng s used o generae rules drecly from ranng daa. In order o avod over fng, CPAR employs expeced accuracy o evaluae rule and bes k rules of each class are used for predcon. There are hree man seps n CPAR whch are gven as follows [3]: /09/$ IEEE 9

2 A. Rule Generaon Some defnons are frs gven as follows: Leral: A leral p s an arbue-value par n he form of (A, v ), where A s an arbue and v s a value. If and only f = v, uple wll sasfy leral p, s he value of h arbue of. In rule generaon algorhm of CPAR, every documen s represened by a se of erms and here s no wegh for each erm. So we make no dfference n leral and erm n hs paper. Rule: A rule r s n he form of p p... pl c, p s a leral and c s a class label. When a uple sasfes all he lerals of rule r, we say ha uple sasfes rule r. The uple can be predced o class c. Gven a rule r:p r C r. Posve examples are examples sasfy P r and belongng o class C r. Negave examples are examples sasfy P r and no belongng o class C r. The gan of leral p s defned as (). P and N are ses of posve and negave examples respecvely. P and N are number of examples n respecve ses P and N. Afer leral p s added o curren rule r, here wll be P * posve and N * negave examples sasfyng he new rule s body. * * P P ga n( p) = P (log log ) * * P + N P + N For an eraon of rule generaon of CPAR, leral wh bes gan s seleced o curren r. A rule s generaed unl he gan value of he leral s smaller han predefned hreshold. The mos me-consumng par of radonal FOIL algorhm les n compung gan of every leral when selecng he bes leral [5]. CPAR uses PNArray o enhance he effcency of he compuaons. If an example s correcly covered by a rule, CPAR decreases s wegh nsead of removng from Fol and CPAR ges more rules. Moreover, CPAR selecs no only he bes leral bu also he lerals geng gan closer o he bes one. I can ge mulple rules smulaneously and concree rule generaon algorhm can be found n [3]. B. Rule Evaluaon Laplace expeced error esmaon s used n CPAR o esmae he exceped accuracy of rules whch belongng o class c. () LaplaceAccuracy = ( n + )/( n + k) () In equaon (), k s he oal class number, no s he number of examples whch sasfy he rule s body and n whch here are nc examples belongng o class c. C. Classfcaon For each example, CPAR selec he bes k rules of each class whch sasfyng. Then compue he average accuracy of c o he k rules and predc he example o he class wh he hghes average accuracy. III. IMPROVED CPAR To overcome he weaknesses of CPAR descrbed n Secon, Class Weghng Adjusmen, Cener Vecor-based Preclassfcaon and Pos-processng wh SVM are proposed. The res of hs secon wll dscuss each approach n deal. A. Class Weghng Adjusmen The processng of rule evaluaon n CPAR only consders abou he expeced accuracy of each rule whou akng n accoun he wegh of every class. Table I lss he number of rules n each class generaed by rule generaon algorhm of CPAR on ranng corpus of TanCorp. Furher, Table I depcs ha he class rule numbers are unbalanced. Therefore, he example wll be easly classfed no class wh more rule number. TABLE I. RULE NUMBER OF EACH CLASS Recrumen Spors Healh Regon Eneranmen Esae Educaon Auomoble Compuer Technque Ar Economc In order o avod hs, a Class Weghng Adjusmen algorhm s used whch has been presened n [6]. Some defnons are gven as follows. Class Rule Inensy: nensy of class c s defned as (3). ρ ( c ) = ε ( c ) + ε ( c ) (0 ρ ) ε s he probably of examples wrongly classfed no c. ε s he probably of examples correcly classfed no c. The sronger class rule nensy s, he larger ρ value of he class wll be. Class Wegh Vecor: W = [ w, w,..., wm] where m ( w n) = n. W s a wegh vecor before he frs mes = weghng adjusmen. w s he wegh of class c and n s he number of rules of class c. The oal number of rules s n. Inally, wegh vecor s se o [,,...,]. For sngle eraon, he weghng facor for every class s defned as (4). New class wegh + of class c wll be compued as (5). w (3) α ( c ) = ρ ( c ) (4) w + = w a ( c ) (5) 9

3 Because new class wegh mus sasfy he m condon + ( w n ) = n, so he normalze expresson of = w + s compued as (6). + w α ( c) n w = (6) Z m Z = w α( c) n (7) = The algorhm repeas eraons unl all he classes rule nensy are equally same or reach he eraon me predeermned. Furher more, he Class Weghng Adjusmen algorhm s presened n Fg.. Inpu: he rules of each class and he ran se T Oupu: class wegh of each class Procedure Class Weghng Adjusmen Algorhm se he wegh of each class o n classwegh n oal rule number of each class 3 eraon me 0 4 whle ++ < eraon me predefned 5 make predcon wh classfcaon algorhm of 6 CPAR for T and record e, e of each class 7 Z0 8 for each class c n all classes 9 p(c) e(c) + e(c) 0 a(c) - p(c) classwegh(c) *= a(c) Z += classwegh(c) * class_rule_number(c) 3 end 4 for each class c n all classes 5 classwegh(c) = classwegh(c) * n / Z 6 end 7 end 8 reurn classwegh Fgure. Class Wegh Algorhm B. Cener Vecor-Based Pre-classfcaon For predcon, he classfcaon n CPAR drecly loads he rules of each class. Classfcaon resul wll be easly affeced by ncorrec class. We can use a sasc for an example belongng o a ceran class o es he nference.e. he smlary beween he example and he class s larger han average smlary of he example wh all classes. There are 97.43% documens n TanCorp sasfyng hs nference. Before loadng class rules, we can compue he smlary beween example wh each class and average smlary hrough all classes. Only he rules of class whose smlary wh example s larger han average smlary are used. Ths mehod can reduce he probably of ncorrec class nerrupon. Durng smlary compuaon, Vecor Space Model [7] s used for documen represenaon. A documen d s made up by a se of feaure ems (,,..., m) and he weghs correspond o hem ( w, w,..., w m). There are several approaches for wegh compuaon; TF-IDF s one of hem. In equaon (8), TF k s he frequency of k urnng up n documen d. There are N documens n ranng corpus. The number of documens whch conan feaure em k s denoed by N. k N wk = TFIDF( k ) = TFk *log (8) N Feaure selecon s a echnology for feaure dmensonal reducon. I can smplfy compuaon and avod over fng [8]. Ch-square sascs s used here. I can measure correlaon degree of erm and class c. Ch-square sascs assumes ha and c have a known dsrbuon. Ths dsrbuon s denoed by χ -dsrbuon. The correlaon degree of and c wll be hgher when hey have a bgger χ value. The formula of χ s shown n (9). N ( AD CB) χ (, c) = ( A+ C)( B+ D)( A+ B)( C+ D) N s he oal documen number n ranng corpus, A s he number of documens whch belong o class c and conan. B s he number of documens no belongng o c bu conanng. C s he number of documens belongng o c whou. D s he number of documens no belongng o c and whou word. The Ch-square value for erm can be compued as descrbed n (0). M s he oal class number. Afer feaure selecon, he erms wll be removed whose Ch-square values are smaller han hreshold predefned. Moreover, he erms whose Chsquare values are larger han hreshold wll be reaned as documen feaure. χ M max = k (9) () = max χ (, c ) (0) Up ll now, he represenaon can be gven o any documen. For a class, we can sum all he documens VSM belongng o and compue he average cener vecor of he class. Towards documen d and cener vecor c j of class j, smlary beween hem can be compued as explaned n (). sm( d, c ) = j M w k jk k = M M wk w jk k= k= w () C. Pos-Processng wh SVM Afer classfcaon wh he class rules, here may sll have some es examples sasfyng no rule. Table II shows he probably of hs knd of documens n es corpus of TanCorp. CPAR s no applcable for hese documens. Under hs condon, Pos-processng wh SVM s proposed for he purpose of classfcaon. 93

4 TABLE II. PROBABILITY OF EXAMPLES BELONGING TO EACH CLASS WITH NO RULE SATISFYING Recrumen Spors Healh Regon Eneranmen Esae.5%.%.9% 0.7% 7.5% Educaon Auomoble Compuer Technque Ar Economc 6.8%.7% 3.% 8.7% 0.9%.5% SVM s a paern recognon approach based on sascal learnng heory frsly proposed by Vapnk e al n 995[9]. I s a unversal learnng approach developed from heory of VC- Dmenson and dea of srucural rsk mnmzaon. In order o ge greaes generalzaon ably, SVM fnds an opmal radeoff beween he complexy of he model and learnng ably accordng o lmed sample nformaon. The basc hough of SVM s descrbed as follows. Suppose some gven daa pons belong o one of he wo classes and s o decde ha o whch class a new daa pon mgh belong. In SVM, a daa pon s vewed as a p- dmensonal vecor and wha needs o be found s ha wheher here s a (p-)-dmensonal hyperplane whch can separae hem. Moreover, he neares dsance beween a pon n one separang hyperplane and a pon n he oher separang hyperplane s maxmzed. I s known as maxmum-margn hyperplane. Suppose here are N daa pons, every daa pon can be n wren as ( x, y ) ( =,,..., N x R y {,} ). Any hyperplane can be wren as wx b = 0. In model ranng of SVM, parameer w and b s chosen o maxmze he margn followng he consran for each, y( wx + b) ( y = ) and y( wx + b) ( y = ).The wo equaons can be generalze as (). y ( wx + b), =,,..., N () Accordng o he rule of geomery, he dsance beween hese wo separang hyperplanes s / w. Maxmzng s equally o mnmze w /. The separang hyperplane sasfyng () and mnmzng w / s called opmal separang hyperplane. Usng Lagrange Mulpler can ge an equvalen problem o maxmze (3). In opmal separang hyperplane, approprae kernel funcon K ( x, xj ) can be seleced o complee lnear classfcaon afer some non-lnear ransformaon. Correspondng classfcaon funcon s gven * below n (4). b s he hreshold of classfcaon. If f( x ) > 0, x wll be predced o he class, oherwse wll no. I has been recognzed ha SVM worked well for ex classfcaon. There are manly four reasons of usng SVM for pos-processng of CPAR [0]. ().SVM doesn depend on he number of feaures because of s applcaon of over fng proecon. Large feaure spaces can be handled n SVM. (). There are only few rrelevan feaures n ex classfcaon. Feaure selecon may resul n a loss of nformaon and SVM can combne many feaures whch are used for classfcaon. (3). The documen vecor of ex classfcaon only conans few ems of value no equal o zero. SVM s well sued for problems wh sparse documen vecor. (4). Mos ex classfcaon problems are lnearly separable and SVM s suable for fndng such lnear separaor. In addon, ICPAR algorhm s defned n Fg.. Inpu: rule se generaed by rule generaon algorhm of CPAR Cener Vecor of each class SVM model raned by ranng corpus Tesng corpus Oupu: class labels predced for each example n esng corpus Procedure ICPAR Algorhm compue he wegh of each class usng class weghng adjusmen for each example n esng corpus 3 vvsm of 4 avg_smaverage smlary of v wh all he classes cener vecor 5 max_excep0 max_excep_label- 6 for each class c n all classes 7 f he smlary of c s cener vecor wh v s larger han avg_sm 8 ruleche rules of c 9 sasfy_ruleche rules of rulec sasfyng example 0 sumcsum up he bes k rules expeced accuracy n sasfy_rulec expeccsumc*class wegh of c f expecc>max_expec 3 max_expecexpecc 4 max_expec_labelc 5 end 6 f max_expec>0 7 predc example no class max_expec_label 8 else 9 predc example wh pos-processng svm model 0 end Fgure. Algorhm of ICPAR IV. EXPERIMENTAL RESULTS Our expermens are conduced on PC wh Wndows XP operang sysem, prmary frequency 3.0G cpu,.0g prmary memory. Chnese ex classfcaon corpus TanCorp V.0 suppled by Insue of Compung Technology, Chnese Academy of Scence s used for expermen []. There are classes n TanCorp and he documen number of each class s lsed n Table III. In order o avod over fng, 5-fold cross valdaon s employed. TABLE III. DOCUMENT NUMBER OF EACH CLASS L = a = aa y y ( x, x ) D j j j, j (3) Recrumen Spors Healh Regon Eneranmen Esae n * * ( ) = sgn( (, ) + ) = f x a yk x x b (4) Educaon Auomoble Compuer Technque Ar Economc In rule generaon of CPAR, we se hreshold of posve wegh facorδ o 0.04, hreshold of leral gan o 0.6, wegh 94

5 decay facorα o /3 and mn_gan_rao (he hreshold rao of a leral s gan o he bes leral s gan, he leral wll be seleced when he rao of s gan o he bes leral s gan s larger han ) o 0.8. CPAR selec he bes k rules of each class for classfcaon, Fg.3 shows accuracy of dfferen k n range from o 0. When k equals o 5, we go a hghes accuracy. So he bes 5 rules are used n CPAR. In Class Weghng Adjusmen algorhm, he eraon me s se o 5. In addon, our SVM mplemenaon s based on LbSVM []. Fgure 3. Dfferen accuracy of dfferen k value Precson and Recall are wdely used n sascal classfcaons. To a ceran class, A denoes he number of examples classfed no he class correcly. B denoes he example number msclassfed o he class. C denoes he number of examples whch belong o he class bu classfed o anoher class. The precson and recall are defned as follow. Precson= A/( A+ B) f( A+ B) > 0; oherwse Precson= Recall= A/( A+ C) f( A+ C) > 0; oherwse Recall= A popular measure ha combnes Precson and Recall s F- score whch s gven n (5). β s a parameer whch can adjus weghs for precson and recall. When=, s known as F measure. The overall performance on he whole daa se s evaluaed by macro average whch s he arhmec average of each class performance ndex. From Fg.4 can be seen ha all of CWA, CVP and PSVM have ncreased he F values of mos classes. Through daa analyss, durng each mprovemen for every class among hem, he classfcaon accuracy of examples belongng o he class s ncreased. Moreover, he probably of examples belongng o oher classes msclassfed o he class s decreased. Therefore, boh recall and precson of he class are ncreased. So he F value of he class s ncreased. Bu excepons do exs. Class label 0 represens he class Recrumen, whch has he larges number of rules. Therefore, examples belongng o oher classes can be easly msclassfed o. Tha s why he precson of Recrumen s lowes of all classes. Even hough CWA can declne hs nfluence, can also reduce he recall of he class because he larger number of rules a class have, he lower wegh of he class wll be. So he classfcaon nensy of he class s weakened by CWA. Ths causes some examples belongng o class Recrumen are msclassfed o oher classes. Therefore, from Fg.4 we can see ha he F value of he class Recrumen s reduced by addng CWA. Moreover, he F value of class Recrumen s lowes n all classes. From error analyss we fnd ha here are abou 0.4% examples of Recrumen msclassfed o he class Compuer and also some ohers msclassfed o he class Educaon or Economc. Ths s he flaw of he classfcaon algorhm based on rules. For nsance, s reasonable ha an example belongng o Recrumen conans he words of nerne, elecronc and company. Bu he hree word can jus make a rule of class Compuer.e. nerneelecronccompanycompuer. When here are enough rules of hs knd, he example s msclassfed o class Compuer. Moreover, reduces he precson of Class Compuer. Clearly, he example of Recrumen has correlaon wh example of Compuer, Educaon or Economc. Therefore, ms-classfcaon s unavodable n hs suaon. In order o mprove classfcaon performance, a modfed rule generaon algorhm s needed whch s our fuure research drecon. Fβ β precson recall β precson recall = ( + ) ( ) /( + ) (5) F= ( precson recall )/( precson + recall ) (6) Table IV llumnaes ha he F-Meassue of macro average s ncreased by addng Class Weghng Adjusmen (CWA), Cener Vecor-based Pre-classfcaon (CVP) and Posprocessng wh SVM (PSVM) o CPAR n sequence. Furher, Table IV depcs our mprovemens owards CPAR. TABLE IV. MACRO_AVERAGE PERFORMANCE INDEX OF DIFFERENT IMPROVEMENTS CPAR CWA added CVP added PSVM added Macro_precso 86.53% 86.93% 86.79% 9.85% n Macro_recall 85.30% 87.5% 89.3% 9.6% Macro_F 85.% 86.58% 87.6% 9.3% Fgure 4. F comparson on each class addng each mprvemen o CPAR Tex classfcaon approaches KNN [3], SVM [] and CPAR [3] are used o compare wh ICPAR. Fgs. 5, 6 and 7 gven below show he comparsons of Precson, Recall and F n four dfferen approaches on each class respecvely. We can see ha ICPAR gves he hghes average performance n each compeon. 95

6 Fgure 5. Precson comparson n dfferen approach on each class Fgure 6. Recall comparson n dfferen approach on each class Fgure 7. F comparson n dfferen approaches on each class Table V shows he resuls of KNN, SVM, CPAR and ICPAR. Furher, Table V depcs ha ICPAR has he hghes classfcaon performance n four classfcaon approaches. We can also conclude ha he classfcaon performance of CPAR wh SVM s more effcen han CPAR or SVM alone. Because of he weakness of rule based classfcaon, CPAR s nadequae o examples sasfyng no rule. Based on sascal learnng heory, SVM can wegh each sngle feaure erm n he process of model ranng. In hs regard, SVM s superor o CPAR n generalzaon ably for new nsances. The negraon of CPAR and SVM n dfferen ways o ge beer classfcaon performance wll be our fuure research drecon. TABLE V. COMPARISON OF DIFFERE NT MACRO_AVERAGE PERFORMANCE INDEX Knn SVM CPAR ICPAR Macro_precson 8.5% 87.56% 86.53% 9.85% Macro_recall 74.% 84.4% 85.30% 9.6% Macro_F 74.80% 85.7% 85.% 9.3% V. CONCLUSIONS In hs paper, we have presened ha ICPAR acheved a hgher accuracy han CPAR. Furher, he proposed ICPAR has he followng dsngushed feaures: () I adjuss he wegh of each class usng Class Weghng Adjusmen algorhm whch balances he classfyng ably of each class. () Insead of loadng he rules of all classes, Cener Vecor-based Preclassfcaon seleced and loaded only hose rules of classes havng hgh level of concern wh he example. Therefore, he probably of ncorrec classfyng for he example has been reduced. (3) Pos-processng wh SVM has been used for examples sasfyng no rules. There s no way o classfy hs knd of examples n CPAR. The proposed algorhm has been llusraed ha s effcen. The mprovemen n exraced rules qualy by combnng he characerscs of Chnese ex, as well as he elmnaon of he rules whch may nerrup he correc classfcaon resul, wll be our fuure research drecon. ACKNOWLEDGMENT Ths work s suppored by he Naonal Hgh-ech R&D Program of Chna (863 Program, No. 007AA0Z94). And apprecae o Insue of Compung Technology, Chnese Academy of Scence for he Chnese ex classfcaon corpus. REFERENCES [] W. L, J. Han, and J. Pe. CMAR: Accurae and effcen classfcaon based on mulple class-assocaon rules. In ICDM'0, pp. 369{376, San Jose, CA, Nov. 00. [] B. Lu, W. Hsu, and Y. Ma. Inegrang classfcaon and assocaon rule mnng. In KDD'98, pp , New York, NY, Aug [3] Yn, X. and Han, J.: CPAR: Classfcaon Based on Predcve Assocaon Rules. Proc SIAM In Conf on Daa Mnng (SDM 03), 003, [4] Renne, J.D.M.[Jason D. M.], Rfkn, R.[Ryan], Improvng Mulclass Tex Classfcaon wh he Suppor Vecor Machne, MIT AIM-00-06, Ocober 00. [5] J. R. Qunlan and R. M. Cameron-Jones. FOIL: A mderm repor. In Proc. 993 European Conf. Machne Learnng, pp. 3{0, Venna, Ausra, 993. [6] Chen and Hu, Tex Assocaon Caegorzaon Based on Self-Adapve Weghng. Journal of Chnese Compuer Sysems. Vol.8, No.. [7] Lu Shao-hu e al. An Approach of Mul-herarchy Tex Classfcaon Based on Vecor Space Model. Journal of Chnese Informaon Processng. Vol.6, No.3. [8] Yang Y M, Pedersen J. A comparave sudy on feaure selecon n ex caegorzaon. In: Proceedngs of he 4h Inernaonal Conference on Machne Learnng (ICML97). San Francsco, USA: Morgan Kaufmann Publshers, [9] C. Cores and V. Vapnk, Suppor-Vecor Neworks, Machne Learnng, 0(3): [0] Joachms T. Tex caegorzaon wh suppor vecor machnes[ C]. Proc of European Conference on Machne Learnng(ECML). [ S. l.] : [ s. n. ],998. [] Songbo Tan e al. A Novel Refnemen Approach for Tex Caegorzaon. ACM CIKM 005. [] Chang C-C ; C-J Ln. C.-C. Chang and C.-J. Ln, LIBSVM: A lbrary for suppor vecor machnes 00. Sofware avalable a <u> hp:// [3] Oh-Woog Kwon, Jong-Hyoek Lee, Web page classfcaon based on k-neares Neghbor approach. 96