Impact of Gradient Ascent and Boosting Algorithm in Classification

Transcription

1 Receved: Sepember 28, Impac of Graden Ascen and Boosng Algorhm n Classfcaon Syed Muzaml Basha Dharmendra Sngh Rapu 2* Vshnu Vandhan School of Compuer scence and Engneerng, Vellore Insue of Technology Unversy, Inda 2 School of Informaon Technology and Engneerng, Vellore Insue of Technology Unversy, Inda * Correspondng auhor s Emal: dharmendrasngh@v.ac.n Absrac: Boosng s he mehod used o mprove he accuracy of any learnng algorhm, whch ofen suffers from over fng problem, because of napproprae coeffcen assocaed o he daa pons. The obecve of our research s o ran he daa, such ha he weghng error of lnear classfer goes o zero and classfy he senmens accuraely. In hs paper, Graden ascen approach s used o mnmze he weghng error of senmen classfer by predcng he proper coeffcens o he daa pons n raned daase. When compared o prevous sudes on desgnng a srong classfer, our research s novel n he followng areas: Esmaon of Maxmum Lkelhood for logsc regresson usng Graden ascen and makng use of weghs of merc n undersandng he behavor of AdaBoos algorhm n classfyng he senmens. In our fndng, he frs decson sump has ranng error of 30.44%. Afer housand eraons, we observed a smooh ranson, where he classfcaon error ends o go down o 8.22% and acually says a same value. Fnally, concludng ha Boosng algorhm ouperforms Random Foress wh lesser Mean squared Tes Errors. Keywords: Graden ascen, AdaBoos, Machne learnng, Classfer.. Inroducon The dea of boosng sars from a queson, ha Kearns and Valan [] posed n 998 "can he weak classfers combned ogeher o ge a sronger classfy"? Rob Schapre [2] a year laer came up wh an algorhm called boosng ha really showed a greaer mpac on machne learnng area. Today, has become a defaul approach for deployng many compuer vson asks a ndusry. Even hough he week classfer has low bas, s no srong enough o classfy he daa pons accuraely because of neffcen coeffcen assocaed o. A Lnear classfer, akes X as an npu n he form of senences from revews and feed hrough s model, makng a predcon y. In whch, posve revew y cap s plus one, or negave revew n whch case y cap s mnus one. In hs process, I assocaes each and every word wh wegh (or) coeffcen o deermne how posvely/negavely nfluenal are hese words. Inally, we are ranng a lnear classfer by learnng he coeffcens. Consder, For Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 a Lnear classfer wh wo non-zero coeffcens have he shape of he decson boundary as lne [3], wh hree non-zero coeffcens he shape s plane and wh many non-zero coeffcens he preferred shape s hyper plane. From he ranng daa, we have seleced some feaure exracor ha gves H(X) n defnng he qualy merc whch s he lkelhood funcon, used graden ascen o opmze o ge weghs (w). In [4] he auhor had defned he qualy merc for logsc regresson. We can nerpre hese lkelhood funcon o ge exac f ranng daa, and maxmze. We dscussed abou he graden ascen algorhm ha does wh really smple updaes, and we derved a graden ascen algorhm [5] from he scrach. Graden ascen s he echnque ha wns a lo of hose machne learnng compeons. So here s a company called Kaggle ha does a bunch of hose compeons. In whch, Boosng wns more han half of hose compeons. Boosng s an amazng echnque used n machne learnng, and s appled o any classfer as booss s qualy by combnng mulple classfers. Ths approach has amazng DOI: /es

2 Receved: Sepember 28, mpac n he machne learnng world. Boosng s ha we can sar o fork ou weak classfers, so hese are he hngs lke a smple logsc regresson, a shallow decson ree or maybe even a decson sump. And so f we look a he learnng curves assocaed wh such models, le s ake a logsc regresson model. In whch, we sar from a very smple weak classfer, Whch s no good f o he daa, resuls n hgh ranng error. Bu he ranng error can be decreased by consderng more feaures. However, he rue error decreases, and hen ncreases as you sar o over f he daa. And our goal here s o fnd knd of hs opmal rade-off beween bas and varance. ow we know he weak classfers are grea because hey have low bas bu we need somehng ha's a lle sronger n order o ge good qualy, low es error. To choose a weak classfer havng lower error, can be done n wo approaches. One approach s o add more feaures. So for example, Insead of usng polynomal feaures n logsc regresson, we can add second order polynomals, hrd order polynomals, fourh order polynomals, and so on, o avod over fng. So le's suppose ha we have a parcular se of weghs and we have mulple decson sumps, so classfers ha have provded her voe. The oher s o mprove he weghs of he daa pons usng Graden ascen mehod. In our research, we would lke o adop he second approach. When comparng o prevous sudes on hs approach, our research s novel n he followng areas: To Model a Lnear Bnary/Mul class classfer whch akes he senences from he produc revew daase and predc he senmen Y, Used a Documen Term Marx o encode a caegorcal npu, Esmaon of Maxmum Lkelhood for logsc regresson usng Graden ascen, Dscussed he effec of sep-sze on Lkelhood funcon. We fnd ha he coeffcen assgned o daa pons are assocae o weghed error. And hen we updae he weghs o reduce classfcaon error. We normalze he weghs by dvdng each value by hs oal sum of he weghs. We oped a smooh ranson where he classfcaon error ends o go down o zero and acually says a zero. Over fng behavor of Ada- Boos algorhm can be predced by he cos funcons derved n [6].Our work n hs paper s organzed as follows: In Relaed work we ams o undersand he research carred ou n predcon usng AdaBoos algorhm, In Mehodology, we elaborae he way n whch he graden ascen helps o AdaBoos algorhm. In resul, we dscussed our fndngs along wh concluson and fuure scope. 2. Relaed work The AdaBoos has proved o be a very effcen ensemble learnng algorhm, whch eravely generaes a se of dverse weak learners and combnes her oupus usng he weghed maory vong rule as he fnal decson. In [7] he auhor proposed a robus mul-class AdaBoos algorhm (Rob_MulAda) whose key ngredens conss n a nose-deecon based mul-class loss funcon and a new wegh updang scheme. Adaboos algorhm can also be used for feaure exracon. In [8] assm e al. 207 proposed a new speech feaure exracon mehod called Mel Modfed Group Delay coeffcens (MMGDCs), In whch adaboos algorhm s used o buld sraegy o make he fuson beween MMGDCs and MFCCs s beer, under nosy envronmens. whereas, In [9] he auhor has modelled capables of he AdaBoos- DT, for he applcaon of neres are evaluaed usng sascal parameers and showed ha he presened AdaBoos-DT models provdes hgh performance n predcon. In [0] he auhor made a sudy o develop an auomaed sysem o mnmze he manual nference and dagnose breas cancer wh good precson. Compared he performance of a eural ework classfer wh Adaboos for esed mages and showed hgh level of overall accuracy (98.68%) and sensvy (80.5%). Whereas, In [] he auhor has proposed Dynamc fnancal dsress predcon (DFDP) approaches, In whch Adaboos suppor vecor machne (SVM) ensemble based on me weghng, oher s Adaboos SVM nernally negraed wh me weghng (ADASVM-TW), based on error-me-based sample wegh updang funcon n he Ada-boos eraon. A boosngbased mehod of learnng a feed-forward arfcal neural nework (A) wh a sngle layer of hdden neurons and a sngle oupu neuron s presened n [2], Where, an algorhm called Boosron s descrbed ha learns a sngle-layer percepron usng Ada-Boos and decson sumps. The proposed mehod uses seres represenaon o approxmae non-lneary of acvaon funcons o learn he coeffcens of nonlnear erms wh Ada-Boos. To address class mbalance n daa, In [3] he auhor proposed a new wegh adusmen facor appled o a weghed suppor vecor machne (SVM) as a weak learner of he AdaBoos algorhm useful for he class-mbalance problem by addressng well-known ssues: overlap, small dsunc, and daa shf. Boosng allows achevng a hghly accurae, robus and fas classfcaon by combnng many relavely smple rules. In [4] he auhor make use of Adaboos algorhm o classfy Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

3 Receved: Sepember 28, Thomson Scaerng mages of he TJ-II fuson devce. Adaboos s ulzed n ranng process o esablsh he color mappng model,in [5] he auhor proposed a mnd evoluonary compuaon (MEC)- back propagaon (BP)-adaboos algorhm (Adaboos) neural nework-based color correcon algorhm for color mage collecng equpmen. To solve he classfcaon problem of he saus box n Sock rend predcon a specal feaures consrucon approach s presened n [6]. Whch s, a new ensemble mehod negraed wh he AdaBoos algorhm, probablsc suppor vecor machne (PSVM), and genec algorhm (GA) s consruced o perform he saus boxes classfcaon. In [7] he auhor addressed, Accurae and mely raffc flow forecasng applcaon, whch s crcal for he successful deploymen of nellgen ransporaon sysems. Developed a ranng samples replcaon sraegy o ran a seres of sacked auo-encoders and an adapve boosng scheme s proposed o ensemble he raned sacked auo-encoders o mprove he accuracy of raffc flow forecasng. In [8] he auhor ams a he problem of raffc accdens, an Adaboos and Conour Crcle (ACC) algorhm was developed based on a radonal Adaboos mehod and he proposed conour crcle (CC) for recognzng wheher eyes are n open sae or closed sae. In whch, Adaboos mehod s used o deec human faces and eye regons, he pxels of he pupl regon are removed by he gven grd mehod, he leas squares mehod s ulzed o f he CC of he upper eyeld, he cener and radus of he CC are exraced as he feaure vecor, and he eyes sae s recognzed accordng o he defned hreshold. In [9] he auhor used Suppor Vecor Machnes (SVM), Genec Algorhms and Parcle Swarm Opmzaon, and sldng wndow approach for parameer selecon. Appled Dscrmnan analyss (ADA) for evaluaon of fnancal nsances and dynamc formaon of bankrupcy classes. Appled correlaon-based feaure subse evaluaor dfferen possble feaure selecon applcaon are researched. Demonsraed a possbly o develop and apply an nellgen classfer based on orgnal dscrmnan analyss (ODA) mehod evaluaon and shows ha mgh perform bankrupcy denfcaon beer han orgnal model. In [20] he auhor ams o solve servce dscovery problem, Bayesan classfer brngs n o web servce dscovery framework, whch can mprove servce queryng speed. Used EM algorhm o esmae pror probably and lkelhood funcons. Concludes ha he EM Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 algorhm and Bayesan classfer suppored mehod ouperforms oher mehods n me complexy. In [2] he auhor proposed a sysem o negrae wo dfferen classfers namely SVM and Gaussan process classfer (GPC) and wo dfferen descrpors lke mul local qunary (MLQ) paerns and mul local phase quanzaon (LPQ) wh ernary codng for exure classfcaon, In whch for each descrpor hey have raned a dfferen classfer, he se of scores of each classfer by normalzng mean o zero and sandard devaon o one, hen all he score ses are combned by he sum rule. Buldng a hgh performance ensemble ha works on dfferen daases whou parameers unng. The auhor obecve n [22] s o se up an opmze soluon for he nrcae algorhmc complexy mposed on learnng he srucure of Bayesan classfers usng sophscaed algorhms. In [23] auhor presened an ear based verfcaon sysem usng a new enropy funcon (EF) o dsplay dfferen characerscs of a Lnear classfer. Consdered feaures lke Effecve Gaussan Informaon source value (EGISV) and Effecve Exponenal Informaon source value (EEISV) funcons whch are derved usng he enropy funcon. Enropy feaures are classfed usng refned scores (RS) mehod n whch scores are generaed usng he Eucldean dsance. In [24] he auhor presened a model ha can provde blockage lkelhood level and verfcaon usng unseen daa, based on prevous decson ree models. The model was developed usng he geographcal groupng of sewers and he applcaon of ensemble echnques. In [25] he auhor presened a possble enhancemen of enropy-based classfers, addressed problem caused by he class mbalance n he orgnal daase and proposed a mehod o es on synhec daa o analyse he robusness wh dfferen class proporons n conrolled envronmen. In [26] auhor derves a lnear classfer, he Gaussan Lnear Dscrmnan (GLD), ha drecly mnmzes he Bayes error for bnary classfcaon and proposed a local neghbourhood search (LS) algorhm o oban a more robus classfer f he daa s known o have a non-normal dsrbuon, Evaluaed he proposed classfers on wo arfcal and en real-world daases, and hen compared he proposed algorhm wh LDA approaches and oher lnear classfers. The GLD ouperforms he orgnal LDA procedure n erms of he classfcaon accuracy. In [27] he auhor, proposed an semsupervsed approach ha exracs and classfes opnon words from one doman called source doman and predcs opnon words of anoher DOI: /es

4 Receved: Sepember 28, doman called arge doman, combned modfed maxmum enropy and bpare graph cluserng. Made a comparson of opnon classfcaon on revews of four dfferen produc domans. And acheved classfcaon accuracy of 88.4%. In [28] he auhor have used Fuzzy Logc o classfy he senmens form Twees, Where as n [29] he auhor made a comparave sudy on predcve models. The research work carred ou by he auhor n [24] had acheved.37 Mean Squared Tes error by weghng he feaure usng Z-Value. Whereas, n our research work ams o reduce he Mean Squared Tes error abou 3% usng proposed boosng algorhm. 3. Mehodology A lnear classfer model s gong o buld a hyperplane, ha separae he posves from he negave samples. And he hyperplane s assocaed wh he score funcon. Whch s weghed combnaon of he coeffcens w 0 mulpled by he feaures ha we have as shown n he Eq. (2). In our model, Le us consder he Inpu as collecon of senences from revews as X={X[], X[],,X[d]} where d s number of revews and Predced oupu as Y conanng possble values as {-,+}. X [] s h npu of X, h (X) s h feaure belongs o X. Y sgmod Score( X )) ( Score( X) w0ho ( X)... wdhd( X) D 0 w h ( X ) w T h( X ) () (2) P( Y X, w) sgmod Score( X )) (3) ( sgmod( Score( X)) (4) T w h ( X ) e n P ( yc:{,2,..., n} X ) n n p y, X, w (5) We should maxmze he qualy merc.e, Lkelhood over all possble weghs ha assgned o all dmensons n he daase. For mul class classfcaon he Eq. (5). Logsc regresson s a specfc case of ha, where we use logsc funcon sgmod o squeeze mnus nfny o plus nfny no he nerval {0,} so we can predc probables for every class. Esmaon of Maxmum lkelhood for logsc regresson: Algorhm : To fnd Max of w ) : Sar 2: Whle no converged ( ) ( ) d 3: w w ( ) w dw 4: End whle 5: Sop ( n ( w) P( Y X, w), (6) where =umber of daa pons. The above funcon can gve larger value, wh possbly good value of w. Fndng he bes lnear classfer wh graden ascen w ) wh n varables. ( n ( wn ) Max ( P( Y X, w)) (7) wn From he Algorhm he Lkelhood funcon reach he opmum, when paral dervave of weghs s equals o zero. whle, he algorhm s repeaed wh sep sze unll paral dervave of an arbue wh respec o ndvdual weghs assgned s less han, where s assumed as olerance value. Dervave of frs erm wh respec o he frs parameer havng wegh (w 0). The paral relave of frs erm wh respec o he second parameer havng wegh (w ) all he way o he dervave las erm he paral dervave wh respec o he las parameer havng wegh (w D) as shown n Eq. (8) for d+ dmenson vecor. ow, he dervave of he lkelhood s gong o be equal o he sum over he daa pons. Therfore, consder ha each daa pon has a conrbuon o he dervave, In frs case he dervave s consdered as bg, n nex case we can consder he dervave as smaller. Bu, We are gong o sum over he daa pons of he dfference beween ermed as ndcaor funcon, ha a daa pon s plus, so ndcaor of wheher hs daa pon s posve as n Eq. (0). Graden ascen algorhm as a knd of hll clmbng algorhm. As per he algorhm, wh one parameer w, you can magne sarng a some pon, le's say w () wh eraon, and hen movng lle b uphll o he nex parameer, w (+). Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

5 Receved: Sepember 28, d dw 0 d dw ( w). d dw d ( w) w where Equaon 0. h ( X )( y P( Y X, w)) (8) (9) y s ndcaor funcon defned as n f y s y (0) 0f y s Algorhm 2: Graden ascen : Sar 2: nalze w () =0 a =. () 3: whle ( w ) 4: for =0,,,d ( w) 5: w ( ) ( ) d 6: w w ( ) w dw 7: 8: 9: End for 0: End whle : Sop where s number of eraons. In whch, we have sared from some pon say, w 0 and we're us gong o follow he graden here unl we ge o he opmal value and sopped, when he value of he graden s suffcenly small wh respec o olerance parameer. Afer every eraon made, we ravers by feaure or by coeffcen o compue he paral dervave, whch s back o coeffcen wh new sepsze. Boosng akes hs weak classfer and makes as a sronger classfer. So le's suppose ha we have a parcular se of weghs and we have mulple decson sumps, so classfers ha have provded her voe as shown n Eq. (). Algorhm 3: Boosng (Greedy learnng ensembles from daa) Sep : sar Sep 2: Consder he Tranng daa Sep 3: Learn classfer f (X) Sep 4: Predcon Y sgn f (X ) Sep 5: Learn classfer and weghs assgned each of he feaure W, f ( X ) Sep 5.: same wegh for all pons: Sep 5.2: for each = {,,T} Sep 5.2.: Learn f (X) wh daa wegh Sep 5.2.2: Compue coeffcen Sep 5.2.3: Recompued wegh Sep 5.2.4: ormalze wegh Sep 5.2.5: End for Sep6: Perform he predcon Y sgn Sep 7: Sop T w f ( X ) W ( n n F X ) sgn( W f ( X )... W f ( X )) () Where X s a daa pon, f s classfer and W s wegh of each classfer assgned based on he mporan of he feaure on whch he classfer do predcon. The predcon may be eher posve (+) (or) negave (-) represened by y cap evaluaed as n Eq. (2). So hnk abou a learnng problem where we ake some daa, we learn a classfer whch gves us some oupu, f(x), and we use o predc on some daa. Y sgn T w f ( X ) (2) We say ha Y cap of f(x). ow, hs dea of learnng from weghed daa s no us abou decson sumps. I's he resul ha mos machne learnng algorhms accep weghed daa. In Eq. (3), we descrbe he way n whch he coeffcens are compued. The exac weghs can be obaned usng graden ascen mehod for logsc regresson. W W ( ) ( ) ( X )([ y ] P( y x, W ( ) )) (3) Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

6 Receved: Sepember 28, where s wegh of each daa pon. So ha's he daa, s he weghs sar wh over bu hey ge dfferen over me. Then we compue he coeffcen wha for hs new classfy f () ha we learned. And hen we should recompue he weghs. Fnally, we say ha he predcon y ha s he sgn of he weghed combnaon of f, f2, f3, f4 weghed by hese coeffcens ha we learn from laer. Measurng error and weghed daa s very smlar o measurng error n regular daa. So, we wan o measure he weghed oal of he correc examples and he weghed oal of he msakes. So we ake our learned classfer f, and we feed ha revew. So keep addng he wegh of he msakes versus he wegh of he correc classfcaons. And use ha o measure he error. Weghed classfcaon error can be compued as n Eq. (4). Weghed_ error Toalweghof msakes Toalweghof alldaa pon s We are compung he coeffcen classfer f (X). W W E weghed_ error( f ) weghed_ error( f ) ln 2 W E W (4) of (5) Based on number of levels n senmens (2, 3, and 5) he coeffcen value of classfer changes as lsed n Table Resuls The algorhm o esmaed o Maxmum he Lkelhood for logsc regresson usng Graden ascen s mplemened and compared wh fve dfferen values ranges from 0-4 (Too Bg) o 0-6 (Too Small). The observaon made from he resuls obaned are, he dfference beween hese values are really small. So, f a classfer s us random, 's no dong anyhng meanngful. I rue ha ges an updae dependng on wheher on f ges he daa pon rgh because hs s correc or wheher f makes a msake. We are gong o ncrease he wegh of daa pons where we made msakes and we are gong o decrease he wegh of daa pons as n Eqs. (6) and (7) usng Ada Boos algorhm. W e, f f ( X ) Y (6) Table. Recompue wegh. f(x)=y W Implcaon w e Correc e 0. Decrease mporance of (x,y ) Correc 0 e 0 Keep mporan he same Msake e Increasng mporan of (x,y ) Msake 0 e 0 Keep mporan he same W e, f f ( X ) Y (7) Fnally, we normalzed he weghs of daa pons sar o over n, when we had unform weghs. Whch s hey should be normalzng weghs of he daa pons hroughou he eraons as n Eq. (8). (8) In classfyng senmens he frs decson sump has ranng error of 20.94%. So, no good a all. Afer hry eraons, we observed a smooh ranson, where he classfcaon error ends o go down o 8.67% and acually says a same value. And ha s a key nsgh of he boosng heorem. So famous AdaBoos Theorem whch underlnes all he choces made n he algorhm and really has had a lo of mpac on machne learnng. From Table 2, one can nerpre ha he coeffcen W cap of an classfer becomes zero, when he weghed_error(f ) reaches o 0.5. Fnally, our am s o generae a Graden Boosng model mplemened n R usng gbm package. The resul obaned can be nerpreed as follows: In Fg. 2 he red lne ndcaes he leas Tes error from he ranng daa consdered n our expermen. The same daase are consdered wh same parameers and came o a concluson ha proposed Boosng algorhm ouperforms Random Foress wh lesser Mean squared Tes Errors sarng wh Tes Error obaned usng 00 rees. The expermen s repeaed by consrucng rees up o 000 and obaned 8.22 Tes Error a las eraon. Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

7 Receved: Sepember 28, Table 2. Compung W cap f 0.5ln(WE) 0.33ln(WE) 0.2ln(WE) Fgure. Performance of Boosng Algorhm. The proposed Boosng algorhm s beer n assgnng bes weghs for each of he feaure n he daase usng Graden Ascen mehod. Ths weghs helps n achevng lesser classfcaon error n classfyng he polares of he senmens. In our approach, we sared wh equal weghs for all feaures exraced from he revews, learn a classfy f. We fnd s coeffcen dependng on how good s n erms of weghed error. And hen updaed he weghs o wegh msakes, mosly he weghs are assgned exacly as n Table. Fnally, normalze he weghs by dvdng each value by hs oal sum of he weghs. In consruc, he radonal Random Fores echnque error n weghng he feaures.e, Table 3. Mean squared es error umber of Trees Tes Error Fgure.2 Influence of arbue usng random fores relave nfluence of each arbue s calculaed based on Z-Score (Sascal Parameer) as ploed n Fg. 2 leads n hgher Tes error as dscussed by he auhor n [24]. Where x-axs represenng he relave source of feaures and y-axs represenng he feaures of he daase. 5. Concluson The model dscussed n our research on Lnear Bnary/Mul class classfer can ake a senences as an npu X from he produc revew daase, encoded a caegorcal ype and gves score o and predc he senmen Y. The exac weghs obaned from he Graden Ascen mehod helps he proposed Boosng algorhm n buldng he sronger classfer by combnng dfferen weak classfer havng her own polares, Whch can ouperform Random Fores algorhm wh lesser Mean squared Tes Errors (8.22) by repeang our expermen. In our Fuure work, we aemps o mplemen our proposed Boosng algorhm on Dsrbued envronmen (Hadoop) Map-Reduce obs and compare s performance wh radonal Machne Learnng algorhms. In whch, we selec a subpar of ha o us pck he magc parameers use cross-valdaon on he same. Acknowledgmens We would lke o hank our VIT Unversy for provdng us he all he research facles requremen for publshng Scopes Indexed ournals. A he same me, we would lke o hank all he Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

8 Receved: Sepember 28, revewers, who help us o mprove he qualy of our paper. References [] A. Ehrenfeuch, D. Haussler, M. Kearns, and L. Valan, A general lower bound on he number of examples needed for learnng, Informaon and Compuaon, Vol.82, o.3, pp , 989. [2] J. Seven, R.P. Anderson, and R.E. Schapre, Maxmum enropy modelng of speces geographc dsrbuons, Ecologcal modellng, Vol.90, o.3, pp , [3] L. Chulhee and D.A. Landgrebe, Decson boundary feaure exracon for nonparamerc classfcaon,ieee ransacons on sysems, man, and cybernecs, Vol.23, o.2, pp , 993. [4] G. Davd and M. Klen, Analyss of mached daa usng logsc regresson, In: Proc. of Inernaonal Conf. on Logsc regresson, ew York, pp [5]. Khanea, T. Ress, C. Kehle, T. Schule- Herbrüggen, and S.J. Glaser, Opmal conrol of coupled spn dynamcs: desgn of MR pulse sequences by graden ascen algorhms, Journal of Magnec Resonance, Vol.72, o.2, pp , [6] L. Mason, J. Baxer, L. Barle, and M.R. Frean, Boosng algorhms as graden descen, In: Proc. of Inernaonal Conf. on neural nformaon processng sysems, ew York, pp.52-58, [7] S. Bo, S. Chen, J. Wang, and H. Chen, A robus mul-class AdaBoos algorhm for mslabeled nosy daa, Knowledge-Based Sysems, Vol.02, o., pp.87-02, 206. [8]. Asba and A. Amrouche, Boosng scores fuson approach usng Fron-End Dversy and adaboos Algorhm, for speaker verfcaon, Compuers & Elecrcal Engneerng, Vol.9, o.2, pp.-2, 207. [9] S. Hamdreza and M. Arabloo, "Modelng of CO 2 solubly n MEA, DEA, TEA, and MDEA aqueous soluons usng AdaBoos-Decson Tree and Arfcal eural ework, Inernaonal Journal of Greenhouse Gas Conrol, Vol.58, o., pp , 207. [0] S. Ghada, A. Khadour, and Q. Kanafan, A and Adaboos applcaon for auomac deecon of mcrocalcfcaons n breas cancer, The Egypan Journal of Radology and uclear Medcne, Vol.47, o.4, pp , 206. [] S. Je, H. Fua, P. Chen, and H. L, Dynamc fnancal dsress predcon wh concep drf based on me weghng combned wh Adaboos suppor vecor machne ensemble, Knowledge-Based Sysems, Vol.20, o., pp.4-4, 207. [2] B. Mrza, M. Awas, and M. El-Alfy, AdaBoos-based arfcal neural nework learnng, eurocompung, Vol.248, o., pp.20-26, 207. [3] L. Won, C.H. Jun, and J.S. Lee, Insance caegorzaon by suppor vecor machnes o adus weghs n AdaBoos for mbalanced daa classfcaon, Informaon Scences, Vol.38, o., pp.92-03, 207. [4] F. Gonzalo, S. Dormdo-Cano, J. Vega, I. Marínez, L. Alfaro, and F. Marínez, Adaboos classfcaon of TJ-II Thomson Scaerng mages, Fuson Engneerng and Desgn, Vol.5, o.42, pp.-5, 207. [5] Z. Jng, Y. Yang, and J. Zhang, A MEC-BP- Adaboos neural nework-based color correcon algorhm for color mage acquson equpmens, Opk-Inernaonal Journal for Lgh and Elecron Opcs, Vol.27, o.2, pp , 206. [6] Z. Xao-dan, A. L, and R. Pan, Sock rend predcon based on a new saus box mehod and AdaBoos probablsc suppor vecor machne, Appled Sof Compung, Vol.49, o., pp , 206. [7] Z. Teng, G. Han, X. Xu, Z. Ln, C. Han, Y. Huang, and J. Qn, δ-agree AdaBoos sacked auoencoder for shor-erm raffc flow forecasng, eurocompung, Vol.247, o., pp.3-38, 207. [8] W. Me, L. Guo, and W. Chen, Blnk deecon usng Adaboos and conour crcle for fague recognon, Compuers & Elecrcal Engneerng, Vol.58, o., pp , 207. [9] M. Pac, L. ann, and S. Sever, An ensemble of classfers based on dfferen exure descrpors for exure classfcaon, Journal of Kng Saud Unversy - Scence, Vol.25, o.7 pp , 203. [20] Y. Peng, Servce Dscovery Framework Suppored by EM Algorhm and Bayesan Classfer, Physcs Proceda, Vol.33, o.7, pp.206-2, 202. [2] M. Pac, L. ann, and S. Sever, An ensemble of classfers based on dfferen exure descrpors for exure classfcaon, Journal of Kng Saud Unversy - Scence, Vol.25, o.7 pp , 203. Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es

9 Receved: Sepember 28, [22] H. Bouhamed, A. Masmoud, and A. Reba, Bayesan Classfer Srucure-learnng Usng Several General Algorhms, Proceda Compuer Scence, Vol.46, o.3, pp , 205. [23] M. Bansal and M.Hanmandlu, A new enropy funcon for feaure exracon wh he refned scores as a classfer for he unconsraned ear verfcaon, Journal of Elecrcal Sysems and Informaon Technology, Vol.2, o.0, pp.74-8, 206. [24] J. Baley, E. Harrs, E. Keedwell, S. Dordevc, and Z. Kapelan, Developng Decson Tree Models o Creae a Predcve Blockage Lkelhood Model for Real-World Wasewaer eworks, Proceda Engneerng, Vol.54, o., pp , 206. [25] A. Krshners, S. Parshun, and H. Gorsks, Enropy-Based Classfer Enhancemen o Handle Imbalanced Class Problem, Proceda Compuer Scence, Vol.04, o., pp , 207. [26] KS. Gyamf, J. Brusey, A. Hun, and E. Gaura, Lnear classfer desgn under heeroscedascy n Lnear Dscrmnan Analyss, Exper Sysems wh Applcaons, Vol.79, o.3, pp.44-52, 207. [27] S.J. Deshmukh and A.K. Trpahy, Enropy based classfer for cross-doman opnon mnng, Appled Compung and Informacs, Vol.53, o.2, pp.2-220, 207. [28] S.M. Basha, Y. Zhennng, D.S. Rapu,.Ch.S.. Iyengar, and D.R. Cayles, Weghed Fuzzy Rule Based Senmen Predcon Analyss on Twees, Inernaonal Journal of Grd and Dsrbued Compung, Vol.0, o.6, pp.4-54, 207. [29] S.M. Basha, Y. Zhennng, D.S. Rapu, R.D. Cayles, and. Ch. S. Iyengar, Comparave Sudy on Performance Analyss of Tme Seres Predcve Models, Inernaonal Journal of Grd and Dsrbued Compung, Vol.0, o.8, pp.37-48, 207. Inernaonal Journal of Inellgen Engneerng and Sysems, Vol., o., 208 DOI: /es