Research Article An Optimized Classification Algorithm by Neural Network Ensemble Based on PLS and OLS

Transcription

1 Mathematical Problems i Egieerig, Article ID , 8 pages Research Article A Optimized Classificatio Algorithm by Neural Network Esemble Based o PLS ad OLS Weikua Jia, 1 Dea Zhao, 1 Yuyag Tag, 1 Chali Hu, 1 ad Yuya Zhao 2 1 School of Electrical ad Iformatio Egieerig, Jiagsu Uiversity, Zhejiag , Chia 2 Chagzhou College of Iformatio Techology, Chagzhou , Chia Correspodece should be addressed to Dea Zhao; dazhao@ujseduc Received 19 May 2014; Accepted 30 Jue 2014; Published 20 July 2014 Academic Editor: Shifei Dig Copyright 2014 Weikua Jia et al This is a ope access article distributed uder the Creative Commos Attributio Licese, which permits urestricted use, distributio, ad reproductio i ay medium, provided the origial work is properly cited Usig the eural etwork to classify the data which has higher dimesio ad fewer samples meas overmuch feature iputs ifluece the structure desig of eural etwork ad fewer samples will geerate icomplete or overfittig pheomeo durig the eural etwork traiig All of the above will restrict the recogitio precisio obviously It is eve better to use eural etwork to classify ad, therefore, propose a eural etwork esemble optimized classificatio algorithm based o PLS ad OLS i this paper The ew algorithm takes some advatages of partial least squares (PLS) algorithm to reduce the feature dimesio of small sample data, which obtais the low-dimesioal ad stroger illustrative data; usig ordiary least squares (OLS) theory determies the weights of each eural etwork i esemble learig system Feature dimesio reductio is applied to simplify the eural etwork s structure ad improve the operatio efficiecy; esemble learig ca compesate for the iformatio loss caused by the dimesio reductio; o the other had, it improves the recogitio precisio of classificatio system Fially, through the case aalysis, the experimet results suggest that the operatig efficiecy ad recogitio precisio of ew algorithm are greatly improved, which is worthy of further promotio 1 Itroductio Neural etwork (NN) is based o the itelliget computig, which makes use of the computer etwork to imitate the biologicaleuraletworkitshowspowerfulfuctioidealigwitholiearprocessadlarge-scalecomputigthe essece of classificatio system is regarded as a iput/output system The trasformatioal relatios are maily about three aspects which icludes umerical fittig, logical reasoig, ad fuzzy trasformig; all of these ca be well expressed by eural etwork The research of eural etwork classificatio has obtaied widespread applicatio i may fields [1] However, while dealig with the small sample, it is a big challege to use eural etwork to classify Overmuch feature iputs will ifluece the structure desig of the eural etwork ad restrict the operatig efficiecy However, fewer umbers of samples easily lead to icomplete traiig or overfittig, which restricts the fial classificatio precisio Thus, i this paper, we itroduce the feature dimesio reductio ad esemble learig ito eural etwork algorithm The priciple of feature dimesio reductio theory [2, 3] is to extract most useful feature from may complex feature variables, which elimiates the ifluece of repetitio or correlatio factors It meas that reducig the feature dimesio as much as possible o the premise of solvig the problems ormally Because feature dimesio reductio will cause the loss of iformatio, but it ca t ifluece the problem solvig To reduce the iputs of the eural etwork, we will desig ad simplify structure coveietly Optimized eural etwork algorithm based o feature dimesio reductio has obtaied gratifyig achievemet i may fields [4 6] The traditioal feature extractio algorithm which depeds o the measure of variace (such as pricipal compoet aalysis, factor aalysis, etc) is hard to get ideal low dimesioal data for small sample, which is with higher dimesio ad fewer samples I this paper, we adopt partial least square (PLS) to feature dimesio reductio algorithm [7, 8],which ca take

2 2 Mathematical Problems i Egieerig the level of correlatio feature variables ad the depedet variables ito cosideratio ad obtai the relatively ideal low-dimesioal data with strog iterpretatio It has a uique advatage for dealig with small sample data Usig PLS to optimize the classificatio algorithm ad eural etwork, it has got some progress [9 11] Hase ad Salamo first proposed eural etwork esemble learig [12] They proved the performace of a series of itegrated eural etwork is better tha the best sigle eural etwork by the experimet; the geeralizatio ability ad recogitio precisio of multiple classifier itegratio system have bee improved obviously I order to improve the classificatio precisio, the method of multiple classifiers itegratio has bee cosidered Meawhile, the esemble learig reaches a climax, a series of cocepts ad algorithms have bee proposed [13 15], ad applied i may fields The history of eural etwork has more tha 70 years; hudreds of models are proposed ad differet models have their ow advatage i dealig with differet problemshowever,itisotperfectfortheatureofeural etwork I this paper, choose BP, RBF, Elma three eural etworks as subclassifier to study The itegrator desig is the key poit of esemble optimizatio algorithm, which is to determie the fial recogitio precisio of itegrated classificatio system The key of esemble learig is how to determie the weights of each subclassifier; it attracts may scholars research iterests [16 18] This study is usig ordiary least squares (OLS) priciple to esemble learig for classificatio system, through establishig the regressio equatio to determie the weights of three subclassifiers Briefadtothepoit,iviewoftheatureofsmall sample ad eural etwork, i this paper, a esemble optimized classificatio algorithm by eural etwork based o PLS ad OLS is proposed The ew algorithm by PLS dimesio reductio is to improve the operatig efficiecy ad the recogitio precisio through OLS esemble learig The ew algorithm aims at providig a high efficiecy ad precisio classificatio system Fially, through the test of two data sets, oe from the actual productio, the other oe form UCI stadard data sets, the experimetal results suggest that the ew algorithm is valid ad worthy of further popularizatio ad applicatio 2 PLS Dimesio Reductio Algorithm Partialleastsquaresisthecharacteristicdevelopmetof ordiary least squares (OLS), its basic idea is that the feature variable matrix X is reduced; at the same time, it gives cosideratio to the correlatio of the depedet variable matrix YSupposethereare feature variables, x 1,x 2,,x, p depedet variables, y 1,y 2,,y p,afterpreprocessig, matrix X is reduced ito X=TP T +E (1) Here, T is the score matrix, P is the load matrix, ad E is the residual error matrix Matrix multiplicatio of TP T ca be expressed as the sum products of score vector t i (the ith colum of matrix T) ad load vector p i (the ith colum of matrix P); the the above formulaca be writte as X= t i p T i +E,,2,, (2) Similarly, matrix Y is decomposed ito Y=UQ T +F (3) Here, U is the score matrix, Q is the load matrix, ad F is the residual error matrix Matrix multiplicatio of UQ T ca be expressed as the sum products of score vector u j (the jth colum of matrix U) adloadvectorq j (the jth colum of matrix Q); the the above formula ca be writte as Y= p j=1 u j q T j +F, j=1,2,,r (4) PLS aalysis separately extracted the scores t ad u form correspodig X ad Y; they are the liear combiatio of feature variables ad depedet variables Ad both scores satisfied the maximum load of variatio iformatio of feature variables ad the depedet variables; the covariace betwee them is the largest Establishmet of the regressio equatio is u j =b k t i (5) Here, b k is regressio coefficiet; the formula ca be expressed i matrix form as Here, B is coefficiets matrix; Y=BX (6) B=W(P T W) 1 Q T (7) Here, W is the weights matrix PLS aims to each dimesio iterative calculatio by usig each other s iformatio; each iteratio cotiuously accordig to residual iformatio of X, Y to adjust t i, u j for the secod roud extracted, util the residual matrix elemet of absolute value approximate to zero The precisio satisfied the requiremets; the the algorithm stops I the iteratio process, t i, u j ca maximize the expressio of variace of X ad Y simultaeously PLS regressio does ot eed to use all the compoets to establish the regressio equatio, which just eed to select the frot l compoets (0 l ) ad the get better regressio equatio Geerally, K-fold cross-validatio method is used to calculate predictio residual sum of squares ad determie theumberofcompoetsextracted,reachigthepurposeof dimesio reductio 3 Neural Network Esemble Optimizatio 31 Subclassifier (Idividual Neural Network) Select BP, RBF, ad Elma regressio three differet types of eural etworks

3 Mathematical Problems i Egieerig 3 Iput layer Hidde layer Output layer Iput layer Hidde layer Output layer Iput layer Hidde layer Output layer w ij wjk w ij w jk w x x u1 y 1 y 1 w Σ y1 x 1 1 x(k 1) y(k) x 2 y 2 x 2 x ym x Figure 1: The topology structures of BP, RBF, ad Elma eural etwork Σ Σ y 2 y m x w 2 u r u c1 u cr Coectig layer ym as subclassifier i this paper These three subclassifiers, to a certai extet, play a complemetary role The topology structure of three eural etworks is as show i Figure 1 BP eural etwork is deoted by subclassifier I; it adopts the error back propagatio learig A three-layer BP eural etwork ca imitate ay oliear fuctio with arbitrary precisio It has good adaptive robustess ad geeralizatio ability With the expadig scope of applicatio, the defects of BP eural etwork emerge gradually, which icludes fixed learig rate that caused log traiig time, easily trapped i local miimum, ad the overfittig illusio, hidde layer, ad its ucertai euros umbers All these above become the iheret defects of BP etwork RBF eural etwork is deoted by subclassifier II; it is a kid of feed forward model, which ca adjust the hidde layer adaptively i the traiig process accordig to the specific problems The distributio of hidde layer uits is determied by the traiig sample s capacity, category, ad distributio It ca dyamically determie the hidde layer, eve its ceter ad width; meawhile, the covergece speed is fast The biggest advatage is to use liear learig algorithm to complete the work doe by oliear learig algorithm ad maitai high precisio of oliear algorithm; it has the characteristics of the best approximatio ad global optimal It ca use the sum of local approximatio to attai the global approximatio of traiig data, so usig the sum of low-order local approximatio ca be fiished the traiig data fittig Durig the traiig process, it is easily appear over-fittig pheomeo, low learig ability, ad so o All of these isufficiet will lead the bigger predictio error, further ifluece the recogitio precisio of RBF eural etwork Elma regressio eural etwork is deoted by subclassifier III; it is a kid of feedback model that adds a cotext layer based o hidde layer of BP eural etwork The cotext layer is regarded as a delay operator; it ca delay ad store the output of hidde layer ad achieve the memory That meas the system has the ability of adaptig the timevaryig dyamic characteristics ad strog global stability Thus, Elma eural etwork optimizatio is always based o BP eural etwork, aturally; it ievitably iherits the iheret defects of BP eural etwork which will lead to the usatisfactory operatig efficiecy 32 Costruct of Neural Network Esemble Algorithm Cosider the complex patter classificatio problems; sigle classifier is usually difficult to achieve the ideal recogitio precisio ad has some of its ow deficiecies The geeralizatio ability ad recogitio precisio of multiple classifier itegrated system will be more outstadig obviously Each classifier is regarded as subclassifier of itegrated system; the mai idea of esemble learig is maily about usig may subclassifiers to solve the same problem ad itegratig the outputs of each subclassifier, fially obtaiig the results of itegrated classificatio system The purpose is to improve the ability of geeralizatio ad recogitio of the learig algorithm Neural etwork esemble is a patter classificatio system, which itegrates a series of sigle eural etworks The performace of the itegrated system is better tha ay sigle eural etwork The mai purpose of eural etwork esemble is to improve the recogitio precisio of classificatio system Obviously, determiig the weights of each subclassifier is the key of esemble algorithm The mai task of eural etwork esemble algorithm is seekig the weights of each subclassifier based o the characteristics of subclassifier ad reducig the recogitio error of itegrated classificatio system There are three subclassifiers for esemble learig, i order to deal with the small sample, adopt dimesio reductio before esemble learig ad optimizig origial data, ad the we further attempt to establish eural etwork esemble learig models Figure 2 presets the established esemble learig model Its operatig priciple cosists of dimesio reductio for small sample by PLS algorithm, gettig the low-dimesioal data as the iput of each subclassifier ad regardig the output of each subclassifier as the iput of itegrator The outputs of sub-classifiers weighted learig by itegrator, fially the system gets relatively optimal classificatio results Figure 2 presets the basic flow chart of eural etwork esemble algorithm; this study applies three subclassifiers, for example Respectively, we use three subclassifiers to recogize the pedig sample data; the cogitio results of subclassifiers I are deoted by A, the cogitio results of subclassifiers II are deoted by B, ad the cogitio results of subclassifiers III are deoted by C Three subclassifiers are idepedet of each other, so are the results Respectively, the weights of three subclassifiers are deoted by w 1, w 2,adw 3,theoutputofitegratedsystem is deoted by Y, adthevalueofy is obtaied by sum

4 4 Mathematical Problems i Egieerig Origial data BP Output 1 Subclassifier 1 w 1 PLS RBF Output 2 w 2 Low-dimesioal data Elma Subclassifier 2 Output 3 w 3 OLS itegrator Subclassifier 3 NN Output w Output result Subclassifier Figure 2: Optimized classificatio algorithm assembled with eural etwork of weighted A, B, C, adtheesemblelearigmodelis established as Y=Aw 1 +Bw 2 +Cw 3 (8) Model (8) reflects the follow discussio obviously While w 1 = 1, w 2 = 0, w 3 = 0, it meas that oly subclassifier I works, the system output results are determied by the subclassifier I While w 1 = 0, w 2 = 1, w 3 = 0,itmeas that oly subclassifier II works; the system output results are determied by the subclassifier II While w 1 = 0, w 2 = 0, w 3 = 1, it meas that oly subclassifier III works, the system output results are determied by subclassifier III The optimized itegrated system, that is, fid out the optimal weights of each sub-classifier, ad the makes the recogitio results of model (8)to achieve optimal state 33 Optimized Weights The study of eural etwork esemble learig, that is, the optimized the model (8), is to determie the value of three idividual eural etworks weights w 1, w 2, w 3 Weusethethoughtofmultipleregressiosto establish terary regressio equatio for the output of three subclassifiers ad determie the optimal estimated value of weights w 1, w 2, w 3 by OLS algorithm Suppose the recogized results of three subclassifiers are deoted by a i, b i, c i,respectively,fortheith traiig sample; the output of itegrated system is deoted by y i ad the actual value of the sample is y o i,wherei = 1,2,,Bytheactual value Y o of traiig sample, the terary regressio equatio is established as follows: Y=θ+αA+βB+γC+ε, ε N(0,σ 2 ) (9) Here, α, β, γ preset the partial regressio coefficiet; usig the method of maximum likelihood estimatio to estimate ukow parameters α, β, γ, θ, it eeds the miimum residual error value Q: Satisfy, amely, mi Q= Q= (y o i y i ) 2 (10) (y 0 i θ αa i βb i γc i ) 2 (11) If we make Q miimum, α, β, γ, θ should satisfy the followig equatio sets: Set Q α = (y 0 i θ αa i βb i γc i )a i =0, Q β = (y 0 i θ αa i βb i γc i )b i =0, Q γ = (y 0 i θ a i α βb i γc i )c i =0, Q θ = (y 0 i θ a i α βb i γc i )=0 1 a 1 b 1 c 1 1 a 2 b 2 c 2 X= [ ] [ 1 a b c ] 4, η = [ θ α [ β] [ γ ]4 1 (12) (13)

5 Mathematical Problems i Egieerig 5 The equatio set (12)traslatesitomatrixform X T Xη = X T Y o (14) Obtai the least squares estimate of coefficiet matrix η: η=(x T X) 1 X T Y o (15) By the coefficiet matrix η, get the values of regressio coefficiets α, β, γ, adthevaluesofweightsw 1,w 2,adw 3 cabefurtherobtaied: w 1 = α α+β+γ, w 2 = β α+β+γ, w 3 = γ α+β+γ (16) Whe values of w 1, w 2, w 3 as formula (16), the substitute them ito formula (8), will makes the output value Y is closer to the actual value Y o, so the residual error Q reach the miimum Cosider mi Q= (y o i y i ) 2 = (a i w 1 +b i w 2 +c i w 3 y o i ) (17) Formula (16) istheoptimalweightsofthethreesubclassifiers by traiig; model (8) achieves the most optimal ad further establish esemble optimizatio algorithm 34 Elemetary Parameters of Esemble Optimized Algorithm Tosumuptheabove,usigPLSforsmallsampledimesio reductio realized the prelimiary optimizatio of the classificatio system ad the optimized the weights of each subclassifier by OLS For the high dimesioal ad small sample problems, we may as well try to establish esemble optimized classificatio algorithm by eural etwork based o PLS ad OLS Some parameters settig of the ew algorithm are as follows (1) Data Preprocessig Data preprocessig elimiates the icommesurability by differet data idex distributio ad umerical differeces, which esures the quality of the applicatio of data from the source Usig stadardized trasformatio obtais the data which is i accordace with the distributio of N(0, 1) The stadardized trasformatio formula is x ij = (x ij x j ) (1/) (x ij x j ) 2 (18) (2) Parameters of PLS PLS feature dimesio reductio is usig K-fold cross validatio (K-CV) method to calculate predictio residual sum of squares This method ca effectively avoid overlearig or uder-fittig ad get more persuasive result (3) Parameters of BP Neural Network The umber of euros usiggao sempiricalformula [19] to determie s = (043m + 012m m + 035) 1/ (19) I the formula, s,, m, respectively, are o behalf of the umber of euros of hidde layer, iput layer, ad output layer The eural etwork traiig uses trailm (LM) algorithm, which is the combiatio of gradiet descet ad quasi-ewto algorithm Its advatage is give full play to the gradiet descet algorithm ca rapid covergece at the begiig traiig, ad the quasi-ewto algorithm ca quickly produce a ideal search directio ear the extremum Coectio weights ad threshold learig use the Leargdm algorithm (4) Set the Ceter of RBF Neural Network Set up the ceter of the RBF etwork ad choose the ceter of the basis fuctio empirically; as log as the distributio of the traiig samples ca represet the give problem, the s ceters of the uiform distributio ca be chose accordig to experiece; the distace is d; choose the width of Gaussia basis fuctio σ= d 2s (20) Select the basis fuctio by K-clusterig method; exploit the ceter of clustered class as the ceter of the basis fuctio (5) Parameters of Elma Neural Network Elma eural etwork, which is optimized by BP eural etwork, the umber of cotext layer is the same as the hidde layer Parameter set is equal to BP eural etwork 35 Steps of Esemble Optimized Algorithm The basic steps of eural etwork esemble algorithm based o PLS ad OLS are as follows Step 1 Normalize the origial data accordig to formula (18); get the characteristic variables matrix X ad depedet variable matrix Y Step 2 Respectively, extract the first pair compoet t 1, u 1 from X, Y ad make up to the maximum correlatio, respectively; establish the regressio equatio of X o t 1 ad Y o u 1 Step 3 Usig residual error matrix E ad F istead of X ad Y, repetitio Step 2, util the absolute value of the residual matrix elemets is close to zero Step 4 With K-CV method, by the priciples of crossvalidatio ad residual sum of squares to determie the umber of compoets extracted Step 5 From the perspective of iformatio feature compressio, get the compressio matrix X ad Y,asewsamples

6 6 Mathematical Problems i Egieerig Table 1: Compariso of every classifier s performace for Test 1 Network model Weights Traiig steps SSE Accuracy, % Subclassifier I (BP NN) Subclassifier II (EBF NN) Subclassifier III (Elma NN) PLS-Elma Assemblig system Step 6 Divide the ew samples ito two parts as traiig samples ad simulatio samples accordig to the eed of the problem Step 7 Set up three subclassifiers, respectively; classificatio traiig, the output of the three subclassifiers are A, B, C; Step 8 By the output of three subclassifiers, establish the terary regressio model (9) Step 9 Trasform (9) ito equatios set(12) by maximum likelihood estimatio Step 10 Get the formula (13)bysolvigequatiosset(12)ad usig OLS theory Step 11 By regressio coefficiet ad formula (16), the weight of three subclassifiers ca be obtaied Step 12 Get the optimal solutio of itegrated model (8)ad termiate the algorithm 4 Case Aalysis Respectively, use three subclassifiers, PLS-Elma eural etwork ad eural etwork esemble algorithm to test the data setadcotrastthetestresults From the follow three aspects to evaluate the performace of each algorithm (model), which icludes covergece steps, sum of squared errors ad recogitio accuracy rate Covergece steps, we test 10 times, the experimet tests 10 times, werecordthebestoce,adlistitablethesumofsquared errors, the sum of squares of the differece of predicted value ad actual value, is usually used to estimate the degree of closeess betwee recogitio value ad actual value I the circumstaces of the same accuracy simulatio, the smaller the error sum of squares is, the higher the precisio of the algorithm is Accuracy rate, the ratio of correct recogized samples ad simulatio samples, which reflects the recogize accuracy of each algorithm I order to illustrate the validity of ew algorithm better, we use two data sets for testig Oe data set is agricultural pests forecastig data, which is from the actual productio [20], ad the other data set is the ioosphere data subset of radar, which is from the UCI machie learig stadard data sets [21] 41 Test 1 This data comes from agricultural productio, usig the meteorological factor to predict the occurrece degree of wheat midge The data set icludes 60 samples from 1941 to 2000, which regards 14 feature variables (meteorological factors) as the iput of eural etwork The sigle output presets the occurrece degree of wheat midge 411 Algorithm Performace Select the last 30 samples to test amog 15 traiig samples ad 15 simulatio samples, i accordace with the characteristics of small sample By PLS dimesio reductio, the data extracts 6 features, which meas the dimesios of data from reduce 14 to 6 The simulatio test results are listed i Table Recogize Precisio of Algorithm I order to better illustrate the classificatio ability of the ew algorithm, we cotiue to do the followig test; divide the selected samples ito 25 traiig samples ad 5 simulatio samples Compared with the simulatio results ad the actual value, the test results are listed i Table 2 42 Test 2 We usig aother UCI data set to test the ew algorithm, the radar data icludes 351 samples that have 34 characteristics ad each sample is used to predict the quality of the radar, which meas 34 iputs ad 1 output 421 Performace of Each Algorithm We selected the frot 40 samples, amog 20 traiig samples ad 20 simulatio samples It is i accordace with the characteristics of small sample By PLS dimesio reductio, this data extracts 19 features; that is, the dimesios of data reduce from 34 to 19 The simulatio test results are listed i Table Every Algorithm s Recogize Precisio I order to better illustrate the classificatio ability of the ew algorithm, we cotiue to do the followig test ad divide the selected samples ito 35 traiig samples ad 5 simulatio samples Comparig the simulatio results with the actual value, the test results are listed i Table 4 43 Results Accordig to the results of Tables 1 to 4, compared amog the three subclassifiers, RBF eural etwork traiig is fastest, but the recogitio accuracy is the worst; Elma eural etwork s recogitio accuracy is the best; BP

7 Mathematical Problems i Egieerig 7 Table 2: Compariso of every classifier s accuracy for Test 1 Network model Weights Traiigsteps SSE Actual value Subclassifier I (BP NN) Subclassifier II (EBF NN) Subclassifier III (Elma NN) PLS-Elma Assemblig system Table 3: Compariso of every classifier s performace for Test 2 Network model Weights Traiig steps SSE Accuracy, % Subclassifier I (BP NN) Subclassifier II (EBF NN) Subclassifier III (Elma NN) PLS-Elma Assemblig system Table 4: Compariso of every classifier s accuracy for Test 2 Network model Weights Traiigsteps SSE Actual value Subclassifier I (BP NN) Subclassifier II (EBF NN) Subclassifier III (Elma NN) PLS-Elma Assemblig system eural etwork s traiig is the slowest; o the traiig speed ad recogitio accuracy, Elma is slightly better tha BP eural etwork Comparig PLS-Elma algorithm with traditioal Elma eural etwork, it has better traiig speed ad is slightly better i recogitio precisio It shows that the recogitio precisio of classifier is ot iflueced; o the cotrary, the operatig efficiecy has bee improved after PLS optimizatio The classificatio ability of esemble optimizatio is the best The data of Tables 1 ad 3 prove that Comparig with other traditioal eural etwork, the esemble learig algorithm with the highest recogitio accuracy rate ad miimum error, it ca suggest that the ew algorithm with the highest recogitio precisio The experimetal results reflect that the recogitio precisio of esemble algorithm is obviously higher tha ay subclassifier, to a certai degree, compesatig for the iformatio loss caused by data dimesio reductio Recogitio results meet the ideal requiremets, which meas the ew algorithm is effective 5 Coclusio ad Discussio From the two groups of experimetal results above, The optimized classificatio algorithm by PLS, algorithm, improves the traiig speed of subclassifier, ad ad the recogitio accuracy is ot reduced It shows that the ew algorithm reduces the iputs of eural etwork by PLS feature dimesio reductio, which is coveiet for desigig the etwork structure ad improvig the operatig efficiecy The recogitio accuracy rate ad test error of itegrated system have bee greatly improved, which shows that the classificatio precisio of the esemble algorithm has bee greatly improved ad higher tha ay subclassifier The purpose of eural etwork esemble is to improve the recogitio precisio of patter classificatio ad the poit of esemble learig is to determie the weights of each

8 8 Mathematical Problems i Egieerig sub,classifier effectively I this paper, usig OLS priciple to desig itegrator, establish multiple regressio model ad further determie the weight of each subclassifier I view of the small sample classificatio problem, this paper proposes a esemble optimized eural etwork classificatio algorithm based o PLS ad OLS PLS has uique advatage to reduce dimesio for small sample data, which obtais ideal low-dimesioal data By OLS algorithm, do eural etwork esemble learig, to determie the weights of each subclassifier The ew algorithm has higher operatig efficiecy ad classificatio precisio, which shows the worthess of further popularizatio ad applicatio Coflict of Iterests The authors declare that there is o coflict of iterests regardig the publicatio of this paper Ackowledgmets ThisworkissupportedbytheNatioalNaturalScieceFoudatio of Chia (os , , ad ); Priority Academic Program Developmet of Jiagsu Higher Educatio Istitutios; Major Projects i the Natioal Sciece & Techology Pillar Program durig the Twelfth Fiveyear Pla Period (o 2011BAD20B06); Major Projects i Jiagsu Provice Sciece & Techology Pillar Program Pla (Idustrial) (o BE ); The Specialized Research Fud for the Doctoral Program of Higher Educatio of Chia (o ), Ordiary Uiversity Graduate Studet Research Iovatio Projects of Jiagsu Provice (o KYLX ) Refereces [1] SFDig,WKJia,CYSu,LWZhag,adZZShi, Neural etwork research progress ad applicatios i forecast, i Advaces i Neural Networks ISNN 2008,vol5264ofLecture Notes i Computer Sciece, pp , Spriger, New York, NY, USA, 2008 [2]SFDig,HZhu,WKJia,adCYSu, Asurveyo feature extractio for patter recogitio, Artificial Itelligece Review,vol37,o3,pp ,2012 [3] FXSog,XMGao,SHLiu,adJYYag, Dimesioality reductio i statistical patter recogitio ad low loss dimesioality reductio, Chiese Joural of Computers, vol 28, o 11, pp , 2005 [4] B Garlik ad M Křiva, Idetificatio of type daily diagrams of electric cosumptio based o cluster aalysis of multidimesioal data by eural etwork, Neural Network World, vol 23, o 3, pp , 2013 [5]KRJaes,SXYag,adRRHacker, Porkfarmodour modellig usig multiple-compoet multiple-factor aalysis adeuraletworks, Applied Soft Computig Joural, vol6, o 1, pp 53 61, 2005 [6]JZhou,AHGuo,BCeller,adSSu, Faultdetectioad idetificatio spaig multiple processes by itegratig PCA with eural etwork, Applied Soft Computig A, vol 14, pp 4 11, 2014 [7] H W Wag, Partial Least Squares Regressio Method ad Applicatio, Natioal Defese Idustry Press, Beijig, Chia, 2000 [8] GZLi,RWZhao,HNQu,adMYYou, Modelselectio for partial least squares based dimesio reductio, Patter Recogitio Letters,vol33,o5,pp ,2012 [9] J Marques ad D Erik, Texture aalysis by a PLS based method for combied feature extractio ad selectio, i Machie Learig i Medical Imagig, vol7009oflecture Notes i Computer Sciece, pp , 2011 [10] SFDig,WKJia,XZXu,adCSu, Elmaeuraletwork algorithm based o PLS, Acta Electroica Siica,vol38,o2, pp71 75,2010 [11] X G Tuo, M Z Liu, ad L Wag, A PLS-based weighted artificial eural etwork approach for alpha radioactivity predictio iside cotamiated pipes, Mathematical Problems i Egieerig,vol2014,ArticleID517605,5pages,2014 [12] LKHaseadPSalamo, Neuraletworkesembles, IEEE Trasactios o Patter Aalysis ad Machie Itelligece, vol 12, o 10, pp , 1990 [13] D Opitz ad R Macli, Popular esemble methods: a empirical study, Joural of Artificial Itelligece Research, vol 11, pp , 1999 [14] Z H Zhou ad S F Che, Neural etwork esemble, Chiese Joural of Computers,vol25,o1,pp1 8,2002 [15] NKouretzes,DKBarrow,adSFCroe, Neuraletwork esemble operators for time series forecastig, Expert Systems with Applicatios,vol41,o9,pp ,2014 [16] C X Zhag ad J S Zhag, A survey of selective esemble learig algorithms, Chiese Joural of Computers,vol34, o 8, pp , 2011 [17]ZZhou,JWu,adWTag, Esembligeuraletworks: may could be better tha all, Artificial Itelligece, vol 137, o 1-2, pp , 2002 [18] M Alhamdoosh ad D Wag, Fast decorrelated eural etwork esembles with radom weights, Iformatio Scieces, vol 264, pp , 2014 [19] D Q Gao, O structures of supervised liear basis fuctio feed forward three-layered eural etworks, Chiese Joural of Computers,vol21,o1,pp80 86,1998 [20] Y M Zhag, The Applicatio of Artificial Neural Network i the Forecastig of Wheat Midge, Northwest A&F Uiversity, Shaaxi Provice, Chia, 2003 [21]

9 Advaces i Operatios Research Advaces i Decisio Scieces Joural of Applied Mathematics Algebra Joural of Probability ad Statistics The Scietific World Joural Iteratioal Joural of Differetial Equatios Submit your mauscripts at Iteratioal Joural of Advaces i Combiatorics Mathematical Physics Joural of Complex Aalysis Iteratioal Joural of Mathematics ad Mathematical Scieces Mathematical Problems i Egieerig Joural of Mathematics Discrete Mathematics Joural of Discrete Dyamics i Nature ad Society Joural of Fuctio Spaces Abstract ad Applied Aalysis Iteratioal Joural of Joural of Stochastic Aalysis Optimizatio