Using Robust Extreme Learning Machines to Predict Cotton Yarn Strength and Hairiness

Transcription

1 Usig Robust Extreme Learig Machies to Predict Cotto Yar Stregth ad Hairiess Diego P. P. Mesquita1, Atoio N. Arau jo Neto1, Jose F. Queiroz Neto1, Joa o P. P. Gomes1 ad Leoardo R. Rodrigues2 1- Federal Uiversity of Ceara - Computer Sciece Departmet Rua Campus do Pici s - Fortaleza-CE - Brazil 2- Istitute of Aeroautics ad Space - Electroics Divisio Prac a Marechal Eduardo Gomes, 50, Sa o Jose dos Campos-SP - Brazil Abstract. Cotto yar is ofte spu from a mixture of distict cotto bales. Although may studies have preseted efforts to predict hairiess ad stregth from cotto properties, the heterogeeity of this mixture ad its ifluece i such values have bee eglected so far. I this work the properties of the cotto bale mixture are modeled as radom variables ad a robust variat of the Extreme Learig Machie (ELM) to address the cotto quality predictio problem is proposed. A real world dataset collected from a textile idustry was used to compare the performace of the proposed model with a traditioal ELM ad a liear regressio model. he results showed that the proposed method outperformed the bechmark methods i terms of Average Root Mea Square Error (ARMSE). 1 Itroductio Predictig cotto yar quality based o the aalysis of the cotto fiber has bee the objective of may studies i last years [1]. Beig able to predict yar quality may reduce productio costs by allowig better productio plaig i textile idustries. I recet years, several works have proposed the use of machie learig methods for this task [2, 3]. Artificial Neural Networks (ANN) have bee used to predict differet cotto yar quality metrics [4]. Amog these metrics, stregth ad hairiess are two of the most importat oes. Stregth is a importat mechaical property of cotto yars which is related to the fiber structure [5]. he measuremet of stregth has become a importat research topic over the years. May factors may affect stregth measuremets such as evirometal coditios (temperature ad moisture), gauge legth, processig history of cotto samples ad timig of the test [6]. Hairiess is characterized by the quatity of freely movig fiber eds or fiber loops projectig from a yar [7]. Usually hairiess is udesirable if it is too high ad some commo problems are breakages, lower machie efficiecy i speed kittig ad a bad appearace of the produced fabrics [8]. he cotto quality predictio problem ca be defied as a stadard regressio problem where the iputs are features extracted from cotto fiber bales ad the outputs are the cotto yar quality metrics. Although good results have his work was supported by the Brazilia Natioal Coucil for Scietific ad echological Developmet (CNPq), grat /

2 bee obtaied so far, previous works do ot address may real world problems faced by may textile compaies. Oe problem relies o the fact that i practice it is very ucommo to produce cotto yars from a homogeeous (with similar properties) cotto mixture. Cotto yars are produced istead from a mixture of differet cotto bales, each with its ow properties. As a cosequece, the mappig betwee fiber properties ad yar quality ca ot be doe i a straightforward way sice there is ot a sigle iput feature vector, but differet feature vectors obtaied from the differet bales. It is worth poitig that the variability of the iput bales may ifluece the fial cotto yar quality. his work proposes a Robust Extreme Learig Machie (R-ELM) to predict cotto yar stregth ad hairiess uder ucertai iputs. he iput feature vectors are modeled as radom variables that ca be defied usig the feature vectors of the bales that belog to the mixture. he algorithm is said to be robust sice the output is robust to this ucertaity o the traiig set Proposed Method Overview of Extreme Learig Machies he ELM model was origially proposed i [9] as a alterative learig method for sigle hidde layer feedforward eural etworks (SLFNs) usig backpropagatio. he mai differece betwee a covetioal SLFN ad a ELM is that the latter assigs radom values to the weights betwee the iput layer ad the hidde layer. he, the weights betwee the hidde layer ad the output layer ca be computed usig the Ordiary Least Squares (OLS) estimate. Cosider a traiig set cotaiig arbitrary distict samples (xi, yi ), where xi Rd 1 ad yi Rm 1. Let wj Rd 1 be the weight vector coectig the j-th hidde ode ad the iput odes, bj be the bias of the j-th hidde ode ad g( ) be the activatio fuctio. he output oi of a SLFN with h hidde euros for the i-th sample is the give by oi h βj g(wj xi + bj ) (1) j1 where βj Rm 1 is the weight vector coectig the j-th hidde euro ad the output euros. Equatio 1 ca be writte i a compact form usig Eq. 2: Hβ O (2) where O [o1, o2,..., on ] ad the elemets of matrix H are computed as defied i Eq. 3. hij g(wj xi + bj ) (3) he output weight vector β [β1, β2,..., βh ] must be estimated i order to miimize the error fuctio preseted i Eq. 4. he OLS solutio, give by Eq. 5, is used to fid a estimate for β which miimizes Eq

3 mi Hβ Y 2 1 β (H H) H Y (4) (5) he ELM approach ca be summarized i the followig steps: Step 1: Radomly assig values for each iput weight vector wj ad each threshold bj ; Step 2: Compute the hidde layer output matrix H usig Eq. 3; Step 3: Estimate the output weight β usig the Moore-Perose pseudoiverse accordig to Eq. 5: 2.2 Robust Extreme Learig Machies (R-ELM) Cosider a regressio problem where the iput data xi are radom variables. I the proposed approach, the goal is to obtai the ELM output give this ucertai iput. For that, it is ecessary to estimate the correspodig hidde layer output vector Hi. It is importat to otice that, sice xi is a radom variable, Hi ca also be modeled as a radom variable. Accordig to Eq. 3, Hi ca be calculated by computig the oliear fuctio g( ) over the iputs xi, the hidde layer weights ad the biases. It ca be oticed that, sice xi is a radom variable with ukow distributio, estimatig the radom variable Hi ca be a o tractable problem. I such situatio, a possible solutio ca be obtaied by usig Mote Carlo techiques. I the curret problem, the radom iputs xi are the feature vectors of each mixture of bales. Cosider that the i-th mixture is composed of P bales, the (p) feature vectors for each bale ca be represeted by xi where p (1,..., P ). (i) I this work, xi is modeled as N (xi, Σ ), where xi ad Σ(i) are calculated accordig to Eqs. 6 ad 7. xi P 1 (p) x P p1 i (6) Σ(i) P 1 (p) (p) (x xi )(xi xi ) P p1 i (7) he Mote Carlo procedure cosists i samplig poits from the distri(k) butio of xi. After that the hidde layer outputs, Hi for k (1,..., ) are calculated accordig to Eq. 3. After obtaiig the samples for a give Hi, ay distributio estimatio procedure (parametric or o-parametric) ca be used. Cosiderig H as a radom matrix with the vectors Hi, it is possible to represet matrix H accordig to: 67

4 H H +U (8) where H is the mea matrix of H ad U is a radom matrix with zero mea. he objective fuctio preseted i Eq. 4 ca be redefied so that H is a radom matrix. I this case, the goal becomes to miimize the expected value of the objective fuctio as defied i Eq. 9. E Hβ Y 2 mi (9) he objective fuctio ca be expressed as: E[ Hβ Y 2 ] E[((H + U )β Y ) ((H + U )β Y )] E[((Hβ Y ) + (U β) )((Hβ Y ) + U β)] (Hβ Y ) (Hβ Y ) + E(β U U β) Hβ Y 2 + β ΣH β where ΣH E[U U ] is the covariace matrix of H. As observed, this problem, termed as robust approximatio problem, ca be see as a regularized least square problem ad its solutio is give by: β (H H + ΣH ) 1 H Y It is possible to obtai the matrix H ad the covariace matrix C i-th trasformed example by the hidde layer usig Eqs. 11 ad 12. Hi (10) (i) for the 1 (k) Hi (11) 1 (k) (k) (Hi H i )(Hi H i ) (12) k1 C (i) k1 he covariace matrix ΣH ca be obtaied by summig C (i) over all i examples, as show i Eq. 13: ΣH C (i) (13) i1 his result ca be obtaied by oticig that for each elemet j, j 0 of ΣH it follows that: ΣH j,j 0 i1 E[Ui,j Ui,j 0 ] i1 Cov(Ui,j Ui,j 0 ) i1 68 Cov(Ui,j Ui,j 0 ) + E[Ui,j ]E[Ui,j 0 ]

5 he fial procedure to implemet the R-ELM model ca be summarized i the followig steps: Step 1: Calculate the mea vectors (xi ) ad the covariace matrices (Σ(i) ) usig the feature vectors that belog to the i-th mixture accordig to Eqs. 6 ad 7. Step 2: Geerate samples from each radom variable xi with distributio defied as N (xi, Σ(i) ). (k) Step 3: For the i-th example, obtai Hi to Eq. 3. for all samples, accordig Step 4: Calculate H ad ΣH usig Eqs. 11, 12 ad 13. Step 5: Fid the weights β of the output layer accordig to Eq Experimets he proposed method was tested o a dataset of cotto yar quality measures collected i 2009 o a facility of a textile idustry i Brazil. A total of 640 mixtures of 10 bales were collected alog with the correspodig stregth ad hairiess of the produced yar. he feature vector for each bale comprises the followig measuremets: rash Code, rash Area, rash Particle Cout, Legth, Uiformity, Short Fiber Idex, Stregth, Elogatio, Microaire, Maturity, White Level ad Yellow Level. he performace of the proposed algorithm was compared agaist a traditioal ELM ad a liear regressio model. he experimets cosisted of 20 similar trials with 90% of the dataset i the traiig set ad 10% i the testig set. he umber of hidde odes for R-ELM ad ELM was selected usig 10-fold cross-validatio. I order to assess the ifluece of the umber of samples used to approximate each Hi, the R-ELM algorithm is evaluated for differet values of. he performace measure adopted was the Average Root Mea Square Error (ARMSE). he results obtaied i the experimets are show i able 1. able 1: ARMSE compariso for cotto yar hairiess ad stregth predictio Model Hairiess Stregth Liear Regressio ELM R-ELM ( 5) R-ELM ( 10) R-ELM ( 15) R-ELM ( 20) R-ELM ( 25)

6 As expected, the proposed R-ELM model preseted more reliable results tha the other methods. he liear model, which is commoly adopted for this task, preseted the worst performace. It is worth oticig that the performace of the R-ELM model icreases with the umber of samples. his is explaied by the fact that whe icreases it is possible to obtai better estimates for H ad ΣH. 4 Coclusios A variat of ELM that ca be traied whe the elemets of the traiig set are radom variables was proposed. he model was desiged to be robust to the ucertaities of the traiig set. he resultig model, amed R-ELM, is a regularized ELM without a regularizatio hyper-parameter. he proposed model was used to predict cotto yar quality measures collected from a real world textile idustry ad was compared to a liear regressio model ad a stadard ELM. he results showed that the proposed model outperformed both bechmark methods i terms of ARMSE. It was also possible to otice that the performace of proposed method is affected by the umber of samples that are used to calculate the matrix H. A possible topic for future research is to develop a method to defie the optimum umber of samples to be used. Refereces [1] Aida Ghosh ad Pritam Chatterjee. Predictio of cotto yar properties usig support vector machie. Fibers ad Polymers, 11:84 88, [2] halid A. A. Abakar ad Chogwe Yu. Applicatio of geetic algorithm for feature selectio i optimisatio of SVMR model for predictio of yar teacity. Fibers & extiles i Easter Europe, 21:95 99, [3] M. Selvaayaki, M. S. Vijaya,. S. Jamua, ad S. arpagavalli. A iteractive tool for yar stregth predictio usig support vector regressio. I proceedigs of the Secod Iteratioal Coferece o Machie Learig ad Computig, pages , Bagalore, February [4] halid A. A. Abakar ad Chogwe Yu. he performace of data miig techiques i predictio of yar quality. Iteratioal Joural of Iovatio, Maagemet ad echology, 4: , [5] You-Lo Hsieh, iao-pig Hu, ad Ajia Wag. Sigle fiber stregth variatios of developig cotto fibers - stregth ad structure of G. hirsutum ad G. barbedese. extile Research Joural, 70: , [6] Farzad Hosseiali. Ivestigatio o the tesile properties of idividual cotto (Gossypium hirsutum L.) fibers. Master s thesis, exas ech Uiversity, USA, August [7] Suil umar Sharma. Yar hairiess questios ad aswers. Spiig extiles, 7, [8] Aridam Basu. Assessmet of yar hairiess. Idia Joural Of Fiber Ad extile Research, [9] G. B. Huag, Q. Y. Zuo, ad C.. Siew. Extreme learig machie: heory ad applicatios. NeuroComputig, 70: ,