Time Series Analysis for Quality Improvement: a Soft Computing Approach

Transcription

1 ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp ime Series Analsis or Qualit Improvement: a Sot Computing Approach K. Xu and S.H. Ng epartment o Industrial and Sstems Engineering National Universit o Singapore, Singapore S.L. Ho he Centre or Qualit and Innovation, Ngee Ann Poltechnic, Singapore, Abstract Qualit improvement provides organizations with signiicant opportunities to reduce costs, increase sales, provide on time deliveries and oster better customer relationships. he design and manuacturing are among the critical processes or continuous qualit improvement. ime series data collected rom these processes are the useul source. While there are various techniques to eplore these processes, Neural Networks (NN) approach is deemed as a promising alternative. However, as NN is a relativel new approach in qualit engineering which is traditionall dominated b statistical analsis, there is still much doubt in its eectiveness compared with statistical modeling. he main ocus here then is to construct a statisticall reliable neural network model with an appropriate architecture to conduct the time series analsis. he purpose o this paper is thus two-old. Firstl we develop the statistical interval analsis or neural network models which provide a statistical guide towards a reliable modeling architecture. Secondl, we appl the developed approach or qualit improvement in various industries. Ke words: Qualit Improvement, ime Series Analsis, Neural Networks, Sot Computing. 1. Introduction Qualit improvement provides organizations with signiicant opportunities to reduce costs, increase sales, provide on time deliveries and oster better customer relationships. Such improvements can contribute considerabl to the bottom line o these organizations. he design and manuacturing are among the critical processes or continuous qualit improvement. While there are various qualitative techniques to eplore these processes, the most common approach is a quantitative approach. As such, the critical qualit data collected in these processes are primar sources or urther investigations into potential causes o poor qualit. Furthermore, these data can provide crucial inormation or decision making. Interestingl, man o these

2 ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp qualit data are time series data. For eample, automobile engine manuacturers are interested in the ailure patterns o its engines over time as it provides important inormation or the lie time predictions, warrant costs evaluations, and maintenance scheduling o its engines. With the rapid advances in microelectronics packaging technolog, electronics assembl manuacturers are collecting crucial on-line process monitoring data to dnamicall track qualit measures. In semiconductor and materials industries, modeling the eperimental data set o chemical vapor deposition process oers an eective approach or the o-line process qualit improvement. In various nonlinear modeling or time series analsis approaches, neural networks (NN) approach is deemed as a promising alternative. here are man successul applications that are widel reported. However, as NN is a relativel new approach in qualit engineering, there is still much doubt in its eectiveness compared with traditional statistical modeling. he main ocus here then is to construct a statisticall reliable neural network model with an appropriate architecture to conduct the time series analsis. he purpose o this paper is thus two-old. Firstl we develop the statistical interval analsis or neural network models which provide a statistical guide towards a reliable modeling architecture, particularl when onl a small number o training samples are available. Secondl, we appl the developed approach or qualit improvement in various industries. he organization o this paper is as ollows. Section introduces a general ramework or the time series modeling approach using neural networks. Section 3 develops the statistical interval analsis or NN models. Using the proposed approach or qualit improvement, an industr application is presented in section 4. he conclusions are presented in section 5.. General ramework or time series modeling o neural networks A general time series orecasting model can be ormulated as X t+1 (X t-m, A, B) where {X t-m m, 1,,, p} represents a time series o lagged variables A denotes the eternal eplanator variables, i.e. variables on which the series is thought to have a dependence i an and B represents the random error variables. X t can be represented in dierent orms. As the modeling progresses and once the NN model has adequatel understand the underling characteristics o the time series, uture outcomes can be predicted when presented with new data patterns. 3. Statistical interval analsis or neural networks computing the statistical intervals such as the prediction intervals allow us to quanti these uncertainties and thereore provide more inormation. In prediction tasks, it is desirable to construct conidence bounds upon the resulting point orecasts.our approach is generall ollowing the line o the non-liner regression. However, we improve the traditional error variance estimation method which ma result in errors when onl limited samples are available. Furthermore, we propose to use the

3 statistical interval analsis and coverage assessment as a guide to select the appropriate NN architecture. Hence, the constructed model ma be statistical reliable Quantiing Uncertainties via Non-linear Regression Given a general multi-response nonlinear model: ( ) +, the least-squares estimate o true value o an actual process is, which is obtained b Levenberg-Marquardt (LM) algorithm through minimizing the error unction (1) in Baesian regularization. he error term, is assumed to be : prediction error vector rom training data sets. Given the prediction rom the model as ( ), ( ) can be approimated in terms o ( ) b irst-order alor epansion: ( ) ( ) ( ) ( ) + where ( ) ( ) ( ) ( ) p... 1 where p is the number o parameters. he dierence between the true value and the predicted value or points that are not used to train the model is given b: ŷ ( ) ( ) ( ) ( ) ( ) he epectation o the dierence is: ( ) ( ) ( ) E E E Assuming statistical independence between and, the variance can be epressed as: ( ) ( ) { } ( ) ( ) { } ( ) ( ) ( ) I I σ σ he estimates or the parameters o NN model obtained via Levenberg-Marquardt (LM) algorithms are: ( ) 1 J J H +, where, H is the Hessian matri and is the Jacobian matri o the training sets error. J hereore, the asmptotic variance o the estimated parameters can be approimated b: ( ) 1 1 JH J H s ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp

4 ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp For large samples, we have approimatel ( Seber and Wild, 1989): where, tnδ is student t-distribution with t nδ n δ degrees o reedom n is the number o training. We will discuss the selection o δ in ollowing section. It ollows that an approimate 1(1γ )% prediction interval or the predicted value ŷ is given b: 1 1 ( σ I + s ( ) H J JH ( )) γ ± tnδ 3.. Error iance Estimation In the discussion o e Veau et al. (1998), or conventional MLP and NN with weight deca method, the estimated error variance is given b E s, where p is n p the number o parameters which could be ver large and, in some cases, ma be larger than n such that estimation o prediction interval is impossible. B stopping the algorithm beore convergence, the weights or bias are shrunk toward, thus reducing the eective number o parameters in the models. However, this would result in overl conservative prediction intervals. NN model using Baesian Regularization (BR) is broadl used to keep the overitting problem in check (MacKa (199) and Foresee and Hagan (1997)). hus, our approach in using the eective parameter δ estimated rom the BR process is a reasonable alternative to the statistical interval analsis. In MacKa (199) and Foresee and Hagan (1997), δ p αrace ( H) 1 is the deined as the eective number o parameters p is the number o parameters o NN. H is the Hessian matri where ( ) ( ) o the objective unction F( ) βe + αe E is the residual sum o squares, E is the sum o squares o the network coeicients (weights and bias), and α and β are the objective unction parameters whose relative size determines the training direction. hereore, in our statistical interval ormulation, E σ s. It implies that the actuall number o weights and bias n δ (eective parameters in the models) would be signiicantl smaller when using the NN modeling with BR. hereore, it ma eectivel prevent over-itting o models when sparse data sets are available or training Statistical Guide o Selecting the NN Architecture Parameters he probabilit o coverage o the prediction interval is approimatel equal to the nominal (1γ ) with large sample sizes (asmptotic inerence). his can be evaluated

5 ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp empiricall using Monte Carlo simulation. We conduct 5 simulations using dierent NN models trained with etra random error to response data settings. he coverage percentage is calculated based on all run designs. he random noise is assumed to be normal with mean and a small standard deviation (in practice, values o σ near.1 are common (e Veau et al., 1998)). aking into account the noise in the training data set, a more suitable measure would beσ.1+ 1 β, where β is the noise o the training data sets and readil available rom the BR process. he coverage probabilit can then be estimated rom the simulation as : Coverage probabilit number o predicted points within intervals. hereore, we can use the 5(number o eperiment run) statistical interval as well as the coverage probabilit as the measure to select the appropriate the NN parameters such as the number o the hidden units etc. his wa, the model constructed is statistical reliabilit in some sense. 4. Cases Stud: Process modeling and improvement In semiconductor and material industries, chemical vapor deposition (CV) process is the essential technique or depositing silicon nitride, silicon dioide, polsilicon, reractor metals etc.( Jaeger, 1988) or materials snthesize. Plasma-enhanced CV (PECV) improves CV b allowing the process to orm a thin ilm on a substrate surace at the relativel low temperature. Fig. 1 shows a tpical laout o a plasma CV apparatus with a parallel plate electrode structure. A substrate is placed on the grounded electrode. he reaction gas is supplied rom the opposite plate in order to orm a uniorm ilm. Figure 1. he tpical laout o a plasma CV apparatus As a widel used technique, modeling and optimizing this process has received much attention or man ears. In this eample, Eight actors: 1 : cleaning method : chamber temperature : batch ater cleaned chamber : low rate o SiH4 : low rate o N : chamber pressure : R.F.power : deposition time are selected to analze two qualit characteristics:, the deposition thickness and a reractive inde. he total number o the training samples is 16. ue to small number o 5

6 ESANN'4 proceedings - European Smposium on Artiicial Neural Networks Bruges (Belgium), 8-3 April 4, d-side publi., ISBN , pp samples, a statistical interval analsis should be used to provide a reliable modeling. Here, we select the three laers neural networks o BR with 8 input nodes, output nodes and 1 hidden laers based on the proposed prediction interval analsis and coverage assessment process. he sum o squares errors or two responses are.6586e+4 ( 1 ) and.66 ( ). he coverage is 94.8% or 1 and 98.8% or. he average hal the length o 95% prediction intervals is or 1 and.514 or. Note that inal solutions are ver close to the deined targets o the two responses (11,.). Note that the developed neural network method can be easil adjusted to modeling dierent controls actors and qualit measures o processes. able 1. Best settings or PECVP Eperiments 1 X Y Conclusions his paper investigates the application o time series approach under the umbrella o sot computing, particularl neural networks, to modeling and improve the process qualit. A statistical interval analsis approach is developed and thus the appropriate neural networks architecture can be selected to provide reliable modeling or qualit improvement. Reerences 1. e Veau, R.., Schumi, J., Schweinsberg, J. and Unger, L. H.,. Prediction Intervals or Neural Networks via Nonlinear Regression, echnometrics, vol. 4 (4), (1998). Foresee, F.. and Hagan, M... Gauss-Newton Approimation to Baesian Learning. in IEEE International Conerence on Neural Networks, Houston, June, USA, (1997) 3. Hwang, J.. G. and ing, A. A.. Prediction Intervals or Artiicial Neural Networks, Journal o the American Statistical Association, 9, (1997) 4. Jaeger, R. Introduction to Microelectronic Fabrication. Addison-Wesle, New York. (1988) 5. MacKa,.J.C.. Baesian Interpolation. Neural Computation 4, (199) 6 Seber, G.A.F. and Wild, C.J., Nonlinear Regression, John Wile & Sons, New York. (1989)