STATISTICAL PROCESS ADJUSTMENT PROBLEMS IN SHORT-RUN MANUFACTURING

Transcription

1 The Pennsylvania State University The Graduate School Department of Industrial and Manufacturing Engineering STATISTICAL PROCESS ADJUSTMENT PROBLEMS IN SHORT-RUN MANUFACTURING A Thesis in Industrial Engineering and Operations Research by Omer Arda Vanli c 2007 Omer Arda Vanli Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2007

2 The thesis of Omer Arda Vanli was reviewed and approved by the following: Enrique Del Castillo Professor of Industrial and Manufacturing Engineering Thesis Adviser Chair of Committee M. Jeya Chandra Professor of Industrial and Manufacturing Engineering Tao Yao Assistant Professor of Industrial and Manufacturing Engineering Murali Haran Assistant Professor of Statistics Nital S. Patel Senior Staff Engineer, Intel Corporation Special Member Richard J. Koubek Professor and Departmental Head of Industrial and Manufacturing Engineering Signatures are on file in the Graduate School.

3 ii ABSTRACT Manufacturing engineers often change or adjust the operating conditions of a production process by manipulating a set of variables, or controllable factors. The goal is usually to keep some other variables of interest, the responses, close to given target values in the presence of uncontrollable variables, the noise factors and the disturbances, that also affect the responses. The performance of a process adjustment technique, which indicates how to change the controllable factors of a process, depends on the amount of information available about the relation between the controllable factors, or inputs, the noise factors and disturbances, and the responses, or outputs. This information usually results in an input-output, or transfer function, model. This dissertation considers problems related to the identification of these models, and proposes new process adjustment techniques when the amount of information available is limited (i.e. process runs are short) and noise factors are present in a process. A specific problem addressed in this dissertation is how to identify the input-output model of a process that is being adjusted. Such a closed-loop identification method is necessary in industrial processes that cannot be left to run without control. Traditional system identification techniques assume open-loop (no control) operation. The first part of this dissertation presents new methodology for the identification of Box- Jenkins transfer function models under closed-loop operating conditions. It is shown how the input-output delay of the process represents crucial information and that, if known a priori, it would facilitate the identification of the rest of the model. Hence, new methods for

4 iii the specific estimation of the input-output delay, while the process operates in closed-loop, are proposed. The methods are based on Time Series change-point detection techniques. In the second part of the dissertation we study a system identification problem frequently found in semiconductor manufacturing. This is the so-called context-based model identification problem, where different process models need to be identified for different batches of products depending on the manufacturing context under which the process data was obtained (e.g. the product type, operation, chamber, tool etc.). A model identification method that uses categorical variable selection methods is developed and applied to a real semiconductor manufacturing data set. The last part of the thesis presents new adjustment methods for processes that involve uncontrollable noise factors. In the statistical process optimization literature, Robust Parameter Design (RPD) methods have been used for designing processes that are insensitive against variation caused by noise factors. These methods, however, are largely applied off-line; that is, they are not process adjustment methods that recommend different controllable factor settings depending on the on-line noise factors measured during production. Instead, they determine the optimal process settings before production starts and they do not alter the optimal settings during production. In this research, new Bayesian process adjustment methods for on-line robust parameter design are proposed. The proposed online RPD controllers are feedforward multiple response controllers that utilize on-line noise factor measurements, assumed available.

5 Contents List of Figures ix List of Tables xi List of Acronyms xiii Acknowledgements xiv 1 Introduction Dissertation Topics and Research Objectives Closed-Loop System Identification Methods Context-Based Model Identification for Run-to-Run Control On-line Robust Parameter Design Dissertation Outline Literature Review Time Series Control and Bayesian Inference Closed-loop System Identification Methods Run-to-Run Control and Model Variable Selection Methods iv

6 2.4 Robust Parameter Design Methods v 3 Closed-Loop System Identification for Small Samples with Constraints Introduction Closed-Loop identification of Box-Jenkins transfer function models Identification and Parameter Estimation of the Process Model Least squares estimation of the process model Identifiability conditions Some constraint types and their specification in practice Effects of adding constraints on the identifiability of a process Simulation Example: Benefits of adding constraints when only small samples are available Estimation of the process model from the simulated closed-loop realizations Assessment of the benefits of using constraints Case Study: A Gas Furnace Process Chapter Summary Change-point Methods for Closed-loop Delay Estimation Introduction Process Assumptions Bayesian approaches Prior specification of the parameters Posterior distribution of the parameters

7 vi Full Bayesian Approach Reduced Bayesian Approach A sequential probability ratio test approach A cumulative sum approach Example 1: Illustration of the proposed methods Performance of the proposed approaches under a second order process Example 2: Comparison with the Laguerre method Guidelines for designing the delay estimation experiment Chapter Summary Appendices of Chapter Appendix 4A: Selection of the reference values of the SPRT and the CUSUM schemes Appendix 4B: Estimation of the parameters required in the change-point methods Appendix 4C: Convergence Diagnostics for the MCMC simulations in the Full Bayesian Approach Context-based Model Identification for Run-to-Run Control Introduction Proposed Model Identification Approach Regression Model Variable Selection ANOVA Model Variable Selection Variable Selection Criterion and the Stopping Rule

8 vii 5.4 Context-Based Model Identification Algorithm Regression Model Variable Selection ANOVA Model Variable Selection Simulation Example Application to a Lithography Alignment Process Chapter Summary Bayesian Approaches for On-line Robust Parameter Design Introduction Process Assumptions Bayesian Posterior Predictive Density Solving the Bayesian Robust Control Problems Existing Approaches Performance evaluation of the proposed and existing methods Extension to Multiple Response Processes Chapter Summary Appendices of Chapter Appendix 6A: Representing the prediction variance as a polynomial in the control factors Appendix 6B: Computing the noise factor forecasts required in the BR control law Appendix 6C: Computing the noise factor posterior moments required in the BRN control law

9 viii Appendix 6D: Obtaining the posterior distribution of the parameters of an MA(1) model using MCMC Summary of contributions and directions for future research Contributions in Closed-loop System Identification Methods Contributions in Context-based Model Identification for Run-to-run Control Contributions in On-line Robust Parameter Design Recommendations for Future Research Bibliography 200

10 List of Figures 2.1 Open-loop and Closed-loop operation of a process Input-output delay of a dynamic process Non-segregated and fully segregated run-to-run controllers Traditional (off-line) and feedforward (on-line) RPD control methods Effect of adding a constraint on the identifiability of the process model The sample squared bias and variance of the identified models Percentages of the estimated controllers Closed-loop identification of the gas furnace process data The unit step response of the dynamics and disturbance transfer functions Input-output delay of a dynamic process Simulated delay estimation experiment(example1) The results of the delay estimation analyses (Example 1) Comparison of the step responses Simulated delay estimation experiments (Example 2) Histograms of the delay estimates (Example 2) Experimentation and operation periods of a hypothetical process ix

11 x 4.8 Sensitivity of the cost functions to operation length The sensitivity of the cost function to number of replications The sensitivity of the cost functions to the true delay Effect of the selection of the reference value on run length Simulated marginal posterior distributions Hypothetical data from a three-context process Sequential regression Flow chart of the model identification algorithm Simulation example disturbance data Pairwise plots of the context variables (Lithography Process) Disturbance values after discounting the autocorrelations (Lithography Process) Disturbance values after removing the offsets (Lithography Process) Predictions and actual measurements (Lithography Process) Flow chart of the BRN control approach Variances of the optimal control factor settings and the empirical cumulative distribution function of MSE(y) Historical noise factor data and the posterior distributions of the parameters The empirical CDF of M SE(y) under different control laws and noise factors The designs and hypothetical process data used in the examples Sample simulation of the Plasma Etch Process over the production period The empirical CDF of the Plasma Etch Process response MSE

12 List of Tables 3.1 Closed-loop equations with PI controllers Constraints used in simulations Cost of identification and cost of operation (gas furnace process) Mean, variance and RMSE of the delay estimates (Example 1) Mean, variance and the root mean square error of the delay estimates of the first order and second order processes Scenarios for prior distributions in the reduced Bayesian approach Mean, Variance and Root Mean Square Error (RMSE) of the delay estimates obtained (Example 2) The variance of the process under MMSE controller as a function of the estimated delay Raftery and Lewis diagnostic and the Geweke diagnostic test results Disturbance data, model error and context variable levels for the 2 tool 2 product example The record numbers of the tools 1,2 and the products 1,2 for the data in Table xi

13 xii 5.3 Drift rates (Simulation Example) Iterations in the first scenario (Simulation example) Iterations in the second scenario (Simulation example) Context variables (Lithography Process) Iterations of the variable selection algorithm (Lithography Process) Autoregressive parameter estimates (Lithography Process) Summary of the estimated model (Lithography Process) Fisher s white noise test results on the residuals Existing and proposed approaches considered in this chapter

14 List of Acronyms xiii RPD SPC APC CDF MMSE MSE ARMA BJ ARMAX CUSUM SPRT ANOVA PI i.i.d AR IMA TF Robust parameter design Statistical process control Automatic process control Cumulative distribution function Minimum mean square error Mean square error Auto Regressive Moving Average Box Jenkins Auto Regressive Moving Average exogeneous Cumulative Sum Sequential Probability Ratio Test Analysis of Variance Proportional Integral independently and identically distributed Autoregressive Integrated Moving Average Transfer Function

15 xiv ACKNOWLEDGEMENTS I would like start by expressing my gratitude to my advisor Dr. Enrique Del Castillo for his genuine interest and attention in my work throughout my doctoral study. Without his constant support, motivation and inspiration this dissertation would not be possible. He not only guided me through my thesis work but also introduced me to many industrial contacts that are very important both in my doctoral study and for my future career. This dissertation work was partially funded by his research grant from Intel Corporation. I would like to thank Dr. Jeya Chandra, my dissertation committee member, for giving me the opportunity to work as his research assistant and as a teaching assistant in the Quality and Manufacturing Management (QMM) program. I convey my thanks to the faculty and staff of the Industrial Engineering department, the QMM program and the Statistics department at Penn State. I thank Dr. Nital S. Patel, my dissertation committee member, for his interest in my dissertation and his sincere support in writing the paper Model Context Selection for Runto-run Control. I also would like to thank the other committee members, Dr. Tao Yao and Dr. Murali Haran. The classes I have taken with my committee members and their feedbacks on my research proposal have been very valuable in improving my dissertation. I also thank Dr. Dennis Lin for reviewing my thesis proposal and his comments. My special thanks are to my colleagues and friends in the Engineering Statistics Lab, Dr. Ramkumar Rajagopal, Dr. Zilong Lian and Eduardo Santiago Duran, for sharing with me many useful research discussions. I thank Dr. Bianca M. Colosimo for collaborating with

16 xv me on the paper Statistical Change-point Methods for Closed-loop Delay Estimation. I also would like thank Dr. Mani Janakiram of Intel Corporation for mentoring me during my internship at Intel and for co-authoring the paper Model Context Selection for Run-to-run Control. Finally, I am grateful to my parents, Ercin Vanli and Gulderen Vanli and my younger brother, Tunca Vanli, for giving me endless emotional support and always being by my side throughout my doctoral study.

17 To My Parents xvi

18 Chapter 1 Introduction Manufacturing engineers often change or adjust the operating conditions of a production process by manipulating a set of variables, or controllable factors. The goal is usually to keep some other variables of interest, the responses, close to given target values in the presence of uncontrollable variables, the noise factors and the disturbances, that also affect the responses. The performance of a process adjustment technique, which indicates how to change the controllable factors of a process, depends on the amount of information available about the relation between the controllable factors, or inputs, the noise factors and disturbances, and the responses, or outputs. This information usually results in an input-output, or transfer function, model. This dissertation considers problems related to the identification of these models, and proposes new process adjustment techniques when the amount of information available is limited and noise factors, which in turn can be explicitly modeled, are present in a process. Throughout this dissertation we assume all measurements occur at discrete, equidistant 1

19 2 points in time. This is a common assumption in statistical process control (SPC) methods. 1 Limited data typically exists in situations in industry when production runs are short. A specific problem addressed in this dissertation is how to identify the input-output model of a process that is being adjusted. Such a closed-loop identification method is necessary in industrial processes that cannot be left to run without control. Traditional system identification techniques assume open-loop (no control) operation. Industrial process models need to be identified and estimated from operating data, hence they involve some level of uncertainty. Automatic process control (APC) methods of feedback and feedforward control can be effective at reducing variation when the process model is accurate, however, their performance can deteriorate to the point of actually increasing the variation if the process model is poorly estimated. According to the modeling methods employed, manufacturing processes are generally classified into two groups: discrete-part and continuous-flow manufacturing processes [16, 14]. In a discrete-part process, the dynamical behavior is not significant and a static transfer function model that reflects an immediate effect of the inputs on the outputs (i.e. an input-output delay of one time period) frequently provides adequate accuracy. These models have a regression model form and are fitted using experimental design methods. By contrast, in continuous-flow processes, the dynamical input-output behavior is significant. In particular, due to the transport time fluids take to flow through piping, the inputoutput delays are longer. Therefore, transfer function models that accurately represent the transient (dynamic) response and multiple periods of delay need to be used for continuous- 1 SPC methods are not control but monitoring methods in the sense that no explicit adjustment methods are utilized and the only goal is instead the detection of process faults and errors.

20 3 flow processes. These models are fitted using Time Series analysis and transfer function model identification methods [13]. The methods presented in this thesis are applicable for both discrete-part and continuous-flow processes. Process variation can be caused by unmeasured disturbances (modeled in statistical models as random errors) or be due to our uncertainty in the parameters of the model we wish to estimate from data. In addition to these sources of variation, some process disturbances can be measured. Measured disturbances, the so-called noise factors, are uncontrollable during actual production. Ambient temperature and raw material variation are two typical examples of noise factors encountered in manufacturing. In the statistical process optimization literature, Robust Parameter Design (RPD) methods [81, 64] have been proposed and studied for designing processes that are insensitive against variation caused by noise factors. These methods, however, are largely applied off-line; that is, they are not process adjustment methods that recommend different controllable factor settings depending on the on-line noise factors measured during production. Instead, they determine the optimal process settings before production starts and they do not alter the optimal settings during production. These settings are hoped to make the process resistent, or robust, to the subsequent variation in the noise factors. An example of discrete-part production processes where the techniques developed in this thesis are applicable is semiconductor manufacturing. In semiconductor manufacturing it is common to find the same tool processing different types of products and operations because of the high capital costs of the tools and the limited capacity of the manufacturing facility [69]. The particular combination of different (categorical) factors related to the process, such as the product, operation, chamber and machine, is defined as the manufacturing

21 context of the process, and the factors that define the context of the process are called 4 the context variables [31]. In production, different products and operations often need to be processed on the same machine or process stage and this causes the parameters of the process model to be non-homogeneous across different contexts. Run-to-run control, a class of process adjustment methods used in semiconductor manufacturing, takes into account the variability in the process parameters by defining a different process model for each context. The necessity to identify the different process models with respect to their context gives rise to a different kind of system identification problem for run-to-run control: the problem of identifying which context variables require a different process model. The contributions of this research focus on system identification and controller design, two widely studied areas in the APC literature. While this dissertation does not consider SPC monitoring problems, some SPC techniques will be used in developing the new identification and control methods. The research in this thesis is motivated by the characteristics of modern manufacturing environments where the processes are operated over relatively short periods of time. In semiconductor manufacturing, for example, the process runs are necessarily short because many different types of products and operations must be processed within a fixed interval of time. As it will be discussed in the following chapters, the limited amount of historical data in short-run processes presents challenges for system identification. In summary, the overall goals of this research are (i) to develop efficient model identification methods based on limited data obtained during the closed-loop operation of a process;

22 (ii) to develop a methodology for context-based model identification problem (as it occurs in semiconductor manufacturing); 5 (iii) to propose new process adjustment methods for on-line Robust Parameter Design control, i.e., methods that utilize noise factor measurements obtained during production to re-optimize the process settings for better control. The next section describes in more detail the problems addressed in this dissertation and the specific objectives related to each of the overall goals mentioned above. 1.1 Dissertation Topics and Research Objectives Closed-Loop System Identification Methods In many industrial settings, it is of interest that the unknown model of the process be identified in closed loop, that is, while the process is being adjusted by a feedback controller. The identified model can then be used to re-design the controller in such a way that the quality characteristic under control better achieves a given objective. Open-loop identification of the process (i.e. observing the process when no feedback control is exercised) would not be feasible or cost effective in an industrial process because the process could be unstable and drift off-target dangerously or could produce expensive scrap when uncontrolled. The main difficulty with closed-loop identification is that the control input level is frequently a linear function of the output and the data collected during closed-loop operation is less informative than open-loop data ([48], pg. 430). As a consequence of this, the

23 6 parameter estimates tend to be less precise and sometimes even a unique solution is not obtainable. Traditional approaches to closed-loop identification of transfer function models require a sufficiently large data set and model forms that are general enough while at the same time requiring some form of external excitation (a dither signal ) be applied to the process. The use of a dither signal, a commonly applied approach in closed-loop identification [18, 19], can unfortunately be too expensive if the cost of changing the controllable factor or the cost of running the process off-target is high, precisely the conditions that justify the use of a controller. This research will concentrate on identifying and estimating the transfer function model of a process in closed-loop without the use of any external excitation. In order to improve the parameter estimates obtained in closed-loop, prior knowledge about the process will be used. In particular, the input-output process delay will be utilized as one form of prior process knowledge in identification. It will be shown that if prior information on the delay is available then identification of the transfer function under closed-loop operation becomes easier. Hence, we investigate methods for estimating the input-output delay only. The first specific objective in this topic is to develop a method for closed-loop identification of transfer function models from small samples that does not rely on the use of a dither signal. The method aims at improving closed-loop model parameter estimates by using some forms of prior process information that are commonly available in practice. The second specific objective in this topic is to develop statistical change-point detection methods for closed-loop estimation of the process delay. The method uses a step-response test to induce a change in the output and applies a change-point detection method to

24 7 estimate the time of the output change and hence, the delay Context-Based Model Identification for Run-to-Run Control Run-to-run control is frequently applied for adjusting semiconductor manufacturing processes. Within each batch (or run) of silicon wafers, the control action is exercised by automatic proportional-integral (PI) controllers, while between batches, the set-points of the automatic controllers are adjusted by a run-to-run controller [27]. As it was mentioned before, due to the many different products and operations that need to be processed simultaneously, it is necessary to define the process model with respect to the manufacturing context of the process. However, due to the large number of products and operations usually encountered in semiconductor manufacturing [31] it is usually not feasible to define a different model for each possible combination of context variables. This research is concerned with model identification for run-to-run control, that is, identifying the model structure that best explains the variability - process offsets or shifts and autocorrelation- in the data with fewest possible explanatory variables. The context of a process will be defined as a set of categorical variables. The specific objective of this topic is to develop a context-based model identification approach that uses statistical linear models and stepwise regression methods to identify the categorical context variables that best explain the autocorrelation and the offsets in the process data.

25 On-line Robust Parameter Design Robust Parameter Design (RPD), a methodology introduced by Taguchi [81], has been widely employed for designing products and processes that are robust against environmental effects. The method consists of determining the levels of controllable factors that make the process least sensitive (or most robust) to the variability transmitted by noise variables. In traditional RPD methods, such as the dual response surface approach (see e.g., Myers et. al., [64]), the robust settings of the controllable factors are computed off-line (i.e. before production starts) and once fixed, the control factor settings are not altered during production. By contrast, automatic process control methods used in production adjust the controllable factors continuously to minimize the variability created by uncontrollable variables, which are not explicitly modeled. In many manufacturing processes, however, the noise factors can be measured during production. This implies that it is possible to use the additional noise factor information in order to continuously update the choice of the controllable factors as production takes place. Furthermore, if the noise factors have strong autocorrelation, then the off-line robust settings (which typically assume i.i.d noise factors) will be ineffective in achieving the desired levels of process variability. It is necessary to recompute the process settings on-line, as production takes place. The specific objective in this topic is to develop Bayesian methods to on-line robust parameter design that incorporate measurements of the noise factors. A Bayesian model accounts for uncertainty in the parameters of the model, therefore, the method is aimed at providing solutions that are robust against both the on-line variability of the uncontrollable noise factors and the uncertainty in the model parameters.

26 9 1.2 Dissertation Outline The remaining chapters of this dissertation are organized as follows. In Chapter 2, the literature relevant to the topics in this dissertation is reviewed. Time Series process control methods and Bayesian methods of inference are reviewed. This is followed by a review of methods for closed-loop system identification (identification of transfer function models, delay estimation and change-point detection), run-to-run control and model variable selection, and robust parameter design. In Chapter 3, a new method for the identification of transfer function models under closed-loop operation is presented. The method is based on constrained non-linear least squares estimation and does not rely on a dither signal. In an effort to improve the parameter estimates obtained from closed-loop data, it incorporates prior knowledge about the process into the estimation procedure as constraints on the parameters. We discuss some forms of prior process knowledge that are commonly available in practice and how they can be represented as constraints. The proposed approach is further compared against a dither signal identification method by using real process data. In Chapter 4, three new methods are proposed to estimate the input-output delay under closed-loop operating conditions. The methods are based on Bayesian, Sequential Probability Ratio Test (SPRT) and Cumulative Sum (CUSUM) change-point detection techniques. The delay is estimated from a step-response test where the target of the process is changed in a step fashion and the output change is measured by a change-point detection method. In Chapter 5, a new model identification method is proposed for use in run-to-run

27 10 control. The method represents the offsets and the autocorrelation in the disturbances as a function of different variables that define the context under which the observations were taken. A fixed effects analysis of variance (ANOVA) model is used to represent the offsets and an autoregressive (AR) time-series model is used to represent the autocorrelations and drift. A simulation example, where the true process model is known, and an application to a real semiconductor lithography process are presented to illustrate the proposed approach. Chapter 6 presents new Bayesian approaches for on-line robust parameter design. Both single and multiple response processes are considered. A Bayesian linear regression model is used to describe the input-output relationship of the process and a vector ARMA time series model is used to model the noise factors. Optimization models are formulated and solved in closed-form to provide an on-line solution to the robust parameter design problem. The proposed approaches are illustrated and compared against existing RPD methods (certainty equivalence and dual response control methods). A semiconductor plasma etch process example is presented to illustrate the proposed multiple response robust control approach. Chapter 7 presents a summary of the research contributions and comments on possible directions for future research.

28 Chapter 2 Literature Review The literature on time series control methods is so extensive that only the themes directly related to this dissertation will be reviewed in this chapter. An overview of Time Series Control methods, and in particular, Bayesian methods of inference for time series models is given in Section 2.1. We review in Section 2.2 the main results existing in the System identification literature, which includes results about the closed-loop identification of transfer function models of the kind discussed in Chapter 3 of this thesis. Existing methods for estimating the input-output delay are also discussed in this section, as this problem is studied in Chapter 4. The run-to-run control problem as it occurs in semiconductor manufacturing is discussed in Section 2.3. Modeling approaches that have been proposed to consider the context of each batch of measurements are discussed. Our approach for identification of contextbased run-to-run models presented in Chapter 5 is based on variable selection methods for Analysis of Variance (ANOVA) models, so these methods are reviewed also in Section 2.3. Finally, we review Robust Parameter Design (RPD) methods in Section 2.4 and include a 11

29 review of recent papers that have been proposed to utilize on-line noise factor information for better control, a problem we study in Chapter Time Series Control and Bayesian Inference Time series analysis In this dissertation Box-Jenkins (BJ) transfer function models [13] will be employed to describe dynamic-stochastic processes. Consider the model y t = G(B)u t k + v t (2.1) where y t is the output (the response variable), u t is the input (the controllable factor) and k is the input-output process delay. Observations are assumed to be obtained (sampled) at equidistant points in time t. The delay is the number of whole time periods that elapses between a change is made in the control input and its effect is observed at the process output. The process transfer function G(B) is assumed to be a linear polynomial in the back shift operator B (BY t = Y t 1 ) and it describes the dynamic (transient) relation between the output and the input. The disturbance v t represents the effect of all sources of variation at the output. The disturbance is unobservable (i.e. not measured) during the operation of the process. In a BJ model, the transfer function is represented as a ratio of two polynomials: G(B) = b(b) a(b) (2.2) and the process disturbance is represented using autoregressive-integrated-moving average

30 13 (ARIMA) time series models. An ARIMA model with drift is given as [13]: v t = δ (1 B) + θ(b) d φ(b)(1 B) ɛ d t (2.3) where {ɛ t } is a Gaussian white noise process, that is, a sequence of an independently and identically distributed (i.i.d) normal random variables with mean 0 and variance σ 2. The parameter d 0 is the degree of integration of the disturbance and δ is a drift constant useful in modelling possible trends in the process mean, such as tool wear which frequently occurs in machining operations. Here θ(b) and φ(b) are polynomials in B with all their roots outside the unit circle. When d 1 the disturbance is said to be non-stationary. For continuous-flow processes with short sampling intervals, the input-output dynamics are significant (i.e. the output depends on the current and past values of itself and those of the input) and a transfer function of the form (2.2) must be used. For discrete-part and run-to-run batch production processes, however, the input-output dynamics are not significant and a static, or pure-gain, transfer function model of the form y t = gu t k + v t. (2.4) usually provides sufficient accuracy [4]. In this model g is the constant gain of the process and it is assumed that all the effect g of a change in the input is completely observed at the output after k time intervals (i.e. there are no transient dynamics). A pure-gain transfer function plus an IMA(1,1) disturbance model v t is widely employed in the control of discrete-part manufacturing processes [16]. A linear regression equation such as y t = β 0 + β 1 u 1,t β r u r,t + v t

31 14 also models only the static (steady-state) response of the process and it is equivalent to a pure-gain transfer function model with r control inputs and a single output. Note that in this model it is implicitly assumed that the input-output delay is one period. Process control A process is said to be under closed-loop control, or to operate in closed-loop, when it operates under the action of a feedback controller. Feedback control can be used to reduce the output variability caused by the unobservable disturbance. A feedback controller determines the control input level as a function of the current and past observations of the output: u t = C(B)(y t T ) (2.5) where T is the target, or set-point, of the process and C(B) is the controller polynomial. In manufacturing processes there are often disturbances that are measured (i.e. observable) during the operation. Observable but uncontrollable factors are referred to as noise factors in the process optimization literature (see Section 2.4). In contrast, feedforward control can be used in reducing the influence of the measured uncontrollable variables on the process variability ([6], pg. 190). For example, consider the process model y t = βu t + γz t + v t where u t is a controllable factor, z t is a noise factor and β and γ are known parameters. Let ẑ t denote the one-step ahead forecast of z t computed from on-line measurements of the noise factor. Then the feedforward control action that will make the expected response to be on target is u t = (T γẑ t )/β.

32 15 Bayesian Inference in time series and regression models In Bayesian models, any unknown quantities, including parameters, are modeled as random variables. Inferences about the parameters are based on the posterior distribution of the parameters, found by applying Bayes rule: p(β y) p(y β)p(β) (2.6) where β is the vector of parameters, p(y β) is the likelihood function of the observed output data y and p(β) is the prior distribution which reflects the knowledge the user has about the parameters before observing the data. Prior distributions that are uniform across all values of the unknown parameter are used to reflect very little prior information about the parameter and are called vague, diffuse or noninformative. Conversely, prior distributions that are not uniform across all values of the parameter are used to express the prior belief that some values of the parameter are more likely than other values and are called informative. The Bayesian method also provides a natural way of making inferences on future values of the response using the posterior predictive distribution. If ỹ is a future value of the response, it is possible to make inferences on ỹ from its posterior distribution given by (Gelman et. al. [34]) p(ỹ y) = p(ỹ β, y)p(β y)dβ. β It is noted that the posterior predictive distribution of the response accounts for the uncertainty in the true value of the parameters β because it assumes that β is a random variable and evaluates its posterior distribution using Bayes rule (2.6).

33 16 In this dissertation, Bayesian time series and regression models will be employed. Bayesian analysis of time series models were studied by Marriott et. al. [53] and Monahan [61]. The method discussed by Monahan [61] uses numerical integration to compute the normalization constant of the posterior distribution of the ARMA parameters. However, he considered only low order ARMA models. Marriott et. al. [53] presented a more general approach that works for any order ARMA models. They used Markov chain Monte Carlo (MCMC) methods to simulate the posterior distribution of the ARMA parameters. In order to account for the stationarity and invertibility constraints in the MCMC sampling procedure, a linear transformation was applied on the ARMA parameters, so that the transformed variables are unconstrained and hence can be more easily sampled. Appendix 6D reviews this method in detail. Press [73] studied Bayesian univariate and multivariate regression models. He showed that the posterior predictive density for a diffuse prior on the parameters is a t distribution in the case of single response, and a multivariate t distribution in the case of multiple responses. 2.2 Closed-loop System Identification Methods In Chapter 3 of this thesis, we present a new method for the identification of Box-Jenkins transfer function models in closed-loop conditions that utilizes prior knowledge about the process. For our purposes, the objective of system identification is to obtain a transfer function (TF) for the process dynamics and an ARIMA model for the disturbance from operating data. Identification methods in both open-loop (i.e. when no feedback control is

34 17 (a) Open-loop operation (b) Closed-loop operation ε t ε t Disturbance ARIMA model Disturbance ARIMA model u t Σ y t -T Σ u t Σ y t Process TF model Controller Process TF model Figure 2.1: Open-loop and Closed-loop operation of a process exercised) and in closed-loop operation have been studied. Figure 2.1 illustrates these two modes of operation. In this section we review methods for closed-loop identification of transfer function models, methods for delay estimation and methods for change-point detection. Identification of transfer function models Parameter Identifiability and Persistency of Excitation Conditions A process is said to be parameter identifiable if the estimators of its model parameters converge to the true values as the sample size tends to infinity (i.e. the estimators are consistent) [45]. The parameter identifiability condition therefore guarantees that all parameters of a process can be uniquely determined from data. The input signal u t that satisfies this requirement is said to be persistently exciting the process ([7], pg. 73). It has been shown that parameter identifiability in closed-loop can be achieved in several ways: (i) By applying an independent signal in the controller, such as, by using a time varying target or set-point T or by adding a random dither signal that is persistently exciting of sufficient order (Söderström et. al., [80]); (ii) By switching between different

35 18 linear controllers (Ljung et. al., [45]); and (iii) By having a complex enough controller in the feedback loop (Söderström et. al., [79]). The persistency of excitation condition on the input is related to the multi-collinearity problem in regression. This will be illustrated in the following example. Example. Persistency of Excitation: This example, taken from Åström and Wittenmark ([6], pg. 82), considers the true process description y t = a 1 y t 1 + b 1 u t 1 + ɛ t. Assuming a model of the correct structure, the input-output data {y 1, u 1,..., y N, u N } can be written in the matrix form as y 2 y 3. y N }{{} y = y 1 u 1 y 2 u 2. y N 1. u N 1 } {{ } X a 1 + b 1 }{{} β ɛ 2 ɛ 3. ɛ N. }{{} ɛ The least squares estimate of the parameter vector is ˆβ = (X X) 1 X y. The parameters β of the true process can be uniquely estimated only when the matrix X X is invertible, that is, when X is of full column rank. If this condition is satisfied than the input signal u t is persistently exciting (of order 2 = dim(β) in this case) and the process is parameter identifiable.

36 19 Suppose a proportional controller u t = c 1 y t is used instead. It can be seen that X = y 1 c 1 y 1 y 2 c 1 y 2.. y N 1 c 1 y N 1 and the columns of X are linearly dependent hence the process is not parameter identifiable. However, if a control input with an added dither signal d t is used, that is, if u t = c 1 y t + d t, then the columns of X are linearly independent and parameter identifiability is restored. It can also be seen that if a higher order controller u t = (c 1 + c 2 B)y t is used then X = y 1 c 1 y 1 + c 2 y 0 y 2 c 1 y 2 + c 2 y 1.. y N 1 c 1 y N 1 + c 2 y N 2 and the columns of X are linearly independent hence parameter identifiability is also restored. Closed-Loop Identification of Box-Jenkins Models: Closed-loop identification with Box-Jenkins transfer function models has been studied by various authors. Box and MacGregor [19] applied the results by Söderström et. al. [80] to processes under minimum mean squared error (MMSE) control. It was assumed that the true input-output delay and model orders were known. They showed that the precision of the parameter estimates can be improved by the addition of a dither signal. Luceño [50] investigated the case of switching between multiple PI controllers during the operation of

37 20 the process and the effects this has on the quality of parameter estimates, but also assumed that the correct model orders and the delay were known. Use of Prior Knowledge in Closed-Loop Identification: The idea of applying prior process knowledge in closed-loop identification to improve parameter estimates was discussed by Box and MacGregor [18] for the cases of prior knowledge on the transfer function dynamics and on the disturbance model. MacGregor and Fogal [51] investigated the benefits of knowing the disturbance model by applying disturbance model pre-filters in identification. Ljung and Forssell [47] showed that complete knowledge of the true disturbance model is sufficient for parameter identifiability. Pan and Del Castillo [68] considered the stationarity and invertibility conditions as a form of prior knowledge and showed by example that closed-loop identification is possible by utilizing these conditions. While there have been several studies on utilizing prior process knowledge in process identification, no systematic study that addresses the relative effectiveness of different forms of prior knowledge was provided. Delay Estimation and Change-point Detection Methods Chapter 4 of this dissertation discusses new methods for the estimation of input-output delay of dynamic processes under closed-loop operating condition. The presented methods use Time Series change-point detection methods. As an illustration of the input-output delay of a transfer function model, consider Figure 2.2. Here, the input u t is increased in a step fashion, the output y t is the response of the process to this input and the input-output delay k is the number of whole time periods

38 21 u t (Input) t y t (Output) t k Figure 2.2: Input-output delay of a dynamic process that elapses between making the change in the input and observing its effect at the output. As it can be seen, due to the dynamics of the process, the output reaches its steady state level exponentially after several time periods rather than in a step fashion after one time period. As it will be discussed in Chapter 4, the process dynamics must be taken into consideration when using a change-point detection method to estimate the delay. Accurate knowledge of the process delay is important in control design and in closedloop process identification. Many optimal control design methods, such as minimum MSE control [6], are very sensitive to the knowledge of the process delay and the closed-loop process under a designed controller may become unstable if the model delay does not coincide with the process delay [44]. For the design of minimum MSE controllers, Bohlin [11] shows that the parameters of such controllers can be uniquely estimated from closedloop data as long as the process delay is known. The knowledge of the delay is also important for the control performance indices that are used to compare the performance of

39 22 a control loop to its best achievable performance and to decide whether it is warranted to re-tune a controller. In the controller performance indices proposed by Harris et. al. [38] and by Horch and Isaksson [40] the delay is the only process information that is required to be known. Delay estimation has been extensively studied in the control systems literature. Björklund and Ljung [10] give a comparison study of some of the recent open-loop delay estimation approaches. Kurz and Goedecke [44] develops a recursive least squares process identification approach for processes with time varying delays. Isaksson, Horch and Dumont [41] propose a closed-loop delay estimation method that uses a Laguerre transfer function model and step response tests. Transfer function models that use Laguerre functions have received considerable attention in the system identification literature. In a Laguerre transfer function model, G(B) is defined as a polynomial in Laguerre filters (which are obtained by taking the Z transform of the Laguerre functions) rather than a polynomial in B. Laguerre functions, through their orthonormality properties, have the advantage over using simple polynomials in B of more accurately approximating transient process behavior and hence are suitable in estimating the delay in a step response test [84]. Delay Estimation Using Laguerre Transfer Function Models The delay estimation method presented by Isaksson et. al. [41] can be summarized as follows. The estimated Laguerre transfer function is first written in the frequency domain and it is factorized into a Minimum Phase part and an All Phase part as G(e iωt s ) = G mp (e iωt s )G ap (e iωt s )