Umea University Report UMINF Department of Computing Science ISSN S Umea January 21, 1997 Sweden Algorithms and Software for th

Size: px

Start display at page:

Download "Umea University Report UMINF Department of Computing Science ISSN S Umea January 21, 1997 Sweden Algorithms and Software for th"

Curtis Hart
5 years ago
Views:

1 Umea University Report UMINF Department of Computing Science ISSN S Umea January 21, 1997 Sweden Algorithms and Software for the Computation of Parameters Occurring in ODE-models Licentiate Thesis Gunilla Wikstrom Submitted to the Department of Computing Science at Umea University in partial fulllment of the requirements for the degree of licentiate of technology.

2 .

3 Abstract One way to describe dynamic systems is to use mathematical models in the form of systems of ordinary dierential equations. These models contain parameters, that must be estimated to obtain the best t to the given data. Nonlinear parameter estimation is often a tedious task, due to integration of dierential equations and calculation of derivatives for the optimization part. It is risky, too, because of the existence of several local minima of the relevant optimization problem. However, it is possible to get around some problems by using short cut methods, i.e. simplied methods that result in approximate parameters. If these approximate parameters are not good enough, they can be used as initial values for more complicated estimation methods, hopefully resulting in more accurate results. When the model parameters appear linearly in the chosen dynamic model, e.g. in applications such as chemical kinetics, theoretical biology and ecology, it is possible to estimate approximate parameters both directly and iteratively. The rst approach includes the use of given data in combination with quadrature formulas. This results in linear least squares problems that can be solved, without iteration, for approximate model parameters. An alternative is to treat the problem as a separable nonlinear least squares problem. In this case, both the dependent variables and the model parameters are unknown. This approach is iterative, but the given data can be used as initial values. In this thesis, approximate methods based on the two previously mentioned approaches are tested and analyzed. It is shown that the direct type of short cut methods, i.e. without iteration, works well and is in agreement with theory in the case of negligible measurement errors and equally spaced values of the independent variable. The dierential equations can be transformed into dierence equations in two ways. Either by using the given model and approximation of derivatives or by integration and approximation of integrals. According to analysis, the latter way is to prefer when errors in the data are dominating. Smoothing can be used to reduce the inuence of errors in the data. However, for these types of short cut methods tests show that the type of cubic smoothing splines used deteriorate the results when truncation errors dominate. By regarding the discretized problem as a constrained separable nonlinear least squares problem, a more advanced approximate method is proposed and it is shown that the results of this iterative method are improved compared to those of the previously mentioned less advanced approximate methods. Furthermore, a Matlab toolbox is developed that can be combined to perform dierent types of tasks: simulation, generation of simulated data, sensitivity analysis, parameter estimation from data (real or generated), statistical analysis of the parameters. Key words: parameter estimation, systems of ordinary dierential equations, short cut methods, linear and separable nonlinear least squares problems, Matlab toolbox (for nonlinear least squares).

4 Short summary in Swedish Modeller, grundade pa matematiska och experimentella principer, ar ett satt att forsoka forsta och beskriva saker i var omgivning. Konstruktionen av denna typ av modeller inkluderar era delmoment, tex forsoksplanering, val av lamplig modellstruktur, numerisk behandling, analys av resultat. I denna rapport behandlas det numeriska delmomentet, dvs hur man pa ett sa bra satt som mojligt ska anvanda datorn som hjalpmedel vid matematiska berakningar samt fa en uppfattning om hur stora fel anvandandet av detta hjalpmedel ger upphov till. De matematiska modeller som studeras ar av generell struktur, dvs de innehaller obekanta parametrar vilka maste bestammas for att man ska fa en praktiskt anvandbar modell motsvarande en specik tillampning. Genom att anpassa den generella modellen till information om den specika tillampningen, dvs valja parametrar sa att skillnaden mellan den generella modellen och motsvarande matdata blir sa liten som mojligt, kan en anvandbar modell bestammas. Tyvarr ar det ibland svart att bestamma de \basta" parametrarna bland ett ertal \acceptabla" dito. Dessutom kan berakningarna, mha dator, av de valda matematiska modellerna vara komplicerade samt resurskravande. I denna rapport studeras darfor satt att komma till ratta med dessa problem. Komplicerade metoder forenklas, vilket som resultat ger approximativa parametrar och darmed approximativa modeller. Ibland kan dock dessa approximativa modeller vara tillrackligt bra, och om inte, kan de anvandas som komplement till mera komplicerade metoder sa att mojligheten att bestamma \ratt" parametrar, och darmed modell, okar samt att tidsatgangen for detta reduceras. Vidare presenteras programvara, innehallande bade forenklade samt mera komplicerade metoder, med vars hjalp man interaktivt kan losa den ovan beskrivna problemtypen.

5 Introduction and Summary v Introduction In applied science it is common to describe dynamic systems by mathematical models in the form of systems of dierential equations (ODEs). These models may contain parameters, that must be estimated to obtain the best t to the given data. The construction of these types of models includes dierent parts, e.g. design of experiments, choice of model structure, numerical treatment and analysis of results. This thesis focuses on the numerical treatment of the problem. Examples of approximate methods, or so called short cut methods, in the case where all parameters enter nonlinearly in the model, can be found in [3]. If some of the parameters enter linearly, then the method described by [5] can be used to reduce the parameter space. Short cut methods are particularly attractive when all the parameters enter linearly in the model, because then the approximate parameter values can be found without iteration, see [2] and [6]. For the type of short cut methods to be studied in this thesis, the model is linear in the parameters. Moreover, the ODEs are approximated by quadrature formulas, resulting in dierence equations. The sum of squared dierences, between the approximate model values and corresponding measurements, is minimized. When both parameters and dependent variables are treated as unknowns, a separable nonlinear least squares (LSQ) problem has to be solved by iteration and the given data can be used as initial values. If only the parameters are considered to be unknown, corresponding approximate values can be found without iteration by solving a linear LSQ problem. This thesis presents some results of a combined research project between the Department of Computing Science, Umea University and the Department of Numerical Analysis and Computing Science, Royal Institute of Technology. As a result, a Matlab toolbox exists called DIFFPAR (DIFFerential equations with unknown PA- Rameters), [1]. This toolbox has now been extended, with a graphical user interface and short cut methods. Summary of papers This thesis consists of four papers. Linear appearance of parameters in the model is assumed in the rst three papers. A special kind of short cut methods, resulting in linear LSQ problems, is analyzed and tested in the rst two papers. In the rst paper, measurement errors are assumed to be negligible and the independent variable equally spaced. In the second paper, both uniformly and normally distributed errors are treated. The third paper presents a short cut method that includes a solution of separable nonlinear LSQ problems. Finally, in the fourth paper, a brief manual for the toolbox DIFFPAR is presented. This toolbox can also be used to solve problems without the restriction of linear appearance of parameters in the model. Scalars are denoted by lower-case letters, vectors by lower-case bold type letters and matrices by upper-case bold type letters.

6 vi Paper II Paper I & II Consider the following parameter estimation problem min k;y(t;k) 1 2 MX (~y i? y(t i ; k)) T W i (~y i? y(t i ; k)) i=1 (1) s:t: _y(t; k) = G(t; y(t)) k y(t0) = y 0 where t is the independent variable. The dependent variable, y(t; k), satises the model at time t. Smoothed or nonsmoothed given data at time t i is denoted by ~y i and W i is diagonal weight matrices with nonnegative elements, where i = 1; : : : ; M. The parameter vector is denoted as k and G(t; y(t)) is a matrix-valued function. By discretization of the ODE-model, e.g. approximate problem is given as min k;y s:t: MX 1 (~y i? y i ) T W i (~y i? y i ) 2 i=1 y i = y 0 + H global;i k = y i?1 + H local;i k by using the Trapeziodal rule, an (2) where y T = (y 1 T ; : : : ; y T M ) is an approximate numerical solution to the constraint. The matrices R H global;i and H local;i are approximations, by use of given data, of the t integrals i R G(; y())d and i G(; y())d; i = 1; : : : ; M. t0 t t i?1 Since H in general is nonlinear in y, because G(t; y(t)) is a function of y(t), we are dealing with a nonlinear parameter estimation problem. However, the purpose of the study in paper I & II is to nd approximate values for the parameters by solving a linear LSQ problem. This is possible to do, when the parameter k enters linearly in the chosen model, by Taylor expansion of the approximate constraint and use of given data. However, these approximations introduce both truncation errors and errors due to errors in the data. In paper I, the eect of truncation errors is studied when errors in the data are assumed to be negligible. It is shown that the two dierent approaches, i.e. \local" and \global" integration by quadrature, give results in agreement with theory. In paper II, also errors in the data are included in the analysis. As a result, it is shown that the \global" approach is to prefer, due to cancellation eects, and that this formulation better corresponds to the original problem (1). It is also shown, by computer simulations, that the use of smoothing, to reduce the eect of errors in the data, is almost negligible when errors in data dominate. When the truncation errors dominate, the special kind of smoothing studied, should not be used. However, when neither of the two types of errors dominate, smoothing can have a positive eect.

7 Introduction and Summary vii Paper III By regarding the discretized problem (2) as a constrained separable nonlinear LSQ problem, when both the parameter k and the dependent variable y are considered unknown, it is possible to construct more advanced short cut methods. In this paper a modied version of L. Kaufman's, [4], algoritm for constrained nonlinear separable LSQ problems is suggested. It is shown that the proposed approximate method works well and is in agreement with theory under \nice" circumstances and satisfyingly under \real" circumstances. Moreover, the results of this method are better than those of the less advanced methods analyzed in paper I & II. Finally, some further improvements and extensions are suggested for the iterative short cut method. Paper IV The toolbox DIF F P AR handles more general problems than (1). It uses the ODEmodel, _y(t; k) = f(t; y(t; k)), and the initial values, y(t0) = y 0, do not have to be exactly known. The toolbox is based on the Gauss-Newton method with local regularization and can handle both non-sti and sti ODE-systems. It also has facilities for 1) iteration in logarithmated values of the parameters (implicit positivity constraint) 2) selection of which parameters that should be estimated 3) giving dierent weights for the measurements 4) handling the situation when the data for some components are missing and 5) facilities for nding initial values for the iteration process (when the parameter vector enters linearly in the model). The graphical user interface (GUI) is based on Matlab's handle graphics, and is mainly coded as one function with separate switches or cases that create user interface elements and dene callbacks. Acknowledgement I am most grateful to Prof. Per-Ake Wedin, my advisor, for giving me the possibility to become a graduate student and for his and Lennart Edsberg's ideas, encouragement and support. I would also like to thank those whom I have had interesting and fruitful discussions with and my family and friends for making me think about other things besides my graduate studies. References [1] Edsberg L. and Wedin P-A. Numerical Tools for Parameter Estimation in ODEsystems, Optimization Methods and Software, Vol. 6, pp , 1995 [2] Foss S. D. Estimates of Chemical Kinetic Rate Constants by Numerical Integration, Chemical Engineering Science, Vol. 26, pp , 1971

8 viii Paper II [3] Hosten L. A. A comparative study of short cut procedures for parameter estimation in dierential equations, Computers & Chemical Engineering, Vol. 3, pp , 1979 [4] Kaufman L. and Pereyra V. A method for separable nonlinear least squares problems with separable nonlinear equality constraints, pp , SIAM J. Numer. Anal., 15, 1978 [5] Lawton W. H. and Sylvestre E. A. Elimination of Linear Parameters in Nonlinear Regression, Technometrics, Vol. 13, pp , 1971 [6] Swartz J. and Bremermann H. Discussion of Parameter Estimation in Biological Modeling: Algorithms for Estimation and Evaluation of the Estimates, Journal of Mathematical Biology, Vol. 1, pp , 1975 "Although this may seem a paradox, all exact science is dominated by the idea of approximation.". - Bertrand Russel

MODELING OF ROCK GROUTING IN SATURATED VARIABLE APERTURE FRACTURES Injekteringsmodellering av vattenfyllda bergsprickor med varierande apertur

MODELING OF ROCK GROUTING IN SATURATED VARIABLE APERTURE FRACTURES Injekteringsmodellering av vattenfyllda bergsprickor med varierande apertur Liangchao Zou, Division of resources, energy and infrastructure,