Numerical Data Fitting in Dynamical Systems

Numerical Data Fitting in Dynamical Systems

Applied Optimization Volume 77 Series Editors: Panos M. Pardalos University of Florida, U.S.A. Donald Hearn University of Florida, U.S.A. The titles published in this series are listed at the end of this volume.

NUlllerical Data Fitting in Dynalllical Systellls A Practical Introduction with Applications and Software by Klaus Schittkowski Department of Mathematics, University of Bayreuth, Bayreuth, Germany SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.

A C.I.P. Catalogue record for this book is available from the Library of Congress. ISBN 978-1-4757-6050-7 ISBN 978-1-4419-5762-7 (ebook) DOI 10.1007/978-1-4419-5762-7 Printed on acid-free paper All Rights Reserved 2002 Springer Science+Business Media Dordrecht Originally published by Kluwer Academic Publishers in 2002 Softcover reprint of the hardcover 1st edition 2002 No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

to Vii

Contents Preface 1. INTRODUCTION 1 2. MATHEMATICAL FOUNDATIONS 7 1 Optimality Criteria 7 1.1 Notation 7 1.2 Convexity and Constraint Qualification 9 1.3 Necessary and Sufficient Optimality Criteria 10 2 Sequential Quadratic Programming Methods 14 2.1 The Quadratic Programming Subproblem 14 2.2 Line Search and Quasi-Newton Updates 16 2.3 Convergence 18 2.4 Systems of Nonlinear Equations 20 3 Least Squares Methods 23 3.1 Optimality Conditions 23 3.2 Gauss-Newton and Related Methods 24 3.3 Solution of Least Squares Problems by SQP Methods 27 3.4 Constrained Least Squares Optimization 31 3.5 Alternative Norms 33 4 Numerical Solution of Ordinary Differential Equations 38 4.1 Explicit Solution Methods 38 4.2 Implicit Solution Methods 40 4.3 Sensitivity Equations 43 4.4 Internal Numerical Differentiation 46 5 Numerical Solution of Differential Algebraic Equations 48 5.1 Algebraic Equations 48 5.2 Index of a Differential Algebraic Equation 50 5.3 Index Reduction and Drift Effect 52 5.4 Projection Methods 55 xi vii

viii NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS 5.5 Consistent Initial Values 60 5.6 Implicit Solution Methods 62 6 Numerical Solution of One-Dimensional Partial Differential Equations 66 6.1 The General Time-Dependent Model 66 6.2 Some Special Classes of Equations 68 6.3 The Method of Lines 74 6.4 Partial Differential Algebraic Equations 78 6.5 Difference Formulae 81 6.6 Polynomial Interpolation 84 6.7 Upwind Formulae for Hyperbolic Equations 85 6.8 Essentially Non-Oscillatory Schemes 93 6.9 Systems of Hyperbolic Equations 98 6.10 Sensitivity Equations 101 7 Laplace Transforms 104 7.1 Basic Properties 104 7.2 Numerical Back-Transformation 107 8 Automatic Differentiation 109 8.1 Forward Mode 109 8.2 Reverse Mode 112 9 Statistical Interpretation of Results 115 3. DATA FITTING MODELS 119 1 Explicit Model Functions 120 2 Laplace Transforms 124 3 Steady State Equations 126 4 Ordinary Differential Equations 128 4.1 Standard Formulation 128 4.2 Differential Algebraic Equations 129 4.3 Switching Points 131 4.4 Constraints 137 4.5 Shooting Method 141 4.6 Boundary Value Problems 146 4.7 Variable Initial Times 148 5 Partial Differential Equations 151 5.1 Standard Formulation 151 5.2 Partial Differential Algebraic Equations 153 5.3 Flux Functions 154 5.4 Coupled Ordinary Differential Algebraic Equations 157 5.5 Integration Areas and Transition Conditions 162 5.6 Switching Points 167

Contents ix 5.7 Constraints 169 6 Optimal Control Problems 175 4. NUMERICAL EXPERIMENTS 181 1 Test Environment 182 2 Numerical Pitfalls 183 2.1 Local Solutions 183 2.2 Slow Convergence 186 2.3 Badly Scaled Data and Parameters 189 2.4 Non-Identifiability of Models 192 2.5 Errors in Experimental Data 195 2.6 Inconsistent Constraints 197 2.7 Non-Differentiable Model Functions 201 2.8 Oscillating Model Functions 205 3 Testing the Validity of Models 208 3.1 Mass Balance and Steady State Analysis 208 3.2 Statistical Analysis 210 3.3 Constraints 212 4 Performance Evaluation 216 4.1 Comparing Least Squares Algorithms 216 4.2 Individual Numerical Results 218 5. CASE STUDIES 231 1 Linear Pharmacokinetics 231 2 Receptor-Ligand Binding Study 236 3 Robot Design 239 4 Multibody System of a Truck 243 5 Binary Distillation Column 248 6 Acetylene Reactor 252 7 Transdermal Application of Drugs 257 8 Groundwater Flow 263 9 Cooling a Hot Strip Mill 266 10 Drying Maltodextrin in a Convection Oven 269 11 Fluid Dynamics of Hydro Systems 273 12 Horn Radiators for Satellite Communication 278

x NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS Appendix A: Software Installation 1 Hardware and Software Requirements 2 System Setup 3 Packing List Appendix B: Test Examples 1 Explicit Model Functions 2 Laplace Transforms 3 Steady State Equations 4 5 6 7 Ordinary Differential Equations Differential Algebraic Equations Partial Differential Equations Partial Differential Algebraic Equations Appendix C: The PCOMP Language Appendix D: Generation of Fortran Code 1 Model Equations 1.1 Input of Explicit Model Functions 1.2 Input of Laplace Transformations 1.3 Input of Systems of Steady State Equations 1.4 Input of Ordinary Differential Equations 1.5 Input of Differential Algebraic Equations 1.6 Input of Time-Dependent Partial Differential Equations 1.7 Input of Partial Differential Algebraic Equations 2 Execution of Generated Code References Index 285 285 285 286 287 288 295 296 299 317 320 331 335 345 345 345 346 347 348 349 350 352 355 359 387

Preface Real life phenomena in engineering, natural, or medical sciences are often described by a mathematical model with the goal to analyze numerically the behaviour of the system. Advantages of mathematical models are their cheap availability, the possibility of studying extreme situations that cannot be handled by experiments, or of simulating real systems during the design phase before constructing a first prototype. Moreover, they serve to verify decisions, to avoid expensive and time consuming experimental tests, to analyze, understand, and explain the behaviour of systems, or to optimize design and production. As soon as a mathematical model contains differential dependencies from an additional parameter, typically the time, we call it a dynamical model. There are two key questions always arising in a practical environment: 1 Is the mathematical model correct? 2 How can I quantify model parameters that cannot be measured directly? In principle, both questions are easily answered as soon as some experimental data are available. The idea is to compare measured data with predicted model function values and to minimize the differences over the whole parameter space. We have to reject a model if we are unable to find a reasonably accurate fit. To summarize, parameter estimation or data fitting, respectively, is extremely important in all practical situations, where a mathematical model and corresponding experimental data are available to describe the behaviour of a dynamical system. The main goal of the book is to give an overview of numerical methods that are needed to compute parameters of a dynamical model by a least squares fit. The mathematical equations that must be provided by the system analyst are explicit model functions or steady state systems in the simplest situations, or responses of dynamical systems defined by ordinary differential equations, differential algebraic equations, or one-dimensional partial differential equations. Many different mathematical disciplines must be combined, and the intention is to present at least some fundamental ideas of the numerical methods needed, so that available software can be applied successfully. It must be noted that there are two alternative aspects not treated in this book. First, we do not emphasize statistical analysis, which is also known as nonlinear regression or xi

xii NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS nonlinear parameter estimation. Moreover, we do not investigate the question whether parameters of a dynamical model can be identified at all, and under which mathematical conditions. It is supposed that a user is able to prepare a well-defined model, where the dynamical system is uniquely solvable and where the parameters can be identified by a least squares fit. There exist numerous qualified textbooks for both topics mentioned, from which additional information can be retrieved. It is assumed that the typical reader is familiar with basic mathematical notation of linear algebra and analysis, as for example learned in elementary calculus lectures. No additional knowledge about mathematical theory is required. New concepts are presented in an elementary form and are illustrated by detailed analytical and numerical examples. Extensive numerical results are included to show the efficiency of modern mathematical algorithms. We also discuss possible pitfalls in the form of warnings that even the most qualified numerical algorithms we know today can fail or produce unacceptable responses. The practical progress of mathematical models and data fitting calculations is illustrated by case studies from pharmaceutics, mechanical, electrical or chemical engineering, and ecology. To be able to repeat all numerical tests presented in the book and to play with algorithms, data, and solution tolerances, an interactive software system is included that runs under Windows 95/98/NT4.0/2000. The program contains the mathematical algorithms described in the book. The database consists of 1,000 illustrative examples, which can be used as benchmark test problems. Among them is a large number of real life models (learning by doing). The book is the outcome of my research activities in this area over the last 20 years with emphasis on the development of numerical algorithms for optimization problems. It would have been impossible to design applicable mathematical algorithms and to implement the corresponding software without intense discussions, contacts, and cooperation with firms, for example Boehringer Ingelheim Pharma, BASF Ludwigshafen, Siemens Munich, Schloemann-Siemag Hilchenbach, EADS Munich, Bayer Sarnia, Dornier Satellite Systems Munich, and many research institutions at universities. Particularly, I would like to thank Dr. M. Wolf from the University of Bonn, Department of Pharmaceutics, for providing many dynamical models describing pharmaceutical applications, and for encouraging the investigation of models based on partial differential equations. Part of my research was supported by projects funded by the BMBF research program Anwendungsbezogene Verbundprojekte in der Mathematik and the DFG research program Echtzeit-Optimierung grafter Systeme. Klaus Schittkowski