Numerical Data Fitting in Dynamical Systems

Transcription

1 Numerical Data Fitting in Dynamical Systems

2 Applied Optimization Volume 77 Series Editors: Panos M. Pardalos University of Florida, U.S.A. Donald Hearn University of Florida, U.S.A. The titles published in this series are listed at the end of this volume.

3 NUlllerical Data Fitting in Dynalllical Systellls A Practical Introduction with Applications and Software by Klaus Schittkowski Department of Mathematics, University of Bayreuth, Bayreuth, Germany SPRINGER-SCIENCE+BUSINESS MEDIA, B.V.

4 A C.I.P. Catalogue record for this book is available from the Library of Congress. ISBN ISBN (ebook) DOI / Printed on acid-free paper All Rights Reserved 2002 Springer Science+Business Media Dordrecht Originally published by Kluwer Academic Publishers in 2002 Softcover reprint of the hardcover 1st edition 2002 No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written permission from the Publisher, with the exception of any material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work.

5 to Vii

6 Contents Preface 1. INTRODUCTION 1 2. MATHEMATICAL FOUNDATIONS 7 1 Optimality Criteria Notation Convexity and Constraint Qualification Necessary and Sufficient Optimality Criteria 10 2 Sequential Quadratic Programming Methods The Quadratic Programming Subproblem Line Search and Quasi-Newton Updates Convergence Systems of Nonlinear Equations 20 3 Least Squares Methods Optimality Conditions Gauss-Newton and Related Methods Solution of Least Squares Problems by SQP Methods Constrained Least Squares Optimization Alternative Norms 33 4 Numerical Solution of Ordinary Differential Equations Explicit Solution Methods Implicit Solution Methods Sensitivity Equations Internal Numerical Differentiation 46 5 Numerical Solution of Differential Algebraic Equations Algebraic Equations Index of a Differential Algebraic Equation Index Reduction and Drift Effect Projection Methods 55 xi vii

7 viii NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS 5.5 Consistent Initial Values Implicit Solution Methods 62 6 Numerical Solution of One-Dimensional Partial Differential Equations The General Time-Dependent Model Some Special Classes of Equations The Method of Lines Partial Differential Algebraic Equations Difference Formulae Polynomial Interpolation Upwind Formulae for Hyperbolic Equations Essentially Non-Oscillatory Schemes Systems of Hyperbolic Equations Sensitivity Equations Laplace Transforms Basic Properties Numerical Back-Transformation Automatic Differentiation Forward Mode Reverse Mode Statistical Interpretation of Results DATA FITTING MODELS Explicit Model Functions Laplace Transforms Steady State Equations Ordinary Differential Equations Standard Formulation Differential Algebraic Equations Switching Points Constraints Shooting Method Boundary Value Problems Variable Initial Times Partial Differential Equations Standard Formulation Partial Differential Algebraic Equations Flux Functions Coupled Ordinary Differential Algebraic Equations Integration Areas and Transition Conditions Switching Points 167

8 Contents ix 5.7 Constraints Optimal Control Problems NUMERICAL EXPERIMENTS Test Environment Numerical Pitfalls Local Solutions Slow Convergence Badly Scaled Data and Parameters Non-Identifiability of Models Errors in Experimental Data Inconsistent Constraints Non-Differentiable Model Functions Oscillating Model Functions Testing the Validity of Models Mass Balance and Steady State Analysis Statistical Analysis Constraints Performance Evaluation Comparing Least Squares Algorithms Individual Numerical Results CASE STUDIES Linear Pharmacokinetics Receptor-Ligand Binding Study Robot Design Multibody System of a Truck Binary Distillation Column Acetylene Reactor Transdermal Application of Drugs Groundwater Flow Cooling a Hot Strip Mill Drying Maltodextrin in a Convection Oven Fluid Dynamics of Hydro Systems Horn Radiators for Satellite Communication 278

9 x NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS Appendix A: Software Installation 1 Hardware and Software Requirements 2 System Setup 3 Packing List Appendix B: Test Examples 1 Explicit Model Functions 2 Laplace Transforms 3 Steady State Equations Ordinary Differential Equations Differential Algebraic Equations Partial Differential Equations Partial Differential Algebraic Equations Appendix C: The PCOMP Language Appendix D: Generation of Fortran Code 1 Model Equations 1.1 Input of Explicit Model Functions 1.2 Input of Laplace Transformations 1.3 Input of Systems of Steady State Equations 1.4 Input of Ordinary Differential Equations 1.5 Input of Differential Algebraic Equations 1.6 Input of Time-Dependent Partial Differential Equations 1.7 Input of Partial Differential Algebraic Equations 2 Execution of Generated Code References Index

10 Preface Real life phenomena in engineering, natural, or medical sciences are often described by a mathematical model with the goal to analyze numerically the behaviour of the system. Advantages of mathematical models are their cheap availability, the possibility of studying extreme situations that cannot be handled by experiments, or of simulating real systems during the design phase before constructing a first prototype. Moreover, they serve to verify decisions, to avoid expensive and time consuming experimental tests, to analyze, understand, and explain the behaviour of systems, or to optimize design and production. As soon as a mathematical model contains differential dependencies from an additional parameter, typically the time, we call it a dynamical model. There are two key questions always arising in a practical environment: 1 Is the mathematical model correct? 2 How can I quantify model parameters that cannot be measured directly? In principle, both questions are easily answered as soon as some experimental data are available. The idea is to compare measured data with predicted model function values and to minimize the differences over the whole parameter space. We have to reject a model if we are unable to find a reasonably accurate fit. To summarize, parameter estimation or data fitting, respectively, is extremely important in all practical situations, where a mathematical model and corresponding experimental data are available to describe the behaviour of a dynamical system. The main goal of the book is to give an overview of numerical methods that are needed to compute parameters of a dynamical model by a least squares fit. The mathematical equations that must be provided by the system analyst are explicit model functions or steady state systems in the simplest situations, or responses of dynamical systems defined by ordinary differential equations, differential algebraic equations, or one-dimensional partial differential equations. Many different mathematical disciplines must be combined, and the intention is to present at least some fundamental ideas of the numerical methods needed, so that available software can be applied successfully. It must be noted that there are two alternative aspects not treated in this book. First, we do not emphasize statistical analysis, which is also known as nonlinear regression or xi

11 xii NUMERICAL DATA FITTING IN DYNAMICAL SYSTEMS nonlinear parameter estimation. Moreover, we do not investigate the question whether parameters of a dynamical model can be identified at all, and under which mathematical conditions. It is supposed that a user is able to prepare a well-defined model, where the dynamical system is uniquely solvable and where the parameters can be identified by a least squares fit. There exist numerous qualified textbooks for both topics mentioned, from which additional information can be retrieved. It is assumed that the typical reader is familiar with basic mathematical notation of linear algebra and analysis, as for example learned in elementary calculus lectures. No additional knowledge about mathematical theory is required. New concepts are presented in an elementary form and are illustrated by detailed analytical and numerical examples. Extensive numerical results are included to show the efficiency of modern mathematical algorithms. We also discuss possible pitfalls in the form of warnings that even the most qualified numerical algorithms we know today can fail or produce unacceptable responses. The practical progress of mathematical models and data fitting calculations is illustrated by case studies from pharmaceutics, mechanical, electrical or chemical engineering, and ecology. To be able to repeat all numerical tests presented in the book and to play with algorithms, data, and solution tolerances, an interactive software system is included that runs under Windows 95/98/NT4.0/2000. The program contains the mathematical algorithms described in the book. The database consists of 1,000 illustrative examples, which can be used as benchmark test problems. Among them is a large number of real life models (learning by doing). The book is the outcome of my research activities in this area over the last 20 years with emphasis on the development of numerical algorithms for optimization problems. It would have been impossible to design applicable mathematical algorithms and to implement the corresponding software without intense discussions, contacts, and cooperation with firms, for example Boehringer Ingelheim Pharma, BASF Ludwigshafen, Siemens Munich, Schloemann-Siemag Hilchenbach, EADS Munich, Bayer Sarnia, Dornier Satellite Systems Munich, and many research institutions at universities. Particularly, I would like to thank Dr. M. Wolf from the University of Bonn, Department of Pharmaceutics, for providing many dynamical models describing pharmaceutical applications, and for encouraging the investigation of models based on partial differential equations. Part of my research was supported by projects funded by the BMBF research program Anwendungsbezogene Verbundprojekte in der Mathematik and the DFG research program Echtzeit-Optimierung grafter Systeme. Klaus Schittkowski