Parameter Estimation of Mathematical Models Described by Differential Equations
|
|
- Lynette Wiggins
- 6 years ago
- Views:
Transcription
1 Parameter Estimation p.1/12 Parameter Estimation of Mathematical Models Described by Differential Equations Hossein Zivaripiran Department of Computer Science University of Toronto
2 Outline Parameter Estimation p.2/12 Modeling with Differential Equations (IVPs, DDEs) Modeling and Parameter Estimation Numerical Parameter Estimation of IVPs DDEs and Parameter Estimation
3 Modeling with DE - Formulation Parameter Estimation p.3/12 An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs) y (t) = f(t, y(t)) y(t 0 ) = y 0
4 Modeling with DE - Formulation Parameter Estimation p.3/12 An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs) y (t) = f(t, y(t)) y(t 0 ) = y 0 Retarded Delay Differential Equations (RDDEs) y (t) = f(t, y(t), y(t σ 1 ),,y(t σ ν )) for t 0 t t F y(t) = φ(t), for t t 0 σ i = σ i (t, y(t)) 0 delay (constant / time dependent / state dependent) φ(t) history function (constant / time dependent)
5 Modeling with DE - Formulation Parameter Estimation p.3/12 An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs) y (t) = f(t, y(t)) y(t 0 ) = y 0 Retarded Delay Differential Equations (RDDEs) y (t) = f(t, y(t), y(t σ 1 ),,y(t σ ν )) for t 0 t t F y(t) = φ(t), for t t 0 σ i = σ i (t, y(t)) 0 delay (constant / time dependent / state dependent) φ(t) history function (constant / time dependent) Neutral Delay Differential Equations (NDDEs) y (t) = f(t, y(t), y(t σ 1 ),,y(t σ ν ), y (t σ ν+1 ),,y (t σ ν+ω )) for t 0 t t F y(t) = φ(t), y (t) = φ (t), for t t 0,
6 Modeling with DE - Examples Parameter Estimation p.4/12 The Van der Pol Oscillator, d 2 x dt 2 µ(1 x2 ) dx dt + x = 0 using y 1 = x and y 2 = dx dt, y 1(t) = y 2 (t) y 2(t) = µ(1 y 2 1)y 2 y 1
7 Modeling with DE - Examples Parameter Estimation p.4/12 The Van der Pol Oscillator with µ = 1, y 1 (0) = 2, y 2 (0) = y t y t
8 Modeling with DE - Examples Parameter Estimation p.4/12 A neutral delay logistic Gause-type predator-prey system [Kuang 1991] y 1(t) = y 1 (t)(1 y 1 (t τ) ρy 1(t τ)) y 2(t)y 1 (t) 2 y 2(t) = y 2 (t) ( y1 (t) 2 ) y 1 (t) α y 1 (t) where α = 1/10, ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functions are for t 0. φ 1 (t) = t φ 2 (t) = t
9 Modeling with DE - Examples Parameter Estimation p.4/12 The predator-prey model y t y t
10 Modeling and Parameter Estimation - Definitions Parameter Estimation p.5/12 Parameterized Models A parameterized IVP y (t;p) = f(t, y(t;p);p) y(t 0 ) = y 0 (p) For example p = [µ] in the Van der Pol oscillator y 1(t) = y 2 (t) y 2(t) = µ(1 y1)y 2 2 y 1 A simple parameterized DDE y (t;p) = f(t, y(t;p), y(t σ(t;p));p) for t 0 (p) t y(t;p) = φ(t;p), for t t 0 (p)
11 Modeling and Parameter Estimation - Definitions Parameter Estimation p.5/12 Parameter Estimation Problem A System of Parameterized IVP y (t;p) = f(t, y(t;p);p) y(t 0 ) = y 0 (p) or DDE y (t;p) = f(t, y(t;p), y(t σ(t;p));p) for t 0 (p) t y(t;p) = φ(t;p), for t t 0 (p) A Set of Data (Observations/Measurements) {Y (γ i ) y(γ i ;p )} Estimate p by minimizing an objective function. e.g. W(p) = i [Y (γ i ) y(γ i ;p)] 2.
12 Modeling and Parameter Estimation - Importance Parameter Estimation p.6/12 Physical/Biological Phenomenon Mathematical Model Physical Laws / Empirical Rules Observed Data Sensitivity Analysis Refined Mathematical Model Parameter Estimation Practical Mathematical Model
13 Modeling and Parameter Estimation - Optimizers Parameter Estimation p.7/12 Algorithms for Nonlinear Least-Squares Unconstrained min p W(p) = i [Y (γ i ) y(γ i ;p)] 2. Constrained Levenberg-Marquardt Variations of Sequential Quadratic Programming (SQP) min p W(p) = i [Y (γ i ) y(γ i ;p)] 2, c j (p) = 0, j E, c j (p) 0, j I. Sequential Quadratic Programming (SQP)
14 Numerical Parameter Estimation of IVPs Parameter Estimation p.8/12 The initial value approach : 1. Choose an initial guess for the parameters 2. Solve model equations 3. Check optimality conditions, (if satisfied stop). 4. Choose a better value for the parameters and continue with (2) The main difficulty : If the parameters are far from the correct ones the trial trajectory soon loses contact to the measurements. If the optimization method needs to compute the gradient or the Hessian of the objective function, ( 2 ) W(p) p l p m ( ) W(p) = 2 ( ) y(γi ;p) [Y (γ i ) y(γ i ;p)] p l p i l = 2 [( ) ( ) ( y(γi ;p) y(γi ;p) 2 )] y(γ i ;p) [Y (γ i ) y(γ i ;p)] p i l p m p l p m the sensitivity equations are usually used to provide the required values. An alternative approach is to use a divided-difference approximation.
15 Numerical Parameter Estimation of IVPs Parameter Estimation p.8/12 A Simple Example, The IVP y 1(t) = ay 2 (t) y 2(t) = ay 1 (t) with initial conditions y 1 (0) = 0 y 2 (0) = 1 the analytical solution is y 1 (t) = sin(at) y 2 (t) = cos(at)
16 Numerical Parameter Estimation of IVPs y1(t) a * =1 0.4 a=1.05 a= a=1.15 a= a=1.5 a= t Parameter Estimation p.8/12
17 Numerical Parameter Estimation of IVPs W(a) a Parameter Estimation p.8/12
18 Numerical Parameter Estimation of IVPs Parameter Estimation p.8/12 Multiple shooting : Partitions the fitting interval into many subintervals, each having its own initial values. The measurements are used to get starting guesses for initial values of each interval. In an iterative process, the algorithm minimizes W(p) on the one hand and enforces the continuity of the full trajectory on the other hand. Advantages: Divergences are avoided and the danger of local minima is reduced.
19 Numerical Parameter Estimation of IVPs y1(t) t Parameter Estimation p.8/12
20 DDEs and Parameter Estimation Parameter Estimation p.9/12 The adapted initial value approach [ Paul, 1997 ] : Assume that jumps in the derivative of y(t;p) with respect to t occur at the points Λ(p) {λ 1 (p), λ 2 (p),...}. Such discontinuities, when arising from the initial point t 0 (p) (and the initial function φ(t;p)), may propagate into W(p) via the solution values {y(γ i ;p)}. The first and second order partial derivatives of the objective function are ( 2 ) W(p) = 2 p l p m ±± i ( ) W(p) = 2 ( ) y(γi ;p) [Y (γ i ) y(γ i ;p)] p l ± p i l [ ( ) ( ) y(γi ;p) y(γi ;p) p l ± p m ± ( 2 ) y(γ i ;p) [Y (γ i ) y(γ i ;p)] p l p m ± ±± ] A general rule for the propagation of discontinuities to W(p) [ Baker & Paul,1997 ] : if a discontinuity point λ r (p) coincides with one of the data points γ i, and λ r (p) varies as some parameters vary, then W(p) has a jump in its partial derivatives that correspond to the varying parameters.
21 DDEs and Parameter Estimation Parameter Estimation p.9/12 Multiple shooting [ Horbet et al., 2002 ] : Use cubic splines to parameterize the initial curves and formulate the continuity of the trajectory in terms of these spline variables. A two-phase procedure: During the first iterations of the optimization, the spline variables are held fixed because they are expected to be estimated well from the data. After the algorithm has converged for the first time, they are released and fitted together with the other variables until the final convergence is achieved. Can attain higher convergence rate in the case of noisy data.
22 DDEs and Parameter Estimation Parameter Estimation p.9/12 The full discretization approach [ Murphy, 1990 ]: Use linear splines to describe solution and delay functions. As the result the problem becomes a very large minimization problem. The number of mesh points is increased gradually to be able to satisfy the specified error tolerance. Although the method has the generality of dealing with any kinds of unknown parameters, it suffers from the heavy computations, slow convergence rate and possibility of being trapped in a local minimum.
23 DDEs and Parameter Estimation Parameter Estimation p.9/12 Non-smooth optimization Consider the continuous function y(t) = 5(t τ) + c, if t < τ 5(t τ) + c, if t τ with discontinuous derivative y (t) = 5, if t < τ 5, if t τ and observed value of y at the discontinuity point τ = 4.
24 DDEs and Parameter Estimation y(t) c τ t Parameter Estimation p.9/12
25 DDEs and Parameter Estimation 60 W(τ) 40 W(τ) τ τ Parameter Estimation p.9/12
26 DDEs and Parameter Estimation Parameter Estimation p.9/12 Try to find τ using MATLAB s unconstrained minimization routine fminunc 31 iterations. Try to use MATLAB s constrained minimization routine fmincon with the added constraint τ 4 2 iterations. Try to use MATLAB s constrained minimization routine fmincon with the added constraint τ 4 2 iterations.
27 DDEs and Parameter Estimation - Safe Minimization Parameter Estimation p.10/12 Appearance of Non-smoothness Partial derivatives(gradient) of the objective function ( ) W(p) = 2 p l ± i ( ) y(γi ;p) [Y (γ i ) y(γ i ;p)] p l ± ( ) 2 W(p) = 2 p l p m ±± i ( ) y(γi ;p) p l ± ( ) y(γi ;p) p m ± [Y (γ i ) y(γ i ;p)] ( ) 2 y(γ i ;p) p l p m ±± Recall the jump equation for sensitivities y (λ + r+1 p ) = y (λ r+1 l p ) + ( y (λ r+1 ) y (λ + r+1 )) λ r+1 (p) l p l The General Rule : A jump occurs in W(p) when a discontinuity point λ r+1 (p) passes a data point γ i
28 DDEs and Parameter Estimation - Safe Minimization Parameter Estimation p.10/12 Algorithms for Nonlinear Least-Squares Unconstrained min p W(p) = i [Y (γ i ) y(γ i ;p)] 2. Levenberg-Marquardt Variations of Sequential Quadratic Programming (SQP) Constrained min p W(p) = i [Y (γ i ) y(γ i ;p)] 2, c j (p) = 0, j E, c j (p) 0, j I. Sequential Quadratic Programming (SQP) The smoothness of functions involved in the problem, the objective function and constraints, is a necessary assumption.
29 DDEs and Parameter Estimation - Safe Minimization Parameter Estimation p.10/12 Avoiding The Non-smoothness y true solution continiously extended solution discontinuity point data point λ r (p) γ i λ r+1 (p) t y λ r (p) γ i λ r+1 (p) t Force the ordering by adding λ r (p) γ i λ r+1 (p) to the set of constraints. The partial derivatives (gradient) of the new constraints, λ r+1(p), can be computed recursively. p
30 DDEs and Parameter Estimation - Safe Minimization Parameter Estimation p.10/12 Safe Conduction of Optimization Let ContinuityConstarints[p Base ] = {λ rpbase (p) γ i λ rpbase +1(p)} Then we can describe the steps for finding a local optimum as the following 1. Start with p c p p new SQP(p c, ContinuityConstarints[p c ]) 3. If some of ContinuityConstarints[p c ] are active, then p c p new and continue at (2), otherwise stop with p = p new.
31 DDEs and Parameter Estimation - A Test Case Parameter Estimation p.11/12 Estimating τ for the predator-prey model y 1(t) = y 1 (t)(1 y 1 (t τ) ρy 1(t τ)) y 2(t)y 1 (t) 2 ( y 2(t) y1 (t) 2 ) = y 2 (t) y 1 (t) α Start with up to 10% random perturbation in original τ, and up to 3% randomly perturbed y(t; τ) as Data (Y ). For γ s we choose 10 random points, one of which is a discontinuity point. Run the parameter estimator 3 times. Results Estimator Choice FCN OBJ Very Simple 72, Using Sensitivities 26, Adding Constraints 9, y 1 (t) 2 + 1
32 DDEs and Parameter Estimation - The Role of Data Parameter Estimation p.12/12 Back to First Talk!
Numerical Methods - Boundary Value Problems for ODEs
Numerical Methods - Boundary Value Problems for ODEs Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Boundary Value Problems for ODEs 2013 1 / 14 Outline 1 Boundary Value
More informationDelay Differential Equations with Constant Lags
Delay Differential Equations with Constant Lags L.F. Shampine Mathematics Department Southern Methodist University Dallas, TX 75275 shampine@smu.edu S. Thompson Department of Mathematics & Statistics Radford
More informationParameter Estimation for ODEs using a Cross-Entropy Approach. Bo Wang
Parameter Estimation for ODEs using a Cross-Entropy Approach by Bo Wang A thesis submitted in conformity with the requirements for the degree of Master of Science Graduate Department of Computer Science
More informationAPPLICATIONS OF FD APPROXIMATIONS FOR SOLVING ORDINARY DIFFERENTIAL EQUATIONS
LECTURE 10 APPLICATIONS OF FD APPROXIMATIONS FOR SOLVING ORDINARY DIFFERENTIAL EQUATIONS Ordinary Differential Equations Initial Value Problems For Initial Value problems (IVP s), conditions are specified
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one
More informationModified Milne Simpson Method for Solving Differential Equations
Modified Milne Simpson Method for Solving Differential Equations R. K. Devkate 1, R. M. Dhaigude 2 Research Student, Department of Mathematics, Dr. Babasaheb Ambedkar Marathwada University, Aurangabad,
More informationDelay Differential Equations Part II: Time and State dependent Lags
Delay Differential Equations Part II: Time and State dependent Lags L.F. Shampine Department of Mathematics Southern Methodist University Dallas, Texas 75275 shampine@smu.edu www.faculty.smu.edu/shampine
More informationCHAPTER 5: Linear Multistep Methods
CHAPTER 5: Linear Multistep Methods Multistep: use information from many steps Higher order possible with fewer function evaluations than with RK. Convenient error estimates. Changing stepsize or order
More informationThe estimation problem ODE stability The embedding method The simultaneous method In conclusion. Stability problems in ODE estimation
Mathematical Sciences Institute Australian National University HPSC Hanoi 2006 Outline The estimation problem ODE stability The embedding method The simultaneous method In conclusion Estimation Given the
More informationHomogeneous Equations with Constant Coefficients
Homogeneous Equations with Constant Coefficients MATH 365 Ordinary Differential Equations J. Robert Buchanan Department of Mathematics Spring 2018 General Second Order ODE Second order ODEs have the form
More informationSection 2.1 Differential Equation and Solutions
Section 2.1 Differential Equation and Solutions Key Terms: Ordinary Differential Equation (ODE) Independent Variable Order of a DE Partial Differential Equation (PDE) Normal Form Solution General Solution
More informationCollocation approximation of the monodromy operator of periodic, linear DDEs
p. Collocation approximation of the monodromy operator of periodic, linear DDEs Ed Bueler 1, Victoria Averina 2, and Eric Butcher 3 13 July, 2004 SIAM Annual Meeting 2004, Portland 1=Dept. of Math. Sci.,
More informationConstrained optimization: direct methods (cont.)
Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a
More informationOrdinary Differential Equations
Ordinary Differential Equations Introduction: first order ODE We are given a function f(t,y) which describes a direction field in the (t,y) plane an initial point (t 0,y 0 ) We want to find a function
More informationFirst Order Linear Ordinary Differential Equations
First Order Linear Ordinary Differential Equations The most general first order linear ODE is an equation of the form p t dy dt q t y t f t. 1 Herepqarecalledcoefficients f is referred to as the forcing
More informationGENG2140, S2, 2012 Week 7: Curve fitting
GENG2140, S2, 2012 Week 7: Curve fitting Curve fitting is the process of constructing a curve, or mathematical function, f(x) that has the best fit to a series of data points Involves fitting lines and
More informationCS520: numerical ODEs (Ch.2)
.. CS520: numerical ODEs (Ch.2) Uri Ascher Department of Computer Science University of British Columbia ascher@cs.ubc.ca people.cs.ubc.ca/ ascher/520.html Uri Ascher (UBC) CPSC 520: ODEs (Ch. 2) Fall
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationDelay Differential Equations Part I: Constant Lags
Delay Differential Equations Part I: Constant Lags L.F. Shampine Department of Mathematics Southern Methodist University Dallas, Texas 75275 shampine@smu.edu www.faculty.smu.edu/shampine Delay Differential
More informationTrajectory-based optimization
Trajectory-based optimization Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2012 Emo Todorov (UW) AMATH/CSE 579, Winter 2012 Lecture 6 1 / 13 Using
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationSMA 208: Ordinary differential equations I
SMA 208: Ordinary differential equations I First Order differential equations Lecturer: Dr. Philip Ngare (Contacts: pngare@uonbi.ac.ke, Tue 12-2 PM) School of Mathematics, University of Nairobi Feb 26,
More informationNumerical algorithms for one and two target optimal controls
Numerical algorithms for one and two target optimal controls Sung-Sik Kwon Department of Mathematics and Computer Science, North Carolina Central University 80 Fayetteville St. Durham, NC 27707 Email:
More informationCh 6.4: Differential Equations with Discontinuous Forcing Functions
Ch 6.4: Differential Equations with Discontinuous Forcing Functions! In this section focus on examples of nonhomogeneous initial value problems in which the forcing function is discontinuous. Example 1:
More informationNumerical Optimization Algorithms
Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms
More informationOptimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections
Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations Subsections One-dimensional Unconstrained Optimization Golden-Section Search Quadratic Interpolation Newton's
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationReview for Exam 2 Ben Wang and Mark Styczynski
Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note
More informationParallelizing large scale time domain electromagnetic inverse problem
Parallelizing large scale time domain electromagnetic inverse problems Eldad Haber with: D. Oldenburg & R. Shekhtman + Emory University, Atlanta, GA + The University of British Columbia, Vancouver, BC,
More informationFirst Order Systems of Linear Equations. or ODEs of Arbitrary Order
First Order Systems of Linear Equations or ODEs of Arbitrary Order Systems of Equations Relate Quantities Examples Predator-Prey Relationships r 0 = r (100 f) f 0 = f (r 50) (Lokta-Volterra Model) Systems
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationIntegration, differentiation, and root finding. Phys 420/580 Lecture 7
Integration, differentiation, and root finding Phys 420/580 Lecture 7 Numerical integration Compute an approximation to the definite integral I = b Find area under the curve in the interval Trapezoid Rule:
More information13. Nonlinear least squares
L. Vandenberghe ECE133A (Fall 2018) 13. Nonlinear least squares definition and examples derivatives and optimality condition Gauss Newton method Levenberg Marquardt method 13.1 Nonlinear least squares
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More information1 Error Analysis for Solving IVP
cs412: introduction to numerical analysis 12/9/10 Lecture 25: Numerical Solution of Differential Equations Error Analysis Instructor: Professor Amos Ron Scribes: Yunpeng Li, Mark Cowlishaw, Nathanael Fillmore
More informationMA3232 Summary 5. d y1 dy1. MATLAB has a number of built-in functions for solving stiff systems of ODEs. There are ode15s, ode23s, ode23t, ode23tb.
umerical solutions of higher order ODE We can convert a high order ODE into a system of first order ODEs and then apply RK method to solve it. Stiff ODEs Stiffness is a special problem that can arise in
More informationElementary Differential Equations
Elementary Differential Equations George Voutsadakis 1 1 Mathematics and Computer Science Lake Superior State University LSSU Math 310 George Voutsadakis (LSSU) Differential Equations January 2014 1 /
More informationLecture 9 Approximations of Laplace s Equation, Finite Element Method. Mathématiques appliquées (MATH0504-1) B. Dewals, C.
Lecture 9 Approximations of Laplace s Equation, Finite Element Method Mathématiques appliquées (MATH54-1) B. Dewals, C. Geuzaine V1.2 23/11/218 1 Learning objectives of this lecture Apply the finite difference
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationINTRODUCTION TO COMPUTER METHODS FOR O.D.E.
INTRODUCTION TO COMPUTER METHODS FOR O.D.E. 0. Introduction. The goal of this handout is to introduce some of the ideas behind the basic computer algorithms to approximate solutions to differential equations.
More informationFirst and Second Order Differential Equations Lecture 4
First and Second Order Differential Equations Lecture 4 Dibyajyoti Deb 4.1. Outline of Lecture The Existence and the Uniqueness Theorem Homogeneous Equations with Constant Coefficients 4.2. The Existence
More informationLeast-Squares Fitting of Model Parameters to Experimental Data
Least-Squares Fitting of Model Parameters to Experimental Data Div. of Mathematical Sciences, Dept of Engineering Sciences and Mathematics, LTU, room E193 Outline of talk What about Science and Scientific
More informationChapter 3 Numerical Methods
Chapter 3 Numerical Methods Part 2 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization 1 Outline 3.2 Systems of Equations 3.3 Nonlinear and Constrained Optimization Summary 2 Outline 3.2
More information8 Example 1: The van der Pol oscillator (Strogatz Chapter 7)
8 Example 1: The van der Pol oscillator (Strogatz Chapter 7) So far we have seen some different possibilities of what can happen in two-dimensional systems (local and global attractors and bifurcations)
More informationNumerical approximation for optimal control problems via MPC and HJB. Giulia Fabrini
Numerical approximation for optimal control problems via MPC and HJB Giulia Fabrini Konstanz Women In Mathematics 15 May, 2018 G. Fabrini (University of Konstanz) Numerical approximation for OCP 1 / 33
More informationBindel, Fall 2011 Intro to Scientific Computing (CS 3220) Week 6: Monday, Mar 7. e k+1 = 1 f (ξ k ) 2 f (x k ) e2 k.
Problem du jour Week 6: Monday, Mar 7 Show that for any initial guess x 0 > 0, Newton iteration on f(x) = x 2 a produces a decreasing sequence x 1 x 2... x n a. What is the rate of convergence if a = 0?
More informationChapter 2: First Order DE 2.4 Linear vs. Nonlinear DEs
Chapter 2: First Order DE 2.4 Linear vs. Nonlinear DEs First Order DE 2.4 Linear vs. Nonlinear DE We recall the general form of the First Oreder DEs (FODE): dy = f(t, y) (1) dt where f(t, y) is a function
More informationSection 2.1 (First Order) Linear DEs; Method of Integrating Factors. General first order linear DEs Standard Form; y'(t) + p(t) y = g(t)
Section 2.1 (First Order) Linear DEs; Method of Integrating Factors Key Terms/Ideas: General first order linear DEs Standard Form; y'(t) + p(t) y = g(t) Integrating factor; a function μ(t) that transforms
More informationThe Plan. Initial value problems (ivps) for ordinary differential equations (odes) Review of basic methods You are here: Hamiltonian systems
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit V Solution of Differential Equations Part 2 Dianne P. O Leary c 2008 The Plan
More informationThe Laplace Transform (Sect. 4.1). The Laplace Transform (Sect. 4.1).
The Laplace Transform (Sect. 4.1). s of Laplace Transforms. The Laplace Transform (Sect. 4.1). s of Laplace Transforms. The definition of the Laplace Transform. Definition The function F : D F R is the
More informationOrdinary Differential Equations
Ordinary Differential Equations Professor Dr. E F Toro Laboratory of Applied Mathematics University of Trento, Italy eleuterio.toro@unitn.it http://www.ing.unitn.it/toro September 19, 2014 1 / 55 Motivation
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationNumerical Methods for Embedded Optimization and Optimal Control. Exercises
Summer Course Numerical Methods for Embedded Optimization and Optimal Control Exercises Moritz Diehl, Daniel Axehill and Lars Eriksson June 2011 Introduction This collection of exercises is intended to
More informationSummer School on Delay Differential Equations and Applications
Summer School on Delay Differential Equations and Applications Dobbiaco (BZ), Italy, June 26 30, 2006 The numerical solution of delay differential equations Page 1 of 211 M. Zennaro Dipartimento di Matematica
More informationMinimization of Static! Cost Functions!
Minimization of Static Cost Functions Robert Stengel Optimal Control and Estimation, MAE 546, Princeton University, 2017 J = Static cost function with constant control parameter vector, u Conditions for
More informationMaths III - Numerical Methods
Maths III - Numerical Methods Matt Probert matt.probert@york.ac.uk 4 Solving Differential Equations 4.1 Introduction Many physical problems can be expressed as differential s, but solving them is not always
More informationSolution Manual for: Numerical Computing with MATLAB by Cleve B. Moler
Solution Manual for: Numerical Computing with MATLAB by Cleve B. Moler John L. Weatherwax July 5, 007 Chapter 7 (Ordinary Differential Equations) Problem 7.1 Defining the vector y as y Then we have for
More informationNumerical Methods - Initial Value Problems for ODEs
Numerical Methods - Initial Value Problems for ODEs Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Initial Value Problems for ODEs 2013 1 / 43 Outline 1 Initial Value
More informationFitting Linear Statistical Models to Data by Least Squares I: Introduction
Fitting Linear Statistical Models to Data by Least Squares I: Introduction Brian R. Hunt and C. David Levermore University of Maryland, College Park Math 420: Mathematical Modeling January 25, 2012 version
More informationHot-Starting NLP Solvers
Hot-Starting NLP Solvers Andreas Wächter Department of Industrial Engineering and Management Sciences Northwestern University waechter@iems.northwestern.edu 204 Mixed Integer Programming Workshop Ohio
More informationThe Direct Transcription Method For Optimal Control. Part 2: Optimal Control
The Direct Transcription Method For Optimal Control Part 2: Optimal Control John T Betts Partner, Applied Mathematical Analysis, LLC 1 Fundamental Principle of Transcription Methods Transcription Method
More informationChapter #4 EEE8086-EEE8115. Robust and Adaptive Control Systems
Chapter #4 Robust and Adaptive Control Systems Nonlinear Dynamics.... Linear Combination.... Equilibrium points... 3 3. Linearisation... 5 4. Limit cycles... 3 5. Bifurcations... 4 6. Stability... 6 7.
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationOutline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems
Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationIntroduction to Nonlinear Optimization Paul J. Atzberger
Introduction to Nonlinear Optimization Paul J. Atzberger Comments should be sent to: atzberg@math.ucsb.edu Introduction We shall discuss in these notes a brief introduction to nonlinear optimization concepts,
More informationThe integrating factor method (Sect. 1.1)
The integrating factor method (Sect. 1.1) Overview of differential equations. Linear Ordinary Differential Equations. The integrating factor method. Constant coefficients. The Initial Value Problem. Overview
More informationUnconstrained Multivariate Optimization
Unconstrained Multivariate Optimization Multivariate optimization means optimization of a scalar function of a several variables: and has the general form: y = () min ( ) where () is a nonlinear scalar-valued
More informationNonlinear Least Squares
Nonlinear Least Squares Stephen Boyd EE103 Stanford University December 6, 2016 Outline Nonlinear equations and least squares Examples Levenberg-Marquardt algorithm Nonlinear least squares classification
More informationLecture 1. Scott Pauls 1 3/28/07. Dartmouth College. Math 23, Spring Scott Pauls. Administrivia. Today s material.
Lecture 1 1 1 Department of Mathematics Dartmouth College 3/28/07 Outline Course Overview http://www.math.dartmouth.edu/~m23s07 Matlab Ordinary differential equations Definition An ordinary differential
More informationFitting Linear Statistical Models to Data by Least Squares: Introduction
Fitting Linear Statistical Models to Data by Least Squares: Introduction Radu Balan, Brian R. Hunt and C. David Levermore University of Maryland, College Park University of Maryland, College Park, MD Math
More informationLevenberg-Marquardt Method for Solving the Inverse Heat Transfer Problems
Journal of mathematics and computer science 13 (2014), 300-310 Levenberg-Marquardt Method for Solving the Inverse Heat Transfer Problems Nasibeh Asa Golsorkhi, Hojat Ahsani Tehrani Department of mathematics,
More informationModelling biological oscillations
Modelling biological oscillations Shan He School for Computational Science University of Birmingham Module 06-23836: Computational Modelling with MATLAB Outline Outline of Topics Van der Pol equation Van
More informationComparative Study of Numerical Methods for Optimal Control of a Biomechanical System Controlled Motion of a Human Leg during Swing Phase
Comparative Study of Numerical Methods for Optimal Control of a Biomechanical System Controlled Motion of a Human Leg during Swing Phase International Master s Programme Solid and Fluid Mechanics ANDREAS
More informationMathematical Models. MATH 365 Ordinary Differential Equations. J. Robert Buchanan. Spring Department of Mathematics
Mathematical Models MATH 365 Ordinary Differential Equations J. Robert Buchanan Department of Mathematics Spring 2018 Ordinary Differential Equations The topic of ordinary differential equations (ODEs)
More informationMultidisciplinary System Design Optimization (MSDO)
Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Today s Topics Sequential
More informationChap. 20: Initial-Value Problems
Chap. 20: Initial-Value Problems Ordinary Differential Equations Goal: to solve differential equations of the form: dy dt f t, y The methods in this chapter are all one-step methods and have the general
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 4, pp. 373-38, 23. Copyright 23,. ISSN 68-963. ETNA A NOTE ON THE RELATION BETWEEN THE NEWTON HOMOTOPY METHOD AND THE DAMPED NEWTON METHOD XUPING ZHANG
More informationMathematical Models. MATH 365 Ordinary Differential Equations. J. Robert Buchanan. Fall Department of Mathematics
Mathematical Models MATH 365 Ordinary Differential Equations J. Robert Buchanan Department of Mathematics Fall 2018 Ordinary Differential Equations The topic of ordinary differential equations (ODEs) is
More informationOrdinary Differential Equations. Session 7
Ordinary Differential Equations. Session 7 Dr. Marco A Roque Sol 11/16/2018 Laplace Transform Among the tools that are very useful for solving linear differential equations are integral transforms. An
More informationNumerical Methods for PDE-Constrained Optimization
Numerical Methods for PDE-Constrained Optimization Richard H. Byrd 1 Frank E. Curtis 2 Jorge Nocedal 2 1 University of Colorado at Boulder 2 Northwestern University Courant Institute of Mathematical Sciences,
More informationCS 450 Numerical Analysis. Chapter 8: Numerical Integration and Differentiation
Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80
More informationUnconstrained minimization of smooth functions
Unconstrained minimization of smooth functions We want to solve min x R N f(x), where f is convex. In this section, we will assume that f is differentiable (so its gradient exists at every point), and
More informationFitting Linear Statistical Models to Data by Least Squares I: Introduction
Fitting Linear Statistical Models to Data by Least Squares I: Introduction Brian R. Hunt and C. David Levermore University of Maryland, College Park Math 420: Mathematical Modeling February 5, 2014 version
More informationMA108 ODE: Picard s Theorem
MA18 ODE: Picard s Theorem Preeti Raman IIT Bombay MA18 Existence and Uniqueness The IVP s that we have considered usually have unique solutions. This need not always be the case. MA18 Example Example:
More informationImplicitly and Explicitly Constrained Optimization Problems for Training of Recurrent Neural Networks
Implicitly and Explicitly Constrained Optimization Problems for Training of Recurrent Neural Networks Carl-Johan Thore Linköping University - Division of Mechanics 581 83 Linköping - Sweden Abstract. Training
More informationSolving ODEs and PDEs in MATLAB. Sören Boettcher
16.02.2009 Introduction Quick introduction to syntax ODE in the form of Initial Value Problems (IVP) what equations can handle how to code into how to choose the right solver how to get the solver to do
More informationOPTIMAL FUSION OF SENSOR DATA FOR DISCRETE KALMAN FILTERING Z. G. FENG, K. L. TEO, N. U. AHMED, Y. ZHAO, AND W. Y. YAN
Dynamic Systems and Applications 16 (2007) 393-406 OPTIMAL FUSION OF SENSOR DATA FOR DISCRETE KALMAN FILTERING Z. G. FENG, K. L. TEO, N. U. AHMED, Y. ZHAO, AND W. Y. YAN College of Mathematics and Computer
More informationChapter 6 - Ordinary Differential Equations
Chapter 6 - Ordinary Differential Equations 7.1 Solving Initial-Value Problems In this chapter, we will be interested in the solution of ordinary differential equations. Ordinary differential equations
More informationInitial value problems for ordinary differential equations
AMSC/CMSC 660 Scientific Computing I Fall 2008 UNIT 5: Numerical Solution of Ordinary Differential Equations Part 1 Dianne P. O Leary c 2008 The Plan Initial value problems (ivps) for ordinary differential
More informationLine Search Methods. Shefali Kulkarni-Thaker
1 BISECTION METHOD Line Search Methods Shefali Kulkarni-Thaker Consider the following unconstrained optimization problem min f(x) x R Any optimization algorithm starts by an initial point x 0 and performs
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationRATIO BASED PARALLEL TIME INTEGRATION (RAPTI) FOR INITIAL VALUE PROBLEMS
RATIO BASED PARALLEL TIME INTEGRATION (RAPTI) FOR INITIAL VALUE PROBLEMS Nabil R. Nassif Noha Makhoul Karam Yeran Soukiassian Mathematics Department, American University of Beirut IRISA, Campus Beaulieu,
More informationNumerical methods in molecular dynamics and multiscale problems
Numerical methods in molecular dynamics and multiscale problems Two examples T. Lelièvre CERMICS - Ecole des Ponts ParisTech & MicMac project-team - INRIA Horizon Maths December 2012 Introduction The aim
More informationA multigrid method for large scale inverse problems
A multigrid method for large scale inverse problems Eldad Haber Dept. of Computer Science, Dept. of Earth and Ocean Science University of British Columbia haber@cs.ubc.ca July 4, 2003 E.Haber: Multigrid
More informationNumerical Questions in ODE Boundary Value Problems
Numerical Questions in ODE Boundary Value Problems M.R.Osborne Contents 1 Introduction 2 2 ODE stability 6 2.1 Initial value problem stability.................. 6 2.2 Boundary value problem stability................
More informationSolving Delay Differential Equations (DDEs) using Nakashima s 2 Stages 4 th Order Pseudo-Runge-Kutta Method
World Applied Sciences Journal (Special Issue of Applied Math): 8-86, 3 ISSN 88-495; IDOSI Publications, 3 DOI:.589/idosi.wasj.3..am.43 Solving Delay Differential Equations (DDEs) using Naashima s Stages
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More information