# Numerical Algorithms as Dynamical Systems

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 A Study on Numerical Algorithms as Dynamical Systems Moody Chu North Carolina State University

2 What This Study Is About? To recast many numerical algorithms as special dynamical systems, whence to derive new understandings and insights. To exploit the notion of dynamical systems as a special realization process for problems arising from the field of numerical analysis, whence to develop possible new schemes. To forge the idea that, while these things" have been differentiated from each other, they can also be integrated together.

3 Topics of Lessons 1. Numerical Analysis versus Dynamical Systems 2. Dynamics and Controls in Solving Algebraic Equations 3. Lax Evolution and Its Equivalents 4. Lotka-Volterra Equations and Singular Values 5. Dynamics via Group Actions 6. Structure-preserving Dynamical Systems and Applications

4 Lecture 1 Numerical Analysis versus Dynamical Systems An Overview Moody T. Chu North Carolina State University June 14, MCM, CAS

5 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

6 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

7 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

8 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

9 Realization Process A logical procedure used to reason or to infer. Usually done deductively or inductively. Aim to draw conclusion or make decision.

10 Dynamical System A realization process in the recursive form. One state develops into another state by following a certain specific rule. Often appears in the form of an iterative procedure or a differential equation.

11 Basic Components Two abstract problems: One is a make-up and is easy. The other is the real problem and is difficult. A bridge: A continuous path connecting the two problems. A path that is easy to follow. A numerical method: A method for moving along the bridge. A method that is readily available.

12 Building the Bridge Specified guidance is available. The bridge is constructed by monitoring the values of certain specified functions. The path is guaranteed to work. e.g. The projected gradient method. Only some general guidance is available. A bridge is built in a straightforward way. No guarantee the path will be complete. e.g. The homotopy method. No guidance at all. A bridge is built seemingly by accident. Usually deeper mathematical theory is involved. e.g. The isospectral flows.

13 Characteristics of a Bridge A bridge, if it exists, usually is characterized by an ordinary differential equation. The discretization of a bridge, or a numerical method in traveling along a bridge, usually produces an iterative scheme.

14 Symmetric Eigenvalue Problem The mathematical problem: A symmetric matrix A 0 is given. Solve the equation A 0 x = λx for a nonzero vector x and a scalar λ.

15 An Iterative Realization The QR decomposition: A = QR where Q is orthogonal and R is upper triangular. The QR algorithm (Francis 61): Theory: A k = Q k R k, A k+1 = R k Q k. Every matrix A k has the same eigenvalues of A 0. The sequence {A k } converges to a diagonal matrix.

16 A Continuous Realization Lie algebra decomposition: X = X o + X + + X where X o is the diagonal, X + the strictly upper triangular, and X the strictly lower triangular part of X. Toda lattice (Symes 82, Deift el al 83): Theory: dx dt = [X, X X T ] X(0) = X 0. Sampled at integer times, {X(k)} gives the same sequence as does the QR algorithm applied to the matrix A 0 = exp(x 0 ).

17 How Is the Bridge Built? The bridge between X 0 and the limit point of Toda flow is built on the basis of maintaining isospectrum. Points to ponder: What motivates the construction of the Toda lattice? Why is convergence guaranteed? What is the advantage of one approach over the other?

18 Nonlinear Algebraic Equations The mathematical problem: A sufficiently smooth function f : R n R n is given. Solve the equation f(x) = 0.

19 An Iterative Realization The Newton method: Theory: x k+1 = x k α k (f (x k )) 1 f(x k ). The sequence {x k } converges quadratically to a solution, if x 0 is sufficiently close to that solution.

20 A Continuous Realization The Newton homotopy (Smale 76, Keller 78, etc.): H(x, t) = f(x) tf(x 0 ). The zero set {(x, t) R n+1 H(x, t) = 0} forms a smooth curve. The homotopy curve: f (x) dx ds 1 dt f(x) t ds = 0, x(0) = x 0, t(0) = 1, where s is the arc length. Suppose f (x) is nonsingular. Then written as Theory: dx ds = dt 1 ds t (f (x)) 1 f(x). With appropriate step size chosen, an Euler step is equivalent to a regular Newton method.

21 How Is the Bridge Built? The bridge is built upon the hope that the obvious solution will be deformed mathematically into the solution that we are seeking for. Points to ponder: Will this idea always work? How to mathematically design an appropriate homotopy?

22 Least Squares Matrix Approximation The mathematical problem: A symmetric matrix N and a set of real values {λ 1,..., λ n} are given. Find a least squares approximation of N that has the prescribed eigenvalues.

23 A Standard Formulation Minimize F (Q) := 1 2 QT ΛQ N 2 Subject to Q T Q = I Equality Constrained Optimization: Augmented Lagrangian methods. Sequential quadratic programming methods. Interior point method. All these techniques employ iterative realization. Linearize the Lagrangian. Sequential quadratic approximation. Follow the central path.

24 A Continuous Realization The projection of the gradient of F can easily be calculated. Projected gradient flow (Chu&Driessel 90): dx dt = [X, [X, N]] X(0) = Λ X := Q T ΛQ. Flow X(t) moves in a descent direction to reduce X N 2. Theory: The optimal solution X can be fully characterized by the spectral decomposition of N and is unique.

25 How Is the Bridge Built? The bridge between a starting point and the optimal point is built on the basis of systematically reducing the difference between the current position and the target position. Points to ponder: How to get to the limit point of the gradient flow efficiently?

26 Mutual Implications Differential System Discrete Approximation Time-1 Sampling Iterative Scheme

27 Technology Limitations Floating-point arithmetic is the most common and effective way for computation. Almost a mandate to discretize a continuous problem. A majority of numerical algorithms in practice are iterative in nature.

28 Iterative Scheme x k+1 = G k (x k,..., x k p+1 ), k = 0, 1,..., A p-step sequential process (Ortega&Rheinboldt 00),: G k : D k V p V is a predetermined map. Could be stationary. A bridge intends to achieve a certain goal. V is a designated set of states. Could be a manifold. Need initial values x 0, x 1,..., x p+1.

29 Initial Value Problem dx dt = f(t, x), x(0) = x 0, f defines a flow moving in a specific direction. In many applications, x(t) is supposed to preserve certain quantities, such as mass, volume, or stay on a certain manifold. Challenging to realize this conservation law.

30 Numerical ODE Techniques Conventional Runge-Kutta method: x n+1 = x n + h R c r k r, r=1 } {{ } G n(x n) R c r = 1. r=1 k r := f(t n + a r h, x n + h P R s=1 brsks), P R s=1 brs = ar. Conventional linear multi-step method: x n+1 := p (α i x n i + hβ i f n i ) +hβ 1 f n+1. i=0 } {{ } G(x n,...,x n p) Special purpose geometric integrator...

31 Sarkovskii s Theorem Consider the iteration x k+1 = f (x k ), where f : R R is continuous. A number x 0 is of period m if x m = x 0 and having least period m if x k x 0 for all 0 < k < m. Arrange positive integers in the following ordering: 3, 5, 7, 9,..., 2 3, 2 5, 2 7,..., 2 2 3, 2 2 5,..., 2 4, 2 3, 2 2, 2, 1. Sharkovskii s theorem: If f has a periodic point of least period m and m n in the above ordering, then f has also a periodic point of least period n. Remarkable for its lack of hypotheses and its qualitative universality.

32 Two Different Views Numerical Analysis Consider systems with only trivial asymptotic behavior. Concerned about convergence and stability. Meant to trace the flow of ODEs with reliable and reasonable accuracy local behavior of the trajectory. Meant to converge to a unique equilibrium point. Dynamical System Interested in systems with more complicated behavior. Study overall trajectories. Want to differentiate the intrinsic geometric structure long-term behavior of the trajectory. Search for limit cycles, bifurcations, or strange attractors.

33 Logistic Equation dx dt = x(1 x), x(0) = x 0. Exact solution: x(t) = x 0 x 0 + e t (1 x 0 ), Converge to the equilibrium x( ) = 1 for any initial value x 0 0.

34 Explicit Euler Iteration x k+1 = x k + ɛx k (1 x k ), ɛ = step size. Fix t, x n x(t) as n in the sense of ɛ = t n. With n = 90 ɛ and 0 < ɛ 3, plot the absolute error x n x(nɛ). Theoretic error estimate O(ɛ). Fix ɛ, iterate the Euler step 5000 times to see the limit points. A cascade of period doubling as ɛ increases. The equilibrium x( ) = 1 is not even an attractor for large ɛ.

35 1.4 Feigenbaum Diagram of Limit Points 10 0 Error of Euler Iterates at t " limit points x n x(n!) ! !

36 Implicit Euler Iterations x k+1 = x k + ɛx k+1 (1 x k+1 ), x k+1 = x k + ɛx k (1 x k+1 ). Converge to the equilibrium x( ) = 1 for any step size ɛ. Points to ponder: Distinguish limiting behavior between an iterative algorithm designed originally to solve a specific problem and a discrete approximation of a differential system formulated to mimic an existing iterative algorithm. Distinguish asymptotic behavior between a differential system developed originally from a specific realization process and its discrete approximation which becomes an iterative scheme.

37 Gradient Flow dx dt = F (x), x(0) = x 0, F : R n R is a specified smooth objective function. Goal: Find the limit point x = lim t x(t) of the gradient flow x(t). Wish list: Do not want to solve the equation F (x) = 0 by Newton-like methods. Ignore the gradient property. Might locate undesirable, dynamically unstable critical points. Do not want to follow the solution curve x(t) closely. Too expensive computation at the transient state.

38 Pseudo-transient Continuation Idea: Stay near the true trajectory, but not strive for accuracy. Employ one implicit Euler step with step size ɛ k : x k+1 = x k ɛ k F (x k+1 ). Perform only one correction using one Newton iteration starting at x k and accept the outcome as x k+1. ( ) 1 1 x k+1 = x k I n + 2 F (x k ) F (x k ). ɛ k The step size changes the nature of iterations. Small values of ɛ k Scheme behaves like a steepest descent method. Large values of ɛ k Scheme behaves like a Newton iteration.

39 Hamiltonian Flow with Poisson bracket { dpi dt = {p i, H}, dq i dt = {q i, H}, {f, g} := n i=1 i = 1,..., n, ( f g f ) g. q i p i p i q i Goal: Find the long-term evolution behavior. Wish list: Want to keep the simplectic form dp dq invariant Preserving qualitative properties of phase space trajectories.

40 Runge-Kutta Methods a B c If m ij := c i b ij + c j b ji c i c j = 0 for all 1 i, j R, then the RK method is simplectic. (Sanz-Serna 88). Gauss-Legengre RK methods are simplectic

### Numerical Methods for ODEs. Lectures for PSU Summer Programs Xiantao Li

Numerical Methods for ODEs Lectures for PSU Summer Programs Xiantao Li Outline Introduction Some Challenges Numerical methods for ODEs Stiff ODEs Accuracy Constrained dynamics Stability Coarse-graining

### Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations

Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations S. Y. Ha and J. Park Department of Mathematical Sciences Seoul National University Sep 23, 2013 Contents 1 Logistic Map 2 Euler and

### Introduction to Dynamical Systems Basic Concepts of Dynamics

Introduction to Dynamical Systems Basic Concepts of Dynamics A dynamical system: Has a notion of state, which contains all the information upon which the dynamical system acts. A simple set of deterministic

Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

### Symplectic integration. Yichao Jing

Yichao Jing Hamiltonian & symplecticness Outline Numerical integrator and symplectic integration Application to accelerator beam dynamics Accuracy and integration order Hamiltonian dynamics In accelerator,

### Lyapunov Stability Theory

Lyapunov Stability Theory Peter Al Hokayem and Eduardo Gallestey March 16, 2015 1 Introduction In this lecture we consider the stability of equilibrium points of autonomous nonlinear systems, both in continuous

### u n 2 4 u n 36 u n 1, n 1.

Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then

### Line Search Methods for Unconstrained Optimisation

Line Search Methods for Unconstrained Optimisation Lecture 8, Numerical Linear Algebra and Optimisation Oxford University Computing Laboratory, MT 2007 Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The Generic

### One Dimensional Dynamical Systems

16 CHAPTER 2 One Dimensional Dynamical Systems We begin by analyzing some dynamical systems with one-dimensional phase spaces, and in particular their bifurcations. All equations in this Chapter are scalar

### An introduction to Birkhoff normal form

An introduction to Birkhoff normal form Dario Bambusi Dipartimento di Matematica, Universitá di Milano via Saldini 50, 0133 Milano (Italy) 19.11.14 1 Introduction The aim of this note is to present an

### MATH 415, WEEKS 14 & 15: 1 Recurrence Relations / Difference Equations

MATH 415, WEEKS 14 & 15: Recurrence Relations / Difference Equations 1 Recurrence Relations / Difference Equations In many applications, the systems are updated in discrete jumps rather than continuous

### Graded Project #1. Part 1. Explicit Runge Kutta methods. Goals Differential Equations FMN130 Gustaf Söderlind and Carmen Arévalo

2008-11-07 Graded Project #1 Differential Equations FMN130 Gustaf Söderlind and Carmen Arévalo This homework is due to be handed in on Wednesday 12 November 2008 before 13:00 in the post box of the numerical

The Conjugate Gradient Method Lecture 5, Continuous Optimisation Oxford University Computing Laboratory, HT 2006 Notes by Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The notion of complexity (per iteration)

### Lecture 15 Newton Method and Self-Concordance. October 23, 2008

Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications

### Newton s Method. Javier Peña Convex Optimization /36-725

Newton s Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and

### BACKGROUND IN SYMPLECTIC GEOMETRY

BACKGROUND IN SYMPLECTIC GEOMETRY NILAY KUMAR Today I want to introduce some of the symplectic structure underlying classical mechanics. The key idea is actually quite old and in its various formulations

### Parallel-in-time integrators for Hamiltonian systems

Parallel-in-time integrators for Hamiltonian systems Claude Le Bris ENPC and INRIA Visiting Professor, The University of Chicago joint work with X. Dai (Paris 6 and Chinese Academy of Sciences), F. Legoll

### Numerical Optimization Algorithms

Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms

### SOLVING ODE s NUMERICALLY WHILE PRESERVING ALL FIRST INTEGRALS

SOLVING ODE s NUMERICALLY WHILE PRESERVING ALL FIRST INTEGRALS G.R.W. QUISPEL 1,2 and H.W. CAPEL 3 Abstract. Using Discrete Gradient Methods (Quispel & Turner, J. Phys. A29 (1996) L341-L349) we construct

### Numerical solution of ODEs

Numerical solution of ODEs Arne Morten Kvarving Department of Mathematical Sciences Norwegian University of Science and Technology November 5 2007 Problem and solution strategy We want to find an approximation

### = 0. = q i., q i = E

Summary of the Above Newton s second law: d 2 r dt 2 = Φ( r) Complicated vector arithmetic & coordinate system dependence Lagrangian Formalism: L q i d dt ( L q i ) = 0 n second-order differential equations

### Period-doubling cascades of a Silnikov equation

Period-doubling cascades of a Silnikov equation Keying Guan and Beiye Feng Science College, Beijing Jiaotong University, Email: keying.guan@gmail.com Institute of Applied Mathematics, Academia Sinica,

### Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Part 3 3.4 Differential Algebraic Systems 3.5 Integration of Differential Equations 1 Outline 3.4 Differential Algebraic Systems 3.4.1 Constrained Dynamics 3.4.2 First and Second

### Newton s Method. Ryan Tibshirani Convex Optimization /36-725

Newton s Method Ryan Tibshirani Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, Properties and examples: f (y) = max x

### Ordinary Differential Equations

Chapter 13 Ordinary Differential Equations We motivated the problem of interpolation in Chapter 11 by transitioning from analzying to finding functions. That is, in problems like interpolation and regression,

### Applied Math Qualifying Exam 11 October Instructions: Work 2 out of 3 problems in each of the 3 parts for a total of 6 problems.

Printed Name: Signature: Applied Math Qualifying Exam 11 October 2014 Instructions: Work 2 out of 3 problems in each of the 3 parts for a total of 6 problems. 2 Part 1 (1) Let Ω be an open subset of R

### Quasi-Newton Methods

Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications

### Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim

Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its

### 10-725/36-725: Convex Optimization Prerequisite Topics

10-725/36-725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the

### 8.1 Bifurcations of Equilibria

1 81 Bifurcations of Equilibria Bifurcation theory studies qualitative changes in solutions as a parameter varies In general one could study the bifurcation theory of ODEs PDEs integro-differential equations

The Conjugate Gradient Method Jason E. Hicken Aerospace Design Lab Department of Aeronautics & Astronautics Stanford University 14 July 2011 Lecture Objectives describe when CG can be used to solve Ax

### Lectures in Discrete Differential Geometry 2 Surfaces

Lectures in Discrete Differential Geometry 2 Surfaces Etienne Vouga February 4, 24 Smooth Surfaces in R 3 In this section we will review some properties of smooth surfaces R 3. We will assume that is parameterized

### Chapter 7 Iterative Techniques in Matrix Algebra

Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition

### Review for Exam 2 Ben Wang and Mark Styczynski

Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note

### which arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i

MODULE 6 Topics: Gram-Schmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.

### Lagrange multipliers. Portfolio optimization. The Lagrange multipliers method for finding constrained extrema of multivariable functions.

Chapter 9 Lagrange multipliers Portfolio optimization The Lagrange multipliers method for finding constrained extrema of multivariable functions 91 Lagrange multipliers Optimization problems often require

### Mathematical Systems Theory: Advanced Course Exercise Session 5. 1 Accessibility of a nonlinear system

Mathematical Systems Theory: dvanced Course Exercise Session 5 1 ccessibility of a nonlinear system Consider an affine nonlinear control system: [ ẋ = f(x)+g(x)u, x() = x, G(x) = g 1 (x) g m (x) ], where

### 6.2 Brief review of fundamental concepts about chaotic systems

6.2 Brief review of fundamental concepts about chaotic systems Lorenz (1963) introduced a 3-variable model that is a prototypical example of chaos theory. These equations were derived as a simplification

### Optimization Methods. Lecture 19: Line Searches and Newton s Method

15.93 Optimization Methods Lecture 19: Line Searches and Newton s Method 1 Last Lecture Necessary Conditions for Optimality (identifies candidates) x local min f(x ) =, f(x ) PSD Slide 1 Sufficient Conditions

### E5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization

E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained

### Trust Regions. Charles J. Geyer. March 27, 2013

Trust Regions Charles J. Geyer March 27, 2013 1 Trust Region Theory We follow Nocedal and Wright (1999, Chapter 4), using their notation. Fletcher (1987, Section 5.1) discusses the same algorithm, but

### 2D-Volterra-Lotka Modeling For 2 Species

Majalat Al-Ulum Al-Insaniya wat - Tatbiqiya 2D-Volterra-Lotka Modeling For 2 Species Alhashmi Darah 1 University of Almergeb Department of Mathematics Faculty of Science Zliten Libya. Abstract The purpose

### 7 Planar systems of linear ODE

7 Planar systems of linear ODE Here I restrict my attention to a very special class of autonomous ODE: linear ODE with constant coefficients This is arguably the only class of ODE for which explicit solution

### Self-Concordant Barrier Functions for Convex Optimization

Appendix F Self-Concordant Barrier Functions for Convex Optimization F.1 Introduction In this Appendix we present a framework for developing polynomial-time algorithms for the solution of convex optimization

### Solutions to Final Exam Sample Problems, Math 246, Spring 2011

Solutions to Final Exam Sample Problems, Math 246, Spring 2 () Consider the differential equation dy dt = (9 y2 )y 2 (a) Identify its equilibrium (stationary) points and classify their stability (b) Sketch

### NATIONAL UNIVERSITY OF SINGAPORE PC5215 NUMERICAL RECIPES WITH APPLICATIONS. (Semester I: AY ) Time Allowed: 2 Hours

NATIONAL UNIVERSITY OF SINGAPORE PC55 NUMERICAL RECIPES WITH APPLICATIONS (Semester I: AY 0-) Time Allowed: Hours INSTRUCTIONS TO CANDIDATES. This examination paper contains FIVE questions and comprises

### 3 Stability and Lyapunov Functions

CDS140a Nonlinear Systems: Local Theory 02/01/2011 3 Stability and Lyapunov Functions 3.1 Lyapunov Stability Denition: An equilibrium point x 0 of (1) is stable if for all ɛ > 0, there exists a δ > 0 such

### EE C128 / ME C134 Feedback Control Systems

EE C128 / ME C134 Feedback Control Systems Lecture Additional Material Introduction to Model Predictive Control Maximilian Balandat Department of Electrical Engineering & Computer Science University of

Conditional Gradient (Frank-Wolfe) Method Lecturer: Aarti Singh Co-instructor: Pradeep Ravikumar Convex Optimization 10-725/36-725 1 Outline Today: Conditional gradient method Convergence analysis Properties

### Chapter 9 Implicit integration, incompressible flows

Chapter 9 Implicit integration, incompressible flows The methods we discussed so far work well for problems of hydrodynamics in which the flow speeds of interest are not orders of magnitude smaller than

### Series Solutions. 8.1 Taylor Polynomials

8 Series Solutions 8.1 Taylor Polynomials Polynomial functions, as we have seen, are well behaved. They are continuous everywhere, and have continuous derivatives of all orders everywhere. It also turns

### Alexander Ostrowski

Ostrowski p. 1/3 Alexander Ostrowski 1893 1986 Walter Gautschi wxg@cs.purdue.edu Purdue University Ostrowski p. 2/3 Collected Mathematical Papers Volume 1 Determinants Linear Algebra Algebraic Equations

### Numerical Mathematics

Alfio Quarteroni Riccardo Sacco Fausto Saleri Numerical Mathematics Second Edition With 135 Figures and 45 Tables 421 Springer Contents Part I Getting Started 1 Foundations of Matrix Analysis 3 1.1 Vector

### Physics 6010, Fall 2016 Constraints and Lagrange Multipliers. Relevant Sections in Text:

Physics 6010, Fall 2016 Constraints and Lagrange Multipliers. Relevant Sections in Text: 1.3 1.6 Constraints Often times we consider dynamical systems which are defined using some kind of restrictions

### Exponential Integrators

Exponential Integrators John C. Bowman (University of Alberta) May 22, 2007 www.math.ualberta.ca/ bowman/talks 1 Exponential Integrators Outline Exponential Euler History Generalizations Stationary Green

### Lecture 13: Series Solutions near Singular Points

Lecture 13: Series Solutions near Singular Points March 28, 2007 Here we consider solutions to second-order ODE s using series when the coefficients are not necessarily analytic. A first-order analogy

### Computational Methods. Eigenvalues and Singular Values

Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations

### Lecture 10 Polynomial interpolation

Lecture 10 Polynomial interpolation Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn

### ODE Runge-Kutta methods

ODE Runge-Kutta methods The theory (very short excerpts from lectures) First-order initial value problem We want to approximate the solution Y(x) of a system of first-order ordinary differential equations

### Practical Optimization: Basic Multidimensional Gradient Methods

Practical Optimization: Basic Multidimensional Gradient Methods László Kozma Lkozma@cis.hut.fi Helsinki University of Technology S-88.4221 Postgraduate Seminar on Signal Processing 22. 10. 2008 Contents

### Variable Step Runge-Kutta-Nyström Methods for the Numerical Solution of Reversible Systems

Variable Step Runge-Kutta-Nyström Methods for the Numerical Solution of Reversible Systems J. R. Cash and S. Girdlestone Department of Mathematics, Imperial College London South Kensington London SW7 2AZ,

### Lecture 6 Positive Definite Matrices

Linear Algebra Lecture 6 Positive Definite Matrices Prof. Chun-Hung Liu Dept. of Electrical and Computer Engineering National Chiao Tung University Spring 2017 2017/6/8 Lecture 6: Positive Definite Matrices

### , b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are

Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues

### Review Higher Order methods Multistep methods Summary HIGHER ORDER METHODS. P.V. Johnson. School of Mathematics. Semester

HIGHER ORDER METHODS School of Mathematics Semester 1 2008 OUTLINE 1 REVIEW 2 HIGHER ORDER METHODS 3 MULTISTEP METHODS 4 SUMMARY OUTLINE 1 REVIEW 2 HIGHER ORDER METHODS 3 MULTISTEP METHODS 4 SUMMARY OUTLINE

### Numerical Methods for Large-Scale Nonlinear Systems

Numerical Methods for Large-Scale Nonlinear Systems Handouts by Ronald H.W. Hoppe following the monograph P. Deuflhard Newton Methods for Nonlinear Problems Springer, Berlin-Heidelberg-New York, 2004 Num.

### Linear Algebra Primer

Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

### High-Order Representation of Poincaré Maps

High-Order Representation of Poincaré Maps Johannes Grote, Martin Berz, and Kyoko Makino Department of Physics and Astronomy, Michigan State University, East Lansing, MI, USA. {grotejoh,berz,makino}@msu.edu

### CHAPTER 5: Linear Multistep Methods

CHAPTER 5: Linear Multistep Methods Multistep: use information from many steps Higher order possible with fewer function evaluations than with RK. Convenient error estimates. Changing stepsize or order

### Mathematical Foundations

Chapter 1 Mathematical Foundations 1.1 Big-O Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for

### Lecture 6 Structure-Preserving Dynamical Systems

Lecture 6 Structure-Preserving Dynamical Systems Algorithm Designs Moody T. Chu North Carolina State University June 14, 2009 @ MCM, CAS Outline Staircase Structure QR Algorithm QZ Algorithm SVD Algorithm

### 4 Film Extension of the Dynamics: Slowness as Stability

4 Film Extension of the Dynamics: Slowness as Stability 4.1 Equation for the Film Motion One of the difficulties in the problem of reducing the description is caused by the fact that there exists no commonly

### THEODORE VORONOV DIFFERENTIABLE MANIFOLDS. Fall Last updated: November 26, (Under construction.)

4 Vector fields Last updated: November 26, 2009. (Under construction.) 4.1 Tangent vectors as derivations After we have introduced topological notions, we can come back to analysis on manifolds. Let M

### Numerical Treatment of Unstructured. Differential-Algebraic Equations. with Arbitrary Index

Numerical Treatment of Unstructured Differential-Algebraic Equations with Arbitrary Index Peter Kunkel (Leipzig) SDS2003, Bari-Monopoli, 22. 25.06.2003 Outline Numerical Treatment of Unstructured Differential-Algebraic

### e j = Ad(f i ) 1 2a ij/a ii

A characterization of generalized Kac-Moody algebras. J. Algebra 174, 1073-1079 (1995). Richard E. Borcherds, D.P.M.M.S., 16 Mill Lane, Cambridge CB2 1SB, England. Generalized Kac-Moody algebras can be

### First and Second Order Differential Equations Lecture 4

First and Second Order Differential Equations Lecture 4 Dibyajyoti Deb 4.1. Outline of Lecture The Existence and the Uniqueness Theorem Homogeneous Equations with Constant Coefficients 4.2. The Existence

### Feedback stabilisation with positive control of dissipative compartmental systems

Feedback stabilisation with positive control of dissipative compartmental systems G. Bastin and A. Provost Centre for Systems Engineering and Applied Mechanics (CESAME Université Catholique de Louvain

### arxiv: v1 [math-ph] 31 Jan 2015

Symmetry, Integrability and Geometry: Methods and Applications SIGMA? (00?), 00?,?? pages Structure relations and Darboux contractions for D nd order superintegrable systems R. Heinonen, E. G. Kalnins,

### 1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)

Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty

### MA22S3 Summary Sheet: Ordinary Differential Equations

MA22S3 Summary Sheet: Ordinary Differential Equations December 14, 2017 Kreyszig s textbook is a suitable guide for this part of the module. Contents 1 Terminology 1 2 First order separable 2 2.1 Separable

### Fourth Order RK-Method

Fourth Order RK-Method The most commonly used method is Runge-Kutta fourth order method. The fourth order RK-method is y i+1 = y i + 1 6 (k 1 + 2k 2 + 2k 3 + k 4 ), Ordinary Differential Equations (ODE)

### FIRST-ORDER SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS III: Autonomous Planar Systems David Levermore Department of Mathematics University of Maryland

FIRST-ORDER SYSTEMS OF ORDINARY DIFFERENTIAL EQUATIONS III: Autonomous Planar Systems David Levermore Department of Mathematics University of Maryland 4 May 2012 Because the presentation of this material

### One dimensional Maps

Chapter 4 One dimensional Maps The ordinary differential equation studied in chapters 1-3 provide a close link to actual physical systems it is easy to believe these equations provide at least an approximate

### Mathematical Modeling and Differential Equations

Mathematical Modeling and Differential Equations A quarter abroad in Paris December 6, 2004 1 Preliminaries 1.1 Overview of the program This is a syllabus for a three-course sequence making up a study-abroad

### HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe

### Going with the flow: A study of Lagrangian derivatives

1 Going with the flow: A study of Lagrangian derivatives Jean-Luc Thiffeault Department of Applied Physics and Applied Mathematics Columbia University http://plasma.ap.columbia.edu/~jeanluc/ 12 February

### 2.3. VECTOR SPACES 25

2.3. VECTOR SPACES 25 2.3 Vector Spaces MATH 294 FALL 982 PRELIM # 3a 2.3. Let C[, ] denote the space of continuous functions defined on the interval [,] (i.e. f(x) is a member of C[, ] if f(x) is continuous

### Lecture Note 1: Background

ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)

### THE GRADIENT PROJECTION ALGORITHM FOR ORTHOGONAL ROTATION. 2 The gradient projection algorithm

THE GRADIENT PROJECTION ALGORITHM FOR ORTHOGONAL ROTATION 1 The problem Let M be the manifold of all k by m column-wise orthonormal matrices and let f be a function defined on arbitrary k by m matrices.

### Lecture2 The implicit function theorem. Bifurcations.

Lecture2 The implicit function theorem. Bifurcations. 1 Review:Newton s method. The existence theorem - the assumptions. Basins of attraction. 2 The implicit function theorem. 3 Bifurcations of iterations.

### Convergence of sequences and series

Convergence of sequences and series A sequence f is a map from N the positive integers to a set. We often write the map outputs as f n rather than f(n). Often we just list the outputs in order and leave

### 10. Unconstrained minimization

Convex Optimization Boyd & Vandenberghe 10. Unconstrained minimization terminology and assumptions gradient descent method steepest descent method Newton s method self-concordant functions implementation

### Tangent spaces, normals and extrema

Chapter 3 Tangent spaces, normals and extrema If S is a surface in 3-space, with a point a S where S looks smooth, i.e., without any fold or cusp or self-crossing, we can intuitively define the tangent

### Nonlinear equations and optimization

Notes for 2017-03-29 Nonlinear equations and optimization For the next month or so, we will be discussing methods for solving nonlinear systems of equations and multivariate optimization problems. We will

### The Continuous Newton-Raphson Method Can Look Ahead

The Continuous Newton-Raphson Method Can Loo Ahead Raphael Hauser and Jelena Nedić Oxford University Computing Laboratory, Wolfson Building, Pars Road, Oxford, OX1 3QD, England,UK; {hauser, jelena}@comlab.ox.ac.u

### Derivation of Itô SDE and Relationship to ODE and CTMC Models

Derivation of Itô SDE and Relationship to ODE and CTMC Models Biomathematics II April 23, 2015 Linda J. S. Allen Texas Tech University TTU 1 Euler-Maruyama Method for Numerical Solution of an Itô SDE dx(t)

### Time Integration Methods for the Heat Equation

Time Integration Methods for the Heat Equation Tobias Köppl - JASS March 2008 Heat Equation: t u u = 0 Preface This paper is a short summary of my talk about the topic: Time Integration Methods for the

### Strong Stability-Preserving (SSP) High-Order Time Discretization Methods

Strong Stability-Preserving (SSP) High-Order Time Discretization Methods Xinghui Zhong 12/09/ 2009 Outline 1 Introduction Why SSP methods Idea History/main reference 2 Explicit SSP Runge-Kutta Methods