# Numerical Algorithms as Dynamical Systems

Save this PDF as:
Size: px
Start display at page:

## Transcription

1 A Study on Numerical Algorithms as Dynamical Systems Moody Chu North Carolina State University

2 What This Study Is About? To recast many numerical algorithms as special dynamical systems, whence to derive new understandings and insights. To exploit the notion of dynamical systems as a special realization process for problems arising from the field of numerical analysis, whence to develop possible new schemes. To forge the idea that, while these things" have been differentiated from each other, they can also be integrated together.

3 Topics of Lessons 1. Numerical Analysis versus Dynamical Systems 2. Dynamics and Controls in Solving Algebraic Equations 3. Lax Evolution and Its Equivalents 4. Lotka-Volterra Equations and Singular Values 5. Dynamics via Group Actions 6. Structure-preserving Dynamical Systems and Applications

4 Lecture 1 Numerical Analysis versus Dynamical Systems An Overview Moody T. Chu North Carolina State University June 14, MCM, CAS

5 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

6 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

7 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

8 Basic Concepts Realization Process Bridge Construction Examples Eigenvalue Problem Root Finding Procrustes Problem Mutual Implications Discretization Discrete Dynamical System Structured Integrators Pseudo-transient Continuation Symplectic Integrator Outline

9 Realization Process A logical procedure used to reason or to infer. Usually done deductively or inductively. Aim to draw conclusion or make decision.

10 Dynamical System A realization process in the recursive form. One state develops into another state by following a certain specific rule. Often appears in the form of an iterative procedure or a differential equation.

11 Basic Components Two abstract problems: One is a make-up and is easy. The other is the real problem and is difficult. A bridge: A continuous path connecting the two problems. A path that is easy to follow. A numerical method: A method for moving along the bridge. A method that is readily available.

12 Building the Bridge Specified guidance is available. The bridge is constructed by monitoring the values of certain specified functions. The path is guaranteed to work. e.g. The projected gradient method. Only some general guidance is available. A bridge is built in a straightforward way. No guarantee the path will be complete. e.g. The homotopy method. No guidance at all. A bridge is built seemingly by accident. Usually deeper mathematical theory is involved. e.g. The isospectral flows.

13 Characteristics of a Bridge A bridge, if it exists, usually is characterized by an ordinary differential equation. The discretization of a bridge, or a numerical method in traveling along a bridge, usually produces an iterative scheme.

14 Symmetric Eigenvalue Problem The mathematical problem: A symmetric matrix A 0 is given. Solve the equation A 0 x = λx for a nonzero vector x and a scalar λ.

15 An Iterative Realization The QR decomposition: A = QR where Q is orthogonal and R is upper triangular. The QR algorithm (Francis 61): Theory: A k = Q k R k, A k+1 = R k Q k. Every matrix A k has the same eigenvalues of A 0. The sequence {A k } converges to a diagonal matrix.

16 A Continuous Realization Lie algebra decomposition: X = X o + X + + X where X o is the diagonal, X + the strictly upper triangular, and X the strictly lower triangular part of X. Toda lattice (Symes 82, Deift el al 83): Theory: dx dt = [X, X X T ] X(0) = X 0. Sampled at integer times, {X(k)} gives the same sequence as does the QR algorithm applied to the matrix A 0 = exp(x 0 ).

17 How Is the Bridge Built? The bridge between X 0 and the limit point of Toda flow is built on the basis of maintaining isospectrum. Points to ponder: What motivates the construction of the Toda lattice? Why is convergence guaranteed? What is the advantage of one approach over the other?

18 Nonlinear Algebraic Equations The mathematical problem: A sufficiently smooth function f : R n R n is given. Solve the equation f(x) = 0.

19 An Iterative Realization The Newton method: Theory: x k+1 = x k α k (f (x k )) 1 f(x k ). The sequence {x k } converges quadratically to a solution, if x 0 is sufficiently close to that solution.

20 A Continuous Realization The Newton homotopy (Smale 76, Keller 78, etc.): H(x, t) = f(x) tf(x 0 ). The zero set {(x, t) R n+1 H(x, t) = 0} forms a smooth curve. The homotopy curve: f (x) dx ds 1 dt f(x) t ds = 0, x(0) = x 0, t(0) = 1, where s is the arc length. Suppose f (x) is nonsingular. Then written as Theory: dx ds = dt 1 ds t (f (x)) 1 f(x). With appropriate step size chosen, an Euler step is equivalent to a regular Newton method.

21 How Is the Bridge Built? The bridge is built upon the hope that the obvious solution will be deformed mathematically into the solution that we are seeking for. Points to ponder: Will this idea always work? How to mathematically design an appropriate homotopy?

22 Least Squares Matrix Approximation The mathematical problem: A symmetric matrix N and a set of real values {λ 1,..., λ n} are given. Find a least squares approximation of N that has the prescribed eigenvalues.

23 A Standard Formulation Minimize F (Q) := 1 2 QT ΛQ N 2 Subject to Q T Q = I Equality Constrained Optimization: Augmented Lagrangian methods. Sequential quadratic programming methods. Interior point method. All these techniques employ iterative realization. Linearize the Lagrangian. Sequential quadratic approximation. Follow the central path.

24 A Continuous Realization The projection of the gradient of F can easily be calculated. Projected gradient flow (Chu&Driessel 90): dx dt = [X, [X, N]] X(0) = Λ X := Q T ΛQ. Flow X(t) moves in a descent direction to reduce X N 2. Theory: The optimal solution X can be fully characterized by the spectral decomposition of N and is unique.

25 How Is the Bridge Built? The bridge between a starting point and the optimal point is built on the basis of systematically reducing the difference between the current position and the target position. Points to ponder: How to get to the limit point of the gradient flow efficiently?

26 Mutual Implications Differential System Discrete Approximation Time-1 Sampling Iterative Scheme

27 Technology Limitations Floating-point arithmetic is the most common and effective way for computation. Almost a mandate to discretize a continuous problem. A majority of numerical algorithms in practice are iterative in nature.

28 Iterative Scheme x k+1 = G k (x k,..., x k p+1 ), k = 0, 1,..., A p-step sequential process (Ortega&Rheinboldt 00),: G k : D k V p V is a predetermined map. Could be stationary. A bridge intends to achieve a certain goal. V is a designated set of states. Could be a manifold. Need initial values x 0, x 1,..., x p+1.

29 Initial Value Problem dx dt = f(t, x), x(0) = x 0, f defines a flow moving in a specific direction. In many applications, x(t) is supposed to preserve certain quantities, such as mass, volume, or stay on a certain manifold. Challenging to realize this conservation law.

30 Numerical ODE Techniques Conventional Runge-Kutta method: x n+1 = x n + h R c r k r, r=1 } {{ } G n(x n) R c r = 1. r=1 k r := f(t n + a r h, x n + h P R s=1 brsks), P R s=1 brs = ar. Conventional linear multi-step method: x n+1 := p (α i x n i + hβ i f n i ) +hβ 1 f n+1. i=0 } {{ } G(x n,...,x n p) Special purpose geometric integrator...

31 Sarkovskii s Theorem Consider the iteration x k+1 = f (x k ), where f : R R is continuous. A number x 0 is of period m if x m = x 0 and having least period m if x k x 0 for all 0 < k < m. Arrange positive integers in the following ordering: 3, 5, 7, 9,..., 2 3, 2 5, 2 7,..., 2 2 3, 2 2 5,..., 2 4, 2 3, 2 2, 2, 1. Sharkovskii s theorem: If f has a periodic point of least period m and m n in the above ordering, then f has also a periodic point of least period n. Remarkable for its lack of hypotheses and its qualitative universality.

32 Two Different Views Numerical Analysis Consider systems with only trivial asymptotic behavior. Concerned about convergence and stability. Meant to trace the flow of ODEs with reliable and reasonable accuracy local behavior of the trajectory. Meant to converge to a unique equilibrium point. Dynamical System Interested in systems with more complicated behavior. Study overall trajectories. Want to differentiate the intrinsic geometric structure long-term behavior of the trajectory. Search for limit cycles, bifurcations, or strange attractors.

33 Logistic Equation dx dt = x(1 x), x(0) = x 0. Exact solution: x(t) = x 0 x 0 + e t (1 x 0 ), Converge to the equilibrium x( ) = 1 for any initial value x 0 0.

34 Explicit Euler Iteration x k+1 = x k + ɛx k (1 x k ), ɛ = step size. Fix t, x n x(t) as n in the sense of ɛ = t n. With n = 90 ɛ and 0 < ɛ 3, plot the absolute error x n x(nɛ). Theoretic error estimate O(ɛ). Fix ɛ, iterate the Euler step 5000 times to see the limit points. A cascade of period doubling as ɛ increases. The equilibrium x( ) = 1 is not even an attractor for large ɛ.

35 1.4 Feigenbaum Diagram of Limit Points 10 0 Error of Euler Iterates at t " limit points x n x(n!) ! !

36 Implicit Euler Iterations x k+1 = x k + ɛx k+1 (1 x k+1 ), x k+1 = x k + ɛx k (1 x k+1 ). Converge to the equilibrium x( ) = 1 for any step size ɛ. Points to ponder: Distinguish limiting behavior between an iterative algorithm designed originally to solve a specific problem and a discrete approximation of a differential system formulated to mimic an existing iterative algorithm. Distinguish asymptotic behavior between a differential system developed originally from a specific realization process and its discrete approximation which becomes an iterative scheme.

37 Gradient Flow dx dt = F (x), x(0) = x 0, F : R n R is a specified smooth objective function. Goal: Find the limit point x = lim t x(t) of the gradient flow x(t). Wish list: Do not want to solve the equation F (x) = 0 by Newton-like methods. Ignore the gradient property. Might locate undesirable, dynamically unstable critical points. Do not want to follow the solution curve x(t) closely. Too expensive computation at the transient state.

38 Pseudo-transient Continuation Idea: Stay near the true trajectory, but not strive for accuracy. Employ one implicit Euler step with step size ɛ k : x k+1 = x k ɛ k F (x k+1 ). Perform only one correction using one Newton iteration starting at x k and accept the outcome as x k+1. ( ) 1 1 x k+1 = x k I n + 2 F (x k ) F (x k ). ɛ k The step size changes the nature of iterations. Small values of ɛ k Scheme behaves like a steepest descent method. Large values of ɛ k Scheme behaves like a Newton iteration.

39 Hamiltonian Flow with Poisson bracket { dpi dt = {p i, H}, dq i dt = {q i, H}, {f, g} := n i=1 i = 1,..., n, ( f g f ) g. q i p i p i q i Goal: Find the long-term evolution behavior. Wish list: Want to keep the simplectic form dp dq invariant Preserving qualitative properties of phase space trajectories.

40 Runge-Kutta Methods a B c If m ij := c i b ij + c j b ji c i c j = 0 for all 1 i, j R, then the RK method is simplectic. (Sanz-Serna 88). Gauss-Legengre RK methods are simplectic

### Numerical Methods for ODEs. Lectures for PSU Summer Programs Xiantao Li

Numerical Methods for ODEs Lectures for PSU Summer Programs Xiantao Li Outline Introduction Some Challenges Numerical methods for ODEs Stiff ODEs Accuracy Constrained dynamics Stability Coarse-graining

### Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations

Logistic Map, Euler & Runge-Kutta Method and Lotka-Volterra Equations S. Y. Ha and J. Park Department of Mathematical Sciences Seoul National University Sep 23, 2013 Contents 1 Logistic Map 2 Euler and

### Although the rich theory of dierential equations can often be put to use, the dynamics of many of the proposed dierential systems are not completely u

A List of Matrix Flows with Applications Moody T. Chu Department of Mathematics North Carolina State University Raleigh, North Carolina 7695-805 Abstract Many mathematical problems, such as existence questions,

### CDS 101 Precourse Phase Plane Analysis and Stability

CDS 101 Precourse Phase Plane Analysis and Stability Melvin Leok Control and Dynamical Systems California Institute of Technology Pasadena, CA, 26 September, 2002. mleok@cds.caltech.edu http://www.cds.caltech.edu/

Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

### Numerical Methods - Numerical Linear Algebra

Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear

### 6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection

6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods

### Lecture 4: Numerical solution of ordinary differential equations

Lecture 4: Numerical solution of ordinary differential equations Department of Mathematics, ETH Zürich General explicit one-step method: Consistency; Stability; Convergence. High-order methods: Taylor

### Introduction to Dynamical Systems Basic Concepts of Dynamics

Introduction to Dynamical Systems Basic Concepts of Dynamics A dynamical system: Has a notion of state, which contains all the information upon which the dynamical system acts. A simple set of deterministic

### CHAPTER 10: Numerical Methods for DAEs

CHAPTER 10: Numerical Methods for DAEs Numerical approaches for the solution of DAEs divide roughly into two classes: 1. direct discretization 2. reformulation (index reduction) plus discretization Direct

### Inverse Eigenvalue Problems: Theory and Applications

Inverse Eigenvalue Problems: Theory and Applications A Series of Lectures to be Presented at IRMA, CNR, Bari, Italy Moody T. Chu (Joint with Gene Golub) Department of Mathematics North Carolina State University

### Optimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30

Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained

### Numerical Optimization

Numerical Optimization Unit 2: Multivariable optimization problems Che-Rung Lee Scribe: February 28, 2011 (UNIT 2) Numerical Optimization February 28, 2011 1 / 17 Partial derivative of a two variable function

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted

### AIMS Exercise Set # 1

AIMS Exercise Set #. Determine the form of the single precision floating point arithmetic used in the computers at AIMS. What is the largest number that can be accurately represented? What is the smallest

### x x2 2 + x3 3 x4 3. Use the divided-difference method to find a polynomial of least degree that fits the values shown: (b)

Numerical Methods - PROBLEMS. The Taylor series, about the origin, for log( + x) is x x2 2 + x3 3 x4 4 + Find an upper bound on the magnitude of the truncation error on the interval x.5 when log( + x)

### Chaotic motion. Phys 750 Lecture 9

Chaotic motion Phys 750 Lecture 9 Finite-difference equations Finite difference equation approximates a differential equation as an iterative map (x n+1,v n+1 )=M[(x n,v n )] Evolution from time t =0to

### Symplectic integration. Yichao Jing

Yichao Jing Hamiltonian & symplecticness Outline Numerical integrator and symplectic integration Application to accelerator beam dynamics Accuracy and integration order Hamiltonian dynamics In accelerator,

### Inverse Singular Value Problems

Chapter 8 Inverse Singular Value Problems IEP versus ISVP Existence question A continuous approach An iterative method for the IEP An iterative method for the ISVP 139 140 Lecture 8 IEP versus ISVP Inverse

### Numerical Linear Algebra Primer. Ryan Tibshirani Convex Optimization /36-725

Numerical Linear Algebra Primer Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: proximal gradient descent Consider the problem min g(x) + h(x) with g, h convex, g differentiable, and h simple

### Lyapunov Stability Theory

Lyapunov Stability Theory Peter Al Hokayem and Eduardo Gallestey March 16, 2015 1 Introduction In this lecture we consider the stability of equilibrium points of autonomous nonlinear systems, both in continuous

### Dynamical Systems and Chaos Part I: Theoretical Techniques. Lecture 4: Discrete systems + Chaos. Ilya Potapov Mathematics Department, TUT Room TD325

Dynamical Systems and Chaos Part I: Theoretical Techniques Lecture 4: Discrete systems + Chaos Ilya Potapov Mathematics Department, TUT Room TD325 Discrete maps x n+1 = f(x n ) Discrete time steps. x 0

### Iterative Methods for Solving A x = b

Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http

### u n 2 4 u n 36 u n 1, n 1.

Exercise 1 Let (u n ) be the sequence defined by Set v n = u n 1 x+ u n and f (x) = 4 x. 1. Solve the equations f (x) = 1 and f (x) =. u 0 = 0, n Z +, u n+1 = u n + 4 u n.. Prove that if u n < 1, then

### Numerical Linear Algebra Primer. Ryan Tibshirani Convex Optimization

Numerical Linear Algebra Primer Ryan Tibshirani Convex Optimization 10-725 Consider Last time: proximal Newton method min x g(x) + h(x) where g, h convex, g twice differentiable, and h simple. Proximal

### Physics 202 Laboratory 5. Linear Algebra 1. Laboratory 5. Physics 202 Laboratory

Physics 202 Laboratory 5 Linear Algebra Laboratory 5 Physics 202 Laboratory We close our whirlwind tour of numerical methods by advertising some elements of (numerical) linear algebra. There are three

### Exam in TMA4215 December 7th 2012

Norwegian University of Science and Technology Department of Mathematical Sciences Page of 9 Contact during the exam: Elena Celledoni, tlf. 7359354, cell phone 48238584 Exam in TMA425 December 7th 22 Allowed

### Outline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems

Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### Introduction to the Numerical Solution of IVP for ODE

Introduction to the Numerical Solution of IVP for ODE 45 Introduction to the Numerical Solution of IVP for ODE Consider the IVP: DE x = f(t, x), IC x(a) = x a. For simplicity, we will assume here that

### Line Search Methods for Unconstrained Optimisation

Line Search Methods for Unconstrained Optimisation Lecture 8, Numerical Linear Algebra and Optimisation Oxford University Computing Laboratory, MT 2007 Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The Generic

### (f(x) P 3 (x)) dx. (a) The Lagrange formula for the error is given by

1. QUESTION (a) Given a nth degree Taylor polynomial P n (x) of a function f(x), expanded about x = x 0, write down the Lagrange formula for the truncation error, carefully defining all its elements. How

### arxiv: v1 [math.na] 5 May 2011

ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and

### Math Ordinary Differential Equations

Math 411 - Ordinary Differential Equations Review Notes - 1 1 - Basic Theory A first order ordinary differential equation has the form x = f(t, x) (11) Here x = dx/dt Given an initial data x(t 0 ) = x

### minimize x subject to (x 2)(x 4) u,

Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for

### Linear Algebra Algorithms as Dynamical Systems

Acta Numerica (2008), pp. 001 c Cambridge University Press, 2008 doi: 10.1017/S0962492904 Printed in the United Kingdom Linear Algebra Algorithms as Dynamical Systems Moody T. Chu Department of Mathematics

### 1 Newton s Method. Suppose we want to solve: x R. At x = x, f (x) can be approximated by:

Newton s Method Suppose we want to solve: (P:) min f (x) At x = x, f (x) can be approximated by: n x R. f (x) h(x) := f ( x)+ f ( x) T (x x)+ (x x) t H ( x)(x x), 2 which is the quadratic Taylor expansion

### An introduction to Birkhoff normal form

An introduction to Birkhoff normal form Dario Bambusi Dipartimento di Matematica, Universitá di Milano via Saldini 50, 0133 Milano (Italy) 19.11.14 1 Introduction The aim of this note is to present an

### MATH 415, WEEKS 14 & 15: 1 Recurrence Relations / Difference Equations

MATH 415, WEEKS 14 & 15: Recurrence Relations / Difference Equations 1 Recurrence Relations / Difference Equations In many applications, the systems are updated in discrete jumps rather than continuous

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 9

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T. Heath Chapter 9 Initial Value Problems for Ordinary Differential Equations Copyright c 2001. Reproduction

### Numerical Analysis Comprehensive Exam Questions

Numerical Analysis Comprehensive Exam Questions 1. Let f(x) = (x α) m g(x) where m is an integer and g(x) C (R), g(α). Write down the Newton s method for finding the root α of f(x), and study the order

### Search Directions for Unconstrained Optimization

8 CHAPTER 8 Search Directions for Unconstrained Optimization In this chapter we study the choice of search directions used in our basic updating scheme x +1 = x + t d. for solving P min f(x). x R n All

The Conjugate Gradient Method Lecture 5, Continuous Optimisation Oxford University Computing Laboratory, HT 2006 Notes by Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The notion of complexity (per iteration)

### Graded Project #1. Part 1. Explicit Runge Kutta methods. Goals Differential Equations FMN130 Gustaf Söderlind and Carmen Arévalo

2008-11-07 Graded Project #1 Differential Equations FMN130 Gustaf Söderlind and Carmen Arévalo This homework is due to be handed in on Wednesday 12 November 2008 before 13:00 in the post box of the numerical

### CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares

CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search

### Chaotic motion. Phys 420/580 Lecture 10

Chaotic motion Phys 420/580 Lecture 10 Finite-difference equations Finite difference equation approximates a differential equation as an iterative map (x n+1,v n+1 )=M[(x n,v n )] Evolution from time t

### Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games

Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games Alberto Bressan ) and Khai T. Nguyen ) *) Department of Mathematics, Penn State University **) Department of Mathematics,

### Fundamentals of Dynamical Systems / Discrete-Time Models. Dr. Dylan McNamara people.uncw.edu/ mcnamarad

Fundamentals of Dynamical Systems / Discrete-Time Models Dr. Dylan McNamara people.uncw.edu/ mcnamarad Dynamical systems theory Considers how systems autonomously change along time Ranges from Newtonian

### MATH 215/255 Solutions to Additional Practice Problems April dy dt

. For the nonlinear system MATH 5/55 Solutions to Additional Practice Problems April 08 dx dt = x( x y, dy dt = y(.5 y x, x 0, y 0, (a Show that if x(0 > 0 and y(0 = 0, then the solution (x(t, y(t of the

### Lecture 15 Newton Method and Self-Concordance. October 23, 2008

Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications

### CS520: numerical ODEs (Ch.2)

.. CS520: numerical ODEs (Ch.2) Uri Ascher Department of Computer Science University of British Columbia ascher@cs.ubc.ca people.cs.ubc.ca/ ascher/520.html Uri Ascher (UBC) CPSC 520: ODEs (Ch. 2) Fall

### Symplectic Runge-Kutta methods satisfying effective order conditions

/30 Symplectic Runge-Kutta methods satisfying effective order conditions Gulshad Imran John Butcher (supervisor) University of Auckland 5 July, 20 International Conference on Scientific Computation and

### Newton s Method. Javier Peña Convex Optimization /36-725

Newton s Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: dual correspondences Given a function f : R n R, we define its conjugate f : R n R, f ( (y) = max y T x f(x) ) x Properties and

### MATH 353 LECTURE NOTES: WEEK 1 FIRST ORDER ODES

MATH 353 LECTURE NOTES: WEEK 1 FIRST ORDER ODES J. WONG (FALL 2017) What did we cover this week? Basic definitions: DEs, linear operators, homogeneous (linear) ODEs. Solution techniques for some classes

### Construction of Lyapunov functions by validated computation

Construction of Lyapunov functions by validated computation Nobito Yamamoto 1, Kaname Matsue 2, and Tomohiro Hiwaki 1 1 The University of Electro-Communications, Tokyo, Japan yamamoto@im.uec.ac.jp 2 The

### One Dimensional Dynamical Systems

16 CHAPTER 2 One Dimensional Dynamical Systems We begin by analyzing some dynamical systems with one-dimensional phase spaces, and in particular their bifurcations. All equations in this Chapter are scalar

### Numerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen

Numerisches Rechnen (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2011/12 IGPM, RWTH Aachen Numerisches Rechnen

### AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 21: Sensitivity of Eigenvalues and Eigenvectors; Conjugate Gradient Method Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis

### Parallel-in-time integrators for Hamiltonian systems

Parallel-in-time integrators for Hamiltonian systems Claude Le Bris ENPC and INRIA Visiting Professor, The University of Chicago joint work with X. Dai (Paris 6 and Chinese Academy of Sciences), F. Legoll

### BACKGROUND IN SYMPLECTIC GEOMETRY

BACKGROUND IN SYMPLECTIC GEOMETRY NILAY KUMAR Today I want to introduce some of the symplectic structure underlying classical mechanics. The key idea is actually quite old and in its various formulations

### that determines x up to a complex scalar of modulus 1, in the real case ±1. Another condition to normalize x is by requesting that

Chapter 3 Newton methods 3. Linear and nonlinear eigenvalue problems When solving linear eigenvalue problems we want to find values λ C such that λi A is singular. Here A F n n is a given real or complex

### Gradient Methods Using Momentum and Memory

Chapter 3 Gradient Methods Using Momentum and Memory The steepest descent method described in Chapter always steps in the negative gradient direction, which is orthogonal to the boundary of the level set

### Numerical Optimization Algorithms

Numerical Optimization Algorithms 1. Overview. Calculus of Variations 3. Linearized Supersonic Flow 4. Steepest Descent 5. Smoothed Steepest Descent Overview 1 Two Main Categories of Optimization Algorithms

### Hessian Riemannian Gradient Flows in Convex Programming

Hessian Riemannian Gradient Flows in Convex Programming Felipe Alvarez, Jérôme Bolte, Olivier Brahic INTERNATIONAL CONFERENCE ON MODELING AND OPTIMIZATION MODOPT 2004 Universidad de La Frontera, Temuco,

### Hamiltonian Dynamics

Hamiltonian Dynamics CDS 140b Joris Vankerschaver jv@caltech.edu CDS Feb. 10, 2009 Joris Vankerschaver (CDS) Hamiltonian Dynamics Feb. 10, 2009 1 / 31 Outline 1. Introductory concepts; 2. Poisson brackets;

### Maps and differential equations

Maps and differential equations Marc R. Roussel November 8, 2005 Maps are algebraic rules for computing the next state of dynamical systems in discrete time. Differential equations and maps have a number

### Multidisciplinary System Design Optimization (MSDO)

Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Today s Topics Sequential

### B5.6 Nonlinear Systems

B5.6 Nonlinear Systems 5. Global Bifurcations, Homoclinic chaos, Melnikov s method Alain Goriely 2018 Mathematical Institute, University of Oxford Table of contents 1. Motivation 1.1 The problem 1.2 A

### Lecture Notes to Accompany. Scientific Computing An Introductory Survey. by Michael T. Heath. Chapter 5. Nonlinear Equations

Lecture Notes to Accompany Scientific Computing An Introductory Survey Second Edition by Michael T Heath Chapter 5 Nonlinear Equations Copyright c 2001 Reproduction permitted only for noncommercial, educational

### Chapter III. Stability of Linear Systems

1 Chapter III Stability of Linear Systems 1. Stability and state transition matrix 2. Time-varying (non-autonomous) systems 3. Time-invariant systems 1 STABILITY AND STATE TRANSITION MATRIX 2 In this chapter,

### Introduction LECTURE 1

LECTURE 1 Introduction The source of all great mathematics is the special case, the concrete example. It is frequent in mathematics that every instance of a concept of seemingly great generality is in

### CS 450 Numerical Analysis. Chapter 5: Nonlinear Equations

Lecture slides based on the textbook Scientific Computing: An Introductory Survey by Michael T. Heath, copyright c 2018 by the Society for Industrial and Applied Mathematics. http://www.siam.org/books/cl80

### How long does it take to compute the eigenvalues of a random symmetric matrix?

How long does it take to compute the eigenvalues of a random symmetric matrix? Govind Menon (Brown University) Joint work with: Christian Pfrang (Ph.D, Brown 2011) Percy Deift, Tom Trogdon (Courant Institute)

### The estimation problem ODE stability The embedding method The simultaneous method In conclusion. Stability problems in ODE estimation

Mathematical Sciences Institute Australian National University HPSC Hanoi 2006 Outline The estimation problem ODE stability The embedding method The simultaneous method In conclusion Estimation Given the

### Introduction to Scientific Computing

Introduction to Scientific Computing Benson Muite benson.muite@ut.ee http://kodu.ut.ee/ benson https://courses.cs.ut.ee/2018/isc/spring 26 March 2018 [Public Domain,https://commons.wikimedia.org/wiki/File1

### SOLVING ODE s NUMERICALLY WHILE PRESERVING ALL FIRST INTEGRALS

SOLVING ODE s NUMERICALLY WHILE PRESERVING ALL FIRST INTEGRALS G.R.W. QUISPEL 1,2 and H.W. CAPEL 3 Abstract. Using Discrete Gradient Methods (Quispel & Turner, J. Phys. A29 (1996) L341-L349) we construct

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 5 Nonlinear Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction

### Outline. Scientific Computing: An Introductory Survey. Nonlinear Equations. Nonlinear Equations. Examples: Nonlinear Equations

Methods for Systems of Methods for Systems of Outline Scientific Computing: An Introductory Survey Chapter 5 1 Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

### . For each initial condition y(0) = y 0, there exists a. unique solution. In fact, given any point (x, y), there is a unique curve through this point,

1.2. Direction Fields: Graphical Representation of the ODE and its Solution Section Objective(s): Constructing Direction Fields. Interpreting Direction Fields. Definition 1.2.1. A first order ODE of the

### EN Nonlinear Control and Planning in Robotics Lecture 3: Stability February 4, 2015

EN530.678 Nonlinear Control and Planning in Robotics Lecture 3: Stability February 4, 2015 Prof: Marin Kobilarov 0.1 Model prerequisites Consider ẋ = f(t, x). We will make the following basic assumptions

### Numerical solution of ODEs

Numerical solution of ODEs Arne Morten Kvarving Department of Mathematical Sciences Norwegian University of Science and Technology November 5 2007 Problem and solution strategy We want to find an approximation

### LECTURE NOTES ELEMENTARY NUMERICAL METHODS. Eusebius Doedel

LECTURE NOTES on ELEMENTARY NUMERICAL METHODS Eusebius Doedel TABLE OF CONTENTS Vector and Matrix Norms 1 Banach Lemma 20 The Numerical Solution of Linear Systems 25 Gauss Elimination 25 Operation Count

### 1. Nonlinear Equations. This lecture note excerpted parts from Michael Heath and Max Gunzburger. f(x) = 0

Numerical Analysis 1 1. Nonlinear Equations This lecture note excerpted parts from Michael Heath and Max Gunzburger. Given function f, we seek value x for which where f : D R n R n is nonlinear. f(x) =

### = 0. = q i., q i = E

Summary of the Above Newton s second law: d 2 r dt 2 = Φ( r) Complicated vector arithmetic & coordinate system dependence Lagrangian Formalism: L q i d dt ( L q i ) = 0 n second-order differential equations

### Chapter 3 Numerical Methods

Chapter 3 Numerical Methods Part 3 3.4 Differential Algebraic Systems 3.5 Integration of Differential Equations 1 Outline 3.4 Differential Algebraic Systems 3.4.1 Constrained Dynamics 3.4.2 First and Second

### On local normal forms of completely integrable systems

On local normal forms of completely integrable systems ENCUENTRO DE Răzvan M. Tudoran West University of Timişoara, România Supported by CNCS-UEFISCDI project PN-II-RU-TE-2011-3-0103 GEOMETRÍA DIFERENCIAL,

### Entrance Exam, Differential Equations April, (Solve exactly 6 out of the 8 problems) y + 2y + y cos(x 2 y) = 0, y(0) = 2, y (0) = 4.

Entrance Exam, Differential Equations April, 7 (Solve exactly 6 out of the 8 problems). Consider the following initial value problem: { y + y + y cos(x y) =, y() = y. Find all the values y such that the

### 10-725/36-725: Convex Optimization Prerequisite Topics

10-725/36-725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 9 Initial Value Problems for Ordinary Differential Equations Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign

### Matrices, Moments and Quadrature, cont d

Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 4 Notes Matrices, Moments and Quadrature, cont d Estimation of the Regularization Parameter Consider the least squares problem of finding x such that

### Arc Search Algorithms

Arc Search Algorithms Nick Henderson and Walter Murray Stanford University Institute for Computational and Mathematical Engineering November 10, 2011 Unconstrained Optimization minimize x D F (x) where

### Solutions for B8b (Nonlinear Systems) Fake Past Exam (TT 10)

Solutions for B8b (Nonlinear Systems) Fake Past Exam (TT 10) Mason A. Porter 15/05/2010 1 Question 1 i. (6 points) Define a saddle-node bifurcation and show that the first order system dx dt = r x e x

### Convex Optimization. Problem set 2. Due Monday April 26th

Convex Optimization Problem set 2 Due Monday April 26th 1 Gradient Decent without Line-search In this problem we will consider gradient descent with predetermined step sizes. That is, instead of determining

### An Introduction to Numerical Continuation Methods. with Application to some Problems from Physics. Eusebius Doedel. Cuzco, Peru, May 2013

An Introduction to Numerical Continuation Methods with Application to some Problems from Physics Eusebius Doedel Cuzco, Peru, May 2013 Persistence of Solutions Newton s method for solving a nonlinear equation

### APPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.

APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product

### Stabilization and Acceleration of Algebraic Multigrid Method

Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration

### Period-doubling cascades of a Silnikov equation

Period-doubling cascades of a Silnikov equation Keying Guan and Beiye Feng Science College, Beijing Jiaotong University, Email: keying.guan@gmail.com Institute of Applied Mathematics, Academia Sinica,

### Outline Introduction: Problem Description Diculties Algebraic Structure: Algebraic Varieties Rank Decient Toeplitz Matrices Constructing Lower Rank St

Structured Lower Rank Approximation by Moody T. Chu (NCSU) joint with Robert E. Funderlic (NCSU) and Robert J. Plemmons (Wake Forest) March 5, 1998 Outline Introduction: Problem Description Diculties Algebraic