Ordinary Differential Equations I CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Ordinary Differential Equations I 1 / 35
Announcements Last Homework (HW7): Due Tuesday 6/6 (before midnight) NOTE: No late HW7 submissions accepted. Final Exam: Sat June 10 @ 3:30-6:30pm in (Room: NVIDIA Auditorium) Covers all material emphasis on 2 nd half Two cheatsheets can reuse one from midterm Similar format to midterm OAE: Tentatively: 2:00pm start (pls confirm time) SCPD: Arrange similar time (or slightly later date) CS 205A: Mathematical Methods Ordinary Differential Equations I 2 / 35
Theme of Last Few Weeks The unknown is an entire function f. CS 205A: Mathematical Methods Ordinary Differential Equations I 3 / 35
New Twist So far: f (or its derivative/integral) known at isolated points CS 205A: Mathematical Methods Ordinary Differential Equations I 4 / 35
New Twist So far: f (or its derivative/integral) known at isolated points Instead: Optimize properties of f CS 205A: Mathematical Methods Ordinary Differential Equations I 4 / 35
Example Problems Approximate f 0 with f but make it smoother (or sharper!) CS 205A: Mathematical Methods Ordinary Differential Equations I 5 / 35
Example Problems Approximate f 0 with f but make it smoother (or sharper!) Simulate some dynamical or physical relationship as f(t) where t is time CS 205A: Mathematical Methods Ordinary Differential Equations I 5 / 35
Example Problems Approximate f 0 with f but make it smoother (or sharper!) Simulate some dynamical or physical relationship as f(t) where t is time Approximate f 0 with f but transfer properties of g 0 CS 205A: Mathematical Methods Ordinary Differential Equations I 5 / 35
Today: Initial Value Problems Find f(t) : R R n Satisfying F [t, f(t), f (t), f (t),..., f (k) (t)] = 0 Given f(0), f (0), f (0),..., f (k 1) (0) CS 205A: Mathematical Methods Ordinary Differential Equations I 6 / 35
Today: Initial Value Problems Find f(t) : R R n Satisfying F [t, f(t), f (t), f (t),..., f (k) (t)] = 0 Given f(0), f (0), f (0),..., f (k 1) (0) Example of canonical form (on board): y = ty cos y CS 205A: Mathematical Methods Ordinary Differential Equations I 6 / 35
Most Famous Example F = m a Newton s second law CS 205A: Mathematical Methods Ordinary Differential Equations I 7 / 35
Most Famous Example F = m a Newton s second law F (t, x, x ) usual expression of force CS 205A: Mathematical Methods Ordinary Differential Equations I 7 / 35
Most Famous Example F = m a Newton s second law F (t, x, x ) usual expression of force n particles = simulation in R 3n CS 205A: Mathematical Methods Ordinary Differential Equations I 7 / 35
Protein Folding http://www.sciencedaily.com/releases/2012/11/121122152910.htm CS 205A: Mathematical Methods Ordinary Differential Equations I 8 / 35
Gradient Descent min x E( x) CS 205A: Mathematical Methods Ordinary Differential Equations I 9 / 35
Gradient Descent min x E( x) = x i+1 = x i h E( x i ) CS 205A: Mathematical Methods Ordinary Differential Equations I 9 / 35
Gradient Descent min x E( x) = x i+1 = x i h E( x i ) = h 0 d x dt = E( x) CS 205A: Mathematical Methods Ordinary Differential Equations I 9 / 35
Crowd Simulation http://video.wired.com/watch/building-a-better-zombie-wwz-exclusive http://gamma.cs.unc.edu/densecrowds/ CS 205A: Mathematical Methods Ordinary Differential Equations I 10 / 35
Examples of ODEs y = 1 + cos t: solved by integrating both sides CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 35
Examples of ODEs y = 1 + cos t: solved by integrating both sides y = ay: linear in y, no dependence on t CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 35
Examples of ODEs y = 1 + cos t: solved by integrating both sides y = ay: linear in y, no dependence on t y = ay + e t : time and value-dependent CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 35
Examples of ODEs y = 1 + cos t: solved by integrating both sides y = ay: linear in y, no dependence on t y = ay + e t : time and value-dependent y + 3y y = t: multiple derivatives of y CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 35
Examples of ODEs y = 1 + cos t: solved by integrating both sides y = ay: linear in y, no dependence on t y = ay + e t : time and value-dependent y + 3y y = t: multiple derivatives of y y sin y = e ty : nonlinear in y and t CS 205A: Mathematical Methods Ordinary Differential Equations I 11 / 35
Reasonable Assumption Explicit ODE An ODE is explicit if can be written in the form f (k) (t) = F [t, f(t), f (t), f (t),..., f (k 1) (t)]. CS 205A: Mathematical Methods Ordinary Differential Equations I 12 / 35
Reasonable Assumption Explicit ODE An ODE is explicit if can be written in the form f (k) (t) = F [t, f(t), f (t), f (t),..., f (k 1) (t)]. Otherwise need to do root-finding! CS 205A: Mathematical Methods Ordinary Differential Equations I 12 / 35
Reduction to First Order f (k) (t) = F [t, f(t), f (t), f (t),..., f (k 1) (t)] CS 205A: Mathematical Methods Ordinary Differential Equations I 13 / 35
Reduction to First Order f (k) (t) = F [t, f(t), f (t), f (t),..., f (k 1) (t)] d dt f 0 (t) f 1 (t). f k 2 (t) f k 1 (t) = f 1 (t) f 2 (t). f k 1 (t) F [t, f 0 (t), f 1 (t),..., f k 1 (t)] CS 205A: Mathematical Methods Ordinary Differential Equations I 13 / 35
Example y = 3y 2y + y CS 205A: Mathematical Methods Ordinary Differential Equations I 14 / 35
Example y = 3y 2y + y d dt y z w = 0 1 0 0 0 1 1 2 3 y z w CS 205A: Mathematical Methods Ordinary Differential Equations I 14 / 35
Time Dependence CS 205A: Mathematical Methods Ordinary Differential Equations I 15 / 35
Time Dependence Visualization: Slope field CS 205A: Mathematical Methods Ordinary Differential Equations I 15 / 35
Autonomous ODE y = F [ y] No dependence of F on t CS 205A: Mathematical Methods Ordinary Differential Equations I 16 / 35
Autonomous ODE y = F [ y] No dependence of F on t g (t) = ( f (t) ḡ (t) ) = ( F [f(t), ḡ(t)] 1 ) CS 205A: Mathematical Methods Ordinary Differential Equations I 16 / 35
Visualization: Phase Space θ = sin θ CS 205A: Mathematical Methods Ordinary Differential Equations I 17 / 35
Visualization: Traces x = σ(y x), y = x(ρ z) y, z = xy βz CS 205A: Mathematical Methods Ordinary Differential Equations I 18 / 35
Existence and Uniqueness dy dt = 2y t Two cases: y(0) = 0, y(0) 0 CS 205A: Mathematical Methods Ordinary Differential Equations I 19 / 35
Existence and Uniqueness Theorem: Local existence and uniqueness Suppose F is continuous and Lipschitz, that is, F [ y] F [ x] 2 L y x 2 for some fixed L 0. Then, the ODE f (t) = F [f(t)] admits exactly one solution for all t 0 regardless of initial conditions. CS 205A: Mathematical Methods Ordinary Differential Equations I 20 / 35
Linearization of 1D ODEs y = F [y] CS 205A: Mathematical Methods Ordinary Differential Equations I 21 / 35
Linearization of 1D ODEs y = F [y] y = ay + b CS 205A: Mathematical Methods Ordinary Differential Equations I 21 / 35
Linearization of 1D ODEs y = F [y] y = ay + b ȳ = aȳ CS 205A: Mathematical Methods Ordinary Differential Equations I 21 / 35
Model Equation y = ay CS 205A: Mathematical Methods Ordinary Differential Equations I 22 / 35
Model Equation y = ay = y(t) = Ce at CS 205A: Mathematical Methods Ordinary Differential Equations I 22 / 35
Stability: Visualization y = ay CS 205A: Mathematical Methods Ordinary Differential Equations I 23 / 35
Three Cases y = ay, y(t) = Ce at 1. a = 0: Stable CS 205A: Mathematical Methods Ordinary Differential Equations I 24 / 35
Three Cases y = ay, y(t) = Ce at 1. a = 0: Stable 2. a < 0: Stable; solutions get closer CS 205A: Mathematical Methods Ordinary Differential Equations I 24 / 35
Three Cases y = ay, y(t) = Ce at 1. a = 0: Stable 2. a < 0: Stable; solutions get closer 3. a > 0: Unstable; mistakes in initial data amplified CS 205A: Mathematical Methods Ordinary Differential Equations I 24 / 35
Intuition for Stability An unstable ODE magnifies mistakes in the initial conditions y(0). CS 205A: Mathematical Methods Ordinary Differential Equations I 25 / 35
Multidimensional Case y = A y, A y i = λ i y i (where A = A ) y(0) = i c i y i CS 205A: Mathematical Methods Ordinary Differential Equations I 26 / 35
Multidimensional Case y = A y, A y i = λ i y i (where A = A ) y(0) = i c i y i = y(t) = i c i e λ it y i CS 205A: Mathematical Methods Ordinary Differential Equations I 26 / 35
Multidimensional Case y = A y, A y i = λ i y i (where A = A ) y(0) = i c i y i = y(t) = i c i e λ it y i Stability depends on max i λ i. CS 205A: Mathematical Methods Ordinary Differential Equations I 26 / 35
Integration Strategies Given y k at time t k, generate y k+1 assuming y = F [ y]. CS 205A: Mathematical Methods Ordinary Differential Equations I 27 / 35
Forward Euler y k+1 = y k + hf [ y k ] Explicit method O(h 2 ) localized truncation error O(h) global truncation error; first order accurate CS 205A: Mathematical Methods Ordinary Differential Equations I 28 / 35
Forward Euler: Stability CS 205A: Mathematical Methods Ordinary Differential Equations I 29 / 35
Model Equation y = ay y k+1 = (1 + ah)y k For a < 0, stable when h < 2 a. CS 205A: Mathematical Methods Ordinary Differential Equations I 30 / 35
Backward Euler y k+1 = y k + hf [ y k+1 ] Implicit method O(h 2 ) localized truncation error O(h) global truncation error; first order accurate CS 205A: Mathematical Methods Ordinary Differential Equations I 31 / 35
Backward Euler: Stability CS 205A: Mathematical Methods Ordinary Differential Equations I 32 / 35
Model Equation y = ay y k+1 = 1 1 ah y k Unconditionally stable! CS 205A: Mathematical Methods Ordinary Differential Equations I 33 / 35
Model Equation y = ay y k+1 = 1 1 ah y k Unconditionally stable! But this has nothing to do with accuracy. CS 205A: Mathematical Methods Ordinary Differential Equations I 33 / 35
Model Equation y = ay y k+1 = 1 1 ah y k Unconditionally stable! But this has nothing to do with accuracy. Good for stiff equations. CS 205A: Mathematical Methods Ordinary Differential Equations I 33 / 35
Numerical Stiffness for IVP ODEs An IVP is said to be numerically stiff if stability requirements dictate a much smaller time step size than is needed to satisfy the approximation requirements alone. [Ascher & Petzold 1998] CS 205A: Mathematical Methods Ordinary Differential Equations I 34 / 35
Forward and Backward Euler on Linear ODE y = A y Forward Euler: y k+1 = (I + ha) y k Backward Euler: y k+1 = (I ha) 1 y k Next CS 205A: Mathematical Methods Ordinary Differential Equations I 35 / 35