Dynamics of Adaptive Time-Stepping ODE solvers

Transcription

1 Dynamics of Adaptive Time-Stepping ODE solvers A.R. Humphries Joint work with: NICK CHRISTODOULOU PAUL FENTON RATNAM VIGNESWARAN ARH acknowledges support of the Engineering and Physical Sciences Research Council (UK), Leverhulme Trust (UK), and McGill. Kansas Dec 2002 p.1/32

2 Abstract For efficiency, variable time-stepping methods are often used to numerically integrate dynamical systems. The flow on chaotic attractors is often organised by the unstable manifolds of the fixed points, and it is thus necessary to obtain good numerical approximations in the neighbourhood of fixed points to reproduce the dynamics. However the standard adaptive algorithm typically fails to do this. Implicit methods designed for stiff problems are also unsuitable; they typically destroy the structure of the unstable manifold unless very small step-sizes are used. We will present examples to illustrate these poor dynamical behaviours, together with theoretical results on the approximation of stable/unstable manifolds, and suggest a phase space/stability based improvement to the standard algorithm. Kansas Dec 2002 p.2/32

3 1. Approximation of a Dynamical System with a Fixed Step-Size Runge-Kutta Method Contents 2. Approximation of Local Unstable Manifolds with a Fixed Step-Size Runge-Kutta Method 3. Approximation of a Dynamical System with a Variable Step-Size Runge-Kutta Embedded Pair 4. Approximation of Local Unstable Manifolds with a Variable Step-Size Runge-Kutta Pair 5. Phase Space Stability Error Control Kansas Dec 2002 p.3/32

4 Approximation of a dynamical system with a fixed step-size Runge-Kutta method The dynamical system u(t) = f(u(t)), u(0) = U IR d, has solution operator S( ) and so u(t) = S(t)U for all t. Kansas Dec 2002 p.4/32

5 Approximation of a dynamical system with a fixed step-size Runge-Kutta method The dynamical system u(t) = f(u(t)), u(0) = U IR d, has solution operator S( ) and so u(t) = S(t)U for all t. c A b T The order p Runge-Kutta method has evolution map S h. Example Forward Euler is defined by u n+1 = u n + hf(u n ) S h (u n ), n 0, u 0 = U. Kansas Dec 2002 p.4/32

6 Approximation of a dynamical system with a fixed step-size Runge-Kutta method The dynamical system u(t) = f(u(t)), u(0) = U IR d, has solution operator S( ) and so u(t) = S(t)U for all t. c A b T The order p Runge-Kutta method has evolution map S h. S h (u) advances the numerical solution with step-size h u n+1 = S h (u n ), n 0, u 0 = U. Each u n is an approximation of S(nh)U = u(nh). Kansas Dec 2002 p.4/32

7 Organisation of the flow 50 Phase Space Dormand Prince 5(4) applied to Lorenz Flow in forward time organised by fixed points and unstable manifolds. u So consider approximation of unstable manifolds by numerical methods u 2 u 1 Kansas Dec 2002 p.5/32

8 Organisation of the flow 50 Phase Space Dormand Prince 5(4) applied to Lorenz Flow in forward time organised by fixed points and unstable manifolds. u So consider approximation of unstable manifolds by numerical methods u u 2 Lorenz vector field is f(x, y, z) = (σ(y x), rx y xz, xy bz). Relationship of flow to fixed points obvious from figure. Kansas Dec 2002 p.5/32

9 Approximation of Local Unstable Manifolds with a Fixed Step-Size Runge-Kutta Method Stable Manifold W s (u^) The unstable manifold of equilibrium point û is W s,δ (u^) δ W u (u^) W u (û) = {u IR d : S( t)u û 0 as t }. W u,δ (u^) Unstable Manifold Let δ > 0. The local unstable manifold of û is W u,δ (û) = {u W u (û) : S( t)u û δ t 0}. Kansas Dec 2002 p.6/32

10 Approximation of Local Unstable Manifolds with a Fixed Step-Size Runge-Kutta Method Stable Manifold W s (u^) The unstable manifold of equilibrium point û is W s,δ (u^) δ W u (u^) W u (û) = {u IR d : S( t)u û 0 as t }. W u,δ (u^) Unstable Manifold Let δ > 0. The local unstable manifold of û is W u,δ (û) = {u W u (û) : S( t)u û δ t 0}. Generate {u n } n 0 using a fixed step-size h. unstable h-manifold of û is The Stable h Manifold W s h (u^) W u h (û) = {u IRd u 0 = u, u k û 0 as k }. W s,δ h (u^) δ W u h (u^) Let δ > 0. The local unstable h-manifold of û is W u,δ h (u^) Unstable h Manifold W u,δ h (û) = {u W h u (û) u 0 = u, u k û δ k}. Kansas Dec 2002 p.6/32

11 Local Unstable Manifold Theorem Theorem Apply a fixed step-size Runge-Kutta method of order p to u = f(u). Let û be a hyperbolic equilibrium and f C p+1 (IR d ). Then there exists C, H, > 0 such that δ (0, ), h (0, H) the following holds: for each u W u,δ (û), there exists a u h W u,δ h (û) such that u u h Ch p u û 2 ; and for each u h W u,δ h (û), there exists u W u,δ (û) such that u u h Ch p u h û 2. Kansas Dec 2002 p.7/32

12 Proof Let Df(û) be the Jacobian of f evaluated at û. Shift the coordinates v = u û. Linearise the solution operator and the Runge-Kutta evolution map about 0 S(h)v = exp(hdf(û))v+g h (v), Ŝ h (v) = R(hDf(û))v+N h (v), where R is matrix generalisation of linear stability function. Show that both W u,δ (û) and W u,δ h (û) are indeed manifolds. Show that both W u,δ (û) and W u,δ h (û) are representable as graphs. Show that the graphs are close. Kansas Dec 2002 p.8/32

13 Remarks A parallel result holds for local stable manifolds. Kansas Dec 2002 p.9/32

14 A parallel result holds for local stable manifolds. Remarks The result is a generalisation of Beyn s result: there exists C > 0 such that u u h Ch p. Kansas Dec 2002 p.9/32

15 A parallel result holds for local stable manifolds. Remarks The result is a generalisation of Beyn s result: there exists C > 0 such that u u h Ch p. Theorem 1 implies that W u,δ (û) and W u,δ h (û) are tangential at the fixed point, and so is a form of local (un)stable manifold theorem. This follows from û 2 on RHS of equations; so distance between numerical and exact manifolds depends on the square of the distance from the fixed point. Kansas Dec 2002 p.9/32

16 Approximation of a Dynamical System with a Variable Step-Size Runge-Kutta Embedded Pair Consider embedded Runge-Kutta pair with p p = 1. c A u n+1 = S hn (u n ) b T order p ũ n+1 = S hn (u n ) bt order p Kansas Dec 2002 p.10/32

17 Approximation of a Dynamical System with a Variable Step-Size Runge-Kutta Embedded Pair Consider embedded Runge-Kutta pair with p p = 1. c A u n+1 = S hn (u n ) b T order p ũ n+1 = S hn (u n ) bt order p S h (u) advances the numerical solution u n+1 = S hn (u n ), n 0, u 0 = U. Kansas Dec 2002 p.10/32

18 Approximation of a Dynamical System with a Variable Step-Size Runge-Kutta Embedded Pair Consider embedded Runge-Kutta pair with p p = 1. c A u n+1 = S hn (u n ) b T order p ũ n+1 = S hn (u n ) bt order p S h (u) advances the numerical solution u n+1 = S hn (u n ), n 0, u 0 = U. Note Does not define dynamical system on IR d as h n varies with n. Kansas Dec 2002 p.10/32

19 Local error approximation With user-defined tolerance, 0 < τ 1, step h n chosen by E(u n, h n ) τ, where E(u n, h n ) = 1 h ρ (u n+1 ũ n+1 ). n with ρ = 0 error per step (EPS) or ρ = 1 error per unit step (EPUS). Kansas Dec 2002 p.11/32

20 Local error approximation With user-defined tolerance, 0 < τ 1, step h n chosen by E(u n, h n ) τ, where E(u n, h n ) = 1 h ρ (u n+1 ũ n+1 ). n with ρ = 0 error per step (EPS) or ρ = 1 error per unit step (EPUS). Algorithm attempts to ensure E(u n, h n ) γτ, γ (0, 1) safety factor Leads to trouble near fixed points since f(u n ) = 0 implies E(u n, h n ) = 0. Kansas Dec 2002 p.11/32

21 Let h k n be a candidate for h n. Step-changing Algorithm If error control condition is satisfied, set h n = h k n, update solution and set [ ( ) ] h 0 γτ (1/p) n+1 = min h max, h n. E(u n, h n ) If error control condition not satisfied set h k+1 n = ( γτ E(u n, h k n) ) (1/p) h k n. p = min(p, p) + 1 ρ γ (0, 1) safety factor. Kansas Dec 2002 p.12/32

22 Adaptive Time-Stepping Algorithm as Dynamical Let τ > 0 be a (user-defined) error tolerance. An acceptable step-size h for u IR d satisfies S h (u) S h (u) τ. The maximum step-size (independent of τ) is h max. System Kansas Dec 2002 p.13/32

23 Adaptive Time-Stepping Algorithm as Dynamical Let τ > 0 be a (user-defined) error tolerance. An acceptable step-size h for u IR d satisfies S h (u) S h (u) τ. System The maximum step-size (independent of τ) is h max. Construct the sequence {(u n, h n )} n 0 using the algorithmic map S τ : IR d (0, h max ] IR d (0, h max ] (u n+1, h n+1 ) = S τ (u n, h n ). S τ finds an acceptable step-size h n and advances the solution. Kansas Dec 2002 p.13/32

24 Variable Time-Stepping Algorithm as Dynamical System S τ : IR d (0, h max ] IR d (0, h max ] (u n+1, h n+1 ) = S τ (u n, h n ). Kansas Dec 2002 p.14/32

25 Variable Time-Stepping Algorithm as Dynamical System S τ : IR d (0, h max ] IR d (0, h max ] (u n+1, h n+1 ) = S τ (u n, h n ). S τ is discontinuous. Kansas Dec 2002 p.14/32

26 Variable Time-Stepping Algorithm as Dynamical S τ : IR d (0, h max ] IR d (0, h max ] S τ is discontinuous. (u n+1, h n+1 ) = S τ (u n, h n ). System (A. Stuart & H. Lamba) There exists γ (0, 1) and C > 0 such that ( ) τ γ h C. f(u) Kansas Dec 2002 p.14/32

27 Variable Time-Stepping Algorithm as Dynamical S τ : IR d (0, h max ] IR d (0, h max ] S τ is discontinuous. (u n+1, h n+1 ) = S τ (u n, h n ). System (A. Stuart & H. Lamba) There exists γ (0, 1) and C > 0 such that ( ) τ γ h C. f(u) (G. Hall & D.J. Higham) f(u) 0 stability restricts the step-size. Kansas Dec 2002 p.14/32

28 Stable Fixed Point Example Consider the method RK1(2) applied to the linear system u = [ ], u = [ u 1 u 2 ], u(0) = [ 1, 10 4] T y n y(t) RK1(2) method with Standard (0, 0) stable fixed point. y y 1 For this method, the numerical solution gives persistent spurious oscillations and the y 1 component has O(τ) oscillation about the fixed point. Kansas Dec 2002 p.15/32

29 u = Methods RK2(3) and RK4(5) applied to Saddle ( ) u, u = ( u 1 u 2 ) Point Example, u(0) = ( 0.99, 10 10) T. Kansas Dec 2002 p.16/32

30 u = Methods RK2(3) and RK4(5) applied to Saddle ( ) u, u = ( u 1 u 2 ) Point Example, u(0) = ( 0.99, 10 10) T RK2(3) with Standard y n y(t) RK2(3) numerical solution does not pass close to fixed point or the local unstable manifold. y y 1 Kansas Dec 2002 p.16/32

31 u = Methods RK2(3) and RK4(5) applied to Saddle ( ) u, u = ( u 1 u 2 ) Point Example, u(0) = ( 0.99, 10 10) T RK2(3) with Standard y n y(t) RK2(3) numerical solution does not pass close to fixed point or the local unstable manifold. y RK4(5) with Standard y n y(t) y 1 RK4(5) has spurious oscillations about the unstable manifold. Numerical solution can ultimately end up either side of 0.02 the unstable manifold. y y 1 Kansas Dec 2002 p.16/32

32 Stable Saddle Point Example! Consider 2-stage RK1(2) method with stability domain 0 1/2 1/ Im(z) 1 0 R(z) < Re(z) Kansas Dec 2002 p.17/32

33 Stable Saddle Point Example! Consider 2-stage RK1(2) method with stability domain 0 1/2 1/ Im(z) 1 0 R(z) < 1 1 Apply this method to u = ( Re(z) ) u, u(0) = U IR 2. Kansas Dec 2002 p.17/32

34 Stable Saddle Point Example! 2 x 10 4 Phase Space using RK1(2). u(0) = (0.1,1e 05) 2 x 10 4 Phase Space using RK1(2). u(0) = (0.1,1e 05) 2.5 Step size Approximate Exact Acceptable Rejection u 2 (t) 0 u 2 (t) 0 h Approximate Exact u (t) u (t) 1 x t In an O(τ)-neighbourhood of the origin, the step-size oscillates about 1. Numerical solution near stable manifold becomes trapped near fixed point. Spurious stable invariant object in numerical flow. Kansas Dec 2002 p.18/32

35 Stable Saddle Point Example! Stability function is R(z) = 1 + z z 2. With z = hλ and here λ = ±1. Consider fixed step-size. For h (0, 1), the numerical manifolds are W s h (0) = {(x, y) IR2 y = 0} and W u h (0) = {(x, y) IR2 x = 0}. For h (1, 2), the numerical manifolds are W s h (0) = {(x, y) IR2 x = 0} and W u h (0) = {(x, y) IR2 y = 0}. Kansas Dec 2002 p.19/32

36 Stable Saddle Point Example! Stability function is R(z) = 1 + z z 2. With z = hλ and here λ = ±1. Consider fixed step-size. For h (0, 1), the numerical manifolds are W s h (0) = {(x, y) IR2 y = 0} and W u h (0) = {(x, y) IR2 x = 0}. For h (1, 2), the numerical manifolds are W s h (0) = {(x, y) IR2 x = 0} and W u h (0) = {(x, y) IR2 y = 0}. When h crosses 1, the manifolds are reversed. In adaptive algorithm this creates a chaotic attractor which persists for all τ > 0. Important to keep the step-size below linear (un)stability limits. Kansas Dec 2002 p.19/32

37 Approximation of Local Unstable Manifolds with a Variable Step-Size Runge-Kutta Pair Let σ > 0. The local unstable set of û, Wτ u,σ (û), is the set of (u, h) such that there exists a backward orbit under S τ {(u n, h n )} n=0 IR d (0, h max ], such that (u, h) = (u 0, h 0 ), u n û σ n 0, and u n û as n. Let σ δ > 0. W u,σ τ,δ (û) is the set of (u, h) such that there exists a finite backward orbit under S τ {(u n, h n )} N n=0 IR d (0, h max ], such that (u, h) = (u 0, h 0 ), u n û σ u N W u,δ h max (û). n = 0,..., N, and Kansas Dec 2002 p.20/32

38 Approximation of Local Unstable Manifolds with a Variable Step-Size Runge-Kutta Pair σ u* σ ~ W u (u*) W u τ (u*) W u,σ~ h max (u*) Kansas Dec 2002 p.20/32

39 An adaptive "Stable Manifold" Theorem Lemma There exists δ = O(τ) and h max sufficiently small and independent of τ such that W u,σ τ,δ u,σ (û) = Wτ (û). Theorem Apply an RKp( p) method with p p = 1 to u = f(u). Let û be a hyperbolic equilibrium and f C max{p, p} (IR d ). Then σ +, H + > 0 such that for h max (0, H + ) & σ (0, σ + ) d H (W u,σ (û), P u W u,σ τ (û)) 0 as τ 0 where P u : (u, h) IR d+1 u IR d is the projection operator. That is, the local unstable manifolds of the dynamical system and the unstable set of the RKp( p) are close for small τ. Kansas Dec 2002 p.21/32

40 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. Kansas Dec 2002 p.22/32

41 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; Kansas Dec 2002 p.22/32

42 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; performs badly around a stable fixed point; Kansas Dec 2002 p.22/32

43 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; performs badly around a stable fixed point; performs badly near saddle points. Kansas Dec 2002 p.22/32

44 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; performs badly around a stable fixed point; performs badly near saddle points. Don t want to restrict h max on whole phase space because of poor behaviour near fixed points Kansas Dec 2002 p.22/32

45 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; performs badly around a stable fixed point; performs badly near saddle points. Don t want to restrict h max on whole phase space because of poor behaviour near fixed points Don t want to introduce expensive algorithm to compute linear stability limit near fixed points. Kansas Dec 2002 p.22/32

46 Phase Space Stability Error Control Standard algorithm performs well during finite-time integration with fixed initial condition. However unless h max is less than the linear stability limit the algorithm admits spurious fixed points; performs badly around a stable fixed point; performs badly near saddle points. Introduce new phase space based error control to automatically control the step-size relative to the stability limit. Kansas Dec 2002 p.22/32

47 Phase Space (PS θ ) Error Control We demand at each step the phase space (PS θ ) error control u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] ϕh n (1 θ)f(u n ) + θf(u n+1 ), ϕ (0, 1). Kansas Dec 2002 p.23/32

48 Phase Space (PS θ ) Error Control We demand at each step the phase space (PS θ ) error control u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] ϕh n (1 θ)f(u n ) + θf(u n+1 ), ϕ (0, 1). h n (1 θ)f(u n ) + θf(u n+1 ) is approximation to arc length evolved over step. Kansas Dec 2002 p.23/32

49 Phase Space (PS θ ) Error Control We demand at each step the phase space (PS θ ) error control u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] ϕh n (1 θ)f(u n ) + θf(u n+1 ), ϕ (0, 1). h n (1 θ)f(u n ) + θf(u n+1 ) is approximation to arc length evolved over step. So PS θ error control bounds an approximation to local error by a fraction ϕ of an approximation to solution arc length in phase space. So is a phase space error control. Kansas Dec 2002 p.23/32

50 Phase Space (PS θ ) Error Control We demand at each step the phase space (PS θ ) error control u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] ϕh n (1 θ)f(u n ) + θf(u n+1 ), ϕ (0, 1). h n (1 θ)f(u n ) + θf(u n+1 ) is approximation to arc length evolved over step. So PS θ error control bounds an approximation to local error by a fraction ϕ of an approximation to solution arc length in phase space. So is a phase space error control. Will show it also acts as a stability control. Kansas Dec 2002 p.23/32

51 Phase Space (PS θ ) Error Control We demand at each step the phase space (PS θ ) error control u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] ϕh n (1 θ)f(u n ) + θf(u n+1 ), ϕ (0, 1). h n (1 θ)f(u n ) + θf(u n+1 ) is approximation to arc length evolved over step. So PS θ error control bounds an approximation to local error by a fraction ϕ of an approximation to solution arc length in phase space. So is a phase space error control. Will show it also acts as a stability control. Will combine this error control with standard error control; and demand both are satisfied at every step. Kansas Dec 2002 p.23/32

52 Key Features of this Error Control Negligible additional computation is needed; Kansas Dec 2002 p.24/32

53 Key Features of this Error Control Negligible additional computation is needed; Away from fixed points the standard error control is sufficient to ensure that the PS θ condition is satisfied. Kansas Dec 2002 p.24/32

54 Key Features of this Error Control Negligible additional computation is needed; Away from fixed points the standard error control is sufficient to ensure that the PS θ condition is satisfied. Prevents spurious fixed points; Kansas Dec 2002 p.24/32

55 Key Features of this Error Control Negligible additional computation is needed; Away from fixed points the standard error control is sufficient to ensure that the PS θ condition is satisfied. Prevents spurious fixed points; Forces convergence to stable fixed points; Kansas Dec 2002 p.24/32

56 Key Features of this Error Control Negligible additional computation is needed; Away from fixed points the standard error control is sufficient to ensure that the PS θ condition is satisfied. Prevents spurious fixed points; Forces convergence to stable fixed points; Gives stable step-size sequence with suitable step-size selection mechanism Kansas Dec 2002 p.24/32

57 Key Features of this Error Control Negligible additional computation is needed; Away from fixed points the standard error control is sufficient to ensure that the PS θ condition is satisfied. Prevents spurious fixed points; Forces convergence to stable fixed points; Gives stable step-size sequence with suitable step-size selection mechanism Good behaviour near saddle points Kansas Dec 2002 p.24/32

58 Step-size selection R(u n, h n ) = u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] h n (1 θ)f(u n ) + θf(u n+1 ) Next step: h θ n+1 = ( χϕ Order p 2 and θ 1/2 q = 1; Order p 3 and θ = 1/2 q = 2; χ (0, 1) is safety factor. ϕ. R(u n,h n )) 1/ qhn, where by Taylor series Kansas Dec 2002 p.25/32

59 Step-size selection R(u n, h n ) = u n+1 u n h n [(1 θ)f(u n ) + θf(u n+1 )] h n (1 θ)f(u n ) + θf(u n+1 ) Next step: h θ n+1 = ( χϕ Order p 2 and θ 1/2 q = 1; Order p 3 and θ = 1/2 q = 2; ϕ. R(u n,h n )) 1/ qhn, where by Taylor series χ (0, 1) is safety factor. New step-size selected as ] h n+1 = min [h s n+1, h θ n+1, αh n, where h s n+1 given by standard time-stepping strategy. α > 1 is a maximum step-size ratio, α = 5. Kansas Dec 2002 p.25/32

60 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: Kansas Dec 2002 p.26/32

61 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: 1. u n d 0 monoty as n ; Kansas Dec 2002 p.26/32

62 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: 1. u n d 0 monoty as n ; 2. If λ i θ(1+ϕ) ϕ λ d, then u n i 0 & un i u n d 0 mono ty as n ; Kansas Dec 2002 p.26/32

63 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: 1. u n d 0 monoty as n ; 2. If λ i θ(1+ϕ) ϕ 3. If θ(1+ϕ) ϕ λ d, then u n i 0 & un i u 0 mono ty as n ; n d [ ] λ d > λ i 2θ(1+ϕ) ϕ 1 λ d, then u n i 0 & un i u n 0; d Kansas Dec 2002 p.26/32

64 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: 1. u n d 0 monoty as n ; 2. If λ i θ(1+ϕ) ϕ 3. If θ(1+ϕ) ϕ λ d, then u n i 0 & un i u 0 mono ty as n ; n d [ ] λ d > λ i 2θ(1+ϕ) ϕ 1 λ d, then u n i 0 & un i u n 0; d 4. Otherwise u n i 0 as n with lim sup n u n i u n d < ϕ θ ϕ 1+ϕ ; Kansas Dec 2002 p.26/32

65 Linear system Theorem Consider forward Euler method under PS θ error control in with ϕ θ/(1 θ) applied to u t = Λu, Λ = Diag[λ 1, λ 2,, λ d ], u(0) = u 0 IR d where λ 1 < λ 2 < < λ d < 0. Then u n 0 as n with: 1. u n d 0 monoty as n ; 2. If λ i θ(1+ϕ) ϕ 3. If θ(1+ϕ) ϕ λ d, then u n i 0 & un i u 0 mono ty as n ; n d [ ] λ d > λ i 2θ(1+ϕ) ϕ 1 λ d, then u n i 0 & un i u n 0; d 4. Otherwise u n u i 0 as n with lim sup n i n u n < ϕ d θ ; ϕ 1+ϕ 5. Let θ n be angle between u n and [0,..., 0, 1] IR d. If u 0 d 0 lim inf cos θ n 1 1 (d 1)ϕ2 n 2 θ 2 + O(ϕ3 ). Kansas Dec 2002 p.26/32

66 Remarks Can extend to arbitrary norms. Kansas Dec 2002 p.27/32

67 Can extend to arbitrary norms. Remarks Can extend to u = Au (i.e. non-diagonal). and nonlinear hyperbolic equilibria. Kansas Dec 2002 p.27/32

68 Can extend to arbitrary norms. Remarks Can extend to u = Au (i.e. non-diagonal). and nonlinear hyperbolic equilibria. Extends to non-stiff saddle points. Kansas Dec 2002 p.27/32

69 Can extend to arbitrary norms. Remarks Can extend to u = Au (i.e. non-diagonal). and nonlinear hyperbolic equilibria. Extends to non-stiff saddle points. Can extend to arbitrary methods. Kansas Dec 2002 p.27/32

70 Can extend to arbitrary norms. Remarks Can extend to u = Au (i.e. non-diagonal). and nonlinear hyperbolic equilibria. Extends to non-stiff saddle points. Can extend to arbitrary methods. Bound in 4 independent of stiffness/eigenvalues and can be made arbitray small by reducing ϕ. Kansas Dec 2002 p.27/32

71 Can extend to arbitrary norms. Remarks Can extend to u = Au (i.e. non-diagonal). and nonlinear hyperbolic equilibria. Extends to non-stiff saddle points. Can extend to arbitrary methods. Bound in 4 independent of stiffness/eigenvalues and can be made arbitray small by reducing ϕ. The exact solution is tangent to [0, 0,..., 0, 1] at fixed point, so 5 gives bound on angle between exact and numerical solutions at fixed point. Reducing ϕ makes angle arbitrarily small (independent of the stiffness/eigenvalues). Kansas Dec 2002 p.27/32

72 Proof Forward Euler method gives u n+1 = R(h n A)u n = Diag[1 + h n λ 1,, 1 + h n λ d ]u n. Kansas Dec 2002 p.28/32

73 Proof Forward Euler method gives u n+1 = R(h n A)u n = Diag[1 + h n λ 1,, 1 + h n λ d ]u n. With -norm, PS θ error control becomes θh 2 nλ 2 1 un 1 θh 2 nλ 2 2 un 2. θh 2 nλ 2 d un d ϕh n λ 1 (1 + θλ 1 h n )u n 1 λ 2 (1 + θλ 2 h n )u n 2. λ d (1 + θλ d h n )u n d. Note Unlike standard control not trivially true at near point. Kansas Dec 2002 p.28/32

74 Proof Forward Euler method gives u n+1 = R(h n A)u n = Diag[1 + h n λ 1,, 1 + h n λ d ]u n. With -norm, PS θ error control becomes θh 2 nλ 2 1 un 1 λ 1 (1 + θλ 1 h n )u n 1 θh 2 nλ 2 2 un 2 λ ϕh n 2 (1 + θλ 2 h n )u n 2.. θh 2 nλ 2 d un d λ d (1 + θλ d h n )u n d hence for some i {1, 2,..., d} h n θλ 2 i u n i ϕλ i 1 + θλ i h n u n i.. Kansas Dec 2002 p.28/32

75 Proof True if and only if h n θλ 2 i u n i ϕλ i 1 + θλ i h n u n i. h n ϕ λ i θ(1 + ϕ) ϕ λ d θ(1 + ϕ). Kansas Dec 2002 p.28/32

76 Proof True if and only if h n θλ 2 i u n i ϕλ i 1 + θλ i h n u n i. h n ϕ λ i θ(1 + ϕ) ϕ λ d θ(1 + ϕ). So the d th condition always holds and monotonic convergence in this component follows. Kansas Dec 2002 p.28/32

77 Proof True if and only if h n θλ 2 i u n i ϕλ i 1 + θλ i h n u n i. h n ϕ λ i θ(1 + ϕ) ϕ λ d θ(1 + ϕ). So the d th condition always holds and monotonic convergence in this component follows. Prove 2 and 3 by showing that d th condition implies (monotonic) convergence for these components. Kansas Dec 2002 p.28/32

78 Proof True if and only if h n θλ 2 i u n i ϕλ i 1 + θλ i h n u n i. h n ϕ λ i θ(1 + ϕ) ϕ λ d θ(1 + ϕ). So the d th condition always holds and monotonic convergence in this component follows. Prove 2 and 3 by showing that d th condition implies (monotonic) convergence for these components. Prove 4 by showing that failure of the i-th condition bounds yi n/yn d and hard work. 5 follows on noting that in limit n all components bounded in terms of d th component. Kansas Dec 2002 p.28/32

79 Stable Fixed Point Example (Revisited) 10 4 RK1(2) method with Standard 10 6 y n y(t) 10 8 y y 1 Recall solution with standard algorithm Kansas Dec 2002 p.29/32

80 Stable Fixed Point Example (Revisited) 10 4 RK1(2) with PS θ & Standard 10 5 y n y(t) 10 6 y y 1 x 10 4 With PS θ spurious oscillation is removed Kansas Dec 2002 p.29/32

81 Stable Fixed Point Example (Revisited) 10 4 RK1(2) with PS θ & Standard 10 0 RK1(2) with PS θ & Standard 10 5 y n y(t) Linear stability limit(λ =5) 10 6 y Step size PS θ Standard Rejected h n for Standard y 1 x t With PS θ spurious oscillation is removed Step-size is kept below stability limit. Kansas Dec 2002 p.29/32

82 Stable Fixed Point Example (Revisited) 10 4 RK1(2) with PS θ & Standard 10 0 RK1(2) with PS θ & Standard RK1(2) with PS θ & Standard,φ =10 1, τ = y n y(t) Linear stability limit(λ =5) y Step size 10 1 h x 10 3 PS θ Standard Rejected h n for Standard y 1 x t With PS θ spurious oscillation is removed Step-size is kept below stability limit. Step-sizes bounded near fixed point. PS θ only determines step-size near fixed point. Y X Kansas Dec 2002 p.29/32

83 Saddle Point Example (Revisited) 10 2 RK2(3) with Standard y n y(t) y y 1 Recall solution with RK2(3) standard algorithm Kansas Dec 2002 p.30/32

84 Saddle Point Example (Revisited) 10 2 RK2(3) with PS θ y n y(t) y y 1 With PS θ spurious oscillation is removed Kansas Dec 2002 p.30/32

85 Saddle Point Example (Revisited) 10 2 RK2(3) with PS θ 10 1 RK2(3) with PS θ & Standard 10 0 y n y(t) PS θ Standard Rejected h n for Standard Linear stability limit(λ =1) y Step size y t With PS θ spurious oscillation is removed Step-size is kept below stability limit. Kansas Dec 2002 p.30/32

86 Saddle Point Example (Revisited) 10 2 RK2(3) with PS θ 10 1 RK2(3) with PS θ & Standard RK2(3) with PS θ & Standard,φ =10 1, τ = y n y(t) PS θ Standard Rejected h n for Standard Linear stability limit(λ =1) y Step size h y t With PS θ spurious oscillation is removed Step-size is kept below stability limit. Step-sizes bounded near fixed point. PS θ only determines step-size near fixed point Y X Kansas Dec 2002 p.30/32

87 Large Average Step-Sizes RK1(2) Method applied to RK1(2) method with PS θ u = u. h stability limit for λ = 1 1 maximum step size (:) stability limit for λ = 10 2 average step size ( ) Step-size oscillates stability limit for λ 3 = 100 minimum step size ( ) χφ Kansas Dec 2002 p.31/32

88 Large Average Step-Sizes RK1(2) Method applied to u = Step-size oscillates. u. All steps below λ = 1 stability limit; monotonic convergence of this component. h stability limit for λ 1 = 1 stability limit for λ 2 = 10 RK1(2) method with PS θ maximum step size (:) stability limit for λ 3 = 100 average step size ( ) minimum step size ( ) χφ Kansas Dec 2002 p.31/32

89 RK1(2) Method applied to u = Step-size oscillates. u. All steps below λ = 1 stability limit; monotonic convergence of this component. Large Average Step-Sizes h stability limit for λ 1 = 1 stability limit for λ 2 = 10 RK1(2) method with PS θ maximum step size (:) stability limit for λ 3 = 100 average step size ( ) minimum step size ( ) χφ Average step-size below λ = 10 stability limit, but for ϕ > 0.1 some step-sizes above this limit. Kansas Dec 2002 p.31/32

90 RK1(2) Method applied to u = Step-size oscillates. u. All steps below λ = 1 stability limit; monotonic convergence of this component. Large Average Step-Sizes h stability limit for λ 1 = 1 stability limit for λ 2 = 10 RK1(2) method with PS θ maximum step size (:) stability limit for λ 3 = 100 average step size ( ) minimum step size ( ) χφ Average step-size below λ = 10 stability limit, but for ϕ > 0.1 some step-sizes above this limit. Except for ϕ tiny, average and maximum step-sizes above λ = 100 stability limit. But convergence to fixed point enforced. Kansas Dec 2002 p.31/32

91 Standard algorithm behaves poorly near saddle points. Stiff methods do not resolve problem for saddles. Conclusions Kansas Dec 2002 p.32/32

92 Standard algorithm behaves poorly near saddle points. Stiff methods do not resolve problem for saddles. Conclusions PS θ proved to give correct behaviour near stable fixed points gives correct behaviour near non-stiff saddles Kansas Dec 2002 p.32/32

93 Standard algorithm behaves poorly near saddle points. Stiff methods do not resolve problem for saddles. Conclusions PS θ proved to give correct behaviour near stable fixed points Ongoing gives correct behaviour near non-stiff saddles PS θ currently being implemented in standard ODE solver Kansas Dec 2002 p.32/32