Generalized mid-point and trapezoidal rules for Lipschitzian RHS

Transcription

1 Generalized mid-point and trapezoidal rules for Lipschitzian RHS Richard Hasenfelder, Andreas Griewank, Tom Streubel Humboldt Universität zu Berlin 7th International Conference on Algorithmic Differentiation September 016, Oxford Richard Hasenfelder (HU Berlin) Numerical integration via PL models 1 / 36

2 Introduction Introduction Consider initial value problem of the form ẋ(t) = F(x,t), x R n,f : R n R m x(0) = x 0 We assume local Lipschitz continuity of RHS F might be non-smooth (e.g. by using max,min) Most classical methods can drop to first order Richard Hasenfelder (HU Berlin) Numerical integration via PL models / 36

3 Introduction Classical Approach Approximation of Integral: h ˆx ˇx := x(h) x(0) = F(x(t))dt hf 0 ( ) ˇx + ˆx h F(ˇx) + F(ˆx) ˆx ˇx := x(h) x(0) = F(x(t))dt h 0 for midpoint rule and for trapezoidal rule, respectively. These methods have global convergence order for smooth functions. Goal: Modify methods such that they preserve convergence properties even for large number of kinks. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 3 / 36

4 Piecewise Linearization Piecewise Linearization Richard Hasenfelder (HU Berlin) Numerical integration via PL models 4 / 36

5 Piecewise Linearization Piecewise Linearization We propose two methods to approximate locally Lipschitz continuous functions with piecewise affine functions: x ˇx ˆx (a) Tangent mode linearization (b) Secant mode linearization Richard Hasenfelder (HU Berlin) Numerical integration via PL models 5 / 36

6 Piecewise Linearization Piecewise Linearization The proposed approximations have the form F(x) = F( x + x) F( x) + F( x; x) F(x) = F( x + x) 1 [F(ˇx) + F(ˆx)] + F(ˇx, ˆx; x) for tangent mode and for secant mode, respectively, where x = 1 (ˇx + ˆx). Definition A function is called composite piecewise smooth, if it can composed out of some set of smooth elementary functions and the absolute value function, i.e. it can be represented as a directed, acyclic graph with nodes (v i ) i I, where v i+1 = ϕ i+1 (v i ) with ϕ i C 1,1 or abs or a binary operation. = We can construct approximations with AD-like methods. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 6 / 36

7 Propagation Rules I Piecewise Linearization We define the following propagation rules to produce the piecewise linearization in tangent mode Propagation Rules v i = v j ± v k for v i = v j ± v k v i = v j v k + v j v k for v i = v j v k v i = ( v j v k v j v k )/ v k for v i = v j /v k v i = c ij v j for v i = ϕ i (v j ) where we have c ij = ϕ i ( v j) for ϕ i C 1,1 and v i = abs( v i 1 + v i 1 ) v i for ϕ i = abs. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 7 / 36

8 Propagation Rules II Piecewise Linearization For secant mode the basic propagation rules stay the same, Propagation Rules v i = v j ± v k for v i = v j ± v k v i = v j v k + v j v k for v i = v j v k v i = ( v j v k v j v k )/ v k for v i = v j /v k v i = c ij v j for v i = ϕ i (v j ) but here ϕ i ( v j) if ˆx = ˇx c ij = ˆv i ˇv i else ˆv j ˇv j and for ϕ i = abs we have that v i = 1 (ˇv i + ˆv i ) = 1 [abs(ˇv j) + abs(ˆv j )]. Obviously for ˇx = ˆx both rules coincide. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 8 / 36

9 Piecewise Linearization Caculation of Lipschitz Constants In a similar way we can propagate a bound on the Lipschitz constant of F. Calculation of β F v = ϕ(u) = β v = β u ϕ (ů) v = abs(u) = β v = β u v = u + w = β v = β u + β w v = uw = β v = β u ẘ + ů β w v = u w = β v = β u ẘ + ů β w ẘ For this constant it holds: F(x) F( x) β F x x Richard Hasenfelder (HU Berlin) Numerical integration via PL models 9 / 36

10 Piecewise Linearization Caculation of Lipschitz Constants II We can also propagate a constant γ F : Calculation of γ F v = ϕ(u) = γ v = γ u ϕ (ů) + βu ϕ (ů) v = abs(u) = γ v = γ u v = u + w = γ v = γ u + γ w v = uw = γ v = γ u ẘ + β u β w + ů γ w v = u w = γ v = γ u ẘ + ẘ β u β w + ů β w + ů ẘ γ w ẘ 3 γ F is a bound on the Lipschitz constant of the derivative of F (everywhere it exists). Richard Hasenfelder (HU Berlin) Numerical integration via PL models 10 / 36

11 Piecewise Linearization Lipschitz Continuity of the Model The Lipschitz constant β F is also a Lipschitz constant for the piecewise linear model: Lipschitz Continuity x F(x) x F( x) β F x x and ˆxˇx F(x) ˆxˇx F( x) β F x x where: x F(x) F( x) + F( x;x x) and ˆxˇx F(x) 1 (F(ˇx) + F(ˆx)) + F(ˇx, ˆx;x x) Richard Hasenfelder (HU Berlin) Numerical integration via PL models 11 / 36

12 Piecewise Linearization Quality of Approximation We assume that F is composite piecewise Lipschitz continuous differentiable on an open neighbourhood of a closed, convex domain K R n. Then: Tangent mode approximation x,x K : F(x) F( x) F( x;x x) γ F x x. Secant mode approximation ˇx, ˆx,x K : F(x) F F(ˇx, ˆx;x x) γ F x ˇx x ˆx. γ F can be computed as mentioned before. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 1 / 36

13 Generalized methods Generalized methods Richard Hasenfelder (HU Berlin) Numerical integration via PL models 13 / 36

14 Generalized methods Generalized Midpoint Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h x(h) x(0) = F(x(t))dt 0 Richard Hasenfelder (HU Berlin) Numerical integration via PL models 14 / 36

15 Generalized methods Generalized Midpoint Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ Richard Hasenfelder (HU Berlin) Numerical integration via PL models 14 / 36

16 Generalized methods Generalized Midpoint Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ }{{} F(x) F( x)+ F( x; x) Richard Hasenfelder (HU Berlin) Numerical integration via PL models 14 / 36

17 Generalized methods Generalized Midpoint Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): and get h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ }{{} F(x) F( x)+ F( x; x) 1 ˆx + ˇx ˆx ˇx = h F( x) + F( x;t(ˆx ˇx))dt with x = 1 Richard Hasenfelder (HU Berlin) Numerical integration via PL models 14 / 36

18 Generalized methods Generalized Trapezoidal Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ Richard Hasenfelder (HU Berlin) Numerical integration via PL models 15 / 36

19 Generalized methods Generalized Trapezoidal Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ }{{} F(x) F+ F(ˆx,ˇx;t(ˆx ˇx)) Richard Hasenfelder (HU Berlin) Numerical integration via PL models 15 / 36

20 Generalized methods Generalized Trapezoidal Rule Idea for construction: We integrate the ODE ẋ(t) = F(x(t)): h 1 ˆx ˇx := x(h) x(0) = F(x(t))dt = h 0 1 F ( x ( h + τh)) dτ }{{} F(x) F+ F(ˆx,ˇx;t(ˆx ˇx)) and get 1 F(ˆx) + F(ˇx) ˆx ˇx = h F + F(ˆx, ˇx;t(ˆx ˇx))dt with F = 1 Richard Hasenfelder (HU Berlin) Numerical integration via PL models 15 / 36

21 Generalized methods Properties Both rules reduce to familiar rules if F is smooth For F piecewise linear we get 1 ( ) ˆx + ˇx ˆx ˇx = h F + t(ˆx ˇx) dt 1 In piecewise linear case method is an AVF method Trapezoidal rule yields a C 1,1 -interpolant that has correct tangents at ˆx and ˇx In both cases to find new point we have to solve a nonlinear, nonsmooth equation Can be solved with Picard like method for sufficiently small h Richard Hasenfelder (HU Berlin) Numerical integration via PL models 16 / 36

22 Convergence Results Generalized methods The above nonlinear equations can be written in fixed point form and with ˇx = 0 we get [ ] x = h F(x/) F (x/; xt) dt =: hg(x) It can be shown that if we bound h suitably, this equation is contractive. Due to Banach we have a fixed point x h and it can be shown that it holds where x(t) solves the equation x h x(h) = O(h 3 ) ẋ(t) = F(x(t)) with x(0) = 0. Richard Hasenfelder (HU Berlin) Numerical integration via PL models 17 / 36

23 A first numerical result Generalized methods Trapezoida Generalize Impl. Midpoint Trapezoidal Generalized Error in x Number of timesteps n Richard Hasenfelder (HU Berlin) Numerical integration via PL models 18 / 36

24 Generalized methods Theoretical expectations Stable second order convergence without event handling should be as good as or outperform Trapezoidal Rule/Impl. Midpoint Should be able to use (local) Richardson extrapolation to gain higher orders Possible geometric integration properties Richard Hasenfelder (HU Berlin) Numerical integration via PL models 19 / 36

25 Generalized methods Richardson extrapolation With transversality condition = finitely many kinks on smooth parts on kinks global Classical Rule O(h 3 ) O(h ) O(h ) Generalized Rule O(h 3 ) O(h 3 ) O(h ) Class. w/ Romberg O(h 5 ) O(h ) O(h ) Gen. w/ Romberg O(h 5 ) O(h 3 ) O(h 3 ) Richard Hasenfelder (HU Berlin) Numerical integration via PL models 0 / 36

26 Generalized methods Richardson extrapolation Without transversality condition on smooth parts on kinks global Classical Rule O(h 3 ) O(h ) O(h) Generalized Rule O(h 3 ) O(h 3 ) O(h ) Class. w/ Romberg??? Gen. w/ Romberg??? Richard Hasenfelder (HU Berlin) Numerical integration via PL models 1 / 36

27 Error Estimator Error Estimator h x(h) x h = F(x(t))dt 0 h With x(t) p(t) O(h 3 ) we get: 0 h 0 F + F(ˇx, ˆx;t(ˆx ˇx))dt F(x(t)) F((1 t h )ˇx + t h ˆx) dt h + F((1 t h )ˇx + t ˆx) F h + F(ˇx, ˆx;t(ˆx ˇx)) dt 0 h x(h) x h β F p(t) ˇx t (ˆx ˇx) dt + 1 γ h 1 F ˆx ˇx h 0 }{{}}{{} η η 1 Richard Hasenfelder (HU Berlin) Numerical integration via PL models / 36

28 Error Estimator Error Estimator Error Local Error and Estimators Error Trap. Error Est. Gen. Error Est t 1e 8 Richard Hasenfelder (HU Berlin) Numerical integration via PL models 3 / 36

29 Experiments Experiments Richard Hasenfelder (HU Berlin) Numerical integration via PL models 4 / 36

30 Experiments Rolling Stone Rolling Stone ẍ = V (x) with 1 (1 x) if x 1 1 V(x) = (1 + x) if x 1 0 else 1 x if x 1 V (x) = x 1 if x 1 0 else = min(max( x 1,0),1 x) = x 1 x x V(x) V (x) x Figure : RHS of Rolling Stone Richard Hasenfelder (HU Berlin) Numerical integration via PL models 5 / 36

31 x Experiments Rolling Stone Exact solution x(0) = 1, ẋ(0) = 1 x(t) = 1 + sin(t) if t [0,π) 1 (t π) if t [π,π + ) sin( t) if t [π +,π + ) t 3 π if t [π +,π + 4) The total period is π t Figure : Exact solution Richard Hasenfelder (HU Berlin) Numerical integration via PL models 6 / 36

32 Convergence Experiments Rolling Stone Trapezoida Generalize Impl. Midpoint Trapezoidal Generalized Error in x Number of timesteps n Richard Hasenfelder (HU Berlin) Numerical integration via PL models 7 / 36

33 Experiments Rolling Stone Convergence with Romberg extrapolation Convergence Generalized Gen. Romberg Trapezoidal Impl. Midpoint Trap. Romberg Error Number of steps Richard Hasenfelder (HU Berlin) Numerical integration via PL models 8 / 36

34 Experiments Rolling Stone Energy of the System Summed energy loss: V(x) + 1 ẋ Impl. Midpoint Trapezoidal Generalized Mean energy loss Number of timesteps n Richard Hasenfelder (HU Berlin) Numerical integration via PL models 9 / 36

35 Experiments Rolling Stone Energy over Time Energy loss over time P.L. Tapezoidal Rule Trapezoidal Rule Energy t Richard Hasenfelder (HU Berlin) Numerical integration via PL models 30 / 36

36 Experiments Diode-LC-Circuit Problem Formulation ẋ 1 ẏ = F(x) = z ż ( y C sin(ωx) + g 1 (Cz) ) 1 LC L C g(x) V (t) I(t) = z(t) with g 1 (x) = { x α if x 0 x β if x < 0 L = 10 6,C = 10 13,ω = , α =,β = Richard Hasenfelder (HU Berlin) Numerical integration via PL models 31 / 36

37 Experiments Diode-LC-Circuit Computed Solution 3 x 10 4 I(t) 1.8 x Q(t) x x 10 8 Figure : Electric Current Figure : Electric Charge Richard Hasenfelder (HU Berlin) Numerical integration via PL models 3 / 36

38 Experiments Diode-LC-Circuit Convergence Impl. Midpoint Trapezoidal Gen. Midpoint Gen. Trapezoidal Error in x Number of timesteps n Richard Hasenfelder (HU Berlin) Numerical integration via PL models 33 / 36

39 Experiments Diode-LC-Circuit Convergence with Romberg extrapolation 10-9 Convergence Error Number of steps Richard Hasenfelder (HU Berlin) Numerical integration via PL models 34 / 36

40 Experiments Diode-LC-Circuit Convergence of adaptive Solver Convergence Generalized Gen. Romberg Gen. with SSC Trapezoidal Trap. Romberg Trap. with SSC Error Number of steps Richard Hasenfelder (HU Berlin) Numerical integration via PL models 35 / 36

41 Outlook Outlook Improve efficiency of implementation Improve error constant and automatic scaling of error norm Regain energy preservation for extrapolation via restoration of time symmetry Exact integration of PL systems Automatic event handling Generalization to discontinuous case Application to DAEs Application to optimal control problems Application to space discretized PDEs ODE sensitivity using Clark can be wrong, PL isn t (cf. K. Khan) Richard Hasenfelder (HU Berlin) Numerical integration via PL models 36 / 36