Parallel-in-time integrators for Hamiltonian systems

Parallel-in-time integrators for Hamiltonian systems Claude Le Bris ENPC and INRIA Visiting Professor, The University of Chicago joint work with X. Dai (Paris 6 and Chinese Academy of Sciences), F. Legoll (ENPC and INRIA) and Y. Maday (Paris 6 and Brown) SSC Seminar, October 2012 The University of Chicago p. 1

Parallel in time algorithm for ODEs ẋ = f(x), x R d Most algorithms are sequential in nature: x n+1 = Φ T (x n ). The parareal algorithm (Lions, Maday and Turinici, 2001) is the first efficient version of a time integrator that is parallel in time. It is based upon two integrators to propagate the system over a time T : a fine integrator F T, to advance of T by many tiny timesteps; a coarse integrator G T, to advance of T by a few large timesteps (hence much cheaper). F T = (Φ δt ) T/δt and G T = (Φ dt ) T/dT with δt dt Φ δt a single step of any one-step integrator. The University of Chicago p. 2

The parareal paradigm - 1 The parareal algorithm combines the two integrators as follows: first, coarse propagation that yields { } x k=0 n Then, iterate over k 0: compute jumps (in parallel): n : n, x k=0 n+1 = G T (x k=0 n ) J k n = F T (x k n) G T (x k n) sequential update to obtain { } x k+1 n : n n, x k+1 n+1 = G T(x k+1 n ) + Jn k The fine solver is called only in the parallel part of the algorithm. The University of Chicago p. 3

The parareal paradigm - 2 x k+1 n+1 = G T(x k+1 n ) + F T (x k n) G T (x k n) If F G, then x k+1 n+1 = F T(x k+1 n ): similar to a fine sequential integration. It turns out that, at iteration k, x k k = (F T ) (k) (x 0 ), as if the fine scheme was used sequentially all the way from x 0 up to time k T. In practice, convergence occurs much faster: only a few iterations are needed to obtain a good accuracy on a large range. The approach has been successfully tested, in the past few years, on many equations (ODEs or PDEs, e.g. heat equation). The University of Chicago p. 4

A simple test-case: ẋ = x (coarse and fine propagators: forward Euler) 1 1e-05 fine integrator (sequential) parareal iteration 0 (coarse integrator) parareal iteration 3 parareal iteration 6 parareal iteration 10 1e-10 1e-15 1e-20 1e-25 1e-30 1e-35 0 10 20 30 40 50 60 Trajectory x k (t) ( T = 0.5, dt = T, δt = 0.005) t At iteration k = 10, trajectory is exact up to time 10 T = 5, and is accurate over a much longer range. The University of Chicago p. 5

Hamiltonian dynamics Consider a Hamiltonian function H(q, p), and the Hamiltonian dynamics q = H (q, p), p ṗ = H q (q, p), q Rd, p R d. Introducing x = (q, p) R 2d, recast the dynamics as ẋ = f(x) = J H(x), J = 0 Id Id 0. Separated case: H(q, p) = 1 2 pt p + V (q). Then q = p, ṗ = V (q). Verlet algorithm (explicit, second order, symmetric, symplectic): p n+1/2 = p n δt 2 V (q n), q n+1 = q n + δt p n+1/2, p n+1 = p n+1/2 δt 2 V (q n+1). The University of Chicago p. 6

Reversibility ẋ = f(x) = J H(x) Let ϕ t (x) be the flow of the equation, namely the solution at time t with I.C. x: dϕ t (x) = J H(ϕ t (x)), ϕ t=0 (x) = x. dt we will focus on the case H(q, p) = 1 2 pt p + V (q), for which the flow is reversible: t, where ρ(x) = ρ(q, p) = (q, p). ρ ϕ t = (ϕ t ) 1 ρ a subset of such systems is the case of integrable reversible systems: up to a reversible change of variables (q, p) (a, θ), the dynamics reads ȧ = 0, θ = ω(a), a R d, θ T d. Thus a(q, p) R d is left invariant by the dynamics. The University of Chicago p. 7

Backward error analysis for ODEs exact flow ϕ δt (x) ẋ = f(x) numerical scheme exact flow Φ δt (x) ẋ = f δt (x) Exact flow of the modified equation at time δt Flow Φ δt (x) given by the numerical scheme after one-time step. The University of Chicago p. 8

Classical analysis for Hamiltonian systems Symmetric integrators on reversible systems: δt, Φ δt Φ δt = Id Symmetric integrator Reversible modified equation Reversible modified equation + integrable reversible system + non resonant condition Conservation of invariants (hence of energy). Another class of efficient schemes: Symplectic integrators The University of Chicago p. 9

Parareal scheme for Hamiltonian dynamics ẋ = f(x) = J H(x), x = (q, p) R 2d Standard parareal algorithm: x k+1 n+1 = G T(x k+1 n ) + F T (x k n) G T (x k n) with F T = (Φ δt ) T/δt and G T = (Φ dt ) T/dT with δt dt, where Φ δt one step of (say) Verlet algorithm with δt time-step. Symplectic or symmetric algorithms are known to be efficient for Hamiltonian dynamics. PROBLEM: Even if F T and G T are symplectic (or symmetric), the resulting parareal algorithm is NOT symplectic (or symmetric)... Long-term (and geometric) properties of the approach are to be clarified! The University of Chicago p. 10

Harmonic oscillator case: q = p, ṗ = q 1e+06 10000 Sequential coarse scheme Parareal it. #1 Parareal it. #4 Parareal it. #10 Sequential fine scheme 100 1 0.01 1e-04 1e-06 1e-08 1e-10 1 10 100 1000 t Error on the energy 1 2 q2 + 1 2 p2 The University of Chicago p. 11

Outline Adapt the parareal algorithm to the Hamiltonian framework by using symmetrization projection on the constant energy manifold M = { (q, p) R 2d ; H(q, p) = H 0 } The University of Chicago p. 12

Symmetric parareal algorithm Symmetrizing by hand a given integrator is easy: consider a numerical flow Φ T, that reads x n+1 = Φ T (x n ) consider the adjoint of Φ T, defined by Φ T = Φ 1 T then the scheme x n+1 = Φ T/2 Φ T/2 (x n) is symmetric: x n+1/2 = Φ T/2 (x n), x n+1 = Φ T/2 (x n+1/2 ) e.g. Φ T/2 (x n+1/2 ) = x n, x n+1 = Φ T/2 (x n+1/2 ) Use this observation to make a symmetric version of parareal: Φ T the standard parareal algorithm; we can write explicitly what is Φ T. Remark: it is much more difficult to make symplectic a given integrator... The University of Chicago p. 13

Symmetric parareal The standard form of the parareal algorithm writes x k+1 n+1 = G T(x k+1 n ) + F T (x k n) G T (x k n) ( ) It is NOT a one-step integrator on x n! The University of Chicago p. 14

Symmetric parareal The standard form of the parareal algorithm writes x k+1 n+1 = G T(x k+1 n ) + F T (x k n) G T (x k n) ( ) It is NOT a one-step integrator on x n! Define U n = (x 0 n, x 1 n,...,x K n ) Knowing U n, the parareal algorithm defines U n+1 : we write U n+1 = Φ T (U n ). The symmetrized form of Φ, that is Φ T/2 Φ T/2, reads U n = Φ T/2 (U n+1/2 ), U n+1 = Φ T/2 (U n+1/2 ). The University of Chicago p. 14

Symmetric parareal: some basic remarks - 1 At the initial iteration (k = 0), we use the symmetric version of G: x k=0 n+1/2 = G 1 T/2 (xk=0 n ), x k=0 n+1 = G T/2 (x k=0 n+1/2 ). we next iterate over k: [ x k+1 n+1/2 = G 1 T/2 x k+1 n F T/2 (x k n+1/2 ) + G T/2(x k n+1/2 ], ) x k+1 n+1 = G T/2 (x k+1 n+1/2 ) + F T/2(x k n+1/2 ) G T/2(x k n+1/2 ) The flow is symmetric in the following sense: if, for any k and any n, (x 0 n,...,x k n, x k+1 n ) (x 0 n+1,...,x k n+1, x k+1 n+1 ) by the previous integrator, then (x 0 n+1,...,x k n+1, x k+1 n+1 ) (x0 n,...,x k n, x k+1 n ) by the exact same algorithm reversing the time. The University of Chicago p. 15

Symmetric parareal: some basic remarks - 2 [ x k+1 n+1/2 = G 1 T/2 x k+1 n F T/2 (x k n+1/2 ) + G T/2(x k n+1/2 ], ) x k+1 n+1 = G T/2 (x k+1 n+1/2 ) + F T/2(x k n+1/2 ) G T/2(x k n+1/2 ) If F G, then the symmetrized form reads x k+1 n+1 = F T/2 F 1 T/2 (xk+1 n ) standard symmetrized version of the fine (= coarse) propagator. Formally taking the limit k + yields x n+1 = F T/2 F 1 T/2 (x n ). Limit of the algorithm in terms of parareal iterations: standard symmetrized form of F T. Note also that we never call F 1. All the expensive computations (involving F) are performed in parallel. The University of Chicago p. 16

Harmonic oscillator case: q = p, ṗ = q, symmetric algorithm 100 1 Sequential coarse scheme Parareal it. #1 Parareal it. #4 Parareal it. #10 Sequential fine scheme 0.01 1e-04 1e-06 1e-08 1 10 100 1000 10000 t Error on the energy 1 2 q2 + 1 2 p2 The University of Chicago p. 17

Parareal integrator interpretation A parareal integrator may be seen, at parareal iteration k, as an integrator of a system consisting of k + 1 identical replicas of the original system. Consider the symmetric parareal integrator, used on an integrable reversible Hamiltonian system: The first replica (k = 0) is integrated by the symmetric algorithm G T/2 G 1 T/2 : energy preservation. Replicas are noninteracting: the system of replicas is an integrable reversible system, with energy = k j=0 H(q j, p j ) At iteration k + 1, we have integrated an integrable reversible system with a symmetric algorithm: its energy is preserved. Hence, for all k, k j=0 H(q j, p j ) is preserved: the energy of each replica H(q j, p j ) is preserved. However, the non resonant condition is NOT satisfied: replicas...! The University of Chicago p. 18

Resonances Consider a set of k identical replicas of the original system: before time discretization, they are resonant (identical!), but uncoupled: energy preservation,... OK the parareal time discretization introduces coupling. these resonances are present in the standard version as well as in the symmetric version of the algorithm. even if the symmetric algorithm is used, these resonances may impede preservation of the invariants (energy,... ) in the long-time limit. One possibility to avoid resonances: use a timestep dt (for G T ) that depends on k. x k=0 n+1 = G (k=0) T (x k=0 n ) x k=1 n+1 = G (k=1) T (x k=1 n ) + F T (x k=0 n ) G (k=1) T (x k=0 n ) + symmetrization. The University of Chicago p. 19

Fix resonances by using dt k 0.1 0.01 Error on energy, symetric parareal (shift), Verlet 0.001 1e-04 1e-05 1e-06 1e-07 1e-08 1e-09 1e-10 10 100 1000 $t$ Kepler, symmetric parareal based on Verlet, dt k Error on the energy H(q, p) = pt p 2 1 q The University of Chicago p. 20

Going from dt k to a smaller dt with no k dependence Error on energy, symetric parareal (shift), Verlet 0.1 0.01 0.001 1e-04 1e-05 1e-06 1e-07 1e-08 1e-09 Error on energy, symetric parareal (without shift), Verlet 0.1 0.01 0.001 1e-04 1e-05 1e-06 1e-07 1e-08 1e-09 Initial coarse scheme Parareal iteration #1 Parareal iteration #4 Parareal iteration #10 At convergence 1e-10 10 100 1000 $t$ 1e-10 10 100 1000 $t$ Kepler, symmetric parareal based on Verlet Error on the energy Left: dt k Right: dt smaller Both pictures with equal cost The University of Chicago p. 21

Another idea On top of symmetrizing, let us project on the constant energy manifold M = { } x = (q, p) R 2d ; H(q, p) = H 0, in a way that preserves symmetry. The University of Chicago p. 22

Another idea On top of symmetrizing, let us project on the constant energy manifold in a way that preserves symmetry. M = { x = (q, p) R 2d ; H(q, p) = H 0 }, Take any symmetric integrator Ψ T, and consider x n x n+1 defined by x n = x n + µ H(x n ), x n+1 = Ψ T ( x n ), with µ chosen such that x n+1 M. x n+1 = x n+1 + µ H(x n+1 ), As the same Lagrange multiplier µ is used in the first and third lines, and as Ψ T is symmetric, the integrator x n x n+1 is symmetric. The University of Chicago p. 22

Symmetric projection x n = x n + µ H(x n ), x n+1 = Ψ T ( x n ), x n+1 = x n+1 + µ H(x n+1 ), u 3 M u 2 u 1 u 0 Need to solve a nonlinear problem to compute x n+1 and µ. In practice: use a Newton-like algorithm, and stop after a few iterations, or when the residu is small enough. In the following examples, stop after 2 iterations. The University of Chicago p. 23

Application to the parareal setting - 1 We need to start from a symmetric algorithm. We again introduce U n := (x 0 n, x 1 n,, x K n ), and consider the symmetric parareal algorithm previously obtained: U n+1 = Ψ sym T (U n), where the map Ψ sym T is symmetric in the classical sense. Introduce H k (U n ) := H(x k n), 1 k K and consider a symmetric algorithm, based on the symmetric map Ψ sym T and on a symmetric projection on the manifold where all the energies H k are preserved. The University of Chicago p. 24

Application to the parareal setting - 2 Ũ n = U n + K µ k H k (U n ) k=1 Ũ n+1 = Ψ sym T (Ũn) U n+1 = K Ũn+1 + µ k H k (U n+1 ), with the Lagrange multipliers µ k such that H k (U n+1 ) = H 0 for any 1 k K. Resulting algorithm: k=1 Initialization: at k = 0, use the symmetrized version of G: Set x 0 n+1/2 = x0 n+1/2. x 0 n+1/2 = G 1 T/2 (x0 n), x 0 n+1 = G T/2 (x 0 n+1/2 ). The University of Chicago p. 25

Symmetric parareal integrator with projection Assume iteration k is completed. We compute the solution at iteration k + 1 by: Set x k+1 0 = u 0 ; For 0 n N 1, compute x k+1 n+1 x k+1 n = x k+1 x k+1 n+1/2 = G 1 T/2 from xk+1 n n ) n + µ k+1 H(x k+1 as [ x k+1 n F T/2 ( x k n+1/2 ) + G T/2( x k n+1/2 ) ] x k+1 n+1 = G T/2 ( x k+1 n+1/2 ) + F T/2( x k n+1/2 ) G T/2( x k n+1/2 ) x k+1 n+1 = x k+1 n+1 + µ k+1 H(x k+1 n+1 ) where µ k+1 is such that H(x k+1 n+1 ) = H 0. As before, µ k+1 and x k+1 n+1 are determined iteratively. Jumps are precomputed in parallel, before the nonlinear iterations at step k + 1. All the expensive computations (fine propagator) are performed in parallel. The University of Chicago p. 26

Kepler problem (N max iter = 2): energy preservation 0.01 1e-04 1e-06 1e-08 1e-10 1e-12 Sequential coarse scheme 1e-14 Parareal it. #1 Parareal it. #4 Parareal it. #10 Sequential fine scheme 1e-16 0.01 0.1 1 10 100 1000 10000 Relative error on the energy H(q, p) (δt = 10 4, dt = 0.01, T = 0.2). t Energy is well preserved at any parareal iteration k 1, although we limit the number of iterations to solve the nonlinear projection step. The University of Chicago p. 27

Kepler problem (N max iter = 2): angular momentum preservation 0.01 1e-04 Sequential coarse scheme Parareal it. #1 Parareal it. #4 Parareal it. #7 Parareal it. #10 Sequential fine scheme 1e-06 1e-08 1e-10 1e-12 1e-14 1e-16 1 10 100 1000 10000 Relative error on the angular momentum q p t For all parareal iterations k 7, the angular momentum is preserved with a relative accuracy of 10 4 over the complete time range. The University of Chicago p. 28

Kepler problem (N max iter = 2): trajectory accuracy 10 1 0.1 Sequential coarse scheme Parareal it. #1 Parareal it. #3 Parareal it. #5 Parareal it. #6 Sequential fine scheme 0.01 0.001 1e-04 1e-05 1e-06 1e-07 1e-08 1 10 100 1000 10000 t Error on the trajectory After only 6 iterations, errors on the trajectory are comparable to the errors on the trajectory of the fine scheme used sequentially on the complete time range. The University of Chicago p. 29

Kepler problem: summary of the results energy is well preserved at any parareal iteration k 1 good preservation of the angular momentum (although we project on the manifold where only the energy is preserved) only 6 parareal iterations are needed to reach the best possible accuracy on the trajectory. These results are better than those obtained with the standard parareal algorithm or its symmetrized version: energy was badly preserved the standard parareal algorithm coupled with a standard, not symmetrized, projection step: better preservation of energy (remember N max iter < ), of the angular momentum, and trajectory convergence is reached for fewer parareal iterations. Both symmetrization and projection are useful. The University of Chicago p. 30

The outer solar system H(q, p) = 1 2 pt M 1 p + V (q), q R 18, p R 18, with V (q) = 1 i<j 6 Gm i m j q i q j, M = diag(m i). First particle Sun, with mass m 1 1000m i for any planet (i 2). We show here that the symmetric parareal algorithm with symmetric projection on the constant energy manifold again behaves well. that the coarse solver can be driven by a simplified dynamics without damaging the good properties of the algorithm. The University of Chicago p. 31

A simplified coarse integrator Consider the simplified dynamics H(q, p) = 1 2 pt M 1 p + V simp (q), q R 18, p R 18, with V simp (q) = 2 j 6 Gm 1 m j q 1 q j. In V simp, we only take into account the gravitational interaction between the Sun (at position q 1 ) and the other planets (at position q j, 2 j 6), and we ignore the interaction between pairs of planets. m 1 m i The exact dynamics is a perturbation of the simplified dynamics. The simplified system is less expensive to simulate than the exact one, since V simp is a sum of 5 terms, whereas V is a sum of 15 terms. The University of Chicago p. 32

Numerical results (N max iter = 2): energy preservation 1e-04 Sequential coarse scheme Parareal it. #1 Parareal it. #4 Parareal it. #8 Parareal it. #10 Sequential fine scheme 1e-06 1e-08 1e-10 1e-12 1e-14 100 1000 10000 100000 t Relative error on the energy (δt = 10 2, dt = 50, T = 200). For k 8, energy is preserved up to the prescribed tolerance. The University of Chicago p. 33

Numerical results: trajectory accuracy 100 1 Sequential coarse scheme Parareal it. #1 Parareal it. #5 Parareal it. #8 Parareal it. #10 Parareal it. #14 Sequential fine scheme 0.01 1e-04 1e-06 1e-08 1e-10 100 1000 10000 100000 t Error on the trajectory For k 5, large trajectory error. At k = 15, the trajectory error is comparable to the error made by the fine propagator used sequentially. The University of Chicago p. 34

Numerical results: angular momentum preservation 0.01 1e-04 Sequential coarse scheme Parareal it. #1 Parareal it. #5 Parareal it. #10 Parareal it. #15 Sequential fine scheme 1e-06 1e-08 1e-10 1e-12 1e-14 1e-16 1000 10000 100000 t Relative error on the angular momentum Good preservation of the angular momentum (error < 1 %), although the trajectory may be wrong. The University of Chicago p. 35

Numerical results: qualitative trajectory 15 10 Sun Jupiter Saturn Uranus Neptune Pluto 15 10 Sun Jupiter Saturn Uranus Neptune Pluto 5 5 0 0-5 -5-10 -10-15 -15-30 -20-10 0 10 20 30 40-30 -20-10 0 10 20 30 40 15 10 Sun Jupiter Saturn Uranus Neptune Pluto 15 10 Sun Jupiter Saturn Uranus Neptune Pluto 5 5 0 0-5 -5-10 -10-15 -15-30 -20-10 0 10 20 30 40-30 -20-10 0 10 20 30 40 k = 1 (top left), k = 5 (top right), k = 10 (bottom left), k = 15 (bottom right). For k = 5, the trajectory is quantitatively wrong but qualitatively correct! For k 10, the trajectory is quantitatively correct. The University of Chicago p. 36

Computational speed-up With our parameters, the computation cost is mostly due to the fine-scale integrations. Hence, the speed-up is G = T K T = number of processors number of parareal iterations For the outer-solar system test-case, provided we have 1000 processors. G 66 The University of Chicago p. 37

Symplectic parareal algorithm Unclear (no standard way to make a given algorithm symplectic). A first possible approach by G. Bal and Q. Wu (2008): build a generating function S by interpolating parareal results: in practice, for some chosen φ i (q, P), S(q, P) = i a i φ i (q, P), a i to determine. derive from this generating function a symplectic algorithm: p = P + S q (q, P), Q = q + S P (q, P). How to go further? The University of Chicago p. 38

Conclusions the standard parareal algorithm is neither symplectic nor symmetric, even if the building blocks are: this creates issues in the long-time behaviour. easy to symmetrize, not enough due to resonances... symmetrize + symmetric projection good results! Both ingredients are needed. Some open questions: Symplectic parareal algorithm? Understand better the behaviour of the integrators through their modified equations (work in progress). X. Dai, CLB, F. Legoll, Y. Maday, Symmetric parareal algorithms for Hamiltonian systems, arxiv preprint 1011.6222, to appear in M2AN. Related work: M. Gander, E. Hairer, Analysis for parareal algorithms applied to Hamiltonian differential equations, preprint The University of Chicago p. 39