Parallel in Time Algorithms for Multiscale Dynamical Systems using Interpolation and Neural Networks

Transcription

1 Parallel in Time Algorithms for Multiscale Dynamical Systems using Interpolation and Neural Networks Gopal Yalla Björn Engquist University of Texas at Austin Institute for Computational Engineering and Sciences April 17, 2018 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

2 Outline 1 Parallel in Time Algorithms 2 Parareal Algorithm 3 Phase-accurate Coarse Solver 4 Numerical Examples 5 Future Work Yalla & Engquist Parallel in Time Algorithms April 17, / 20

3 Parallel in Time Algorithms 1 Required for massively parallel simulations of systems governed by time-dependent dynamical systems, e.g., molecular dynamics. Challenge arises due to causality in time. 1 Martin J Gander. 50 years of time parallel time integration. In: Multiple Shooting and Time Domain Decomposition Methods. Springer, 2015, pp Yalla & Engquist Parallel in Time Algorithms April 17, / 20

4 Parallel in Time Algorithms 1 1 Multiple Shooting Methods 2 Domain Decomposition & Waveform Relaxation Methods 3 Multigrid Based Methods 1 Martin J Gander. 50 years of time parallel time integration. In: Multiple Shooting and Time Domain Decomposition Methods. Springer, 2015, pp Yalla & Engquist Parallel in Time Algorithms April 17, / 20

5 The Parareal Algorithm Consider the dynamical system: u (t) = f(u(t)), t (t 0, t f ) u(t 0 ) = u 0 Divide the time domain (t 0, t f ) into N equal subdomains Ω n = (T n, T n+1 ). u n(t) = f(u n (t)) t (T n, T n+1 ) u(t n ) = U n This is simply a shooting method in time, and it equivalent to solving: U 0 u 0 U 1 φ T0 (U 0 ). = 0 (3) U N 1 φ TN 2 (U N 2 )) where φ Tn (U n ) is solution of (1) with initial condition U n after time T n. Yalla & Engquist Parallel in Time Algorithms April 17, / 20 (1) (2)

6 The Parareal Algorithm Consider the dynamical system: u (t) = f(u(t)), t (t 0, t f ) u(t 0 ) = u 0 Divide the time domain (t 0, t f ) into N equal subdomains Ω n = (T n, T n+1 ). Parareal Algorithm (1) U k+1 0 = u 0 U k+1 n = G(U k+1 n 1 ) + [F(U k n 1) G(U k n 1) ] Un k is the solution u(t n) on the k th iteration of the algorithm. G denotes a coarse solver F denotes a fine solver Yalla & Engquist Parallel in Time Algorithms April 17, / 20

7 U k 0 = u 0 U k n = G(U k n 1 ) + [ F(Un 1 k 1 ] k 1 ) G(Un 1 ) Video Source: Yalla & Engquist Parallel in Time Algorithms April 17, / 20

8 U k 0 = u 0 U k n = G(U k n 1 ) + [ F(Un 1 k 1 ] k 1 ) G(Un 1 ) Efficiency = Iterations for Convergence / Total Number of Steps Video Source: Yalla & Engquist Parallel in Time Algorithms April 17, / 20

9 Drawbacks of the Parareal Algorithm For Hamiltonian systems, very high accuracy is required for the coarse solver! 2 A good coarse approximation is needed. Typical coarse integrators fail to predict phase correctly, leading to blow up of error during correction step with fine solver. 2 Martin J Gander and Ernst Hairer. Analysis for parareal algorithms applied to Hamiltonian differential equations. In: Journal of Computational and Applied Mathematics 259 (2014), pp Yalla & Engquist Parallel in Time Algorithms April 17, / 20

10 Phase Plane Map Consider a set of M initial conditions (training points) {u i 0 }M i=1 and a set of M corresponding (target points) {v i } M i=1 defined by, v i = F ( u i ) 0 i = 1,..., M The set {u i 0 vi } M i=1 acts as a type of look-up table for future initials conditions 3. Definition Define a phase plane map G map ( { u i 0 vi} M i=1, U n) to be a coarse solver that uses the information contained in { u i 0 vi} M i=1 and a point U n at time T n, to determine the solution U n+1 at time T n+1. G map can be defined through, e.g., Interpolation or a Neural Network. 3 Jürg Nievergelt. Parallel methods for integrating ordinary differential equations. In: Communications of the ACM 7.12 (1964), pp Yalla & Engquist Parallel in Time Algorithms April 17, / 20

11 Phase Plane Map v i = F ( u i ) 0 U n+1 G map ( { u i 0 vi} M i=1, U n) Yalla & Engquist Parallel in Time Algorithms April 17, / 20

12 Phase Plane Map The computation of v i = F(u i 0 ) is embarrassingly parallel. Yalla & Engquist Parallel in Time Algorithms April 17, / 20

13 Phase Plane Map For autonomous ODEs, G map can be applied over each subdomain Ω n. Else, generate {u i j vi } M i=1 for each Ω j, j = 0,..., N 1. Yalla & Engquist Parallel in Time Algorithms April 17, / 20

14 Phase Plane Map 1 How to select {u i 0 }M i=1? 2 How to select M? 3 Which method is best for G map? Yalla & Engquist Parallel in Time Algorithms April 17, / 20

15 Numerical Examples Traditional parareal algorithm = G = RK4 with step size T/N Modified parareal algorithm = G = phase plane map with step size T/N For both, the fine solver is an adaptive RK45 solver. T = Total Simulation Time. N = Number of time-subdomains Ω n. Yalla & Engquist Parallel in Time Algorithms April 17, / 20

16 Harmonic Oscillator Consider a simple harmonic oscillator: with ε = 0.01 T = 70 (1000 revolutions) x(0) = [ 2/9, 2/3] T x 1 = 1 ε x 2 x 2 = 1 ε x 1 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

17 Harmonic Oscillator N = 100, M = 2 Iterations: 1 N = 10, 000 Iterations: 42 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

18 Lennard-Jones System r = 2 m F (r) F (r) = V (r) [ (σ ) 12 ( σ ) ] 6 V (r) = 4ε r r ε = well depth σ = zero distance r min = 1.12 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

19 Lennard-Jones System Iterations: 10 Iterations: 53 T = 50 N = 400 r 0 = [1.2, 0] T M=50 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

20 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

21 ( ) r = 1.12 M Iterations ( ) r = 1.2 M Iterations ( ) r = 1.4 M Iterations Yalla & Engquist Parallel in Time Algorithms April 17, / 20

22 High Dimensional Harmonic Oscillator ẋ = 1 ε Ax where x = [x 1, v 1, x 2, v 2, x 3, v 3, x 4, v 4 ] T, ε = 0.01, and A R 8 8 is defined as, 0 a a b A = 0 0 b c c d d 0 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

23 High Dimensional Harmonic Oscillator ẋ = 1 ε Ax where x = [x 1, v 1, x 2, v 2, x 3, v 3, x 4, v 4 ] T, ε = 0.01, and A R 8 8 is defined as, 0 a a b A = 0 0 b c c d d 0 *Neural network defined with logistic activation function, size , with a limited memory BFGS solver for optimization. Yalla & Engquist Parallel in Time Algorithms April 17, / 20

24 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

25 Localized Multiscale Problem 1 Type A: Problems that contain local defects or singularities. Microscale model needed only near defects. 2 Type B: Microscale model needed everywhere as a supplement to macroscale model, e.g., resolve constitutive model. 4. Type A Example: An N-body Problem: m i q i = n j i m i m j (q j q i ) q j q i 3 m i denotes the mass of the i th body. q i denotes the position of the i th body. Let n = 2 and assume m 2 >> m 1. 4 Weinan E. Principles of multiscale modeling. Cambridge University Press, Yalla & Engquist Parallel in Time Algorithms April 17, / 20

26 Apply a traditional parareal algorithm with RK45 as fine scale solver with adaptive step size. RK4 as coarse solver with step size T/N. T = 0.5 N = 5 Iterations = 2 Yalla & Engquist Parallel in Time Algorithms April 17, / 20

27 Conclusion & Future Work Parallel in time algorithms needed for massively parallel simulations of time dependent dynamical systems. Results for parareal + phase plane map show promise. Demonstrate a proof of concept. Future Work Potential Improvements include use of modern sparse grid and adaptive methods for interpolation or optimizing neural network approach. More realistic examples in the realm of molecular dynamics. Scaling/speedup results on modern supercomputing platforms. Yalla & Engquist Parallel in Time Algorithms April 17, / 20

28 References Weinan E. Principles of multiscale modeling. Cambridge University Press, Martin J Gander. 50 years of time parallel time integration. In: Multiple Shooting and Time Domain Decomposition Methods. Springer, 2015, pp Martin J Gander and Ernst Hairer. Analysis for parareal algorithms applied to Hamiltonian differential equations. In: Journal of Computational and Applied Mathematics 259 (2014), pp Jacques-Louis Lions, Yvon Maday, et al. A parareal method in time discretization of pde s. In: Comptes Rendus de l Académie des Sciences-Series I-Mathematics (2001), pp Jürg Nievergelt. Parallel methods for integrating ordinary differential equations. In: Communications of the ACM 7.12 (1964), pp *Code will be available at Yalla & Engquist Parallel in Time Algorithms April 17, / 20