Nonlinear Preconditioning in PETSc

Transcription

1 Nonlinear Preconditioning in PETSc Matthew Knepley PETSc Team Computation Institute University of Chicago Challenges in 21st Century Experimental Mathematical Computation ICERM, Providence, RI July 22, 2014 M. Knepley (UC) NPC ICERM 14 1 / 1

2 The PETSc Team Bill Gropp Barry Smith Satish Balay Jed Brown Matt Knepley Lisandro Dalcin Hong Zhang Mark Adams Peter Brune M. Knepley (UC) NPC ICERM 14 3 / 1

3 Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP, 180, pp (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM 14 4 / 1

4 Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP, 180, pp (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM 14 4 / 1

5 Why Experiment with Solvers? Asymptotic convergence rate Rarely get asymptotics in practice Really want the constant Manner of convergence Stationary iterative methods decrease high frequency error quickly Tuminaro, Walker, Shadid, JCP, 180, pp (2002). Robustness Line search Solver composition M. Knepley (UC) NPC ICERM 14 4 / 1

6 Outline Algorithmics M. Knepley (UC) NPC ICERM 14 5 / 1

7 Abstract System Algorithmics Out prototypical nonlinear equation is: F( x) = b (1) and we define the residual as r( x) = F( x) b (2) M. Knepley (UC) NPC ICERM 14 6 / 1

8 Abstract System Algorithmics Out prototypical nonlinear equation is: F( x) = b (1) and we define the (linear) residual as r( x) = A x b (3) M. Knepley (UC) NPC ICERM 14 6 / 1

9 Algorithmics Linear Left Preconditioning The modified equation becomes P (A x 1 ) b = 0 (4) M. Knepley (UC) NPC ICERM 14 7 / 1

10 Algorithmics Linear Left Preconditioning The modified defect correction equation becomes P (A x 1 i ) b = x i+1 x i (5) M. Knepley (UC) NPC ICERM 14 7 / 1

11 Algorithmics Additive Combination The linear iteration x i+1 = x i (αp 1 + βq 1 )(A x i b) (6) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM 14 8 / 1

12 Algorithmics Additive Combination The linear iteration x i+1 = x i (αp 1 + βq 1 ) r i (7) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM 14 8 / 1

13 Algorithmics Additive Combination The linear iteration x i+1 = x i (αp 1 + βq 1 ) r i (7) becomes the nonlinear iteration x i+1 = x i +α(n (F, x i, b) x i )+β(m(f, x i, b) x i ) (8) M. Knepley (UC) NPC ICERM 14 8 / 1

14 Algorithmics Nonlinear Left Preconditioning From the additive combination, we have P 1 r = x i N (F, x i, b) (9) so we define the preconditioning operation as r L x N (F, x, b) (10) M. Knepley (UC) NPC ICERM 14 9 / 1

15 Algorithmics Multiplicative Combination The linear iteration x i+1 = x i (P 1 + Q 1 Q 1 AP 1 ) r i (11) becomes the nonlinear iteration M. Knepley (UC) NPC ICERM / 1

16 Algorithmics Multiplicative Combination The linear iteration x i+1/2 = x i P 1 r i (12) becomes the nonlinear iteration x i = x i+1/2 Q 1 r i+1/2 (13) M. Knepley (UC) NPC ICERM / 1

17 Algorithmics Multiplicative Combination The linear iteration x i+1/2 = x i P 1 r i (12) x i = x i+1/2 Q 1 r i+1/2 (13) becomes the nonlinear iteration x i+1 = M(F, N (F, x i, b), b) (14) M. Knepley (UC) NPC ICERM / 1

18 Algorithmics Nonlinear Right Preconditioning For the linear case, we have AP 1 y = b (15) x = P 1 y (16) so we define the preconditioning operation as y = M(F(N (F,, b)), x i, b) (17) x = N (F, y, b) (18) M. Knepley (UC) NPC ICERM / 1

19 Algorithmics Nonlinear Preconditioning Type Sym Statement Abbreviation Additive + x + α(m(f, x, b) x) M + N + β(n (F, x, b) x) Multiplicative M(F, N (F, x, b), b) M N Left Prec. L M( x N (F, x, b), x, b) M L N Right Prec. R M(F(N (F, x, b)), x, b) M R N Inner Lin. Inv. \ y = J( x) 1 r( x) = K( J( x), y 0, b) NEWT\K M. Knepley (UC) NPC ICERM / 1

20 Algorithmics Nonlinear Richardson 1: procedure NRICH( F, x i, b) 2: d = r( x i ) 3: x i+1 = x i + λ d λ determined by line search 4: end procedure 5: return x i+1 M. Knepley (UC) NPC ICERM / 1

21 Line Search Algorithmics Equivalent to NRICH L N : NRICH L N M. Knepley (UC) NPC ICERM / 1

22 Line Search Algorithmics Equivalent to NRICH L N : NRICH L N NRICH( x N (F, x, b), x, b) M. Knepley (UC) NPC ICERM / 1

23 Line Search Algorithmics Equivalent to NRICH L N : NRICH L N NRICH( x N (F, x, b), x, b) x i+1 = x i λ r L M. Knepley (UC) NPC ICERM / 1

24 Line Search Algorithmics Equivalent to NRICH L N : NRICH L N NRICH( x N (F, x, b), x, b) x i+1 = x i λ r L x i+1 = x i + λ(n (F, x i, b) x i ) M. Knepley (UC) NPC ICERM / 1

25 Newton-Krylov Algorithmics 1: procedure NEWT\K( F, x i, b) 2: d = J( x i ) 1 r( x i, b) solve by Krylov method 3: x i+1 = x i + λ d λ determined by line search 4: end procedure 5: return x i+1 M. Knepley (UC) NPC ICERM / 1

26 Algorithmics Left Preconditioned Newton-Krylov 1: procedure NEWT\K( x M(F, x, b), x i, 0) 2: d = ( x i M(F, x i, 1 b)) x ( x i M(F, x i, b)) i 3: x i+1 = x i + λ d 4: end procedure 5: return x i+1 M. Knepley (UC) NPC ICERM / 1

27 Algorithmics Jacobian Computation ( x M(F, x i, b)) x i = I M(F, x i, b) x i, Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM / 1

28 Algorithmics Jacobian Computation ( x M(F, x i, b)) x i = I M(F, x i, b) x i, Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM / 1

29 Algorithmics Jacobian Computation Impractical! ( x M(F, x i, b)) x i = I M(F, x i, b) x i, Direct differencing would require one inner nonlinear iteration per Krylov iteration. M. Knepley (UC) NPC ICERM / 1

30 Algorithmics Jacobian Computation Approximation for NASM ( x M(F, x, b)) x = ( x ( x b J b( x b ) 1 F b ( x b ))) x J b ( x b ) 1 J( x) b This would require one inner nonlinear iteration small number of block solves per outer nonlinear iteration. X.-C. Cai and D. E. Keyes, SIAM J. Sci. Comput., 24 (2002), pp M. Knepley (UC) NPC ICERM / 1

31 Algorithmics Right Preconditioned Newton-Krylov 1: procedure NK( F( M( F,, b)), y i, b) 2: x i = M( F, y i, b) 3: d = J( x) 1 r( x i ) 4: x i+1 = x i + λ d λ determined by line search 5: end procedure 6: return x i+1 M. Knepley (UC) NPC ICERM / 1

32 Algorithmics Jacobian Computation First-Order Approximation Only the action of the original Jacobian is needed at first order: y i+1 = y i λ M(F, 1 y i ) J(M(F, y i )) 1 F(M(F, y i )) y i M(F, y i+1 ) = M(F, y i λ M(F, 1 y i ) J(M(F, y i )) 1 F(M(F, y i ))) y i M(F, y i ) λ M(F, 1 y i ) M(F, y i ) J(M(F, y i )) 1 F(M(F, y i ))) y i y i = M(F, y i ) λj(m(f, y i )) 1 F(M(F, y i )) x i+1 = x i λj( x i ) 1 F( x i ) NEWT\K R M is equivalent to NEWT\K M at first order M. Knepley (UC) NPC ICERM / 1

33 Algorithmics Jacobian Computation First-Order Approximation Only the action of the original Jacobian is needed at first order: y i+1 = y i λ M(F, 1 y i ) J(M(F, y i )) 1 F(M(F, y i )) y i M(F, y i+1 ) = M(F, y i λ M(F, 1 y i ) J(M(F, y i )) 1 F(M(F, y i ))) y i M(F, y i ) λ M(F, 1 y i ) M(F, y i ) J(M(F, y i )) 1 F(M(F, y i ))) y i y i = M(F, y i ) λj(m(f, y i )) 1 F(M(F, y i )) x i+1 = x i λj( x i ) 1 F( x i ) NEWT\K R M is equivalent to NEWT\K M at first order M. Knepley (UC) NPC ICERM / 1

34 Algorithmics Jacobian Computation Direct Approximation F(M(F, y i, b)) = J(M(F, y i, b)) M(F, y i, b) y i ( y i+1 y i ) J(M(F, y i, b))(m(f, y i + d, b) x i ) Solve for d Requires inner nonlinear solve for each Krylov iterate Needs FGMRES P. Birken and A. Jameson, J. Num. Meth. in Fluids, 62 (2010), pp M. Knepley (UC) NPC ICERM / 1

35 Outline M. Knepley (UC) NPC ICERM / 1

36 Outline Composition M. Knepley (UC) NPC ICERM / 1

37 SNES ex16 3D Large Deformation Elasticity Composition Ω F S : v dω + loading e y v dω = 0 (19) Ω F Deformation gradient S Second Piola-Kirchhoff tensor Saint Venant-Kirchhoff model of hyperelasticity Ω -arc angle subsection of a cylindrical shell -height thickness -rad inner radius -width width M. Knepley (UC) NPC ICERM / 1

38 Large Deformation Elasticity Composition Unstressed and stressed configurations for the elasticity test problem. M. Knepley (UC) NPC ICERM / 1

39 Large Deformation Elasticity Composition Coloration indicates vertical displacement in meters. M. Knepley (UC) NPC ICERM / 1

40 Large Deformation Elasticity Composition P. Wriggers, Nonlinear Finite Element Methods, Springer, M. Knepley (UC) NPC ICERM / 1

41 Large Deformation Elasticity Running Composition SNES example 16: cd src/snes/examples/tutorials make ex16./ex16 -da_grid_x 401 -da_grid_y 9 -da_grid_z 9 -height 3 -width 3 -rad 100 -young 100 -poisson 0.2 -loading -1 -ploading 0 M. Knepley (UC) NPC ICERM / 1

42 Plain SNES Convergence Composition (NEWT\K MG) and NCG (NEWT\K MG) NCG r(x) Time (sec) M. Knepley (UC) NPC ICERM / 1

43 Plain SNES Convergence Composition Solver T N. It L. It Func Jac PC NPC NCG (NEWT\K MG) M. Knepley (UC) NPC ICERM / 1

44 Composition Composed SNES Convergence NCG(10) + (NEWT\K MG) and NCG(10) (NEWT\K MG) NCG(10)+(NEWT\K MG) NCG(10)*(NEWT\K MG) r(x) Time (sec) M. Knepley (UC) NPC ICERM / 1

45 Composition Composed SNES Convergence Solver T N. It L. It Func Jac PC NPC NCG (NEWT\K MG) NCG(10) (NEWT\K MG) NCG(10) (NEWT\K MG) M. Knepley (UC) NPC ICERM / 1

46 Composition Peconditioned SNES Convergence NGMRES R (NEWT\K MG) and NCG L (NEWT\K MG) NGMRES R(NEWT\K MG) NCG L(NEWT\K MG) r(x) Time (sec) M. Knepley (UC) NPC ICERM / 1

47 Composition Peconditioned SNES Convergence Solver T N. It L. It Func Jac PC NPC NCG (NEWT\K MG) NCG(10) (NEWT\K MG) NCG(10) (NEWT\K MG) NGMRES R (NEWT\K MG) NCG L (NEWT\K MG) M. Knepley (UC) NPC ICERM / 1

48 Outline Multilevel M. Knepley (UC) NPC ICERM / 1

49 SNES ex19 Driven Cavity Flow Multilevel u + Ω = 0 Ω + ( uω) GR x T = 0 T + PR ( ut ) = 0 M. Knepley (UC) NPC ICERM / 1

50 SNES ex19 Driven Cavity Flow Multilevel u + Ω = 0 Ω + ( uω) GR x T = 0 T + PR ( ut ) = 0 M. Knepley (UC) NPC ICERM / 1

51 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e2 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view M. Knepley (UC) NPC ICERM / 1

52 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e2 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm e-08 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 7 M. Knepley (UC) NPC ICERM / 1

53 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e4 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view M. Knepley (UC) NPC ICERM / 1

54 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e4 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm 4.417e-08 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 7 M. Knepley (UC) NPC ICERM / 1

55 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e5 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view M. Knepley (UC) NPC ICERM / 1

56 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e5 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -snes_monitor_short -snes_converged_reason -snes_view lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve did not converge due to DIVERGED_LINEAR_SOLVE iterations 0 M. Knepley (UC) NPC ICERM / 1

57 Driven Cavity Problem Multilevel SNES ex19.c./ex19 -lidvelocity 100 -grashof 1e5 -da_grid_x 16 -da_grid_y 16 -da_refine 2 -pc_type lu -snes_monitor_short -snes_converged_reason -snes_view lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm M. Knepley (UC) NPC ICERM / 1

58 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type newtonls -snes_converged_reason -pc_type lu lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm M. Knepley (UC) NPC ICERM / 1

59 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type fas -snes_converged_reason -fas_levels_snes_type gs -fas_levels_snes_max_it 6 lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm Nonlinear solve did not converge due to DIVERGED_INNER iterations 4 M. Knepley (UC) NPC ICERM / 1

60 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type fas -snes_converged_reason -fas_levels_snes_type gs -fas_levels_snes_max_it 6 -fas_coarse_snes_converged_reason lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 12 1 SNES Function norm Nonlinear solve did not converge due to DIVERGED_MAX_IT its 50 2 SNES Function norm Nonlinear solve did not converge due to DIVERGED_MAX_IT its 50 3 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 22 4 SNES Function norm Nonlinear solve did not converge due to DIVERGED_LINE_SEARCH its 42 Nonlinear solve did not converge due to DIVERGED_INNER iterations 4 M. Knepley (UC) NPC ICERM / 1

61 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type fas -snes_converged_reason -fas_levels_snes_type gs -fas_levels_snes_max_it 6 -fas_coarse_snes_linesearch_type basic -fas_coarse_snes_converged_reason lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 5 48 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 6 49 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 5 50 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 6. M. Knepley (UC) NPC ICERM / 1

62 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type nrichardson -npc_snes_max_it 1 -snes_converged_reason -npc_snes_type fas -npc_fas_coarse_snes_converged_reason -npc_fas_levels_snes_type gs -npc_fas_levels_snes_max_it 6 -npc_fas_coarse_snes_linesearch_type basic lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 6 1 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 27 2 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its SNES Function norm e-05 Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE its 2 44 SNES Function norm e-05 Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE its 2 45 SNES Function norm e-06 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 45 M. Knepley (UC) NPC ICERM / 1

63 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type ngmres -npc_snes_max_it 1 -snes_converged_reason -npc_snes_type fas -npc_fas_coarse_snes_converged_reason -npc_fas_levels_snes_type gs -npc_fas_levels_snes_max_it 6 -npc_fas_coarse_snes_linesearch_type basic lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 6 1 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 13 2 SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its SNES Function norm Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 2 28 SNES Function norm e-05 Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE its 2 29 SNES Function norm e-05 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 29 M. Knepley (UC) NPC ICERM / 1

64 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type ngmres -npc_snes_max_it 1 -snes_converged_reason -npc_snes_type fas -npc_fas_coarse_snes_converged_reason -npc_fas_levels_snes_type newtonls -npc_fas_levels_snes_max_it 6 -npc_fas_levels_snes_linesearch_type basic -npc_fas_levels_snes_max_linear_solve_fail 30 -npc_fas_levels_ksp_max_it 20 -npc_fas_levels_snes_converged_reason -npc_fas_coarse_snes_linesearch_type basic lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm Nonlinear solve did not converge due to DIVERGED_MAX_IT its 6. Nonlinear solve converged due to CONVERGED_SNORM_RELATIVE its 1. 1 SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm e-06 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5 M. Knepley (UC) NPC ICERM / 1

65 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type composite -snes_composite_type additiveoptimal -snes_composite_sneses fas,newtonls -snes_converged_reason -sub_0_fas_levels_snes_type gs -sub_0_fas_levels_snes_max_it 6 -sub_0_fas_coarse_snes_linesearch_type basic -sub_1_snes_linesearch_type basic -sub_1_pc_type mg lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm e-08 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 7 M. Knepley (UC) NPC ICERM / 1

66 Nonlinear Preconditioning Multilevel./ex19 -lidvelocity 100 -grashof 5e4 -da_refine 4 -snes_monitor_short -snes_type composite -snes_composite_type multiplicative -snes_composite_sneses fas,newtonls -snes_converged_reason -sub_0_fas_levels_snes_type gs -sub_0_fas_levels_snes_max_it 6 -sub_0_fas_coarse_snes_linesearch_type basic -sub_1_snes_linesearch_type basic -sub_1_pc_type mg lid velocity = 100, prandtl # = 1, grashof # = SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm SNES Function norm e-08 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE iterations 5 M. Knepley (UC) NPC ICERM / 1

67 Nonlinear Preconditioning Multilevel Solver T N. It L. It Func Jac PC NPC (NEWT\K MG) NGMRES R (NEWT\K MG) FAS FAS + (NEWT\K MG) FAS (NEWT\K MG) NRICH L FAS NGMRES R FAS M. Knepley (UC) NPC ICERM / 1

68 Nonlinear Preconditioning Multilevel See discussion in: Composing scalable nonlinear solvers, Peter Brune, Matthew Knepley, Barry Smith, and Xuemin Tu, ANL/MCS-P , Argonne National Laboratory, M. Knepley (UC) NPC ICERM / 1

69 Outline Magma Dynamics M. Knepley (UC) NPC ICERM / 1

70 Magma Dynamics Magma Dynamics Couples scales Subduction Magma Migration Physics Incompressible fluid Porous solid Variable porosity Deforming matrix Compaction pressure Code generation FEniCS Multiphysics Preconditioning PETSc FieldSplit a Katz, Speigelman M. Knepley (UC) NPC ICERM / 1

71 Magma Dynamics Magma Dynamics Couples scales Subduction Magma Migration Physics Incompressible fluid Porous solid Variable porosity Deforming matrix Compaction pressure Code generation FEniCS Multiphysics Preconditioning PETSc FieldSplit a Katz, Speigelman M. Knepley (UC) NPC ICERM / 1

72 Dimensional Formulation Magma Dynamics φ t (1 φ) v S = 0 ( K ) φ µ p + v S = 0 ) p ζ φ ( v S ) (2η φ ɛ S = 0 M. Knepley (UC) NPC ICERM / 1

73 Closure Conditions Magma Dynamics K φ = K 0 ( φ φ 0 ) n η φ = η 0 exp ( λ(φ φ 0 )) ζ φ = ζ 0 ( φ φ 0 ) m M. Knepley (UC) NPC ICERM / 1

74 Nondimensional Formulation Magma Dynamics φ t (1 φ) v S = 0 ( ) n ) ( R2 φ p + v S = 0 r ζ + 4/3 φ 0 ( ( ) ) m φ ) p v S (2e λ(φ φ 0) ɛ S = 0 φ 0 M. Knepley (UC) NPC ICERM / 1

75 Initial and Boundary conditions Magma Dynamics Initially φ = φ 0 + A cos( k x) where A φ 0 and on the top and bottom boundary K φ p ˆn = 0 v S = ± γ 2 ˆx M. Knepley (UC) NPC ICERM / 1

76 Newton options Magma Dynamics -snes_monitor -snes_converged_reason -snes_type newtonls -snes_linesearch_type bt -snes_fd_color -snes_fd_color_use_mat -mat_coloring_type greedy -ksp_rtol 1.0e-10 -ksp_monitor -ksp_gmres_restart 200 -pc_type fieldsplit -pc_fieldsplit_0_fields 0,2 -pc_fieldsplit_1_fields 1 -pc_fieldsplit_type schur -pc_fieldsplit_schur_precondition selfp -pc_fieldsplit_schur_factorization_type full -fieldsplit_0_pc_type lu -fieldsplit_pressure_ksp_rtol 1.0e-9 -fieldsplit_pressure_pc_type gamg -fieldsplit_pressure_ksp_monitor -fieldsplit_pressure_ksp_gmres_restart 100 -fieldsplit_pressure_ksp_max_it 200 M. Knepley (UC) NPC ICERM / 1

77 Newton options Separate porosity Magma Dynamics -pc_type fieldsplit -pc_fieldsplit_0_fields 0,1 -pc_fieldsplit_1_fields 2 -pc_fieldsplit_type multiplicative -fieldsplit_0_pc_type fieldsplit -fieldsplit_0_pc_fieldsplit_type schur -fieldsplit_0_pc_fieldsplit_schur_precondition selfp -fieldsplit_0_pc_fieldsplit_schur_factorization_type full -fieldsplit_0_fieldsplit_velocity_pc_type lu -fieldsplit_0_fieldsplit_pressure_ksp_rtol 1.0e-9 -fieldsplit_0_fieldsplit_pressure_pc_type gamg -fieldsplit_0_fieldsplit_pressure_ksp_monitor -fieldsplit_0_fieldsplit_pressure_ksp_gmres_restart 100 -fieldsplit_fieldsplit_0_pressure_ksp_max_it 200 M. Knepley (UC) NPC ICERM / 1

78 Early Newton convergence Magma Dynamics 0 TS dt 0.01 time 0 0 SNES Function norm e-03 Linear pressure_ solve converged due to CONVERGED_RTOL its 10 0 KSP Residual norm e+00 Linear pressure_ solve converged due to CONVERGED_RTOL its 10 1 KSP Residual norm e-03 Linear pressure_ solve converged due to CONVERGED_RTOL its 11 2 KSP Residual norm e-13 Linear solve converged due to CONVERGED_RTOL its 2 1 SNES Function norm e-06 Linear pressure_ solve converged due to CONVERGED_RTOL its 8 0 KSP Residual norm e-02 Linear pressure_ solve converged due to CONVERGED_RTOL its 8 1 KSP Residual norm e-07 Linear pressure_ solve converged due to CONVERGED_RTOL its 8 2 KSP Residual norm e-14 Linear solve converged due to CONVERGED_RTOL its 2 2 SNES Function norm e-11 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 2 1 TS dt 0.01 time 0.01 M. Knepley (UC) NPC ICERM / 1

79 Later Newton convergence Magma Dynamics 0 TS dt 0.01 time SNES Function norm e-03 Linear pressure_ solve converged due to CONVERGED_RTOL its 16 Linear pressure_ solve converged due to CONVERGED_RTOL its 16 Linear pressure_ solve converged due to CONVERGED_RTOL its 16 Linear solve converged due to CONVERGED_RTOL its 2 1 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL its 2 2 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL its 2 3 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL its 2 4 SNES Function norm e-04 Linear solve converged due to CONVERGED_RTOL its 2 5 SNES Function norm e-06 Linear solve converged due to CONVERGED_RTOL its 2 6 SNES Function norm e-08 Linear solve converged due to CONVERGED_RTOL its 2 7 SNES Function norm e-14 Nonlinear solve converged due to CONVERGED_FNORM_RELATIVE its 7 1 TS dt 0.01 time 0.64 M. Knepley (UC) NPC ICERM / 1

80 Newton failure Magma Dynamics 0 TS dt 0.01 time 0.64 Time 0.64 L_2 Error: [ , , ] 0 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 2 1 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 3 2 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 5 3 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 6 4 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 7 5 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 7. 9 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 7 10 SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations SNES Function norm e-03 Linear solve converged due to CONVERGED_RTOL iterations 15. M. Knepley (UC) NPC ICERM / 1

81 NCG+Newton options Magma Dynamics -snes_monitor -snes_converged_reason -snes_type composite -snes_composite_type multiplicative -snes_composite_sneses ncg,newtonls -sub_0_snes_monitor -sub_1_snes_monitor -sub_0_snes_type ncg -sub_0_snes_linesearch_type cp -sub_0_snes_max_it 5 -sub_1_snes_linesearch_type bt -sub_1_snes_fd_color -sub_1_snes_fd_color_use_mat -mat_coloring_type greedy -sub_1_ksp_rtol 1.0e-10 -sub_1_ksp_monitor -sub_1_ksp_gmres_restart 200 -sub_1_pc_type fieldsplit -sub_1_pc_fieldsplit_0_fields 0,2 -sub_1_pc_fieldsplit_1_fields 1 -sub_1_pc_fieldsplit_type schur -sub_1_pc_fieldsplit_schur_precondition selfp -sub_1_pc_fieldsplit_schur_factorization_type full -sub_1_fieldsplit_0_pc_type lu -sub_1_fieldsplit_pressure_ksp_rtol 1.0e-9 -sub_1_fieldsplit_pressure_pc_type gamg -sub_1_fieldsplit_pressure_ksp_gmres_restart 100 -sub_1_fieldsplit_pressure_ksp_max_it 200 M. Knepley (UC) NPC ICERM / 1

82 NCG+Newton convergence Magma Dynamics 0 TS dt 0.01 time SNES Function norm e-03 0 SNES Function norm e-03 1 SNES Function norm e-02 2 SNES Function norm e-02 3 SNES Function norm e-02 4 SNES Function norm e-02 5 SNES Function norm e-02 0 SNES Function norm e-02 0 KSP Residual norm e+00 1 KSP Residual norm e-05 2 KSP Residual norm e-09 3 KSP Residual norm e-12 1 SNES Function norm e-03 1 SNES Function norm e-03 2 SNES Function norm e-02 3 SNES Function norm e-02 4 SNES Function norm e-03 5 SNES Function norm e-04 6 SNES Function norm e-05 7 SNES Function norm e-08 8 SNES Function norm e-13 1 TS dt 0.01 time 0.65 M. Knepley (UC) NPC ICERM / 1

83 FAS options Magma Dynamics Top level -snes_monitor -snes_converged_reason -snes_type fas -snes_fas_type full -snes_fas_levels 4 -fas_levels_3_snes_monitor -fas_levels_3_snes_converged_reason -fas_levels_3_snes_atol 1.0e-9 -fas_levels_3_snes_max_it 2 -fas_levels_3_snes_type newtonls -fas_levels_3_snes_linesearch_type bt -fas_levels_3_snes_fd_color -fas_levels_3_snes_fd_color_use_mat -fas_levels_3_ksp_rtol 1.0e-10 -mat_coloring_type greedy -fas_levels_3_ksp_gmres_restart 50 -fas_levels_3_ksp_max_it 200 -fas_levels_3_pc_type fieldsplit -fas_levels_3_pc_fieldsplit_0_fields 0,2 -fas_levels_3_pc_fieldsplit_1_fields 1 -fas_levels_3_pc_fieldsplit_type schur -fas_levels_3_pc_fieldsplit_schur_precondition selfp -fas_levels_3_pc_fieldsplit_schur_factorization_type full -fas_levels_3_fieldsplit_0_pc_type lu -fas_levels_3_fieldsplit_pressure_ksp_rtol 1.0e-9 -fas_levels_3_fieldsplit_pressure_pc_type gamg -fas_levels_3_fieldsplit_pressure_ksp_gmres_restart 100 -fas_levels_3_fieldsplit_pressure_ksp_max_it 200 M. Knepley (UC) NPC ICERM / 1

84 FAS options Magma Dynamics 2nd level -fas_levels_2_snes_monitor -fas_levels_2_snes_converged_reason -fas_levels_2_snes_atol 1.0e-9 -fas_levels_2_snes_max_it 2 -fas_levels_2_snes_type newtonls -fas_levels_2_snes_linesearch_type bt -fas_levels_2_snes_fd_color -fas_levels_2_snes_fd_color_use_mat -fas_levels_2_ksp_rtol 1.0e-10 -fas_levels_2_ksp_gmres_restart 50 -fas_levels_2_pc_type fieldsplit -fas_levels_2_pc_fieldsplit_0_fields 0,2 -fas_levels_2_pc_fieldsplit_1_fields 1 -fas_levels_2_pc_fieldsplit_type schur -fas_levels_2_pc_fieldsplit_schur_precondition selfp -fas_levels_2_pc_fieldsplit_schur_factorization_type full -fas_levels_2_fieldsplit_0_pc_type lu -fas_levels_2_fieldsplit_pressure_ksp_rtol 1.0e-9 -fas_levels_2_fieldsplit_pressure_pc_type gamg -fas_levels_2_fieldsplit_pressure_ksp_gmres_restart 100 -fas_levels_2_fieldsplit_pressure_ksp_max_it 200 M. Knepley (UC) NPC ICERM / 1

85 FAS options Magma Dynamics 1st level -fas_levels_1_snes_monitor -fas_levels_1_snes_converged_reason -fas_levels_1_snes_atol 1.0e-9 -fas_levels_1_snes_type newtonls -fas_levels_1_snes_linesearch_type bt -fas_levels_1_snes_fd_color -fas_levels_1_snes_fd_color_use_mat -fas_levels_1_ksp_rtol 1.0e-10 -fas_levels_1_ksp_gmres_restart 50 -fas_levels_1_pc_type fieldsplit -fas_levels_1_pc_fieldsplit_0_fields 0,2 -fas_levels_1_pc_fieldsplit_1_fields 1 -fas_levels_1_pc_fieldsplit_type schur -fas_levels_1_pc_fieldsplit_schur_precondition selfp -fas_levels_1_pc_fieldsplit_schur_factorization_type full -fas_levels_1_fieldsplit_0_pc_type lu -fas_levels_1_fieldsplit_pressure_ksp_rtol 1.0e-9 -fas_levels_1_fieldsplit_pressure_pc_type gamg M. Knepley (UC) NPC ICERM / 1

86 FAS options Magma Dynamics Coarse level -fas_coarse_snes_monitor -fas_coarse_snes_converged_reason -fas_coarse_snes_atol 1.0e-9 -fas_coarse_snes_type newtonls -fas_coarse_snes_linesearch_type bt -fas_coarse_snes_fd_color -fas_coarse_snes_fd_color_use_mat -fas_coarse_ksp_rtol 1.0e-10 -fas_coarse_ksp_gmres_restart 50 -fas_coarse_pc_type fieldsplit -fas_coarse_pc_fieldsplit_0_fields 0,2 -fas_coarse_pc_fieldsplit_1_fields 1 -fas_coarse_pc_fieldsplit_type schur -fas_coarse_pc_fieldsplit_schur_precondition selfp -fas_coarse_pc_fieldsplit_schur_factorization_type full -fas_coarse_fieldsplit_0_pc_type lu -fas_coarse_fieldsplit_pressure_ksp_rtol 1.0e-9 -fas_coarse_fieldsplit_pressure_pc_type gamg M. Knepley (UC) NPC ICERM / 1

87 FAS convergence Magma Dynamics 0 TS dt 0.01 time SNES Function norm e-03 2 SNES Function norm e-03 2 SNES Function norm e-15 1 SNES Function norm e-04 1 SNES Function norm e-11 1 SNES Function norm e-10 2 SNES Function norm e-08 1 SNES Function norm e-15 0 SNES Function norm e-15 0 SNES Function norm e-15 1 SNES Function norm e-14 2 SNES Function norm e-02 2 SNES Function norm e-12 0 SNES Function norm e-12 0 SNES Function norm e-13 0 SNES Function norm e-12 0 SNES Function norm e-12 2 SNES Function norm e-05 1 SNES Function norm e-05 2 SNES Function norm e-08 3 SNES Function norm e-11 1 TS dt 0.01 time 0.65 M. Knepley (UC) NPC ICERM / 1

88 Magma Dynamics See discussion in: Composing scalable nonlinear solvers, Peter Brune, Matthew Knepley, Barry Smith, and Xuemin Tu, ANL/MCS-P , Argonne National Laboratory, M. Knepley (UC) NPC ICERM / 1

89 What Are We Missing? Magma Dynamics We need a short time convergence theory: Most iterations never enter the asymptotic regime Most complex solvers are composed We need a viable nonlinear smoother: GS is too expensive for FEM NASM is a possibility, M. Knepley (UC) NPC ICERM / 1

90 Nonlinear GMRES Extra 1: procedure NGMRES(F, x i x i m+1, b ) 2: di = r( x i ) 3: x i M = x i + λ d i 4: Fi M = r( x i M ) 5: minimize r((1 i 1 k=i m α i) x M i + {α i m α i 1 } 6: x A i = (1 i 1 k=i m α i) x M + 7: x i+1 = x A or x M 8: end procedure 9: return x i+1 i i if x A i i 1 k=i m is insufficient. i 1 k=i m α k x k α k x k ) 2 over Can emulate Anderson mixing and DIIS M. Knepley (UC) NPC ICERM / 1

91 Nonlinear GMRES Extra 1: procedure NGMRES(F, x i x i m+1, b ) 2: di = r( x i ) 3: x i M = x i + λ d i 4: Fi M = r( x i M ) 5: minimize r((1 i 1 k=i m α i) x M i + {α i m α i 1 } 6: x A i = (1 i 1 k=i m α i) x M + 7: x i+1 = x A or x M 8: end procedure 9: return x i+1 i i if x A i i 1 k=i m is insufficient. i 1 k=i m α k x k α k x k ) 2 over Can emulate Anderson mixing and DIIS M. Knepley (UC) NPC ICERM / 1

92 Extra Full Approximation Scheme (FAS) Nonlinear Multigrid 1: procedure FAS( F, x i, b) 2: x s = M s (F, x i, b) 3: x c = R x s 4: bc = F c ( x c ) R[F( x s ) b] 5: x s = x s + P[FAS( F c, x c, b c ) x c ] 6: x i+1 = M s (F, x s, b) 7: end procedure 8: return x i+1 M. Knepley (UC) NPC ICERM / 1

93 Other Nonlinear Solvers Extra NASM Nonlinear Additive Schwarz NGS Nonlinear Gauss-Siedel NCG Nonlinear Conjugate Gradients QN Quasi-Newton methods M. Knepley (UC) NPC ICERM / 1