Multi-grid Methods in Lattice Field Theory

Transcription

1 Multi-grid Methods in Lattice Field Theory Rich Brower, Physics, Boston University Jan 10, 2017 (SciDAC Software Co-director and NVIDIA CUDA Fellow) Machine Learning/Multi-scale Physics smoothing prolongation (interpolation) Fine Grid restriction The Multigrid V-cycle Smaller Coarse Grid

2 Lattice Field Theory has Come of Age CM Mflops (1989) 10 7 increase in 25 years BF/Q 1 Pflops (2012)

3 Lagrangian for QCD What so difficult about this! Z S = d 4 xl (only explicit scale) L(x) = 1 4g 2 F ab µ F ab µ + a ab µ(@ + A ab µ ) b + m 3x3 Maxwell matrix field & 2+ Dirac quarks 1 color charge g & small quark masses m. Prob Sample quantum probability of gluonic plasma : Z DA µ (x)det[d quark (A)D quark(a)]e R d 4 xf 2 /2g 2

4 All prediction from Quantum Field Theory require Algorithms Z = Path Integral exp[ - Action] Feynman Diagrams Wilson Lattice QCD PDE/FEM Schwartz Schurs thooft Dim Reg Twisters Bootstrap OPE & Renormalization Group AdS/CFT Domain Wall Wilson Flow Multi- Grid DD

5 Peter s Multiscale P. Boyle Yesterday What about multi-scales inside of Lattice QCD? a(lattice) 1/M proton 1/m L(box) 0.06 fermi 0.2 fermi 1.4 fermi 6.0 fermi =) L = O(100) or Minimum Lattice Volume 100 4!

6 Multigrid: Case History in Algorithm Development * MG is always the Future : Anonymous, JLab 2008 ** But, the future has arrived! : Rich, Oak Ridge 2013 History Lessons ( ) * - Cause of early failure Modern Era ( ) - 5 years to put into production the QCD MG Solver for Wilson-clover) Future** ( ) - Domain Wall & Staggered Solvers, HMC evolution, etc - Adaptation to heterogeneous architectures, etc.

7 Outline *Combining Renormalization Group And Multigrid Methods 1988 R. Brower, R. Giles, K.J.M. Moriarty, P. Tamayo 1. Lattice Gauge Multi-grid: Why was it so difficult? Lessons from 1990s: The scaling & RG metaphor* 2. The Adaptive Geometric MG break through - Wilson clover (twisted mass) - Staggered (?) - Domain Wall (overlap/peter Boyle!) 3. Multi-scale extensions: - Monte Carlo MG (Endres et al?) - Quantum Finite Elements (FEM + quantum) - SUSY/Graphene/etc

8 A faithful Discrete Dirac PDE on Lattice is not trivial Standard Finite Difference or Finite Element Methods Fail There are several popular choice and trade offs Each poses different Multigrid Challenges Wilson Clover & Twisted Mass Staggered (or Kogut-Susskind, SUSY, Graphene) Domain Wall & Overlap (exact chiral) Simplicial Lattice (Random Flat Christ et al) (General Riemann Brower et al)

9 Dirac Equation: Math Preliminaries Continuum: µ ia µ ) (x)+m = b(x) or X µ ij µ ia ab µ ) jb (x)+m ia = b ia (x) 3 by 3 gauged colors 4 x 4 (d/2 x d/2) spin matrices Even more compact Linear Op form (D + m) = b D is anti-hermitian implies spectral (D + m) i =(i + m) i Normal Operator (D + m) (D + m) = (@ µ ia µ ) 2 µ F µ + m 2

10 Putting it on hypercubic lattice D quark = d D(U, m) = 1 2 just phases! 1 µ 2 dx µ=1 U(x, x + µ) Wilson µ (~x) U µ (~x) ~x,~y ˆµ U µ(~x ˆµ) ~x,~y+ˆµ + m ~x,~y Staggered µ =( 1) x 2+ x µ 1+ µ 2 U(x + µ, x)+m q Domain Wall is 5d Wilson-like x x+µ U glue (x, x + µ) =e iaa µ(x)

11 Scaling & Ren Group Metaphor Massless Laplace: (x)! 2 d ( x) r 2 (x, y, ]=e2 d (x, y, ) Solution (Green s function) (x, y, )= 1 S d e 2 ( p x 2 + y 2 + z) d 2 (x, y) = 1 2 log(p x 2 + y 2 ) for d =2

12 Naive Scaling and 1-d MG Toy Example (D )(x) = (x + a) 2 (x)+ (x a)+a 2 m 2 (x) =b(x) a 2a Restriction R = P 2 a a Prolongation P (1) Blocking preserves the scale invariant const solutions (null state) (2) Coarse operator is renormalized: m 2 m ( in units a = 1)

13 Dwilson = d 1 = ( 2 ( where µ 1 µ 2 µ U (x, x + µ) µ) µ (x) = U (x, x + µ) (x + µ) 1+ 2 µ U (x + µ, x) + mq µ (x) ' a(@µ Wilson Spectra in 2d Aµ (x)) (x) )

14 QCD MG attempts in 1990 s See Thomas Kalkretuer hep-lat/ review on MG Methods for Propagators in LGT. Israel: Ben-Av, M. Harmatz, P.G. Lauwers & S.Solomon Boston: Brower, Edwards, Rebbi & Vicari Amsterdam: A. Hulsebos, J Smit J. C. Vick Amsterdam: A. Hulsebos, J Smit J. C. Vick

15 QCD MG failure in 1990s:

16 2x2 Blocks for U(1) Dirac β = 1 2-d Lattice, U µ (x) on links Ψ(x) on sites Gauss-Jacobi (Diamond), CG (circle), V cycle (square), W cycle (star)

17 Universal of Critical Slowing down:! = 3 (cross) 10(plus) 100( square) Gauss-Jacobi (Diamond), CG(circle), 3 level (square & star) = F (ml )

18 Lessons from MG attempts in 1990s Partial success at weak coupling. Local real space RG blocking but not perfect. Maintain Gauge invariance 5 Maintain Hermiticity BUT why did it fail at small mass?

19 Perturbative RG Metaphor? Education is knowing how far to push a metaphor!

20 Why Didn t It work?

21 Instantons, Topological Zero Modes (Atiyah-Singer index) and Confinement length l σ ia 0 (x) = 1 x ˆxµ ik µ (1 + ) kj 2 ja l σ

22 A little knowledge is a dangerous thing Lagrangian (i.e. PDE s) Lattice (a >0) (i.e.computer) Quantum Theory * (i.e.nature) Rotational(Lorentz) Invariance Gauge Invariance Scale Invariance Chiral Invariance / *Must take lattice spacing to zero for the quantum averages.

23 2. Adaptive Multigrid Revolution? 2. Machine Learning Multigrid Revolution!

24 Adaptive Smooth Aggregations Algebraic MultiGrid Slow convergence of Dirac solver is due small eigenvalues for vectors in near null subspace: S. smoothing prolongation (interpolation) D : S ' 0 Fine Grid restriction The Multi-grid V-cycle Spilt the vector space into near null space and the complement S S? Smaller Coarse Grid

25 2-level Multi-grid Cycle (simplified) Smooth: x 0 =(1 D)x + b r 0 =(1 D)r Project: ˆD = P DP ˆr = P r Approx. Solve: Prolongate Update ˆDê =ˆr =) ê ' ˆD 1 P r e = P ê x 00 = x 0 + e Petrov-Galerkin Oblique Projector: Exact Upscale r exact =[1 DP 1 P DP P ]r =) P r exact =0

26 Adaptive Smooth Aggregation Algebraic Multigrid Adaptive*multigrid*algorithm*for*the*lattice*Wilson7Dirac*operator *R.*Babich,*J.*Brannick,*R.*C.* Brower,*M.*A.*Clark,*T.*Manteuffel,*S.*McCormick,*J.*C.*Osborn,*and*C.*Rebbi,**PRL.**(2010).*

27 Good News/Bad News More Data: Should Save MG projectors with lattice Actually MG error is smaller at fixed Residual

28 Adaptive Smooth Aggregation Algebraic Multigrid Adaptive*multigrid*algorithm*for*the*lattice*Wilson7Dirac*operator *R.*Babich,*J.*Brannick,*R.*C.* Brower,*M.*A.*Clark,*T.*Manteuffel,*S.*McCormick,*J.*C.*Osborn,*and*C.*Rebbi,**PRL.**(2010).*

29 Good News/Bad News More Data: Should Save MG projectors with lattice Actually MG error is smaller at fixed Residual

30 Staggered Multigrid: Preliminary (Brower, Clark, Strelchenko and Weinberg) 1 µ (x)[uµ (x, x + µ) U (x + µ, x)] + m 2 1 = µ (~x)[ µ µ] + m 2 D(U, m) = 2d Staggered Spectrum

31 Normal Equation (e/o precond): Free field is 2a Laplace D (U, m)d(u, m) = apple m Doe m D eo apple m D eo Doe m = apple m 2 D eo D oe 0 0 m 2 D oe D eo Total D eo,d oe applications, Normal eqn 128 2, CG on Even-Odd 256 2, CG on Even-Odd 128 2, MG-GCR 256 2, MG-GCR D applications m

32 Spurious Galerkin Eigenmodes 0.2 Eigenvalues 32 2, =6.0, Naive Galerkin 0.15 Im( ) D ˆD ˆD ˆD = P DP ˆD = P ˆDP

33 Removing Spurious Galerkin Eigenmodes Eigenvalues 32 2, =6.0, HybridAlgorithm L1: D L2: ˆD +0.16[D d D] T L3: ˆD [D d D] T Im( ) Re( ) ˆD = P DP +[P D DP] T

34 Truncated Normal Stabilized Staggered MG Total D eo,d oe applications, D solve 128 2, CG on Even-Odd 256 2, CG on Even-Odd 128 2, MG-GCR 256 2, MG-GCR D applications m

35 Domain Wall Hierarchically deflated conjugate gradient P A Boyle (Edinburgh U.). Feb 11, pp. EDINBURGH arxiv: QCD Multigrid Algorithms for Domain-Wall Fermions Saul D. Cohen (Washington U., Seattle), R.C. Brower (Boston U., Ctr. Comp. Sci.), M.A. Clark (Harvard-Smithsonian Ctr. Astrophys.), J.C. Osborn (Argonne). May pp. Published in PoS LATTICE2011 (2011) 030 2d U(1)

36 Domain Wall: with extra 5th dimension of size Ls

37 2 + 1 Domain Wall Spectrum: Violently Non-Normal Operator

38 Non Normal Non Hermitian Non Pos. Def. (Saul Cohen)

39 High Performance on GPUs Cost in $s reduced by a factor of at least 100+ GPU O(10+) MG O(10+)

40 QUDA: NVIDIA GPU QCD on CUDA team Ron Babich (BU-> NVIDIA)! Kip Barros (BU ->LANL)! Rich Brower (Boston University)! Michael Cheng (Boston University)! Mike Clark (BU-> NVIDIA)! Justin Foley (University of Utah)! Steve Gottlieb (Indiana University)! Bálint Joó (Jlab)! Claudio Rebbi (Boston University)! Guochun Shi (NCSA -> Google)! Alexei Strelchenko (Cyprus Inst.-> FNAL)! Hyung-Jin Kim (BNL)! Mathias Wagner (Bielefeld -> Indiana Univ)! Frank Winter (UoE -> Jlab)

41 Mapping Multi-scale Algorithms to Multi-scale Architecture

42 First Step Wilson-Dslash on GPUs REDUCE MEMORY TRAFFIC: (1) Lossless Data Compression: SU(3) matrices are all unitary complex matrices with det = number parameterization: reconstruct full matrix on the fly in registers Additional 384 (free) flops per site Also have an 8-number parameterization of SU(3) manifold (requires sin/cos and sqrt) ( ) ( ) c = (axb)* Group Manifold:S 3 S 5 a1 a2 a3 b1 b2 b3 c1 c2 c3 a1 a2 a3 b1 b2 b3 (2) Similarity Transforms to increase sparsity (3) Mixed Precision: Use 16-bit fixed-point representation. No loss in precision with mixedprecision solves (Almost a free lunch:small increase in iteration count)

43 Latest Results from SuperComputing 2016 (Kate Clark) Accelerating Lattice QCD Multigrid on GPUs Using Fine-Grained Parallelization M.A. Clark, Bálint Joó, Alexei Strelchenko, Michael Cheng, Arjun Gambhir, Richard Brower. Dec 22, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '16),

44 3. Multi-scale Extensions Monte Carlo Equilibration HMC dynamics Simplicial Lattice Riemann Manifolds Recursive spherical lattice applied Conformal Field Theory Deep learning: simplicial renormalization.

45 Multigrid ideas for HMC Very important and difficult problem Major focus of US Exascale Software project (see Poster by Mike Endres) New simulation strategies for lattice gauge theory Michael G. Endres Lattice 2016 Multiscale Monte Carlo equilibration: Pure Yang-Mills theory Michael G. Endres, Richard C. Brower, William Detmold, Kostas Orginos, Andrew V. Pochinsky

46 Deep Learning Challenges Lattice Quantum corrected Finite Elements (QFE) S = 1 2 X h i,ji V ij l ij [ ie (i)j a a ij j j ji e (i)j a a i ] mv i i i + S W ilsont erm Lattice Dirac Fermions on a Simplicial Riemannian Manifold Richard C. Brower (Boston U.), George T. Fleming, Andrew D. Gasbarro (Yale U.), Timothy G. Raben, Chung-I Tan (Brown U.), Evan S. Weinberg (Boston U.). Oct 26, pp.

47 Solving CFT with Deep Learning? Back to the Bootstrap Neural Like Equations (i.e. Data: spectra + couplings to conformal blocks) Exact 2 and 3 correlators Only tree diagrams! partial waves exp: sum over conformal blocks CFT Bootstrap: OPE & factorization completely fixed the theory

48 Thanks Let s Make Multigrid Great Again!

49

50 Devils in the Details! Karl Hermann Amandus Schwarz (25 January November 1921) Boris Grigoryevich Galerkin (Russian: Бори с Григо рьевич Галёркин, surname more accurately romanized as Galyorkin; March 4 [O.S. February 20, 1871] 1871 July 12, 1945),

51 First Small Success: Applied Math/Physics Collaboration First Success: Applied Math/Physics Collaboration Collaboration NVIDIA Mike Clark Ron Babich Saul Cohen Saul Cohen Michael Cheng Oliver Witzel INT Seattle Saul Cohen MIT Andrew Pochinsky Chris Schroeder

52 What is the New Idea? Math Speak: A Schur/Schwarzian DD splitting of the vector space: - How do you spit the space into Fine vs Cause Space? - Classical MG vs Adaptive MG ker(p )! UV! But P P = 1 cc so Ker(P) = 0! fine space! S?! P! span(p)! S IR! span(p )! P! (see Front cover of Strang s Undergraduate MIT math text!) In Physic Speak: The Wilsonian Renormalization Groups: - How to find sepearate UV (short scales) from IR (long scales) - Conformal (Scale Inv) vs Non-perturbative RG