Contributors: T. Dannert (1), F. Jenko (1),F. Merz (1), D. Told (1), X. Lapillonne (2), S. Brunner (2), and others T. Görler (1) The gyrokinetic turbulence code GENE - Numerics and applications (1) Max-Planck-Institut für Plasmaphysik, Garching (2) CRPP/EPFL, Lausanne 3 rd summer school of the Large Scale Initiative FUSION September 26 th -30 th, 2011 Paris
What to expect from this lecture? A brief motivation for and introduction to gyrokinetic theory A discussion of a massively parallelized implementation - the plasma microturbulence code GENE Examples of simulation results Summary & Outlook
Plasma turbulence modeling and gyrokinetic theory in brief
Plasma microturbulence Energy confinement time set by anomalous heat and particle transport one of the key physics problems Plasma density Plasma temperature Ion temperature (kev) Commonly attributed to plasma microturbulence ρ Microinstabilities driven by strong temperature and density background gradients Quasistationary state far from thermodynamic equilibrium
Plasma turbulence: Kinetic theory! Dilute and/or hot plasmas are almost collisionless. Thus, if kinetic effects (finite Larmor radius, Landau damping, magnetic trapping etc.) play a role, magnetohydrodynamics is not applicable, and one has to use a kinetic description! Vlasov-Maxwell equations Removing the fast gyromotion leads to a dramatic speed-up Charged rings as quasiparticles; gyrocenter coordinates; keep kinetic effects Brizard & Hahm, Rev. Mod. Phys. 79, 421 (2007)
The gyrokinetic ordering The gyrokinetic model is a Vlasov-Maxwell on which the GK ordering is imposed Slow time variation as compared to the gyromotion time scale Spatial equilibrium scale much larger than the Larmor radius Strong anisotropy, i.e. only perpendicular gradients of the fluctuating quantities can be large Small amplitude perturbations, i.e. energy of the perturbation much smaller than the thermal energy:
Theoretical framework: Gyrokinetic theory Gyrokinetic Vlasov equation Poisson equation with gyrocenter position Ampère s law parallel velocity and magnetic moment Nonlinear, 5-dimensional, partial integro-differential system of equations to be solved!
Major theoretical speedups G.Hammett relative to original Vlasov/pre-Maxwell system on a naïve grid, for ITER * = /a ~ 1/1000 Nonlinear gyrokinetic equations eliminate plasma frequency: pe / i ~ m i /m e x10 3 eliminate Debye length scale: ( i / De ) 3 ~ (m i /m e ) 3/2 x10 5 average over fast ion gyration: i / ~ 1/ * x10 3 Field-aligned coordinates adapt to elongated structure of turbulent eddies: / ~ 1/ * x10 3 Reduced simulation volume reduce toroidal mode numbers (i.e., 1/15 of toroidal direction) L r ~ a/6 ~ 160 r ~ 10 correlation lengths x15 x6 Total speedup x10 16 For comparison: Massively parallel computers (1984-2009) x10 7
Status quo in gyrokinetic simulation over the last decade or so, GK has emerged as the standard approach to plasma turbulence a variety of nonlinear GK codes is being used and (further) developed these codes differ with respect to their numerics, physics, parallel scalability, and public availability GENE GYSELA ORB5 ELMFIRE... Europe US GYRO GTC GEM GS2... GKV G5D... Japan
The gyrokinetic Vlasov code GENE
The gyrokinetic code GENE (gene.rzg.mpg.de) GENE is a physically comprehensive Vlasov code: allows for kinetic electrons & electromagnetic fluctuations, collisions, and external ExB shear flows is coupled to various MHD codes and the transport code TRINITY can be used as initial value or eigenvalue solver supports local (flux-tube) and global (full-torus), gradient- and fluxdriven simulations GENE is well benchmarked and hyperscalable Strong scaling on BG/P 11
Concepts used within GENE for speed-up field-aligned (Clebsch-type) coordinate system to exploit the high anisotropy of plasma turbulence; parallel (z) derivatives can be taken small compared to perpendicular (x,y) ones(!) f-splitting: Apply same approach as in the derivation of the GKE and split the distribution function f = F 0 + f where F 0 : stationary background, here: local Maxwellian f: fluctuating part with f/f 0 << 1 Lowest-order nonlinearity kept next order can be switched on for testing (electrostatic version) 12
Numerical methods time scheme I Method of Lines: turn PDEs into ODEs by discretizing the spatial derivative first solve for the continuous time coordinate Time Solver: Full eigenvalue spectrum of the GK lin. operator Linear system: Iterative eigenvalue solver based on PETSc/SLEPc/Scalapack lib s solve for largest abs/re/im eigenvalues gain insights in linear stability/physics Explicit Runge-Kutta (ERK) schemes 13
Numerical methods time scheme II Time Solver: (Full) Nonlinear system: Linear stability regions for low-order RK-schemes Several ERK-methods, e.g. 4 th order Optimum linear time step can be precomputed using iterative EV-Solver Adaptive CFL time step adaption for nonlinearity 14
Phase space discretization GENE is a Eulerian code; thus solving the 5D (δf-splitted) distribution function on a fixed grid in (x, v, ) trapped/passing boundary radial direction x: toroidal direction y: parallel direction z: v -velocity space: -velocity space: equidistant grid (either configuration or Fourier space) equidistant grid in Fourier space equidistant grid points equidistant grid Gauss-Legendre or Gauss-Laguerre knots 15
Phase space discretization finite differences 4 th order finite differences are basic choice however, the more elaborate Arakawa-scheme is employed, if possible [A. Arakawa, JCP 135, 103 (1997), reprint] Exception: spectral methods are employed in the y direction - always x direction - depends on the type of operation (local/global) 16
Local vs. global GENE Local: in the radial direction Simulation domain small compared to machine size; thus, constant temperatures/densities and fixed gradients a Periodic boundary conditions; allows application of spectral methods Local sim. domain ρ*=ρ s /a << 1 Global sim. domain Global: adding nonlocal features in the radial direction Consider full temperature & density profiles; radially varying metric Dirichlet or v. Neumann boundary conditions Heat sources & sinks 17
Local vs. global GENE numerical point of view Local approach: Spectral methods derivatives: Gyroaverage & field operators can be given analytically: T(x) / T 0 Global approach: derivatives: finite difference scheme, typically 4 th order Use interpolation schemes for gyroaverage & field operators with gyromatrix
Gyro-averaging procedure in more detail discretize gyro-angle integration coordinate transform interpolation between grid-cells required (!) here: 1-dimensional problem (y remains in Fourier space) use finite-elements which allows easy extraction of gyroaveraged quantities at original grid points (~ Hermite polynomial interpol.) Gyromatrix is constructed at initialization only:
Boundary conditions - overview radial direction x: local: periodic global: Dirichlet v. Neumann T T r toroidal direction y: periodic r [Beer, PoP 95] parallel direction z: quasi-periodic v -velocity space: Dirichlet b.c. magnetic shear tilts the simulation box phase factor -velocity space: Not required (if collisions are neglected)
Sources/sink models in GENE Full domain: Radially dependent heat source/sink in whole domain (density and parallel momentum are unaffected) Similar to ORB5 [McMillan et al., PoP `08] Background temperature gradient is approx. maintained while avoiding direct influence on microturbulence (similar operator exists for density) Localized sources: Krook operator, boundaries, to damp fluctuations close to the Heat source model (cmp. GYSELA, [Grandgirard et al., PPCF 07]) with Example of a heat source profile: and floating boundary conditions 21
Geometry arbitrary flux-surface shapes can be considered as long as the metric is provided to compute, e.g., standard choice (implemented): circular concentric flux surfaces others can be read via interfaces to TRACER/GIST (field line tracer) and the equilibrium code CHEASE
Code parallelization and optimization MPI (message passing interface) parallelization available in all phase space directions + species; however, difficult to estimate interference, especially on large numbers of processors (>1k) automatic comparison at code initialization For better utilization of different architectures (different cache sizes!), strip mining techniques and message block adaption FFTW-like performance optimization during initialization
Applications
Why global? Cover a larger radial domain (instead of using several flux tubes) Check validity of local simulations: When do meso-/large scale events, i.e. avalanches or turbulence spreading, occur? Do they affect the transport scaling? machine-size events: Bohm scaling Gyroradius scale turbulence: Gyro-Bohm scaling Re-assess earlier results by [Z. Lin et al., PRL, 2002] and [Candy et al., PoP 2004] Allow for flux-driven simulations 25
Local limit test (ρ* 0) with L x /a =const. for 2 profile types local value Here: CBC-like parameters with kinetic electrons at k y ρ s ~ 0.3 The global results do approach the local limit
Local limit test (ρ* 0) with L x /a =const. for 2 profile types local value The profile shape (linear driving region) matters! Data points align with
Nonlinear investigation of finite size effects Candy et al., PoP 11 (2004) GENE-local using the circular, concentric equilibrium GENE-global ORB5 GENE-local using the s-α equilibrium McMillan et al., PRL 105, 155001 (2010) ORB5 (Lagrangian) and GENE (Eulerian) agree if the same geometry model is used long lasting controversy probably resolved Both, GENE and ORB5 converge towards the local limit Deviations (global/local) < 10% at ρ* < 1/300 28
Finite system size: Profile shape matters logarithmic temperature gradient Local (flux-tube) limit PRL 105, 155001 (2010) Both codes also show that it is the parameter which really matters this should be kept in mind when dealing, e.g. with Internal Transport Barriers Scaling cannot be explained with profile shearing (only weak dependence) Turbulence spreading, avalanches? 29
Can avalanches break gyro-bohm scaling? Global flux-driven simulations of ITG-ae turbulence with GENE (for ρ*=1/140) time flux-surface averaged radial heat flux associated ExB shearing rate (radial shear layers) radius radius avalanches are mesoscale ; radial extent ~20-40 ρ i their propagation speed is found to be ~ρ*v ti propagation direction is correlated with sign(ω E ) importance of low-frequency zonal flows & mean flows Same phenomenology as, e.g., McMillan (PoP 2009) 30
Heat flux for global, gradient-driven ITG-ae Radial extent and propagation velocities do not depend much on ρ* Δt = 100 R0/cs ρ*=1/140 ρ*=1/560 ρ*=1/280 Δt = 100 R0/cs ρ*=1/1000 (ITER-like) 31
Interesting insights from local simulations ITG turbulence (adiabatic electrons) ITG turbulence (kinetic electrons) Δt = 100 R 0 /c s Avalanches are not inherently nonlocal ETG turbulence (adiabatic ions) Avalanches are less pronounced in more complete model TEM turbulence Δt = 100 R 0 /c s Avalanches are not to be identified with streamers Avalanches tend to be absent
Conclusions
Summary & Outlook The nonlinear gyrokinetic plasma turbulence code GENE has been introduced and its features and recent extension to a nonlocal code have been discussed; Examples for applications of this code suite have been presented: Transition from nonlocal to local turbulence (ρ* 0) has been revisited cooperatively via Lagrangian & Eulerian codes; linear driving region important Heat flux avalanches seem to be mesoscale (ρ i -related) phenomena and are found not to break the gyro-bohm scaling Next applications: Take advantage of the comprehensive physics model (electromagnetic effects, collisions, etc.) and address specific experimental discharges, e.g. eitb in TCV consider non-axisymmetric stellarator equilibria keeping finite-size effects References: http://gene.rzg.mpg.de ( publications) [Görler et al., JCP 230, 7053 (2011)]