[Toward] Simulating Cosmological Reionization with Enzo

Transcription

1 [Toward] Simulating Cosmological Reionization with Enzo Texas Cosmology Network Meeting October 2009 Daniel R. Reynolds M.L. Norman, P. Paschos, G. So [UCSD] J.C. Hayes [LLNL] Sponsor: NSF Astronomy & Astrophysics October 29, 2009 (1)

2 Personal Introduction Assistant professor in Mathematics at SMU. Computational/applied math background; expertise in nonlinear and linear solvers for multi-scale problems in computational physics. This work focuses on collaborations with UCSD s Laboratory for Computational Astrophysics (M. Norman et al.). Goal is to develop high-accuracy, scalable solvers for coupled simulations of radiation, hydrodynamics and ionization. Efforts focused on developing solver modules for Enzo, an open-source code for cosmological hydro., dark-matter, gravity and chemical ionization (advertised by Kim Tran earlier). Target applications are mainly in cosmological reionization, though goal is to develop numerical methods that span to other regimes as well. October 29, 2009 (2)

3 Enzo-RT Goals I. Extend Enzo to include radiation transport & ionization, to enable: studies of self-regulated star formation, predictions on the epoch of cosmic reionization, and predictions on the observed properties of early galaxies. II. Accurately model stiff cosmological RT and chemical ionization processes. III. Integrate new solver in Enzo code so that coupling respects hydrodynamics approach (shocks), but tightly couples physics. IV. Enable very-large-scale simulations (O(10 6 ) processors). October 29, 2009 (3)

4 Coupled Matter-Radiation System We consider the coupled cosmological PDE system, 2 φ = 4πG a (ρ b + ρ dm ρ 0 ), t ρ b + 1 a v b ρ b = 1 a ρ b v b, t v b + 1 a (v b ) v b = ȧ a v b 1 aρ b p 1 a φ, t e + 1 a v b e = 2ȧ a e 1 aρ b (pv b ) 1 a v b φ + G Λ, t n i + 1 a (n iv b )= 3 ȧ a n i n i Γ ph i + α rec i,j n en j, t E ν + 1 a (E νv b )= (D E ν )+ν ȧ a νe ν +4πη ν ck ν E ν. Here, G(E,n i ) and Λ(E,n i ) are the heating and cooling rates. The E ν equation approximates the radiative flux as a function of the energy density gradient, F ν = D E ν, where D(E ν, E ν ) IR 3 3 is the flux limiter. [Bryan et al., Comp. Phys. Comm., 1995; R. et al., J. Comput. Phys., 2009] October 29, 2009 (4)

5 Enzo Operator-Split Numerics We solve this coupled system in an operator-split framework, solving one component of the system at a time within a time step: (i) Project dark matter particles onto mesh to generate ρ dm. (ii) Solve for the gravitational potential φ via FFT or MG methods. (iii) Advect dark matter particles via Particle-Mesh method. (iv) Explicitly evolve (ρ b, v b,e) and advect (E ν, n i ) with a high-order PPM method. (v) Implicitly evolve a stiffreaction-di ffusion PDE system that updates (E ν, e, n i ). October 29, 2009 (5)

6 Enzo Operator-Split Numerics We solve this coupled system in an operator-split framework, solving one component of the system at a time within a time step: (i) Project dark matter particles onto mesh to generate ρ dm. (ii) Solve for the gravitational potential φ via FFT or MG methods. (iii) Advect dark matter particles via Particle-Mesh method. (iv) Explicitly evolve (ρ b, v b,e) and advect (E ν, n i ) with a high-order PPM method. (v) Implicitly evolve a stiffreaction-di ffusion PDE system that updates (E ν, e, n i ). October 29, 2009 (5)

7 Gray FLD Radiation Transfer Equation Prior to investigating the multi-frequency case, we begin using a single, frequency-integrated ( grey ) radiation energy, E(x, t) = Z ν 0 E ν (x, t,ν ) dν = Ẽ(x, t) Z ν 0 χ E (ν) dν, where χ E : IR IR is an assumed radiation energy density spectrum. With this approximation, the integrated radiation energy equation becomes t E + 1 a (Ev b) = (D E) ȧ E +4πη cke, a where the new coupling terms are integrated over frequency: η = k = Z ν 0 Z ν 0 η ν dν «. Z «k ν E ν dν E ν dν. ν 0 October 29, 2009 (6)

8 Rad-Hydro Verification Results Radiation Energy Density (erg cm -3 ) Distance (cm) Smearing of a radiation front in a vacuum due to FLD approximation (convergence wrt x). T Distance From Shock (cm) Radiating shock front convergence wrt x. [Lowrie & Edwards, ShWav, 2008]. Thermal equilibration: energy conserv. error, Z E tot dx«. Z «E tot (0)dx, L H and H L equilibration, under various nonlinear and linear tolerances (ε, δ). [Turner & Stone, ApJ, 2001] ε δ L H H L 1e-7 1e-9 7.0e e-12 1e-10 1e-9 3.1e e-14 1e-4 1e-9 3.6e e-11 1e-7 1e e e-12 1e-7 1e-3 7.0e e-12 October 29, 2009 (7)

9 Isothermal Radiative Ionization Test Tests of isothermal ionization of a static, initially-neutral hydrogen region: L =6.6 kpc. Monochromatic emission: Ṅ γ = photons s 1, at hν = 13.6 ev. Fixed gas temperature, T = 10 4 K. Case-B recombination rate, α B = cm 3 s 1. Constant number density n = 10 3 cm 3. Analytical solution given by r I = r Sˆ1 e tα B n H, where " # 1/3 3Ṅγ r S = 4πα B n 2. H [Iliev et al., MNRAS, 2006] October 29, 2009 (8)

10 Isothermal Ionization Results $!") $( & 6;-:< &% & 6;-:< (' & 6;-:< $%) & 6;-:< =3=>?*7.=> 1234-, /!8,23*692:7*723 6 NW: I-front position history.!"+!"&!"*!"% -./, , ,4,:,'!!,;80 ',!')!#!!#)!(!, / +, 0!"(!"'!"%! 6!!"# $ $"# % %"# *+*,-. & &"# ' '"# NE: Coarse radiation field (N x = 16).!")!"$!"(!"#!"'!,!!"#!"$!"%!"& '!()!$!!$)!)! /, 0!, 0 1 /+,2 3,,4,)56)0!7,46*)8495*546 # ()$!!& ) $: & );-9< '"# &% & );-9< ' :' & );-9< $%= & );-9< &"# & %"# % $"# $!"#! )!!"# $ $"# % *+*,-. %"# & &"# ' SW: r I convergence vs x. SE: Fine radiation field (N x = 128).!"+!"&!"*!"%!")!"$!"(!"#!"' -./, , ,4,:,'!!,;80 ',!#!!(!!$!!)!!%!!*!!&!!+!!,!!"#!"$!"%!"& ' October 29, 2009 (9)

11 Isothermal Cosmological Ionization Test Repeat of previous test, but in a cosmologically expanding universe. Four tests: q 0 z i L i [kpc] ρ b,i [g cm 3 ] H 0 Ωm Ω Λ Ω b e e e e Analytical solution given by r I (t) =r S,i Z 1/3 a(t) λe τ(t) e τ(b) [1 2q 0 +2q 0 (1 + z i )/b] db! 1/2, 1 where τ(a) =λ [F (a) F (1)] ˆ6q 0(1 2 + z i ) 2 1,»» 1+z i 1+z 1/2 i F (a) = 2 4q 0 2q 0 1 2q 0 +2q 0, a a λ = α B n H,i /H 0 /(1 + z i ). [Shapiro & Giroux, ApJ, 1987] October 29, 2009 (10)

12 $!"+!"*!") Cosmological Ionization Results : NW: I-front radii vs scaled z. ##,-#!!% #! + * >99/9-7?-9 7 2:569 = 2:5-@=-9<A=B7C:D-E!-F-!"'D-4!-F-& #( % -G<=B %$ % -G<=B (& % -G<=B #$* % -G<=B !"(!"#!"'!"&!"%!"$ NE: I-front error vs scaled z. 29 :9;<!9569 = ) ( ' & % $! :!!"# $ $"# % %"# &!,-./0$12340$ :;6<3=7>80?& 3 6 3!"( 8!"' SW: I-front radii for z i =4. # -!!"#!"$!"%!"&!"'!"(!")!./012#34562# Weak Scaling Results for Cosmological Ionization Test 3000!"& !!"#!!"#!"$!"%!"&!"'!"(!")!*+,-.#/012.#/ SE: Weak CPU scaling [Kraken] N src N CPU (O(N log N)) time (s) procs October 29, 2009 (11)

13 Hydrodynamic Radiative Ionization Test Re-do first test in a dynamic medium (L = 15 kpc); combines radiation, hydrodynamics and hydrogen ionization: Emission source has a T = 10 5 K blackbody spectrum. Initial temperature set to T = 10 2 K. Front transitions from R-type to D-type as it reaches Strömgren radius. Eventually stalls at r f = r S 2Ti T e 2/3, where Ti and T e are the temperatures behind and ahead of the i-front. R-type and D-type propgation have different analytical solutions, r R I = r S i» 1/3 h1 e tα B(T i )n H, r D I = r S 1+ 7c 4/7 st, 4r S with true solution somewhere between the two. [Whalen & Norman, ApJS, 2006; Iliev et al., MNRAS, 2009] October 29, 2009 (12)

14 Hydrodynamic Radiative Ionization Results %"# % $"# /012+*3+1,+4514-!6*01(47085(501 $9 & 4:+8; &% & 4:+8; 9' & 4:+8; $%< & 4:+8; 4 NW: I-front position history Temperature profile, t = 175 Myr * - )*. log(t) 3 $!"#! 4!!"# $ $"# % %"# ()( *+, & &"# ' '"# NE: T profile at 200 Myr r/l box -*.!*. / -)*0 1**2*3453.!6*25(37284(425!"!# 3 $9 & 3:+8;!"!'# &% & 3:+8;!"!' 9' & 3:+8;!"!&#!"!&!"!%#!"!%!"!$#!"!$!"!!#! 3!!"# $ $"# % %"# ()( *+, & &"# ' '"# SW: r I convergence wrt x. SE: T convergence wrt t /+ #!!# #!!$ #!!% #!!& #!!' #!!* #!!) #!!( 7 '1!& $1!& #1!& '1!' $1!' #1!' 819:1+34;+171++/+7:+/<521=747>7#)'7?@+!!"#!"$!"%!"&!"' +,-./0 7 October 29, 2009 (13)

15 The Need for Multi-Frequency However, in comparison with other RT codes, we get peculiar results: Right: n profiles at 250 Myr (cyan is Enzo) [1]. Below: Enzo n profiles results at 200, 500 Myr. Lower right: Zeus-MP results (solid is MF, dashed is monochromatic). '!!# 012,3)453/67894:)-;7<3 #!!4=9) >!!4=9) 4 / '!!( n (cm -3 ) 10-3 '!!$ 4!!"#!"$!"%!"& ' '"# )*+, r/l box October 29, 2009 (14)

16 Summary of Current Results We ve achieved a second-order accurate (space and time) coupled solver for cosmological radiation, hydrodynamics, self gravity and chemical ionization. Captures shock fronts due to trusted PPM hydrodynamics approach. Accurately solves couplings between radiation, ionization and gas energy, due to implicit formulation and coupled solvers. Highly scalable, with current tests using > 4000 processors, thanks to reliance on optimal multigrid methods. However, the grey radiation approximation has its shortcomings: Single radiation field allows full absorption by hydrogen, even though higher-frequency radiation should pass through. Leads to increased absorption in optically-thick regions, with no pre-heating ahead of I-fronts. October 29, 2009 (15)

17 Continuing Work Enhance stability of chemistry computations at large t Fastest dynamics occur in ionization, ostensibly allowing complete ionization of a volume in a single step [O(1) O(10 5 )]. Time inaccuracy can over-shoot to give negative densities. Extend radiation approximation to multi-frequency case, E E ν New physical couplings and interpolation of ν-space, based offof [Ricotti, Gnedin & Shull, ApJ, 2002] Large-scale solvers for coupled reaction-diffusion systems Extend implicit solver software to AMR grids Regular-grid geometric multigrid AMG or FAC methods October 29, 2009 (16)