Particle In Cell Beam Dynamics Simulations With a Wavelet based Poisson Solver

Transcription

1 Particle In Cell Beam Dynamics Simulations With a Wavelet based Poisson Solver Balša Terzić Beam Physics and Astrophysics Group, Department of Physics Northern Illinois University in collaboration with Ilya Pogorelov (Lawrence Berkeley Lab) Courtlandt Bohn, Daniel Mihalcea, Benjamin Sprague (NIU) University of New Mexico, Department of Mathematics and Statistics November 2, 2006

2 Motivation Gaining insight into the dynamics of multi particle systems (galaxies, charged particle beams, plasma...) heavily relies on N body simulations It is important for N body codes to: account for multiscale dynamics minimize numerical noise due to Nsimulation<< Nphysical be as efficient as possible, without compromising accuracy for some applications: have a compact representation of system's history We present a wavelet based Poisson equation solver which, as a part of an Particle In Cell (PIC) N body code, addresses all of these requirements

3 Outline of the Talk PIC algorithm for N body simulations origin and profile of noise Brief introduction to wavelets and their properties denoising and compression Wavelet based Poisson equation solver Applications to real machines and comparison to existing codes Summary and discussion of further work

4 Algorithms for N body Simulations Main algorithms for N body simulations: direct summation (DS): CPU cost scales as N2 tree: DS nearby and statistical treatment farther away Particle In Cell (PIC): particles binned in cells (grid) system at t=t0 binning particle distribution N particles in box Newton's equations advance particles by t system at t=t0+ t on Nx Ny Nz grid interpolation, F for each particle F = solve the Poisson equation = on Nx Ny Nz grid

5 Noise in PIC Simulations Any numerical simulation will have numerical noise Origin of numerical noise in PIC simulations: graininess of the distribution function: Nsimulation<< Nphysical discreteness of the computational domain: and specified on a finite, discrete grid Each macroparticle is deposited onto a finite grid by either: Nearest Grid Point (NGP) dep. scheme Cloud In Cell (CIC) dep. scheme x location of the macroparticle location of gridpoints

6 Noise in PIC Simulations For NGP, at each gridpoint, particle dist. is Poissonian: n j P= n! 1 n nj e nj is the expected number in jth cell; n integer For CIC, at each gridpoint, particle dist. is contracted Poissonian: D / 2 1 n an P= n! an j e where a= 2 /3 j Measure of error (noise) in depositing macroparticles onto a grid: N grid 2= N grid 1 Var qi i=1 2 NGP = 2 Qtotal N N grid 2 2 CIC = 2 a Qtotal N N grid where qj = q0nj, q0= Qtotal/N, Qtotal is the total charge and Ngrid total the number of gridpoints This error/noise estimate is crucial for optimal wavelet denoising IDEA: Solve the Poisson equation in such a way so as to minimize numerical noise USE WAVELETS

7 Wavelets What are wavelets? orthogonal basis of functions composed of scaled and translated versions of the mother wavelet (x) and the scaling function (x) f(x) =s00 00 (x) + d00 00 (x) + i k dik ik (x) ik(x) = 2k/2 (2kx i) = j gik 2i+jk+1 (x) ik(x) = 2k/2 (2kx i) = j hik 2i+jk+1 (x) family of perfect low and high pass filters hi and gi (enable DWT) Why wavelets? simultaneous time (space) and frequency localization (FFT only frequency) data compression: computational speed up (FBI uses it for storing fingerprints) signal denoising: natural setting in which noise can be partially removed denoised simulation simulation with more macroparticles

8 One Dimensional Wavelets (a) Haar wavelet, (b) Morlet wavelet, (c) Daubechies wavelet. Two Dimensional Daubechies least symmetric N = 10 two dimensional wavelet.

9 Wavelet Denoising and Compression In wavelet space: signal noise few large wavelet coefficients ci,j many small wavelet coefficients ci,j Anscombe transformation Poissonian noise Denoising by wavelet thresholding: Gaussian noise if ci,j < T, set to ci,j =0 (choose threshold T carefully!) A great deal of study has been devoted to estimating optimal T or T = 2 log N grid T =2 log N grid where was estimated earlier w/ Anscombe transformation w/out Anscombe transformation

10 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc ANALYTICAL Nppc: avg. # of particles per cell Nppc= 3 SNR = 2.02 Nppc= 205 SNR = 16.89

11 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc Nppc: avg. # of particles per cell 2D superimposed Gaussians on grid ANALYTICAL Nppc= 3 SNR = 2.02 Nppc= 205 SNR = 16.89

12 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc Nppc: avg. # of particles per cell 2D superimposed Gaussians on grid ANALYTICAL Nppc= 3 SNR = 2.02 Nppc= 205 SNR = 16.89

13 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc Nppc: avg. # of particles per cell 2D superimposed Gaussians on grid ANALYTICAL Nppc= 3 SNR = 2.02 Nppc= 205 SNR = 16.89

14 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc Nppc: avg. # of particles per cell 2D superimposed Gaussians on grid DENOISED ANALYTICAL Nppc= 3 SNR = 2.02 Nppc= 205 SNR = denoising by wavelet thresholding: if ci,j < T, set to 0 Nppc= 3 SNR = 16.83

15 Example: Wavelet Denoising and Compression Whenever the discrete signal is analytically known, one can compute Signal to Noise Ratio (SNR) which measures its quality SNR ~ Nppc Nppc: avg. # of particles per cell 2D superimposed Gaussians on grid COMPACT: only 0.12% of coeffs WAVELET THRESHOLDING ANALYTICAL Nppc= 3 SNR = 2.02 Nppc= 205 SNR = DENOISED Nppc= 3 SNR = denoising by wavelet thresholding: if ci,j < T, set to 0 Increase in SNR by b equivalent to running simulation with b2 more particles! Particle distribution is compactly stored in wavelet space (in the example b=8.3 equivalent to 68 times more particles; 79/65536 wavelet coeffs retained)

16 Wavelet based Poisson Solver Poisson equation in physical space u=f BCs: using Green's functions Discretize Poisson equation on the Nx Ny Nz grid LU=F transform discretized Poisson equation to wavelet space WT Lw Uw = Fw ubnd = g PRECONDITIONED Conjugate Gradient (PCG) in wavelet space A X B 2 ε2 B 2 iwt solution U on the Nx Ny Nz grid wavelet threshold source Fw, operator Lw AX=B (PLwP) P 1Uw = PFw A compressed operator PLwP PRECONDITION Laplacian Lw B denoised distribution PFw with diagonal preconditioner X denoised solution P 1Uw k (Lw) ~ O(Nx2) k (PLwP) ~ O(Nx) P

17 Preconditioned Conjugate Gradient (PCG) i convergence rate depends on k u u i 2 k 1 u 2 preconditioning (diagonal in wavelet space): k ~ O(Nx2) k ~ O(Nx ) k 1 good initial approximation: solution at previous time step

18 Preconditioned Conjugate Gradient (PCG) i convergence rate depends on k u u i 2 k 1 u 2 preconditioning (diagonal in wavelet space): k ~ O(Nx2) k ~ O(Nx ) k 1 good initial approximation: solution at previous time step typical realistic beam simulation no preconditioning Ui=0 initial guess average over step run 75.2

19 Preconditioned Conjugate Gradient (PCG) i convergence rate depends on k u u i 2 k 1 u 2 preconditioning (diagonal in wavelet space): k ~ O(Nx2) k ~ O(Nx ) k 1 good initial approximation: solution at previous time step typical realistic beam simulation average over step run no preconditioning Ui=0 initial guess 75.2 preconditioned Ui=0 initial guess 60.7

20 Preconditioned Conjugate Gradient (PCG) i convergence rate depends on k u u i 2 k 1 u 2 preconditioning (diagonal in wavelet space): k ~ O(Nx2) k ~ O(Nx ) k 1 good initial approximation: solution at previous time step typical realistic beam simulation average over step run no preconditioning Ui=0 initial guess 75.2 preconditioned Ui=0 initial guess 60.7 no preconditioning Ui=Ui 1 initial guess 4.8

21 Preconditioned Conjugate Gradient (PCG) i convergence rate depends on k u u i 2 k 1 u 2 preconditioning (diagonal in wavelet space): k ~ O(Nx2) k ~ O(Nx ) k 1 good initial approximation: solution at previous time step typical realistic beam simulation average over step run no preconditioning Ui=0 initial guess 75.2 preconditioned Ui=0 initial guess 60.7 no preconditioning Ui=Ui 1 initial guess 4.8 preconditioned Ui=Ui 1 initial guess 2.4

22 Applications Our goal: develop wavelet based Poisson solver which can easily be integrated into existing PIC codes First: test the PCG as a stand alone Poisson solver on (idealized) examples from beam dynamics (3D 'fuzzy cigar' beam charge distribution) galactic dynamics (mass density of an elliptical galaxy as idealized by the 3D Plummer sphere)

23 3D 'fuzzy cigar' on a 32x32x32 grid BCs: grounded rectangular pipe (V=0 on the sides), open in z direction (Green's function solution)

24 3D Plummer sphere on a 32x32x32 grid BCs: open in all directions (analytically specified)

25 Applications Second: insert the PCG Poisson solver into an existing PIC code (IMPACT T) and run realistic charged particle beam simulations compare (conventional FFT based) IMPACT T Vs. IMPACT T with PCG: rms properties level of detail computational speed

26 IMPACT T Vs. IMPACT T with PCG: rms properties Fermilab/NICADD photoinjector ( grid) 1 nc charge rms beam radius rms normalized transverse emittance rms bunch length rms normalized longitudinal emittance

27 IMPACT T Vs. IMPACT T with PCG: level of detail IMPACT T transverse charge distribution for the Fermilab/NICADD photoinjector simulation very non axisymmetric beam grid, N=200,000 very good agreement in detail IMPACT T w/ PCG initial distribution z = 1.0 m z = 2.0 m z = 4.0 m

28 IMPACT T Vs. IMPACT T with PCG: level of detail IMPACT T transverse charge distribution for the Fermilab/NICADD photoinjector simulation 5 beamlets (masked photocathode) grid, N=200,000 very good agreement in detail IMPACT T w/ PCG initial distribution z = 2.0 m z = 4.0 m z = 6.0 m

29 IMPACT T Vs. IMPACT T with PCG: computational speed IMPACT T w/ PCG ~ 10% faster than the conventional serial IMPACT T Areas of further computational optimization: code parallelization (almost completed) further optimizing boundary conditions computations finding new ways to exploit sparseness of operators and data sets finding a more effective preconditioner for PCG

30 Data Compression: Clear Advantage of PCG IMPACT T w/ PCG provides excellent compression of data sets and operators in wavelet space Fermilab/NICADD photoinjector: Real Simulations particle distribution Laplacian operator grid, N=125000, Nppc=4.58 ~ 3.5% coefficients retained on average grid, N= , Nppc=4.58 ~ 1.75% coefficients retained on average compact storage of beam's distribution history needed for CSR simulations

31 Conclusions Presented an iterative wavelet based Poisson solver (PCG) wavelet compression and denoising achieves computational speedup preconditioning and sparsity of operators in wavelet space reduce CPU load integrated PCG into a PIC code (IMPACT T) for beam dynamics simulations Original contribution: first wavelet based solver for 3D Poisson equation with Dirichlet BCs first application of wavelets to 3D simulation of beams better understanding of noise in PIC simulations and its removal modeled real machines: Fermilab/NICADD photoinjector Future plans: use PCG in N body simulations of self gravitating systems (galaxies, clusters...) apply wavelet formalism to other PDEs: Fokker Planck, Vlasov Poisson, Maxwell explore use of compact representation of charge distribution in CSR simulations

32 On line Resources This paper: Our group's (BPAG) website:

33 Auxiliary Slides

34 Computing Boundary Conditions Green's function corresponding to a grounded rectangular pipe V=0 y computational grid EIGENFUNCTIONS pipe walls sin l x sin m y V=0 V=0 EIGENVALUES l =l / a V=0 lm z = 2 lm z z2 4 ab a x b 0 0 x, y, z sin l x sin m y dx dy 2lm lm z = lm z Nx Ny m=m / b x, y, z = l =1 m=1 lm z sin l x sin m y 2 pipe walls 2 2 l m = l m

35 Preconditioner in Wavelet Space

36 Wavelets (x) What are wavelets? 4th order Daubechies orthogonal basis of functions composed of scaled and translated versions of the mother wavelet (x) and the scaling function (x) f(x) =s00 00 (x) + d00 00 (x) + dik ik (x) ik(x) = 2k/2 (2kx i) ik(x) = 2k/2 (2kx i) family of perfect low and high pass filters hi and gi (enable DWT) Why wavelets? simultaneous time (space) and frequency localization (FFT only frequency) data compression: computational speed up signal denoising: natural setting in which noise can be partially removed denoised simulation simulation with more macroparticles 2(x)

37 Wavelet Decomposition The continuous wavelet transform of a function f (t) is s, = f t s, t dt s, t = 1 s t s t mother wavelet with scale and translation dimensions s and respectively

38 How Do Wavelets Work? Wavelet analysis (wavelet transform): S signal A approximation D detail Approximation apply low pass filter to Signal and down sample Detail apply high pass filter to Signal and down sample Wavelet synthesis (inverse wavelet transform): up sampling & filtering Complexity: 4MN, M the size of the wavelet, N number of cells Recall: FFT complexity 4N log2n

39 2D EXAMPLE: MICROBUNCHED DENSITY DISTRIBUTION (with U=0 starting guess) BCs: open in all directions (analytically specified)

40 IMPACT T Vs. IMPACT T with PCG: rms properties Fermilab/NICADD photoinjector ( grid) 1 nc charge bunch compressor at z=5.65m from cathode rms beam radius rms normalized transverse emittance rms bunch length rms normalized longitudinal emittance

41 IMPACT T Vs. IMPACT T with PCG: level of detail Impact IMPACT T IMPACT Tresidue w/ PCG Impact with CPCG transverse charge distribution for the AES/JLab low charge photoinjector simulation very non axisymmetric beam grid, N= very good agreement in detail initial distribution z = 0.5 m z = 1.9 m z = 2.5 m