Higher Order Statistics in Cosmology

Transcription

1 Higher Order Statistics in Cosmology and István Szapudi Institute for Astronomy University of Hawaii Statistical Frontiers of Astrophysics, Kashiwa, U. of Tokyo

2 Outline Introduction 1 Introduction Definitions 2

3 Definitions Random Fields Definition A random field is a spatial field with an associated probability measure: P(A)DA. Gaussian and non-gaussian random fields are abundant in Cosmology. CMB, Dark Matter Distribution, Galaxy Distribution, etc. Astronomers measure particular realization of a random field (ergodicity helps but we cannot avoid cosmic errors ) Clusters of galaxies correspond to a biased version of the underlying galaxy (or Dark Matter) random field.

4 Non-Gaussianity Introduction Definitions Higher Order Statistics Studies and characterizes the spatial distribution random fields beyond Gaussianity. Gaussian: statistics up to second order. Non-Gaussian: connected moments are non-zero beyond second order. Gaussian/Non-Gaussian: correlated/random phases. LSS is highly non-gaussian, as evidenced by clusters, superclusters, filaments.

5 Definitions Gaussian vs. Non-Gaussian distributions These two have the same two-point correlation function or P(k) Non-Gaussian information is essential for LSS. The clusters are pretty much gone!

6 Definitions Introduction Definitions The ensemble average A corresponds to a functional integral over the probability measure. Physical meaning: average over independent realizations. Ergodicity: (we hope) ensemble average can be replaced with spatial averaging. Joint Moments F (N) (x 1,..., x N ) = T (x 1 ),..., T (x N )

7 Definitions Connected Moments These are the most frequently used spatial statistics Typically we use fluctuation fields δ = T / T 1 Connected moments are defined recursively δ 1,..., δ N c = δ 1,..., δ N P δ δ i c... δ j... δ k c... With these the N-point correlation functions are ξ (N) (1,..., N) = δ 1,..., δ N c

8 Basic Objects N-point correlation functions. Introduction Definitions Special Cases Two-point functions δ 1 δ 2 Three-point functions δ 1 δ 2 δ 3 Cumulants δr N c = S N δr 2 N 1 Cumulant Correlators δ1 NδM 2 c Conditional Cumulants δ(0)δr N c Non-linear transformations f (δ 1 )f (δ 2 ) In the above δ R stands for the fluctuation field smoothed on scale R (different R s could be used for each δ s). Host of alternative statistics exist: e.g. Minkowski functions, void probability, minimal spanning trees, phase correlations, etc.

9 Complexities Combinatorial explosion of terms Introduction Definitions N-point quantities have a large configuration space: measurement, visualization, and interpretation become complex. CPU intensive measurement: M N scaling for N-point statistics of M objects. At the moment there is still only a small overlap between the reliable region of the theory with the measurements (this is changing). Redshift distortions/projection effects, bias, errors, and non-linearities. As we will see, a close look at symmetries helps with these issues.

10 Definitions Cosmic Errors in Galaxy Surveys Classification of Cosmic Errors (Szapudi & Colombi 1996) Discreteness errors: from finite sampling (with galaxies) of the underlying continous (Dark Matter) field Finite volume effects: due to the density fluctuations on the scale of the survey Edge effects: due to the particular geometry of the survey The latter two affects the CMB as well (cosmic variance).

11 Cosmic Errors Introduction Definitions

12 Definitions Edge-effects as Uneven Weighting Edge effects are due to a weighting problem objects near the edge will receive different weights from objects near the middle Geometry directly influences optimal weighting E.g. quadratic estimator: optimal edge correction for Gaussian fields P(k) < x C 1 PC 1 x > In a full survey (full sky, torus, etc) C is diagonal in Fourier space. Each pair is weighted by inverse variance.

13 Realistic Survey Boundaries A real bad case Definitions

14 Definitions Edge-effects as Symmetry breaking Realistic survey geometry breaks the underlying symmetries of space (translation and rotation) Edge effects are really a manifestation of this symmetry breaking E.g. in the quadratic estimator, the Fourier modes will be no longer orthogonal. C 1 for optimal weighting has to be calculated for the individual survey. It will depend both on geometry and theory (edge effects are separate only approximately.

15 Edge Correction Introduction Definitions Edge effects can cause bias and variance Special significance for BAO/DE detection: resolution on large scales is more important than ever. Edge corrections (for bias) are simplest (diagonal) in real (or pixel) space (division) In Fourier space edge correction corresponds to inverting a matrix (deconvolution) Since the theory is simplest in Fourier space we need machinery to shift between them easily.

16 in General Invariance under a (Lie) group G Definitions Expansion according to irreducible representations ξ(gx, gy) = ξ(x, y) δ(x) = ai α Ψ α i Irreducible reps diagonalize corr. matrix ai α a β j = δ ij δ αβ dxξ(0, x) s Ψα s (0)Ψ α s (x) 1 f α dg a l1 m 1 al 2 m 2 = δ l1 l 2 δ m1 m 2 C l1

17 Definitions : Translation Invariance The Dirac δ s are expressing the symmetry Fourier Transform δ(k 1 )δ(k 2 ) = (2π) D δ D (k 1 + k 2 )P(k 1 ) δ(k 1 )δ(k 2 )δ(k 3 ) = (2π) D δ D (k 1 + k 2 + k 3 )B(k 1, k 2, k 3 ) Generalized Wiener-Khinchin Theorem ξ(x 1, x 2 ) = d D k (2π) D P(k)e ik(x 1 x 2 ) d D k i ξ(x 1, x 2, x 3 ) = Π 3 i=1 B(k (2π) D 1, k 2, k 3 )e i(p k j x j ) δ D ( k j ).

18 Expansions for the sphere Statistics under SO(3) invariance Definitions Angular Power Spectrum ξ(θ) = 2l + 1 4π C lp l (cos θ) Legendre polynomials P l bipolar spherical harmonics L = 0 Bispectrum: Tripolar Expansion S l1 l 2 l 3 (ˆx 1, ˆx 2, ˆx 3 ) P m 1,m 2,m l1 l 2 l 3 m 1 m 2 m 3 «Y l1 m 1 (ˆx 1 )Y l2 m 2 (ˆx 2 )Y l3 m 3 (ˆx) ξ 3 (ˆx 1, ˆx 2, ˆx 3 ) = P l 1 l 2 l 3 B l1,l 2,l 3 S l1 l 2 l 3 (ˆx 1, ˆx 2, ˆx 3 )

19 Definitions : Rotation Invariance These further simplify things... The two-point function depends on the distance of the two variables. The integral is 1D only ξ(r) ξ(r) = kdk 2π P(k)J 0(kr) 2D, = k 2 dk 2π 2 P(k)j 0 (kr) 3D Can we use the same idea for the 3 point statistics? They depend on a triangle size and shape.

20 Definitions Rotation Invariance: 3D SO(3) If we parametrize the triangle by two unit vectors, and two sizes, bipolar spherical expansion can capture the rotation invariance. In the bipolar expansion only L = 0 (scalar) can happen because of SO(3) invariance: Legendre polynomials. Multipole Expansion For 3-point Statistics B(k 1, k 2, θ) = l B l (k 1, k 2 )P l (cos θ) 2l + 1 4π, Note that tripolar expansion is also possible and more convenient in some cases.

21 Definitions Consequence of Rotation Invariance in 3D The above expansion simplifies the connection between Fourier and real space 3-point statistics. In particular the 6 dimensional (intractable integral) becomes 2 dimensional. Double Hankel Transform k 2 2 k ξl 3 2 (r 1, r 2 ) = 1 2π 2 dk 1 2π 2 dk 2( 1) l B l (k 1, k 2 )j l (k 1 r 1 )j l (k 2 r 2 ) Multiples do not mix in the transformation!

22 Rotational Invariance in 2D U(1) invariance Definitions In 2 dimensions SO(3) U(1), we need to use cosine expansion. 2D Expansion B(k 1, k 2, θ) = B 0(k 1,k 2 ) 2 + n<0 B n(k 1, k 2 ) cos(nθ) ξn(r 3 1, r 2 ) = k 1 2π dk 1 k 2 2π dk 2( 1) n B n (k 1, k 2 )J n (k 1 r 1 )J n (k 2 r 2 ) The same expansion can be applied to spherical geometry in the flat sky limit, and for the (redshift space) projected three-point function.

23 Radical Information Compression Multipole expansion of the bispectrum It appears that (except for degenerate configurations) the first few orders (up to quadrupole) yield accurate description.

24 Bias The non-linear bias only affects the monopole! The Reduced Bispectrum Q = B/(P 1 P 2 + P 2 P 3 + P 3 P 1 ) Biasing the in the perturbative regime: f (δ) = b 0 + b 1 δ + b 2 /2δ 2 The reduced bispectrum will transform as q = Q/b + b 2 /b 2 Biasing of Multipole Moments b = Q 1 q 1 = Q 2 q 2 b 2 = q 0 b 2 Q 0 b. This means Q 1 /Q 2 is independent of bias!

25 Theoretical Prediction: Baryonic Oscillations Large Scales: Eulerian Perturbation Theory (Pan & Szapudi 2005b) Eulerian Perturbation theory: works for large scales.

26 Theoretical Predictions Small scales: halo model, multipole view: (Fosalba, Pan & Szapudi 2005)

27 Theoretical Predictions Small scales: halo model, traditional view: (Fosalba, Pan & Szapudi 2005)

28 Redshift Distortions Multipole expansion simplifies prediction of redshift space statistics We observe the redshift of galaxies: the velocities add to the distance One calculates the effect of redshift distortions for each multipole Some multipoles are effected more then others. Redshift space is difficult, but there are partial results. We can approximately predict Q 1 /Q 2 and Conditional cumulants (l = 0) to up to 5% accuracy.

29 Q 1 /Q 2 Redshift and Real Space Approximate Redshift Corrections, (Pan & Szapudi 2005a)

30 Conditional Cumulants Redshift and Real Space Approximate Redshift Corrections

31 Fast Algorithm for the Power Spectrum Full edge correction Use fast algorithm (N log N) for measuring the edge corrected two-point function at high resolution Use the Bessel integration formula of Ogata (2005) Bessel Integration 0 f (x)j ν (x) π k=1 w νkf (πψ(hr νk )/h) J ν (πψ(hr νk )/h)ψ (hr νk ), The two point function is spline interpolated at the roots r νk the Bessel function J ν,

32 Espice Powerspectrum From the Virgo VLS simulations P(k) k[hmpc 1 ]

33 SpICE vs espice Introduction This is analogous to the edge corrected power spectrum estimator for CMB: SpICE C l vs P(k) Euclidean vs Spherical item 3D vs 2D Gauss Legendre vs Bessel Integration However, in the flat sky approximation, 2D Bessel transform can be used instead of Legendre transform

34 CMB Angular Power Spectrum Recovered with Bessel integration C l l

35 Recipe for Measuring the Bispectrum With Edge Correction Measure the full three-point correlation function All configurations included with a fast algorithm (scaling is N log N times a factor depending resolution Calculate Legendre moments of the three-point functions Double-Bessel transform each moment with j l (or J l for 2D) Optionally, recover the full bispectrum from its moments at a given resolution

36 Introduction Full WMAP-bispectrum 3 Bispectrum(mK ) With Gang Chen index for (l1,l2,l3) I. Szapudi Higher Order Statistics

37 Measurements in the 2dF With Jun Pan

38 The 2dF Data Introduction We use the 2dF230k (Colless etal 2001) with galaxies with redshift quality Q 3 Both SGP and NGP, with completeness f 0.7 We build volume limited samples (integral magnitudes from 18-22) using k + e corrections from Norberg etal (2002)

39 Volume Limited Samples Sample Properties M bj z min z max d min d max N g (SGP/NGP) / / / / /2113

40 What do we estimate Introduction The monopole l = 0 moment of the three-point correlation function. Definition ζ 0 (r 1, r 2 ) = 2π d cos θζ(r 1, r 2, θ) Physical meaning: angle averaged three-point correlation function. In addition ξ, condition cumulants of order N = 2, 3 have been estimated.

41 Estimators Introduction We have used the fully edge corrected estimators Landy and Szalay (1993) for the two-point statistics Szapudi and Szalay (1998) ˆζ = DDD 3DDR + 3DRR RRR RRR Uniform weighting, 20 times the number of galaxies for randoms. An easy to implement N 2 algorithm (like two-point function): angle average neighbor counts in shells (Pan and Szapudi 2005)

42 Example of Measurements Magnitude Slice

43 Theoretical Model Introduction Perturbative bias expansion δ g = bδ + b 2 /2δ is assumed. To leading order Bias Model ( ξ g = f 2 b 2 σ ) 8 2 ξ 0.9 ( σ8 ) 4 [ ] ζ g = b 3 f 3 ζ + b 2 b 2 f (ξ 1ξ

44 Theoretical Model Cont d Redshift distortions: Distant observer limit. with f = Ω 0.6/b Kaiser shift f 2 = ( f + 15 ) f 2 Pan & Szapudi 2005 f 3 = 5( f f 2 + 9f 3 14f 4 ) 98( f + 3f 2 ) f 2 2

45 Theoretical Model Cont d Dynamical model: Eulerian perturbation theory The leading order calculation of e.g., any third order statistic can be obtained simply by δ 3 c = (δ (1) + δ (2) +...) 3 c = 3 (δ (1) ) 2 δ (2) +higher orders. For Gaussian initial conditions, this leads to formula in terms of P(k), the initial (or linear) power spectrum, and the F 2 kernel. µ = 3/7Ω 1/140 and x = cos θ The first permutation of the F 2 kernel {( ) ( 3 µ k1 P 0 (x) + + k ) 2 P 1 (x) + 2 } k 2 k 1 3 (1 µ) P 2(x) P(k 1 )P(k 2 )

46 Theoretical Model Cont d A permutation of the ζ is calculated as Bessel integrals A permutation of the ζ is calculated as Bessel integrals ξ 3 0 (r 1, r 2 ) = k 2 1 2π 2 dk 1 k 2 2 2π 2 dk 2 ( µ) )P(k 1 )P(k 2 )j 0 (k 1 r 1 )j 0 (k 2 r 2 ) ξ1 3 (r 1, r 2 ) = ( ) k 2 1 k dk 2 2π dk k1 2π 2 2 k 2 + k 2 k 1 P(k 1 )P(k 2 )j 1 (k 1 r 1 )j 1 (k 2 r 2 ) ξ2 3 (r k 2 1 k 1, r 2 ) = dk 2 2π dk 2π (1 µ) P(k 1)P(k 2 )j 2 (k 1 r 1 )j 2 (k 2 r 2 ) Ooura (1999) high accuracy oscillatory integrator used Finally ζ 0 and (ξ 1 ξ ) 0 is calculated by direct numerical integration. In addition, direct redshift space measurements from the VLS simulations are used as phenomenological model with f 2 = f 3 = 1 formally.

47 Maximum likelihood estimation Parameters b, b 2, σ 8 Data vectors ξ + ζ 0 measured in logarithmic bins from 1h 1 Mpc 70h 1 Mpc. Our measurement contain information from up to 140h 1 Mpc scales: possibly the largest scale ever measured for 3-point statistics In contrast with previous studies, we do not use ratio statistics of the kind ζ/ξ 2 On the large scales we consider, these statistics cross over 0 No ratio bias. We can measure σ 8

48 Structure of the Covariance Matrix

49 The Covariance Matrix Introduction We want to calculate χ dc 1 d, and the corresponding probability. Covariance matrix C is estimated from 22 simulations. This means that less than 22 eigenvectors can be meaningful at most from SVD. We checked that left and right eigenvectors are the same for large eigenvalues. From numerical experiments we found that approximately half of the eigenvectors are useful. (results are robust to using a few more or less) Our results are robust with respect to how many eigenvalues we keep. We have checked 8,9,10,11,12 eigenvalues, and all calculations were completed with 10.

50 The First Four Modes Introduction

51 Gaussianity of the SVD modes We will use Gaussian Maximum Likelihood

52 Likelihood Contours Vol. Lim. Sample, 8 35h 1 Mpc, phenomenological model

53 Three Parameter Fits Phenomenological model Introduction This corresponds to a direct comparison with simulations. In particular, redshift distortions are modelled by redshift space analysis of the simulations. Best Fit Parameters M bj σ 8 b b 2 χ ( 0.80) ( 0. 89) (-0.06) ( 0.79) ( 0.85) ( 1.01) ( 1.08) (-0.04) 3.11 (-0.06) 0.97 Brightest and faintest vol. limited sample omitted as the fit was not stable (although statistically consistent).

54 Likelihood Contours Vol. Lim. Sample, 8 35h 1 Mpc, phenomenological and theoretical models

55 Two Parameter Fits Phenomenological model Introduction For these measurements σ 8 = 0.9 was assumed. Best Fit Parameters M bj b b 2 χ (0.833) ( ) (0.993) (0.968) (1.053) (1.274) (0. 003) ( ) ( ) (0. 010) 0.206

56 Two Parameter Fits Theoretical model Introduction Again, σ 8 = 0.9 was assumed. Best Fit Parameters M bj b b 2 χ (0.358) ( ) (0.939) (0.935) (1.094) (1.375) ( ) (0.184) ( ) ( ) 0.977

57 Evolution of the Bias Comparison with Norberg etal (2002)

58 Logarithmic Mapping Szapudi & Kaiser (2003) Introduction From the Schrödinger Equation Ȧ = 1 [ 2 B + 2 A B] 2ma 2 Ḃ = 2 [ 2 A+ A 2 ] 1 B 2 mv 2ma 2 2ma 2 2 V = 4πG ρa 2 (e 2A 1) where A = 1 2 log(1 + δ). Tree level perturbation theory in A corresponds to infinite partial loop summation.

59 Densities Neyrinck, Szapudi, & Szalay 2009 δ Introduction δinitial log(1+δ) δgauss

60 Lognormality Introduction P[(x x)/ (x)] log(1 + ) Gaussian (x x >)/ (x)

61 Theoretical Calculations of the Bias A Perturbative and a Non-Perturbative Result P log(1+δ) (k) ξ lim = 2 [ ( ) s k 0 P δ (k) 4 + ξ l 1 + ξs 2 C1,2 + ( )] + ξs 2 7 2S 3 4C 1,2 + 2C 1,3 3 + C 2,2 4 + O(ξs 3 ) e Var[log(1+δ cell)]

62 Bias Introduction P log(1 + ) (k 0)/P (k 0) First-order Second-order exp( Var[log(1 + )]) P log(1 + ) bias P Gauss bias z s = < 2 cell >

63 Powerspectrum Amplitude 4 2 z =4.2 z =0.69 z =0 P z =127 P log(1 + ) P Gauss P init /P nowig

64 Information Introduction z =127 P P log(1 + ) P Gauss global mean z =127 z =1.2 z = S/N k max [h/mpc]

65 Phases Similar to Cepstrum Introduction

66 Flat Powerspectrum with Phases Preserved

67 Summary High Precision Cosmology With Higher Order Statistics Taking into account symmetries help to deal with the combinatorial explosion of higher order statistics. The resulting new technology helps with visualization, interpretation, estimation, algorithms, bias, redshift distortions, and predictions and edge corrrections. New algorithms (SpICE, espice), and theoretical prediction tools (cosmopy). We have presented the first edge corrected measurement of the full bispectrum of WMAP, and ζ 0 in 2dF First joint 2 and 3 point parameter estimation, σ 8 without CMB, but in agreement with CMB results Non-linear transformations (log, Gauss) appear to pump cosmological information leaked to higher order back into two-point.

68 Integrated Sachs-Wolf Effect Photons passing through changing gravitational potentials are becoming slightly hotter or colder The effect is linear if Ω 1 If the universe is flat (e.g., from CMB), linear effect signals Dark Energy Caviat: there can be a non-linear effect as well

69 Effect and Cross-Correlation Galaxy catalogs and the CMB are correlated due to the effect This is a tiny correlation compared to the CMB fluctuations, but it has been detected in several galaxy catalogs Combining all available data sets gives up to 4 σ result (Giannantonio etal, Ho etal) Evaluating the detection significance requires full knowledge of the covariances (between bins and catalogs)

70 Effect and Cross-Correlation Galaxy catalogs and the CMB are correlated due to the effect This is a tiny correlation compared to the CMB fluctuations, but it has been detected in several galaxy catalogs Combining all available data sets gives up to 4 σ result (Giannantonio etal, Ho etal) Evaluating the detection significance requires full knowledge of the covariances (between bins and catalogs)

71 SDSS DR6 LRG Introduction 7500 square degree 2SLAQ cuts objects 0.45 < z < 0.75 with median z = 0.52 median photo-z errors σ z 0.04

72 The SDSS DR6 LRG catalog

73

74 WMAP 5-year data set Introduction co-added Q,V,W ILC map MCMC map (joint fit to temperature, polarization and foregrounds) KQ75 galactic foreground masks maps are smoothed to 1 FWHM resolution Healpix N side = 64 maps, or 55 arcminute resolution

75

76 Cross-correlation results with SpICE and MLHood Agreement with Giannantonio etal, Ho etal, e.g., 2.1 σ from the MCMC map..

77

78 Beyond Cross-correlation Why in the linear regime Non-optimal weighting No redshift information was used Cosmic variance of galaxy data, even though we have access to a particular realization Perhaps more than linear signal

79 Finding Superstructures (100Mpc scales) The magic of Voboz Voboz/Zobov algorithms to find supervoids and superclusters int the LRG catalog Cutting out the highest signal-to-noise areas (simple weighting) Photometric redshift information is used Actual realization of the galaxy (DM) field is used Linear use of the data Possibility of localizing the signal, especially if there is a non-linear component

80 Introduction Supervoids and Superclusters I. Szapudi Higher Order Statistics

81 Back to the Basics: Image-stacking Granett, Neyrinck, & Szapudi 2008 Two different Monte Carlos to determine significance: agree within 2%

82 Detection significance Robust against number, radius, color. N Radius T µk T /σ

83 Granett, Neyrinck, & Szapudi 2009 What does this mean, how do we do cosmology with our results? Understand Voboz/Zobov or simplify technique Keep advantages: weighting,redshift information and realization. Calculate the potential from the galaxies (N-body engine) Raytracing, using the linear growth factor to calculate derivatives in 2 φ Use maximum likelihood to reveal the signal in the CMB data.

84 Potential map corresponding to the LRG catalog

85 Max. Likelihood/Matched Filter if Y = λx plus some noise (here the CMB) Maximum likelihood gives ˆλ = XC 1 Y YC 1 Y, (1) with variance σ 2 = (YC 1 Y ) 1, where C = X i X j λ is related to the bias (and any numerical factors missed in the prediction) looks like an optimal sum over a two-point quantity

86

87 Detection Significance from Potential Map Map Amplitude σ Q Coadd 2.96 ± V Coadd 3.33 ± W Coadd 3.01 ± Q FG reduced 3.43 ± V FG reduced 3.52 ± W FG reduced 3.20 ± MCMC 3.75 ± ILC 4.33 ±

88 Disappearing the cross-correlations

89 Potential + Superstructures Formally a 5.3 σ signal

90 Summary for more Over 2σ detection of the linear from cross-correlation and (marginally better) potential maps from the LRG s Cross-correlation disappears when the best fit map is subtracted Signal from superstructures over 4σ very robustly This appears to be in addition to the linear signal The nature of the signal from superstructures is yet to be determined (astrophysical, non-linear, or...? ) Pan-STARRS will be able to confirm all these measurements with overwhelming statistical significance: 6σ for linear. Theoretical investigations of the non-linear are on-going.