Estimating the Power Spectrum of the Cosmic Microwave Background

Transcription

1 Estimating the Power Spectrum of the Cosmic Microwave Background J. R. Bond 1,A.H.Jaffe 2,andL.Knox 1 1 Canadian Institute for Theoretica Astrophysics, Toronto, O M5S 3H8, CAADA 2 Center for Partice Astrophysics, 301 LeConte Ha, University of Caifornia, Berkeey, CA (February 25, 1998) We deveop two methods for estimating the power spectrum, C, of the cosmic microwave background (CMB) from data and appy them to the COBE/DMR and Saskatoon datasets. One method invoves a direct evauation of the ikeihood function, and the other is an estimator that is a minimum-variance weighted quadratic function of the data. Appied iterativey, the quadratic estimator is not distinct from ikeihood anaysis, but is rather a rapid means of finding the power spectrum that maximizes the ikeihood function. Our resuts bear this out: direct evauation and quadratic estimation converge to the same C s. The quadratic estimator can aso be used to directy determine cosmoogica parameters and their uncertainties. Whie the two methods both require O( 3 ) operations, the quadratic is much faster, and both are appicabe to datasets with arbitrary chopping patterns and noise correations. We aso discuss approximations that may reduce it to O( 2 ) thus making it practica for forthcoming megapixe datasets. I. ITRODUCTIO Observations of the cosmic microwave background (CMB) anisotropy are providing strong constraints on theories of cosmoogica structure formation. Panned observations have the potentia of providing constraints on the parameters of these theories at the percent eve [1 3]. Predictions of theories for CMB anisotropy are statistica in nature. For many theories, the compete description is given by the power spectrum, C, defined beow. Thus extraction of C from the data is of utmost importance as an end in itsef and for purposes of radica compression [4,5]. With the assumption of the Gaussianity of the data, the ikeihood function the probabiity of the data given a particuar theory takes a simpe form; with the further assumption of a prior uniform in the parameters, the ikeihood is proportiona to the posterior distribution of the parameters, given the data. This is precisey the quantity one wants and thus ikeihood anaysis has been used extensivey for cacuating the constraints on parameters given by CMB data. This is true whether the parameters are those of the power spectrum itsef or cosmoogica parameters. Another approach has been to form estimators that are quadratic functions of the data, e.g., [6]. This procedure has been improved recenty by the use of minimumvariance weighting of a the pairs of data points [7,8]. In this paper we present a unification of the quadratic and ikeihood approaches. We show that, when used iterativey, the minimum-variance weighted quadratic estimator is a fast technique for finding the maximum of the ikeihood function. In Section II we introduce the ikeihood function, expain our method for evauating it directy, and derive the quadratic estimator. We appy quadratic estimation and direct evauation to the case of COBE/DMR [9] in Section III. Both methods invove iteration and we find that for both, the iteration converges rapidy, with exceent agreement between the two methods on the fina C s and their variances. However, the higher moments of the probabiity distribution cannot be estimated with the quadratic approach and we find that there are significant deviations from Gaussianity in the ikeihood as a function of C. We discuss these differences, probems arising from them and possibe soutions. For COBE/DMR we estimate every individua C (for 2 24) since the data aow us to determine these with some precision. The quadrupoe, C 2, has received more attention in previous work than any of the other moments because of its sma vaue and because it is the most susceptibe to contamination by emission from our gaaxy [10]. We aso find the quadrupoe to be quite sma, C 2 = 149 ± 126 µk 2,comparedtoC 2 = 810 µk 2 for COBE-normaized standard cod dark matter (CDM). However, due to the strong skewness of the probabiity distribution for C 2, 25% of the probabiity is actuay above the COBE-normaized CDM vaue of C 2. Thus consistency with reativey fat modes ike standard CDM does not require the quadrupoe power to have been reduced by systematic errors. For most observations, which ony cover a sma fraction of the sky, estimating every C is not possibe. One must be content with estimating the power spectrum either with some binning in or through some other parameterization. Therefore in Section IV we discuss binning and rebinning. Then in Section V we appy the methods to estimate, from the Saskatoon (SK) data [11], the power in ten bins from =19to= 499. Power spectrum estimation can be used as a form of 1

2 data compression where the estimates of C and their covariance matrix are then used to constrain cosmoogica parameters. Because of the great simpifications invoved in working with power spectrum estimates instead of pixeized data, this is currenty the ony practica procedure for using a the CMB data. Such exercises have been conducted, e.g., [12 14]. In Section VI we discuss the approximations invoved in such a procedure and methods for reducing the resuting inaccuracies, and in Section VII we appy these resuts to future baoon- and sateite-borne experiments. Unfortunatey, direct evauation of the ikeihood function is an O( 3 ) operation, where is the number of data points. And it must be evauated many times. Thus for > 10, 000 this procedure becomes rapidy intractabe on modern workstations at east for the most straightforward impementations. Athough the speed of ikeihood anaysis has been greaty increased by use of signa-to-noise eigenmode compression [13,15 18], this procedure sti requires an O( 3 ) operation to be performed at east once. Further speed is necessary if we are to be abe to anayze forthcoming megapixe datasets. The quadratic estimator may offer a means of achieving this speed. We emphasize that as we have appied it here it is sti an O( 3 ) operation, but beieve that approximations may be made in a controed manner to reduce it to O( 2 ). We discuss these probems and possibe soutions in Section VIII, as we as expicity outine our agorithm for power spectrum estimation from CMB data. II. METHODS: LIKELIHOOD AALYSIS We begin by estabishing the notation used for describing the pixeized data of a CMB observation. We aso define the power spectrum, C, and the ikeihood function. With this common groundwork compete, we then move on to a description of the two different methods for estimating C. A. The Likeihood Function In genera, CMB observations are reduced to a set of binned observations of the sky, or pixes, i, i =1... together with a noise covariance matrix, C nii.wemode the observations as contributions from signa and noise, i = s i + n i. (2.1) We assume that the signa and noise are independent with zero mean, with correation matrices given by so C Tii = s i s i ; C nii = n i n i (2.2) where... indicate an ensembe average. With the further assumption that the data are Gaussian, these two point functions are a that is necessary for a compete statistica description of the data. One important compication to the above description comes from the existence of constraints. Often the data, i, are susceptibe to a arge source of noise, or a notwe-understood source of noise that contaminates ony one mode of the data. For exampe, the average vaue of i may be very poory determined. In this case, the average is usuay subtracted from i. Simiary, the monopoe and dipoe are expicity subtracted from the a-sky COBE/DMR data, because the monopoe is not determined by the data and the dipoe is oca in origin. In genera, pacing any constraint on the data or some subset thereof, such as insisting that its average be zero, resuts in additiona correations in i.wetakethisinto account by adding these additiona correations, C C,to the noise matrix to create a generaized noise matrix, C,whereC =C n +C C. In the imit that the ampitude of C C gets arge, this is equivaent to projecting out those modes which are now unconstrained by the data [19], but this scheme is numericay much simper to impement. Thus in the text beow we aways write the noise matrix as C instead of C n. The detais of this procedure for handing the effect of constraints are expained in Appendix A. Due to finite anguar resoution and switching strategies designed to minimize contributions from spurious signas (such as from the atmosphere), the signa is generay not simpy the temperature of the sky in some direction, T (ˆx), but a inear combination of temperatures: s i = dωh(ˆx, ˆx i )T (ˆx) (2.4) where H(ˆx, ˆx i ) is sometimes caed the beam map, antenna pattern or synthesis vector. If we discretize the temperature on the sky then we can write the beam map in matrix form, s i = n H int n. The temperature on the sky, ike any scaar fied on a sphere, can be decomposed into spherica harmonics T (θ, φ) = a m Y m (θ, φ). (2.5) m If the anisotropy is statisticay isotropic, i.e., there are no specia directions in the mean, then the variance of the mutipoe moments, a m, is independent of m and we can write: a m a m = C δ δ mm. (2.6) For theories with statisticay isotropic Gaussian initia conditions, the anguar power spectrum, C,istheentire statistica content of the theory in the sense that any possibe predictions of the theory for the temperature of i i = C Tii +C nii (2.3) 2

3 the microwave sky can be derived from it. Even for non-gaussian theories, the anguar power spectrum is a very important statistic, probaby the most important one for determining the viabiity of the most popuar non- Gaussian theories. However, the techniques we present in this paper for estimating the power spectrum assume that the fuctuations in both the sky signa and experimenta noise are Gaussian. The theoretica covariance matrix, C Tii, is reated to the anguar power spectrum by where C Tii = 2+1 4π C W ii (), (2.7) W ii () = nn H in H i n P (cos θ nn ) (2.8) is caed the window function of the observations and θ nn is the anguar separation between the points on the sphere abeed by n and n. Let us define the quantity C ( +1)C /(2π). This is usefu for two reasons: it is the ogarithmic average of C that gives the variance of the data and (therefore) for scae-invariant theories of structure formation, C is roughy constant at arge scaes. Within the context of a mode, the C depend on some parameters, a p, p =1... p which coud be the Hubbe constant, baryon density, redshift of reionization, etc. The theoretica covariance matrix wi depend on these parameters through its dependence on C. We can now write down the ikeihood function for a p, which is equa to the probabiity of the data given a p. 1 L (a p )=P( a p )= (2π) /2 C T (a p )+C 1/2 [ exp 1 ] 2 T (C T (a p )+C ) 1. (2.9) One can then search for the parameters a p that maximize this ikeihood. B. Direct Evauation of the Likeihood Function First, we must choose a set of parameters to characterize the theoretica covariance, C T. For a given cass of cosmoogica theories (e.g., adiabatic perturbations from infation) we can cacuate the power spectrum from some set of parameters ike the densities of various components, Ω x, the shape of the primordia power spectrum, the Hubbe constant, etc. A detaied exporation on-inear evoution wi produce non-gaussianity from Gaussian initia conditions but this is quite sub-dominant for < of the cosmoogica parameter space constrained by current CMB and arge-scae structure data is given in [13]. Aternatey, we can describe the power spectrum by its actua vaue at some discrete mutipoes or bands of. Moreover, a of the information in the experiment (again, for Gaussian theories) is captured in the ikeihood function for the power spectrum: P ( {a p }) P ( {C (a p )}) (2.10) In this paper, we concentrate on the C parameterization in order to determine the power spectrum directy from the data. In principe, we woud ike to cacuate the fu ikeihood as a function of the power spectrum P ( {C }) for some max ; at the very east we woud ike to find the maximum of this max -dimensiona function, and its properties (e.g., curvature or width ) around this maximum. Searching such muti-dimensiona spaces can be difficut; in this case, each evauation of the ikeihood function is an expensive O( 3 ) matrix manipuation and a brute force search through the parameter space woud take of order (C /δc ) max such evauations to reach an accuracy of δc. In our appications, we have found that the space is sufficienty structureess that a simpe iteration procedure works we for finding the maximum. In addition, we do not use a of the individua C vaues as separate parameters, since experiments do not have uncorreated information about bands of width < 2π/θ, where θ is the anguar extent of the survey [20]. For COBE/DMR, we bin in bands of width =2 3for 25 and ony consider 35; above this mutipoe we give the power spectrum a constant shape and ampitude (that of COBE-normaized standard CDM, in this case). For SK, we have tried bins of various widths, the choice of which we wi discuss beow. At the first iteration, we choose some appropriate starting C. For each (or band), we hod a other C s fixed whie the one of interest is aowed to vary; in the appropriate signa-to-noise basis, the ikeihood as a function of this singe parameter is trivia to compute (see Appendix A). That is, for each band abeed by B, we rewrite the correation matrix as C T + C = q B C B + C (2.11) (no sum over B) where the effective signa and noise matrices are given by C Bii C ii = B π C W ii (); = C ii + L B 2L +1 4π C LW ii (L). (2.12) and cacuate the ikeihood as a function of the adjustment factor q B aone. After going through a the bands of interest, we then update the starting power spectrum 3

4 by mutipying the C s in each band by the q B that maximized the ikeihood function. We then repeat. Convergence is achieved when a the q B s equa unity. For COBE/DMR, starting from COBE-normaized standard CDM (aready a good fit) we achieved convergence at the few percent eve after ony two such iterations for 20; after 10 iterations, convergence is everywhere better than There is a drawback to the procedure as described so far, compared to what coud be achieved by more ambitious methods such as simuated anneaing [21,1]. Even though we find the maximum of the ikeihood function, we haven t accuratey determined its shape ony the shape aong each C whie the others are hed constant (i.e., parae to the axes of the max -dimensiona space). And we have no estimate for the correations between the uncertainties in each estimate of C. Beow, we sha see how to use the Fisher matrix for an estimate of these correations. Ceary, a more ambitious minimization strategy woud be preferabe; we have chosen not to impement one since the quadratic estimator to be derived beow achieves this end without any expicit ikeihood cacuation. We have aso considered the possibiity of estimating each C assuming no other knowedge of a of the others. That is, we have attempted to marginaize over the C vaues outside of each band. This is equivaent to the procedure outined in Appendix A for marginaizing over removed constraints (averages, dipoes, etc.) and foreground tempates. However, in this case, the method fais to constrain the power spectrum. In performing this marginaization, we effectivey aow an arbitrary amount of noise consistent with any power spectrum at a outside of the band of interest. That is, we mutipy the second term in Eq by a very arge number to make the variance in those modes arger than the noise or (expected) signa. For a perfect, a-sky observation, this woud not be a hindrance since a the mutipoes are independent. For any reaistic observation, however, there is aiasing of different mutipoes together; some modes of the data (defined, for exampe, by the eigenmodes of Appendix A) that are being marginaized over wi have nonzero contributions from within the -band of interest. Thus, the new noise spectrum aone wi span the space of possibe signas, consistent with having no power at a in the band. This just reinforces the idea that any unknown noise in the observation shoud ideay be competey orthogona to the quantities we are attempting to estimate (which wi often be the case when the marginaization technique is used for experimenta constraints or foreground remova). C. Gaussian Approximation to the Likeihood Function If the ikeihood function is continous and has a peak then it can be approximated as a Gaussian near the peak. For we-constrained parameters this approximation shoud be good except in the tais of the distribution. A Gaussian approximation to the ikeihood function can be obtained by truncating the Tayor expansion of n L about a p at second order in δa p : n L(a + δa) =nl(a)+ p pp n L(a) δa p a p 2 n L(a) δa p δa p. (2.13) a p a p This Gaussian approximation is usefu because now, instead of making mutipe evauations of the ikeihood function, we can directy sove for the δa p that maximize it: δa p = [ 2 ] 1 n L(a) n L(a). (2.14) a p a p p a p The first derivative is given by: n L(a) = 1 a p 2 Tr [( T C )( C 1 C T,p C 1)] (2.15) and the second derivative by F (a) n L(a) pp 2 a p a p =Tr [( T C ) (C 1 C T,p C 1 C T,p C Tr ( C 1 C T,p C 1 ) C T,p 1 2 C 1 C T,pp C 1 ) ] (2.16) where Tr is the trace, C C T +C is the tota covariance matrix and,p / a p. We ca the second derivative the curvature matrix and give it the symbo F (a) where the (a) indicates that we have taken the derivative of n L with respect to a. To the extent that the ikeihood function is not Gaussian, we wi not have correcty soved for its maximum. Thus we iterate. The coser we get to the maximum, the better the quadratic approximation to n L wi become. This is exacty the ewton-raphson method for finding the zero of n L/ a p. The procedure is not foo-proof there is the risk of getting trapped in a oca extremum. In practice we have found the ikeihood function to be sufficienty structureess that this is not a probem. D. Quadratic Estimator The above procedure is not exacty what we do in practice. Cacuating the curvature matrix is a computationay intensive procedure. Matters simpify significanty if we sette for the ensembe average quantity, caed the Fisher matrix, F : 4

5 F (a) pp F(a) pp = 1 2 Tr ( C 1 C T,p C 1 C T,p ). (2.17) When taking this ensembe average, denoted by..., we assume that the theory is correct and therefore that T = C. ote that the Fisher matrix, ike the curvature matrix, is defined with respect to particuar parameter choices. If we transform to a new set of parameters, ã p then the Fisher matrix for these new parameters is F (ã) = Z 1 F (a) (Z 1 ) T,whereZ pp = ã p / a p. Tegmark offers a proof of this [7]; with our approach it is obvious from the definition of the curvature matrix in Eq Repacing the curvature matrix with the Fisher matrix makes our estimator for a p quadratic in the data, : δa p = 1 (F (a) ) 1 pp 2 Tr [( T C )( C 1 C T,p C 1)]. p (2.18) This is what we ca the quadratic estimator. The right hand-side depends on a p,sowepickaninitiaa p,cacuate the correction δa p, and then repeat for the new vaue of a p. ote that the power spectrum estimate is not constrained to be positive-definite a point we discuss beow. If we assume that the input theory is correct, then T = C and therefore Eq impies δa p =0. Simiary, one can work out that δa p δa p =(F (a) ) 1 pp. This is to be expected since for a Gaussian distribution, the two-point function is the inverse of the curvature matrix. Athough the quadratic invoves using the Fisher matrix F as an approximation to the fu curvature matrix F, both procedures iterate to the same parameters, the maximum of the ikeihood function. This is because both F and F are invertibe, so δa p = 0 from either procedure impies n L/ a p = 0. Thus, when appied iterativey, the quadratic estimator wi find the exact ocation of the ikeihood peak; the ony approximation comes in using the Fisher matrix to approximate the errors, rather than the fu curvature matrix (and beow we show that in the cases studied, this is a very good approximation; moreover, having found the ocation of the peak, the curvature there can be cacuated expicity if necessary). Our procedure is very simiar to that of the Levenberg- Marquardt method [21] for minimizing a χ 2 with noninear parameter dependence. There the curvature matrix (second derivative of the χ 2 ) is repaced by its expectation vaue and then scaed according to whether the χ 2 is reduced or increased from the previous iteration. Simiar manipuations of the Fisher matrix may possiby speed convergence of the ikeihood maximization, athough one woud want to do this without direct evauation of the ikeihood function. In our appications to COBE/DMR and SK we have found that iteration converges quicky. Iteration is especiay important for the cacuation of the error covariance matrix. Without iteration, the errors are determined entirey by the initia theoretica assumptions and are not infuenced by the data. (Of course, this is exacty why the Fisher matrix has been so usefu in determining how we future observations wi be abe to determine parameters.) As we have defined it so far, the quadratic estimator with the iteration procedure is a method for finding the maximum of the ikeihood. Ony if one takes the prior probabiity to be uniform in the parameters is this equivaent to maximizing the posterior probabiity. We coud, of course, incude different priors directy in the definition of the estimator. The derivation woud then begin by changing Eq to a Tayor expansion of n P post where P post LP prior is the posterior probabiity distribution and P prior is the (differentiabe) prior distribution. To see how the quadratic estimator works, we can take a one-dimensiona exampe. Consider a function f, that is approximatey quadratic. If we take its first and second derivatives about some point, x 0 (= 0.7 in the figure), we can construct the function f Q which approximates f. By finding the vaue of x that maximizes f Q we have a guess as to the maximum of f. ow, for a further refinement of the estimate, a new f Q can be cacuated based upon the properties of f at this new vaue of x. (ote that the fu quadratic estimator of Eq incudes the further approximation of using the Fisher matrix (Eq. 2.17) rather than the actua curvature matrix (Eq. 2.16) for the second derivative of the og-ikeihood.) FIG. 1. A one-dimensiona exampe of quadratic estimation. The appications we discuss in the foowing a use the C s as the parameters a p.inthiscase, C Tii, C T = +1/2 C (+1) W ii () W ii ()/. (2.19) 5

6 We aso consider the power spectrum averaged over some bands B with some assumed shape C shape ;inthatcase, we average the above weighted by the shape: C Tii,B = B C Tii,C shape. (2.20) However, there is aso the interesting possibiity of taking the a p as the cosmoogica parameters that affect the spectrum, Ω, h, n S,Ω b, etc. Iteration in this case shoud aso converge to the ikeihood maximum. We note that the quadratic estimator discussed here can aso be derived by finding the quadratic function of the data that is unbiased and has minimum variance. For a fu discussion of the quadratic in this context, see [7,22,23]. The quadratic function of the data derived this way is the same as Eq However, the estimate is ony unbiased if there is no iteration. Since the end point of (successfu) iteration is the maximum ikeihood, the iterated estimator is, ike a maximum ikeihood estimators, ony asymptoticay unbiased. The methods we have used can aso be appied to optima determination of the correation function in anguar bins. The optima signa pus noise weighting suggested for correation function determination differs from the usua diag[cn 1 ] weighting appied to COBE/DMR. E. Singe Bandpower Estimation It has now become conventiona to characterize switching experiments which covered sma patches of the sky by a singe bandpower [15], pacing the estimated power at a ocation reated to the window function of the experiment. In this case, there is just one parameter to determine. The quadratic statistic reduces to Q B = C 1 C T C 1 Tr C C 1 C T C 1 Tr C T C 1 C T C 1. (2.21) If the optima weight C 1 is repaced by the diagona part of Cn 1, then this is reated to the quadratic statistic proposed by Boughn and Cottingham [24], which has been appied to the COBE/DMR and FIRS data using Monte Caro simuations to define its distribution. With the optima weighting and the proper incusion of constraints in C,thevauesofQ B and its error estimation are of direct use. As discussed above, the iterated quadratic estimator for the ampitude wi converge to the maximum ikeihood vaue. The parameter Q B coud be any squared ampitude characterizing the assumed theoretica C, such as the σ8 2 used to characterize the strength of the power spectrum on custer scaes. To transate to an average bandpower one must evauate Q B C shape B, using an appropriatey weighted average of C shape over the singe band B. Issues associated with such averaging are addressed in IV. Current and future experiments cover arge enough patches of the sky that characterizing their resuts by singe bandpowers is not usefu, but evauation of power spectrum normaization ampitudes (such as σ 8 ) for particuar theories wi aways be of use. III. APPLICATIO TO COBE/DMR We first appy these methods to the anisotropy measurements of the COBE/DMR instrument [9,25]. The DMR instrument actuay measured a compicated set of temperature differences 60 apart on the sky, but the data were reported in the much simper form of a temperature map, aong with appropriate errors (which we have expanded to take into account correations generated by the differencing strategy, as treated in [16], foowing [26]). The cacuation of the theoretica correation matrix incudes the effects of the beam, digitization of the time stream, and an isotropized treatment of pixeization, using the tabe given by Kneiss and Smoot [27], modified for resoution 5. We use a weighted combination of the 31, 53 and 90 GHz maps. Because most of the information in the data is at arge anguar scaes, we use the maps degraded to resoution 5 which has 1536 pixes. Further, we cannot of course observe the entire CMB sky; we use the most recent gaactic cut suggested by the DMR team [9], eaving us with 999 pixes to anayze. We use the gaactic, as opposed to eciptic, pixeization. For both methods we iterated 28 parameters: C 2 to C 24 individuay, C 25 to C 32 grouped into bins of width 2 and finay C 33 through C 35 grouped into one bin. Binning is described in more detai prior to the Saskatoon appication where it is much more important. FIG. 2. Maximum-ikeihood power spectra from iterative direct evauation of the ikeihood function. The curve is the zeroth iteration: COBE-normaized standard CDM. The points with error bars are, from eft to right, the resuts of the first to third iterations. Here, we define the error bars by a ikeihood ratio of e 1/2 from the peak. 6

7 FIG. 3. Iterative quadratic estimation. The curve is the zeroth iteration: COBE-normaized standard CDM. The points with error bars are, from eft to right, the resuts of the first to third iterations. FIG. 4. We compare the resuts of the quadratic and direct evauation iteration schemes. At each, the eft error bar (square symbo) is for the quadratic, the right (triange) is for the direct evauation. In Figures 2 and 3 we see the resuts of the iterative procedures described in the previous section. Figure 2 shows the resuts of direct evauation and Fig. 3 shows the resuts of quadratic estimation. Moments >10 are not shown to avoid cutter. From eft to right are the first to third iterations, together with their error bars. The soid ine is the starting point we chose, the power spectrum for COBE-normaized standard CDM. For this method, we define the estimated C as the maximum of the ikeihood function, and the errors by the vaue of C where the ikeihood drops by a factor e 1/2 from that maximum. First we wi discuss the direct evauation method. The iteration converges rapidy. The maximum ikeihood vaues of a fourth iteration (shown in Fig. 4) typicay differ from the third by 1 3% of the error bars (for 2 19) with a maximum deviation of 7% at = 12. In the imit that the moments were independent, there woud be no need for iteration; iteration is ony necessary because of the infuence the vaue of one band has on the best vaue of another. The rapidity of the convergence is expected because, as we wi see beow, the moments are in fact fairy uncorreated. We remind the reader that the error bars given by this method indeed the whoe probabiity distribution for each C are cacuated by hoding the others fixed. Iteration is aso quite rapid for the quadratic estimator: the maximum ikeihood vaues of a fourth iteration (shown in Fig. 4) differ from the third by better than 1% of the square root of the variance for 24, except for the quadrupoe and = 20 which are sighty worse, converging to 3%. Just ike the direct method, most of the change in the maximum ikeihood estimate occurs in the first iteration. Unike the direct method, the error bars of the first iteration are quite different from the error bars of the ater iterations. That is because the error bars (the Fisher matrix) do not depend on the data, but ony on the input power spectrum. Therefore the data have had no effect on the error bars unti the second iteration is reached. To the extent that the distribution is Gaussian, these error bars accuratey represent the uncertainty on each parameter; they take into account the correations with the other parameters. The argest changes in the error bars from 1st to 2nd, 2nd to 3rd and 3rd to 4th are 610% ( = 2), 60% ( = 2) and 6.5% ( = 6), respectivey. From the 3rd to the 4th, most of the changes are ess than 1%. In the previous section it was caimed that the curvature matrix is a good approximation to the Fisher matrix. We have expicity checked this for the fina iteration and find that for <20 most of the Fisher matrix and curvature matrix derived error bars agree with each other to better than 4%. The worst cases are =4and=5at 13% and 15%. ot ony do these methods converge, but they converge to the same power spectrum, as we see in Fig. 4. The differences between the fina iterations are ess than 2% of the quadratic error bars for <20, except for a 4% difference at = 18; at higher moments, the methods often do not detect positive power. ote that at mutipoes where both methods do detect nonzero power, the quadratic method gives error bars which are systematicay smaer (than those of the direct method) in the direction of positive power, and systematicay arger towards ower power. This can be understood as a resut of the considerabe non-gaussian skewness of the distribution of power, as seen in Fig. 6. Aso note that when the ikeihood maximum is at zero power, the quadratic estimate is at (physicay meaningess) negative power. This is to be expected since the existence of a maximum 7

8 at C = 0 impies n L/ C 0, and therefore the Gaussian fit to n L at C = 0 wi peak at C 0. We have aso checked that using the fu resoution 6 data (3881 pixes after the gaactic cut) changes the resuts of the maximum-ikeihood estimate for the power spectrum by much ess than one sigma. We have checked in detai using the direct evauation, for which the resoution 6 resuts differ from those at resoution 5 by ess than 5% for 15, except at = 6 9 where the difference is amost 10 20% and at =12and= 14, where the difference is neary 50%, sti smaer than the arge error at these ; the higher resoution data give an overa normaization that differs by 4% (compared with an error of 14%) from that of the best quadratic computed at resoution 5. These differences are consistent with those observed for different pixeizations and gaactic cuts [9,25]; note that both the direct evauation and quadratic procedures converge with consideraby higher precision than these intrinsic errors, even for > 15 where the pixeization differences become important and, simutaneuousy, the noise begins to dominate. We aso agree at east quaitativey with other cacuations that we have compared to, in a cases (with detected power) we within the various reported error bars. In Fig. 5 we show a comparison of our quadratic resuts with those of [17,25], both of whom use a maximum ikeihood method. Gorski [25] uses a compete search through parameter space with cut-sky spherica harmonics to speed up the cacuation; Bunn & White [17] aso use the Signa-to-oise transformation of Appendix A to increase speed. The resuts of our first quadratic iteration aso have quaitative agreement with Tegmark s impementation of the quadratic estimator [7]. the fina estimates are unaffected by the choice of initia starting pace, and the stronger caim that they woud have resuted from any starting pace. From the Fisher matrix and from the probabiity distributions of Fig. 6 it shoud be evident that this ikeihood space is fairy structureess. We coud have started anywhere and converged to the same pace, athough perhaps sighty ess rapidy. We note though that if the correations were stronger between the different C s, the direct method woud be ess robust. In particuar, if the initia power spectrum were much too arge, then each mutipoe moment woud try to make up for this a by itsef by coming out very sma. Thus there coud be arge osciations conceivaby without convergence. In addition, these correations, combined with the width of the ikeihood function, impy that our iterative direct evauation method for finding the peak may not converge to a unique maximum, as vaues osciate between iterations; in practice, we have found that the changes remain much smaer than the size of the error bars, as noted above. Such a broad ikeihood function indicates that the data do not strongy prefer a unique maximum. onetheess, if we desire to find the exact ocation of the peak, a more compete search through the many-parameter space (as in [17,25]) or the use of the quadratic method wi be necessary. The probabiity distributions of the parameters are different for the two different methods because of the approximation of independence by the direct method and the approximation of Gaussianity by the quadratic method. We can see those differences in Fig. 6. The departure from Gaussianity is most dramatic for the quadrupoe. According to the Gaussian distribution of C 2, COBE-normaized CDM with C 2 = 770 µk 2 is over five standard deviations away from the mean, highy rued out. But the strong skewness of the exact ikeihood function has 25% of the probabiity for C 2 above 770 µk 2. This is more probabiity than there is above ony 1σ for a Gaussian distribution! As increases the distributions become more Gaussian. The distribution for = 21is we-approximated by a Gaussian as expected from the centra imit theorem since there are approximatey 30 independent modes of roughy equa weight contributing to the constraint. FIG. 5. Comparison of different groups power spectrum estimates, as marked. Gorski computes power spectra in both eciptic and gaactic pixeizations of the sky. Aso these quadrupoe probabiity distributions do not take into account the possibiity of foreground contamination. The DMR team [10] have carefuy anayzed the foreground contamination and report C 2 = (273 ± 185 ± 360) µk 2 with statistica and systematic errors. The fact that three competey different methods achieve simiar resuts ends support to the caim that 8

9 FIG. 6. Probabiity Distributions for individua C vaues, as abeed, for a prior uniform in C. The soid curve is the true ikeihood from the ast iteration of the fu evauation; the dotted curve is the Gaussian approximation from the ast iteration of the quadratic procedure. For = 2, we aso show the cumuative probabiity distribution, propery normaized to unit probabiity as C. The highy non-gaussian nature of some of these distributions impies that other definitions of the point estimation and the error bars are possibe. First, we coud consider the mean or median of the distribution, rather than its maximum, and define errors by the amount of encosed probabiity. Second, we coud aso have used different prior probabiities for the C. Throughout the paper, we use a prior uniform in C, equivaent to equating the posterior distribution with the ikeihood itsef. When the data constrain the power strongy (i.e., sma error bars), the resut is insensitive to the choice of the prior; in other regimes, such as the quadrupoe, C 2,the prior has more significance. To investigate this, we have aso tried other possibe prior distributions, aong with the definition of the point estimate by the median of the distribution. A prior P(C )dc dc / C (which is equivaent to a prior uniform in σ th =(C ) 1/2 )gives a median C 2 60% higher than the ikeihood maximum; the highy skewed distribution means that for a constant prior the median is 166% higher, whie a prior uniform in n C 2 has a median ony 5% higher. Finay, we have aso tried a Fisher Prior, which uses the eement of the Fisher matrix (Eq. 2.17) corresponding to a p = σ th to determine the expected ampitude, )] 2 1/2 P (σ 2 th) F 1/2 σσ [ ( n C + σth Tr 2 C T σ 2 th (3.1) which is uniform in C σth 2 at ow ampitudes, but uniform in n C at high ampitudes, where the smooth transition is determined by the scae at which signa-to-noise becomes about one. For this prior, the median is about 20% higher than the maximum ikeihood. FIG. 7. Rows of the normaized DMR Fisher matrix (see text), at =2,10, 21. The soid ines show the matrix at the zeroth iteration; the dashed ines for the fina iteration. F (C) In Fig. 7 we show the normaized Fisher matrix, / F (C) F (C) to indicate the eve of correations between the different C s. The off-diagona terms are due to the inhomogeneous coverage, the most drastic component of which is due to the gaactic cut. This cut discards a map pixes with gaactic atitude b 20,withsome modifications motivated by the DIRBE dust map [9]. A map with a b 20 cut and otherwise homogeneous coverage woud resut in zero overap between Y m swith opposite parity which expains the near zero vaues of the Fisher matrix for odd [17]. Modes with simiar parity do mix and hence the non-zero eements at = ± 2. Even these off-diagona terms though are much smaer than the diagona, especiay for the ower mutipoe moments which are determined by modes with higher signa-to-noise. Iteration does not have much effect on the normaized Fisher matrix; the off-diagona components are argey a resut of the coverage geometry. IV. METHODS: BIIG AD REBIIG For the same reason that imited extent in the time domain eads to imited spectra resoution in the frequency domain, uncertainties in C and C are strongy correated when < 2π/θ where θ is the inear extent of the observed region [20]. Thus binning moments together in bins of width π/θ is a sensibe thing to do. Because of the experimenta noise, fina bins may need to be even coarser to prevent the error bars from being excessivey arge. We view binning as a two-step procedure: an initia fine binning foowed by a rebinning to coarser bins. The reason for the first step is that we want to know, within each coarser bin, where the constraining information is. 9

10 The finer binning gives us this knowedge. For pedagogica reasons, we start with a discussion of rebinning and then discuss the initia binning. A. Rebinning We assume here that the initia binning is the finest possibe, = 1, since this makes for the simpest exposition. It is easiy generaized to arbitrary initia binning. For reasons that wi become cear ater, we begin our discussion of this rebinning procedure by reparameterizing the power spectrum in terms of an assumed spectra shape, C shape. Thus the parameters we are trying to estimate are no onger C directy, but the deviation from the assumed shape, given by q : C = q C shape. (4.1) If our estimates of individua q are too noisy then we can average them together into coarser bins, which we wi abe by the subscript B. Wewishtodothisina minimum variance manner. That is, we want to find Q B that minimizes χ 2 = (Q B q ) F (q) (Q B q ) (4.2) where the sum (ike a sums in this subsection) extends over the width of the new and coarser bin. The Fisher matrix appears here because, in the Gaussian approximation to the ikeihood function, the Fisher matrix is the inverse of the parameter covariance matrix. Compications due to non-gaussianity are discussed in VI. It is easy to show that the soution to this minimization probem is given by Q B = q F (q) F (q). (4.3) The new parameters Q B have the Fisher matrix, F (Q) BB = F (q) where the sum over extends across bin B and the sum over extends across bin B. We see that Q B averages q over the fiter f (q) B = F (q). The ( q) superscript indicates that this fiter is for averaging q s. As the constraints on the power spectrum become tighter, it is inevitabe that we wi move from potting averages of C (band-powers) to potting q in what we ca q-space, or deviation space. We show some exampes of this ater in VII where we simuate future data sets. Therefore it is worth exporing this space a itte further. One question to answer is: what vaue shoud be used for ocating Q B horizontay on a graph? We advocate choosing this eff so that for a band ranging from 1 to 2 eff 2 f (q) B = f (q) B. (4.4) = 1 = eff With this definition, 50% of the weight that constrains q B comes from 1 << eff and the other 50% comes from eff << 2. Athough comparison of theories with the data wi occur in q space, we wish to transate our vaues into the famiiar C -space. To do this we must define a suitabe average of C shape over bin B, C shape B, with which to mutipy Q B and a suitabe vaue at which to pot the error bar, eff. The best weighting to use for this is debatabe. We emphasize that the ambiguities associated with the transation from Q B to a power estimate, C B ony affect potting not the comparison of theory with data. Furthermore, we have tried severa different weighting schemes and found negigibe differences in their vaues of eff and C B, so ong as they are proportiona to f (q) B which encodes the signa-to-noise information in the band. To motivate a particuar averaging we first rewrite Eq. 4.3 in terms of C and its Fisher matrix: Q B = C F (C) Cshape F (C) C shape C shape. (4.5) The reation between Q B and C in the above equation suggests that the foowing fiter be used to cacuate C shape B : B = F (C) C shape = f (q) B /Cshape (4.6) since this is the weighting of each C in Eq Therefore to make our power estimates we use C shape B = B Cshape f. (4.7) (C) B with the resut that C B = Q B C shape B = B C. (4.8) f(c) B The roe of the fiter function, B is exacty that of W / in the band-power procedure of [15], where W is the trace of the window function matrix defined in Eq We wi deveop this connection more ater. For now, we define eff, + and, exacty as was done in [15], so that we can pot data points propery ocated in space with horizonta error bars: eff = (C) f B f(c) B (4.9) and and + are where B has faen to e 1/2 of its maximum vaue. We remind the reader that every sum over in this section is ony over the vaues of within band B. 10

11 B. Initia Binning One may wish to estimate fewer parameters than every mutipoe moment right from the beginning. In this case one woud parameterize the spectrum as C = q B C shape χ B () (4.10) where χ B () isonewhenis within the range of band B, and zero otherwise. To convert q B to a power estimate, C B, we need an average of the shaped spectrum over band B. A usefu conversion factor is given by Eq Of course, in order to cacuate C shape B by Eq. 4.7 one needs to know the Fisher matrix at every which is a cacuation we re trying to avoid by using coarse binning. Once again though, as ong as the binning is not too coarse, the detais of the averaging are unimportant. If the binning is fine enough, then a simpe average (uniform in ) wi suffice that is, take C shape B = Cshape χ B () χ ; (4.11) B() here, the denominator is simpy the width of the bin. This is what we have done in our appications (athough see VI for how this can be improved by use of anaytic knowedge of the Fisher matrix). As is usuay the case with binning, we want to make the bins as fine as necessary to capture a the information but no finer since that means extra work. A ower imit to the bin sizes comes from the fact that fuctuation power from C wi be indistinguishabe from that from C if < 2π/θ, whereθis the inear extent of the observed region, as aready mentioned. We may wish to make our initia bins even coarser. Some considerations to keep in mind are that if one is trying to reduce sensitivity to uncertainty in the power-aw index then ogarithmic spacing produces equa shape sensitivity in each bin. If the chief shape uncertainty comes from features with a characteristic waveength, e.g., Dopper peaks, then a inear spacing produces equa shape sensitivity in each bin. of supernova remnant, Cassiopeia-A. Leitch and coaborators [28] have recenty measured the fux and find that the remnant is 5% brighter than the previous best determination. We have adjusted the Saskatoon data accordingy. In Fig. 8 we show the resuts of our iterated quadratic estimator on the SK data, in ten eveny spaced bins from =19to= 499. Again, the convergence proceeds quite rapidy, athough not quite as rapidy as for COBE/DMR. Evauation of the Fisher matrix shows that there are approximatey 20% anti-correations between neighboring bins. We note in passing that the faing power spectrum seen for < 100 has been noticed by the experimenters themseves [29]. What we directy estimate is the adjustment factor q B of Eq As mentioned above, in order to convert this to a power spectrum ampitude we need some measure of the average power in the bin. Here we have used an average uniform in C across the bin (Eq. 4.11). For the first bin, the averaging shoud probaby be weighted more to the higher mutipoe moments than to the ower ones in the bin because the sensitivity to the spectrum is increasing rapidy with increasing. We wi see this rapid rise in sensitivity to the power spectrum in the next section where we pot the Fisher matrix for a finer binning. There is very itte information in the three highest bins. Thus, for the fina iteration we binned them together and potted the resut as the point with the horizonta error bar. Because of the coarseness of the bins, the fiter function for the rebinning is coarse and therefore eff, + and are not determined very we. To get the fiter function more finey, we need to do a finer initia binning, which wi be done in the next section. V. APPLICATIO TO SASKATOO We now appy our methods to the Saskatoon (SK) dataset [11]. The SK data are reported as compicated chopping patterns (i.e., beam patterns, H, above) in a circe of radius about 8 around the orth Ceestia Poe. The data were taken over (athough we ony use the data) at an anguar resoution of FWHM at approximatey 30 GHz and 40 GHz. More detais can be found in [11]. The combination of the beam size, chopping pattern, and sky coverage mean that SK is sensitive to the power spectrum over the range = The Saskatoon dataset is caibrated by observations FIG. 8. Quadratic estimates of the power in 10 bins, derived from the SK data. The curve is the zeroth iteration, tited CDM with n =1.45 and σ 8 =2.16. The squares are from eft to right, the resuts of the first to third iterations. The data point with the horizonta error bar is a rebinning of the top three bins. 11

12 To investigate the probabiity distributions beyond the mean and the variance, we used our direct ikeihood evauation procedure, starting from the fina quadratic iteration. The resuts are shown in Fig. 9. The uncertainties in the first bin are strongy sampe-variance dominated. In the sampe-variance imit the fractiona variance, (δc ) 2 /C 2, is inversey proportiona to the number of independent modes contributing to the estimate. Since the first bin is not we-determined we can therefore surmise that ony a few modes contribute to it. With so few modes we cannot expect the distribution to be Gaussian and thus the strong non-gaussianity for the first band, shown in Fig. 9, is not surprising. FIG. 9. Probabiity Distributions for the power in bands, C B, as abeed, for a prior uniform in C B. The soid curve is the true ikeihood from the direct evauation; the dotted curve is the Gaussian approximation from the third iteration of the quadratic procedure. VI. METHODS: RADICAL COMPRESSIO As mentioned above, for Gaussian theories, P ( C ) contains a the information that is in the map. If the probabiity distribution were Gaussian in C,thenathe information in the probabiity distribution coud be compressed into a mean and a covariance matrix: P ( C ) Ĉ, δc δc. (6.1) By the definition of a Gaussian probabiity distribution, this compression invoves no oss of information. The ossess nature of this compression was pointed out by Tegmark [7] athough here we emphasize that it is ony true in the Gaussian imit. We refer to compression to the power spectrum as radica compression because the data reduction is impressive: the information in a map with pixes and an noise covariance matrix is now hed in ess than power estimates and their covariance matrix. With compression to numbers and a covariance matrix, anaysis of constraints on cosmoogica parameters becomes quite rapid. One simpy forms the χ 2 : χ 2 ({a}) = ) ( ) (C ({a}) Ĉ M 1 C ({a}) Ĉ (6.2) and simpy evauates it to find the minimum and aso the one sigma and possiby two sigma confidence regions of the parameter space. Here, C ({a}) is the cacuated spectrum for the paramters a p and M δc δc is some appropriatey determined correation matrix, e.g., the inverse of the Fisher matrix or the exact curvature matrix for the quadratic method, or a ikeihood ratio or Bayesian determination for the direct evauation of the ikeihood. Unfortunatey, the probabiity distribution is non- Gaussian, as we have seen. One might think that this ony causes minor inaccuracies to the method of Eq In fact, the probems are of a systematic nature and can be quite important. To see this we need ony examine the case of COBE/DMR. Say we wanted to use our power spectrum estimates to measure the best fit ampitude of standard CDM, expressed as a prediction for σ 8,byusing Eq Using our estimates of C from the fina iteration of either the direct or quadratic estimation procedures together with the Fisher matrix from the fina iteration, we find σ 8 =1.1 instead of the correct vaue of σ 8 = 1.2. This exampe does not mean that non- Gaussianity has made radica compression useess, but rather that we must proceed with some care. The decrease in power is a systematic effect due to the skewness of the probabiity distributions which aow more positive and ess negative fuctuations reative to a Gaussian distribution with the same variance. Another way of thinking about it is that those ampitudes that fuctuate downwards have their variance reduced and thus their weight increased whie those that fuctuate upward have their variance increased and therefore their weight decreased. Contrast this to a Gaussian probabiity distribution for which the curvature is independent of ocation. Thus one can see that the non-gaussianity of the probabiity distribution can be very important and some care must be used in attempting this radica compression. One soution to the probem may be to find a function of C whose distribution is more Gaussian than that of C itsef. Motivation for one particuar form comes from considering the sources of the variance. There is a sampe-variance contribution which is proportiona to the power and a noise contribution which is independent of the power, thus δc C +x for some appropriate x reated to the experimenta noise. According to this proportionaity, the probabiity distribution for n (C + x ) might be we-approximated by a Gaussian since its variance is independent of C. This procedure is under investigation [4]. 12