TESTS OF STATISTICAL SIGNIFICANCE AND BACKGROUND ESTIMATION IN GAMMA-RAY AIR SHOWER EXPERIMENTS

Transcription

1 The Astrophysical Journal, 603: , 2004 March 1 # The American Astronomical Society. All rights reserved. Printed in U.S.A. TESTS OF STATISTICAL SIGNIFICANCE AND BACKGROUND ESTIMATION IN GAMMA-RAY AIR SHOWER EXPERIMENTS R. Fleysher, L. Fleysher, P. Nemethy, and A. I. Mincer Department of Physics, New York University, New York, NY 10003; roman.fleysher@physics.nyu.edu, lazar.fleysher@physics.nyu.edu, peter.nemethy@nyu.edu, allen.mincer@nyu.edu and T. J. Haines Physics Division, Los Alamos National Laboratory, Los Alamos, NM 87545; haines@lanl.gov Received 2003 May 29; accepted 2003 November 14 ABSTRACT In this paper we discuss established methods of significance calculation for testing the existence of a signal in the presence of unknown background and point out the limits of their applicability. We then introduce a new selfconsistent scheme for source detection and discuss some of its properties. The method overcomes weaknesses of those used previously and allows incorporating background anisotropies by vetoing existing localized sources and sinks on the sky and compensating for known large-scale anisotropies. By giving an example using the Milagro gamma-ray observatory data, we demonstrate how the method can be employed to relax the detector stability assumption. The new method is universal and can be used with any large field-of-view detector, in which the object of investigation, steady or transient, point or extended, traverses its field of view. Subject headings: atmospheric effects methods: data analysis methods: numerical methods: statistical 1. INTRODUCTION The method is applicable to point and extended source The problem of evaluating the statistical significance of searches, as well as to searches for transient phenomena. In x 6 we show how the new method allows compensating for known observations when one searches for gamma-ray sources by large-scale anisotropies and incorporating existing localized using air shower experiments remains one of highest importance. The emission from a source would appear as an excess sources and sinks by vetoing small regions around them, a feature not available in any of the present methods. In x 7we number of events coming from the directions of the candidate present a method for testing the detector stability assumption, over the background level. The difficulty arises because the the foundation for the background estimation methods. In x 7.1 signal-to-background ratio as registered by the detectors in we illustrate how this test can be applied and how it helps to this energy range is often quite unfavorable (often less than 10 3 ), requiring careful examination of data. Therefore, one resolve practical problems specific to an experiment. of the problems in gamma-ray astronomy using air shower The methods described in this paper were developed for and techniques is to be able to determine the level of background. applied in two gamma-ray searches (Fleysher 2003a, 2003b) This problem is rather difficult if one tries to calculate the level using the Milagro water Cerenkov air shower detector (Atkins from first principles because it would require exact knowledge et al. 2000). The methods are universal and can be used in a of the details of the detector operation. Instead, the problem is wide variety of applications in which the object of investigation, steady or transient, point or extended, traverses the solved by measuring the background level using the same instrument. detector s field of view. In summary, xx 3, 5, 6, and 7 report Thus, in a typical experiment, two measurements are performed one the new contributions of this paper. corresponding to the observation of the candi- 2. LI MA STATISTIC date (so-called on-source observation) and the other the measurement of the corresponding background level (socalled off-source observation). Then the decision about the pothesis of the absence of a source given two independent Many statistical tests have been used to test the null hy- plausibility of the existence of the source is addressed by a counts N 1 from the on-source and N 2 from the off-source hypothesis test. regions accumulated during time periods t 1 and t 2, respectively, with all other conditions being equal. An improvement In x 2 of this paper we review the standard procedure of the significance calculation, which uses the statistic of Li & Ma was proposed by Li & Ma (1983) and is based on the test (1983); in x 3 we discuss the conditions of its applicability. We statistic show that the required approximations are not always justified in experiments. U ¼ N 1 N 2 p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; In x 4 we outline two of the most widely used methods of ðn 1 þ N 2 Þ ¼ t 1 =t 2 > 0: ð1þ background estimation: direct integration and time-swapping methods (Alexandreas et al. 1993). In x 5 we show that none of these methods are compatible with the Li & Ma (1983) significance formula. We then introduce a new self-consistent scheme for source detection and discuss some of its properties. Because each event carries no information about another, each of the observed counts can be regarded as being drawn from a Poisson distribution. It has been argued by the authors Li & Ma (1983) that for large N 1;2 ðn 1;2 > 10Þ, if the null 355

2 356 FLEYSHER ET AL. Vol. 603 hypothesis is true, the distribution of U becomes Gaussian with zero mean and unit variance. With this assumption, given ameasuredvalueuof U, the calculation of the p-value (which we denote by ) becomessimple: ¼ 1 Z þ1 pffiffiffiffiffi e x2 =2 dx 2 u when looking for a source and ¼ 1 Z u pffiffiffiffiffi e x2 =2 dx 2 1 when looking for a sink. The null hypothesis is rejected with significance c if < c. Because of the one-to-one correspondence between and u, the significance of a measurement can be quoted in the units of U. 3. CONDITIONS OF APPLICABILITY OF LI MA STATISTIC When the null hypothesis is true, the distribution of statistic U (eq. [1]) approaches the normal distribution in the limit of large numbers. Indeed, by substituting the factorial in the Poisson distribution using the Stirling formula n! ð2nþ 1=2 n n e n and using Taylor expansion in n in the vicinity of, one obtains (Fleysher 2003a) P ðnþ ¼ n n! e pffiffiffiffiffiffiffiffi 1 e ðn Þ2 =2 e ðn Þ3 =6 2 : 2 Thus, it is seen that the Poisson distribution approaches that of a Gaussian in a narrow region around its mean: ðn Þ 3 =6 2 T1 with 6. Substituting N1;2 for n and corresponding estimates of ¼ðN 1 þ N 2 Þt 1;2 =ðt 1 þ t 2 Þ for, we obtain the region around zero juj < ju b j where the distribution of statistic U is approximately normal. That is, for q 6 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi juj < ju b jt min 36ð1 þ Þ 2 ðn 1 þ N 2 Þ; q ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð1 þ Þ 2 ðn 1 þ N 2 Þ ð2þ the error on the p-value due to this approximation does not exceed 1= ð2þ 1=2R þ1 u b e x2 =2 dx. This calculation shows how large the numbers of detected events should be for the Gaussian approximation to be useful, which can be much greater than the value of 10 suggested by Li & Ma (1983). Figure 1 shows the results of Monte Carlo simulations for the distribution of the statistic U. It can be seen that the distribution is approximately normal in the vicinity of zero. By the same arguments it may be shown that within essentially the same region around zero, another statistic, U 0 ¼ N 1 N 2 p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; ð3þ N 1 þ 2 N 2 is also distributed normally. The motivation for the statistic U 0 is similar to that of statistic U (eq. [1]) in that the numerator may be interpreted as the excess number of events from the source over the expected background but the denominator is the maximum likelihood estimate of the standard deviation of the numerator given that the alternative hypothesis is true. (The alternative hypothesis in this case is that both observations N 1 and N 2 are from Poisson distributions with Fig. 1. Distributions of the statistic U (eq. [1]) obtained in two runs of Monte Carlo simulations (histograms). Curves are the best fit to a Gaussian distribution; parameters of the fit are listed in the box. The histograms and curves should agree in the u b -neighborhood around zero. The number of entries in each run (about 10 9 ) was chosen to provide reasonable accuracy in the region plotted. Top: Values of N 1 and N 2 drawn from a Poisson distribution with averages equal to 500 and 5000, respectively ( ¼ 0:1). According to eq. (2), u b T5:3. Bottom: Values of N 1 and N 2 drawn from a Poisson distribution with averages equal to and , respectively ( ¼ 0:1). According to eq. (2), u b T20:3. unrelated means.) Although this motivation appears to be incorrect and the statistic was abandoned by Li & Ma (1983), the critical range of U 0 maybedefinedasu 0 > u0 0 for testing the null hypothesis against the presence of a source and u 0 < u0 0 for testing against the presence of a sink. Because under the conditions of applicability of the Li Ma statistic (eq. [2])

3 No. 1, 2004 SIGNIFICANCE TESTS AND BACKGROUND ESTIMATION 357 statistic U 0 is distributed normally, the p-value calculation is identical to that of the statistic U. Figure 2 presents the results of Monte Carlo simulations of the distribution of statistic U 0. Thus, both statistics U and U 0 have the same conditions of applicability. Depending on the situation, application of one of them may be easier than the other (see xx 5 and 5.1). In general, a hypothesis test may be based on any statistic if its distribution under the null hypothesis is known. 4. BACKGROUND ESTIMATION, ISOTROPY, AND STABILITY ASSUMPTIONS In a typical gamma-ray experiment on- and off-source observations can be performed at the same time by utilizing the wide field of view of a detector, or they can be performed at different times by making measurement in the same local directions of the field of view. (Because of the Earth s rotation, the off-source bin may present itself in the directions of local coordinates previously pointed at the source bin.) Both of these stipulations could contradict the conditions of being equal : if observations are done at the same time, then nonuniformity in the acceptance of the array to air showers due to detector geometry must be compensated for; if observations are done at different times, then any time variation in detector operation must be addressed. Under these varying conditions, the meaning of the parameter must be changed to the effective ratio of exposures of the bins. The mechanism of such an equalization and -determination is called background estimation. The name is due to interpretation of the second term of the numerator of equation (1) as the expected number of background events in the source region: N b N 2.Correspondingly, the number of events N 1 obtained from the direct source observation will be denoted N s N 1. A widely accepted method of background estimation (Alexandreas et al. 1993) recognizes that usually no major changes in the detector configuration are made on short timescales. It also recognizes that most air showers detected are produced by charged cosmic rays, which can be regarded as forming isotropic background radiation. Detector configuration stability implies that the acceptance of the detector is time independent, although variations in the overall rate of detected events are allowed. (An example of such rate variations could be an event rate decrease caused by a temporary data acquisition system overload.) Therefore, the average number of detected events as a function of local coordinates x and time t on the short timescale can be written in the form dnðx; tþ ¼ GðxÞRðtÞ dx dt: ð4þ Here RðtÞ is the overall event rate, GðxÞ acceptance of the array such that R FOV GðxÞ dx ¼ 1, both to be estimated from the data. The local coordinates x could be either hour angle and declination or zenith and azimuth. The average number of background events expected in the source bin is then given by Z Z N b ¼ ½1 ðx; tþš GðxÞ RðtÞ dx dt; ð5þ Fig. 2. Distributions of the statistic U 0 (eq. [3]) obtained in two runs of Monte Carlo simulations similar to that presented in Fig. 1. where ðx; tþ is equal to zero if x and t are such that they translate into inside the source bin and equal to 1 otherwise. The isotropy and stability assumptions (eq. [4]) become part of the null hypothesis being tested. Extending the time integration window in equation (5) is equivalent to increasing exposure to the off-source bin, which leads to a decrease in the value of and improved sensitivity. Assumption (4), however, must hold during the entire integration period, placing a constraint on the maximum size of the off-source bin. (See x 7 for an example method of detector stability duration determination.) The time integration window is limited by 24 hr of sidereal day. The fluctuations in N b are dominated by the ones in GðxÞ because the event rate RðtÞ is collected from the entire sky and may be deemed as known to high precision. The significance test is based on either statistic (1) or (3) Direct Integration Method The direct integration method of source detection is the method in which the integration of equation (5) is performed

4 358 FLEYSHER ET AL. Vol. 603 numerically by discretizing both GðxÞ and RðtÞ on a fine grid and replacing integrals by sums. The acceptance and the event rate are estimated by histogramming local coordinates x and event times t of the events collected during the integration time period from the entire sky and normalizing GðxÞ. Inthis scheme, the source region defined by ðx; tþ also gets discretized; therefore, the source count N s must be obtained by using the same discretized definition of the source region Time-swapping Method In the time-swapping method of source detection the integration of equation (5) is performed by means of Monte Carlo methods, which leads to N b ¼ N 0 N X N i¼1 ½1 ðx i ; t i ÞŠ ¼ 1 X N i¼1 ½1 ðx i ; t i ÞŠ; ð6þ where N generated events ðx i ; t i Þ are distributed according to joint probability density GðxÞ RðtÞ with RðtÞ ¼RðtÞ=N 0, N 0 being the total number of events detected during the integration time window. A list of all coordinates of the detected events is regarded as a sample from the GðxÞ distribution, while a similar list of all times is regarded as the one from RðtÞ. Therefore, a sample from the GðxÞ RðtÞ distribution can be generated from the data by randomly associating an event s local coordinate x with another event s time t among the pool of detected events. In practice, Monte Carlo integration is performed by substituting each real event s arrival time with a new time from the list of registered times of collected events in a finite time window. This is why the method is referred to as the time-swapping method. The swapping is repeated times per each real event, typically being around 10. The accuracy of the method approaches that of direct integration when is increased. Accounting for the number of swaps is not discussed in the literature (we are aware that this effect was accounted for but not discussed in Alexandreas et al. 1991a); we present our calculation of this effect in x 5.1. In the time-swapping method, the function ðx; tþ defining the source region does not have to be discretized as it had to be in the direct integration method. 5. SOURCE REGION EXCLUSION METHOD The standard realization of the direct integration and timeswapping methods described includes on-source events N s in the calculation of expected background N b [via GðxÞ and RðtÞ]. This, however, is inconsistent with their independence required by the Li Ma statistic (eq. [1]) and was already recognized but not addressed by Alexandreas et al. (1993). This problem has been ignored in its applications (see, e.g., Cassiday et al. 1989; Alexandreas et al. 1991a; Atkins et al. 1999; Wang et al. 2001). The incompatibility of the standard methods of background estimation with those of the standard significance calculation is illustrated in Figure 3 (top), where results of the computer simulations for a Galactic plane observation are presented. Detection of an extended source such as the Galactic plane presents an additional difficulty because the ratio of on- and off-source exposures varies dramatically over the area of the source, which is not tracked by the standard background estimation methods. The figure shows the excess number of events ðn s N b Þ extracted from a simulated Galactic signal as a function of Galactic latitude by using standard implementation of the time-swapping method (Alexandreas et al. 1993). Fig. 3. Excess number of events ðn s N b Þ as a function of Galactic latitude obtained in Monte Carlo simulations with uniform Galactic signal flux being that of background in the region of 5 around the Galactic equator. The expected Galactic bin content is about 205,000. Top, Standard time-swapping implementation is used (Alexandreas et al. 1993); bottom, exclusion method from this paper is used, where the region of 7 around Galactic equator is excluded from the background estimation. Use of the standard methods would lead to a 25% loss in both the excess number of events and in the value of the statistic U. The excess is recovered correctly by the exclusion method proposed here (Fig. 3, bottom). In order to be able to use either of the statistics (1) or (3) the events from the source bin should be excluded from the background estimation. However, simply removing all of these events from the procedure will destroy its fundamental assumption that the lists of local coordinates and times

5 No. 1, 2004 SIGNIFICANCE TESTS AND BACKGROUND ESTIMATION 359 represent samples from GðxÞ and RðtÞ, respectively. A solution to this problem follows. Denote by ðx; tþ a function similar to ðx; tþ that defines the region of the sky from events which are to be excluded from the background estimation. The excluded region should contain the candidate source bin but need not be limited to it. Also denote by N out ðx; tþ the number of detected events originating from outside the excluded region, and by R out ðtþ their total event rate; then it is readily seen that dn out ðx; tþ ¼ ðx; tþgðxþrðtþ dx dt: By integrating this equation with respect to t and x, a system of equations on unknown GðxÞ and RðtÞ is obtained [N out ðxþ and R out ðtþ are available experimentally]: ( N out ðxþ ¼ GðxÞ R ðx; t 0 Þ Rðt 0 Þ dt 0 R out ðtþ ¼ RðtÞ R ðx 0 ; tþ Gðx 0 Þ dx 0 : ð7þ The numerical solution of these integral equations provides RðtÞ and GðxÞ basedondatan out ðxþ and R out ðtþ from outside theexcludedregiontobeusedinequation(5).thesituationis illustrated on Figure 4. The heavily shaded area is the outside of the excluded region bounded by ðx; tþ in its discrete form, events from which may be used for the off-source observation. The region of interest, the on-source region, is defined by some other conditions ðx; tþ that are irrelevant for the background equations (7) as long as it is contained in the excluded region. It can be noted that both GðxÞ and RðtÞ enter into equations (7) and (5) only as a product GðxÞRðtÞ; therefore, normalization of either of them does not make any difference as long as the product is preserved. In addition, if there are points fx 0 g in the local coordinates that are always inside the excluded region, then acceptance Gðfx 0 gþ will not be determined. This may happen if the detector was operational during a short time period and/or the excluded region was large. On-source events with local coordinates from these regions must be discarded as having no corresponding background estimate. It is thus seen that the method entails that the second, off-source region be defined by the regions of the local sky that have the opportunity to present themselves into the directions of the source region because of the Earth s rotation during the time period of integration. Different parts of the source region have different corresponding off-source regions. This leads to the ratio of exposures of on- and off-source regions that is dependent on the local coordinates x: R ½1 ðx; tþšrðtþdt R : ðx; tþrðtþdt ðxþ ¼ N bðxþ N out ðxþ ¼ The off-source region corresponding to a given on-source region ðx; tþ is not a celestial bin, it is a set of local directions with ðxþ > 0. Because the measurements made from different local directions x are independent, all measurements can be combined to obtain the compound statistic U: P x U ¼ N sðxþ P x N bðxþ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P x ðxþn sðxþþ P x N bðxþ N s N b U ¼ p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P x ðxþn : sðxþþn b The described method is the integration scheme, which is based on the direct integration method and which properly Fig. 4. Illustration of the source region exclusion method. See x 5 for a description of the method.

6 360 FLEYSHER ET AL. Vol. 603 estimates the ratio of exposures ðxþ and accounts for the source events Accounting for Number of Swaps in the Time-swapping Method In the same way, in the time-swapping method (eq. [6]) the acceptance GðxÞ and event rate RðtÞ must be solutions of the equations (7) to account for on-source events. The event rate RðtÞ is saved as a histogram, and generated event times are drawn from it. The sample from GðxÞ is generated by using events from outside the excluded region and should contain NðxÞ ¼GðxÞ R RðtÞ dt events with given local coordinates x. However, the number of events available is N out ðxþ ¼ GðxÞ R ðx; tþrðtþ dt. Therefore, instead of swapping each event times, missing events are created by choosing the actual number of swaps from a Poisson distribution with parameter ½1 þ 0 ðxþš, where 1 þ 0 ðxþ ¼ R RðtÞ dt : GðxÞ R RðtÞ dt GðxÞ R ðx; tþrðtþ dt ¼ GðxÞ N out ðxþ The significance calculation has to reflect the fact that the time-swapping method is a Monte Carlo integration and thus introduces additional fluctuations in the estimate of N b.the integration error decreases as the number of generated events increases, or equivalently as increases and the fluctuations in N b approach that of the direct integration method. The use of the statistic U 0 (eq. [3]) provides a transparent way of including these additional fluctuations. It can be shown (Fleysher 2003b) that the statistic U 0 within the framework of time swapping must be substituted by U 0 ðxþ ¼ N s N b p ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi N s þ P x ðxþn ; ð8þ bðxþþn b = ðxþ ¼ N bðxþ N out ðxþ : The fact that the source region defined by ðx; tþ does not have to be discretized is the advantage of the time-swapping method. 6. KNOWN ANISOTROPIES AND KNOWN DETECTOR INSTABILITIES It was assumed in the above discussion that no anisotropy on the sky is present. This, together with the stability assumption, had led to equation (4). In fact, if there are known anisotropies on the sky, then the number of registered events is given by dnðx; tþ ¼½1 þ Sðx; tþšgðxþrðtþ dx dt; where Sðx; tþ describes the strength of the sources as a function of local coordinates and time. It has to be recognized that the presence of sources or large-scale anisotropies on the sky and the instability of the detector may mimic each other; thus, in general, the function Sðx; tþ describes both phenomena. Incorporation of the improved stability assumption (9) into the framework of the direct integration method is straightforward: Z Z N b ¼ ½1 ðx; tþš½1 þ Sðx; tþšgðxþrðtþ dx dt: ð9þ The anisotropy function Sðx; tþ must be discretized on the same grid as GðxÞ and RðtÞ are. In order to incorporate the improved stability assumption into the framework of the timeswapping method, the generated events ðx i ; t i Þ must represent a sample from ½1 þ Sðx; tþšgðxþrðtþ, which can be achieved with the help of the rejection method (Press et al. 1994). Handling the existing localized sources and sinks for which the function Sðx; tþ is not available is possible with the proposed exclusion method. The examples are the Crab Nebula, which is known to emit gamma rays in the TeV energy region (Weeks et al. 1989), and the Sun and the Moon, two known cosmic-ray sinks on the sky (Alexandreas et al. 1991b). Not only do the latter present a source of anisotropy, they also traverse the sky, on their way blocking potential candidates and perturbing the on-source count N s,aswellasn b.these can be handled by vetoing certain size regions around the objects, that is, treating them as part of the excluded region during integration (eqs. [5] and [7]) and disregarding events if they fall within the veto region when counting on-source events N s.inotherwords,ifðx; tþ is the function describing the veto region where it is equal to zero and equal to 1 everywhere else, then the excluded region ðx; tþ and source region ðx; tþ have to be redefined as ( ðx; tþ ðx; tþðx; tþ ðx; tþ 1 ½1 ðx; tþšðx; tþ : In general, existing localized sources and sinks can be excluded or vetoed as described; known large-scale anisotropies and detector instability have to be incorporated into the stability assumption. These will become a part of the null hypothesis being tested. 7. DURATION OF VALIDITY OF DETECTOR STABILITY ASSUMPTION For the purpose of background estimation it is desirable to maximize the time integration window over which the background is calculated; however, the timescale during which the stability assumption (4) holds must first be ascertained. For example, even if no reconfigurations to the detector on the short timescale are made, the acceptance of the array GðxÞ depends on transmission properties of the atmosphere, which may vary. Thus, the atmosphere must be considered an integral part of the detector, and we refer to the phenomenon in general as detector instability. A test of the stability assumption would be a comparison of two acceptances G i ðxþ and G j ðxþ, wherei 6¼ j, measured at different times t i and t j. Such a test can be implemented as a series of 2 tests of G i ðxþ and G j ðxþ [yielding 2 ðt i ; t j Þ and then obtaining the combined 2 tot ðtþ for time separation t ¼ t i t j ]: 2 tot ðtþ ¼ X 2 ðt i ; t j Þ: t¼t i t j The test statistic 2 tot ðtþ so obtained follows a 2 distribution with m tot ðtþ ¼ P t mðt i; t j Þ degrees of freedom, where t ¼ t i t j, if observed differences are of random nature only. Here mðt i ; t j Þ are the number of degrees of freedom in the corresponding 2 ðt i ; t j Þ tests. By examining the dependence of 2 tot =m tot on the time separation t it is possible to test the detector stability assumption and to ascertain the proper integration time window. The duration of the validity of the detector stability assumption is given by the T stb ¼ t

7 No. 1, 2004 SIGNIFICANCE TESTS AND BACKGROUND ESTIMATION 361 at which the test statistic 2 tot =m tot begins to deviate significantly from the expected 1.0. If detector instability is recognized, the stability assumption (4) must be replaced by assumption (9), where Sðx; tþ describes the measured or known variations, or the time integration window must be shortened Illustration with Diurnal Modulations in Milagro Detector In this section we illustrate the use of the stability assumption test for the Milagro data, determine atmospheric variations, and show how they are incorporated into improved stability assumption. (For a description of Milagro, please see Atkins et al ) Figure 5 is an example of the results of the detector stability assumption test using Milagro data with regard to the zenith coordinate. It is seen from the plot that the degree of violation of the assumption grows with time separation t almost immediately, which indicates the need for determination of atmospheric variations Sðx; tþ, but then it drops before growing again. This can be interpreted as the presence of a periodic component that ensured that two acceptances separated by 24 UT hr are closer to each other than, say, those separated by only 12. Thus, despite the fact that no human intervention on the short timescale is made, the acceptance of the detector changes. Because the diurnal periodicity is noted, the investigation of the modulation can be performed by comparing a particular distribution with its daily average. It was observed that the shape of the modulation [ðzþ] of the zenith distribution is approximately constant, with amplitude varying from halfhour to half-hour. Therefore, the modulation model to be used in equation (9) is chosen to be of the form Sðx; tþ ¼ðtÞðzÞ; ð10þ Fig. 6. Example of the zenith correction function ðzþ derived from Milagro data. where ðtþ is the amplitude of the correction at time t; ðzþ is the polynomial zenith angle correction function, coefficients of which are obtained from the modulation shape study. The example of the correction function is shown in Figure 6. The example of the average daily amplitude dependence is shown in Figure 7. The value of the amplitude is typically within the range. The plot can also be used to justify the choice of half-hour intervals for the amplitude measurement. The diurnal modulation model (10) becomes part of the null hypothesis. Fig. 5. Results of stability assumption test with regard to zenith coordinate using Milagro data. Horizontal axis is time separation t with 30 minute bins; vertical axis is corresponding 2 tot ðtþ=m totðtþ. Solid horizontal line is the expected value of 1 if the stability assumption holds. Fig. 7. Example of the average daily dependence of the zenith correction amplitude derived from Milagro data.

8 362 FLEYSHER ET AL. 8. CONCLUSIONS We have considered a typical air shower measurement conducted by means of two observations and have discussed two commonly used tests (based on statistics U and U 0 ) and the conditions of their applicability. A careful look at the situation in which an astrophysical object traverses the large field of view of a detector had led us to the subject of background estimation. We have developed the source region exclusion method of background estimation, which is consistent with the use of either statistic U or U 0, and have discussed two implementations of it: direct integration and time swapping. The background estimation method is based on widely adopted assumptions of short-timescale stability of the detector operation and that of isotropy of the cosmic-ray background. We have discussed a way to test and to relax the short-timescale stability assumption and used Milagro data to illustrate the situation in which the presence of zenith diurnal modulations can easily be incorporated into the background estimation method. More generally, this is also the way to incorporate known large-scale anisotropies. The effect of localized sources and sinks does not have to be known; existing ones can be handled by excluding or vetoing the regions around them. While the methods and ideas presented in this paper were developed for a gamma-ray air shower array, the methods can also find their applications outside the field of gamma-ray astronomy. The properties of the significance test can be useful for any counting-type experiment in which the number of events follows a Poisson distribution; the background estimation method can be used with any large field of view detector, for which the object of investigation traverses the field of view, such as in solar neutrino monitors, or is transient, such as in supernova neutrino observatories. We would like to thank the Milagro collaboration for permitting us to use Milagro data for the illustration of the zenith diurnal modulation and for their help. This work is supported by the Department of Energy Office of High Energy Physics, the National Science Foundation (grants PHY , PHY , PHY , PHY , and PHY ), the LDRD program at Los Alamos National Laboratory, Los Alamos National Laboratory, and the University of California, Institute of Nuclear and Particle Astrophysics and Cosmology. Alexandreas, D. E., et al. 1991a, ApJ, 383, L b, Phys. Rev. D, 43, , Nucl. Instrum. Methods Phys. Res. A, 328, 570 Atkins, R., et al. 1999, ApJ, 525, L , Nucl. Instrum. Methods Phys. Res. A, 449, 478 Cassiday, G. L., et al. 1989, Phys. Rev. Lett., 62, 383 Fleysher, L. 2003a, Ph.D. thesis, New York Univ. (astro-ph/ ) REFERENCES Fleysher, R. 2003b, Ph.D. thesis, New York Univ. (astro-ph/ ) Li, T., & Ma, Y. 1983, ApJ, 272, 317 Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. 1994, Numerical Recipes (Cambridge: Cambridge Univ. Press) Wang, K., et al. 2001, ApJ, 558, 477 Weeks, T. C., et al. 1989, ApJ, 342, 379