SPATIAL BOOTSTRAP WITH INCREASING OBSERVATIONS IN A FIXED DOMAIN

Transcription

1 Statistica Sinica 1(00), 7- SPATIAL BOOTSTRAP WITH INCREASING OBSERVATIONS IN A FIXED DOMAIN Ji Meng Loh and Michael L. Stein Columbia Univesity and Univesity of Chicago Abstact: Vaious methods, such as the moving block bootstap, have been developed to esample dependent data. These contain featues that attempt to etain the dependence stuctue in the data. Asymptotic esults fo these methods have been based on inceasing the numbe of obsevations as the obsevation egion inceases, combined with estictions on the ange of dependence of the data. In this wok, we conside esampling a one-dimensional Gaussian andom field obseved on a egula lattice, with inceasing numbe of obsevations within a fixed obsevation egion. We show a consistency esult fo esampling the vaiogam when the undelying pocess has a vaiogam of the fom γ(t) = θt + S(t), S(t) Dt. Fo a class of smoothe pocesses, specifically, fo a pocess with vaiogam of the fom γ(t) = βt + θt 3 + R(t), R(t) Dt, we povide a simila esult when esampling the second-ode vaiogam. We pefomed a simulation study in one and two dimensions. We used the Matén model fo the covaiance function with vaying values of the smoothness paamete ν. We find that the empiical coveage of confidence intevals of the vaiogam appoaches the nominal 95% level as the numbe of obsevations inceases fo models with small ν, but as ν is inceased the empiical coveage deceases, with the coveage becoming significantly lowe than the nominal level when ν 1. When we conside the second-ode vaiogam, we find that the empiical coveage appoaches the nominal level fo lage values of ν than fo the fist-ode vaiogam, with the empiical coveage noticeably lowe than the nominal level only when ν is about. Key wods and phases: Fixed domain asymptotics, Gaussian pocesses, spatial bootstap, vaiogam. 1. Intoduction The spatial bootstap has been studied by a numbe of eseaches, Davison and Hinkley (1997) give a bief oveview of methods. Hall (195) appeas to have been the fist to use some kind of a block esampling pocedue to bootstap spatial data. Künsch (199) intoduced the block-esampling bootstap to esample one-dimensional pocesses fo finding the sample mean of weakly stationay obsevations. With N obsevations in a line, he suggested foming N n + 1 blocks, each consisting of n consecutive obsevations, and sampling fom these blocks with eplacement. Lahii (199) consideed the

2 JI MENG LOH AND MICHAEL L. STEIN second ode popeties of the moving block bootstap pocedue of Künsch (199) and showed that it was second-ode coect unde appopiate conditions. Liu and Singh (199) independently poposed a simila method in a pape on bootstapping m-dependent data, showing consistency of estimates unde m- dependence if the size of the blocks b as the sample size N, with b/n 0. Politis and Romano (199a) developed the method futhe and suggested wapping the data, which we call tooidal wapping, befoe pefoming block esampling. Shi and Shao (19) and othes poposed vaious vesions of the spatial bootstap. Politis and Romano (199) studied subsampling, which involves calculating estimates using egions that ae smalle than the obsevation window, and then escaling the estimates. In his pesentation of the block bootstap, Künsch (199) povided a way to extend the method to a block of blocks bootstap method. See also Politis and Romano (199b) and Bühlmann and Künsch (1995). Loh and Stein (00) intoduced a vey simila method, called the maked point bootstap, fo esampling point pocesses whee maks ae computed and assigned to obseved points befoe pefoming the bootstap. The maks ecod the contibutions made by neighboing points to the estimate of inteest. By using the maks in the calculation of bootstap estimates, the method aims to etain some of the stuctue inheent in the obsevations, that might othewise be patially destoyed by the egula block esampling methods. Theoetical esults that have been obtained ae dependent on the undelying pocess exhibiting only shot ange dependence. Lahii (1993) showed that with longe ange dependence pesent, the block esampling method begins to fail, since putting independent blocks togethe to geneate the bootstapped estimate destoys the long-ange dependence pesent in the oiginal obsevations. Futhemoe, these esults conside inceasing domain asymptotics, whee the obsevation egion o domain inceases as the numbe of obsevations inceases. In this wok, we focus on fixed domain asymptotics and deive consistency esults fo the block-of-blocks bootstap in two specific cases. This bootstap method was intoduced by Künsch (199) and is biefly descibed in Section. It is well known that estimates fo dependent data need not be consistent when the domain is fixed (Stein (1999)). We cannot expect esampling methods to wok well fo estimates that ae not consistent. Thus ou esults ae necessaily esticted to cetain models and estimated quantities. We conside only Gaussian pocesses, and fo simplicity estict theoetical esults to one dimension. In Section 3, we wok with the vaiogam γ defined by γ(t) = Va[Z(s i + t) Z(s i )] fo distance t 0, whee Z(s) epesents a Gaussian pocess obseved at point s. In paticula we conside t = hl/n whee h is a fixed positive intege, L the

3 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 9 length of the obsevation egion and N the numbe of obsevations. We show a consistency esult fo esampling the vaiogam γ of a pocess that has vaiogam γ(t) = θt + S(t), whee S(t) Dt, and has a bounded second deivative on (0, L]. Fo a smoothe pocess, highe ode diffeences can emove the long ange dependence inheent in the pocess (Kent and Wood (1997) and Istas and Lang (1997)). In Section, we conside a model with vaiogam γ(t) = βt +θt 3 +R(t), whee R(t) Dt and has a bounded fouth deivative on (0,L]. This vaiogam is popotional to t nea the oigin and the pocess is exactly once mean squae diffeentiable. We show a consistency esult fo the block-of-blocks bootstap when we conside the second-ode vaiogam, defined as φ(t) = Va[Z(s i + t) Z(s i ) + Z(s i t)]. (1) Geneal asymptotic esults fo bootstap of dependent data equie the obsevation domain to incease as the numbe of obsevations incease, togethe with some mixing condition fo the pocess. This ensues that the esampling scheme poduces eplicates that ae asymptotically independent, identically distibuted. Fo the case of inceasing obsevations in a fixed obsevation domain, thee is no geneal analogue to mixing conditions. Futhemoe, it is well known (e.g., Stein (1999)) that quantities associated with the pocess often cannot be consistently estimated. Since andomly selected blocks emain close to each othe if the domain has fixed size, it cannot be expected that a geneal asymptotic esult can be poven in the fixed domain setting. Howeve, thee ae some quantities elated to the andom pocess that can be consistently estimated. Fo example, Ying (1991) showed that fo the pocess with vaiogam γ(t) = a[1 exp( θt)], aθ can be consistently estimated. A quantity that can be ecoveed with pobability one fom obsevations in a bounded egion is called micoegodic (Matheon (1971, 199) and Stein (1999)). Fo micoegodic quantities, it may be possible that bootstap estimates of standad eos ae consistent unde the fixed domain setting. We ae not awae of any pevious esults demonstating the validity of bootstapping unde fixed domain asymptotics. The esults hee exploit the fact that fist-ode and non-ovelapping second-ode diffeences of the espective pocesses we conside ae nealy independent. Whethe some vesion of the bootstap will wok fo moe geneal micoegodic quantities is unknown. To investigate how well these vaious bootstap pocedues wok in pactice, and to investigate thei potential validity fo pocesses in two dimensions, we simulated Gaussian pocesses on the unit inteval and the unit squae, and examined the empiical coveage of nominal 95% confidence intevals fo the

4 70 JI MENG LOH AND MICHAEL L. STEIN fist- and second-ode vaiogams. We used the Matén model with covaiance function given by C(t) = σ ν 1 Γ(ν) ( ν 1 t ρ ) νkν ( ν 1 t ρ ), () whee Γ is the Gamma function and K ν is a modified Bessel function. The paamete ρ is elated to the ange of the coelation in the pocess while ν is a smoothness paamete, so that the pocess is exactly m times mean squae diffeentiable if m < ν < m + 1. We consideed values of ν =,,...,.0 and ρ = 0.01 and We found that, fo the fist-ode vaiogam, the empiical coveage of confidence intevals appoaches the nominal level as the numbe of obsevations inceases fo models with small ν. We see a gadual dop in empiical coveage as ν is inceased, with the coveage becoming substantially lowe than the nominal level when ν 1. With the second-ode vaiogam, we find that the empiical coveage of confidence intevals appoaches the nominal level with inceasing numbe of obsevations fo lage values of ν, with the empiical coveage becoming noticeably lowe than the nominal level only when ν is about. Section 5 gives details of ou simulation pocedue and findings.. The Block-of-Blocks Bootstap Method As an extension to the block bootstap method fo esampling one-dimensional pocesses, Künsch (199) intoduced the block-of-blocks bootstap in ode to educe the effect of joining independent blocks. We descibe the pocedue below, using a slightly diffeent fomulation fom that in Künsch (199). Suppose Z is a andom field with N obseved values Z(s i ) at points s i,i = 1,...,N, on a egula lattice of length L. Suppose futhe that a quantity of inteest θ can be estimated via a statistic that can be expessed in the fom ˆθ = i Y i (Z). The quantity Y i (Z) can include obsevations of Z othe than at s i, but geneally only obsevations that ae nea to s i. Thus each point s i has a quantity Y i (Z) associated with it that is a function of values of the pocess Z nea s i. The bootstap pocedue consists of fist assigning maks Y i (Z) to each point s i. The points ae then esampled, fo example, by deciding in advance the length and numbe of sampling intevals to use, then picking andomly stating positions fo these intevals. The obsevation points coveed by these andomly placed intevals fom the bootstap sample. Often the total length of the sampling intevals is chosen to be equal to the actual length of the obsevation egion,

5 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 71 although this does not necessaily have to be the case. With the esampled points s 1,...,s N, the bootstap estimate of θ is given by ˆθ = i Y i (Z). (3) Note that Yi (Z) is the mak computed fom the oiginal obsevations of Z. This is not equal to Yi (Z ), which would be what is used if the esampled points s i wee put back on a lattice and a new estimate computed diectly fom the new sample. 3. Resampling the Vaiogam in a Fixed Domain Hee we povide a consistency esult fo the block-of-blocks bootstap when we esample fo the vaiogam of a pocess Z with vaiogam γ(t) = θt + S(t), whee S(t) Dt and has a bounded second deivative on (0,L]. Specifically, suppose Z is obseved at N equally spaced points in the inteval (0, L], h fixed. The obsevation points ae denoted by s 1 = L/N,s = L/N,...,s N = L. Fo fixed intege h > 0, we conside estimating the quantity γ(hl/n) = Va[Z(s i+h ) Z(s i )]. We wite Z(s i+h ) as Z i+h fo shot. Note that as N inceases, we ae estimating the vaiogam at smalle and smalle distances, specifically the distance between points that ae h numbe of neighbos away. An unbiased estimate of γ(hl/n) is given by ( hl ) ˆγ = N N h 1 (N h) i=1 (Z i+h Z i ) = 1 N h N h i=1 Y i, () whee Y i = (Z i+h Z i ). Each point s i,i = 1,...,N h is assigned the mak Y i. Set M = N h. We conside beaking the inteval (0,L h] into B blocks, whee M/B is an intege, and esampling the points by sampling these B blocks with eplacement. In block bootstap teminology, we ae using fixed athe than moving blocks. It is conceptually staightfowad to extend this to the moving block case. Note that the use of blocks hee is only to select the points that will be included in the new sample. In computing the bootstap estimate, the maks Y i computed fom the eal sample ae used. In the usual block bootstap, the Z s coesponding to the esampled points ae used to compute new values of Y i to give the bootstap estimate. It is this new computation of the Y i s that affects the dependence stuctue. Note also that we do not esample points N h + 1 to N, since these points have no maks. The obsevations at these points ae used in the estimate, howeve, since they ae pat of the maks assigned to othe points.

6 7 JI MENG LOH AND MICHAEL L. STEIN Fo the k-th block, k = 1,...,B, let ˆγ k = B M M B Y k,i, i=1 whee Y k,i is the i-th value of Y in the k-th block. Thus ˆγ k is the aveage of the Y values in the k-th block. If B blocks ae dawn with eplacement, the bootstap estimate of γ(hl/n) is given by ˆγ = 1 B B ˆγ j = 1 M j=1 B M B j=1 i=1 Y j,i. Hee, ˆγ j denotes the aveage of the Y values of the j-th sampled block, and Yj,i the actual Y values of that block. Let U j,j = 1,...,B, be independent with P(U j = ˆγ k ) = 1/B,k = 1,...,B. Thus ˆγ = U j /B and E(U j Z) = ˆγ. Define ˆσ as ˆσ = Va(ˆγ Z) = 1 B B (ˆγ k ˆγ) = 1 B ( B Bˆγ k We then have the following. k=1 k=1 B ) ˆγ j. (5) j=1 Theoem 1. With ˆγ and ˆσ with defined in () and (5) espectively, ˆγ is unbiased Va(Nˆγ) = θ L h(h + 1) + O(N ), 3N () N[E(N ˆσ ) Va(Nˆγ)] = O(B 1 ) + O(BN 1 ), (7) Va(N ˆσ ) = O(B N 1 ). () Poof. In what follows we suppess the tem hl/n and wite ˆγ and γ to denote ˆγ(hL/N) and γ(hl/n) espectively. It is clea that the estimato ˆγ is unbiased. Futhemoe, M M Va(Mˆγ) = Va(Y i ) + Cov (Y i,y j ) i=1 = Mγ + i<j M Cov (Z i+h Z i,z j+h Z j ), (9) i<j

7 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 73 using the fact that Z i+h Z i N(0,γ) and the esult that fo (U,V ) bivaiate nomal, Cov (U,V ) = Cov (U,V ). Fo i,j such that j i = k 0, C k Cov(Z i+h Z i,z j+h Z j ) (k + h)l h k L = [γ( ) + γ( N N ) γ(kl N )] 1 { O(N ) if k h, = θl(h k) N + O(N ) if k < h. Putting the expessions fo γ and C k into (9) gives (). To find E(N ˆσ ) we fist define k = Bˆγ k B j=1 ˆγ j. We note that E( k ) = 0, so that E(N ˆσ ) = N B Fom the expession () B Va( k ). (11) k=1 M B 1 = (B 1)(Y Y M ) (Y M B B Y M), (1) we have M ( ) M Va B 1 = M(B 1)Va(Y 1 ) + (B 1) + M M B 1 l=1 (M MB 1 ) C l [ M M M B (B 1) lcl + B l=1 l= M B +1 B 1 l=1 M B C l + ( ) M B l Cl M 1 l=m M B +1 (M l)c l ], (13) whee C l is as defined in (). In the ight-hand side of (13), the second and thid tems coespond to coss-tems within the espective backeted tems in (1), and the last tem coesponds to coss-tems between tems in the two backets in (1). Reaanging the tems and noting that fo l h, the sum of the coefficients of C l is O(M ) while C l is O(N ), we have ( ) M Va B 1 h 1 = M(B 1)Va(Y 1 )+ +O(N ), l=1 {BM M (B B+1)l}C l = MBθ L h(h + 1)(3N ) 1 + O(B N ) + O(N 1 ).

8 7 JI MENG LOH AND MICHAEL L. STEIN Expessions fo M k /B,k =,...,B 1, ae the same as the above, diffeing only in the smalle ode tems. Putting these into (11) yields E(N ˆσ ) = θ L h(h + 1) 3N which in tun gives (7). Next, we have Va(ˆσ ) = 1 B B Va( k ) + B k=1 = 1 B B k=1 + O(N 1 B 1 ) + O(BN ), B k,l:k<l Va( k ) + [ B 1 B B + k=1 Cov( k, l ) Cov ( k, k+1 ) Cov ( k, k+ ) + + Cov ( 1, B ]. ) (1) k=1 The quantities Va( k ),Cov ( k, k+1 ) and Cov ( k, k+l ) fo l =,...,B 1, have tems of the fom Cov(Y i Y j,y u Y v ). Fist conside the case whee at least one of {u,v} is less than h away fom {i,j}. The covaiance tems Cov (Y i Y j,y u Y v ) can be classified into fou types, epesented by Cov (Y i Y i,y i Y i ), Cov (Y i Y i, Y i Y j ), Cov (Y i Y j,y i Y k ) and Cov(Y i Y j,y i Y j ), whee i denotes integes such that i i < h, and similaly fo i,i. These ae in tun bounded by Cov (Yi, Yi ), Cov(Y i,y iy j ), Cov(Y i Y j,y i Y k ) and Cov (Y i Y j,y i Y j ) espectively, whee each is O(N ). Using (1) and simila expessions fo M k /B, we can find the sums of the coefficients of each type fo Va(M k /B ),Cov (M k /B,M k+1 /B ) and Cov (M k /B,M k+l /B ) fo l =,...,B 1. Fo Va(M k /B ), the sum of the coefficients coesponding to Cov (Y i Y i, Y i Y i ), Cov(Y i Y i,y i Y j ), Cov(Y i Y j,y i Y k ) and Cov (Y i Y j,y i Y j ) ae, espectively, O(MB), O(M B ), O(M 3 B) and O(M ). Similaly, these coefficients ae O(MB + B ), O(MB 3 + M B), O(M 3 + M B ) and O(M ) fo Cov (M k /B,M k+1 /B ), and O(MB), O(M B), O(M 3 ) and O(M ) fo Cov (M k /B, M k+l /B ), l =,...,B 1. Thus the contibution of these tems to (1) is O(N 5 B ). Repeating the same agument, we find that the contibution of tems Cov (Y i Y j,y u Y v ) whee both {u,v} ae both at least h away fom {i,j} add up to O(N B ). Combining these findings gives (). Togethe with the asymptotic nomality of ˆγ (Kent and Wood (1997)), we can futhe show asymptotic nomality of (ˆγ γ)/ˆσ as B inceases with N, N

9 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 75 inceasing slightly faste than B: ˆγ γ ˆσ = ˆγ γ Va(ˆγ) ( Va(Nˆγ) N ˆσ )1 = ˆγ γ Va(ˆγ) [1 + O p (N 1 B 1 ) + O(BN 1 )] 1 (using Theoem 1) = ˆγ γ Va(ˆγ) + O p (N 1 B 1 ) + O p (BN 1 ), whee (ˆγ γ)/ Va(ˆγ) is asymptotically standad nomal (Kent and Wood (1997)). Specifically, (ˆγ γ)/ˆσ is asymptotically nomal if we let B = N α such that < α < 1 and N Resampling in the pesence of a tend Theoem 1 still holds if the obseved pocess has a tend, povided the tend is sufficiently smooth. Specifically, suppose X(t) = µ(t) + Z(t), whee Z is a zeo mean pocess with vaiogam γ(t) = θt + S(t),S(t) Dt, with a bounded second deivative on (0,L]. Fist, we note that Va(X i+h X i ) = γ(hl/n). Setting ˆγ = N h i=1 (X i+h X i ) /(N h), we find that E(ˆγ) = γ ( ) hl + 1 N h (µ i+h µ i ). N N h If, fo example, the fist deivative of µ is bounded, then the second tem in the expession above is O(N ). Then ˆγ is asymptotically unbiased and Va(Nˆγ) is as given in (), with the additional tems in µ being O(N 3 ). Futhemoe, the steps in the poof leading to Equations (7) and () cay though unchanged.. Resampling the Second-Ode Vaiogam in a Fixed Domain Kent and Wood (1997) showed that fo pocesses with vaiogam given by i=1 γ(t) = θt α + o(t α ) as t 0, the fixed domain asymptotic behavio of estimates of α is diffeent fo α < 1.5 and α 1.5. This diffeence in asymptotic behavio can be attibuted to longe ange dependence in pocesses with α 1.5. Kent and Wood (1997) also showed that highe ode diffeencing of the data can emove this long ange dependence. See also Istas and Lang (1997).

10 7 JI MENG LOH AND MICHAEL L. STEIN In this section, we show how the idea of diffeencing to emove dependence can apply to infeence via esampling with a specific model whee α =. Specifically, we conside the model fo a Gaussian pocess Z in one dimension, with vaiogam given by γ(t) = βt + θt 3 + R(t) t 0, (15) whee R(t) Dt and γ(t) has a bounded fouth deivative. We conside estimating the second-ode vaiogam, which we denote by φ, defined in (1). Fo this model, the second-ode vaiogam φ(hl/n) = C(0) C(hL/N) + C(hL/N) has the fom N 3 φ = θh 3 L 3 + N 3 R( hl N ) N3 R( hl N ) = θh 3 L 3 + O(N 1 ), (1) so that the fist tem of N 3 φ is independent of N. Note also that (1) does not contain β. An estimato of φ is ˆφ ( ) hl = 1 N M M Y i, (17) i=1 whee Y i = [Z i+h Z i + Z i h ],i = h + 1,...,N h, and M = N h. Using the block-of-blocks bootstap as in the pevious section we get the following. Theoem. The estimato ˆφ given in (17) is unbiased and Va(N 3 θ ˆφ) L = 5N P(h) + O(N ), (1) whee P(h) = 90h 7 700h +33h h 1790h h +17h is a fixed polynomial in h, so that N 3 ˆφ has N 1 convegence. Futhemoe, N[E(N ˆσ ) Va(N3 ˆφ)] = O(B 1 ) + O(BN 1 ), (19) Va(N ˆσ ) = O(B N 1 ). (0) Poof. The estimate ˆφ is clealy unbiased. Futhemoe, Va(ˆφ) = 1 M Va(Y 1) + M = ( ) hl M φ + N M M Cov (Z i+h Z i + Z i h,z j+h Z j + Z j h ) i<j M C i j, (1) i<j

11 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 77 whee ( ) ( kl (k + h)l C k = γ + γ N N ( k h L +γ N ) γ ( (k + h)l ) γ ( ) k h L N N ). () Using (15), we can evaluate () fo the thee cases k = i j < h,h k < h and k h, giving θl 3 (k 3 1k h+h 3 )N 3 + O(N ) fo k < h, C k = θl 3 ( k 3 +1k h kh +1h 3 )N 3 +O(N ) fo h k<h, (3) O(N ) fo k h. Thee ae M k pais of Y i and Y j such that j i = k, thus thee ae ode M tems with k h. The sum of the coesponding covaiance tems within the double sum in (1) fo k h is of ode N, wheeas fo k < h, M i<j: i j <h C i j = θ L N h 1 + [ h 1 (k 3 1k h + h 3 ) (M k) k=1 ] ( k 3 +1k h kh +1h 3 ) (M k) +O(N ). k=h Putting this togethe with (1) into (1) gives (1). Fo the bootstap calculations, many of the expessions fo the bootstap vaiance estimato follow fom the pevious section. Hee, we find that the tems involving C k fo k h sum up to a lowe ode than the tems coesponding to k < h. Specifically, fo (13), thee ae O(M ) numbe of tems involving C l with l h, of ode O(N ). Thus we find that, fo M/B h, ( ) M Va B 1 = M(B 1)Va(Y 1 ) + θ L h 1 N [MB M (B B+1)l] l=1 [(l 3 1l h + h 3 ) + O(N 1 )] + θ L N [MB M (B B + 1)l] h 1 l=h [( l 3 +1l h lh +1h 3 ) +O(N 1 )]+O(N ) = MBθ L 5N P(h) + O(N 5 ) + O(B N ),

12 7 JI MENG LOH AND MICHAEL L. STEIN whee P(h) is defined just afte (1). Putting this into E(N ˆσ ) = N B B Va( k ) k=1 and compaing with Va(N 3 ˆφ) in (1) gives (19). To obtain (0), we note that the agument of the two paagaphs afte (1) applies hee as well. The tems Cov (Y i Y j,y u Y v ), whee at least one of {u,v} is at most h away fom {i,j}, ae bounded by one of Cov (Yi,Y i ),Cov (Y i,y iy j ), Cov(Y i Y j,y i Y k ) o Cov(Y i Y j,y i Y j ), which ae O(N 1 ). The sums of coefficients coesponding to these tems ae of the same ode as those in the pevious section. This gives (0), afte noting that tems Cov(Y i Y j,y u Y v ) fo u and v both h o moe away fom i and j ae O(N 1 ) and thei total is o(b N 1 ). As in Section 3.1, the bootstap is consistent in the pesence of a tend if the tend is sufficiently smooth. Specifically, if the second deivative of µ is bounded, ˆφ is asymptotically unbiased. Equations (1), (19) and (0) of Theoem emain unchanged. Note that the second-ode vaiogam φ contains θ but not β. Unde fixed domain asymptotics φ, and thus θ, can be consistently estimated and we showed above that the block-of-blocks bootstap yielded asymptotically coect infeences unde fixed domain asymptotics. On the othe hand, β cannot be consistently estimated using obsevations in a fixed domain and we do not expect any bootstap method to yield asymptotically coect infeences fo β unde fixed domain asymptotics. The esults of ou simulation study (Section 5, Figue 3) also show that as we conside smoothe pocesses, consistency is achieved only if we diffeence enough, e.g., by consideing the second-ode vaiogam athe than the fist-ode vaiogam. 5. Simulation Study We pesent hee the esults of a simulation study to bootstap a Gaussian andom field using the block-of-blocks bootstap. We pefomed simulations in one and two dimensions, using equally spaced obsevations on the unit inteval and the unit squae. In ou simulations we used the Matén model with covaiance function given by (), with ρ = 0.01 and 0.15 and ν =,,...,.0. The values ν = and 1.5 coespond to models belonging to those consideed in Sections 3 and, espectively. Nomally a simulation of a ealization of a andom field equies pefoming a Cholesky decomposition of the covaiance matix and, fo lage data sets, is vey computationally intensive. Howeve, fo obsevations on a egula lattice, simulations can be done vey quickly using the Fast Fouie Tansfom.

13 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 79 5 x Figue 1. Fo each point x in the two dimensional simulations, the distances at which the vaiogams ae estimated ae the distances to the neaest neighbos as shown hee. Wood and Chan (199) and Dietich and Newsam (1997) povide slightly diffeent methods to do this (see also Stein (00)). We used the method of Dietich and Newsam (1997). With a ealization of the pocess, we computed the fist and second-ode vaiogams at distances coesponding to the neaest neighbos of the obsevations. In one dimension, h = 1,...,, while in two dimensions, h = 1,,, 5,,3,, 13,, 1 (see Figue 1). The bootstap method descibed in Section was then used to esample the obsevations to obtain bootstap estimates. Fo each ealization, we obtained R = 999 bootstap samples. With the 999 bootstap estimates, a nominal 95% confidence inteval is then constucted using what Davison and Hinkley (1997) call the basic bootstap inteval. Specifically, fo each ealization of a pocess, we have, fo the fist ode vaiogam, say, an estimate ˆγ = ˆγ(h/N) and R bootstap estimates ˆγ k,k = 1,...,R. If ˆγ (R+1)(1 α/) and ˆγ (R+1)α/ ae espectively the (R + 1)(1 α/)th and (R + 1)(α/)th odeed values of ˆγ k,k = 1,...,R, the 0(1 α)% basic bootstap inteval is given by [ˆγ ˆγ (R+1)(1 α/),ˆγ ˆγ (R+1)α/ ]. () The above inteval is obtained by assuming that the sampling distibution of ˆγ ˆγ is simila to that of ˆγ γ, so that the (1 α/)th and (α/)th quantiles of ˆγ γ can be estimated by the (R+1)(1 α/)th and (R+1)(α/)th odeed values of ˆγ ˆγ. The bootstap pocedue is pefomed 500 times, giving 500 nominal 95% confidence intevals. The empiical coveage of the confidence intevals can then be examined. We implemented a few modifications to the esampling scheme descibed in the ealie sections. Fist, we used ovelapping blocks instead of non-ovelapping ones. In analogy to the block bootstap, this means we used moving blocks instead of fixed blocks. Thus, in the one-dimensional simulations, block 1 consists

14 0 JI MENG LOH AND MICHAEL L. STEIN Figue. This figue shows how tooidal wapping is implemented in two dimensions. The dots epesent the obsevation points. The smalle squae is a esampling block that does not fully lie within the obsevation egion. The dotted potion is wapped aound and samples the obsevations on the othe side as well. of obsevations 1 though N/B, block of obsevations though N/B+1, and so on, and a new sample is obtained by sampling B of these blocks with eplacement. Also, to ensue that evey point has equal pobability of being selected, we implemented a tooidal wapping pocedue and allow blocks to fall patly outside the obsevation egion. Such blocks ae wapped aound so that they also sample points on the othe side of the obsevation egion. So, fo example, in one dimension, block N consists of obsevations N,1,...,N/B 1. Figue shows tooidal wapping implemented in two dimensions. Note that the maks ae calculated fom the oiginal obsevations without any tooidal wapping. The tooidal wapping is only applied to the esampling of the points. Any othe pocedue fo selecting points with equal pobability can be used, if desied. Fo the simulations in one dimension, we use N equal to 3,,1,5,51,, and,0, and blocks of length 1/,1/,1/, and 1/1. In two dimensions, we use N = 1,3,..., and blocks of aea 1/,1/,1/, and 1/ Results Figue 3 shows the esults of ou simulation study in one dimension, specifically, plots of the empiical coveage of nominal 95% confidence intevals fo the fist-ode and second-ode vaiogams fo the Matén pocess with ρ = 0.15 and ν fom to.0. The esults fo ρ = 0.01 ae vey simila and ae not shown. We only show plots fo epesentative values of N and B, specifically, N = 1 with B =, N = 51 with B =, and N =,0 with B = 1. We did not include plots fo N = 3 as the empiical coveages attained fo this N ae unifomly poo, especially fo lage. When estimating the fist-ode vaiogam, we find that the empiical coveage of nominal 95% confidence intevals appoaches the nominal level as N

15 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 1 Figue 3. Plots of empiical coveage of nominal 95% confidence intevals fo the fist-ode vaiogam γ (thin lines) and second-ode vaiogam φ (thick lines), obtained by esampling using the block-of-blocks bootstap method fo the Matén models in one dimension, fo ρ = 0.15, ν =,,...,. Diffeent numbes of obsevations N and block sizes 1/B wee consideed: N = 1, B = (solid), N = 51, B = (dashed) and N =, 0, B = 1 (dotted).

16 JI MENG LOH AND MICHAEL L. STEIN and B get lage, with N/B inceasing, fo small ν, specifically, ν =, and 5. This holds fo both ρ = 0.01 and As we incease ν futhe, we find a gadual degadation, with the empiical coveage becoming noticeably less than 95% fo all N when ν is about 1 and lage, especially so when ρ = Futhemoe, thee is little o no impovement in coveage pobabilities when N inceases fom 51 to,0. Fom the esults in the pevious sections, we expect, fo the ν = 1.5 model, that the empiical coveage will be close to the nominal level when we conside the second-ode vaiogam fo lage values of N. We find fom Figue 3 that this is indeed the case. Again, we see a gadual dop in empiical coveage as ν is inceased to. Fo ν =, we find that the empiical coveage of 95% confidence intevals fo the second-ode vaiogam does not appoach the 95% level as the numbe of obsevations is inceased. We also examine how the empiical coveage pobabilities of nominal 95% confidence intevals vay when B is vaied, with N fixed. In geneal, we find that fo a paticula value of N, empiical coveage impoves as B inceases. With small B, the bootstap samples tend to be moe alike, so that bootstap estimates do not have the appopiate vaiance. Howeve, we find that the empiical coveage may dop if B is inceased too much. Figue shows the empiical coveage of nominal 95% confidence intevals fo the second-ode vaiogam fo N = 3 (left), and N =,0 (ight), with the Matén ν = 1.5,ρ = 0.15 model. The ight plot shows the common situation of empiical coveage inceasing as B inceases. The left plot shows an example of empiical coveage dopping when B gets too lage. This dop in empiical coveage when B is too lage is slightly moe ponounced in ou simulations in two dimensions. Figue 5 shows plots of the empiical coveage of nominal 95% confidence intevals fo φ of the Matén ν = 1.5,ρ = 0.15 model, with N = 3 and 51. Notice that empiical coveage dops when the smallest block sizes wee used. Figue shows plots of the empiical coveage of nominal 95% bootstap confidence intevals of the fist- and second-ode vaiogams in the two-dimensional case, with diffeent combinations of the numbe of obsevations N and block aea 1/B : N = 3 with B =,N = 1 with B =, and N = 51 with B =. In two dimensions, we see that we equie lage sample sizes fo the asymptotics to become appaent. Note that the actual empiical coveages attained in the one- and two-dimensional simulations ae difficult to compae. Fo example, it is not clea whethe N = 1 in the one-dimensional case should be compaed with N = 1 o N = 1 in two dimensions. It is also not clea whethe to make compaisons using block width o numbe of blocks. We do find the same qualitative behavio of the empiical coveage as in the one-dimensional case: as N and B ae inceased, with N/B inceasing, the

17 SPATIAL BOOTSTRAP IN A FIXED DOMAIN ν =, =0.01, N =3 ν =, =0.01, N = ν =, =0.15, N =3 ν =, =0.15, N = ν =1.5, =0.01, N =3 ν =1.5, =0.01, N = ν =1.5, =0.15, N =3 ν =1.5, =0.15, N = Figue. Plots of empiical coveage of nominal 95% confidence intevals fo the second-ode vaiogam φ, obtained by esampling using the blockof-blocks bootstap method fo the Matén model with ν = 1.5, ρ = 0.15 in one dimension. The numbes of obsevations N ae 3 (left) and,0 (ight). Fo each plot, the diffeent lines coespond to diffeent block size 1/B: B = (solid), B = (dashed), B = (dotted), and B = 1 (dotted and dashed).

18 JI MENG LOH AND MICHAEL L. STEIN ν =, =0.01, N =3 3 ν =, =0.01, N = ν =, =0.15, N =3 3 ν =, =0.15, N = ν =1.5, =0.01, N =3 3 ν =1.5, =0.01, N = ν =1.5, =0.15, N =3 3 ν =1.5, =0.15, N = Figue 5. Plots of empiical coveage of nominal 95% confidence intevals fo the second-ode vaiogam φ, obtained by esampling using the blockof-blocks bootstap method fo the Matén model with ν = 1.5, ρ = 0.15 in two dimensions. The numbes of obsevations N ae 3 3 (left) and (ight). Fo each plot, the diffeent lines coespond to blocks with aea 1/B : B = (solid), B = (dashed), B = (dotted), and B = 1 (dotted and dashed).

19 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 5 Figue. Plots of empiical coveage of nominal 95% confidence intevals fo the 1st ode vaiogam γ (thin lines) and nd ode vaiogam φ (thick lines), obtained by esampling using the block-of-blocks bootstap method in two dimensions fo the Matén model with ρ = 0.15 and ν =,,...,. Diffeent numbes of obsevations N and block aeas 1/B wee consideed: N = 3 3, B = (solid), N = 1 1, B = (dashed), N = 51 51, B = (dotted and dashed).

20 JI MENG LOH AND MICHAEL L. STEIN empiical coveage inceases towad the nominal level povided ν < 1 fo the fist-ode vaiogam, and ν < fo the second-ode vaiogam. This once again suggests that consistency is achieved only if we diffeence enough. The esults in Chan and Wood (000) show diffeent asymptotic behavio of incement-based estimatos in one and two dimensions fo a elated setting. Howeve, it is difficult to say anything conclusive about diffeences between the one- and twodimensional cases fom ou simulation study.. Conclusion In spatial statistics, asymptotics can be consideed in the context of an inceasing o fixed domain as the numbe of obsevations inceases. As fa as we ae awae, thee has not been any asymptotic esults fo the spatial bootstap unde the fixed-domain setting. In this wok, we showed that the block-of-blocks bootstap gives asymptotically consistent esults in the fixed domain case, when appopiate quantities ae consideed. Specifically, we consideed two Gaussian pocesses, one with vaiogam of the fom γ(t) = θt + S(t), S(t) Dt and the othe of the fom γ(t) = βt +θt 3 +R(t), R(t) Dt, and showed that the blockof-blocks bootstap gives consistent esults fo the vaiogam and second-ode vaiogam espectively. The esults hee ae not specific to the block-of-blocks bootstap and should apply to othe simila esampling schemes. In ou simulations with Matén pocesses, we find that as the smoothness paamete ν is inceased, the empiical coveage of nominal 95% confidence intevals fo the vaiogam deceased. Fo ν about 1 and lage, the coveage of confidence intevals was noticeably lowe than the nominal level even fo lage N. Fo confidence intevals of the second-ode vaiogam, howeve, the nominal level is attained as N is inceased, fo lage values of ν, with the degadation becoming noticeable at a highe value of ν than fo the fist-ode vaiogam. Following Matheon (1971), Stein (1999) defines the pincipal iegula tem as the fist tem in the expansion of the covaiance function C(t) in t about 0 that is not an even powe of t. Fo example, fo the pocesses consideed in Section 3 and, the pincipal iegula tems ae θt and θt 3 espectively. Fo the Matén model, the pincipal iegula tem has powe given by ν. With the second-ode vaiogam in Section, we emoved the βt tem in the expansion of the covaiance function by taking second diffeences and wee able to make consistent infeences fo the pincipal iegula tem unde fixed-domain asymptotics. Kent and Wood (1997) consideed estimating α in γ(t) = θt α +o(t α ). They showed that estimates fo α obtained fom fist diffeences of the pocess ae n 1/ -consistent only if α (0,1.5), but when second diffeences ae used, n 1/ - consistent estimates ae obtained fo α (0, ). Ou simulations suggest a simila

21 SPATIAL BOOTSTRAP IN A FIXED DOMAIN 7 esult fo the bootstap: the bootstap can wok easonably well fo a high enough ode of the vaiogam elative to the smoothness of the pocess. Covaiance functions with simila high fequency behavio of thei spectal densities yield vey similia pedictions (Stein (1999)). The high fequency behavio is in tun elated to the pincipal iegula tem in the expansion of the covaiance function in t about 0. We showed hee that the bootstap estimate is consistent fo the standad eo of the pincipal iegula tem estimate, but not necessaily fo the empiical vaiogam. Acknowledgement The fist autho s eseach was patly suppoted by National Science Foundation awad Although the second autho s eseach was funded wholly o in pat by the United States Envionmental Potection Agency though STAR coopeative ageement R to the Univesity of Chicago, it has not been subjected to the Agency s equied pee and policy eview and theefoe does not necessaily eflect the views of the Agency, and no official endosement should be infeed. Refeences Bühlmann, P. and Künsch, H. R. (1995). The blockwise bootstap fo geneal paametes of a stationay time seies. Scand. J. Statist., Chan, G. and Wood, A. T. A. (000). Incement-based estimatos of factal dimension fo two-dimensional suface data. Statist. Sinica, Davison, A. C. and Hinkley, D. V. (1997). Bootstap Methods and thei Applications. Cambidge Univesity Pess, Cambidge. Dietich, C. R. and Newsam, G. N. (1997). Fast and exact simulation of stationay Gaussian pocesses though ciculant embedding of the covaiance matix. SIAM J. Sci. Comput. 1, -17. Hall, P. (195). Resampling a coveage patten. Stochastic Pocess. Appl. 0, 31-. Istas, J. and Lang, G. (1997). Quadatic vaiations and estimation of the local Hölde index of a Gaussian pocess. Ann. Inst. H. Poincaé Pobab. Statist. 33, Kent, J. T. and Wood, A. T. A. (1997). Estimating the factal dimension of a locally self-simila Gaussian pocess by using incements. J. Roy. Stat. Soc. Se. B 59, Künsch, H. R. (199). The jackknife and the bootstap fo geneal stationay obsevations. Ann. Statist. 17, Lahii, S. N. (199). Edgewoth coection by moving block bootstap fo stationay and nonstationay data. In Exploing the Limits of Bootstap (Edited by R. LePage and L. Billad), Wiley, New Yok. Lahii, S. N. (1993). On the moving block bootstap unde long ange dependence. Statist. Pobab. Lett. 1, Liu, R. Y. and Singh, K. (199). Moving blocks jackknife and bootstap captue weak dependence. In Exploing the Limits of Bootstap (Edited by R. LePage and L. Billad), 5-. Wiley, New Yok.

22 JI MENG LOH AND MICHAEL L. STEIN Loh, J. M. and Stein, M. L. (00). Bootstapping a spatial point pocess. Statist. Sinica 1, 9-1. Matheon, G. (1971.) The Theoy of Regionalized Vaiables and its Applications. Ecole des Mines, Fontainebleau. Matheon, G. (199). Estimating and Choosing: An Essay on Pobability in Pactice, Tans. A.M. Hasofe. Spinge-Velag, Belin. Politis, D. N. and Romano, J. P. (199a). A cicula block-esampling pocedue fo stationay data. In Exploing the Limits of Bootstap (Edited by R. LePage and L. Billad), Wiley, New Yok. Politis, D. N. and Romano, J. P. (199b). A geneal esampling scheme fo tiangula aays of α-mixing andom vaiables with application to the poblem of spectal density estimation. Ann. Statist. 0, Politis, D. N. and Romano, J. P. (199). Lage sample confidence egions based on subsamples unde minimal assumptions. Ann. Statist., Shi, X. and Shao, J. (19). Resampling estimation when obsevations ae m-dependent. Comm. Statist. A 17, Stein, M. L. (1999). Intepolation of Spatial Data. Spinge, New Yok. Stein, M. L. (00). Fast and exact simulation of factional Bownian sufaces. J. Comput. Gaph. Statist. 11, Wood, A. T. A. and Chan, G. (199). Simulation of stationay Gaussian pocesses in [0, 1] d. J. Comput. Gaph. Statist. 3, Ying, Z. (1991). Asymptotic popeties of a maximum likelihood estimato with data fom a Gaussian pocess. J. Multivaiate Anal. 3, Depatment of Statistics, Columbia Univesity, 155 Amstedam Ave, th Fl., MC 90, New Yok, NY 07, U.S.A. meng@stat.columbia.edu Depatment of Statistics, Univesity of Chicago, 573 Univesity Ave, Chicago, IL 037, U.S.A. stein@galton.uchicago.edu (Received May 00; accepted Decembe 00)