Equivalence Theory for Density Estimation, Poisson Processes and Gaussian White Noise With Drift

Transcription

1 University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 4 Equivalence Theory for Density Estimation, Poisson Processes and Gaussian White Noise With Drift Lawrence D. Brown University of Pennsylvania Andrew V. Carter University of California, Santa Barbara Mark G. Low University of Pennsylvania Cun-Hui Zhang Rutgers University Follow this and additional works at: Part of the Statistics and Probability Commons Recommended Citation Brown, L. D., Carter, A. V., Low, M. G., & Zhang, C. (4. Equivalence Theory for Density Estimation, Poisson Processes and Gaussian White Noise With Drift. The Annals of Statistics, 3 (5, This paper is posted at ScholarlyCommons. For more information, please contact repository@pobox.upenn.edu.

2 Equivalence Theory for Density Estimation, Poisson Processes and Gaussian White Noise With Drift Abstract This paper establishes the global asymptotic equivalence between a Poisson process with variable intensity and white noise with drift under sharp smoothness conditions on the unknown function. This equivalence is also extended to density estimation models by Poissonization. The asymptotic equivalences are established by constructing explicit equivalence mappings. The impact of such asymptotic equivalence results is that an investigation in one of these nonparametric models automatically yields asymptotically analogous results in the other models. Keywords asymptotic equivalence, decision theory, local limit theorem, quantile transform, white noise model Disciplines Statistics and Probability This journal article is available at ScholarlyCommons:

3 The Annals of Statistics 4, Vol. 3, No. 5, DOI 1.114/ Institute of Mathematical Statistics, 4 EQUIVALENCE THEORY FOR DENSITY ESTIMATION, POISSON PROCESSES AND GAUSSIAN WHITE NOISE WITH DRIFT BY LAWRENCE D. BROWN, 1 ANDREW V. CARTER, MARK G. LOW AND CUN-HUI ZHANG 3 University of Pennsylvania, University of California, Santa Barbara, University of Pennsylvania and Rutgers University This paper establishes the global asymptotic equivalence between a Poisson process with variable intensity and white noise with drift under sharp smoothness conditions on the unknown function. This equivalence is also extended to density estimation models by Poissonization. The asymptotic equivalences are established by constructing explicit equivalence mappings. The impact of such asymptotic equivalence results is that an investigation in one of these nonparametric models automatically yields asymptotically analogous results in the other models. 1. Introduction. The purpose of this paper is to give an explicit construction of global asymptotic equivalence in the sense of Le Cam (1964 between a Poisson process with variable intensity and white noise with drift. The construction is extended to density estimation models. It yields asymptotic solutions to both density estimation and Poisson process problems based on asymptotic solutions to white noise with drift problems and vice versa. Density estimation model. A random vector V n of length n is observed such that V n (V 1,...,V n is a sequence of i.i.d. variables with a common density f F. Poisson process. A random vector of random length {N,X N } is observed such that N N n is a Poisson variable with EN = n and that given N = m, X N = X m (X 1,...,X m is a sequence of i.i.d. variables with a common density f F. The resulting observations are then distributed as a Poisson process with intensity function nf. Received April ; revised February 4. 1 Supported by NSF Grant DMS Supported by NSF Grant DMS Supported by NSF Grants DMS-1-59 and DMS AMS subject classifications. Primary 6B15; secondary 6G7, 6G. Key words and phrases. Asymptotic equivalence, decision theory, local limit theorem, quantile transform, white noise model. 74

4 White noise. such that EQUIVALENCE THEORY FOR DENSITY ESTIMATION 75 A Gaussian process Z Zn {Z n (t, t 1} is observed t Zn (t B (1.1 (t f(xdx+ n, t 1, with a standard Brownian motion B (t and an unknown probability density function f F in [, 1]. Asymptotic equivalence. For any two experiments ξ 1 and ξ with a common parameter space, (ξ 1,ξ ; denotes Le Cam s distance [cf., e.g., Le Cam (1986 or Le Cam and Yang (199] defined as (ξ 1,ξ ; sup max sup inf sup E (j L j=1, δ (j δ (k θ L ( θ,δ (j E (k θ L ( θ,δ (k, θ where (a the first supremum is taken over all decision problems with loss function L 1, (b given the decision problem and j = 1,, k 3 j (k = forj = 1 and k = 1forj = the maximin value of the maximum difference in risks over is computed over all (randomized statistical procedures δ (l for ξ l and (c the expectations E (l θ are evaluated in experiments ξ l with parameter θ, l = j,k. The statistical interpretation of the Le Cam distance is as follows: If (ξ 1,ξ ; < ε, then for any decision problem with L 1 and any statistical procedure δ (j with the experiment ξ j, j = 1,, there exists a (randomized procedure δ (k with ξ k, k = 3 j, such that the risk of δ (k evaluated in ξ k nearly matches (within ε that of δ (j evaluated in ξ j. Two sequences of experiments {ξ 1,n,n 1} and {ξ,n,n 1}, with a common parameter space F, are asymptotically equivalent if (ξ 1,n,ξ,n ; F asn. The interpretation is that the risks of corresponding procedures converge. A key result of Le Cam (1964 is that this equivalence of experiments can be characterized using random transformations between the probability spaces. A random transformation, T(X,U which maps observations X into the space of observations Y (with possible dependence on an independent, uninformative random component U also maps distributions in ξ 1 to approximations of the distributions in ξ via P (1 θ T P( θ. For the mapping between the Poisson and Gaussian processes we shall restrict ourselves to transformations T with deterministic inverses, T 1 (T (X, U = X. The experiments are asymptotically equivalent if the total-variation distance between P ( θ and the distribution of T under P (1 θ converges to uniformly in θ. As explained in Brown and Low (1996 andbrown, Cai, Lowand Zhang (, knowingan appropriate T allows explicit construction of estimation procedures in ξ 1 by applying statistical procedures from ξ to T(X,U.

5 76 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG In general, asymptotic equivalence also implies a transformation from the P ( θ to the P (1 θ and the corresponding total-variation distance bound. However, in the case of the equivalence between the Poisson process and white noise with drift, by requiring that the transformation be invertible, we have saved ourselves a step. The transformation in the other direction is T 1,and P (1 θ T P( θ P (1 θ TT 1 P ( θ T 1 = P (1 θ P ( θ T 1. Therefore, it is sufficient if sup θ P (1 θ T P( θ. The equivalence mappings T n constructed in this paper from the sample space of the Poisson process to the sample space of the white noise are invertible randomized mappings such that ( (1. sup H f Tn (N, X N, Zn f F under certain conditions on the family F.HereH f (Z 1,Z denotes the Hellinger distance of stochastic processes or random vectors Z 1 and Z living in the same sample space, when the true unknown density is f.sincet n are invertible randomized mappings, T n (N, X N are sufficient statistics for the Poisson processes and their inverses Tn 1 are necessarily many-to-one deterministic mappings. Similar considerations apply for the mapping of the density estimation problem to the white noise with drift problem, although in that case there are two mappings, one from the density estimation to the white noise with drift model and another from the white noise with drift model back to the density estimation model. These mappings are given in Section. There have recently been several papers on the global asymptotic equivalence of nonparametric experiments. Brown and Low (1996 established global asymptotic equivalence of the white noise problem with unknown drift f to a nonparametric regression problem with deterministic design and unknown regression f when f belongs to a Lipschitz class with smoothness index α> 1. It has also been demonstrated that such nonparametric problems are typically asymptotically nonequivalent when the unknown f belongs to larger classes, for example, with smoothness index α 1. Brown and Low (1996 showed the asymptotic nonequivalence between the white noise problem and nonparametric regression with deterministic design for α 1, Efromovich and Samarov (1996 showed that the asymptotic equivalence may fail when α< 1 4. Brown and Zhang (1998 showed the asymptotic nonequivalence for α 1 between any pair of the following four experiments: white noise, density problem, nonparametric regression with random design, and nonparametric regression with deterministic design. In Brown, Cai, Low and Zhang ( the asymptotic equivalence for nonparametric regression with random design was shown under Besov constraints which include Lipschitz classes with any smoothness index α> 1. Gramma and Nussbaum (1998 solved the fixed-design nonparametric regression problem for nonnormal errors. Milstein

6 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 77 and Nussbaum (1998 showed that some diffusion problems can be approximated by discrete versions that are nonparametric autoregression models, and Golubev and Nussbaum (1998 established a discrete Gaussian approximation to the problem of estimating the spectral density of a stationary process. Most closely related to this paper is the work in Nussbaum (1996 where global asymptotic equivalence of the white noise problem to the nonparametric density problem with unknown density g = f /4 is shown. In this paper the global asymptotic equivalence was established under the following smoothness assumption: f belongs to the Lipschitz classes with smoothness index α> 1. The parameter spaces. The class of functions F will be assumed throughout to be densities with respect to Lebesgue measure on [, 1] that are uniformly bounded away from. The smoothness conditions on F can be described in terms of Haar basis functions of the densities. Let (1.3 θ k,l θ k,l (f fφ k,l, l=,..., k 1, k=, 1,..., be the Haar coefficients of f,where φ k,l k/( (1.4 1 Ik+1,l 1 Ik+1,l+1 are the Haar basis functions with I k,l [l/ k,(l+ 1/ k. The convergence of the Hellinger distance in (1. is established via an inequality in Theorem 3 in terms of the tails of the Besov norms f 1/,, and f 1/,4,4 of the Haar coefficients θ k,l θ k,l (f in (1.3. The Besov norms f α,p,q for the Haar coefficients, with smoothness index α and shape parameters p and q, are defined by (1.5 [ 1 { ( q k f α,p,q f + k(α+1/ 1/p 1 1/p } q ] 1/q θ k,l (f p. k= l= Let f k be the piecewise average of f at resolution level k, that is, the piecewise constant function defined by (1.6 Since f k written as k 1 f k f k (t 1{t I k,l } I k f. k,l l= f k+1 p p = l θ k,l φ k,l p = l θ k,l p k(p/ 1, (1.5 can be { } 1/q f α,p,q f q + ( kα f k f k+1 p q, k=

7 78 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG and its tail at resolution level k is f f k α,p,q, k, with f f k q ( α,p,q = { k k(α+1/ 1/p 1 1/p } q (1.7 θ k,l p. k=k Let B(α,p,q be the Besov space B(α,p,q ={f : f α,p,q < }. The following two theorems on the equivalence of white noise with drift, density estimation and Poisson estimation models are corollaries of our main result, Theorem 3, which bounds the squared Hellinger distance between particular invertible randomized mappings of the Poisson process and white noise with drift models. The randomized mappings are given in Section. Proofs of these theorems are given in the Appendix. THEOREM 1. Let Zn, {N,X N} and V n be the Gaussian process, Poisson process and density estimation experiments, respectively. Suppose that H is compact in both B(1/,, and B(1/, 4, 4 and that H {f :inf <x<1 f(x ε } for some ε >. Then (1.8 lim n (Z n, {N,X N}; H = and (1.9 lim n (Z n, V n ; H =. Our construction also shows that asymptotic equivalence holds for a class F if F is boundedin the Lipschitz norm with smoothnessindex β and compact in the Sobolev norm with smoothness index α β such that α + β 1, α 3 4 or β>1. by For <β 1 the Lipschitz norm f (L β l=1 and Sobolev norm f (S α f (L f(x f(y β sup x<y 1 x y β, f (S α n α c n (f, n= where c n (f 1 f(xe inπx dx are the Fourier coefficients of f. are defined THEOREM. Let Zn, {N,X N} and V n be the Gaussian process, Poisson process and density estimation experiments, respectively, and let F be bounded in the Lipschitz norm with smoothness index β and compact in the Sobolev norm with smoothness index α β. Suppose F {f :inf <x<1 f(x ε } for some ε >. Then if α + β 1, α 3 4 or β>1, lim n (Z n, {N,X N}; F = and lim n (Z n, V n ; F =.

8 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 79. The equivalence mappings. This section describes in detail the mappings which provide the asymptotic equivalence claimed in this paper. The fact that these mappings yield asymptotic equivalence is a consequence of our major result, Theorem 3. The construction is broken into several stages. From observations of the white noise (1.1, define random vectors { ( ( } Z k {Z k,l, l<k }, Z l + 1 l (.1 k,l k Z Z k, (. W k {W k,l, l<k }, W k,l W k,l+1 (Z k,l Z k,l+1 /. Let k k,n be suitable integers with lim n k,n =. Following Brown, Cai, Low and Zhang (, we construct equivalence mappings by finding the counterparts of Z k and W k, k>k, with the Poisson process (N, X N, to strongly approximate the Gaussian variables. It can be easily verified from (1.1 that {Z k,l, l<k,w k,l, l< k 1,k>k } are uncorrelated normal random variables with k EZ k,l = h k,l k I k,l h, h f, (.3 Var(Z k,l = σ k k /(4n, for l =,..., k 1, and for l =,..., k 1 1, EW k,l = 1 (h k,l h k,l+1 = k 1 hφ k 1,l, (.4 Var(W k,l = σ k 1. Let Ũ ={Ũk,l,k k,l } be a sequence of i.i.d. uniform variables in [ 1/, 1/ independent of (N, X N.Fork =, 1,...and l =,..., k 1define (.5 N k {N k,l, l< k }, N k,l #{X i : X i I k,l }. We shall approximate Z k in (.1 in distribution by (.6 Z k {Z k,l, l< k }, Z k,l σ k sgn (N k,l + Ũk,l N k,l + Ũk,l, at the initial resolution level k = k.sincen k,l are Poisson variables with (.7 λ k,l EN k,l = n k f k,l = f k,l 4σk, f k,l I k f, k,l by Taylor expansion and central limit theory Z k,l σ k ( λk,l + N k,l λ k,l λ 1/ k,l N ( f k,l,σk

9 8 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG as λ k,l, compared with (.3. Note that fk,l h k,l under suitable smoothness conditions on f, in view of (.3 and (.7. The Poisson variables N k,l can be fully recovered from Z k,l, while the randomization turns N k,l into continuous variables. Approximation of Wk,l for k>k is more delicate, since the central limit theorem is not sufficiently accurate at high resolution levels. Let F m be the cumulative distribution function of the independent sum of a binomial variable X m,1/ with parameter (m, 1 and a uniform variable Ũ in [ 1, 1, (.8 F m (x P { X m,1/ + Ũ x}, with F being the uniform distribution in [ 1, 1.Let be the N(, 1 cumulative distribution. We shall approximate W k by using a quantile transformation of randomized versions of the Poisson random variables. More specifically, let (.9 W k {W k,l, l< k }, W k,l σ k 1 1( F Nk 1,l (N k,l + Ũk,l withw k,l W k,l+1, l =,..., k 1 1, and theσ k in (.3. Given N k 1,l = m, I N k,l Bin(m, p k,l, p k,l k,l f I k 1,l f = f k,l (.1, f k,l + f k,l+1 so that W k,l is distributed exactly according to N(,σk 1 for p k,l = 1, compared with (.4. Thus, the distributions of W k,l and Wk,l are close at high resolution levels as long as f is sufficiently smooth, even for small N k 1,l = m. The equivalence mappings T n, with randomization through Ũ, are defined by T n : {N,X N, Ũ} W [k, Z n {Z n (t : t 1}, where for k k, W [k,k {Z k, W j,k <j<k}, andz k and W k are as in (.6 and (.9. The inverse of T n is a deterministic many-to-one mapping defined by Tn 1 : Zn W [k, (N, X N, where for k k, W [k,k {Z k, W j,k <j<k}. REMARK 1. One need only carry out the above construction to k = k 1 : k 1 > εn since we shall assume that f B( 1,, and then the observations W [k,k {Z k, W j,k <j<k} and W [k,k {Z k, W j,k <j<k} are asymptotically sufficient for the Gaussian process and Poisson process experiments. See Brown and Low (1996 for a detailed argument in the context of nonparametric regression.

10 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 81 Mappings for the density estimation model. The constructive asymptotic equivalence between density estimation experiments and Gaussian experiments is established by first randomizing the density estimation experiment to an approximation of the Poisson process and then applying the randomized mapping as given above. Set γ k = sup f H f f k 1/,, and note that since H is compact in B(1/,,, γ k. Now let k be the smallest integer such that 4 k /n γ k and divide the unit interval into subintervals of equal length with length equal to k. Let f n be the corresponding histogram estimate based on V n. Now note that since functions f H are bounded below by ε > it follows that 1 ( f n f 1 ( f n f ( f n + f 1 ( f n f (.11 =. ε ε Now 1 1 E ( f n f ( 1 = E f n f ( k + f (.1 f k and simple calculations show that the histogram estimate f n satisfies Ef n (x = f k (x and Var f n (x f k (x k n. Hence, 1 n 1/ ( E f n f k n 1/ k (.13 n Now n 1/ k and hence, from (1.7, γ 1/ k (.14 1 n 1/ ( f f k 1 γ 1/ k It thus follows from (.11 to (.14 that (.15 n 1/ sup E f H 1 γ 1/ k. f f k 1/,, γ 1/ k. ( f n f. Hence the density estimate is squared Hellinger consistent at a rate faster than square root of n. Now generate Ñ, a Poisson random variable with expectation n and independent of V n.ifñ>ngenerate Ñ n conditionally independent observations Vn+1,...,V Ñ with common density f n. Finally let (Ñ, XÑ = (Ñ,V1,V,..., V Ñ and write R1 n for this randomization from V n to (Ñ, XÑ, R 1 n : V n (Ñ, XÑ. A map from the Poisson number of independent observations back to the fixed number of observations is obtained similarly. This time let fˆ n be the histogram

11 8 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG estimator based on (N, X N.IfN<ngenerate n N additional conditionally independent observations with common density ˆ f n. It is also easy to check that (.16 n 1/ sup E f H 1 ( ˆ f n f. Now label these observations V n = (V 1,...,V n and write Rn for this randomization from (N, X N to V n, Rn : (N, X N V n. REMARK. It should also be possible to map the density estimation problem directly into an approximation of the white noise with drift model. Dividing the interval into k subintervals and conditioning on the number of observations falling in each subinterval, the conditional distribution within each subinterval is the same as for the Poisson process. Therefore, it is only necessary to have a version of Theorem 4 for a k -dimensional multinomial experiment. Carter ( provides a transformation from a k -dimensional multinomial to a multivariate normal as in Theorem 4 such that the total-variation distance between the distributions is O(k k n 1/. The transformation is similar to ours in that it adds uniform noise and then uses the square root as a variance-stabilizing transformation. However, the covariance structure of the multinomial complicates the issue and necessitates using a multi-resolution structure similar to the one applied here to the conditional experiments. The Carter ( result can be used in place of Theorem 4 to get a slightly weaker bound on the error in the approximation in Theorem 3 (because of the extra k factor when the total number of observations is fixed. This is enough to establish Theorem if the inequalities bounding α and β are changed to strictly greater than. It is also enough to establish Theorem 1 if H is a Besov space with α> 1. Carter ( also showed that a somewhat more complicated transformation leads to a deficiency bound on the normal approximation to the multinomials without the added k factor. 3. Main theorem. The theorems in Section 1 on the equivalence of white noise with drift experiments and Poisson process experiments are consequences of the following theorem which uniformly bounds the Hellinger distance between the randomized mappings described in Section. THEOREM 3. Suppose inf <x<1 f(x ε >. Let W [k,k {Z k, W j,k < j<k} with the variables in (.1 and (., and W [k,k {Z k, W j,k <j<k} with the variables in (.6 and (.9. Then there exist universal constants C,

12 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 83 D 1 and D such that for all k 1 >k, H ( W [k,k 1, W [k,k 1 C ε 4 k n + D 1 ε k k=k k 1 l= θ k,l + D ε 3 n 4 k 3k k 1 θ k,l 4 k=k l= C 4 k ε n + D 1 f ε f k 1/,, + D n f ε 3 4 k f k 4 1/,4,4, where θ k,l are the Haar coefficients of f as in (1.3, f k is as in (1.6 and 1/,p,p are the Besov norms in (1.5. REMARK 3. Here the universal constant C is the same as the one in Theorem 4, while D 1 = 3D 8 + andd = D for the D in Theorem 5. The proof of Theorem 3 is based on the inequalities established in Sections 4 and 5 for the normal approximation of Poisson and Binomial variables. Some additional technical lemmas are given in the Appendix. Let X m,p be a Bin(m, p variable, X λ be a Poisson variable with mean λ,andũ be a uniform variable in [ 1, 1 independent of X m,p and X λ.define (3.1 g m,p (x d dx P { 1( F m ( X m,p + Ũ x } with the F m in (.8 and the N(, 1 distribution function, and define g λ (x d dx P { (3. sgn( X λ + Ũ X λ + Ũ x}. Write ϕ b for the density of N(b,1 variables. PROOF OF THEOREM 3. Let g[k,k (w [k,k and g [k,k(w [k,k be the joint densities of W [k,k and W [k,k, gk (w k be the joint density of W k, and g k (w k w [k,k be the conditional joint density of W k given W [k,k. SinceW k is independent of W [k,k, g [k,k g [k,k g[k,k+1 g [k,k+1 = g[k,k g ( [k,k 1 g k g k, so that the Hellinger distance can be written as Hf ( W [k,k 1, W [k,k 1 ( g = 1 [k,k 1 g [k,k 1 (3.3 ( g = 1 [k,k +1 g [k,k +1

13 84 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG + g ( [k,k g [k,k 1 k <k<k 1 = Hf ( Z k, Z k + k <k<k 1 g k g k g [k,k g [k,kh (g k,g k. At the initial resolution level k, N k,l are independent Poisson variables by (.5, so that Z k,l are independent. This and the independence of Z k,l from (.1 imply Hf ( Z k 1 k, Z k l= Hf ( Z k,l, Z k,l. By (.6 and (3. Z k,l/σ k have densities g λk,l,whilez k,l /σ k are N(h k,l/ σ k, 1 variables by (.3. Thus, Theorem 4 can be used to obtain Hf ( Z k,l, Z k,l = H ( f gλk,l,ϕ C h k,l /σ k + 1 ( λ k,l h k,l. λ k,l σ k Since λ k,l = f k,l /(4σ k by (.7 and σ k = k /n by (.3, the above calculation yields (3.4 Hf ( k 1 Z k, Z k C l= k nf k,l C k k 1 + nε l= by Lemma 1(i and the bound f ε. For k>k and l< k 1 1, define k 1 + l= n k ( ε 3 n ( fk,l h k,l k I k,l ( f fk,l (3.5 µ k,l m k,l (p k,l 1, β k,l λ k 1,l (p k,l 1, where p k,l are as in (.1, λ k,l = f k,l n/ k are as in (.7, and the functions m k,l m k,l (w [k,k are defined by N k 1,l = m k,l (W [k,k. Atafixedresolution level k>k,andforl =,..., k 1 1, N k,l are independent binomial variables conditionally on W [k,k, so that by (.9 and (3.1 W k,l /σ k 1 are independent variables with densities g mk,l,p k,l under the conditional density g k.in addition, W k,l are independent normal variables with variance σ k 1 under g k. Thus, (3.6 H (gk k 1,g 1 k H ( g mk,l,p k,l,ϕ β k,l, l=

14 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 85 by (.4, where βk,l EW k,l /σ k 1 = 4n hφ k 1,l. It follows from Theorem 5 and (3.5 that for fixed w [k,k, H ( g mk,l,p k,l,ϕ β k,l (3.7 {[ D p k,l 1 ] [ + m k,l p k,l 1 ] 4 } + (µ k,l βk,l. Furthermore, it follows from Lemma 3 that g [k,k g ( [k,k mk,l λ k 1,l ( g [k,k mk,l 4 λ k 1,l = E ( N k 1,l 4 λ k 1,l, so that by (3.5, g [k,k g [k,k(µ k,l β k,l 4(p k,l 1 + (β k,l β k,l. Similarly, g [k,k g [k,km k,l EN k 1,l λ k 1,l + 1/. Thus, by (3.7, g [k,k g [k,kh ( g mk,l,p k,l,ϕ β k,l (3.8 [ 4D 1 pk,l 1 ] [ + Dλk 1,l pk,l 1 ] 4 + (βk,l βk,l, with D 1 = 3D/8 +. Now, by (.1 and (1.3, p k,l 1 = I k,l f I k,l+1 f (3.9 = I k 1,l f k 1 θ k 1,l f k 1,l, so that by (3.5, (.7, the definition of βk,l in (3.6 and Lemma 1(ii, β k,l βk,l = nf k 1,l k 1 θ k 1,l k 1 4n hφ k 1,l (3.1 f k 1,l θ k 1,l hφ k 1,l = 4n f k 1,l 4n (k 1/ 1 f 3/ k 1,l I k 1,l (f f k 1,l.

15 86 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG Inserting (3.9 and (3.1 into (3.8 and summing over l via (3.6, we find g [k,k g [k,kh (g k,g k (3.11 k 1 1 l= k 1 1 l= k 1 1 l= g [k,k g [k,kh ( g mk,l,p k,l,ϕ β k,l [ k θk 1,l 4 k θk 1,l 4 4D 1 8fk 1,l + Dλ k 1,l 64fk 1,l 4 + nk ( ] fk 1,l 3 (f f k 1,l I k 1,l [ D1 ε ( D n k 1 θk 1,l k 1 ( ε 3 I k 1,l (f f k 1,l due to λ k,l = nf k,l / k in (.7 and θk,l I k,l (f f k,l. Finally, inserting (3.4 and (3.11 into (3.3 and then using Lemma yields Hf ( W [k,k 1 +1, W [k,k 1 +1 C k + D k 1 1 nε ε k k 1 k=k ( D k 1 k=k l= C 4 k ε n + D k 1 1 ε k k=k θk,l l= k 1 k 1 l= n k ( ε 3 θ k,l + D ε 3 I k,l (f f k,l n 4 k 3k k 1 k=k l= θ 4 k,l, ], with D ( D /(1 1 4 = D and the theorem follows. 4. Approximation of Poisson variables. Let X λ be a Poisson random variable with mean λ and Ũ be a uniform variable on [ 1, 1 independent of X λ.define (4.1 Z λ sgn( X λ + Ũ X λ + Ũ, g λ(y d dy P { Z λ y}. The main result of this section is a local limit theorem which bounds the squared Hellinger distance between this transformed Poisson random variable and a normal random variable.

16 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 87 THEOREM 4. Let Z λ and g λ be as in (4.1. Let Zλ N( λ, 1 and ϕ µ be the density of N(,µ. Let H(, be the Hellinger distance. Then, as λ, (4. H ( Z λ,zλ = H ( ( 1 g λ,ϕ λ = 7 + o(1 96λ. Consequently, there exists a universal constant C< such that (4.3 H ( g λ,ϕ µ C/λ + ( λ µ / λ>,µ. REMARK 4. The theorem remains valid if Z λ is replaced by Z λ X λ + Ũ + 1, since H ( Z λ, Z λ is bounded by { f Xλ+Ũ f Xλ+Ũ+1/ 1 + = 1 E e λ λ j+1 } j!(j + 1! j= X λ λ min (1, C PROOF OF THEOREM 4. The second inequality of (4.3 follows immediately from (4., since H (ϕ µ1,ϕ µ = (µ 1 µ /4[cf.Brown,Cai,LowandZhang (, Lemma 3] and H ( g λ,ϕ µ. Let t(x sgn(x x, a strictly increasing function. Define λ. (4.4 X λ t 1 (Z λ = sgn(z λ (Z λ /4. Let f λ and fλ denote the densities of X λ + Ũ and X, respectively. Since t( is invertible, H ( Z λ,zλ = H( X λ + Ũ, X λ = f λ fλ, so that it suffices to show fλ (4.5 A λ fλ = 1 C λ λ, lim C λ = 7 λ 19. Since Ũ is uniform, f λ(x = e λ λ j /j! on [j 1/,j + 1/,sothat (4.6 A λ = j= j+1/ f λ (j {fλ (x/f λ(j} 1/ dx. j 1/ Since t (x = x 1/, by (4.4 fλ (x = x 1/ ϕ(t(x λ.thisgives fλ (x f λ (j = exp{ ( x λ /} = exp[ψ j (x], j 1 πxe λ λ j /j! x<j+ 1,

17 88 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG for j 1, in view of the Stirling formula j!= πj j+1/ exp( j + ε j,where ψ j (x ( x λ log x + λ 4 + j [ ] j log + log j j λ 4 + ε j (4.7 with 1/(1j + 1 <ε j < 1/(1j,forj = 1,,... Now, by the mean-value theorem, j+1/ { f } λ (x 1/ dx j 1/ f λ (j j+1/ ψ j j = exp [ψ j (j + ψ (j(x j+ (x j ] (x j dx j 1/ for some x j j 1,with ψ j (x = λ x 1 1 λ (4.8 4x, ψ j (x = x 3/ + 1 4x. Since exp[ψ j (j + ψ j (x j(x j /] is symmetric about j, itfollowsthat j+1/ { f } λ (x 1/ dx j 1/ f λ (j (4.9 j+1/ = exp [ψ j (j + ψ j (x j(x j ] (ψ j (j(x jk dx. j 1/ (k! k= Now, we shall take uniform Taylor expansions of ψ j and their derivatives in J λ {j : j/λ 1 λ /5 }. By (4.7, ψ j (j = λψ(j/λ + ε j /with ψ(x ( x 1 1 x + + x log x. Since ψ(1 = ψ (1 = ψ (1 =, ψ (1 = 1/4 andψ (1 = 7 8, ( j λψ = λ (j λ 3 λ 4 3!λ 3 7λ (j λ 4 ( 1 + o(1 = o(1. 8 4!λ 4 Since 1/(1j + 1<ε j < 1/(1j, ε j / = (1 + o(1/(4λ = o(1. Thus, (j λ3 ψ j (j = 4λ 7 (j λ 4 ( 1 + o(1 1 + o( λ 3 = o(1 4λ uniformly in J λ as λ. Similarly, by (4.8 and x j j 1, {ψ j (j} = ( (j λ 1 + o(1 4λ + o(1 λ = o(1, ψ j (x 1 + o(1 j = = o(1. λ

18 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 89 These expansions and (4.9 imply that uniformly in J λ, j+1/ { f } λ (x 1/ dx j 1/ f λ (j j+1/ [ = 1 + ψ j (j + { ψ j (x j + ( ] ψ j (j }(x j dx j 1/ + o(1 = o(1 (j λ k k= λ k+1 (j λ3 4λ 7 (j λ 4 8 4λ λ k= (j λ k λ k+1, [ 1 λ ] (j λ + 4λ as j+1/ j 1/ (x j dx = 1 1.Sincef λ(j is the Poisson probability mass function of X λ, j+1/ { f } f λ (j λ (x 1/ dx j J j 1/ f λ (j λ (4.1 = λ [ ] λ + 1 4λ 1 96λ + o(1 λ = o(1 19λ as j J λ f λ (j = 1 + o(1/λ. Note that E( X λ λ 3 = λ and E( X λ λ 4 = 3λ + λ. Hence, (4.5 follows from (4.6, (4.1 and the fact that j+1/ { f } f λ (j λ (x 1/ ( 1 dx P { X λ / J λ }P { X λ j/ J j 1/ f λ (j / J λ}=o. λ λ 5. Approximation of binomial variables. The strong approximation of a normal by a binomial depends on the cumulative distribution function F m in (.8. The addition of the independent uniform Ũ in (.8 to the binomial X m,1/ makes the c.d.f. continuous and thus 1 F m is a one-to-one function on ( 1,m+ 1 that maps symmetric binomials to standard normals. Let ϕ b be the N(b,1 density and g m,p be the probability density of (5.1 1 (F m [ X m,p + Ũ], X m,p Bin(m, p, as in (3.1, where Ũ is an independent uniform on [ 1, 1. THEOREM 5. There is a constant C 1 > such that, for all m, ( H ( g m,p,ϕ b = gm,p ( ϕ b b dz C1 m + b8 (5. m,

19 9 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG where b = ( m/ log(p/(1 p. Consequently, [( H ( g m,p,ϕ β D p 1 ( + m p 1 4 ] + ( m(p 1 β (5.3. PROOF. The case when m = is trivial because X = with probability 1 and therefore g,p is exactly an N (, 1. Thus, the following assumes that m 1. It follows from (3.1 that (5.4 g m,p (z = p j (1 p m j m ϕ (z, where j = j(zis the integer between and m such that (5.5 1[ F m ( j 1 ] z< 1 [ F m ( j + 1 ]. Let θ = log(p/q so that log g m,p(z ϕ (z ( = θ j m + m log(4pq, and the second term can be approximated by θ 4 θ 4 [ + e θ 4 log(4pq = log + e θ ] θ θ 4 ( Let h 1 (θ = ( + e θ + e θ /4. The second inequality in (5.6 follows from log(h 1 (θ log(1 + θ /4 θ /4 θ 4 /3. The first inequality in (5.6 follows from h 1 (θ 1 + θ /4 + θ 4 /4 for θ 4, and from log(h 1 (θ θ θ /4for θ > 4. Now, let z = z j(z m/ m (5.7 (z = and b = θ m/. Then for some 1/4 h (θ 1/3 the log ratio is log g m,p(z ϕ (z = z b b + h (θmθ 4. The log ratio of normals with different means is log ( ϕ /ϕ b = zb + b /. Therefore the ratio with respect to the normal with mean b is (5.8 log g m,p = h (θmθ 4 b(z z, h (θ 1 ϕ 4. b Since y log(x/y x y x log(x/y, for all positive x and y, ( 1 ϕb gm,p log g m,p ϕ b g m,p 1 ϕb log ( ϕb g m,p,

20 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 91 so that by (5.8, H ( { ( } 1 ϕb g p,m,ϕ b log (ϕ b + g m,p dz 4 g m,p (5.9 ( mθ 4 + b (z z (ϕ b + g m,p dz. 4 It follows from Carter and Pollard (4 that the difference between z and z = z (z is bounded by { z z C (m 1/ + m 1 z 3, for all z, (5.1 C (m 1/ + m 1 z 3, if z m, for some constant C. Thus, ( (z z 1 z g m,p dz C m + 6 (5.11 m g m,p dz+ Since g m,p I{z = (j m// m } dz = P { X m,p = j}, z 6 g m,p dz = E ( X m,p m/ 6 ( = O m 1 + m 3 ( p 1 z >m z 6 m g m,p dz. 6 = O(1 + b 6 uniformly in (m, p. It follows from (5.4 that z 6 g m,p dz m z 6 ϕ dz = O( m m 6 e m = O(m 1. z >m z >m The above two inequalities and (5.11 imply (z z g m,p dz C O(1/m + b 6 /m. Similarly, (z z ϕ b dz C O(1/m + b 6 /m. Inserting these two inequalities into (5.9 yields (5. in view of (5.7. Now let us prove (5.3. The Hellinger distance is bounded by, so that b 8 /m in (5. can be replaced by b 4 /m and it suffices to consider p for the proof of (5.3. By inspecting the infinite series expansion of log( p q = log(1+x log(1 x for x = p 1, we find that for p 1 1 4, log( p q 8 3 p 1 and log( p q 4(p p 1 3. These inequalities, respectively, imply b m + b4 m 16 9 (p m(p and b m(p m p m p 1 4, in view of the definition of b, which then imply (5.3 via (5. and the fact that H (ϕ b ϕ β = (b β /4.

21 9 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG APPENDIX A.1. The Tusnády inequality. The coupling of symmetric binomials and normals maps the integers j onto intervals [β j,β j+1 ] such that the normal(m/,m/4 probability in the interval is equal to the binomial probability at ( m j j.takingthe standardized values z j = (β j m/ (j 1/ m/, u j =, m m Carter and Pollard (4 showed that for m/ <j<mandcertain universal finite constants C ± u j + 1 C m z u ( j j u j 1 + m γ uj log(1 u j /m u j + log m C + m cu j m where c = log and γ is an increasing function with γ( = 1/1 and γ(1 = log 1/. This immediately implies that z j u j C m ( u j 3 + log m u j (A.1 m 1 for a certain universal constant C <. We shall prove (5.1 here based on (A.1. Because of the symmetry in both distributions, it is only necessary to consider z>. It follows from (5.5 and (5.7 that z j z<z j+1 u j z = z (z < u j+1. Let z j z<z j+1.sinceu j+1 u j = / m,foru j+1 m/ (A.1 implies z z z j u j z j+1 u j+1 + ( 1 (A. C m m + z 3 z 3 m Since u j and z j are both increasing in j, it follows that (z z / m are uniformly bounded away from zero for u j+1 m/, so that z z z j u j z j+1 u j+1 + C z 3 z 3 (A.3 m m for (m + 1/ m = u m+1 u j+1 m/andz m.sincez z z u m+1 z for z> m, (A. and (A.3 imply { z z C (m 1/ + m 1 z 3, for all z, C (m 1/ + m 1 z 3, if z m, for a certain universal C <, that is, (5.1..

22 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 93 A.. Technical lemmas. proof of Theorem 3. The following three lemmas simplify the rest of the LEMMA 1. (i Let f k,l and h k,l be as in (.7 and (.3. Then (A.4 f k,l h k,l k 1 f 3/ k,l (f f k,l. I k,l (ii Let θ k,l be the Haar coefficients of f as in (1.3. Then (A.5 hφ k,l θ k,l f k/ 1 f 3/ k,l (f f k,l. k,l I k,l PROOF. LetT = (f f k,l /f k,l 1. By algebra, T 1 + T 1 = T = T T ( T. It follows from (.3 and (.7 that h k,l = k f k,l I k,l 1 + T = k f k,l I k,l ( 1 + f f k,l f k,l (f f k,l f k,l ( T which implies (A.4 as k I k,l = 1 and by (.7 I k,l (f f k,l =. For (ii we have hφ k,l = f k,l φ k,l 1 + T = f k,l ( φ k,l 1 + f f k,l f k,l (f f k,l, f k,l ( T, which implies (A.5 as φ k,l = and φ k,l k by (1.4. LEMMA. Let θ k,l be the Haar coefficients in (1.3 and f k,l be as in (.7. Then k k=k k 1 l= ( (f f k,l I k,l ck (1 1/ c k(1+c k 1 θk,l 4 k=k l= c>. PROOF. Define δ i,j,k,l { 1, if Ii,j I k,l,, otherwise.

23 94 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG Since j δ i,j,k,l = i k for i k, using Cauchy Schwarz twice yields ( ( (f f k i 1 = δ i,j,k,l θi,j I k,l i=k j= [ ( ic/ ic i k i 1 1/ ] δ i,j,k,l θi,j 4 i=k j= ic ic i k i 1 δ i,j,k,l θi,j 4 i=k i=k k(1+c 1 1/ c i=k j= i(1+c i 1 j= δ i,j,k,l θ 4 i,j. Since k 1 l= δ i,j,k,l = 1fori k, the above inequality implies k k=k k 1 l= ( I k,l (f f k,l k k(1+c 1 1/ k=k c i=k ( i = ck 1 1/ i=k k=k c i(1+c i 1 j= i(1+c i 1 k 1 l= θi,j 4 j= δ i,j,k,l θ 4 i,j ck (1 1/ c i=k i(1+c i 1 j= θ 4 i,j. LEMMA 3. Let X λ be a Poisson random variable with mean λ. Then E ( X λ λ 4 4. PROOF. Since E( X λ λ 4 = λ(3λ + 1, E ( X λ λ 4 E( X λ λ 4 ( λ λ P ( X λ = 3λ + 1 λ

24 and EQUIVALENCE THEORY FOR DENSITY ESTIMATION 95 A.3. Proof of Theorem 1. First note that H(T n Rn 1 V n, Z n H ( T n Rn 1 V n,t n(n, X N + H ( T n (N, X N, Z n H(V n,r n T n 1 Z n H ( V n,r n (N, X N + H ( Rn (N, X N, Rn T n 1 Z n. Note also that since for any randomization T and random X and Y, H(TX,TY H(X,Y, it follows that H ( T n Rn 1 V n,t n(n, X N H ( Rn 1 V n,(n,x N and H ( Rn (N, X N, Rn T n 1 ( Z n H (N, XN, Tn 1 ( n Z = H Tn (N, X N, Z n. For the class H and the randomizations Rn 1 and R n it follows from (.15, (.16 and the proof of Proposition 3 on page 58 of Le Cam (1986 that and sup H ( Rn 1 V n,(n,x N f H sup H ( V n,r n (N, X N. f H Hence (1.9 and (1.8 will follow once sup H ( (A.6 T n (N, X N, Z n f H is established. By Theorem 3, for (A.6 to hold it is sufficient to show that ( 4 k sup f H n + f f k 1/,, + n f 4 k f k 4 1/,4,4. If the class of functions H is a compact set in the Besov spaces, then the partial sums converge uniformly to, sup f f k 1/,p,p f H for p = or4ask. This implies that there is a sequence γ k such that γk 1 sup f H f f k 4 1/,4,4. To be specific, let γ k = sup f f k 1/,4,4. f H

25 96 L. D. BROWN, A. V. CARTER, M. G. LOW AND C.-H. ZHANG It is necessary to choose the sequence of integers k (n that will be the critical dimension that divides the two techniques. Let k be the smallest integer such that 4 k n γ k. Therefore, k (n,andasn, ( 4 k sup f H n + f f k 1/,, + n f 4 k f k 4 1/,4,4 ( sup 4γ k + f f k 1/,, + 1 f f k 4 1/,4,4 f H γ k. A.4. Proof of Theorem. Theorem follows from Theorem 1 and the fact that the Lipschitz and Sobolev spaces described are compact in the Besov spaces. The Lipschitz class is equivalent to B β,, and therefore is compact in B 1/,p,p if β> 1. The Sobolev class is equivalent to B α,, and f f k α,, C α c n (f n α, where C α depends only on α. Thus if F is compact in Sobolev(α for α 1 then it is compact in B 1/,,. Further restrictions are required to show that the Sobolev(α class is compact in B 1/,4,4.If f (L β C (L,then f k f k+1 C (L kβ,sothat f f k 4 1/,4,4 C (L k(1 β k=k n f k f k+1 dx = C(L f f k (1 β,,. Therefore, for F bounded in Lipschitz(β, a compact Sobolev(α set is also compact in B 1/,4,4 if α 1 β. Finally, if F is compact in Sobolev(α, α 3/4, then it immediately follows from the Sobolev embedding theorem that the function is bounded in Lipschitz(1/4 [e.g., Folland (1984, pages 7 and 73], and it follows that F is compact in B 1/,4,4. Acknowledgments. We thank the referees and an Associate Editor for several suggestions which led to improvements in the final manuscript. REFERENCES BROWN, L. D., CAI, T., LOW, M. G. and ZHANG, C.-H. (. Asymptotic equivalence theory for nonparametric regression with random design. Ann. Statist BROWN, L. D. and LOW, M. G. (1996. Asymptotic equivalence of nonparametric regression and white noise. Ann. Statist BROWN, L. D. and ZHANG, C.-H. (1998. Asymptotic nonequivalence of nonparametric experiments when the smoothness index is 1/. Ann. Statist

26 EQUIVALENCE THEORY FOR DENSITY ESTIMATION 97 CARTER, A. V. (. Asymptotic equivalence of nonparametric experiments. Ph.D. dissertation, Yale Univ. CARTER, A. V. (. Deficiency distance between multinomial and multivariate normal experiments. Ann. Statist CARTER, A. V. and POLLARD, D. (4. Tusnády s inequality revisited. Ann. Statist. 3. To appear. EFROMOVICH, S. and SAMAROV, A. (1996. Asymptotic equivalence of nonparametric regression and white noise has its limits. Statist. Probab. Lett FOLLAND, G. B. (1984. Real Analysis. Wiley, New York. GOLUBEV, G. and NUSSBAUM, M. (1998. Asymptotic equivalence of spectral density and regression estimation. Technical report, Weierstrass Institute, Berlin. GRAMMA, I. and NUSSBAUM, M. (1998. Asymptotic equivalence for nonparametric generalized linear models. Probab. Theory Related Fields LE CAM, L. (1964. Sufficiency and approximate sufficiency. Ann. Math. Statist LE CAM, L. (1986. Asymptotic Methods in Statistical Decision Theory. Springer, New York. LE CAM,L.andYANG, G. (199. Asymptotics in Statistics. Springer, New York. MILSTEIN, G. and NUSSBAUM, M. (1998. Diffusion approximation for nonparametric autoregression. Probab. Theory Related Fields NUSSBAUM, M. (1996. Asymptotic equivalence of density estimation and Gaussian white noise. Ann. Statist L. D. BROWN M. G. LOW DEPARTMENT OF STATISTICS THE WHARTON SCHOOL UNIVERSITY OF PENNSYLVANIA PHILADELPHIA,PENNSYLVANIA USA lbrown@wharton.upenn.edu lowm@wharton.upenn.edu A. V. CARTER DEPARTMENT OF STATISTICS AND APPLIED PROBABILITY UNIVERSITY OF CALIFORNIA,SANTA BARBARA SANTA BARBARA, CALIFORNIA USA carter@pstat.ucsb.edu C.-H. ZHANG DEPARTMENT OF STATISTICS 54 HILL CENTER, BUSCH CAMPUS RUTGERS UNIVERSITY PISCATAWAY,NEW JERSEY USA