IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS

Size: px

Start display at page:

Download "IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS"

Damon Ford
5 years ago
Views:

1 IMPROVING EFFICIENT MARGINAL ESTIMATORS IN BIVARIATE MODELS WITH PARAMETRIC MARGINALS HANXIANG PENG AND ANTON SCHICK Abstract. Suppose we have data from a bivariate model with parametric margials. Efficiet estimators of the parameters i the margial models are geerally ot efficiet i the bivariate model. I this article, we propose a method of improvig these margial estimators ad demostrate that the magitude of this improvemet ca be as large as 1 percet i some cases. 1. Itroductio Let (X, Y ) be a bivariate radom vector with distributio Q. Let F deote the distributio of X ad G the distributio of Y. We assume that F ad G belog to regular parametric models, but that Q is ukow otherwise. The F = F α ad G = G β. The parameter α ca typically be estimated efficietly by the maximum likelihood estimator ˆα based o a radom sample X 1,..., X from F α. Similarly, the parameter β ca be estimated efficietly by the maximum likelihood estimator ˆβ based o a radom sample Y 1,..., Y from G β. If a radom sample (X 1, Y 1 ),..., (X, Y ) from Q is observed, the the margial estimator ˆα may o loger be efficiet as there may be iformatio about α cotaied i the data Y 1,..., Y. A similar observatio applies to ˆβ. I this paper we propose a simple method of usig possible iformatio about α cotaied i Y 1,..., Y. More precisely, we propose to subtract from ˆα a properly weighted stochastic term that is a asymptotically ubiased estimator of zero. The resultig estimator has a asymptotic dispersio matrix that is at most as large as the asymptotic dispersio matrix of ˆα. A variat of our approach has already be used i improvig oparametric estimators i the presece of a costrait, see e.g. Schick ad Wefelmeyer (28) for a recet overview of such methods. We give the details for our method i Sectio 2. I Sectio 3 we discuss the magitude of improvemet possible by this method. There it is demostrated that improvemets of up to 1 percet are possible. Sectio 4 discusses expoetial margials. We coclude this itroductio by specifyig the otio of a regular parametric model as used throughout this paper. We use regularity i the sese of Bickel et al (1993, page 12-13) to mea cotiuous Helliger-differetiability with positive defiite iformatio matrices. Here is the formal defiitio. Defiitio 1. A model H = {H θ : θ Θ} parametrized by a ope subset Θ of R k is regular if it is domiated by a measure ν, if the map θ h θ = Haxiag Peg is supported by NSF Grat DMS Ato Schick was supported by NSF Grat DMS

2 2 HANXIANG PENG AND ANTON SCHICK dhθ /dν is cotiuously differetiable i L 2 (ν) (whose derivative we may write as θ (1/2) κ(, θ) h θ for some κ(, θ) L k 2(H θ ) with κ(x, θ) dh θ (x) = ), ad if J(θ) = κ(x, θ) κ (x, θ) dh θ (x) is positive for each θ. The κ(, θ) is called the score fuctio at θ ad J(θ) the iformatio matrix at θ. 2. The Method Let (X, Y ) be a bivariate radom vector with distributio Q. Let F deote the distributio of X ad G the distributio of Y. We assume that F ad G belog to parametric models, but that Q is ukow otherwise. Let F = {F s : s Θ 1 } deote the parametric model for F with Θ 1 a ope subset of R k1 ad G = {G t : t Θ 2 } the parametric model for G, with Θ 2 a ope subset of R k2. We assume that the models F ad G are regular. Sice we assume that F belogs to F ad G belogs to G, there are α Θ 1 ad β Θ 2 such that F = F α ad G = G β. We write κ 1 (, s) for the score fuctio of the model F at s, deote the correspodig iformatio matrix by J 1 (s) = κ 1 (x, s) κ 1 (x, s) df s (x) ad set ψ 1 (x, s) = J 1 1 (s) κ 1(x, s), s Θ 1. We write κ 2 (, t) for the score fuctio of the model G at t, deote the correspodig iformatio matrix by J 2 (t) = κ 2 (y, t) κ 2 (y, t) dg t (y) ad set ψ 2 (y, t) = J 1 2 (t) κ 2(y, t), t Θ 2. Suppose we observe idepedet copies X 1,..., X of X oly. The a efficiet estimator ˆα of α must satisfy (1) ˆα = α + 1 ψ 1 (X j, α) + o p ( 1/2 ). Similarly, if oly idepedet copies Y 1,..., Y of Y are observable, the a efficiet estimator ˆβ of β must satisfy (2) ˆβ = β + 1 ψ 2 (Y j, β) + o p ( 1/2 ). Typically maximum likelihood estimators possess these properties. We assume from ow o that we have available estimators ˆα ad ˆβ that satisfy (1) ad (2). Now suppose that we observe idepedet copies (X 1, Y 1 ),..., (X, Y ) of the pair (X, Y ). The above estimators use oly the margial iformatio. Due to depedece, iformatio about α may also be cotaied i the data Y 1,..., Y, ad iformatio about β may be cotaied i the data X 1,..., X. Thus the estimators ˆα ad ˆβ may o loger be efficiet ad better estimators may be available. I this paper we demostrate a simple approach of usig the data Y 1,..., Y to improve the estimator ˆα of α. By symmetry this also provides a method of improvig the estimator ˆβ of β. By our assumptio o the models, G is domiated by a σ-fiite measure µ. Write g t for the desity of G t. Now let us look at a fuctio W from R Θ 2 to R m such that, for each t i Θ 2, (3) W (y, t) dg t (y) =,

3 (4) IMPROVING EFFICIENT MARGINAL ESTIMATORS WITH BIVARIATE DATA 3 W (y, t) κ 2 (y, t) dg t (y) =, (5) W (y, t)w (y, t) dg t (y) = I m, with I m the m m idetity matrix. Suppose also that (6) W (y, β )g 1/2 β (y) W (y, β)g 1/2 β (y) 2 dµ(y) as β β. It the follows from Schick (21) that 1/2 [W (Y j, β ) W (Y j, β) + E[W (Y, β) κ 2 (Y, β)](β β)] = o p (1) for every sequece β i Θ 2 such that 1/2 (β β) is bouded. This eve holds if we replace β by a discretized versio of ˆβ. To simplify otatio we require that this holds without discretizatio. I view of (4) we the have (7) 1/2 W (Y j, ˆβ ) = 1/2 W (Y j, β) + o p (1). For a k 1 m matrix D, cosider the estimator ˆα (D) of α defied by ˆα (D) = ˆα 1 DW (Y j, ˆβ ). It has expasio ˆα (D) = α + 1 [ψ 1 (X j, α) DW (Y j, β)] + o p ( 1/2 ). Thus 1/2 (ˆα (D) α) coverges i distributio to a ormal radom vector with mea zero ad the same dispersio matrix Ψ(D) as ψ 1 (X, α) DW (Y, β). It is straightforward to check that this dispersio matrix is miimized by D = D = E[ψ 1 (X, α)w (Y, β)], resultig i the miimal dispersio matrix Ψ(D ) = J 1 1 (α) D D. Sice D D is o-egative defiite, the estimator ˆα (D ) is asymptotically o more dispersed tha the estimator ˆα. I the case that X ad Y are idepedet, there is o iformatio o α cotaied i the data Y 1,..., Y ad o improvemet over ˆα is possible. I this sceario, we have D = ad Ψ(D ) = J1 1 (α). The matrix D is ukow ad must be estimated. This ca be doe by ˆD = 1 ψ 1 (X j, ˆα )W (Y j, ˆβ ). This estimator is cosistet if 1 (8) ψ 1 (X j, ˆα ) ψ 1 (X j, α) 2 = o p (1),

4 4 HANXIANG PENG AND ANTON SCHICK ad (9) 1 W (Y j, ˆβ ) W (Y j, β) 2 = o p (1). Thus, uder assumptios (7)-(9) we fid that the estimator satisfies (1) ˆα = α + 1 ˆα = ˆα ˆD 1 W (Y j, ˆβ ) [ψ 1 (X j, α) D W (Y j, β)] + o p ( 1/2 ). Hece (ˆα α) is asymptotically ormal with zero mea vector ad dispersio matrix Ψ(D ). Thus ˆα improves upo ˆα uless D =. We have already see that D = if X ad Y are idepedet. But D = ca also happe with depedet data. For example, if X ad Y have ormal margials ad are joitly ormal, the the maximum likelihood estimators of the meas ad variaces based o the margial observatios coicide with the likelihood estimators based o the joit distributio ad the margial estimators are also efficiet for the bivariate data ad thus caot be improved. I the ext sectio we shall discuss the magitude of the improvemets i some special cases. There we shall see that improvemets of up to 1 percet are possible. Remark 1. Our method ca also be used if the margial distributio G is kow. I this case we replace W (Y j, ˆβ ) by W (Y j ) where W eeds to oly satisfy W dg = ad W W dg = I m. 3. O the Magitude of the Improvemet To see how large a gai ca be achieved with our improvemet procedure we cosider ow a bivariate locatio model. Let f be a symmetric desity that has fiite Fisher iformatio I(f) for locatio. This meas that f is absolutely cotiuous ad I(f) = l 2 f (x)f(x) dx is fiite, where l f = f /f is the score fuctio for locatio. We take F to be the locatio model F = {F s : s R} geerated by f i which df s (x) = f(x s) dx. It is well kow that this model is regular with κ 1 (x, s) = l f (x s), J 1 (s) = I(f), ψ 1 (x, s) = l f (x s)/i(f). For G we take the locatio model geerated by a symmetric desity g with fiite Fisher iformatio I(g) for locatio. The G is regular with κ 2 (y, t) = l g (y t), J 2 (t) = I(g), ψ 2 (y, t) = l g (y t)/i(g).

5 IMPROVING EFFICIENT MARGINAL ESTIMATORS WITH BIVARIATE DATA 5 As the uderlyig joit distributio we take the bivariate probability measure Q which has a desity q of the form (11) q(x, y) = (1 + ρu(x α)v(y β))f(x α)g(y β), x, y R, for some (ukow) reals α ad β ad some ρ i [ 1, 1]. Here u ad v are measurable fuctios with values i [ 1, 1] such that u(x)f(x) dx = ad v(y)g(y) dy =. Some of the copula models are of this form. Ideed, the Farlie-Gumbel- Morgester copula model is of this type. See Hutchiso ad Lai (199) ad Nelse (1999) for properties ad applicatios of this copula model as well as for additioal refereces. We take W (y, t) to be of the form w(y t) for some fuctio w. The above distributio Q allows ow for a simple calculatio of D. Ideed, uder Q, we calculate I(f)D = ρe[u(x α)l f (X α)]e[v(y β)w(y β)] = ρ u(x)l f (x)f(x) dx v(y)w(y)g(y) dy. The required coditios (3)-(6) are implied if we take w so that (12) w(y)g(y) dy =, w 2 (y)g(y) dy = 1, w(y)g (y) dy =. A possible choice for w is w (x) = 21 [ c,c] (x) 1 = { 1, x c, 1, x > c, with c the third quartile of g. Ideed, the the first part of (12) follows from the choice of c, the secod part follows from the fact that w = 1, ad the third part follows as w is eve ad g is odd. The asymptotic relative efficiecy (ARE) of the proposed estimator ˆα to the margial estimator ˆα is the ratio of the asymptotic variace 1/I(f) D 2 of ˆα ad the asymptotic variace 1/I(f) of ˆα : ARE = 1 I(f)D 2 = 1 ρ2 I(f) ( u(x)l f (x)f(x) dx As v is bouded by 1, we see that ( ARE 1 ρ2 2 u(x)l f (x)f(x) dx) I(f) v(y)w(y)g(y) dy) 2. with equality if v = w = w. Note that u = sig(l f ) is a odd fuctio i view of the symmetry of f so that sig(l f (x))f(x) dx = ad that u = 1. Note also that this u maximizes u(x)l f (x)f(x) dx subject to u 1 ad u(x)f(x) dx =. Thus we obtai ( (13) ARE 1 ρ2 2 f (x) dx) I(f) with equality if u = sig(l f ) ad v = w = w. Example 1. Let us choose f ad g to be the stadard ormal desity p give by p(x) = (1/ 2π) exp( x 2 /2). This desity has fiite Fisher iformatio I(p) = 1 ad score fuctio l p (x) = x. Based o the margial observatios X 1,..., X, the maximum likelihood estimator ˆα is the sample mea. The sample mea is

6 6 HANXIANG PENG AND ANTON SCHICK efficiet i the (stadard) ormal locatio model F with asymptotic variace 1. For v = w = w ad u(x) = sig(x), we obtai ARE = 1 ρ 2 2 ( ) 2 x exp( x 2 2ρ 2 /2) dx = 1 π π. Thus the efficiecy gai ca be as large as is 2/π or about 63.7 percet. Example 2. Let us choose f ad g to be the Cauchy desity give by p(x) = 1/(π(1+x 2 )). This desity has fiite Fisher iformatio for locatio I(p) = 1/2 ad score fuctio l p (x) = 2x/(1 + x 2 ). Based o the margial observatios X 1,..., X, the maximum likelihood estimator ˆα is a solutio to the score equatio X i ˆα 1 + (X i ˆα ) 2 =. i=1 This estimator ˆα is efficiet i the Cauchy-locatio model F with asymptotic variace 2. The third quartile of the Cauchy desity p is c = 1. For v = w = w = 21 [ 1,1] 1 ad u(x) = sig(x), we have ARE = 1 2ρ 2( 2 2x ) 2 π(1 + x 2 ) 2 dx 8ρ 2 = 1 π 2. Thus the efficiecy gai ca be as large as is 8/π 2 or about 81 percet. Example 3. Let us choose f ad g to be the Laplace (or double expoetial) desity p give by p(x) = (1/2) exp( x ). This desity has fiite Fisher iformatio I(p) = 1 ad score fuctio l p (x) = sig(x). The maximum likelihood estimator ˆα is the sample media of X 1,..., X. It is efficiet i the Laplace locatio model F ad has asymptotic variace 1. The third quartile of the Laplace desity p is c = log 2. For v = w = w = 21 [ log 2,log 2] 1 ad u(x) = sig(x), we obtai ARE = 1 ρ 2( 2 e dx) x = 1 ρ 2. Thus the efficiecy gai ca be as large as 1 or 1 percet. 4. Expoetial margials Let us ow assume that F = G = {F s : s > }, where F s is the expoetial distributio with mea s, i.e., F s has desity f s (x) = f(x/s)/s, where f(x) = exp( x)1 (, ) (x), x R. This model is regular with score fuctio κ(x, s) = (x s)/s 2 ad iformatio J(s) = 1/s 2. Thus ψ 1 (x, α) = x α ad ψ 2 (y, β) = y β. The maximum likelihood estimator based o a radom sample from this model is the sample mea ad is efficiet. Thus we take ˆα = X = (X 1 + +X )/ ad ˆβ = Ȳ = (Y 1 + +Y )/. Here we take W to be of the form W (y, t) = w(y/t) for some fuctio w. The required coditios (3)-(6) are implied if w satisfies (14) w(y)f(y) dy =, w 2 (y)f(y) dy = 1, w(y)(y 1)f(y) dy =.

7 IMPROVING EFFICIENT MARGINAL ESTIMATORS WITH BIVARIATE DATA 7 Possible choices for w are give by w = w (c,d) = (1 (c,d) p)/ p(1 p), where p = exp( c) exp( d) ad < c < 1 < d < are chose to satisfy c exp( c) = d exp( d). We shall look at two differet joit distributios Q. The first joit distributio has distributio fuctio H give by (15) H(x, y) = γf (x/α)f (y/β) + (1 γ) mi(f (x/α), F (y/β)), x, y R, for some < γ < 1, where F is the distributio fuctio of f. The secod joit distributio has a desity q of the form (16) q(x, y) = (1 + ρu(x/α)v(y/β))f(x/α)f(y/β)/(αβ), x, y R, for some ρ i [ 1, 1]. Here u ad v are measurable fuctios with values i [ 1, 1] such that u(x)f(x) dx = ad v(y)f(y) dy =. We shall see that for the first joit distributio our method does ot provide ay improvemet. Ideed, H is the distributio fuctio of (α X, β(1[u γ]ỹ + 1[U > γ] X) where X, Ỹ ad U are idepedet, X ad Ỹ are expoetially distributed with mea 1, ad U is uiformly distributed o (, 1). Thus we ca express D = αe[( X 1)w(1[U γ]ỹ + 1[U > γ] X)] = α(e[1[u γ]( X 1)w(Ỹ )] + E[1[U > γ]( X 1)w( X)]. I view of the idepedece of U, X, Ỹ ad the last coditio i (14), we fid D =. Thus o improvemet is possible. For the secod distributio we calculate D = ρe[u(x/α)(x α)]e[v(y/β)w(y/β)] = αρ u(x)(x 1) exp( x) dx v(y)w(y) exp( y) dy. The asymptotic relative efficiecy of the proposed estimator ˆα to the margial estimator ˆα is the ratio of the asymptotic variace α 2 D 2 of ˆα ad the asymptotic variace α 2 of ˆα, amely, ARE = 1 D2 α 2 = 1 ρ2( u(x)(x 1) exp( x) dx v(y)w(y) exp( y) dy) 2. Let us ow take u = u = 1 (1, ) 1 (,1 δ) with δ = log(e 1). The the itegral ivolvig u becomes δ = exp( 1) + (1 δ) exp(δ 1) = 1 (1 e 1 ) log(e 1), ad accordigly, ( 2. ARE = 1 ρ 2 δ 2 v(y)w(y) exp( y) dy) For the choice w = w (c,d) we obtai p(1 p) ARE 1 ρ 2 δ 2 max(p, 1 p) with equality for v = (1 (c,d) p)/ max(p, 1 p). If c ad d are such that p = 1/2, the ARE = 1 ρ 2 δ 2. Sice δ , we see that improvemets of up to 56.7 percet are possible.

8 8 HANXIANG PENG AND ANTON SCHICK Refereces [1] Bickel, P.J., Klaasse, C.A.J., Ritov, Y. ad Weller, J.A. (1993). Efficiet ad adaptive estimatio i semiparametric models. Johs Hopkis Uiv. Press, Baltimore. [2] Hutchiso, T. P., ad Lai, C. D. (199) Cotiuous bivariate distributios, emphasisig applicatios. Rumsby Scietific Publishig, Adelaide. [3] Nelse, R. (1999). A itroductio to copulas. Lecture Notes i Statistics, 139. Spriger- Verlag, New York. [4] Haxiag Peg ad Ato Schick (24). Estimatio of Liear Fuctioals of a Bivariate Probability with Ukow Parametric Margials. Statistics & Decisios [5] Schick, A. (21). O asymptotic differetiability of averages. Statist. Probab. Lett., 51, [6] Schick, A. ad Wefelmeyer, W. (28). Some developmets i semiparametric models. To appear i: J. Stat. Theory Pract. Haxiag Peg, Departmet of Mathematical Scieces, Purdue School of Scieces, Idiaa Uiversity-Purdue Uiversity at Idiaapolis, Idiaapolis, IN , USA Ato Schick, Departmet of Mathematical Scieces, Bighamto Uiversity, Bighamto, NY , USA

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials