On Predictive Density Estimation for Location Families under Integrated Absolute Error Loss

Transcription

1 On Preitive Density Estimation for Loation Families uner Integrate Absolute Error Loss Tatsuya Kubokawa a, Éri Marhanb, William E. Strawerman a Department of Eonomis, University of Tokyo, 7-3- Hongo, Bunkyo-ku, Tokyo , JAPAN ( tatsuya@e.u-tokyo.a.jp) b Université e Sherbrooke, Département e mathématiques, Sherbrooke Q, CANADA, JK R ( eri.marhan@usherbrooke.a) Rutgers University, Department of Statistis an Biostatistis, 50 Hill Center, Bush Campus, Pisataway, N.J., USA, ( straw@stat.rutgers.eu) Summary This paper is onerne with estimating a preitive ensity uner integrate absolute error (L ) loss. Base on a spherially symmetri observable X p X ( x µ ), x, µ R, we seek to estimate the (unimoal) ensity of Y q Y ( y µ ), y R. We fous on the benhmark (an maximum likelihoo for unimoal p) plug-in ensity estimator q Y ( y X ) an, for 4, we establish its inamissibility, as well as provie plug-in ensity improvements, as measure by the frequentist risk taken with respet to X. Sharper results are obtaine for the sublass of sale mixtures of normal istributions whih inlue the normal ase. The finings rely on the uality between the preitive ensity estimation problem with a point estimation problem of estimating µ uner a loss whih is a onave funtion of ˆµ µ, Stein estimation results an tehniques appliable to suh losses, an further properties speifi to sale mixtures of normal istributions. Finally, (i) we aress univariate impliations for ases where there exist parametri restritions on µ, an (ii) we show quite generally for logonave q Y that improvements on the benhmark mle an always be foun among the sale expane preitive ensities q Y ( (y x) ), with positive but not too large. AMS 00 subjet lassifiations: 6C0, 6C86, 6F0, 6F5, 6F30 Keywors an phrases: Conave loss; Dominane; Frequentist risk; Inamissibility; L loss; Multivariate normal; Plug-in; Preitive ensity; Restrite parameter spae; Sale mixture of normals; Stein estimation. Introution The evelopments of this paper relate to spherially symmetri an inepenently istribute X µ p X ( x µ ), Y µ q Y ( y µ ) ; x, y, µ R ; () with p an q known Lebesgue ensities, not neessarily equal, an µ is unknown. The set-up in () inlues the normal moel with X µ N (µ, σ XI ), Y µ N (µ, σ Y I ), () as well as sale mixtures of normal istributions (Definition.).

2 For preitive analysis purposes, researhers are intereste in speifying a preitive ensity ˆq(y; x), base on observation x, as an estimate of the ensity q Y ( y µ ). In turn, suh a ensity may play a surrogate role for generating either future or missing values of Y. Our interest an motivation here lies in assessing the effiieny of suh preitive ensities with integrate absolute error loss (hereafter referre to as L ) an orresponing frequentist risk, with L (µ, ˆq) = q Y ( y µ ) ˆq(y) y, (3) R R(µ, ˆq) = q Y ( y µ ) ˆq(y; x) y} p X ( x µ ) x. (4) R R { L loss is a quite reognizable an an appealing istane. It has playe a prominent role in assessing the effiieny of ensity estimators over the years, both in a nonparametri an a parametri setting (e.g., Devroye an Gyrfi, 985; DasGupta an Lahiri, 0; among many others). It is also intrinsi in the sense that, for one-to-one funtions g : R R with inverse jaobian J, the L istane between the ensities of g(y ) an g(y ) is inepenent of g; i.e., R q Y ( g (y) µ ) J ˆq(g (y)) J y is inepenent of g. The tratability of the L istane an its assoiate risk is another matter, an analytial results relative to the performane of preitive ensity estimators are laking. In terms of a benhmark proeure, the evaluation of the minimum risk equivariant preitive ensity estimator, whih is the Bayes proeure assoiate with the onstant prior measure π 0 (µ) = is quite hallenging. An more generally, we have been unable to provie the form of Bayesian preitive ensity estimators for a given prior π. With suh a pauity of results an bearings, we fous on the performane of the plug-in preitive ensity estimator q Y ( y X ), y R, whih is also for unimoal p X the maximum likelihoo preitive ensity estimator (mle) of q Y ( y µ ), y, µ R. Our main objetive an ommon theme is to provie ominating preitive ensity estimators of q Y ( y X ), y R. This is ahieve in Setion for 4, quite generally with respet to moel () by substituting the estimator X by a more effiient ˆµ(X) as a plug-in estimator. In Setion 3, we obtain improvements in the univariate ase by variane expansion of the preitive ensity estimator. Here are further etails. In Setion, we fous on the performane of plug-in preitive ensity estimators q Y ( y ˆµ(X) ), y R, with ˆµ(X) an estimator of µ. For 4, we provie ominating estimators of the plug-in q Y ( y X ), y R. This is ahieve by apitalizing on an expliit representation for the L istane (Lemma.) between two ensities of the same spherially symmetri family, whih implies that our preitive ensity estimation problem for plug-in estimators is ual to a point estimation problem uner a loss whih is a onave funtion of ˆµ µ (Corollary.). Using Stein estimation results an tehniques appliable to suh onave losses (e.g., Branwein an Strawerman, 99, 980; Branwein, Ralesu, Strawerman, 993), we establish the inamissibility of plug-in ensities q Y ( y X ) for 4 an obtain ominating preitive ensity estimators. In Two-point priors are an exeption with the Bayes estimator given by q Y ( y me(µ x) ), with me(µ x) being the posterior meian.

3 subsetion., we provie further sharper evelopments for sale mixtures of normals p X an q Y, whih inlue of ourse the normal ase. The ual loss funtions that intervene are of interest on their own an our finings also represent ontributions from the point estimation perspetive. Namely, the ual loss for the normal moel turns out to be the interesting loss L(µ, ˆµ) = 4Φ( ˆµ µ ), where Φ is the stanar normal f. In subsetion.3, for univariate situations where µ is either restrite to an interval (a, b), or restrite to a half-interval (a, ), we make use of existing results for strit-bowle shape losses an our uality results to show that the plug-in ensity estimator q Y ( y ˆµ πu (X) ) ominates the plug-in q Y ( y X ) for log-onave ensity p X, where ˆµ πu (X) is the Bayes point estimator of µ assoiate with a uniform prior on the restrite parameter spae an the given ual loss. For Kullbak-Leibler loss an normal moels as in (), similar inamissibility results appliable to the MRE preitive ensity estimator as well as plug-in preitive ensity estimators, as well as onnetions between preitive ensity estimation an Stein estimation have been obtaine by Komaki (00), George, Liang an Xu (006), Brown, George an Xu (008), an Fourrinier et al. (0), among others. Various finings for integrate square error loss an spherially symmetri istributions are given in Kubokawa, Marhan an Strawerman (05A). In Setion 3, we fous on the univariate ase an sale expansions of the form ˆq (y; x) = q Y ( y x ), y R, with >. We show the plug-in ensity q Y ( y X ), y R, is ominate by a sublass of suh sale expansions ˆq, with positive but not too large, as long as q is logonave. This applies to the normal ase, as well as moels like Logisti an Laplae, among others. This is paraoxial in the sense that the variane assoiate with the plug-in ensity q Y ( y X ) mathes the variane of the true ensity q Y ( y µ ), but that improvements an be nevertheless foun among the ˆq s with >. Suh a result goes bak to Aithison (975) who showe iretly, for Kullbak-Leibler loss an univariate normal moels, that the plug-in ensity ˆq N(X, σy ) is ominate by the sale expansion (an MRE preitive ensity estimator) ˆq mre N(X, σy + σ X ). This type of phenomenon was generalize, for multivariate normal moels, plug-in ensity estimators q Y ( y ˆµ(X) ) an Kullbak-Leibler loss by Fourrinier et al. (0), an aresse reently for integrate square error loss by Kubokawa, Marhan an Strawerman (05A). In the former Kullbak-Leibler ase, the authors showe that, universally for any (non-egenerate) estimator ˆµ(X), any imension, any restrite or not parameter spae, ominating preitive ensity estimators q Y ( y ˆµ(X) ) of q Y ( y ˆµ(X) ) an always be foun among hoies >. In the latter ase, the authors provie results for ˆµ(X) = ax with 0 < a with similar sale expansion improvements always available. L loss an plug-in estimators. An ientity for L istane an general ominane results of plug-in preitive ensity estimators We begin with a useful L istane ientity. 3

4 Lemma.. Let Y = (Y,..., Y ) be a spherially symmetri istribute ranom vetor with unimoal, Lebesgue ensity f µ (y) = q Y ( y µ ); y R. Then for any µ, µ R, the L istane between f µ an f µ is given by ρ L = q Y ( y µ ) q Y ( y µ ) y = 4F ( µ µ R ), (5) where F (t) = P 0 (Y t), t R, is the univariate umulative istribution funtion of Y when µ = 0. Proof. We have q Y ( y µ ) q Y ( y µ ) y µ y µ L(y) 0, where L(y) = (µ µ ) y + µ µ. Setting A = {y R : L(y) 0}, we obtain splitting the integration on A an its omplement A ρ L = P µ (Y A) P µ (Y A) + P µ (Y A ) P µ (Y A ) = {P µ (L(Y ) 0) + P µ (L(Y ) 0) }. (6) Observe that L(Y ) is a linear funtion of the spherially symmetri istribute Y. For suh linear funtions, we have (e.g., Muirhea, 005) (l Y + k) (l µ + k) l F, for all l R {0}, k R. We thus obtain P µ (L(Y ) 0) = F ( (L(µ ))) = F ( µ µ ). Similarly, we obtain P µ (L(Y ) 0) = F ( µ µ ), an the esire expression for ρ L follows from (6). Remark.. For the multivariate normal ase, ientity (5) was given an erive in a ifferent manner by DasGupta an Lahiri (0). An existing referene for the general ase seems likely to us, but we oul not fin suh a referene. Observe that the istane ρ L is always a onave funtion of µ µ on (0, ) sine F is unimoal, an also of µ µ given that F is inreasing. Remark.. The L istane formula above also provies an expliit form for the muh stuie overlap oeffiient (e.g., Weizman, 970) OVL between two spherially symmetri ensities. The latter is efine for ensities g an g as OV L(g, g ) = min(g (y), g (y)) y, (7) R an is relate to the L istane through the ientity OV L(g, g ) = ρ L (g, g ) given that min(g (y), g (y)) = g (y) + g (y) g (y) g (y) for all y. Corollary.. For estimating a unimoal spherially symmetri Lebesgue ensity q Y ( y µ ), y R, uner L loss an base on X p X ( x µ ), the frequentist risk of the plug-in ensity estimator q Y ( y ˆµ(X) ) is equal to the frequentist risk of the point estimator ˆµ(X) of µ uner loss 4F ( ˆµ µ ), with F being the ommon marginal f assoiate with q Y. Consequently, q Y ( y ˆµ (X) ) ominates q Y ( y ˆµ (X) ) iff ˆµ (X) ominates ˆµ (X) uner loss F ( ˆµ µ ). 4

5 Proof. This is a iret onsequene of Lemma.. Sine the ual problem esribe above involves loss funtions l( µ ) with l(t) = F ( t ) being onave (see Remark.), we onsier using Stein estimation tehniques an results for suh onave losses ( Branwein an Strawerman 99, 980); Branwein, Ralesu an Strawerman, 993), along with Corollary., to obtain ominating estimators of the plug-in ensity q Y ( y X ), y R, whih we now proee to o, elaborate on, an illustrate. For what follows, we enote f as the ensity of X µ uner p X an we reall that f(t) = π/ Γ(/) t p x (t ) (e.g., Muirhea, 005). Here is an aaptation of Theorem. of Branwein an Strawerman (980) appliable to Baranhik type estimators, an followe by relate inferenes for improving on plug-in ensity estimators uner L loss. Theorem.. (Branwein an Strawerman, 980) Let X have a spherially symmetri istribution with ensity p X ( x µ ), x R, with respet to σ-finite measure ν. For 4 an for estimating µ R uner loss l( ˆµ µ ) with l non-ereasing an onave on, estimators ˆµ a,r( ) (X) = ( a r(x X) )X ominate X, an are thus minimax, X X provie: (i) 0 r( ) an r( ) 0; (ii) r(t) is non-ereasing for t > 0; (iii) r(t)/t is non-inreasing for t > 0; (iv) 0 < E px l ( X µ ) < ; (v) 0 < a ( ) E h, where the expetation is taken with respet to the ensity (R ) h(s) on proportional to l (s ) f(s) = π/ Γ(/) l (s ) s p x (s ). This following result follows from Corollary. an Theorem.. Corollary.. For estimating a unimoal spherially symmetri Lebesgue ensity q Y ( y µ ), y, µ R an 4, uner integrate L loss an base on X p X ( x µ ), a plug-in Baranhik ensity estimator q Y ( y ˆµ a,r( ) (X) ), with ˆµ a,r( ) (X) = ( a r(x X) )X, X X ominates the plug-in q Y ( y X ) provie onitions (i), (ii), an (iii) of Theorem. are satisfie as well as: (iv ) 0 < E px ( q Y ( X µ /6) X µ ) < ; (v ) 0 < a ( ) (0, ) u 3 p X (u) F ( u 4 ) ν(u). (0, ) u 5 p X (u) F ( u 4 ) ν(u) Proof. This follows from Corollary. an Theorem. with l(u) = F ( u ) an l (u) = F ( u ), as well as the hange of variables u = u s. 5

6 Remark.3. In our set-up, the moel ensity q Y etermines the loss l via Lemma. an is thus taken to be unimoal an Lebesgue. On the other han, there no restritions on p X other than risk-finiteness for the estimators ˆµ a,r( ) (X). Conition (iv ) is weak. For instane, it is satisfie when both the ensities q Y an p X are boune. The upper boun for the multiplier a of the estimator ˆµ a,r( ) (X) in (v ) epens on both q Y an p X. Here is an evaluation for the partiular ase when both p X an q Y are normal ensities. Example.. (normal ase) For the normal ase () with q Y (u) = (πσ Y ) / e u/σ Y, F (t) = (πσ Y ) / e t /σ Y, an px (u) = (πσ X ) / e u/σ X, Corollary. applies with (iv ) satisfie an (v ) speializing to 0 < a ( ) 0, u 3 e u/σ X e u/8σ Y u 0, u 5 e u/σ Xe u/8σ Y u = ( )( 3) 8σ X σ Y σ X + 4σ Y. (8) We point out that a simultaneous ominane result is available for a family of p X moels by taking the infimum with respet to p X on the rhs of (v ). For the normal ase, if we have for instane X N (µ, σx I ) with σx unknown, but known to boune below by a X > 0, then simultaneous ominane ours for all suh p X s by taking 0 < a ( )( 3). Improvements for sale mixture of normals 8a X σ Y a X +4σ Y Distributions in () inlue the sublass of sale mixture of normals, with examples given by the multivariate Cauhy, Stuent, Logisti, Laplae, Generalize Hyperboli an Exponential Power istributions, among others (e.g., Anrews an Mallows, 974). Definition.. Moel () is referre to as a sale mixture of normals moel whenever p X (t) = (πv) / e t v G(v), qy (t) = (πw) / e t w H(w), (9) for t R an W G, V H are inepenently istribute mixing ranom variables on, for whih we further assume that E(V / ) an E(W / ) are finite. We enote suh moels as X µ SN (G) an Y µ SN (H). Further evelopments for sale mixtures of normals are provie in this setion an lea to wier lasses of ominating estimators than those given by Corollary.. We revisit this latter orollary for situations in () where X µ SN (G), Y µ SN (H). (0) We efine Z as a ranom variable, F Z as its f, an τ as a bivariate f suh that Z = 4Z Z Z + 4Z, with (Z, Z ) τ(z, z ) z( )/ (z + 4z ) / G(z ) H(z ). () Theorem.. Let X p X ( x µ ) an Y q Y ( y µ ), x, y, µ R, be sale mixtures of normals as in (0) an onsier estimating q Y ( y µ ) uner L loss base on X. 6.

7 (a) For >, the plug-in ensity estimator q Y ( y ˆµ(X) ) ominates the plug-in ensity q Y ( y X ) provie ˆµ(X ) ominates X uner loss ˆµ µ an for X p ( x µ ), with p ( s ) = K s (0, ) where K is a normalization onstant. (πz) / e s z F Z (z), s R, () (b) In partiular, a plug-in Baranhik ensity estimator q Y ( y ˆµ a,r( ) (X) ), with ˆµ a,r( ) (X) = ( a r(x X) )X, ominates the plug-in ensity q X X Y ( y X ) provie onitions (i), (ii), an (iii) of Theorem. are satisfie, 4, the expetations E(Z / ) an E(Z 3/ ) are finite, an 0 < a ( 3) E(Z / ). E(Z 3/ ) Proof. (a) We apply Corollary.. We thus seek onitions for whih the ifferene in risks (µ, ˆµ) = E µ [ l( ˆµ(X) µ ) l( X µ ) ] is less than 0, where l( ˆµ µ ) = F ( ˆµ µ ). We apply the inequality l(s) l(t) < l (t)(s t) for stritly onave l an s t, whih implies for the ifferene in losses that l( ˆµ(x) µ ) l( x µ ) < l ( x µ ) ( ˆµ(x) µ x µ ), (3) for all x, µ R suh that x ˆµ(x). Observe that l ( x µ ) = = x µ F x µ ( ) (πw) / e x µ 8w H(w), (4) x µ 0 sine the marginal istributions assoiate with a sale mixture of normals as in (0) are themselves univariate sale mixtures of normals with the same mixing istribution. 3 Now, using (3) an (4), it follows that [ ( ˆµ(X) µ (µ, ˆµ) < Eµ X X µ ) X µ ( ˆµ(x) µ x µ ) = R x µ 0 ( ˆµ(x) µ x µ ) R x µ 0 ( ˆµ(x) µ x µ ) R x µ whih establishes part (a) (πw) / e X µ 8w (π) + (wv ) ( 8πwv v + 4w ) ] H(w) e x µ ( v+4w 8wv ) G(v)H(w) x x µ e ( 8wv v+4w ) τ(v, w) x (πz) / e x µ z F Z (z) x, For =, the ensity in () is not well efine. 3 It is not the ase that spherially symmetri istributions share a similar onsisteny property, but it is true inee for sale mixtures of normals istributions (e.g., Kano, 994). 7

8 (b) We apply part (a). We show below that the ensity p ( s ) is a sale mixture of normals. This permits us to apply the ominane result of Strawerman (974) for Baranhik estimators satisfying onitions (i), (ii), (iii) of Theorem., in ases where both E p 0 X an E p 0 X are finite, an for 0 < a /(E p 0 X ). The finiteness onitions are satisfie for 4 an with the finiteness of E(Z / ) an E(Z 3/ ), an a alulation yiels E p 0 X = = = R x 3 R x 0 R 0 0 (πz) / e x z F Z (z) x 0 (πz) / e x z F Z (z) x (πz) / e x x 3 z R x (πz) / e x z E(Z 3/ ) ( 3) E(Z / ), x F Z (z) x F Z (z) using expetation expressions for a entral χ istribution. This yiels the esire result. It remains to show that p ( s ), s R, is a sale mixture of normals ensity. Reall that, in general, a spherially symmetri ensity f( t µ ) is a sale mixture of normals if an only if f is ompletely monotone on (0, ), i.e., ( ) k f (k) (t) 0 for t > 0 an k = 0,,,... (e.g., Berger, 975). Sine both t / an (0, ) (πz) / e t z τ(z) are ompletely monotone, it follows that their prout is ompletely monotone (e.g., Feller, 966, page 47) an that the ensity in () is inee a sale mixture of normals. Remark.4. Written in terms of the mixing variane fs (H, G) in (0), Theorem. (b) s boun on a is, using (), equal to 8( 3) z R / z / + (z +4z G(z ) ( )/ ) H(z ) z R 3/ z / + (z +4z G(z ) ( 3)/ ) H(z ). Example.. (normal ase) In the normal ase () whih arises as a partiular ase of (0) for egenerate V, W, we obtain that Z in () is also egenerate with P (Z = z 0 ) =, with z 0 = 4σ X σ Y. In this ase as well, we obtain σx +4σ Y p ( s ) s (πz 0) / e s z 0, whih is the ensity of a Kotz istribution (see for instane Naarajah, 003). By virtue of part (a) of Theorem., minimax or ominane results appliable to this partiular Kotz istribution generate plug-in N (ˆµ(X), σy I ) ensity estimators (suh as those in part b) whih ominate the plug-in ensity of a N (X, σy I ) uner L loss. The ut-off point in part (b) reues to ( 3)z 0 = 8( 3)σ X σ Y. In omparison to Corollary. s σx +4σ Y utoff point given in (8), the ut-off point here is larger by a multiple of /( ). 8

9 Proeeing with a numerial illustration, we set σx = (without loss of generality) an onsier the (smooth) Baranhik estimator ˆµ a (X) = ( X X )X, orresponing to r(t) = X X+a t. whih satisfies the ominane onition as long as 0 < a a t+a 0(, σ Y ) = 8( 3)σ Y. +σy We selet the upper ut-off point a 0 (, σ Y ) an ompare the risks of the plug-in preitive ensity estimators q Y ( y X ) an q Y ( y ˆµ a0 (X) ). Figure (left) shows the ratio of these risks for = 6, σ Y = 0.5,,. Our theoretial results tells us that the ratio is less than one. In aorane with the traitional performane of shrinkage estimators, empirial finings show that (relative) improvement is most important when λ = µ is lose to 0 an suh an improvement issipates for large λ. Furthermore, these gains are amplifie when σy inreases (i.e., larger unertainty in Y is better mitigate by the shrinkage proeure). Similarly, these gains are amplifie as the imension rises. With the maximum gain at 0, Figure (right) illustrates these above features of the ratio of risks evaluate at µ = 0 as a funtion of σ Y, an illustrates how important the gains an be. In pratie, one may shrink to any prior plausible value µ 0 of µ (by using ˆµ(X) = µ 0 + ( r( X µ 0 ) X µ 0 )(X µ 0 ), with improvement expete to be most important for small µ µ 0. Figure : Ratios R(µ,q Y ( y ˆµ a0 (X) ) R(µ,q Y of risks for = 6, σ ( y X ) X =, σ Y =,, 0.5, as funtions of λ = µ (on the left). For fixe λ, ratios inrease in σy. Same ratios at µ = 0 for = 4, 6, 8 (on the right). These ratios erease in. Example.3. (ases where the mixing istributions are lower boune) Suppose in (0) that the mixing varianes are lower boune by positive values, in the sense that there exists known positive onstants a X an a Y suh that G (a X ) = 0 an H (a Y ) = 0 (G an H nee not be known). In suh ases, Theorem. s utoff point ( 3) E(Z 3/ ) for the onstant a of the plug-in Baranhik ensity estimator ˆµ E(Z / ) a,r( )(X) an be lower boune as follows. We have by the ovariane inequality Cov (g (Z), g (Z)) 0 9

10 for ereasing g (z) = z 3/, inreasing g (z) = z, an by making use of (): ( E(Z / ) E(Z 3/ ) E(Z) 4 E ( + 4 ) ) 4 ( + 4 ) = Z Z a Y a X 4a Xa Y 4a Y + a X Theorem. s utoff point is thus boune below by ( 3)E(Z) whih in turn is boune below by 8( 3) a Xa Y 4a Y +a X. In the normal ase, these bouns are exat an take us bak to the boun given in Example.. The range of preitive ensity Baranhik estimators whih ominate the plug-in q Y ( y X ) is thus narrower with the lower boun, but the boun is simple, an ominane applies simultaneously for all sale mixture of normals in (0) suh that the lower bouns a X an a Y on the mixing variane apply..3 Improvements in the ase of univariate parametri restritions We briefly expan on ominane results appliable to univariate ( = ) ases where µ is either restrite to an interval (a, b), or to a half-interval (a, ). Combining Corollary. s uality with point estimation loss F ( ˆµ µ ), whih is a stritly bowle shape funtion of ˆµ µ on R, with finings of Marhan an Strawerman (005), or again Kubokawa an Saleh (994), we erive the following ominane result for estimating an univariate ensity q Y ( y µ ) base on X p X ( x µ ) for ases (suh as the normal ase) where the family of ensities for X has an inreasing monotone likelihoo ratio (or equivalently p X (t ) is logonave in t ). Corollary.3. For estimating an unimoal an univariate symmetri Lebesgue ensity q Y ( y µ ), y R, µ (a, b) (or µ (a, )) uner L loss an base on X p X ( x µ ) with p X (t ) logonave, the plug-in ensity estimator q Y ( y ˆµ U (X) ) with ˆµ U (X) the Bayes estimator of µ with respet to the uniform prior on (a, b) (or on (a, )) ominates the plug-in ensity estimator q Y ( y X ). Proof. Sine ˆµ πu (X) ominates the MRE estimator X as shown by Marhan an Strawerman (005) for loss funtions ρ( µ) with ρ strit bowle shape, ρ(t) > ρ(0) = 0 for all t 0, an logonave ensities, the result follows from part (a) of Corollary.. 3 Sale expansion improvements on plug-in preitive ensity estimators We onsier here moel () in the univariate ase an simplify the notation for onveniene writing X p(x µ), Y q(y µ), x, y, µ R, with p, q known an even. With normal pf s p an q representing the key example for further referene, we investigate the performane of preitive ensity estimators y x q( ) of q(y µ), for >, uner L loss as in (3) q(y µ) q(y x ) y, (5) R 0

11 an assoiate frequentist risk { R (µ) = q(y µ) q(y x } ) y R R p(x µ) x, µ R. By the hange of variables (x, y) (x µ, y µ) an by exploiting the assumption that p an q are even, the above risk may be expresse as { R (µ) = q(y) q(y x } ) y p(x) x. (6) R Observe that the risk is thus onstant as a funtion of µ an that an optimal hoie of exists for any given (p, q). We are partiularly intereste in seeking improvements in terms of risk of the maximum likelihoo plug-in estimator q(y x), y R, by sale expansions q( y x ) with >. The rossings of the ensities q(y) an ) will be ritial in eomposing the loss in (5). One may verify without muh iffiulty that suh ensities ross one on R an one on for ases suh as normal (also see Example 3.) an Laplae (with q(y) = y e σ ). σ The next result establishes suh behaviour quite generally uner the assumption that q is logonave. q( y x Lemma 3.. Suppose q is an even, ifferentiable a.e., an logonave ensity on R. Then, for any fixe (x, ) with > an x 0, (i) there is exatly one positive (y + ) an one negative (y ) solution of q(y) y x q( ) = 0. Furthermore, (ii) q(y) > y x q( ) iff y < y < y +. Proof. With the assumptions, we may write q(y) = Ae h(y) with h even, h(y) inreasing in y, an h (y) inreasing in y. We break up the proof into separate parts: (A) y > 0 an (B) y < 0 an we assume x > 0 without loss of generality. Note that sgn{q(y) q(y x ) } = sgn{d(y)}, (7) with D(y) = h(y) h( y x ) log(). If (i) hols, part (ii) follows as D(0) = h(0) ) log() < 0, given the properties of h an sine >. h( x (A) Case y > 0. For y x, we have D (y) = h (y) h ( y x ) h (y)( ) h (x)( ) > 0. (8) Note also that D (y) 0 for 0 < y < x sine h(y) is inreasing in y for y > 0, an h( y x ) is ereasing in y for y < x. Hene, D( ) is inreasing on R +. Sine D(0) < 0, we onlue that there exists exatly one positive solution of the equation q(y) y x q( ) = 0.

12 (B) Case y < 0. Set z = y. By symmetry, for y < 0, D(y) = h(z) h( z + x ) log() = T (z) (say). Set z 0 = x. For z z 0, we have z z+x an T (z) = h(z) h( z+x) log() log() < 0 given that h is inreasing on. There is hene no root of D on [0, z 0 ]. For z > z 0, we have T (z) = h (z) h ( z + x ) h (z)( ) h (z 0 )( ) > 0. (9) Therefore T is stritly monotone on (z 0, ) with T (z) as z, an we infer that there exists a unique root of T in the interval (z 0, ) an none in the interval (0, z 0 ), whih establishes the result. Theorem 3.. Suppose q is an even, ifferentiable a.e., an logonave ensity on R. Suppose p is an even ensity. Then, the risk R (µ) of the preitive ensity y x q( ) in estimating q(y µ), y R, is given by R (µ) = 4(A + A ), with ( A = F ( y ) x ) F (y ) p(x) x, ( A = F (y + ) F ( y ) + x ) p(x) x, where F is the f assoiate with q, an y an y + are quantities epening on x representing the negative an positive rossing respetively of y x q( ) an q(y). Proof. With q(y) ) iff y y y + by virtue of Lemma 3., the given expression for the risk R (µ) follows by evaluating the inner integral in (6) separately on the omains (, y ), [y, y + ], (y +, ), an by olleting terms. q( y x Remark 3.. Taking in Theorem 3., we obtain y an y + x/. This yiels A 0, A (F (x/) ) p(x) x an R (µ) = (8F (x/) 4) p(x) x = 8 F (x/) p(x) x. (0) The above may be written as 4 F ( x /) p(x) x, an we point out that this also R follows from Corollary.. Example 3.. (normal ase) For a normal pf q(y) = π e y, the onlusions of Lemma 3. may be iretly verifie with rossings y an y + given expliitly by y = x D, y + = x + D, () where D = x + ( ) log( ). This is so as q(y) ) y ( ) log( ) 0 y y y + by taking logarithms. Theorem 3. thus applies in the x + xy q( y x

13 normal ase with the above values of y an y +. The risk R (µ) of the plug-in preitive ensity estimator, given by (0) atually reues to arsin( 5) To justify π 5 this, write x/ x/ F (x/) p(x) x = p(y) p(x) y x = p(y) p(x) y x = P(Z 0, Z 0), with (Z, Z ) istribute as bivariate normal with means 0, varianes an orrelation ρ = 5 5. The result for R (µ) follows with the quarant probability ientity P(Z 0, Z 0) = 4 + arsin(ρ) π for bivariate normal vetors (e.g., Muirhea, 98, page 44). We now proee with the main result of this setion. Theorem 3.. Suppose q is an even, ifferentiable a.e., an logonave ensity on R. Suppose p is an even ensity. Then, for estimating the ensity q(y µ), y R base on X p(x µ), x R, uner L loss, the preitive ensity estimator q(y x) is inamissible an ominate by a sublass of preitive ensity estimators > an small enough. R q( y x ) for Remark 3.. Observe that the above result is quite general as long as q is logonave. It applies namely for the normal ase with Y N(µ, σy ). Little, not even unimoality, is require of p exept the evenness. Proof of Theorem 3.. We show that R (µ) =+ < 0, whih will suffie. We have from Theorem 3. by ifferentiating uner the integral sign ( A = F ( y ) x ) F (y ) p(x) x ( = q( y ) x ) (y (y x)) q(y )y p(x) x = q(y ) (x y ) p(x) x, where y = y, an where we have mae use of the property q(y ) = q( y x ) whih y satisfies by efinition. Similarly, we obtain A = q(y +) (y + x) p(x) x. Taking +, we have lim + y =, lim + y + = x (Remark 3.), an hene lim + y q(y ) = lim t t q(t) = 0. We thus have R 4 (µ) =+ = lim (q(y ) (x y ) + q(y + ) (y + x)) p(x) x + R + = x q( x ) p(x) x < 0, ompleting the proof. 3

14 Example 3.. (normal ase ontinue) As an illustration, for σy = σ X =, we obtain numerially that R (µ) ereases for < < 0 an inreases for 0, with 0.8 representing the optimal hoie of among the estimators ˆq (y; x) = y x q( ). We also obtain that ˆq ominates ˆq iff < < with.936. However, the gains are small. For instane R 0 (µ) while R (µ) (see Example 3.), representing a improvement of aroun.7%. 4 Conluing Remarks We have shown that, for estimating a -imensional unimoal spherially symmetri ensity q Y ( y µ ) base on X p X ( x µ ), an uner L loss, the benhmark plug-in q Y ( y X ) preitive ensity estimator is quite generally inamissible for 4 in terms of frequentist risk, an ominate by a lass of plug-in preitive ensity estimators q Y ( y ˆµ(X) ), with the ˆµ(X) being James-Stein an more generally Baranhik-type estimators of µ. We have apitalize on a L istane formula (Lemma.) to establish the link between the preitive ensity estimation problem an a point estimation of µ problem base on X uner a ual loss of the form ρ( µ ) with onave ρ. Our inamissibility an ominane results are obtaine by making use of tehniques in Stein estimation for suh onave losses, an by working with the speifi form of ρ. The finings also represent multivariate mean point estimation ontributions on their own, an further reent work in this regar appears in Kubokawa, Marhan an Strawerman (05B). We have also shown in the univariate normal ase, an more generally for log-onave ensity q, that a sale expansion, inue by the preitive ensity estimator q (y; x) = y x q( ) with >, ominates the plug-in ˆq for s slightly larger than. Although L istane arises in many varie theoretial an pratial situations, its analytial treatment appears to be quite iffiult. The tehniques an results presente here aress suh a iffiulty an, we believe, pave the way for further finings. Aknowlegements The authors are grateful to an Assoiate Eitor an referee for helpful suggestions. Tatsuya Kubokawa s researh is supporte in part by Grant-in-Ai for Sientifi Researh Nos. 5H0943 an from the Japan Soiety for the Promotion of Siene, Eri Marhan s researh is supporte in part by the Natural Sienes an Engineering Researh Counil of Canaa, an William Strawerman s researh is partially supporte by a grant from the Simons Founation (#09035). Finally, thanks to the Center for International Researh on the Japanese Eonomy whih provie finanial support for 0 an 05 visits by Marhan to the University of Tokyo. 4

15 Referenes [] Aithison, J. (975). Gooness of preition fit. Biometrika, 6, [] Anrews, D.R. an Mallows, C.L. (974). Sale mixtures of normal istributions. Journal of the Royal Statistial Soiety B, 36, [3] Baranhik, A.J. (970). A family of minimax estimators of the mean of a multivariate normal istribution. Annals of Mathematial Statistis, 4, [4] Berger, J. (975). Minimax estimation of loation vetors for a wie lass of ensities. Annals of Statistis, 3, [5] Branwein, A.C., Ralesu, S. an Strawerman, W.E. (993). Shrinkage estimators of the loation parameter for ertain spherially symmetri istributions. Annals of the Institute of Statistial Mathematis, 45, [6] Branwein, A.C an Strawerman, W.E. (99). Generalization of James-Stein estimators uner spherial symmetry. Annals of Statistis, 9, [7] Branwein, A.C an Strawerman, W.E. (980). Minimax estimation of loation parameters for spherially symmetri istributions with onave loss. Annals of Statistis, 8, [8] Brown, L.D., George, E.I., an Xu, X. (008). Amissible preitive ensity estimation. Annals of Statistis, 36, [9] DasGupta, A. an Lahiri, S.N. (0). Density estimation in high an ultra imensions, regularization, an the L asymptotis. Contemporary Developments in Bayesian analysis an Statistial Deision Theory: A Festshrift for William E. Strawerman, IMS Colletions, 8, -3. [0] Devroye, L. an Gyrfi, L. (985). Nonparametri ensity estimation. The L view, Wiley, New York. [] Feller, W. (966). An Introution to Probability Theory an its Appliation,, seon eition, Wiley, New York. [] Fourrinier, D., Marhan, É., Righi, A. an Strawerman, W.E. (0). On improve preitive ensity estimation with parametri onstraints. Eletroni Journal of Statistis, 5, 7-9. [3] George, E. I., Liang, F. an Xu, X. (006). Improve minimax preitive ensities uner Kullbak-Leibler loss. Annals of Statistis, 34, [4] Kano, Y. (994). Consisteny property of elliptial probability ensity funtions. Journal of Multivariate Analysis, 5, [5] Komaki, F. (00). A shrinkage preitive istribution for multivariate normal observables. Biometrika, 88, [6] Kubokawa, T., Marhan, É., an Strawerman, W.E. (05A). On preitive ensity estimation for loation families uner integrate square error loss. Journal of Multivariate Analysis, 4, [7] Kubokawa, T., Marhan, É., an Strawerman, W.E. (05B). On improve shrinkage estimators for onave loss. Statistis & Probability Letters, 96, [8] Kubokawa, T. an Saleh, A.K.MD.E. (998). Estimation of loation an sale parameters uner orer restritions. Journal of Statistial Researh, 8,

16 [9] Marhan, É., an Strawerman, W. E. (005). On improving on the minimum risk equivariant estimator of a loation parameter whih is onstraine to an interval or a half-interval. Annals of the Institute of Statistial Mathematis, 57, [0] Muirhea, R.J. (005). Aspets of multivariate statistial theory. Wiley, seon eition. [] Naarajah, S. (003). The Kotz-type istribution with appliations. Statistis, 37, [] Strawerman, W.E. (974). Minimax estimation of loation parameters for ertain spherially symmetri istributions. Journal of Multivariate Analysis, 4, [3] Weizman, M.S. (970). Measures of overlap of inome istributions of white an negro families in the Unite States. Tehnial Report, US Department of Commere. 6