AN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS

Statistica Sinica 6 016, 1709-178 doi:http://dx.doi.org/10.5705/ss.0014.0034 AN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS Nilabja Guha 1, Anindya Roy, Yaakov Malinovsky and Gauri Datta 3 1 Texas A & M University, University of Maryland, Baltiore County and 3 University of Georgia Abstract: The proble of predicting a vector of ordered paraeters or its part arises in contexts such as easureent error odels, signal processing, data disclosure, and sall area estiation. Often estiators of functions of the ordered rando effects are obtained under strong distributional assuptions, e.g., norality. We discuss a siple generalized shrinkage estiator for predicting ordered rando effects. The proposed approach is distribution free and has significant advantage when there is odel isspecification. We give expression to and characterization of the optial shrinkage paraeter; the expression involves the Wasserstein distance between two odel-related distributions. We provide a fraework for estiating the distance and thereby estiating an epirical version of the oracle optial estiator. We copare the risk for the optial predictor to that of other distribution-free estiators. Extensive siulation results are provided to support the theoretical results. Key words and phrases: Epirical Bayes predictor, linear predictor, order statistics, shrinkage. 1. Introduction A coon odel of interest is y i = θ i + e i, i = 1,,...,, 1.1 where the σ 1 i e i are assued independent and identically distributed as H0, 1, a ean zero unit variance distribution, with the constants σ i assued to be known. Independent of the e i, the θ i are given as θ i = µ i + u i, where the u i are independently and identically distributed i.i.d. as ean zero, finite variance rando variables with distribution G. This odel finds wide-spread applications ranging fro easureent error odels, signal processing, data disclosure, sall area estiation to nae a few. Here, we develop ethodology for predicting the ordered rando effects, θ i, in the context of Model 1.1. Model 1.1 is a special case of the Fay-Herriot odel Fay and Herriot 1979 in sall area estiation SAE, where the area eans θ i are further odeled using area specific covariate inforation x i = x i1,..., x ip, and area

1710 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA specific rando effects u i as θ i = x i β + u i. Following SAE terinology we refer to θ = θ 1,..., θ as the vector of area eans. While predicting ηθ = θ is coon, investigators have also studied prediction of other functions; vector of ranks, the epirical distribution of the area eans Shen and Louis 1998 and the range of area eans Judkins and Liu 000. Here we are interested in predicting the vector of order statistics ηθ = θ = θ 1 θ θ. Prediction of the ordered eans is significantly harder than prediction of the linear function of the area eans Pfefferann 013. When G and H are correctly specified, the posterior ean ˆη = Eηθ y iniizes the prediction ean squared error PMSE, Rˆη = E[ˆη ηθ ˆη ηθ] where y = y 1,..., y i.i.d. is the data. When e i N0, σ i.i.d. and θ i Nµ, σθ the Bayes estiator of θ is ˆθ B = y 1 γy µ1, where γ = σ θ σ θ + σ and 1 is a vector of ones. The epirical Bayes EB estiator is obtained by replacing µ by ȳ, ˆθ EB = y 1 γy ȳ1, which is also the Best Linear Unbiased Predictor BLUP in the class of all {H, G} with finite second oents. Brown 1971 and Brown and Greenshtein 009 have looked at Bayes/epirical Bayes estiation under general prior. The plugged-in version η ˆθ B, however is not the Bayes predictor and can result in substantial bias in prediction. Wright, Stern, and Cressie 003 considered a Bayesian schee for predicting ordered eans but their procedure is sensitive to prior choice and requires substantial coputation. When G and H are partially specified up to lower order oents Stein s shrinkage estiators Stein 1956 can be used for a variety of paraetric functions but for the ordered paraeters no suitable predictors are available. When error variances are assued equal, Malinovsky and Rinott 010 proposed a class of shrinkage estiators θ i λ = λy i + 1 λµ. 1. They showed that the risk iniizing value of λ lies in the interval [γ, γ] and, based on siulation evidence, conjectured the asyptotic optial value to be γ. The weight γ also appears in Louis 1984, who proposed Bayes and epirical Bayes predictors that iniize an expected distance function between the epirical cdf of predictors of θ and epirical cdf of its true value. We use siilar shrinkage estiators under odel 1.1 and derive expressions for the optial shrinkage paraeter. The optiu estiator is shown to have good finite saple perforance with respect to ean squared prediction error, even in coparison to the best estiator when G and H are known to be noral. Thus, we provide an estiator that can predict the order paraeters with

AN OPTIMAL SHRINKAGE FACTOR 1711 reasonable accuracy and does not ake strong distributional assuptions. For the equal error variance case we show that the optial choice of λ in 1. is not necessarily γ, and characterize the cases when γ is indeed the asyptotically optial choice for λ. Based on the derived expression for the optial value of λ, we propose a new class of predictors for the ordered paraeters and provide a fraework for estiation of the optial predictor. We illustrate its finite saple perforance via siulation.. Prediction of Ordered Rando Effects It is instructive to begin with a special case of odel 1.1 in which the design variances are all assued to be equal. Under the assued odel, constant error variance would iply that the errors are i.i.d.. Since we later consider the case when the error variances are not equal, we do not separately consider the case where the errors have equal variance but are not necessarily identically distributed. We assue that θ i arise following soe distribution G, with ean µ and variance σθ, but we do not specify the fors of G and H..1. Prediction in the equal variance odel Assue odel 1.1 with constant variances, σ1 = = σ = σ. The arginal distribution of y i is denoted by F which under the assued odel has ean µ and variance σy = vary i = σθ + σ. For prediction of the ordered paraeters, we consider the class of shrinkage predictors 1.. Under squared error loss the PMSE for a saple of size is R λ = 1 E θ i λy i 1 λµ and the optial risk iniizing value of λ is λ = 1 E y i µθ i µ 1 E y i µ..1 We study the liiting for of.1 as and evaluate the relative efficiency of the optial shrinkage coefficient with respect to other predictors. Let { 1 WF, G = [F 1 t G 1 t] }1/ dt 0 denote the L Wasserstein etric between the distributions F and G, assued to have finite variance. We consider the predictors θ i γ = γy i + 1 γµ and θ i γ = γy i + 1 γµ.

171 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Following the for of the BLUP for the unordered paraeters, a natural choice for the predictor of the ordered quantities would be θ = θ 1 γ θ γ, while the predictor θ 1 = θ 1 γ θ γ is the for conjectured in Malinovsky and Rinott 010 to have asyptotically optiu perforance. The PMSE associated with the predictors θ 1 and θ are R 1 = R γ and R = R γ, respectively. For any estiator of the for θ i λ define the relative efficiency with respect to θ i γ as RE 1 λ = R 1 /R λ, and that of θ i λ with respect to θ i γ as RE λ = R /R λ. Let the distribution of the standardized observations, y i µ/σ y, be F and that of the standardized paraeters, θ i µ/σ θ, be G where σy = σθ + σ. We require soe conditions on F and G. A1: The distributions F and G have finite fourth oents. A: For all 0 < t < 1/, F x and G x have continuous, positive derivatives on x F 1 t, F 1 1 t and x G 1 t, G 1 1 t, respectively. Theore 1. Under A1 A, as, λ λ = γ 1 W F, G R λ R = σθ W F, G W 4 F, G 4 [ ] 1, RE 1 λ RE 1 = RE 1 W F, G 4 λ RE = 1 + [1 W F, G / γ] [1 1 W F, G /. ] The gain in PMSE at the optial shrinkage value over that at γ, is [1 W F, G /4] 1. This iproveent can be quite significant if the Wasserstein distance between F and G is large and, as 0 W F, G, potentially there can be a two fold reduction in the PMSE of the optial predictor over that of θ i γ. We find in the siulations that the gain in efficiency fro the optial predictor can be substantial. Reark 1. If F and G are equal, then W F, G = 0 and λ = γ, and the PMSE of the optial predictor goes to zero as goes to infinity. Here, the distribution of the centered y i is a scaled version of that of the centered θ i, and a siple scaling of the observed values provides the optial prediction. In the context of equal error variance Malinovsky and Rinott 010 conjectured the optiu value of λ in 1. to be γ. Theore 1 iplies that the,,

AN OPTIMAL SHRINKAGE FACTOR 1713 result holds iff W F, G = 0. As F and G are distributions of the standardized quantities, to derive necessary and sufficient condition for W F, G = 0, without loss of generality, we take µ = 0. Theore. Suppose 1.1 holds and the errors e i are independently and identically distributed as H with ean zero and variance σ. Then the Wasserstein distance etric W F, G between the distributions of standardized θ and standardized y is zero if and only if θ i has the sae distribution as that of k=1 ck e k where c = γ = σ θ /σ y. Proof. If part : If θ = k=1 ck e k, then y = k=0 ck e k, where e k s are i.i.d for all k. Hence, cy has sae distribution as θ and after standardization u and y have the sae distribution. Hence, WF, G = 0. Only if part : We can write cy i = cθ i + ce i. Fro W F, G = 0, follows that yi = cy i has sae distribution as θ i. Iterating the procedure we see that θ has the sae distribution as k=1 ck e k. Because c < 1, the series representation is valid in ean squared sense. Reark. Noral distributions on θ and e give W F, G = 0, and hence Gaussianity is a sufficient condition for Theore to hold. However, as shown in Theore, the class of distribution pairs H, G that will give W F, G = 0 is a uch wider class containing the noral distribution. In such cases, F is a self-decoposable distribution Lukacs 1970 and exaples of such distribution could be found in Shanbhag and Sreehari 1977... Optial prediction with unequal design variances With a slight odification, a shrinkage predictor siilar to 1. can be proposed in the unequal variance case as well. In order to derive the liiting for of the optial estiator we have to ake assuptions about the convergence of the epirical distribution of the standardized responses. Such assuptions autoatically hold in the i.i.d. case considered in Section.1. Let v i = vary i = σ θ + σ i and z i = y i µ/v i. Then we propose a class of shrinkage predictors for the ordered area eans as θ i λ = λz i + µ.. The PMSE at λ is R λ = 1 E θ i λz i µ.

1714 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA If σi s are the sae, the class of predictors in. reduces to the class 1.. Let γ i = σθ /σ θ + σ i. Then analogous to the equal variance case, one could look at the predictors θ 1 = θ 1 1 θ1 where θ 1 i = σ θ z i + µ and θ = θ 1 θ where θ i = γ i y i + 1 γ i µ. Unlike the equal variance case, the predictor θ i does not belong to the class. but rather it is the ordered version of the area specific BLUP for the unordered area eans. Let R 1 and R denote the PMSE of θ 1 and θ, respectively, with RE1 λ = R 1 /R λ and RE λ = R /R λ. Let w i = θ i µ/σ θ denote the standardized area eans. Then the shrinkage coefficient with iniu PMSE is given by w iz i λ 1 E = σ θ 1 E z i..3 We establish a sipler liiting for for the optial shrinkage coefficient, leading to suitable predictor that can be used once the unknown paraeters have been substituted with data estiates. Let F and K denote the epirical distributions of z i and γ i z i, respectively. A3 The sequence of distributions F and K are uniforly integrable and converge in distribution to ean zero distributions F and K with finite fourth oents, respectively. The distribution of w i, G has finite fourth oent. Theore 3. Under A3, if A holds for F, G and K, as [ λ λ = σ θ 1 W F, G ], [ R λ R = σθ 1 1 W F, G ], [ RE 1 λ RE 1 = 1 W F, G ] 1, 4 RE λ RE = W K, G [1 1 W F, G / ]. Based on the optial value of the shrinkage coefficient, the proposed predictor for the ordered θ i is θ = θ 1 θ where θ i = σ θ [ 1 W F, G ] z i + µ..4

AN OPTIMAL SHRINKAGE FACTOR 1715 Reark 3. Our results hold for the ore general loss function in which different ordered effects have different weights for their corresponding risks. More details of this can be found in the suppleentary docuent..3. An application to sall area estiation We consider SAE where a fixed area level effect is present, and the ean value of the ith area, θ i = Ey i θ i, potentially depends on the characteristics of the area. Let θ i = µ i + u i and hence y i = µ i + u i + e i. The µ i are fixed effects and the u i are rando effects. Typically area specific fixed effects are odeled as µ i = x i β. We take the u i as i.i.d. N0, σθ and the e i as i.i.d. N0, σi. Write the standardized response as z i = y i x i β/v i where vi = σ θ + σ i is the variance of y i. Following the generalized shrinkage estiation developent, we can predict u i = σ θ z i. For predicting θ i s we propose θ i = σ θ z i + x iβ,.5 and let θ = θ 1 θ be the ordered values of θ i. Reark 4. Use of θ in the equal variance case is justified because axiu a posteriori order for the latent rando effects is the sae as the order of the observed quantities under ild distributional assuptions. More details are provided in the suppleentary docuent. We do not address the rank estiation issue directly. A short discussion on rank estiation is included in the suppleentary aterials, in the context of odel 1.1. 3. Epirical Version of the Predictors In practice, the unknown paraeters in the expression for the optial shrinkage predictor are replaced with their estiators. Thus, at.4, one plugs in the estiated values of µ, W F, G and σ θ. 3.1. Epirical predictor Unless otherwise entioned we use the saple ean ȳ to estiate µ. Other estiators, such as the saple edian can be considered. Estiation of σ θ is straightforward, but estiation of W is ore involved. A consistent ethod-ofoent estiator of σ θ is ˆσ θ = ax { 1 y i ȳ 1 } σi, 0. Based on the estiated σ θ, we can replace v i by ˆv i = ˆσ θ + σ i. We also use ẑ i = y i ȳ/ˆv i as the observed standardized response in order to copute the Wasserstein distance.

1716 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA If the faily of distributions G0, σθ is known up to σ θ, then W F, G can be estiated epirically once F is estiated based on ˆσ θ. In cases where G is unknown we can proceed as follows. We assue that the error distribution is a known finite location-scale ixture of noral distributions a good approxiation to H0, σi and that each ixture coponent is independent of the unobserved θ. We also use a finite noral location scale ixture representation for the distribution of θ. Thus, we take L e i p e,l,i Nµ e,l,i, σe,l,i, 3.1 l=1 where p e,l,i, µ e,l,i, and σe,l,i are all known and K θ i G = p θ,k Nµ θ,k, σθ,k. 3. Then y i F i = k=1 K k=1 l=1 L p k,l,i Nµ k,l,i, σk,l,i, where p k,l,i = p θ,k p e,l,i, µ k,l,i = µ θ,k + µ e,l,i, and σk,l,i = σ θ,k + σ e,l,i. One can then use the EM algorith to estiate the distributions and hence estiate the Wasserstein distance based on the estiated distributions, say Ŵ F, G. For coputation and ipleentation, it is ore efficient to use the finite saple version of the Wasserstein etric associated with the finite saple version of the optial shrinkage and estiate that to plug-in into the predictor. Set, WF, G = 1/ F 1 i/ + 1 G 1 i/ + 1 and W F, G = E1/ F 1 i/ + 1 G 1 i/ + 1. Given the noral location scale ixture representation of G we can generate independent observations fro the distribution of θ and generate a copy of observed y s using the known error distribution. Then, W F, G is estiated by its Monte-Carlo estiator. Let, F,j and G,j be the epirical distribution for standardized θ and y in j th replication. We estiate W F, G = 1 R 1 F 1,j R i + 1 G 1,j i + 1, j=1 where R is the nuber of replications. Let ˆλ be the value of the optial shrinkage coefficient when σ θ and W F, G have been replaced by their estiators ˆσ θ and W F, G. Then the estiated optial predictor.4 for the ordered area eans is ˆθ = ˆθ 1 ˆθ where

AN OPTIMAL SHRINKAGE FACTOR 1717 ˆθ i = ˆσ θ [ 1 W F, G ] ẑ i + ȳ. 3.. Accuracy of the epirical predictor The epirical estiator is a plug-in version of the optial predictor. To judge its accuracy we derive asyptotic expression for the PMSE of the epirical predictor. We need soe assuptions to establish the asyptotic rates. A4: For all 0 < t < 1/, F x and G x have continuous and positive derivative on x F 1 t, F 1 1 t and x G 1 t, G 1 1 t, respectively. A5: Let SF = {x : 0 < F x < 1} and SG = {x : 0 < G x < 1} be the open supports of F and G. F and G are twice differentiable on their open supports and their corresponding densities, f and g, are strictly positive on their respective open supports. A6: A7: 1 0 t1 t f F 1 dt < and t 1 0 t1 t g G 1 dt <. t t1 t f F 1 t t1 t g G 1 t sup 0<t<1 f F 1 t < and sup 0<t<1 g G 1 t <. A8: The densities fx and gy are onotone for x / F 1 t, F 1 1 t and y / G 1 t, G 1 1 t for soe 0 < t < 1/. A9: There exists c > 0, such that inf i σ i > c, and ˆσ θ, 1ˆσ θ, >0 < K for all > 0 for soe 0 and K, where ˆσ θ, is the estiate of σ θ based on observations. A10: Assue that consistent estiators of W, and W, Ŵ, and W, are available. Assuptions A4 A8 can be found in Barrio, Gin, and Utzet 005 in the context of convergence of integrated quantile differences. Here A9 is needed for the case when σ θ is being estiated. Assuption A10 is plausible since, if the assued location scale representation is correct, as in that case the MLE of the paraeters in the ixture odel is - consistent for the true value and W, is a continuous function of the paraeters. Proposition 1. Under A1 A, A10, for the equal error variance case at.1, σ θ /σ y 1 W F, G / = λ + O P 1/. If A4 A8 and A10 hold, then σ θ /σ y 1 Ŵ F, G / = λ + O P 1/.

1718 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Proposition. Under A A3, A10, for the unequal error variance case at.3, σ θ 1 W F, G / = λ + O p 1/. Under A A8 and A10, σ θ 1 Ŵ F, G / = λ + O p 1/. The following results generalize Proposition 1 and when the optial shrinkage predictor is based on plugged-in estiators for W, σ θ and µ. Theore 4. Under A1 A10, for the equal variance case at.1, ˆσ θ 1 ˆσ y ˆσ θ ˆσ y If W F, G > 0, [1 W F, G = λ + O p 1/, 1 Ŵ F, G = λ + O p 1/. W W F, G ] 1 = RE 1 λ 4 + O p 1/, K, G [1 1 W F, G / ] = RE λ + O p 1/. Theore 5. Under A A10, for the unequal design variance case at.3, If W F, G > 0, [ 1 W ˆσ θ 1 F, G = λ + O P 1/, ˆσ θ 1 Ŵ F, G = λ + O P 1/. W W F, G 4 K, G [1 1 W F, G / ] ] 1 = RE 1 λ + O p 1/, = RE λ + O p 1/. Theores 4 and 5 provide approxiations to the relative efficiency of the estiated version of the optial shrinkage predictor. In ters of PMSE the proposed optiu shrinkage estiator perfors better than the BLUP type estiator in the equal error variance case. But for the unequal variances this ay not happen as the BLUP type estiator does not belong to the class of estiators represented by.. In our odel-based approach to estiating

AN OPTIMAL SHRINKAGE FACTOR 1719 W F, G, once the noral location-scale ixture odel for θ is estiated we can estiate W K, G as well. Thus, the relative efficiency of the BLUP type estiator copared to the proposed optiu estiator can be estiated and the estiator with lower value of estiated asyptotic PMSE can be used. 4. Siulation Study 4.1. Optiu shrinkage and Wasserstein correction We carried out three exaples. The saple sizes considered were =,000 and = 10,000. The larger saple size was chosen to evaluate the accuracy of the estiation of the Wasserstein distance, and copare with the theoretical asyptotic value. Each reported Monte Carlo value was based on 500 replications. Exaple 1. The first experiental scenario was designed to evaluate the effect of the Wasserstein distance on the perforance of the different predictors. The area ean distributions were chosen to be two coponent noral scale ixtures paraeterized by a single paraeter: θ i a 1 N0, a 1 + 1 a 1 N0, a 1 1. Here Eθ = 0 and V arθ = 1, and W F, G is an increasing function of a [,, with W = 0 for a =. The error distribution was fixed as standard noral. For the siulation we took a {, 5, 10, 100, 1,000}. The scale ixture odels for different values of a are denoted by Nixa. Exaple. We look at two distributions for the area eans: a Double Exponential distribution to reflect possible heavy tails in the distribution and a two coponent location ixture of norals to account for possible ultiodality in the area ean distribution. In particular we took θ i 0.5N4, 1 + 0.5N 4, 1 and θ i DE, where, in each case, the errors were generated independently using e i N0, σi. For the noral ixture case, we considered two cases, constant error variances, σi = 16, and σi Unifor0, 16. For the double exponential, we took σi = 1 and σi Unifor0, 1. The double exponential odels and noral location ixture odels with equal and unequal variances are denoted by DE E, DE U, Nix E and Nix U, respectively. The optiu shrinkage coefficient was used in each exaple. We considered the shrinkage predictors θ, θ1 version of the predictors where σθ, and θ. For prediction we used the estiated and W are obtained following the procedure described in Section 3 and plugged into the expression of the predictors. The value K = 6 was used.

170 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 1. Relative perforance of the different shrinkage predictors. θ vs θ1 θ vs θ Model W Ŵ RE,000 RE 10,000 RE RE,000 RE 10,000 RE Nixa a = 0 0.0 1.00 1.00 1.00 40.4 199 a = 5 0.5 0.4 1.01 1.01 1.01.19.1.15 a = 10 0.41 0.40 1.04 1.05 1.05 1.5 1.4 1.6 a = 0 0.53 0.47 1.07 1.08 1.08 1.09 1.08 1.07 a = 50 0.6 0.5 1.09 1.09 1.10 1.0 1.0 1.0 a = 100 0.67 0.61 1.10 1.1 1.13 1.00 1.01 1.01 DE E 0.14 0.13 1.00 1.00 1.00 4.84 5.13 5.13 DE U 0.11 0.10 1.00 1.00 1.00 3.67 3.94 3.94 Nix E 0.37 0.3 1.03 1.03 1.04 1.33 1.35 1.35 Nix U 0.9 0.8 1.0 1.0 1.03 1.18 1.18 1.18 Table 1 gives the ratio of PMSE of the optial predictors copared with the other predictors at the two saple sizes. Colun gives the value of the true Wasserstein distance and colun 3 gives the estiate of W averaged over the Monte Carlo replications. Coluns 4-6 give the relative efficiency of the optial shrinkage estiator θ copared to the estiator θ1 at saple sizes =,000, 10,000, and =, respectively. The value at = is the theoretical value given in the Theore 1. Coluns 7 9 give the relative efficiency value of the optial estiator copared to θ at =,000, 10,000, and =, respectively. In the noral scale ixture odels, for saller values of the Wasserstein distance, the optial shrinkage predictor θ and the one ignoring the Wasserstein correction, θ 1, are nearly identical. This is expected since for values of W close to zero, the correction factor is close to one and the two predictors essentially coincide. However, the BLUP-type estiator θ is uch inferior to the other estiators in the sall W scenario. When W is large, the optial estiator is considerably better than θ 1, because ignoring the Wasserstein correction has a significant effect on the predictor. Here BLUP-type estiator θ is nearly identical to the optial predictor. For oderate W, the optial shrinkage provide substantial gains over both θ 1 and θ. In the noral ean-ixture exaple for unequal or equal variances θ and θ 1 perfor better than the θ and, with Wasserstein correction, θ perfors better than θ 1 in the equal variance case with relatively higher value of W. For double exponential scenario θ and θ1 perfor uch better than the θ and, because of the sall value of W, the Wasserstein correction is unnecessary for all practical purposes.

AN OPTIMAL SHRINKAGE FACTOR 171 Table. Relative perforance of the different shrinkage predictors. θ vs θ1 θ vs θ α W Ŵ RE 500 RE ˆ RE RE 500 RE ˆ RE α = 0 0.30 0.7 1.0 1.0 1.0.40.70.64 α = 0.5 0.33 0.31 1.0 1.0 1.03.8 3.31 3.03 α = 1 0.34 0.33 1.0 1.03 1.03.97 3.7 3.38 Figure 1. Plot for the nonparaetric fit for density of G. The left hand panel shows a typical fit for shifted Gaa distribution for skewed G, the right hand panel is a typical fit in exaple 1 with a = 10. The solid line is the fitted and the dashed line is the true target density. Exaple 3. We investigated the effect of skewness in the area ean distribution. We took θ i Gaa1.5, 1.5 1, an asyetric distribution for the rando effect part around ean zero and with support 1,. Error ters were noral and we took σi = bα + 1 αc i, with c i = 1 i/, b = 3 and 0 α 1. We considered the cases α = 0, 0.5 and 1. The relative efficiency of θ with respect to θ1 and θ is given in the Table for = 500 with the data estiate of the relative efficiencies given by RE, ˆ where the nuber of replication was 100. In these exaples the proposed ixture odel approach was able to estiate G. Figure 1 shows the estiated density in two cases; in all cases the proposed estiator provided a reasonable approxiation to G. 4.. Sall area estiation For the SAE odel θ i and e i are assued to be noral. By Theore, the Wasserstein distance between the standardized distributions of the area eans and the responses is zero. Thus, by Theore 3, the optial shrinkage estiator

17 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 3. Relative efficiency of the proposed predictor to BLUP-type predictor. c = 100 = 300 = 500 1 1.0 1.08 1.09 3 1.08 1.6 1.44 5 1.18 1.6 1.87 θ is identical to θ1. We considered two cases. Case 1: In this case = 100, 300, and 500 sall areas were considered. We took the case with a single covariate and, for the i th area we had y i = α + βx i + u i + e i. We took u i N0, 16 with α = 1, β =. We generated x i N0, 1 and e i N0, σi. The error variance values were generated as σi U0, c. We chose c fro c = 1, 3, 5. The siulation results are reported for 500 Monte Carlo replications. The proposed estiator θ is copared with the BLUP-type predictor θ. The relative efficiencies are reported in Table 3. The proposed predictor outperfors the BLUP-type predictor. The difference is significantly higher when the γ i are further fro one, that is when c is 3 or 5. For large, the percentage of iproveent is generally greater with the optial predictor providing 10 50% gain in efficiency of prediction. Case : The gain in the perforance of the optial predictor is due to better prediction of the order statistics of the rando effects u i. When the observed values are highly influenced by the fixed effects, then the BLUP-type predictor is expected to perfor coparably with the optial predictor since the ain reduction in risk is achieved by accurate prediction of the fixed effect part. To evaluate the effect of the correlation of the responses with the fixed effect on the perforance of the optial predictor we considered a ore general case with different values of β. For higher β the fixed effect α + βx i doinates and the response y i is higher and this effects the perforance of the shrinkage estiator. We use the sae odel as before Case 1, with α = 1, β = i, where i = 1,,..., 15. Also, we varied the nuber of sall areas as = 100j with j = 1,, 3, 4, 5. The area specific variances were generated as U0, c, with c = 1, 3, 5, 8. The relative efficiencies of the BLUP-type predictor θ copared to the optial predictor θ 1 are given in Figure. The lower the relative efficiency, the better the perforance of the proposed predictor in predicting the order statistics. The proposed predictor has significantly saller PMSE for a variety of cases where the correlation between the fixed effect and the response is oderate to sall. For larger values of β the responses are essentially the fixed effects, and

AN OPTIMAL SHRINKAGE FACTOR 173 Figure. Fro left to right the figures correspond to c = 1, 3, 5, 8, respectively. The X-axis denotes i = 1,..., 30. Here = 100j and j = 1,, 3, 4, 5 are denoted by,,, and, respectively. the relative efficiency approaches one. Since the optial predictor is being replaced by its plug-in version, the relative efficiency of the BLUP-type estiator is actually slightly larger than one, when the area eans are essentially of the sae agnitude as that of the fixed part α + βx i. 4.3. Coparison with the full Bayesian estiator If the distributions G and H are known, Bayesian coputation can be used to generate the posterior saples of θ i and estiate the posterior expectation of the ordered eans by taking the ean of the ordered posterior saples. The posterior ean is the best estiator in ters of PMSE. It is interesting to copare the Bayesian ethod to our distribution free approach. We also study the sensitivity of the Bayesian estiator to odel isspecification, especially in the odel for the rando effects paraeter θ. We considered Students t distribution with various degrees of freedo for G and assued it to be isspecified as noral distribution. Also, a ixture noral distribution was considered for θ with G.5N1.5, 1 +.5N 1.5, 1. We took, equal variances E σi = 1 and unequal variances U σi U0, 1. In the Bayesian ethodology we assued the prior on θ, G, to be Nµ, σ and assued the following non-inforative priors Πµ, σ 1. The ratio of the square root of the PMSE s for the shrinkage and the Bayesian ethods are reported in Table 4. If the odel is truly specified then the ratio should be greater than one. Under odel isspecification the shrinkage generally perfors better than the Bayesian ethod, and for the correctly specified case the Bayesian is better, as expected. However, even in the correctly specified odel, the odel free estiator continues to have reasonable perforance.

174 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 4. Relative efficiency of the optial shrinkage and Bayes predictor. G =100 =400 =900 U E U E U E T. 0.87 0.81 0.71 0.68 0.66 0.63 T 3 0.9 0.87 0.77 0.73 0.69 0.68 T 4 1.00 0.93 0.8 0.79 0.75 0.7 T 6 1.0 1.04 0.9 0.9 0.8 0.81 N0, 1 1.19 1. 1.3 1. 1.3 1.1 Mixture 1.04 0.98 0.86 0.85 0.77 0.78 5. Conclusion We propose an optial estiator of ordered rando effects in the class of siple shrinkage estiators. The ain attraction of the proposed estiator is that it is distribution free. It is very robust to odel isspecification and, as evident fro siulations the distribution free estiator is reasonably efficient with respect to the best Bayesian estiator in a correctly specified noral odel. The estiator provides an easy and direct way for predicting ordered rando effects under both equal and unequal error variances. Fro siulations, in any setups we see significant relative gain in efficiency for the optial shrinkage estiator over other shrinkage estiators in the sae class. We also derived liiting for of the risk of the estiator and proposed a ethod for estiating the risk. The estiator depends on the Wasserstein distance between two standardized distributions and a ethod based on the observed y i s is also given for estiating it. The optial estiator based on the estiated Wasserstein distance is shown to have reasonable asyptotic properties. We also address the situation when area specific covariate inforation is available. Alternative classes of estiators could involve linear shrinkage estiators with different shrinkage for the different order statistics, or classes of shrinkage estiators that directly account for the joint dependence aong the order statistics. Inference in such classes ay be ore difficult, and this is an interesting area for further exploration. Generalization to classes of estiators that can effectively account for area specific covariates while having the advantage of being distribution free is a topic of future research. The proposed estiator gives a good starting point and a preliinary fraework for ore general classes of distribution-free estiators. Suppleentary Materials The suppleentary docuent has four sections. Details about Reark 3 are given in Section 1. The justification and proof of results related to the sall area odel in Reark 4 are given in Section. In the third section we prove soe

AN OPTIMAL SHRINKAGE FACTOR 175 results fro the anuscript and in the fourth, we discuss the rank estiation issue. Acknowledgeent We thank an associate editor and two reviewers for their thoughtful coents and suggestions. The authors would like to thank Dr. A. M. Kagan for pointing out the connection to self-decoposable distributions. Appendix Proof of Theore 1. Fro.1 Hence, where λ = 1 E y i µθ i µ 1 E y i µ. λ = 1 σ θ σ y E [y i µ/σ y ][θ i µ/σ θ ] 1 E y i µ/σ y = σ θ ES σ, y S = 1 { 1 z i + 1 w i 1 z i w i }. Because E1/ z i = 1 and E1/ w i = 1 we have S = 1 { 1 z i + 1 w i T 1 }, where Then, T 1 = 1 F 1 i + 1 G 1 i. + 1 where T 1 = d F, F + d G, G + d F, G + C 1 + C + C 3, d F, F = 1 F 1 i + 1 F 1 i + 1, d G, G = 1 G 1 i + 1 i G 1 + 1, d F, G = 1 F 1 i + 1 i G 1 + 1, A.1

176 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA C 1 = C = C 3 = F 1 i + 1 F 1 i + 1 G 1 i + 1 i G 1 + 1, F 1 i + 1 i G 1 + 1 G 1 i + 1 i G 1 + 1, F 1 i + 1 F 1 i + 1 F 1 i + 1 i G 1 + 1. Fro Lea A.1, we have Ed F, F 0, Ed G, G 0, and d F, G W F, G. We state and prove Lea A.1 later. For C 1, EC1 E F 1 i + 1 F 1 i + 1 E G 1 i + 1 i G 1 + 1. Fro Lea A.1 EC 1 0. Siilarly EC, EC 3 0. Fro A.1, Lea A.1, and A. we have 1 E F 1 i + 1 G 1 i + 1 W F, G. Then, λ λ = γ1 W F, G / and thus, R 1 = Also, Hence, θ i µ γy i µ = σθ A. w i z i σθ W F, G. γ λ = σ θ W F, G. σ y R λ R 1 σθ W 4 F, G = R. 4 To find a direct expression for R γ, we use R γ = σθ 1 E w i γz i = σθ 1+γ γ 1 E z i w i. We have shown that 1 E z iw i 1 W F, G /. Hence, R γ σθ 1 + γ 1 W F, G 1 W F, G

AN OPTIMAL SHRINKAGE FACTOR 177 γ = R + σθ W F, G 1. Lea A.1. Under A1, and A, as goes to infinity, d F, G W F, G and Ed F, F, Ed G, G converges to zero. The proof is in the suppleentary aterials. Proof of Theore 3. Let S = 1 { 1 z i + 1 w i 1 z i w i } = 1 { 1 z i + 1 w i 1 F 1 i + 1 G 1 i + 1 }.A.3 Here, E1/ z i = 1 and E1/ w i = 1. The last suation converges to W F, G by Lea A.1, siilar to Theore 1. Thus, R 1 Siilarly, R = θ i µ z i = σ θ w i z i σ θ W F, G. σθ W K, G. As, σ θ λ = σ θ W F, G /, R λ σθ W F, G σθ W 4 F, G /4. References Brown, L. D. 1971. Adissible estiators, recurrent diffusions, and insoluble boundary value probles. Ann. Math. Statist. 4, 855-904. Brown, L. D. and Greenshtein, E. 009. Nonparaetric epirical Bayes and copound decision approaches to estiation of a high diensional vector of noral eans. Ann. Statist. 37, 1685-1704. Barrio, E. D., Gin, E. and Utzet, F. 005. Asyptotics for L functionals of the epirical quantile process, with applications to tests of fit based on weighted Wasserstein distances. Bernoulli 11, 131-189. Fay, R. E. and Herriot, R. A. 1979. Estiates of incoe for sall places: An application of Jaes-Stein procedures to census data. J. Aer. Statist. Assoc. 74, 69-77. Judkins, D. R. and Liu, J. 000. Correcting the bias in the range of a statistic across sall areas. J. Official Statist. 16, 1-13. Louis, T. A. 1984. Estiating a population of paraeter values using Bayes and epirical Bayes ethods. J. Aer. Statist. Assoc. 79, 393-398. Lukacs, E. 1970. Characteristic Functions. nd edition. Griffin, London. Malinovsky, Y. and Rinott, Y. 010. Prediction of ordered rando effects in a siple sall area odel. Statist. Sinica 0, 697-714. Pfefferann, D. 013. New iportant developents in sall area estiation. Statist. Sci. 8, 40-68. Shanbhag, D. N. and Sreehari, M. 1977. On certain self-decoposable distributions. Z. Wahrsch. Verw. Gebiete 38, 17-.

178 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Shen, W. and Louis, T. A. 1998. Triple-goal estiates in two-stage hierarchical odels. J. Roy. Statist. Soc. Ser. B 60, 455-471. Stein, C. 1956. Inadissibility of the usual estiator for the ean of a ultivariate distribution. Proc. Third Berkeley Syp. Math. Statist. Probab. 1, 197-06. Wright, D. L., Stern, H. S. and Cressie, N. 003. Loss function for estiation of extree with an application to disease apping. Canad. J. Statist. 31, 51-66. Departent of Statistics, Texas A & M University, College Station, TX 77843, USA. E-ail: nguha@ath.tau.edu Departent of Matheatics and Statistics, University of Maryland, Baltiore County Baltiore, MD 150, USA. E-ail: anindya@ubc.edu Departent of Matheatics and Statistics, University of Maryland, Baltiore County Baltiore, MD 150, USA. E-ail: yaakov@ubc.edu Departent of Statistics, University of Georgia, 101 Cedar Street, Athens, GA 3060, USA. E-ail: gauri@uga.edu Received August 014; accepted Noveber 015