AN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS
|
|
- Chrystal Warren
- 6 years ago
- Views:
Transcription
1 Statistica Sinica 6 016, doi: AN OPTIMAL SHRINKAGE FACTOR IN PREDICTION OF ORDERED RANDOM EFFECTS Nilabja Guha 1, Anindya Roy, Yaakov Malinovsky and Gauri Datta 3 1 Texas A & M University, University of Maryland, Baltiore County and 3 University of Georgia Abstract: The proble of predicting a vector of ordered paraeters or its part arises in contexts such as easureent error odels, signal processing, data disclosure, and sall area estiation. Often estiators of functions of the ordered rando effects are obtained under strong distributional assuptions, e.g., norality. We discuss a siple generalized shrinkage estiator for predicting ordered rando effects. The proposed approach is distribution free and has significant advantage when there is odel isspecification. We give expression to and characterization of the optial shrinkage paraeter; the expression involves the Wasserstein distance between two odel-related distributions. We provide a fraework for estiating the distance and thereby estiating an epirical version of the oracle optial estiator. We copare the risk for the optial predictor to that of other distribution-free estiators. Extensive siulation results are provided to support the theoretical results. Key words and phrases: Epirical Bayes predictor, linear predictor, order statistics, shrinkage. 1. Introduction A coon odel of interest is y i = θ i + e i, i = 1,,...,, 1.1 where the σ 1 i e i are assued independent and identically distributed as H0, 1, a ean zero unit variance distribution, with the constants σ i assued to be known. Independent of the e i, the θ i are given as θ i = µ i + u i, where the u i are independently and identically distributed i.i.d. as ean zero, finite variance rando variables with distribution G. This odel finds wide-spread applications ranging fro easureent error odels, signal processing, data disclosure, sall area estiation to nae a few. Here, we develop ethodology for predicting the ordered rando effects, θ i, in the context of Model 1.1. Model 1.1 is a special case of the Fay-Herriot odel Fay and Herriot 1979 in sall area estiation SAE, where the area eans θ i are further odeled using area specific covariate inforation x i = x i1,..., x ip, and area
2 1710 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA specific rando effects u i as θ i = x i β + u i. Following SAE terinology we refer to θ = θ 1,..., θ as the vector of area eans. While predicting ηθ = θ is coon, investigators have also studied prediction of other functions; vector of ranks, the epirical distribution of the area eans Shen and Louis 1998 and the range of area eans Judkins and Liu 000. Here we are interested in predicting the vector of order statistics ηθ = θ = θ 1 θ θ. Prediction of the ordered eans is significantly harder than prediction of the linear function of the area eans Pfefferann 013. When G and H are correctly specified, the posterior ean ˆη = Eηθ y iniizes the prediction ean squared error PMSE, Rˆη = E[ˆη ηθ ˆη ηθ] where y = y 1,..., y i.i.d. is the data. When e i N0, σ i.i.d. and θ i Nµ, σθ the Bayes estiator of θ is ˆθ B = y 1 γy µ1, where γ = σ θ σ θ + σ and 1 is a vector of ones. The epirical Bayes EB estiator is obtained by replacing µ by ȳ, ˆθ EB = y 1 γy ȳ1, which is also the Best Linear Unbiased Predictor BLUP in the class of all {H, G} with finite second oents. Brown 1971 and Brown and Greenshtein 009 have looked at Bayes/epirical Bayes estiation under general prior. The plugged-in version η ˆθ B, however is not the Bayes predictor and can result in substantial bias in prediction. Wright, Stern, and Cressie 003 considered a Bayesian schee for predicting ordered eans but their procedure is sensitive to prior choice and requires substantial coputation. When G and H are partially specified up to lower order oents Stein s shrinkage estiators Stein 1956 can be used for a variety of paraetric functions but for the ordered paraeters no suitable predictors are available. When error variances are assued equal, Malinovsky and Rinott 010 proposed a class of shrinkage estiators θ i λ = λy i + 1 λµ. 1. They showed that the risk iniizing value of λ lies in the interval [γ, γ] and, based on siulation evidence, conjectured the asyptotic optial value to be γ. The weight γ also appears in Louis 1984, who proposed Bayes and epirical Bayes predictors that iniize an expected distance function between the epirical cdf of predictors of θ and epirical cdf of its true value. We use siilar shrinkage estiators under odel 1.1 and derive expressions for the optial shrinkage paraeter. The optiu estiator is shown to have good finite saple perforance with respect to ean squared prediction error, even in coparison to the best estiator when G and H are known to be noral. Thus, we provide an estiator that can predict the order paraeters with
3 AN OPTIMAL SHRINKAGE FACTOR 1711 reasonable accuracy and does not ake strong distributional assuptions. For the equal error variance case we show that the optial choice of λ in 1. is not necessarily γ, and characterize the cases when γ is indeed the asyptotically optial choice for λ. Based on the derived expression for the optial value of λ, we propose a new class of predictors for the ordered paraeters and provide a fraework for estiation of the optial predictor. We illustrate its finite saple perforance via siulation.. Prediction of Ordered Rando Effects It is instructive to begin with a special case of odel 1.1 in which the design variances are all assued to be equal. Under the assued odel, constant error variance would iply that the errors are i.i.d.. Since we later consider the case when the error variances are not equal, we do not separately consider the case where the errors have equal variance but are not necessarily identically distributed. We assue that θ i arise following soe distribution G, with ean µ and variance σθ, but we do not specify the fors of G and H..1. Prediction in the equal variance odel Assue odel 1.1 with constant variances, σ1 = = σ = σ. The arginal distribution of y i is denoted by F which under the assued odel has ean µ and variance σy = vary i = σθ + σ. For prediction of the ordered paraeters, we consider the class of shrinkage predictors 1.. Under squared error loss the PMSE for a saple of size is R λ = 1 E θ i λy i 1 λµ and the optial risk iniizing value of λ is λ = 1 E y i µθ i µ 1 E y i µ..1 We study the liiting for of.1 as and evaluate the relative efficiency of the optial shrinkage coefficient with respect to other predictors. Let { 1 WF, G = [F 1 t G 1 t] }1/ dt 0 denote the L Wasserstein etric between the distributions F and G, assued to have finite variance. We consider the predictors θ i γ = γy i + 1 γµ and θ i γ = γy i + 1 γµ.
4 171 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Following the for of the BLUP for the unordered paraeters, a natural choice for the predictor of the ordered quantities would be θ = θ 1 γ θ γ, while the predictor θ 1 = θ 1 γ θ γ is the for conjectured in Malinovsky and Rinott 010 to have asyptotically optiu perforance. The PMSE associated with the predictors θ 1 and θ are R 1 = R γ and R = R γ, respectively. For any estiator of the for θ i λ define the relative efficiency with respect to θ i γ as RE 1 λ = R 1 /R λ, and that of θ i λ with respect to θ i γ as RE λ = R /R λ. Let the distribution of the standardized observations, y i µ/σ y, be F and that of the standardized paraeters, θ i µ/σ θ, be G where σy = σθ + σ. We require soe conditions on F and G. A1: The distributions F and G have finite fourth oents. A: For all 0 < t < 1/, F x and G x have continuous, positive derivatives on x F 1 t, F 1 1 t and x G 1 t, G 1 1 t, respectively. Theore 1. Under A1 A, as, λ λ = γ 1 W F, G R λ R = σθ W F, G W 4 F, G 4 [ ] 1, RE 1 λ RE 1 = RE 1 W F, G 4 λ RE = 1 + [1 W F, G / γ] [1 1 W F, G /. ] The gain in PMSE at the optial shrinkage value over that at γ, is [1 W F, G /4] 1. This iproveent can be quite significant if the Wasserstein distance between F and G is large and, as 0 W F, G, potentially there can be a two fold reduction in the PMSE of the optial predictor over that of θ i γ. We find in the siulations that the gain in efficiency fro the optial predictor can be substantial. Reark 1. If F and G are equal, then W F, G = 0 and λ = γ, and the PMSE of the optial predictor goes to zero as goes to infinity. Here, the distribution of the centered y i is a scaled version of that of the centered θ i, and a siple scaling of the observed values provides the optial prediction. In the context of equal error variance Malinovsky and Rinott 010 conjectured the optiu value of λ in 1. to be γ. Theore 1 iplies that the,,
5 AN OPTIMAL SHRINKAGE FACTOR 1713 result holds iff W F, G = 0. As F and G are distributions of the standardized quantities, to derive necessary and sufficient condition for W F, G = 0, without loss of generality, we take µ = 0. Theore. Suppose 1.1 holds and the errors e i are independently and identically distributed as H with ean zero and variance σ. Then the Wasserstein distance etric W F, G between the distributions of standardized θ and standardized y is zero if and only if θ i has the sae distribution as that of k=1 ck e k where c = γ = σ θ /σ y. Proof. If part : If θ = k=1 ck e k, then y = k=0 ck e k, where e k s are i.i.d for all k. Hence, cy has sae distribution as θ and after standardization u and y have the sae distribution. Hence, WF, G = 0. Only if part : We can write cy i = cθ i + ce i. Fro W F, G = 0, follows that yi = cy i has sae distribution as θ i. Iterating the procedure we see that θ has the sae distribution as k=1 ck e k. Because c < 1, the series representation is valid in ean squared sense. Reark. Noral distributions on θ and e give W F, G = 0, and hence Gaussianity is a sufficient condition for Theore to hold. However, as shown in Theore, the class of distribution pairs H, G that will give W F, G = 0 is a uch wider class containing the noral distribution. In such cases, F is a self-decoposable distribution Lukacs 1970 and exaples of such distribution could be found in Shanbhag and Sreehari Optial prediction with unequal design variances With a slight odification, a shrinkage predictor siilar to 1. can be proposed in the unequal variance case as well. In order to derive the liiting for of the optial estiator we have to ake assuptions about the convergence of the epirical distribution of the standardized responses. Such assuptions autoatically hold in the i.i.d. case considered in Section.1. Let v i = vary i = σ θ + σ i and z i = y i µ/v i. Then we propose a class of shrinkage predictors for the ordered area eans as θ i λ = λz i + µ.. The PMSE at λ is R λ = 1 E θ i λz i µ.
6 1714 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA If σi s are the sae, the class of predictors in. reduces to the class 1.. Let γ i = σθ /σ θ + σ i. Then analogous to the equal variance case, one could look at the predictors θ 1 = θ 1 1 θ1 where θ 1 i = σ θ z i + µ and θ = θ 1 θ where θ i = γ i y i + 1 γ i µ. Unlike the equal variance case, the predictor θ i does not belong to the class. but rather it is the ordered version of the area specific BLUP for the unordered area eans. Let R 1 and R denote the PMSE of θ 1 and θ, respectively, with RE1 λ = R 1 /R λ and RE λ = R /R λ. Let w i = θ i µ/σ θ denote the standardized area eans. Then the shrinkage coefficient with iniu PMSE is given by w iz i λ 1 E = σ θ 1 E z i..3 We establish a sipler liiting for for the optial shrinkage coefficient, leading to suitable predictor that can be used once the unknown paraeters have been substituted with data estiates. Let F and K denote the epirical distributions of z i and γ i z i, respectively. A3 The sequence of distributions F and K are uniforly integrable and converge in distribution to ean zero distributions F and K with finite fourth oents, respectively. The distribution of w i, G has finite fourth oent. Theore 3. Under A3, if A holds for F, G and K, as [ λ λ = σ θ 1 W F, G ], [ R λ R = σθ 1 1 W F, G ], [ RE 1 λ RE 1 = 1 W F, G ] 1, 4 RE λ RE = W K, G [1 1 W F, G / ]. Based on the optial value of the shrinkage coefficient, the proposed predictor for the ordered θ i is θ = θ 1 θ where θ i = σ θ [ 1 W F, G ] z i + µ..4
7 AN OPTIMAL SHRINKAGE FACTOR 1715 Reark 3. Our results hold for the ore general loss function in which different ordered effects have different weights for their corresponding risks. More details of this can be found in the suppleentary docuent..3. An application to sall area estiation We consider SAE where a fixed area level effect is present, and the ean value of the ith area, θ i = Ey i θ i, potentially depends on the characteristics of the area. Let θ i = µ i + u i and hence y i = µ i + u i + e i. The µ i are fixed effects and the u i are rando effects. Typically area specific fixed effects are odeled as µ i = x i β. We take the u i as i.i.d. N0, σθ and the e i as i.i.d. N0, σi. Write the standardized response as z i = y i x i β/v i where vi = σ θ + σ i is the variance of y i. Following the generalized shrinkage estiation developent, we can predict u i = σ θ z i. For predicting θ i s we propose θ i = σ θ z i + x iβ,.5 and let θ = θ 1 θ be the ordered values of θ i. Reark 4. Use of θ in the equal variance case is justified because axiu a posteriori order for the latent rando effects is the sae as the order of the observed quantities under ild distributional assuptions. More details are provided in the suppleentary docuent. We do not address the rank estiation issue directly. A short discussion on rank estiation is included in the suppleentary aterials, in the context of odel Epirical Version of the Predictors In practice, the unknown paraeters in the expression for the optial shrinkage predictor are replaced with their estiators. Thus, at.4, one plugs in the estiated values of µ, W F, G and σ θ Epirical predictor Unless otherwise entioned we use the saple ean ȳ to estiate µ. Other estiators, such as the saple edian can be considered. Estiation of σ θ is straightforward, but estiation of W is ore involved. A consistent ethod-ofoent estiator of σ θ is ˆσ θ = ax { 1 y i ȳ 1 } σi, 0. Based on the estiated σ θ, we can replace v i by ˆv i = ˆσ θ + σ i. We also use ẑ i = y i ȳ/ˆv i as the observed standardized response in order to copute the Wasserstein distance.
8 1716 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA If the faily of distributions G0, σθ is known up to σ θ, then W F, G can be estiated epirically once F is estiated based on ˆσ θ. In cases where G is unknown we can proceed as follows. We assue that the error distribution is a known finite location-scale ixture of noral distributions a good approxiation to H0, σi and that each ixture coponent is independent of the unobserved θ. We also use a finite noral location scale ixture representation for the distribution of θ. Thus, we take L e i p e,l,i Nµ e,l,i, σe,l,i, 3.1 l=1 where p e,l,i, µ e,l,i, and σe,l,i are all known and K θ i G = p θ,k Nµ θ,k, σθ,k. 3. Then y i F i = k=1 K k=1 l=1 L p k,l,i Nµ k,l,i, σk,l,i, where p k,l,i = p θ,k p e,l,i, µ k,l,i = µ θ,k + µ e,l,i, and σk,l,i = σ θ,k + σ e,l,i. One can then use the EM algorith to estiate the distributions and hence estiate the Wasserstein distance based on the estiated distributions, say Ŵ F, G. For coputation and ipleentation, it is ore efficient to use the finite saple version of the Wasserstein etric associated with the finite saple version of the optial shrinkage and estiate that to plug-in into the predictor. Set, WF, G = 1/ F 1 i/ + 1 G 1 i/ + 1 and W F, G = E1/ F 1 i/ + 1 G 1 i/ + 1. Given the noral location scale ixture representation of G we can generate independent observations fro the distribution of θ and generate a copy of observed y s using the known error distribution. Then, W F, G is estiated by its Monte-Carlo estiator. Let, F,j and G,j be the epirical distribution for standardized θ and y in j th replication. We estiate W F, G = 1 R 1 F 1,j R i + 1 G 1,j i + 1, j=1 where R is the nuber of replications. Let ˆλ be the value of the optial shrinkage coefficient when σ θ and W F, G have been replaced by their estiators ˆσ θ and W F, G. Then the estiated optial predictor.4 for the ordered area eans is ˆθ = ˆθ 1 ˆθ where
9 AN OPTIMAL SHRINKAGE FACTOR 1717 ˆθ i = ˆσ θ [ 1 W F, G ] ẑ i + ȳ. 3.. Accuracy of the epirical predictor The epirical estiator is a plug-in version of the optial predictor. To judge its accuracy we derive asyptotic expression for the PMSE of the epirical predictor. We need soe assuptions to establish the asyptotic rates. A4: For all 0 < t < 1/, F x and G x have continuous and positive derivative on x F 1 t, F 1 1 t and x G 1 t, G 1 1 t, respectively. A5: Let SF = {x : 0 < F x < 1} and SG = {x : 0 < G x < 1} be the open supports of F and G. F and G are twice differentiable on their open supports and their corresponding densities, f and g, are strictly positive on their respective open supports. A6: A7: 1 0 t1 t f F 1 dt < and t 1 0 t1 t g G 1 dt <. t t1 t f F 1 t t1 t g G 1 t sup 0<t<1 f F 1 t < and sup 0<t<1 g G 1 t <. A8: The densities fx and gy are onotone for x / F 1 t, F 1 1 t and y / G 1 t, G 1 1 t for soe 0 < t < 1/. A9: There exists c > 0, such that inf i σ i > c, and ˆσ θ, 1ˆσ θ, >0 < K for all > 0 for soe 0 and K, where ˆσ θ, is the estiate of σ θ based on observations. A10: Assue that consistent estiators of W, and W, Ŵ, and W, are available. Assuptions A4 A8 can be found in Barrio, Gin, and Utzet 005 in the context of convergence of integrated quantile differences. Here A9 is needed for the case when σ θ is being estiated. Assuption A10 is plausible since, if the assued location scale representation is correct, as in that case the MLE of the paraeters in the ixture odel is - consistent for the true value and W, is a continuous function of the paraeters. Proposition 1. Under A1 A, A10, for the equal error variance case at.1, σ θ /σ y 1 W F, G / = λ + O P 1/. If A4 A8 and A10 hold, then σ θ /σ y 1 Ŵ F, G / = λ + O P 1/.
10 1718 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Proposition. Under A A3, A10, for the unequal error variance case at.3, σ θ 1 W F, G / = λ + O p 1/. Under A A8 and A10, σ θ 1 Ŵ F, G / = λ + O p 1/. The following results generalize Proposition 1 and when the optial shrinkage predictor is based on plugged-in estiators for W, σ θ and µ. Theore 4. Under A1 A10, for the equal variance case at.1, ˆσ θ 1 ˆσ y ˆσ θ ˆσ y If W F, G > 0, [1 W F, G = λ + O p 1/, 1 Ŵ F, G = λ + O p 1/. W W F, G ] 1 = RE 1 λ 4 + O p 1/, K, G [1 1 W F, G / ] = RE λ + O p 1/. Theore 5. Under A A10, for the unequal design variance case at.3, If W F, G > 0, [ 1 W ˆσ θ 1 F, G = λ + O P 1/, ˆσ θ 1 Ŵ F, G = λ + O P 1/. W W F, G 4 K, G [1 1 W F, G / ] ] 1 = RE 1 λ + O p 1/, = RE λ + O p 1/. Theores 4 and 5 provide approxiations to the relative efficiency of the estiated version of the optial shrinkage predictor. In ters of PMSE the proposed optiu shrinkage estiator perfors better than the BLUP type estiator in the equal error variance case. But for the unequal variances this ay not happen as the BLUP type estiator does not belong to the class of estiators represented by.. In our odel-based approach to estiating
11 AN OPTIMAL SHRINKAGE FACTOR 1719 W F, G, once the noral location-scale ixture odel for θ is estiated we can estiate W K, G as well. Thus, the relative efficiency of the BLUP type estiator copared to the proposed optiu estiator can be estiated and the estiator with lower value of estiated asyptotic PMSE can be used. 4. Siulation Study 4.1. Optiu shrinkage and Wasserstein correction We carried out three exaples. The saple sizes considered were =,000 and = 10,000. The larger saple size was chosen to evaluate the accuracy of the estiation of the Wasserstein distance, and copare with the theoretical asyptotic value. Each reported Monte Carlo value was based on 500 replications. Exaple 1. The first experiental scenario was designed to evaluate the effect of the Wasserstein distance on the perforance of the different predictors. The area ean distributions were chosen to be two coponent noral scale ixtures paraeterized by a single paraeter: θ i a 1 N0, a a 1 N0, a 1 1. Here Eθ = 0 and V arθ = 1, and W F, G is an increasing function of a [,, with W = 0 for a =. The error distribution was fixed as standard noral. For the siulation we took a {, 5, 10, 100, 1,000}. The scale ixture odels for different values of a are denoted by Nixa. Exaple. We look at two distributions for the area eans: a Double Exponential distribution to reflect possible heavy tails in the distribution and a two coponent location ixture of norals to account for possible ultiodality in the area ean distribution. In particular we took θ i 0.5N4, N 4, 1 and θ i DE, where, in each case, the errors were generated independently using e i N0, σi. For the noral ixture case, we considered two cases, constant error variances, σi = 16, and σi Unifor0, 16. For the double exponential, we took σi = 1 and σi Unifor0, 1. The double exponential odels and noral location ixture odels with equal and unequal variances are denoted by DE E, DE U, Nix E and Nix U, respectively. The optiu shrinkage coefficient was used in each exaple. We considered the shrinkage predictors θ, θ1 version of the predictors where σθ, and θ. For prediction we used the estiated and W are obtained following the procedure described in Section 3 and plugged into the expression of the predictors. The value K = 6 was used.
12 170 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 1. Relative perforance of the different shrinkage predictors. θ vs θ1 θ vs θ Model W Ŵ RE,000 RE 10,000 RE RE,000 RE 10,000 RE Nixa a = a = a = a = a = a = DE E DE U Nix E Nix U Table 1 gives the ratio of PMSE of the optial predictors copared with the other predictors at the two saple sizes. Colun gives the value of the true Wasserstein distance and colun 3 gives the estiate of W averaged over the Monte Carlo replications. Coluns 4-6 give the relative efficiency of the optial shrinkage estiator θ copared to the estiator θ1 at saple sizes =,000, 10,000, and =, respectively. The value at = is the theoretical value given in the Theore 1. Coluns 7 9 give the relative efficiency value of the optial estiator copared to θ at =,000, 10,000, and =, respectively. In the noral scale ixture odels, for saller values of the Wasserstein distance, the optial shrinkage predictor θ and the one ignoring the Wasserstein correction, θ 1, are nearly identical. This is expected since for values of W close to zero, the correction factor is close to one and the two predictors essentially coincide. However, the BLUP-type estiator θ is uch inferior to the other estiators in the sall W scenario. When W is large, the optial estiator is considerably better than θ 1, because ignoring the Wasserstein correction has a significant effect on the predictor. Here BLUP-type estiator θ is nearly identical to the optial predictor. For oderate W, the optial shrinkage provide substantial gains over both θ 1 and θ. In the noral ean-ixture exaple for unequal or equal variances θ and θ 1 perfor better than the θ and, with Wasserstein correction, θ perfors better than θ 1 in the equal variance case with relatively higher value of W. For double exponential scenario θ and θ1 perfor uch better than the θ and, because of the sall value of W, the Wasserstein correction is unnecessary for all practical purposes.
13 AN OPTIMAL SHRINKAGE FACTOR 171 Table. Relative perforance of the different shrinkage predictors. θ vs θ1 θ vs θ α W Ŵ RE 500 RE ˆ RE RE 500 RE ˆ RE α = α = α = Figure 1. Plot for the nonparaetric fit for density of G. The left hand panel shows a typical fit for shifted Gaa distribution for skewed G, the right hand panel is a typical fit in exaple 1 with a = 10. The solid line is the fitted and the dashed line is the true target density. Exaple 3. We investigated the effect of skewness in the area ean distribution. We took θ i Gaa1.5, 1.5 1, an asyetric distribution for the rando effect part around ean zero and with support 1,. Error ters were noral and we took σi = bα + 1 αc i, with c i = 1 i/, b = 3 and 0 α 1. We considered the cases α = 0, 0.5 and 1. The relative efficiency of θ with respect to θ1 and θ is given in the Table for = 500 with the data estiate of the relative efficiencies given by RE, ˆ where the nuber of replication was 100. In these exaples the proposed ixture odel approach was able to estiate G. Figure 1 shows the estiated density in two cases; in all cases the proposed estiator provided a reasonable approxiation to G. 4.. Sall area estiation For the SAE odel θ i and e i are assued to be noral. By Theore, the Wasserstein distance between the standardized distributions of the area eans and the responses is zero. Thus, by Theore 3, the optial shrinkage estiator
14 17 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 3. Relative efficiency of the proposed predictor to BLUP-type predictor. c = 100 = 300 = θ is identical to θ1. We considered two cases. Case 1: In this case = 100, 300, and 500 sall areas were considered. We took the case with a single covariate and, for the i th area we had y i = α + βx i + u i + e i. We took u i N0, 16 with α = 1, β =. We generated x i N0, 1 and e i N0, σi. The error variance values were generated as σi U0, c. We chose c fro c = 1, 3, 5. The siulation results are reported for 500 Monte Carlo replications. The proposed estiator θ is copared with the BLUP-type predictor θ. The relative efficiencies are reported in Table 3. The proposed predictor outperfors the BLUP-type predictor. The difference is significantly higher when the γ i are further fro one, that is when c is 3 or 5. For large, the percentage of iproveent is generally greater with the optial predictor providing 10 50% gain in efficiency of prediction. Case : The gain in the perforance of the optial predictor is due to better prediction of the order statistics of the rando effects u i. When the observed values are highly influenced by the fixed effects, then the BLUP-type predictor is expected to perfor coparably with the optial predictor since the ain reduction in risk is achieved by accurate prediction of the fixed effect part. To evaluate the effect of the correlation of the responses with the fixed effect on the perforance of the optial predictor we considered a ore general case with different values of β. For higher β the fixed effect α + βx i doinates and the response y i is higher and this effects the perforance of the shrinkage estiator. We use the sae odel as before Case 1, with α = 1, β = i, where i = 1,,..., 15. Also, we varied the nuber of sall areas as = 100j with j = 1,, 3, 4, 5. The area specific variances were generated as U0, c, with c = 1, 3, 5, 8. The relative efficiencies of the BLUP-type predictor θ copared to the optial predictor θ 1 are given in Figure. The lower the relative efficiency, the better the perforance of the proposed predictor in predicting the order statistics. The proposed predictor has significantly saller PMSE for a variety of cases where the correlation between the fixed effect and the response is oderate to sall. For larger values of β the responses are essentially the fixed effects, and
15 AN OPTIMAL SHRINKAGE FACTOR 173 Figure. Fro left to right the figures correspond to c = 1, 3, 5, 8, respectively. The X-axis denotes i = 1,..., 30. Here = 100j and j = 1,, 3, 4, 5 are denoted by,,, and, respectively. the relative efficiency approaches one. Since the optial predictor is being replaced by its plug-in version, the relative efficiency of the BLUP-type estiator is actually slightly larger than one, when the area eans are essentially of the sae agnitude as that of the fixed part α + βx i Coparison with the full Bayesian estiator If the distributions G and H are known, Bayesian coputation can be used to generate the posterior saples of θ i and estiate the posterior expectation of the ordered eans by taking the ean of the ordered posterior saples. The posterior ean is the best estiator in ters of PMSE. It is interesting to copare the Bayesian ethod to our distribution free approach. We also study the sensitivity of the Bayesian estiator to odel isspecification, especially in the odel for the rando effects paraeter θ. We considered Students t distribution with various degrees of freedo for G and assued it to be isspecified as noral distribution. Also, a ixture noral distribution was considered for θ with G.5N1.5, 1 +.5N 1.5, 1. We took, equal variances E σi = 1 and unequal variances U σi U0, 1. In the Bayesian ethodology we assued the prior on θ, G, to be Nµ, σ and assued the following non-inforative priors Πµ, σ 1. The ratio of the square root of the PMSE s for the shrinkage and the Bayesian ethods are reported in Table 4. If the odel is truly specified then the ratio should be greater than one. Under odel isspecification the shrinkage generally perfors better than the Bayesian ethod, and for the correctly specified case the Bayesian is better, as expected. However, even in the correctly specified odel, the odel free estiator continues to have reasonable perforance.
16 174 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Table 4. Relative efficiency of the optial shrinkage and Bayes predictor. G =100 =400 =900 U E U E U E T T T T N0, Mixture Conclusion We propose an optial estiator of ordered rando effects in the class of siple shrinkage estiators. The ain attraction of the proposed estiator is that it is distribution free. It is very robust to odel isspecification and, as evident fro siulations the distribution free estiator is reasonably efficient with respect to the best Bayesian estiator in a correctly specified noral odel. The estiator provides an easy and direct way for predicting ordered rando effects under both equal and unequal error variances. Fro siulations, in any setups we see significant relative gain in efficiency for the optial shrinkage estiator over other shrinkage estiators in the sae class. We also derived liiting for of the risk of the estiator and proposed a ethod for estiating the risk. The estiator depends on the Wasserstein distance between two standardized distributions and a ethod based on the observed y i s is also given for estiating it. The optial estiator based on the estiated Wasserstein distance is shown to have reasonable asyptotic properties. We also address the situation when area specific covariate inforation is available. Alternative classes of estiators could involve linear shrinkage estiators with different shrinkage for the different order statistics, or classes of shrinkage estiators that directly account for the joint dependence aong the order statistics. Inference in such classes ay be ore difficult, and this is an interesting area for further exploration. Generalization to classes of estiators that can effectively account for area specific covariates while having the advantage of being distribution free is a topic of future research. The proposed estiator gives a good starting point and a preliinary fraework for ore general classes of distribution-free estiators. Suppleentary Materials The suppleentary docuent has four sections. Details about Reark 3 are given in Section 1. The justification and proof of results related to the sall area odel in Reark 4 are given in Section. In the third section we prove soe
17 AN OPTIMAL SHRINKAGE FACTOR 175 results fro the anuscript and in the fourth, we discuss the rank estiation issue. Acknowledgeent We thank an associate editor and two reviewers for their thoughtful coents and suggestions. The authors would like to thank Dr. A. M. Kagan for pointing out the connection to self-decoposable distributions. Appendix Proof of Theore 1. Fro.1 Hence, where λ = 1 E y i µθ i µ 1 E y i µ. λ = 1 σ θ σ y E [y i µ/σ y ][θ i µ/σ θ ] 1 E y i µ/σ y = σ θ ES σ, y S = 1 { 1 z i + 1 w i 1 z i w i }. Because E1/ z i = 1 and E1/ w i = 1 we have S = 1 { 1 z i + 1 w i T 1 }, where Then, T 1 = 1 F 1 i + 1 G 1 i. + 1 where T 1 = d F, F + d G, G + d F, G + C 1 + C + C 3, d F, F = 1 F 1 i + 1 F 1 i + 1, d G, G = 1 G 1 i + 1 i G 1 + 1, d F, G = 1 F 1 i + 1 i G 1 + 1, A.1
18 176 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA C 1 = C = C 3 = F 1 i + 1 F 1 i + 1 G 1 i + 1 i G 1 + 1, F 1 i + 1 i G G 1 i + 1 i G 1 + 1, F 1 i + 1 F 1 i + 1 F 1 i + 1 i G Fro Lea A.1, we have Ed F, F 0, Ed G, G 0, and d F, G W F, G. We state and prove Lea A.1 later. For C 1, EC1 E F 1 i + 1 F 1 i + 1 E G 1 i + 1 i G Fro Lea A.1 EC 1 0. Siilarly EC, EC 3 0. Fro A.1, Lea A.1, and A. we have 1 E F 1 i + 1 G 1 i + 1 W F, G. Then, λ λ = γ1 W F, G / and thus, R 1 = Also, Hence, θ i µ γy i µ = σθ A. w i z i σθ W F, G. γ λ = σ θ W F, G. σ y R λ R 1 σθ W 4 F, G = R. 4 To find a direct expression for R γ, we use R γ = σθ 1 E w i γz i = σθ 1+γ γ 1 E z i w i. We have shown that 1 E z iw i 1 W F, G /. Hence, R γ σθ 1 + γ 1 W F, G 1 W F, G
19 AN OPTIMAL SHRINKAGE FACTOR 177 γ = R + σθ W F, G 1. Lea A.1. Under A1, and A, as goes to infinity, d F, G W F, G and Ed F, F, Ed G, G converges to zero. The proof is in the suppleentary aterials. Proof of Theore 3. Let S = 1 { 1 z i + 1 w i 1 z i w i } = 1 { 1 z i + 1 w i 1 F 1 i + 1 G 1 i + 1 }.A.3 Here, E1/ z i = 1 and E1/ w i = 1. The last suation converges to W F, G by Lea A.1, siilar to Theore 1. Thus, R 1 Siilarly, R = θ i µ z i = σ θ w i z i σ θ W F, G. σθ W K, G. As, σ θ λ = σ θ W F, G /, R λ σθ W F, G σθ W 4 F, G /4. References Brown, L. D Adissible estiators, recurrent diffusions, and insoluble boundary value probles. Ann. Math. Statist. 4, Brown, L. D. and Greenshtein, E Nonparaetric epirical Bayes and copound decision approaches to estiation of a high diensional vector of noral eans. Ann. Statist. 37, Barrio, E. D., Gin, E. and Utzet, F Asyptotics for L functionals of the epirical quantile process, with applications to tests of fit based on weighted Wasserstein distances. Bernoulli 11, Fay, R. E. and Herriot, R. A Estiates of incoe for sall places: An application of Jaes-Stein procedures to census data. J. Aer. Statist. Assoc. 74, Judkins, D. R. and Liu, J Correcting the bias in the range of a statistic across sall areas. J. Official Statist. 16, Louis, T. A Estiating a population of paraeter values using Bayes and epirical Bayes ethods. J. Aer. Statist. Assoc. 79, Lukacs, E Characteristic Functions. nd edition. Griffin, London. Malinovsky, Y. and Rinott, Y Prediction of ordered rando effects in a siple sall area odel. Statist. Sinica 0, Pfefferann, D New iportant developents in sall area estiation. Statist. Sci. 8, Shanbhag, D. N. and Sreehari, M On certain self-decoposable distributions. Z. Wahrsch. Verw. Gebiete 38, 17-.
20 178 NILABJA GUHA, ANINDYA ROY, YAAKOV MALINOVSKY AND GAURI DATTA Shen, W. and Louis, T. A Triple-goal estiates in two-stage hierarchical odels. J. Roy. Statist. Soc. Ser. B 60, Stein, C Inadissibility of the usual estiator for the ean of a ultivariate distribution. Proc. Third Berkeley Syp. Math. Statist. Probab. 1, Wright, D. L., Stern, H. S. and Cressie, N Loss function for estiation of extree with an application to disease apping. Canad. J. Statist. 31, Departent of Statistics, Texas A & M University, College Station, TX 77843, USA. E-ail: nguha@ath.tau.edu Departent of Matheatics and Statistics, University of Maryland, Baltiore County Baltiore, MD 150, USA. E-ail: anindya@ubc.edu Departent of Matheatics and Statistics, University of Maryland, Baltiore County Baltiore, MD 150, USA. E-ail: yaakov@ubc.edu Departent of Statistics, University of Georgia, 101 Cedar Street, Athens, GA 3060, USA. E-ail: gauri@uga.edu Received August 014; accepted Noveber 015
Keywords: Estimator, Bias, Mean-squared error, normality, generalized Pareto distribution
Testing approxiate norality of an estiator using the estiated MSE and bias with an application to the shape paraeter of the generalized Pareto distribution J. Martin van Zyl Abstract In this work the norality
More informationA Simple Regression Problem
A Siple Regression Proble R. M. Castro March 23, 2 In this brief note a siple regression proble will be introduced, illustrating clearly the bias-variance tradeoff. Let Y i f(x i ) + W i, i,..., n, where
More informationMachine Learning Basics: Estimators, Bias and Variance
Machine Learning Basics: Estiators, Bias and Variance Sargur N. srihari@cedar.buffalo.edu This is part of lecture slides on Deep Learning: http://www.cedar.buffalo.edu/~srihari/cse676 1 Topics in Basics
More informationTEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES
TEST OF HOMOGENEITY OF PARALLEL SAMPLES FROM LOGNORMAL POPULATIONS WITH UNEQUAL VARIANCES S. E. Ahed, R. J. Tokins and A. I. Volodin Departent of Matheatics and Statistics University of Regina Regina,
More informationNon-Parametric Non-Line-of-Sight Identification 1
Non-Paraetric Non-Line-of-Sight Identification Sinan Gezici, Hisashi Kobayashi and H. Vincent Poor Departent of Electrical Engineering School of Engineering and Applied Science Princeton University, Princeton,
More informationare equal to zero, where, q = p 1. For each gene j, the pairwise null and alternative hypotheses are,
Page of 8 Suppleentary Materials: A ultiple testing procedure for ulti-diensional pairwise coparisons with application to gene expression studies Anjana Grandhi, Wenge Guo, Shyaal D. Peddada S Notations
More informationTesting equality of variances for multiple univariate normal populations
University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Inforation Sciences 0 esting equality of variances for ultiple univariate
More informationAN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME
J. Japan Statist. Soc. Vol. 35 No. 005 73 86 AN EFFICIENT CLASS OF CHAIN ESTIMATORS OF POPULATION VARIANCE UNDER SUB-SAMPLING SCHEME H. S. Jhajj*, M. K. Shara* and Lovleen Kuar Grover** For estiating the
More informationBest Linear Unbiased and Invariant Reconstructors for the Past Records
BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY http:/athusy/bulletin Bull Malays Math Sci Soc (2) 37(4) (2014), 1017 1028 Best Linear Unbiased and Invariant Reconstructors for the Past Records
More informationDEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS
ISSN 1440-771X AUSTRALIA DEPARTMENT OF ECONOMETRICS AND BUSINESS STATISTICS An Iproved Method for Bandwidth Selection When Estiating ROC Curves Peter G Hall and Rob J Hyndan Working Paper 11/00 An iproved
More informationBiostatistics Department Technical Report
Biostatistics Departent Technical Report BST006-00 Estiation of Prevalence by Pool Screening With Equal Sized Pools and a egative Binoial Sapling Model Charles R. Katholi, Ph.D. Eeritus Professor Departent
More informationTail Estimation of the Spectral Density under Fixed-Domain Asymptotics
Tail Estiation of the Spectral Density under Fixed-Doain Asyptotics Wei-Ying Wu, Chae Young Li and Yiin Xiao Wei-Ying Wu, Departent of Statistics & Probability Michigan State University, East Lansing,
More informationProbability Distributions
Probability Distributions In Chapter, we ephasized the central role played by probability theory in the solution of pattern recognition probles. We turn now to an exploration of soe particular exaples
More informationEstimation of the Population Mean Based on Extremes Ranked Set Sampling
Aerican Journal of Matheatics Statistics 05, 5(: 3-3 DOI: 0.593/j.ajs.05050.05 Estiation of the Population Mean Based on Extrees Ranked Set Sapling B. S. Biradar,*, Santosha C. D. Departent of Studies
More informationCS Lecture 13. More Maximum Likelihood
CS 6347 Lecture 13 More Maxiu Likelihood Recap Last tie: Introduction to axiu likelihood estiation MLE for Bayesian networks Optial CPTs correspond to epirical counts Today: MLE for CRFs 2 Maxiu Likelihood
More informationSoft Computing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis
Soft Coputing Techniques Help Assign Weights to Different Factors in Vulnerability Analysis Beverly Rivera 1,2, Irbis Gallegos 1, and Vladik Kreinovich 2 1 Regional Cyber and Energy Security Center RCES
More informatione-companion ONLY AVAILABLE IN ELECTRONIC FORM
OPERATIONS RESEARCH doi 10.1287/opre.1070.0427ec pp. ec1 ec5 e-copanion ONLY AVAILABLE IN ELECTRONIC FORM infors 07 INFORMS Electronic Copanion A Learning Approach for Interactive Marketing to a Custoer
More informationInteractive Markov Models of Evolutionary Algorithms
Cleveland State University EngagedScholarship@CSU Electrical Engineering & Coputer Science Faculty Publications Electrical Engineering & Coputer Science Departent 2015 Interactive Markov Models of Evolutionary
More informationEstimation of the Mean of the Exponential Distribution Using Maximum Ranked Set Sampling with Unequal Samples
Open Journal of Statistics, 4, 4, 64-649 Published Online Septeber 4 in SciRes http//wwwscirporg/ournal/os http//ddoiorg/436/os4486 Estiation of the Mean of the Eponential Distribution Using Maiu Ranked
More informationSharp Time Data Tradeoffs for Linear Inverse Problems
Sharp Tie Data Tradeoffs for Linear Inverse Probles Saet Oyak Benjain Recht Mahdi Soltanolkotabi January 016 Abstract In this paper we characterize sharp tie-data tradeoffs for optiization probles used
More informationThe proofs of Theorem 1-3 are along the lines of Wied and Galeano (2013).
A Appendix: Proofs The proofs of Theore 1-3 are along the lines of Wied and Galeano (2013) Proof of Theore 1 Let D[d 1, d 2 ] be the space of càdlàg functions on the interval [d 1, d 2 ] equipped with
More informationarxiv: v1 [cs.ds] 3 Feb 2014
arxiv:40.043v [cs.ds] 3 Feb 04 A Bound on the Expected Optiality of Rando Feasible Solutions to Cobinatorial Optiization Probles Evan A. Sultani The Johns Hopins University APL evan@sultani.co http://www.sultani.co/
More informationGEE ESTIMATORS IN MIXTURE MODEL WITH VARYING CONCENTRATIONS
ACTA UIVERSITATIS LODZIESIS FOLIA OECOOMICA 3(3142015 http://dx.doi.org/10.18778/0208-6018.314.03 Olesii Doronin *, Rostislav Maiboroda ** GEE ESTIMATORS I MIXTURE MODEL WITH VARYIG COCETRATIOS Abstract.
More informationA Note on the Applied Use of MDL Approximations
A Note on the Applied Use of MDL Approxiations Daniel J. Navarro Departent of Psychology Ohio State University Abstract An applied proble is discussed in which two nested psychological odels of retention
More informationE0 370 Statistical Learning Theory Lecture 6 (Aug 30, 2011) Margin Analysis
E0 370 tatistical Learning Theory Lecture 6 (Aug 30, 20) Margin Analysis Lecturer: hivani Agarwal cribe: Narasihan R Introduction In the last few lectures we have seen how to obtain high confidence bounds
More informationEstimating Parameters for a Gaussian pdf
Pattern Recognition and achine Learning Jaes L. Crowley ENSIAG 3 IS First Seester 00/0 Lesson 5 7 Noveber 00 Contents Estiating Paraeters for a Gaussian pdf Notation... The Pattern Recognition Proble...3
More informationBlock designs and statistics
Bloc designs and statistics Notes for Math 447 May 3, 2011 The ain paraeters of a bloc design are nuber of varieties v, bloc size, nuber of blocs b. A design is built on a set of v eleents. Each eleent
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optiization and Approxiation Instructor: Moritz Hardt Eail: hardt+ee227c@berkeley.edu Graduate Instructor: Max Sichowitz Eail: sichow+ee227c@berkeley.edu October
More informationStatistics and Probability Letters
Statistics and Probability Letters 79 2009 223 233 Contents lists available at ScienceDirect Statistics and Probability Letters journal hoepage: www.elsevier.co/locate/stapro A CLT for a one-diensional
More informationBayes Decision Rule and Naïve Bayes Classifier
Bayes Decision Rule and Naïve Bayes Classifier Le Song Machine Learning I CSE 6740, Fall 2013 Gaussian Mixture odel A density odel p(x) ay be ulti-odal: odel it as a ixture of uni-odal distributions (e.g.
More informationProc. of the IEEE/OES Seventh Working Conference on Current Measurement Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES
Proc. of the IEEE/OES Seventh Working Conference on Current Measureent Technology UNCERTAINTIES IN SEASONDE CURRENT VELOCITIES Belinda Lipa Codar Ocean Sensors 15 La Sandra Way, Portola Valley, CA 98 blipa@pogo.co
More informationThis model assumes that the probability of a gap has size i is proportional to 1/i. i.e., i log m e. j=1. E[gap size] = i P r(i) = N f t.
CS 493: Algoriths for Massive Data Sets Feb 2, 2002 Local Models, Bloo Filter Scribe: Qin Lv Local Models In global odels, every inverted file entry is copressed with the sae odel. This work wells when
More informationA Poisson process reparameterisation for Bayesian inference for extremes
A Poisson process reparaeterisation for Bayesian inference for extrees Paul Sharkey and Jonathan A. Tawn Eail: p.sharkey1@lancs.ac.uk, j.tawn@lancs.ac.uk STOR-i Centre for Doctoral Training, Departent
More informationExtension of CSRSM for the Parametric Study of the Face Stability of Pressurized Tunnels
Extension of CSRSM for the Paraetric Study of the Face Stability of Pressurized Tunnels Guilhe Mollon 1, Daniel Dias 2, and Abdul-Haid Soubra 3, M.ASCE 1 LGCIE, INSA Lyon, Université de Lyon, Doaine scientifique
More informationCOS 424: Interacting with Data. Written Exercises
COS 424: Interacting with Data Hoework #4 Spring 2007 Regression Due: Wednesday, April 18 Written Exercises See the course website for iportant inforation about collaboration and late policies, as well
More informationOn Conditions for Linearity of Optimal Estimation
On Conditions for Linearity of Optial Estiation Erah Akyol, Kuar Viswanatha and Kenneth Rose {eakyol, kuar, rose}@ece.ucsb.edu Departent of Electrical and Coputer Engineering University of California at
More informationSupport recovery in compressed sensing: An estimation theoretic approach
Support recovery in copressed sensing: An estiation theoretic approach Ain Karbasi, Ali Horati, Soheil Mohajer, Martin Vetterli School of Coputer and Counication Sciences École Polytechnique Fédérale de
More informationFairness via priority scheduling
Fairness via priority scheduling Veeraruna Kavitha, N Heachandra and Debayan Das IEOR, IIT Bobay, Mubai, 400076, India vavitha,nh,debayan}@iitbacin Abstract In the context of ulti-agent resource allocation
More informationResearch Article On the Isolated Vertices and Connectivity in Random Intersection Graphs
International Cobinatorics Volue 2011, Article ID 872703, 9 pages doi:10.1155/2011/872703 Research Article On the Isolated Vertices and Connectivity in Rando Intersection Graphs Yilun Shang Institute for
More informationOBJECTIVES INTRODUCTION
M7 Chapter 3 Section 1 OBJECTIVES Suarize data using easures of central tendency, such as the ean, edian, ode, and idrange. Describe data using the easures of variation, such as the range, variance, and
More informationESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS. A Thesis. Presented to. The Faculty of the Department of Mathematics
ESTIMATING AND FORMING CONFIDENCE INTERVALS FOR EXTREMA OF RANDOM POLYNOMIALS A Thesis Presented to The Faculty of the Departent of Matheatics San Jose State University In Partial Fulfillent of the Requireents
More informationStochastic Subgradient Methods
Stochastic Subgradient Methods Lingjie Weng Yutian Chen Bren School of Inforation and Coputer Science University of California, Irvine {wengl, yutianc}@ics.uci.edu Abstract Stochastic subgradient ethods
More informationSome Proofs: This section provides proofs of some theoretical results in section 3.
Testing Jups via False Discovery Rate Control Yu-Min Yen. Institute of Econoics, Acadeia Sinica, Taipei, Taiwan. E-ail: YMYEN@econ.sinica.edu.tw. SUPPLEMENTARY MATERIALS Suppleentary Materials contain
More informationKernel-Based Nonparametric Anomaly Detection
Kernel-Based Nonparaetric Anoaly Detection Shaofeng Zou Dept of EECS Syracuse University Eail: szou@syr.edu Yingbin Liang Dept of EECS Syracuse University Eail: yliang6@syr.edu H. Vincent Poor Dept of
More informationLecture October 23. Scribes: Ruixin Qiang and Alana Shine
CSCI699: Topics in Learning and Gae Theory Lecture October 23 Lecturer: Ilias Scribes: Ruixin Qiang and Alana Shine Today s topic is auction with saples. 1 Introduction to auctions Definition 1. In a single
More informationTesting the lag length of vector autoregressive models: A power comparison between portmanteau and Lagrange multiplier tests
Working Papers 2017-03 Testing the lag length of vector autoregressive odels: A power coparison between portanteau and Lagrange ultiplier tests Raja Ben Hajria National Engineering School, University of
More informationarxiv: v1 [stat.ot] 7 Jul 2010
Hotelling s test for highly correlated data P. Bubeliny e-ail: bubeliny@karlin.ff.cuni.cz Charles University, Faculty of Matheatics and Physics, KPMS, Sokolovska 83, Prague, Czech Republic, 8675. arxiv:007.094v
More informationBootstrapping Dependent Data
Bootstrapping Dependent Data One of the key issues confronting bootstrap resapling approxiations is how to deal with dependent data. Consider a sequence fx t g n t= of dependent rando variables. Clearly
More informationInternational Journal of Pure and Applied Mathematics Volume 37 No , IMPROVED DATA DRIVEN CONTROL CHARTS
International Journal of Pure and Applied Matheatics Volue 37 No. 3 2007, 423-438 IMPROVED DATA DRIVEN CONTROL CHARTS Wille Albers 1, Wilbert C.M. Kallenberg 2 1,2 Departent of Applied Matheatics Faculty
More informationMoments of the product and ratio of two correlated chi-square variables
Stat Papers 009 50:581 59 DOI 10.1007/s0036-007-0105-0 REGULAR ARTICLE Moents of the product and ratio of two correlated chi-square variables Anwar H. Joarder Received: June 006 / Revised: 8 October 007
More informationASSUME a source over an alphabet size m, from which a sequence of n independent samples are drawn. The classical
IEEE TRANSACTIONS ON INFORMATION THEORY Large Alphabet Source Coding using Independent Coponent Analysis Aichai Painsky, Meber, IEEE, Saharon Rosset and Meir Feder, Fellow, IEEE arxiv:67.7v [cs.it] Jul
More informationGeneralized Augmentation for Control of the k-familywise Error Rate
International Journal of Statistics in Medical Research, 2012, 1, 113-119 113 Generalized Augentation for Control of the k-failywise Error Rate Alessio Farcoeni* Departent of Public Health and Infectious
More informationResearch in Area of Longevity of Sylphon Scraies
IOP Conference Series: Earth and Environental Science PAPER OPEN ACCESS Research in Area of Longevity of Sylphon Scraies To cite this article: Natalia Y Golovina and Svetlana Y Krivosheeva 2018 IOP Conf.
More informationIn this chapter, we consider several graph-theoretic and probabilistic models
THREE ONE GRAPH-THEORETIC AND STATISTICAL MODELS 3.1 INTRODUCTION In this chapter, we consider several graph-theoretic and probabilistic odels for a social network, which we do under different assuptions
More informationRandomized Recovery for Boolean Compressed Sensing
Randoized Recovery for Boolean Copressed Sensing Mitra Fatei and Martin Vetterli Laboratory of Audiovisual Counication École Polytechnique Fédéral de Lausanne (EPFL) Eail: {itra.fatei, artin.vetterli}@epfl.ch
More informationFDR- and FWE-controlling methods using data-driven weights
FDR- and FWE-controlling ethods using data-driven weights LIVIO FINOS Center for Modelling Coputing and Statistics, University of Ferrara via N.Machiavelli 35, 44 FERRARA - Italy livio.finos@unife.it LUIGI
More informationRecovering Data from Underdetermined Quadratic Measurements (CS 229a Project: Final Writeup)
Recovering Data fro Underdeterined Quadratic Measureents (CS 229a Project: Final Writeup) Mahdi Soltanolkotabi Deceber 16, 2011 1 Introduction Data that arises fro engineering applications often contains
More informationMonte Carlo simulation is widely used to measure the credit risk in portfolios of loans, corporate bonds, and
MANAGMNT SCINC Vol. 5, No., Noveber 005, pp. 643 656 issn 005-909 eissn 56-550 05 5 643 infors doi 0.87/nsc.050.045 005 INFORMS Iportance Sapling for Portfolio Credit Risk Paul Glasseran, Jingyi Li Colubia
More informationNonmonotonic Networks. a. IRST, I Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I Povo (Trento) Italy
Storage Capacity and Dynaics of Nononotonic Networks Bruno Crespi a and Ignazio Lazzizzera b a. IRST, I-38050 Povo (Trento) Italy, b. Univ. of Trento, Physics Dept., I-38050 Povo (Trento) Italy INFN Gruppo
More informationBayes Estimation of the Logistic Distribution Parameters Based on Progressive Sampling
Appl. Math. Inf. Sci. 0, No. 6, 93-30 06) 93 Applied Matheatics & Inforation Sciences An International Journal http://dx.doi.org/0.8576/ais/0063 Bayes Estiation of the Logistic Distribution Paraeters Based
More informationFixed-to-Variable Length Distribution Matching
Fixed-to-Variable Length Distribution Matching Rana Ali Ajad and Georg Böcherer Institute for Counications Engineering Technische Universität München, Gerany Eail: raa2463@gail.co,georg.boecherer@tu.de
More informationNonlinear Log-Periodogram Regression for Perturbed Fractional Processes
Nonlinear Log-Periodogra Regression for Perturbed Fractional Processes Yixiao Sun Departent of Econoics Yale University Peter C. B. Phillips Cowles Foundation for Research in Econoics Yale University First
More informationNecessity of low effective dimension
Necessity of low effective diension Art B. Owen Stanford University October 2002, Orig: July 2002 Abstract Practitioners have long noticed that quasi-monte Carlo ethods work very well on functions that
More informationC na (1) a=l. c = CO + Clm + CZ TWO-STAGE SAMPLE DESIGN WITH SMALL CLUSTERS. 1. Introduction
TWO-STGE SMPLE DESIGN WITH SMLL CLUSTERS Robert G. Clark and David G. Steel School of Matheatics and pplied Statistics, University of Wollongong, NSW 5 ustralia. (robert.clark@abs.gov.au) Key Words: saple
More informationPseudo-marginal Metropolis-Hastings: a simple explanation and (partial) review of theory
Pseudo-arginal Metropolis-Hastings: a siple explanation and (partial) review of theory Chris Sherlock Motivation Iagine a stochastic process V which arises fro soe distribution with density p(v θ ). Iagine
More informationUsing EM To Estimate A Probablity Density With A Mixture Of Gaussians
Using EM To Estiate A Probablity Density With A Mixture Of Gaussians Aaron A. D Souza adsouza@usc.edu Introduction The proble we are trying to address in this note is siple. Given a set of data points
More informationThe Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Parameters
journal of ultivariate analysis 58, 96106 (1996) article no. 0041 The Distribution of the Covariance Matrix for a Subset of Elliptical Distributions with Extension to Two Kurtosis Paraeters H. S. Steyn
More informationSimultaneous critical values for t-tests in very high dimensions
Bernoulli 17(1, 2011, 347 394 DOI: 10.3150/10-BEJ272 Siultaneous critical values for t-tests in very high diensions HONGYUAN CAO 1 and MICHAEL R. KOSOROK 2 1 Departent of Health Studies, 5841 South Maryland
More informationPattern Recognition and Machine Learning. Learning and Evaluation for Pattern Recognition
Pattern Recognition and Machine Learning Jaes L. Crowley ENSIMAG 3 - MMIS Fall Seester 2017 Lesson 1 4 October 2017 Outline Learning and Evaluation for Pattern Recognition Notation...2 1. The Pattern Recognition
More informationLower Bounds for Quantized Matrix Completion
Lower Bounds for Quantized Matrix Copletion Mary Wootters and Yaniv Plan Departent of Matheatics University of Michigan Ann Arbor, MI Eail: wootters, yplan}@uich.edu Mark A. Davenport School of Elec. &
More informationEffective joint probabilistic data association using maximum a posteriori estimates of target states
Effective joint probabilistic data association using axiu a posteriori estiates of target states 1 Viji Paul Panakkal, 2 Rajbabu Velurugan 1 Central Research Laboratory, Bharat Electronics Ltd., Bangalore,
More informationFeature Extraction Techniques
Feature Extraction Techniques Unsupervised Learning II Feature Extraction Unsupervised ethods can also be used to find features which can be useful for categorization. There are unsupervised ethods that
More informationIN modern society that various systems have become more
Developent of Reliability Function in -Coponent Standby Redundant Syste with Priority Based on Maxiu Entropy Principle Ryosuke Hirata, Ikuo Arizono, Ryosuke Toohiro, Satoshi Oigawa, and Yasuhiko Takeoto
More informationBoosting with log-loss
Boosting with log-loss Marco Cusuano-Towner Septeber 2, 202 The proble Suppose we have data exaples {x i, y i ) i =... } for a two-class proble with y i {, }. Let F x) be the predictor function with the
More informationIntelligent Systems: Reasoning and Recognition. Perceptrons and Support Vector Machines
Intelligent Systes: Reasoning and Recognition Jaes L. Crowley osig 1 Winter Seester 2018 Lesson 6 27 February 2018 Outline Perceptrons and Support Vector achines Notation...2 Linear odels...3 Lines, Planes
More informationLost-Sales Problems with Stochastic Lead Times: Convexity Results for Base-Stock Policies
OPERATIONS RESEARCH Vol. 52, No. 5, Septeber October 2004, pp. 795 803 issn 0030-364X eissn 1526-5463 04 5205 0795 infors doi 10.1287/opre.1040.0130 2004 INFORMS TECHNICAL NOTE Lost-Sales Probles with
More informationA note on the multiplication of sparse matrices
Cent. Eur. J. Cop. Sci. 41) 2014 1-11 DOI: 10.2478/s13537-014-0201-x Central European Journal of Coputer Science A note on the ultiplication of sparse atrices Research Article Keivan Borna 12, Sohrab Aboozarkhani
More informationOptimal nonlinear Bayesian experimental design: an application to amplitude versus offset experiments
Geophys. J. Int. (23) 155, 411 421 Optial nonlinear Bayesian experiental design: an application to aplitude versus offset experients Jojanneke van den Berg, 1, Andrew Curtis 2,3 and Jeannot Trapert 1 1
More informationRobustness and Regularization of Support Vector Machines
Robustness and Regularization of Support Vector Machines Huan Xu ECE, McGill University Montreal, QC, Canada xuhuan@ci.cgill.ca Constantine Caraanis ECE, The University of Texas at Austin Austin, TX, USA
More informationThis article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and
This article appeared in a ournal published by Elsevier. The attached copy is furnished to the author for internal non-coercial research and education use, including for instruction at the authors institution
More information3.8 Three Types of Convergence
3.8 Three Types of Convergence 3.8 Three Types of Convergence 93 Suppose that we are given a sequence functions {f k } k N on a set X and another function f on X. What does it ean for f k to converge to
More informationMeta-Analytic Interval Estimation for Bivariate Correlations
Psychological Methods 2008, Vol. 13, No. 3, 173 181 Copyright 2008 by the Aerican Psychological Association 1082-989X/08/$12.00 DOI: 10.1037/a0012868 Meta-Analytic Interval Estiation for Bivariate Correlations
More informationLARGE DEVIATIONS AND RARE EVENT SIMULATION FOR PORTFOLIO CREDIT RISK
LARGE DEVIATIONS AND RARE EVENT SIMULATION FOR PORTFOLIO CREDIT RISK by Hasitha de Silva A Dissertation Subitted to the Graduate Faculty of George Mason University In Partial fulfillent of The Requireents
More informationAsymptotically Unbiased Estimation of the Coefficient of Tail Dependence
Asyptotically Unbiased Estiation of the Coefficient of Tail Dependence Yuri Goegebeur, Arelle Guillou To cite this version: Yuri Goegebeur, Arelle Guillou. Asyptotically Unbiased Estiation of the Coefficient
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 2: PAC Learning and VC Theory I Fro Adversarial Online to Statistical Three reasons to ove fro worst-case deterinistic
More informationBayesian Approach for Fatigue Life Prediction from Field Inspection
Bayesian Approach for Fatigue Life Prediction fro Field Inspection Dawn An and Jooho Choi School of Aerospace & Mechanical Engineering, Korea Aerospace University, Goyang, Seoul, Korea Srira Pattabhiraan
More informationExperimental Design For Model Discrimination And Precise Parameter Estimation In WDS Analysis
City University of New York (CUNY) CUNY Acadeic Works International Conference on Hydroinforatics 8-1-2014 Experiental Design For Model Discriination And Precise Paraeter Estiation In WDS Analysis Giovanna
More informationOptimal Jackknife for Discrete Time and Continuous Time Unit Root Models
Optial Jackknife for Discrete Tie and Continuous Tie Unit Root Models Ye Chen and Jun Yu Singapore Manageent University January 6, Abstract Maxiu likelihood estiation of the persistence paraeter in the
More informationOn the block maxima method in extreme value theory
On the bloc axiethod in extree value theory Ana Ferreira ISA, Univ Tecn Lisboa, Tapada da Ajuda 349-7 Lisboa, Portugal CEAUL, FCUL, Bloco C6 - Piso 4 Capo Grande, 749-6 Lisboa, Portugal anafh@isa.utl.pt
More informationComputational and Statistical Learning Theory
Coputational and Statistical Learning Theory Proble sets 5 and 6 Due: Noveber th Please send your solutions to learning-subissions@ttic.edu Notations/Definitions Recall the definition of saple based Radeacher
More informationConstrained Consensus and Optimization in Multi-Agent Networks arxiv: v2 [math.oc] 17 Dec 2008
LIDS Report 2779 1 Constrained Consensus and Optiization in Multi-Agent Networks arxiv:0802.3922v2 [ath.oc] 17 Dec 2008 Angelia Nedić, Asuan Ozdaglar, and Pablo A. Parrilo February 15, 2013 Abstract We
More informationBounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks
Bounds on the Miniax Rate for Estiating a Prior over a VC Class fro Independent Learning Tasks Liu Yang Steve Hanneke Jaie Carbonell Deceber 01 CMU-ML-1-11 School of Coputer Science Carnegie Mellon University
More informationExamining the Effects of Site Selection Criteria for Evaluating the Effectiveness of Traffic Safety Countermeasures
Exaining the Effects of Site Selection Criteria for Evaluating the Effectiveness of Traffic Safety Countereasures Doinique Lord* Associate Professor and Zachry Developent Professor I Zachry Departent of
More informationSupplement to: Subsampling Methods for Persistent Homology
Suppleent to: Subsapling Methods for Persistent Hoology A. Technical results In this section, we present soe technical results that will be used to prove the ain theores. First, we expand the notation
More informationBest Procedures For Sample-Free Item Analysis
Best Procedures For Saple-Free Ite Analysis Benjain D. Wright University of Chicago Graha A. Douglas University of Western Australia Wright s (1969) widely used "unconditional" procedure for Rasch saple-free
More informationEnsemble Based on Data Envelopment Analysis
Enseble Based on Data Envelopent Analysis So Young Sohn & Hong Choi Departent of Coputer Science & Industrial Systes Engineering, Yonsei University, Seoul, Korea Tel) 82-2-223-404, Fax) 82-2- 364-7807
More informationAn Introduction to Meta-Analysis
An Introduction to Meta-Analysis Douglas G. Bonett University of California, Santa Cruz How to cite this work: Bonett, D.G. (2016) An Introduction to Meta-analysis. Retrieved fro http://people.ucsc.edu/~dgbonett/eta.htl
More informationIdentical Maximum Likelihood State Estimation Based on Incremental Finite Mixture Model in PHD Filter
Identical Maxiu Lielihood State Estiation Based on Increental Finite Mixture Model in PHD Filter Gang Wu Eail: xjtuwugang@gail.co Jing Liu Eail: elelj20080730@ail.xjtu.edu.cn Chongzhao Han Eail: czhan@ail.xjtu.edu.cn
More informationGrafting: Fast, Incremental Feature Selection by Gradient Descent in Function Space
Journal of Machine Learning Research 3 (2003) 1333-1356 Subitted 5/02; Published 3/03 Grafting: Fast, Increental Feature Selection by Gradient Descent in Function Space Sion Perkins Space and Reote Sensing
More informationThe degree of a typical vertex in generalized random intersection graph models
Discrete Matheatics 306 006 15 165 www.elsevier.co/locate/disc The degree of a typical vertex in generalized rando intersection graph odels Jerzy Jaworski a, Michał Karoński a, Dudley Stark b a Departent
More information