Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities Peter M. Aronow and Cyrus Samii Forthcoming at Survey Methodology Abstract We consider conservative variance estimation for the Horvitz-Thompson estimator of a population total in sampling designs with zero pairwise inclusion probabilities, known as non-measurable designs. We decompose the standard Horvitz-Thompson variance estimator under such designs and characterize the bias precisely. We develop a bias correction that is guaranteed to be weakly conservative (non-negatively biased) regardless of the nature of the non-measurability. The analysis sheds light on conditions under which the standard Horvitz-Thompson variance estimator performs well despite non-measurability and the conservative bias correction may outperform commonly-used approximations. Keywords: Horvitz-Thompson estimation, non-measurable designs, variance estimation 1 Introduction Sampling designs sometimes result in pairs of units having zero probability of being jointly included in the sample. Horvitz and Thompson (1952) s statement of the properties of the finite population total makes clear that general, unbiased variance estimation for estimators of population totals is impossible for such non-measurable designs (Särndal et al., 1992, 33). Optimal methods for variance estimation in these cases remains an open problem to this day. This paper analyzes the nature of the biases that non-measurability introduces for the standard Horvitz-Thompson estimator and studies an approach to correct for this bias in a manner that is guaranteed to be conservative. While our results cannot offer a solution to the non-measurability problem for all practical applications, we Peter M. Aronow, Department of Political Science, Yale University, 77 Prospect St., New Haven, CT 652, Email: peter.aronow@yale.edu; Cyrus Samii, Department of Politics, New York University, 19 West 4th St., New York, NY 112, Email: cds283@nyu.edu 1
do clarify conditions under which the standard estimator performs well and where the conservative bias correction outperforms commonly-used approximations. Despite their theoretical drawbacks, sampling designs with zero pairwise inclusion probabilities are quite common. A common example of a non-measurable design is one that draws only single units or clusters from a set of strata. This may occur if the population or a subpopulation of interest is incidentally sparse over stratification cells. Another instance of a non-measurable design is a systematic sample in which unit indices are sampled from a list in multiples from a random starting value. In these designs, units whose indices are multiples from different starting values have zero joint probability of inclusion. Approximate methods have been proposed for special cases, as discussed in Hansen et al. (1953, Section 9.15), Särndal et al. (1992, Chapter 3), and Wolter (27, Chapters 2 and 8). In the single-unit per stratum case, a common approach is to collapse strata and assume units were drawn via a simple random sample from the larger, collapsed stratum. For systematic samples, the standard approach is to use an approximation based on an assumption of simple random sampling with replacement. These approximate methods are generally biased to a degree that cannot be determined from the data. In some cases, it can be shown that the bias will tend to be positive, but such is not the case generally, and especially so when the zero pairwise inclusion probabilities occur in a haphazard manner. This paper begins by decomposing the bias of the Horvitz-Thompson variance estimator under non-measurability. This exposes precisely how conditions on the underlying data result in more or less bias. We also show how a simple application of Young s inequality yields a bias correction and a class of estimators guaranteed to have weakly positive bias as well as no bias under special conditions. We discuss implications for applied work. 2 Variance estimation for the Horvitz-Thompson estimator Consider a population U indexed by 1,..., k,..., N and a sampling design such that the probability of inclusion in the sample for unit k is given by π k, and the joint inclusion probability for units k and l is given by π kl. Under a measurable design, two conditions obtain: (1) π k > and π k is known for all k U and (2) π kl > and π kl is known for all k, l U. Non-measurable designs include those for which either of the two conditions for a measurable design do not hold. Failure to meet the former condition precludes unbiased estimation of totals. The Horvitz-Thompson estimator of a population total is given by ˆt = k s y k = y k I k, π k π k 2
where I k {, 1} is unit k s inclusion indicator, the only stochastic component of the expression, with E (I k ) = π k, the inclusion probability, and s and U refer to the sample and the population, respectively. Define E (I k I l ) = π kl, which is the probability that both units k and l from U are included in the sample. Since I k I k = I k, E (I k I k ) = π kk = π k by construction. When condition (1) holds, as we assume throughout, the Horvitz-Thompson estimator is unbiased. 2.1 Properties of the Horvitz-Thompson variance estimator under measurability By Horvitz and Thompson (1952), the variance of the Horvitz-Thompson estimator for the total, Var (ˆt) = Cov (I k, I l ) y k y l π k π l l U = ( ) 2 yk Var (I k ) + Cov (I k, I l ) y k y l. π k π l π k l U\k We label a sample from a measurable design, s M, and an unbiased estimator for Var (ˆt) on s M is given by, Var (ˆt) = Cov (I k, I l ) y k y l = Cov (I k, I l ) I k I l π kl π k π l π kl k s M l s M l U y k y l, π k π l where the only stochastic part of the latter expression is I k I l, and unbiasedness is due to E (I k I l ) = π kl. 2.2 Properties of the Horvitz-Thompson variance estimator under non-measurability We now examine the case where condition 2 does not hold: π kl = for some units k, l U. Because I k is a Bernoulli random variable with probability π k, Cov (I k, I l ) = π kl π k π l for k l, and Cov (I k, I k ) = Var (I k ) = π k (1 π k ). Then, we can re-express the variance above as, Var (ˆt) = ( ) 2 yk π k (1 π k ) + (π kl π k π l ) y k y l π k = π k (1 π k ) π k ( yk π k ) 2 + l U\k l {U\k:π kl >} 3 π l (π kl π k π l ) y k y l π k π l y k y l. } {{ } A
For k and l such that π kl =, the sampling design will never provide information on the component of the variance labeled as A above, since we will never observe y k and y l together. We label a sample from a design where condition 2 fails as s. When Var (ˆt) is applied to s, the result is unbiased for Var (ˆt) + A. We state this formally as follows: Proposition 1. When s refers to a sample from a design with some π kl =, we have, E [ Var (ˆt)] = Var (ˆt) + y k y l = Var (ˆt) + A. Proof. The result follows from, [ ] Cov (I k, I l ) y k y l E = E π kl π k π l k s l s l {U:π kl >} I k I l Cov (I k, I l ) π kl y k y l π k π l = Var (I k ) = Var (ˆt) + = Var (ˆt) + A. ( yk π k ) 2 + l {U\k:π kl >} y k y l Cov (I k, I l ) y k π k y l π l The standard Horvitz-Thompson variance estimator, if applied to designs with zero pairwise inclusion probabilities, can therefore have a positive or a negative bias. If the y k, y l values are always nonnegative (or always nonpositive), then the bias is always nonnegative. When values may be positive or negative, A is the sum of cross-products of outcomes that never appear together under the design sample. If the jointly exclusive outcomes are centered over zero, then no correlation in these outcomes would tend to result in small bias, positive correlation in positive bias, and negative correlation in negative bias. 3 Conservative bias correction for the Horvitz- Thompson estimator under non-measurability The case where A may be less than zero for a non-measurable design suggests the need for some adjustment that will guarantee a bias that is weakly bounded below by zero. We first develop a general bias correction that is guaranteed to be conservative, later providing a special simplified case for practical usage. 4
3.1 General formulation Consider the following variance estimator: Var C (ˆt) = l {U:π kl >} I k I l Cov (I k, I l ) π kl y k y l + π k π l ( I k y k a kl a kl π k ) y l bkl + I l, b kl π l 1 where a kl, b kl are positive real numbers such that a kl + 1 b kl = 1 for all pairs k, l with π kl =. The estimator is guaranteed to produce an expected value greater than or equal to the true variance for all designs, and is thus conservative. We state this condition of the estimator formally: Proposition 2. E [ Var C (ˆt)] Var (ˆt). Proof. By Young s inequality, y k a kl a kl + y l bkl b kl y k y l, if 1 a kl + 1 b kl A = and Therefore = 1. Define A such that, A y k a kl + y l bkl a kl b kl y k y l Var (ˆt) + A + A Var (ˆt). y k y l The associated Horvitz-Thompson estimator of A would be  = ( ) y k a kl y l bkl I k + I l, a kl π k b kl π l which is unbiased [ ] by E (I k ) = π k and E (I l ) = π l. Since E = A, by Proposition 1, y k y l = A. y k y l = A [ ] Cov (I k, I l ) y k y l E + π kl π k π  = Var (ˆt) + A + A l k s l s E [ Var C (ˆt)] Var (ˆt). (1) 5
Substituting terms, E l {U:π kl >} I k I l Cov (I k, I l ) π kl y k y l + π k π l ( I k y k a kl a kl π k ) y l bkl + I l Var (ˆt). b kl π l Var C (ˆt) is justified as a conservative estimator for the case when A is not known to be positive. This estimator is unbiased under a special condition: Corollary 1. If, for all pairs k, l such that π kl =, (i) y k a kl = y l b kl and (ii) yk y l = y k y l, E [ Var C (ˆt)] = Var (ˆt). Proof. By (i), (ii) and Young s inequality, Therefore, A = and It follows that y k a kl a kl y k a kl + y l bkl a kl b kl + y l bkl b kl = y k y l = y k y l. = Var (ˆt) + A + A = Var (ˆt) E [ Var C (ˆt)] = Var (ˆt). y k y l = y k y l = A If any units k, l are in clusters (i.e., Pr (I k I l ) = ), these units should be totaled into one larger unit before estimation. Combining units will, in general, reduce the bias of the variance estimator because only pairs of cluster-level totals will be included in A, as opposed to all constituent pairs. 3.2 Simplified special case In general, it would be difficult to assign optimal values of a kl and b kl for all pairs k, l such that π kl =. Instead, we examine one intuitive case, assigning all a kl = b kl = 2: Var C2 (ˆt) = Cov (I k, I l ) y k y l I k I l + ( ) yk 2 yl 2 I k + I l. π kl π k π l 2π k 2π l l {U:π kl >} As a special case of Var C (ˆt), Var C2 (ˆt) is also conservative: 6
Corollary 2. E [ Var C2 (ˆt)] Var (ˆt). 1 Proof. For all pairs k, l such that π kl =, holds. a kl + 1 b kl = 1 2 + 1 2 = 1. Proposition 1 therefore The choice to set all a kl = b kl = 2 is justified by the fact that it will yield the lowest value of the estimator Var C (ˆt) subject to the constraint that a kl and b kl are fixed as constants a and b over all k, l. Corollary 3. Among the class of estimators Var Cab (ˆt), defined as the set of estimators Var C (ˆt) such that all a kl = a and all b kl = b, Var ] C2 (ˆt) = min a,b [ Var Cab (ˆt). Proof. By simple algebra, Var Cab (ˆt) = Cov (I k, I l ) y k y l I k I l + y k (I a k π kl π k π l aπ k l {U:π kl >} = Var (ˆt) + ) y k (I a y l b k + I l aπ k bπ l = Var (ˆt) + 1 ) y k (I a y l b y k b y l a k + I l + I k + I l 2 aπ k bπ l bπ k aπ l = Var (ˆt) + 1 ( ( Ik yk a 2 π k a + y ) k b + I ( l yl a b π l a + y l b b ) y l b + I l bπ l )). By Young s inequality, given 1 + 1 = 1, y k a + y k b y 2 a b a b k, but equality must hold if a = b = 2. Similarly, y l a + y l b y 2 a b l, but equality must hold if a = b = 2. Since I k π k and I k π k, any choice a b can only yield Var Cab (ˆt) Var C2 (ˆt). Given all values of y k and y l, it is possible to derive an optimal vector of a kl and b kl values that varies over k, l, but such a derivation may not be of practical value. 4 Applications Proposition 1 indicates that the bias of the Horvitz-Thompson estimator under nonmeasurability is given by, A = y k y l. This expression, along with the fact that A A, makes it evident that the degree of bias in Var (ˆt) and Var C (ˆt) depends a great deal on the number of pairs with zero pairwise 7
inclusion probabilities. For designs where this number is small, Var (ˆt) may provide a reasonable and conservative estimator for cases where y k takes the same sign for all k, and Var C (ˆt) may provide a reasonable and conservative estimator for cases where y k may take different signs for some k. An example that arises frequently is stratified sampling where for a relatively small proportion of cases, we have small strata from which we draw only one unit. For designs that result in many pairs having zero inclusion probabilities, Var (ˆt) and Var C (ˆt) could be wildly over-conservative and other estimators may be preferred in terms of criteria such as mean square error. A prominent example is systematic sampling. Indeed, Särndal et al. (1992, 76) propose that under systematic sampling, the Horvitz- Thompson variance estimator, Var (ˆt), can give a non-sensical result. The expression for A makes it clear why this would be the case. Wolter (27, Ch. 8) shows that simpler biased estimators, such as the with-replacement (Hansen-Hurwitz) variance estimator, can be reliable, if slightly conservative, in a broad range of data scenarios under equal probability and probability proportional to size (PPS) systematic sampling. Nonetheless, it is known that the with-replacement estimator fails to account adequately for sampling variance when outcome variance within systematic sample clusters is smaller than the between cluster variance. In such cases, Var (ˆt) would bound this variance in expectation when outcomes are all of the same sign, and Var C (ˆt) would always bound this variance in expectation. Of course, it may still be the case that the bias is too large to be of much use, and so we would not suggest that Var (ˆt) and Var C (ˆt) provides a full solution to the variance estimation problem for systematic sampling under high intra-cluster correlation. Results from simulation studies are available in a supplement and Var C2 (ˆt) perform relative to other, commonly-used approximations in these applied scenarios. The simulations demonstrate situations when these estimators are preferable to commonly used alternatives. In the case of one-unit-per-stratum sampling, we show that these estimators are less biased than the commonly used collapsed stratum estimator in a range of scenarios. In the case of PPS systematic sampling, these estimators perform favorably when the population exhibits substantial periodicity, a case when the commonly-used with-replacement estimator may be grossly negatively biased. [These simulation results are appended to the manuscript.] 5 Conclusion We have characterized precisely the bias of the Horvitz-Thompson estimator under nonmeasurability and used this characterization to develop a conservative bias correction. These estimators reflect the fundamental uncertainty inherent to non-measurable designs. Compared to available approximate methods, these estimators may sometimes perform better and sometimes worse from a practical perspective. But available approximate methods may be biased in ways that cannot always be evaluated in terms of either magnitude or 8
sign. The estimators developed in this paper may therefore provide an informative measure of sampling variability with which analysts can agree without invoking additional assumptions or resorting to methods that carry the potential for negative bias. The bias term, A, has a simple form that suggests the possibility of refinements to the estimators developed here, something that we leave open for future research. References Hansen, M. H., W. N. Hurwitz, and W. G. Madow (1953). Sample Survey Methods and Theory: Volume 1 Methods and Theory. New York: Wiley. Horvitz, D. and D. Thompson (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association 47(26), 663 685. Särndal, C.-E., B. Swensson, and J. Wretman (1992). Model Assisted Survey Sampling. New York: Springer. Wolter, K. (27). Introduction to Variance Estimation, Second Edition. New York: Springer. 9
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities Supplement: Simulation Studies A Introduction This supplement contains a simulation study of applications of the estimators discussed in the paper, Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities. In the simulations, expected values of Var (ˆt) and Var C (ˆt) were computed exactly using expressions from Propositions 1 and 2 in the main text. Expected values for alternative estimators were computed using exact expressions for bias when available and via simulated sample draws otherwise. All simulations study expected values of estimators for the sampling variance of the Horvitz-Thompson estimator for the population mean, ˆµ HT 1 N ˆt. All simulations were carried out in the R statistical computing environment (the code is available from the authors). B Lonely primary sampling units in stratified sampling The first set of simulation studies examines variance estimation when single units are drawn from strata under stratified sampling. This situation arises in a number of applied contexts, including stratified sampling under proportional allocation (Lohr, 1999, 14-6) as well as subgroup analyses using subgroups that are sparsely distributed over strata. We build our simulations to match the context considered by Wolter (27, 5-57) in his study of the collapsed stratum estimator. Simplifying that context, we consider two strata each of size M, in which case N = 2M. From each stratum a single unit is randomly sampled. Thus, for unit k in stratum s, π ks = π = 1 M, and for units k and l in strata 1 or 2, we have { if s = t π ks,lt = 1 if s t. M 2 A1
In stratum s, outcomes y s = (y 1s,..., y Ms ) are drawn as, y ks = α s + ɛ is, where α s is a stratum-specific constant and ɛ ks N (, 1). For the two strata, 1 and 2, we keep things simple by setting α 1 = α = α 2, in which case the intra-class correlation over strata is given by, α 2 ρ ICC = α 2 + 1/2. We simulated 1, sets of outcomes, (y 1, y 2 ) over varying values of M and ρ ICC. To each set, we applied the single-unit-per-stratum design. Then, we computed three approximations of the sampling variance of the Horvitz-Thompson estimator of the population mean: 1. The Horvitz-Thompson estimator for the sampling variance of the estimator for the population mean, (1/N 2 ) Var (ˆt). 2. The bias-adjusted estimator, (1/N 2 ) Var C2 (ˆt). 3. The collapsed stratum estimator (Wolter, 27, 51-2), which for this case, given unit k from stratum 1 and unit l from stratum 2 selected into s, equals, V CS (ˆµ HT ) = 1 ( yk1 y ) 2 l2 = 1 N 2 π 1 π 2 4 (y k1 y l2 ) 2. Because of the zero inclusion probabilities within strata, each of these estimators is biased. No generally unbiased estimator exists for the sampling variance of the population mean estimator in this scenario. The collapsed stratum estimator is a commonly used approximation in the applied literature. Table A1 and Figure A1 display results from these simulations. For each ρ ICC and M value, we used the empirical variance of ˆµ HT as an estimate of the true variance (Var or True ), and means of the computed (1/N 2 ) Var (ˆt) ( Var HT or Horvitz-Thompson ), (1/N 2 ) Var C2 (ˆt) ( Var C2 or Conservative (A ) ), and V CS (ˆµ HT ) values as estimates of the expected values of these estimators. In these simulations, (1/N 2 ) Var (ˆt) strictly dominates (1/N 2 ) Var C2 (ˆt) and V CS (ˆµ HT ) as an approximation to the true sampling variance. The (1/N 2 ) Var C2 (ˆt) estimator dominates V CS (ˆµ HT ) for smaller stratum sizes and the two perform similarly for larger stratum sizes. When ρ ICC is close to 1, all approximations perform very badly. The reasons for this differ for the estimators, however. The bias of V CS (ˆµ HT ) is driven up directly by between stratum variance. The bias of (1/N 2 ) Var (ˆt) and (1/N 2 ) Var C2 (ˆt) is driven by the fact that within-stratum values become less symmetric over zero as α increases. A2
C Systematic probability-proportional-to-size sampling A second set of simulations is based on systematic probability-proportional-to-size (PPS) sampling, another commonly used design (Lohr, 1999, 185-6; Wolter, 27, 332-5). The problem that arises for variance estimation under systematic sampling, whether PPS or not, is that it produces zero pairwise inclusion probabilities among population units that are proximate on the list used to take the systematic sample. For the simulations, a size variable, x = (x 1,..., x N ), is produced as a vector of N random draws from a negative binomial distribution with mean 6, variance 16, and offset of 1 to avoid zero values. These settings were meant to recreate a data scenario roughly comparable to Wolter (27, Ch. 8). An outcome variable, y, is created as, y k = βx k + ɛ k, for k = 1,..., N, where β = ρ(σ y /σ x ), ɛ k is Gaussian noise with mean zero and standard deviation σ y 1 ρ2, and σ x and σ y are the standard deviation of the N values in x and y respectively, where E x (σ x ) = 4 and we fix σ y = 3 (again, to mimic data distributed as in Wolter (27, Ch. 8)). Thus, x and y have a linear relationship through the origin and their correlation is given by ρ. Systematic PPS sampling was implemented such that each unit s first order inclusion probability is given by, π k = x k N l=1 x m, l where m is the size of the systematic sample, and joint inclusion probabilities are computed as per Madow (1949). For the simulations, sampling and computation of inclusion probabilities were carried out using functions provided by Tillé and Matei (211). Because y is proportional to x with x 1, Proposition 1 in the main text indicates that the Horvitz-Thompson variance estimator for ˆµ HT, (1/N 2 ) Var (ˆt), provides a conservative approximation of the sampling variance of ˆµ HT. For comparison, we also computed (via 1 simulated sample draws) the expected value of the with-replacement variance estimator for ˆµ HT (Wolter, 27, expr. 8.7.2), Var W R (ˆµ HT ) = 1 ( ) 2 yk ˆt, N 2 m(m 1) p k k s where p k = π k /m is the notional per-draw selection probability. Var W R (ˆµ HT ) is regularly used for systematic samples in applied settings. Table A2 and Figure A2 display results from simulations for this scenario. Each graph plots the true variance (Var or True ), expected value of the (1/N 2 ) Var (ˆt) ( Var HT or Conservative (HT) ), and expected value of Var W R (ˆµ HT ) ( With-repl. ) over values of ρ and for varying sample sizes (m) and sampling proportions (m/n). As expected, A3
Var (ˆt) tends to produce very over-conservative estimates; this is due to the large number of pairs with zero-inclusion probabilities, each of which contributes to the bias. Given that the size variable is randomly sorted, this data scenario is very favorable to Var W R (ˆµ HT ), as is also evident from the results. For systematic sampling, a more challenging inference scenario arises when the data exhibit periodicity. In such a case, the within-sample variance fails to reflect the total of within and between-sample variance, potentially rendering approximations such as the with-replacement estimator anti-conservative (Särndal et al., 1992, 78-83). To illustrate this point, we performed a second set of simulations where the size variable, x, was drawn as an N-length vector of repeating 2 s and 1 s, thus maintaining variance values that are comparable to the previous simulation. Also, so that we could provide an illustration of the performance of Var C2 (ˆt), we produced y k values in the same manner as above, but then worked with the demeaned value, ỹ k y k ȳ, as the variable with which ˆµ HT was computed. By Propositions 1 and 2, Var (ˆt) may have either positive or negative bias, while Var C2 (ˆt) yields a conservative approximation of the sampling variance of ˆµ HT. Table A3 and Figure A3 display the results of these simulations. Each graph plots the true variance (Var or True ), expected value of (1/N 2 ) Var (ˆt) ( Var HT or Conservative (HT) ), expected value of (1/N 2 ) Var C2 (ˆt) ( Var C2 or Conservative (A ) ), and expected value of Var W R (ˆµ HT ) ( With-repl. ) over values of ρ and for varying sample sizes (m) and sampling proportions (m/n). As expected, Var W R (ˆµ HT ) tends to greatly understate the sampling variance of ˆµ HT, while Var C2 (ˆt) is generally quite conservative. Interestingly, Var (ˆt) tends to match the true value of the variance almost perfectly. This occurs because of the special nature of the data generating process, which results in the elements in the bias term, A, canceling each other out, in which case A tends to. References Lohr, S. L. (1999). Sampling: Design and Analysis. Pacific Grove: Brooks/Cole. Madow, W. G. (1949). On the theory of systematic sampling, II. Annals of Mathematical Statistics 2(3), 333 354. Särndal, C.-E., B. Swensson, and J. Wretman (1992). Model Assisted Survey Sampling. New York: Springer. Tillé, Y, and A. Matei. (211). sampling: Survey Sampling. R package, version 2.4. http://cran.r-project.org/package=sampling A4
Wolter, K. M. (27) Introduction to Variance Estimation, Second Edition. New York: Springer. A5
Table A1: Simulation results for lonely primary sampling units in stratified sampling for different values of ρ ICC and strata sizes (M). Sim. Params. True Mean from sims. Relative Bias Nominal Bias ρ ICC M Var Var HT Var C2 Var CS Var HT Var C2 Var CS Var HT Var C2 Var CS 1. 2.25.25.49.49. 1. 1.1..25.25 6.33 2.25.3.61.73.24 1.48 1.97.6.36.48 11.5 2.25.38.76 1..5 2.1 2.93.13.51.75 16.67 2.25.5 1. 1.51 1.4 3.9 5.14.26.76 1.26 21.82 2.25.82 1.64 2.77 2.33 5.65 1.27.57 1.39 2.53 26.9 2.25 1.4 2.79 5.9 4.6 1.2 19.4 1.15 2.54 4.84 31.99 2.25 14.3 28.6 56.7 55.77 112.54 224.12 14.5 28.34 56.44 2. 5.39.39.79.49. 1.3.26..4.1 7.33 5.4.5.99.75.24 1.48.86.1.59.35 12.5 5.4.6 1.21 1.1.52 2.5 1.55.21.81.61 17.67 5.4.79 1.57 1.47.98 2.97 2.69.39 1.18 1.7 22.82 5.4 1.3 2.61 2.76 2.28 5.56 5.94.91 2.21 2.36 27.9 5.41 2.26 4.52 5.15 4.58 1.16 11.72 1.86 4.12 4.75 32.99 5.4 22.94 45.87 56.84 56.59 114.17 141.71 22.54 45.48 56.44 3. 2.47.47.95.5. 1..5..47.3 8.33 2.47.59 1.18.74.25 1.5.58.12.71.27 13.5 2.48.72 1.44 1.1.5 2.1 1.12.24.96.53 18.67 2.47.94 1.89 1.49.99 2.99 2.14.47 1.41 1.1 23.82 2.47 1.53 3.5 2.72 2.26 5.53 4.82 1.6 2.59 2.25 28.9 2.48 2.67 5.35 5.13 4.63 1.26 9.8 2.2 4.87 4.65 33.99 2.47 27.23 54.46 56.83 56.74 114.47 119.5 26.76 53.99 56.36 4. 1.49.5.99.5. 1..1..5.1 9.33 1.5.62 1.24.75.25 1.5.51.12.74.25 14.5 1.5.74 1.49 1..5 2. 1.2.25.99.51 19.67 1.49.99 1.98 1.5 1. 2.99 2.2.49 1.48 1. 24.82 1.5 1.61 3.21 2.75 2.24 5.49 4.54 1.11 2.72 2.25 29.9 1.5 2.79 5.57 5.13 4.61 1.23 9.33 2.29 5.8 4.63 34.99 1.49 28.36 56.71 56.79 56.47 113.94 114.9 27.86 56.22 56.29 5. 5.5.5 1..5. 1....5. 1.33 5.5.62 1.25.75.25 1.5.5.13.75.25 15.5 5.5.75 1.5 1..5 2.1 1.1.25 1..5 2.67 5.5 1. 2. 1.5 1. 3. 2.1.5 1.5 1. 25.82 5.5 1.62 3.24 2.75 2.25 5.5 4.51 1.12 2.74 2.25 3.9 5.5 2.8 5.6 5.11 4.62 1.24 9.26 2.3 5.1 4.61 35.99 5.5 28.57 57.14 56.76 56.15 113.29 112.52 28.7 56.64 56.26 A6
Figure A1: Expected value of estimators of sampling variance of ˆµ HT under lonely primary sampling units in stratified sampling for different values of ρ ICC and strata sizes (M). M = 2 M = 5 M = 2 1 2 3 4 5 6 True Horvitz Thompson Conservative (A*) Collapsed stratum 1 2 3 4 5 6 1 2 3 4 5 6 ρ ICC ρ ICC ρ ICC M = 1 M = 5 1 2 3 4 5 6 1 2 3 4 5 6 ρ ICC ρ ICC A7
Table A2: Simulation results for PPS systematic sampling: random size case. Simulation Parameters True Mean from sims. Relative Bias Nominal Bias N m ρ y Var Var HT Var W R Var HT Var W R Var HT Var W R 1 8 2.5 3.29 529.96 544. 65.2.3.23 14.4 12.24 2 8 2.1 5.87 56.26 529.46 61.87.5.21 23.2 14.61 3 8 2.2 11.59 532.5 614.33 64.17.15.2 82.28 18.12 4 8 2.3 17.82 484.35 671.21 581.15.39.2 186.86 96.8 5 8 2.4 23.72 459.18 811.4 539.63.77.18 351.86 8.45 6 8 2.5 29.55 47.41 941.85 489.26 1.31.2 534.44 81.85 7 8 2.6 36.19 328.73 1147.58 386.2 2.49.17 818.85 57.29 8 8 2.8 47.88 189.28 1581.4 227.39 7.35.2 1392.12 38.11 9 8 2.95 55.97 53.28 1983.83 62.73 36.23.18 193.55 9.45 1 8 2.99 6.1 1.71 2236.5 13.31 27.82.24 2225.79 2.6 11 4 2.5 2.91 639.61 648.53 67.23.1.5 8.92 3.62 12 4 2.1 5.32 639.85 665.15 662.25.4.4 25.3 22.4 13 4 2.2 1.75 615.12 72.87 638.69.17.4 15.75 23.57 14 4 2.3 15.98 577.96 811.99 66.96.4.5 234.3 29. 15 4 2.4 21.58 538.98 967.71 559.55.8.4 428.73 2.57 16 4 2.5 26.84 476.99 1138.38 495.28 1.39.4 661.39 18.29 17 4 2.6 32.11 41.38 1356.77 428.89 2.31.5 946.39 18.51 18 4 2.8 43. 235.35 1936.15 245.59 7.23.4 17.8 1.24 19 4 2.95 51.21 63.69 2473.52 64.9 37.84.2 249.83 1.21 2 4 2.99 53.77 12.86 2667.59 13.45 26.43.5 2654.73.59 21 12 3.5 2.81 35.85 39.38 44.17.1.23 3.53 8.32 22 12 3.1 5.15 34.86 44.36 44.8.27.26 9.5 9.22 23 12 3.2 1.65 35.65 77.22 43.42 1.17.22 41.57 7.77 24 12 3.3 15.94 33.88 128.49 4.7 2.79.2 94.61 6.82 25 12 3.4 21.44 3.45 199.61 36.76 5.56.21 169.16 6.31 26 12 3.5 26.51 28.58 285.1 32.94 8.98.15 256.52 4.36 27 12 3.6 31.83 23.5 394.3 28.47 16.9.24 37.98 5.42 28 12 3.8 42.53 13.78 678.24 16.47 48.22.2 664.46 2.69 29 12 3.95 5.4 3.44 92.67 4.35 266.64.26 917.23.91 3 12 3.99 52.26.75 1.5.9 1333..2 999.75.15 31 6 3.5 2.67 42.2 48.32 44.56.15.6 6.12 2.36 32 6 3.1 5.18 43.1 66.48 44.23.54.3 23.38 1.13 33 6 3.2 1.42 41.99 136.45 42.38 2.25.1 94.46.39 34 6 3.3 15.6 39.41 251.13 41.9 5.37.4 211.72 1.68 35 6 3.4 21.3 36.99 422.26 37.91 1.42.2 385.27.92 36 6 3.5 26.11 32.6 626.57 33.3 18.22.2 593.97.7 37 6 3.6 31.65 26.96 899.8 28.63 32.38.6 872.84 1.67 38 6 3.8 42.15 15.49 1563.71 16.15 99.95.4 1548.22.66 39 6 3.95 49.89 4.26 2173.8 4.32 59.11.1 2168.82.6 4 6 3.99 52.29.87 2383.53.9 2738.69.3 2382.66.3 41 24 6.5 2.73 18.19 21.1 22.57.16.24 2.82 4.38 42 24 6.1 5.14 18.15 27.77 22.27.53.23 9.62 4.12 43 24 6.2 1.56 18.8 58.36 21.47 2.23.19 4.28 3.39 44 24 6.3 15.97 16.86 18.81 2.42 5.45.21 91.95 3.56 45 24 6.4 21.19 15.55 176.91 18.5 1.38.19 161.36 2.95 46 24 6.5 26.44 13.51 264.64 16.77 18.59.24 251.13 3.26 47 24 6.6 31.82 11.77 373.63 14.46 3.74.23 361.86 2.69 48 24 6.8 42.18 6.66 646.21 8.1 96.3.2 639.55 1.35 49 24 6.95 5.23 1.82 99.42 2.18 498.68.2 97.6.36 5 24 6.99 52.24.37 98.7.44 2647.84.19 979.7.7 51 12 6.5 2.62 21.61 27.51 22.24.27.3 5.9.63 52 12 6.1 5.27 21.31 45.42 22.1 1.13.4 24.11.79 53 12 6.2 1.45 2.88 115.68 21.32 4.54.2 94.8.44 54 12 6.3 15.78 19.42 236.2 2.27 11.15.4 216.6.85 55 12 6.4 21.15 18.15 47.25 18.66 21.44.3 389.1.51 56 12 6.5 26.28 16.18 616.64 16.75 37.11.4 6.46.57 57 12 6.6 31.43 13.81 872.7 14.32 62.19.4 858.89.51 58 12 6.8 42.5 7.71 1545.25 8.7 199.42.5 1537.54.36 59 12 6.95 49.95 2.8 2171.46 2.18 142.97.5 2169.38.1 6 12 6.99 51.99.44 235.79.45 5341.7.2 235.35.1 A8
Table A3: Simulation results for PPS systematic sampling: periodic size case. Simulation Parameters True Mean from sims. Relative Bias Nominal Bias N m ρ y Var Var HT Var C2 Var W R Var HT Var C2 Var W R Var HT Var C2 Var W R 1 8 2.5 -. 68.5 527.29 111.16 819.45 -.13.81.35-8.76 493.11 211.4 2 8 2.1 -. 644.24 557.15 1146.35 86.8 -.14.78.25-87.9 52.11 162.56 3 8 2.2. 625.78 542.86 199.9 765.79 -.13.76.22-82.92 473.31 14.1 4 8 2.3. 661.57 573.37 1123.95 727.74 -.13.7.1-88.2 462.38 66.17 5 8 2.4. 726.7 631.77 1199.1 693.81 -.13.65 -.5-94.93 472.4-32.89 6 8 2.5. 816.15 713.11 1282.44 611.1 -.13.57 -.25-13.4 466.29-25.14 7 8 2.6 -. 89.58 776.59 1344.6 523.27 -.13.51 -.41-113.99 454.2-367.31 8 8 2.8 -. 1145.46 13.38 1578.73 291.78 -.12.38 -.75-142.8 433.27-853.68 9 8 2.95. 1336.81 1173.42 1739.33 8.9 -.12.3 -.94-163.39 42.52-1255.91 1 8 2.99. 1397.5 1227.24 1791.48 16.2 -.12.28 -.99-169.81 394.43-1381.3 11 4 2.5. 764.97 743.7 1574.64 8.6 -.3 1.6.5-21.27 89.67 35.63 12 4 2.1 -. 777.42 755.92 1586.43 83.44 -.3 1.4.3-21.5 89.1 26.2 13 4 2.2. 798.12 775.99 1614.4 799.17 -.3 1.2. -22.13 815.92 1.5 14 4 2.3 -. 852.85 829.37 1667.29 733.92 -.3.95 -.14-23.48 814.44-118.93 15 4 2.4. 94.72 879.71 1716.9 67.94 -.3.9 -.26-25.1 811.37-233.78 16 4 2.5. 975.72 948.93 1782.24 6.88 -.3.83 -.38-26.79 86.52-374.84 17 4 2.6 -. 162. 132.97 1865.95 515.88 -.3.76 -.51-29.3 83.95-546.12 18 4 2.8 -. 1295.1 1259.81 294.14 291.75 -.3.62 -.77-35.2 799.13-13.26 19 4 2.95. 153.69 1462.95 2297.2 77.76 -.3.53 -.95-4.74 793.33-1425.93 2 4 2.99. 1559.54 1517.3 2347.71 16.14 -.3.51 -.99-42.24 788.17-1543.4 21 12 3.5 -. 42.55 36.78 653.92 53.7 -.14 14.37.26-5.77 611.37 11.15 22 12 3.1. 58.69 51.57 669.42 53.35 -.12 1.41 -.9-7.12 61.73-5.34 23 12 3.2 -. 1.87 89.35 71.66 5.59 -.11 5.96 -.5-11.52 6.79-5.28 24 12 3.3. 181.49 161.43 78.15 49.3 -.11 3.3 -.73-2.6 598.66-132.46 25 12 3.4 -. 296.35 264.5 881.44 45.27 -.11 1.97 -.85-32.3 585.9-251.8 26 12 3.5. 431.48 385.26 995.71 4.7 -.11 1.31 -.91-46.22 564.23-391.41 27 12 3.6. 66.48 541.65 116.33 34.87 -.11.91 -.94-64.83 553.85-571.61 28 12 3.8 -. 144.4 933.46 1549.4 19.51 -.11.48 -.98-11.94 54.64-124.89 29 12 3.95. 145.34 1297.21 191.48 5.12 -.11.32-1. -153.13 46.14-1445.22 3 12 3.99. 1578.87 1412.15 229.28 1.7 -.11.29-1. -166.72 45.41-1577.8 31 6 3.5. 56.35 54.9 891.43 54.67 -.3 14.82 -.3-1.45 835.8-1.68 32 6 3.1 -. 63.91 62.36 893.22 52.77 -.2 12.98 -.17-1.55 829.31-11.14 33 6 3.2 -. 114.48 112.18 949.68 52.71 -.2 7.3 -.54-2.3 835.2-61.77 34 6 3.3. 196.46 193.7 125.17 48.5 -.2 4.22 -.75-3.39 828.71-147.96 35 6 3.4 -. 32.6 297.22 1131.46 46.29 -.2 2.75 -.85-4.84 829.4-255.77 36 6 3.5 -. 441.46 434.71 1262.96 4.83 -.2 1.86 -.91-6.75 821.5-4.63 37 6 3.6. 614.43 65.26 1436.63 34.54 -.1 1.34 -.94-9.17 822.2-579.89 38 6 3.8 -. 148.24 133.1 1861.12 19.59 -.1.78 -.98-15.23 812.88-128.65 39 6 3.95 -. 1464.6 1443.65 2275.35 5.36 -.1.55-1. -2.95 81.75-1459.24 4 6 3.99 -. 1587.1 1564.34 2396.58 1.8 -.1.51-1. -22.67 89.57-1585.93 41 24 6.5 -. 23.14 2.39 622.28 27.2 -.12 25.89.17-2.75 599.14 3.88 42 24 6.1. 34.57 3.43 631.34 27.15 -.12 17.26 -.21-4.14 596.77-7.42 43 24 6.2. 8.14 71.94 671.15 25.81 -.1 7.37 -.68-8.2 591.1-54.33 44 24 6.3 -. 166.64 15.2 758.2 24.83 -.1 3.55 -.85-16.44 591.38-141.81 45 24 6.4. 268.72 242.4 839.63 22.6 -.1 2.12 -.92-26.32 57.91-246.12 46 24 6.5. 41.6 37.89 971. 2.42 -.1 1.36 -.95-39.71 56.4-39.18 47 24 6.6. 587.24 53.72 1132.75 17.26 -.1.93 -.97-56.52 545.51-569.98 48 24 6.8. 146.57 947.17 1551.1 9.7 -.9.48 -.99-99.4 54.44-136.87 49 24 6.95 -. 1459.15 132.2 1922.45 2.62 -.1.32-1. -138.95 463.3-1456.53 5 24 6.99. 1581.13 143.59 232.7.52 -.1.29-1. -15.54 45.94-158.61 51 12 6.5. 28.58 27.82 861.11 26.77 -.3 29.13 -.6 -.76 832.53-1.81 52 12 6.1. 41.36 4.42 874.69 26.11 -.2 2.15 -.37 -.94 833.33-15.25 53 12 6.2. 88.89 87.3 918.99 25.9 -.2 9.34 -.71-1.59 83.1-62.99 54 12 6.3 -. 168.73 165.94 997.42 24.28 -.2 4.91 -.86-2.79 828.69-144.45 55 12 6.4 -. 278.6 274.9 116.83 22.81 -.2 2.97 -.92-4.51 828.23-255.79 56 12 6.5 -. 424.5 417.44 1253.59 2.61 -.2 1.96 -.95-6.61 829.54-43.44 57 12 6.6. 599.83 59.55 1424.26 17.67 -.2 1.37 -.97-9.28 824.43-582.16 58 12 6.8 -. 147.24 131.27 1866.45 9.89 -.2.78 -.99-15.97 819.21-137.35 59 12 6.95. 1463.81 1441.68 2275.64 2.62 -.2.55-1. -22.13 811.83-1461.19 6 12 6.99 -. 1586.32 1562.38 2395.86.53 -.2.51-1. -23.94 89.54-1585.79 A9
Figure A2: Expected value of estimators of sampling variance of ˆµ HT under PPS systematic sampling for different values of ρ, sample sizes (m), and sampling proportions (m/n): random size case. N=8 m=2 N=4 m=2 2 True Conservative (HT) With repl. 2 15 15 1 1 5 5 N=12 m=3 N=6 m=3 2 2 15 15 1 1 5 5 N=24 m=6 N=12 m=6 2 2 15 15 1 1 5 5 A1
Figure A3: Expected value of estimators of sampling variance of ˆµ HT under PPS systematic sampling, for different values of ρ, sample sizes (m), and sampling proportions (m/n): periodic size case. N=8 m=2 N=4 m=2 2 15 1 5 True Conservative (HT) Conservative (A*) With repl. 2 15 1 5 N=12 m=3 N=6 m=3 2 2 15 1 5 15 1 5 N=24 m=6 N=12 m=6 2 2 15 1 5 15 1 5 A11