Coordinating the PRN: Combining Sequential and Bernoulli-Type Sampling Schemes in Business Surveys

Transcription

1 Coordnatng the PRN: Combnng Sequental and Bernoull-Type Samplng Schemes n Busness Surveys Ront Nrel, Aryeh Reter, Tzah Makovky and Moshe Kelner Central Bureau of Statstcs, Israel 66 Kanfey Nesharm St., Jerusalem 95464, Israel nrelr@cc.hu.ac.l; areter@cbs.gov.l; tzahm@cbs.gov.l; moshek@cbs.gov.l. Abstract In ths paper we use permanent random numbers (PRNs for the mantenance of a sngle sample over a long perod of tme. The sample s stratfed by ndustry and sze levels and unts are selected wth equal probabltes wthn strata. Two types of updates are carred out: (a annual updates reflectng all changes n the frame durng the precedng year and (b ntra-annual updates accountng for newborns durng the year. A sequental PRN scheme s used for the annual sample and a Bernoull scheme for the ntra-annual samples. Ths desgn causes two problems. Frst, because of the dfferences between the sequental and Bernoull schemes, the upper bound of PRNs of newborns may dffer from that of the old unts, thus resultng n dfferent selecton probabltes for the two types of unts n the next annual samplng. Second, snce generally the number of newborns n each update s small, the cumulatve sample sze over updates may have large devatons from ts expected value. Here we propose adusted PRNs for unts ncluded n the ntra-annual frames that have a unform dstrbuton n the unt nterval, and a maxmal value for the sampled unts that s approxmately equal to that of the annual sample. We also suggest a samplng scheme that controls the overall sample sze of the ntra-annual updates, whle keepng the expected weghts unchanged over tme. Fnally, we llustrate the proposed method for data from the Israel Manufacturng Indces Survey. Keywords: Beta dstrbuton; Collocated samplng; Coordnaton of samples; Order statstcs.. Introducton Natonal Statstcal Agences (NSAs publsh tme seres of economc ndcators based on perodc surveys of busness establshments. Whle effcent estmaton of perodc change requres maxmal overlap between successve samples, samples should also be contnuously updated to account for rapd changes n the busness populaton (e.g. brths, changes n sze. Other common problems nclude the need to coordnate between several samples and between unts wthn a sngle sample, to reduce response burden. Samplng schemes based on Permanent Random Numbers (PRNs provde smple and flexble solutons to such problems. Ohlsson (995 revews the man schemes based on PRNs and descrbes ther applcaton n a number of NSAs. The basc dea s that each unt n the frame s assgned a random number n the nterval (, that s permanently assocated wth that unt. Wthn strata, unts are ordered by ther PRNs and the samplng s based on the ordered lst. For example, n a sequental scheme for selectng a smple random sample wthout replacement (srswor of n unts from a populaton of sze N the frst n unts n the ordered lst are selected. In a Bernoull scheme, wth the same samplng rate, the sample comprses all unts wth a PRN not larger than n/n. Here we use PRNs for the mantenance of a sngle sample over a long perod of tme (e.g. a decade. The sample s stratfed by ndustry and sze levels and unts are selected wth equal probabltes wthn strata. Two types of updates are carred out: (a an annual update reflectng all changes n the frame durng the precedng year (e.g. deaths, splts, and changes n ndustry and (b ntra-annual updates accountng for newborns durng the year. The desgn of the annual update sample s guded by a requrement for a maxmal overlap of successve annual samples. The desgn of the ntra-annual samples s guded by the need to keep the selecton probabltes wthn strata constant throughout the year. To attan the frst requrement a sequental PRN scheme s appled each year to an updated frame. Unformty of the probabltes over the year s acheved by usng a Bernoull scheme for the ntra-annual updates. Ths desgn causes two problems. Frst, because of the dfferences between the sequental and Bernoull schemes, the upper bound of PRNs of newborns may dffer from that of the old unts (persstants resultng n dfferent selecton probabltes for the two types of unts n the next annual samplng. Second, snce generally the number of newborns n each update s small, the cumulatve sample sze over updates may have a large devaton from ts expected value.

2 In ths paper we address these problems analytcally. In Secton 2 we propose a method for combnng two frames from whch a sequental and a Bernoull samples were selected, respectvely, wth the same probabltes. The combned frame s then used to select a new sequental sample that has a szeable overlap wth the two former samples. Ths problem s related to the ssue of brth bas dscussed by Ernst, Vallant and Casady (2 and by Ohlsson (995, p. 66. Both papers are concerned wth bas n the selecton probablty of newborns under sample rotaton when the newborns and persstants are sampled together. For specfc stuatons these papers suggest solutons such as a correcton factor for the selecton probablty of newborns, and an adusted samplng scheme. In our problem, persstants and newborns are sampled separately and the two frames are then combned for a later samplng occason. Our method s based on an adustment of the newborns random numbers. The adusted numbers have a unform dstrbuton n the unt nterval, and the maxmal value for the sampled unts s approxmately equal to that of the sequental sample. Secton 3 deals wth the problem of sample sze varablty over several ntra-annual updates. The PRN lterature consders the queston of choosng between fxed sample szes and fxed ncluson probabltes, partcularly n the context of Posson samplng. Brewer, Early and Hanf (984 suggest collocaton of the random numbers so that they are more evenly dstrbuted n the unt nterval and thus reduce, but not elmnate, the varaton n sample sze. Ohlsson (995 suggests a sequental Posson scheme that ensures a fxed sample sze but results n approxmate selecton probabltes. The approxmaton s mproved by Saavedra (995 but no soluton s suggested for the calculaton of the true selecton probabltes. The problem of controllng the cumulatve sample sze of a sequence of Bernoull samples mposes an addtonal challenge. We propose a constructve method that calculates the desred sample sze for each update, and ensures that the cumulatve sample sze does not devate from ts expected value by more than one unt. Furthermore, the selecton probabltes are kept at ther expected values. The method s based on collocaton of the random numbers and on randomzaton of the condtonal ncluson probabltes. In Secton 4 we llustrate the suggested method for the Israel Manufacturng Indces Survey. Fnally, Secton 5 contans some concludng remarks. 2. A Sngle Intra-Annual Update We deal here wth annual busness samples that are stratfed by ndustry and sze levels. Generally there s a take-all stratum and several (e.g. one to fve take-some strata for each ndustry. To acheve statstcal effcency and reduce feldwork burden the same base sample s used for several years. Ths sample s updated perodcally (e.g. once n several months for new establshments (newborns and once a year for other changes n the frame, ncludng merges and splts as well as changes n ndustry and sze level. Formally, for each stratum h we have a base frame lst L h comprsng Nh unts. An annual sample Sh of nh unts s selected wth equal probabltes ( nh Nhfor the take-all strata. Unts are selected usng a sequental PRN scheme; that s, each unt n stratum h s assgned ndependently a permanent random number xh from a unform dstrbuton on (,. A random startng pont r h s selected, and we assume here wthout loss of generalty that r h. Hence, unts wth the nh smallest random numbers are ncluded n the sample. Let x ( n h be the largest random number n the sample. To abbrevate we drop herenafter the ndex h. We begn wth the smple case where the sample s updated for newborns once durng a gven year. A frame update L comprsng N newborn unts s obtaned, and a PRN s assgned to each unt n L. To smplfy the estmaton procedure t s requred that the update selecton probabltes are retaned at the annual sample value π n / N. A Bernoull scheme may be used to select the ntra-annual sample update S ; that s, all unts wth x π are ncluded n the sample. The update sample sze n s thus random, wth expected value Nπ and varance Nπ ( π. Collocaton of the random numbers may reduce the varablty of n. For example, we can sort the unts n L n a random order and assgn to unt k a random number from a unform dstrbuton on ( ( k / N, k/ N. As before, all unts wth PRN not greater than π are selected, but the sample sze varablty s reduced greatly because the random numbers are more equally spaced. In fact, n s equal to [ Nπ ] or [ ] Nπ +, wth respectve probabltes Nπ and a and a are the nteger and fractonal part of a, respectvely. We refer to ths modfed scheme as a Bernoull-Type (B-T scheme. N π, where [ ] In preparaton for the selecton of the next annual sample, a new frame L2 s constructed combnng lvng unts n L and L, and unts that were born n-between L and L 2. Generally, a coordnaton of some degree s desred between subsequent annual samples, rangng from a complete overlap to no overlap. A sequental srswor PRN scheme s appled to the new

3 frame, and the startng pont r 2 controls the degree of overlap. In our case, a maxmal overlap s desred and we thus assume r 2. However, unts n S have generally dfferent lkelhood of beng ncluded n the new sample as compared to unts n S, snce generally x( n π. The reason s that the order statstc ( k of a seres of N unform (, random varables has a Beta( kn, k+ dstrbuton (e.g. Stone, 996, p. 78. In partcular, ( n has a Beta( n, N n + dstrbuton wth expectaton p n /( N + and varance p( p /( N + 2. Thus, the scales of random numbers n L and L have to be coordnated before samplng from L 2. We construct now a transformaton of the random numbers of unts n L so that the transformed numbers have two propertes: (a they have a unform dstrbuton on the unt nterval and (b the maxmal value of the transformed numbers of unts n S s approxmately equal to x. Let be the PRN assgned to unt L and defne a rescaled random ( n varable by x( n f S π ( x( n ( π ( x( n + x( n f S. π For S, maps from the nterval (, π to (, x ( n, and for S from ( π, to ( x ( n,. The condtonal dstrbuton of (n x s a mxture of U(, x and U( x, dstrbutons wth respectve probabltes π and π. The uncondtonal dstrbuton s gven by n f ( t Pr{Beta( n,n n + > t}+pr{beta( n,n n t}, t (2 n (For more detals see the Appendx. Analyss of the frst and second dervatves of f ( t ndcates a mnmum at t π and two turnng ponts < t < π < t2 <, between whch the functon s convex, and otherwse t s concave. Thus, s not unformly dstrbuted on (,. Defne the U(, random varable Y F (, where F( s the cumulatve dstrbuton functon (CDF. The values of Y can be computed from n n2 N y ( t F (t Pr{ Beta(, } Pr{ Beta(, }, + N t + + N t t (3 N n n (See the Appendx for detals. The transformed varable Y has a unform (, dstrbuton by defnton and therefore holds property (a. To verfy property (b we study emprcally the relatve devaton of y( p from p, where p s the expected value ( n. Table shows the relatve devates RD ( y( p p / p and the relatve standard devaton of ( n, RSTD Var( ( n / p, for selected values of n and N. It s seen that the values of RD are consderably smaller than the respectve values of RSTD. For the least favorable row n the Table, n 2, the rato RSTD/RD s equal to 4.4, 4.9, and 7 for N 2, 5 and, respectvely. We conclude that the dfferences between x( n and π may be substantal and that as, on average, the maxmal value of for unts s S s p, and as the devaton of the CDF of from p s small, the adusted PRNs satsfy property (b. 3. Several Intra-Annual Updates In ths secton we deal wth the more realstc case of several ntra-annual updates of the frame (e.g. quarterly. We assume that for each update the number of new busness unts n a gven stratum, f any, s small. As before, t s requred that the selecton probablty n each stratum s kept at ts annual sample value, and that the sample sze varaton for each update and

4 Table. Expected values of x ( n ( n (RSTD for selected values of N and n. ( p, values of y(p, relatve devates of y(p from p (RD, and relatve standard devaton of N 2 N 5 N n p y(p RD RSTD p y(p RD RSTD p y(p RD RSTD over updates s mnmal. We begn by explanng how to determne the sze of the sample for each update and then show how to sample from the updates usng PRNs. Fnally we descrbe the adusted PRNs for the annual update. 3. Sample Sze Determnaton For any take-some stratum let N be number of samplng unts n the -th update,, K,J, n the actual sample sze n update and π the desred ncluson probablty. As before, denote by [ a ] the ntegral part of a, that s [ a] a < [ a] +, and by a the fractonal part of a, so that a [ a] + a. The expected sample sze for the frst update ( s E N π. Thus, the actual sample sze n s equal to [ E ] or [ E ] + wth respectve probabltes E and E. For >, let m n + K + n, E E( n N π, and F E + K+ E be the actual and expected cumulatve sample szes after samplng from the -th update, respectvely. Assume that the cumulatve sample sze m for update has the desred expected value wth mnmal varaton,.e. P( m F F and P( m F + F. (4 Clearly E( m F. We show now how to determne n so that m has the same optmal dstrbuton as m, that s, wth the ndex replacng the ndex n (4. Consder frst the case F > F. Gven m, n s selected at random from the followng condtonal dstrbuton: P( n F F m F F, P( n F F m F + F, P( n F F m F F, and P( n F m F + + F. Ths ensures that the uncondtonal dstrbuton of m m + n s the desred one. If F F then n wth condtonal probabltes ( ( ( P n m F E F and P n m F +, n otherwse. The expectaton of n s thus equal to E. and 3.2 Samplng for the Intra-Annual Updates Once the sample sze s determned for each stratum wthn update we sample usng the Bernoull-Type method. For the frst update, the unts,...,n are randomly permuted and a random number between (/N and /N s assgned to unt n the permuted lst. Unts wth numbers not greater than π are sampled. The random numbers are U(, dstrbuted and the sample sze n s as desred. For the -th update we assgn the random numbers as for the frst update, wth N replacng N. The samplng process depends, agan, on the relatonshp between F and F. If F > F then for every value of m we have

5 (. Denote by π ( F m / N so that E( n m E n m F m N π. Unts wth random numbers not greater than π are sampled. Note that here we have a samplng scheme wth a random samplng probablty and a fxed expected probablty. The sample sze varablty s mnmzed, wth a maxmal devaton of one unt between the actual and expected cumulatve sample szes. The values of π are ( F F N F- and F - greater than one. Such a case happens when m F + F one and sample n N unts. If F F and ( F F N wth probabltes, respectvely. These values are non-negatve, and n rare cases may be equal to zero, or be equal or, and then π and n. If π >, we truncate t to we bascally proceed as before, except for the case m F + where we take π and n, and the case when m F samplng scheme over updates as a condtonal Bernoull-Type (CB-T scheme. { } where we take π E N ( F. We refer to ths 3.3 Samplng for the Annual Update For the annual update a new frame based on L, L, K, LJ and addtonal newborns s constructed. As n Secton 2, n preparaton for the next annual samplng we adust the random numbers assgned to unts n the update frames. Frst, let be the random varable defned for the -th unt n the -th update frame as n (, wth π and S replacng π and S, respectvely. The varable s well defned when < π <. When π, no unts are sampled for the -th update and hence s computed only for unts S. Smlarly when π, s computed only for unts S. As we have seen n the sngle update scenaro of Secton 2, does not have a U(, dstrbuton and thus we further adust t usng Y F (. To extract F ( t, we frst note that the condtonal dstrbuton of, π s, as before, a (n mxture of U(, x (n and U( x (n, dstrbutons, but wth respectve probabltes π and π. Usng the ndependence of, π and (n, we wrte ther ont dstrbuton as a product of f and the margnal dstrbutons of (n and π. Integratng over the values of E( π π. Therefore, the CDF of (n, π π t s seen n the Appendx that the uncondtonal dstrbuton of s as n Equaton (3. 4. An Illustratve Example: The Manufacturng Indces Survey s as n (2, because The proposed method s mplemented for the monthly Manufacturng Indces Survey. The survey provdes measurements of the development n manufacturng and n the economy of Israel n general by estmatng the total number of employees, labor cost, turnover and other economc ndcators, by ndustry and sector. The samplng frame s extracted from the Israel Busness Regster (BR. Data for each samplng unt nclude the ndustry classfcaton, the annual turnover (T and the annual average number of employees (E. To determne the unt sze, a regresson model of T on E s ftted (n square root scale for each ndustry. The unt sze s equal to the observed annual turnover T, except for cases wth a negatve Studentzed resdual wth absolute value larger than a predefned value. In these cases the sze s set to the model predcton plus a random nose. For the 24 base sample, unts were stratfed by 77 manufacturng ndustres and by 2 to 4 sze strata. The number of sze strata depends on the number of unts n each ndustry, where the top sze group s a take-all stratum. The Lavallée-Hdroglou (988 stratfcaton algorthm wth a unform coeffcent of varaton (CV across ndustres s used to set the strata boundares and to allocate the sample to the take-some strata. The allocaton may then be adusted to meet addtonal methodologcal constrants, such as a mnmal number of sampled cases n a stratum, a maxmal samplng weght and a maxmal CV of the estmated number of employees. In each take-some stratum a srswor s drawn usng a sequental scheme. The total number of sampled unts for the 24 base sample was 2,2. The sample s updated for newborns once n two months, usng the condtonal B-T (CB-T scheme descrbed n Secton 3.2.

6 Table 2 compares the relatve standard devatons RSTD Var( m5 / F5 of the cumulatve sample sze obtaned up to and ncludng the ffth update, for selected ndustres and three samplng schemes: Bernoull (B, Bernoull-Type (B-T and the proposed condtonal B-T (CB-T scheme. The varances of the cumulatve sample sze m 5 are computed as follows: 5 5 ( m N π ( π ( m 5 E ( - E ( m F ( F Var, Var, Var. B 5 B-T CB-T We present ndustres that had a substantal number of newborns throughout 24. It s seen that on average usng the Bernoull scheme yelds sample szes wth RSTDs rangng between about 5 to 2 percent for the selected ndustres, as compared to -85% for the CB-T method. Over all take-some strata, the B-T scheme reduces the RSTD by about 7 percent, and the CB-T by further 3 percent. Equalty between values for the B-T and CB-T, as seen for ndustry 265 stratum 2, occurs when all newborn unts are n the same update. Equalty between the Bernoull and B-T RSTD values (ndustry 36, stratum 3 occurs when there s at most one newborn unt per update. 5. Concludng Remarks Revsons of monthly busness surveys that account for new unts are common practce. Frequently, however, t s desred that these sample revsons nterfere as lttle as possble wth the ongong man sample. The technque suggested by Ohlsson (995, Secton 9.. for coordnaton over tme s based on recurrng standard sequental samplng from an updated man frame. Ths scheme may affect persstants and change the selecton probabltes. We have seen that t s possble to sample newborns separately from the man sample, whle preservng fxed selecton probabltes throughout the year as well as mnmal sample sze varablty. We have also shown how to combne correctly sequental and Bernoull samples to form a subsequent sample wth a desred overlap. Once all unts have random numbers that are U(, and have a comparable scale for all prevously sampled (and non sampled unts, we can contnue wth the standard sequental scheme. Ernst, Vallant and Casady (2 dscuss varous PRN-based technques n the context of brths and deaths and argue that Though the methods are n common use, there appears to be a lmted lterature on ther propertes, partcularly regardng the treatment of populaton changes due to brths and deaths In ths paper we have suggested a workable scheme that s based on exact dstrbutonal consderatons. The devate y(p-p s not defned analytcally and we leave ths queston for future work. It would also be nterestng to study the bas nduced by usng or for samplng, rather than usng Y. Table 2. Relatve standard devatons (RSTD of the cumulatve sample sze for take-some strata n selected ndustres usng the Bernoull, Bernoull-Type (B-T and Condtonal B-T (CB-T schemes. Data from fve updates, the Manufacturng Indces Survey 24. Industry Stratum Cumulatve number of newborns Expected cumulatve sample sze RSTD Bernoull B-T CB-T All take-some strata

7 Appendx: Densty and Cumulatve Dstrbuton Functon of The densty of ( n s N! g x x x n Nn ( (, ( n ( n!( N n! and the condtonal densty of xs ( n π x f ( t ( n x π x f t x f x<t. Notng that the densty functon of a Beta( α, β varable on (, wth nteger parameters s uncondtonal densty s equal to f ( t f( t x x g( xdx The random varable ( n N! t n 2 N n n N n { π x ( x d x ( π x ( x dx} + ( n!( N n! t n + > ( n Pr{Beta( n, N n t}+pr{beta( n, N n t} has the same densty. To see ths, recall that the condtonal densty of n (5 wth π replacng π. Let h( π denote the probablty functon of π. The ont dstrbuton of π π f t,x, I, ( n,,x t I x, t g x h π x x ( π [ ]( + ( ] ( ( ( π ( α + β! ( ( α!( β! x x, α β ( n, the x,π π s as, (n and π s where I A (t s the ndcator functon of a set A at t. Integratng the ont dstrbuton over π yelds f ( t x g( x snce E( π π. To extract the CDF t s useful to recall that Pr{Beta( km, k+ t} Pr{Bn( mt, k}. Hence, t ( ( d n t t < + ( n F t f u u Pr{Bn( N, u n }d u Pr{Bn( N, u n }d u t N N N ( ( n t n2 N N u (- u d u+ u (- u du ( n n n n2 N Pr{ Beta( +, N t} + Pr Beta( +, N t N n n { }. ( n (5 References Ernst, L. R., Vallant, R. and Casady, R. J. (2. Permanent and collocated random number samplng and the coverage of brths and deaths. Journal of Offcal Statstcs, 6, Lavalée, P. and Hdroglou, M. A. (988. On the stratfcaton of skewed populatons. Survey Methodology, 4, Ohlsson, E. (995 Coordnaton of samples usng permanent random numbers. In Busness Survey Methods Ed B. G. Cox, D. A. Bnder, D. N. Chnnappa, A. Chrstanson, M. J. Colledge, and P. S. Kott. New York: John Wley,

8 Saavedra, P. J. (995. Fxed sample sze PPS approxmatons wth a permanent random number. In Proceedngs of the Secton on Survey Research Methods, Amercan Statstcal Assocaton, Orlando. Stone, C. J. (996. A Course n Probablty and Statstcs. Belmont: Duxbury Press.