Unequal probability inverse sampling

Transcription

1 Ctlogue o X ISSN Suvey Methodology Uequl pobblty vese smplg by Yves Tllé Relese dte: Decembe 20, 2016

2 How to obt moe fomto Fo fomto bout ths poduct o the wde ge of sevces d dt vlble fom Sttstcs Cd, vst ou webste, You c lso cotct us by eml t STATCAN.fostts-fostts.STATCAN@cd.c telephoe, fom Mody to Fdy, 8:30.m. to 4:30 p.m., t the followg toll-fee umbes: Sttstcl Ifomto Sevce Ntol telecommuctos devce fo the heg mped Fx le Depostoy Sevces Pogm Iques le Fx le Stdds of sevce to the publc Sttstcs Cd s commtted to sevg ts clets pompt, elble d couteous me. To ths ed, Sttstcs Cd hs developed stdds of sevce tht ts employees obseve. To obt copy of these sevce stdds, plese cotct Sttstcs Cd toll-fee t The sevce stdds e lso publshed o ude Cotct us > Stdds of sevce to the publc. Note of ppecto Cd owes the success of ts sttstcl system to log stdg pteshp betwee Sttstcs Cd, the ctzes of Cd, ts busesses, govemets d othe sttutos. Accute d tmely sttstcl fomto could ot be poduced wthout the cotued co opeto d goodwll. Stdd tble symbols The followg symbols e used Sttstcs Cd publctos:. ot vlble fo y efeece peod.. ot vlble fo specfc efeece peod... ot pplcble 0 tue zeo o vlue ouded to zeo 0 s vlue ouded to 0 (zeo) whee thee s megful dstcto betwee tue zeo d the vlue tht ws ouded p pelmy evsed x suppessed to meet the cofdetlty equemets of the Sttstcs Act E use wth cuto F too uelble to be publshed * sgfctly dffeet fom efeece ctegoy (p < 0.05) Publshed by uthoty of the Mste esposble fo Sttstcs Cd Mste of Idusty, 2016 All ghts eseved. Use of ths publcto s goveed by the Sttstcs Cd Ope Lcece Ageemet. A HTML veso s lso vlble. Cette publcto est uss dspoble e fçs.

3 Suvey Methodology, Decembe Vol. 42, No. 2, pp Sttstcs Cd, Ctlogue No X Uequl pobblty vese smplg Yves Tllé 1 Abstct I ecoomc suvey of smple of etepses, occuptos e domly selected fom lst utl umbe of occuptos locl ut hs bee detfed. Ths s vese smplg poblem fo whch we e poposg few solutos. Smple desgs wth d wthout eplcemet e pocessed usg egtve boml dstbutos d egtve hypegeometc dstbutos. We lso popose estmtos fo whe the uts e selected wth uequl pobbltes, wth o wthout eplcemet. Key Wods: Locto; Hovtz-Thompso estmto; Negtve boml; Negtve hypegeometc; Ivese desg; Icluso pobblty; Wge. 1 Poblem The poblem ose s pt of questo o Sttstcs Cd s ew Job Vccy d Wge Suvey (JVWS). The JVWS compses wge compoet d job vccy compoet. The wge compoet looks t vege wges, mmum wges, mxmum wges d sttg wges fo vous occuptos. The objectve s to povde wge sttstcs by ecoomc egos (ecoomc egos e subdvsos of povces). I the fst stge, smple of 100,000 busess loctos (lso kow s locl uts of etepses) e selected usg Posso desg sttfed by dusty d ecoomc ego. Fo smplcty, the tem etepse wll be used the est of the documet sted of locto, keepg md tht Sttstcs Cd defes locto s poducto ut locted t sgle geogphcl locto t o fom whch ecoomc ctvty s coducted d fo whch mmum of employmet dt e vlble. Fo puposes of mgg espose bude, t s ot possble to detfy evey occupto ech etepse. Theefoe, poposg lst of occuptos d skg whethe the lsted occuptos exst etepse hs bee cosdeed. Occuptos c the be domly dw fom the lst d poposed successvely to the hed of the etepse utl occuptos hve bee eched. Sce the most commo occuptos e of specfc teest, t s useful to cosde cses whch occuptos e selected wth uequl pobbltes fom the lst popoto to the pevlece the totl populto. Note tht ths method ws ot mplemeted fo Sttstcs Cd s Job Vccy d Wge Suvey. The suvey decded to peset lst, of fxed legth, of occuptos to the suveyed etepses. Nevetheless, the theoetcl popetes of the poposed method em of teest. Ivese smplg efes to scheme whch uts e selected successvely utl pedetemed umbe of uts wth cet chctestc s obted. Ivese smplg must ot be cofused wth ejectve smplg. I ejectve smplg, smple s selected ccodg to desg, d the smple s ejected f t does ot hve the desed chctestc (e.g., specfc smple sze o vege equl to tht of the populto). The selecto of smples s epeted utl smple wth the desed popety s obted. 1. Yves Tllé, Isttute of Sttstcs, Uvesty of Neuchâtel, Aveue de Bellevux 51, 2000 Neuchâtel, Swtzeld. E-ml: yves.tlle@ue.ch.

4 284 Tllé: Uequl pobblty vese smplg Ivese smplg ses cet umbe of theoetcl questos. How c such desg be mplemeted wth equl o uequl cluso pobbltes? Wht s the pobblty of cluso of occupto wth ech etepse? How c vble of teest be estmted usg smple cosstg of few etepses d few occuptos wth them? How c the umbe of occuptos the etepse be estmted? Moe geelly, how c ths suvey be mplemeted d how c estmto be doe? The key ssue s the wy whch the occuptos e selected. They my be selected usg smple desg wth o wthout eplcemet, o wth uequl pobbltes. Oe opto would be to select the uts wth uequl pobbltes usg the sequetl Posso smplg method poposed by Ohlsso (1998) o the Peto smplg method poposed by Rosé (1997). The vese smplg poblem hs ledy bee dscussed by Muthy (1957), Smpfod (1962), Pthk (1964), Chkkgoud (1966, 1969), d Sleh d Sebe (2001). Howeve, the pmete to be estmted hee s uque, sce estmtes of vege eveue mog ll etepses hvg specfc occupto e desed. We lso popose ew uequl-pobblty vese desg wthout eplcemet. Ths tcle s ogzed s follows: I Secto 2, the poblem s stted d the otto s defed. The equl pobblty cse wth eplcemet s dscussed Secto 3, d the equl pobblty cse wthout eplcemet s dscussed Secto 4. The uequl pobblty cse wth eplcemet s developed Secto 5. A ew selecto method fo the uequl pobblty cse wthout eplcemet s peseted Secto 6. Flly, Secto 7 cots shot dscusso. 2 Fomlzto of the poblem The followg otto s used: U : populto of N etepses,.e., U 1,,,, N (U my deote the populto of etepses ecoomc ego), L : the lst of occuptos, M : the umbe of occuptos the lst,.e., the sze of L, F : the lst of occuptos etepse, wth F L, D : the lst of occuptos bset fom etepse, wth D L, F D L d D F, Mp : the umbe of occuptos etepse,.e., the sze of F, : the umbe of dstct occuptos to be obted ech etepse, X : the umbe of flues befoe the occuptos etepse e obted by selectg the occuptos usg gve desg. The m objectve s to estmte the vege wge fo occupto the totl populto. Let y be k the vege wge fo occupto k etepse, d let z be the umbe of employees wth occupto k k etepse. The objectve s to estmte the vege wge fo occupto k gve by Sttstcs Cd, Ctlogue No X

5 Suvey Methodology, Decembe Y k U F k z y k k. z k U F k Assume tht smple of etepses S 1 s selected fom U usg some gve desg wth cluso pobbltes. I etepse, smple of occuptos S s selected usg oe of the desgs descbed 1 bove wth cluso pobblty. If the desg s wth eplcemet, epesets the expected umbe k k of tmes tht occupto k s selected etepse. Y c be estmted usg to type estmto (Hájek 1971): k ˆ S1 S 1 F k k k. zk Y S S F k 1 k 1 Theefoe, the pobblty tht occupto wll be selected etepse must be kow. Howeve, wth vese type desg, the pobblty s ukow d must theefoe be estmted ode to estmte Y. Sce the cluso pobbltes ppe the deomto, t s pefeble to estmte the veses of k I etepse, occupto s pobblty of beg selected deceses s the umbe of occuptos. k ceses. I ddto, the pobblty depeds o the vese smplg desg used ech etepse. z y k k 3 Smple dom smplg wth eplcemet Assume tht etepse hs popoto p of the occuptos the lst the etepse. If the smple of occuptos s dw wth eplcemet etepse utl occuptos the etepse hve bee detfed, the X NB, p. I tht cse, X hs egtve boml dstbuto deoted by x 1 x P X x p 1 p, x * wth x 0,1,2,3,, p 0,1, 1,2,3,. Futhemoe, 1 p 1 p E X d v X. p p Let A, k L, be the umbe of tmes tht ut k s selected the smple tke fom etepse. I k smple desg wth eplcemet of sze, the vlues of whee A 0,,, d k! 1 P A, k L, M k k! k L k 2 A hve multoml dstbuto. Theefoe, k Sttstcs Cd, Ctlogue No X

6 286 Tllé: Uequl pobblty vese smplg kl k. If ths multoml vecto s codtoed o fxed sze gve pt of the populto, the P A, k F A k k k kf P A, k F d k k A k kf P A k kf p p! 1 p!!!1 1! M! k F k 1 1!, Mp k F! k wth kf k. Ths shows tht, f the sum of A s codtoed o oe pt of the populto, the dstbuto ems k multoml d codtolly thee s stll smple desg wth eplcemet. Wth the pocedue whch we dw wth eplcemet utl we obt occuptos etepse, we hve Mp E A X k X M Mp f k F f k D. I fct, codtolly o X, F of sze Mp, occuptos e selected d, X occuptos e selected. D of sze M 1 p, I the cse wth eplcemet, wht s clculted s ot elly cluso pobblty, but the the expected vlue of A whch s deoted s, k k EE A X, Mp k k k L. The poblem s tht we kow M, d X, but ot p. We c estmte p usg the method of E X X, whch yelds momets by solvg Sttstcs Cd, Ctlogue No X

7 Suvey Methodology, Decembe d theefoe X 1 pˆ pˆ pˆ. 1 X The mxmum lkelhood method povdes the sme estmto s the method of momets, but ths estmto s bsed (Mkulsk d Smth 1976; Johso, Kemp d Kotz 2005, pge 222). If 2, the ubsed mmum vce estmto of p s Howeve, 1 p ˆ s ubsed fo 1 p. 1 1 pˆ. 2 X 1 Sce we e usg weghts tht e veses of, the veses of e thus estmted s follows: k k Mpˆ M 1 2 f k F X 1 1 k M 1 pˆ 2 M f k D. X X 1 Howeve, the cse wth eplcemet s ot vey stsfctoy, becuse selectg occuptos wth eplcemet does ot ecessly esult dstct occuptos, sce the sme occupto my be selected moe th oce. Futhemoe, smplg my be especlly log f eplcemet s pefeed. Mp s smll. Theefoe, smplg wthout 4 Smple dom smplg wthout eplcemet Fo the cse wthout eplcemet, the otto used s the sme s fo the dw wth eplcemet. The umbe of flues X theefoe hs egtve hypegeometc dstbuto. Ths pobblty dstbuto s lttle kow, to the pot tht t hs bee peseted s fogotte dstbuto by Mlle d Fdell (2007). Ths dstbuto s the coutept to the egtve boml fo the dw wthout eplcemet. The geel fmewok s s follows: We cosde populto of sze M whch thee e Mp fvouble uts, mely the occuptos the lst tht exst the etepse. If the dws e equl pobblty wthout eplcemet utl fvoble uts ppe, the the egtve hypegeometc vble, X NH( M,, Mp ), couts the umbe of flues befoe fvouble evets occu. The pobblty dstbuto s x 1 M x x Mp P X x px; M,, Mp, M Mp Sttstcs Cd, Ctlogue No X

8 288 Tllé: Uequl pobblty vese smplg whee 0,, 1, M 1,2,, Mp 1,2,, M, d x M p 1,2,, Mp. 2 M 1 p M 1 p M 1 Mp 1 E X,v X. Mp 1 Mp 1 Mp 2 Ag, A deotes the umbe of tmes tht ut k s selected the smple. Now, the vlue of A c k k be oly 0 o 1. If uts e selected usg smple desg wthout eplcemet L, the smple desg s defed s 1 M P A, k L, k k whee 0,1, d k kl k. If the vecto of A s codtoed o fxed sze oe pt of the populto, we hve k P A, k F A k k k kf P A, k F d A k k k kf P A kf 1 MpM Mp 1 M kd M A k F k A 0,1 MpM Mp M Mp M M 1 Mp, k 1 k wth kf k. Ths shows tht, f the sum of A s codtoed o oe pt of the populto, we stll hve smple k desg wthout eplcemet. I the pocedue whch we dw wthout eplcemet utl we obt occuptos etepse, we theefoe hve Sttstcs Cd, Ctlogue No X

9 Suvey Methodology, Decembe Mp E A X k X M Mp f k F f k D. The cluso pobblty s theefoe f k F Mp EE A X E X f k D, M Mp Mp 1 k k fo ll k L. Ag, the poblem s tht we kow M, d X, but ot p. We c estmte p usg the mxmum lkelhood method, though umecl method. Usg the method of momets, estmte c be obted by solvg fo X E X, tht s, p the equto X M 1 pˆ. Mpˆ 1 Hece Howeve, few les t s vefed tht, f 2, M X pˆ. 1 M X pˆ 2 1 X 1 s ubsed fo p. Ag, sce we e usg weghts tht e veses of. The veses of the cluso pobbltes k e thus estmted s follows: Mpˆ M 1 2 f k F 1 1 X k M 1 pˆ 2 M f k D. X X 1 These weghts e lso used the estmto by Muthy (1957), whch s ubsed (see lso Sleh d Sebe 2001). If Mp <, ll occuptos wll be selected etepse d the estmted cluso pobbltes e the equl to 1. Sttstcs Cd, Ctlogue No X

10 290 Tllé: Uequl pobblty vese smplg 5 Uequl pobblty smplg wth eplcemet Uequl pobblty smplg s ot elly moe dffcult to pocess whe the dw s wth eplcemet. Now let p deote the pobblty of occupto beg dw ech dw wth k Let P be the sum of kl p k 1. p lmted to the occuptos etepse : k P p. kf k I ths cse, X hs egtve boml dstbuto wth pmetes d P. Theefoe, 1 P 1 P E X d v X. P P Let A, k L be the umbe of tmes tht ut k s selected the smple. I uequl pobblty k desg wth eplcemet of sze, the vlues of A hve multoml dstbuto. Theefoe, k whee A 0,,, d k p k k P A, k L!, k k! kl k kl k. If ths multoml vecto s codtoed o fxed sze oe pt of the populto, the P A, k F A k k k kf P A, k F d A k k k kf P! 1 P A!1 P p!!!! kf k p 1 k!, kf P! k k k k kf k P wth kf k. Sttstcs Cd, Ctlogue No X

11 Suvey Methodology, Decembe Ths shows tht, f the sum of A s codtoed o oe pt of the populto, the dstbuto ems k multoml d codtolly thee s stll uequl pobblty desg wth eplcemet. Wth the pocedue whch we dw wth eplcemet utl we obt occuptos etepse, we hve The expected vlue of A s k pk P E A X k Xp k 1 P f k F f k D. pk EE A X, P k k k L. The poblem s tht we kow p, d X, but ot P. We c estmte P usg the method of k E X X, whch gves momets by solvg X 1 Pˆ Pˆ d theefoe Pˆ. 1 X The mxmum lkelhood method povdes the sme estmto s the method of momets, but ths estmto s bsed (Mkulsk d Smth 1976; Johso et l. 2005, pge 222). I fct, the ubsed mmum vce estmto s Howeve, 1 P ˆ s ubsed fo P. 1 1 Pˆ. 2 X 1 Ag, sce we e usg weghts tht e veses of. The veses of e thus estmted s k k follows: Pˆ 1 2 f k F 1 1 p X p k k k 1 Pˆ 1 2 f k D. Xp X 1 k pk (5.1) Sttstcs Cd, Ctlogue No X

12 292 Tllé: Uequl pobblty vese smplg 6 Uequl pobblty smplg wthout eplcemet 6.1 Sequetl smplg wthout eplcemet Fo the dw wthout eplcemet, the fst poblem s detemg the desg. Oe opto s to use the method by Ohlsso (1995) clled sequetl Posso smplg. Ths method volves geetg M ufom dom vbles the tevl 0,1, deoted u. Next, we select the uts coespodg to k the smllest vlues of u. Ths method hs the dvtge of beg usble fo y smple sze d k k povdg sequece of smples tht e cluded ech othe. Ufotutely, t oly stsfes ppoxmtely the fxed cluso pobbltes. Howeve, the ppoxmtos e vey ccute ccodg to the smultos gve Ohlsso (1995). Methods hve lso bee poposed by Smpfod (1962) d Pthk (1964). We popose exct soluto to the poblem the sese tht the cluso pobbltes e exctly stsfed. We beg by clcultg the cluso pobbltes fo desg of fxed sze wth cluso pobbltes popotol to stctly postve uxly vble b, k L. The pobbltes e detemed by whee C s detemed such tht k kl k k bk m 1, C, b L bk m 1, C. kl b L A smple lgothm fo clcultg these pobbltes s descbed Tllé (2006, pge 19), mog othes. The pobbltes c be clculted smply usg the fucto clusopobbltes the R smplg pckge. A sequetl selecto method must theefoe select smple of sze wth cluso pobbltes k. It must the mke t possble to go fom sze to sze 1 by smply selectg ddtol ut such tht the completed smple hs cluso pobblty of 1. k It ppes tht the oly method tht llows tht to be cheved s the elmto method (Tllé 1996). Ths method stts wth the ete populto (the lst of occuptos) d elmtes oe ut ech step. I step j 1,, N, the ut s elmted fom mog the emg uts wth the pobblty 1 N j k N j 1. k Ths method c thus be used to cete sequece of smples cluded ech othe tht vefy the cluso pobbltes elto to the sze. Theefoe, we c smply pply the elmto method fo smple sze 1 so tht the lgothm successvely elmtes ll the uts. Tkg them the evese ode of elmto, we obt sequece of uts. The fst uts of the sequece e selected wth cluso pobblty. The ppedx k Sttstcs Cd, Ctlogue No X

13 Suvey Methodology, Decembe cots fucto wtte R tht c be used to geete ths sequece. The code s executed smulto tht shows tht the pobbltes obted though smultos by pplyg ths fucto e equl to the fxed cluso pobbltes fo ll smple szes. 6.2 Ivese o egtve desg wth uequl pobbltes Now tht the desg s defed, the vese desg c be defed. The uts the lst of occuptos e tke usg the elmto method utl occuptos the etepse e selected. I ths cse, the pobblty dstbuto of the umbe of flues codtol cluso pobblty E k A X s lso poblemtc. X seems mpossble to clculte. Clcultg the Howeve, we c poceed by logy d estmte the cluso pobbltes o the bss of expesso (5.1) developed fo the cse wth eplcemet, whee p c smply be eplced by k Theefoe, we obt 7 Dscusso k X. X 1 X X 1 X 1 k X X 1 X k k f k F f k D. The selecto poblem c theefoe be esolved fo ll cses, wth o wthout eplcemet d wth equl o uequl pobbltes. The poposed soluto bsed o the elmto method espects the cluso pobbltes exctly, whch s ot tue fo Ohlsso s sequetl smplg. The mplemetto s especlly smple, sce the pogm povdes odeed sequece of occuptos to popose utl the objectve hs bee met. The estmto ssue s slghtly moe dffcult. Fo the uequl pobblty smplg wthout eplcemet, we must mke do wth heustc soluto. As well, t c be see tht, the secod stge, thee teds to be lowe cluso pobbltes etepses tht hve my occuptos. Ths should led us to select wth gete pobbltes the etepses tht my hve lge umbe of occuptos, to vod selectg occuptos wth pobbltes tht e too uequl. Ackowledgemets The utho wshes to thk Pee Lvllée fo submttg ths teestg poblem d povdg thoughtful commets o ele veso of ths tcle. The utho lso thks Audey-Ae Vllée fo he metculous poofedg, d efeee d wte of Suvey Methodology fo the petet emks, whch mde t possble to mpove ths tcle. Sttstcs Cd, Ctlogue No X

14 294 Tllé: Uequl pobblty vese smplg Appedx Lod smplg pckge, whch cots the fucto clusopobbltes(). lby(smplg) The fucto etus vecto wth the sequece umbes of the elmtos. The lst (esp. fst) ut elmted s the fst (esp. lst) compoet of the vecto. The fucto theefoe povdes the umbes of the uts to be peseted successvely fo the vese selecto. The gumet x s the vecto of vlues of the uxly vble used to clculte the cluso pobbltes. elmto<-fucto(x) { pkbx/sum(x) M legth(pkb) sum(pkb) sb ep(1, M) b ep(1, M) esep(0, M) fo ( 1:(M)) { clusopobbltes(pkb, M - ) v 1 - /b b p v * sb p cumsum(p) u uf(1) fo (j 1:legth(p)) f (u < p[j]) bek sb[j] 0 es[]j } es[m:1] } 500,000 smultos wth sze lst of sze M20. By tkg the fst m compoets of vecto v, we obt smple of sze m. M20 xuf(m) Pky(0,c(M,M)) Clculte the cluso pobbltes fo ll smple szes fom 1 to 20. fo( 1:M) Pk[,]clusopobbltes(x, ) owsums(pk) SIM50000 SSy(0,c(M,M)) fo( 1:SIM) { Sy(0,c(M,M)) velmto(x) fo( 1:M) S[,v[1:]]1 SSSS+S } SSSS/SIM Compe ctul d empcl cluso pobbltes. Pk SS SS-Pk Sttstcs Cd, Ctlogue No X

15 Suvey Methodology, Decembe Refeeces Chkkgoud, M.S. (1966). A ote o vese smplg wth equl pobbltes. Skhyā, A28, Chkkgoud, M.S. (1969). Ivese smplg wthout eplcemet. Austl Joul of Ststc, 11, Hájek, J. (1971). Dscusso of essy o the logcl foudtos of suvey smplg, pt o by D. Bsu. I Foudtos of Sttstcl Ifeece, (Eds., V.P. Godmbe d D.A. Spott), pge 326, Tooto, Cd. Holt, Reht, Wsto. Johso, N.L., Kemp, A.W. d Kotz, S. (2005). Uvte Dscete Dstbutos. New Yok: Joh Wley & Sos, Ic. Mkulsk, P.W., d Smth, P.J. (1976). A vce boud fo ubsed estmto vese smplg. Bometk, 63(1), Mlle, G.K., d Fdell, S.L. (2007). A fogotte dscete dstbuto? Revvg the egtve hypegeometc model. The Amec Sttstc, 61(4), Muthy, M.N. (1957). Odeed d uodeed estmtos smplg wthout eplcemet. Skhyā, 18, Ohlsso, E. (1995). Sequetl Posso smplg. Resech epot 182, Stockholm Uvesty, Swede. Ohlsso, E. (1998). Sequetl Posso smplg. Joul of Offcl Sttstcs, 14, Pthk, P.K. (1964). O vese smplg wth uequl pobbltes. Bometk, 51, Rosé, B. (1997). O smplg wth pobblty popotol to sze. Joul of Sttstcl Plg d Ifeece, 62, Sleh, M.M., d Sebe, G.A.F. (2001). A ew poof of Muthy s estmto whch pples to sequetl smplg. The Austl d New Zeld Joul of Sttstcs, 43, Smpfod, M.R. (1962). Methods of cluste smplg wth d wthout eplcemet fo clustes of uequl szes. Bometk, 49(1/2), Tllé, Y. (1996). A elmto pocedue of uequl pobblty smplg wthout eplcemet. Bometk, 83, Tllé, Y. (2006). Smplg Algothms. New Yok: Spge. Sttstcs Cd, Ctlogue No X