Dscusson of Extensons of the Gauss-arkov Theorem to the Case of Stochastc Regresson Coeffcents Ed Stanek Introducton Pfeffermann (984 dscusses extensons to the Gauss-arkov Theorem n settngs where regresson coeffcents are stochastc. In the process, he comments on errors n prevously publshed developments of theorems, partcularly on Porter s (973 paper that deals wth estmaton of cluster means n a two stage sample settng. Pfeffermann (984, p42 suggests that estmators of the type dscussed by Porter do not exst, and attrbutes the error to not recognzng the stochastc nature of the regresson coeffcents. We dscuss these results relatve to predctors of realzed random effects n fnte balanced populatons wth clustered samples by Stanek and Snger (es02ed3v6.doc. Comment by the Revewer The context s a two stage cluster sample, where we assume that the observatons on a selected unt n a selected cluster ncludes response error. In ths context, the revewer s dscusson s as follows: Porter s paper and Pfeffermann s dscusson of t may be especally pertnent to the present manuscrpt. In partcular, the 2 nd paragraph on page 42 of Pfeffermann s (984 work seems to suggest thtat a SLUE of the latent cluster means may not exst, unless one makes an nference condtonal on the realzed PSUs and SSUs. Revew of odels and otaton for Pfeffermann and Stanek and Snger We revew the model and notaton used by Pfeffermann (984 and Stanek and Snger (2003. Frst, we present the model and assumptons gven by Pfeffermann. We refer to equatons n the models usng the equaton numbers gven by Pfeffermann (or Stanek and Snger, precedng the equaton numbers by P or S to desgnate the authors. We use the notaton used by each author to make the correspondence of ths dscusson to ther papers transparent. Pfeffermann s odel Pfeffermann assumes a lnear model of the form Y = X β + ε (P2. k k where X s a fxed desgn matrx, and β s a vector of random coeffcents such that E ( β = A γ, E ( β Aγ ( β Aγ =, E k l ( εβ = 0, E ( ε = 0, and var ε = Σ. C03ed6.doc 6/2/2003 0:2 A
Pfeffermann descrbes ths model as a model for =,..., domans, where the model for the th doman s gven by Y = X β + ε. (P2.7 k easures are made on all domans. There s no fnte populaton samplng. We can express ths model n a slghtly dfferent manner. Usng E ( β = Aγ, we can express equaton (P2. as Y = XAγ+ X( β Aγ + ε = XAγ +µ (P3. X A + E = 0 var µ = X X + Σ = Λ. where µ= ( β γ ε. For equaton (P3., ( µ and Three nterpretatons are gven to the model (whch we do not dscuss n detal. The nterpretatons correspond to a random coeffcent model, a statonary stochastc process model, and a arkovan Kalman flter model. Pfeffermann focuses on the problem of predctng lnear functons gven by w ( γ,β = w γ + w2β by lnear unbased estmators of the form T( Y = t0 + t Y that mnmze the SE. The unbased requrement s defned by Eξ T( Y w( γ,β = 0. (P2.5 The SE s gven by E T w 2 ξ Y γ,β. (P2.6 If we defne the WLS estmator as γ ˆ = ( XA Λ ( XA ( XA Λ Y (P3.3 (the WLS estmate, then (referencng Harvlle (976, the best lnear unbased predctor s gven by ŵ( γ,β = w ˆ ˆ γ + w 2Aγ + w2 XΛ Y ( XA γ ˆ. (P5.2 Pfeffermann dscusses ths predctor relatve to the rank of A, ndcatng that ths s mportant. We postpone further dscusson of ths pont, but note that Pfeffermann relates ths dscusson to papers by Duncan and Horn (972, and Chpman (964 and Rao (965. We also note that Pfeffermann also develops an extenson of ths theorem that appears to be related to what we have done, but that we don t explore further here. Porter s odel (as descrbed by Pfeffermann Pfeffermann dscusses Porter s (973 model and concludes that there s a problem wth Porter s development of a predctor. Porter s problem s predcton n the context of a model for a sngle stage sample. Snce the problem addressed by Stanek and Snger also consders samplng, the problem noted by Pfeffermann wth Porter s results s relevant to the results of Stanek and Snger. We recount Pfeffermann s summary of Porter s problem. Smlar to equaton (P2.7, Pfeffermann descrbes Porter s model for a sample of =,..., unts (.e. clusters, where the model for the th unt s gven by C03ed6.doc 6/2/2003 0:2 A 2
Y = Xβ + ε. In ths model, the coeffcents β are random as s the desgn matrx, Pfeffermann dstngushes these coeffcents from the fxed coeffcents for a partcular unt by denotng the coeffcents for a gven unt by β and the desgn matrx for a partcular unt by X. Over samplng, E ( β = β, and hence the model can be expressed as Y = X β + X ( β β + ε, = X β+µ µ = X β β + ε. where Suppose we defne X = X. m X, and concatenate the vectors for response, and error = for the unts. Then smlar to Pfeffermann s model (P3., Y = Xβ + X( β β + ε. = Xβ+µ j Assumng X X the matrx X s stochastc. Pfeffermann notes that for ths reason, the usual Gauss-arkov theorem can not be appled to the model. Pfeffermann concludes that n order to apply the Gauss-arkov theorem, t s necessary to condton j on X. Wth the further assumpton that β β for j, ( k E = Xk ( βk β Thus, f one condtons on X, then ( X Xk Ε µ X = X = X β β + ε X = X k. 0 Ε µ = 0 whch volates the assumpton of the model. For ths reason, Pfeffermann concludes that Porter s results are not vald. Pfeffermann s odel for Random Coeffcents We consder Pfeffermann s model, as he dscusses t for random coeffcents (on page 40, snce the model s closely related to Porter s model. The purpose of the dscusson s to see why Pfeffermann clams that hs model does not suffer from the same problem as Porter s model. The stochastc regresson model gven by Pfeffermann s composed of dstnct regresson models gven by Y = X β + ε. (P2.7 K0 K0 whch get concatenated to form the model Y = X β + ε (P2. k k where Pfeffermann states that X = X s a known explanatory varable matrx, and 2 = β = β β β s a vector of unobservable stochastc coeffcents. It appears that as stated, the matrx X defned by Pfeffermann suffers from the same problem as Porter s desgn matrx snce f β s stochastc, the accompanyng X s also stochastc. C03ed6.doc 6/2/2003 0:2 A 3
The specal cases consdered by Pfeffermann, however, smplfy ths problem. The frst model, referred to as a random coeffcent model, assumes that A = I. K 0 Wth ths assumpton, E ( β = Aγ = ( IK 0 γ. Then XA = X ( I K or = 0 K0 XA = X X 2 X, whch stll s stochastc, snce t depends on the K0 2 K0 K0 orderng of the selectons of. In two specal cases, ths matrx wll not be stochastc. One case corresponds to the settng where X =. A second case correspond to the settng where for all =,...,, = 0 and X = X 0. We also note that other settngs are possble where X s non-stochastc, and yet β s stochastc. Consder for example a populaton of schools, where response s the proporton of students n a school that pass a test on a year. Suppose the schools are lsted n order, and that certan schools are measured, but that a randomly selected year s th selected. Introducng the random varable U that ndcates when the k selected year s year t, we can represent β β U β, where the subscrpt k s dropped when kt k kt t t= T = = the range s smply k=, β k s stochastc, and β t s non-stochastc. It seems that ths settng would correspond to the settng descrbed by Pfeffermann. The key to ths nterpretaton s an addtonal dmenson (tme for the stochastc regresson coeffcents n Pfeffermann s model. Stanek and Snger s odel Stanek and Snger develop a model for a clustered populaton wth equal sze clusters. As an example, they represent the non-stochastc response for unt t n cluster s as y st = µ + βs + εst or y = µ + β + ε whch we summarze for the populaton as s s s y = Xµ + Zβ + ε, (S2.4 where X = and Z = I. ote that all terms n ths equaton are non-stochastc. A stochastc model s ntroduced by ncludng samplng of clusters and unts n clusters, and addng response error. The resultng model s gven by Y * = Xµ + ZB+ ( E+ W * (S2.3 where B= Uβ. (S2.2 In the model (S2.3, the matrces X and Z are fxed. C03ed6.doc 6/2/2003 0:2 A 4
The process of ntroducng the random varables that result n model (S2.3 from model (S2.4 nvolve ntroducng samplng, and addng response error. Snce the addton of response error does not mpact the matrces X and Z, we consder the smpler model gven by Y = Xµ + ZB+ E. ovng from model (S2.4 to ths model nvolves pre-multplyng the terms n the model (S2.4 by the samplng ndcator matrces, such that Y= ( U I U y. (S2.9 Pfeffermann s crtcsm of Porter s results apples smlarly n Stanek and Snger s settng. The queston smplfes to whether the expressons ( U I U X and ( U I U Z are stochastc, or non-stochastc. For the problem consder by Stanek and Snger, snce ( U I U X= X, the frst term s non-stochastc. The second term smplfes to ( U I U Z= U = ZU where Z s non-stochastc, but U s stochastc. Usng (S2.2, the random porton of ths product s assocated wth the random coeffcents, llustratng that Z remans non-stochastc. Thus, Pfeffermann s crtcsm of Porter s results does not apply to the results of Stanek and Snger. Extenson of Stanek and Snger s odel We brefly consder an nterestng extenson of Stanek and Snger s model. Suppose that y = Z β + ε s s s s where all terms n ths model are non-stochastc. Assemblng all clusters, we express the non-stochastc model as y = Zs β + ε ( where β= ( β β β 2 = s β. Ths model s more general than model (S2.4. We now add samplng to the model by pre-multplyng terms n the model by ( U I U. We focus our attenton on the expresson U I U Z, expand ths ( s s expresson, and consder settngs where t can be parttoned smlarly to model (S2.3. Frst, note that C03ed6.doc 6/2/2003 0:2 A 5
s ( s U I U Z = U I U Z ( ( 2 ( UU Z U2U Z2 U U Z ( ( 2 ( U U Z U U Z U U Z U ( U Z U ( 2 U Z U ( U Z 2 22 2 = 22 2 s Suppose that Zs = zs for all s =,...,. Then snce U =, zu zu 2 2 zu 2 2 22 ( zu zu z U U I U Zs = zu zu 2 22 zu = UD z z 0 0 0 z2 0 where D z =. We can express ths equvalently as 0 0 z U I U Zs = I UD = Z UD z z where Z = I. Usng ths result, we can ntroduce samplng n model ( to result n Y = Z( UDz β + E. Let * * * * β = D z β where elements of β are zsβ s, and B = Uβ. Then the samplng model * s equvalent to Y = ZB + E. We wsh to express ths model as a mxed model. To do so, let us defne zsβs * β = and let X =. The model s then gven by ( β Y = Xβ * + ZB * X * + E. As a second applcaton, suppose that Z s has dmenson K0 for all s =,...,. Also, suppose that there s only one stage of samplng. Ths s equvalent to ( s settng U = I for all s =,...,. Wth these assumptons, and expressng β= β we can express s. C03ed6.doc 6/2/2003 0:2 A 6
s s ( s s U I U Z β= β = U I Z β. ow note that (( ( 2 2 Zsβs = Z β Z β Zβ s s s U 2sZsβs sβs U I Z = express U U U Z β Z β s s s. Then, or equvalently. Fnally, note that we can ( Z ( Z vec vec. Then vec( Z 2 szsβs = U sβs I Usβ s I vec ( Z U2sβ s vec 2 I Z sβ s U I Z = whch s equvalent to vec ( Z U sβs I Usβ s vec ( Z U2s s vec( 2 ( β Z U I Zsβs = I. vec ( Z U s s β Usng ths result n the samplng model that arses from y = Z β + ε, for s =,...,, whch we summarze by ( as y = Zs β + ε, whch we express as s s s s K0 ( U I y = ( U I Z β + ( U I s ε C03ed6.doc 6/2/2003 0:2 A 7
Usβ s vec( Z U2sβ s vec( Z2 Y = I + E. vec( Z U s s β Ths model s lke a mxed model where the random effects can once agan be separated from the random effects. C03ed6.doc 6/2/2003 0:2 A 8
References Chpman, J.S. (964. On tleast squares wth nsuffcent observatons, Journal of the Amercan Statstcal Assocaton, 59:078-. Duncan, D.B. and Horn, S.D. (972. Lnear dynamc recursve estmaton from the vewpont of regresson analyss, Journal of the Amercan Statstcal Assocaton, 67:85-822. Harvlle, D. (976. Extenson of the Gauss arkov theorem to nclude the estmaton of random effects, Annals of Statstcs, 4:384-396. Pfeffermann, D. (984. On extensons of the Gauss-arkov Theorem to the case of stochastc regresson coeffcents, Journal of the Royal Statstcal Socety B, 46, o.:39-48. Porter, R.. (973. On the use of survey sample weghts n the lnear model, Annals of Economc and Socal easurement, 2:4-58. Rao, C.R. (965. The theory of least squares when the parameters are stochastc and ts applcaton to the analyss of growth curves, Bometrka, 52:447-458. C03ed6.doc 6/2/2003 0:2 A 9