4.5 Multiple Imputation

Size: px

Start display at page:

Download "4.5 Multiple Imputation"

Elmer Norman
5 years ago
Views:

1 45 ultiple Imputatio Itroductio Assume a parametric model: y fy x; θ We are iterested i makig iferece about θ I Bayesia approach, we wat to make iferece about θ from fθ x, y = πθfy x, θ πθfy x, θdθ where πθ is a prior distributio, which is assumed to be kow for simplicity here The poit estimator is ˆθ = E{θ x, y} ad its variace estimator is ˆV = V {θ x, y} 2 We may express ˆθ = ˆθ x, y ad ˆV = ˆV x, y Now, cosider the case whe x is always observed ad y is subject to missigess Let y = y obs, y mis be the observed, missig part of the sample We have two approaches of makig iferece about θ usig the observed data x, y obs : Direct Bayesia approach: Cosider fθ x, y obs = πθfy obs x, θ πθfyobs x, θdθ

2 ad use ˆθ r = E{θ x, y obs } 3 ad its variace estimator is ˆV r = V {θ x, y obs } 4 2 ultiple imputatio approach: Compariso a For each k, geerate y k mis from fy mis x, y obs b Apply the k-th imputed values to ˆθ i to obtai ˆθ k Also, apply the k-th imputed values to ˆV i 2 to obtai ˆV x, y obs, y k mis c Combie the poit estimators to get as a poit estimator of θ ˆθ I = d The variace estimator of ˆθ I is where ˆV I = W + W = B = ˆθ k + B ˆV k ˆθ k θ I 2 Bayesia Frequetist odel Posterior distributio Predictio model flatet, θ data flatet data, θ Computatio Data augmetatio E algorithm Predictio I-step E-step Parameter update P-step -step Parameter est Posterior mode L estimatio Imputatio ultiple imputatio Fractioal imputatio Variace estimatio Rubi s formula Liearizatio or Bootstrap = ˆθ x, y obs, y k ˆV k = mis 2

3 2 ai Result Bayesia Properties Rubi, 987: For sufficietly large, we have ˆθ I = ˆθ k = Eθ x, y obs, y k mis = E{Eθ x, y obs, Y mis x, y obs } = Eθ x, y obs, which is equal to ˆθ r i 3 Also, for sufficietly large, ˆV I = W + B = ˆV k + ˆθ k θ 2 I = E{V θ x, y obs, Y mis x, y obs } + V {Eθ x, y obs, Y mis x, y obs }, which is equal to ˆV r i 4 2 Frequetist Properties Wag ad Robis, 998 Assume that, uder complete data, ˆθ is the LE of θ ad ˆV = {Iˆθ } is asymptotically ubiased for V ˆθ Uder the existece of missig data, ˆθ I is asymptotically equivalet to the LE of θ ad ˆV I is approximately ubiased for V ˆθ I That is, ˆθ I = ˆθLE 5 ad E{ ˆV I } = V ˆθ I 6 for sufficietly large ad See Appedix A for a sketched proof 3

4 3 Computatio Gibbs samplig Gema ad Gema 984: the Gibbs sampler for Bayesia image recostructio Taer ad Wog 987: data augmetatio for Bayesia iferece i geeric missig-data problems Gelfad ad Smith 990: simulatio of margial distributios by repeated draws from coditioals Idea for Gibbs samplig: Sample from coditioal distributios Give Z t = Z t, Z t 2,, Z t J, draw Z t+ by samplig from the full coditioals of f, Z t+ P Z t+ 2 P Z t+ J P Z Z t 2, Z t 3,, Z t J Z 2 Z t, Z t 3,, Z t J Z J Z t, Z t 2,, Z t J Uder mild regularity coditios, P Z t f as t Data augmetatio: Applicatio of the Gibbs samplig to missig data problem y = observed data z = missig data θ = model parameters Predictive distributio: P z y = Posterior distributio: P θ y = P z y, θ dp θ y P y y, z dp z y 4

5 Algorithm: Iterative method of data augmetatio I-step: Draw P-step: Draw z t+ P z y, θ t θ t+ P θ y, z t+ Two uses of data augmetatio Parameter simulatio: collect ad summarize a sequece of depedet draws of θ, θ t+, θ t+2,, θ t+n, where t is large eough to esure statioarity ultiple imputatio: collect idepedet draws of z, z t, z 2t,, z mt Parameter simulatio Bayesia approach θ = compoet of fuctio of θ of iterest Collect iterates of θ from data augmetatio θ t+, θ t+2,, θ t+n, where t is large eough to esure statioarity ad N is the ote Carlo sample size θ = N N N N θt+k estimates the posterior mea E θ y 2 θ t+k θ estimates the posterior variace V θ y The 25th ad 975th percetiles of θ t+, θ t+2,, θ t+n estimate the edpoits of a 95% equal-tailed Bayesia iterval for θ 5

6 4 Examples 4 Example Uivariate Normal distributio Let y,, y be IID observatios from Nµ, σ 2 ad oly the first r elemets are observed ad the remaiig r elemets are missig Assume that the respose mechaism is igorable Bayesia imputatio: the j-th posterior values of µ, σ 2 are geerated from σ j2 y r rˆσ r/χ 2 2 r 7 ad µ j y r, σ j2 N ȳ r, r σ j2 8 where y r = y,, y r, ȳ r = r r i= y i, ad ˆσ 2 r = r r i= y i ȳ r 2 Give the posterior sample µ j, σ j2, the imputed values are geerated from idepedetly for i = r +,, y j i y r, µ j, σ j2 N µ j, σ j2 9 Let θ = EY be the parameter of iterest ad the I estimator of θ ca be expressed as where The, ˆθ I = ȳ r + r ˆθ j I j= ˆθ I = { r = y i + i= j= µ j ȳ r + ˆθ j I i=r+ y j i } i=r+ j= y j i µ j 0 Asymptotically, the first term has mea µ ad variace r σ 2, the secod term has mea zero ad variace r/ 2 σ 2 /mr, the third term has mea zero ad variace σ 2 r/ 2 m, ad the three terms are mutually idepedet Thus, the variace of ˆθ I is V ˆθI = r σ2 + 2 r r σ2 + r σ2 6

7 For variace estimatio, ote that V y j i = V ȳ r + V µ j ȳ r + V y j i µ j = r σ2 + r + r + r σ2 + σ 2 r r = σ 2 Writig ˆV j I ˆθ = { i= ỹ j i ỹ j k } 2 i= 2 = ỹ j i µ where ỹi = δ i y i + δ i y j i, we have E { } j ˆV I ˆθ = { i= E = [σ 2 = σ 2 2 ỹ j i µ V { r σ2 + r ỹ j k µ ỹ j k 2 } 2 r σ2 + r σ2 }] which shows that EW = V ˆθ Also, EB m = V = V = = ˆθ I { r 2 r r ˆθ Cov I, 2 ˆθ I µ ȳ r + σ 2 σ 2 r + r Thus, Rubi s variace estimator satisfies { } E ˆVI ˆθ I = r σ2 + r which is cosistet with the geeral result i 6 i=r+ y i µ } 2 r σ2 + r σ2 = V ˆθI, 7

8 42 Example 2 Cesored regressio model, or Tobit model odel z i = x iβ + ɛ i ɛ i N 0, σ 2 y i = { zi if z i c i c i if z i < c i Data augmetatio I-step: Give θ t = β t, σ t, geerate the imputed value for δ i = 0 from with ɛ t i If δ i =, the z t+ i = z i 2 P-step: Give z t+ = z t+ i = x iβ t + ɛ t i φ s Φ [c i x i βt /σ t ] z t+, z t+ 2,, z t+ θ t+ P θ z t+, geerate θ t+ from That is, geerate σ 2t+ z t+ p ˆσ 2t+ /χ 2 p ad where ad β t+ z t+, σ 2t+ N ˆβt+, x i x i σ 2t+, ˆβ t+ = i= x i x i i= i= x i z t+ i ˆσ 2t+ = p z t+ I P x z t+ 8

9 43 Example 3 Bayesia bootstrap Noparametric approach to Bayesia imputatio First proposed by Rubi 98 Assume that a elemet of the populatio takes oe of the values d,, d K with probability p,, p K, respectively That is, we assume P Y = d k = p k, K p k = 2 Let y,, y be a IID sample from 2 ad let k be the umber of y i equal to d k The parameter is a vector of probabilities p = p,, p K, such that K i= p i = I this case, the populatio mea θ = EY ca be expressed as θ = K i= p id i ad we oly eed to estimate p If the improper Dirichlet prior with desity proportioal to K p k is placed o the vector p, the the posterior distributio of p is proportioal to K p k k which is a Dirichlet distributio with parameter,, K This posterior distributio ca be simulated usig idepedet uiform radom umbers Let u,, u be IID U0,, ad let g i = u i u i, i =, 2,, where u k is the k-th order statistic of u,, u with u 0 = 0 ad u = Partitio the g,, g ito K collectios, with the k-th oe havig k elemets, ad let p k be the sum of the g i i the k-th collectio The, the realized value of p,, p k follows a K -variate Dirichlet distributio with parameter,, K I particular, if K =, the g,, g is the vector of probabilities to attach to the data values y,, y i that Bayesia bootstrap replicatio To implemet Rubi s Bayesia bootstrap to multiple imputatio, assume that the first r elemets are observed ad the remaiig r elemets are missig The imputed values ca be geerated with the followig steps: 9

10 [Step ] From y r = y,, y r, geerate p r = p,, p r from the posterior distributio usig the Bayesia bootstrap as follows Geerate u,, u r idepedetly from U0, ad sort them to get 0 = u 0 < u < < u r < u r = 2 Compute p i = u i u i, i =, 2,, r ad p r = r i= p i [Step 2] Select the imputed value of y i by y with probability p yi = y r with probability p r idepedetly for each i = r +,, Rubi ad Scheker 986 proposed a approximatio of this Bayesia bootstrap method, called the approximate Bayesia boostrap ABB method, which provides a alterative approach of geeratig imputed values from the empirical distributio The ABB method ca be described as follows: [Step ] From y r = y,, y r, geerate a door set y r = y,, y r by bootstrappig That is, we select y with probability /r yi = y r with probability /r idepedetly for each i =,, r [Step 2] From the door set yr = y,, yr, select a imputed value of y i by y i = idepedetly for each i = r +,, y with probability /r yr with probability /r 0

11 Referece Gelfad, AE ad Smith, AF 990 Samplig-based approaches to calculatig margial desities, Joural of the America Statistical Associatio, 85, Gema, S ad Gema, D 984 Stochastic relaxatio, Gibbs distributios, ad the Bayesia restoratio of images IEEE Trasactios o Patter Aalysis ad achie Itelligece, 62, Rubi, D B 98 The Bayesia bootstrap The Aals of Statistics, 9, Rubi, D B 987 ultiple Imputatio for Norespose i Surveys, Joh Wiley & Sos, New York Rubi, DB ad Scheker, N 986 ultiple imputatio for iterval estimatio from simple radom samples with igorable orespose Joural of the America Statistical Associatio 8, Taer, A ad Wog, WH 987 The calculatio of posterior distributio by data augmetatio Joural of the America Statistical Associatio 82, Wag, N ad Robis, J 998 Large-sample theory for parametric multiple imputatio procedure, Biometrika, 85,

12 Appedix A Proof of 5 ad 6 Let S com θ = Sθ; x, y be the score fuctio of θ uder complete respose The LE uder complete respose, deoted by ˆθ, is asymptotically equivalet to where I com = E{ S com θ/ θ } Thus, ˆθ I = ˆθ k where y k mis = θ + I com = θ + I + I com com ˆθ = θ + I coms com θ, Sθ; x, y obs, y k mis E{Sθ; x, y obs, Y mis x, y obs ; θ k } [ ] Sθ; x, y obs, y k mis E{Sθ; x, y obs, Y mis x, y obs ; θ k }, fy mis x, y obs ; θ k ad θ k pθ x, y obs Uder some regularity coditios, the posterior distributio coverges to a ormal distributio with mea ˆθ LE ad variace I obs = V ˆθ LE This is ofte called Berstei vo ises theorem Thus, we ca apply Taylor liearizatio o Sθ; x, y obs, y k mis with respect to θ k aroud the true θ to get E{Sθ; x, y obs, Y mis x, y obs ; θ k } = E{Sθ; x, y obs, Y mis x, y obs ; θ} + I mis θ k θ = S obs θ + I mis ˆθ LE θ + I mis θ k ˆθ LE, 3 where I mis is the iformatio matrix associated with fy mis x, y obs ; θ Sice ˆθ LE is the solutio to S obs θ = 0, we have ad 3 further simplifies to ˆθ LE = θ + I obs S obsθ E{Sθ; x, y obs, Y mis x, y obs ; θ k } = S obs θ + I mis I obs S obsθ + I mis θ k ˆθ LE = I com I obs S obsθ + I mis θ k ˆθ LE 4 2

13 Thus, combiig all terms together, we have ˆθ k = ˆθ LE + I comi mis θ k ˆθ LE + I com { S k ES x, y obs ; θ k } 5 where S k = Sθ; x, y obs, y k mis Therefore, ˆθ I = ˆθ LE + IcomI mis θ k ˆθ LE + I com = ˆθ LE + I comi mis { +I com [ ] Sθ; x, y obs, y k mis E{Sθ; x, y obs, Y mis x, y obs ; θ k } θ k ˆθ LE } { S k θ ES x, y obs ; θ k } Note that the secod term reflects the variability due to geeratig θ ad the third therm reflects the variability due to geeratig y k mis three terms are idepedet ad so we obtai from fy mis x, y obs ; θ k The V {ˆθ I } = V ˆθ LE + I comi mis I obs I misicom + I comi mis Icom 6 The last two terms are egligible for large To prove 6, ote first that EV = V ˆθ = I com ad so we have EW = I com Now, isertig 5 ito we have B = EB = V ˆθ ˆθ k ˆθ I 2, Covˆθ 2, ˆθ = I comi mis I obs I misi com + I comi mis I com = I obs I com, where the last equality follows from A + BCB = A A BCB A + A BCB A + BCB BCB A with A = I com, B = I, ad C = I mis Therefore, E ˆV I = I obs = V ˆθ LE = V ˆθ I 3

5. Fractional Hot deck Imputation

5. Fractional Hot deck Imputation 5. Fractioal Hot deck Imputatio Itroductio Suppose that we are iterested i estimatig θ EY or eve θ 2 P ry < c where y fy x where x is always observed ad y is subject to missigess. Assume MAR i the sese