A note on multiple imputation for general purpose estimation
|
|
- Opal Owens
- 5 years ago
- Views:
Transcription
1 A note on multiple imputation for general purpose estimation Shu Yang Jae Kwang Kim SSC meeting June 16, 2015 Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
2 Introduction Basic Setup Assume simple random sampling, for simplicity. Under complete response, suppose that ˆη n,g = n 1 n g(y i ) is an unbiased estimator of η g = E{g(Y )}, for known g( ). δ i = 1 if y i is observed and δ i = 0 otherwise. y i : imputed value for y i for unit i with δ i = 0. Imputed estimator of η g ˆη I,g = n 1 n i=1 i=1 {δ i g(y i ) + (1 δ i )g(y i )} Need E {g(y i ) δ i = 0} = E {g(y i ) δ i = 0}. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
3 Introduction ML estimation under missing data setup Often, find x (always observed) such that Missing at random (MAR) holds: f (y x, δ = 0) = f (y x) Imputed values are created from f (y x). Computing the conditional expectation can be a challenging problem. 1 Do not know the true parameter θ in f (y x) = f (y x; θ): E {g (y) x} = E {g (y) x; θ}. 2 Even if we know θ, computing the conditional expectation can be numerically difficult. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
4 Introduction Imputation Imputation: Monte Carlo approximation of the conditional expectation (given the observed data). E {g (y i ) x i } = 1 m m ( g j=1 y (j) i ) 1 Bayesian approach: generate yi from f (y i x i, y obs ) = f (y i x i, θ) p(θ x i, y obs )dθ 2 Frequentist approach: generate yi consistent estimator. from f ( y i x i ; ˆθ ), where ˆθ is a Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
5 Introduction Basic Setup (Cont d) Thus, imputation is a computational tool for computing the conditional expectation E{g(y i ) x i } for missing unit i. To compute the conditional expectation, we need to specify a model f (y x; θ) evaluated at θ = ˆθ. Thus, we can write ˆη I,g = ˆη I,g (ˆθ). To estimate the variance of ˆη I,g, we need to take into account of the sampling variability of ˆθ in ˆη I,g = ˆθ I,g (ˆθ). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
6 Introduction Basic Setup (Cont d) Three approaches Bayesian approach: multiple imputation by Rubin (1978, 1987), Rubin and Schenker (1986), etc. Resampling approach: Rao and Shao (1992), Efron (1994), Rao and Sitter (1995), Shao and Sitter (1996), Kim and Fuller (2004), Fuller and Kim (2005). Linearization approach: Clayton et al (1998), Shao and Steel (1999), Robins and Wang (2000), Kim and Rao (2009). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
7 Comparison Bayesian Frequentist Model Posterior distribution Prediction model f (latent, θ data) f (latent data, θ) Computation Data augmentation EM algorithm Prediction I-step E-step Parameter update P-step M-step Parameter est n Posterior mode ML estimation Imputation Multiple imputation Fractional imputation Variance estimation Rubin s formula Linearization or Bootstrap Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
8 MultipleImputation Step 1. (Imputation) Create M complete datasets by filling in missing values with imputed values generated from the posterior predictive distribution. To create the jth imputed dataset, first generate θ (j) from the posterior distribution p(θ X n, y obs ), and then generate y (j) i from the imputation model f (y x i ; θ (j) ) for each missing y i. Step 2. (Analysis) Apply the user s complete-sample estimation procedure to each imputed dataset. Let ˆη (j) be the complete-sample estimator of η = E{g(Y )} applied to the jth imputed dataset and ˆV (j) be the complete-sample variance estimator of ˆη (j). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
9 Multiple Imputation Step 3. (Summarize) Use Rubin s combining rule to summarize the results from the multiply imputed datasets. The multiple imputation estimator of η, denoted by ˆη MI, is ˆη MI = 1 M M j=1 ˆη (j) I Rubin s variance estimator is ˆV MI (ˆη MI ) = W M + ( ) B M, M where W M = M 1 M j=1 ˆV (j) and B M = (M 1) 1 M j=1 (ˆη(j) I ˆη MI ) 2. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
10 Multiple imputation Motivating Example Suppose that you are interested in estimating η = P(Y 3). Assume a normal model for f (y x; θ) for multiple imputation. Two choices for ˆη n : 1 Method-of-moments (MOM) estimator: ˆη n1 = n 1 n i=1 I (y i 3). 2 Maximum-likelihood estimator: ˆη n2 = n 1 n P(Y 3 x i ; ˆθ), i=1 where P(Y 3 x i ; ˆθ) = 3 f (y x i; ˆθ)dy. Rubin s variance estimator is nearly unbiased for ˆη n2, but provide conservative variance estimation for ˆη n1 (30-50% overestimation of the variances in most cases). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
11 Introduction The goal is to 1 understand why MI provide biased variance estimation for MOM estimation. 2 characterize the asymptotic bias of MI variance estimator when MOM estimator is used in the complete-sample analysis; 3 give an alternative variance estimator that can provide asymptotically valid inference for MOM estimation. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
12 Multiple imputation Rubin s variance estimator is based on the following decomposition, var(ˆη MI ) = var(ˆη n ) + var(ˆη MI, ˆη n ) + var(ˆη MI ˆη MI, ), (1) where ˆη n is the complete-sample estimator of η and ˆη MI, is the probability limit of ˆη MI for M. Under some regularity conditions, W M term estimates the first term, the B M term estimates the second term, and the M 1 B M term estimates the last term of (1), respectively. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
13 Multiple imputation In particular, Kim et al (2006, JRSSB) proved that the bias of Rubin s variance estimator is Bias( ˆV MI ) = 2cov(ˆη MI ˆη n, ˆη n ). (2) The decomposition (1) is equivalent to assuming that cov(ˆη MI ˆη n, ˆη n ) = 0, which is called the congeniality condition by Meng (1994). The congeniality condition holds when ˆη n is the MLE of η. In such cases, Rubin s variance estimator is asymptotically unbiased. If method of moment (MOM) estimator is used to estimate η = E{g(Y )}, Rubin s variance estimator can be asymptotically biased. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
14 MI for MOM estimator: No x variable Assume that y is observed for the first r elements. MI for MOM estimator of η = E{g(Y )}: ˆη MI = 1 n r g(y i ) n M i=1 n M i=r+1 j=1 g(y (j) i ) where y (j) i are generated from f (y θ (j) ) and θ (j) are generated from p(θ y obs ). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
15 MI for MOM estimator: No x variable (Cont d) Since, conditional on the observed data 1 p lim M M M j=1 g(y (j) i ) = E [E {g(y ); θ } y obs ] = E{g(Y ); ˆθ} and the MI estimator of η (for M ) can be written ˆη MI = r n ˆη MME,r + n r n ˆη MLE,r. Thus, ˆη MI is a convex combination of ˆη MME,r and ˆη MLE,r, where ˆη MLE,r = E{g(Y ); ˆθ} and ˆη MME,r = r 1 r i=1 g(y i). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
16 MI for MOM estimator: No x variable (Cont d) Since Bias( ˆV MI ) = 2Cov(ˆη MME,n, ˆη MI ˆη MME,n ), we have only to evaluate the covariance term. Writing ˆη MME,n = p ˆη MME,r + (1 p) ˆη MME,n r, where p = r/n, we can obtain Cov(ˆη MME,n, ˆη MI ˆη MME,n ) = Cov{ˆη MME,n, (1 p)(ˆη MLE,r ˆη MME,n r )} = p(1 p)cov{ˆη MME,r, ˆη MLE,r } (1 p) 2 V {ˆη MME,n r } = p(1 p) {V (ˆη MLE,r ) V (ˆη MME,r )}, which proves Bias( ˆV MI ) 0. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
17 Bias of Rubin s variance estimator Theorem Let ˆη n = n 1 n i=1 g(y i) be the method of moments estimator of η = E{g(Y )} under complete response. Assume that E( ˆV (j) ) = var(ˆη n ) holds for j = 1,..., M. Then for M, the bias of Rubin s variance estimator is Bias( ˆV ( ) MI ) = 2n 1 (1 p) E [var{g(y ) X } δ = 0] ṁθ,0 T I 1 θ ṁ θ,1, where p = r/n, I θ = E{ 2 log f (Y X ; θ)/ θ θ T }, m(x; θ) = E{g(Y ) x; θ}, ṁ θ (x) = m(x; θ)/ θ, ṁ θ,0 = E{ṁ θ (X ) δ = 0}, and ṁ θ,1 = E{ṁ θ (X ) δ = 1}. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
18 Bias of Rubin s variance estimator Under MCAR, the bias simplifies to Bias( ˆV MI ) = 2p(1 p){var(ˆη r,mme ) var(ˆη r,mle )}, where ˆη r,mme = r 1 r i=1 g(y i) and ˆη r,mle = r 1 r i=1 E{g(Y ) x i; ˆθ}. This shows that Rubin s variance estimator is unbiased if and only if the method of moments estimator is as efficient as the maximum likelihood estimator, that is, var(ˆη r,mme ) = var(ˆη r,mle ). Otherwise, Rubin s variance estimator is positively biased. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
19 Rubin s variance estimator can be negatively biased Now consider a simple linear regression model which contains one covariate X and no intercept. By Theorem 1, Bias( ˆV MI ) = 2(1 p)σ2 n { 1 which can be zero, positive or negative. If } E(X δ = 0)E(X δ = 1) E(X 2, δ = 1) E(X δ = 0) > E(X 2 δ = 1) E(X δ = 1) the Rubins variance estimator can be negatively biased. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
20 Alternative variance estimation Decompose where and var (ˆη MI, ) = n 1 V 1 + r 1 V 2, V 1 = var{g(y )} (1 p)e[var{g(y ) X } δ = 0], V 2 = ṁθ T I 1 θ ṁ θ p 2 ṁθ,1 T I 1 θ ṁ θ,1. The first term n 1 V 1, is the variance of the sample mean of δ i g(y i ) + (1 δ i )m(x i ; θ). The second term r 1 V 2, reflects the variability associated with the estimated value of θ instead of the true value θ in the imputed values. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
21 Alternative variance estimation Estimate n 1 V 1 The first term n 1 V 1 can be estimated by W M = W M C M, where W M = M 1 M j=1 ˆV (j), and C M = 1 n 2 (M 1) M n k=1 i=r+1 { g(y (k) i ) 1 M since E{W M } = n 1 var{g(y )} and E(C M ) = n 1 (1 p)e[var{g(y ) X } δ = 0]. M k=1 g(y (k) i )} 2. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
22 Alternative variance estimation Estimate r 1 V 2 To estimate the second term term, we use over-imputation: Table: Over-Imputation Data Structure Row 1... M average 1 g (1) 1... g (M) 1 ḡ1. r r + 1. n average... gr (1)... gr (M) ḡr g (1) r+1... g (M) r+1 ḡr+1.. gn (1)... gn (M) η n (1)... η n (M). ḡ n η n Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
23 Alternative variance estimation Estimate r 1 V 2 The second term r 1 V 2 can be estimated by D M = D M,n D M,r, where D M,n = (M 1) 1 D M,r = (M 1) 1 with d (k) i M k=1 M k=1 (n 1 n i=1 (n 1 r i=1 d (k) i d (k) i ) 2 (M 1) 1 ) 2 (M 1) 1 = g(y (k) i ) M 1 M (l) l=1 g(yi ), since E(D M,n ) = r 1 ṁ T θ I 1 θ M k=1 M k=1 n 2 n 2 n i=1 r i=1 (d (k) i ) 2, (d (k) i ) 2. ṁ θ and E(D M,r ) = r 1 p 2 ṁθ,1 T I 1 θ ṁ θ,1. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
24 Alternative variance estimation Theorem Under the assumptions of Theorem 1, the new multiple imputation variance estimator is ˆV MI = W M + D M + M 1 B M, where W M = W M C M, and B M being the usual between-imputation variance. ˆV MI is asymptotically unbiased for estimating the variance of the multiple imputation estimator as n. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
25 Simulation study Set up 1 Samples of size n = 2, 000 are independently generated from Y i = βx i + e i, where β = 0.1, X i exp(1) and e i N(0, σ 2 e) with σ 2 e = 0.5. Let δ i be the response indicator of y i and δ i Bernoulli(p i ), where p i = 1/{1 + exp( φ 0 φ 1 x i )}. We consider two scenarios: (i) (φ 0, φ 1 ) = ( 1.5, 2) and (ii) (φ 0, φ 1 ) = (3, 3) The parameters of interest are η 1 = E(Y ) and η 2 = pr(y < 0.15). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
26 Simulation study Result 1 Table: Relative biases of two variance estimators and mean width and coverages of two interval estimates under two scenarios in simulation one Relative bias Mean Width Coverage (%) for 95% C.I. for 95% C.I. Scenario Rubin New Rubin New Rubin New 1 η η η η C.I., confidence interval; η 1 = E(Y ); η 2 = pr(y < 0.15). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
27 Simulation study Result 1 For η 1 = E(Y ), under scenario (i), the relative bias of Rubin s variance estimator is 96.8%, which is consistent with our result with E 1 (X 2 ) E 0 (X )E 1 (X ) > 0, where E 1 (X 2 ) = 3.38, E 1 (X ) = 1.45, and E 0 (X ) = Under scenario (ii), the relative bias of Rubin s variance estimator is 19.8%, which is consistent with our result with E 1 (X 2 ) E 0 (X )E 1 (X ) < 0, where E 1 (X 2 ) = 0.37, E 1 (X ) = 0.47, and E 0 (X ) = The empirical coverage for Rubin s method can be over or below the nominal coverage due to variance overestimation or underestimation. On the other hand, the new variance estimator is essentially unbiased for these scenarios. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
28 Simulation study Set up 2 Samples of size n = 200 are independently generated from Y i = β 0 + β 1 X i + e i, where β = (β 0, β 1 ) = (3, 1), X i N(2, 1) and e i N(0, σ 2 e) with σ 2 e = 1. The parameters of interest are η 1 = E(Y ) and η 2 = pr(y < 1). We consider two different factors: 1 The response mechanism: MCAR and MAR: For MCAR, δ i Bernoulli(0.6). For MAR, δ i Bernoulli(p i ), where p i = 1/{1 + exp( x i )} with the average response rate about The size of multiple imputation, with two levels M = 10 and M = 30. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
29 Simulation study Result 2 Table: Relative biases of two variance estimators and mean width and coverages of two interval estimates under two scenarios of missingness in simulation two Relative Bias Mean Width Coverage (%) for 95% C.I. for 95% C.I. M Rubin New Rubin New Rubin New Missing completely at random η η Missing at random η η C.I., confidence interval; η 1 = E(Y ); η 2 = pr(y < 1). Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
30 Simulation study Result 2 Both Rubin s variance estimator and our new variance estimator are unbiased for η 1 = E(Y ). Rubin s variance estimator is biased upward for η 2 = pr(y < 1), with absolute relative bias as high as 24%; whereas our new variance estimator reduces absolute relative bias to less than 1.74%. For Rubin s method, the empirical coverage for η 2 = pr(y < 1) reaches to 98% for 95% confidence intervals, due to variance overestimation. In contrast, our new method provides more accurate coverage of confidence interval for both η 1 = E(Y ) and η 2 = pr(y < 1) at 95% levels. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
31 Conclusion Investigate asymptotic properties of Rubin s variance estimator. If method of moment is used, Rubin s variance estimator can be asymptotically biased. New variance estimator, based on multiple over-imputation, can provide valid variance estimation in this case. Our method can be extended to a more general class of parameters obtained from estimating equations. n U(η; x i, y i ) = 0. i=1 This is a topic of future study. Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
32 The end Shu Yang, Jae Kwang Kim Multiple Imputation June 16, / 32
Fractional Imputation in Survey Sampling: A Comparative Review
Fractional Imputation in Survey Sampling: A Comparative Review Shu Yang Jae-Kwang Kim Iowa State University Joint Statistical Meetings, August 2015 Outline Introduction Fractional imputation Features Numerical
More informationFractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling
Fractional Hot Deck Imputation for Robust Inference Under Item Nonresponse in Survey Sampling Jae-Kwang Kim 1 Iowa State University June 26, 2013 1 Joint work with Shu Yang Introduction 1 Introduction
More informationShu Yang and Jae Kwang Kim. Harvard University and Iowa State University
Statistica Sinica 27 (2017), 000-000 doi:https://doi.org/10.5705/ss.202016.0155 DISCUSSION: DISSECTING MULTIPLE IMPUTATION FROM A MULTI-PHASE INFERENCE PERSPECTIVE: WHAT HAPPENS WHEN GOD S, IMPUTER S AND
More informationParametric fractional imputation for missing data analysis
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 Biometrika (????),??,?, pp. 1 15 C???? Biometrika Trust Printed in
More informationRecent Advances in the analysis of missing data with non-ignorable missingness
Recent Advances in the analysis of missing data with non-ignorable missingness Jae-Kwang Kim Department of Statistics, Iowa State University July 4th, 2014 1 Introduction 2 Full likelihood-based ML estimation
More informationStatistical Methods for Handling Missing Data
Statistical Methods for Handling Missing Data Jae-Kwang Kim Department of Statistics, Iowa State University July 5th, 2014 Outline Textbook : Statistical Methods for handling incomplete data by Kim and
More informationLikelihood-based inference with missing data under missing-at-random
Likelihood-based inference with missing data under missing-at-random Jae-kwang Kim Joint work with Shu Yang Department of Statistics, Iowa State University May 4, 014 Outline 1. Introduction. Parametric
More informationStatistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach
Statistical Methods for Handling Incomplete Data Chapter 2: Likelihood-based approach Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Observed likelihood 3 Mean Score
More informationChapter 4: Imputation
Chapter 4: Imputation Jae-Kwang Kim Department of Statistics, Iowa State University Outline 1 Introduction 2 Basic Theory for imputation 3 Variance estimation after imputation 4 Replication variance estimation
More informationAn Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data
An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data Jae-Kwang Kim 1 Iowa State University June 28, 2012 1 Joint work with Dr. Ming Zhou (when he was a PhD student at ISU)
More information6. Fractional Imputation in Survey Sampling
6. Fractional Imputation in Survey Sampling 1 Introduction Consider a finite population of N units identified by a set of indices U = {1, 2,, N} with N known. Associated with each unit i in the population
More informationMiscellanea A note on multiple imputation under complex sampling
Biometrika (2017), 104, 1,pp. 221 228 doi: 10.1093/biomet/asw058 Printed in Great Britain Advance Access publication 3 January 2017 Miscellanea A note on multiple imputation under complex sampling BY J.
More informationarxiv:math/ v1 [math.st] 23 Jun 2004
The Annals of Statistics 2004, Vol. 32, No. 2, 766 783 DOI: 10.1214/009053604000000175 c Institute of Mathematical Statistics, 2004 arxiv:math/0406453v1 [math.st] 23 Jun 2004 FINITE SAMPLE PROPERTIES OF
More informationPlausible Values for Latent Variables Using Mplus
Plausible Values for Latent Variables Using Mplus Tihomir Asparouhov and Bengt Muthén August 21, 2010 1 1 Introduction Plausible values are imputed values for latent variables. All latent variables can
More informationData Integration for Big Data Analysis for finite population inference
for Big Data Analysis for finite population inference Jae-kwang Kim ISU January 23, 2018 1 / 36 What is big data? 2 / 36 Data do not speak for themselves Knowledge Reproducibility Information Intepretation
More informationIntroduction An approximated EM algorithm Simulation studies Discussion
1 / 33 An Approximated Expectation-Maximization Algorithm for Analysis of Data with Missing Values Gong Tang Department of Biostatistics, GSPH University of Pittsburgh NISS Workshop on Nonignorable Nonresponse
More informationTwo-phase sampling approach to fractional hot deck imputation
Two-phase sampling approach to fractional hot deck imputation Jongho Im 1, Jae-Kwang Kim 1 and Wayne A. Fuller 1 Abstract Hot deck imputation is popular for handling item nonresponse in survey sampling.
More informationOn the bias of the multiple-imputation variance estimator in survey sampling
J. R. Statist. Soc. B (2006) 68, Part 3, pp. 509 521 On the bias of the multiple-imputation variance estimator in survey sampling Jae Kwang Kim, Yonsei University, Seoul, Korea J. Michael Brick, Westat,
More informationNonresponse weighting adjustment using estimated response probability
Nonresponse weighting adjustment using estimated response probability Jae-kwang Kim Yonsei University, Seoul, Korea December 26, 2006 Introduction Nonresponse Unit nonresponse Item nonresponse Basic strategy
More informationRobustness to Parametric Assumptions in Missing Data Models
Robustness to Parametric Assumptions in Missing Data Models Bryan Graham NYU Keisuke Hirano University of Arizona April 2011 Motivation Motivation We consider the classic missing data problem. In practice
More informationCombining data from two independent surveys: model-assisted approach
Combining data from two independent surveys: model-assisted approach Jae Kwang Kim 1 Iowa State University January 20, 2012 1 Joint work with J.N.K. Rao, Carleton University Reference Kim, J.K. and Rao,
More informationCombining multiple observational data sources to estimate causal eects
Department of Statistics, North Carolina State University Combining multiple observational data sources to estimate causal eects Shu Yang* syang24@ncsuedu Joint work with Peng Ding UC Berkeley May 23,
More informationBootstrap inference for the finite population total under complex sampling designs
Bootstrap inference for the finite population total under complex sampling designs Zhonglei Wang (Joint work with Dr. Jae Kwang Kim) Center for Survey Statistics and Methodology Iowa State University Jan.
More informationChapter 5: Models used in conjunction with sampling. J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70
Chapter 5: Models used in conjunction with sampling J. Kim, W. Fuller (ISU) Chapter 5: Models used in conjunction with sampling 1 / 70 Nonresponse Unit Nonresponse: weight adjustment Item Nonresponse:
More informationBayesian methods for missing data: part 1. Key Concepts. Nicky Best and Alexina Mason. Imperial College London
Bayesian methods for missing data: part 1 Key Concepts Nicky Best and Alexina Mason Imperial College London BAYES 2013, May 21-23, Erasmus University Rotterdam Missing Data: Part 1 BAYES2013 1 / 68 Outline
More informationEM Algorithm II. September 11, 2018
EM Algorithm II September 11, 2018 Review EM 1/27 (Y obs, Y mis ) f (y obs, y mis θ), we observe Y obs but not Y mis Complete-data log likelihood: l C (θ Y obs, Y mis ) = log { f (Y obs, Y mis θ) Observed-data
More informationBasics of Modern Missing Data Analysis
Basics of Modern Missing Data Analysis Kyle M. Lang Center for Research Methods and Data Analysis University of Kansas March 8, 2013 Topics to be Covered An introduction to the missing data problem Missing
More informationANALYSIS OF ORDINAL SURVEY RESPONSES WITH DON T KNOW
SSC Annual Meeting, June 2015 Proceedings of the Survey Methods Section ANALYSIS OF ORDINAL SURVEY RESPONSES WITH DON T KNOW Xichen She and Changbao Wu 1 ABSTRACT Ordinal responses are frequently involved
More informationPropensity score adjusted method for missing data
Graduate Theses and Dissertations Graduate College 2013 Propensity score adjusted method for missing data Minsun Kim Riddles Iowa State University Follow this and additional works at: http://lib.dr.iastate.edu/etd
More informationMeasurement error as missing data: the case of epidemiologic assays. Roderick J. Little
Measurement error as missing data: the case of epidemiologic assays Roderick J. Little Outline Discuss two related calibration topics where classical methods are deficient (A) Limit of quantification methods
More informationStatistical Methods. Missing Data snijders/sm.htm. Tom A.B. Snijders. November, University of Oxford 1 / 23
1 / 23 Statistical Methods Missing Data http://www.stats.ox.ac.uk/ snijders/sm.htm Tom A.B. Snijders University of Oxford November, 2011 2 / 23 Literature: Joseph L. Schafer and John W. Graham, Missing
More informationMathematical statistics
October 4 th, 2018 Lecture 12: Information Where are we? Week 1 Week 2 Week 4 Week 7 Week 10 Week 14 Probability reviews Chapter 6: Statistics and Sampling Distributions Chapter 7: Point Estimation Chapter
More informationA measurement error model approach to small area estimation
A measurement error model approach to small area estimation Jae-kwang Kim 1 Spring, 2015 1 Joint work with Seunghwan Park and Seoyoung Kim Ouline Introduction Basic Theory Application to Korean LFS Discussion
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationBetter Bootstrap Confidence Intervals
by Bradley Efron University of Washington, Department of Statistics April 12, 2012 An example Suppose we wish to make inference on some parameter θ T (F ) (e.g. θ = E F X ), based on data We might suppose
More informationCombining Non-probability and Probability Survey Samples Through Mass Imputation
Combining Non-probability and Probability Survey Samples Through Mass Imputation Jae-Kwang Kim 1 Iowa State University & KAIST October 27, 2018 1 Joint work with Seho Park, Yilin Chen, and Changbao Wu
More informationCOS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION
COS513: FOUNDATIONS OF PROBABILISTIC MODELS LECTURE 9: LINEAR REGRESSION SEAN GERRISH AND CHONG WANG 1. WAYS OF ORGANIZING MODELS In probabilistic modeling, there are several ways of organizing models:
More informationA Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,
A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. Heckman-Type
More informationFractional hot deck imputation
Biometrika (2004), 91, 3, pp. 559 578 2004 Biometrika Trust Printed in Great Britain Fractional hot deck imputation BY JAE KWANG KM Department of Applied Statistics, Yonsei University, Seoul, 120-749,
More informationReview. December 4 th, Review
December 4 th, 2017 Att. Final exam: Course evaluation Friday, 12/14/2018, 10:30am 12:30pm Gore Hall 115 Overview Week 2 Week 4 Week 7 Week 10 Week 12 Chapter 6: Statistics and Sampling Distributions Chapter
More informationGeneralized Linear Models. Kurt Hornik
Generalized Linear Models Kurt Hornik Motivation Assuming normality, the linear model y = Xβ + e has y = β + ε, ε N(0, σ 2 ) such that y N(μ, σ 2 ), E(y ) = μ = β. Various generalizations, including general
More informationA weighted simulation-based estimator for incomplete longitudinal data models
To appear in Statistics and Probability Letters, 113 (2016), 16-22. doi 10.1016/j.spl.2016.02.004 A weighted simulation-based estimator for incomplete longitudinal data models Daniel H. Li 1 and Liqun
More informationBiostat 2065 Analysis of Incomplete Data
Biostat 2065 Analysis of Incomplete Data Gong Tang Dept of Biostatistics University of Pittsburgh October 20, 2005 1. Large-sample inference based on ML Let θ is the MLE, then the large-sample theory implies
More informationMISSING or INCOMPLETE DATA
MISSING or INCOMPLETE DATA A (fairly) complete review of basic practice Don McLeish and Cyntha Struthers University of Waterloo Dec 5, 2015 Structure of the Workshop Session 1 Common methods for dealing
More informationarxiv: v5 [stat.me] 13 Feb 2018
arxiv: arxiv:1602.07933 BOOTSTRAP INFERENCE WHEN USING MULTIPLE IMPUTATION By Michael Schomaker and Christian Heumann University of Cape Town and Ludwig-Maximilians Universität München arxiv:1602.07933v5
More informationA Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness
A Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness A. Linero and M. Daniels UF, UT-Austin SRC 2014, Galveston, TX 1 Background 2 Working model
More informationModification and Improvement of Empirical Likelihood for Missing Response Problem
UW Biostatistics Working Paper Series 12-30-2010 Modification and Improvement of Empirical Likelihood for Missing Response Problem Kwun Chuen Gary Chan University of Washington - Seattle Campus, kcgchan@u.washington.edu
More informationGraphical Models for Collaborative Filtering
Graphical Models for Collaborative Filtering Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Sequence modeling HMM, Kalman Filter, etc.: Similarity: the same graphical model topology,
More informationCalibration estimation using exponential tilting in sample surveys
Calibration estimation using exponential tilting in sample surveys Jae Kwang Kim February 23, 2010 Abstract We consider the problem of parameter estimation with auxiliary information, where the auxiliary
More informationDiscussing Effects of Different MAR-Settings
Discussing Effects of Different MAR-Settings Research Seminar, Department of Statistics, LMU Munich Munich, 11.07.2014 Matthias Speidel Jörg Drechsler Joseph Sakshaug Outline What we basically want to
More informationBayesian inference for factor scores
Bayesian inference for factor scores Murray Aitkin and Irit Aitkin School of Mathematics and Statistics University of Newcastle UK October, 3 Abstract Bayesian inference for the parameters of the factor
More informationInference with Imputed Conditional Means
Inference with Imputed Conditional Means Joseph L. Schafer and Nathaniel Schenker June 4, 1997 Abstract In this paper, we develop analytic techniques that can be used to produce appropriate inferences
More informationMax. Likelihood Estimation. Outline. Econometrics II. Ricardo Mora. Notes. Notes
Maximum Likelihood Estimation Econometrics II Department of Economics Universidad Carlos III de Madrid Máster Universitario en Desarrollo y Crecimiento Económico Outline 1 3 4 General Approaches to Parameter
More informationEstimation in Generalized Linear Models with Heterogeneous Random Effects. Woncheol Jang Johan Lim. May 19, 2004
Estimation in Generalized Linear Models with Heterogeneous Random Effects Woncheol Jang Johan Lim May 19, 2004 Abstract The penalized quasi-likelihood (PQL) approach is the most common estimation procedure
More informationLast lecture 1/35. General optimization problems Newton Raphson Fisher scoring Quasi Newton
EM Algorithm Last lecture 1/35 General optimization problems Newton Raphson Fisher scoring Quasi Newton Nonlinear regression models Gauss-Newton Generalized linear models Iteratively reweighted least squares
More informationInferences on a Normal Covariance Matrix and Generalized Variance with Monotone Missing Data
Journal of Multivariate Analysis 78, 6282 (2001) doi:10.1006jmva.2000.1939, available online at http:www.idealibrary.com on Inferences on a Normal Covariance Matrix and Generalized Variance with Monotone
More informationBAYESIAN METHODS TO IMPUTE MISSING COVARIATES FOR CAUSAL INFERENCE AND MODEL SELECTION
BAYESIAN METHODS TO IMPUTE MISSING COVARIATES FOR CAUSAL INFERENCE AND MODEL SELECTION by Robin Mitra Department of Statistical Science Duke University Date: Approved: Dr. Jerome P. Reiter, Supervisor
More informationParametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
More informationBayesian Analysis of Latent Variable Models using Mplus
Bayesian Analysis of Latent Variable Models using Mplus Tihomir Asparouhov and Bengt Muthén Version 2 June 29, 2010 1 1 Introduction In this paper we describe some of the modeling possibilities that are
More informationMultiple Imputation Methods for Treatment Noncompliance and Nonresponse in Randomized Clinical Trials
UW Biostatistics Working Paper Series 2-19-2009 Multiple Imputation Methods for Treatment Noncompliance and Nonresponse in Randomized Clinical Trials Leslie Taylor UW, taylorl@u.washington.edu Xiao-Hua
More informationEric Shou Stat 598B / CSE 598D METHODS FOR MICRODATA PROTECTION
Eric Shou Stat 598B / CSE 598D METHODS FOR MICRODATA PROTECTION INTRODUCTION Statistical disclosure control part of preparations for disseminating microdata. Data perturbation techniques: Methods assuring
More informationg-priors for Linear Regression
Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 g-priors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and g-priors In the last lecture,
More informationBayesian Additive Regression Tree (BART) with application to controlled trail data analysis
Bayesian Additive Regression Tree (BART) with application to controlled trail data analysis Weilan Yang wyang@stat.wisc.edu May. 2015 1 / 20 Background CATE i = E(Y i (Z 1 ) Y i (Z 0 ) X i ) 2 / 20 Background
More informationTopics and Papers for Spring 14 RIT
Eric Slud Feb. 3, 204 Topics and Papers for Spring 4 RIT The general topic of the RIT is inference for parameters of interest, such as population means or nonlinearregression coefficients, in the presence
More informationLECTURE 5 NOTES. n t. t Γ(a)Γ(b) pt+a 1 (1 p) n t+b 1. The marginal density of t is. Γ(t + a)γ(n t + b) Γ(n + a + b)
LECTURE 5 NOTES 1. Bayesian point estimators. In the conventional (frequentist) approach to statistical inference, the parameter θ Θ is considered a fixed quantity. In the Bayesian approach, it is considered
More informationInterval Estimation III: Fisher's Information & Bootstrapping
Interval Estimation III: Fisher's Information & Bootstrapping Frequentist Confidence Interval Will consider four approaches to estimating confidence interval Standard Error (+/- 1.96 se) Likelihood Profile
More informationComparison of multiple imputation methods for systematically and sporadically missing multilevel data
Comparison of multiple imputation methods for systematically and sporadically missing multilevel data V. Audigier, I. White, S. Jolani, T. Debray, M. Quartagno, J. Carpenter, S. van Buuren, M. Resche-Rigon
More informationConfidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods
Chapter 4 Confidence Intervals in Ridge Regression using Jackknife and Bootstrap Methods 4.1 Introduction It is now explicable that ridge regression estimator (here we take ordinary ridge estimator (ORE)
More informationChapter 8: Estimation 1
Chapter 8: Estimation 1 Jae-Kwang Kim Iowa State University Fall, 2014 Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 1 / 33 Introduction 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch.
More informationA Non-parametric bootstrap for multilevel models
A Non-parametric bootstrap for multilevel models By James Carpenter London School of Hygiene and ropical Medicine Harvey Goldstein and Jon asbash Institute of Education 1. Introduction Bootstrapping is
More informationTheory of Maximum Likelihood Estimation. Konstantin Kashin
Gov 2001 Section 5: Theory of Maximum Likelihood Estimation Konstantin Kashin February 28, 2013 Outline Introduction Likelihood Examples of MLE Variance of MLE Asymptotic Properties What is Statistical
More informationParameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
More informationMonte Carlo Studies. The response in a Monte Carlo study is a random variable.
Monte Carlo Studies The response in a Monte Carlo study is a random variable. The response in a Monte Carlo study has a variance that comes from the variance of the stochastic elements in the data-generating
More informationDownloaded from:
Hossain, A; DiazOrdaz, K; Bartlett, JW (2017) Missing binary outcomes under covariate-dependent missingness in cluster randomised trials. Statistics in medicine. ISSN 0277-6715 DOI: https://doi.org/10.1002/sim.7334
More informationPart 4: Multi-parameter and normal models
Part 4: Multi-parameter and normal models 1 The normal model Perhaps the most useful (or utilized) probability model for data analysis is the normal distribution There are several reasons for this, e.g.,
More informationAn Empirical Comparison of Multiple Imputation Approaches for Treating Missing Data in Observational Studies
Paper 177-2015 An Empirical Comparison of Multiple Imputation Approaches for Treating Missing Data in Observational Studies Yan Wang, Seang-Hwane Joo, Patricia Rodríguez de Gil, Jeffrey D. Kromrey, Rheta
More informationBayesian inference for multivariate extreme value distributions
Bayesian inference for multivariate extreme value distributions Sebastian Engelke Clément Dombry, Marco Oesting Toronto, Fields Institute, May 4th, 2016 Main motivation For a parametric model Z F θ of
More informationBayesian Dropout. Tue Herlau, Morten Morup and Mikkel N. Schmidt. Feb 20, Discussed by: Yizhe Zhang
Bayesian Dropout Tue Herlau, Morten Morup and Mikkel N. Schmidt Discussed by: Yizhe Zhang Feb 20, 2016 Outline 1 Introduction 2 Model 3 Inference 4 Experiments Dropout Training stage: A unit is present
More informationBayesian Analysis (Optional)
Bayesian Analysis (Optional) 1 2 Big Picture There are two ways to conduct statistical inference 1. Classical method (frequentist), which postulates (a) Probability refers to limiting relative frequencies
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationEstimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing
Estimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing Alessandra Mattei Dipartimento di Statistica G. Parenti Università
More informationA Review of Pseudo-Marginal Markov Chain Monte Carlo
A Review of Pseudo-Marginal Markov Chain Monte Carlo Discussed by: Yizhe Zhang October 21, 2016 Outline 1 Overview 2 Paper review 3 experiment 4 conclusion Motivation & overview Notation: θ denotes the
More informationSTATISTICAL INFERENCE WITH DATA AUGMENTATION AND PARAMETER EXPANSION
STATISTICAL INFERENCE WITH arxiv:1512.00847v1 [math.st] 2 Dec 2015 DATA AUGMENTATION AND PARAMETER EXPANSION Yannis G. Yatracos Faculty of Communication and Media Studies Cyprus University of Technology
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationFitting Multidimensional Latent Variable Models using an Efficient Laplace Approximation
Fitting Multidimensional Latent Variable Models using an Efficient Laplace Approximation Dimitris Rizopoulos Department of Biostatistics, Erasmus University Medical Center, the Netherlands d.rizopoulos@erasmusmc.nl
More informationMultiscale Adaptive Inference on Conditional Moment Inequalities
Multiscale Adaptive Inference on Conditional Moment Inequalities Timothy B. Armstrong 1 Hock Peng Chan 2 1 Yale University 2 National University of Singapore June 2013 Conditional moment inequality models
More informationBrandon C. Kelly (Harvard Smithsonian Center for Astrophysics)
Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics) Probability quantifies randomness and uncertainty How do I estimate the normalization and logarithmic slope of a X ray continuum, assuming
More informationarxiv: v3 [stat.me] 20 Feb 2016
Posterior Predictive p-values with Fisher Randomization Tests in Noncompliance Settings: arxiv:1511.00521v3 [stat.me] 20 Feb 2016 Test Statistics vs Discrepancy Variables Laura Forastiere 1, Fabrizia Mealli
More informationSTAT 135 Lab 2 Confidence Intervals, MLE and the Delta Method
STAT 135 Lab 2 Confidence Intervals, MLE and the Delta Method Rebecca Barter February 2, 2015 Confidence Intervals Confidence intervals What is a confidence interval? A confidence interval is calculated
More informationStreamlining Missing Data Analysis by Aggregating Multiple Imputations at the Data Level
Streamlining Missing Data Analysis by Aggregating Multiple Imputations at the Data Level A Monte Carlo Simulation to Test the Tenability of the SuperMatrix Approach Kyle M Lang Quantitative Psychology
More informationMeasurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007
Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η
More informationNonrespondent subsample multiple imputation in two-phase random sampling for nonresponse
Nonrespondent subsample multiple imputation in two-phase random sampling for nonresponse Nanhua Zhang Division of Biostatistics & Epidemiology Cincinnati Children s Hospital Medical Center (Joint work
More informationMS&E 226: Small Data. Lecture 11: Maximum likelihood (v2) Ramesh Johari
MS&E 226: Small Data Lecture 11: Maximum likelihood (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 18 The likelihood function 2 / 18 Estimating the parameter This lecture develops the methodology behind
More informationMachine Learning CSE546 Carlos Guestrin University of Washington. September 30, 2013
Bayesian Methods Machine Learning CSE546 Carlos Guestrin University of Washington September 30, 2013 1 What about prior n Billionaire says: Wait, I know that the thumbtack is close to 50-50. What can you
More informationTopic 12 Overview of Estimation
Topic 12 Overview of Estimation Classical Statistics 1 / 9 Outline Introduction Parameter Estimation Classical Statistics Densities and Likelihoods 2 / 9 Introduction In the simplest possible terms, the
More informationLecture 5 September 19
IFT 6269: Probabilistic Graphical Models Fall 2016 Lecture 5 September 19 Lecturer: Simon Lacoste-Julien Scribe: Sébastien Lachapelle Disclaimer: These notes have only been lightly proofread. 5.1 Statistical
More information,..., θ(2),..., θ(n)
Likelihoods for Multivariate Binary Data Log-Linear Model We have 2 n 1 distinct probabilities, but we wish to consider formulations that allow more parsimonious descriptions as a function of covariates.
More informationMathematical statistics
October 18 th, 2018 Lecture 16: Midterm review Countdown to mid-term exam: 7 days Week 1 Chapter 1: Probability review Week 2 Week 4 Week 7 Chapter 6: Statistics Chapter 7: Point Estimation Chapter 8:
More information