A Note on Bayesian Inference After Multiple Imputation

Save this PDF as:

Size: px
Start display at page:

Transcription

1 A Note on Bayesian Inference After Multiple Imputation Xiang Zhou and Jerome P. Reiter Abstract This article is aimed at practitioners who plan to use Bayesian inference on multiplyimputed datasets in settings where posterior distributions of the parameters of interest are not approximately Gaussian. We seek to steer practitioners away from a naive approach to Bayesian inference, namely estimating the posterior distribution in each completed dataset and averaging functionals of these distributions. We demonstrate that this approach results in unreliable inferences. A better approach is to mix draws from the posterior distributions from each completed dataset, and use the mixed draws to summarize the posterior distribution. Using simulations, we show that for this second approach to work well, the number of imputed datasets should be large. In particular, five to ten imputed datasets which is the standard recommendation for multiple imputation is generally not enough to result in reliable Bayesian inferences. Xiang Zhou is PhD candidate, Department of Neurobiology, and MA, Department of Statistical Science, Duke University, Durham, NC ( and Jerome P. Reiter is Associate Professor, Department of Statistical Science, Duke University, Durham, NC ( This work was supported by the Environmental Protection Agency grant R The authors thank the associate editor and a referee for suggestions on how to explain the bias in the naive approach. 1

2 Keywords: Missing, Nonresponse, Sample 1 INTRODUCTION When some data values are missing, one approach to statistical inference is multiple imputation (Rubin, 1987; Reiter and Raghunathan, 2007). The basic idea is for the analyst to fill in any missing values by repeatedly sampling from the predictive distributions of the missing values. When the posterior distribution of the parameter of interest, or, for likelihood-oriented statisticians, the sampling distribution of the complete-data estimator, is approximately Gaussian, the analyst can obtain inferences by computing point and variance estimates of interest with each dataset and combining these estimates using simple formulas. These formulas serve to propagate the uncertainty introduced by imputation through the analyst s inferences. When presuming normality of the posterior/sampling distribution is not justifiable, the distribution is not adequately summarized by the mean and variance, so that Rubin s (1987) rules are not appropriate for inference. Nonetheless, some practitioners continue to use Rubin s (1987) rules even when they are theoretically invalid. For example, in a literature review of applications of multiple imputation involving parameters not adequately modeled with normal distributions, Marshall et al. (2009) find that, Rubin s rules without applying any transformations were the standard approach used, when any method was stated. They go on to cite several examples where Rubin s (1987) rules are used to estimate functionals of distributions, such as percentiles of survival distributions. 2

3 When normality is not justifiable, Bayesian approaches are viable options for inference. In multiple imputation contexts, the analyst must appropriately utilize the information from the multiple datasets in the inferences; again, simply applying Rubin s (1987) rules to posterior means and variances is generally not correct. An approach suggested by Gelman et al. (2004, p. 520) is (i) simulate many draws from the posterior distribution in each imputed dataset, and (ii) mix all the draws. The mixed draws approximate the posterior distribution. Gelman et al. (2004) do not evaluate the properties of this approximation, nor do the prominent texts on multiple imputation of Schafer (1997) and Little and Rubin (2002). In this article, we examine the approximation of Gelman et al. (2004, p. 520) using simulation studies. We find that the approach works well with large numbers of multiplyimputed datasets. However, the usual advice for multiple imputation for modest fractions of missing data that five or ten completed datasets are adequate for inferences can result in unreliable estimates of posterior distributions. We also point out the pitfalls of incorrectly using Rubin s (1987) rules on functionals of posterior distributions. Specifically, we examine an approach akin to some of those observed by Marshall et al. (2009): (i) estimate posterior quantiles in each completed dataset, and (ii) average them across the datasets. We argue and demonstrate that this approach produces unreliable estimates of posterior distributions. 2 DESCRIPTION OF THE APPROACHES In this section, we motivate the approach of Gelman et al. (2004, p. 520). We begin with brief reviews of Bayesian inference with incomplete data and of multiple imputation. Let 3

4 Y inc = (Y obs, Y mis ) be the n p matrix of data values for n units, where Y obs is the portion of Y inc that is observed, and Y mis is the portion of Y inc that is missing. Let Q be a parameter of interest, for example a regression coefficient. Let Q α be the value of Q such that Qα f(q Y obs )dq = Qα Y mis f(q Y obs, Y mis )f(y mis Y obs )dy mis dq = α, (1) where α is a desired quantile of the posterior distribution of Q. Analysts can approximate this integral with Monte Carlo methods. First, draw Y mis from its posterior predictive distribution. Second, draw a value of Q from its posterior distribution, given the drawn Y mis. Third, repeat these two steps K times, where K is very large. Fourth, sort the K simulated values of Q, and select the (αk)th element of the sorted list. The result is an estimate for Q α. In multiple imputation, the analyst creates m completed datasets, D (l) = (Y obs, Y (l) mis ) where 1 l m, which are used for analysis. Here, Y (l) mis is a draw from the posterior predictive distribution of (Y mis Y obs ), or from an approximation of that distribution such as the approach of Raghunathan et al. (2001). Typically, m is much smaller than K would be for Bayesian inference for non-gaussian distributions. Thus, with small m, drawing one value of Q for each D (l) results in too few draws of Q to get reasonable estimates of Q α. Instead, we can utilize each completed dataset for more than just one draw of Q. To motivate this, we re-express the integral in (1) as 1 α = lim m m m l=1 Qα f(q Y obs, Y (l) mis )dq. (2) This suggests that, for any value of Q in the upper limit of the integral, we can find the associated cumulative probability by (i) sampling J values of Q in each D (l), where J is 4

5 large, (ii) finding the percentage of the J draws in each D (l) less than the upper limit value, and (iii) averaging those percentages across all m datasets, where m. Equivalently, the analyst could mix all of the sampled draws from each dataset, and find the percentage of elements less than the upper limit in the combined draws. This process can be easily adapted to find Q α : try different upper limits until one reaches the desired α probability. The approximation of Gelman et al. (2004, p. 520), which we denote as Q α, is essentially an approximation of (2) for finite m. Specifically, for each D (l) where l = 1,..., m, the analyst simulates J values of Q from f(q D (l) ), where J is large. Let ˆf(Q (l) ) represent the J draws of Q obtained with D (l). The analyst mixes all ˆf(Q (l) ) together to create ˆf(Q all ). The analyst sorts the mj draws in ˆf(Q all ), and the α(mj)th element of the sorted list is the estimate of Q α. We now use simulations to illustrate the properties of Q α. We also use the simulations to emphasize that the naive approach of averaging posterior quantiles can produce poor estimates of Q α in comparison to Q α. To fix notation for the naive approach, let Q (l) α be the value of Q in D (l) such that Q (l) α f(q Y obs, Y (l) mis )dq = α. Then, Qα = m l=1 Q(l) α /m. Clearly, Q α has nothing to do with (2). It is derived from convenience rather than theory. 3 ILLUSTRATIVE SIMULATIONS The complete data, Y inc, comprise n = 50 values generated independently from Bernoulli trials with π =.2. We introduce missing data by randomly deleting 10%, 30% or 50% of the data completely at random (Rubin, 1976). We use multiple imputation to generate m = 5, 5

6 m = 20, or m = 100 completed datasets. We seek Q α, where α {.025,.25,.75,.975}, for the posterior distribution of π. We generate Y inc and multiple imputations 5000 times to approximate the sampling distributions of Q α and Q α. To create each completed dataset, we first sample a value of π from the appropriate Beta distribution based on Y obs. We use a uniform prior distribution for π. We then draw Y mis from a Bernoulli distribution using the sampled π. After the imputation steps, in each D (l) we draw J = values of π from Beta( (Y obs +Y (l) mis )+1, n (Y obs +Y (l) mis )+1), which is the posterior computed with D (l). To get Q α, we mix and sort the mj draws of π, and select the α(mj)th element. To get Q α, we compute the α-quantile in each D (l) and average them across the m datasets. Figure 1 shows the distributions of Q α Q α and Q α Q α for α {.025,.25} across the 5000 replications with m = 100. Here, Q α is computed from f(π Y obs ). For each scenario, Q α is nearly centered on Q α. There do not appear to be any trends with the percentage of missing data, apart from the expected increase in variability as the percentage of missing data increases. However, in additional simulations with m = 5 and m = 20, typically Q α > Q α for small α, and Q α < Q α for large α. This is evident in Figure 2, which shows that when m = 5 and to a lesser extent when m = 20, the posterior intervals based on Q α tend to be tighter than warranted for modest m. This problem disappears when m = 100. The inaccuracy when m = 5 merits closer inspection, because often practitioners only create five multiple imputations for analysis. Across all missing data scenarios, the median lengths of the 50% and 95% posterior intervals are smaller when m = 5 than when m =

7 Put another way, analysts appear to obtain sharper inferences by using five imputations than using one hundred imputations. This does not imply that analysts should use small m for Bayesian inference after multiple imputation; on the contrary, it implies that approximations Q α based on small m are not reliable. Hence, analysts planning on Bayesian inference after multiple imputation should generate a large number of completed datasets. What about Q α? As evident in Figure 1, Qα can differ substantially from Q α, and its performance worsens as the percentage of missing values increases. More often than not, Q α > Q α for small α, and Q α < Q α for large α. Hence, as also evident in Figure 2, analysts who construct posterior intervals based on Q α tend to have tighter ranges than warranted. What is wrong with the naive approach of averaging posterior quantiles? Each Q (l) α is a summary of the posterior distribution of Q estimated as if D (l) was in fact genuine data with n records. However, the observed data comprise fewer than n records, so that the actual posterior distribution of Q is more dispersed than the complete-data posterior distribution. Thus, each Q (l) α is biased towards the median, and so is their average. 4 APPLICATION TO BIOASSAY DATA To illustrate Bayesian inference after multiple imputation on genuine data, we modify data from a bioassay experiment that appears in Gelman et al. (2004, p ), who took them from Racine et al. (1986). The data comprise two measurements on n = 20 animals. Let x i be the natural logarithm of the dose of a chemical compound administered to animal i, where x i {.86,.30,.05,.73}. There were five animals at each dose level. Let y i = 1 if 7

8 Relative Value of Two Estimates for Quantile Relative Value of Two Estimates for 0.25 Quantile Differences Q, 10 Q, 30 Q, 50 Q ~, 10 Q ~, 30 Q ~, 50 Differences Q, 10 Q, 30 Q, 50 Q ~, 10 Q ~, 30 Q ~, 50 Figure 1: Box plots of Q α Q α and Q α Q α in different settings with n = 50, m = 100, and 10%, 30%, or 50% missing data. The first three plots in each panel are for Q α Q α, and the second three plots in each panel are for Q α Q α. The labels on the horizontal axis show the percentage of missing data. Generally, Qα is substantially different than Q α, whereas Q α estimates Q α reasonably well. animal i dies shortly after receiving the dose, and let y i = 0 otherwise. There are no missing data in the study. Therefore, we deleted a randomly selected 20% of the y i values. The goal of the analysis is to learn about the toxicity of the compound, which we do with a logistic regression of Y on X. Because of the small sample size, it is doubtful that the sampling distributions of the estimated regression coefficients are well-approximated by normal distributions. Following Gelman et al. (2004), we therefore use a Bayesian logistic regression model to learn about the toxicity of the compound, so that y i π i Bernoulli(π i ) where logit(π i ) = β 0 + β 1 x i. We use the non-informative prior distribution f(β 0, β 1 ) 1. 8

9 The primary targets of scientific interest are the posterior distributions of β 0 and β 1. To multiply-impute the four missing values, we first draw a value of (β 0, β 1 ) from its approximate posterior distribution using grid sampling. We substitute the drawn values into the equation for π i for each of the four animals with missing data. We then draw values of Y mis from each animal s Bernoulli distribution to create the completed dataset, D (l), where l = 1,...,m. We examine three scenarios: m = 5, m = 20, and m = 100. For each D (l), we determine quantiles of f(β 0, β 1 D (l) ) by using grid sampling again. We sample J = 10, 000 values from the joint distribution for each completed dataset. By mixing the mj draws of (β 0, β 1 ), we can compute values of Q α. Table 1 displays several quantiles for β 0 and β 1. When m = 100, the values of Qα are close to the corresponding values of Q α. As expected, the differences between Q α and Q α are largest when m = 5. For both β 0 and β 1, the posterior intervals are too narrow when m = 5. The Table also displays estimates based on Q α. Once again, they are less reliable than those based on Q α. To see if the results in Table 1 for m = 5 are unusual, we repeated the posterior simulation 100 times. In 57% of these replications, Q.975 Q.025 for β 1 with m = 5 was shorter than Q.975 Q.025 for β 1 from the observed data; roughly the same trend held for the interquartile range for β 1 and for the intervals involving β 0. The lengths of the one hundred Q.975 Q.025 for β 1 with m = 5 ranged from 15.8 (1.5 to 17.3) to 21.4 (3.0 to 24.4), as compared to a length of 19.5 for Q.975 Q.025. Thus, there are substantial chances of estimating inappropriately short posterior intervals with Q.975 Q.025, although the risks appear to be random rather 9

10 than systematic. Given the potential for overstating accuracy, we would be reluctant to recommend or use m = 5 for this analysis. We also repeated the analysis using an imputation model that differs from the analysis model. Specifically, for imputations we assume that f(y ij ) Bernoulli(π j ) in each of the j = 1,...,4 dosage strata. The results show the themes seen in Table 1. 5 CLOSING REMARKS As both multiple imputation and Bayesian inference grow in popularity, we anticipate that practitioners will commonly use Bayesian inference after multiple imputation. We hope that this article reduces the number of practitioners who naively and incorrectly average posterior quantiles and other functionals, and encourages practitioners to use the approach of Gelman et al. (2004, p. 520) with large m. References Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004). Bayesian Data Analysis. London: Chapman & Hall. Little, R. J. A. and Rubin, D. B. (2002). Statistical Analysis with Missing Data: Second Edition. New York: John Wiley & Sons. Marshall, A., Altman, D. G., Holder, R. L., and Royston, P. (2009). Combining estimates 10

11 of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Medical Research Methodology 9:57. Racine, A., Grieve, A., Fluhler, H., and Smith, A. (1986). Bayesian methods in practice: experiences in the pharmaceutical industry (with discussion). Applied Statistics 35, Raghunathan, T. E., Lepkowski, J. M., van Hoewyk, J., and Solenberger, P. (2001). A multivariate technique for multiply imputing missing values using a series of regression models. Survey Methodology 27, Reiter, J. P. and Raghunathan, T. E. (2007). The multiple adaptations of multiple imputation. Journal of the American Statistical Association 102, Rubin, D. B. (1976). Inference and missing data (with discussion). Biometrika 63, Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons. Schafer, J. L. (1997). Analysis of Incomplete Multivariate Data. London: Chapman & Hall. 11

12 95% Posterior Interval with Q ~ α 50% Posterior Interval with Q ~ α (Q ~ Q ~ 0.025) (Q Q0.025) (Q ~ 0.75 Q ~ 0.25) (Q 0.75 Q0.25) _1 10_2 10_3 30_1 30_2 30_3 50_1 50_2 50_3 10_1 10_2 10_3 30_1 30_2 30_3 50_1 50_2 50_3 95% Posterior Interval with Q α 50% Posterior Interval with Q α (Q Q0.025) (Q0.975 Q0.025) _1 10_2 10_3 30_1 30_2 30_3 50_1 50_2 50_3 (Q 0.75 Q0.25) (Q0.75 Q0.25) _1 10_2 10_3 30_1 30_2 30_3 50_1 50_2 50_3 Figure 2: Box plots of differences in lengths of the approximate and true 50% and 95% posterior intervals with n = 50. The labels on the horizontal axis show the percentage of missing data followed by the value of m, where 1 represents m = 5, 2 represents m = 20, and 3 represents m = 100. The intervals based on Q α (top panel) are relatively poor approximations for modest m but good for large m. The intervals based on Q α (bottom panel) are always unreliable. 12

13 Q α for MI with Qα for MI with Q α m = 5 m = 20 m = 100 m = 5 m = 20 m = 100 Posterior quantiles for β 0 α = α = α = α = % interval length Posterior quantiles for β 1 α = α = α = α = % interval length Table 1: Quantile estimates for the bioassay data from one set of multiple imputations using m = 5, m = 20, and m = 100. The imputation model is the same as the analysis model. Here, Q α is estimated with the observed data. Inferences based on Q α are reliable with m = 100 but less so with m = 5 or m = 20. Inferences based on Q α are generally unreliable. 13

Quantile POD for Hit-Miss Data

Quantile POD for Hit-Miss Data Yew-Meng Koh a and William Q. Meeker a a Center for Nondestructive Evaluation, Department of Statistics, Iowa State niversity, Ames, Iowa 50010 Abstract. Probability of detection

F-tests for Incomplete Data in Multiple Regression Setup

F-tests for Incomplete Data in Multiple Regression Setup ASHOK CHAURASIA Advisor: Dr. Ofer Harel University of Connecticut / 1 of 19 OUTLINE INTRODUCTION F-tests in Multiple Linear Regression Incomplete

Multiple Imputation for Missing Data in Repeated Measurements Using MCMC and Copulas

Multiple Imputation for Missing Data in epeated Measurements Using MCMC and Copulas Lily Ingsrisawang and Duangporn Potawee Abstract This paper presents two imputation methods: Marov Chain Monte Carlo

Toutenburg, Fieger: Using diagnostic measures to detect non-mcar processes in linear regression models with missing covariates

Toutenburg, Fieger: Using diagnostic measures to detect non-mcar processes in linear regression models with missing covariates Sonderforschungsbereich 386, Paper 24 (2) Online unter: http://epub.ub.uni-muenchen.de/

PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.

PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.. Beta Distribution We ll start by learning about the Beta distribution, since we end up using

BAYESIAN ANALYSIS OF DOSE-RESPONSE CALIBRATION CURVES

Libraries Annual Conference on Applied Statistics in Agriculture 2005-17th Annual Conference Proceedings BAYESIAN ANALYSIS OF DOSE-RESPONSE CALIBRATION CURVES William J. Price Bahman Shafii Follow this

LOGISTIC REGRESSION Joseph M. Hilbe

LOGISTIC REGRESSION Joseph M. Hilbe Arizona State University Logistic regression is the most common method used to model binary response data. When the response is binary, it typically takes the form of

Hossain, A; DiazOrdaz, K; Bartlett, JW (2017) Missing binary outcomes under covariate-dependent missingness in cluster randomised trials. Statistics in medicine. ISSN 0277-6715 DOI: https://doi.org/10.1002/sim.7334

eqr094: Hierarchical MCMC for Bayesian System Reliability

eqr094: Hierarchical MCMC for Bayesian System Reliability Alyson G. Wilson Statistical Sciences Group, Los Alamos National Laboratory P.O. Box 1663, MS F600 Los Alamos, NM 87545 USA Phone: 505-667-9167

Bagging During Markov Chain Monte Carlo for Smoother Predictions

Bagging During Markov Chain Monte Carlo for Smoother Predictions Herbert K. H. Lee University of California, Santa Cruz Abstract: Making good predictions from noisy data is a challenging problem. Methods

R-squared for Bayesian regression models

R-squared for Bayesian regression models Andrew Gelman Ben Goodrich Jonah Gabry Imad Ali 8 Nov 2017 Abstract The usual definition of R 2 (variance of the predicted values divided by the variance of the

Comparison of Three Calculation Methods for a Bayesian Inference of Two Poisson Parameters

Journal of Modern Applied Statistical Methods Volume 13 Issue 1 Article 26 5-1-2014 Comparison of Three Calculation Methods for a Bayesian Inference of Two Poisson Parameters Yohei Kawasaki Tokyo University

STATISTICAL INFERENCE WITH DATA AUGMENTATION AND PARAMETER EXPANSION

STATISTICAL INFERENCE WITH arxiv:1512.00847v1 [math.st] 2 Dec 2015 DATA AUGMENTATION AND PARAMETER EXPANSION Yannis G. Yatracos Faculty of Communication and Media Studies Cyprus University of Technology

Statistical Practice. Selecting the Best Linear Mixed Model Under REML. Matthew J. GURKA

Matthew J. GURKA Statistical Practice Selecting the Best Linear Mixed Model Under REML Restricted maximum likelihood (REML) estimation of the parameters of the mixed model has become commonplace, even

Known unknowns : using multiple imputation to fill in the blanks for missing data

Known unknowns : using multiple imputation to fill in the blanks for missing data James Stanley Department of Public Health University of Otago, Wellington james.stanley@otago.ac.nz Acknowledgments Cancer

ASA Section on Survey Research Methods

REGRESSION-BASED STATISTICAL MATCHING: RECENT DEVELOPMENTS Chris Moriarity, Fritz Scheuren Chris Moriarity, U.S. Government Accountability Office, 411 G Street NW, Washington, DC 20548 KEY WORDS: data

Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions

R U T C O R R E S E A R C H R E P O R T Item Parameter Calibration of LSAT Items Using MCMC Approximation of Bayes Posterior Distributions Douglas H. Jones a Mikhail Nediak b RRR 7-2, February, 2! " ##\$%#&

A Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness

A Bayesian Nonparametric Approach to Monotone Missing Data in Longitudinal Studies with Informative Missingness A. Linero and M. Daniels UF, UT-Austin SRC 2014, Galveston, TX 1 Background 2 Working model

Gregory J. Matthews a, Ofer Harel a & Robert H. Aseltine b a Department of Statistics, University of Connecticut, 215

This article was downloaded by: [University of Massachusetts, Amherst], [Greg Matthews] On: 05 November 2013, At: 14:30 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered

Approximate Median Regression via the Box-Cox Transformation

Approximate Median Regression via the Box-Cox Transformation Garrett M. Fitzmaurice,StuartR.Lipsitz, and Michael Parzen Median regression is used increasingly in many different areas of applications. The

Bayesian Inference. Chapter 1. Introduction and basic concepts

Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master

Measurement Error and Linear Regression of Astronomical Data. Brandon Kelly Penn State Summer School in Astrostatistics, June 2007

Measurement Error and Linear Regression of Astronomical Data Brandon Kelly Penn State Summer School in Astrostatistics, June 2007 Classical Regression Model Collect n data points, denote i th pair as (η

Estimating complex causal effects from incomplete observational data

Estimating complex causal effects from incomplete observational data arxiv:1403.1124v2 [stat.me] 2 Jul 2014 Abstract Juha Karvanen Department of Mathematics and Statistics, University of Jyväskylä, Jyväskylä,

BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS

BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS Andrew A. Neath 1 and Joseph E. Cavanaugh 1 Department of Mathematics and Statistics, Southern Illinois University, Edwardsville, Illinois 606, USA

On the Conditional Distribution of the Multivariate t Distribution

On the Conditional Distribution of the Multivariate t Distribution arxiv:604.0056v [math.st] 2 Apr 206 Peng Ding Abstract As alternatives to the normal distributions, t distributions are widely applied

REVISED PAGE PROOFS. Logistic Regression. Basic Ideas. Fundamental Data Analysis. bsa350

bsa347 Logistic Regression Logistic regression is a method for predicting the outcomes of either-or trials. Either-or trials occur frequently in research. A person responds appropriately to a drug or does

arxiv: v5 [stat.me] 13 Feb 2018

arxiv: arxiv:1602.07933 BOOTSTRAP INFERENCE WHEN USING MULTIPLE IMPUTATION By Michael Schomaker and Christian Heumann University of Cape Town and Ludwig-Maximilians Universität München arxiv:1602.07933v5

Bayesian Estimation of log N log S

Bayesian Estimation of log N log S Paul Baines, Irina Udaltsova Department of Statistics University of California, Davis July 12th, 2011 Introduction What is log N log S? Cumulative number of sources detectable

CTDL-Positive Stable Frailty Model

CTDL-Positive Stable Frailty Model M. Blagojevic 1, G. MacKenzie 2 1 Department of Mathematics, Keele University, Staffordshire ST5 5BG,UK and 2 Centre of Biostatistics, University of Limerick, Ireland

INVERTED KUMARASWAMY DISTRIBUTION: PROPERTIES AND ESTIMATION

Pak. J. Statist. 2017 Vol. 33(1), 37-61 INVERTED KUMARASWAMY DISTRIBUTION: PROPERTIES AND ESTIMATION A. M. Abd AL-Fattah, A.A. EL-Helbawy G.R. AL-Dayian Statistics Department, Faculty of Commerce, AL-Azhar

Bayesian Learning. HT2015: SC4 Statistical Data Mining and Machine Learning. Maximum Likelihood Principle. The Bayesian Learning Framework

HT5: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Maximum Likelihood Principle A generative model for

ON THE CONSEQUENCES OF MISSPECIFING ASSUMPTIONS CONCERNING RESIDUALS DISTRIBUTION IN A REPEATED MEASURES AND NONLINEAR MIXED MODELLING CONTEXT

ON THE CONSEQUENCES OF MISSPECIFING ASSUMPTIONS CONCERNING RESIDUALS DISTRIBUTION IN A REPEATED MEASURES AND NONLINEAR MIXED MODELLING CONTEXT Rachid el Halimi and Jordi Ocaña Departament d Estadística

Other Noninformative Priors

Other Noninformative Priors Other methods for noninformative priors include Bernardo s reference prior, which seeks a prior that will maximize the discrepancy between the prior and the posterior and minimize

6. Assessing studies based on multiple regression

6. Assessing studies based on multiple regression Questions of this section: What makes a study using multiple regression (un)reliable? When does multiple regression provide a useful estimate of the causal

The Bayesian Approach to Multi-equation Econometric Model Estimation

Journal of Statistical and Econometric Methods, vol.3, no.1, 2014, 85-96 ISSN: 2241-0384 (print), 2241-0376 (online) Scienpress Ltd, 2014 The Bayesian Approach to Multi-equation Econometric Model Estimation

Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities

Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities Peter M. Aronow and Cyrus Samii Forthcoming at Survey Methodology Abstract We consider conservative variance

Implications of Missing Data Imputation for Agricultural Household Surveys: An Application to Technology Adoption

Implications of Missing Data Imputation for Agricultural Household Surveys: An Application to Technology Adoption Haluk Gedikoglu Assistant Professor of Agricultural Economics Cooperative Research Programs

Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features. Yangxin Huang

Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features Yangxin Huang Department of Epidemiology and Biostatistics, COPH, USF, Tampa, FL yhuang@health.usf.edu January

Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events

Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events Liam F. McGrath September 15, 215 Abstract When separation is a problem in binary

Tools for Parameter Estimation and Propagation of Uncertainty

Tools for Parameter Estimation and Propagation of Uncertainty Brian Borchers Department of Mathematics New Mexico Tech Socorro, NM 87801 borchers@nmt.edu Outline Models, parameters, parameter estimation,

A nonparametric two-sample wald test of equality of variances

University of Wollongong Research Online Faculty of Informatics - Papers (Archive) Faculty of Engineering and Information Sciences 211 A nonparametric two-sample wald test of equality of variances David

Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

For right censored data with Y i = T i C i and censoring indicator, δ i = I(T i < C i ), arising from such a parametric model we have the likelihood,

A NOTE ON LAPLACE REGRESSION WITH CENSORED DATA ROGER KOENKER Abstract. The Laplace likelihood method for estimating linear conditional quantile functions with right censored data proposed by Bottai and

Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

Title stata.com logistic postestimation Postestimation tools for logistic Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

STATISTICAL INFERENCE FOR SURVEY DATA ANALYSIS

STATISTICAL INFERENCE FOR SURVEY DATA ANALYSIS David A Binder and Georgia R Roberts Methodology Branch, Statistics Canada, Ottawa, ON, Canada K1A 0T6 KEY WORDS: Design-based properties, Informative sampling,

An EM-Algorithm Based Method to Deal with Rounded Zeros in Compositional Data under Dirichlet Models. Rafiq Hijazi

An EM-Algorithm Based Method to Deal with Rounded Zeros in Compositional Data under Dirichlet Models Rafiq Hijazi Department of Statistics United Arab Emirates University P.O. Box 17555, Al-Ain United

Summary and discussion of The central role of the propensity score in observational studies for causal effects

Summary and discussion of The central role of the propensity score in observational studies for causal effects Statistics Journal Club, 36-825 Jessica Chemali and Michael Vespe 1 Summary 1.1 Background

Multicollinearity and A Ridge Parameter Estimation Approach

Journal of Modern Applied Statistical Methods Volume 15 Issue Article 5 11-1-016 Multicollinearity and A Ridge Parameter Estimation Approach Ghadban Khalaf King Khalid University, albadran50@yahoo.com

Authors and Affiliations: Nianbo Dong University of Missouri 14 Hill Hall, Columbia, MO Phone: (573)

Prognostic Propensity Scores: A Method Accounting for the Correlations of the Covariates with Both the Treatment and the Outcome Variables in Matching and Diagnostics Authors and Affiliations: Nianbo Dong

Sensitivity study of dose-finding methods

to of dose-finding methods Sarah Zohar 1 John O Quigley 2 1. Inserm, UMRS 717,Biostatistic Department, Hôpital Saint-Louis, Paris, France 2. Inserm, Université Paris VI, Paris, France. NY 2009 1 / 21 to

STATISTICS 3A03. Applied Regression Analysis with SAS. Angelo J. Canty

STATISTICS 3A03 Applied Regression Analysis with SAS Angelo J. Canty Office : Hamilton Hall 209 Phone : (905) 525-9140 extn 27079 E-mail : cantya@mcmaster.ca SAS Labs : L1 Friday 11:30 in BSB 249 L2 Tuesday

Luke B Smith and Brian J Reich North Carolina State University May 21, 2013

BSquare: An R package for Bayesian simultaneous quantile regression Luke B Smith and Brian J Reich North Carolina State University May 21, 2013 BSquare in an R package to conduct Bayesian quantile regression

Approximating Bayesian Posterior Means Using Multivariate Gaussian Quadrature

Approximating Bayesian Posterior Means Using Multivariate Gaussian Quadrature John A.L. Cranfield Paul V. Preckel Songquan Liu Presented at Western Agricultural Economics Association 1997 Annual Meeting

Stat 705: Completely randomized and complete block designs

Stat 705: Completely randomized and complete block designs Timothy Hanson Department of Statistics, University of South Carolina Stat 705: Data Analysis II 1 / 16 Experimental design Our department offers

A Comparative Study of Imputation Methods for Estimation of Missing Values of Per Capita Expenditure in Central Java

IOP Conference Series: Earth and Environmental Science PAPER OPEN ACCESS A Comparative Study of Imputation Methods for Estimation of Missing Values of Per Capita Expenditure in Central Java To cite this

PARAMETER ESTIMATION FOR THE LOG-LOGISTIC DISTRIBUTION BASED ON ORDER STATISTICS

PARAMETER ESTIMATION FOR THE LOG-LOGISTIC DISTRIBUTION BASED ON ORDER STATISTICS Authors: Mohammad Ahsanullah Department of Management Sciences, Rider University, New Jersey, USA ahsan@rider.edu) Ayman

4. STATISTICAL SAMPLING DESIGNS FOR ISM

IRTC Incremental Sampling Methodology February 2012 4. STATISTICAL SAMPLING DESIGNS FOR ISM This section summarizes results of simulation studies used to evaluate the performance of ISM in estimating the

BIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY

BIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY Ingo Langner 1, Ralf Bender 2, Rebecca Lenz-Tönjes 1, Helmut Küchenhoff 2, Maria Blettner 2 1

Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error Distributions

Journal of Modern Applied Statistical Methods Volume 8 Issue 1 Article 13 5-1-2009 Least Absolute Value vs. Least Squares Estimation and Inference Procedures in Regression Models with Asymmetric Error

Lecture 5: Spatial probit models. James P. LeSage University of Toledo Department of Economics Toledo, OH

Lecture 5: Spatial probit models James P. LeSage University of Toledo Department of Economics Toledo, OH 43606 jlesage@spatial-econometrics.com March 2004 1 A Bayesian spatial probit model with individual

The Expectation Maximization Algorithm

The Expectation Maximization Algorithm Frank Dellaert College of Computing, Georgia Institute of Technology Technical Report number GIT-GVU-- February Abstract This note represents my attempt at explaining

WinLTA USER S GUIDE for Data Augmentation

USER S GUIDE for Version 1.0 (for WinLTA Version 3.0) Linda M. Collins Stephanie T. Lanza Joseph L. Schafer The Methodology Center The Pennsylvania State University May 2002 Dev elopment of this program

Selection Criteria Based on Monte Carlo Simulation and Cross Validation in Mixed Models

Selection Criteria Based on Monte Carlo Simulation and Cross Validation in Mixed Models Junfeng Shang Bowling Green State University, USA Abstract In the mixed modeling framework, Monte Carlo simulation

Noninformative Priors for the Ratio of the Scale Parameters in the Inverted Exponential Distributions

Communications for Statistical Applications and Methods 03, Vol. 0, No. 5, 387 394 DOI: http://dx.doi.org/0.535/csam.03.0.5.387 Noninformative Priors for the Ratio of the Scale Parameters in the Inverted

An Overview of the Pros and Cons of Linearization versus Replication in Establishment Surveys

An Overview of the Pros and Cons of Linearization versus Replication in Establishment Surveys Richard Valliant University of Michigan and Joint Program in Survey Methodology University of Maryland 1 Introduction

The Generalized Likelihood Uncertainty Estimation methodology

CHAPTER 4 The Generalized Likelihood Uncertainty Estimation methodology Calibration and uncertainty estimation based upon a statistical framework is aimed at finding an optimal set of models, parameters

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP

INTRODUCTION TO BAYESIAN INFERENCE PART 2 CHRIS BISHOP Personal Healthcare Revolution Electronic health records (CFH) Personal genomics (DeCode, Navigenics, 23andMe) X-prize: first \$10k human genome technology

Principal Component Analysis, A Powerful Scoring Technique

Principal Component Analysis, A Powerful Scoring Technique George C. J. Fernandez, University of Nevada - Reno, Reno NV 89557 ABSTRACT Data mining is a collection of analytical techniques to uncover new

Fractional Polynomial Regression

Chapter 382 Fractional Polynomial Regression Introduction This program fits fractional polynomial models in situations in which there is one dependent (Y) variable and one independent (X) variable. It

Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

A Single Series from the Gibbs Sampler Provides a False Sense of Security

A Single Series from the Gibbs Sampler Provides a False Sense of Security Andrew Gelman Department of Statistics University of California Berkeley, CA 9472 Donald B. Rubin Department of Statistics Harvard

Some methods for handling missing values in outcome variables. Roderick J. Little

Some methods for handling missing values in outcome variables Roderick J. Little Missing data principles Likelihood methods Outline ML, Bayes, Multiple Imputation (MI) Robust MAR methods Predictive mean

Chapter 1 Statistical Inference

Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

Pubh 8482: Sequential Analysis

Pubh 8482: Sequential Analysis Joseph S. Koopmeiners Division of Biostatistics University of Minnesota Week 10 Class Summary Last time... We began our discussion of adaptive clinical trials Specifically,

BMA-Mod : A Bayesian Model Averaging Strategy for Determining Dose-Response Relationships in the Presence of Model Uncertainty

BMA-Mod : A Bayesian Model Averaging Strategy for Determining -Response Relationships in the Presence of Model Uncertainty A. Lawrence Gould KOL 20 October, 2017 Overview and Motivation -Response Trials

Variable selection and machine learning methods in causal inference

Variable selection and machine learning methods in causal inference Debashis Ghosh Department of Biostatistics and Informatics Colorado School of Public Health Joint work with Yeying Zhu, University of

p(d θ ) l(θ ) 1.2 x x x

p(d θ ).2 x 0-7 0.8 x 0-7 0.4 x 0-7 l(θ ) -20-40 -60-80 -00 2 3 4 5 6 7 θ ˆ 2 3 4 5 6 7 θ ˆ 2 3 4 5 6 7 θ θ x FIGURE 3.. The top graph shows several training points in one dimension, known or assumed to

Impact of covariate misclassification on the power and type I error in clinical trials using covariate-adaptive randomization

Impact of covariate misclassification on the power and type I error in clinical trials using covariate-adaptive randomization L I Q I O N G F A N S H A R O N D. Y E A T T S W E N L E Z H A O M E D I C

Misspecification in Nonrecursive SEMs 1. Nonrecursive Latent Variable Models under Misspecification

Misspecification in Nonrecursive SEMs 1 Nonrecursive Latent Variable Models under Misspecification Misspecification in Nonrecursive SEMs 2 Abstract A problem central to structural equation modeling is

Probability and Statistics

Probability and Statistics Kristel Van Steen, PhD 2 Montefiore Institute - Systems and Modeling GIGA - Bioinformatics ULg kristel.vansteen@ulg.ac.be CHAPTER 4: IT IS ALL ABOUT DATA 4a - 1 CHAPTER 4: IT

Random Effects Models for Network Data

Random Effects Models for Network Data Peter D. Hoff 1 Working Paper no. 28 Center for Statistics and the Social Sciences University of Washington Seattle, WA 98195-4320 January 14, 2003 1 Department of

Directly testing the linearity assumption for assay validation. Steven Novick / Harry Yang May, 2013

Directly testing the linearity assumption or assay validation Steven Novick / Harry Yang May, 2013 Manuscript accepted March 2013 or publication in Journal o Chemometrics 1 Steven Novick Associate Director

3.1 Measures of Central Tendency: Mode, Median and Mean. Average a single number that is used to describe the entire sample or population

. Measures of Central Tendency: Mode, Median and Mean Average a single number that is used to describe the entire sample or population. Mode a. Easiest to compute, but not too stable i. Changing just one

A Brief Review of Probability, Bayesian Statistics, and Information Theory

A Brief Review of Probability, Bayesian Statistics, and Information Theory Brendan Frey Electrical and Computer Engineering University of Toronto frey@psi.toronto.edu http://www.psi.toronto.edu A system

Handling Missing Data in R with MICE

Handling Missing Data in R with MICE Handling Missing Data in R with MICE Stef van Buuren 1,2 1 Methodology and Statistics, FSBS, Utrecht University 2 Netherlands Organization for Applied Scientific Research

Empirical Risk Minimization is an incomplete inductive principle Thomas P. Minka

Empirical Risk Minimization is an incomplete inductive principle Thomas P. Minka February 20, 2001 Abstract Empirical Risk Minimization (ERM) only utilizes the loss function defined for the task and is

An Introduction to Reversible Jump MCMC for Bayesian Networks, with Application

An Introduction to Reversible Jump MCMC for Bayesian Networks, with Application, CleverSet, Inc. STARMAP/DAMARS Conference Page 1 The research described in this presentation has been funded by the U.S.

STRmix - MCMC Study Page 1

SDPD Crime Laboratory Forensic Biology Unit Validation of the STRmix TM Software MCMC Markov Chain Monte Carlo Introduction The goal of DNA mixture interpretation should be to identify the genotypes of

Dose-response modeling with bivariate binary data under model uncertainty

Dose-response modeling with bivariate binary data under model uncertainty Bernhard Klingenberg 1 1 Department of Mathematics and Statistics, Williams College, Williamstown, MA, 01267 and Institute of Statistics,

Inferences about Parameters of Trivariate Normal Distribution with Missing Data

Florida International University FIU Digital Commons FIU Electronic Theses and Dissertations University Graduate School 7-5-3 Inferences about Parameters of Trivariate Normal Distribution with Missing

Should Non-Sensitive Attributes be Masked? Data Quality Implications of Data Perturbation in Regression Analysis

Should Non-Sensitive Attributes be Masked? Data Quality Implications of Data Perturbation in Regression Analysis Sumitra Mukherjee Nova Southeastern University sumitra@scis.nova.edu Abstract Ensuring the

Efficient Robbins-Monro Procedure for Binary Data

Efficient Robbins-Monro Procedure for Binary Data V. Roshan Joseph School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332-0205, USA roshan@isye.gatech.edu SUMMARY

Chris Fraley and Daniel Percival. August 22, 2008, revised May 14, 2010

Model-Averaged l 1 Regularization using Markov Chain Monte Carlo Model Composition Technical Report No. 541 Department of Statistics, University of Washington Chris Fraley and Daniel Percival August 22,

CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA

STATISTICS IN MEDICINE, VOL. 17, 59 68 (1998) CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA J. K. LINDSEY AND B. JONES* Department of Medical Statistics, School of Computing Sciences,

5.3. STATISTICAL ANALYSIS OF RESULTS OF BIOLOGICAL ASSAYS AND TESTS

EUROPEAN PHARMACOPOEIA 6.0 5.3. Statistical analysis 01/2008:50300 5.3. STATISTICAL ANALYSIS OF RESULTS OF BIOLOGICAL ASSAYS AND TESTS 1. INTRODUCTION This chapter provides guidance for the design of bioassays

Bayesian modelling. Hans-Peter Helfrich. University of Bonn. Theodor-Brinkmann-Graduate School

Bayesian modelling Hans-Peter Helfrich University of Bonn Theodor-Brinkmann-Graduate School H.-P. Helfrich (University of Bonn) Bayesian modelling Brinkmann School 1 / 22 Overview 1 Bayesian modelling

Lecture 4 January 23

STAT 263/363: Experimental Design Winter 2016/17 Lecture 4 January 23 Lecturer: Art B. Owen Scribe: Zachary del Rosario 4.1 Bandits Bandits are a form of online (adaptive) experiments; i.e. samples are

Parametric Techniques

Parametric Techniques Jason J. Corso SUNY at Buffalo J. Corso (SUNY at Buffalo) Parametric Techniques 1 / 39 Introduction When covering Bayesian Decision Theory, we assumed the full probabilistic structure