A COEFFICIENT OF DETERMINATION FOR LOGISTIC REGRESSION MODELS
|
|
- Lauren Clarke
- 6 years ago
- Views:
Transcription
1 A COEFFICIENT OF DETEMINATION FO LOGISTIC EGESSION MODELS ENATO MICELI UNIVESITY OF TOINO After a brief presentation of the main extensions of the classical coefficient of determination ( ), a new index is proposed that can be used with Logistic for ungrouped data. This index is a direct extension of the classical coefficient of determination for linear models (link function identity and normal distribution for errors), and they share the same properties. Index performances (including the one proposed here) are compared by means of simulated data. Key words: Model Fit; Coefficient of Determination; Logistic regression models; Generalized Linear Models; Log likelihood. Correspondence concerning this article should be addressed to enato Miceli, Dipartimento di Psicologia, Università degli Studi di Torino, Via Verdi, 4 TOIO (TO), Italy. miceli@psych.unito.it INTODUCTION A large number of research studies in psychology applies models with categorical and limited dependent variables in statistical analysis. Such models usually belong to the large family of Generalized Linear Models (GLM) (McCullagh & Nelder, 983; Nelder & Wedderburn, 97). When data are gathered with non-experimental research methods (as in many studies using logistic regression models), the assessment of the goodness-of-fit raises problems due to the lack of a summary measure that can be easily interpreted, such as the coefficient of determination in classical regression linear models. The coefficient of determination ( ) in classical linear models (link function identity and normal distribution for errors) is widely used as a goodness-of-fit measure because of its interesting properties (ao, 973): (i) it ranges between and (the higher the fit, the more approximates, which is reached when the model perfectly reproduces the observed data); (ii) it is dimensionless, i.e., it is independent of the unit of measurement used for variables; (iii) it is independent of sample size (); (iv) it can be immediately and easily interpreted in that it can be expressed as the proportion of the deviance explained by the model with respect to the total deviance to be explained. In classical linear models, the parameters ( θˆ,ˆ θ,...,ˆθ K) can be estimated by Ordinary Least Squares (OLS) criterion and can be expressed as the ratio between explained deviance and deviance to explain ( observations and K variables): TPM Vol. 4, No., Summer 7 7 Cises 83
2 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. y) yi) = y) ) () where: ŷ i =θ ˆ K + θ ˆ k xik ; y = k= y i. Numerous suggestions were made for generalizing to various models, other than the classical linear one, even when deviance has to be replaced by the more general concept of variability, and the parameters are Maximum Likelihood (ML) estimates. Efforts were primarily made to extend to discrete models, in particular to logistic regression models for ungrouped data (Aldrich & Nelson, 984; Cox & Snell, 989; Maddala, 983; Magee, 99; Nagelkerke, 99). The index (here referred to as ), originally suggested by Maddala (983), and subsequently by Cox and Snell (989), and Magee (99), can be expressed as: L = L () where is the sample size; L and L denote the likelihoods of the fitted and the null (intercept only) model, respectively. The index (here referred to as A ), proposed by Aldrich and Nelson (984), can be expressed as: c A = + c L where c = log, generally referred to as likelihood ratio. L Even if both indexes present interesting aspects, they do not have property (i). In both cases, the maximum value is less than. In particular, the maximum value of equals: max = L Nagelkerke (99) proposed to correct ) that satisfies property (i), and that can be expressed as: (3), suggesting an index (here referred to as = max (4) It is easily found (Nagelkerke, 99) that not (i) and while properties (i), (ii) and (iii) hold for has the (ii), (iii) and (iv) properties, but, the same is not true of property (iv), which is of fundamental importance in providing a clear interpretation of the index values. Given that is a popular diagnostic tool in research and it varies between and, there is a high risk that its values may be interpreted as explained variation. Furthermore, this 84
3 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. risk could be even higher if as suggested by our simulations the index values always tend to suggest an optimistic interpretation of the explanatory power of the model under consideration. Obviously, in order to claim this, a measure having all the four properties mentioned above is needed. For this reason it appears useful to propose a new index here referred to as M or Maximal atio Index. THE MAXIMAL ATIO (M) INDEX It is useful to start thinking about a metric dependent variable (y) and a group of K metric explanatory variables, independent variables, or covariates. In such a context, K nested linear models (link function identity and normal distribution for errors) and the intercept only model can be estimated: besides the intercept, model M will contain only the variable x ; model M will contain x and x, and so forth. Equation () shows the strict proportionality linking K values of to as many values of the deviance explained by each model. In addition, by obtaining parameters through the ML estimator, the explained deviance is equivalent to the likelihood ratio (omitting the scale factor ) often referred to as c (Aldrich & Nelson, 984, p. 55); such ratio σ can be expressed as: Λ = L L (where L denotes the likelihood of the fitted model, and L denotes the likelihood of the null or intercept only model). The deviance explained by the fitted model can thus be expressed as: c [ log( L ) ( )] logλ= log = L (5) Therefore, within classical linear models (link function identity and normal distribution for errors), can be interpreted as in (iv) taking into account the increments in the explained deviance, as well as the increments in the likelihood ratio. In the context of logistic regression models the concept of explained deviance has to be replaced by the more general concept of explained variability and, given that c measures the latter, it seems obvious to develop a measure of fit proportional to this statistic. On the other hand, within GLM, a statistic also indicated as likelihood ratio (see Dobson, 99, p6) is often used, but its meaning is completely different from that of statistic c. Such ratio can be expressed as: λ = L max L (where L denotes the likelihood of the fitted model, but L max denotes the likelihood of the maximal or full model). Nelder and Wedderburn (97) proposed to use twice the logarithm of such ratio as measure of fit of any generalized linear model. They indicated such statistic with the term deviance, so as to evoke the statistic that has the same name in classical linear models, and to underline the extension of such concept to the whole generalized linear models family, even when the simple residual sum of squares can no longer be calculated, or is meaningless. Such statistic, in relation to a generic fitted model, can thus be expressed as: [ log( L ) ( )] D = L (6) logλ= max log While statistic c expressed the contribution of the covariates to the model fit of the dependent variable (so to speak, the way that has been gone thanks to the model), now statistic 85
4 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. D expresses the amount of discrepancy that, in spite of the model, is still present ( the way that still has to be gone ). The use of a maximal model in the assessment of fit is commonly associated with a certain type of models (for example, log-linear models), or with particular research contexts (confirmatory or experimental methods), when the model may comprise as many covariates as there are observations. Vice versa, a maximal model is not suitable for studies conducted with nonexperimental methods when, for exploratory purposes, researchers deal with a great amount of observations and no a priori defined group of covariates as it often happens when using a logistic regression model. This may be the reason why, in research practice, each statistic (both c and D ) is exclusively restricted to a specific world or domain. Nonetheless, there is a point in which the two worlds meet: this is the intercept model. Thus, the calculation of statistic D for the intercept model (of any model belonging to the GLM family and hence even for logistic regression models) yields a measure of the variability that covariates still have to explain. Such statistic is here referred to as D : D [ log( L ) ( )] logλ= max log = L (7) Now, in the context of logistic regression models, having a measure of explained variability (c ) and a measure of variability to explain (D ) at our disposal, the Maximal atio (M) can be expressed as: c M= D Thus, it is easy to demonstrate that in the case of classical linear models (link function identity and normal distribution for errors), this ratio coincides with (Miceli,, p. 6-6), and obviously it has the same well known properties, including the one of varying between and and of being proportional to the amount of explained variability. The main steps of the demonstration are reported below; for classical linear models (link function identity and normal distribution for errors) we can write the log-likelihood function of the generic model with k covariates (k < ) and σ for dispersion parameter as: l = ) ( y y ) log( πσ ) i σ where: ŷ i =θ ˆ + K θˆ x k ik ; k= the log-likelihood function of the maximal or full model, when y = yˆ ( i ), is: ( πσ ) l max = log the log-likelihood function of the null or intercept only model, when = y ( i ) and y = y i, is: ( y y) log( πσ ) = i σ l i i ŷ i (8) 86
5 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. Then c = ( l l) = y) log( πσ ) + ŷi) + log( πσ ) =+ σ σ y) ŷi) σ = D = ( lmax l) = log( πσ ) + y) + log( πσ ) =+ y) σ σ And c M= D = σ y) ŷi) σ y) = y) ŷi) y) = EMPIICAL COMPAISON BETWEEN THE DIFFEENT INDEXES Through simulated data, it is now possible to compare the performance of the different indexes. Simulations were conducted by generating, for different sample sizes ( = ; = 3; exp( X i) = 3), a continuous latent variable (y), obtained from yi =, where X i denotes + exp X the linear combination of 5 normally distributed random variables, and as many coefficients (plus the intercept). For each sample size, two types of continuous variable (y) were generated, as shown in Figure : simulation type A with about 36% of its values falling into the. interval, thus presenting a clear-cut logistic trend; and simulation type B, with about 86% of its values falling into the same interval, presenting a like linear trend. For each simulation type (A and B) and for each sample size (, 3, and 3), nine cutting points were then defined, in order to generate as many dummy variables (D, D,..., D9), so that each of them had a different frequency of value, as illustrated below: ( ) i Dummy variable D D D3 D4 D5 D6 D7 D8 D9 Frequencies of (%) Each of the 54 dummies thus generated was then used as dependent variable in 5 logistic regression models, thus computing an overall ML estimate of 8 models. The variables of the various logistic regression models were organized so as to define, for each dummy, a group of 5 nested models (M, M,..., M5). 87
6 TPM Vol. 4, No., Summer 7 7 Cises Miceli, ote. Simulation type A ( = ): latent dependent variable (y) ote. Simulation type B ( = ): latent dependent variable (y). FIGUE Two Types of Latent Dependent Variable y. The obtained results, partially reported in Table a, b, c, and Figure, permit us to express subsequent considerations (due to space limitations, Table a, b, c only report some results from simulation type A estimates, with = 3 (dependent variable: D, D3, and D5); Figure reports simulation type A graphs. The remaining results are in line with the ones presented here): (a) the four indexes provide different indications on the model fit; and A even show discordant values; (b) offers a model-data fit value closer to M, compared to the other indexes, yielding higher values in all occasions; 88
7 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. (c) M and yield very similar values in almost all simulations; however, discrepancies (with increasingly higher values of ) become larger in proximity of central values ( ), and when the frequency of value in the dependent variable is more or less balanced (4% 6%). TABLE A Comparison among Fit Indexes from Simulation Type A ( = 3) Model D c M A M M M M M M M M M M M M M M M ote. Fifteen nested models were simulated for dependent variable D (frequencies of value = 3%). TABLE B Comparison among Fit Indexes from Simulation Type A ( = 3) Model D c M A M M M M M M M M M M M M M M M ote. Fifteen nested models were simulated for dependent variable D3 (frequencies of value = 5%). 89
8 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. TABLE C Comparison among Fit Indexes from Simulation Type A ( = 3) Model D c M A M M M M M M M M M M M M M M M ote. Fifteen nested models were simulated for dependent variable D5 (frequencies of value = 5%). Dependent variable D.. ote M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 ; ; ; γ M A (figure continues) 9
9 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. FIGUE (continued) Dependent variable D.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 Dependent variable D3.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 Dependent variable D4.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 (figure continues) 9
10 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. FIGUE (continued) Dependent variable D5.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 Dependent variable D6.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 Dependent variable D7.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 (figure continues) 9
11 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. FIGUE (continued) Dependent variable D8.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 Dependent variable D9.. M M M3 M4 M5 M6 M7 M8 M9 M M M M3 M4 M5 FIGUE Comparison among Fit Indexes from Simulation Type A ( = 3) If it is important that the fit index may be interpreted as a proportion of explained variation, then it should be noted that always tends to suggest an optimistic interpretation of the explanatory power of the fitted model, that is to say a larger proportion of explained variation. In addition, this optimistic interpretation is not constant when data and models vary. This aspect can be verified by assessing the congruence between the increments in the variability explained by each model (expressed by statistic c ) and the corresponding increments in the fit index. Such evaluation can be done with nested models, as in this study. The strict proportionality between c and M can be derived by formula (8). On the contrary, as shown in Table, never strictly follows the increments in the explained variability: above all, the relation is not constant, and larger differences (with r values considerably lower than +) are observed for those dependent 93
12 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. variables that present a higher balance between and (D4, D5, and D6). Further, Table suggests that increasingly larger discrepancies can be observed as the sample size increases, and the more the latent variable (y) moves away from linearity (discrepancies in simulation type A are larger than in simulation type B). Table reports r values calculated across the increments of c and in relation to the 5 nested models estimated for each dependent variable. The values of the other two indexes ( and A ), were not reported due to space limitations. However, they are very similar to those of ; usually, values are remarkably lower. A TABLE Pearson Correlations between Likelihood atio c and (for each simulation type and each dependent variable) Simulation type D D D3 D4 D5 A B A B A B Simulation type D6 D7 D8 D9 A B A B A B The results of the present study are summarized in Figure 3. For each dependent variable, the values of the four fit indexes (on the ordinate) for each estimated model are shown, so that the trend of these values can be compared with the trend of the likelihood ratio c (the explained variation) on the abscissa. CONCLUSIONS The assessment of the goodness-of-fit for Logistic (ungrouped data) can be facilitated by an index allowing an easy interpretation, such as the coefficient of determination for classical linear regression models. The new index developed in this study (M) can be used as an alternative for the common indexes (proposed by Cox & Snell, 989, and by Nagelkerke, 99) that today are supplied by the most common statistical software packages. 94
13 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. This paper compares the performance of M with the other known indexes by means of simulated data. Dependent variable D ote. ; ; ; γ M A Dependent variable D Dependent variable D (figure continues) 95
14 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. FIGUE 3 (continued) Dependent variable D Dependent variable D Dependent variable D (figure continues) 96
15 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. FIGUE 3 (continued) Dependent variable D Dependent variable D Dependent variable D FIGUE 3 Comparison among Fit Indexes, with Likelihood atio c on the abscissa from Simulation Type A ( = 3) 97
16 TPM Vol. 4, No., Summer 7 7 Cises Miceli,. In particular, the main distinctive features of the M index are the following: it is easy to compute; in the case of classical linear models (link function identity and normal distribution for errors) it coincides with the classical coefficient of determination ( ); it varies between and ; its values may be interpreted as explained variation by the fitted model with respect to the total variation to be explained. EFEENCES Aldrich, J. H., & Nelson, F. D. (984). Linear Probability, Logit, and Probit Models. Sage University Paper Series on Quantitative Applications in the Social Sciences (pp-45). Beverly Hills and London: Sage Publications. Cox, D.., & Snell, E. J. (989). The Analysis of Binary Data ( nd ed.). London: Chapman & Hall. Dobson, A. J. (99). An Introduction to Generalized Linear Models. London: Chapman & Hall. Maddala, G. S. (983). Limited-dependent and Qualitative Variables in Econometrics. New York: Cambridge University Press. Magee, L. (99). Measures Based on Wald and Likelihood atio Joint Significance Test. American Statistician, 44, McCullagh, P., & Nelder, J. A. (983). Generalized Linear Models. New York: Chapman & Hall. Miceli,. (). Percorsi di icerca e Analisi dei Dati [esearch methods and data analysis]. Torino: Bollati Boringhieri. Nagelkerke, N. J. D. (99). A Note on a General Definition of the Coefficient of Determination. Biometrika, 78, Nelder, J. A., & Wedderburn,. W. M. (97). Generalized Linear Models. Journal of oyal Statistical Society, A, 35, ao, C.. (973). Linear Statistical Inference and its Applications ( nd ed.). New York: Wiley. 98
ONE MORE TIME ABOUT R 2 MEASURES OF FIT IN LOGISTIC REGRESSION
ONE MORE TIME ABOUT R 2 MEASURES OF FIT IN LOGISTIC REGRESSION Ernest S. Shtatland, Ken Kleinman, Emily M. Cain Harvard Medical School, Harvard Pilgrim Health Care, Boston, MA ABSTRACT In logistic regression,
More informationLOGISTIC REGRESSION Joseph M. Hilbe
LOGISTIC REGRESSION Joseph M. Hilbe Arizona State University Logistic regression is the most common method used to model binary response data. When the response is binary, it typically takes the form of
More informationGeneralized Linear Models
York SPIDA John Fox Notes Generalized Linear Models Copyright 2010 by John Fox Generalized Linear Models 1 1. Topics I The structure of generalized linear models I Poisson and other generalized linear
More information11. Generalized Linear Models: An Introduction
Sociology 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Copyright 2014 by John Fox Generalized Linear Models: An Introduction 1 1. Introduction I A synthesis due to Nelder and
More informationGeneralized Linear Models (GLZ)
Generalized Linear Models (GLZ) Generalized Linear Models (GLZ) are an extension of the linear modeling process that allows models to be fit to data that follow probability distributions other than the
More informationINFORMATION AS A UNIFYING MEASURE OF FIT IN SAS STATISTICAL MODELING PROCEDURES
INFORMATION AS A UNIFYING MEASURE OF FIT IN SAS STATISTICAL MODELING PROCEDURES Ernest S. Shtatland, PhD Mary B. Barton, MD, MPP Harvard Medical School, Harvard Pilgrim Health Care, Boston, MA ABSTRACT
More informationST3241 Categorical Data Analysis I Generalized Linear Models. Introduction and Some Examples
ST3241 Categorical Data Analysis I Generalized Linear Models Introduction and Some Examples 1 Introduction We have discussed methods for analyzing associations in two-way and three-way tables. Now we will
More informationLogistic Regression. Continued Psy 524 Ainsworth
Logistic Regression Continued Psy 524 Ainsworth Equations Regression Equation Y e = 1 + A+ B X + B X + B X 1 1 2 2 3 3 i A+ B X + B X + B X e 1 1 2 2 3 3 Equations The linear part of the logistic regression
More informationModel Estimation Example
Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions
More informationLOGISTICS REGRESSION FOR SAMPLE SURVEYS
4 LOGISTICS REGRESSION FOR SAMPLE SURVEYS Hukum Chandra Indian Agricultural Statistics Research Institute, New Delhi-002 4. INTRODUCTION Researchers use sample survey methodology to obtain information
More informationGeneralized Linear Models
Generalized Linear Models Lecture 3. Hypothesis testing. Goodness of Fit. Model diagnostics GLM (Spring, 2018) Lecture 3 1 / 34 Models Let M(X r ) be a model with design matrix X r (with r columns) r n
More informationSingle-level Models for Binary Responses
Single-level Models for Binary Responses Distribution of Binary Data y i response for individual i (i = 1,..., n), coded 0 or 1 Denote by r the number in the sample with y = 1 Mean and variance E(y) =
More informationGLM models and OLS regression
GLM models and OLS regression Graeme Hutcheson, University of Manchester These lecture notes are based on material published in... Hutcheson, G. D. and Sofroniou, N. (1999). The Multivariate Social Scientist:
More informationLogistic Regression: Regression with a Binary Dependent Variable
Logistic Regression: Regression with a Binary Dependent Variable LEARNING OBJECTIVES Upon completing this chapter, you should be able to do the following: State the circumstances under which logistic regression
More information9 Generalized Linear Models
9 Generalized Linear Models The Generalized Linear Model (GLM) is a model which has been built to include a wide range of different models you already know, e.g. ANOVA and multiple linear regression models
More informationGeneralized Linear Models 1
Generalized Linear Models 1 STA 2101/442: Fall 2012 1 See last slide for copyright information. 1 / 24 Suggested Reading: Davison s Statistical models Exponential families of distributions Sec. 5.2 Chapter
More information8 Nominal and Ordinal Logistic Regression
8 Nominal and Ordinal Logistic Regression 8.1 Introduction If the response variable is categorical, with more then two categories, then there are two options for generalized linear models. One relies on
More informationPackage rsq. January 3, 2018
Title R-Squared and Related Measures Version 1.0.1 Date 2017-12-31 Author Dabao Zhang Package rsq January 3, 2018 Maintainer Dabao Zhang Calculate generalized R-squared, partial
More informationChapter 1 Statistical Inference
Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations
More informationNormal distribution We have a random sample from N(m, υ). The sample mean is Ȳ and the corrected sum of squares is S yy. After some simplification,
Likelihood Let P (D H) be the probability an experiment produces data D, given hypothesis H. Usually H is regarded as fixed and D variable. Before the experiment, the data D are unknown, and the probability
More informationSCHOOL OF MATHEMATICS AND STATISTICS. Linear and Generalised Linear Models
SCHOOL OF MATHEMATICS AND STATISTICS Linear and Generalised Linear Models Autumn Semester 2017 18 2 hours Attempt all the questions. The allocation of marks is shown in brackets. RESTRICTED OPEN BOOK EXAMINATION
More informationCorrelation and regression
1 Correlation and regression Yongjua Laosiritaworn Introductory on Field Epidemiology 6 July 2015, Thailand Data 2 Illustrative data (Doll, 1955) 3 Scatter plot 4 Doll, 1955 5 6 Correlation coefficient,
More informationClass Notes: Week 8. Probit versus Logit Link Functions and Count Data
Ronald Heck Class Notes: Week 8 1 Class Notes: Week 8 Probit versus Logit Link Functions and Count Data This week we ll take up a couple of issues. The first is working with a probit link function. While
More informationGeneralized Linear Models: An Introduction
Applied Statistics With R Generalized Linear Models: An Introduction John Fox WU Wien May/June 2006 2006 by John Fox Generalized Linear Models: An Introduction 1 A synthesis due to Nelder and Wedderburn,
More informationCategorical data analysis Chapter 5
Categorical data analysis Chapter 5 Interpreting parameters in logistic regression The sign of β determines whether π(x) is increasing or decreasing as x increases. The rate of climb or descent increases
More informationApplication of Poisson and Negative Binomial Regression Models in Modelling Oil Spill Data in the Niger Delta
International Journal of Science and Engineering Investigations vol. 7, issue 77, June 2018 ISSN: 2251-8843 Application of Poisson and Negative Binomial Regression Models in Modelling Oil Spill Data in
More informationStatistical Models for Management. Instituto Superior de Ciências do Trabalho e da Empresa (ISCTE) Lisbon. February 24 26, 2010
Statistical Models for Management Instituto Superior de Ciências do Trabalho e da Empresa (ISCTE) Lisbon February 24 26, 2010 Graeme Hutcheson, University of Manchester GLM models and OLS regression The
More informationSAS Software to Fit the Generalized Linear Model
SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling
More informationCHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA
STATISTICS IN MEDICINE, VOL. 17, 59 68 (1998) CHOOSING AMONG GENERALIZED LINEAR MODELS APPLIED TO MEDICAL DATA J. K. LINDSEY AND B. JONES* Department of Medical Statistics, School of Computing Sciences,
More information1. Hypothesis testing through analysis of deviance. 3. Model & variable selection - stepwise aproaches
Sta 216, Lecture 4 Last Time: Logistic regression example, existence/uniqueness of MLEs Today s Class: 1. Hypothesis testing through analysis of deviance 2. Standard errors & confidence intervals 3. Model
More informationModels for Binary Outcomes
Models for Binary Outcomes Introduction The simple or binary response (for example, success or failure) analysis models the relationship between a binary response variable and one or more explanatory variables.
More informationRepeated ordinal measurements: a generalised estimating equation approach
Repeated ordinal measurements: a generalised estimating equation approach David Clayton MRC Biostatistics Unit 5, Shaftesbury Road Cambridge CB2 2BW April 7, 1992 Abstract Cumulative logit and related
More informationLongitudinal Modeling with Logistic Regression
Newsom 1 Longitudinal Modeling with Logistic Regression Longitudinal designs involve repeated measurements of the same individuals over time There are two general classes of analyses that correspond to
More informationNATIONAL UNIVERSITY OF SINGAPORE EXAMINATION. ST3241 Categorical Data Analysis. (Semester II: ) April/May, 2011 Time Allowed : 2 Hours
NATIONAL UNIVERSITY OF SINGAPORE EXAMINATION Categorical Data Analysis (Semester II: 2010 2011) April/May, 2011 Time Allowed : 2 Hours Matriculation No: Seat No: Grade Table Question 1 2 3 4 5 6 Full marks
More informationStandard Errors & Confidence Intervals. N(0, I( β) 1 ), I( β) = [ 2 l(β, φ; y) β i β β= β j
Standard Errors & Confidence Intervals β β asy N(0, I( β) 1 ), where I( β) = [ 2 l(β, φ; y) ] β i β β= β j We can obtain asymptotic 100(1 α)% confidence intervals for β j using: β j ± Z 1 α/2 se( β j )
More informationA NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL
Discussiones Mathematicae Probability and Statistics 36 206 43 5 doi:0.75/dmps.80 A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Tadeusz Bednarski Wroclaw University e-mail: t.bednarski@prawo.uni.wroc.pl
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models Generalized Linear Models - part III Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs.
More informationNon-maximum likelihood estimation and statistical inference for linear and nonlinear mixed models
Optimum Design for Mixed Effects Non-Linear and generalized Linear Models Cambridge, August 9-12, 2011 Non-maximum likelihood estimation and statistical inference for linear and nonlinear mixed models
More informationMathematical Modelling of RMSE Approach on Agricultural Financial Data Sets
Available online at www.ijpab.com Babu et al Int. J. Pure App. Biosci. 5 (6): 942-947 (2017) ISSN: 2320 7051 DOI: http://dx.doi.org/10.18782/2320-7051.5802 ISSN: 2320 7051 Int. J. Pure App. Biosci. 5 (6):
More informationGeneralized Linear Models. Last time: Background & motivation for moving beyond linear
Generalized Linear Models Last time: Background & motivation for moving beyond linear regression - non-normal/non-linear cases, binary, categorical data Today s class: 1. Examples of count and ordered
More informationEconometrics II. Seppo Pynnönen. Spring Department of Mathematics and Statistics, University of Vaasa, Finland
Department of Mathematics and Statistics, University of Vaasa, Finland Spring 2018 Part III Limited Dependent Variable Models As of Jan 30, 2017 1 Background 2 Binary Dependent Variable The Linear Probability
More informationCOMPOSITIONAL IDEAS IN THE BAYESIAN ANALYSIS OF CATEGORICAL DATA WITH APPLICATION TO DOSE FINDING CLINICAL TRIALS
COMPOSITIONAL IDEAS IN THE BAYESIAN ANALYSIS OF CATEGORICAL DATA WITH APPLICATION TO DOSE FINDING CLINICAL TRIALS M. Gasparini and J. Eisele 2 Politecnico di Torino, Torino, Italy; mauro.gasparini@polito.it
More informationSOS3003 Applied data analysis for social science Lecture note Erling Berge Department of sociology and political science NTNU.
SOS3003 Applied data analysis for social science Lecture note 08-00 Erling Berge Department of sociology and political science NTNU Erling Berge 00 Literature Logistic regression II Hamilton Ch 7 p7-4
More informationGeneralized linear models
Generalized linear models Outline for today What is a generalized linear model Linear predictors and link functions Example: estimate a proportion Analysis of deviance Example: fit dose- response data
More informationClassification. Chapter Introduction. 6.2 The Bayes classifier
Chapter 6 Classification 6.1 Introduction Often encountered in applications is the situation where the response variable Y takes values in a finite set of labels. For example, the response Y could encode
More informationExperimental Design and Statistical Methods. Workshop LOGISTIC REGRESSION. Jesús Piedrafita Arilla.
Experimental Design and Statistical Methods Workshop LOGISTIC REGRESSION Jesús Piedrafita Arilla jesus.piedrafita@uab.cat Departament de Ciència Animal i dels Aliments Items Logistic regression model Logit
More informationGeneralized linear models for binary data. A better graphical exploratory data analysis. The simple linear logistic regression model
Stat 3302 (Spring 2017) Peter F. Craigmile Simple linear logistic regression (part 1) [Dobson and Barnett, 2008, Sections 7.1 7.3] Generalized linear models for binary data Beetles dose-response example
More informationSample size determination for logistic regression: A simulation study
Sample size determination for logistic regression: A simulation study Stephen Bush School of Mathematical Sciences, University of Technology Sydney, PO Box 123 Broadway NSW 2007, Australia Abstract This
More informationStatistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationSimple ways to interpret effects in modeling ordinal categorical data
DOI: 10.1111/stan.12130 ORIGINAL ARTICLE Simple ways to interpret effects in modeling ordinal categorical data Alan Agresti 1 Claudia Tarantola 2 1 Department of Statistics, University of Florida, Gainesville,
More information12 Modelling Binomial Response Data
c 2005, Anthony C. Brooms Statistical Modelling and Data Analysis 12 Modelling Binomial Response Data 12.1 Examples of Binary Response Data Binary response data arise when an observation on an individual
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationMULTINOMIAL LOGISTIC REGRESSION
MULTINOMIAL LOGISTIC REGRESSION Model graphically: Variable Y is a dependent variable, variables X, Z, W are called regressors. Multinomial logistic regression is a generalization of the binary logistic
More informationMultilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2
Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do
More informationMultinomial Logistic Regression Models
Stat 544, Lecture 19 1 Multinomial Logistic Regression Models Polytomous responses. Logistic regression can be extended to handle responses that are polytomous, i.e. taking r>2 categories. (Note: The word
More informationGeneralized Linear Models for Non-Normal Data
Generalized Linear Models for Non-Normal Data Today s Class: 3 parts of a generalized model Models for binary outcomes Complications for generalized multivariate or multilevel models SPLH 861: Lecture
More informationGeneralized Linear. Mixed Models. Methods and Applications. Modern Concepts, Walter W. Stroup. Texts in Statistical Science.
Texts in Statistical Science Generalized Linear Mixed Models Modern Concepts, Methods and Applications Walter W. Stroup CRC Press Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint
More informationExperimental Design and Data Analysis for Biologists
Experimental Design and Data Analysis for Biologists Gerry P. Quinn Monash University Michael J. Keough University of Melbourne CAMBRIDGE UNIVERSITY PRESS Contents Preface page xv I I Introduction 1 1.1
More informationH-LIKELIHOOD ESTIMATION METHOOD FOR VARYING CLUSTERED BINARY MIXED EFFECTS MODEL
H-LIKELIHOOD ESTIMATION METHOOD FOR VARYING CLUSTERED BINARY MIXED EFFECTS MODEL Intesar N. El-Saeiti Department of Statistics, Faculty of Science, University of Bengahzi-Libya. entesar.el-saeiti@uob.edu.ly
More informationGeneralized Linear Models
Generalized Linear Models Methods@Manchester Summer School Manchester University July 2 6, 2018 Generalized Linear Models: a generic approach to statistical modelling www.research-training.net/manchester2018
More informationpoisson: Some convergence issues
The Stata Journal (2011) 11, Number 2, pp. 207 212 poisson: Some convergence issues J. M. C. Santos Silva University of Essex and Centre for Applied Mathematics and Economics Colchester, United Kingdom
More informationOutline of GLMs. Definitions
Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density
More informationBIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY
BIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY Ingo Langner 1, Ralf Bender 2, Rebecca Lenz-Tönjes 1, Helmut Küchenhoff 2, Maria Blettner 2 1
More informationSurvival Analysis Math 434 Fall 2011
Survival Analysis Math 434 Fall 2011 Part IV: Chap. 8,9.2,9.3,11: Semiparametric Proportional Hazards Regression Jimin Ding Math Dept. www.math.wustl.edu/ jmding/math434/fall09/index.html Basic Model Setup
More informationChapter 9 Regression with a Binary Dependent Variable. Multiple Choice. 1) The binary dependent variable model is an example of a
Chapter 9 Regression with a Binary Dependent Variable Multiple Choice ) The binary dependent variable model is an example of a a. regression model, which has as a regressor, among others, a binary variable.
More informationNATIONAL UNIVERSITY OF SINGAPORE EXAMINATION (SOLUTIONS) ST3241 Categorical Data Analysis. (Semester II: )
NATIONAL UNIVERSITY OF SINGAPORE EXAMINATION (SOLUTIONS) Categorical Data Analysis (Semester II: 2010 2011) April/May, 2011 Time Allowed : 2 Hours Matriculation No: Seat No: Grade Table Question 1 2 3
More informationDISPLAYING THE POISSON REGRESSION ANALYSIS
Chapter 17 Poisson Regression Chapter Table of Contents DISPLAYING THE POISSON REGRESSION ANALYSIS...264 ModelInformation...269 SummaryofFit...269 AnalysisofDeviance...269 TypeIII(Wald)Tests...269 MODIFYING
More informationLogistic Regression. Fitting the Logistic Regression Model BAL040-A.A.-10-MAJ
Logistic Regression The goal of a logistic regression analysis is to find the best fitting and most parsimonious, yet biologically reasonable, model to describe the relationship between an outcome (dependent
More informationCh 6: Multicategory Logit Models
293 Ch 6: Multicategory Logit Models Y has J categories, J>2. Extensions of logistic regression for nominal and ordinal Y assume a multinomial distribution for Y. In R, we will fit these models using the
More informationConfirmatory Factor Analysis: Model comparison, respecification, and more. Psychology 588: Covariance structure and factor models
Confirmatory Factor Analysis: Model comparison, respecification, and more Psychology 588: Covariance structure and factor models Model comparison 2 Essentially all goodness of fit indices are descriptive,
More informationPsychology 282 Lecture #4 Outline Inferences in SLR
Psychology 282 Lecture #4 Outline Inferences in SLR Assumptions To this point we have not had to make any distributional assumptions. Principle of least squares requires no assumptions. Can use correlations
More informationSTA 303 H1S / 1002 HS Winter 2011 Test March 7, ab 1cde 2abcde 2fghij 3
STA 303 H1S / 1002 HS Winter 2011 Test March 7, 2011 LAST NAME: FIRST NAME: STUDENT NUMBER: ENROLLED IN: (circle one) STA 303 STA 1002 INSTRUCTIONS: Time: 90 minutes Aids allowed: calculator. Some formulae
More informationLongitudinal and Panel Data: Analysis and Applications for the Social Sciences. Table of Contents
Longitudinal and Panel Data Preface / i Longitudinal and Panel Data: Analysis and Applications for the Social Sciences Table of Contents August, 2003 Table of Contents Preface i vi 1. Introduction 1.1
More informationModel Based Statistics in Biology. Part V. The Generalized Linear Model. Chapter 16 Introduction
Model Based Statistics in Biology. Part V. The Generalized Linear Model. Chapter 16 Introduction ReCap. Parts I IV. The General Linear Model Part V. The Generalized Linear Model 16 Introduction 16.1 Analysis
More informationTento projekt je spolufinancován Evropským sociálním fondem a Státním rozpočtem ČR InoBio CZ.1.07/2.2.00/
Tento projekt je spolufinancován Evropským sociálním fondem a Státním rozpočtem ČR InoBio CZ.1.07/2.2.00/28.0018 Statistical Analysis in Ecology using R Linear Models/GLM Ing. Daniel Volařík, Ph.D. 13.
More informationGeneralized linear models
Generalized linear models Douglas Bates November 01, 2010 Contents 1 Definition 1 2 Links 2 3 Estimating parameters 5 4 Example 6 5 Model building 8 6 Conclusions 8 7 Summary 9 1 Generalized Linear Models
More informationGroup comparisons in logit and probit using predicted probabilities 1
Group comparisons in logit and probit using predicted probabilities 1 J. Scott Long Indiana University May 27, 2009 Abstract The comparison of groups in regression models for binary outcomes is complicated
More information11. Generalized Linear Models: An Introduction
Sociolog 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Generalized Linear Models: An Introduction 1 1. Introduction I A snthesis due to Nelder and Wedderburn, generalized linear
More informationLecture 12: Effect modification, and confounding in logistic regression
Lecture 12: Effect modification, and confounding in logistic regression Ani Manichaikul amanicha@jhsph.edu 4 May 2007 Today Categorical predictor create dummy variables just like for linear regression
More informationEconometric Analysis of Cross Section and Panel Data
Econometric Analysis of Cross Section and Panel Data Jeffrey M. Wooldridge / The MIT Press Cambridge, Massachusetts London, England Contents Preface Acknowledgments xvii xxiii I INTRODUCTION AND BACKGROUND
More informationWU Weiterbildung. Linear Mixed Models
Linear Mixed Effects Models WU Weiterbildung SLIDE 1 Outline 1 Estimation: ML vs. REML 2 Special Models On Two Levels Mixed ANOVA Or Random ANOVA Random Intercept Model Random Coefficients Model Intercept-and-Slopes-as-Outcomes
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models Generalized Linear Models - part II Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs.
More informationAnalysis of Categorical Data. Nick Jackson University of Southern California Department of Psychology 10/11/2013
Analysis of Categorical Data Nick Jackson University of Southern California Department of Psychology 10/11/2013 1 Overview Data Types Contingency Tables Logit Models Binomial Ordinal Nominal 2 Things not
More informationReview: what is a linear model. Y = β 0 + β 1 X 1 + β 2 X 2 + A model of the following form:
Outline for today What is a generalized linear model Linear predictors and link functions Example: fit a constant (the proportion) Analysis of deviance table Example: fit dose-response data using logistic
More informationStructural Equation Modeling and Confirmatory Factor Analysis. Types of Variables
/4/04 Structural Equation Modeling and Confirmatory Factor Analysis Advanced Statistics for Researchers Session 3 Dr. Chris Rakes Website: http://csrakes.yolasite.com Email: Rakes@umbc.edu Twitter: @RakesChris
More informationLecture notes to Chapter 11, Regression with binary dependent variables - probit and logit regression
Lecture notes to Chapter 11, Regression with binary dependent variables - probit and logit regression Tore Schweder October 28, 2011 Outline Examples of binary respons variables Probit and logit - examples
More informationLinear Regression Models P8111
Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started
More informationGeneralized logit models for nominal multinomial responses. Local odds ratios
Generalized logit models for nominal multinomial responses Categorical Data Analysis, Summer 2015 1/17 Local odds ratios Y 1 2 3 4 1 π 11 π 12 π 13 π 14 π 1+ X 2 π 21 π 22 π 23 π 24 π 2+ 3 π 31 π 32 π
More informationSTA 4504/5503 Sample Exam 1 Spring 2011 Categorical Data Analysis. 1. Indicate whether each of the following is true (T) or false (F).
STA 4504/5503 Sample Exam 1 Spring 2011 Categorical Data Analysis 1. Indicate whether each of the following is true (T) or false (F). (a) T In 2 2 tables, statistical independence is equivalent to a population
More informationInvestigating Models with Two or Three Categories
Ronald H. Heck and Lynn N. Tabata 1 Investigating Models with Two or Three Categories For the past few weeks we have been working with discriminant analysis. Let s now see what the same sort of model might
More informationIntroduction to Generalized Linear Models
Introduction to Generalized Linear Models Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2018 Outline Introduction (motivation
More informationModel Based Statistics in Biology. Part V. The Generalized Linear Model. Chapter 18.1 Logistic Regression (Dose - Response)
Model Based Statistics in Biology. Part V. The Generalized Linear Model. Logistic Regression ( - Response) ReCap. Part I (Chapters 1,2,3,4), Part II (Ch 5, 6, 7) ReCap Part III (Ch 9, 10, 11), Part IV
More informationStatistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
More informationA Practitioner s Guide to Generalized Linear Models
A Practitioners Guide to Generalized Linear Models Background The classical linear models and most of the minimum bias procedures are special cases of generalized linear models (GLMs). GLMs are more technically
More informationProcedia - Social and Behavioral Sciences 109 ( 2014 )
Available online at www.sciencedirect.com ScienceDirect Procedia - Social and Behavioral Sciences 09 ( 04 ) 730 736 nd World Conference On Business, Economics And Management - WCBEM 03 Categorical Principal
More informationRegression models for multivariate ordered responses via the Plackett distribution
Journal of Multivariate Analysis 99 (2008) 2472 2478 www.elsevier.com/locate/jmva Regression models for multivariate ordered responses via the Plackett distribution A. Forcina a,, V. Dardanoni b a Dipartimento
More informationImproving the Precision of Estimation by fitting a Generalized Linear Model, and Quasi-likelihood.
Improving the Precision of Estimation by fitting a Generalized Linear Model, and Quasi-likelihood. P.M.E.Altham, Statistical Laboratory, University of Cambridge June 27, 2006 This article was published
More informationTreatment Variables INTUB duration of endotracheal intubation (hrs) VENTL duration of assisted ventilation (hrs) LOWO2 hours of exposure to 22 49% lev
Variable selection: Suppose for the i-th observational unit (case) you record ( failure Y i = 1 success and explanatory variabales Z 1i Z 2i Z ri Variable (or model) selection: subject matter theory and
More informationExam Applied Statistical Regression. Good Luck!
Dr. M. Dettling Summer 2011 Exam Applied Statistical Regression Approved: Tables: Note: Any written material, calculator (without communication facility). Attached. All tests have to be done at the 5%-level.
More informationGeneralized Linear Models I
Statistics 203: Introduction to Regression and Analysis of Variance Generalized Linear Models I Jonathan Taylor - p. 1/16 Today s class Poisson regression. Residuals for diagnostics. Exponential families.
More information