# Residuals and model diagnostics

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Residuals and model diagnostics Patrick Breheny November 10 Patrick Breheny Survival Data Analysis (BIOS 7210) 1/42

2 Introduction Residuals Many assumptions go into regression models, and the Cox proportional hazards model, despite making no assumptions about the baseline hazard, is no exception Diagnostic methods are useful in all types of regression models to investigate the validity of those assumptions and identify ways in which they might be violated Residuals play a big role in regression method diagnostics To build model diagnostics for Cox regression, we first need to discuss methods for extending residuals to the case of censored data Patrick Breheny Survival Data Analysis (BIOS 7210) 2/42

3 Cumulative hazard transformation We begin with the following useful theorem: Theorem: Suppose T is a continuous nonnegative random variable with cumulative hazard function Λ. Then the random variable Y = Λ(T ) follows an exponential distribution with rate λ = 1. Thus, one way of checking the validity of a model is by comparing the model s estimates {ˆΛ(t i )} against the standard exponential distribution Patrick Breheny Survival Data Analysis (BIOS 7210) 3/42

4 In the context of the proportional hazards model, we have ê i = ˆΛ 0 (t i ) exp(x T i β), although the idea is very general and can be applied to any kind of model The terms {ê i } are called the, although residual is perhaps a misnomer, in that it s more of a transformation than a residual Patrick Breheny Survival Data Analysis (BIOS 7210) 4/42

5 Diagnostic plot of : PBC data Diagnostics based on are based on fitting a Kaplan-Meier (or Nelson-Aalen) curve to {ê i } and comparing it to that of the standard exponential For the PBC data with trt, stage, hepato, and bili included, we have 2.0 Cumulative hazard Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 5/42

6 Simulated example: Lack of fit To illustrate the utility of the plot, let s simulate some data from a lognormal AFT model and fit a Cox model (note that the PH assumption is violated here): 6 5 Cumulative hazard Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 6/42

7 Zooming in Residuals One shortcoming of this plot is that violations at small e values tend to be hidden: Cumulative hazard Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 7/42

8 Plotting the standardized difference Alternatively, one may consider plotting the standardized difference between the fitted cumulative hazard and the standard exponential, which reveals the lack of fit much more clearly: 3 2 (Λ^(e) e) SE Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 8/42

9 Standardized differences: PBC data Here is what the standardized plot looks like for the PBC data: 3 2 (Λ^(e) e) SE Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 9/42

10 GVHD data Residuals Stratification of the residuals according to one of the variables in the model can also help to discover model violations For example, in the GVHD data, here is what the overall plot looks like: Cumulative hazard Residual Patrick Breheny Survival Data Analysis (BIOS 7210) 10/42

11 GVHD data: Stratifying by treatment And here is the plot stratifying by treatment: Cumulative hazard Residual We will discuss stratification in more detail in the next lecture Patrick Breheny Survival Data Analysis (BIOS 7210) 11/42

12 Shortcomings of the Cox-Snell residual One drawback to the is that they don t provide much insight into why the model s assumptions are violated It would be more appealing if each residual took on a positive or negative value indicating whether they patient survived longer, as opposed to shorter, that the model predicts, and by how much Patrick Breheny Survival Data Analysis (BIOS 7210) 12/42

13 Consider, then, the following residual: ˆm i = d i ˆΛ i (t i ) = d i ê i ; we have seen this quantity a few times already (one-sample logrank test, score function for exponential regression) This represents the discrepancy between the observed value of a subject s failure indicator and its expected value, integrated over the time for which that patient was at risk Positive values mean that the patient died sooner than expected (according to the model); negative values mean that the patient lived longer than expected (or were censored) Patrick Breheny Survival Data Analysis (BIOS 7210) 13/42

14 Martingales Residuals A stochastic process M(t) that satisfies (i) EM(t) = 0 for all t and (ii) E{M(t) M(s)} = M(s)} for all s < t is known in statistics as a martingale The stochastic process N(t) t 0 Y (t)dλ(t), where N(t) is the counting process that records whether the subject has failed by time t or not, satisfies these two properties For this reason, the residuals { ˆm i } defined on the previous slide are known as martingale residuals Patrick Breheny Survival Data Analysis (BIOS 7210) 14/42

15 for the PBC model Censored Died / Liver failure Martingale residual Time The large outlier is a patient with stage 4 cirrhosis and a bilirubin concentration of 14.4 (96th percentile), yet survived 7 years Patrick Breheny Survival Data Analysis (BIOS 7210) 15/42

16 residuals Residuals can be obtained from the survival package by calling residuals(fit), where fit is a fitted coxph model (resid(fit) also works as a shortcut) The martingale residuals are returned by default, although seven other options are available and can be requested by specifying the type option It may be noted that type= coxsnell is not one of the options, although you can easily calculate the Cox-Snell residuals from the martingale residuals Patrick Breheny Survival Data Analysis (BIOS 7210) 16/42

17 Comments Residuals are very useful and can be used for many of the usual purposes that we use residuals for in other models (identifying outliers, choosing a functional form for the covariate, etc.) However, the primary drawback to the martingale residual is its clear asymmetry (its upper bound is 1, but it has no lower bound) For this reason, I ll hold off on these plots until we discuss a more symmetric, normally distributed residual Patrick Breheny Survival Data Analysis (BIOS 7210) 17/42

18 : Motivation A technique for creating symmetric, normalized residuals that is widely used in generalized linear modeling is to construct a deviance residual The idea behind the deviance residual is to examine the difference between the log-likelihood for subject i under a given model and the maximum possible log-likelihood for that subject: 2( l i l i ), in a sense, constructing a miniature likelihood ratio test for individual i Patrick Breheny Survival Data Analysis (BIOS 7210) 18/42

19 : Definition As it is essentially a likelihood ratio test, the quantity on the previous slide should approximately follow a χ 2 1 distribution To turn it into a quantity that approximately follows a normal distribution, we can use ˆd i = sign( ˆm i ) 2( l i l i ); this is known as the deviance residual I am leaving the details of deriving l i and working out a simple expression for the deviance residual as homework In R: residuals(fit, type= deviance ) Patrick Breheny Survival Data Analysis (BIOS 7210) 19/42

20 for the PBC data The deviance residuals are much more symmetric: Time Deviance residual Censored Died / Liver failure Patrick Breheny Survival Data Analysis (BIOS 7210) 20/42

21 Outliers Residuals have several uses that we will now illustrate One is identifying outliers For the PBC data, there are no extreme outliers; the largest residuals are only 2.5 SDs away from zero Note that the skewness of the martingale residual makes one subject look like an extreme outlier; according to the deviance residuals, however, that subject is only the 5th largest outlier by absolute value Patrick Breheny Survival Data Analysis (BIOS 7210) 21/42

22 Outliers (cont d) The five largest negative residuals: time status trt stage hepato bili ˆd And the five largest positive residuals: time status trt stage hepato bili ˆd Patrick Breheny Survival Data Analysis (BIOS 7210) 22/42

23 Residuals vs. covariates The other main use of residuals is to plot them against covariates to assess the relationship between a covariate and unexplained variation This can be done using covariates that are already in the model as well as new covariates that one is considering adding to the model; we will consider examples of both Patrick Breheny Survival Data Analysis (BIOS 7210) 23/42

24 Residuals vs. albumin Albumin is a protein synthesized by the liver and often used as a marker of liver disease: Albumin Deviance residual Patrick Breheny Survival Data Analysis (BIOS 7210) 24/42

25 Assessing functional forms Albumin Bilirubin Stage As the previous figure shows, deviance residuals are helpful not only for checking whether new variables should be added to a model, but also for assessing whether the relationship between the predictor and the (log) hazard is linear In the case of albumin, the exploratory plot, as well as biological insight (the normal range of serum albumin is g/dl) suggest a piecewise linear model with a change point (sometimes called a knot ) at 3.4 Patrick Breheny Survival Data Analysis (BIOS 7210) 25/42

26 Implementation details Albumin Bilirubin Stage This can be implemented in various ways; here is one: f <- function(x) {pmin(x, 3.5)} fit <- coxph(s ~... + f(albumin), PBC) For nonlinear functional forms, it is typically helpful to plot the modeled relationship between a covariate and the outcome Again, this can be done in various ways, but the visreg package is useful here: visreg(fit, 'albumin') Patrick Breheny Survival Data Analysis (BIOS 7210) 26/42

27 Albumin Bilirubin Stage Changepoint model for albumin: Illustration The dots here are the partial deviance residuals, η i + ˆd i Albumin Linear predictor Patrick Breheny Survival Data Analysis (BIOS 7210) 27/42

28 Albumin summary Albumin Bilirubin Stage The AIC values for four possible models: No albumin: Linear: Changepoint, no effect in normal range: Changepoint, linear effect in normal: The changepoint model that assumes a flat line after 3.5 is the reasonably clear choice based on AIC, as well as being a very reasonable model from a biological perspective Patrick Breheny Survival Data Analysis (BIOS 7210) 28/42

29 Albumin Bilirubin Stage Residuals for bilirubin: In and out of the model Bilirubin Deviance residual without bili Bilirubin Deviance residual with bili Patrick Breheny Survival Data Analysis (BIOS 7210) 29/42

30 Albumin Bilirubin Stage log(bilirubin) The residual plots on the previous page suggest log(bili) as a functional form: Bilirubin Linear predictor Patrick Breheny Survival Data Analysis (BIOS 7210) 30/42

31 More on logs Residuals Albumin Bilirubin Stage One advantage of the log scale as a functional form is that it has a convenient interpretation in proportional hazards models Consider comparing two individuals, one with double the bilirubin concentration of the other, but all other covariates equal; the hazard ratio comparing the two is: HR = 2 β j For log(bili), β j = 0.895; thus, doubling the bilirubin concentration increases the hazard by 86% ( = 1.86) Patrick Breheny Survival Data Analysis (BIOS 7210) 31/42

32 Splines Residuals Albumin Bilirubin Stage Splines are also an attractive option for incorporating nonlinear functional forms The basic idea of splines is similar to our albumin model from earlier: fit a piecewise polynomial model, with restrictions that make the resulting functional form smooth and continuous The details of this are very interesting, but unfortunately we don t have time to get into them in this course Still, it is worth knowing that the survival package has a nice utility built in for fitting Cox models with spline terms: fit <- coxph(s ~... + pspline(bili), PBC) Patrick Breheny Survival Data Analysis (BIOS 7210) 32/42

33 Albumin Bilirubin Stage Spline illustration Bilirubin Linear predictor Patrick Breheny Survival Data Analysis (BIOS 7210) 33/42

34 Bilirubin: Summary Albumin Bilirubin Stage The AIC values for four possible models: No bilirubin: Linear: Log(bilirubin) Splines: The log model seems to be the clear choice here Patrick Breheny Survival Data Analysis (BIOS 7210) 34/42

35 Albumin Bilirubin Stage Stage Before moving on, let s quickly revisit the effect of stage: Stage Linear predictor Patrick Breheny Survival Data Analysis (BIOS 7210) 35/42

36 Remarks Residuals Albumin Bilirubin Stage The effect of stage seems to be very linear among stages 2, 3, and 4 Stage 1 patients, however, seem to be at substantially lower risk Still, it is difficult to tell from the data alone whether this phenomenon is real, since we have few stage 1 patients in the study (just 16 out of 312 patients), which explains why AIC and the likelihood ratio test prefer the simpler model Patrick Breheny Survival Data Analysis (BIOS 7210) 36/42

37 Influence Residuals One final issue on the topic of regression diagnostics is the assessment of influence An influential measurement is one that has a large effect on the model fit; this can be measured in a variety of ways, both absolute and relative A variety of residuals (score residuals, Schoenfeld residuals, delta-beta residuals) have been proposed as ways of quantifying and assessing influence; we ll focus on delta-beta residuals Patrick Breheny Survival Data Analysis (BIOS 7210) 37/42

38 Delta-beta plots The idea behind delta-beta residuals is very simple: let β (i) j denote the estimate of β j obtained if we leave subject i out of the model The delta-beta residual for coefficient j, subject i is therefore defined as ˆ ij = β j β (i) j This might seem computer intensive, but there are various computational tricks that allow one to fairly quickly refit models leaving individual observations out In R: resid(fit, type= dfbeta ) returns a matrix of delta-beta residuals Patrick Breheny Survival Data Analysis (BIOS 7210) 38/42

39 Treatment Residuals Treatment: ^ij Index (ordered by time on study) Patrick Breheny Survival Data Analysis (BIOS 7210) 39/42

40 Stage Residuals Stage: ^ij Index (ordered by time on study) Patrick Breheny Survival Data Analysis (BIOS 7210) 40/42

41 Bilirubin Residuals High bili Low bili ^ij Index (ordered by time on study) Patrick Breheny Survival Data Analysis (BIOS 7210) 41/42

42 Remarks Residuals Delta-beta plots are always interesting to look at and offer a great deal of insight into the inner mechanics of complicated models Furthermore, they can indicate whether the estimation of a coefficient is dominated by just a few individuals, which would be clear cause for concern What action to take in the presence of influential observations is often a complicated decision; my advice, however, is to strongly prefer modifying the model to fit the data as opposed to manipulating the data to fit the model (i.e., by removing outliers) Patrick Breheny Survival Data Analysis (BIOS 7210) 42/42

### Tied survival times; estimation of survival probabilities

Tied survival times; estimation of survival probabilities Patrick Breheny November 5 Patrick Breheny Survival Data Analysis (BIOS 7210) 1/22 Introduction Tied survival times Introduction Breslow approximation

### Cox regression: Estimation

Cox regression: Estimation Patrick Breheny October 27 Patrick Breheny Survival Data Analysis (BIOS 7210) 1/19 Introduction The Cox Partial Likelihood In our last lecture, we introduced the Cox partial

### Continuous case Discrete case General case. Hazard functions. Patrick Breheny. August 27. Patrick Breheny Survival Data Analysis (BIOS 7210) 1/21

Hazard functions Patrick Breheny August 27 Patrick Breheny Survival Data Analysis (BIOS 7210) 1/21 Introduction Continuous case Let T be a nonnegative random variable representing the time to an event

### Müller: Goodness-of-fit criteria for survival data

Müller: Goodness-of-fit criteria for survival data Sonderforschungsbereich 386, Paper 382 (2004) Online unter: http://epub.ub.uni-muenchen.de/ Projektpartner Goodness of fit criteria for survival data

### Lecture 8 Stat D. Gillen

Statistics 255 - Survival Analysis Presented February 23, 2016 Dan Gillen Department of Statistics University of California, Irvine 8.1 Example of two ways to stratify Suppose a confounder C has 3 levels

### Extensions of Cox Model for Non-Proportional Hazards Purpose

PhUSE 2013 Paper SP07 Extensions of Cox Model for Non-Proportional Hazards Purpose Jadwiga Borucka, PAREXEL, Warsaw, Poland ABSTRACT Cox proportional hazard model is one of the most common methods used

### Chapter 4 Regression Models

23.August 2010 Chapter 4 Regression Models The target variable T denotes failure time We let x = (x (1),..., x (m) ) represent a vector of available covariates. Also called regression variables, regressors,

### Log-linearity for Cox s regression model. Thesis for the Degree Master of Science

Log-linearity for Cox s regression model Thesis for the Degree Master of Science Zaki Amini Master s Thesis, Spring 2015 i Abstract Cox s regression model is one of the most applied methods in medical

### Chapter 1 Statistical Inference

Chapter 1 Statistical Inference causal inference To infer causality, you need a randomized experiment (or a huge observational study and lots of outside information). inference to populations Generalizations

### Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

Title stata.com stcrreg postestimation Postestimation tools for stcrreg Description Syntax for predict Menu for predict Options for predict Remarks and examples Methods and formulas References Also see

### Correlation and regression

1 Correlation and regression Yongjua Laosiritaworn Introductory on Field Epidemiology 6 July 2015, Thailand Data 2 Illustrative data (Doll, 1955) 3 Scatter plot 4 Doll, 1955 5 6 Correlation coefficient,

### The coxvc_1-1-1 package

Appendix A The coxvc_1-1-1 package A.1 Introduction The coxvc_1-1-1 package is a set of functions for survival analysis that run under R2.1.1 [81]. This package contains a set of routines to fit Cox models

### Survival Analysis. Lu Tian and Richard Olshen Stanford University

1 Survival Analysis Lu Tian and Richard Olshen Stanford University 2 Survival Time/ Failure Time/Event Time We will introduce various statistical methods for analyzing survival outcomes What is the survival

### A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL

Discussiones Mathematicae Probability and Statistics 36 206 43 5 doi:0.75/dmps.80 A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Tadeusz Bednarski Wroclaw University e-mail: t.bednarski@prawo.uni.wroc.pl

### Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features. Yangxin Huang

Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features Yangxin Huang Department of Epidemiology and Biostatistics, COPH, USF, Tampa, FL yhuang@health.usf.edu January

### STAT331. Cox s Proportional Hazards Model

STAT331 Cox s Proportional Hazards Model In this unit we introduce Cox s proportional hazards (Cox s PH) model, give a heuristic development of the partial likelihood function, and discuss adaptations

### Multistate Modeling and Applications

Multistate Modeling and Applications Yang Yang Department of Statistics University of Michigan, Ann Arbor IBM Research Graduate Student Workshop: Statistics for a Smarter Planet Yang Yang (UM, Ann Arbor)

### ADVANCED STATISTICAL ANALYSIS OF EPIDEMIOLOGICAL STUDIES. Cox s regression analysis Time dependent explanatory variables

ADVANCED STATISTICAL ANALYSIS OF EPIDEMIOLOGICAL STUDIES Cox s regression analysis Time dependent explanatory variables Henrik Ravn Bandim Health Project, Statens Serum Institut 4 November 2011 1 / 53

### Logistic regression: Miscellaneous topics

Logistic regression: Miscellaneous topics April 11 Introduction We have covered two approaches to inference for GLMs: the Wald approach and the likelihood ratio approach I claimed that the likelihood ratio

### Optimal Treatment Regimes for Survival Endpoints from a Classification Perspective. Anastasios (Butch) Tsiatis and Xiaofei Bai

Optimal Treatment Regimes for Survival Endpoints from a Classification Perspective Anastasios (Butch) Tsiatis and Xiaofei Bai Department of Statistics North Carolina State University 1/35 Optimal Treatment

### Grouped variable selection in high dimensional partially linear additive Cox model

University of Iowa Iowa Research Online Theses and Dissertations Fall 2010 Grouped variable selection in high dimensional partially linear additive Cox model Li Liu University of Iowa Copyright 2010 Li

### Lecture 11. Interval Censored and. Discrete-Time Data. Statistics Survival Analysis. Presented March 3, 2016

Statistics 255 - Survival Analysis Presented March 3, 2016 Motivating Dan Gillen Department of Statistics University of California, Irvine 11.1 First question: Are the data truly discrete? : Number of

### Stat 642, Lecture notes for 04/12/05 96

Stat 642, Lecture notes for 04/12/05 96 Hosmer-Lemeshow Statistic The Hosmer-Lemeshow Statistic is another measure of lack of fit. Hosmer and Lemeshow recommend partitioning the observations into 10 equal

### 8 Nominal and Ordinal Logistic Regression

8 Nominal and Ordinal Logistic Regression 8.1 Introduction If the response variable is categorical, with more then two categories, then there are two options for generalized linear models. One relies on

### Understanding the Cox Regression Models with Time-Change Covariates

Understanding the Cox Regression Models with Time-Change Covariates Mai Zhou University of Kentucky The Cox regression model is a cornerstone of modern survival analysis and is widely used in many other

### Exploratory Factor Analysis and Principal Component Analysis

Exploratory Factor Analysis and Principal Component Analysis Today s Topics: What are EFA and PCA for? Planning a factor analytic study Analysis steps: Extraction methods How many factors Rotation and

### Frailty Modeling for clustered survival data: a simulation study

Frailty Modeling for clustered survival data: a simulation study IAA Oslo 2015 Souad ROMDHANE LaREMFiQ - IHEC University of Sousse (Tunisia) souad_romdhane@yahoo.fr Lotfi BELKACEM LaREMFiQ - IHEC University

### Analysis of competing risks data and simulation of data following predened subdistribution hazards

Analysis of competing risks data and simulation of data following predened subdistribution hazards Bernhard Haller Institut für Medizinische Statistik und Epidemiologie Technische Universität München 27.05.2013

### 2/26/2017. PSY 512: Advanced Statistics for Psychological and Behavioral Research 2

PSY 512: Advanced Statistics for Psychological and Behavioral Research 2 When and why do we use logistic regression? Binary Multinomial Theory behind logistic regression Assessing the model Assessing predictors

### Statistics 262: Intermediate Biostatistics Regression & Survival Analysis

Statistics 262: Intermediate Biostatistics Regression & Survival Analysis Jonathan Taylor & Kristin Cobb Statistics 262: Intermediate Biostatistics p.1/?? Introduction This course is an applied course,

### Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification. Todd MacKenzie, PhD

Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification Todd MacKenzie, PhD Collaborators A. James O Malley Tor Tosteson Therese Stukel 2 Overview 1. Instrumental variable

### Typical Survival Data Arising From a Clinical Trial. Censoring. The Survivor Function. Mathematical Definitions Introduction

Outline CHL 5225H Advanced Statistical Methods for Clinical Trials: Survival Analysis Prof. Kevin E. Thorpe Defining Survival Data Mathematical Definitions Non-parametric Estimates of Survival Comparing

### Regression: Main Ideas Setting: Quantitative outcome with a quantitative explanatory variable. Example, cont.

TCELL 9/4/205 36-309/749 Experimental Design for Behavioral and Social Sciences Simple Regression Example Male black wheatear birds carry stones to the nest as a form of sexual display. Soler et al. wanted

### Multiple samples: Modeling and ANOVA

Multiple samples: Modeling and Patrick Breheny April 29 Patrick Breheny Introduction to Biostatistics (171:161) 1/23 Multiple group studies In the latter half of this course, we have discussed the analysis

### Introduction to Empirical Processes and Semiparametric Inference Lecture 25: Semiparametric Models

Introduction to Empirical Processes and Semiparametric Inference Lecture 25: Semiparametric Models Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations

### Variable Selection and Model Choice in Survival Models with Time-Varying Effects

Variable Selection and Model Choice in Survival Models with Time-Varying Effects Boosting Survival Models Benjamin Hofner 1 Department of Medical Informatics, Biometry and Epidemiology (IMBE) Friedrich-Alexander-Universität

### A COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky

A COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky Empirical likelihood with right censored data were studied by Thomas and Grunkmier (1975), Li (1995),

### 11. Generalized Linear Models: An Introduction

Sociology 740 John Fox Lecture Notes 11. Generalized Linear Models: An Introduction Copyright 2014 by John Fox Generalized Linear Models: An Introduction 1 1. Introduction I A synthesis due to Nelder and

### Model Estimation Example

Ronald H. Heck 1 EDEP 606: Multivariate Methods (S2013) April 7, 2013 Model Estimation Example As we have moved through the course this semester, we have encountered the concept of model estimation. Discussions

### Ridge regression. Patrick Breheny. February 8. Penalized regression Ridge regression Bayesian interpretation

Patrick Breheny February 8 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/27 Introduction Basic idea Standardization Large-scale testing is, of course, a big area and we could keep talking

### Remedial Measures Wrap-Up and Transformations Box Cox

Remedial Measures Wrap-Up and Transformations Box Cox Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 1, Slide 1 Last Class Graphical procedures for determining appropriateness

### Generalized Linear Models with Functional Predictors

Generalized Linear Models with Functional Predictors GARETH M. JAMES Marshall School of Business, University of Southern California Abstract In this paper we present a technique for extending generalized

### Multinomial Logistic Regression Models

Stat 544, Lecture 19 1 Multinomial Logistic Regression Models Polytomous responses. Logistic regression can be extended to handle responses that are polytomous, i.e. taking r>2 categories. (Note: The word

### Multivariable Fractional Polynomials

Multivariable Fractional Polynomials Axel Benner September 7, 2015 Contents 1 Introduction 1 2 Inventory of functions 1 3 Usage in R 2 3.1 Model selection........................................ 3 4 Example

### ST495: Survival Analysis: Maximum likelihood

ST495: Survival Analysis: Maximum likelihood Eric B. Laber Department of Statistics, North Carolina State University February 11, 2014 Everything is deception: seeking the minimum of illusion, keeping

### Linear Regression Models P8111

Linear Regression Models P8111 Lecture 25 Jeff Goldsmith April 26, 2016 1 of 37 Today s Lecture Logistic regression / GLMs Model framework Interpretation Estimation 2 of 37 Linear regression Course started

### Survival models and health sequences

Survival models and health sequences Walter Dempsey University of Michigan July 27, 2015 Survival Data Problem Description Survival data is commonplace in medical studies, consisting of failure time information

### Outline of GLMs. Definitions

Outline of GLMs Definitions This is a short outline of GLM details, adapted from the book Nonparametric Regression and Generalized Linear Models, by Green and Silverman. The responses Y i have density

### Analyzing Residuals in a PROC SURVEYLOGISTIC Model

Paper 1477-2017 Analyzing Residuals in a PROC SURVEYLOGISTIC Model Bogdan Gadidov, Herman E. Ray, Kennesaw State University ABSTRACT Data from an extensive survey conducted by the National Center for Education

### Frailty Modeling for Spatially Correlated Survival Data, with Application to Infant Mortality in Minnesota By: Sudipto Banerjee, Mela. P.

Frailty Modeling for Spatially Correlated Survival Data, with Application to Infant Mortality in Minnesota By: Sudipto Banerjee, Melanie M. Wall, Bradley P. Carlin November 24, 2014 Outlines of the talk

### Assessment of time varying long term effects of therapies and prognostic factors

Assessment of time varying long term effects of therapies and prognostic factors Dissertation by Anika Buchholz Submitted to Fakultät Statistik, Technische Universität Dortmund in Fulfillment of the Requirements

### A new strategy for meta-analysis of continuous covariates in observational studies with IPD. Willi Sauerbrei & Patrick Royston

A new strategy for meta-analysis of continuous covariates in observational studies with IPD Willi Sauerbrei & Patrick Royston Overview Motivation Continuous variables functional form Fractional polynomials

### Regression I: Mean Squared Error and Measuring Quality of Fit

Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving

### STAT 331. Martingale Central Limit Theorem and Related Results

STAT 331 Martingale Central Limit Theorem and Related Results In this unit we discuss a version of the martingale central limit theorem, which states that under certain conditions, a sum of orthogonal

### Numerical Methods Lecture 7 - Statistics, Probability and Reliability

Topics Numerical Methods Lecture 7 - Statistics, Probability and Reliability A summary of statistical analysis A summary of probability methods A summary of reliability analysis concepts Statistical Analysis

### Package ICBayes. September 24, 2017

Package ICBayes September 24, 2017 Title Bayesian Semiparametric Models for Interval-Censored Data Version 1.1 Date 2017-9-24 Author Chun Pan, Bo Cai, Lianming Wang, and Xiaoyan Lin Maintainer Chun Pan

### Lecture 3. Truncation, length-bias and prevalence sampling

Lecture 3. Truncation, length-bias and prevalence sampling 3.1 Prevalent sampling Statistical techniques for truncated data have been integrated into survival analysis in last two decades. Truncation in

### Evaluating Predictive Accuracy of Survival Models with PROC PHREG

Paper SAS462-2017 Evaluating Predictive Accuracy of Survival Models with PROC PHREG Changbin Guo, Ying So, and Woosung Jang, SAS Institute Inc. Abstract Model validation is an important step in the model

### Generalized Linear Models

Generalized Linear Models Advanced Methods for Data Analysis (36-402/36-608 Spring 2014 1 Generalized linear models 1.1 Introduction: two regressions So far we ve seen two canonical settings for regression.

### Using Mplus individual residual plots for. diagnostics and model evaluation in SEM

Using Mplus individual residual plots for diagnostics and model evaluation in SEM Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 20 October 31, 2017 1 Introduction A variety of plots are available

### Boosting. Ryan Tibshirani Data Mining: / April Optional reading: ISL 8.2, ESL , 10.7, 10.13

Boosting Ryan Tibshirani Data Mining: 36-462/36-662 April 25 2013 Optional reading: ISL 8.2, ESL 10.1 10.4, 10.7, 10.13 1 Reminder: classification trees Suppose that we are given training data (x i, y

### Estimating Treatment Effects and Identifying Optimal Treatment Regimes to Prolong Patient Survival

Estimating Treatment Effects and Identifying Optimal Treatment Regimes to Prolong Patient Survival by Jincheng Shen A dissertation submitted in partial fulfillment of the requirements for the degree of

### Survival Distributions, Hazard Functions, Cumulative Hazards

BIO 244: Unit 1 Survival Distributions, Hazard Functions, Cumulative Hazards 1.1 Definitions: The goals of this unit are to introduce notation, discuss ways of probabilistically describing the distribution

### Flexible parametric alternatives to the Cox model, and more

The Stata Journal (2001) 1, Number 1, pp. 1 28 Flexible parametric alternatives to the Cox model, and more Patrick Royston UK Medical Research Council patrick.royston@ctu.mrc.ac.uk Abstract. Since its

### Research Design: Topic 18 Hierarchical Linear Modeling (Measures within Persons) 2010 R.C. Gardner, Ph.d.

Research Design: Topic 8 Hierarchical Linear Modeling (Measures within Persons) R.C. Gardner, Ph.d. General Rationale, Purpose, and Applications Linear Growth Models HLM can also be used with repeated

### Time-Invariant Predictors in Longitudinal Models

Time-Invariant Predictors in Longitudinal Models Topics: Summary of building unconditional models for time Missing predictors in MLM Effects of time-invariant predictors Fixed, systematically varying,

### Efficient Estimation of Censored Linear Regression Model

2 3 4 5 6 7 8 9 2 3 4 5 6 7 8 9 2 2 22 23 24 25 26 27 28 29 3 3 32 33 34 35 36 37 38 39 4 4 42 43 44 45 46 47 48 Biometrika (2), xx, x, pp. 4 C 28 Biometrika Trust Printed in Great Britain Efficient Estimation

### Multiple Linear Regression II. Lecture 8. Overview. Readings

Multiple Linear Regression II Lecture 8 Image source:https://commons.wikimedia.org/wiki/file:autobunnskr%c3%a4iz-ro-a201.jpg Survey Research & Design in Psychology James Neill, 2016 Creative Commons Attribution

### BIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY

BIAS OF MAXIMUM-LIKELIHOOD ESTIMATES IN LOGISTIC AND COX REGRESSION MODELS: A COMPARATIVE SIMULATION STUDY Ingo Langner 1, Ralf Bender 2, Rebecca Lenz-Tönjes 1, Helmut Küchenhoff 2, Maria Blettner 2 1

### Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

### Kernel density estimation in R

Kernel density estimation in R Kernel density estimation can be done in R using the density() function in R. The default is a Guassian kernel, but others are possible also. It uses it s own algorithm to

### The lasso. Patrick Breheny. February 15. The lasso Convex optimization Soft thresholding

Patrick Breheny February 15 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/24 Introduction Last week, we introduced penalized regression and discussed ridge regression, in which the penalty

### 36-309/749 Experimental Design for Behavioral and Social Sciences. Dec 1, 2015 Lecture 11: Mixed Models (HLMs)

36-309/749 Experimental Design for Behavioral and Social Sciences Dec 1, 2015 Lecture 11: Mixed Models (HLMs) Independent Errors Assumption An error is the deviation of an individual observed outcome (DV)

### Introduction to Generalized Models

Introduction to Generalized Models Today s topics: The big picture of generalized models Review of maximum likelihood estimation Models for binary outcomes Models for proportion outcomes Models for categorical

### BIOS 312: Precision of Statistical Inference

and Power/Sample Size and Standard Errors BIOS 312: of Statistical Inference Chris Slaughter Department of Biostatistics, Vanderbilt University School of Medicine January 3, 2013 Outline Overview and Power/Sample

### Exam C Solutions Spring 2005

Exam C Solutions Spring 005 Question # The CDF is F( x) = 4 ( + x) Observation (x) F(x) compare to: Maximum difference 0. 0.58 0, 0. 0.58 0.7 0.880 0., 0.4 0.680 0.9 0.93 0.4, 0.6 0.53. 0.949 0.6, 0.8

### Linear Regression Models

Linear Regression Models Model Description and Model Parameters Modelling is a central theme in these notes. The idea is to develop and continuously improve a library of predictive models for hazards,

### MS&E 226: Small Data

MS&E 226: Small Data Lecture 6: Model complexity scores (v3) Ramesh Johari ramesh.johari@stanford.edu Fall 2015 1 / 34 Estimating prediction error 2 / 34 Estimating prediction error We saw how we can estimate

### Linear Models (continued)

Linear Models (continued) Model Selection Introduction Most of our previous discussion has focused on the case where we have a data set and only one fitted model. Up until this point, we have discussed,

### SAMPLE SIZE ESTIMATION FOR SURVIVAL OUTCOMES IN CLUSTER-RANDOMIZED STUDIES WITH SMALL CLUSTER SIZES BIOMETRICS (JUNE 2000)

SAMPLE SIZE ESTIMATION FOR SURVIVAL OUTCOMES IN CLUSTER-RANDOMIZED STUDIES WITH SMALL CLUSTER SIZES BIOMETRICS (JUNE 2000) AMITA K. MANATUNGA THE ROLLINS SCHOOL OF PUBLIC HEALTH OF EMORY UNIVERSITY SHANDE

### Randomization Tests for Regression Models in Clinical Trials

Randomization Tests for Regression Models in Clinical Trials A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University By Parwen

### Ron Heck, Fall Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October 20, 2011)

Ron Heck, Fall 2011 1 EDEP 768E: Seminar in Multilevel Modeling rev. January 3, 2012 (see footnote) Week 8: Introducing Generalized Linear Models: Logistic Regression 1 (Replaces prior revision dated October

### The First Derivative and Second Derivative Test

The First Derivative and Second Derivative Test James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 8, 2017 Outline Extremal Values The

### The covariate order method for nonparametric exponential regression and some applications in other lifetime models.

The covariate order method for nonparametric exponential regression and some applications in other lifetime models. Jan Terje Kvaløy and Bo Henry Lindqvist Department of Mathematics and Natural Science

### Faster Kriging on Graphs

Faster Kriging on Graphs Omkar Muralidharan Abstract [Xu et al. 2009] introduce a graph prediction method that is accurate but slow. My project investigates faster methods based on theirs that are nearly

### High-dimensional regression

High-dimensional regression Advanced Methods for Data Analysis 36-402/36-608) Spring 2014 1 Back to linear regression 1.1 Shortcomings Suppose that we are given outcome measurements y 1,... y n R, and

### STA 108 Applied Linear Models: Regression Analysis Spring Solution for Homework #6

STA 8 Applied Linear Models: Regression Analysis Spring 011 Solution for Homework #6 6. a) = 11 1 31 41 51 1 3 4 5 11 1 31 41 51 β = β1 β β 3 b) = 1 1 1 1 1 11 1 31 41 51 1 3 4 5 β = β 0 β1 β 6.15 a) Stem-and-leaf

### Frailty Models and Copulas: Similarities and Differences

Frailty Models and Copulas: Similarities and Differences KLARA GOETHALS, PAUL JANSSEN & LUC DUCHATEAU Department of Physiology and Biometrics, Ghent University, Belgium; Center for Statistics, Hasselt

### From semi- to non-parametric inference in general time scale models

From semi- to non-parametric inference in general time scale models Thierry DUCHESNE duchesne@matulavalca Département de mathématiques et de statistique Université Laval Québec, Québec, Canada Research

### Classification 1: Linear regression of indicators, linear discriminant analysis

Classification 1: Linear regression of indicators, linear discriminant analysis Ryan Tibshirani Data Mining: 36-462/36-662 April 2 2013 Optional reading: ISL 4.1, 4.2, 4.4, ESL 4.1 4.3 1 Classification

### Machine Learning Linear Classification. Prof. Matteo Matteucci

Machine Learning Linear Classification Prof. Matteo Matteucci Recall from the first lecture 2 X R p Regression Y R Continuous Output X R p Y {Ω 0, Ω 1,, Ω K } Classification Discrete Output X R p Y (X)

### , (1) e i = ˆσ 1 h ii. c 2016, Jeffrey S. Simonoff 1

Regression diagnostics As is true of all statistical methodologies, linear regression analysis can be a very effective way to model data, as along as the assumptions being made are true. For the regression

### Analysing categorical data using logit models

Analysing categorical data using logit models Graeme Hutcheson, University of Manchester The lecture notes, exercises and data sets associated with this course are available for download from: www.research-training.net/manchester

### Introduction to Regression

Regression Introduction to Regression If two variables covary, we should be able to predict the value of one variable from another. Correlation only tells us how much two variables covary. In regression,

### STATISTICS 174: APPLIED STATISTICS TAKE-HOME FINAL EXAM POSTED ON WEBPAGE: 6:00 pm, DECEMBER 6, 2004 HAND IN BY: 6:00 pm, DECEMBER 7, 2004 This is a

STATISTICS 174: APPLIED STATISTICS TAKE-HOME FINAL EXAM POSTED ON WEBPAGE: 6:00 pm, DECEMBER 6, 2004 HAND IN BY: 6:00 pm, DECEMBER 7, 2004 This is a take-home exam. You are expected to work on it by yourself

### Psychology Seminar Psych 406 Dr. Jeffrey Leitzel

Psychology Seminar Psych 406 Dr. Jeffrey Leitzel Structural Equation Modeling Topic 1: Correlation / Linear Regression Outline/Overview Correlations (r, pr, sr) Linear regression Multiple regression interpreting

### Lecture 9: Predictive Inference for the Simple Linear Model

See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 9: Predictive Inference for the Simple Linear Model 36-401, Fall 2015, Section B 29 September 2015 Contents 1 Confidence intervals

### The assumptions are needed to give us... valid standard errors valid confidence intervals valid hypothesis tests and p-values

Statistical Consulting Topics The Bootstrap... The bootstrap is a computer-based method for assigning measures of accuracy to statistical estimates. (Efron and Tibshrani, 1998.) What do we do when our