# A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Discussiones Mathematicae Probability and Statistics doi:0.75/dmps.80 A NOTE ON ROBUST ESTIMATION IN LOGISTIC REGRESSION MODEL Tadeusz Bednarski Wroclaw University Abstract Computationally attractive Fisher consistent robust estimation methods based on adaptive explanatory variables trimming are proposed for the logistic regression model. Results of a Monte Carlo experiment and a real data analysis show its good behavior for moderate sample sizes. The method is applicable when some distributional information about explanatory variables is available. Keywords: logistic model, robust estimation. 200 Mathematics Subject Classification: 62F35, 62J2.. Introduction The logistic model plays important role in predictive inference in many fields: diagnostics in medicine, behavioral prediction in social and economic processes. A more recent area of its interesting applications concern the uplift modeling in marketing. The binary response variable Y of the model satisfies the logistic equation P β Y = X = x = e[,x ]β + e [,x ]β, where X is a k-vector of explanatory variables and β = [β 0, β,..., β k ] is a vector of regression parameters. The logistic function, as well as the normal distribution in the probit model, is supposed to measure the relationship between the probability of the event of interest and the value of its linear predictor. This mathematical relationship does not seem to follow precisely from any real phenomena. It is rather a mixture of mathematical convenience and some intuitive knowledge about a monotone regularity between probability of success and explanatory variables.

2 44 T. Bednarski As for any statistical model, inference based on maximum likelihood may be very sensitive to erroneous observations and therefore supplementary robust data analysis is highly recommended. There is a vast literature on robust inference for the generalized linear models and in particular for the logistic one [, 2, 3, 5, 8, 0,, 2]. The methods are either based on downweighting the influential observations or on modification of the likelihood function or both approaches are used. Some gain in inferential efficiency for the logistic or probit model can be achieved by adjustment of robust methodology to a particular experimental situation. In contrast to a standard linear regression model, where outliers in dependent variable alone can do much harm, this is unlikely to happen for the logistic model, where true oulyingness is possible only in independent variable. The estimation method proposed here for the logistic model is related to [9, 2] with weighting based exclusively on the design or explanatory variable. It assumes some prior knowledge about the distribution of the explanatory variables. In repeated experiments such knowledge is sometimes available and it either has a distributional character or is implied by, say, physical limitations of studied objects. Further on we shall assume that this prior knowledge would be an approximately normal distribution for all or some of the explanatory variables. 2. The method If X, Y,..., X n, Y n denote a random sample of explanatory and dependent variables then the likelihood function for the logistic model is equal to n e [,X i ]β Yi Yi L βx,y = + e [,X i ]β + e [,X i ]β i= with the corresponding score function n [ ] S β = Y X i e[,x i ]β i + e [,X i ]β. i= There are two basic modifications of the score function proposed here. In both of them it is initially assumed that the constant term in the linear predictor is equal to zero. The first modification has the form n Sβ w = wx i Y i ewx i β + e wx, i β i= where wx = xi B x for B a Borel set. The proposed method is equivalent to the maximum likelihood estimation for the logistic regression for possibly modified

3 A note on robust estimation in logistic regression model 45 observations in explanatory variables and under the assumption of zero shift in the linear predictor. The second modification assumes the following form of the score function 2 S w β = n hwx i Y i ewx i β + e wx. i β i= Theorem. Let us consider the estimation methods given by score functions and 2. a The data modifying function wx = xi B x for B a Borel set such that P X B > 0 gives the conditional Fisher consistency of method. b Assume the distribution of X is symmetric around the origin, w is an odd function, [ h is even and for some ] β β 0 we have E β0 hwx i Y i ewx i β +e wx i β 0. Then method 2 is Fisher consistent. Proof. Justification of the two facts is elementary. following immediate equalities: In case a we have the wx y ewx β dp + e wx β β0 y xdgx = wx ewx β P + e wx β β0 y = x + wx 0 ewx β P + e wx β β0 y = 0 x]dgx = = e x β 0 wx wx dgx + e wx β + e x β 0 + e wx β + e x β 0 e wx β wx + e wx β e x β 0 e wx β dgx, + e x β 0 where dp β0 y xdgx denotes the true model distribution. By the definition of w we get the zero value of the expression under the integral if β = β 0. The second fact follows then from the symmetry assumption and oddness of the pseudo-score function. Notice that

4 46 T. Bednarski hw x + e w x β e x β 0 e w x β + e x β 0 e wx β e x β = hwx + e wx β e x β 0 e w x β + e x β 0 = hwx + e wx β e x β 0 e wx β. + e x β 0 A drawback of the first method is a non-smooth structure of the data modifying function w. The second method allows smooth w s which is a merit in estimation since the variance assessment becomes more reliable. Further on we shall discuss properties of the first method only. The method is meant to be an intermediate step in any practical implementation of a standard logistic model with arbitrary shift when the distribution of explanatory variables is approximately normal. In the first step the original data are robustly standardized with a consequent reparametrization of the model. Then the maximum likelihood method with standardized data modified by wx = xi B x is applied. The final step consists in returning to the original parametrization. 3. Monte Carlo and real data case study A simulation study was conducted to compare efficiency of the above described simplified robust estimation procedure with mle and some standard robust estimates. A logistic model with two dimensional explanatory vector of independent standard normal variables X = X, X 2 was assumed so that P β Y = X = x = eβ 0+β x +β 2 x 2 + e β 0+β x +β 2 x 2 and β = β 0, β, β 2 = 0,,. The sample size was taken n = 300. In contaminated samples 5% of the original standard normal vector observations were replaced by the vectors 0, k where for each k taking values, 2,..., 7 the estimation process was simulated 500 times for the original and contaminated data. This sort of contamination was chosen because it strongly exhibit efficiency differences. The mle was used for the original data and for the contaminated data to give a reference point. The contaminated data were also estimated with standard robust methods for the logistic model and the proposed robust method. The data simulation and estimation process was conducted with R software. The maximum likelihood estimation was done with glm function while standard robust estimation with glmrob function of the robust package. Two robust

5 A note on robust estimation in logistic regression model 47 methods were applied: mallows for Mallow s leverage downweighting estimator and cubif for the conditionally unbiased bounded influence estimator. For the simplified method the mle was applied to transformed explanatory variables: in the first step robust location and covariance matrix were computed using covrob procedure with constrained M estimator. Then for standardized data the mle was applied under zero value of the shift and B = [ 3, 3] 3. Finally using the obtained robust estimates of shift and covariance matrix final estimates were obtained. Figure gives two graphs summarizing the estimation effects. The one on the left-hand side shows plots of the mean Euclidian distance between estimates and the vector β = 0,, at different contaminating values k. The one on the righthand side is the mean angle formed by the vector of estimates ˆβ, ˆβ 2 and,. In each case there are four plotted curves: the mle for non-contaminated, the mle for contaminated samples, robust mallows estimate for contaminated data and finally simplified estimation for contaminated samples. The mean distance and angle of estimators with respect to the true parameter value were employed as stability characteristics since they may behave differently depending on the data distribution. Figure. Mean distance left graph and angle right graph for the mle, standard robust rob and simplified robust method rob2. In spite of the fact that the simplified method completely ignores the information from outlying observations, we can see its good behavior compared to other estimation methods. The cubif procedure showed similar results. Also changes in sample size from 00 to 500 did not essentially changed the situation. We can see that as the contamination values increase, the standard robust methods start to work better. However, they seem to achieve the efficiency of the simple method at extremely large outliers values, detectable probably by any method. The procedures were used in their default form, therefore there might be a potential for their better behavior. The following graph depicts assignment of sample units to two classes depending on the estimated success probability value, smaller or greater than 0.5. To make the visual comparison possible only the non-contaminated data are plotted. The black dots correspond to sample elements for which estimated success probability was greater than 0.5. We can see that even for relatively small contamination size k = 4.75 the angular effect differs very much for the presented methods. The simplified method is very stable and it gives results similar to mle for non-contaminated data. These classification results would be rather typical for contamination size k between 4 and 7.

6 48 T. Bednarski Figure 2. Exemplary result of classification. Figure 3. Differences between dependent variable and estimated probabilities for the Feigl and Zelen 965 data. The real data case analyzed below originates from [6] and it was discussed in [4]. The data are available in R as BGPhazard package. There are four variables of 33 leukemia patients: time survival times in weeks from diagnosis, AG indicator of positive result of a test related to the white blood cell characteristics, wbc white blood cells counts in thousands and delta a status variable indicating censoring. As in [4] the dependent variable indicating survival longer than 52 weeks was introduced and logistic model was applied with explanatory variables AG and wbc. The mle, standard robust and simplified robust methods were employed. The inferential problem for this data set is that five patients extreme values of wbc = 00 lead to contradictory predictions about survival time and AG. The trimming value for the explanatory variable in the simplified method was 69.7 the mean plus 3 standard deviations computed without the outlying values of wbc = 00. The three methods indicate significance of AG variable. The simplified method gave the smallest p-value to wbc p = for standard robust and for mle and it led to smallest null and residual deviance. The graph below shows the residual curves for the three methods. The simplified method behaves very well for unimodal elliptically symmetric distributions close to the normal one. Simulations show that the variability of the method is comparable to other robust methods for the logistic model except for the case when outliers of robustly standardized data are around the points of discontinuity of the function w, where the simplified method shows higher variability. The method also looses its advantage when the distribution of the vector of explanatory variables is for instance uniformly distributed or it comes from an asymmetric distribution. The asymptotic distributions of the proposed estimators are relatively straightforward under the assumption that explanatory variables come indeed from the standard normal population. Otherwise they become more difficult to compute due to transformations of observed variables by robust estimates of shift and covariance. A description of the asymptotic distributions along with their validity for moderate sample sizes will be given in a subsequent paper. References [] A.M. Bianco and E. Martinez, Robust testing in the logistic regression model, Computational Statistics and Data Analysis

7 A note on robust estimation in logistic regression model 49 [2] A.M. Bianco and V.J. Yohai, Robust estimation in the logistic regression model, Lecture Notes in Statistics, Springer Verlag, New York [3] E. Cantoni and E. Ronchetti, Robust inference for generalized linear models, Journal of the American Statistical Association [4] R.D. Cook and S. Weisberg, Residuals and Influence in Regression Chapman and Hall, London, 982. [5] C. Croux, G. Haesbroeck and K. Joossens, Logistic discrimination using robust estimators: An influence function approach, Canadian J. Statist [6] P. Feigl and M. Zelen, Estimation of exponential probabilities with concomitant information, Biometrics [7] D.J. Finney, The estimation from individual records of the relationship between dose and quantal response, Biometrika [8] H.R. Kunsch, L.A. Stefanski and R.J. Carroll, Conditionally Unbiased Bounded Influence Estimation in General Regression Models, with Applications to Generalized Linear Models, J. Amer. Statist. Assoc [9] C.L. Mallows, On some topics in robustness Tech. Report, Bell Laboratories, Murray Hill, NY, 975. [0] S. Morgenthaler, Least-absolute-deviations fits for generalized linear model, Biometrika [] D. Pregibon, Resistant Fits for some commonly used Logistic Models with Medical Applications, Biometrics

8 50 T. Bednarski [2] L. Stefanski, R. Carroll and D. Ruppert, Optimally bounded score functions for generalized linear models with applications to logistic regression, Biometrika Received 2 December 205

### STA216: Generalized Linear Models. Lecture 1. Review and Introduction

STA216: Generalized Linear Models Lecture 1. Review and Introduction Let y 1,..., y n denote n independent observations on a response Treat y i as a realization of a random variable Y i In the general

### Analysis of 2 n Factorial Experiments with Exponentially Distributed Response Variable

Applied Mathematical Sciences, Vol. 5, 2011, no. 10, 459-476 Analysis of 2 n Factorial Experiments with Exponentially Distributed Response Variable S. C. Patil (Birajdar) Department of Statistics, Padmashree

### Multilevel Statistical Models: 3 rd edition, 2003 Contents

Multilevel Statistical Models: 3 rd edition, 2003 Contents Preface Acknowledgements Notation Two and three level models. A general classification notation and diagram Glossary Chapter 1 An introduction

### Repeated ordinal measurements: a generalised estimating equation approach

Repeated ordinal measurements: a generalised estimating equation approach David Clayton MRC Biostatistics Unit 5, Shaftesbury Road Cambridge CB2 2BW April 7, 1992 Abstract Cumulative logit and related

### Analysis of 2 n factorial experiments with exponentially distributed response variable: a comparative study

Analysis of 2 n factorial experiments with exponentially distributed response variable: a comparative study H. V. Kulkarni a and S. C. Patil (Birajdar) b Department of Statistics, Shivaji University, Kolhapur,

### Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon

Discussion of the paper Inference for Semiparametric Models: Some Questions and an Answer by Bickel and Kwon Jianqing Fan Department of Statistics Chinese University of Hong Kong AND Department of Statistics

### A Course in Applied Econometrics Lecture 18: Missing Data. Jeff Wooldridge IRP Lectures, UW Madison, August Linear model with IVs: y i x i u i,

A Course in Applied Econometrics Lecture 18: Missing Data Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. When Can Missing Data be Ignored? 2. Inverse Probability Weighting 3. Imputation 4. Heckman-Type

### Model comparison and selection

BS2 Statistical Inference, Lectures 9 and 10, Hilary Term 2008 March 2, 2008 Hypothesis testing Consider two alternative models M 1 = {f (x; θ), θ Θ 1 } and M 2 = {f (x; θ), θ Θ 2 } for a sample (X = x)

### Using Estimating Equations for Spatially Correlated A

Using Estimating Equations for Spatially Correlated Areal Data December 8, 2009 Introduction GEEs Spatial Estimating Equations Implementation Simulation Conclusion Typical Problem Assess the relationship

### The betareg Package. October 6, 2006

The betareg Package October 6, 2006 Title Beta Regression. Version 1.2 Author Alexandre de Bustamante Simas , with contributions, in this last version, from Andréa Vanessa Rocha

### 8 Nominal and Ordinal Logistic Regression

8 Nominal and Ordinal Logistic Regression 8.1 Introduction If the response variable is categorical, with more then two categories, then there are two options for generalized linear models. One relies on

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions

### A COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky

A COMPARISON OF POISSON AND BINOMIAL EMPIRICAL LIKELIHOOD Mai Zhou and Hui Fang University of Kentucky Empirical likelihood with right censored data were studied by Thomas and Grunkmier (1975), Li (1995),

### Continuous Time Survival in Latent Variable Models

Continuous Time Survival in Latent Variable Models Tihomir Asparouhov 1, Katherine Masyn 2, Bengt Muthen 3 Muthen & Muthen 1 University of California, Davis 2 University of California, Los Angeles 3 Abstract

### AFT Models and Empirical Likelihood

AFT Models and Empirical Likelihood Mai Zhou Department of Statistics, University of Kentucky Collaborators: Gang Li (UCLA); A. Bathke; M. Kim (Kentucky) Accelerated Failure Time (AFT) models: Y = log(t

### Outlier Robust Nonlinear Mixed Model Estimation

Outlier Robust Nonlinear Mixed Model Estimation 1 James D. Williams, 2 Jeffrey B. Birch and 3 Abdel-Salam G. Abdel-Salam 1 Business Analytics, Dow AgroSciences, 9330 Zionsville Rd. Indianapolis, IN 46268,

### Testing Statistical Hypotheses

E.L. Lehmann Joseph P. Romano, 02LEu1 ttd ~Lt~S Testing Statistical Hypotheses Third Edition With 6 Illustrations ~Springer 2 The Probability Background 28 2.1 Probability and Measure 28 2.2 Integration.........

### Investigating Models with Two or Three Categories

Ronald H. Heck and Lynn N. Tabata 1 Investigating Models with Two or Three Categories For the past few weeks we have been working with discriminant analysis. Let s now see what the same sort of model might

### Heteroskedasticity-Robust Inference in Finite Samples

Heteroskedasticity-Robust Inference in Finite Samples Jerry Hausman and Christopher Palmer Massachusetts Institute of Technology December 011 Abstract Since the advent of heteroskedasticity-robust standard

### Statistics for Python

Statistics for Python An extension module for the Python scripting language Michiel de Hoon, Columbia University 2 September 2010 Statistics for Python, an extension module for the Python scripting language.

### arxiv: v2 [stat.me] 8 Jun 2016

Orthogonality of the Mean and Error Distribution in Generalized Linear Models 1 BY ALAN HUANG 2 and PAUL J. RATHOUZ 3 University of Technology Sydney and University of Wisconsin Madison 4th August, 2013

### Statistics - Lecture One. Outline. Charlotte Wickham 1. Basic ideas about estimation

Statistics - Lecture One Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Outline 1. Basic ideas about estimation 2. Method of Moments 3. Maximum Likelihood 4. Confidence

### A Course in Applied Econometrics Lecture 14: Control Functions and Related Methods. Jeff Wooldridge IRP Lectures, UW Madison, August 2008

A Course in Applied Econometrics Lecture 14: Control Functions and Related Methods Jeff Wooldridge IRP Lectures, UW Madison, August 2008 1. Linear-in-Parameters Models: IV versus Control Functions 2. Correlated

### Diagnostics for heteroscedasticity in regression

Biornetrika. (1983), 70, 1, pp. 1-10 1 Printed in (reat Britain Diagnostics for heteroscedasticity in regression BY R. DENNIS COOK AND SANFORD WEISBERG Department of Applied Statistics, University of Minnesota,

### A nonparametric test for trend based on initial ranks

Journal of Statistical Computation and Simulation Vol. 76, No. 9, September 006, 89 837 A nonparametric test for trend based on initial ranks GLENN HOFMANN* and N. BALAKRISHNAN HSBC, USA ID Analytics,

### BIOS 312: Precision of Statistical Inference

and Power/Sample Size and Standard Errors BIOS 312: of Statistical Inference Chris Slaughter Department of Biostatistics, Vanderbilt University School of Medicine January 3, 2013 Outline Overview and Power/Sample

### Simple Estimators for Semiparametric Multinomial Choice Models

Simple Estimators for Semiparametric Multinomial Choice Models James L. Powell and Paul A. Ruud University of California, Berkeley March 2008 Preliminary and Incomplete Comments Welcome Abstract This paper

### Frailty Models and Copulas: Similarities and Differences

Frailty Models and Copulas: Similarities and Differences KLARA GOETHALS, PAUL JANSSEN & LUC DUCHATEAU Department of Physiology and Biometrics, Ghent University, Belgium; Center for Statistics, Hasselt

### Incorporating published univariable associations in diagnostic and prognostic modeling

Incorporating published univariable associations in diagnostic and prognostic modeling Thomas Debray Julius Center for Health Sciences and Primary Care University Medical Center Utrecht The Netherlands

### Spatial Variation in Infant Mortality with Geographically Weighted Poisson Regression (GWPR) Approach

Spatial Variation in Infant Mortality with Geographically Weighted Poisson Regression (GWPR) Approach Kristina Pestaria Sinaga, Manuntun Hutahaean 2, Petrus Gea 3 1, 2, 3 University of Sumatera Utara,

### Latent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent

Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary

### A Power Analysis of Variable Deletion Within the MEWMA Control Chart Statistic

A Power Analysis of Variable Deletion Within the MEWMA Control Chart Statistic Jay R. Schaffer & Shawn VandenHul University of Northern Colorado McKee Greeley, CO 869 jay.schaffer@unco.edu gathen9@hotmail.com

### COMS 4771 Introduction to Machine Learning. James McInerney Adapted from slides by Nakul Verma

COMS 4771 Introduction to Machine Learning James McInerney Adapted from slides by Nakul Verma Announcements HW1: Please submit as a group Watch out for zero variance features (Q5) HW2 will be released

### Previous lecture. Single variant association. Use genome-wide SNPs to account for confounding (population substructure)

Previous lecture Single variant association Use genome-wide SNPs to account for confounding (population substructure) Estimation of effect size and winner s curse Meta-Analysis Today s outline P-value

### Chapter 3: Maximum Likelihood Theory

Chapter 3: Maximum Likelihood Theory Florian Pelgrin HEC September-December, 2010 Florian Pelgrin (HEC) Maximum Likelihood Theory September-December, 2010 1 / 40 1 Introduction Example 2 Maximum likelihood

### Comparison between conditional and marginal maximum likelihood for a class of item response models

(1/24) Comparison between conditional and marginal maximum likelihood for a class of item response models Francesco Bartolucci, University of Perugia (IT) Silvia Bacci, University of Perugia (IT) Claudia

### For Bonnie and Jesse (again)

SECOND EDITION A P P L I E D R E G R E S S I O N A N A L Y S I S a n d G E N E R A L I Z E D L I N E A R M O D E L S For Bonnie and Jesse (again) SECOND EDITION A P P L I E D R E G R E S S I O N A N A

### The Fundamentals of Heavy Tails Properties, Emergence, & Identification. Jayakrishnan Nair, Adam Wierman, Bert Zwart

The Fundamentals of Heavy Tails Properties, Emergence, & Identification Jayakrishnan Nair, Adam Wierman, Bert Zwart Why am I doing a tutorial on heavy tails? Because we re writing a book on the topic Why

### Regression models for multivariate ordered responses via the Plackett distribution

Journal of Multivariate Analysis 99 (2008) 2472 2478 www.elsevier.com/locate/jmva Regression models for multivariate ordered responses via the Plackett distribution A. Forcina a,, V. Dardanoni b a Dipartimento

### Multi-period credit default prediction with time-varying covariates. Walter Orth University of Cologne, Department of Statistics and Econometrics

with time-varying covariates Walter Orth University of Cologne, Department of Statistics and Econometrics 2 20 Overview Introduction Approaches in the literature The proposed models Empirical analysis

### Mixed effects models

Mixed effects models The basic theory and application in R Mitchel van Loon Research Paper Business Analytics Mixed effects models The basic theory and application in R Author: Mitchel van Loon Research

### Lecture 2 Linear Regression: A Model for the Mean. Sharyn O Halloran

Lecture 2 Linear Regression: A Model for the Mean Sharyn O Halloran Closer Look at: Linear Regression Model Least squares procedure Inferential tools Confidence and Prediction Intervals Assumptions Robustness

### Bayesian course - problem set 5 (lecture 6)

Bayesian course - problem set 5 (lecture 6) Ben Lambert November 30, 2016 1 Stan entry level: discoveries data The file prob5 discoveries.csv contains data on the numbers of great inventions and scientific

### Least Squares Estimation-Finite-Sample Properties

Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions

### Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events

Problems with Penalised Maximum Likelihood and Jeffrey s Priors to Account For Separation in Large Datasets with Rare Events Liam F. McGrath September 15, 215 Abstract When separation is a problem in binary

### Partial factor modeling: predictor-dependent shrinkage for linear regression

modeling: predictor-dependent shrinkage for linear Richard Hahn, Carlos Carvalho and Sayan Mukherjee JASA 2013 Review by Esther Salazar Duke University December, 2013 Factor framework The factor framework

### The loss function and estimating equations

Chapter 6 he loss function and estimating equations 6 Loss functions Up until now our main focus has been on parameter estimating via the maximum likelihood However, the negative maximum likelihood is

### Unbiased Estimation. Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others.

Unbiased Estimation Binomial problem shows general phenomenon. An estimator can be good for some values of θ and bad for others. To compare ˆθ and θ, two estimators of θ: Say ˆθ is better than θ if it

### WALD LECTURE II LOOKING INSIDE THE BLACK BOX. Leo Breiman UCB Statistics

1 WALD LECTURE II LOOKING INSIDE THE BLACK BOX Leo Breiman UCB Statistics leo@stat.berkeley.edu ORIGIN OF BLACK BOXES 2 Statistics uses data to explore problems. Think of the data as being generated by

### A SHORT COURSE ON ROBUST STATISTICS. David E. Tyler Rutgers The State University of New Jersey. Web-Site dtyler/shortcourse.

A SHORT COURSE ON ROBUST STATISTICS David E. Tyler Rutgers The State University of New Jersey Web-Site www.rci.rutgers.edu/ dtyler/shortcourse.pdf References Huber, P.J. (1981). Robust Statistics. Wiley,

### Chapter 4 Regression Models

23.August 2010 Chapter 4 Regression Models The target variable T denotes failure time We let x = (x (1),..., x (m) ) represent a vector of available covariates. Also called regression variables, regressors,

### Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood

Maximum Likelihood Estimation; Robust Maximum Likelihood; Missing Data with Maximum Likelihood PRE 906: Structural Equation Modeling Lecture #3 February 4, 2015 PRE 906, SEM: Estimation Today s Class An

### COMS 4771 Regression. Nakul Verma

COMS 4771 Regression Nakul Verma Last time Support Vector Machines Maximum Margin formulation Constrained Optimization Lagrange Duality Theory Convex Optimization SVM dual and Interpretation How get the

### Sigmaplot di Systat Software

Sigmaplot di Systat Software SigmaPlot Has Extensive Statistical Analysis Features SigmaPlot is now bundled with SigmaStat as an easy-to-use package for complete graphing and data analysis. The statistical

### Introduction to Empirical Processes and Semiparametric Inference Lecture 25: Semiparametric Models

Introduction to Empirical Processes and Semiparametric Inference Lecture 25: Semiparametric Models Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations

### Accounting for measurement uncertainties in industrial data analysis

Accounting for measurement uncertainties in industrial data analysis Marco S. Reis * ; Pedro M. Saraiva GEPSI-PSE Group, Department of Chemical Engineering, University of Coimbra Pólo II Pinhal de Marrocos,

### Multivariable Fractional Polynomials

Multivariable Fractional Polynomials Axel Benner September 7, 2015 Contents 1 Introduction 1 2 Inventory of functions 1 3 Usage in R 2 3.1 Model selection........................................ 3 4 Example

### Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence

Generalized Linear Model under the Extended Negative Multinomial Model and Cancer Incidence Sunil Kumar Dhar Center for Applied Mathematics and Statistics, Department of Mathematical Sciences, New Jersey

### Models for Ordinal Response Data

Models for Ordinal Response Data Robin High Department of Biostatistics Center for Public Health University of Nebraska Medical Center Omaha, Nebraska Recommendations Analyze numerical data with a statistical

PENALIZING YOUR MODELS AN OVERVIEW OF THE GENERALIZED REGRESSION PLATFORM Michael Crotty & Clay Barker Research Statisticians JMP Division, SAS Institute Copyr i g ht 2012, SAS Ins titut e Inc. All rights

### SAS/STAT 13.1 User s Guide. Introduction to Survey Sampling and Analysis Procedures

SAS/STAT 13.1 User s Guide Introduction to Survey Sampling and Analysis Procedures This document is an individual chapter from SAS/STAT 13.1 User s Guide. The correct bibliographic citation for the complete

### Some Monte Carlo Evidence for Adaptive Estimation of Unit-Time Varying Heteroscedastic Panel Data Models

Some Monte Carlo Evidence for Adaptive Estimation of Unit-Time Varying Heteroscedastic Panel Data Models G. R. Pasha Department of Statistics, Bahauddin Zakariya University Multan, Pakistan E-mail: drpasha@bzu.edu.pk

### Multinomial Logistic Regression Models

Stat 544, Lecture 19 1 Multinomial Logistic Regression Models Polytomous responses. Logistic regression can be extended to handle responses that are polytomous, i.e. taking r>2 categories. (Note: The word

### Generalized Linear Models with Functional Predictors

Generalized Linear Models with Functional Predictors GARETH M. JAMES Marshall School of Business, University of Southern California Abstract In this paper we present a technique for extending generalized

### Data Preprocessing. Cluster Similarity

1 Cluster Similarity Similarity is most often measured with the help of a distance function. The smaller the distance, the more similar the data objects (points). A function d: M M R is a distance on M

### Graphical Diagnosis. Paul E. Johnson 1 2. (Mostly QQ and Leverage Plots) 1 / Department of Political Science

(Mostly QQ and Leverage Plots) 1 / 63 Graphical Diagnosis Paul E. Johnson 1 2 1 Department of Political Science 2 Center for Research Methods and Data Analysis, University of Kansas. (Mostly QQ and Leverage

### Propensity Score Analysis with Hierarchical Data

Propensity Score Analysis with Hierarchical Data Fan Li Alan Zaslavsky Mary Beth Landrum Department of Health Care Policy Harvard Medical School May 19, 2008 Introduction Population-based observational

### Lecture 14 Simple Linear Regression

Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent

### DIC: Deviance Information Criterion

(((( Welcome Page Latest News DIC: Deviance Information Criterion Contact us/bugs list WinBUGS New WinBUGS examples FAQs DIC GeoBUGS DIC (Deviance Information Criterion) is a Bayesian method for model

### Bayesian Inference: Probit and Linear Probability Models

Utah State University DigitalCommons@USU All Graduate Plan B and other Reports Graduate Studies 5-1-2014 Bayesian Inference: Probit and Linear Probability Models Nate Rex Reasch Utah State University Follow

### Robust statistics. Michael Love 7/10/2016

Robust statistics Michael Love 7/10/2016 Robust topics Median MAD Spearman Wilcoxon rank test Weighted least squares Cook's distance M-estimators Robust topics Median => middle MAD => spread Spearman =>

### Random Effects Models for Network Data

Random Effects Models for Network Data Peter D. Hoff 1 Working Paper no. 28 Center for Statistics and the Social Sciences University of Washington Seattle, WA 98195-4320 January 14, 2003 1 Department of

### Applied Regression Modeling

Applied Regression Modeling A Business Approach Iain Pardoe University of Oregon Charles H. Lundquist College of Business Eugene, Oregon WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION CONTENTS

### Stein-Rule Estimation under an Extended Balanced Loss Function

Shalabh & Helge Toutenburg & Christian Heumann Stein-Rule Estimation under an Extended Balanced Loss Function Technical Report Number 7, 7 Department of Statistics University of Munich http://www.stat.uni-muenchen.de

### The linear mixed model: introduction and the basic model

The linear mixed model: introduction and the basic model Analysis of Experimental Data AED The linear mixed model: introduction and the basic model of 39 Contents Data with non-independent observations

### AP Statistics Unit 6 Note Packet Linear Regression. Scatterplots and Correlation

Scatterplots and Correlation Name Hr A scatterplot shows the relationship between two quantitative variables measured on the same individuals. variable (y) measures an outcome of a study variable (x) may

### The General Linear Model (GLM)

he General Linear Model (GLM) Klaas Enno Stephan ranslational Neuromodeling Unit (NU) Institute for Biomedical Engineering University of Zurich & EH Zurich Wellcome rust Centre for Neuroimaging Institute

### Robust repeated median regression in moving windows with data-adaptive width selection SFB 823. Discussion Paper. Matthias Borowski, Roland Fried

SFB 823 Robust repeated median regression in moving windows with data-adaptive width selection Discussion Paper Matthias Borowski, Roland Fried Nr. 28/2011 Robust Repeated Median regression in moving

### Econometrics Multiple Regression Analysis: Heteroskedasticity

Econometrics Multiple Regression Analysis: João Valle e Azevedo Faculdade de Economia Universidade Nova de Lisboa Spring Semester João Valle e Azevedo (FEUNL) Econometrics Lisbon, April 2011 1 / 19 Properties

### High Breakdown Analogs of the Trimmed Mean

High Breakdown Analogs of the Trimmed Mean David J. Olive Southern Illinois University April 11, 2004 Abstract Two high breakdown estimators that are asymptotically equivalent to a sequence of trimmed

### Asymptotic Relative Efficiency in Estimation

Asymptotic Relative Efficiency in Estimation Robert Serfling University of Texas at Dallas October 2009 Prepared for forthcoming INTERNATIONAL ENCYCLOPEDIA OF STATISTICAL SCIENCES, to be published by Springer

### Regression I: Mean Squared Error and Measuring Quality of Fit

Regression I: Mean Squared Error and Measuring Quality of Fit -Applied Multivariate Analysis- Lecturer: Darren Homrighausen, PhD 1 The Setup Suppose there is a scientific problem we are interested in solving

### Robust regression in Stata

The Stata Journal (2009) 9, Number 3, pp. 439 453 Robust regression in Stata Vincenzo Verardi 1 University of Namur (CRED) and Université Libre de Bruxelles (ECARES and CKE) Rempart de la Vierge 8, B-5000

### Multivariate Lineare Modelle

0-1 TALEB AHMAD CASE - Center for Applied Statistics and Economics Humboldt-Universität zu Berlin Motivation 1-1 Motivation Multivariate regression models can accommodate many explanatory which simultaneously

### Time Series. Anthony Davison. c

Series Anthony Davison c 2008 http://stat.epfl.ch Periodogram 76 Motivation............................................................ 77 Lutenizing hormone data..................................................

### Statistical Methods in Clinical Trials Categorical Data

Statistical Methods in Clinical Trials Categorical Data Types of Data quantitative Continuous Blood pressure Time to event Categorical sex qualitative Discrete No of relapses Ordered Categorical Pain level

### Comparing IRT with Other Models

Comparing IRT with Other Models Lecture #14 ICPSR Item Response Theory Workshop Lecture #14: 1of 45 Lecture Overview The final set of slides will describe a parallel between IRT and another commonly used

### Analysis of Fast Input Selection: Application in Time Series Prediction

Analysis of Fast Input Selection: Application in Time Series Prediction Jarkko Tikka, Amaury Lendasse, and Jaakko Hollmén Helsinki University of Technology, Laboratory of Computer and Information Science,

### A simple graphical method to explore tail-dependence in stock-return pairs

A simple graphical method to explore tail-dependence in stock-return pairs Klaus Abberger, University of Konstanz, Germany Abstract: For a bivariate data set the dependence structure can not only be measured

### ASA Section on Survey Research Methods

REGRESSION-BASED STATISTICAL MATCHING: RECENT DEVELOPMENTS Chris Moriarity, Fritz Scheuren Chris Moriarity, U.S. Government Accountability Office, 411 G Street NW, Washington, DC 20548 KEY WORDS: data

### TRANSFORMATION MODELS FOR INTERVAL SCALE GROUPED DATA WITH APPLICATIONS

Statistica Sinica 7(1997), 841-854 TRANSFORMATION MODELS FOR INTERVAL SCALE GROUPED DATA WITH APPLICATIONS Henry Horng-Shing Lu and Fushing Hsieh National Chiao Tung University and National Taiwan University

### CURE MODEL WITH CURRENT STATUS DATA

Statistica Sinica 19 (2009), 233-249 CURE MODEL WITH CURRENT STATUS DATA Shuangge Ma Yale University Abstract: Current status data arise when only random censoring time and event status at censoring are

### An EM-Algorithm Based Method to Deal with Rounded Zeros in Compositional Data under Dirichlet Models. Rafiq Hijazi

An EM-Algorithm Based Method to Deal with Rounded Zeros in Compositional Data under Dirichlet Models Rafiq Hijazi Department of Statistics United Arab Emirates University P.O. Box 17555, Al-Ain United

### The regression model with one stochastic regressor.

The regression model with one stochastic regressor. 3150/4150 Lecture 6 Ragnar Nymoen 30 January 2012 We are now on Lecture topic 4 The main goal in this lecture is to extend the results of the regression

### Solutions for Examination Categorical Data Analysis, March 21, 2013

STOCKHOLMS UNIVERSITET MATEMATISKA INSTITUTIONEN Avd. Matematisk statistik, Frank Miller MT 5006 LÖSNINGAR 21 mars 2013 Solutions for Examination Categorical Data Analysis, March 21, 2013 Problem 1 a.

### Mean squared error matrix comparison of least aquares and Stein-rule estimators for regression coefficients under non-normal disturbances

METRON - International Journal of Statistics 2008, vol. LXVI, n. 3, pp. 285-298 SHALABH HELGE TOUTENBURG CHRISTIAN HEUMANN Mean squared error matrix comparison of least aquares and Stein-rule estimators

### Lecture 4: Training a Classifier

Lecture 4: Training a Classifier Roger Grosse 1 Introduction Now that we ve defined what binary classification is, let s actually train a classifier. We ll approach this problem in much the same way as