The Logit Model: Estimation, Testing and Interpretation
|
|
- Lisa Stone
- 6 years ago
- Views:
Transcription
1 The Logit Model: Estimation, Testing and Interpretation Herman J. Bierens October 25, Introduction to maximum likelihood estimation 1.1 The likelihood function Consider a random sample Y 1,..., Y n from the Bernoulli distribution: Pr[Y j 1] p 0 Pr[Y j 0] 1 p 0, where p 0 is unknown. For example, toss n times a coin for which you suspect that it is unfair: p , and for each tossing j assign Y j 1if the outcome is heads and Y j 0if the outcome is tails. The question is how to estimate p 0 and how to test the null hypothesis that the coin is fair: p The probability function involved can be written as f(y p 0 ) Pr[Y j y] p y 0 (1 p 0 ) 1 y ( p0 if y 1, 1 p 0 if y 0. Next, let y 1,..., y n be a given sequence of zeros and ones. Thus, each y j is either 0 or 1. The joint probability function of the random sample Y 1,Y 2,..., Y n is defined as f n (y 1,..., y n p 0 )Pr[Y 1 y 1 and Y 2 y 2... and Y n y n ]. 1
2 Because the random variables Y 1,Y 2,..., Y n are independent, we can write hence Pr[Y 1 y 1 and Y 2 y 2... and Y n y n ] Pr[Y 1 y 1 ] Pr[Y 2 y 2 ]... Pr[Y n y n ] f(y 1 p 0 ) f(y 2 p 0 )... f(y n p 0 ) ny f(y j p 0 ), f n (y 1,..., y n p 0 ) ny ny p y j 0 (1 p 0 ) 1 y j p y j 0 P n y j ny (1 p 0 ) 1 y j p0 (1 p 0 ) n P n y j. Replacing the given non-random sequence y 1,..., y n by the random sample Y 1,Y 2,..., Y n and the unknown probability p 0 by a variable p in the interval (0, 1) yields the likelihood function P n L n (p) f n (Y 1,..., Y n p) p Y j (1 p) n P n Y j For the case p p 0 the likelihood function can be interpreted as the joint probability that we draw a particular sample Y 1,..., Y n. 1.2 Maximum likelihood estimation The idea of maximum likelihood (ML) estimation is now to choose p such that L n (p) is maximal. In other words, choose p such that the probability of drawing this particular sample Y 1,..., Y n is maximal. Note that maximizing L n (p) is equivalent to maximizing ln (L n (p)), i.e., ln (L n (p)) Y j ln(p)+ n Y j ln(1 p) n ³ Y ln(p)+(1 Y )ln(1 p), 2
3 where Y 1 n Y j isthesamplemean. Therefore,theMLestimator bp in this case can be obtained from the first-order condition for a maximum of ln (L n (p)) in p bp: 0 d ln (L à n(bp)) n Y d ln(bp)! d ln(1 bp) +(1 Y ) dbp dbp dbp à n Y d ln(bp)! d ln(1 bp) d(1 bp) +(1 Y ) dbp d(1 bp) dbp à n Y 1! bp +(1 Y ) 1 1 bp ( 1) à Y n bp 1 Y! n Y (1 bp) bp ³ 1 Y 1 bp bp (1 bp) Ã! Y bp n bp (1 bp) wherewehaveusedthefactthatd ln(x)/dx 1/x. Thus, in this case the ML estimator bp of p 0 isthesamplemean: bp Y. Note that this is an unbiased estimator: E (bp) 1 n P n E (Y j )p Large sample statistical inference It can be shown (but this requires advanced probability theory) that if the sample size n is large then n (bp p 0 ) is approximately normally distributed, i.e., n (bp p0 ) 1 X n (Y j p 0 ) N[0, σ 2 n 0 ], where σ 2 0 var(y j )E h (Y j p 0 ) 2i (1 p 0 ) 2 p 0 +( p 0 ) 2 (1 p 0 ) p 0 (1 p 0 ). 3
4 Thus, for large sample size n, n (bp p0 ) N[0, 1]. (1) qp 0 (1 p 0 ) This result can be used to test hypotheses about p 0. In particular, under the null hypothesis that the coin is fair, p 0 0.5, wehave 2 n (bp 0.5) n (bp 0.5) N[0, 1], Therefore, 2 n (bp 0.5) can be used as the test statistic of the standard normal test of the null hypothesis p 0 1/2, as follows. Recall that for a standard normal random variable U, Pr [ U > 1.96] Thus, under the null hypothesis p 0 1/2 one would expect that Pr h 2 i n (bp 0.5) > Pr h 2 i n (bp 0.5) If 2 n (bp 0.5) > 1.96 then we reject the null hypothesis p 0 1/2at the 5% significance level, because this is not what one would expect if the null hypothesis is true, and if 2 n (bp 0.5) 1.96 then we accept this null hypothesis, as this result is then in accordance with the null hypothesis p 0 1/2. The result (1) can also be used to endow the unknown probability p 0 with aconfidence interval, for example the 95% confidence interval, as follows. The result (1) implies Pr n (bp p0 ) q p0 (1 p 0 ) , which, after some straightforward calculations, can be shown to be equivalent to Pr h i p n p 0 p n 0.95 where q p n n.bp +(1.96)2 / n.bp (1 bp)+(1.96) 2 /4 n +(1.96) 2 q p n n.bp +(1.96)2 / n.bp (1 bp)+(1.96) 2 /4 n +(1.96) 2 4
5 The interval h p n, p n i is now the 95% confidence interval for p An application election polls Consider a presidential election with two candidates, candidate A and candidate B, and let p 0 be the fraction of likely voters who favor candidate A, just before the election is held. To predict the outcome of the election, a polling agency draws a random sample of size n 3000, for example, from the population of likely voters. 1 Suppose that 1800 of the respondents express a preference for candidate A. Thus, the fraction of respondents favoring candidate A is bp 0.6. Substituting n 3000 and bp 0.6 in the formulas for p n and p n yields p n 0.58, p n 0.62 Thus, the 95% confidence interval of 100 p 0 is [58, 62]. The polling results are therefore stated as: 60% of the likely voters will vote for candidate A, with a margin of error of ±2 points. 2 Motivation for maximum likelihood estimation A more formal motivation for ML estimation is based on the fact that for 0 <x<1 and x>1, ln(x) <x 1. This is illustrated in the following picture: 1 How to draw such a sample is beyond the scope of this lecture note. 5
6 ln(x) x 1. The inequality ln(x) <x 1 is strict for x 6 1, and ln(1) 0. Consequently, taking x f(y j p)/f(y j p 0 ), we have the inequality Ã! f(yj p) ln f(y j p) f(y j p 0 ) f(y j p 0 ) 1. Taking expectations, it follows that " Ã!# " # f(yj p) f(yj p) E ln E 1 f(y j p 0 ) f(y j p 0 ) f(1 p) f(1 p 0 ) Pr[Y j 1]+ f(0 p) f(0 p 0 ) Pr[Y j 0] 1 hence p p p (1 p 0 ) 1 p 0 1 p 0 p +1 p 10, (2) " E [ln (f(y j p))] E [ln (f(y j p 0 ))] E ln Ã!# f(yj p) 0, f(y j p 0 ) and therefore, E [ln (L n (p))] E [ln (L n (p 0 ))]. (3) Thus, E [ln (L n (p))] is maximal for p p 0, anditcanbeshownthatthis maximum is unique. 6
7 3 Maximum likelihood estimation of the Logit model 3.1 The Logit model with one explanatory variable Next, let (Y 1,X 1 ),..., (Y n,x n ) be a random sample from the conditional Logit distribution: Pr[Y j 1 X j ] 1 1+exp( α 0 β 0 X j ), (4) Pr[Y j 0 X j ] 1 Pr[Y j 1 X j ] exp ( α 0 β 0 X j ) 1+exp( α 0 β 0 X j ) where the X j s are the explanatory variables and α 0 and β 0 are unknown parameters to be estimated. This model is called a Logit model, because Pr[Y j 1 X j ]F (α 0 + β 0 X j ) (5) where 1 F (x) 1+exp( x) is the distribution function of the logistic (Logit) distribution. The conditional probability function involved is (6) f(y X j, α 0, β 0 ) Pr[Y j y X j ] F (α 0 + β 0 X j ) y (1 F (α 0 + β 0 X j )) 1 y ( F (α0 + β 0 X j ) if y 1, 1 F (α 0 + β 0 X j ) if y 0. Now the conditional log-likelihood function is ln (L n (α, β)) ln (f(y j X j, α, β)) Y j ln (F (α + βx j )) + (1 Y j )(α + βx j ) 7 (1 Y j )ln(1 F (α + βx j )) ln (1 + exp ( α βx j )). (7)
8 Similar to (3) we have E [ln(l n (α, β)) X 1,..., X n ] E [ln(l n (α 0, β 0 )) X 1,..., X n ]. Again, this result motivates to estimate α 0 and β 0 by maximizing ln (L n (α, β)) to α and β: ln ³ L n (bα, β) b max ln (L n(α, β)). α,β However, there is no longer an explicit solution for bα and β. b These ML estimators have to be solved numerically. Your econometrics software will do that for you. 3.2 Pseudo t-values It can be shown that if the sample size n is large then n (bα α0 ) N(0, σ 2 α), n ³b β β 0 N(0, σ 2 β ). Given consistent estimators bσ 2 α and bσ 2 β of the unknown variances σ 2 α and σ 2 β, respectively (which are computed by your econometrics software), we then have ³ n (bα α0 ) n β b β 0 N(0, 1), N(0, 1). bσ α bσ β These results can be used to test whether the coefficients α 0 and β 0 are zero or not. In particular the null hypothesis β 0 0is of interest, because this hypothesis implies that the conditional probability Pr[Y j 1 X j ] does not depend on X j. Under the null hypothesis β 0 0we have bt β n b β bσ β N(0, 1). Recall that the 5% critical value of the two-sided standard normal test is Thus, for example, the null hypothesis β 0 0is rejected at the 5% significance level in favor of the alternative hypothesis β 0 60if bt β > 1.96, andacceptedif bt β The statistic t b β is called the pseudo t-value of β b because it is used in the same way as the t-value in linear regression, and bσ β is called the standard error of β. b Your econometric software will report the ML estimators together with their corresponding pseudo t-values and/or standard errors. 8
9 3.3 The general Logit model The general Logit model takes the form Pr[Y j 1 X 1j,...X k,j ] 1 1+exp( β1x 0 1j... βk 0X kj) 1 1+exp ³ P k i1 βi 0 X ij, (8) where one of the X ij equals 1 for the constant term, for example, let X kj 1, and the β 0 i s are the true parameter values. This model can be estimated by ML in the same way as before. Thus, the log-likelihood function is ln (L n (β 1,..., β k )) (1 Y j ) kx β i X ij i1 Ã Ã ln 1+exp!! kx β i X ij, (9) and the ML estimators b β 1,..., b β k are obtained by maximizing ln (L n (β 1,..., β k )): ln ³ L n ( b β 1,..., b β k ) max β 1,...,β k ln (L n (β 1,..., β k )). Again, it can be shown that if n is large then for i 1,..., k, ³ n β b i βi 0 N[0, σ 2 i ]. i1 Given consistent estimators bσ 2 i of the variances σ2 i, it follows then that ³ n β b i βi 0 bσ i N[0, 1] for i 1,..., k. Your econometrics software will report the ML estimators bβ i together with their corresponding pseudo t-values t b i nβ b i /bσ i and/or standard errors bσ i. 3.4 Testing joint significance Now suppose you want to test the joint null hypothesis H 0 : β1 0 0, β0 2 0,..., β0 m 0, (10) 9
10 where m<k. There are two ways to do that. One way is akin to the F test in linear regression: Re-estimate the Logit model under the null hypothesis: ln ³ L n (0,..,0, e β m+1,..., e β k ) max β m+1,...,β k ln (L n (0,.., 0, β m+1,..., β k )). and compare the log-likelihoods 2. Itcanbeshownthatunderthenullhypothesis (10) and for large samples, Ã Ln (0,.., 0, β e LR m 2ln m+1,..., β e! k ) L n ( β b 1,..., β b χ 2 m k ), where the degrees of freedom m corresponds to the number of restrictions imposed under the null hypothesis. This is the so-called likelihood ratio test, which is conducted right-sided. For example, choose the 5% significance level, look up in the table of the χ 2 distribution the critical value c such that for a χ 2 m distributed random variable Z m, Pr[Z m >c]0.05. Then the null hypothesis (10) is rejected at the 5% significance level if LR m >cand accepted if LR m c. An alternative test of the null hypothesis (10) is the Wald test, which is conducted in the same way as for linear regression models. 3 Under the null hypothesis (10) the Wald test statistic has also a χ 2 m distribution. 4 Interpretation of the coefficients of the Logit model 4.1 Marginal effects Consider the Logit model (5). If β 0 > 0 then Pr[Y j 1 X j ]F (α 0 + β 0 X j ) is an increasing function of X j : where F 0 is the derivative of (6): dp [Y j 1 X j ] dx j β 0.F 0 (α 0 + β 0 X j ), 2 Your econometric software will report the log-likelihood function value. 3 In EasyReg International the Wald test can be conducted simply by point-and-click. 10
11 F 0 (x) exp( x) (1 + exp( x)) 2 1+exp( x) (1 + exp( x)) 2 1 (1 + exp( x)) exp( x) 1 2 F (x) F (x)2 (1 + exp( x)) F (x)(1 F (x)). Therefore, the marginal effect of X j on Pr[Y j 1 X j ] depends on X j : dp [Y j 1 X j ] β 0.F (α 0 + β 0 X j )(1 F (α 0 + β 0 X j )), dx j which renders the interpretation of β 0 difficult. However, the coefficient β 0 can be interpreted in terms of relative changes in odds. 4.2 Odds and odds ratios The odds is the ratio of the probability that something is true divided by the probability that it is not true. Thus, in the Logit case (4), Odds (X) Pr[Y j 1 X j ] Pr[Y j 0 X j ] F (α 0 + β 0 X j ) 1 F (α 0 + β 0 X j ) exp(α 0 + β 0 X j ). (11) The odds ratio is the ratio of two odds for different values of X j, say X j x and X j x + x: Odds (x + x) Odds (x) exp(α + βx + β x) exp(α + βx) exp(β x), where x is a small change in x. Then Ã! 1 Odds (x + x) Odds (x) exp(β 0 x) 1 lim lim x 0 x Odds (x) x 0 x exp(β 0 x) 1 β 0 lim β 0 d exp(u) β 0 exp(0) β 0. β 0 x 0 β 0 x du u0 Thus, β 0 may be interpreted as the relative change in the odds due to a small change x in X j : Odds (x + x) Odds (x) Odds (x) 11 Odds (x + x) Odds (x) 1 β 0 x (12)
12 If X j is a binary variable itself, X j 0or X j 1, then the only reasonable choices for x + x and x are 1 and 0, respectively, so that then Odds (1) (1) Odds (0) 1Odds Odds (0) Odds (0) exp(β 0 ) 1. Only if β 0 is small we may then use the approximation exp(β 0 ) 1 β 0. If not, one has to interpret β 0 in terms of the log of the odds ratio involved: Ã! Odds (1) ln β 0. Odds (0) The interpretation of the coefficients β 0 i,i 1,..., k 1 in the general Logit model (8) is similar as in the case (12): Odds (X 1j,..., X i 1,j,X i,j + X i,j,x i+1,j,..., X k,j ) Odds (X 1j,..., X i 1,j,X i,j,x i+1,j,..., X k,j ) 1 β 0 i X i,j if X i,j is small. For example, β 0 i may be interpreted as the percentage change in Odds(X 1j,.., X k,j ) due to a small percentage change 100 X i,j 1 in X i,j. 12
Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing
Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October
More informationInferences About Two Proportions
Inferences About Two Proportions Quantitative Methods II Plan for Today Sampling two populations Confidence intervals for differences of two proportions Testing the difference of proportions Examples 1
More informationIntroductory Econometrics. Review of statistics (Part II: Inference)
Introductory Econometrics Review of statistics (Part II: Inference) Jun Ma School of Economics Renmin University of China October 1, 2018 1/16 Null and alternative hypotheses Usually, we have two competing
More informationHigh-Throughput Sequencing Course
High-Throughput Sequencing Course DESeq Model for RNA-Seq Biostatistics and Bioinformatics Summer 2017 Outline Review: Standard linear regression model (e.g., to model gene expression as function of an
More informationDirection: This test is worth 250 points and each problem worth points. DO ANY SIX
Term Test 3 December 5, 2003 Name Math 52 Student Number Direction: This test is worth 250 points and each problem worth 4 points DO ANY SIX PROBLEMS You are required to complete this test within 50 minutes
More informationLECTURE 10: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING. The last equality is provided so this can look like a more familiar parametric test.
Economics 52 Econometrics Professor N.M. Kiefer LECTURE 1: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING NEYMAN-PEARSON LEMMA: Lesson: Good tests are based on the likelihood ratio. The proof is easy in the
More informationEconometrics Lecture 5: Limited Dependent Variable Models: Logit and Probit
Econometrics Lecture 5: Limited Dependent Variable Models: Logit and Probit R. G. Pierse 1 Introduction In lecture 5 of last semester s course, we looked at the reasons for including dichotomous variables
More informationGoals. PSCI6000 Maximum Likelihood Estimation Multiple Response Model 1. Multinomial Dependent Variable. Random Utility Model
Goals PSCI6000 Maximum Likelihood Estimation Multiple Response Model 1 Tetsuya Matsubayashi University of North Texas November 2, 2010 Random utility model Multinomial logit model Conditional logit model
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationLecture 10: Introduction to Logistic Regression
Lecture 10: Introduction to Logistic Regression Ani Manichaikul amanicha@jhsph.edu 2 May 2007 Logistic Regression Regression for a response variable that follows a binomial distribution Recall the binomial
More informationHypothesis testing Goodness of fit Multicollinearity Prediction. Applied Statistics. Lecturer: Serena Arima
Applied Statistics Lecturer: Serena Arima Hypothesis testing for the linear model Under the Gauss-Markov assumptions and the normality of the error terms, we saw that β N(β, σ 2 (X X ) 1 ) and hence s
More informationIntroduction to Machine Learning. Lecture 2
Introduction to Machine Learning Lecturer: Eran Halperin Lecture 2 Fall Semester Scribe: Yishay Mansour Some of the material was not presented in class (and is marked with a side line) and is given for
More informationSTAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots. March 8, 2015
STAT 135 Lab 6 Duality of Hypothesis Testing and Confidence Intervals, GLRT, Pearson χ 2 Tests and Q-Q plots March 8, 2015 The duality between CI and hypothesis testing The duality between CI and hypothesis
More informationSingle-level Models for Binary Responses
Single-level Models for Binary Responses Distribution of Binary Data y i response for individual i (i = 1,..., n), coded 0 or 1 Denote by r the number in the sample with y = 1 Mean and variance E(y) =
More informationECON 594: Lecture #6
ECON 594: Lecture #6 Thomas Lemieux Vancouver School of Economics, UBC May 2018 1 Limited dependent variables: introduction Up to now, we have been implicitly assuming that the dependent variable, y, was
More informationBinary Dependent Variables
Binary Dependent Variables In some cases the outcome of interest rather than one of the right hand side variables - is discrete rather than continuous Binary Dependent Variables In some cases the outcome
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationBayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence
Bayesian Inference in GLMs Frequentists typically base inferences on MLEs, asymptotic confidence limits, and log-likelihood ratio tests Bayesians base inferences on the posterior distribution of the unknowns
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models Generalized Linear Models - part III Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs.
More informationChapter 10. Hypothesis Testing (I)
Chapter 10. Hypothesis Testing (I) Hypothesis Testing, together with statistical estimation, are the two most frequently used statistical inference methods. It addresses a different type of practical problems
More informationEconomics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1,
Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem
More information1/24/2008. Review of Statistical Inference. C.1 A Sample of Data. C.2 An Econometric Model. C.4 Estimating the Population Variance and Other Moments
/4/008 Review of Statistical Inference Prepared by Vera Tabakova, East Carolina University C. A Sample of Data C. An Econometric Model C.3 Estimating the Mean of a Population C.4 Estimating the Population
More informationStatistical Analysis of List Experiments
Statistical Analysis of List Experiments Graeme Blair Kosuke Imai Princeton University December 17, 2010 Blair and Imai (Princeton) List Experiments Political Methodology Seminar 1 / 32 Motivation Surveys
More informationChapter 9 Regression with a Binary Dependent Variable. Multiple Choice. 1) The binary dependent variable model is an example of a
Chapter 9 Regression with a Binary Dependent Variable Multiple Choice ) The binary dependent variable model is an example of a a. regression model, which has as a regressor, among others, a binary variable.
More informationSome Basic Concepts of Probability and Information Theory: Pt. 2
Some Basic Concepts of Probability and Information Theory: Pt. 2 PHYS 476Q - Southern Illinois University January 22, 2018 PHYS 476Q - Southern Illinois University Some Basic Concepts of Probability and
More informationEcon 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE
Econ 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE Eric Zivot Winter 013 1 Wald, LR and LM statistics based on generalized method of moments estimation Let 1 be an iid sample
More informationBinomial Model. Lecture 10: Introduction to Logistic Regression. Logistic Regression. Binomial Distribution. n independent trials
Lecture : Introduction to Logistic Regression Ani Manichaikul amanicha@jhsph.edu 2 May 27 Binomial Model n independent trials (e.g., coin tosses) p = probability of success on each trial (e.g., p =! =
More informationExercises Chapter 4 Statistical Hypothesis Testing
Exercises Chapter 4 Statistical Hypothesis Testing Advanced Econometrics - HEC Lausanne Christophe Hurlin University of Orléans December 5, 013 Christophe Hurlin (University of Orléans) Advanced Econometrics
More informationHypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3
Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest
More informationECON 4160, Autumn term Lecture 1
ECON 4160, Autumn term 2017. Lecture 1 a) Maximum Likelihood based inference. b) The bivariate normal model Ragnar Nymoen University of Oslo 24 August 2017 1 / 54 Principles of inference I Ordinary least
More informationMaximum-Likelihood Estimation: Basic Ideas
Sociology 740 John Fox Lecture Notes Maximum-Likelihood Estimation: Basic Ideas Copyright 2014 by John Fox Maximum-Likelihood Estimation: Basic Ideas 1 I The method of maximum likelihood provides estimators
More information22s:152 Applied Linear Regression. Example: Study on lead levels in children. Ch. 14 (sec. 1) and Ch. 15 (sec. 1 & 4): Logistic Regression
22s:52 Applied Linear Regression Ch. 4 (sec. and Ch. 5 (sec. & 4: Logistic Regression Logistic Regression When the response variable is a binary variable, such as 0 or live or die fail or succeed then
More informationUC Berkeley Math 10B, Spring 2015: Midterm 2 Prof. Sturmfels, April 9, SOLUTIONS
UC Berkeley Math 10B, Spring 2015: Midterm 2 Prof. Sturmfels, April 9, SOLUTIONS 1. (5 points) You are a pollster for the 2016 presidential elections. You ask 0 random people whether they would vote for
More informationChapter 8 Heteroskedasticity
Chapter 8 Walter R. Paczkowski Rutgers University Page 1 Chapter Contents 8.1 The Nature of 8. Detecting 8.3 -Consistent Standard Errors 8.4 Generalized Least Squares: Known Form of Variance 8.5 Generalized
More informationSolutions for Examination Categorical Data Analysis, March 21, 2013
STOCKHOLMS UNIVERSITET MATEMATISKA INSTITUTIONEN Avd. Matematisk statistik, Frank Miller MT 5006 LÖSNINGAR 21 mars 2013 Solutions for Examination Categorical Data Analysis, March 21, 2013 Problem 1 a.
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationChapter 11. Regression with a Binary Dependent Variable
Chapter 11 Regression with a Binary Dependent Variable 2 Regression with a Binary Dependent Variable (SW Chapter 11) So far the dependent variable (Y) has been continuous: district-wide average test score
More information9 Generalized Linear Models
9 Generalized Linear Models The Generalized Linear Model (GLM) is a model which has been built to include a wide range of different models you already know, e.g. ANOVA and multiple linear regression models
More informationChapter 14 Logistic Regression, Poisson Regression, and Generalized Linear Models
Chapter 14 Logistic Regression, Poisson Regression, and Generalized Linear Models 許湘伶 Applied Linear Regression Models (Kutner, Nachtsheim, Neter, Li) hsuhl (NUK) LR Chap 10 1 / 29 14.1 Regression Models
More informationProbability & Statistics - FALL 2008 FINAL EXAM
550.3 Probability & Statistics - FALL 008 FINAL EXAM NAME. An urn contains white marbles and 8 red marbles. A marble is drawn at random from the urn 00 times with replacement. Which of the following is
More informationLECTURE 5. Introduction to Econometrics. Hypothesis testing
LECTURE 5 Introduction to Econometrics Hypothesis testing October 18, 2016 1 / 26 ON TODAY S LECTURE We are going to discuss how hypotheses about coefficients can be tested in regression models We will
More informationFundamental Probability and Statistics
Fundamental Probability and Statistics "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are
More informationECON Introductory Econometrics. Lecture 11: Binary dependent variables
ECON4150 - Introductory Econometrics Lecture 11: Binary dependent variables Monique de Haan (moniqued@econ.uio.no) Stock and Watson Chapter 11 Lecture Outline 2 The linear probability model Nonlinear probability
More informationHT Introduction. P(X i = x i ) = e λ λ x i
MODS STATISTICS Introduction. HT 2012 Simon Myers, Department of Statistics (and The Wellcome Trust Centre for Human Genetics) myers@stats.ox.ac.uk We will be concerned with the mathematical framework
More informationPOLI 7050 Spring 2008 March 5, 2008 Unordered Response Models II
POLI 7050 Spring 2008 March 5, 2008 Unordered Response Models II Introduction Today we ll talk about interpreting MNL and CL models. We ll start with general issues of model fit, and then get to variable
More informationSection 9.4. Notation. Requirements. Definition. Inferences About Two Means (Matched Pairs) Examples
Objective Section 9.4 Inferences About Two Means (Matched Pairs) Compare of two matched-paired means using two samples from each population. Hypothesis Tests and Confidence Intervals of two dependent means
More informationStandard Errors & Confidence Intervals. N(0, I( β) 1 ), I( β) = [ 2 l(β, φ; y) β i β β= β j
Standard Errors & Confidence Intervals β β asy N(0, I( β) 1 ), where I( β) = [ 2 l(β, φ; y) ] β i β β= β j We can obtain asymptotic 100(1 α)% confidence intervals for β j using: β j ± Z 1 α/2 se( β j )
More informationMath 416 Lecture 3. The average or mean or expected value of x 1, x 2, x 3,..., x n is
Math 416 Lecture 3 Expected values The average or mean or expected value of x 1, x 2, x 3,..., x n is x 1 x 2... x n n x 1 1 n x 2 1 n... x n 1 n 1 n x i p x i where p x i 1 n is the probability of x i
More informationLecture 2: Review of Probability
Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................
More informationLing 289 Contingency Table Statistics
Ling 289 Contingency Table Statistics Roger Levy and Christopher Manning This is a summary of the material that we ve covered on contingency tables. Contingency tables: introduction Odds ratios Counting,
More informationPOLI 443 Applied Political Research
POLI 443 Applied Political Research Session 6: Tests of Hypotheses Contingency Analysis Lecturer: Prof. A. Essuman-Johnson, Dept. of Political Science Contact Information: aessuman-johnson@ug.edu.gh College
More informationTesting and Model Selection
Testing and Model Selection This is another digression on general statistics: see PE App C.8.4. The EViews output for least squares, probit and logit includes some statistics relevant to testing hypotheses
More informationMath 151. Rumbos Fall Solutions to Review Problems for Exam 2. Pr(X = 1) = ) = Pr(X = 2) = Pr(X = 3) = p X. (k) =
Math 5. Rumbos Fall 07 Solutions to Review Problems for Exam. A bowl contains 5 chips of the same size and shape. Two chips are red and the other three are blue. Draw three chips from the bowl at random,
More informationMaximum Likelihood (ML) Estimation
Econometrics 2 Fall 2004 Maximum Likelihood (ML) Estimation Heino Bohn Nielsen 1of32 Outline of the Lecture (1) Introduction. (2) ML estimation defined. (3) ExampleI:Binomialtrials. (4) Example II: Linear
More informationBinary Models with Endogenous Explanatory Variables
Binary Models with Endogenous Explanatory Variables Class otes Manuel Arellano ovember 7, 2007 Revised: January 21, 2008 1 Introduction In Part I we considered linear and non-linear models with additive
More informationThis paper is not to be removed from the Examination Halls
~~ST104B ZA d0 This paper is not to be removed from the Examination Halls UNIVERSITY OF LONDON ST104B ZB BSc degrees and Diplomas for Graduates in Economics, Management, Finance and the Social Sciences,
More informationStatistical Distribution Assumptions of General Linear Models
Statistical Distribution Assumptions of General Linear Models Applied Multilevel Models for Cross Sectional Data Lecture 4 ICPSR Summer Workshop University of Colorado Boulder Lecture 4: Statistical Distributions
More informationGeneralized logit models for nominal multinomial responses. Local odds ratios
Generalized logit models for nominal multinomial responses Categorical Data Analysis, Summer 2015 1/17 Local odds ratios Y 1 2 3 4 1 π 11 π 12 π 13 π 14 π 1+ X 2 π 21 π 22 π 23 π 24 π 2+ 3 π 31 π 32 π
More informationUnit 9: Inferences for Proportions and Count Data
Unit 9: Inferences for Proportions and Count Data Statistics 571: Statistical Methods Ramón V. León 12/15/2008 Unit 9 - Stat 571 - Ramón V. León 1 Large Sample Confidence Interval for Proportion ( pˆ p)
More informationThis exam contains 13 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability and Statistics FS 2017 Session Exam 22.08.2017 Time Limit: 180 Minutes Name: Student ID: This exam contains 13 pages (including this cover page) and 10 questions. A Formulae sheet is provided
More informationGeneralized linear models for binary data. A better graphical exploratory data analysis. The simple linear logistic regression model
Stat 3302 (Spring 2017) Peter F. Craigmile Simple linear logistic regression (part 1) [Dobson and Barnett, 2008, Sections 7.1 7.3] Generalized linear models for binary data Beetles dose-response example
More informationSTAT 7030: Categorical Data Analysis
STAT 7030: Categorical Data Analysis 5. Logistic Regression Peng Zeng Department of Mathematics and Statistics Auburn University Fall 2012 Peng Zeng (Auburn University) STAT 7030 Lecture Notes Fall 2012
More information8 Nominal and Ordinal Logistic Regression
8 Nominal and Ordinal Logistic Regression 8.1 Introduction If the response variable is categorical, with more then two categories, then there are two options for generalized linear models. One relies on
More informationBinary Logistic Regression
The coefficients of the multiple regression model are estimated using sample data with k independent variables Estimated (or predicted) value of Y Estimated intercept Estimated slope coefficients Ŷ = b
More informationApplied Econometrics (QEM)
Applied Econometrics (QEM) based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #3 1 / 42 Outline 1 2 3 t-test P-value Linear
More informationPsychology 282 Lecture #4 Outline Inferences in SLR
Psychology 282 Lecture #4 Outline Inferences in SLR Assumptions To this point we have not had to make any distributional assumptions. Principle of least squares requires no assumptions. Can use correlations
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationTest Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics
Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics The candidates for the research course in Statistics will have to take two shortanswer type tests
More informationInference and Regression
Inference and Regression Assignment 4 - Solutions Department of IOMS Professor William Greene Phone: 212.998.0876 Office: KMC 7-90 Home page:www.stern.nyu.edu/~wgreene Email: wgreene@stern.nyu.edu Course
More informationUnit 9: Inferences for Proportions and Count Data
Unit 9: Inferences for Proportions and Count Data Statistics 571: Statistical Methods Ramón V. León 1/15/008 Unit 9 - Stat 571 - Ramón V. León 1 Large Sample Confidence Interval for Proportion ( pˆ p)
More informationST3241 Categorical Data Analysis I Generalized Linear Models. Introduction and Some Examples
ST3241 Categorical Data Analysis I Generalized Linear Models Introduction and Some Examples 1 Introduction We have discussed methods for analyzing associations in two-way and three-way tables. Now we will
More informationMath 180B Problem Set 3
Math 180B Problem Set 3 Problem 1. (Exercise 3.1.2) Solution. By the definition of conditional probabilities we have Pr{X 2 = 1, X 3 = 1 X 1 = 0} = Pr{X 3 = 1 X 2 = 1, X 1 = 0} Pr{X 2 = 1 X 1 = 0} = P
More informationExample. χ 2 = Continued on the next page. All cells
Section 11.1 Chi Square Statistic k Categories 1 st 2 nd 3 rd k th Total Observed Frequencies O 1 O 2 O 3 O k n Expected Frequencies E 1 E 2 E 3 E k n O 1 + O 2 + O 3 + + O k = n E 1 + E 2 + E 3 + + E
More informationLecture 14: Introduction to Poisson Regression
Lecture 14: Introduction to Poisson Regression Ani Manichaikul amanicha@jhsph.edu 8 May 2007 1 / 52 Overview Modelling counts Contingency tables Poisson regression models 2 / 52 Modelling counts I Why
More informationModelling counts. Lecture 14: Introduction to Poisson Regression. Overview
Modelling counts I Lecture 14: Introduction to Poisson Regression Ani Manichaikul amanicha@jhsph.edu Why count data? Number of traffic accidents per day Mortality counts in a given neighborhood, per week
More informationSo far our focus has been on estimation of the parameter vector β in the. y = Xβ + u
Interval estimation and hypothesis tests So far our focus has been on estimation of the parameter vector β in the linear model y i = β 1 x 1i + β 2 x 2i +... + β K x Ki + u i = x iβ + u i for i = 1, 2,...,
More informationA discussion on multiple regression models
A discussion on multiple regression models In our previous discussion of simple linear regression, we focused on a model in which one independent or explanatory variable X was used to predict the value
More informationWooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics
Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).
More informationBMI 541/699 Lecture 22
BMI 541/699 Lecture 22 Where we are: 1. Introduction and Experimental Design 2. Exploratory Data Analysis 3. Probability 4. T-based methods for continous variables 5. Power and sample size for t-based
More informationInference for Proportions, Variance and Standard Deviation
Inference for Proportions, Variance and Standard Deviation Sections 7.10 & 7.6 Cathy Poliak, Ph.D. cathy@math.uh.edu Office Fleming 11c Department of Mathematics University of Houston Lecture 12 Cathy
More informationData Mining 2018 Logistic Regression Text Classification
Data Mining 2018 Logistic Regression Text Classification Ad Feelders Universiteit Utrecht Ad Feelders ( Universiteit Utrecht ) Data Mining 1 / 50 Two types of approaches to classification In (probabilistic)
More informationUsing the Delta Method to Construct Confidence Intervals for Predicted Probabilities, Rates, and Discrete Changes 1
Using the Delta Method to Construct Confidence Intervals for Predicted Probabilities, Rates, Discrete Changes 1 JunXuJ.ScottLong Indiana University 2005-02-03 1 General Formula The delta method is a general
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationSTA 2101/442 Assignment 2 1
STA 2101/442 Assignment 2 1 These questions are practice for the midterm and final exam, and are not to be handed in. 1. A polling firm plans to ask a random sample of registered voters in Quebec whether
More informationSampling, Confidence Interval and Hypothesis Testing
Sampling, Confidence Interval and Hypothesis Testing Christopher Grigoriou Executive MBA HEC Lausanne 2007-2008 1 Sampling : Careful with convenience samples! World War II: A statistical study to decide
More informationCourse: ESO-209 Home Work: 1 Instructor: Debasis Kundu
Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear
More informationHypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006
Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312 This draft, April 20, 2006 1 1 A Brief Review of Hypothesis Testing and Its Uses values and pure significance tests (R.A. Fisher)
More informationNaïve Bayes classification
Naïve Bayes classification 1 Probability theory Random variable: a variable whose possible values are numerical outcomes of a random phenomenon. Examples: A person s height, the outcome of a coin toss
More information14.30 Introduction to Statistical Methods in Economics Spring 2009
MIT OpenCourseWare http://ocw.mit.edu 4.0 Introduction to Statistical Methods in Economics Spring 009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationLecture 12: Effect modification, and confounding in logistic regression
Lecture 12: Effect modification, and confounding in logistic regression Ani Manichaikul amanicha@jhsph.edu 4 May 2007 Today Categorical predictor create dummy variables just like for linear regression
More informationLogistic Regressions. Stat 430
Logistic Regressions Stat 430 Final Project Final Project is, again, team based You will decide on a project - only constraint is: you are supposed to use techniques for a solution that are related to
More informationHierarchical Generalized Linear Models. ERSH 8990 REMS Seminar on HLM Last Lecture!
Hierarchical Generalized Linear Models ERSH 8990 REMS Seminar on HLM Last Lecture! Hierarchical Generalized Linear Models Introduction to generalized models Models for binary outcomes Interpreting parameter
More informationIntroduction Large Sample Testing Composite Hypotheses. Hypothesis Testing. Daniel Schmierer Econ 312. March 30, 2007
Hypothesis Testing Daniel Schmierer Econ 312 March 30, 2007 Basics Parameter of interest: θ Θ Structure of the test: H 0 : θ Θ 0 H 1 : θ Θ 1 for some sets Θ 0, Θ 1 Θ where Θ 0 Θ 1 = (often Θ 1 = Θ Θ 0
More informationFöreläsning /31
1/31 Föreläsning 10 090420 Chapter 13 Econometric Modeling: Model Speci cation and Diagnostic testing 2/31 Types of speci cation errors Consider the following models: Y i = β 1 + β 2 X i + β 3 X 2 i +
More informationEPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7
Introduction to Generalized Univariate Models: Models for Binary Outcomes EPSY 905: Fundamentals of Multivariate Modeling Online Lecture #7 EPSY 905: Intro to Generalized In This Lecture A short review
More informationMath 494: Mathematical Statistics
Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/
More informationModeling Binary Outcomes: Logit and Probit Models
Modeling Binary Outcomes: Logit and Probit Models Eric Zivot December 5, 2009 Motivating Example: Women s labor force participation y i = 1 if married woman is in labor force = 0 otherwise x i k 1 = observed
More informationCount data page 1. Count data. 1. Estimating, testing proportions
Count data page 1 Count data 1. Estimating, testing proportions 100 seeds, 45 germinate. We estimate probability p that a plant will germinate to be 0.45 for this population. Is a 50% germination rate
More informationBinary choice 3.3 Maximum likelihood estimation
Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Output of the estimation We explain here the various outputs from the maximum likelihood estimation procedure. Solution of the maximum likelihood
More information