δ -method and M-estimation
|
|
- Nickolas Neal
- 5 years ago
- Views:
Transcription
1 Econ 2110, fall 2016, Part IVb Asymptotic Theory: δ -method and M-estimation Maximilian Kasy Department of Economics, Harvard University 1 / 40
2 Example Suppose we estimate the average effect of class size on student exam grades, using the project STAR data. What is the variance of our estimator? Can we form a confidence set for the size of the effect? Can we reject the null hypothesis of a zero average effect? Also if exam scores are not normally distributed? 2 / 40
3 Example Suppose we estimate the top 1% income share using data on the number of individuals in different tax brackets, assuming that top incomes are Pareto distributed. Suppose we calculate the implied optimal top tax rate. Can we form a 95% confidence interval for this optimal tax rate? 3 / 40
4 Takeaways for this part of class How we get our formulas for standard deviations in many settings. When and why we can expect asymptotic normality for many estimators (and what that means). When we might expect problems to arise for asymptotic approximations. 4 / 40
5 Roadmap IVa IVb Types of convergence Laws of large numbers (LLN) and central limit theorems (CLT) The delta method M- and Z -Estimators Special M-Estimators Ordinary least squares (OLS) Maximum likelihood estimation (MLE) Confidence sets 5 / 40
6 Part IVb The delta method M- and Z-Estimators Consistency Asymptotic normality Special M-Estimators Least squares Maximum likelihood Confidence sets 6 / 40
7 The delta method The delta method Suppose we know the asymptotic behavior of sequence X n, we are interested in Y n = g(x n ), and g is smooth. Often a Taylor expansion of g around the probability limit of X n yields the answer, where we can ignore higher order terms in the limit. Y n = g(β) + g (β) (X n β) + o( X n β ). This idea is called the delta method. 7 / 40
8 The delta method Theorem (Delta method) Assume that r n (X n β) d X for some sequence r n and some random variable X. Let Y n = g(x n ) for a function g which is differentiable at β. Then r n (Y n g(β)) d g (β) X. 8 / 40
9 The delta method Proof: By differentiability of g, Rearranging gives Y n = g(β) + g (β) (X n β) + o( X n β ). r n (Y n g(β)) = r n g (β) (X n β) + r n o( X n β ). The second term vanishes asymptotically, since r n (X n β) converges in distribution. The continuous mapping theorem applied to matrix multiplication by g (β) now yields the claim. 9 / 40
10 The delta method Leading special case Let X n be a sequence of random variables such that n(xn b) d N (0,σ 2 ). Let g : R R be continuously differentiable at a. Then n(g(xn ) g(b)) d N (0,(g (b)) 2 σ 2 ). 10 / 40
11 The delta method Attention There are important cases where the delta method provides poor approximations Examples: near β = 0, for 1. g(x) = X 2. g(x) = 1/X 3. g(x) = X Relevant for: 1. weak instruments 2. inference under partial identification / moment inequalities 11 / 40
12 The delta method Practice problem Suppose X i are iid with mean 1 and variance 2, and n = 25. Let Y = X 2. Provide an approximation for the distribution of Y. Now suppose X i has mean 0 and variance 2. Provide an approximation for the distribution of Y. 12 / 40
13 M- and Z-Estimators M- and Z-Estimators Many interesting objects β can be written in the form β 0 = argmax β This defines a mapping from the probability distribution of X to a parameter β. In our decision theory notation: β 0 = β(θ) E[m(β,X)]. (1) 13 / 40
14 M- and Z-Estimators Example - Least squares The coefficients β 0 of the best linear predictor Ŷ = X β 0 minimize the average squared prediction error, β 0 = argmin β E[(Y X β) 2 ]. Thus m(β,x,y ) = (Y X β) / 40
15 M- and Z-Estimators Example - Maximum likelihood Suppose Y is distributed according to the density Y f(y,β 0 ). Then β 0 maximizes the expected log likelihood, We will show this later. β 0 = argmax β E[log(f(Y,β))]. Thus m(β,x) = log(f(y,β)). 15 / 40
16 M- and Z-Estimators M-Estimator Use E n to denote the sample average, e.g. E n [X] = 1 n n i=1 We can define an estimator for β which solves the analogous conditions replacing the population expectation by a sample average, that is β = argmax β X i. E n [m(β,x)]. (2) Such an estimator is called an M-estimator (for maximizer ). 16 / 40
17 M- and Z-Estimators Examples continued 1. Least squares: ordinary least squares (OLS) estimator β = argmin β 2. Maximum likelihood: maximum likelihood estimator (MLE) β = argmax β E n [(Y X β) 2 ] E n [log(f(y,β))] 17 / 40
18 M- and Z-Estimators Z-Estimator If m is differentiable and β is an interior maximizer equation (1) implies the first order conditions β E[m(β,X)] = E[m (β 0,X)] = 0. If we directly define the estimator via then β is called a Z-estimator (for zero ). E n [m ( β,x i )] = 0, (3) 18 / 40
19 M- and Z-Estimators Practice problem Find the first order conditions for MLE and for OLS 19 / 40
20 M- and Z-Estimators Solution: 1. Least squares: where E n [ê X] = 0 ê = Y X β is the regression residual. 2. Maximum likelihood: where is called the score. E n [S(Y, β) ] = 0 S(Y,β) := log(f(y,β)) β 20 / 40
21 M- and Z-Estimators Consistency Consistency Basic requirement for good estimators: That they are close to the population estimand with large probability as sample sizes get large: P( β β 0 < ε) 1 ε. Thus: β p β 0 This property is called consistency. 21 / 40
22 M- and Z-Estimators Consistency Theorem (Consistency of M-Estimators) M-estimators are consistent if sup E n [m(β,x)] E[m(β,X)] p 0 β sup E[m(β,X)] < E[m(β 0,X)]. β: β β 0 >ε The first condition holds in many case by some uniform law of large numbers. The second condition states that the maximum is well separated. 22 / 40
23 M- and Z-Estimators Consistency Figure: Proof of consistency δ δ E[m(β,X)] β 0 ε ε β 23 / 40
24 M- and Z-Estimators Consistency Sketch of proof: By assumption (2), for every ε there is a δ, such that if then β β 0 < ε. By assumption (1), sup E n [m(β,x)] E[m(β,X)] < δ β sup E n [m(β,x)] E[m(β,X)] < δ β happens with probability going to 1 as n. 24 / 40
25 M- and Z-Estimators Asymptotic normality Asymptotic normality What is the (approximate) distribution of M-estimators? Consistency just states that they converge to a point. But if we blow up the scale appropriately? For instance by n? Then we get convergence to a normal distribution! 25 / 40
26 M- and Z-Estimators Asymptotic normality Theorem Under suitable differentiability conditions, M-estimators and Z-estimators are asymptotically normal, n( β β0 ) d N(0,V) for some V. 26 / 40
27 M- and Z-Estimators Asymptotic normality Figure: Proof of asymptotic normality E[m'(β,X)] E n [m'(β,x)] β^ β 0 β E n [m'(β 0,X)] 27 / 40
28 M- and Z-Estimators Asymptotic normality Sketch of proof: follows by arguments similar to our derivation of the delta method. if m is twice differentiable, by the intermediate value theorem 0 = E n [m ( β,x)] = E n [m (β 0,X)] + E n [m ( β,x)] ( β β 0 ) for some β between β and β 0. Rearranging yields n( β β0 ) = ( E n [m ( β,x)]) 1 nen [m (β 0,X)]. Consistency of β and a uniform law of large numbers for m imply ( E n [m ( β,x)]) 1 p ( E[m (β 0,X)] ) / 40
29 M- and Z-Estimators Asymptotic normality The central limit theorem implies nen [m (β 0,X)] d N(0,Var(m (β 0,X))). Slutsky s theorem then yields the asymptotic distribution of β as n( β β0 ) d N(0,V) where V = ( E[m (β 0,X)] ) 1 Var(m (β 0,X)) (E[m (β 0,X)] ) 1. (4) 29 / 40
30 M- and Z-Estimators Asymptotic normality Estimators of the asymptotic variance Asymptotic variance: sandwich form Estimators for this variance: sample analogs of both components For instance: ( ) 1 ] ( ) 1 V = E n [m ( β,x)] En [(m ( β,x)) 2 E n [m ( β,x)] This is the kind of variance estimator you get when you type, robust after some estimation commands in Stata. 30 / 40
31 Special M-Estimators Least squares Least squares Recall OLS: First order condition: β = argmin β E n [(Y X β) 2 ] E n [e X] = 0 where e := Y X β In our general notation: m(y,x,β) = e 2 = (Y X β) 2 m (Y,X,β) = 2e X m (Y,X,β) = 2 XX t 31 / 40
32 Special M-Estimators Least squares Apply the asymptotic results for general M-estimators β is consistent for β0, the best linear predictor, β 0 = argmin β E[(Y X β) 2 ]. β is asymptotically normal n ( β β0 ) d N(0,V) 32 / 40
33 Special M-Estimators Least squares Asymptotic variance V = ( E[m (β 0,X)] ) 1 Var(m (β 0,X)) (E[m (β 0,X)] ) 1 = E[XX t ] 1 E[e 2 XX t ] E[XX t ] 1 heteroskedasticity robust variance estimator for ordinary least squares: 1 n E n[xx t ] 1 E n [ê 2 XX t ] E n [XX t ] 1 (5) Factor of 1/n to get variance of β rather than n β 33 / 40
34 Special M-Estimators Maximum likelihood Maximum likelihood Lemma Suppose Y f(y,β 0 ), where f denotes a family of densities indexed by β. Then E[log(f(Y,β 0 ))] E[log(f(Y,β))]. (6) The inequality is strict if f(y,β 0 ) f(y,β) with positive probability. 34 / 40
35 Special M-Estimators Maximum likelihood Sketch of proof: Want to show: 0 log(f(y,β))f(y,β 0 )dy ( ) f(y,β) = log f(y,β 0 )dy. f(y,β 0 ) log(f(y,β 0 ))f(y,β 0 )dy Jensen s inequality, applied to the concave function log: ( ) f(y,β) log f(y,β 0 )dy f(y,β 0 ) ( ) f(y,β) log f(y,β 0 ) f(y,β 0)dy =log(1) = / 40
36 Special M-Estimators Maximum likelihood Terminology for maximum likelihood Log likelihood: L n (β) = n E n [m(y,β)] = log(f(y i,β)) i Score: Information: S i (β) = m (Y i,β) = β log(f(y i,β)) I(β) = E[m (Y,β)] = E[ S/ β] 36 / 40
37 Special M-Estimators Maximum likelihood Lemma If Y i f(y,β 0 ), then Var(S(β 0 )) = I(β 0 ) = E[ S(β 0 )/ β]. Proof: Differentiate 0 = E[S] = S(y,β 0 )f(y,β 0 )dy with respect to β 0 to get 0 = S fdy + Sf dy = E[S ] + E[S 2 ]. But: Parametric models are usually wrong. So don t trust this equality. If it holds, the asymptotic variance for the MLE simplifies to V = E[S ] 1 E[S 2 ] E[S ] 1 = I(β 0 ) / 40
38 Confidence sets Confidence sets Confidence set C: a set of β s, which is calculated as a function of data Y Confidence set C for β of level α: for all distributions of Y (i.e., all θ ) and corresponding β 0. P(β 0 C) 1 α. (7) In this expression β 0 is fixed and C is random. Confidence set C n for β of asymptotic level α: lim P(β C n) 1 α. (8) n 38 / 40
39 Confidence sets Confidence sets for M-estimators can use asymptotic normality to get asymptotic confidence sets Suppose n( β β0 ) d N(0,V) V p V Define β := n V 1/2 ( β β 0 ). Slutsky s theorem and therefore where k = dim(β). β d N(0,I), β 2 d χ 2 k, 39 / 40
40 Confidence sets Let χ 2 k,1 α be the 1 α quantile of the χ 2 k distribution. Define { C n = β : } n V 1/2 ( β β) 2 χk,1 α 2. (9) We get P(β 0 C n ) 1 α. C n is a confidence set for β of asymptotic level α. C n is an ellipsoid. 40 / 40
Institute of Actuaries of India
Institute of Actuaries of India Subject CT3 Probability & Mathematical Statistics May 2011 Examinations INDICATIVE SOLUTION Introduction The indicative solution has been written by the Examiners with the
More informationComprehensive Examination Quantitative Methods Spring, 2018
Comprehensive Examination Quantitative Methods Spring, 2018 Instruction: This exam consists of three parts. You are required to answer all the questions in all the parts. 1 Grading policy: 1. Each part
More informationOrdinary Least Squares Regression
Ordinary Least Squares Regression Goals for this unit More on notation and terminology OLS scalar versus matrix derivation Some Preliminaries In this class we will be learning to analyze Cross Section
More informationthe error term could vary over the observations, in ways that are related
Heteroskedasticity We now consider the implications of relaxing the assumption that the conditional variance Var(u i x i ) = σ 2 is common to all observations i = 1,..., n In many applications, we may
More informationP n. This is called the law of large numbers but it comes in two forms: Strong and Weak.
Large Sample Theory Large Sample Theory is a name given to the search for approximations to the behaviour of statistical procedures which are derived by computing limits as the sample size, n, tends to
More informationLecture Notes 15 Prediction Chapters 13, 22, 20.4.
Lecture Notes 15 Prediction Chapters 13, 22, 20.4. 1 Introduction Prediction is covered in detail in 36-707, 36-701, 36-715, 10/36-702. Here, we will just give an introduction. We observe training data
More informationp y (1 p) 1 y, y = 0, 1 p Y (y p) = 0, otherwise.
1. Suppose Y 1, Y 2,..., Y n is an iid sample from a Bernoulli(p) population distribution, where 0 < p < 1 is unknown. The population pmf is p y (1 p) 1 y, y = 0, 1 p Y (y p) = (a) Prove that Y is the
More informationInference in non-linear time series
Intro LS MLE Other Erik Lindström Centre for Mathematical Sciences Lund University LU/LTH & DTU Intro LS MLE Other General Properties Popular estimatiors Overview Introduction General Properties Estimators
More informationEcon 2140, spring 2018, Part IIa Statistical Decision Theory
Econ 2140, spring 2018, Part IIa Maximilian Kasy Department of Economics, Harvard University 1 / 35 Examples of decision problems Decide whether or not the hypothesis of no racial discrimination in job
More informationRegression Models - Introduction
Regression Models - Introduction In regression models, two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent variable,
More informationAsymptotic Theory. L. Magee revised January 21, 2013
Asymptotic Theory L. Magee revised January 21, 2013 1 Convergence 1.1 Definitions Let a n to refer to a random variable that is a function of n random variables. Convergence in Probability The scalar a
More informationGraduate Econometrics I: Maximum Likelihood I
Graduate Econometrics I: Maximum Likelihood I Yves Dominicy Université libre de Bruxelles Solvay Brussels School of Economics and Management ECARES Yves Dominicy Graduate Econometrics I: Maximum Likelihood
More informationLECTURE 10: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING. The last equality is provided so this can look like a more familiar parametric test.
Economics 52 Econometrics Professor N.M. Kiefer LECTURE 1: NEYMAN-PEARSON LEMMA AND ASYMPTOTIC TESTING NEYMAN-PEARSON LEMMA: Lesson: Good tests are based on the likelihood ratio. The proof is easy in the
More informationQualifying Exam in Probability and Statistics. https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf
Part : Sample Problems for the Elementary Section of Qualifying Exam in Probability and Statistics https://www.soa.org/files/edu/edu-exam-p-sample-quest.pdf Part 2: Sample Problems for the Advanced Section
More informationEstimation of Dynamic Regression Models
University of Pavia 2007 Estimation of Dynamic Regression Models Eduardo Rossi University of Pavia Factorization of the density DGP: D t (x t χ t 1, d t ; Ψ) x t represent all the variables in the economy.
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2009 Prof. Gesine Reinert Our standard situation is that we have data x = x 1, x 2,..., x n, which we view as realisations of random
More informationEconomics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
More informationSTAT 135 Lab 5 Bootstrapping and Hypothesis Testing
STAT 135 Lab 5 Bootstrapping and Hypothesis Testing Rebecca Barter March 2, 2015 The Bootstrap Bootstrap Suppose that we are interested in estimating a parameter θ from some population with members x 1,...,
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationAnalogy Principle. Asymptotic Theory Part II. James J. Heckman University of Chicago. Econ 312 This draft, April 5, 2006
Analogy Principle Asymptotic Theory Part II James J. Heckman University of Chicago Econ 312 This draft, April 5, 2006 Consider four methods: 1. Maximum Likelihood Estimation (MLE) 2. (Nonlinear) Least
More informationHypothesis Testing. 1 Definitions of test statistics. CB: chapter 8; section 10.3
Hypothesis Testing CB: chapter 8; section 0.3 Hypothesis: statement about an unknown population parameter Examples: The average age of males in Sweden is 7. (statement about population mean) The lowest
More informationEcon 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE
Econ 583 Homework 7 Suggested Solutions: Wald, LM and LR based on GMM and MLE Eric Zivot Winter 013 1 Wald, LR and LM statistics based on generalized method of moments estimation Let 1 be an iid sample
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More informationCherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009 there were participants
18.650 Statistics for Applications Chapter 5: Parametric hypothesis testing 1/37 Cherry Blossom run (1) The credit union Cherry Blossom Run is a 10 mile race that takes place every year in D.C. In 2009
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More informationTesting Restrictions and Comparing Models
Econ. 513, Time Series Econometrics Fall 00 Chris Sims Testing Restrictions and Comparing Models 1. THE PROBLEM We consider here the problem of comparing two parametric models for the data X, defined by
More informationEcon 2148, spring 2019 Statistical decision theory
Econ 2148, spring 2019 Statistical decision theory Maximilian Kasy Department of Economics, Harvard University 1 / 53 Takeaways for this part of class 1. A general framework to think about what makes a
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationChapter 4: Asymptotic Properties of the MLE
Chapter 4: Asymptotic Properties of the MLE Daniel O. Scharfstein 09/19/13 1 / 1 Maximum Likelihood Maximum likelihood is the most powerful tool for estimation. In this part of the course, we will consider
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationSimple Linear Regression
Simple Linear Regression Reading: Hoff Chapter 9 November 4, 2009 Problem Data: Observe pairs (Y i,x i ),i = 1,... n Response or dependent variable Y Predictor or independent variable X GOALS: Exploring
More informationReview. December 4 th, Review
December 4 th, 2017 Att. Final exam: Course evaluation Friday, 12/14/2018, 10:30am 12:30pm Gore Hall 115 Overview Week 2 Week 4 Week 7 Week 10 Week 12 Chapter 6: Statistics and Sampling Distributions Chapter
More informationSTATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
More information11 Survival Analysis and Empirical Likelihood
11 Survival Analysis and Empirical Likelihood The first paper of empirical likelihood is actually about confidence intervals with the Kaplan-Meier estimator (Thomas and Grunkmeier 1979), i.e. deals with
More informationRecent Advances in the Field of Trade Theory and Policy Analysis Using Micro-Level Data
Recent Advances in the Field of Trade Theory and Policy Analysis Using Micro-Level Data July 2012 Bangkok, Thailand Cosimo Beverelli (World Trade Organization) 1 Content a) Classical regression model b)
More information(a) (3 points) Construct a 95% confidence interval for β 2 in Equation 1.
Problem 1 (21 points) An economist runs the regression y i = β 0 + x 1i β 1 + x 2i β 2 + x 3i β 3 + ε i (1) The results are summarized in the following table: Equation 1. Variable Coefficient Std. Error
More informationEcon 583 Final Exam Fall 2008
Econ 583 Final Exam Fall 2008 Eric Zivot December 11, 2008 Exam is due at 9:00 am in my office on Friday, December 12. 1 Maximum Likelihood Estimation and Asymptotic Theory Let X 1,...,X n be iid random
More informationCentral Bank of Chile October 29-31, 2013 Bruce Hansen (University of Wisconsin) Structural Breaks October 29-31, / 91. Bruce E.
Forecasting Lecture 3 Structural Breaks Central Bank of Chile October 29-31, 2013 Bruce Hansen (University of Wisconsin) Structural Breaks October 29-31, 2013 1 / 91 Bruce E. Hansen Organization Detection
More informationLinear Regression. Junhui Qian. October 27, 2014
Linear Regression Junhui Qian October 27, 2014 Outline The Model Estimation Ordinary Least Square Method of Moments Maximum Likelihood Estimation Properties of OLS Estimator Unbiasedness Consistency Efficiency
More informationMultivariate Analysis and Likelihood Inference
Multivariate Analysis and Likelihood Inference Outline 1 Joint Distribution of Random Variables 2 Principal Component Analysis (PCA) 3 Multivariate Normal Distribution 4 Likelihood Inference Joint density
More informationEconomics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationNotes on Asymptotic Theory: Convergence in Probability and Distribution Introduction to Econometric Theory Econ. 770
Notes on Asymptotic Theory: Convergence in Probability and Distribution Introduction to Econometric Theory Econ. 770 Jonathan B. Hill Dept. of Economics University of North Carolina - Chapel Hill November
More information1 Extremum Estimators
FINC 9311-21 Financial Econometrics Handout Jialin Yu 1 Extremum Estimators Let θ 0 be a vector of k 1 unknown arameters. Extremum estimators: estimators obtained by maximizing or minimizing some objective
More informationLecture notes on statistical decision theory Econ 2110, fall 2013
Lecture notes on statistical decision theory Econ 2110, fall 2013 Maximilian Kasy March 10, 2014 These lecture notes are roughly based on Robert, C. (2007). The Bayesian choice: from decision-theoretic
More informationEconomics 620, Lecture 18: Nonlinear Models
Economics 620, Lecture 18: Nonlinear Models Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 18: Nonlinear Models 1 / 18 The basic point is that smooth nonlinear
More informationUniformity and the delta method
Uniformity and the delta method Maximilian Kasy January, 208 Abstract When are asymptotic approximations using the delta-method uniformly valid? We provide sufficient conditions as well as closely related
More informationML Testing (Likelihood Ratio Testing) for non-gaussian models
ML Testing (Likelihood Ratio Testing) for non-gaussian models Surya Tokdar ML test in a slightly different form Model X f (x θ), θ Θ. Hypothesist H 0 : θ Θ 0 Good set: B c (x) = {θ : l x (θ) max θ Θ l
More informationDA Freedman Notes on the MLE Fall 2003
DA Freedman Notes on the MLE Fall 2003 The object here is to provide a sketch of the theory of the MLE. Rigorous presentations can be found in the references cited below. Calculus. Let f be a smooth, scalar
More informationMaximum Likelihood Asymptotic Theory. Eduardo Rossi University of Pavia
Maximum Likelihood Asymtotic Theory Eduardo Rossi University of Pavia Slutsky s Theorem, Cramer s Theorem Slutsky s Theorem Let {X N } be a random sequence converging in robability to a constant a, and
More informationInformation in a Two-Stage Adaptive Optimal Design
Information in a Two-Stage Adaptive Optimal Design Department of Statistics, University of Missouri Designed Experiments: Recent Advances in Methods and Applications DEMA 2011 Isaac Newton Institute for
More informationSection 8.2. Asymptotic normality
30 Section 8.2. Asymptotic normality We assume that X n =(X 1,...,X n ), where the X i s are i.i.d. with common density p(x; θ 0 ) P= {p(x; θ) :θ Θ}. We assume that θ 0 is identified in the sense that
More informationRegression Models - Introduction
Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent
More informationCAS MA575 Linear Models
CAS MA575 Linear Models Boston University, Fall 2013 Midterm Exam (Correction) Instructor: Cedric Ginestet Date: 22 Oct 2013. Maximal Score: 200pts. Please Note: You will only be graded on work and answers
More informationFinal Exam. Question 1 (20 points) 2 (25 points) 3 (30 points) 4 (25 points) 5 (10 points) 6 (40 points) Total (150 points) Bonus question (10)
Name Economics 170 Spring 2004 Honor pledge: I have neither given nor received aid on this exam including the preparation of my one page formula list and the preparation of the Stata assignment for the
More informationSTAT763: Applied Regression Analysis. Multiple linear regression. 4.4 Hypothesis testing
STAT763: Applied Regression Analysis Multiple linear regression 4.4 Hypothesis testing Chunsheng Ma E-mail: cma@math.wichita.edu 4.4.1 Significance of regression Null hypothesis (Test whether all β j =
More informationEconomics 582 Random Effects Estimation
Economics 582 Random Effects Estimation Eric Zivot May 29, 2013 Random Effects Model Hence, the model can be re-written as = x 0 β + + [x ] = 0 (no endogeneity) [ x ] = = + x 0 β + + [x ] = 0 [ x ] = 0
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationwhere x and ȳ are the sample means of x 1,, x n
y y Animal Studies of Side Effects Simple Linear Regression Basic Ideas In simple linear regression there is an approximately linear relation between two variables say y = pressure in the pancreas x =
More informationA Very Brief Summary of Statistical Inference, and Examples
A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)
More informationSTA 2101/442 Assignment 3 1
STA 2101/442 Assignment 3 1 These questions are practice for the midterm and final exam, and are not to be handed in. 1. Suppose X 1,..., X n are a random sample from a distribution with mean µ and variance
More informationWhy experimenters should not randomize, and what they should do instead
Why experimenters should not randomize, and what they should do instead Maximilian Kasy Department of Economics, Harvard University Maximilian Kasy (Harvard) Experimental design 1 / 42 project STAR Introduction
More informationsimple if it completely specifies the density of x
3. Hypothesis Testing Pure significance tests Data x = (x 1,..., x n ) from f(x, θ) Hypothesis H 0 : restricts f(x, θ) Are the data consistent with H 0? H 0 is called the null hypothesis simple if it completely
More informationAsymptotic Statistics-VI. Changliang Zou
Asymptotic Statistics-VI Changliang Zou Kolmogorov-Smirnov distance Example (Kolmogorov-Smirnov confidence intervals) We know given α (0, 1), there is a well-defined d = d α,n such that, for any continuous
More informationIV Estimation and its Limitations: Weak Instruments and Weakly Endogeneous Regressors
IV Estimation and its Limitations: Weak Instruments and Weakly Endogeneous Regressors Laura Mayoral, IAE, Barcelona GSE and University of Gothenburg U. of Gothenburg, May 2015 Roadmap Testing for deviations
More informationTopic 7: HETEROSKEDASTICITY
Universidad Carlos III de Madrid César Alonso ECONOMETRICS Topic 7: HETEROSKEDASTICITY Contents 1 Introduction 1 1.1 Examples............................. 1 2 The linear regression model with heteroskedasticity
More informationLeast Squares Estimation-Finite-Sample Properties
Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions
More informationLECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]
LECTURE 7 NOTES 1. Convergence of random variables. Before delving into the large samle roerties of the MLE, we review some concets from large samle theory. 1. Convergence in robability: x n x if, for
More informationMaster s Written Examination
Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth
More informationStatistical inference
Statistical inference Contents 1. Main definitions 2. Estimation 3. Testing L. Trapani MSc Induction - Statistical inference 1 1 Introduction: definition and preliminary theory In this chapter, we shall
More informationStat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS
Stat 135, Fall 2006 A. Adhikari HOMEWORK 6 SOLUTIONS 1a. Under the null hypothesis X has the binomial (100,.5) distribution with E(X) = 50 and SE(X) = 5. So P ( X 50 > 10) is (approximately) two tails
More informationProbability and Statistics Notes
Probability and Statistics Notes Chapter Seven Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Seven Notes Spring 2011 1 / 42 Outline
More informationStatistics Ph.D. Qualifying Exam: Part II November 9, 2002
Statistics Ph.D. Qualifying Exam: Part II November 9, 2002 Student Name: 1. Answer 8 out of 12 problems. Mark the problems you selected in the following table. 1 2 3 4 5 6 7 8 9 10 11 12 2. Write your
More informationWooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics
Wooldridge, Introductory Econometrics, 4th ed. Appendix C: Fundamentals of mathematical statistics A short review of the principles of mathematical statistics (or, what you should have learned in EC 151).
More informationLet us first identify some classes of hypotheses. simple versus simple. H 0 : θ = θ 0 versus H 1 : θ = θ 1. (1) one-sided
Let us first identify some classes of hypotheses. simple versus simple H 0 : θ = θ 0 versus H 1 : θ = θ 1. (1) one-sided H 0 : θ θ 0 versus H 1 : θ > θ 0. (2) two-sided; null on extremes H 0 : θ θ 1 or
More informationLECTURE 10. Introduction to Econometrics. Multicollinearity & Heteroskedasticity
LECTURE 10 Introduction to Econometrics Multicollinearity & Heteroskedasticity November 22, 2016 1 / 23 ON PREVIOUS LECTURES We discussed the specification of a regression equation Specification consists
More informationMISCELLANEOUS TOPICS RELATED TO LIKELIHOOD. Copyright c 2012 (Iowa State University) Statistics / 30
MISCELLANEOUS TOPICS RELATED TO LIKELIHOOD Copyright c 2012 (Iowa State University) Statistics 511 1 / 30 INFORMATION CRITERIA Akaike s Information criterion is given by AIC = 2l(ˆθ) + 2k, where l(ˆθ)
More informationEcon 2120: Section 2
Econ 2120: Section 2 Part I - Linear Predictor Loose Ends Ashesh Rambachan Fall 2018 Outline Big Picture Matrix Version of the Linear Predictor and Least Squares Fit Linear Predictor Least Squares Omitted
More informationLecture 8 Inequality Testing and Moment Inequality Models
Lecture 8 Inequality Testing and Moment Inequality Models Inequality Testing In the previous lecture, we discussed how to test the nonlinear hypothesis H 0 : h(θ 0 ) 0 when the sample information comes
More informationy ˆ i = ˆ " T u i ( i th fitted value or i th fit)
1 2 INFERENCE FOR MULTIPLE LINEAR REGRESSION Recall Terminology: p predictors x 1, x 2,, x p Some might be indicator variables for categorical variables) k-1 non-constant terms u 1, u 2,, u k-1 Each u
More informationStatistical Inference
Statistical Inference Classical and Bayesian Methods Revision Class for Midterm Exam AMS-UCSC Th Feb 9, 2012 Winter 2012. Session 1 (Revision Class) AMS-132/206 Th Feb 9, 2012 1 / 23 Topics Topics We will
More informationThe Simple Linear Regression Model
The Simple Linear Regression Model Lesson 3 Ryan Safner 1 1 Department of Economics Hood College ECON 480 - Econometrics Fall 2017 Ryan Safner (Hood College) ECON 480 - Lesson 3 Fall 2017 1 / 77 Bivariate
More informationLinear regression COMS 4771
Linear regression COMS 4771 1. Old Faithful and prediction functions Prediction problem: Old Faithful geyser (Yellowstone) Task: Predict time of next eruption. 1 / 40 Statistical model for time between
More informationPh.D. Preliminary Examination Statistics June 2, 2014
Ph.D. Preliminary Examination Statistics June, 04 NOTES:. The exam is worth 00 points.. Partial credit may be given for partial answers if possible.. There are 5 pages in this exam paper. I have neither
More informationEconometrics II - EXAM Answer each question in separate sheets in three hours
Econometrics II - EXAM Answer each question in separate sheets in three hours. Let u and u be jointly Gaussian and independent of z in all the equations. a Investigate the identification of the following
More informationEconometrics. Week 8. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague
Econometrics Week 8 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 25 Recommended Reading For the today Instrumental Variables Estimation and Two Stage
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationNonconcave Penalized Likelihood with A Diverging Number of Parameters
Nonconcave Penalized Likelihood with A Diverging Number of Parameters Jianqing Fan and Heng Peng Presenter: Jiale Xu March 12, 2010 Jianqing Fan and Heng Peng Presenter: JialeNonconcave Xu () Penalized
More informationLecture 6: Linear Regression
Lecture 6: Linear Regression Reading: Sections 3.1-3 STATS 202: Data mining and analysis Jonathan Taylor, 10/5 Slide credits: Sergio Bacallado 1 / 30 Simple linear regression Model: y i = β 0 + β 1 x i
More informationPh.D. Qualifying Exam Friday Saturday, January 3 4, 2014
Ph.D. Qualifying Exam Friday Saturday, January 3 4, 2014 Put your solution to each problem on a separate sheet of paper. Problem 1. (5166) Assume that two random samples {x i } and {y i } are independently
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationQuantitative Methods in Economics Conditional Expectations
Quantitative Methods in Economics Conditional Expectations Maximilian Kasy Harvard University, fall 2016 1 / 19 Roadmap, Part I 1. Linear predictors and least squares regression 2. Conditional expectations
More informationUses of Asymptotic Distributions: In order to get distribution theory, we need to norm the random variable; we usually look at n 1=2 ( X n ).
1 Economics 620, Lecture 8a: Asymptotics II Uses of Asymptotic Distributions: Suppose X n! 0 in probability. (What can be said about the distribution of X n?) In order to get distribution theory, we need
More informationChapter 4: Constrained estimators and tests in the multiple linear regression model (Part III)
Chapter 4: Constrained estimators and tests in the multiple linear regression model (Part III) Florian Pelgrin HEC September-December 2010 Florian Pelgrin (HEC) Constrained estimators September-December
More informationEconomics 520. Lecture Note 19: Hypothesis Testing via the Neyman-Pearson Lemma CB 8.1,
Economics 520 Lecture Note 9: Hypothesis Testing via the Neyman-Pearson Lemma CB 8., 8.3.-8.3.3 Uniformly Most Powerful Tests and the Neyman-Pearson Lemma Let s return to the hypothesis testing problem
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationMS&E 226: Small Data
MS&E 226: Small Data Lecture 15: Examples of hypothesis tests (v5) Ramesh Johari ramesh.johari@stanford.edu 1 / 32 The recipe 2 / 32 The hypothesis testing recipe In this lecture we repeatedly apply the
More information