What regression does. x is called the regressor The linear form is a natural first assumption, unless theory rejects it. positive or negative.

Similar documents
This is natural first assumption, unless theory rejects it.

What regression does. = β + β x : The conditional mean of y given x is β + β x 1 2

Inference in Simple Regression

Section 10 Regression with Stochastic Regressors

Regression with Stochastic Regressors

Reproducing kernel Hilbert spaces. Nuno Vasconcelos ECE Department, UCSD

Physics 107 HOMEWORK ASSIGNMENT #20

ENGI 4421 Probability & Statistics

Section 14 Limited Dependent Variables

Fall 2010 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede. (n.b. for now, we do not require that k. vectors as a k 1 matrix: ( )

element k Using FEM to Solve Truss Problems

4DVAR, according to the name, is a four-dimensional variational method.

Chapter 3, Solution 1C.

Problem 1. Refracting Surface (Modified from Pedrotti 2-2)

Chapter 2 - The Simple Linear Regression Model S =0. e i is a random error. S β2 β. This is a minimization problem. Solution is a calculus exercise.

Basics of heteroskedasticity

e i is a random error

Chapter 7. Systems 7.1 INTRODUCTION 7.2 MATHEMATICAL MODELING OF LIQUID LEVEL SYSTEMS. Steady State Flow. A. Bazoune

Economics 130. Lecture 4 Simple Linear Regression Continued

Econ107 Applied Econometrics Topic 3: Classical Model (Studenmund, Chapter 4)

Theory of a vertically loaded Suction Pile in SAND

1. Inference on Regression Parameters a. Finding Mean, s.d and covariance amongst estimates. 2. Confidence Intervals and Working Hotelling Bands

Chapter 6 : Gibbs Free Energy

The Simple Linear Regression Model: Theory

Internal vs. external validity. External validity. This section is based on Stock and Watson s Chapter 9.

Lecture 4 Hypothesis Testing

Lecture 12. Heat Exchangers. Heat Exchangers Chee 318 1

Section 3: Detailed Solutions of Word Problems Unit 1: Solving Word Problems by Modeling with Formulas

55:041 Electronic Circuits

A/2 l,k. Problem 1 STRATEGY. KNOWN Resistance of a complete spherical shell: r rk. Inner and outer radii

1. An incident ray from the object to the mirror, parallel to the principal axis and then reflected through the focal point F.

_J _J J J J J J J J _. 7 particles in the blue state; 3 particles in the red state: 720 configurations _J J J _J J J J J J J J _

Interval Estimation in the Classical Normal Linear Regression Model. 1. Introduction

β0 + β1xi. You are interested in estimating the unknown parameters β

Spring 2002 Lecture #17

Design of Analog Integrated Circuits

Lecture 3 Stat102, Spring 2007

Chapter 11: Simple Linear Regression and Correlation

A New Method for Solving Integer Linear. Programming Problems with Fuzzy Variables

The Ordinary Least Squares (OLS) Estimator

Lucas Imperfect Information Model

Limited Dependent Variables

Transient Conduction: Spatial Effects and the Role of Analytical Solutions

CHAPTER 3 ANALYSIS OF KY BOOST CONVERTER

Introduction to Electronic circuits.

Circuits Op-Amp. Interaction of Circuit Elements. Quick Check How does closing the switch affect V o and I o?

[The following data appear in Wooldridge Q2.3.] The table below contains the ACT score and college GPA for eight college students.

Wp/Lmin. Wn/Lmin 2.5V

Feedback Principle :-

CONVEX COMBINATIONS OF ANALYTIC FUNCTIONS

1 GSW Iterative Techniques for y = Ax

Shell Stiffness for Diffe ent Modes

The support vector machine. Nuno Vasconcelos ECE Department, UCSD

p(z) = 1 a e z/a 1(z 0) yi a i x (1/a) exp y i a i x a i=1 n i=1 (y i a i x) inf 1 (y Ax) inf Ax y (1 ν) y if A (1 ν) = 0 otherwise

Chapter 9: Statistical Inference and the Relationship between Two Variables

Exercises H /OOA> f Wo AJoTHS l^»-l S. m^ttrt /A/ ?C,0&L6M5 INFERENCE FOR DISTRIBUTIONS OF CATEGORICAL DATA. tts^e&n tai-ns 5 2%-cas-hews^, 27%

Lecture 13: Markov Chain Monte Carlo. Gibbs sampling

The Multiple Classical Linear Regression Model (CLRM): Specification and Assumptions. 1. Introduction

Conduction Heat Transfer

Problem Set 5 Solutions - McQuarrie Problems 3.20 MIT Dr. Anton Van Der Ven

PHYSICS 536 Experiment 12: Applications of the Golden Rules for Negative Feedback

Statistics for Economics & Business

since [1-( 0+ 1x1i+ 2x2 i)] [ 0+ 1x1i+ assumed to be a reasonable approximation

Statistics MINITAB - Lab 2

Exploiting vector space properties for the global optimization of process networks

Chapter 14 Simple Linear Regression

Statistics Chapter 4

STAT 3008 Applied Regression Analysis

Approach: (Equilibrium) TD analysis, i.e., conservation eqns., state equations Issues: how to deal with

Department of Quantitative Methods & Information Systems. Time Series and Their Components QMIS 320. Chapter 6

Lecture 6: Introduction to Linear Regression

Here is the rationale: If X and y have a strong positive relationship to one another, then ( x x) will tend to be positive when ( y y)

SIMULATION OF THREE PHASE THREE LEG TRANSFORMER BEHAVIOR UNDER DIFFERENT VOLTAGE SAG TYPES

Big Data Analytics! Special Topics for Computer Science CSE CSE Mar 31

Inference in the Multiple-Regression

Interference is when two (or more) sets of waves meet and combine to produce a new pattern.

Experiment 6: Constructing a Microscope

Basic Business Statistics, 10/e

ENGI 4430 Parametric Vector Functions Page 2-01

β0 + β1xi and want to estimate the unknown

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede

Conservation of Energy

The Support Vector Machine

AP Statistics Notes Unit Two: The Normal Distributions

Lecture 3 Specification

Modelling of Clock Behaviour. Don Percival. Applied Physics Laboratory University of Washington Seattle, Washington, USA

A Matrix Representation of Panel Data

Which Separator? Spring 1

β0 + β1xi. You are interested in estimating the unknown parameters β

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

A Note on Equivalences in Measuring Returns to Scale

NUMERICAL DIFFERENTIATION

THE ROYAL STATISTICAL SOCIETY 2006 EXAMINATIONS SOLUTIONS HIGHER CERTIFICATE

Lecture 6 More on Complete Randomized Block Design (RBD)

Chapter 2 Transformations and Expectations. , and define f

Kinetic Model Completeness

princeton univ. F 13 cos 521: Advanced Algorithm Design Lecture 3: Large deviations bounds and applications Lecturer: Sanjeev Arora

Fall 2012 Analysis of Experimental Measurements B. Eisenstein/rev. S. Errede

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

A Note on the Linear Programming Sensitivity. Analysis of Specification Constraints. in Blending Problems

Transcription:

Sectn Smple Regressn What regressn es Relatnshp between varables Often n ecnmcs we beleve that there s a (perhaps causal) relatnshp between tw varables Usually mre than tw, but that s eferre t anther ay We call ths the ecnmc mel Eample: Grae n Ecn 01 vs number f rm-mates takng Ecn 01 Functnal frm Is the relatnshp lnear? y1 s calle the regressr The lnear frm s a natural frst assumptn, unless thery rejects t s slpe, whch etermnes whether relatnshp between an y s pstve r negatve 1 s ntercept r cnstant term, whch etermnes where the lnear relatnshp ntersects the y as Is t plausble that ths s an eact, etermnstc relatnshp? Data (almst) never ft eactly alng lne Why? Measurement errr (ncrrect efntn r msmeasurement) Other varables that affect y Relatnshp s nt purely lnear Relatnshp may be fferent fr fferent bservatns S the ecnmc mel must be mele as etermnng the epecte value f y E y 1 : The cntnal mean f y gven s 1 te that ths says nthng abut ther aspects f the strbutn (ther than the epecte value) Hw es a change n affect the varance f y? (We assume that t es nt) Hw es a change n affect the mean, r the 75 th percentle, r any ther aspect f the strbutn f y? (If y s assume t be nrmal, then everythng abut the strbutn epens nly n mean an varance) ~ 15 ~

Other regressn technques (n partcular, quantle regressn) allw us t eamne the mpact f n aspects f the strbutn f y ther than the mean Ang an errr term fr a stchastc relatnshp gves us the actual value f y: y1 e Errr term e captures all f the abve prblems Errr term s cnsere t be a ranm varable an s nt bserve rectly Varance f e s, whch s the cntnal varance f y gven, the varance f the cntnal strbutn f y gven The smplest, but ften nt val, assumptn s that the cntnal varance s the same fr all bservatns n ur sample (hmskeastcty) E y, whch means that the epecte value f y ncreases by unts when ncreases by ne unt Des t matter whch varable s n the left-han se? At ne level, n: 1 y 1 e, s 1 1 1 1 y v, where 1,, v e Fr purpses f mst estmatrs, yes: We shall see that a crtcally mprtant assumptn s that the errr term s nepenent f the regressrs r egenus varables Are the errrs shcks t y fr gven r shcks t fr gven y? It mght nt seem lke there s much fference, but the assumptn s crucal t val estmatn Egenety: s egenus wth respect t y f shcks t y nt affect, e, y es nt cause Where the ata cme frm? Sample an ppulatn We bserve a sample f bservatns n y an Depenng n cntet these samples may be Drawn frm a larger ppulatn, such as census ata r surveys Generate by a specfc ata-generatng prcess (DGP) as n tmeseres bservatns We usually wul lke t assume that the bservatns n ur sample are cv y, y 0, j statstcally nepenent, r at least uncrrelate: j ~ 16 ~

We wll assume ntally (fr a few weeks) that the values f are chsen as n an eperment: they are nt ranm We wll a ranm regressrs sn an scver that they n t change thngs much as lng as s nepenent f e Gals f regressn True regressn lne: actual relatnshp n ppulatn r DGP True an f (e ) Sample f bservatns cmes frm rawng ranm realzatns f e frm f (e ) an plttng pnts apprprately abve an belw the true regressn lne We want t fn an estmate regressn lne that cmes as clse t the true regressn lne as pssble, base n the bserve sample f y an pars: Estmate values f parameters 1 an Estmate prpertes f prbablty strbutn f errr term e Make nferences abut the abve estmates Use the estmates t make cntnal frecasts f y Determne the statstcal relablty f these frecasts Summarzng assumptns f smple regressn mel Assumptn #0: (Implct an unstate) The mel as specfe apples t all unts n the ppulatn an therefre all unts n the sample All unts n the ppulatn uner cnseratn have the same frm f the relatnshp, the same ceffcents, an errr terms wth the same prpertes If the Unte States an Mal are n the ppulatn, they really have the same parameters? Ths assumptn unerles everythng we n ecnmetrcs, an thus t must always be cnsere very carefully n chsng a specfcatn an a sample, an n ecng fr what ppulatn the results carry mplcatns SR1: y1 e SR: Ee 0, s Ey1 te that f s ranm, we make these cntnal epectatns Ee 0 E y 1 var e var y SR3: If s ranm, ths becmes var e var y We shul (an wll) cnser the mre general case n whch varance vares acrss bservatns: heterskeastcty ~ 17 ~

SR4: e ej y yj cv, cv, 0 Ths, t, can be relae: autcrrelatn SR5: s nn-ranm an takes n at least tw values We wll allw ranm later an see that Ee 0 mples that e must be uncrrelate wth e ~ 0, SR6: (ptnal) Ths s cnvenent, but nt crtcal snce the law f large numbers assures that fr a we varety f strbutns f e, ur estmatrs cnverge t nrmal as the sample gets large Eample: Assess the valty f these assumptns fr 01 rm-mate mel Strateges fr btanng regressn estmatrs What s an estmatr? A rule (frmula) fr calculatng an estmate f a parameter ( 1,, r ) base n the sample values y, Estmatrs are ften ente by ^ ver the varable beng estmate: An estmatr f mght be ente ˆ Hw mght we estmate the ceffcents f the smple regressn mel? Three strateges: Meth f least-squares Meth f mamum lkelh Meth f mments All three strateges wth the SR assumptns lea t the same estmatr rule: the rnary least-squares regressn estmatr: (b 1, b, s ) Meth f least squares Estmatn strategy: Make sum f square y-evatns ( resuals ) f bserve values frm the estmate regressn lne as small as pssble Gven ceffcent estmates b 1, b, resuals are efne as e y b1 b Or eˆ y yˆ, wth y b1 b ˆ ˆ Why nt mnmze the sum f the resuals? We n t want sum f resuals t be large negatve number: Mnmze sum f resuals by havng all resuals nfntely negatve Many alternatve lnes that make sum f resuals zer (whch s esrable) because pstves an negatves cancel ut Why use square rather than abslute value t eal wth cancellatn f pstves an negatves? ~ 18 ~

Square functn s cntnuusly fferentable; abslute value functn s nt Least-squares estmatn s much easer than least-absluteevatn estmatn Prmnence f Gaussan (nrmal) strbutn n nature an statstcal thery fcuses us n varance, whch s epectatn f square Least-abslute-evatn estmatn s ccasnally ne (specal case f quantle regressn), but nt cmmn Least-abslute-evatn regressn gves less mprtance t large utlers than least-squares because squarng gves large emphass t resuals wth large abslute value Tens t raw the regressn lne twar these pnts t elmnate large square resuals Least-squares crtern functn: ˆ ~ 19 ~ S e y b b 1 1 1 Least-squares estmatrs s the slutn t mns Snce S s a cntnuusly fferentable functn f the estmate parameters, we can fferentate an set the partal ervatves equal t zer t get the leastsquares nrmal equatns: S y b1 b0, b 1 y b b 0 1 1 1 1 S y b1 b0 b 1 1 y b b 0 1 1 1 y b1 b 0 b1 y b te that the b 1 cntn assures that the regressn lne passes thrugh the pnt, y b1, b Substtutng the secn cntn nt the frst ve by : y y b b y y b 0 0 b y y y y ˆ XY ˆ X The b estmatr s the sample cvarance f an y ve by the sample varance f What happens f s cnstant acrss all bservatns n ur sample?

Denmnatr s zer an we can t calculate b Ths s ur frst encunter wth the prblem f cllnearty: f s a cnstant then s a lnear cmbnatn f the ther regressr the cnstant ne that s multple by b 1 Cllnearty (r multcllnearty) wll be mre f a prblem n multple regressn If t s etreme (r perfect), t means that we can t calculate the slpe estmates The abve equatns are the rnary least-squares (OLS) ceffcent estmatrs Meth f mamum lkelh Cnser the jnt prbablty ensty functn f y an, f (y, 1, ) The functn s wrtten s cntnal n the ceffcents t make eplct that the jnt strbutn f y an are affecte by the parameters Ths functn measures the prbablty ensty f any partcular cmbnatn f y an values, whch can be lsely thught f as hw prbable that utcme s, gven the parameter values Fr a gven set f parameters, sme bservatns f y an are less lkely than thers Fr eample, f 1 = 0 an < 0, then t s less lkely that we wul see bservatns where y > 0 when > 0, than bservatns wth y < 0 The ea f mamum-lkelh estmatn s t chse a set f parameters that makes the lkelh f bservng the sample that we actually have as hgh as pssble The lkelh functn s just the jnt ensty functn turne n ts hea: L,, y f, y, 1 1 If the bservatns are nepenent ranm raws frm entcal prbablty strbutns (they are IID), then the verall sample ensty (lkelh) functn s the pruct f the ensty (lkelh) functn f the nvual bservatns: f 1, y1,, y,, n, yn 1, f, y 1, 1 n L,, y,, y,,, y L,, y 1 1 1 n n 1 1 If the cntnal prbablty strbutn f e cntnal n s Gaussan (nrmal) wth mean zer an varance :,,,, 1 f y 1 L 1 y e y 1 1 Because f the epnental functn, Gaussan lkelh functns are usually manpulate n lgs ~ 0 ~

te that because the lg functn s mntnc, mamzng the lg-lkelh functn s equvalent t mamzng the lkelh functn tself 1 1 Fr an nvual bservatn: ln L ln y Aggregatng ver the sample: 1 ln L,, y ln L,, y 1 1 1 1 1 ln y 1 1 1 ln y 1 1 1 The nly part f ths epressn that epens n r n the sample s the fnal summatn Because f the negatve sgn, mamzng the lkelh functn (wth respect t ) s equvalent t mnmzng the summatn But ths summatn s just the sum f square resuals that we mnmze n OLS Thus, OLS s MLE f the strbutn f e cntnal n s Gaussan wth mean zer an cnstant varance, an f the bservatns are IID Meth f mments Anther general strategy fr btanng estmatrs s t set estmates f selecte ppulatn mments equal t ther sample cunterparts Ths s calle the meth f mments In rer t emply the meth f mments, we have t make sme specfc assumptns abut the ppulatn/dgp mments Assume Ee 0, Ths means that the ppulatn/dgp mean f the errr term s zer Crrespnng t ths assumptn abut the ppulatn mean f e s the sample mean cntn 1 eˆ 0 Thus we set the sample mean t the value we have assume fr the ppulatn mean Assume cv e, 0, whch s equvalent t E E( ) e 0 Crrespnng t ths assumptn abut the ppulatn cvarance between the regressr an the errr term s the sample 1 cvarance cntn: ˆ e 0 Agan, we set the sample mment t the zer value that we have assume fr the ppulatn mment ~ 1 ~

Pluggng the epressn fr the resual nt the sample mment epressns abve: 1 y b1 b0, b1 y b Ths s the same as the ntercept estmate equatn fr the least-squares estmatr abve 1 y b1 b0, y yb b 0, y yb 0, y y b Ths s eactly the same equatn as fr the OLS estmatr Thus, f we assume that Ee 0, an cv e, 0 n the ppulatn, then the OLS estmatr can be erve by the meth f mments as well (te that bth f these mment cntns fllw frm the etene assumptn SR that E(e ) = 0) Evaluatng alternatve estmatrs (nt mprtant fr cmparsn here snce all three are same, but are they any g?) Desrable crtera Unbaseness: estmatr s n average equal t the true value E ˆ Small varance: estmatr s usually clse t ts epecte value var ˆE ˆ Eˆ Small RMSE can balance varance wth bas: RMSE MSE MSE E ˆ We wll talk abut BLUE estmatrs as mnmum varance wthn the class f unbase estmatrs Samplng strbutn f OLS estmatrs b 1 an b are ranm varables: they are functns f the ranm varables y an e We can thnk f the prbablty strbutn f b as ccurrng ver repeate ranm samples frm the unerlyng ppulatn r DGP ~ ~

In many (mst) cases, we cannt erve the strbutn f an estmatr theretcally, but must rely n Mnte Carl smulatn t estmate t (See belw) Because OLS estmatr (uner ur assumptns) s lnear, we can erve ts strbutn We can wrte the OLS slpe estmatr as b 1 1 1 1 y y 1 e y 1 1 e 1 e 1 1 1 1 1 e The thr step uses the prperty y 1, snce the epecte value f e s zer Fr nw, we are assumng that s nn-ranm, as n a cntrlle eperment If s fe, then the nly part f the frmula abve that s ranm s e The frmula shws that the slpe estmate s lnear n e Ths means that f e s Gaussan, then the slpe estmate wll als be Gaussan Even f e s nt Gaussan, the slpe estmate wll cnverge t a Gaussan strbutn as lng as sme mest assumptns abut ts strbutn are satsfe Because all the varables are nn-ranm, they can cme utse when we take epectatns, s e Ee E b 1 1 E 1 1 What abut the varance f b? ~ 3 ~

We wll the etals f the analytcal wrk n matr frm because t s easer var b E b E 1 Ee 1 1 HGL equatns 14 an 16 prve frmulas fr varance f b 1 an the cvarance between the ceffcents: var b 1 b b 1 1 cv 1, 0 1 te that the cvarance between the slpe an ntercept estmatrs s negatve f 0 : verestmatng ne wll ten t cause us t unerestmate the ther What etermnes the varance f b? Smaller varance f errr mre precse estmatrs Larger number f bservatns mre precse estmatrs Mre spersn f bservatns arun mean mre precse estmatrs What we knw abut the verall prbablty strbutn f b? If assumptn SR6 s satsfe an e s nrmal, then b s als nrmal because t s a lnear functn f the e varables an lnear functns f nrmally strbute varables are als nrmally strbute If assumptn SR6 s nt satsfe, then b cnverges t a nrmal strbutn as prve sme weak cntns n the strbutn f e are satsfe These epressns are the true varance/cvarance f the estmate ceffcent vectr Hwever, because we nt knw, t s nt f practcal use t us We ~ 4 ~

nee an estmatr fr n rer t calculate a stanar errr f the ceffcents: an estmate f ther stanar evatn 1 The requre estmate n the classcal case s s eˆ We ve by because ths s the number f egrees f freem n ur regressn Degrees f freem are a very mprtant ssue n ecnmetrcs It refers t hw many ata pnts are avalable n ecess f the mnmum number requre t estmate the mel In ths case, t takes mnmally tw pnts t efne a lne, s the smallest pssble number f bservatns fr whch we can ft a bvarate regressn s Any bservatns beyn make t (generally) mpssble t ft a lne perfectly thrugh all bservatns Thus, s the number f egrees f freem n the sample We always ve sums f square resuals by the number f egrees f freem n rer t get unbase varance estmates Fr eample, n calculatng the sample varance, we use 1 s z z because there are 1 egrees f 1 1 freem left after usng ne t calculate the mean Here, we have tw ceffcents t estmate, nt just ne, s we ve by The stanar errr f each ceffcent s the square rt f the crrespnng agnal element f that estmate cvarance matr te that the HGL tet uses an alternatve frmula base n 1 ˆ eˆ 1 Hw g s the OLS estmatr? Ths estmatr fr s base because there are nly egrees f freem n the resuals are use up n estmatng the parameters In large samples they are equvalent 1 Is OLS the best estmatr? Uner what cntns? Uner classcal regressn assumptns SR1 SR5 (but nt necessarly SR6) the Gauss- Markv Therem shws that the OLS estmatr s BLUE Any ther estmatr that s unbase an lnear n e has hgher varance than b te that (5, 0) s an estmatr wth zer varance, but t s base n the general case ~ 5 ~

Vlatn f any f the SR1 SR5 assumptns usually means that there s a better estmatr Intructn t Stata Stata wrks n a ataset (ta fle) Stata cmmans: Enter at prmpt Chse frm menu/wnws Enter nt a fle fr batch eecutn The Stata screen Lg fles Results wnw Cmman wnw Varables wnw Revew wnw Prpertes wnw Set ne up s stuents can see t later Openng a ata set Shw ata etr/brwser Cmmans t statstcal analyss summarze reg Graphcs cmmans Use menus t get see all ptns wthut rememberng hw t type Sample analyss: Ree Ecn 01 graes Depenent varable gpnts Shw summary statstcs Pnt ut screte strbutn: Is ths a prblem? Regressn n sngle varable: hsgpa Interpretng ceffcents (nte that ntercept s autmatcally nclue: nnt ptn) Pnt ut stanar errr, t statstc, p value, cnfent lmts te mssng bservatns Shw utreg usng graeregs, se Alternatve: regress n rr Shw hw utreg as clumns utreg usng graeregs, se merge Calculate precte values wth prect prect gpahat Graph actual an precte vs rr ~ 6 ~

Dsplay hypthetcal precte values wth margns margns, at(rr=(5 4 3 )) Transfrmatn: satc100 = satv100 + satm100 Regress n satc100 Cmpare t hsgpa regressn Regressn n ummy varable Regress n female Interpretatn f ceffcents Categry mean prectns: margns female Multple regressn emnstratn Reg gpnts rr satv100 satm100 female Shw utreg wth multple varables utreg usng graeregs, se merge A takng t regressn an nterpret Use margns t slate prectns f hypthetcal nvual varables wth thers at means margns, at(rr = (5 4 3 )) atmeans margnsplt Mnte Carl meths Base n HGL Appen G Hw we evaluate an estmatr such as OLS? Uner smple assumptns, we can smetmes calculate the estmatr s theretcal prbablty strbutn We can ften calculate the theretcal strbutn t whch the estmatr cnverges n large samples even when we cannt calculate the small-sample strbutn In general (an, n partcular, when we cannt calculate the true strbutn), we can smulate the mel ver thusans f samples t estmate ts strbutn The estmatn f the prbablty strbutn f an estmatr thrugh smulatn s calle Mnte Carl smulatn an s an ncreasngly mprtant tl n ecnmetrcs Cnser smple Mnte Carl eample: (MC Class Demta) Let s suppse that we are wrkng wth a gven, fe = 157 We have fe, gven values f the varable fr all 157 bservatns Usng HGL s e9-13ta wth avertsng varable as We assume that the true ppulatn values f 1 an are 10 an 3 Clse t estmate values fr regressn f sales n avertsng The true errr term s IID nrmal wth varance 009 (stanar evatn 03) ~ 7 ~

T use Mnte Carl t smulate the strbutn f the OLS estmatrs, we generate M replcatns f the samplng eperment: M sets f 157 IID (0, 009) smulate bservatns n e usng ranm number generatr (We wul generate sample values fr f t were nt beng taken as fe) Calculate the M sets f 157 values f y fr each bservatn as 1 + + e wth knwn values f the parameters an an smulate values f e Run M regressns fr the M smulate samples, keepng the estmate values f nterest (presumably ˆ 1 an ˆ, but pssbly als ther values) Lk at strbutn f the estmatrs ver M replcatns t apprmate the actual strbutn Mean Varance/stanar evatn/stanar errr Quantles fr use n nference Demnstrate usng Stata Setup ata Create fle s alreay n MC Class Demta prgram lstest g e=rnrmal(0, 03) g y=10 + 3*+e reg y rp e y en La t nt memry: run lstest Run smulatn wth 5000 replcatns smulate b=_b[], reps(5000): lstest Shw summary stats, hstgram, centles (5, 975) Least-squares regressn mel n matr ntatn (Frm Grffths, Hll, an Juge, Sectn 54) We can wrte the th bservatn f the bvarate lnear regressn mel as y e 1 Arrangng the bservatns vertcally gves us such equatns: y e, 1 1 1 1 y e 1 y e 1, ~ 8 ~

Ths s a system f lnear equatns that can be cnvenently rewrtten n matr frm There s n real nee fr the matr representatn wth nly ne regressr because the equatns are smple, but when we a regressrs the matr ntatn s mre useful Let y be an 1 clumn vectr: y1 y y y Let X be an matr: 1 1 1 X 1 s a 1 clumn vectr f ceffcents: 1 An e s an 1 vectr f the errr terms: e1 e e e Then y Xβ e epresses the system f equatns very cmpactly (Wrte ut matrces an shw hw multplcatn wrks fr sngle bservatn) In matr ntatn, eˆ yxb s the vectr f resuals Summng squares f the elements f a clumn vectr n matr ntatn s just the nner pruct: eˆ ˆˆ, 1 ee where prme entes matr transpse Thus we want t mnmze ths epressn fr least squares ee ˆˆ yxb yxb ybx yxb yy bxy bxxb Dfferentatng wth respect t the ceffcent vectr an settng t zer yels Xy XXb 0, r XXb Xy Pre-multplyng by the nverse f XX yels the OLS ceffcent frmula: 1 b XX Xy (Ths s ne f the few frmulas that yu nee t memrze) ~ 9 ~

te symmetry between matr frmula an scalar frmula Xy s the sum f the crss pruct f the tw varables an XX s the sum f squares f the regressr The frmer s n the numeratr (an nt nverte) an the latter s n the enmnatr (an nverte) In matr ntatn, we can epress ur estmatr n terms f e as 1 1 XX XXβ e β 1 β XX Xe b XX Xy 1 1 XX XX XX Xe When s nn-stchastc, the cvarance matr f the ceffcent estmatr s als easy t cmpute uner the OLS assumptns Cvarance matrces: The cvarance f a vectr ranm varable s a matr wth varances n the agnal an cvarances n the ffagnals Fr an M 1 vectr ranm varable z, the cvarance matr s t the fllwng uter pruct: cv( z) E zez zez E z1 Ez E z1 Ez z Ez E z1 Ez z M Ez E z1 Ez z Ez E z Ez E z Ez z M Ez E z1 Ez z M Ez E z Ez z M Ez E z M Ez In ur regressn mel, f e s IID wth mean zer an varance, then Ee = 0 an cv e E ee I, wth I beng the rer- entty matr We can then cmpute the cvarance matr f the (unbase) estmatr as cvb E bβbβ 1 1 E XX Xe XX Xe 1 1 E XX XeeX XX 1 1 XX XE eexxx XX XX XX XX 1 1 1 What happens t var b as gets large? Summatns n XX have atnal terms, s they get larger Ths means that nverse ~ 30 ~

matr gets smaller an varance ecreases: mre bservatns mples mre accurate estmatrs te that varance als ncreases as the varance f the errr term ges up Mre mprecse ft mples less precse ceffcent estmates Our estmate cvarance matr f the ceffcents s then s 1 XX The (, ) element f ths matr s s ˆ 1 1 e 1 1 1 Ths s the frmula we calculate n class fr the scalar system Thus, t summarze, when the classcal assumptns hl an e s nrmally strbute, b~, 1 β XX Asympttc prpertes f OLS bvarate regressn estmatr (Base n S&W, Chapter 17 t cvere n class Sprng 014) Cnvergence n prbablty (prbablty lmts) Assume that S 1, S,, S, s a sequence f ranm varables In practce, they are gng t be estmatrs base n 1,,, bservatns p S f an nly f lm Pr S 0 fr any > 0 Thus, fr any small value f, we can make the prbablty that S s further frm than arbtrarly small by chsng large enugh p If S, then we can wrte plm S = Ths means that the entre prbablty strbutn f S cnverges n the value as gets large Estmatrs that cnverge n prbablty t the true parameter value are calle cnsstent estmatrs Cnvergence n strbutn If the sequence f ranm varables {S } has cumulatve prbablty strbutns F 1, F,, F,, then S lm F t F t, fr all t at whch F s cntnuus S f an nly f If a sequence f ranm varables cnverges n strbutn t the nrmal strbutn, t s calle asympttcally nrmal ~ 31 ~

Prpertes f prbablty lmts an cnvergence n strbutn Prbablty lmts are very frgvng: Slutsky s Therem states that plm (S + R ) = plm S + plm R plm (S R ) = plm S plm R plm (S / R ) = plm S / plm R The cntnuus-mappng therem gves us Fr cntnuus functns g, plm g(s ) = g(plm S ) An f S g S g S S, then Further, we can cmbne prbablty lmts an cnvergence n strbutn t get If plm a = a an S, then a S S as a S a S S / a S/ a These are very useful snce t means that asympttcally we can treat any Central lmt therems cnsstent estmatr as a cnstant equal t the true value There s a varety wth slghtly fferent cntns Basc result: If {S } s a sequence f estmatrs f, then fr a we varety f unerlyng strbutns, S the unerlyng statstc Applyng asympttc thery t the OLS mel 0,, where s the varance f Uner the mre general cntns than the nes that we have typcally assume (nclung, specfcally, the fnte kurtss assumptn, but nt the hmskeastcty assumptn r the assumptn f fe regressrs), the OLS estmatr satsfes the cntns fr cnsstency an asympttc nrmalty var E( ) e 0, Ths s general case wth var heterskeastcty Wth hmskeastcty, the varable reuces t the usual frmula: b 0, var b plm ˆ, as prven n Sectn 173 b b t b se b Chce fr t statstc: 0, 1 ~ 3 ~

If hmskeastc, nrmal errr term, then eact strbutn s t If heterskeastc r nn-nrmal errr (wth fnte 4 th mment), then eact strbutn s unknwn, but asympttc strbutn s nrmal Whch s mre reasnable fr any gven applcatn? Lnearty an nnlnearty The OLS estmatr s a lnear estmatr because b s lnear n e (whch s because y s lnear n ), nt because y s lnear n OLS can easly hanle nnlnear relatnshps between y an lny = 1 + y = 1 + etc Dummy (ncatr) varables take the value zer r ne Eample: MALE = 1 f male an 0 f female y MALE e 1 Fr females, E y MALE 1 Fr males, E y MALE 1 Thus, s the fference between the epecte value f males an females ~ 33 ~