Maximum Likelihood Estimation

Transcription

1 Chapter 9 Maximum Likelihood Estimatio 9.1 The Likelihood Fuctio The maximum likelihood estimator is the most widely used estimatio method. This chapter discusses the most importat cocepts behid maximum likelihood estimatio alog with some examples. Let the probability desity fuctio (pdf) of a radom variable, y, coditioal o a set of parameters, θ, be deoted by f(y θ). The fuctio idetifies the datageeratig process that uderlies a observed sample ad provides a mathematical descriptio of the data that the process will produce. The joit desity of idepedet ad idetically distributed (i.i.d.) observatios from this process is the product of idividual desities f(y 1,...,y )= f(y i θ)=l(θ y) (9.1) This joit desity is the likelihood fuctio ad it is defied as a fuctio of the ukow parameter vector, θ, ad the collectio of sample data, y. Writig the parameters as a fuctio of the data, the log of the likelihood fuctio ca be writte as: ll(θ y)= l f(y i θ). (9.2) We ca geeralize the cocept ad allow the likelihood fuctio to deped o other coditioig variables, x. Suppose that i the classical liear regressio model the disturbace follows a ormal distributio. The, coditioal o x i, y i is ormally distributed with mea µ i = x i β ad variace σ 2. The log likelihood is: ll(θ y,x) = l f(y i x i,θ) (9.3) 77

2 78 9 Maximum Likelihood Estimatio = 1 2 [lσ 2 + l(2π)+ (y i x i β)2 ] σ 2, where X is the K matrix of data with the ith row equal to x i. We say that the parameter vector θ is idetified (estimable) if for ay other parameter vector, θ θ, for some data y, L(θ y) L(θ y). 9.2 Properties of Maximum Likelihood Estimators Maximum likelihood estimators (MLEs) are most attractive because of their largesample or asymptotic properties. Give certai regularity coditios MLE is asymptotically efficiet. That is, it is cosistet, asymptotically ormally distributed, ad has a asymptotic covariace matrix that is ot larger tha the asymptotic covariace matrix of ay other cosistet, asymptotically ormally distributed estimator. 9.3 Maximum Likelihood Estimators: Two examples MLE i Stata: A liear regressio model Suppose we wat to estimate the followig model The Stata MLE program is foreig i = β 0 + β 1 mpg i + β 2 weight i + ε i (9.4) capture program drop myols program myols versio 11 args lf xb lsigma local y "$ML_y1" quietly replace lf = l(ormalde( y, xb, exp( lsigma ))) ed To ru this program we eed to load the data ad the type use ml model lf myols (xb: foreig = mpg weight) (lsigma:) ml maximize This yields the followig regressio output iitial: log likelihood = Iteratio 5: log likelihood = Number of obs = 74 Wald chi2(2) = Log likelihood = Prob > chi2 = foreig Coef. Std. Err. z P> z [95% Cof. Iterval]

3 9.3 Maximum Likelihood Estimators: Two examples 79 xb mpg weight _cos lsigma _cos We ca check these results with the OLS that you will obtai with the commad reg foreig mpg weight. reg foreig mpg weight This yields the followig regressio output Source SS df MS Number of obs = F( 2, 71) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = foreig Coef. Std. Err. t P> t [95% Cof. Iterval] mpg weight _cos Biary choice models: Probit A probit is a model to explai biary choice variables. The idea is that whe the depedet variable is either Y = 1 or Y = 0 we should use biary choice models. For example, if a perso chooses to buy a foreig car (foreig = 1), the probability of choosig this type of car ca be modeled as a fuctio ofmpg adweight. I geeral, Prob(Y = 1 x) = F(x,β) (9.5) Prob(Y = 0 x) = 1 F(x,β) The set of parameters β reflects the impact of the chages i x o the probability. The problem at this poit is to devise a suitable model for the right-had side of the equatio. The simplest approach is to retai the familiar liear regressio, F(x,β)=x β. (9.6) Because E[y x]=f(x,β), we ca costruct the regressio model. y=e[y x]+(y E[y x])=x β + ε. (9.7)

4 80 9 Maximum Likelihood Estimatio This liear probability model has a umber of shortcomigs. The first complicatio is that ε is heteroscedastic. Because x β + ε must be either 0 of 1, ε equals either xβ or 1 xβ with probabilities 1 F( ) ad F( ), respectively. 1 The secod complicatio is that we caot costrai x β to be i the 0 1 iterval. This meas that the model ca geerate probabilities outside[0, 1] ad potetially egative variaces. Fig. 9.1 Model for a Probability, from [Greee (2008)]. As Figure 9.1 suggests, for a give regressor vector, x β, we would like lim Prob(Y = 1 x)=1 (9.8) x β + lim Prob(Y = 1 x)=0 x β A cotiuous probability distributio defied over the real lie should work. Whe the ormal distributio is used, this gives rise to the probit model, x β Prob(Y = 1 x)= φ(t)dt = Φ(x β), (9.9) where φ( ) ad Φ( ) are the pdf ad the cdf of the ormal distributio, respectively. 1 It ca be show that the variace is: Var[ε x]=x β(1 x β).

5 9.3 Maximum Likelihood Estimators: Two examples Probit via step by step MLE i Stata I a probit model the log likelihood fuctio for Equatio 9.5 is give by { lφ(θ1 j ) if y j = 1 ll i = l(1 Φ(θ 1 j )) if y j = 0 where θ 1 j = x j b 1. The probit program is capture program drop myprobit program myprobit versio 11 args lf xb local y "$ML_y1" quietly replace lf = l( ormal( xb )) if y ==1 quietly replace lf = l(1-ormal( xb )) if y ==0 ed To ru this program use ml model lf myprobit (foreig = mpg weight) ml maximize The probit regressio output is iitial: log likelihood = Iteratio 5: log likelihood = Number of obs = 74 Wald chi2(2) = Log likelihood = Prob > chi2 = foreig Coef. Std. Err. z P> z [95% Cof. Iterval] mpg weight _cos You ca check these results with Stata built-i procedure for probit estimatio usig the Stata built-i commad probit foreig mpg weight The probit regressio output is Iteratio 0: log likelihood = Iteratio 5: log likelihood = Probit regressio Number of obs = 74 LR chi2(2) = Prob > chi2 = Log likelihood = Pseudo R2 = foreig Coef. Std. Err. z P> z [95% Cof. Iterval] mpg weight _cos

6 82 9 Maximum Likelihood Estimatio The margial effect i the logistic regressio are give by E[y x] x I Stata these oe ca be computed usig mfx compute The probit regressio output is = φ(x β)β. (9.10) Margial effects after probit y = Pr(foreig) (predict) = variable dy/dx Std. Err. z P> z [ 95% C.I. ] X mpg weight This commad assumes that x β i Equatio 9.10 is evaluated i the sample meas of the regressors. As a fial commet, MLEs ca be used i a large umber of cases. Some examples iclude latet regressio models, SUR, simultaeous equatios models, oliear regressio models, stochastic frotier models, cout data models, dyamic discrete choice models, GARCHs, etc.