Birkbeck Economics MSc Economics, PGDip Econometrics Autumn 2003 QUANTITATIVE METHODS 1. Background 2. Excercises 3. Notes 4. Advice on QM projects

Transcription

1 Birkbeck Economics MSc Economics, PGDip Econometrics Autumn 2003 QUANTITATIVE METHODS 1. Background 2. Excercises 3. Notes 4. Advice on QM projects Ron Smith

2 1. Introduction 1.1. Aims The QM course provides an introduction to theoretical and applied econometrics which focuses on time-series methods. Cross-section methods are covered in more detail in the Advanced Econometrics option of the Economics MSc. However all the basic results of the linear regression model apply to cross-section as well as time series data. The di erence is that issues of temporal dependence, dynamics, are more important in time-series, while non-linearity is more important in cross section. The course emphasises actually doing applied econometrics. This involves combining economic theory, statistical methods and an understanding of the data with the ability to use the appropriate software Learning Outcomes Derive standard estimators (OLS, ML, GLS) and understand their properties Explain the basis for standard exact and asymptotic tests and use them in practice Develop and analyse basic univariate and multivariate time-series models for integrated and cointegrated data and know how to choose between alternative models Use standard econometrics packages and interpret their output Read understand and explain empirical articles in the literature of the sort that appear in the Economic Journal or American Economic Review Conduct and report on an independent piece of empirical research that uses advanced econometric techniques Structure of the Booklet After this introduction, the rest of Section 1 sets out what is being covered in the course and recommended reading. Section 2 contains two sets of exercises, one mainly theoretical to be done for class, one applied to be done on your own. 2

3 Section 3 contains some theoretical notes. Section 4 contains advice on how to do your applied econometric project (which has to be handed in after Easter). Most of the techniques that you will need to do your projects will be taught in the rst term. Assessment is based on two thirds exam one third project. The course emphasises actually doing applied econometrics. The skill to combine economic theory, statistical methods and an understanding of the data with the ability to use the appropriate software is something that is learnt through experience rather than taught in lectures. The applied exercise in Section 2 is designed to give you that experience. It is essential that you start it as soon as possible. There will be classes in using econometric packages on the computer early in the Autumn term. The applied exercise contains a lot of information that we will assume that you have learned and the theory in the lectures will make a lot more sense if you have seen how it is used. It is essential that you learn how to use an econometric package, either EViews, Micro t, or any other one you wish to use during the Autumn term. We assume that you have done the applied exercise and you will notice that lots of questions using these data appear on past exam questions. These notes cover the material taught in the Autumn term, in the Spring term there will be one lecture and a class each week covering more advanced and more applied topics Provisional Outline Autumn Term By Weeks: 1. Least Squares and the Linear Regression Model (LRM). 2. Maximum Likelihood estimation of the LRM. 3. Test procedures, Asymptotic and Exact. 4. Speci cation and Diagnostic Tests. 5. Univariate Stochastic Processes: ARIMA 6. ARDL models. 7. Unit roots and Cointegration. 3

4 8. Vector Autoregressions and Johansen estimation of Cointegrating Vectors. 9. Endogenous regressors and Generalised Instrumental Variable Estimation (GIVE). 10. Applications and Revision Reading You will need to use a text book to supplement the lectures. There are a large number of good texts which you can use, some of which are discussed below. Get used to using the index to locate the topics, they are covered in di erent places in di erent texts. In most cases you can use earlier editions of the texts. The text book that is closest to this course is A Guide to Modern Econometrics, Marno Verbeek, Wiley, William Greene, Econometric Analysis, (5th edition, 2002 Prentice Hall) provides a more extended treatment. Greene, like most mainstream econometrics texts does not emphasise time-series methods. The best introductory text on modern time series issues is G S Maddala and In-Moo Kim, Unit Roots, Cointegration and Structural Change, Cambridge However, it does not cover a range of other econometric topics that we will deal with. W Enders Applied Econometric Time Series, (Wiley) is a good introductory treatment but is now somewhat out of date. To follow the course you will need to choose a text that uses matrix algebra; but if you have not done any econometrics before you might want to start with one that does not. Examples are Maddala, Introduction to Econometrics, 3rd edition, Wiley; Seddighi, Lawler and Katos, Econometrics A practical approach, Routledge, 2000; Gujarati Basic Econometrics or Essential Econometrics (McGraw Hill); Stewart and Gill, Econometrics, 2nd edition, Prentice Hall 1998; Stock and Watson Introduction to Econometrics, Addison Wesley (2003); Dougherty Introduction to Econometrics 2nd ed Oxford You will also need to use a text-book to deal with issues in your project that are not covered in the lectures. If your current or future work involves econometrics, which it tends to for many of our students, you will nd that you will continue to use the textbook for work and Greene is a good general reference. You should also read Peter Kennedy, A Guide to Econometrics (5th edition Blackwell, 2003). This is not a textbook, it leaves out all the derivations, but it has a brilliant explanation of what econometrics is all about. Past students have 4

5 always found it very helpful. The fth edition has an excellent chapter on doing applied econometrics. J D Hamilton, Time Series Analysis (Princeton/Wiley) is a more advanced time-series text. Je rey Wooldridge, Econometric Analysis of Cross-section and panel data, MIT press 2002, is a more advanced cross-section and panel text. This course is loosely based on the Hendry methodology, David Hendry Dynamic Econometrics (Oxford 1995) provides the authorised version of the methodology. An early exposition of the methodology is Aris Spanos, Statistical Foundations of Econometric Modelling, (Cambridge University Press 1986), which is based on the QM course he gave here at Birkbeck. Davidson and MacKinnon, Estimation and Inference in Econometrics, (Oxford University Press 1993) is very good, particularly on testing, but uses a geometric approach and notation which is rather di erent from most books. Davidson Econometrics is a more conventional and very good exposition. Kerry Patterson, An Introduction to Applied Econometrics, a time series approach Macmillan 2000, has a lot of good applied examples. Keith Cuthbertson, Quantitative Financial Economics, Wiley 1996 has good nance examples. Favero Applied Macroeconometrics, Oxford 2001 is very good at linking the macroeconomic theory to the econometrics. The Micro t manual contains a lot of useful applied tutorials and a clear explanation of the techniques and tests. EViews is not so well documented. Ernst R Berndt, The Practice of Econometrics, (Addison Wesley 1991) is very good on older applied topics. You should also read applied econometric articles. The Journal of Economic Literature, Journal of Economic Perspectives and the Journal of Economic Surveys are good places to look. The December 1998 Journal of Economic Surveys has a range of articles on practical issues in cointegration analysis. 5

6 2. Exercises 2.1. Class exercises These exercises should be attempted before coming to class. Unless you try to do them yourself you will not know what problems you are having Week 1, No class Week 2. (a) Explain the following terms used to describe estimators: Least Squares; Maximum Likelihood; Unbiased; Minimum Variance; Consistent; Asymptotically Normal. (b) Explain what a conditional expectation is. Suppose that there is a disease which a icts 1% of the population without them showing obvious symptoms. There is a test which is 99% accurate, i.e. it shows negative for 99% of those without the disease and positive for 99% of those with the disease. What is the conditional probability that you have the disease given that you have tested positive? How would you evaluate the expected value of a screening programme for this disease? (c) We will not do this in the lectures, but it can be found in any econometrics text. Consider the Linear Regression Model y = X + u where y is a T 1 vector of observations on a dependent variable; X a T k rank k matrix of observations on non-stochastic exogenous variables; u a T 1 vector of unobserved disturbances with E(u) = 0; E(uu 0 ) = 2 I; and a k 1 vector of unknown coe cients. The Least Squares estimator of is b = (X 0 X) 1 X 0 y: The Gauss-Markov theorem is that b is the Best Linear Unbiased Estimator (BLUE) of : Prove this and explain what role each of the assumptions play in the proof Week 3 Suppose we have data on Y t and X t both scalars, t = 1; 2; :::; T and de ne y t = Y t Y, where Y = P T i=1 Y i=n and similarly for x t = X t X: Consider the bivariate regression Y t = + X t + u t 6

7 where the observations are independent and the conditional distribution of Y t is given by f(y t j X t ) = (2 2 ) 1=2 exp 1 2 (Y t X t (a) Show the model can also be written y t = x t + u t : (b) Using the form in (a), derive the maximum likelihood estimators of = (; 2 ): (c) Derive the information matrix. (d) How would you estimate the standard error of b : (e) Compare your derivations to the matrix form in the notes Week 4. (a) In the Linear Regression Model in the week 2 exercise, suppose that the disturbances are also normally distributed and there are k prior restrictions of the form q = 0, where q is a known vector of order k 1: Derive a test statistic to test these restrictions. Explain how you would calculate the restricted and unrestricted sums of squares to carry out the test. (b) The following equations were estimated on 24 observations , where D t is dividends and E t is earnings in year t. Standard errors are given in parentheses, SSR is sum of squared residuals, MLL is maximised log likelihood. D t = 0:59 +0:40E t SSR = 2:1849 (0:20) (0:10) M LL = 5:297 D t = 0:14 +0:32E t 0:10E t 1 +0:70D t 1 SSR = 0:84821 (0:17) (0:08) (0:10) (0:14) M LL = 6:0576 Test the hypothesis that the coe cient of earnings in the rst equation (i) equals one (ii) equals zero. Test the hypotheses that the coe cients of lagged earnings and dividends in the second equation equal zero, (i) individually (ii) jointly. For the joint test use both F and LR tests. Suppose the coe cients in the second equation are labelled = ( 1 ; 2 ; 3 ; 4 ): Write the restrictions that the coe cients of lagged earnings and dividends equal zero in the form R q = 0: ) 2 7

8 Week 5 Explain what e ect the following problems have on the properties of the least squares estimates of the coe cients and their standard errors. How would you detect whether each problem was present: (a) Heteroskedasticity. (b) Serial correlation. (d) Non-normality. (e) Non-linearity. (f) Exact multicollinearity Week 6 Consider the general model d t = d t e t + 1 e t p t + 1 p t 1 + u t where d t is log nominal dividends, e t is log nominal earnings and p t is the log of the producer price index. (a) How would you calculate the long-run elasticity of dividends to prices and earnings, i in: d t = e t + 2 p t :. (b) How would you estimate the model subject to the restrictions that the long-run elasticity to earnings is unity and to prices zero. (c) Suppose it was believed that the appropriate model was d t = + e t + p t + v t ; v t = v t 1 + " t ; where is less than one in absolute value. Show that this is a restricted form of the general model and derive the two restrictions. (d) Suppose it was believed that the adjustment to the long-run relationship was given by: d t = 1 d t + 2 (d t 1 d t 1 ) + u t what restriction does this impose on the general model. (e) Using the Shiller data, estimate the models over the period and test the three sets of restrictions. 8

9 Week 7 Consider the following models estimated over a sample t = 1; 2; :::; T. In each case " t is white noise and and are less than one in absolute value. y t = + y t 1 + " t (1) y t = + y t 1 + " t (2) y t = + " t + " t 1 (3) y t = + y t 1 + " t + " t 1 (4) (a) Suppose you had estimates of the parameters, how would you forecast y T +1 and y T +2 in each case, given data up to y T? (b) In cases (1) and (2) substitute back to remove the y t i : (c) What is the expected value of y t in each case? (d) For (1) and (3) derive the variance, covariances and autocorrelations of the series. (e) For which of the models is y t I(1)? Week 8. The Augmented Dickey Fuller Tests for non-stationarity uses a regression of the form: TX y t = + y t 1 + t + i y t i + " t (a) What is the null hypothesis tested; what is the test statistic and what is its 95% critical value? (b) Suppose all the i = 0. Substitute back to express y t in terms of " t i ; t and y 0 : (c) What is the rationale behind the P T i=1 iy t i term. (d) Test whether the log payout ratio (log(nd=ne) has a unit root (i) over , (ii) over Weeks 9 and 10. Consider the VAR i=1 y t = A 0 + A 1 y t 1 + " t ; t = 1; 2; :::; T 9

10 where y t = (y 1t ; y 2t ) 0 ; A 0 is a 2x1 vector, (a o ; a 1 ) 0 ; a11 a A 1 = 12 0 ; " a 21 a t ~N 22 0 ;!11! 12! 21! 22 (a) Write down each equation of the VAR and explain how you would estimate the coe cients and covariance matrix. (b) Given estimates of the parameters, how would you forecast y T +1 and y T +2? (c) What condition is required for y 1t to be Granger non-causal with respect to y 2t? (d) Write the VAR as y t = A 0 + y t 1 + " t ; t = 1; 2; :::; T: Explain the relation between A 1 and : (e) What are the implications for if y it are (i) I(0); (ii) I(1) and cointegrated; (iii) I(1) and not cointegrated? What restrictions does case (iii) put on A 1? (f). Suppose the y it are I(1) with cointegrating relationship z t = y 1t y 2t. Write out the restricted system and explain the restrictions this imposes on : Show that it has rank one. (g) Derive the parameters of the ARDL(1,1) model: y 1t = y 2t + 1 y 2t y 1t 1 + u t from the VAR. Hint note that E(" 1t j " 2t ) = (! 12 =! 22 )" 2t and use this in the rst equation of the VAR substituting for " 2t : Additional Questions (based on old exams). 1. Suppose you have data for two groups, i = 1; 2 with models y i = X i i + u i ; u i N(0; 2 i I) where X i are T k full rank matrices of observations on exogenous variables. (a) How would you test 2 1 = 2 2? (b) Given 2 1 = 2 2 how would you test 1 = 2? (c) If 1 = 2 but 2 1 6= 2 2; how would you estimate the model? 2. Suppose Gross investment, I; equals the change in the capital stock, K t plus depreciation, K t 1 and the change in the capital stock is given by partial adjustment to desired capital stock: K t K t 1 = (K t K t 1 ) + u t 10

11 where u t is a white noise error and Kt = + Y t, where Y t are sales. Neither actual nor desired capital stock are observed. (a) Show that the investment equation can be written: I t = + Y t (1 )Y t 1 + (1 )I t 1 + u t (1 )u t 1 : Hint use the procedure for adaptive expectations in the notes to express K in terms of I. (b) How would you estimate the model, given data on I t and Y t? (c) Show that is overidenti ed. 3. In the Linear Regression Model y = X + u where X is a T k matrix, de ne P x = X(X 0 X) 1 X 0 and M = I P x: The Least Squares residuals are bu = y X; b b = (X 0 X) 1 X 0 y: (a) Show that P x and M are both idempotent and that they are orthogonal. (b) Show that bu = My = Mu: 4. In the Linear Regression Model suppose E(X 0 u) 6= 0; where X is a T k matrix but that there are a set of i instruments, W such that E(W 0 u) = 0 and E(W 0 X) 6= 0: (a) Suppose i = k derive the IV estimator of. What does it minimise? Show that the IV minimand is zero. (b) Suppose i > k derive the GIV estimator of. How would you estimate the standard errors of the elements of : 5. Call the logarithm of the real Standard and Poor Stock price index y t : The following models were estimated over the period Numbers in parentheses below coe cients are estimated standard errors and numbers in brackets are the maximised log-likelihoods (MLL), " t is assumed to be a white noise error in each case. 11

12 A: y t = 0: " t [35:3726] (0:0167) B: y t = 0: :098 " t 1 ; + " t [35:7223] (0:0181) (0:83) C: y t = 0: :0641 y t 1 + " t [35:6056] (0:0168) (0:0947) D: y t = 0: :822 " t 1 0:642 y t 1 + " t [37:8298] (0:0294) (0:113) (0:149) E: y t = 0:266 +0:002 t 0:135 y t 1 +0:131 y t 1 + " t [39:9282] (0:102) (0:0008) (0:046) (0:095) (a) Use model E to test for the presence of a unit root in y t : The 5% critical value for the Augmented Dickey-Fuller test with trend is (b) In model D, test at the 5% level whether the autoregressive and moving average coe cients are signi cant: (i) individually; (ii) jointly. Explain the con ict between the individual and joint results. (c) Brie y indicate how model D can be used to make a two-period ahead forecast, given data up to (d) Which of these models would you choose on the basis of the AIC = MLL i k i ; where k i is the number of parameters estimated in model i. 6. Consider the general model d t = d t e t + 1 e t p t + 1 p t 1 + u t ; where d t is log nominal dividends, e t is log nominal earnings, p t is the log of the producer price index, and u t is a white noise error. Estimation on US annual data from 1950 to 1986 gave a maximised log-likelihood (MLL) of (a) What is the long-run elasticity of dividends to prices and earnings? (b) A Wald test of the restrictions 1 = 0 and = 0 gave a test statistic of (i) Are the restrictions rejected at the 5% level? 12

13 (ii) Interpret these restrictions. (iii) What is the restricted model? (c) A Wald test of the restrictions 0 = 1 = 0 = 1 gave a test statistic of When the model was estimated subject to the non-linear restrictions 0 = 1 = 0 = 1 the MLL was A Wald Test of the restrictions 0 1 = 1 0 gave a test statistic of (i) Interpret these restrictions. (ii) Conduct a Likelihood Ratio test of the restrictions 0 = 1 = 0 = 1 : (iii) Comment on the results of the two Wald tests and the Likelihood Ratio test. 8. A second-order cointegrating vector error-correction model (VECM), with unrestricted intercepts and restricted trends, was estimated on quarterly US data from 1947Q3 to 1988Q4. The variables included were the logarithm of real consumption (c t ), the logarithm of real investment (i t ), and the logarithm of real income (y t ). The Johansen maximal eigenvalue tests for, r; the number of cointegrating vectors, were: The Johansen Trace Tests were: H o H 1 Statistic 10%CV r = 0 r = 1 34:6 23:1 r 1 r = 2 15:8 17:2 r 2 r = 3 3:3 10:5 H o H 1 Statistic 10%CV r = 0 r 1 53:7 39:3 r 1 r 2 19:1 23:1 r 2 r = 3 3:3 10:5 Assuming that r = 2, the following two just-identi ed cointegrating vectors (standard errors in parentheses) were estimated: c i y t 1 0 1:13 0:0003 (0:16) (0:0006) 0 1 1:14 0:0007 (0:26) (0:001) 13

14 The system maximised log-likelihood (MLL) was The system was then estimated subject to the over-identifying restrictions that: (i) both coe cients of income were unity, giving a MLL of ; and (ii) not only were the income coe cients unity, but that the trend coe cients were also zero, giving a MLL of (a) How many cointegrating vectors do the tests indicate? (b) If there are r cointegrating vectors, how many restrictions on each vector do you need to identify it. (c) Interpret the just identifying restrictions used above. (d) Test the two sets of overidentifying restrictions. (e) The VECM was estimated with unrestricted intercepts and restricted trends. What does this mean? 14

15 2.2. Applied Exercise Data There is a data set available in Excel format as Shiller.xls; in Micro t format as Shiller. t and Eviews format as Shiller.wf1. You can use any program you wish to analyse it. This contains annual US data from 1871 to 2000 on NSP nominal Standard and Poors stock price index (January value); ND nominal dividends per share for the year; NE nominal earnings per share for the year; R the average interest rate for the year; PPI producer price index, January value. The data is updated from Robert J Shiller Market Volatility, MIT Press 1989 and we will use it to re-examine the hypotheses in a famous paper J Lintner Distribution of Income of Corporations among Dividends, Retained Earnings and Taxes, American Economic Review May The early exercises are quite speci c, telling you exactly what to do. The later ones ask more general questions and leave you to decide how to answer them Theoretical background. Lintner suggested that there was a target or long-run dividend pay-out ratio, say, ; such that D t = E t : We will take logs of this relationship, using lower case letters for logs, e.g. d t = log(d t ), etc. Notice natural logs are almost universally used. Taking logs we get d t = log() + e t. This can be written in an unrestricted form as d t = e t ; where his theory suggests that 1 = 1 and 0 = log(): To this he added a Partial Adjustment Model (PAM) and a random error d t = (d t d t 1 ) + u t d t = e t + (1 )d t 1 + u t : d t = b 0 + b 1 e t + b 2 d t 1 + u t The PAM can be justi ed if, for instance, rms smooth dividends, not adjusting them completely to short term variations in earnings. We estimate the b i and then work out the theoretical parameters from them: = 1 b 2 ; 1 = b 1 =(1 b 2 ); 0 = b 0 =(1 b 2 ) Notice the restricted model (with 1 = 1) can be estimated after creating a new variable earnings minus lagged dividends as: d t = a 0 + a 1 (e t d t 1 ) + v t : 15

16 More general adjustment processes such as Error Correction Models (ECM) can be used and other variables, e.g. in ation and stock market prices included. The model above is in terms of nominal dividends and earnings, it could be done in terms of real dividends and earnings with rd t = d t p t, and written: d t p t = b 0 + b 1 (e t p t ) + b 2 (d t 1 p t 1 ) + u t removing the restrictions gives the unrestricted model: d t = c 0 + c 1 e t + c 2 p t + c 3 d t 1 + c 4 p t 1 + v t : The restricted model has three parameters, the unrestricted model has ve parameters, so there are two homogeneity restrictions: c 1 + c 2 = 1; c 3 + c 4 = 0: Check this by working out the c i in terms of the b i : Data Analysis Load the data. In Eviews, choose File, New, Work le, In the dialog box specify annual data, and put in the boxes for beginning and end. You will then get a box with two variables, C for the constant and Resid for the residuals. Choose File, Import, Read Text Lotus Excel. In the dialog box where it asks for names or number, put 5 and OK. The 5 variables will appear in the list of variables. Create the following new variables: Use Quick Generate in EViews and give the formula to generate the new variables. Just put the formula in the editor in M t. After you have generated the new variables, save the work le with a di erent name. C=1 (Eviews provides this automatically, in M t you have to generate it yourself). LD=log(ND); LE=log(NE); LSP=log(NSP); LP=log(PPI); LRD=LD-LP; LRE=LE-LP; LRSP=LSP-LP; LDY=LD-LSP; LPE=LSP-LE; LPO=LD-LE; DLD=LD-LD(-1); DLE=LE-LE(-1); DLSP=LSP-LSP(-1); DLP=LP-LP(-1); DLRD=LRD-LRD(-1); DLRE=LRE-LRE(-1); DLRSP=LRSP-LRSP(-1). Interpret these variables. Note DY, PE and PO are the initials for three ratios: Dividend Yield, Price-Earnings and Pay-Out. The starting D indicates a rst di erence, change. L indicates logs. R indicates real. So DLRSP is the change in the log of the real stock price index. Note changes in logs correspond 16

17 to growth rates, so DLP is in ation measured as a proportionate change not a percentage change. Plot these series. Which have trends, which do not? On which series can you see the e ects of World War I, the 1929 crash, World War II, the Oil shock of 1973? Plot dividends and earnings together, were there occasions when rms paid out more in dividends than they earned? Are there shifts in the average levels of the three ratios? Look at the Price Earnings Ratio (LPE) for the 1990s. Get summary statistics on LRD, LRE, LP, LRSP and their changes: DLRD, DLRE, DLP, DLRSP. Which seem more normally distributed? Normally distributed variables have skewness of zero and kurtosis of 3. Plot histograms to con rm the impression given by the summary statistics. Are dividends more or less volatile than earnings? Compare the correlations of LD with LE and DLD with DLE. Look at the scatter diagrams for the two pairs and compare the degree of association. Whenever you do any empirical work, start with a data analysis like this: look at ratios, time-series plots, histograms, scatter diagrams, descriptive statistics Regression output. Over the sample, t = 1; 2; :::; T : estimate the following static regression: d t = + e t + u t (1) You do this by choosing Quick, Estimate an equation in Eviews, Univariate in M t, in both cases when you get to the appropriate editor type in: LD C LE and check that you have the right sample. If you are getting di erent results from below the most likely cause is having a di erent sample. Print or save the results to compare with the rst di erence equation below. The programs will give you: Estimates of the coe cients, and, their standard errors, t ratios (ratio of coe cient to the standard error which is the test statistic for testing the null hypothesis that the coe cient is zero) and perhaps p values for the null hypothesis that the coe cients are zero. Various descriptive statistics such as the mean of the dependent variable y = 17

18 P T t=1 y t=t; and its standard deviation v ux s y = t T (y t y) 2 =(T 1) The Sum of Squared Residuals: t=1 TX t=1 the standard error of regression: v ux s = t T bu 2 t =(T k) t=1 where k = 2 in this case. The ordinary coe cient of determination and the version corrected for degrees of freedom (R bar squared): P T P T R 2 t=1 = 1 bu2 t P T t=1 (y t y) ; 2 R2 t=1 = 1 bu2 t =(T k) P T t=1 (y t y) 2 =(T 1) R 2 measures the proportion of variation in the dependent variable y t explained by the regression. An F test for the hypothesis that all the slope coe cients (i.e. other than the intercept) are equal to zero: bu 2 t [ P T t=1 (y t y) 2 P T t=1 bu2 t ]=(k 1) P T t=1 bu2 t =(T k) ~ F (k 1; T k) The Maximised Log-likelihood and some model selection criteria such as Schwarz or Akaike are often given (see theoretical notes). Note that EViews and M t use di erent formuale for these model selection criteria. In Eviews you choose the model with the smallest value, in M t you choose the model with the largest value. The Durbin Watson statistic is DW = P T t=2 bu2 t P T t=1 bu2 t This measures serial correlation in the residuals. It should be about 2 and is roughly equal to 2(1 ) where is the serial correlation coe cient. It is only appropriate for rst order serial correlation when there are no lagged dependent variables in the equation. 18

19 Comparison of levels and rst di erence regression. Now estimate the rst di erence version over : d t = e t + e t (2) Type in DLD C DLE. Print and save the results. The coe cient of LE (0.86) is a lot larger than the coe cient of DLE (0.23). The intercept in the levels equation is negative and signi cant, in the rst di erence equation positive and signi cant. Interpret the intercepts. Although the R 2 suggests that the levels equation ts better (0.96 versus 0.29) in fact the standard error of the regression of the rst di erence equation is substantially smaller (0.043 versus 0.107). This means that we would make roughly a 4% error in predicting dividends using the rst di erence equation and a 10% error in the levels equation. In log models multiply the standard error by 100 and interpret it as a percentage error. The only reason the R 2 of the level version looks better is that the level has a larger variance than the change (standard deviation of 0.55 as against 0.05). Finally the level regression has an appalling Durbin Watson statistic 0.65, while that of the change regression, 1.44, is a lot better. Plot the actual and predicted values for the level regression and you will see that the tted (predicted value) is above the actual through the 50s; below during the 60s and early 70s; then above to the early 80s. The errors are not random, but systematically above or below for long periods of time Adding Dynamics: Estimate over the partial adjustment model: d t = b 0 + b 1 e t + b 2 d t 1 + u t (3a) i.e. type in LD C LE LD(-1). Calculate the long-run coe cient 1 = b 1 =(1 b 2 ); use the Wald test to test the hypothesis that b 1 =(1 b 2 ) 1 = 0 or equivalently b 1 + b 2 1 = 0: In Eviews if you click on View when the regression results are showing, you will get a set of options which include coe cient tests. In M t you can do it either by linear or non-linear function of the parameters in the postregression menu or Wald test in the hypothesis testing menu. Try the Wald test using both forms of the restriction. Wald tests are not invariant to the way you write the restriction and you will get di erent answers. 19

20 Estimate DLD C LE LD(-1) d t = a 0 + a 1 e t + a 2 d t 1 + u t (3b) Compare the two equations. What is the relationship between their coe cients, which of the summary statistics are the same, which are di erent? Why? Estimate over the ARDL(1,1): LD C LE LE(-1) LD(-1): the ECM1: DLD C DLE LE(-1) LD(-1): d t = e t + 1 e t d t 1 + u t (4a) d t = a 0 + b 0 e t + b 1 e t 1 + a 1 d t 1 + u t (4b) and the ECM2: DLD C DLE LE LD(-1): d t = a 0 + b 0 e t + b 1 e t + a 1 d t 1 + u t : (4c) Compare the ARDL(1,1), ECM1 and ECM2; what is the relationship between their coe cients, which of the summary statistics are the same, which are di erent? Compare these with the PAM, rst di erence and levels models. The various version of 4 or 3 are just reparameterizations, statistically identical. Whereas 3, 2, and 1 are all restricted versions of 4. 1 is also a restricted version of 3, but 2 is not nested within 3. Set out the restrictions on 4a that give 3, 2 and 1 and test the restrictions. Note the test of (4) against (2) is non-standard because the series are probably I(1) and to do it properly di erent critical values would be needed Diagnostics M t automatically provides four diagnostic tests, and more can be got through the hypothesis testing menu. In Eviews, click the View button when the regression results are showing and it will give you a menu of options for testing. Estimate (4a) and conduct tests for Heteroskedasticity (White without cross-products in EViews), second order serial correlation, Normality, Non-linearity (Ramsey Reset test) and structural stability: Cusum, Cusumsquared, Chow break point and Chow forecasting (predictive failure) with the break point at 1973 (this starts the second period in 1973). In M t estimate over and specify for the forecasting tests. The tests are explained in section 3.4 of the theoretical notes. 20

21 Notice that on tests which use tted values (e.g. Ramsey RESET) you will get di erent test statistics if you parameterise the equation using the levels or the changes of the dependent variable. The diagnostics using the change as the dependent variable are more reliable. Does the equation seem well speci ed? Look at the actual and tted values and residuals. Eviews and Micro t have di erent diagnostic tests, so an equation that looks OK on one program may not on another. This equation shows a number of problems and does not seem well speci ed. The failure on normality seems due to the 1950 observation which has a large residual. Try a more general model: DLD C LE DLE DLE(-1) LD(-1) DLD(-1) LP DLP DLP(-1) Test that the long-run e ect of earnings is unity and the long run e ect of prices is zero. Drop insigni cant variables and try imposing the long-run unit earnings elasticity and long-run zero price elasticity. Experiment with models with disturbance serial correlation e.g. in EViews LD C LE LP AR(2). In M t choose the exact ML option for serial correlation in the estimator choice menu ARIMA models For the period estimate the following ve models for the logarithm of the real stock price index: 1. Random Walk (ARIMA(0,1,0). Just run DLRSP C: 2. AR2, (ARIMA(2,0,0)). Run LRSP C LRSP(-1) LRSP(-2); 3. ARIMA( 1,1,0). Run DLRSP C DLRSP(-1); 4. ARIMA(1,1,1). Run DLRSP C DLRSP(-1) MA(1) in Eviews, in M t choose the MA option in the estimator menu set the order at one and run the equation; 5. ARIMA(0,1,1); as for 4, without including.dlrsp(-1) Use hypothesis tests and model selection criteria to choose between the ve models. Some points to note. For the ARIMA(1,1,1) EViews and M t give somewhat di erent answers, though the general impression is similar. In EViews you can 21

22 also put in DLRSP C AR(1) MA(-1), and get the same estimates apart from the constant which will di er by the factor (1 ): M t gives you the plot of the log-likelihood function and you will see it is multimodal. M t will not converge from the initial value selected by the program, zero. Put in a value like 0.9. Notice that although the AR and MA components seem signi cant in the ARIMA(1,1), they cancel making the random walk the pre ered model. The random walk is y t = + " t multiply each side by a common factor (1 L) and we get (1 L)y t = (1 L) + (1 L)" t y t = (1 ) + y t 1 + " t " t 1 with signi cant AR and MA terms of equal and opposite signs. This seems to be what is happening here. If the MA estimate is unity, you will not get standard errors, since this is on the boundary of the possible parameter values. Try using the models to forecast the 1990s Unit Roots etc Estimate equations of the form y t = 0 + y t 1 + t + y t 1 + u t for the logarithms of nominal and real dividends and earnings and the stock price level over and Test the hypothesis = 0, remember the critical values are non-standard. Do the same for: 2 y t = 0 + y t y t 1 + u t : Are these series I(0), I(1) or I(2)? Estimate a second order VAR for these real earnings, real dividends and real stock prices over and test for Granger causality between the variables. Use the Johansen procedure to test for cointegration assuming unrestricted intercept and restricted trend, option 4 in Eviews. Assume that there is a single cointegrating vector, estimate and interpret it. How does the cointegrating vector feedback on the three variables? How sensitive is the evidence for cointegration to the sample chosen? Try other periods. 22

23 Use the data up to 1995 to estimate a var between log real stock prices and log real earnings. Does the log price-earnings ratio constitute a cointegrating vector? Develop a model for these two variables and use it to forecast stock prices and comment on your results. 23

24 3. Theoretical Notes 3.1. Least squares and The Linear Regression Model Econometrics is mainly about estimating linear regression models of the form: y t = x t + u t for t = 1; 2; :::; T, that is a set of T equations. The residuals, errors, are u t = y t 1 2 x t : Least squares chooses i to minimise P u 2 t : In matrix form the model is and we will use y 1 y 2 :: y T (X 0 X) 1 = = The sum of squared residuals is (X 0 X) = y = X + u 3 1 x 1 1 x 2 7 :: :: 5 1 x T 1 1 T P x 2 t ( P x t ) 2 u 0 u = (y X) 0 (y X) P P T P xt xt x 2 t u 1 u 2 :: u T P x 2 t P xt P xt T = y 0 y + 0 X 0 X 2 0 X 0 y TX u 2 t = X X X yt 2 + ( 2 1T x 2 t xt ) t=1 2( 1 X yt + 2 X xt y t ) Notice that if A is a n m matrix, and B is an m k matrix the transpose of the product (AB) 0 is B 0 A 0 the product of a k m matrix with a m n matrix, A 0 B 0 is not conformable. This is a scalar, which is always equal to its transpose, so we can set y 0 X = 0 X 0 y. The term 0 X 0 X is a quadratic form, i.e. of the form x 0 Ax: Quadratic forms play a big role in econometrics. A matrix, A; is postive de nite if for any a; a 0 Aa > 0: Matrices with the structure X 0 X are always positive de nite, since they can be written as a sum of squares. De ne z = Xa; then z 0 z = a 0 X 0 Xa is the sum of the squared elements of z. 24

25 Di erentiation with vectors and matrices Consider the equation: P = x 1xn 0 a nx1 Then the di erential of P with respect to x or x 0 is de ned as : dp dx = a and dp dx 0 = a 0 To see this make sense consider the case n= 2. Then we can write: a1 P = [x 1 ; x 2 ] Then So and a 2 = x 1 a 1 + x 2 a 2 dp = a 1 and dp = a 2 dx 1 dx 2 dp dx = Consider the quadratic form: dp dx 1 dp dx 2 = a1 a 2 = a dp dp = ; dp = [a dx 0 1 ; a 2 ] = a 0 dx 1 dx 2 Q = x 1xn 0 A nxn x nx1 Then the di erential of Q with respect to x or x 0 is de ned as : dq dx = 2Ax and dq dx 0 = 2x 0 A To see this make sense consider the case n= 2. Then we can write: a11 a Q = [x 1 ; x 2 ] 12 x1 a 12 a 22 x 2 25

26 where for simplicity I have assumed A is symmetric. a11 x Q = [x 1 ; x 2 ] 1 + a 12 x 2 a 12 x 1 + a 22 x 2 Expanding this gives: = a 11 x a 12 x 1 x 2 + a 22 x 2 2 So: Then and dq dx = dq dx 1 = 2a 11 x 1 + 2a 12 x 2 and dq dx 2 = 2a 12 x 1 + 2a 22 x 2 " dq dx 1 dq dx 2 = 2 A 2x2 x 2x1 dq dx 0 = # 2a11 x = 1 + 2a 12 x 2 2a 12 x 1 + 2a 22 x 2 a11 a = 2 12 a 12 a 22 x1 dq ; dq = [2a 11 x 1 + 2a 12 x 2 ; 2a 12 x 1 + 2a 22 x 2 ] dx 1 dx 2 a11 a 12 = 2 [x 1 ; x 2 ] = 2 x 1x2 0 A 2x2 a 12 a 22 x The linear regression model (LRM) Consider the linear regression model y = X + u T x1 T xk kx1 T x1 The problem is to minimize the sum of squared residuals with respect to u 0 u = (y X) 0 (y X) = (y 0 0 X 0 ) (y X) = y 0 y 0 X 0 y y 0 X + 0 X 0 X These are all scalars the second and third terms are equal and we can write u 0 u = y 0 y 2 0 X 0 y + 0 X 0 X 26

27 The second term is: P = 2 0 (X 0 y) 1xk kx1 From above we know that if P = x 0 a, dp dx = a so The third term is a quadratic form dp d = 2X0 y Q = 1xk 0 (X 0 X) kxk From above 2 we know that if Q = x 0 Ax, dq dx And the rst order condition is so kx1 dq d = 2X0 X 0 = 2X 0 y + 2X 0 X = (X 0 X) 1 X 0 y = 2Ax so: In the simple model to minimise u 0 u we have to di erentiate it twice, with respect to 1 and 2 ; to get the 2 1 vector of derivatives and set them equal to 0 The two elements of the vector are = 2X 0 X 2X 0 y = 0 = 2X 0 (y X) = 2X 0 u = 0 1 = 2 1 T X xt 2 X y t = = 2 2 X x 2 t X xt 2 X x t y t = 0 (3.2) 27

28 Check that this corresponds to the matrix formula. We can also write these as 2 X (y t [ x t ]) = 2 X u t = 0 2 X x t (y t [ x t ]) = 2 X x t u t = 0 The least squares estimates make the residuals uncorrelated with the regressors. To nd the least squares estimates, we cancel the 2s and solve for 1 and 2 from the two equations of the rst order condition 2X 0 X 2X 0 y = 0 b = (X 0 X) 1 X 0 y This requires that X 0 X is non-singular, so an inverse exists. Our least squares estimate of is denoted b and is a 2x1 vector. P P P (X 0 X) 1 X 0 1 x 2 y = T P x 2 t ( P P t xt P yt x t ) 2 xt T xt y t 1 = P x 2 t P yt P xt P xt y t T P x 2 t ( P x t ) 2 2 = P xt P yt + T P x t y t T P x 2 t ( P x t ) 2 These can be expressed in more intuitive form. From the rst equation (3:1) P yt P xt 1 = T 2 T = y 2 x substituting for 1 in the second equation (3:2) can be written 2 X x 2 t + (y 2 x) X x t X xt y t = 0 2 X xt (x t x) X xt (y t y) = 0 2 = P xt (y t y) P xt (x t x) = P (xt x)(y t y) P (xt x) 2 28

29 Dividing top and bottom by T, this is the ratio of the covariance to the variance of x t : Note that X (xt x)(y t y) = X X X x t y t + T yx xt y yt x P P xt yt = X x t y t + T T T = X x t (y t y) X xt P yt T X yt P xt T Statistical properties of the LRM. Least squares is a purely arithmetic procedure, to establish its statistical properties we need to make some statistical assumptions. Suppose we have a sample of data of observations on random variables y t a scalar and x t a k 1 vector. The joint distribution of the random variables, y t ; x t ; can be written as the product of the distribution of y t conditional on x t and the marginal distribution of x t : D j (y t ; x t ; j ) = D c (y t j x t ; c )D m (x t ; m ) (3.3) j is a vector of parameters of the joint distribution, c of the conditional distribution, m of the marginal. The distribution that we will be interested in is the distribution of y t conditional on x t and the parameters that we will be interested in are the parameters of the conditional distribution c which we will usually denote by : We will assume that the x is exogenous, which means that there is no information in the marginal distribution for x about the parameters of the conditional distribution that we are interested in. Usually we are only interested in the rst two moments of the distribution, the conditional expectation (the regression function) and the conditional variance. If y t and x t are jointly Normally distributed, say: yt y 2 = N ; y yx x x t xy xx then the conditional expectation of y t is a linear function of x t : E(y t j x t ) = 0 x t where is a k 1 vector of regression coe cients, which are a function of the parameters of the joint distribution. We can decompose y t into two components, the systematic part given by the conditional expectation and the unsystematic part, the error. The error is: u t = y t E(y t j x t ) = y t 0 x t 29

30 so: y t = 0 x t + u t ; t = 1; 2; :::; T: (3.4) If the random variables are jointly normally distributed and the observations are independent, the conditional variance is a constant: E(y t E(y t j x t )) 2 = E(u 2 t ) = 2 : (3.5) The parameters of the conditional distribution which we will want to estimate are c = = (; 2 ): We will write the conditional expectation in a number of ways other than (3.4). In terms of the original variables it is: E(y t j x t ) = x 2t + ::: + k x kt : (3.6) Notice that x 1t is a variable that always takes the value unity, one, to pick up the intercept. We can also write it in matrix form y = X + u; (3.7) where y is a T 1 vector; X is a T k matrix; is a k 1 vector of unknown parameters; u is a T 1 vector. Note that the rst column of X, like the rst element of x t ; is a column of ones to pick up the intercept. In matrix form the T T conditional variance covariance matrix of y is: E(y E(y j X))(y E(y j X)) 0 = E(uu 0 ) = 2 I T : This is a T T matrix with 2 on the diagonal and zeros on the o -diagonals. Distinguish uu 0 a T T variance covariance matrix and u 0 u the scalar sum of squared residuals. If the joint distribution of y t and x t is independent normal, the conditional distribution is also normal and we can write the distribution for an observation: D c (y t j x t ; ) IN( 0 x t ; 2 ) 1 = (2 2 ) 1=2 exp 2 (y t 0 x t ) 2 or in matrix form for the whole sample: D c (y j X; ) N(X; 2 I) (3.8) 1 = (2 2 ) T=2 exp 2 (y 2 X)0 (y X) : (3.9) 30

31 Notice that we do not need to specify independence in the matrix form, the fact that the variance covariance matrix is 2 I implies that the conditional covariances between y t and y t i are zero. We need to make some assumptions about X: First it should be of full rank k, there should be no exact linear dependences between the columns of X; the various right hand side variables. This is required for (X 0 X) 1 to exist. Secondly, the right hand side variables should be exogenous, roughly independent of the errors. Exogeneity is discussed in more detail below in the notes for week Maximum Likelihood estimation Introduction Suppose we have a random variable y with a known probability density function f(y; ), where is a vector of parameters (e.g. mean (expected value) and variance). We can use this formula to tell us the probability of particular values of y, given known parameters. For instance, given that a coin has a probability of getting a head of p = 0:5, what is the probability of observing 10 heads in a row? Answer (0:5) 10 : Alternatively, we can use the same formula to tell us the likelihood of particular values of the parameters, given that we have observed a sample of realisations of y; say y 1 ; y 2 ; ::; y T : Given that we observe ten heads in a row, how likely is it that the coin is unbiased (i.e p = 0:5)? Again the answer is (0:5) 10. In the rst case we interpret f(y; ) as a function of y given. In the second case we interpret f(y; ) as a function of given y: The maximum likelihood (ML) procedure estimates as the value of most likely to have given the observed sample. In the coin example, p = 0:5 is very unlikely to have generated the observed sample of 10 heads. We will denote the ML estimator by b : If the sample is independent we can just multiply the probabilities for each observation together as we did in the coin example and write the Likelihood as: L() = f(y 1 ; )f(y 2 ; ):::f(y T ; ) We then choose that maximises this value for our observed sample y 1 ; y 2 ; :::; y T : In practice, it is more convenient to work with the logarithm of the likelihood function. Since logs are a monotonic function the value of that maximises the log-likelihood will also maximise the likelihood. Thus the log-likelihood is: LL() = TX log f(y t ; ): t=1 31

32 To nd the maximum we take the derivatives of LL(); and set them to zero: S( b P log f(y t ; = 0 then solve for the value of ; b that makes the derivatives equal to zero. Notice that LL() is a scalar, and if is a k 1 will be a k 1 of derivatives. S( b ) is often called the Score vector. For simple examples, like the LRM below we can solve these equations analytically, for more complicated examples we solve them numerically. To check that we have found a maximum, we need to check the second order conditions and calculate the kxk matrix of second 0 : For a maximum this matrix should be negative de nite. From this we can derive the Information matrix, the negative of the expected value of the matrix of second derivatives: I() = LL() 0 ): A useful result is that for any unbiased estimator (in small samples) or consistent estimator (asymptotically when T! 1) the inverse of the information matrix provides a lower bound (the Cramer-Rao lower bound) on the variance covariance matrix of the estimator V ( b ) I( b ) 1 : General properties of ML estimators Under certain conditions (which usually hold in economic examples) the ML estimator b is consistent, that is lim T! 1 Pr(j b T j> ) = 0 it is asymptotically normally distributed and asymptotically attains the Cramer- Rao lower bound (i.e. it is e cient), it is asymptotically N(; I() 1 ): I( b ) 1 is often used to provide estimates of the asymptotic variance covariance matrix of the 32

33 ML estimator. When we evaluate asymptotic distributions we look at p T ( b ) as T! 1, because since it is consistent the distribution of b collapses to a point. S() is also asymptotically normal N(0; I()): We will use these two asymptotic normality properties in testing. In addition, E(S()S() 0 ) = I(): ML estimators are also invariant in that for any function of, say g(), the ML estimator of g() is g( b ). Partly because of this ML estimators are not necessarily unbiased, some are many are not ML estimation of the LRM For the LRM, the likelihood of the sample is given by (3.8) above, but now interpreted as a function of = (; 2 ), the unknown parameters: 1 L(; 2 ) = (2 2 ) T=2 exp 2 (y 2 X)0 (y X) : The Log-likelihood function is : LL(; 2 ) = T 2 log(2) T 2 log(2 ) (y X)0 (y X): and to nd the estimates that maximise this we di erentiate it with respect to and 2 and set the derivatives equal zero. Notice that u 0 u = (y X) 0 (y X) = y 0 y + 0 X 0 X 2 0 X 0 y: When we transpose we reverse the order to maintain the correct dimensions and 0 X 0 y = y 0 X because both are scalars. 2 = (2X0 X 2X 0 y) (3.10) 2 ) T u0 u: (3.11) The derivative of log( 2 ) is 1= 2 and the derivative of 1=2 2 = (2 2 ) 1 is ( 1)( (2 2 ) 2 ). Setting (3.10) equal to zero gives one First Order Conditions, FOC 1 2b 2 (2X0 X b 2X 0 y) = 0 1 b 2 (X0 y X 0 X b ) = 0 33

34 where the hats denote that these are the values of and 2 that make the FOCs equal to zero. Notice that this can be written 1 b 2 X0 (y X b ) = 1 b 2 X0 bu = 0 (3.12) the rst order conditions choose that makes the estimated residuals, bu = y X b ; uncorrelated with (orthogonal to) the explanatory variables. This estimate is b = (X 0 X) 1 X 0 y: Notice that we need X to be of full rank for the inverse of (X 0 X) to exist. If (X 0 X) is singular, b is not de ned. This is called exact multicollinearity. Setting (3.11) equal to zero gives T 2b b 4 bu0 bu = 0 multiply through by 2b 4 T b 2 + bu 0 bu = 0 so our maximum likelihood estimator of the variance is: b 2 = bu0 bu T : The ML estimator is biased and we usually use the unbiased estimator s 2 = bu 0 bu=(t k): To check second order conditions and construct the information matrix we take derivates of (3.10) and (3.11) Notice the derivative of ( 2 ) 1 X 0 u 2 LL(; 0 = 1 2 X0 X 2 2 = 1 4 X0 u: (3.14) ( 2 ) 2 X 0 u: 2 LL(; 2 2 ) 2 = T 2 4 u 0 u 6 : (3.15) 34

35 To get the information matrix we take the negative of the expected value of the second derivative matrix. Notice that E(X 0 u) = 0; E(u 0 u) = T 2 so the expected value of the nal second derivative can be written: I() = T 2 4 T 2 6 = T 2 4 LL() 0 ) = I(; 2 ) 1 = T 4 = T X 0 X 0 T (X 0 X) T This gives the lower bound for the Variance-covariance matrix for ; 2 : Notice that and 2 are independent, their covariances are zero. But there will be non-zero covariances between the elements of : We can put the ML estimates into the Log-likelihood function, to get the Maximised Log-Likelihood, MLL, reported by most programs MLL = = = : T 2 log(2) T 1 2 log(b2 ) 2b 2 bu0 bu T 2 log(2) T T b 2 2 log(b2 ) 2b 2 T 2 (log(2) + 1) T 2 log(b2 ) apart from the constant this is just the negative of half the sample size times the log of the ML estimate of the variance Properties of the ML estimators in the LRM General asymptotic properties of ML estimators were discussed above, to derive the speci c small sample properties of the LRM estimators we will use two results repeatedly. Firstly, linear functions of normally distributed variables are normally distributed. If y is IN(; 2 ) then a + by is N(a + b; b 2 2 ): The multivariate generalisation of this is that if the T 1 vector Y N(M; ), where M is T 1; is a T T variance covariance matrix. Then for given A and B of order K 1 and K T : A + BY N(A + BM; BB 0 ): (3.16) 35