Bayesian Econometrics - Computer section Leandro Magnusson Department of Economics Brown University Leandro Magnusson@brown.edu http://www.econ.brown.edu/students/leandro Magnusson/ April 26, 2006 Preliminary Version Abstract This material is used as a reference for the computer exercises of the Bayesian econometrics course taught by Professor Tony Lancaster. It covers a variety of models (linear regression models, limited depended variable models (logit, probit and Tobit), etc). The Bayesian computation is done in BUGS using R as its interface. Both softwares can be download freely from their respective web pages: http://www.mrc-bsu.cam.ac.uk/bugs/ and http://www.r-project.org/. w q 1
1 First Order Auto Regressive Model Model: The AR(1) and ARMA(1,1) model are: y t = ρy t 1 + ε t, ε t N (0, τ) (1.1) y t = ρy t 1 + ε t, ε t = u t θu t 1, {u t } T t=0 is i.i.d., u t N (0, σ) (1.2) where 1 < ρ < 1. The Durbin-Watson statistic of the AR(1) errors is: dw = T t=2 (ε t ε t 1 ) T t=2 (ε2 t 1 ) ε t = y t ρy t 1 (1.3) Exercise 1. (Based on Example 2.15, page 95, Lancaster (2004)) In R, generate a sequence of AR(1) and a sequence of ARMA(1,1) observations with the same initial value. Let s assume the both sample are AR(1), the prior distribution of ρ is improper and τ is known. (a) What is the likelihood for the AR(1) model (p(y ρ))? (b) Generate ρ from its posterior distribution (ρ N (ˆρ, T t=1 y2 t 1)). (c) Generate a sequence of y rep from its predictive distribution and compute diff(ρ, y rep, y obs ) = dw(ρ, y rep ) dw(ρ, y obs ) (d) Repeat items (b), (c) n times separately for AR(1) and ARMA(1,1). (e) Plot the distributions of df(ρ, y rep, y obs ) and draw your conclusions. 2 Linear Regression Models 2.1 Independent, normal, homoscedastic errors Database: See Mankiw et al. (1992). Model: Solow Model without human capital: with human capital: Y t = K α t (A t L t ) 1 α Y t = K α t H β t (A t L t ) 1 α β where L t = L 0 exp(nt) and A t = A 0 exp(gt). Consider that A 0 = a + ɛ, where a is constant and ɛ is a country-specific shock. The basic empirical specifications for the Solow model are: ( ) Y ln = a + α L 1 α ln(s k) α ln(n + g + δ) + ɛ (2.1) 1 α ( ) Y α ln = a + L 1 α β ln(s β k) + 1 α β ln(s h) α + β ln(n + g + δ) + ɛ (2.2) 1 α β where s k is the capital savings rate, s h is the human capital savings rate, n is the population growth rate, g is technical growth rate, δ depreciation. Mankiw et al. (1992) assume that g + δ = 0.05. 2
Exercise 2. (a) Consider the following unrestricted specification ( ) Y ln = a + b 1 ln(s k ) + b 2 ln(n + g + δ) + ɛ L What is the likelihood for the above model? How would you choose the priors? (b) Write and run a BUGS program for the linear regression model. Do separately for nonoil, intermediate and oecd countries. Compare your answers with the maximum likelihood estimates. (c) Propose and run a test for the coefficients b 1 and b 2 for each group of countries. How could you test the hypothesis about the returns to scale? (d) Include human capital and repeat parts (a), (b) and (c). 2.2 Heteroscedasticy Database: See Yatchew & No (2001) Model: Consider the log linear form of the demand of gasoline: log(gas) = β 0 + β 1 log(price) + β 2 log(income) + u (2.3) In matrix notation, the OLS terms are: v = N k (the degrees of freedom), ˆβ = (X X) 1 X y and s 2 = (y X ˆβ) (y X ˆβ) v. Assuming normality and homoscedasticity of the residuals, the likelihood function is: 1 p(y β, τ) = 1 (2π) N 2 [ {τ 1 2 exp τ 2 (β ˆβ)X X(β ˆβ) ]} ( {τ v 2 exp τv ]} 2s 2 If the residuals have the same variance we have: E [ u 2 σ 2] 2 = 2σ 4 where σ 2 is the variance (σ 2 = 1 τ ). Therefore a simple statistic to verify if the residuals have equal variance is: N T (β, σ 2 i=1 ) = ((y i x i β) 2 σ 2 ) 2σ 4 N Student linear model Take a look at Lancaster (2004), chapter 3, pages 159-162. Assume that: u i X, β, τ, λ i N (0, τλ i ) where {λ i } N i=1 is independently gamma distributed with mean 1 and scalar hyperparameter d. Multiplying the density of u i X, β, τ, λ i by the density of {λ i } N i=1 and integrating out λ s one may show that u i X, β, τ follows a t- distribution with mean 0 and degrees of freedom equals to d. 1 For a formal derivation see Koop (2003) 3
Exercise 3. (a) Assuming that the errors have the same variance, suggest a natural conjugate prior for this model and derive the posterior distribution of β and τ. Write a program in R for sampling of the posterior distribution of β and τ. (b) Simulate β and σ 2 from the joint distribution and evaluate T (β, σ 2 ). Repeat nrep times and the draw the histogram of the test statistic. What are you conclusions? (c) Write down a BUGS program for the student linear model. Explore how the posterior inferences about β change by using different values of d. Remember that low values of d approximates the t-distribution to a Cauchy (have tails) and large values of d approximates to a normal distribution. 3 Limited Dependent Variable Models 3.1 Logit Database: See Slonim & Roth (1998). Definition: (Ultimatum Game) The ultimatum game consists of two players bargaining over the amount of money which it will be called pie. One player, the proposer, proposes a division of the pie, and the second player, the responder, accepts or rejects it. If the responder accepts, each player earns the amount specified in the proposal and if the responder rejects, each player earns zero. At perfect equilibrium the proposer receives all or almost all of the pie. Model: Rejection behavior Prob (R = 1) = Λ (a + b 0 Of + b M pm + b H ph) (3.1) where Λ is the logistic distribution function, R equals 1 if the offer is reject and 0 otherwise, Of is the proportion of the pie offered (from 0 to 49,5%), pm = 1 if stakes are medium and ph = 1 is stakes are high. Exercise 4. (a) What is the likelihood for the above model? How would you choose the prior? (b) Write and run a BUGS program for the logit model. Study the convergence of the sampler (try 3000 interactions burning 1000 and 10000 interactions burning 8000). Compare your answers with the maximum likelihood estimates. (c) Evaluate the rejection probability at various offer values: of = [ ] 0.1 0.2 0.3 0.4 0.5 ; do this separately for the low, medium, and high stakes conditions. (d) Does the level of the stakes affect the rejection probability. Propose a test and compute it. What is your conclusion? 4
3.2 Tobit Database: See Mroz (1987) Model: (Labor supply of Married Women) The reduced form of a woman s labor supply is given by: 2 h i = a 0 + a 1 Y i + a 2Z i + u i where h i is the ith woman s hour of work during a given year, Y i is a measure of other income received by the household, Z i is a set of control variables which includes her age, the number of children less than six years old, the number of children between ages of five and nineteen her years of schooling, her experience in years and the square of experience; and u i is a stochastic term which is assumed to be normally distributed. Define the labor force participation as d i = 1 if h 1 > 0 and d i = 0 otherwise. The Tobit likelihood is given by: p(h a s, τ) = N i=1 [ Φ ( τ.5 (a 0 + a 1 Y i + a 2Z i ) ) 1 d i τ.5 φ ( τ.5 (h i (a 0 + a 1 Y i + a 2Z i )) ) d i ] (3.2) where Φ and φ are the normal distribution and density functions, respectively. Sampling with data augmentation: Suppose we could observe h the potential hours of work for every married woman which may also be negative. Thence p(a s, τ h ) = p(a s, τ h, h) is the posterior for a s and τ derived from the linear normal likelihood model. We do not observe h but its conditional density p(h h = 0, a s, τ) is normally left truncated. Therefore we have the following Gibbs Sampler Algorithm with the parameter set augmented by h for censored observations: 1) Choose a initial value for h, a s and τ; 2) Sample a s and τ from p(a s, τ h ); 3) Sample h from the truncated normal p(h h = 0, a s, τ); 4) Repeat 2 and 3 nrep times. Exercise 5. (a) Write a BUGS program for the Tobit model. (b) Write a R program using data augmentation. (In the library msm provides the function tnorm(µ,σ 2 )) truncated normal 2 The woman s wage rate, the endogenous explanatory variable, is assumed to be a linear function of Y i and Z i. 5
4 Instrumental Variable Model 4.1 Recursive Equation Model Database: See Romer (1993) Model: In a closed economy, the Lucas model for the relationship between output and inflation is: y = y + β(π π e ) (4.1) where y is actual output, y the natural rate, π inflation, and π e expected inflation, and where β > 0. Unanticipated monetary shocks affect both prices and real output. The policy-maker s objective (welfare) function is: W = 1 2 π2 + γy (4.2) where γ > 0. The policy-maker chooses inflation π. Assuming rational expectations, i.e. private agents known optimization problem of the policy maker, the equilibrium is π = π e = γβ and y = y : inflation is positive and output is at natural rate (suboptimal equilibrium). Openness affects the output-inflation trade-off (increased openness raises de amount of inflation associated with a given expansion of domestic output; that is, it reduces β) and the benefit of higher output relative to the cost of higher inflation (γ is decreasing in the degree of openness). Romer (1993) considers the following linear relation between inflation and output: log π = b 0 + b 1 I Y + b 2 log y + b3 dd + b4 rd (4.3) where log π is the log of the average inflation 73-93, I Y is the average share of imports in GDP or GNP, log y is the log of real income per capita, dd are dummies for alternative measures of openness and inflation and rd are regional dummies. A priori we expect that b 1 < 0. According to the author, I Y is endogenous since the adoption of protectionism policies that benefit some interest groups leads to larger budget deficits and therefore inflation. As instrument he uses land area (in logarithms). Exercise 6. (a) How would you translate the instrumental variable model into a recursive system model? likelihood and the priors for the recursive system. Define the (b) Write a Bugs program for the recursive model. Could you propose an exogeneity test for openness? 4.2 Multinomial Approach (Bayesian Bootstrap) Database: See Romer (1993) Model: Another approach for the previous model is to consider only moment conditions. Under mean independence we have: E [Z (y Xβ)] = 0 6
The likelihood is derived from the multinomial distribution and its natural conjugate prior is the dirichlet distribution. For more details, take a look at Lancaster (2004) chapter 3, section 3.4, pages 141-147. Exercise 7. (a) Sample β from its posterior and compare to the posterior distribution of the previous exercise. (Remember that you can sample β from β = [Z GX] 1 Z Gy where G is a diagonal matrix whose components are i.i.d exponentially distributed) (b) How would you propose and exogeneity test in this context? 7
References Koop, G. (2003), Bayesian Econometrics, first edn, Wiley. Lancaster, T. (2004), An Introduction to Modern Bayesian Econometrics, first edn, Blackwell Publishing. Mankiw, N. G., Romer, D. & Weil, D. N. (1992), A contribution to the empirics of economic growth, Quartely Journal of Economics 107(2), 407 437. Mroz, T. A. (1987), The sensitivity of an empirical model of married women s hours of work to economic and statistical assumptions, Econometrica 55(4), 765 799. Romer, D. (1993), Openness and inflation: 108(4), 869 903. Theory and evidence, Quartely Journal of Economics Slonim, R. & Roth, A. E. (1998), Learning in high stakes ultimum games: an experiment in the slovak republic, Econometrica 66(3), 569 596. Yatchew, A. & No, J. (2001), Household gasoline demand in canada, Econometrica 69(6), 1697 1709. 8