Bayesian IV: the normal case with multiple endogenous variables

Transcription

1 Baesian IV: the normal case with multiple endogenous variables Timoth Cogle and Richard Startz revised Januar 05 Abstract We set out a Gibbs sampler for the linear instrumental-variable model with normal errors and normal priors, and we show how to compute the marginal likelihood. Introduction In this note we present a Gibbs sampler for a Baesian instrumental-variable estimator with normal errors and priors, together with an algorithm for computing the marginal likelihood. While the seemingl unrelated regression formulation of instrumental variables is nonlinear in its parameters, Gibbs sampling is possible if parameters are blocked in the right wa. For models with a single endogenous regressor, textbook expositions of Gibbs sampling include Lancaster (00) and Rossi, et al. (005). Here we describe a Gibbs sampler for a model with multiple instruments and endogenous right-hand variables. The literature on substantive issues about Baesian estimation of instrumental variable models is large. Two good places to start are Geweke (99) and Kleibergen and Zivot (00). Conle et. al. (008) allow for a semi-parametric model for the error terms in the single endogenous regressor model of Rossi et. al. (005). Hooerheide at. al. (00) propose a natural conjugate prior for the IV problem. For an Tim Cogle: Department of Economics, New York Universit, 9 W. th St., FL, New York, NY 00, tim.cogle@nu.edu. Dick Startz: Department of Economics, North Hall, Universit of California, Santa Barbara, CA 90, startz@ucsb.edu. Matlab implementations of the Gibbs sampler and marginal likelihood calculator are available from the second author. Our thanks to Peter Rossi, Galli Alain, and Gerdie Everaert for helpful comments.

2 exposition of MCMC with nonnormal priors, see Gao and Lahiri (000). Hooerheide at. al. (00b) discuss the properties of conditional distributions for IV estimators in MCMC settings. Zellner et. al. (0) discuss an alternative to MCMC procedures with a at prior and a single endogenous regressor. Nonlinear SUR formulation and Gibbs sampling Consider a model with a single structural equation = X + "; () where is an N vector of observations on a dependent variable, X is an N k matrix of endogenous regressors, is a k vector of parameters, and " is an N vector of residuals. The rst-stage equations are where Z is an N q matrix of instruments, X = Z + v; () is a q k matrix of coe cients, and v is an N k matrix of errors. Exogenous variables in the structural equation ma be included in both the X and Z matrices without loss of generalit. So that the model satis es an order condition for identi cation, we assume that q is at least as large as k: Substituting equations () into equation () gives a restricted reduced form, = Z + v 0 ; () where v 0 = " + v. Together, equations () and () ield a seemingl-unrelated regression with nonlinear cross-equation parameter restrictions, x 5 = (I k+ Z) 5 + x k where x i is the ith column of X and j is the jth column of : k v 0 v 5 ; () Let u n = [v n0 ; v n ; ; v nk ] 0 represent the vector of SUR errors for observation n: We assume that u n is i.i.d normal with mean zero and covariance : Although u n is seriall uncorrelated, its elements can be correlated contemporaneousl. Hence need not be diagonal and tpicall won t be.

3 Gibbs sampling Gibbs sampling is accomplished in three blocks: j ; ; j ; ; and assume that parameters are independent a priori across blocks, j; : We p(; ; ) = p()p( )p(); (5) Marginal priors are speci ed so that the conditional posterior for each block has a convenient form. The data are denoted D = (; X; Z).. Block : p(j ; ; D) Conditional on ( ; ; D); the residuals u = (v0; 0 v; 0 vk 0 )0 in equation () are observable. We assume the prior p() is inverse Wishart with scale matrix and degrees S of freedom df k +. Since the conditional likelihood function is Gaussian, the posterior is also inverse Wishart with scale matrix S =S +uu 0 and degrees of freedom DF = df + N. Hence can be drawn b sampling from a IW ( S; DF ) distribution.. Block : p(j ; ; D) Since is known, the restricted reduced form (equation ) can be written as Z Z k v 0 x Z 5 = v 5 ; () x k Z k 0 0 or, letting ^x i = Z i be the analog of the tted values from the rst-stage regression given ; x ^x x k ^x k 5 = ^x ^x k v 0 v 5 : () Since the regressors in rows through k are zero, the residuals in those equations are conditionall observable. Given joint normalit of the errors, the conditional mean and variance of v 0 can be found b projecting v 0 onto v = (v ; ; ): Partition as = 0 v0 v ; (8) vv0 vv

4 where 0 = var(v 0 ); vv = var(v); and v0 u = cov(v 0 ; v): Then v 0 is conditionall normal with mean and variance E(v 0 jv ; ; ) = v vv vv0 ; (9) var(v 0 jv ; ; ) = 0 v0 v vv vv0 : (0) The residual from this projection, 0 v 0 E(v 0 jv ; ; ); has conditional mean zero and is conditionall independent of (v ; ; ): After subtracting E(v 0 jv ; ; ) from both sides of the rst equation in (), we nd E(v 0 jv ; ; ) x ^x 5 = x k ^x k ^x ^x k v 5 : () Because the transformed residual 0 is independent of (v ; ; ); the k bottom rows are irrelevant for estimating : Assume a normal prior p() = N( 0 ;V ): Let V = 0 = V V is 0 + ^X 0 ( E(v 0 jv ; ; )). Block : p( j; ; D) + ^X 0 ^X and 0 V 0 : Then the conditional posterior p(j ; ; D) = N( ; V ): () Since is known, the restricted reduced form (equation ) can be written as a seemingl unrelated regression that is linear in the unknown parameters, v x = Z 5 + v I k Z 5 : () x k k Call the left-hand side variables ~ i and the right hand side variables ~ X i. Assuming a normal prior p(vec( )) = N( 0 ;V ); the conditional posterior p( j; ; D) is normal with variance V = + V P k+ ~ i= Xi 0 Xi ~ ; () To verif the equivalence with (), write 0 Z longhand and rearrange terms.

5 and mean = V V P k+ ~ i= Xi 0 ~ i : (5) Marginal Likelihood Calculation The marginal likelihood of the model can be computed b appling Chib s (995) method. Let be the three blocks f; is ; g. The basic marginal likelihood identit p(; XjZ) = p(; Xj ; Z)p( ) p( : () jd) While the identit in equation () holds for an value of, we want to be sure that is positive de nite and should also recognize that, since the frequentist estimate of does not have a nite rst moment in the just-identi ed case, the MCMC mean of ma not be a wise choice for. As a recommendation, let = median( (s) ) and = median( (s) ); where the superscript (s) indicates draws from the MCMC distribution. Then, using the residuals e i ( ; ), let = P N i= e ie 0 i=n: The values of the likelihood and prior in equation () can be computed directl. Note that the log likelihood is given b log p(; Xj ; Z) = N(k + ) The third term simpli es as follows, log() N log det P N i= e0 i( ) e i : P N i= e0 i( ) e i = P N i= tr e0 i( ) e i ; () = tr ( ) P N i= e ie 0 i ; = tr (NI k+ ) ; = N(k + ): Hence log p(; Xj N(k + ) ; Z) = ( + log()) Break up the posterior densit for as follows. N log det : (8) p( ; ; jd) = p( jd) p( jd; ) p( jd; ; ): (9) The order of parameters in equation (9) re ects computational e cienc rather than anthing more fundamental. The evaluation of (9) proceeds in three steps. If the 5

6 draws from the MCMC are indexed (s) = ; S, then p(vec( )jd) S P S (s)= f N(vec( ); (s) ; V (s) ): (0) where f N () is the densit of a normal pdf and (s) ; V (s) in Block above in the original MCMC. Calculation of p( jd; have alread been calculated ) the second term in (9) requires a second run of the MCMC procedure outlined above, except that Step is omitted and draws of where (s) ; V (s) replaces. Index the draws of this auxiliar sampler b (s ) = ; S : Then p( P jd; ) S S (s )= f N(vec( ); (s) ; V (s ) ); () are calculated as in Block above. The conditional posterior p(jd; ; nal term in (9) can be evaluated as ) is simulated in Block above. Hence the p( jd; ; ) = f IW ( jdf; S); () where the scale matrix S is evaluated at ; ;as in Block above. Collecting the results of (0), (), and () and multipling them together delivers the denominator of (). Dividing the posterior kernel p(; Xj ; Z)p( ) b the result delivers the marginal likelihood p(; XjZ). 5 References Chib, S., 995. Marginal Likelihood From the Gibbs Output, Journal of the American Statistical Association 90, -. Conle, T., C. Hansen, R. McCulloch, and P. Rossi, 008. A semi-parametric baesian approach to the instrumental variable problem, Journal of Econometrics,, Geweke, J., 99. Baesian reduced rank regression in econometrics. Journal of Econometrics 5,. Gao, C. and K. Lahiri, 000. MCMC algorithms for two recent Baesian limited information estimators. Economics Letters, -. Hoogerheide L., Kleibergen, F., and van Dijk, H. 00 Natural conjugate priors for the instrumental variables regression model applied to the Angrist Krueger data. Journal of Econometrics 8, 0.

7 , Kaashoek, J., and van Dijk, H. 00b. On the shape of posterior densities and credible sets in instrumental variable regression models with reduced rank: An application of exible sampling methods using neural networks. Journal of Econometrics 9, Kleibergen, F., and Zivot, E., 00. Baesian and classical approaches to instrumental variable regression. Journal of Econometrics, 9. Lancaster, T., 00. An Introduction to Modern Baesian Econometrics, Blackwell Publishing. Rossi, P., G. M. Allenb, and R. McCulloch, 005. Baesian Statistics and Marketing, John Wile & Sons. Zellner, A., T. Ando, N. Basturk, L. Hoogerheide, and H. van Dijk, 0. Instrumental variables, errors in variables, and simultaneous equations models: applicabilit and limitations of direct monte carlo. Tinbergen Institute Discussion Paper 0- /.