Macroeconomics and Model Uncertainty

Transcription

1 Macroeconomics and Model Uncertainty William A. Brock and Steven N. Durlauf October 12, 2004 JEL Classification Codes: C52, E6 Keywords: model uncertainty, robustness, stabilization policy, ambiguity aversion, design limits

2 Macroeconomics and Model Uncertainty Abstract This paper provides some reflections on the new macroeconomics of model uncertainty. Model uncertainty is argued to have important positive and normative implications for macroeconomic analysis. We provide a general discussion of the relationship between macroeconomic models that assume a specific model and those that do not and provide some examples from the existing literature. Our discussion also provides some suggestions on directions for future research in this area. William A. Brock Department of Economics University of Wisconsin 1180 Observatory Drive Madison, WI wbrock@ssc.wisc.edu Steven N. Durlauf Department of Economics University of Wisconsin 1180 Observatory Drive Madison, WI sdurlauf@ssc.wisc.edu

3 The owl of Minerva begins its flight only with the onset of dusk. G. W. F. Hegel, Elements of the Philosophy of Right I. Introduction This paper provides some reflections on the new macroeconomics of model uncertainty. Research on model uncertainty represents a broad effort to understand how the ignorance of individuals and policymakers about the true structure of the economy should and does affect their respective behaviors. As such, this approach challenges certain aspects of modern macroeconomics, most notably the rational expectations assumptions that are generally made in modeling aggregate outcomes. Our goal is to explore some of the most interesting implications of model uncertainty for positive and normative macroeconomic analysis. Our discussion will suggest directions along which work on model uncertainty may be fruitfully developed; as such it is necessarily somewhat speculative as we cannot say to what extent these directions are feasible. Our interest in model uncertainty is certainly not unique; in fact, within macroeconomics, studies of model uncertainty have become one of the most active areas of research. Much of this new work was stimulated by the seminal contributions of Hansen and Sargent of which (2001,2003a,2003b) are only a subset of their contributions. Hansen and Sargent s work explores the question of robustness in decisionmaking. In their approach, agents are not assumed to know the true model of the economy, in contrast to standard rational expectations formulations. Instead, agents are modeled as possessing the more limited information that the true model lies within a model space that is defined as all models local to some baseline. This approach in turns leads to their adoption of minimax methods for decisionmaking; an agent chooses an action that works best under the assumption that the least favorable model (for the agent) in the model space is in fact the true one. This approach has been shown to have important implications for the equilibrium trajectory of macroeconomic aggregates. Our discussion will consider both local and nonlocal forms of model uncertainty. Thus for much of our analysis, rather than consider spaces of models defined relative to 1

4 some baseline, we consider the incorporation of model uncertainty into environments where the potential models are quite different. As such, this approach will be complementary to robustness analyses. We do not claim our approach is better than the robustness one; assumptions about the type of model uncertainty that is present cannot be assessed outside the objectives of the researcher and the particular economic environment under study. Model uncertainty has also become a major area of research in statistics and econometrics. Draper (1995) is a conceptual analysis that lays out many of the implications of model uncertainty for data analysis. As Draper observed, many of the implications of model uncertainty for empirical practice are present in Leamer (1978) and had (and still have) yet to be fully integrated into empirical work. One area where statistical methods that account for model uncertainty are now well developed is the determination of regressors in linear models. Raftery, Madigan and Hoeting (1997) and Fernandez, Ley, and Steel (2001a) present a range of methods for implementing empirical analyses that account for uncertainty in control variable choice in evaluating regression coefficients. These tools have been used in a range of empirical contexts in economics, especially economic growth; contributions include Brock and Durlauf (2001), Brock, Durlauf and West (2003), Doppelhofer, Miller, and Sala-i-Martin (2000), Fernandez, Ley and Steel (2001b). This work has shown that many of the claims in the empirical growth literature concerning the predictive value of various aggregate variables in cross-country growth regressions does not hold up when one accounts for uncertainty in the control variables that should be included in the regression. On the other hand, this approach has also strengthened certain claims; for example, the negative partial correlation between initial income and growth, also known as β convergence, has proven strongly robust with respect to variable choice, see Durlauf, Johnson, and Temple (2004) for a summary of this evidence. The growing class of theoretical and empirical studies of model uncertainty does not lend itself to ready summary. Our intent in this essay is to provide a discussion of the broad themes that one finds in this literature. Section 2 of the paper describes the relationship between model uncertainty and total uncertainty with respect to predicting an 2

5 unobserved variable such as a future macroeconomic aggregate. Section 3 discusses some implications of model uncertainty for macroeconomic theory. Section 4 discusses the relationship between model uncertainty and policy evaluation. Section 5 provides an extended example of how model uncertainty can be incorporated into an abstract modeling framework and how its presence affects the conclusions one draws the model. Section 6 offers some conclusions. 2. Model uncertainty and total uncertainty a. general ideas In this section, we consider the ways in which model uncertainty affects the formal description of uncertain economic outcomes. Our goal in this section is to integrate model uncertainty into standard characterization of uncertainty. At an abstract level, we work with θ =vector of outcomes of interest d =data representing history of the economy η =innovations that affect outcomes; these are independent of d m=model of the economy, element of model space M Conventional macroeconomic modeling may be abstractly understood as producing probability descriptions of the vector θ given a model of the economy, its history, and various shocks. Abstractly, one can think about a data generating process ( η, d) θ = m (1) 3

6 For our purposes, we assume that the probability measure describing η is known and that the model m is associated with a set of parameters β m. Hence, one can think of a given model producing probability statements about θ of the form ( m, m, d) µθ β (2) The uncertainty associated with this probability is fundamental as it is exclusively driven by lack of knowledge of η. As a description of the uncertainty about θ, (2) fails to properly account for the degree of uncertainty that a modeler faces at time t 1. The first level at which this is so is that the parameter vector describe uncertainty about outcomes via β m is generally not known. Hence, it is more natural to ( md, ) µθ. (3) In contrast to (2), the probabilities about outcomes described by (3) are conditioned only on the model and the available data. In standard (frequentist) practice, the data are used to construct estimates of β m. The analysis of the differences between (3) and (2) is an important question in the forecasting literature: West (1996) is a well known example of an analysis that considers how forecast distributions need to account for parameter uncertainty. For questions of using conditional probabilities to compare policies, on the other hand, parameter uncertainty is not commonly evaluated, exceptions whose findings call into question this standard practice include Giannoni (2001) and Onatski and Williams (2003). Model uncertainty extends this type of reasoning to eliminate the assumption that a modeler knows the form of the true data generating process. In other words, in evaluating model uncertainty, one attempts to construct µθ ( d ) (4) 4

7 How does one do this? The key insight, initially recognized in Leamer (1978) and subsequently developed in Draper (1995), is that model uncertainty may be treated like any other unobservable. Specifically, one thinks of the true model as lying in some space M over which probabilities are defined. The probability assigned to an element m of M may be thought of as the probability that model m is the true one. 1 Under the assumption that the space is countable, one uses Bayes rule to eliminate the dependence of (3) on a specific model, thereby producing (4). Formally, ( d) = ( d, m) ( m d) µθ µθ µ (5) m M This conditional probability introduces a new argument into macroeconomic analysis, µ ( md), the probability that a given model is the correct one given the available data. By Bayes rule, this probability in turn can be decomposed as ( md) ( dm) ( m) µ µ µ (6) where " " depends on two factors: means is proportional to. The probability of a model given the data thus µ (dm), the probability of the data given the model and µ ( m), the prior probability of the model. The first term summarizes the implications of the data for the relative likelihoods of each model; the second term embodies the prior information that exists on which model is correct. b. macroeconomics and forecast errors These calculations suggest a possible hierarchy for understanding the uncertainty that exists for macroeconomic outcomes, an argument made in Brock, Durlauf, and West 1 We will later address some issues that arise when none of the models in the space is the correct one. 5

8 t + 1 (2004). Let the vector denote a macroeconomic outcome of interest. Suppose that the data generating process for this vector depends on information available at time t,, and η +, information that is realized between t + 1 and t. In parallel to eq. (1), the data t 1 d t generating process for t + 1 is (, η, β ) t+ 1 m dt t+ 1 m = (7) for some model m. This data generating process suggests different levels of prediction errors. The intrinsic uncertainty that is associated with future data realizations produces prediction errors of the form ( β ) t+ 1 E t+ 1 dt, m, m (8) Parameter uncertainty leads to a definition of prediction errors when predictions are conditioned only on the model and data, i.e. ( ) E d m (9), t+ 1 t t+ 1 Finally, model uncertainty means that prediction errors should be defined by ( t ) E dt (10) t= c. implications for empirical practice The probability statements we have described provide predictive statements about unknowns when one only conditions on the data. As such, when applied to data analysis, they represent a different way of reporting empirical results. One way to think about the bulk of standard empirical practice is that it involves reporting results that are modelspecific whereas the analysis we have provided argues that these probability statements 6

9 should integrate out dependence on a particular model. What does this mean in practice? For a Bayesian, the answer is straightforward, since Bayesian statistics is based on the reporting of conditional probability statements that map knowns (the data) to unknowns t + 1 (θ or ). Accounting for model uncertainty is straightforward for Bayesians since one simply averages over the model-specific probabilities using posterior model probabilities. For frequentists, the answer is less clear since frequentist statistics reports probabilities of knowns (the data or transformations of the data, e.g. a parameter estimate) conditional on unknowns (the true parameter, for example.) That being said, one can certainly construct frequentist estimates that account for model uncertainty. Doppelhofer, Miller and Sala-i-Martin (2000) do this in the context of cross-country growth regressions; Brock, Durlauf and West (2003) argue that this is possible generally. Suppose that one has a frequentist model-specific estimate ˆm θ. In principle, one can always construct ˆ θ = ˆ θ pˆ (11) m M m m where pˆm is a proxy for a posterior model probability. One candidate for these proxies is a BIC-adjusted (normalized) likelihood statistic for each model. This approach may be interpreted as treating each model as equally likely. Alternatively, one can modify these adjusted likelihoods to allow for greater prior weight for certain models. Some authors have made proposals to do this. Doppelhofer, Miller, and Sala-i-Martin (2000), for example, argue that less complex models deserve greater prior weights. Brock, Durlauf, and West (2003) in contrast argue that economic theory should be used to account for similarities in the model space when assigning probabilities. This is an inadequately researched area. For a Bayesian, these frequentist calculations are incoherent as they mix probability statements about observables and unobservables. However, we do not regard this as an important criticism with respect to empirical work. We believe that what is critical in an empirical exercise is that a researcher establishes clear procedures for 7

10 analyzing data and that the procedures be interpretable relative to the objective of the data exercise. This pseudo-frequentist approach does this. Our discussion of empirical practice has been somewhat too facile in that it has not addressed the issue of how to interpret model averaging results when the true model is not an element of the model space; Bernardo and Smith (1994) call this the M-open case. There is an extensive literature on maximum likelihood estimation of misspecified models which suggests that for model weighting schemes such as one based on BICadjusted likelihoods, asymptotically, the averaging procedure will place all weight on the model that in a Kullback-Leibler sense best approximates the data, see Brock and Durlauf (2001) for more discussion. Further, our analysis has treated the model space as fixed. Of course, part of the evolution of empirical work is defined precisely by the evolution of the models under consideration. The question of how inferences will evolve in response to an evolving model space has yet to be researched as far as we know, despite its obvious importance. 3. Implications for macroeconomic theory In this section we identify some areas where model uncertainty may prove to have interesting implications for macroeconomic theory. In particular, we believe that model uncertainty raises a number of issues related to modeling individual behavior that can enrich standard macroeconomic formulations. a. expectations Model uncertainty calls into question the ways in which expectations are formulated in macroeconomic theory. The dominant rational expectation paradigm in macroeconomics assumes that agents possess common knowledge of the model of the economy in which they operate. Efforts to generalize allow for learning, well surveyed in Evans and Honkpohja (2001), commonly assume that the structure of the economy is known even if the values of the parameters of the structural model need to be learned. 8

11 Model uncertainty introduces issues of expectations formation that are quite different from the standard learning paradigm. One reason, recognized by Hansen and Sargent in (2003a,b) and elsewhere, is that model uncertainty raises questions of how one even defines rational expectations. My beliefs about the model of the economy will depend on what your beliefs are etc., which requires that I have information about your prior about models, etc. The incorporation of model uncertainty in macroeconomic modeling can have important implications for understanding the historical record. Cogley and Sargent (2004) explore the implications of theory uncertainty for understanding Federal Reserve behavior. They show that uncertainty on the part of the Fed in determining the correct model of inflation can explain monetary policy during the 1970 s even when evidential and theoretical support for the conventional Phillips curve had begun to disintegrate. If one treats parameter uncertainty as a form of model uncertainty (albeit of a localized nature), one can further this claim. Weitzman (2004) shows how a range of asset price puzzles, notably the equity premium puzzle and excess volatility of stock prices may be resolved using parameter uncertainty. Specifically, he shows that if economic actors did not know the parameters of the relevant dividends process for stock returns and instead, were learning about the parameters following Bayesian updating rules, these puzzles no longer exist. Weitzman s results are very striking because they so strongly reverse various claims about the inadequacy of dynamic representative agent general equilibrium modeling for understanding asset markets simply by considering how agents will behave in the presence of a narrow type of model uncertainty. Further, model uncertainty introduces two additional factors in the mapping of individual characteristics into equilibrium aggregate outcomes: the formation of initial beliefs and the ways in which these initial beliefs are updated. As indicated by our analysis, aggregate equilibria will depend on the distribution of ( m) µ across i. One can easily imagine cases where this distribution is nontrivial. For example, in transitions for high inflation to low inflation regimes, it seems natural to expect substantial heterogeneity in beliefs as to whether the change has occurred. This is the sort of event that available data cannot adjudicate. And unlike cases in which individuals have heterogeneous valuations of a common object, there is no market mechanism to facilitate i 9

12 aggregation, nor is it clear how agents would react to information about the beliefs of others. In the asset case, the implications of one agent s beliefs about the value on the beliefs of another presupposes a model of how the other agent s beliefs are determined, which itself is subject to model uncertainty. Nor is this question resolved by recourse to the interesting work on rational beliefs initiated by Kurz (1997) since the issue here is not what types of heterogeneity are consistent with long-term data properties. Rather our concern is with the heterogeneity of prior beliefs when the data are not sufficiently informative to impose long run restrictions, at least in the sense that the data do not reveal the true model of the economy. Beyond the issue of priors, model uncertainty suggests the importance of determining how agents acquire information about the economy. If an agent is confident that he knows the true model of the economy, he is presumably going to treat the costs and benefits of information acquisition differently than when the true model is unknown. Put differently, the value to an agent of superior information may depend on the degree of model uncertainty. This seems particularly relevant when the economy moves across regimes, which in our context may be interpreted as shifting across models. Brock and Hommes (1997) suggest some ways to think about this problem by modeling environments in which agents make discrete choices about what sorts of information to acquire before forming expectations. They show how these decisions will correlate across individuals and induce interesting aggregate price dynamics. Brock and Hommes (1997) introduce an information cost for the acquisition of structural rational expectations in contrast to a zero cost for acquisition of backwards-looking expectations. Once one introduces costly rational expectations versus costless backwards expectations, then it is natural to introduce a notion of past net profit to having used either expectational scheme. Each agent chooses an expectational scheme according to a discrete choice mechanism which puts more probability weight on an expectational scheme the higher the accrued profits measure for that scheme. Dynamics emerge naturally. To see how this mechanism can matter for macroeconomic contexts, consider inflation dynamics. If inflation has been stable for a long enough time, the net gain of acquiring rational expectations over simple backwards expectations will become 10

13 negative. Hence the economy eventually moves to a state where most people are choosing backwards looking expectations. This state will persist until substantial shocks hit the economy (either exogenous or endogenous) which will cause the net benefit measure to shift in favor of rational expectations. If the economy goes through another quiet period then the net benefit measure will shift in favor of backwards expectations and the whole story repeats. Such a perspective is very much what one would expect when agents experience shifting degrees of model uncertainty, in this case with respect to the prevailing inflation regime. If the economy goes through a period like that discussed in Sargent (1999a) where people got "fooled" by the government in the late 60's and early 70's most agents will switch to rational expectations under the Brock and Hommes (1997) mechanism. But after a sustained episode of price stability, more and more agents will rationally become inert, i.e. they will use backwards-looking expectations in order to economize on expensive rational expectations. We believe that generalizing the usual Taylor rule and related monetary policy analysis to settings where the dynamics of the distribution of prediction schemes across agents is endogenously determined by model uncertainty and which further takes into account the simultaneity between the monetary rule chosen and the predictor dynamics is an important direction for future research. It would be an approach to endogenizing at least part of the model uncertainty we have been discussing in this paper. It could also lead to an approach to defining which parts of the model space that are more probable or less probable conditional upon the economy's history. For example, the theory above suggests that one should expect most of the economy's mass to be on expectational schemes close to rational expectations following a history of recent price inflation. One reason why economists have found the rational expectations assumption appealing is that it imposes powerful discipline on modeling; in contrast, when expectations are not anchored by the logic of a model or some other well defined rule, then it is difficult to empirically falsify a model. Hence, the introduction of nontrivial model space priors and rules for information acquisition into macroeconomic modeling will require disciplining as well, in order to avoid a situation where any set of empirical findings may be explained by ad hoc choices of prior beliefs and updating behavior. 11

14 b. preferences The analysis in section 2 treated model uncertainty in a fashion that is equivalent to other forms of uncertainty faced by a policymaker. This assumption may not be appropriate. One reason for this is strictly positive: individual preferences do not necessarily treat model uncertainty in this fashion. Evidence of this claim comes from experiments such as those that generated the classic Ellsberg Paradox. The Ellsberg paradox comes from the following experimental observation. Suppose individuals are asked to compare bets in which there is a given payoff based on having correctly chosen the color of a ball drawn from an urn. In case 1, the urn contains an equal number of red and black balls. In case two, the proportions in the urn are unknown before the individual is allowed to choose the color. Individuals systematically prefer the bet where they know the urn proportions to the bet where they do not, despite the fact that there is no expected payoff difference between them. This sort of finding has helped motivate recent efforts to axiomatize Knightian uncertainty in the economic theory literature, cf. Gilboa and Schmeidler (1989) and Epstein and Wang (1994). In this work, Knightian uncertainty is represented as a situation in which probabilities cannot be assigned to a set of unknowns. The absence of these probabilities leads to minimax types of behavior on the part of an agent. One may think of this Knightian uncertainty as representing model uncertainty. These considerations suggest that model uncertainty affects preferences differently from other types of uncertainty. This idea may be formalized by considering how an agent makes a choice a from some set A in the presence of model uncertainty. Suppose that the agent s payoff is represented by a loss function l( a, θ ) which depends on the action and an unknown θ. If the agent is a standard expected loss minimizer, he will make a choice in order to minimize (, θ ) µθ ( ) l a d dθ (12) Θ where the conditional probability µθ ( d ) accounts for model uncertainty in a way symmetric to all other sources of uncertainty, as is done in the derivation of eq. (4) via 12

15 the argument described by eq. (5). How can these preferences account for ambiguity aversion? One way to do this is to follow the approach taken by Epstein and Wang (1994) and model preferences so that additional weight is placed on the least favorable model in M beyond the weight that is assigned in the expected loss calculation. Formally, preferences may be modeled as ( ) ( θ ) µθ ( ) θ m M ( θ ) µθ ( ) θ (13) (1 e) l a, d d + e sup l a, d, m d Θ Θ In this equation, e measures the degree of ambiguity aversion. If e = 0, then eq. (13) reduces to the expected loss calculation; if preferences with respect to model uncertainty. e = 1, then the policymaker exhibits minimax The minimax case has been studied in the growing literature on robustness in macroeconomic analysis, a literature which we have noted was launched by Hansen and Sargent. This approach assumes that the model space is local in the sense described in the Introduction. Beyond the work of Hansen and Sargent, standard references now include Giannoni (2002), Marcellino and Salmon (2002), Onatski and Stock (2002), Tetlow and von zur Muehlen, (2001). This work has yielded a number of valuable insights. For example, Giannoni (2002) provides a comprehensive analysis of how ambiguity aversion increases the aggressiveness of optimal stabilization policies in the sense that the magnitudes of the sensitivity of changes in control variables to lagged states may increase for a minimax policymaker when model uncertainty is present. Brock, Durlauf, and West (2003) provide some simple examples of this behavior. That paper notes that ambiguity aversion is different from risk aversion in the sense that it is a first-order phenomenon rather than a second-order one. What this means is that if one starts with an environment with no model uncertainty and then defines a O( ε ) term that represents model uncertainty, the effects are O( ε ) whereas for the classic Arrow-Pratt analysis of risk, the introduction of risk in a risk free environment produces effects of 2 O( ε ). The reason for this is that minimax preferences imply that the least favorable model is always assumed for the model space, hence the element that embodies 13

16 uncertainty always has non-zero expected value (modulo hairline cases) whereas in risk aversion analysis it is assumed that the risk term has an expected value of zero. As indicated for eq. (13), minimax preferences are a special case of a more general ambiguity aversion formulation. We would argue that for nonlocal model spaces it is important to model ambiguity aversion where e 1. One basic problem for nonlocal model spaces is that it is natural to worry that highly implausible (low prior probability) models will completely control decisionmaking. (For local model spaces, one typically does not think that the models will differ greatly in terms of prior probability.) Beyond this, there is a widespread belief that mnimax preferences are inappropriately risk averse; in fact Hurwicz (1951) proposed loss functions similar to (13) specifically to reduce the implicit risk aversion involved in the minimax formulation. For these reasons, we believe a valuable direction is the exploration of behavioral rules for preferences where e 1. A move away from minimax preferences towards less extreme forms of ambiguity aversion raises the issue of how e is determined. It seems unsatisfactory to treat the degree of ambiguity aversion as a deep parameter. We regard this as an important next step in research. Further, it seems important to study alternative ways account for model uncertainty in preference specification. Manski (2004) suggests the use of minimax regret for the study of treatment effects, an approach that may prove interesting in other contexts as well. However, minimax regret is not a panacea. Brock (2004) shows that minimax regret preferences will also exhibit significant sensitivity to implausible models, although not to the extent that is possible with minimax preferences. Minimax regret also suffers from the problem that it does not obey independence of irrelevant alternatives; specifically, the ordering between two models as to which is less may be affected under minimax by the presence of a third, see Chernoff (1954). We conjecture that further work on preferences will need to consider ways to avoid highly implausible models from dominating behavioral decisions. There are good reasons to believe that the evaluation of macroeconomic policy rules will be especially sensitive to the interaction of model uncertainty and policymaker preferences. For many macroeconomic models, it is possible for policy rules to induce instability, e.g. infinite variances for outcomes that one wants to stabilize. Hence a policy 14

17 rule that is optimal under one model may produce instability under other models. Instability naturally is naturally associated very high losses for a policymaker. This leads to the question of whether a rule should be rejected because it produces instability for models with very low posterior probabilities. The tradeoff of high losses and low model probabilities is the precise case where the deviations from expected loss calculations matter. This situation arises in Brock, Durlauf, and West (2004) which we discuss below. 4. Model uncertainty and policy evaluation How does model uncertainty affect the evaluation of macroeconomic policies? t t + 1 Suppose that a policymaker is interested at time in influencing the level of using a policy p, which lies in some set P. Assume that the policymaker s preferences are associated with the loss function ( ) l,, t+ 1 dt p (14) Note that we do not work with a loss function that extends over periods beyond t ; this is done strictly for convenience. In standard policy evaluation exercises, each policy is associated with an expected loss ( ( t+ 1 t ) t ) = ( t+ 1 t ) µ ( t 1 t ) E l, d, p d, m l, d, p + d, m. (15) The optimal policy choice is therefore defined by ( 1 ) µ ( 1 ) min p P l t, p, d t t p, d t, m + + (16) 15

18 where the conditional probability ( t 1 p, dt, m) µ + explicitly reflects the dependence of outcomes at t on the policy choice. Accounting for model uncertainty simply means replacing these formulas with ( ( + 1,, ), ) ( + 1,, ) µ ( 1 ) E l pd pd = l pd + d (17) t t t t t t t and ( 1 ) µ ( 1 ) min p P l t, p, d t t p, d + + t (18) For our purposes, the important observation to make about eqs. (17) and (18) is that there is no model selection involved. In response to ignorance of the true model of the economy, a policymaker does not first, evaluate potential models and then determine which best fits the data (subject to some penalty for model complexity, in order to ensure the selection rule does not reward overfitting) and second, evaluate policies conditional on that selection. Rather, policies should be evaluated according to their effects for each model, with the model-specific performances averaged. This may be seen explicitly when one rewrites (17) as follows: l ( t+ 1, pd, t) µ ( t+ 1 dt) = l ( t 1, pd, t) µ ( t 1 dt, m) + + m M (19) = m M (,, ) µ (, ) l p d d m t+ 1 t t+ 1 t In fact, one could go so far as to argue that for a policymaker, model selection has no intrinsic value. This is true even if the payoff function is generalized to be model dependent, i.e. the loss function is written as l(, d, p) t t 1. From the perspective of (17) -(19), it is reasonable to conclude that model selection is generally inappropriate for policy analysis. Conditioning on a model in essence means replacing the correct posterior model probabilities with a degenerate posterior (all probability assigned to the 16

19 selected model.) Such a substitution thus amounts to using incorrect posterior model probabilities. As argued in great detail by Draper, ignoring this source of uncertainty can lead to very inaccurate model assessments. Levin and Williams (2003) provide a very valuable analysis of the robustness of different monetary policy rules in the presence of model uncertainty. They evaluate simple policy rules using standard forward-looking, backwards-looking and hybrid (both forward- and backwards-looking) models. The exercise uses parameters that appear to be reasonable given previous studies and further weight all models equally, so in this sense their exercise is more theoretical rather than empirical. Interestingly, they find that no rule in the class they study 2 performs well across all models when a policymaker is only concerned about stabilizing inflation; on the other hand, when a policymaker places substantial weight on both inflation and output stabilization, robust rules do exist. The implications of model uncertainty for monetary policy rule evaluation is explored in an empirical context in Brock, Durlauf, and West (2004). In one exercise, they compare the stabilization properties of the classic Taylor rule with an interest rate rule in which current interest rates are determined by a linear combination of lagged interest rates, lagged inflation, and lagged output (measured as a deviation from trend). The optimized three-variable interest rate rule substantially outperforms the Taylor rule for most models (and indeed almost all models if their posterior probabilities are summed) in a model space characterized by uncertainty in lag structure for the IS and Phillips curves. However, the space also contains models for which the optimized rule induces instability where the Taylor rule does not. This suggests a rationale for the Taylor rule, robustness to instability, which would never have been apparent had model selection been done prior to policy evaluation. It turns out that the models for which instability occurs for optimized models versus the Taylor rule have posterior probabilities of less than 1%, which is why the specification of the policymaker s preferences with respect to model uncertainty is so important. 2 Levin and Williams (2003) study rules in which the nominal interest at time t + 1 depends on t levels of the interest rate, inflation, and output relative to trend; different rules correspond to different parameters for the time t variables. 17

20 5. An example To illustrate the power of the model uncertainty perspective, we summarize an analysis in Brock and Durlauf (2004b) that explores the role of local model uncertainty in the choice of optimal controls. Consider a general state equation for some outcome of t + 1 interest where a policymaker has available some control variable : u 3 t ( ) t+ 1 A L t But ξ t + 1 = + + (20) ξ + is an unobservable component of the equation and is assumed to have an invertible t 1 moving average representation ( ) ξ = ν t (21) t+ 1 w L + 1 A policymaker chooses a feedback rule of the form ut ( ) = F L t (22) We assume that the policymaker is interested in minimizing E + 2 t 1, the unconditional variance of t + 1. When this feedback rule is substituted into the state equation, one has the following representation of the state: ( ( ) ( ) ) t ( ) t 1 A L F L B w L ν t 1 + = + + (23) We assume that the policymaker is interested in minimizing E + 2 t 1, the unconditional variance of t + 1. In order to understand how different feedback rules affect 3 See Kwaakernak and Sivan (1972) for a detailed description of linear feedback systems of this type. 18

21 E +, it is useful to contrast the system we have described to one where there is no 2 t 1 control, i.e. ( ) NC NC t+ 1 A L t ξ t + 1 = + (24) The variable NC t + 1 is known as the free dynamics of the system. Stabilization policy may be thought of as transforming the free dynamics in such a way as to minimize the t + 1 unconditional variance of. In order to understand the properties of this transformation, we work in the frequency domain. One advantage of the frequency domain is that it allows on to think about the variance components of fluctuations that occur for different cycles. In terms of notation, for any lag polynomial i ( ) C e ω ij = Ce ω j j= C( L) denote its Fourier transform. One can show (cf. Brock and Durlauf (2004b) for a complete argument) that let 2 ( ) ( ) 2 π t+ 1 π NC E = S ω f ω dω (25) where S ( ω) = iω iω iω ( e A( e )) BF( e ) (26) The function S ( ω ) is known as the sensitivity function and illustrates an important idea: a feedback rule transforms the spectral density of the free dynamics by altering the variance contributions of the individual frequencies. The effect of the feedback rule is produced by its effect on the sensitivity function. One can therefore think of the design problem for controls as asking how a policymaker will want to transform the spectral density of the free dynamics. 19

22 From the perspective of designing optimal stabilization policies, it is therefore essential to know what constraints exist on the possible sensitivity functions that a policymaker may implicitly shape by the choice of a feedback rule. An important result in the control theory literature, known as the Bode integral constraint characterizes these restrictions. Under the assumption that the system has no explosive roots, the constraint may be written (using a discrete time version due to Wu and Jonckheere (1992)) as π 2 ( S( ω) ) ln dω = 0 (27) π This form implies that a policymaker cannot set S ( ω ) 2 < 1 for all frequencies; hence all stabilization policies necessarily tradeoff higher volatility at some frequencies for lower volatility at others. This is a fundamental tradeoff that all policymakers face as any sensitivity function that a policymaker wishes to produce via the choice of feedback rule must fulfill (27). This constraint has powerful implications; for example, it indicates how a policy designed to minimize low (high) frequency fluctuations will necessarily exacerbate some high (low) frequency fluctuations. See Brock and Durlauf (2004a) for a discussion of its implications in a range of macroeconomic contexts. The Bode integral constraint allows one to prove an extremely interesting result concerning model uncertainty and robustness. Suppose that the spectral density of the innovations is not known, but rather that it lies in some set defined around a baseline spectral density ( ) f ξ ω ( ( ) ( )) 2 2 fξ ω fξ ω dω ε π (28) π Eq. (27) implies that a policymaker does not know the true time series structure of the state equation. This form of model uncertainty is also studied in Sargent (1999b) and is one way of capturing Friedman s (1948) concern about long and variable lags in the relationship between monetary policy and aggregate activity. One can show that for a 20

23 policymaker who wishes to miminize those in the set defined by (28) is E +, the least favorable spectral density among 2 t 1 ( ω) ( ω) 1 f * ξ fξ ( ) fξ ( ) o 1 ( ) ω = ω + ε + ε (29) f ξ ( ) 1/2 π where for any function g ( ω ), ( ω) π ( ω) ( ω) g = g g d ω. Equation (29) defines the spectral density that a policymaker will assume is true one when choosing a feedback rule. As such, the policymaker behaves in a way that follows Wald s (1950) idea that minimax preferences may be interpreted as the Nash equilibrium of a noncooperative game, an idea that is critical in the development of the modern macroeconomic robustness research program. To understand the least favorable spectral density for the policymaker, notice that relative to the baseline, the term f f ξ ξ 1 ( ω) is large (small) when f 1 ξ ( ) ( ω) ω is small (large). This means that when there is model uncertainty the least favorable model is the one that flattens the spectral density, i.e. shifts the spectral density closer to white noise. This makes intuitive sense as a policymaker can never minimize the effects of white noise shocks when he is constrained to follow policy rules that set the policy variable before the white noise shocks are realized, as occurs with rules of the form (22); put differently, ν + is the component of t 1 t + 1 that cannot be affected by the choice of feedback rule. We conjecture that this general result may help to explain the presence of various deviations of aggregate time series from the behavior implied by the absence of model uncertainty. Intuitively, if an agent guards against the least favorable model by choosing the one that is closest to white noise, this suggests that the state variable will have more persistence in it than would be expected to occur if the true model were known. This type of finding may be relevant for empirical rejections of models because of violations of Euler equation or other first order conditions that imply some combination of time 21

24 series should be white noise; if agents react to model uncertainty by optimizing relative to the least favorable model in the way we have described, then it seems clear that violations of white noise conditions will be generated. 6. Conclusions The new macroeconomics of model uncertainty has shown itself to be a fruitful direction for theoretical and empirical research. In one sense, this direction would seem to be an inevitable one given the absence of resolution of so many macroeconomic debates, debates that include the role of incomplete markets in aggregate outcomes, the degree of price rigidity in the economy, the importance of increasing returns to scale in the aggregate production function in producing cycles and growth, etc. 4 An appealing aspect of this new program is the interconnected development of theory and empirics, a path of development that contrasts, for example, with the modern economic growth literature where theory and empirics have to a large extent evolved in parallel (see Durlauf, Johnson, and Temple (2004) for a defense of this claim). We therefore expect substantial progress in this area over time. 4 Of course, much of what we have discussed about model uncertainty applies to any branch of economics, not just macroeconomics. That being said, macroeconomics seems particularly likely to benefit from this perspective. 22

25 Bibliography Bernardo, J. and A. Smith, (1994), Bayesian Theory, New ork: John Wiley. Brock, W., (2004), Profiling Problems with Partially Identified Structure, mimeo, Department of Economics, University of Wisconsin. Brock, W. and S. Durlauf, (2001), Growth Economics and Reality, World Bank Economic Review, 15, 2, Brock, W. and S. Durlauf, (2004a), Elements of a Theory of Design Limits to Optimal Policy, The Manchester School, 72, Supplement 2, Brock, W. and S. Durlauf, (2004b), Local Robustness Analysis: Theory and Application, mimeo, Department of Economics, University of Wisconsin. Brock, W., S. Durlauf, and K. West, (2003), Policy Evaluation in Uncertain Economic Environments (with discussion), Brookings Papers on Economic Activity, 1, Brock, W., S. Durlauf, and K. West, (2004), Model Uncertainty and Policy Evaluation: Some Theory and Empirics, mimeo, Department of University of Wisconsin. Brock, W. and C. Hommes, (1997), A Rational Route to Randomness, Econometrica, 65, 5, Chernoff, H., (1954), Rational Selection of Decision Functions, Econometrica, 22, 4, 422, 443. Cogley, T. and T. Sargent, (2004) The Conquest of U.S. Inflation: Learning and Robustness to Model Uncertainty, mimeo, University of California at Davis. Doppelhofer, G., R. Miller, and X. Sala-i-Martin, (2000), Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach, Working Paper Cambridge, Mass.: National Bureau of Economic Research. Draper, D., (1995), Assessment and Propagation of Model Uncertainty, Journal of the Royal Statistical Society, series B, 57, Durlauf, S., P. Johnson, and J. Temple, (2004), Growth Econometrics, mimeo Department of Economics, University of Wisconsin. Epstein, L. and T. Wang, (1994), Intertemporal Asset Pricing Behavior Under Knightian Uncertainty, Econometrica, 62,

26 Evans, G. and S. Honkapohja, (2001), Learning and Expectations in Macroeconomics, Princeton: Princeton University Press. Fernandez, C., E. Ley and M. Steel, (2001a), Benchmark Priors for Bayesian Model Averaging. Journal of Econometrics, 100, 2, Fernandez. C., E. Ley and M. Steel, (2001b) Model Uncertainty in Cross-Country Growth Regressions, Journal of Applied Econometrics, 16, 5, Friedman, M., (1948), A Monetary and Fiscal Framework for Economic Stability, American Economic Review, 38, Giannoni, M., (2001), Robust Optimal Monetary Policy in a Forward-Looking Model with Parameter and Shock Uncertainty, mimeo, Federal Reserve Bank of New ork. Giannoni, M., (2002), Does Model Uncertainty Justify Caution? Robust Optimal Monetary Policy in a Forward-Looking Model, Macroeconomic Dynamics, 6, Gilboa, I. and D. Schmeidler, (1989), Maximin Expected Utility with Non-Unique Priors, Journal of Mathematical Economics, 18, Hansen, L. and T. Sargent, (2001), Acknowledging Misspecification in Macroeconomic Theory, Review of Economic Dynamics, 4, Hansen, L. and T. Sargent, (2003a), Robust Control and Economic Model Uncertainty. Book manuscript, Hoover Institution, Stanford University and forthcoming, Princeton University Press. Hansen, L. and T. Sargent, (2003b) Robust Control of Forward-Looking Models, Journal of Monetary Economics, 50, Hurwicz, L., (1951), Some Specification Problems and Applications to Econometric Models, Econometrica, 19, Kurz, M., (1997), Endogenous Economic Fluctuations: Studies in the Theory of Rational Beliefs, New ork: Springer-Verlag. Kwakernaak, H. and R. Sivan, (1972), Linear Optimal Control Systems, New ork: John Wiley and Sons. Leamer, E., (1978), Specification Searches, New ork: John Wiley and Sons. Levin, A. and J. Williams, (2003), Robust Monetary Policy with Competing Reference Models, Journal of Monetary Economics, 50,

27 Manski, C., (2004), Statistical Treatment Rules for Heterogeneous Populations, Econometrica, 72, 4, Marcellino, M. and M. Salmon, (2002), Robust Decision Theory and the Lucas Critique, Macroeconomic Dynamics, 6, 1, Onatski, A. and J. Stock, (2002), Robust Monetary Policy Under Model Uncertainty in a Small Model of the U.S. Economy, Macroeconomic Dynamics, 6, Onatski, A. and N. Williams, (2003), Modeling Model Uncertainty, Journal of the European Economic Association, 1, Raftery, A., D. Madigan, and J. Hoeting, (1997), "Bayesian Model Averaging for Linear Regression Models, Journal of the American Statistical Association, 92, 437, Sargent, T., (1999a), The Conquest of American Inflation, Princeton: Princeton University Press. Sargent, T., (1999b), Comment, in Monetary Policy Rules, J. Taylor, ed. Chicago: University of Chicago Press. Tetlow, R. and P. von zur Muehlen, (2001), Robust Monetary Policy With Misspecified Models: Does Model Uncertainty Always Call for Attenuated Policy?, Journal of Economic Dynamics and Control, 25, 6-7, Wald, A., (1950), Statistical Decision Functions, New ork: John Wiley. Weitzman, M., (2004), The Bayesian Family of Equity Non-Puzzles, mimeo, Department of Economics, Harvard University. West, K., (1996), Asymptotic Inference About Predictive Ability, Econometrica, 64, Wu, B.-F. and E. Jonckheere, (1992), A Simplified Approach to Bode s Theorem for Continuous and Discrete Time Systems, IEEE Transactions on Automatic Control, 37, 100,