Haavelmo, Marschak, and Structural Econometrics

Transcription

1 Haavelmo, Marschak, and Structural Econometrics James J. Heckman University of Chicago Trygve Haavelmo Centennial Symposium December 13 14, 2011 University of Oslo This draft, October 13, 2011 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 1 / 111

2 Causality and Policy Evaluation James J. Heckman Haavelmo, Marschak, and Structural Econometrics 2 / 111

3 An example of a structural relationship: Haavelmo, 1943, Econometrica Y outcome = X β + U causes All Causes model, (X, U) being the causes Fixing vs. Conditioning External manipulations: Variations in X that hold U fixed Do operator (Pearl) ceteris paribus notion Marshall (1890). ( ) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 3 / 111

4 OLS is the linear projection of Y on X. A linear approximation to E(Y X ) = X β + E(U X ) If E(U X ) = 0, OLS identifies β. E(Y X, U) = X β + U When is a model structural? If the coefficients β are invariant to a class of shifts in X, then ( ) is structural for that class of shifts (Haavelmo, 1943; Hurwicz, 1962). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 4 / 111

5 Causality & Policy Problems Prototypical Model Treatment vs. Policy Rejection Common Causes, Exclusions, and Nonexclusions Ob s e r v e d Z ( 4 ) ( 5 )? X Y ( 1 ) ( 3 )? ( 2 ) U Un o b s e r v e d Z U? James J. Heckman Haavelmo, Marschak, and Structural Econometrics 5 / 111

6 Causality & Policy Problems Prototypical Model Treatment vs. Policy Rejection Unidentified Model Considered by Marschak s Circle Ob s e r v e d Z ( 4 ) ( 4 ) ( 2 ) X Y ( 1 ) ( 3 ) ( 3 ) U Un o b s e r v e d Z U? Model not in general identified even if U = 0 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 6 / 111

7 Causality & Policy Problems Prototypical Model Treatment vs. Policy Rejection One Solution of the Cowles Group Z Z 1 2 ( 5) ( 6) ( 2) X Y ( 1) ( 4) ( 3) U 1 U 2 Z1 6= Z2, (Z1, Z2 ) (U1, U2 ), (U1, U2 ) arbitrarily correlated James J. Heckman Haavelmo, Marschak, and Structural Econometrics 7 / 111

8 First Application of the Cowles Structural Approach Marschak and Andrews (1944) Archetypal production function relating inputs to outputs. Idealized problem: Firm choosing inputs Profit π = P 0 X 0 P 1 X 1 P 2 X 2 X 0 output with price P 0 ; X 1 input with price P 1 ; X 2 input with price P 2 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 8 / 111

9 X 0 = α 0 X α 1 1 X α 2 2 ɛ }{{} f (Cobb Douglas) (1) measure of firm efficiency Firm Inputs For Interior Solution Case X 1 = (k)p X 2 = (η)p α 2 1 α α 1 α 2 1 α 1 P 1 α 2 1 α 2 ɛ 1 α 2 f ɛ 1 α 1 α α 1 α 2 1 α 1 P 1 α 2 1 α 2 ɛ 1 α 2 f ɛ 2 (ɛ 1, ɛ 2 ): Include departures from rationality given ɛ f James J. Heckman Haavelmo, Marschak, and Structural Econometrics 9 / 111

10 In logs the production function is ln X 0 = ln α 0 + α 1 ln X 1 + α 2 ln X 2 + ln ɛ f Questions of interest: α 1 + α 2 = 1? (Returns to scale) Wages equal marginal product? Variables related through common shocks X 0 ɛ f X 1 X 2 OLS does not recover the structural relationship. Does not identify α 1, α 2 the causal effects. Depending on the environment in factor markets we may have ɛ f (P 1, P 2 ) ( ) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 10 / 111

11 For his data, he has no confidence in ( ). Instead, using some prior information, he partially identified α 1 and α 2 to determine scale parameters. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 11 / 111

12 Partial Identification Analysis Identified Region Marschak and Andrews (1944) Figure 1: The effect of various side conditions on the parameters α 1 and α 2 (U.S.A., 1909). Point A indicates the single-equation estimates of the parameters and the shaded area covers the estimates compatible with certain economic considerations. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 12 / 111

13 Key Ideas In The Structural Approach Formulate a model or class of models within which to pose and answer precise economic and policy questions Give measures of precision concerning the exactitude with which the questions are answered (identification and estimation) Causality is not an absolute concept; can only be defined within models (either implicit or explicit) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 13 / 111

14 Well-posed models make explicit the assumptions used by analysts regarding preferences, technology and the information available to agents, the constraints under which they operate, the rules of interaction among agents in market and social settings and the sources of variability among agents. These explicit features make structural econometric models useful vehicles a for interpreting empirical evidence using theory; b for collating and synthesizing evidence across studies using economic theory; c for measuring the various effects of policies on outcomes d for forecasting the welfare and direct effects of previously implemented policies in new environments and the effects of new policies. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 14 / 111

15 Table 1: Three Distinct Tasks Arising in the Analysis of Causal Models and Counterfactual Policy Analysis Task Description Requirements 1 Defining the Set of A Scientific Theory Hypotheticals or (An Interpretive Framework) Counterfactuals 2 Identifying Causal Parameters Mathematical Analysis of from Hypothetical Population Point or Set Identification Data 3 Estimating Parameters in Estimation and Testing Real World Samples Theory James J. Heckman Haavelmo, Marschak, and Structural Econometrics 15 / 111

16 A Well Understood Problem: Predicting future outcomes from the past in a stationary environment, e.g., one in which all policies of interest have already been tried out. Policy Evaluation Problems P1: Evaluating the Impacts (constructing counterfactual states) of Implemented Interventions on Outcomes Including Their Impacts on the Well-Being of the Treated and Society at Large. (Internal Validity) P2: Forecasting the Impacts (Constructing Counterfactual States) of Interventions Implemented in One Environment in Other Environments, Including Impacts on Well-Being. (External Validity) P3: Forecasting the Impacts of Interventions (Constructing Counterfactual States Associated with Interventions) Never Historically Experienced, Including Their Impacts on Well-Being. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 16 / 111

17 An Amendment to the Structural Program Suggested by Marschak (1953) Structural econometricians often overly ambitious and unfocused. Seek parameters to answer a vague collection of policy problems. Marschak s Maxim: Economists should solve well-posed problems defined within models using minimal assumptions. All that is required to conduct many policy analyses or to answer many well-posed economic questions are policy invariant combinations of the structural parameters that are often much easier to identify than the individual parameters themselves and that do not necessarily require knowledge of individual structural parameters. This approach focuses attention on stating clearly the economic and policy questions being addressed. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 17 / 111

18 This approach is often less computationally intensive and focuses on a more limited range of policy questions than the very large range of policy questions contemplated by the Cowles pioneers in the original case for structural estimation. The computationally less demanding models, more transparent sources of identifiability and the relative ease of performing replication and sensitivity analyses give credibility to this approach. Models frame the question addressed even if all of the components of the model are not estimated or even identified. Recent appeals to Marschak s Maxim: Chetty(2009) and Weyl(2009) Early example: Harberger s (1954) welfare cost calculations James J. Heckman Haavelmo, Marschak, and Structural Econometrics 18 / 111

19 A Prototypical Structural Model Roy (1951): Agents face two potential outcomes: ( Y 0 }{{} high school, Y 1 }{{} college ) Individual Treatment Effect: Y 1 Y 0 Distribution F Y0,Y 1 (y 0, y 1 ) Y 0, Y 1 may depend on X : E(Y 0 X ) = µ 0 (X ) ; Y 0 = µ 0 (X ) + U 0 E(Y 1 X ) = µ 1 (X ) ; Y 1 = µ 1 (X ) + U 1 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 19 / 111

20 Costs, C, can depend on cost shifters (e.g., Z) C = µ C (Z) + U C E(C Z) = µ C (Z) Agent subjective evaluations: Y 1 Y 0 C Choice of outcome state generated by agent subjective evaluations D =1(Y 1 Y 0 C > 0) (2) =1(µ 1 (X ) µ 0 (X ) µ C (Z) > V ) V = [U 1 U 0 U C ]. The observed outcome Y : Switching Regression Quandt (1958) : Y = DY 1 + (1 D)Y 0 (3) The evaluation problem: Analysts observe either Y 0 or Y 1, but not both, for any person. The selection problem. What is observed is not a random sample of the population. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 20 / 111

21 Adding Uncertainty I: information set of the agent making the decision. Agents may be uncertain about components of (Y 0, Y 1, C). I D = E(Y 1 Y 0 C I): Ex Ante Valuation of Treatment D = 1(I D > 0). (4) ex post objective outcomes: (Y 0, Y 1 ). ex ante outcomes are E(Y 0 I) and E(Y 1 I). ex ante evaluation is I D = E(Y 1 Y 0 C I). ex post valuation is Y 1 Y 0 C. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 21 / 111

22 Consider problem P1: Evaluating a program in place. Population Data F Y0 (y 0 D = 0, X, Z) F Y1 (y 1 D = 1, X, Z) Pr(D = 1 X, Z) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 22 / 111

23 Population Parameters of Interest Average Treatment Effect (ATE Gross = E(Y 1 Y 0 ); ATE Net = E(Y 1 Y 0 C)) The effect of Treatment on The Treated (TT = E(Y 1 Y 0 D = 1) or TT Net = E(Y 1 Y 0 C D = 1)) The voting criterion: Pr(I D > 0) = Pr(E(Y 1 Y 0 C I) > 0). ex post: Pr(Y 1 Y 0 C > 0). The mean effect of treatment for those at the margin of indifference is E(Y 1 Y 0 I D = 0). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 23 / 111

24 Most social programs offer incentives to participate and do not force people into treatment states. Treatment Effects Versus Policy Effects Most social programs offer incentives to participate and do not force people into treatment states. Policy Relevant Treatment Effect Designed to address problems P2 and P3. b : baseline policy ( before ) and a represent a policy being evaluated ( after ). Y a : outcome under policy a; Y b is the outcome under the baseline. D a and D b : choice taken under each policy regime. The observed outcomes under each policy regime are Y a = Y 0 D a + Y 1 (1 D a ) and Y b = Y 0 D b + (1 D b ). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 24 / 111

25 The Policy Relevant Treatment Effect (PRTE) is PRTE = E(Y a Y b ). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 25 / 111

26 Despite the wide array of questions addressed by this approach, there is widespread rejection of the structural method used to generate these and other structural estimates by many applied economists. Move to the program evaluation approach which does not formulate or estimate explicit economic models. An appeal to transparent statistical identification, away from the use of explicit economic models based on choice either in formulating economic policy questions or in suggesting frameworks for estimating models to answer policy questions. Effects not parameters Objective evaluations not subjective evaluations James J. Heckman Haavelmo, Marschak, and Structural Econometrics 26 / 111

27 A second response to the perceived failure of parametric structural models was development of a more robust nonparametric version of the structural approach. (Matzkin, ) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 27 / 111

28 Table 1: Comparison of the Aspects of Evaluating Social Policies that are Covered by the Program Evaluation Approach and the Structural Econometric Approach Program Evaluation Approach The Structural Econometric Approach Counterfactuals for objective outcomes (Y 0, Y 1 ) Yes Yes Agent valuations of subjective outcomes (I D ) No (choice-mechanism implicit) Yes Net Benefits (Benefit-Cost) No Yes Models for the causes of potential outcomes No Yes Ex ante versus ex post counterfactuals No Yes Treatment assignment rules that recognize voluntary No Yes nature of participation Evaluation of returns at the margin of various policies No Yes Social interactions, general equilibrium effects and contagion No (assumed away) Yes (modeled) Internal validity (problem P1) Yes Yes External validity (problem P2) No Yes Forecasting effects of new policies (problem P3) No Yes Distributional treatment effects No a Yes (for the general case) Analyze relationship between outcomes and choice equations No (implicit) Yes (explicit) a An exception is the special case of common ranks of individuals across counterfactual states: rank invariance. See the discussion in Abbring and Heckman (2007). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 28 / 111

29 Is Randomization the Gold Standard? In the program evaluation approach, the experiment is the ideal. The beauty of randomized evaluations is that the results are what they are: we compare the outcome in the treatment [group] with the outcome in the control group, see whether they are different, and if so by how much. Interpreting quasiexperiments sometimes requires statistical legerdemain, which makes them less attractive... Banerjee (2006) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 29 / 111

30 Different experiments identify different parameters. At best, commonly used forms of social experiments identify but not F Y0,Y 1 (y 0, y 1 X ) Can estimate E(Y 1 Y 0 X ) F Y1 (y 1 X ) and F Y0 (y 0 X ) Cannot in general identify Pr(Y 1 Y 0 > 0 X ). Cannot recover agent preferences subjective evaluations or surpluses in a noncompulsory setting. Cannot address returns at the margin. Noncompliance is a source of information on subjective evaluations. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 30 / 111

31 Instrumental variables play a key role in the program evaluation literature. Better LATE than nothing, Imbens, 2010 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 31 / 111

32 Recast the discussion into a familiar-looking regression framework. Equation for ex post outcome Y as a function of participation status, D Y = α + βd + ε (5) Recall Y = Y 0 (1 D) + Y 1 D Y 1 = µ 1 (X ) + U 1 Y 0 = µ 0 (X ) + U 0 C = µ C (Z) + U C D = 1(Y 1 Y 0 C > 0) β = Y 1 Y 0 = µ 1 (x) µ 0 (X ) + U 1 U 0 }{{} Individual Level Treatment Effect α = µ 0 (x) + U 0 ε = U 0 D = 1(β C > 0) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 32 / 111

33 E (β) = β, Y = α + βd + {ε + (β β)d} (6) β = µ 1 µ 0, β β = U 1 U 0. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 33 / 111

34 Choice Equation I D = E(Y 1 Y 0 C I) = µ D (Z) V µ D (Z) = E(µ 1 (X ) µ 0 (X ) µ C (Z) I) V = E(U 1 U 0 U C I). Choice equation: D = 1(µ D (Z) > V ). (7) U D = F V (V ) P(z) = F V (µ D (z)), D(z) = 1(P(z) > U D ) Z (U 0, U 1, U D ) X Variables in Z not in X are instruments. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 34 / 111

35 Distribution Ch. 71: Econometric of gains Evaluation in the of Social Generalized Programs, Part II Roy economy 4893 U 1 U 0 D TT = 2.666, TUT = Return to marginal agent = C = 1.5 ATE = μ 1 μ 0 = β = 0.2 The model Source: Heckman, Urzua and Vytlacil (2006). Outcomes Choice model James J. Heckman Haavelmo, Marschak, and Structural { Econometrics 35 / 111

36 Understanding Instrumental Variables in Models With Heterogeneous Responses to Treatment Assume Classical IV Conditions Z (U 0, U 1, V ) Pr(D = 1 Z) is a nondegenerate function of Z Then E (Y Z) = α + βe (D Z) + E [( β β ) D = 1, Z ] Pr(D = 1 Z) }{{} 0 if agents even partially anticipate β β Draft lottery example James J. Heckman Haavelmo, Marschak, and Structural Econometrics 36 / 111

37 If E[(β β) D = 1, Z] = 0 E(Y Z) = α + βe(d Z) β = E(Y Z = z 2 ) E(Y Z = z 1 ) E(D Z = z 2 ) E(D Z = z 1 ) = IV Otherwise IV not interpretable without additional assumptions. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 37 / 111

38 LATE LATE is defined by the variation of an instrument, not as an economic parameter answering a well posed economic question. It provides an intuitive interpretation for what IV estimates in a model with heterogeneous responses. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 38 / 111

39 Y 0 and Y 1 are potential ex post outcomes. Instrument Z assumes values in Z, z Z. D(z) is an indicator of hypothetical choice representing what choice the individual would have made had the individual s Z been exogenously set to z. D(z) = 1 if the person chooses (is assigned to) state 1. D(z) = 0, otherwise. All policies are assumed to operate through their effects on Z. It is assumed that Z can be varied conditional on X. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 39 / 111

40 (IA-1) Three assumptions define LATE. (Y 0, Y 1, {D(z)} z Z ) Z X (IA-2) Pr(D = 1 Z = z) is a nontrivial function of z conditional on X. (IA-3) Monotonicity or Uniformity For any two values of Z, say Z = z 1 and Z = z 2, either D(z 1 ) D(z 2 ) for all persons, or D(z 1 ) D(z 2 ) for all persons. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 40 / 111

41 For two distinct values of Z, z 1 and z 2, IV applied to (5) identifies LATE(z 2, z 1 ) = E(Y 1 Y 0 D(z 2 ) = 1, D(z 1 ) = 0), if the change from z 1 to z 2 induces people into the program (D(z 2 ) D(z 1 )). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 41 / 111

42 LATE does not identify which people are induced to change their treatment status by the change in the instrument. It does not identify a margin of choice in terms of agent preferences. It leaves unanswered many policy questions. If a proposed program changes the same components of vector Z as used to identify LATE but at different values of Z (say z 4, z 3 ), LATE(z 2,z 1 ) does not identify LATE(z 4, z 3 ). Different instruments identify different margins: difficult to compare them or place them on common scale James J. Heckman Haavelmo, Marschak, and Structural Econometrics 42 / 111

43 Applying Some Economics to Interpret LATE: The LATE conditions are equivalent to a nonparametric generalized Roy model. Vytlacil (2002): LATE is equivalent to a nonparametric version of the generalized Roy model. The LATE conditions the generalized Roy model Generalized Roy model the LATE model. LATE existence of V and µ D (Z) D = 1(µ D (Z) > V ) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 43 / 111

44 D(z) = 1(µ D (z) > V ). P(z) Pr(D = 1 Z = z) = Pr(µ D (z) > V ) P(z) = Pr(µ D (z) > V ) = F V (µ D (z)). Define: U D = F V (V ) Uniformly distributed over the interval [0, 1] D = 1 (µ D (z) > V ) (8) D = 1 (Fv (µ D (z)) > Fv (V )) (9) D = 1(P(z) > U D ). (10) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 44 / 111

45 From P(z), one can identify the ex ante net benefit I D up to scale and determine for each value of Z = z, what proportion of people perceive that they will benefit from the program and the intensity of their benefit. From agent choices, one can supplement the information in LATE and ascertain ex ante valuation evaluations. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 45 / 111

46 Index Sufficiency The LATE assumptions imply E(Y Z = z) = E(Y P(Z) = P(z)): index sufficiency. E (Y Z = z) = E [Y 0 (1 D) + Y 1 D Z = z] = E [Y 0 + D (Y 1 Y 0 ) Z = z] = E (Y 0 ) + E ([Y 1 Y 0 ] D = 1, P (Z) = P (z)) P(z) = E (Y 0 ) + E (Y 1 Y 0 P (z) > U D ) P(z) }{{} Selective effect due to sorting on heterogeneous responses: The surplus from sorting = S(P(z)) = E(Y 0 ) + S(P(z)) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 46 / 111

47 If agents do not sort on idiosyncratic gains E(Y 1 Y 0 P(z) > U D ) = β = µ 1 µ 0 S(P(z)) = βp(z) E(Y Z) = µ 0 + βp(z) Z enters only through P(Z) E(Y Z) is a linear function of P(Z) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 47 / 111

48 One can define LATE(z 2, z 1 ) using the latent variable U D and the values taken by P(Z) when Z = z 1 and Z = z 2. LATE(z 2, z 1 ) = E(Y 1 Y 0 P(z 1 ) U D P(z 2 )). (11) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 48 / 111

49 Marginal increment in outcomes as we change the mean scale utility function P(z). E(Y P(Z) = p) = E(Y 1 Y 0 U D = p) p }{{}}{{} Identified in the data Marginal Treatment Effect(MTE) A willingness to pay parameter = S(p) p. (12) MTE: (Mean gross gain in moving from 0 1 for a person indifferent between 1 and 0 at mean scale utility p = U D. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 49 / 111

50 Figure 2: (a) Plots of the MTE derived from E(Y P(Z) = p) E[Y P(Z)=p] p (a) Plot of the E(Y P(Z) = p Source: Heckman and Vytlacil (2005) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 50 / 111

51 Figure 2: (b) Plots of the MTE derived from E(Y P(Z) = p) MTE Source: Heckman and Vytlacil (2005) u D (b) Plot of MTE(u D ): The derivative of E(Y P(Z) = p) evaluated at points p = u D James J. Heckman Haavelmo, Marschak, and Structural Econometrics 51 / 111

52 Figure 3: MTE = E(Y 1 Y 0 X, U D ) for Returns to College MTE U D Source: Carneiro et al. (2011) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 52 / 111

53 E (Y P(Z) = p) = E(Y 0 ) + p 0 MTE(u D ) du D } {{ } S(p) (13) E(Y P(Z) = p 2 ) E(Y P(Z) = p 1 ) p 2 = S(p 2 ) S(p 1 ) = MTE(u D ) du D. p 1 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 53 / 111

54 LATE(p 2, p 1 ) = E(Y P(Z) = p 2) E(Y P(Z) = p 1 ) p 2 p 1 = S(p 2) S(p 1 ) p 2 p 1 p 2 MTE(u D ) du D p 1 =. (14) p 2 p 1 Under standard regularity conditions lim LATE(p 2, p 1 ) = MTE(p 1 ). p 2 p 1 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 54 / 111

55 MTE(u D ) and the model-generated LATE (11) are structural parameters in the sense that changes in Z (conditional on X ) do not affect MTE(u D ) or theoretical LATE. They are invariant with respect to all policy changes that operate through Z. Conditional on X, one can transport MTE and the derived theoretical LATEs across different policy environments and different data sets. (Solve versions of P2 and P3) These policy-invariant parameters implement Marschak s Maxim since they are defined for combinations of the parameters of the generalized Roy model and do not require estimating the full structural model. They can be estimated in cases where the full structural model is not identified. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 55 / 111

56 This deeper understanding of LATE facilitates its use in answering out of sample policy questions P2 and P3 for policies that operate through changing Z. Thus if one computes a LATE for any two pairs of values Z = z 1, and Z = z 2, with associated probabilities Pr(D = 1 Z = z 1 ) = P(z 1 ) = p 1 and Pr(D = 1 Z = z 2 ) = P(z 2 ) = p 2, one can use it to evaluate any other pair of policies z and z such that and Pr(D = 1 Z = z 1 ) = Pr(D = 1 Z = z) = p 1 Pr(D = 1 Z = z 2 ) = Pr(D = 1 Z = z) = p 2. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 56 / 111

57 One can use an empirical LATE determined for one set of instrument configurations to identify outcomes for other sets of instrument configurations that produce the same p 1 and p 2, i.e., we can compare any policy described by z {z P(z) = p 1 } with any policy z {z P(z) = p 2 } and not just the policies associated with z 1 and z 2 that identify the sample LATE. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 57 / 111

58 Analysts can aggregate the variation in different components of Z into the induced variation in P(Z) to trace out MTE(u D ) over more of the support of u D than would be possible using variation in any particular component of Z. Using the economics of the problem, one can understand what different instruments identify and at what margin of u D. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 58 / 111

59 Figure 4: MTE as a function of u D : What sections of the MTE different values of the instruments and different instruments approximate. Mean Marginal Gain LATE(p 2, p 1 ) MTE LATE(p 4, p 3 ) u D (p 2, p 1 ) u D (p 4, p 3 ) 0 p 1 p 2 u D p 3 p 4 1 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 59 / 111

60 Figure 5: Understanding What Variation in Different Instruments Identifies: Returns to College, Fully Nonparametric Specification 1.5 Distance Wage Unemp. Tuition All 1 support(p X),MTE P,U D Source: Carneiro et al. (2011). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 60 / 111

61 Implications for Specification Tests A test of whether MTE(u D ) depends on u D, or a test of nonlinearity of E(Y P(Z) = p) in p, is a test of the whether different instruments estimate the same parameter. The LATE model and its extensions overturn the logic of the Durbin (1954) Wu (1973) Hausman (1978) test for overidentification. Variability among the estimates from IV estimators based on different instruments may have nothing to do with the validity of any particular instrument, but may just depend on what stretch of the MTE they approximate. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 61 / 111

62 All Treatment Effects Are Weighted Averages of the MTE For treatment effect e, Treatment Effect(e) = 1 0 MTE(u D ) h e (u D ) du D. (15) }{{}}{{} identified from previous analysis Determined from sample data James J. Heckman Haavelmo, Marschak, and Structural Econometrics 62 / 111

63 For ATE, h e (u D ) = 1 For LATEs over disjoint, inclusive intervals (u D,j 1, u D,j ) ATE = M LATE(u D,j, u D,j 1 )η j, j=1 where η j = u D,j u D,j 1. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 63 / 111

64 ATE weights u D evenly. TT oversamples low values of u D (associated with persons more likely to participate in the program). TUT oversamples high u D. In this example, because MTE(u D ) is decreasing in u D, TT > ATE > TUT. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 64 / 111

65 Figure 6: Weights for the marginal treatment effect for different parameters for the model graphed in Figure 2 h (u D ) 3.5 MTE MTE TT TUT ATE u D Source: Heckman and Vytlacil (2005) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 65 / 111

66 The Policy Relevant Treatment Effect (PRTE) is E(Y Alternative Policy a) E(Y Baseline Policy b) = E(Y a ) E(Y b ) 1 = MTE(u D )h PRTE (u D ) du D, 0 h PRTE (u D ) = F b P (u D) F a P (u D). FP b is the distribution of P(Z) under policy b, and F P a is the distribution of P(Z) under policy a. The same MTE(u D ) can be used to evaluate the impacts of a variety of different policies. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 66 / 111

67 Table 2: (a) MTE Weights For Different Treatment Parameter and IVs using P(Z) as an instrument. (F p is the distribution of P. f P is its density.) Treatment Parameter Weights h ATE (u D ) = 1 [ 1 ] h TT 1 (u D ) = f P (p)dp u D E(P) h TUT (u D ) = [ ud 0 ] 1 f P (p) dp E (1 P) h PRTE (u D ) = F a P (u D) F b P (u D) f J,P (j, p) is the joint density of J and P. For derivations of these weights, see Heckman and Vytlacil (1999, 2005, 2007). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 67 / 111

68 What Does IV Based on P(Z) Estimate? IV is also a weighted average of MTEs. Consider a linear regression approximation of E (Y P(Z) = p): b = E (Y P(Z) = p) = a + bp, Cov(Y, P(Z)) Var(P(Z)) = Cov(E(Y P(Z)), P(Z)). Var(P(Z)) b is the IV estimate of the causal effect of D on Y using P(Z) as an instrument since Cov(P(Z), D) = Var(P(Z)). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 68 / 111

69 Cov(Y, P(Z)) Cov(S(P(Z)), P(Z)) b = = Var(P(Z)) Var(P(Z)) ( ) P(Z) Cov MTE(u D )du D, P(Z) 0 =. (16) Var(P(Z)) When MTE(u D ) = β = µ 1 µ 0 S(P(Z)) = βp(z), b = β James J. Heckman Haavelmo, Marschak, and Structural Econometrics 69 / 111

70 Table 3: MTE Weights For IV (P(Z) as IV and J(Z) as IV) [ 1 ] h IV 1 (u D ) = (p E(P))f P (p) dp u D Var(P) for P(Z) as an instrument 1 (j E(J))f J,P (j, p) dj dp h IV u (u D ) = D Cov(J,P) for a general instrument J(Z), a function of Z f J,P (j, p) is the joint density of J and P. For derivations of these weights, see Heckman and Vytlacil (1999, 2005, 2007). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 70 / 111

71 Figure 7: Marginal treatment effect vs. linear instrumental variables based on P(z) and the OLS weights 4902 J.J. Heckman and E.J. Vytlacil Y 1 = α + β + U 1 U 1 = σ 1 τ α = 0.67 σ 1 = Y 0 = α + U 0 U 0 = σ 0 τ β = 0.2 σ 0 = D = 1ifZ V 0 V = σ V τ τ N(0, 1) σ V = U D = Φ( V σ ) V στ Z N( , ) Source: Heckman and Vytlacil (2005). James J. Heckman Figure 2B. Marginal treatment effect vs. Haavelmo, linear instrumental Marschak, variables and and Structural ordinary least Econometrics squares weights. 71 / 111

72 More General Instruments Than P(Z) Typically, economists use a variety of instruments one at a time and not just P(Z) as an instrument and compare the resulting estimates (see, e.g, Card, 1999, 2001). Different instruments identify different parameters. IV is a weighted average of MTEs where the weights integrate to 1 and can be estimated from sample data. However, in the case of general instruments, the weights can be negative over stretches of u D. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 72 / 111

73 Consider using the first component of Z, Z 1, as an instrument for D. Suppose that Z contains two or more elements (Z = (Z 1,..., Z K ), K 2). The economics implicit in LATE informs us that Z determines the distribution of Y through P(Z). Any correlation between Y and Z 1 arises from the statistical dependence between Z 1 and P(Z) operating to determine Y. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 73 / 111

74 The IV estimator based on Z 1 is IV Z1 = Cov(Y, Z 1) Cov(D, Z 1 ) = Cov(E(Y Z 1), Z 1 ). Cov(D, Z 1 ) Note, however, that choices (and hence Y ) are generated by the full vector of Z operating through P(Z). The analyst may only use Z 1 as an instrument but the underlying economic model informs us that the full vector of Z determines observed Y. The only case where this is irrelevant is when E(Y 1 Y 0 P(z) > U D ) = E(Y 1 Y 0 ) = β, the traditional case. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 74 / 111

75 Using Z 1 as an instrument and not conditioning on Z 2,..., Z K leaves uncontrolled the influence of the other elements of Z on Y. This is a new phenomenon in IV that would not be present if D did not depend on β (i.e. D (Y 1 Y 0 )). An IV based on Z 1 identifies an effect of Z 1 on Y as it operates directly through Z 1 (Z 1 changing P(Z 1,..., Z K )) holding other elements in Z constant and indirectly through the effect of Z 1 as it covaries with (Z 2,..., Z K ), and how those variables affect Y through their effect on P(Z). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 75 / 111

76 A linear regression analogy: Suppose outcome Q can be expressed as a linear function of W = (W 1,..., W L ), an L-dimensional regressor: Q = L φ l W l + ε, l=1 where E(ε W ) = 0. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 76 / 111

77 If we regress Q only on W 1, we obtain in the limit the standard omitted variable result that the estimated effect of W 1 on Q is Cov(Q, W 1 ) Var(W 1 ) = φ 1 + L l=2 Cov(W l, W 1 ) φ l, (17) Var(W 1 ) where φ 1 is the ceteris paribus direct effect of W 1 on Q and the summation captures the rest of the effect (the effect on Q of W 1 operating through covariation between W 1 and the other values W l, l 1). An analogous problem arises in using one instrument at a time to identify the effect of Z 1. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 77 / 111

78 Thus if the analyst does not condition on the other elements of Z in using Z 1 as an instrument, the margin identified by variations of Z 1 does not in general correspond to variations arising solely from variations in Z 1, holding the other instruments constant. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 78 / 111

79 Thus in general an IV based on Z 1 mixes causal effects with sample dependence effects among the correlated regressors. In a study of college going, if Z 1 and Z 2 are tuition and distance to college, respectively, the instrument Z 1 identifies the direct effect of variation in tuition on college attendance and the effect of distance to college on college attendance as it covaries with tuition in the sample used by the analyst. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 79 / 111

80 This is not the ceteris paribus causal effect of a variation in tuition. It produces what Marschak called Mongrel Functions It does not correspond to the answer needed to predict the effects of a policy that operates solely through an effect on tuition. In models in which D depends on β, the traditional instrumental variable argument that analysts do not need a model for D and can ignore other possible determinants of D besides the instrument being used, breaks down. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 80 / 111

81 Example: Figure 8: Joint Distribution of Instruments Z = (Z 1, Z 2 ) 0.20 Joint Density Z Z1 2 2 James J. Heckman Haavelmo, Marschak, and Structural Econometrics 81 / 111

82 Figure 9: MTE and IV weights for a general instrument Z 1, a component of Z = (Z 1, Z 2 ). IV Weights IV Weights MTE u D James J. Heckman Haavelmo, Marschak, and Structural Econometrics 82 / 111

83 Table 4: IV estimator for three different distributions of Z but the same generalized Roy model. Data Distribution IV ATE Source: Heckman, Urzua, et al. (2006, Table 3). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 83 / 111

84 To interpret which margin is identified by different instruments requires that the analyst specify and account for all of the Z that form P(Z). Since different economists may disagree on the contents of Z, different economists using Z 1 on the same data will obtain the same point estimate but will disagree about the interpretation of the margin identified by variation in Z 1. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 84 / 111

85 The Problem of Limited Support P(Z) may not be identified over the full unit interval. P(Z) may only assume discrete values. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 85 / 111

86 Many proposed policy changes are incremental in nature, and a marginal version of the PRTE is all that is required to answer questions of economic interest. When some instruments are continuous, it is possible under the conditions in their paper to identify a marginal version of PRTE (MPRTE). Application of these data sensitive nonparametric approaches enables analysts to avoid one source of instability of the estimates of policy effects that plagued 1980s econometrics. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 86 / 111

87 Estimating Marginal Policy Changes: IV PRTE in general. PRTE depends on the policy change only through the distribution of P after the policy change. The PRTE is defined for a discrete change from a baseline policy to a fixed alternative. Identifying it in any sample can be a challenging task because it often requires that the support of P(Z) be the full unit interval. Marginal version of the PRTE parameter (MPRTE). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 87 / 111

88 Consider the following sequences of policies: i a policy that increases the probability of attending college by an amount α, so that P α = P 0 + α and F α (t) = F 0 (t α); ii a policy that changes each person s probability of attending college by the proportion (1 + α), so that P α = (1 + α)p 0 and F α (t) = F 0 ( t 1+α ); and iii a policy intervention that has an effect similar to a shift in one of the components of Z, say Z [k], so that Z α [k] = Z [k] + α and Z α [j] = Z [j] for j k. For example, the kth element of Z might be college tuition, and the policy under consideration subsidizes college tuition by the fixed amount α. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 88 / 111

89 The MPRTE is the appropriate parameter with which to conduct cost-benefit analysis of marginal policy changes. It is a weighted average of MTEs. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 89 / 111

90 Table 5: Weights for MPRTE Measure of Distance Definition of Weight for People Near the Margin Policy Change µ S (Z) V < e Zα k = Z k + α h MPRTE (x, u S ) = f P X (u S )f V X (F 1 V X (u S )) E(f V X (µ S (Z)) X ) P U < e P α = P + α h MPRTE (x, u S ) = f P X (u S ) P U 1 < e P α = (1 + α) P h MPRTE (x, u S ) = u S f P X (u S ) E(P X ) Source: Carneiro, Heckman, and Vytlacil (2010). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 90 / 111

91 Figure 10: Weights for three different versions of the MPRTE MTE Za = Z+a Pa = P+a Pa = (1+a)P 0.3 MTE,Weights Us Source: Carneiro et al. (2011). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 91 / 111

92 Figure 11: Weights for IV and MPRTE MTE IV MPRTE 0.3 MTE,Weights Us Source: Carneiro et al. (2011). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 92 / 111

93 Table 6: Returns to a year of college Model Semi-Parametric ATE= E(β) Not Identified TT= E(β D = 1) Not Identified TUT= E(β D = 0) Not Identified MPRTE Policy Perturbation Metric Zα k = Z k + α Zγ V < e P α = P + α P U < e P α = (1 + α) P P U 1 < e Linear IV (Using P(Z) as the instrument) OLS James J. Heckman Haavelmo, Marschak, and Structural Econometrics 93 / 111

94 Identifying Returns To Persons at The Margin MPRTE is closely related to the return for the person at the margin : AMTE. What margin? How to identify returns at the margin? There are technical issues that arise in identifying the marginal gain to persons in indifference sets that arise from the thinness of these sets. (Borel Paradox) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 94 / 111

95 Different ways to approximate the indifference set P(Z) = U D through limit operations determine different values of the AMTE. The effect of a marginal policy change for a particular perturbation of P(Z) is the same as the average effect of treatment for those who are arbitrarily close to being indifferent between treatment or not, using a metric m(p, U D ) measuring the distance between P(Z) and U S. This parameter is defined as AMTE = lim e 0 E[Y 1 Y 0 m(p, U D ) e]. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 95 / 111

96 Econometric Cost-Benefit Analysis The program effect literature focuses on gross benefits B. For decision making, need marginal benefits and marginal costs. Some aspects of marginal costs may be subjective e.g., the pain and suffering associated with a medical procedure or going to college. Not easily measured. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 96 / 111

97 Identifying Costs and Surplus Assume two types of exclusion restrictions: 1 At least one variable that affects costs but not outcomes 2 At least one variable that affects outcomes but not costs James J. Heckman Haavelmo, Marschak, and Structural Econometrics 97 / 111

98 Benefit-MTE B-MTE(X, U D ) = E(Y 1 Y 0 X, U D ) Cost-MTE C-MTE(Z, U D ) = E(C Z, U D ) Surplus-MTE S-MTE(X, Z, U D ) = E(S X, Z, U D ) = B-MTE(X, U D ) C-MTE(Z, U D ) James J. Heckman Haavelmo, Marschak, and Structural Econometrics 98 / 111

99 Identification Analysis: C-MTE B-MTE(x, P(x, z)) = C-MTE(z, P(x, z)) implies p E(Y X = x, P = p) p=p(x,z) = C-MTE(z, P(x, z)). For a fixed z, we identify C-MTE(z, u D ) for u D Supp(P X = x, Z = z) by varying X. The more variation in regressors that shift outcomes, the more variation in propensity scores conditional on Z = z, and the larger the set of evaluation points for which we identify C-MTE(z, u D ). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 99 / 111

100 Figure 12: Local Identification of C MTE(z, u D ) Marginal Treatment Effect Source: Eisenhauer, Heckman, and Vytlacil (2011). u D James J. Heckman Haavelmo, Marschak, and Structural Econometrics 100 / 111

101 Figure 13: Relationships Among MTE Parameters. Marginal Treatment Effect P(x, z) = 0.55 B MTE(x,u S ) C MTE(z,u S ) S MTE(x,z,u S ) u S Source: Eisenhauer, Heckman, and Vytlacil (2011). James J. Heckman Haavelmo, Marschak, and Structural Econometrics 101 / 111

102 Multiple Choices Multiple margins of entry into a state James J. Heckman Haavelmo, Marschak, and Structural Econometrics 102 / 111

103 In the special case where the analyst seeks to estimate the mean return to those induced into a choice state by a change in an instrument compared to their next best option, LATE remains useful (see Heckman, Urzua, et al., 2006; Heckman and Urzua, 2010; Heckman and Vytlacil,2007). If, however, one is interested in identifying the mean returns to any pair of outcomes, unaided IV will not do the job. Structural methods are required. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 103 / 111

104 In general unordered choice models, agents attracted into a state by a change in an instrument come from many origin states, so there are many margins of choice. Structural models can identify the gains arising from choices at these separate margins. This is a difficult task for IV without invoking structural assumptions. Structural models can also identify the fraction of persons induced into a state coming from each origin state. IV alone cannot. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 104 / 111

105 Policy Effects, Treatment Effects, and IV Policy effects are not generally the same as treatment effects, In general, neither are produced from IV estimators. Since randomized assignments of components of Z are instruments, this analysis also applies to the output of randomized experiments. The economic approach to policy evaluation pioneered by Marschak and his collaborators formulates policy questions using well-defined economic models. It then uses whatever statistical tools it takes to answer these questions. Policy questions and not statistical methods drive analyses. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 105 / 111

106 Well-posed economic models are scarce in the program evaluation approach. In contrast to the structural approach, it features statistical methods over economic content. Credibility in the program evaluation literature is assessed by statistical properties of estimators and not economic content or policy relevance. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 106 / 111

107 We can do better than hoping that an instrument or an estimator answers policy problems. By recovering economic primitives, we can distinguish the objects various estimators identify from the questions that arise in addressing policy problems. An alternative approach developed in Heckman and Vytlacil (2005) constructs combinations of instruments using sample data on Z that address specific policy questions. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 107 / 111

108 Summary This lecture compares the structural approach developed by Haavelmo, Koopmans and the Cowles group to empirical policy analysis with the program evaluation approach. It offers a way to do policy analysis that combines the best features of both approaches based on Marschak s Maxim. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 108 / 111

109 Marschak s Maxim advocates placing the economic and policy questions being addressed front and center. Use economics to frame questions and the statistics to help address them. Economic theory helps to define and sharpen statements about policy questions. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 109 / 111

110 Both the program evaluation approach and the structural approach have desirable features. Program evaluation approaches are generally computationally simpler than structural approaches, and it is often easier to conduct sensitivity and replication analyses with them. Identification of program effects is often more transparent than identification of structural parameters. At the same time, the economic questions answered and the policy relevance of the treatment effects featured in the program evaluation approach are often very unclear. Structural approaches produce more interpretable parameters that are better suited to conduct counterfactual policy analyses. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 110 / 111

111 Approach advocated in this lecture is to use Marschak s Maxim to identify the policy-relevant combinations of structural parameters that answer well-posed policy and economic questions. This approach often simplifies the burden of computation, facilitates replication and sensitivity analyses, and makes identification more transparent. At the same time, application of this approach forces analysts to clearly state the goals of the policy analysis something many economists (structural or program evaluation) have difficulty doing. That discipline is an added bonus of this approach. James J. Heckman Haavelmo, Marschak, and Structural Econometrics 111 / 111