Structural Econometrics: Dynamic Discrete Choice. Jean-Marc Robin

Transcription

1 Structural Econometrics: Dynamic Discrete Choice Jean-Marc Robin 1. Dynamic discrete choice models 2. Application: college and career choice Plan 1

2 Dynamic discrete choice models See for example the presentation by Wolpin (AER, 1996). At each date t discrete, an individual has to choose one action among K possible actions. Let 1 if k is the chosen action, d k (t) = 0 otherwise. Let d(t) = (d 1 (t); :::; d K (t)) or d (t) = P K k=1 kd k (t) be the choice variable. Let S(t) 2 S be the state variable (i.e; the information at the beginning of period t when the action is chosen). Assume S discrete: S = fs 1 ; :::; s N g (in any case the computer will require a discrete state space). Action k yields payo R k (S(t); t). The state transition probability matrix is p ij (k; t) = Pr fs(t + 1) = s j js(t) = s i ; d k (t) = 1g : 2

3 Strategies A strategy is a sequence of functions D(; t) : S! f0; 1g K s 7! D(s; t) = (D 1 (s; t); :::; D K (s; t)) Individuals seek for the strategy D to maximise the expected discounted sum of future payos: " TX K # V (S(t); t) = max E X t D k (S(); ) R k (S(); ) D(;) S(t) : =t k=1 3

4 Bellman principle Write, for s 2 S, V (s; t) = max fv 1 (s; t); :::; V K (s; t)g where V k (s; t) is the present value if action k is chosen at t when S(t) = s: V k (s; t) = R k (S(t); t) + E [V (S(t + 1); t + 1)j S(t) = s; d k (t) = 1] and V k (s; T ) = R k (s; T ): The optimal strategy is D k (s; t) = 1 i V k (s; t) = max fv 1 (s; t); :::; V K (s; t)g and then V (s; t) = KX D k (s; t)v k (s; t): k=1 4

5 Solution Start from terminal period T and, for all s 2 S, determine the action which maximises payo R k (s; T ): and D k (s; T ) = 1 i R k (s; T ) = max fr 1 (s; T ); :::; R K (s; T )g V (s; T ) = KX D k (s; T )R k (s; T ): k=1 Then determine D(s; t) recursively: for all s 2 S, where, for all s 1 ; :::; s N ; D k (s; t) = 1 i V k (s; t) = max fv 1 (s; t); :::; V K (s; t)g V k (s i ; t) = R k (s i ; t) + E [V (S(t + 1); t + 1)j S(t) = s i ; d k (t) = 1] NX = R k (s; t) + p ij (k; t) V (s j ; t + 1) {z } j=1 = P K k=1 D k(s j ; t + 1)V k (s j ; t + 1) Curse of dimensionality: huge number of computations and large memory size required to compute V k (s; t) 8k; s; t. 5

6 Estimation Parameters: in the payo functions R k (s; t) and transition probabilities p ij (k; t). Inference: maximum likelihood or (simulated) method of moments. Data: individual sequences y h = x h (t h 0); d h (t h 0); x h (t h 0 + 1); d h (t h 0 + 1); :::; x h (t h 1); d h (t h 1) for individuals h = 1; :::; H and t 2 t h 0; t h 0 + 1; :::; t h 1, where x h (t) 2 fx 1 ; :::; x I g is the observed part of the state variables, i.e. S h (t) = x h (t); " h (t), with the following......assumptions on the process of shocks " h (t): " h (t) = " h 1(t); :::; " h K (t) iid; R k (S h (t) ; t) = R k (x h (t) ; t) + " h k (t); conditional independence: Pr x h (t + 1); " h (t + 1)jx h (t); " h (t); d k (t) = 1 = Pr(" h (t + 1)) Pr x h (t + 1) = x j jx h (t) = x i ; d h k(t) = 1 : {z } p ij (k;t) 6

7 Likelihood The conditional likelihood of y h given x h (t h 0) is `(y h jx h (t h 0)) = Pr d h (t h 0)jx h (t h 0) Pr x h (t h 0 + 1)jx h (t h 0); d h (t h 0) Pr d h (t h 0 + 1)jx h (t h 0 + 1) P x h (t h 0 + 2)jx h (t h 0 + 1); d h (t h 0 + 1) Pr d h (t h 1)jx h (t h 1) where Pr d h k(t) = 1jx h (t) = Pr " h (t) s.t. D k (x h (t); " h (t); t) = 1jx h (t) : The conditional likelihood of the sample is HY `(y h jx h (t h 0)): h=1 7

8 Choice probabilities Pr d h k(t) = 1jx h (t) = x i = Pr " h k(t) " h m(t) + V m (x i ; t) V k (x i ; t); 8m 6= kjx h (t) = x i : where V k (x i ; t) = R k (x i ; t) + NX p ij (k; t)v (s j ; t + 1); p ij (k; t) = Pr x h (t + 1) = x j jx h (t) = x i ; d h k(t) = 1 ; V (x j ; t + 1) = E max K V k (x j ; t + 1) + " h k(t + 1) : k=1 j=1 For instance, for (X 1 ; X 2 ) Gaussian, E max fx 1 ; X 2 g = X 2 + E max fx 1 X 2 ; 0g m1 m 2 m1 m 2 = m 2 + (m 1 m 2 ) + ' where = Std (X 1 X 2 ) = p

9 Two stage estimation One can proceed in two stages to save computer time, although t the cost of some eciency loss. 1. Maiximise partial likelihood of state changes: HY Pr x h (t h 0 + 1)jx h (t h 0); d h (t h 0) Pr x h (t h 0 + 2)jx h (t h 0 + 1); d h (t h 0 + 1) h=1 Pr x h (t h 1)jx h (t h 1 1); d h (t h 1 1) ; with respect to parameters of Pr fx(t + 1)jx(t); d(t); tg. 2. Maximise the likelihood of the sequence of decisions: HY Pr d h (t h 0)jx h (t h 0) Pr d h (t h 1)jx h (t h 1) h=1 using the estimated Pr fx(t + 1)jx(t); d(t); tg to compute the present value functions necessary to calculate choice probabilities. 9

10 Unobserved heterogeneity The two-stage estimation procedure does not work if there exists unobserved heterogeneity. Assume that S h (t) = x h (t); " h (t); h where h 2 f1; :::; Mg indicates a particular way of grouping individuals. All individuals with the same h have a specic value of the parameters governing payo functions and state probabilities. Let Pr h = m = m, m 2 f1; :::; Mg. The likelihood becomes HY `(y h jx h (t h 0)) = HY! MX m`(y h jx h (t h 0); m) h=1 h=1 m=1 where `(y h jx h (t h 0); h ) = Pr d h (t h 0)jx h (t h 0); h Pr x h (t h 0 + 1)jx h (t h 0); h ; d h (t h 0) Pr d h (t h 0 + 1)jx h (t h 0 + 1); h Pr x h (t h 0 + 2)jx h (t h 0 + 1); h ; d h (t h 0 + 1) Pr d h (t h 1)jx h (t h 1); h : 10

11 EM algorithm Let y = (y 1 ; ; y H ) be a vector of observations. Let z = (z 1 ; ; z H ) be unobserved covariates. The likelihood of (y; z) is f(y; z; ). Since z is not observed one estimates by maximixing the integrated likelihood: Z f(y; ) = f(y; z; )(dz). This integral may be dicult to compute and the numerical approximation may yield unstable Newtontype optimisation algorithms (numerical errors accumulate instead of averaging). The EM algorithm is often preferable. The EM algorithm iterates the following steps until numerical convergence (generally slowly) where Q(j (p (p) = arg max 1) ) = E Z = Q(j (p 1) ); (p hln f(y; z; )jy; 1)i p n zjy; (p 1)o ln f(y; z; )(dz): Each iteration increases the likelihood and converges toward a local maximum of the likelihood. 11

12 Assume z i 2 f1; :::; Mg and m = Pr fz i = mg. EM algorithm: discrete mixtures Then = (; ) where indexes f(y i jz i ; ) and = ( 1 ; :::; M ). We have f(y; z; ) = HY f(y i ; z i ; ) = " HY X M # m f(y i jz i = m; ) : i=1 i=1 m=1 Step E (expectation): Use Bayes rule to compute posterior probabilities: and p Q(j (p 1) ) = n z i = mjy i ; (p 1)o = = Z p HX i=1 n zjy; (p MX p m=1 P M m=1 (p 1) m f(y i jz i = m; (p 1) ) (p 1) m f(y i jz i = m; (p 1) ) 1)o ln f(y; z; )(dz) n z i = mjy i ; (p 1)o ln [ m f(y i jz i = m; )] : 12

13 Step M (maximisation): Update by constrained ML: (p) = arg max HX i=1 MX p m=1 n z i = mjy i ; (p 1)o ln f(y i jz i = m; ); (i.e. n duplicate individual observations K times and aect a weight equal to posterior probability p z i = mjy i ; (p 1)o ) and update as (p) m = 1 H HX p i=1 n z i = mjy i ; (p 1)o : 13

14 Application: education and career choice See for example the presentation by Keane et Wolpin (JPE, 1997). Model of education and career choices. Data: 11-year panel (National Longitudinal Survey of Youths): cohort of youths aged 16 in 1979 and followed until Objective: evaluate policy eects such as education subsidies. Population studied is a cohort of individuals starting at the age of 16 and retiring at 65. Choices: blue collar worker (k = 1), white collar worker (k = 2), military (k = 3), education (k = 4) or inactivity (k = 5). 14

15 Model Payos associated to choices k = 1; 2; 3 are the corresponding wages, the log of which are ln R k (t) = e k (16) + e k1 EDUC(t) + e k2 EXP k (t) e k3 [EXP k (t)] 2 + " k (t) where e k (16) is the intercept (initial condition), EDUC(t) is the number of years of education, EXP k (t) is occupation-k specic experience (= nb of years spent working as k; with EXP k (16) = 0). Education's instantaneous payo (or cost if negative): Leisure utility: R 4 (t) = e 4 (16) c 1 1 [EDUC(t) 12] {z } HS graduate R 5 (t) = e 5 (16) + " 5 (t): c 2 1 [EDUC(t) 16] + " {z } 4 (t): college graduate State variable: S(t) = (e(16); EDUC(t); EXP (t); "(t)) with 8 >< e(16) = (e 1 (16); :::; e 5 (16)) ; EXP (t) = (EXP 1 (t); EXP 2 (t); EXP 3 (t)) ; >: "(t) = (" 1 (t); :::; " 5 (t)) : 15

16 Model (continued) Heterogeneity { four groups m = 1; 2; 3; 4. { e(16) = (e 1 (16); :::; e 5 (16)) group-specic. { as EDUC(16) = 9 or 10, assume dierent proportions of each type given EDUC(16): Pr f = mjeduc(16)g m;educ(16) : State probabilities: { "(t) = (" 1 (t); :::; " 5 (t)) iid and N (0; ), with Cov (" k (t); "`(t)) = 0 for ` or k 4 (i.e. only " 1 (t); " 2 (t); " 3 (t) corresponding to employment spells are correlated). { Education: EDUC(t + 1) = EDUC(t) + d 4 (t): { Experience: EXP k (t + 1) = EXP k (t) + d k (t). Value functions: V k (S(t); t) = R k (t) + E [V (S(t + 1); t + 1)jd k (t) = 1] where "(t + 1) is the only risk factor (not predetermined) in V (S(t + 1); t + 1) given d(t): 16

17 Value functions V k (S(t); t) = R k (t) + E [V (S(t + 1); t + 1)jd k (t) = 1] where "(t + 1) is the only risk factor (not predetermined) in V (S(t + 1); t + 1) given d(t), a Pour k = 1; 2; 3, ( EXP`(t + 1) = EXP`(t) + 1(` = k); ` = 1; 2; 3; EDUC(t + 1) = EDUC(t): Pour k = 4, ( Pour k = 5, ( EXP`(t + 1) = EXP`(t); ` = 1; 2; 3; EDUC(t + 1) = EDUC(t) + 1: EXP`(t + 1) = EXP`(t); ` = 1; 2; 3; EDUC(t + 1) = EDUC(t): 17

18 Likelihood Individual observations: y h (t) = d h (t); w h (t), t = 16; :::; 26, where d h (t) = d h 1(t); :::; d h 5(t) is occupation choice and w h (t) = P 3 k=1 dh k (t)rh k (t) is current wage (missing if not working). Sample likelihood: " HY H # X L = m;educ (16)`h(y h h (16); :::; y h (26)je h (16); EDUC h (16)) : h=1 h=1 Likelihood for individual h: `h(y h (16); y h (17); :::; y h (26)je h (16); EDUC h (16)) = Y26 t=16 `h y h (t)je h (16); EDUC h (t); EXP h (t) : 18

19 Likelihood (continued) Likelihood for individual h at time t: `h y h (t)je h (16); EDUC h (t); EXP h (t) is computed as follows (we omit conditioning to simplify notations). Dierent as general studied above as the wage information tells us about shocks " h k (t). Case d h (t) = k 2 f1; 2; 3g: one thus knows that w h (t) = R h k (t) and V k(s(t); t) V`(S(t); t), ` 6= k: 8 9 `h y h (t) >< >= = Pr >: V k(s h (t); t) V`(S h (t); t); 8` 6= kjrk(t) h = w h (t) {z } >; pdf Rk(t) h = w h (t) {z } determines " h k (t) Other cases: one only knows that V k (S h (t); t) V`(S h (t); t), ` 6= k: `h y h (t) = Pr V k (S h (t); t) V`(S h (t); t); 8` 6= k : i.e. density of R h k (t) at observation wh (t) : 19

20 Given e h (16); EDUC h (t); EXP h (t), 0 Likelihood (continued) pdf Rk(t) h = w h (t) = 1 z } { 1 ' ln w h (t) e k (16) e k1 EDUC(t) e k2 EXP k (t) + e k3 [EXP k (t)] 2 w h (t) k k C A " h k (t) 1 where 2 k = Var(" k(t)), and Pr V k (S h (t); t) V`(S h (t); t); 8` 6= k j " h k(t) = Pr " h` (t) g`(t); 8` 6= k j " k (t) where g`(t) = ln V k (S h (t); t) E V (S h (t + 1); t + 1)jd`(t) = 1 e`(16) e`1 EDUC(t) e`2 EXP k (t) + e`3 [EXP k (t)] 2 ; ` = 1; 2; 3 ; g 4 (t) = V k (S h (t); t) e 4 (16) c 1 1 [EDUC(t) 12] c 2 1 [EDUC(t) 16] ; g 5 (t) = V k (S h (t); t) e 5 (16): One has to compute the cdf of a vector of 4 normal r.v.'s. (Computation simplied by the fact that " h 4 (t) and " h 5 (t) are assumed independent and independent of " h 1 (t) ; " h 2 (t) and " h 3 (t). 20

21 Lastly, Pr V k (S h (t); t) V`(S h (t); t); 8` 6= k w.r.t. " h k (t). can be computed by numerical integration of Pr V k (S h (t); t) V 21

22 Results See article. The t is excellent. They nd a very limited eect of college tuition subsidies (exogenous change in c 2 ). 22