Discrete panel data. Michel Bierlaire

Transcription

1 Discrete panel data Michel Bierlaire Transport and Mobility Laboratory School of Architecture, Civil and Environmental Engineering Ecole Polytechnique Fédérale de Lausanne M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 1 / 40

2 Outline Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 2 / 40

3 Introduction Introduction Panel data Type of data used so far: cross-sectional. Cross-sectional: observation of individuals at the same point in time. Time series: sequence of observations. Panel data is a combination of comparable time series. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 3 / 40

4 Introduction Introduction Panel data Data collected over multiple time periods for the same sample of individuals. Multidimensional Individual Day Price of stock 1 Price of stock 2 Purchase n t x 1nt x 2nt i int M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 4 / 40

5 Introduction Introduction Examples of discrete panel data People are interviewed monthly and asked if they are working or unemployed. Firms are tracked yearly to determine if they have been acquired or merged. Consumers are interviewed yearly and asked if they have acquired a new cell phone. Individual s health records are reviewed annually to determine onset of new health problems. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 5 / 40

6 Introduction Model: single time period x ε U i M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 6 / 40

7 Static model Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 7 / 40

8 Static model Static model x t 1 x t ε t 1 ε t U t 1 U t i t 1 i t M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 8 / 40

9 Static model Static model Utility U int = V int +ε int, i C nt. Logit P(i nt ) = e V int j C nt e V jnt Estimation: contribution of individual n to the log likelihood T P(i n1,i n2,...,i nt ) = P(i n1 )P(i n2 ) P(i nt ) = P(i nt ) T lnp(i n1,i n2,...,i nt ) = lnp(i n1 )+lnp(i n2 )+ +lnp(i nt ) = lnp(i nt ) t=1 M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 9 / 40 t=1

10 Static model Static model: comments Views observations collected through time as supplementary cross sectional observations. Standard software for cross section discrete choice modeling may be used directly. Simple, but there are two important limitations: Serial correlation unobserved factors persist over time, in particular, all factors related to individual n, ε in(t 1) cannot be assumed independent from ε int. Dynamics Choice in one period may depend on choices made in the past. e.g. learning effect, habits. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 10 / 40

11 Static model with panel effect Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 11 / 40

12 Static model with panel effect Dealing with serial correlation x t 1 x t ε t 1 ε t U t 1 U t i t 1 i t M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 12 / 40

13 Static model with panel effect Panel effect Relax the assumption that ε int are independent across t. Assumption about the source of the correlation individual related unobserved factors, persistent over time. The model ε int = α in +ε int It is also known as agent effect, unobserved heterogeneity. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 13 / 40

14 Static model with panel effect Panel effect Assuming that ε int are independent across t, we can apply the static model. Two versions of the model: with fixed effect: α in are unknown parameters to be estimated, with random effect: α in are distributed. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 14 / 40

15 Static model with panel effect Static model with fixed effect Utility U int = V int +α in +ε int, i C nt. Logit P(i nt ) = e V int+α in j C nt e V jnt+α jn Estimation: contribution of individual n to the log likelihood T P(i n1,i n2,...,i nt ) = P(i n1 )P(i n2 ) P(i nt ) = P(i nt ) T lnp(i n1,i n2,...,i nt ) = lnp(i n1 )+lnp(i n2 )+ +lnp(i nt ) = lnp(i nt ) t=1 M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 15 / 40 t=1

16 Static model with panel effect Static model with fixed effect Comments α in capture permanent taste heterogeneity. For each n, one α in must be normalized to 0. The α s are estimated consistently only if T. This has an effect on the other parameters that will be inconsistently estimated. In practice, T is usually too short, the number of α parameters is usually too high, for the model to be consistently estimated and practical. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 16 / 40

17 Static model with panel effect Static model with random effect Denote α n the vector gathering all parameters α in. Assumption: α n is distributed with density f(α n ). For instance: α n N(0,Σ). We have a mixture of static models. Given α n, the model is static, as ε int are assumed independent across t. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 17 / 40

18 Static model with panel effect Static model with random effect Utility Conditional choice probability U int = V int +α in +ε int, i C nt. P(i nt α n ) = e V int+α in j C nt e V jnt+α jn M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 18 / 40

19 Static model with panel effect Static model with random effect Contribution of individual n to the log likelihood, given α n T P(i n1,i n2,...,i nt α n ) = P(i nt α n ). Unconditional choice probability P(i n1,i n2,...,i nt ) = α t=1 t=1 T P(i nt α)f(α)dα. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 19 / 40

20 Static model with panel effect Static model with random effect Estimation Mixture model. Requires simulation for large choice sets. Generate draws α 1,...,α R from f(α). Approximate P(i n1,i n2,...,i nt ) = T P(i nt α)f(α)dα 1 R α t=1 R T P(i nt α r ) r=1 t=1 The product of probabilities can generate very small numbers. ( R T R T ) P(i nt α r ) = exp lnp(i nt α r ). r=1 t=1 r=1 t=1 M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 20 / 40

21 Static model with panel effect Static model with random effect Comments Parameters to be estimated: β s and σ s Maximum likelihood estimation leads to consistent and efficient estimators. Ignoring the correlation (i.e. assuming that α n is not present) leads to consistent but not efficient estimators (not the true likelihood function). Accounting for serial correlation generates the true likelihood function and, therefore, the estimates are consistent and efficient. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 21 / 40

22 Dynamic model Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 22 / 40

23 Dynamic model Dynamics Choice in one period may depend on choices made in the past e.g. learning effect, habits. Simplifying assumption: the utility of an alternative at time t is influenced by the choice made at time t 1 only. It leads to a dynamic Markov model. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 23 / 40

24 Dynamic model Dynamic Markov model x t 1 x t ε t 1 ε t U t 1 U t i t 1 i t M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 24 / 40

25 Dynamic model Dynamic Markov model The model U int = V int +γy in(t 1) +ε int, i C nt. { 1 if alternative i was chosen by n at time t 1 y in(t 1) = 0 otherwise. Captures serial dependence on past realized state Example - utility of bus today depends on whether consumer took bus yesterday (habit). Fails if utility of bus today depends on permanent individual taste for bus (tastes) and whether consumer took bus yesterday. No serial correlation. Estimation: same as for the static model except that observation t = 0 is lost M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 25 / 40

26 Dynamic model with panel effect Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 26 / 40

27 Dynamic model with panel effect Dynamic Markov model with serial correlation x t 1 x t ε t 1 ε t U t 1 U t i t 1 i t M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 27 / 40

28 Dynamic model with panel effect Dynamic Markov model Extension: combine Markov with panel effect U int = V int +α in +γy in(t 1) +ε int, i C nt. Dynamic Markov model with fixed effect Similar to the static model with FE. Similar limitations. Dynamic Markov model with random effect Difficulties depending on how the Markov chain starts. If the first choice i 0 is truly exogenous similar to the static model with RE. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 28 / 40

29 Dynamic model with panel effect Dynamic Markov model What if i n0 is not exogenous (i.e. stochastic)? U in1 = V in1 +α in +γy in0 +ε in1, i C n1. The first choice i n0 is dependent on the agent s effect α in. So, the explanatory variable y in0 is correlated with α in. This is called endogeneity. Solution: use the Wooldridge approach. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 29 / 40

30 Dynamic model with panel effect Dynamic Markov model with RE - Wooldridge Conditional on y in0, we have a dynamic Markov model with RE as before U int = V int +α in +γy in(t 1) +ε int, i C nt. Contribution of individual n to the log likelihood, given i n0 and α n T P(i n1,i n2,...,i nt i n0,α n ) = P(i nt i n0,α n ). t=1 We integrate out α n P(i n1,i n2,...,i nt i n0 ) = T P(i nt i n0,α)f(α i n0 )dα. α t=1 M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 30 / 40

31 Dynamic model with panel effect Dynamic Markov model with RE - Wooldridge The main difference between static model with RE and dynamic model with RE is the term f(α i n0 ) It captures the distribution of the panel effects, knowing the first choice. This can be approximated by, for instance, α n = a+by n0 +cx n +ξ n, ξ n N(0,Σ α ). a, b and c are vectors and Σ α a matrix of parameters to be estimated. x n capture the entire history (t = 1,...,T) for agent n. This addresses the endogeneity issue. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 31 / 40

32 Application Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 32 / 40

33 Application Application Cherchi and Ortuzar (2002) Mixed RP/SP models incorporating interaction effects, Transportation 29(4), pp M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 33 / 40

34 Application Application Context Study done in 1998, Sardinia Island, Italy Cagliari-Assimini corridor (20km) Modal shares: car (75%), bus (20%), train (3%), other (2%) RP/SP data. Not time series, but panel structure of SP data. t is the index of the choice experiment instead of time. t = 0 corresponds to the RP observation. Panel effect is captured. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 34 / 40

35 Application Application Estimation results Logit with panel effect Variable Estimate t-test Estimate t-test Cte. train Cte. car Travel time (min) Travel cost/wage rate (euros) Waiting time (min) Comfort low Comfort avg Transfers Panel effect std. dev Log likelihood ρ M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 35 / 40

36 Application Application Average value of time by purpose (euros/min) Logit with panel effect Work 321 obs Study 285 obs Personal business 164 obs Leisure 64 obs M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 36 / 40

37 Application Application Comments Panel effect is significant. Significant improvement of the fit. With small samples, the gain in efficiency obtained from the panel effect may significantly improve the estimates. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 37 / 40

38 Summary Outline 1 Introduction 2 Static model 3 Static model with panel effect 4 Dynamic model 5 Dynamic model with panel effect 6 Application 7 Summary M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 38 / 40

39 Summary Summary Static model Straightforward extension of cross-sectional specification. Two main limitations: serial correlation and dynamics. Panel effect Deals with serial correlation. Fixed effect: Static model with additional parameters. Not operational in most practical cases. Random effect: Modifies the log likelihood function. Must integrate the product of the choice probabilities over time. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 39 / 40

40 Summary Summary Dynamic model, with a Markov assumption Static model with an additional variable: the previous choice. Dynamic model with panel effect Both can be combined. Must capture the relation between the first choice and the panel effect. Application Illustrates the importance of the panel effect. M. Bierlaire (TRANSP-OR ENAC EPFL) Discrete panel data 40 / 40