Outline. Clustering. Capturing Unobserved Heterogeneity in the Austrian Labor Market Using Finite Mixtures of Markov Chain Models

Transcription

1 Capturing Unobserved Heterogeneity in the Austrian Labor Market Using Finite Mixtures of Markov Chain Models Collaboration with Rudolf Winter-Ebmer, Department of Economics, Johannes Kepler University Linz Supported by the Austrian Science Foundation (FWF) under grant P ( Gibbs Sampling for Discrete Data ) Sylvia Frühwirth-Schnatter and Christoph Pamminger Department of Applied Statistics and Econometrics Johannes Kepler University Linz, Austria UseR! 2006 p. 1 UseR! 2006 p. 2 Clustering Motivating Example Research Question Data Description Markov Chain Model Outline Clustering Clustering is a widely used statistical tool to determine subsets Frequently used clustering methods are based on distance-measures However, distance-measures are difficult to define for more complex data (e.g. time series) Dirichlet Multinomial Model Bayesian Analysis MCMC-Estimation Estimation Results Model-based clustering methods (mixture models) We present an approach for model-based clustering of discrete-valued time series data following ideas discussed in Frühwirth-Schnatter and Kaufmann (2004) UseR! 2006 p. 3 UseR! 2006 p. 4

2 Motivating Example Wage Mobility in the Austrian labor market Describes chances but also risks of an individual to move between wage categories Assumption of different career progressions or income careers of employees Task: Find groups of employees with similar behavior in terms of transition probabilities (focus on one-year transitions) Data provided by the Austrian social security authority Data Description Time series for N = 9, 809 individuals (only men, because of data inconsistencies with e.g. female part-time workers) Gross monthly wage at May of successive years (with individual length T i ) divided into 6 categories corresponding to quintiles of the particular income distribution (1-5) and zero-income (0) according to Weber (2002) y i = (y i0,y i1,y i2,...,y it,...,y i,ti ), i = 1,...,N Income careers of the first four employees in the data set [1] [2] [3] [4] Illustration UseR! 2006 p. 5 y it = k Markov Chain Model if subject i {1,...,N} belongs to wage category k {0, 1,...,K} in year t {0,...,T i } Markov chain y i is modeled with a (time-homogeneous) Markov process with unknown transition matrix ξ, where K ξ jk = P{y it = k y i,t 1 = j} and ξ jk = 1 ξ = ξ 0 ξ 1. ξ K k=0 ξ 00 ξ 01 ξ 0K = ξ 10 ξ 11 ξ 1K..... ξ K0 ξ K1 ξ KK UseR! 2006 p Figure 1: Individual wage mobility time series of nine selected employees. UseR! 2006 p. 7 UseR! 2006 p. 8

3 Bayesian Analysis Prior-distribution of ξ j, j = 0,...,K: Posterior-distribution of ξ j : ξ j D(e 0,j0,...,e 0,jK ). ξ j D(e N,j0,...,e N,jK ) with e N,jk = e 0,jk + N jk, where N jk = #{y it = k,y i,t 1 = j} is the number of transitions from state j to state k over all subjects i = 1,...,N Modeling Hidden Groups Assumptions and notations H hidden groups with group-specific transition matrices ξ h, h = 1,...,H Individual transition matrices ξ s i, i = 1,...,N Latent indicator variable S = (S 1,...,S N ) for group membership: S i = h, if subject i belongs to group h Relative group sizes η = (η 1,...,η H ): P{S i = h η} = η h, h = 1,...,H ξ product of (K + 1 indep.) Dirichlet-distributions UseR! 2006 p. 9 UseR! 2006 p. 10 Modeling Heterogeneity 1. Simple model: ξ s i (S i = h) = ξ h (fixed) ξ h S product of (K + 1 indep.) Dirichlet-distributions 2. Apply a multinomial logit model with random effects (Rossi et al., 2005). High-parametrical model including high-dimensional covariance matrices 3. Dirichlet Multinomial Model: ξ s i,j (S i = h) D(e h,j0,...,e h,jk ) with group-specific parameter e h = {e h,j }, j = 0,...,K UseR! 2006 p. 11 Dirichlet Multinomial Model Group-specific transition matrix ξ h is given by ξ h,jk = E(ξ s i,jk S i = h,e h ) = e h,jk K k=0 e h,jk So each row of e h determines the corresponding row of ξ h Finite mixture model representation: Y i p h (y i e h )... product of K + 1 Dirichlet-distributions Unconditional density: p(y i e 1,...,e H ) = H η h p h (y i e h ) h=1 UseR! 2006 p. 12

4 Group-specific parameter e h The variance of ξ s i,jk is given by V ar(ξ s i,jk S i = h,e h ) = ξ 2 h,jk l k e h,jl K k=0 e h,jk (1 + ) K k=0 e h,jk If K k=0 e h,jk is very large (for each row in each group) amount of heterogeneity (in each group) is small leads to the simple model with fixed ξ h If K k=0 e h,jk is small the individual transition matrices are allowed to deviate from the group mean within each group Prior-assumptions: Bayesian Analysis All e h,j are independent and e h,j 1 0 (to avoid problems with empty groups and non-informative priors) e h,j 1 is a discrete-valued multivariate random variable e h,j 1 negative multinomial distribution η Dirichlet-distribution All parameters e 1,...,e H, S, η are jointly estimated by means of MCMC-Sampling UseR! 2006 p. 13 MCMC-Estimation (Gibbs Sampler) Choose initial values for η and e 1,...,e H (H fixed in advance) and repeat following steps (m = 1,...,M): 1. Bayes-classification for each subject i: draw S (m) i from p(s i y i,η (m 1),e (m 1) 1,...,e (m 1) H ). 2. sample Group sizes η: draw η (m) from D(α (m) 1,...,α (m) α (m) h = N (m) h = h}. 3. sample group-specific parameters e 1,...,e H : + α 0 and N (m) h H ) with = #{S (m) i draw e (m) h,j row-by-row from p(e h,j y,s (m) ) (not of closed form!) using a Metropolis-Hastings step (with discrete random walk proposal). Estimation Results Here we show the results for 3 groups which allow very sensible interpretations according to our economist (M = 10,000 with 2,000 burn-in) Transition probabilities Typical group members Classification probabilities Equilibrium distributions UseR! 2006 p. 14 UseR! 2006 p. 15 UseR! 2006 p. 16

5 Transition Probabilities Typical Group Members member of group 1 member of group 1 member of group 1 ti.1 ti 0 1 S = 1 ( ) ti.1 ti 0 1 S = 2 ( ) ti.1 ti 0 1 S = 3 ( ) member of group member of group member of group Figure 2: 3D-Visualizations of transition probabilities ˆξ h (volumes of balls are proportional to probs) and estimated group sizes ˆη indicated in brackets (posterior means). member of group 3 member of group 3 member of group 3 Classification Probabilities UseR! 2006 p. 17 Figure 3: Selected typical group members (with high classification prob). Equilibrium Distributions UseR! 2006 p. 18 i\h j\h Table 2: Equilibrium distributions in each group. Table 1: Classification probabilities for each individual. UseR! 2006 p. 19 UseR! 2006 p. 20

6 Open Problem Further research has to be done to find formal criterions to determine the number of groups. Possible approaches: Model selection based on marginal likelihoods Classification likelihood information criterion (using entropy) Integrated classification likelihood Summary Discrete-valued time series Categorical variable Markov chains Individual transition matrices Dirichlet multinomial model (allows for heterogeneity within groups): mixture model with (products of) Dirichlet-distributions with group-specific parameters Estimation via MCMC (number of groups fixed) Group-specific transition matrices UseR! 2006 p. 21 UseR! 2006 p. 22 References Frühwirth-Schnatter, Sylvia (2006). Finite Mixture and Markov Switching Models. Springer Series in Statistics. New York: Springer (to appear). Frühwirth-Schnatter, Sylvia and Kaufmann, Sylvia (2004). Model-Based Clustering of Multiple Time Series. IFAS Research Paper Series, , Rossi, Peter E., Allenby, Greg and McCulloch, Rob (2005). Bayesian Statistics and Marketing. John Wiley and Sons. Weber, Andrea (2002). State Dependence and Wage Dynamics: A Heterogeneous Markov Chain Model for Wage Mobility in Austria, Economics Series 114, Institute for Advanced Studies. UseR! 2006 p. 23