ECAS Summer Course. Quantile Regression for Longitudinal Data. Roger Koenker University of Illinois at Urbana-Champaign

Transcription

1 ECAS Summer Course 1 Quantile Regression for Longitudinal Data Roger Koenker University of Illinois at Urbana-Champaign La Roche-en-Ardennes: September 2005 Part I: Penalty Methods for Random Effects Part II: Estimation of Reference Growth Charts Part II is based on joint work with: Ying Wei, Columbia, Biostatistics Anneli Pere, Children s Hospital, Helsinki and Xuming He, UIUC.

2 Penalty Methods and Random Effects 2 Classical Models with Independent Random Effects Additive random effects have nice conditional mean functions and nasty conditional quantile functions. Transformations to annihilate fixed effects fail for quantile regression models

3 Penalty Methods and Random Effects 2 Classical Models with Independent Random Effects Additive random effects have nice conditional mean functions and nasty conditional quantile functions. Transformations to annihilate fixed effects fail for quantile regression models Classical Models with Fixed Effects Fixed effect models are computationally tractable Penalty methods for fixed effect models offer an alternative to classical random effects estimation

4 A Classical Result 3 Model: or, y ij = x ijβ + α i + u ij j = 1,...m i, i = 1,..., n, y = Xβ + Zα + u. If u and α are independent Gaussian vectors with u N (0, R) and α N (0, Q), the optimal weighted least squares estimator is, ˆβ = (X (R + ZQZ ) 1 X) 1 X (R + ZQZ ) 1 y.

5 A Classical Result 3 Model: or, y ij = x ijβ + α i + u ij j = 1,...m i, i = 1,..., n, y = Xβ + Zα + u. If u and α are independent Gaussian vectors with u N (0, R) and α N (0, Q), the optimal weighted least squares estimator is, ˆβ = (X (R + ZQZ ) 1 X) 1 X (R + ZQZ ) 1 y. Theorem ˆβ = argmin (α,β) y Xβ Zα 2 R 1 + α 2 Q 1, where x 2 A = x Ax.

6 Commentary 4 History Henderson (1950) on dairy cows, Goldberger (1962) on BLUP, Lindley and Smith (1972) on Bayesian hierarchical models, and Robinson (1991) for a survey.

7 Commentary 4 History Henderson (1950) on dairy cows, Goldberger (1962) on BLUP, Lindley and Smith (1972) on Bayesian hierarchical models, and Robinson (1991) for a survey. Interpretation Maximum likelihood estimations for the Gaussian random effects models is identical to penalized least squares estimation of the fixed effects model. Random effects is just shrinkage estimation.

8 Commentary 4 History Henderson (1950) on dairy cows, Goldberger (1962) on BLUP, Lindley and Smith (1972) on Bayesian hierarchical models, and Robinson (1991) for a survey. Interpretation Maximum likelihood estimations for the Gaussian random effects models is identical to penalized least squares estimation of the fixed effects model. Random effects is just shrinkage estimation. Ideology Our focus is on design of effective algorithms for data analysis. Models are useful as an interpretative device, but they shouldn t be allowed to become sacred objects.

9 Quantile Regression with Fixed Effects 5 Model: Q yij (τ x ij ) = α i + x ijβ(τ) j = 1,...m i, i = 1,..., n.

10 Quantile Regression with Fixed Effects 5 Model: Q yij (τ x ij ) = α i + x ijβ(τ) j = 1,...m i, i = 1,..., n. Estimator: With (the inevitable) ρ τ (u) = u(τ I(u < 0)), min (α,β) q k=1 n j=1 m i i=1 w k ρ τk (y ij α i x ijβ(τ k ))

11 Quantile Regression with Fixed Effects 5 Model: Q yij (τ x ij ) = α i + x ijβ(τ) j = 1,...m i, i = 1,..., n. Estimator: With (the inevitable) ρ τ (u) = u(τ I(u < 0)), min (α,β) q k=1 n j=1 m i i=1 w k ρ τk (y ij α i x ijβ(τ k )) Implementation: Linear programming formulation Interior point algorithms exploiting sparse linear algebra

12 Quantile Regression with Penalized Fixed Effects 6 Model: (as before) Q yij (τ x ij ) = α i + x ijβ(τ) j = 1,...m i, i = 1,..., n. Estimator: min (α,β) q k=1 Limiting Forms: n j=1 m i i=1 w k ρ τk (y ij α i x ijβ(τ k )) + λ q α i i=1 as λ 0 we obtain fixed effects, as λ we force all α i to 0,

13 L 1 versus L 2 Shrinkage 7 α α λ λ Figure 1: Shrinkage of the fixed effect parameter estimates, ˆα i. The left panel illustrates an example of the l 1 shrinkage effect. The right panel illustrates an example of the l 2 shrinkage effect. There are m = 5 observations on each of n = 50 cross-sectional units.

14 Sparsity 8 row column Figure 2: Sparsity of the design matrix for the penalized fixed effects quantile regression estimator. Grey areas indicate portions of the matrix that are nonzero, white space indicates zeros. Here, n = 50 and m i = 5 for all i. Three quantiles are being simultaneously estimated with the fixed effects restricted to be identical across quantiles.

15 Regularity Conditions 9 A1. The y ij are independent with conditional distribution functions, F ij, given x ij, and differentiable conditional densities, 0 < f ij <, with bounded derivatives f ij, at ξ ij(τ), for j = 1,..., m, i = 1,..., n, A2. Let ω = τ(1 τ) and denote Φ = diag(f ij (ξ ij (τ))). The limiting forms of the following matrices are positive definite: D 0 = lim m n ω m ( ( D 1 = lim m 1 m n A3. max 1 i n x ij < M. 1 j m Z Z X Z/ n Z ΦZ X ΦZ/ n Z X/ n X X/n ), Z ΦX/ n X ΦX/n ).

16 A Theorem 10 Theorem 1. Under conditions A1-3, with n a /m 0 for some a > 0, the ˆδ 1 component of the minimizer, ˆδ, converges in distribution to a Gaussian random vector with mean zero and covariance matrix given by the lower p by p block of the matrix D1 1 D 0D1 1.

17 More Regularity Conditions 11 B1. The y ij are independent with conditional distribution functions, F ij, given x ij, and differentiable conditional densities, 0 < f ij <, with bounded derivatives f ij, at ξ ij(τ), for j = 1,..., m, i = 1,..., n, B2. Let Ω denote the q by q matrix with typical element τ k τ l τ k τ l and Φ k = diag(f ij (ξ ij (τ k ))). The limiting forms of the following matrices are positive definite: ( D 0 = lim m 1 m n D 1 = lim m 1 m n w ΩwZ Z W Ωw X Z/ n w ΩW Z X/ n W ΩW X X/n ), wk Z Φ k Z w 1 Z Φ 1 X/ n w q Z Φ q X/ n w 1 X Φ 1 Z/ n w 1 X Φ 1 X/n w q X Φ q Z/ n 0 w q X Φ q X/n.

18 B3. max 1 i n x ij < M. 1 j m 12

19 Another Theorem 13 Theorem 2. Under conditions B1-3, provided that λ m / m λ 0, and n a /m 0 for some a > 0, the first component ˆδ 1 minimizing V mn has the same limiting distribution as the as the first component of the minimizer of, V 0 (δ) = δ B δ D 1 δ + λ 0 δ s where B is a zero mean Gaussian vector with covariance matrix D 0, and s = (s 0 0 pq) and s 0 = (sgn(α i )).

20 Monte-Carlo Models 14 We consider two very simple models: and where y ij = α i + x ij β + u ij (1) y ij = α i + x ij β + (1 + x ij γ)u ij. (2) x ij = δ i + v ij (3) with γ i and v ij independent and identically distributed over i and i, j respectively, then the interclass correlation coefficient, ρ x = σ 2 δ/(σ 2 δ + σ 2 u),

21 Monte Carlo: Location Shift Model 15 LS PLS LSFE QR PQR QRFE N Bias RMSE t 3 Bias RMSE χ 2 3 Bias RMSE

22 Monte Carlo: Location-Scale Shift Model 16 LS PLS LSFE QR PQR QRFE N Bias RMSE t 3 Bias RMSE χ 2 3 Bias RMSE

23 Conclusions 17 Classical random effects in Gaussian models is just penalized least squares estimation in which fixed effects are shrunken toward their prior mean of zero. Other penalties may be useful in other settings.

24 Conclusions 17 Classical random effects in Gaussian models is just penalized least squares estimation in which fixed effects are shrunken toward their prior mean of zero. Other penalties may be useful in other settings. L 1 penalty methods have attractive properties in a variety of model selection, image analysis and function estimation problems. Interior point algorithms exploiting inherently sparse linear algebra facilate efficient computations.

25 Conclusions 17 Classical random effects in Gaussian models is just penalized least squares estimation in which fixed effects are shrunken toward their prior mean of zero. Other penalties may be useful in other settings. L 1 penalty methods have attractive properties in a variety of model selection, image analysis and function estimation problems. Interior point algorithms exploiting inherently sparse linear algebra facilate efficient computations. There is more to longitudinal data analysis than the estimation of location shift effects. Many interesting questions can only be addressed with broader models that incorporate distributional effects for treatments and other covariates.

26 Quantile Regression Methods for Reference Growth Charts 18 Roger Koenker University of Illinois at Urbana-Champaign Based on joint work with: Ying Wei, Columbia, Biostatistics Anneli Pere, Children s Hospital, Helsinki Xuming He, UIUC.

27 Quetelet s (1871) Growth Chart 19

28 Penalized Maximum Likelihood Estimation 20 References: Cole (1988), Cole and Green (1992), and Carey(2002) Data: {Y i (t i,j ) : j = 1,..., J i, i = 1,..., n.} Model: Z(t) = (Y (t)/µ(t))λ(t) 1 λ(t)σ(t) N (0, 1) Estimation: max l(λ, µ, σ) ν λ (λ (t)) 2 dt ν µ (µ (t)) 2 dt ν σ (σ (t)) 2 dt, l(λ, µ, σ) = n [λ(t i ) log(y (t i )/µ(t i )) log σ(t i ) 1 2 Z2 (t i )], i=1

29 Quantiles as Argmins 21 The τ th quantile of a random variable Y having distribution function F is: α(τ) = argmin ρ τ (y α)df (y) where ρ τ (u) = u (τ I(u < 0)).

30 Quantiles as Argmins 21 The τ th quantile of a random variable Y having distribution function F is: α(τ) = argmin ρ τ (y α)df (y) where ρ τ (u) = u (τ I(u < 0)). The τth sample quantile is thus: ˆα(τ) = argmin ρ τ (y α)df n (y) n = argmin n 1 ρ τ (y i α) i=1

31 Quantile Regression 22 The τth conditional quantile function of Y X = x is g(τ x) = argmin g G ρ τ (y g(x))df A natural estimator of g(τ x) is ĝ(τ x) = argmin g G n i=1 ρ τ (y i g(x i )) with G chosen as a finite dimensional linear space, g(x) = p ϕ j (x)β j. j=1

32 Choice of Basis 23 There are many possible choices for the basis expansion {ϕ j }. We opt for the (very conventional) cubic B-spline functions: Age In R these quantile regression models can be estimated with the command. fit <- rq(y bs(x,knots=knots),tau = 1:9/10) Similar functionality in SAS is coming real soon now.

33 Data 24 Longitudinal measurements on height for 2514 Finnish children,

34 Data 24 Longitudinal measurements on height for 2514 Finnish children, 1143 boys, 1162 girls all healthy, full-term, singleton births,

35 Data 24 Longitudinal measurements on height for 2514 Finnish children, 1143 boys, 1162 girls all healthy, full-term, singleton births, About 20 measurements per child,

36 Data 24 Longitudinal measurements on height for 2514 Finnish children, 1143 boys, 1162 girls all healthy, full-term, singleton births, About 20 measurements per child, Two cohorts: 1096 born between , 1209 born between

37 Data 24 Longitudinal measurements on height for 2514 Finnish children, 1143 boys, 1162 girls all healthy, full-term, singleton births, About 20 measurements per child, Two cohorts: 1096 born between , 1209 born between Sample constitutes 0.5 percent of Finns born in these periods.

38 Unconditional Reference Quantiles Boys Years Box Cox Parameter Functions λ(t) Height (cm) µ(t) LMS edf = (7,10,7) LMS edf = (22,25,22) QR edf = 16 σ(t) Age (years) Age (years)

39 26

40 Unconditional Reference Quantiles Boys 2 18 Years Box Cox Parameter Functions λ(t) µ(t) Height (cm) LMS edf = (7,10,7) σ(t) LMS edf = (22,25,22) QR edf = Age (years) Age (years)

41 28

42 Unconditional Reference Quantiles Girls Years Box Cox Parameter Functions λ(t) µ(t) 90 Height (cm) LMS edf = (7,10,7) σ(t) LMS edf = (22,25,22) QR edf = Age (years) Age (years)

43 30

44 31 Unconditional Reference Quantiles Girls 2 18 Years Box Cox Parameter Functions λ(t) Height (cm) µ(t) LMS edf = (7,10,7) σ(t) LMS edf = (22,25,22) QR edf = Age (years) Age (years)

45 32

46 33 Growth Velocity Curves Boys years Girls years τ Boys 2 18 years Girls 2 18 years LMS (7,10,7) LMS (22,25,22) QR Velocity (cm/year) Age (years)

47 34

48 35 Estimated Age Specific Density Functions Age = 0.5 Age = 1 Age = 1.5 N= 235 N= 204 N= Boys Data QR LMS N= 215 N= 191 N= Girls Height (cm)

49 36

50 37 Height Density at Age Boys Pooled Born in 1960 Born in Girls Height(cm)

51 Conditioning on Prior Growth 38 It is often important to condition not only on age, but also on prior growth and possibly on other covariates. Autoregressive models are natural, but complicated due to the irregular spacing of typical longitudinal measurements. Data: {Y i (t i,j ) : j = 1,..., J i, i = 1,..., n.} Model: Q Yi (t i,j )(τ t i,j, Y i (t i,j 1 ), x i ) = g τ (t i,j ) + [α(τ) + β(τ)(t i,j t i,j 1 )]Y i (t i,j 1 ) + x i γ(τ).

52 AR Components of the Infant Conditional Growth Model 39 τ Boys Girls ˆα(τ) ˆβ(τ) ˆγ(τ) ˆα(τ) ˆβ(τ) ˆγ(τ) (0.020) (0.011) (0.011) (0.024) (0.011) (0.020) (0.019) (0.025) (0.029) (0.024) (0.024) (0.007) (0.006) (0.009) (0.009) (0.016) (0.016) (0.007) (0.009) (0.013) (0.017) (0.017) (0.018) (0.022) (0.021) (0.027) (0.027) (0.030) (0.038) (0.007) (0.006) (0.008) (0.012) (0.015) (0.024) (0.010) (0.009) (0.008) (0.009) (0.015) (0.018) (0.027)

53 AR Components of the Childrens Conditional Growth Model 40 τ Boys Girls ˆα(τ) ˆβ(τ) ˆγ(τ) ˆα(τ) ˆβ(τ) ˆγ(τ) (0.010) (0.002) (0.013) (0.012) (0.002) (0.005) (0.006) (0.004) (0.004) (0.009) (0.014) (0.001) (0.001) (0.001) (0.001) (0.001) (0.002) (0.007) (0.006) (0.004) (0.006) (0.009) (0.015) (0.006) (0.005) (0.007) (0.007) (0.008) (0.013) (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) (0.015) (0.007) (0.006) (0.006) (0.006) (0.012) (0.018)

54 Transformation to Normality 41 A presumed advantage of univariate (age-specific) transformation to normality is that once observations are transformed to univariate Z-scores they are automatically prepared to longitudinal autoregression: Z t = α 0 + α 1 Z t 1 + U t Premise: Marginal Normality Joint Normality

55 Transformation to Normality 41 A presumed advantage of univariate (age-specific) transformation to normality is that once observations are transformed to univariate Z-scores they are automatically prepared to longitudinal autoregression: Z t = α 0 + α 1 Z t 1 + U t Premise: Marginal Normality Joint Normality Of course we know it isn t true, but we also think we know that counterexamples are pathological, and don t occur in nature.

56 Boys age 6.5 Theoretical Quantiles Sample Quantiles Boys age 6 Theoretical Quantiles Sample Quantiles Boys AR(1) residuals Theoretical Quantiles Sample Quantiles Girls age 6.5 Theoretical Quantiles Sample Quantiles Girls age 6 Theoretical Quantiles Sample Quantiles Girls AR(1) residuals Theoretical Quantiles Sample Quantiles

57 43

58 44 Height (cm) Subject 89 Subject Age (years)

59 45 1 Subject 89 Subject Quantile Unconditional Conditional Observed Height (cm) Height (cm) 0

60 Conclusions 46 Nonparametric quantile regression using B-splines offers a reasonable alternative to parametric methods for constructing reference growth charts.

61 Conclusions 46 Nonparametric quantile regression using B-splines offers a reasonable alternative to parametric methods for constructing reference growth charts. The flexibility of quantile regression methods exposes features of the data that are not easily observable with conventional parametric methods. Even for height data.

62 Conclusions 46 Nonparametric quantile regression using B-splines offers a reasonable alternative to parametric methods for constructing reference growth charts. The flexibility of quantile regression methods exposes features of the data that are not easily observable with conventional parametric methods. Even for height data. Longitudinal data can be easily accomodated into the quantile regression framework by adding covariates, including the use of autoregressive effects for unequally spaced measurements.