Interval Regression with Sample Selection

Transcription

1 Interval Regresson wth Sample Selecton Arne Hennngsen, Sebastan Petersen, Géraldne Hennngsen February 9, 09 Ths vgnette s largely based on Petersen et al. 07. Model Specfcaton The general specfcaton of an nterval regresson model wth sample selecton s: ε S ε O y S β S x S + ε S { y S 0 f y S 0 otherwse y O β O x O + ε O 3 unknown f y S 0 f < y O and y S y O f < y O 3 and y S. M 0 N 0, [ ρ ρ 4 f M < y O M+ and y S ], 5 where subscrpt ndcates the observaton, y O s a latent outcome varable, y O s a partally observed categorcal varable that ndcates n whch nterval y O les, M s the number of ntervals,,..., M+ are the boundares of the ntervals whereas frequently but not necessarly and M+, y S s a bnary varable that ndcates whether y O s observed, y S s a latent varable that ndcates the tendency that y S s one, x S and x O are column vectors of explanatory varables for the selecton equaton and outcome equaton, respectvely, ε S and ε O are random dsturbance terms that have a jont bvarate normal dstrbuton, and β S and β O are column vectors and ρ and are scalars of unknown model parameters.

2 Log-Lkelhood Functon The probablty that y O s unobserved s: P y S 0 P y S 0 6 P β S x S + ε S 0 7 P ε S β S x S 8 The probablty that y O s observed and ndcates that y O les n the mth nterval s: P y S y O m P y S > 0 m < y O m+ 9 β S x S + ε S > 0 m < β O x O + ε O m+ P P ε S > β S x S m β O x O < ε O m+ β O x O 0 The log-lkelhood contrbuton of the th observaton s: [ ] l y S ln Φ β S x S + y S y O m+ β m ln [Φ O x O, β S x S, ρ m ] Φ m β O x O, β S x S, ρ, where Φ. ndcates the cumulatve dstrbuton functon of the unvarate standard normal dstrbuton and Φ. ndcates the cumulatve dstrbuton functon of the bvarate standard normal dstrbuton. 3 Restrctng coeffcents ρ and The parameter ρ needs to be n the nterval,. In order to restrct ρ to be n ths nterval, we estmate arctanρ nstead of ρ so that the derved parameter ρ tanarctanρ s always n the nterval,. We use the delta method to calculate approxmate standard errors of the derved parameter ρ, whereas the correspondng element of the Jacoban matrx s: tanarctanρ arctanρ ρ arctanρ + ρ 3 The parameter needs to be strctly postve,.e. > 0. In order to restrct to be strctly postve, we estmate log nstead of or so that the derved parameters explog and exp log are always strctly postve. We use the delta

3 method to calculate approxmate standard errors of the derved parameters and, whereas the correspondng elements of the Jacoban matrx are: explog log exp log log explog 4 exp log 5 4 Gradents of the CDF of the bvarate standard normal dstrbuton In order to facltate the calculaton of the gradents of the log-lkelhood functon, we calculate the partal dervatves of the cumulatve dstrbuton functon CDF of the bvarate standard normal dstrbuton: Φ x, x, ρ x x φ a, a, ρ da da, 6 where φ. s the probablty densty functon PDF of the bvarate standard normal dstrbuton: φ x, x, ρ π ρ exp x ρx x + x ρ 7 In the followng, we check equaton 7 by a smple numercal example: > lbrary "mvtnorm" > lbrary "maxlk" > x <- 0.4 > x < > rho < > sgma <- matrx c, rho, rho,, nrow > dens <- dmvnorm c x, x, sgma sgma > prnt dens [] > all.equal dens, * p * sqrt - rho^ ^- * + exp - x^ - * rho * x * x + x^ / * - rho^ [] TRUE 3

4 4. Gradents wth respect to the lmts x and x Φ x, x, ρ x x x x x x φ a, x, ρ da 8 φa x, ρφx da 9 φ a, ρx, ρ φx da 0 a ρx φ ρ φx da ρ φ x ρx ρ Φ a ρx ρ ρ da φx x ρx ρ φa da φx 3 φx, 4 where φ, µ, ndcates the densty functon of a normal dstrbuton wth mean µ and varance. In the followng, we use the same smple numercal example as n the begnnng of secton 4 to check the above dervatons. Frst, we check whether the PDF of the bvarate standard normal dstrbuton,.e. φ x, x, ρ part of equaton 8, s equal to φ x, ρx, ρ φx part of equaton 0 and equal to φ x ρx / ρ ρ φx part of equatons and : > all.equal dens, dnorm x, rho * x, sqrt - rho^ * dnormx [] TRUE > all.equal dens, dnorm x - rho * x / sqrt - rho^ / + sqrt - rho^ * dnormx [] TRUE In the followng, we wll numercally calculate the dervatve of the cumulatve dstrbuton functon of the bvarate normal dstrbuton equaton 6 wth respect to x and check wehther ths partal dervatve s equal to the rght-hand sdes of equatons 8,,, and 4: > funx <- functon a { + prob <- pmvnorm upper c x, a, sgma sgma 4

5 + return prob + } > grad <- c numercgradent funx, x > prnt grad [] > funx <- functon a { + dens <- rep NA, length a + for n :length a { + dens[] <- dmvnorm c a[], x, sgma sgma + } + return dens + } > all.equal grad, ntegrate funx, lower -Inf, upper x $value [] TRUE > funxa <- functon a { + dens <- rep NA, length a + for n :length a { + dens[] <- dnorm a[] - rho * x / sqrt - rho^ / + sqrt-rho^ * dnormx + } + return dens + } > all.equal grad, ntegrate funxa, lower -Inf, upper x $value [] TRUE > funxb <- functon a { + dens <- rep NA, length a + for n :length a { + dens[] <- dnorm a[] - rho * x / sqrt - rho^ / + sqrt-rho^ + } + return dens + } > all.equal grad, + ntegrate funxb, lower -Inf, upper x $value * dnormx [] TRUE > all.equal grad, + pnorm x - rho * x / sqrt - rho^ * dnorm x [] TRUE 5

6 4. Gradents wth respect to the coeffcent of correlaton ρ Φ x, x, ρ ρ [ x ] x φ a, a, ρ da da ρ 5 6 x x φ a, a, ρ da da 7 ρ x x ρ exp a ρa a + a ρ π ρ da da 8 x x π ρ exp a ρa a + a ρ ρ da da 9 x x ρ π exp a ρa a + a ρ ρ ρ ρ exp a ρa a + a ρ ρ da da ρ 30 x x ρ π a ρa a + a ρ exp a ρa a + a ρ ρ ρ ρ ρ exp a ρa a + a ρ ρ da da 3 6

7 x x π 4ρa ρa a + a ρ a a 4 ρ exp ρ a ρa a + a ρ ρ ρ ρ exp a ρa a + a ρ ρ da da 3 4ρa x x ρa a + a ρ a a 4 ρ ρ ρ ρ π ρ ρ exp a ρa a + a ρ da da x x 4ρa ρa a + a ρ a a ρ + π 4 ρ 5 3 ρ 3 exp a ρa a + a ρ da da x x 4ρa ρa a + a ρ a a ρ + π 4 ρ 5 ρ 3 exp a ρa a + a ρ da da x x ρ ρa ρa a + a a a π ρ 3 ρ 5 exp a ρa a + a ρ da da x x ρ π ρ ρ ρa ρa a + a a a ρ exp a ρa a + a ρ da da

8 π ρ π ρ lm a x x a ρa ρ exp a ρa a + a x ρ a ρx ρ exp a ρa x + x ρ a + ρa ρ a exp ρa a + a da ρ Applyng L Hosptal on the last term leads to x π a ρx ρ ρ exp a ρa x + x ρ 0 π ρ x a ρx ρ exp a ρa x + x ρ da da 40 da 4 π ρ exp a ρa x + x x ρ π ρ exp x ρx x + x ρ 4 43 φ x, x, ρ 44 Ths result s n lne wth Sbuya 960 and Sungur 990. In the followng, we wll numercally calculate the dervatve of the cumulatve dstrbuton functon of the bvarate normal dstrbuton equaton 6 wth respect to ρ and check whether ths partal dervatve s equal to the rght-hand sdes of equaton 44: > # Numercal gradent of the PDF w.r.t. rho > funrho <- functon p { + prob <- dmvnorm x c x, x, + sgma matrx c, p, p,, nrow + return prob + } > grad <- c numercgradent funrho, rho > prnt grad [] > # Comparson wth analytcal gradent for rho > efun <- exp-x^ - * rho * x * x + x^/* - rho^ 8

9 > all.equal grad, + -*rho*-*rho*x*x+x^+x^ - *x*x*-rho^ * efun/ + *-rho^^3/ + + rho*efun/sqrt-rho^ / + *p*-rho^ [] TRUE [] TRUE [] TRUE [] TRUE > # Numercal gradent of the CDF w.r.t. rho > cdfrho <- functon p, xa x, xb x { + prob <- pmvnorm upper c xa, xb, + sgma matrx c, p, p,, nrow + return prob + } > grad <- c numercgradent cdfrho, rho > prnt grad [] > # comparson wth analytcal gradent > all.equal grad, dmvnorm x c x, x, + sgma matrx c, rho, rho,, nrow [] TRUE > # comparsons wth other values > compdervrho <- functon xa, xb, p { + dn <- c numercgradent cdfrho, p, xa xa, xb xb + da <- dmvnorm x c xa, xb, + sgma matrx c, p, p,, nrow + return all.equal dn, da + } > compdervrho x, x, rho [] TRUE > compdervrho 0.5, x, rho [] TRUE 9

10 > compdervrho.5, x, rho [] TRUE > compdervrho x, -, rho [] TRUE > compdervrho x, x, 0. [] TRUE > compdervrho x, x, 0.98 [] TRUE 0

11 5 Gradents of the Log-Lkelhood Functon 5. Gradents wth respect to the parameters of the selecton equaton β S Frst, we use equaton 4, to determne the dervatve of the bvarate standard normal dstrbuton wth respect to the parameter β S as part of the loglkelhood functon: Φ m β O x O, β S x S, ρ Φ m β O x O m β O x O β S Φ ρβ S x S φβ S x S ρ βs x S β S 45 + ρβ S x S φβ S x S ρ x S 46 Usng ths result we can now derve the gradent for β S n the log-lkelhood functon: l β S + [ ] β S y S ln Φ β S x S y S y O m m ln m+ β [Φ O x O m β O x O Φ, β S x S, ρ y S β S + y S y O m ], β S x S, ρ 47 [ ] ln Φ β S x S 48 m m+ β β S ln [Φ O x O ], β S x S, ρ m β O x O Φ, β S x S, ρ β S x S x S 49 Φ β S x S φ y S + y S y O β m S β S Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ β S x S x S 50 Φ β S x S m φ y S Φ m+ β O x O,β S x S, ρ Φ m β O x O,β S x S, ρ

12 + y S y O m m Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ m+ β O x O Φ + ρβ S x S φ β S x S ρ x S Φ + ρβ S x S φ β S x S ρ x S m β O x O φ β S x S y S Φ β S x S + y S y O m m x S 5 Φ m+ β O x O +ρβ S x S ρ Φ m β O x O +ρβ S x S ρ φ β S x S x S Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ

13 5. Gradents wth respect to the parameters n the outcome equaton β O Analogous to β S and by usng equaton 4 we derve the gradent of β O : Φ m β O x O, β S x S, ρ β O Φ βs x S + ρ m βo x O ρ φ m β O x O xo Usng ths result we derve the gradent for the outcome parameter β O for the loglkelhood functon: l β O [ ] β O y S ln Φ β S x S + y S y O m m ln Φ m β O x O y S y O m m β O Φ m β O x O y S y O m m y S y O m m Φ m+ β [Φ O x O ], β S x S, ρ ln βs x S + ρ m+ β O x O ρ Φ m+ β [Φ O x O ], β S x S, ρ Φ m+ β O x O,β S x S, ρ β O, β S x S, ρ, β S x S, ρ Φ m β O x O,β S x S, ρ β O Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ βs x S + ρ m βo x O ρ Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ φ φ m+ β O x O m β O x O xo xo 56 3

14 y S y O m m Φ βs x S + ρ m+ β O x O ρ Φ βs x S + ρ m βo x O ρ Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ φ φ m+ β O x O m β O x O xo Gradents wth respect to the coeffcent of correlaton ρ Gven the result that the dervatve of the CDF wth respect to ρ s equal to the PDF see equaton 44, we can also derve the gradent of the correlaton parameter ρ: l ρ ρ + y S y O m y S ln m ln [ ] Φ β S x S m+ β [Φ O x O ], β S x S, ρ m β O x O Φ, β S x S, ρ y S y O m m+ β ln [Φ O x O, β S x S, ρ ρ ] m Φ m β O x O y S y O m m, β S x S, ρ φ m β O x O, β S x S, ρ φ m+ β O x O, β S x S, ρ Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ 60 l arctanhρ l ρ ρ arctanhρ l ρ ρ Gradents wth respect to the standard devaton used for normalsaton Fnally, we derve the gradent for n the same way as we dd for β S and β O : 4

15 Φ m β O x O, β S x S, ρ Φ βs x S + ρ m βo x O ρ φ m β O x O βo x O m 6 Φ m β O x O, β S x S, ρ lm m lm Φ Φ Φ m Φ βs x S + ρ m βo x O ρ m β O x O βs x S + ρ m βo x O φ ρ lm φ m β O x O m βs x S + ρ m βo x O ρ βs x S + ρ m βo x O ρ lm m β O x O m φ m β O x O βo x O m 63 βo x O m Φ Φ lm m φ m β O x O βs x S + ρ m βo x O ρ βs x S + ρ m βo x O ρ lm m lm m m βo x O φ m β O x O φ m β O x O φ m β O x O m β O x O m β O x O Smlarly: Φ m β O x O, β S x S, ρ lm m

16 l [ ] y S ln Φ β S x S + y S y O m+ β m ln [Φ O x O m ], β S x S, ρ m β O x O Φ, β S x S, ρ y S y O m m+ β ln [Φ O x O, β S x S, ρ ] m Φ m β O x O, β S x S, ρ y S y O m Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ m Φ m+ β O x O,β S x S, ρ m β Φ O x O,β S x S, ρ y S yo m 74 m Φ m+ β O x O, β S x S, ρ Φ m β O x O, β S x S, ρ Φ βs x S + ρ m+ β O x O m+ β O x O φ βo x O m+ ρ Φ βs x S + ρ m βo x O ρ φ m β O x O βo x O m Example wth a Generated Dataset 7 Generate Dataset > lbrary "mvtnorm" > # number of observatons > nobs <- 300 > # parameters l log l log l 75 6

17 > betas <- c,, - > betao <- c 0, 4 > rho <- 0.4 > sgma <- 5 > # boundares of the ntervals > bound <- c-inf,5,5,inf > # set 'seed' of the pseudo random number generator > # n order to always generate the same pseudo random numbers > set.seed3 > # generate varables x and x > dat <- data.frame x rnorm nobs, x rnorm nobs > # varance-covarance matrx of the two error terms > vcovmat <- matrx c, rho*sgma, rho*sgma, sgma^, nrow > # generate the two error terms > eps <- rmvnorm nobs, sgma vcovmat > dat$epss <- eps[,] > dat$epso <- eps[,] > # generate the selecton varable > dat$ys <- wth dat, betas[] + betas[] * x + betas[3] * x + epss > 0 > table dat$ys FALSE TRUE 9 09 > # generate the unobserved/latent outcome varable > dat$you <- wth dat, betao[] + betao[] * x + epso > dat$you[!dat$ys ] <- NA > # obtan the ntervals of the outcome varable > dat$yo <- cut dat$you, bound > table dat$yo -Inf,5] 5,5] 5, Inf] Estmaton of the Model In the followng estmaton, the startng values are obtaned by a maxmum-lkelhood ML estmaton of a tobt- model, where the dependent varable of the outcome equaton s set to the md ponts of the ntervals: > lbrary "sampleselecton" > res <- selecton ys ~ x + x, yo ~ x, data dat, boundares bound > res Call: selectonselecton ys ~ x + x, outcome yo ~ x, data dat, boundares bou 7

18 Coeffcents: S:Intercept S:x S:x O:Intercept O:x logsgma atanhrho sgma sgmasq rho > summary res Tobt model wth nterval outcome sample selecton model Maxmum Lkelhood estmaton BHHH maxmsaton, teratons Return code : successve functon values wthn tolerance lmt Log-Lkelhood: observatons 9 censored and 09 observed Intervals of the dependent varable of the outcome equaton: YO lower upper count -Inf,5] -Inf 5 6 5,5] , Inf] 5 Inf 53 7 free parameters df 93 Probt selecton equaton: Estmate Std. Error t value Pr> t Intercept < e-6 *** x e-0 *** x < e-6 *** Outcome equaton: Estmate Std. Error t value Pr> t Intercept < e-6 *** x e-05 *** Error terms: Estmate Std. Error t value Pr> t logsgma < e-6 *** atanhrho sgma < e-6 *** sgmasq e-0 *** rho Sgnf. codes: 0 *** 0.00 ** 0.0 * In the followng estmaton, the startng values are obtaned by a two-step estmaton of a tobt- model, where the dependent varable of the outcome equaton s set to the md ponts of the ntervals: 8

19 > res <- selecton ys ~ x + x, yo ~ x, data dat, boundares bound, + start "step" > res Call: selectonselecton ys ~ x + x, outcome yo ~ x, data dat, start "step", Coeffcents: S:Intercept S:x S:x O:Intercept O:x logsgma atanhrho sgma sgmasq rho > summary res Tobt model wth nterval outcome sample selecton model Maxmum Lkelhood estmaton BHHH maxmsaton, teratons Return code : successve functon values wthn tolerance lmt Log-Lkelhood: observatons 9 censored and 09 observed Intervals of the dependent varable of the outcome equaton: YO lower upper count -Inf,5] -Inf 5 6 5,5] , Inf] 5 Inf 53 7 free parameters df 93 Probt selecton equaton: Estmate Std. Error t value Pr> t Intercept < e-6 *** x e-0 *** x < e-6 *** Outcome equaton: Estmate Std. Error t value Pr> t Intercept < e-6 *** x e-05 *** Error terms: Estmate Std. Error t value Pr> t logsgma < e-6 *** atanhrho sgma < e-6 *** sgmasq e-0 *** rho

20 --- Sgnf. codes: 0 *** 0.00 ** 0.0 * The followng commands compare the startng values and the estmated coeffcents: > # compare startng values small dfferences > cbnd res$start, res$start, res$start - res$start [,] [,] [,3] Intercept x x Intercept x logsgma atanhrho > # combare estmated coeffcents vrtually dentcal > cbnd coef res, coef res, coef res - coef res [,] [,] [,3] Intercept e-07 x e-07 x e-07 Intercept e-06 x e-06 logsgma e-07 atanhrho e-06 sgma e-07 sgmasq e-06 rho e-06 8 Example wth the Smoke dataset The followng command loads the dataset: > data "Smoke" The followng command creates a vector wth the boundares of the ntervals: > bounds <- c 0, 5, 0, 0, 50, Inf The followng command estmates the model wth few explanatory varables: > SmokeRes <- selecton smoker ~ educ + age, + cgs_ntervals ~ educ, data Smoke, boundares bounds 0

21 The followng command estmates the model wth more explanatory varables: > SmokeRes <- selecton smoker ~ educ + age + restaurn, + cgs_ntervals ~ educ + ncome + restaurn, data Smoke, + boundares bounds The followng commands test whether addng further explanatory varables sgnfcantly mproves the explanatory power of the model: > lbrary "lmtest" > lrtest SmokeRes, SmokeRes Lkelhood rato test Model : selectonselecton smoker ~ educ + age + restaurn, outcome cgs_ntervals ~ Model : selectonselecton smoker ~ educ + age, outcome cgs_ntervals ~ educ, data Smoke, boundares bounds educ + ncome + restaurn, data Smoke, boundares bounds #Df LogLk Df Chsq Pr>Chsq * --- Sgnf. codes: 0 *** 0.00 ** 0.0 * > waldtest SmokeRes, SmokeRes Wald test Model : selectonselecton smoker ~ educ + age + restaurn, outcome cgs_ntervals ~ Model : selectonselecton smoker ~ educ + age, outcome cgs_ntervals ~ educ, data Smoke, boundares bounds educ + ncome + restaurn, data Smoke, boundares bounds Res.Df Df Chsq Pr>Chsq * --- Sgnf. codes: 0 *** 0.00 ** 0.0 * Both tests ndcate that at 5% sgnfcance level the model wth more explanatory varables SmokeRes has sgnfcantly hgher explanatory power than the model wth fewer explanatory varables SmokeRes. References Petersen, S., Hennngsen, G., and Hennngsen, A. 07. Can changes n households lfe stuatons predct partcpaton n energy audts and nvestments n energy savngs? Unpublshed Manuscrpt. Department of Management Engneerng, Techncal Unversty of Denmark.

22 Sbuya, M Bvarate extreme statstcs,. Annals of the Insttute of Statstcal Mathematcs, :95 0. Sungur, E Dependence nformaton n parameterzed copulas. Communcatons n Statstcs - Smulaton and Computaton, 94: