Applied Asymptotics Case studies in higher order inference

Transcription

1 Applied Asymptotics Case studies in higher order inference Nancy Reid May 18, 2006 A.C. Davison, A. R. Brazzale, A. M. Staicu

2 Introduction likelihood-based inference in parametric models higher order approximations are uncannily accurate perhaps seem difficult to use illustration of these approximations in a variety of practical examples such as: generalized linear models; canonical and non-canonical link functions non-normal linear regression, nonlinear normal regression variance components models calibration studies here concentrate on Bayesian examples; inspired by Bickel & Ghosh 1990

3 Cumulative distribution function y

4

5 Main results I: Bartlett correction vector parameter of interest; model f (y; θ), y R n, θ R p log-likelihood ratio statistic: w(θ) = 2{l(ˆθ) l(θ)} d χ 2 p, n Ew(θ) = p{1 + b(θ) n + O(n 2 )} w (θ) = w(θ) 1 + b(θ)/n d χ 2 p{1 + O(n 2 )} w (θ) = w(θ) 1 + b( θ)/n d χ 2 p{1 + O(n 2 )}

6 ... Bartlett correction Bickel & Ghosh showed that this holds under the posterior distribution elegant shrinkage argument leads to frequentist result suggestion to use the jackknife to estimate the mean; no explicit formula explain the detailed structure of the expansion to show why the result holds contrast to Lawley (1956); see also DiCiccio and Stern (1994) for explicit Bayesian formula w(θ 1 ) = 2{l( θ 1, θ 2 ) l(θ 1, θ 2 )} d χ 2 p 1

7

8

9 Model... Bartlett correction f (y ij ; β i, µ i ) = 1 ( ) βi βi y β i 1 Γ(β i ) µ ij e β i y ij /µ i i Are data consistent with a common value for β i? w(β c ) = 2{l( β, µ) l(β c 1, µ)}. χ 2 9 First order p-value 0.073; using Bartlett correction 0.107; using Skovgaard s correction Bartlett correction estimated via 1000 parametric bootstrap samples

10 log likelihood ratios Quantiles of χ

11 log likelihood ratios Quantiles of χ

12 log likelihood ratios Quantiles of χ

13 Main results II: scalar parameter of interest f (y; θ); y R n, θ = (ψ, λ) p-value for inference re ψ approximated to O(n 1 ) by Φ(r ) error O(n 3/2 ) r = r + 1 r log Q r r = ± [2{l( ˆψ, ˆλ) l(ψ, ˆλ ψ )}] Q =... something like a standardized mle or score function Bayesian version: Pr(Ψ ψ) =. Φ(rB { ) } 1/2 Q = q B = l p(ψ)j 1/2 j λλ (ψ, ˆλ ψ ) π( ˆψ, ˆλ) p ( ˆψ) j λλ ( ˆψ, ˆλ) π(ψ, ˆλ ψ )

14 Logistic regression The first ten out of 79 sets of observations on the physical characteristics of urine. Presence/absence of calcium oxalate crystals is indicated by 1/0. Two cases with missing values. 1 Case Crystals Specific gravity ph Osmolarity Conductivity Urea Calcium Andrews & Herzberg, 1985

15 Model: Independent binary responses Y 1,..., Y n with Pr(Y i = 1) = Fitting generalized linear model in R: exp(x T i β) 1 + exp(x T i β) data(urine) fit <- glm(r gravity+ph+osmo+conduct+urea+calc, family = binomial, data=urine) summary(fit) Estimate Std. Error z value Pr(> z ) (Intercept) gravity ph osmo conduct urea * calc **

16 A closer look at coefficient of calc method lower bound upper bound p-value for 0 Φ(q) Φ(r) Φ(r ) library(cond) # part of package hoa on cran-r urine.cond.calc <- cond.glm(urine.glm,offset=calc) > summary(urine.cond.calc,test=0)... Test statistics hypothesis : coef( calc ) = 0 statistic tail prob. Wald pivot e-04 Wald pivot (cond. MLE) e-04 Likelihood root e-06 Modified likelihood root e-06 Modified likelihood root (cont. corr.) e-06...

17 A closer look at coefficient of calc method lower bound upper bound p-value for 0 Φ(q) Φ(r) Φ(r ) Φ(rB ) Profile and modified profile log likelihoods log likelihood profile log likelihood modified profile log likelihood

18 Comparison of posterior limits for ψ with frequentist p-value

19 Choice of prior A matching prior ensures that to O(n 3/2 ), the posterior upper α limit has approximately correct frequentist coverage: Pr m (ψ > ψ (1 α) (y) y) = α Pr(ψ (1 α/2). (Y ) < ψ θ) = α There is a family of such matching priors when ψ is scalar π(ψ, λ) i ψψ (ψ, η)g(η) θ = (ψ, λ) (ψ, η(ψ, λ)) η orthogonal to ψ with respect to expected Fisher information i(θ)

20 ... matching prior for logistic regression η easily computed: full exponential family ˆη ψ = ˆη (in exponential families only) When used in q B matching prior is unique { } 1/2 q B = q B = l p(ψ)j 1/2 j ηη (ψ, ˆη ψ ) π( ˆψ, ˆη) p ( ˆψ) j ηη ( ˆψ, ˆη) π(ψ, ˆη ψ ) can be computed without transforming to (ψ, η) i ψψ (ψ, η) = iψψ 0 (ψ, λ) i0 ψλ (ψ, λ)i0 λλ (ψ, λ) 1 iλψ 0 (ψ, λ) for any orthogonal parameter (not only exponential family) ˆη ψ = ˆη(1 + O p (n 1 ) this seems to be enough to ensure uniqueness of matching prior in Laplace approximation Sweeting, Biometrika, 2004; Staicu & R, in prep.

21 Conclusions/Further work Reasonable range of examples where r type approximations very easy to use Most numerical work is very convincing Important exception: proportional hazards model Non-parametric or semi-parametric models? Frequentist version relies on the log likelihood function (and in particular on differentiating log likelihood on the sample space) Not clear how/if this can be adapted to pseudo-likelihoods Vector version of Bayesian matching argument using Bickel & Ghosh (1990) Careful proofs still needed of many hoa results!! Best wishes for Peter, and warm thanks for many and inspiring contributions to our field