Last week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36

Transcription

1 Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) = l(ψ, ˆλ ψ ) exact conditional density f (y; ψ, λ) = exp{ψ T s 1 + λ T s 2 k(ψ, λ)}h(y) f (s 1 s 2 ; ψ). = c (2π) q elp(ψ) lp( ˆψ) j p ( ˆψ) 1/2 j λλ( ˆψ, ˆλ) 1/2 j λλ (ψ, ˆλ ψ ) 1/2 LTCC Likelihood Theory Week 3 November 19, /36

2 ... last week Tail area approximation: p(ψ). = Φ(r ) = Φ(r + 1 r log Q r ), r = ± [2{l(ˆθ) l(θ)}] or ± [2{l p ( ˆψ) l p (ψ)}] Q = q B = l (θ)j 1/2 (ˆθ) π(ˆθ) π(θ) l p ( or 1/2 ˆψ)j p ( ˆψ) π( ˆψ, ˆλ) π(ψ, ˆλ ρ(ψ, ˆψ) ψ ) Q = q = (ˆθ θ)j 1/2 (ˆθ) or ( ˆψ ψ)j 1/2 p ( ˆψ)ρ 1 (ψ, ˆψ) ρ(ψ, ˆψ) = j λλ(ψ,ˆλ ψ ) 1/2 j λλ ( ˆψ,ˆλ) 1/2 LTCC Likelihood Theory Week 3 November 19, /36

3

4 This week 1. elimination of nuisance parameters and adjusted profile likelihood 2. approximate inference in location and transformation models 3. tail area approximations again 4. tangent exponential model 5. examples LTCC Likelihood Theory Week 3 November 19, /36

5 Nuisance parameters π m (ψ). = = c (2π) q elp(ψ) lp( ˆψ) j 1/2 p ( ˆψ) π(ψ, ˆλ ψ ) π( ˆψ, ˆλ) c (2π) q el A(ψ) l A ( ˆψ) j 1/2 p ( ˆψ) π(ψ, ˆλ ψ ) π( ˆψ, ˆλ) l A (ψ) = l p (ψ) 1 2 log j λλ(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 j λλ (ψ, ˆλ ψ ) 1/2 { f (s 1 s 2 ; ψ) =. c (2π) q j p( ˆψ) 1/2 lp(ψ) lp( ˆψ) j λλ ( ˆψ, ˆλ) e j λλ (ψ, ˆλ ψ ) c (2π) q j p( ˆψ) 1/2 e l A(ψ) l A ( ˆψ) = } 1/2 l A (ψ) = l p (ψ) log j λλ(ψ, ˆλ ψ ) LTCC Likelihood Theory Week 3 November 19, /36

6 Adjusted profile log-likelihood l A (ψ) = l p (ψ) + A(ψ) = l(ψ, ˆλ ψ ) + A(ψ) A(ψ) assumed to be O p (1) generic form is A FR (ψ) = log j λλ(ψ, ˆλ ψ ) log d(λ) d ˆλ ψ Fraser, 2003 closely related A BN (ψ) = 1 2 log j λλ(ψ, ˆλ ψ ) + log d ˆλ d ˆλ ψ SM , BN 1983 if i ψλ (θ) = 0, then ˆλ ψ = ˆλ + O p (n 1 ), suggesting we ignore last term if ψ is scalar, then in principle we can find a parametrization (ψ, λ) in which i ψλ (θ) = 0 SM LTCC Likelihood Theory Week 3 November 19, /36

7 Log-likelihoods marginal l m (ψ) = log f m (t; ψ) f (y; ψ, λ) f m (t; ψ)f c (s t; ψ, λ) conditional l c (ψ) = log f c (s t; ψ) f (y; ψ, λ) f m (t; ψ, λ)f c (s t; ψ) adjusted l A (ψ) = l p (ψ) 1 2 log j λλ(ψ, ˆλ ψ ) λ ψ refinements l FR (ψ), l BN (ψ),... Sartori, 2003; Nov12 Prob 2.2 LTCC Likelihood Theory Week 3 November 19, /36

8

9

10 Marginal inference conditional inference linear exponential families saddlepoint approx marginal inference? Bayesian posterior transformation models Laplace approx exponential families non-linear parameter of interest tangent exponential models everything else LTCC Likelihood Theory Week 3 November 19, /36

11 Location model location model f (y i ; θ) = f 0 (y i θ), y i, θ R f (y; θ) = l(θ; y) = change of variable (y 1,..., y n ) (ˆθ, a 1,..., a n ) a i = f (ˆθ, a; θ) = LTCC Likelihood Theory Week 3 November 19, /36

12 ... location models f (ˆθ, a; θ) = f (a; θ) = f (ˆθ a; θ) = LTCC Likelihood Theory Week 3 November 19, /36

13 Approximate p-values ˆθ ˆθ f ( ˆϑ a; ϑ)d ˆϑ = c j(ˆθ) 1/2 e l(θ;ˆθ,a) l(ˆθ;ˆθ,a) d ˆθ r = ce 1 2 r 2 j(ˆθ) 1/2 dr = r e 1 2 r 2 r q dr = Φ(r + 1 r log q r ) l ;ˆθ(ˆθ; ˆθ, a) l ;ˆθ(θ; ˆθ, a) d ˆθ = {l ;ˆθ(ˆθ; ˆθ, a) l ;ˆθ(θ; ˆθ, a)}j(ˆθ) 1/2 = l θ (θ)j(ˆθ) 1/2 location model LTCC Likelihood Theory Week 3 November 19, /36

14 ... approximate p-values LTCC Likelihood Theory Week 3 November 19, /36

15 Nuisance parameters Regression-scale models: y i = x T i β + σɛ i, ɛ i f 0 ( ) f (y; β, σ 2 ) = f c ( ˆβ, ˆσ a; σ, β)f m (a) a = parameter of interest ˆβ j, say t j = ˆβ j β j v j has marginal density free of β (j), σ t p+1 = log ˆσ log σ has marginal density free of β v j = Var( ˆβ j ) = p-value functions using Laplace approximation again LTCC Likelihood Theory Week 3 November 19, /36

16 These are all special cases to compute p-value we need to integrate a density approximation this density approximation is to either a marginal or a conditional density the integration involves the derivative of the log-likelihood, with respect to the data simplify the density approximation to incorporate only this first derivative use the resulting simpler form as the basis for approximate likelihood based inference LTCC Likelihood Theory Week 3 November 19, /36

17 Tangent exponential model f TEM (s; θ) = c e l{ϕ(θ);y}+ϕ(θ)t s j ϕϕ { ˆϕ(s); s} 1/2 p-value = Φ(r ) = Φ(r + 1 r log Q r ) Q = {ϕ(ˆθ) ϕ(θ)}j 1/2 ϕϕ ( ˆϕ) = ϕ(ˆθ) ϕ(θ) ϕ θ (ˆθ) j 1/2 θθ (ˆθ) Q = ϕ(ˆθ) ϕ(ˆθ ψ ) ϕ λ (ˆθ ψ ) ϕ θ (ˆθ) j(ˆθ) 1/2 j λλ (ˆθ ψ ) 1/2 LTCC Likelihood Theory Week 3 November 19, /36

18

19 .. tangent exponential model ϕ(θ) = l ;V (θ; y) = n i=1 l(θ; y) V i y i V i = ( zi y i ) 1 z i θ (ˆθ,y) BDR, Ch. 8.4, 8.5 LTCC Likelihood Theory Week 3 November 19, /36

20 Example: top quark model Y Poisson (µ + b), data y = 27 b = 6.7 b known mid p-value Pr(Y > 27) + 1 2Pr(Y = 27) = ( approximation: Φ(r ) = Φ r + 1 r log q ) = r continuity correction: Φ{r (y )} = Pr(Y 27) = Abe et al., 1995 LTCC Likelihood Theory Week 3 November 19, /36

21 ... top quark pr(y y 0 ; µ = 0) pr(y > y 0 ; µ = 0) Exact Mid-p Φ(r ) N(θ, θ) N(θ, ˆθ) LTCC Likelihood Theory Week 3 November 19, /36

22 Two-sample comparison data on cost of treatment: standard vs new treatment Group Group model Y 1i Exp(µ 1 ), Y 2i Exp(µ 2 ) ψ = µ 1 /µ 2 Exact inference (Ȳ1. µ 1 ) / (Ȳ2. µ 2 ) F 2n,2m exact 95% confidence interval for ψ: (0.98, 4.19) approx 95% confidence interval for ψ: (0.98, 4.185) Evans et al LTCC Likelihood Theory Week 3 November 19, /36

23 Logistic regression The first ten out of 79 sets of observations on the physical characteristics of urine. Presence/absence of calcium oxalate crystals is indicated by 1/0. Two cases with missing values. Case Crystals Specific gravity ph Osmolarity Conductivity Urea Calcium Andrews & Herzberg, 1985 LTCC Likelihood Theory Week 3 November 19, /36

24 Model: Independent binary responses Y 1,..., Y n with Pr(Y i = 1) = Fitting generalized linear model in R: exp(x T i β) 1 + exp(x T i β) data(urine) fit <- glm(r gravity+ph+osmo+cond+urea+calc, family = binomial, data=urine) summary(fit) Estimate Std. Error z value Pr(> z ) (Intercept) gravity ph osmo conduct urea * calc ** LTCC Likelihood Theory Week 3 November 19, /36

25 A closer look at coefficient of urea method lower bound upper bound p-value for 0 Φ(q) Φ(r) Φ(r ) library(cond) # part of package hoa on cran-r urine.cond.urea <- cond.glm(urine.glm,offset=urea) > summary(urine.cond.urea,test=0)... Test statistics hypothesis : coef( urea ) = 0 statistic tail prob. Wald pivot Wald pivot (cond. MLE) Likelihood root Modified likelihood root Modified likelihood root (cont. corr.) LTCC Likelihood Theory Week 3 November 19, /36

26 A closer look at coefficient of urea method lower bound upper bound p-value for 0 Φ(q) Φ(r) Φ(r ) Profile and modified profile log likelihoods log likelihood profile log likelihood modified profile log likelihood LTCC Likelihood Theory Week 3 November 19, /36

27 Several 2 2 tables. Institution y 1 m 1 y 2 m 2 Institution y 1 m 1 y 2 m Lipsitz et al. 1988: Biometrics LTCC Likelihood Theory Week 3 November 19, /36

28 ... matched pairs Model: Y 1i Binomial(m 1i, p 1i ) Y 2i Binomial(m 2i, p 2i ) parameter of interest ψ = p 2i p 1i nuisance parameters p 1i, i = 1, inference for ψ : lower upper point estimate p-value for ψ = 0 Φ(r) Φ(r ) LTCC Likelihood Theory Week 3 November 19, /36

29 Pivot (psi) Log likelihood psi likelihood root, r, q psi profile log likelihood, modified profile

30 Example G: Cox & Snell, 1980 cost date T1 T2 cap PR NE CT BW N PT n = 32, d = 8 LTCC Likelihood Theory Week 3 November 19, /36

31 ν log(n) PT First order Third order First order Third order LTCC Likelihood Theory Week 3 November 19, /36 Linear regression, non-normal error Model Y i = β 0 + x T i β + σɛ i ɛ N(0, 1) or ɛ t ν Normal t 4, first order t 4, third order Est (SE) z Est (SE) z Est (SE) z Constant (3.140) (3.67) (3.70) 3.21 date (0.043) (0.048) (0.049) 4.02 log(cap) (0.119) (0.113) (0.129) 5.31 NE (0.074) (0.077) (0.080) 2.97 CT (0.060) (0.054) (0.063) 2.26 log(n) (0.042) (0.043) (0.048) 1.51 PT (0.114) (0.101) (0.110) 2.42

32 library(marg) # part of package hoa on cran-r data(nuclear) # Fit normal-theory linear model and examine its contents: nuc.norm <- lm( log(cost) date + log(cap) + NE + CT + log(n) + PT, + data = nuclear ) summary(nuc.norm) # Fit linear model with t errors and 4 df and examine its contents: nuc.t4 <- rsm( log(cost) date + log(cap) + NE + CT + log(n) + PT, + data = nuclear, family = student(4) ) summary(nuc.t4) plot(nuc.t4) # Conditional analysis for partial turnkey guarantee: nuc.t4.pt <- cond( nuc.t4, offset = PT ) summary(nuc.t4.pt) plot(nuc.t4.pt) # For conditional analysis for other covariates, replace pt by # log(n),...

33 Type II censored data 40 units on test, 28 failures at (log) times Weibull model: f (y; µ, σ) = e (y µ)/σ exp{ e (y µ)/σ } 90% confidence intervals µ σ Φ(r) ( 0.116, 0.476) (0.700, 1.217) Φ(r ) ( 0.107, 0.510) (0.743, 1.320) Exact (num. int.) ( 0.11, 0.51) (0.724, 1.277) Lawless 2003 Ch.5; Wong & Wu 2003 LTCC Likelihood Theory Week 3 November 19, /36

34 Vector parameter of interest use tangent exponential model (or usual exponential family model) construct a scalar parameter of interest representing direction in sample space apply higher order approximation Example y N(µ, Σ); H 0 : Σ 1 is tri-diagonal First-order Markov dependence in a graphical model Nominal (%) First order Second order LTCC Likelihood Theory Week 3 November 19, /36

35

36