Approximating models. Nancy Reid, University of Toronto. Oxford, February 6.

Size: px

Start display at page:

Download "Approximating models. Nancy Reid, University of Toronto. Oxford, February 6."

Sherman Dickerson
5 years ago
Views:

1 Approximating models Nancy Reid, University of Toronto Oxford, February 6 1

2 1. Context Likelihood based inference model f(y; θ), log likelihood function l(θ; y) y = (y 1,..., y n ), l(θ; y) = log f(y i ; θ) assume l(θ; y) = O(n), θ ˆθ = O(n 1/2 ) goal to improve the approximation given by the limiting distribution e.g. r = ±[2{l(ˆθ) l(θ)}] 1/2 d N(0, 1) E(r) = an 1/2 + O(n 3/2 ) var(r) = 1 + bn 1 + O(n 3/2 ) implies r an 1/2. (1 + bn 1 ) 1/2 N(0, 1) is better than r. N(0, 1) 2

3 2. Local exponential family models f(x; ϕ) = exp{ϕx c(ϕ) d(x)} l ;x = l ϕ;x = Start with arbitrary f(y; θ), y, θ scalar Expand l(θ; y) about (θ 0 ; y 0 ) θ 0 = Represent coefficients l(θ 0 ; y 0 ), l ;y (θ 0 ; y 0 ), etc. by a ij a 00 a 01 a 02 a 03 a 04 0 a 11 a 12 a 13 a 20 a 21 a 22 a 30 a 31 a 40 3

4 1. Standardize θ (θ θ 0 )ĵ 1/2 y (y y 0 )a 11 ĵ 1/2 a 20 1, a 11 1, a ij ã ij 2. Reparametrize θ θ + ã 21 θ 2 /2 + ã 31 θ 3 /6 ã 21 0, ã New variable ỹ ỹ + ã 12 ỹ 2 /2 + ã 13 ỹ 3 /6 ã 12 0, ã Notation a 30 = α 3 n 1/2, a 40 = a 22 = a 00 a 01 a 02 a 03 a γ/n α 3 /n 1/2 0 α 4 /n 4

5 5. Density must integrate to 1 3α 4 5α γ 24n α 3 2 n 1 + α 4 2α 2 3 5γ 2n α 3 n α 4 3α 2 3 n γ/n α 3 /n 1/2 0 α 4 /n α 3 = α 4 = γ =

6 First row is log(2π) + (3α 4 5α γ)/24n, α 3/2n 1/2, 1 + (α 4 2α 2 3 5γ)/2n, α 3/n 1/2, (α 4 3α 2 3 6γ)/n 4-1

7 ...2 Local exponential family models New density looks like with cdf f(x, ϕ). = φ(x ϕ) exp{...}. = φ(x ϕ){1 +...} F (x, ϕ) = Φ(x ϕ) + [ α3 φ(x ϕ) 6 n {...} + α 4 24n {...} + α2 3 72n + γ { 2x + ϕx 2 + x 3}] 4n {...} Free of γ at x = 0 (y = y 0 ) p-value does not depend on γ Andrews, Fraser, Wong,

8 3. Tangent exponential model p T EM (x; θ) = c j(ˆϕ) 1/2 exp[l(θ; y 0 ) l(ˆθ 0 ; y 0 ) +, {ϕ(θ) ϕ(ˆθ 0 )}x] ϕ = l(θ, y) y y=y 0 x = l(θ; y) θ θ=ˆθ 0 j(ˆϕ) = 2 l(ϕ) ϕ 2 ˆϕ 6

9 l(θ; y 0 ) is first column (ignoring (0,0) entry) ϕ(θ) is second column (ignoring (0,1) entry) These 2 columns determine the rest of the array, except the γ/n term Easy to use p T EM to get a p-value (saddlepoint type approximation) 6-1

10 ...3 Tangent exponential model How to get a scalar variable y? Condition on an (approximate) ancillary, so l ;y is taken for fixed ancillary a(y). This can be computed by finding a vector V = (V 1,..., V n ) T tangent to the ancillary at y 0 : ϕ(θ) = l ;V (θ; y) y 0 = l ;yi (θ; y 0 i )V i Example y i f(y i µ) a i = y i ˆµ, say, V i = 1 ϕ(θ) = log f(y i µ) y i y 0 = l θ (θ; y 0 ) 7

11 Example f(y 1, y 2 ; θ) = e y θ {1 + e (y θ) } 2 exp[γ(θ)(y θ) c{γ(θ)}], 1 θ 1 γ(θ) = 0.5 tanh(θ) c(θ) = log{(πθ)/ sin(πθ)} y density y y2 y1 8

12 ...3 Tangent exponential model Vector θ? Use the same approach, now V = (V 1, V 2,..., V n ) T V i is 1 d, l ;V (θ) is also 1 d Example Example y i = x T i β + σe i V i = (x T i ê i ) y i = µ i (β) + σe i V i = {µ i (ˆβ) ê i } Inference re nuisance parameters uses p T EM twice to get a marginal distribution Example House price data (Srivastava and Sen); 4 covariates, 26 observations, model y i = x T i β + σe i, e i t 5 9

13 marginal inference for β 4 and for log σ, (conditional on usual ancillary), uses Alessandra Brazzale s Splus library HOA Lugannani-Rice tail approximations 95 % Confidence Intervals MLE normal approximation cond. MLE normal approximation marg. MLE normal approximation directed deviance modified directed deviance conditional marginal approximate marginal deviance Coefficient of front Coefficient of front 10

14 Profile and modified profile log-liks profile log-likelihood modified profile log-likelihood approximate marginal log-likelihood Coefficient of front

15 Lugannani-Rice tail approximations 95 % Confidence Intervals MLE normal approximation cond. MLE normal approximation marg. MLE normal approximation directed deviance modified directed deviance conditional marginal approximate marginal deviance log(scale) scale

16 Profile and modified profile log-liks profile log-likelihood modified profile log-likelihood approximate marginal log-likelihood log(scale)

17 > houses.marg.front <- cond.rsm(mod.obj=houses.rsm,offset=fron > summary(houses.marg.front) FORMULA: FAMILY : OFFSET : price ~ bdroom + floor + rooms + front student front COEFFICIENTS Value Std. Error uncond cond marg CONFIDENCE INTERVALS level = 95 % lower two-sided MLE normal approx Cond. MLE normal approx Marg. MLE normal approx Directed deviance Modified directed deviance Marginal directed deviance

18 4. Local location models f(x; β) = f(x β) l β = l ;x l ββ = l βx = l ;xx 4.1 If y f(y; θ) then x = y F y (y; θ 0 ) F θ (y; θ 0 ) dy has a density which is a location model near θ 0, g(x ), say. Satisfies l = l ;x, but not higher order. This model has an exact ancillary This ancillary can be used for the original model, for computing p-values. (This is where V above came from.) 11

19 ...4. Local location models 4.2 As with exponential model we can carry this further to get an array of coefficients for the double expansion about (y 0, ˆθ 0 ) of the form: a a 3 /n 1/2 a 4 /n 0 1 a 3 /n 1/2 a 4 /n 1 a 3 /n 1/2 a 4 /n+ a 3 /n 1/2 a 4 /n a 4 /n Andrews, Fraser, Wong, 2003 A more compact notation f{x β(θ)}, β(θ) = θ l θ (θ) ϕ(θ) dθ Existence (algorithm) for vector θ Fraser, Yi,

20 a + 3α 4 5α γ 24n γ 2n α 3 n 1/2 α 4 6γ n 0 1 α 3 /n 1/2 α 4 /n 1 α 3 /n 1/2 α 4+γ n α 4 /n x = y F y(y; 0) dy, G(x; θ) = F {y(x); θ} F ;θ (y; 0) G x (x; 0) = F y {y(x); 0} = F ;θ (y; 0) = G ;θ (x; 0) { F ;θ(y; 0) F y (y; 0) } 12-1

21 ...4 Local location model Bayesian analysis of location model uses flat prior for location parameter, in our case π(θ) dβ(θ) and this will give posterior p-values equal to those from tangent exponential model to O(n 3/2 ) if non-location term γ = 0, to O(n 1 ) if γ 0 With nuisance parameters, can only obtain strong matching priors for a single parameter of interest, using π(ψ, ˆλ ψ ) ψ β 1 (ψ,ˆλ ψ ) j λλ(ˆθ ψ ) ϕ λ (ˆθ ψ ) Fraser & Reid,

22 Example Location model with curved parameter of interest Y 1 N(θ 1, 1), Y 2 N(θ 2, 1) independent ψ 2 = (R + θ 1 ) 2 + θ 2 2 ; R known r 2 = {(R + y 1 ) 2 + y 2 2 } Bayesian posterior under usual flat prior (θ 1, θ 2 y) N(y 1, y 2 ) frequentist p-value (marginal) Pr{r r 0 ; ψ 0 ) Bayesian p-value Pr{ψ ψ 0 y) Will be quite different: matching prior using information adjustment gives π(θ) r ψ 14

23 frequentist = Pr{χ 2 2 (ψ0 ) 2 (y 1 + R) 2 + y 2 2 } Bayesian = Pr{χ 2 2 ((y 1 + R) 2 + y 2 2 ) ψ02 } Bayesian frequentist = Pr{X 1 X 2 = 0} X 1 P o((y 1 + R) 2 + y 2 2 ), X 2 P o(ψ 02 ) 14-1

24 R=1 R= p-value 0.4 p-value psi psi R=3 R= p-value 0.4 p-value psi psi 15

25 16

26 References Andrews, D.A., Fraser, D.A.S., Wong, A. Computation of distribution functions from likelihood information near observed data. Brazzale, A. brazzale Fraser, D.A.S., Reid, N. Strong matching of frequentist and Bayesian parametric inference. Fraser, D.A.S., Yi, G. Location reparametrization and default priors for statistical analyses. Reid, N. Asymptotics and the theory of inference. 17

Last week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36

Last week. posterior marginal density. exact conditional density. LTCC Likelihood Theory Week 3 November 19, /36 Last week Nuisance parameters f (y; ψ, λ), l(ψ, λ) posterior marginal density π m (ψ) =. c (2π) q el P(ψ) l P ( ˆψ) j P ( ˆψ) 1/2 π(ψ, ˆλ ψ ) j λλ ( ˆψ, ˆλ) 1/2 π( ˆψ, ˆλ) j λλ (ψ, ˆλ ψ ) 1/2 l p (ψ) =