Accurate directional inference for vector parameters

Size: px

Start display at page:

Download "Accurate directional inference for vector parameters"

Mae Sanders
5 years ago
Views:

1 Accurate directional inference for vector parameters Nancy Reid February 26, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Nancy Reid Accurate directional inference for vector parameters York University February /29

2 Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,..., y n ) independent observations log-likelihood function l(θ; y) = log f (y; θ) parameter of interest θ = (ψ, λ), ψ R d likelihood inference w(ψ) = 2{l( ˆψ, ˆλ) l(ψ, ˆλ ψ )}. χ 2 d Likelihood Contours psi w B (ψ) = w(ψ) 1 + B(ψ) { w (ψ) = w(ψ) 1. χ 2 d O p (n 2 ) B(ψ) = E{w(ψ)}/d } log γ(ψ) w(ψ) psi1 Skovgaard, 2001 Nancy Reid Accurate directional inference for vector parameters York University February /29

3 Example: 2 3 contingency table activity amongst psychiatric patients Everitt, 1992 Affective disorders Schizophrenics Neurotics Retarded Not retarded model: log-linear y Poisson, log{e(y)} = X θ, θ R 6 log-likelihood l(θ; y) = θ X y 1 e X θ = θ s c(θ) θ = (ψ, λ) ψ R 2, λ R 4 (ψ 1, ψ 2 ) interaction parameters H 0 : ψ = ψ 0 = (0, 0) independence log-likelihood l(ψ, λ; y) = ψ s 1 + λ s 2 c(ψ, λ) Nancy Reid Accurate directional inference for vector parameters York University February /29

4 Testing ψ = 0 Likelihood Contours psi w(ψ 0 ). χ 2 2 p-value w (ψ 0 ) p-value psi1 directional p-value exact p-value t Nancy Reid Accurate directional inference for vector parameters York University February /29

5 model f (y; θ) = exp{ϕ(θ) T u(y) c{ϕ(θ)} d(y)} y 1,..., y n i.i.d sufficient statistic f (s; θ) = {y:s(y)=s} f (y; θ)dy = exp{ϕ(θ)t s nc{ϕ(θ)} d(s)} reduce dimension from n to p by marginalization linear parameter of interest ϕ(θ) = θ = (ψ, λ) model f (s 1, s 2 ; ψ, λ) = exp{ψ T s 1 + λ T s 2 nc(ψ, λ) d(s)} conditional density f (s 1 s 2 ; ψ) = exp{ψ T s 1 n c 2 (ψ) d 2 (s 1 )} reduce dimension from p to d by conditioning Nancy Reid Accurate directional inference for vector parameters York University February /29

6 ... conditional density f (s 1 s 2 ; ψ) = exp{ψ T s 1 n c 2 (ψ) d 2 (s 1 )} f (s 1 s 2 ; ψ) = f (s 1, s 2 ; ψ, λ) f (s 2 ; ψ, λ) f (s; ψ, λ) with s 2 held fixed s 2 held fixed ˆλ ψ held fixed (ˆθ ψ = (ψ, ˆλ ψ ) saddlepoint approximation: f (s 1 s 2 ; ψ). = c exp[l(ˆθ 0 ψ ; s) l{ˆθ(s); s}] j{ˆθ(s); s} 1/2, s L 0 ψ centering l(θ; s) = ψ T s 1 + λ T s 2 + l(θ; y 0 ) fix s 0 = s(y 0 ) = 0 plane L 0 ψ = {s Rp : s 2 = 0} = {s R p : ˆλ ψ = ˆλ 0 ψ } ˆθ(s) : l(θ; s)/ θ = 0 m.l.e. as a function of the variable Nancy Reid Accurate directional inference for vector parameters York University February /29

7 ... conditional density f (s 1 s 2 ; ψ). = c exp[l(ˆθ 0 ψ ; s) l{ˆθ(s); s}] j{ˆθ(s); s} 1/2, s L 0 ψ Affective disorders Schizophrenics Neurotics Retarded Not retarded s 1 R 2, s 2 R 4 L 0 ψ : all 2 3 tables with the same row and column totals Nancy Reid Accurate directional inference for vector parameters York University February /29

8 Directional tests measure the directed departure from H 0 in L 0 ψ s ψ : expected value of s 1, under H 0 s 0 : observed value of s 1 = 0 from centering L ψ : line through these two points L ψ = ts0 + (1 t)(s ψ s 0 ), t R Relative log likelihood S(t) psi S Nancy Reid Accurate directional inference for vector parameters York University February /29 psi1 S1

9 ... directional tests L ψ = ts0 + (1 t)s ψ Relative log likelihood S(t) psi S psi S1 null hypothesis of independence t = 0 x observed value of s t = 1 p-value compares probability from to to that from to along the line in the sample space curve in the parameter space like a 2-sided p-value Pr ( response > observed response > 0) Nancy Reid Accurate directional inference for vector parameters York University February /29

10 ... directional p-value p-value = 1 t d 1 f {s(t); ψ}dt 0 t d 1 f {s(t); ψ}dt t d 1 from change to polar coordinates Simplifications: s(t) along the line L ψ s, conditional on s/ s 1: L ψ L0 ψ Rp 2: ratio of two integrals drop any terms that don t depend on t 3: saddlepoint approximation f (s; ψ). = c exp[l(ˆθ 0 ψ; s) l{ˆθ(s); s}] j{ˆθ(s); s} 1/2, s L 0 ψ Nancy Reid Accurate directional inference for vector parameters York University February /29

11 2 3 table h(t) t = t d 1 h(t) t = t = t = Nancy Reid Accurate directional inferencetfor vector parameters York University February /29

12 table t=0 t=0.5 psi psi psi psi1 t=1 t=2 psi psi Nancy Reid Accurate directional inference for vector parameters York University February /29 psi1 psi1

13 table simulations Nominal LRT Directional Skovgaard, Nominal LRT Directional Skovgaard, Nancy Reid Accurate directional inference for vector parameters York University February /29

14 Example: comparison of normal variances diameter F -test batch Likelihood ratio statistic Directional Skovgaard, Bartlett s test Nancy Reid Accurate directional inference for vector parameters York University February /29

15 Simulations 3 groups, 10 observations per group Likelihood ratio statistic Bartlett s test p values p values Uniform quantiles Uniform quantiles directional Skovgaard s W* p values p values Uniform quantiles Uniform quantiles Nancy Reid Accurate directional inference for vector parameters York University February /29

16 Simulations 3 groups, 5 observations per group Likelihood ratio statistic Bartlett s test p values p values Uniform quantiles Uniform quantiles directional Skovgaard s W* p values p values Uniform quantiles Uniform quantiles Nancy Reid Accurate directional inference for vector parameters York University February /29

17 Simulations 1000 groups, 5 observations per group Likelihood ratio statistic Bartlett s test p values p values Uniform quantiles Uniform quantiles directional Skovgaard s W* p values p values Uniform quantiles Uniform quantiles dimension ψ 999; dimension nuisance 1001 Nancy Reid Accurate directional inference for vector parameters York University February /29

18 Example: covariance selection model y i N q (µ, Λ 1 ) inverse covariance matrix linear exponential family l(θ; y) = n 2 log Λ 1 2 tr(λy T y) + 1 T yξ n 2 ξt Λξ s 1, s 2 θ = (ξ, Λ) = (Λµ, Λ) H 0 : some off-diagonal elements of Λ are 0 ψ 1 = = ψ d = 0 conditional independence need constrained m.l.e: use fitcongraph in ggm ˆΛ 1 (t) = t ˆΛ 1 + (1 t)ˆλ 1 0 m.l.e. along the line f {s(t); ψ} t ˆΛ 1 + (1 t)ˆλ 1 0 (n q 2)/2 Nancy Reid Accurate directional inference for vector parameters York University February /29

19 ... covariance selection simulations Nominal LRT Directional Skovgaard, Nominal LRT Directional Skovgaard, Covariance matrix 11 11; dimension of ψ = 45 first order Markov dependence Nancy Reid Accurate directional inference for vector parameters York University February /29

20 Tangent exponential model n p (dimension of y to dimension of θ) Every model f (y; θ) on R n can be approximated by an exponential family model: f TEM (s; θ)ds = exp{ϕ(θ) s + l(θ)}h(s)ds s is a score variable on R p s(y) = l ϕ(ˆθ 0 ; y) l(θ) = l(θ; y 0 ) is the observed log-likelihood function ϕ(θ) = ϕ(θ; y 0 ) is the canonical parameter R p to be described matches log-likelihood function at y 0, and its first derivative on the sample space, at y 0 by construction implements conditioning on an approximate ancillary statistic contrast with exp fam Nancy Reid Accurate directional inference for vector parameters York University February /29

21 Aside: canonical parameter ϕ(θ) if f (y; θ) is an exponential family, ϕ is sitting in the model if not find a pivotal quantity z i = z i (y i ; θ) with a fixed distribution ( zi define V i = y i ) 1 z i θ y=y 0,θ=ˆθ 0 example: (y i µ)/σ a vector of length p ϕ(θ) = ϕ(θ; y 0 ) = n i=1 l(θ; y 0 ) y i V i = l ;V (θ; y 0 ) Nancy Reid Accurate directional inference for vector parameters York University February /29

22 Testing a value for ψ eliminating nuisance parameter: p d θ = (ψ, λ); ϕ = ϕ(θ) is the canonical parameter of the TEM with ψ fixed by the hypothesis, we can integrate out the nuisance parameter by Laplace approximation define the same plane in the score space L 0 ψ, where ˆϕ ψ is fixed at its observed value f (s; ψ) = c exp{l( ˆϕ 0 ψ ; s) l( ˆϕ(s); s)} j ϕϕ( ˆϕ(s); s) 1/2 j (λλ) ( ˆϕ 0 ψ ; s) 1/2, s L 0 ψ p-value = 1 t d 1 f {s(t); ψ}dt 0 t d 1 f {s(t); ψ}dt s(t) along the line L ψ Nancy Reid Accurate directional inference for vector parameters York University February /29

23 Example: marginal independence y i N q (µ, Σ), H 0 : some entries of Σ are 0 estimate Σ under H 0 using fitcovgraph l(σ) = n 1 2 n 1 [log ϕ(σ) 2 tr {ϕ(σ)s}], ϕ(σ) = Σ 1 S = sample covariance f {s(t); ψ) t ˆΣ + (1 t)ˆσ 0 (n q 2)/2 j (λλ) {ˆΣ 0 ; s(t)} 1/2 j (λλ) needs l(σ)/ (σ jk ) for the non-zero elements j λj λ k {ˆΣ 0 ; s(t)} = n 1 2 A kj = Σ 1 0 ( Σ/ λ k)σ 1 0 ( Σ/ λ j) ( [ tr(a kj + t tr{(a kj + A jk )(ˆΣ 1 ˆΣ ]) 0 I q )} Nancy Reid Accurate directional inference for vector parameters York University February /29

24 ... independence simulations; n = 60; 600, d = 1000 Nominal LRT Directional Skovgaard, Nominal LRT Directional Skovgaard, Nominal LRT Directional Skovgaard, Nominal LRT Directional Skovgaard, Nancy Reid Accurate directional inference for vector parameters York University February /29

25 Conclusion different way to assess vector parameters incorporates information in the direction of departure easy to compute: two model fits, plus 1-d numerical integration accurate conditionally, by construction, and unconditionally can be used in models of practical interest simulations exponential family model not necessary easily generalized using approximate exponential family model Nancy Reid Accurate directional inference for vector parameters York University February /29

26 ... conclusion psi Nancy Reid Accurate directional inference for vector parameters psi1 York University February /29

27 ... conclusion s s1 Nancy Reid Accurate directional inference for vector parameters York University February /29

28 ... conclusion t Nancy Reid Accurate directional inference for vector parameters York University February /29

29 References Fraser, D.A.S. and Massam, H. (1985). Conical tests. Stat. Hefte Skovgaard, I.M. (1988). Saddlepoint expansions for directional test probabilities. JRSS B Skovgaard, I.M. (2001). Likelihood asymptotics. SJS Cheah, P.K., Fraser, D.A.S., Reid, N. (1994). Multiparameter testing in exponential models. Biometrika Davison, A.C., Fraser, D.A.S., Reid, N., Sartori, N. (2014). Accurate directional inference for vector parameters in linear exponential.jasa Fraser, D.A.S., Reid, N., Sartori, N. (2016). Accurate directional inference for vector parameters. submitted Nancy Reid Accurate directional inference for vector parameters York University February /29

Accurate directional inference for vector parameters

Accurate directional inference for vector parameters Nancy Reid October 28, 2016 with Don Fraser, Nicola Sartori, Anthony Davison Parametric models and likelihood model f (y; θ), θ R p data y = (y 1,...,