Summary. Ancillary Statistics What is an ancillary statistic for θ? .2 Can an ancillary statistic be a sufficient statistic?

Transcription

1 Biostatistics 62 - Statistical Inference Lecture 5 Hyun Min Kang 1 What is an ancillary statistic for θ? 2 Can an ancillary statistic be a sufficient statistic? 3 What are the location parameter and the scale parameter? 4 In which case ancillary statistics would be helpful? January 24th, 213 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Last Lecture : Example : Uniform Definition 6216 A statistic S(X) is an ancillary statistic if its distribution does not depend on θ Examples of iid X 1,, X n N (µ, σ 2 ) where σ 2 is known s 2 X = 1 n 1 n i=1 (X 1 X) 2 is an ancillary statistic X 1 X 2 N (, 2σ 2 ) is ancillary (X 1 + X 2 )/2 X 3 N (, 15σ 2 ) is ancillary (n 1)s2 X σ 2 χ 2 n 1 is ancillary iid X 1,, X n Uniform(θ, θ + 1) Show that R = X (n) X (1) is an ancillary statistic Possible Strategies Method 1 : Obtain the distribution of R and show that it is independent of θ Method 2 : Represent R as a function of ancillary statistics, which is independent of θ Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

2 Method 2 - A Simpler Proof Location-Scale Family and Parameters f X (x θ) = I(θ < x < θ + 1) = I( < x θ < 1) Let Y i = X i θ Uniform(, 1) Then X i = Y i + θ, dx dy = 1 holds f Y (y) = I( < y + θ θ < 1) dx dy = I( < y < 1) Then, the range statistic R can be rewritten as follows R = X (n) X (1) = (Y (n) + θ) (Y (1) + θ) = Y (n) Y (1) As Y (n) Y (1) is a function of Y 1,, Y n Any joint distribution of Y 1,, Y n does not depend on θ Therefore, R is an ancillary statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Definition Let T = {f T (t θ), θ Ω} be a family of pdfs or pmfs for a statistic T(X) Equivalently, T(X) is called a complete statistic Example T(X) N (, 1) The family of probability distributions is called complete if E[g(T) θ] = for all θ implies Pr[g(T) = θ] = 1 for all θ In other words, g(t) = almost surely g 1 (T(X)) = = Pr[g 1 (T(X)) = ] = 1 g 2 (T(X)) = I(T(X) = ) = Pr[g 2 (T(X)) = ] = 1 Pr[T(X) = )] In this case, g 2 (T(X)) = is almost surely true Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Definition 355 Let f(x) be any pdf Then for any µ, < µ <, and any σ > the family of pdfs f((x µ)/σ)/σ, indexed by the parameter (µ, σ) is called the location-scale family with standard pdf f(x), and µ is called the location parameter and σ is called the scale parameter for the family Example f(x) = 1 2π e x2 /2 N (, 1) f((x µ)/σ)/σ = 1 2πσ 2 e (x µ)2 /2σ 2 N (µ, σ 2 ) Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Notes on Notice that completeness is a property of a family of probability distributions, not of a particular distribution For example, X N (, 1) and g(x) = x makes E[g(X)] = EX =, but Pr(g(X) = ) = instead of 1 The above example is only for a particular distribution, not a family of distributions If X N (θ, 1), < θ <, then no function of X except for g(x) = satisfies E[g(X) θ] for all θ Therefore, the family of N (θ, 1) distributions, < θ <, is complete Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

3 Why Complete Statistics? Example - Poisson distribution Stigler (1972) Am Stat 26(2):28-9 While a student encountering completeness for the first time is very likely to appreciate its usefulness, he is just as likely to be puzzled by its name, and wonder what connection (if any) there is between the statistical use of the term complete, and the dictionary definition: lacking none of the parts, whole, entire Requiring g(t) to satisfy the definition puts a restriction on g The larger the family of pdfs/pmfs, the greater the restriction on g When the family of pdfs/pmfs is augmented to the point that E[g(T)] = for all θ, it rules out all g except for the trivial g(t) =, then the family is said to be complete { } Suppose T = f T : f T (t λ) = λt e λ for t {, 1, 2, } Let λ Ω = {1, 2} Show that this family is NOT complete Proof strategy We need to find a counter example, which is a function g such that E[g(T) λ] = for λ = 1, 2 but g(t) Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Poisson distribution example : Proof Poisson distribution example : Proof (cont d) The function g must satisfy for λ {1, 2} Thus, E[g(T) λ] = t= g(t) λt e λ = E[g(T) λ = 1] = t= g(t) 1t e 1 = E[g(T) λ = 2] = t= g(t) 2t e 2 = The above equation can be rewritten as t= g(t)/ = t= 2t g(t)/ = Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Define g(t) as Then g(t) = 2 t = t = 2 3 t = 1 otherwise g(t)/ = g()/! + g(1)/1! + g(2)/2! = /2 = t= 2 t g(t)/ = g()/! + 2g(1)/1! g(2)/2! = /2 = t= There exists a non-zero function g that satisfies E[g(T)λ] = for all λ Ω Therefore this family is NOT complete Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

4 Another example with Poisson distribution Proof : Finding the moment-generating function of X iid X 1,, X n Poisson(λ), λ > Show that T(X) = n i=1 X i is a complete statistic Proof strategy Need to find the distribution of T(X) Show that there is no non-zero function g such that E[g(T) λ] = for all λ M X (s) = E[e sx ] = x= e sx e λ λ x x! = e λ (es λ) x e esλ e es λ x! x= = e λ e es λ (es λ) x e es λ x! x= = e λ e es λ f Poisson (x e s λ) = e λ(es 1) x= Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Proof : Finding the MGF of T(X) = n i=1 X i M T (s) = E(e s X i ) = E = = ( n i=1 e sx i ) n E ( e sx ) i = [ E ( e sx )] n i i=1 [ e λ(es 1) ] n = e nλ(e s 1) Theorem 2311 (b) Let F X (x) and F Y (y) be two cdfs all of whose moments exists If the moment generating functions exists and M X (t) = M Y (t) for all t in some neighborhood of, then F X (u) = F Y (u) for all u By Theorem 2311, T(X) Poisson(nλ) Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Proof : Showing E[g(T) λ] = Pr[g(T) = ] = 1 Suppose that there exists a g(t) such that E[g(T) λ] = for all λ > e nλ (nλ) t E[g(T) λ] = Which is equivalent to t= = e nλ g(t)(nλ) t = t= g(t)n t λ t = t= for all λ > Because the function above is a power series expansion of λ, g(t)n t / = for all t and g(t) = for all t Therefore T(X) = n i=1 X i is a complete statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

5 Example : Uniform Distribution Proof : Uniform Distribution (cont d) iid Let X 1,, X n Uniform(, θ), θ >, Ω = (, ) Show that X (n) is complete Proof We need to obtain the distribution of T(X) = X (n) Let f X (x) = 1 θ I( < x < θ), then its cdf is F X(x) = x θ I( < x < θ) + I(x θ) n! f T (t θ) = (n 1)! f X (t)f X(t) n 1 = n ( ) t n 1 I( < t < θ) = nθ n t n 1 I( < t < θ) θ θ Consider a function g(t) such that E[g(T) θ] = for all θ > E[g(T) θ] = θ Taking derivative of both sides, ng(θ) θ g(t)nθ n t n 1 I( < t < θ)dt = n θ θ n g(t)t n 1 dt = n θ θ n g(θ)θn 1 n2 θ n+1 g(t)t n 1 dt = = n n θ θ θ n g(t)t n 1 dt = n θ E[g(T) θ] = Because g(t) = holds for all θ >, T(X) = X (n) is a complete statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 A simpler proof (how it was solved in the class) Another Example of Uniform Distribution Consider a function g(t) such that E[g(T) θ] = for all θ > θ E[g(T) θ] = g(t)t n 1 dt = Taking derivative of both sides, θ g(t)nθ n t n 1 I( < t < θ)dt = n θ θ n g(t)t n 1 dt = g(θ)θ n 1 = g(θ) = for all θ > Because g(t) = holds for all θ >, T(X) = X (n) is a complete statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 iid Let X 1,, X n Uniform(θ, θ + 1), θ R We have previously shown that T(X) = (X (1), X (n) ) is a minimal sufficient statistic for θ Show that T(X) is not a complete statistic Proof - Using a range statistic Define R = X (n) X (1) We have previously shown that f R (r θ) = n(n 1)r (n 2) (1 r), < r < 1 Then R Beta(n 1, 2), and E[R θ] = n 1 n+1 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

6 Proof Even Simpler Proof Define g(t(x)) = X (n) X (1) n 1 n+1 E[g(T) θ] = E[X (n) X (1) θ] n 1 n + 1 = n 1 n + 1 n 1 n + 1 = Therefore, there exist a g(t) such that Pr[g(T) θ] < 1 for all θ, so T(X) = (X (1), X (n) ) is not a complete statistic We know that R = X (n) X (1) is an ancillary statistic, which do not depend on θ Define g(t) = X (n) X (1) E(R) Note that E(R) is constant to θ Then E[g(T) θ] = E(R) E(R) =, so T is not a complete statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Example from Stigler (1972) Am Stat Solution (cont d) Let X is a uniform random sample from {1,, θ} where θ Ω = N Is T(X) = X a complete statistic? Solution Consider a function g(t) such that E[g(T) θ] = for all θ N Note that f X (x) = 1 θ I(x {1,, θ}) = 1 θ I N θ (x) E[g(T) θ] = E[g(X) θ] = g(x) = 1 θ g(x) = 1 θ g(x) = for all θ N, which implies if θ = 1, θ g(x) = g(1) = if θ = 2, θ g(x) = g(1) + g(2) = g(2) = if θ = k, θ g(x) = g(1) + + g(k 1) + g(2) = g(k) = Therefore, g(x) = for all x N, and T(X) = X is a complete statistic for θ Ω = N Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26

7 Is the previous example barely complete? Modified Let X is a uniform random sample from {1,, θ} where θ Ω = N {n} Is T(X) = X a complete statistic? Solution Define a nonzero g(x) as follows g(x) = E[g(T) θ] = 1 θ 1 x = n 1 x = n + 1 otherwise { θ n g(x) = θ = n 1 θ Today - Examples of complete statistics Two Poisson distribution examples Two Uniform distribution examples Example of barely complete statistics Next Lecture Basu s Theorem Because Ω does not include n, g(x) = for all θ Ω = N {n}, and T(X) = X is not a complete statistic Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26 Hyun Min Kang Biostatistics 62 - Lecture 5 January 24th, / 26