Modeling: How do we capture the uncertainty in our data and the world that produced it?

Transcription

1 Location-scale families January 4, 209 Debdeep Pati Overview data statistics inferences. As statisticians, we are tasked with turning the large amount of data generated by eperiments and observations into inferences about the world. This simple directive gives rise to a number of core statistical questions: Modeling: How do we capture the uncertainty in our data and the world that produced it? Methodology: What are the right mathematical and computational tools that allow us to draw these statistical inferences? Analysis: How do we compare and evaluate the statistical inferences we make and the procedures we use to make them? In particular, how do we do optimal inference? 2 Location-Scale Family Definition. A location-scale family of distributions has densities (pdf s) of the form g( µ, ) = ( ) µ ψ where ψ is a pdf, > 0, < µ <. 2. Properties. g( 0, ) = ψ() 2. If X g( µ, ), then X µ g( 0, ). 3. If X g( 0, ), then X + µ g( µ, ). 2.2 Eamples. Gaussian: Take ψ() = e 2 /2, the standard normal density. Then g( µ, ) = ( ) µ ψ = ( e µ ) 2/2 = e ( µ)2 2 2, < <

2 is the pdf of a N(µ, 2 ) distribution. 2. Cauchy: Take ψ() = π + 2, Then g( µ, ) = ψ ( µ ) = π + ( µ ) 2, < < defines the Cauchy L-S family. Note: for this family of distributions, µ is not the mean and is not the standard deviation. Same remark applies in the net eample 3. Uniform: Take ψ() = I (0,) () =, if 0 < < 0, otherwise. ψ is the pdf of the Unif(0, ) distribution. g( µ, ) = ( ) µ ψ = ( ) µ I (0,) = I (µ,µ+)() where the last equality follows from the fact that µ is the pdf of the of the Unif(µ, µ + ). (0, ) iff (µ, µ + ). This 3 Location / Scale Family Let ψ be a pdf. Definition 2. A scale family of distributions has densities of the form g( ) = ψ( ) where > 0. is the scale parameter. Definition 3. A location family of distributions has densities of the form g( µ) = ψ( µ) where < µ <. 3. Eamples. The N(µ, ) distributions form a location family and N(0, 2 ) distributions form a scale family. Take ψ() = e 2 /2. Then ψ( µ) = e ( µ)2 /2, which is the pdf of N(µ, ) and ψ( ) = e 2 /2 2, which is the pdf of N(0, 2 ). 2

3 2. The family of Gamma(α 0, β) distributions (where α 0 is any fied value of α forms a scale family. Take ψ() = α 0 e Γ(α 0 ), > 0. Then ( ) ψ = ( ) α0 e / Γ(α 0 ) = α 0 e / α 0 Γ(α0 ) which is the pdf of the Gamma(α 0, ) distribution. Note: If we permit both α and β to vary, the family of Gamma(α, β) distributions does not form a location-scale family. (α is a shape parameter). Suppose g( µ, ) is a location-scale family of density and X g( µ, ), Z g( 0, ). Then so that X µ d = Z, X d = Z + µ P (X > b) = ( X µ P > b µ ) ( = P Z > b µ ) E(X) = E(Z + µ) = EZ + µ, if EZ is finite V ar(x) = V ar(z + µ) = 2 V ar(z) if V ar(z) is finite Similar facts hold for location families and scale families. Erase µ (set µ = 0) for facts for scale families. Erase (set = ) for facts about location families. 3.2 Eamples. The N(, 2 ), > 0, distributions form a scale family. The density of the N(, 2 ) distribution is: [ ] ( µ) 2 ep 2 2 = ep ( ) 2 2 where ψ is the N(, ) pdf. 2. The N(, λ), λ > 0, distributions do not form a scale family. One way to see this is to note that if X N(, λ) then EX = for all λ (it is constant). But a scale family with scale parameter satisfies EX = EZ which cannot be constant (unless EZ = 0). 3

4 4 Eponential families Definition 4. The family of pdf s or pmf s f( θ) : θ Θ, where Θ is the parameter space and θ can represent a single parameter or a vector of parameters is an eponential family if we can write k f( θ) = h() ep w i (θ)t i () for real valued functions, h(),, t() with, h() 0 for all and all θ Θ. In addition > 0 for all θ Θ. This is the general k-parameter eponential family(kpef). For k =, the general one parameter eponential family(pef) has the form i= f( θ) = h() ep t() for all and all θ Θ. Note: We allow h to be degenerate (constant), but require all the other functions to be nondegenerate (nonconstant). 4. Eamples of pef s. Eponential Distributions: The pdf is given by f( β) = β e /β, > 0, β > 0. In this eample, θ = β and Θ = (0, ) and f( β) = I (0, ) ().. ep β h() β. t() This f( β) forms a pef with the parts as identified above. 2. Binomial distributions: The family of Binomial(n, p) distributions with n known (fied) is a pef. The pmf is ( ) n f( p) = p ( p) n, = 0,,..., n, 0 < p < ) ( p) n ( p p 4

5 In this eample, θ = p and Θ = (0, ) and ( ) n f( p) = I 0,,...,n (). ( p) n h(). ep t() This f( p) forms a pef with the parts as identified above. p. log p 4.2 Eamples of 2pef s. The family of N(µ, 2 ) distributions: The pdf is given by f( µ, 2 ) = e ( µ)2 /2 2, < < valid for > 0 and < µ <. In this eample, θ = (µ, 2 ) and Θ = (µ, 2 ) : 2 > 0, and < µ <. f( µ, 2 ) = = ep µ 2 µ2 h() 2 2. ( ) ep µ ep 2 2 w (θ). 2 t () + µ 2 w 2 (θ). t 2 () This f( µ, 2 ) forms a 2pef with the parts as identified above. 4.3 Non-eponential families There are many families of distributions which are not eponential families. The Cauchy L-S family family f( µ, ) =. ( π + µ cannot be written as an eponential family. (Try it!). ) 2, < < A trickier Eample: Consider this shifted eponential distribution with pdf β f( µ, β) = e ( µ)/β, > µ 0, otherwise. 5

6 Note that f( µ, β) = h(). eµ/β β ep β. t(), Is it correct? This is not valid for all, but only for > µ. To get an epression valid for all, we need an indicator function. f( µ, β) = ep This is not an eponential family. I (µ, ) () not a function of alone. eµ/β β β.. t() Definition 5. The support of an eponential family of a pdf or pmf f() is the set : f() > 0. Fact: The support of an eponential family of pdf s (pmf s) f( θ) is the same for all θ. Proof. (For pef) The support of f( θ) = h() ept() is : h() > 0 which does not involve θ. Fact: If f( θ) is pef, then for any 4 distinct points, 2, 3 and 4 in the support, g(, θ) = logf( θ)/h() satisfies independent of θ. Proof. Observe that if f( θ) is pef and the proof follows immediately. g(, θ) g( 2, θ) g( 3, θ) g( 4, θ) g(, θ) g( 2, θ) g( 3, θ) g( 4, θ) = t( ) t( 2 ) t( 3 ) t( 4 ) 6

7 4.4 Eamples. The pdf f( µ, β) = β e ( µ)/β, > µ 0, otherwise. has support : > µ which depends on θ = (µ, β) through the value µ. Thus (without further work) we know that this is not an eponential family. 2. The family of Unif(a, b) distributions with < a < b < is not an eponential family. The Unif(a, b) density f( a, b) = b a, for a < < b 0, otherwise has support : a < < b which depends on θ = (a, b). Thus (without further work) we know that this is not an eponential family. 3. The Cauchy L-S family is not an eponential family, but its support is the same for all θ = (µ, ). 7