Chapter 4: Modelling

Transcription

1 Chapter 4: Modelling Exchangeability and Invariance Markus Harva / Reading Circle on Bayesian Theory

2 Outline 1 Introduction 2 Models via exchangeability 3 Models via invariance 4 Exercise

3 Statistical models Events of interest defined in terms of random quantities x 1,..., x n Individual s degrees of belief specified as a joint distribution function P(x 1,..., x n ) (or density p(x 1,..., x n )) In an application a specific form for p is chosen Here we study more general belief structures that lead to a mathematical representation of a model

4 Exchangeability Sometimes the indices of the random quantities x 1,..., x n are judged not to be significant This leads to the notion of finite exchangeability Definition (Finite exchangeability) The random quantities x 1,..., x n are finitely exchangeable under a probability measure P if for all permutations π. P(x 1,..., x n ) = P(x π(1),..., x π(n) ) Partial and infinite exchangeability

5 The Bernoulli model Infinite exchangeability for 0-1 random quantities = the Bernoulli model Theorem (Representation theorem for 0-1 random quantities) If x 1, x 2,... are 0-1 random quantities and infinitely exchangeable under P, their joint mass function p is of the form p(x 1,..., x n ) = 1 0 n θ x i (1 θ) 1 x i dq(θ), where Q(θ) = lim n P(y n /n θ) with y n = x x n. i=1

6 The multinomial model Infinite exchangeability for 0-1 random vectors = the multinomial model Theorem If x 1, x 2,... are 0-1 random vectors and infinitely exchangeable under P, their joint mass function p is of the form p(x 1,..., x n ) = n Θ i=1 ( θ x i θx ik k 1 k ) 1 Pj θ x ij j dq(θ), where Q(θ) = lim n P(( x 1n θ 1 ) ( x kn θ k )), with x in = n 1 (x 1i + + x ni ). j=1

7 The general model Infinite exchangeability for real-valued random quantities = something rather abstract: Theorem If x 1, x 2,... are real-valued infinitely exchangeable random quantities under probability measure P, the form of P is P(x 1,..., x n ) = F i=1 n F(x i )dq(f), where Q(F) = lim n P(F n ), F n being the empirical distribution defined by x 1,..., x n.

8 Spherical symmetry The general representation theorem does not provide a concrete usable model More assumptions needed in addition to infinite exchangeability Definition (Spherical symmetry) A random vector x = [x 1,..., x n ] has spherical symmetry under P, if P(x) = P(Ax) for all orthogonal matrices A.

9 The normal model Spherical symmetry = the normal model Theorem (Representation theorem under spherical symmetry) If x 1, x 2,... is an infinite sequence of real-valued random quantities with probability measure P, and if, for any n, x n := [x 1,..., x n ] has spherical symmetry, the distribution of x n has the form P(x n ) = 0 n Φ(λ 1/2 x i )dq(λ), i=1 where Φ is the standard normal distribution function and Q(λ) = lim n P(s 2 n λ) with s 2 n := n 1 (x x 2 n ).

10 Origin invariance Exchangeability of positive random quantities = symmetry of events w.r.t. the 45 line through the origin Extension of this: Definition (Origin invariance) An infinitely exchangeable sequence x 1, x 2,... of positive real-valued random quantities with probability measure P has origin invariance if for all n and any event A R n + P((x 1,..., x n ) A) = P((x 1,..., x n ) A + a) for all a R n such that a T 1 = 0.

11 The exponential model Origin invariance = the exponential model Theorem (Continuous representation under origin invariance) If the sequence x 1, x 2,... of positive real-valued random quantities has origin invariance under P, the joint density has the form p(x 1,..., x n ) = 0 n θ exp( θx i )dq(θ), i=1 where Q(θ) = lim n P( x 1 n θ) with x n = n 1 (x x n ).

12 Exercise Come up or find in literature a model for a sequence of real-valued random variables x 1, x 2,... that is infinitely exchangeable and whose representation in terms of the general representation theorem genuinely involves an integral over measures. Write down the model using the notation of Proposition 4.3.