Data, Estimation and Inference

Transcription

1 Data, Estimation and Inference Pedro Piniés Michaelmas

2 2

3 p(x) ( = ) = δ 0 ( < < + δ ) δ ( ) =1. x x+dx (, ) = ( ) ( ) = ( ) ( ) 3

4 ( ) ( ) 0 ( ) =1 ( = ) = ( ) ( < < ) = ( ) ( ) 4

5 ( ) = ( ;, ) = 1 ( ) ( ) ( ) 1 b a p x (x) a b x 5

6 B / 44 Consider the Binomial Distribution. Perform n independent trials (e.g. biased coin tosses), the outcome of each is success with probability p and failure with probability q =(1 p) (that is, each has a Bernoulli distribution.) Probability of x successes is P(x n,p)=b(x;n,p)= n! x!(n x)! px (1 p) n x Mean: µ = n n!  x x=0 x!(n x)! px (1 p) n x = np Variance: s 2 = n  (x µ) 2 n! x=0 x!(n x)! px (1 p) n x = np(1 p) 6

7 B / 44 The Poisson distribution emerges from the Binomial for p 1. Assume p 1, and µ = np is a constant. Write P(x n,p)= 1 x! If x n and n large then n! (n x)! n! (n x)! px (1 p) x (1 p) n. = n(n 1)(n 2)...(n x + 1)(n x)(n x 1)...1 (n x)(n x 1)...1 n x (1 p) x 1 + px 1 µ lim (1 p!0 p)n = lim (1 p) 1 /p =( 1/e) µ = e µ p!0 ) P(x µ)=po(x; µ)= µx x! e µ s 2 = µ = np. 7

8 B / 44 The Poisson distribution is used for events that occur randomly at independent times, but at an average intensity µ of events per unit time. Example: Number of telephone calls to a call centre in an hour. Example: Shot noise in electronics. Example: Number of direct fire on NATO ISAF coalition forces in Afghanistan per day. More generally, it is a distribution over the natural numbers N = {0,1,2,3,...}. 8

9 µ σ ( ) =N ( ; µ, σ 2 )= 1 ( µ)2 ( 2πσ 2σ 2 ) µ = 2 2σ 1.2σ p(x) x 9

10 B / 44 This result can be seen as a special case of the Central Limit Theorem. The first reason for the importance of the Gaussian is the following theorem. Central Limit Theorem: If one takes a suitably normalized sum of an (infinitely) large number of independent random variables x i, then that normalised sum will be Gaussian distributed. This is the case, no matter what the distributions of the individual random variables. Indeed, they can have different distributions. There are some caveats: we require the distributions to have finite means and variances: the Cauchy would break the theorem. 10

11 ( ) = σ 0 N ( ; µ, σ 2 )=δ( µ). = ( ) ( ) =δ ( ( ) ). 11

12 B / 44 The set to which a distribution assigns probability mass is known as its support. Distribution Support Bernoulli Be(x; p) x 2 {0, 1} Binomial B(x;n,p) x 2 {1,2,...,n} Poisson Po(x; µ) x 2 N Gaussian N (x; µ,s 2 ) x 2 R Those relationships between distributions again: Bernoulli multiple trials! Binomial p 1, and np=µ! Poisson µ 0! Gaussian. 12

13 B / 44 Assigning prior distributions is a crucial part of inference. Which might be an appropriate distribution for the number of firsts awarded by the department this year? 1 Bernoulli 2 Binomial 3 Poisson 4 Gaussian. 13

14 B / 44 Assigning prior distributions is a crucial part of inference. Which might be an appropriate distribution for the number of buses going past my window in an hour? 1 Bernoulli 2 Binomial 3 Poisson 4 Gaussian. 14

15 B / 44 Assigning prior distributions is a crucial part of inference. Which might be an appropriate distribution for the length of a piece of string? 1 Bernoulli 2 Binomial 3 Poisson 4 Gaussian. 15

16 E[ ] = ( ) =µ = ( ). E[ ( )] = ( ) ( ). E[ ]= ( ). E[( µ) ]= ( µ) ( ). ( ) =E[( µ) 2 ]= ( µ)2 ( ). 16

17 ( =, = ) = (, ). (, ) 0 (, ) =1. ( ) = (,, ) 17

18 0 ( 1, 2, 3,...) ( ) 18

19 (, ) =E [ ( E[ ]) ( E[ ]) ]. Σ = E[( µ)( µ) ] ( 1 ) ( 1, 2 ) ( 1, 3 ) Σ = ( 2, 1 ) ( 2 ) ( 2, 3 ) ( 3, 1 ) ( 3, 2 ) ( 3 ) (, )= (, ) Σ 19

20 >µ >µ Positive covariance Zero covariance Negative covariance Zero covariance 20

21 In summary, pdfs allow us to work with continuous variables. pdfs can be manipulated just like probabilities except that sums are replaced with integrals. Assigning prior distributions is a crucial part of inference. Moments generalise readily to pdfs. 21