ESS011 Mathematical statistics and signal processing

Transcription

1 ESS011 Mathematical statistics and signal processing Lecture 9: Gaussian distribution, transformation formula for continuous random variables, and the joint distribution Tuomas A. Rajala Chalmers TU April 1, 2014

2 1/17 Where are we Yesterday we looked at some popular distribution families. Today we take a closer look at the normal distribution. In addition we see a rule for transforming r.v. s, and talk about vectors of random variables.

3 2/17 Normal distribution: definition Recall: Normal distribution A continuous random variable X follows the normal distribution N(µ, σ 2 ) iff the density is of form f() = 1 1 σ 2π e 2 ( µ)2 /σ 2, R Also called the Gaussian distribution (google hits: normal 2.3m, Gaussian 0.9m) Also known as the bell curve due to its shape: f() (de Moivre 18th c., Gauss, Laplace, Pearson 19th c.)

4 3/17 Gaussian distribution properties The parameters have a direct role: for X N(µ, σ 2 ) E(X) = µ Var(X) = σ 2 It is symmetric around its mean: f(µ ) = f(µ + ) for all. The mode, ma. point of f, is µ N(0,1) N(0,0.09) N(3,1) f() f() f()

5 Standard normal distribution If µ = 0 and σ 2 = 1 for normal distribution, i.e. N(0, 1), it is called the standard normal distribution. So often used that its CDF has its own notation: Φ() := 1 2π Standardization If X N(µ, σ 2 ), then e t2 /2 dt Proof: later today. Z := X µ σ N(0, 1) Then we can find the value P (X < a) = Φ( a µ σ ) for any a R using tabulations of Φ. 4/17

6 5/17 Using standard normal Eample: Let X N(1.8, ). What is P (X < 1)? We write z = (1 1.8)/ and check from table/computer Φ(z) In statistics we want to know when event is significant. Significant is often translated to happends less than 5% of time. Rule of 2σ: For X N(µ, σ 2 ), N(0,1) or P (µ 2σ < X < µ + 2σ) 0.95 P ( 2 < Z < 2) 0.95 f() ~

7 6/17 Approimating probabilities In general, we can use Chebyshev s inequality For any X with mean µ and variance σ 2, and any positive number t, P ( X µ < tσ) 1 1 t 2 or put in another way P (µ tσ < X < µ + tσ) 1 1 t 2 m=2.1, s=1.4 This works for any distribution. f() >0.95 (=0.997)

8 For eample: A polio vaccine is produced with an average 75 active molecules out of n = 100 molecules. In order to be effective, 67% of the molecules must be active. Your doctor gives you a shot. Does it work? Let X Binom(100, 0.75), we need to find P (X > 67). With µ = 75, σ 4.3, we get t = (75 67)/ , and Chebyshev P (µ tσ < X < µ + tσ) 1 1/t 2 becomes This implies P (67 < X < 83) 0.71 P (X < 67 or X > 83) = P (X < 67) + P (X > 83) 0.29 Now we can only say that the vaccine does not work with probability P (X < 67) < 0.29 The true prob. of not working is 0.04 so the approimation is not very good. 7/17

9 / Gaussian distribution as an approimation For many distributions, normal distribution provides an approimation. For the binomial eample, we will later see that when p > 0.5 and n(1 p) > 5 Binom(n, p) N(np, np(1 p)) E.g. prev. eample P (X > 67) 0.97, (true 0.96) binom(n=100,p=0.75) and N(pn, np(1 p)) Poisson(5) Poisson(15) 0.10 Similarly, for large λ P oisson(λ) N(λ, λ) f() f() (the fact that > 0 is not a problem) * * *

10 9/17 Transformation of a random variable Consider a continuous random variable X with some density f, and some differentiable and monotonic function g. What is the distribution of g(x)? f() g()=^0.5? f() g0() g() Note: When g is non-linear, Eg(X) g(e(x))

11 10/17 The change of variable method The idea: P (g(x) < ) = P (X < g 1 ()). Differentiation leads to the density of a transformed variable: Change of variable formula Let r.v. X have a density f X with domain D, and let g : D D be a differentiable one-to-one mapping. Then the density of Y := g(x) is with J(y) = dg 1 (y)/dy. f Y (y) = f X (g 1 (y)) J(y), y D Remember to check the change of support.

12 11/17 Eamples of transformations Standard normal: Let X N(µ, σ 2 ) and let Z := g(x) = X µ. Then σ g 1 (z) = σz + µ, Jacobian is J(z) = σ, and f z (z) = f (g 1 (z)) J(z) = σ 1 σ 2π e 2 (σz+µ µ)2 /σ 2 = 1 e 1 2 z2 N(0, 1) 2π Simulation: Say we want to simulate numbers Y Ep(1/λ). It is enough that a computer can simulate randomness through numbers U Unif(0, 1). Why? Set g(u) := log(u)/λ. Then g 1 (y) = e yλ and J(u) = λe yλ so f y (y) = f u (g 1 (u)) J(u) = 1 λe yλ Ep(1/λ)

13 12/17 Joint distributions So far we have studied univariate r.v. s. Let s go to the multivariate case. Discrete joint density Let X 1, X 2,..., X n be discrete r.v. s with domains D i. The vector X = (X 1, X 2,..., X n ) is called a multivariate r.v., and a function f( 1, 2,..., n ) = P (X 1 = 1, X 2 = 2,..., X n = n ) is called the joint density of X. Cont. joint density Let X i in previous def. be continuous. Function f such that 1 f( 1,..., n ) 0, i D i 2 D 1... D n f( 1,..., n )d 1... d n = 1 3 P (X 1 [a 1, b 1 ],..., X n [a n, b n ]) = b 1 a 1... b n a n f( 1,..., n )d 1... n for a i < b i D i is called the joint density X.

14 13/17 Joint distribution: Marginals To get back to univariates, we can marginalize: The marginal densities For a multivariate random variable X = (X 1,..., X n ), the functions f i ( i ) = f( 1,..., n ) (discrete) f i ( i ) = j D j:j i D j:j i f( 1,..., n )d 1...d i 1 d i+1...d n for i = 1,..., n are called the marginal distributions of X i. Any combination of discrete and continuous similarly. (continuous)

15 14/17 Eample of joint distribution Coins: Toss a fair coin 3 times. Let X =1st is heads (1), Y =total number of heads. The sample space is {000, 001, 010, 100, 011, 110, 101, 111}, and the joint distribution f(, y) is as in the table. All probabilities can be derived from the table: e.g. P (X = 0 and Y > 0) = f(0, 1) + f(0, 2) + f(0, 3) = 3/8 The marginals: f X (0) = y f(0, y) = 0.5 f X (1) = y f(1, y) = 0.5 for X, and similarly for Y /8 2/8 1/ /8 2/8 1/ f X 0 1/8 2/8 1/8 0 4/ /8 2/8 1/8 4/8 f Y 1/8 3/8 3/8 1/8 Σ = 1

16 15/17 Another eample of joint distribution Let X [8.5, 10.5] denote individual s blood calsium level and Y [120, 240] colestherol level. Assume that the two have a jointly uniform distribution, f(, y) = c with some constant c > 0. First,to be a density, solve c: c ddy = 1 c = 1/ Marginals: f X () = c dy = 1/2, f Y (y) = c d = 1/120. The marginals are also uniform.

17 16/17 Joint distribution: Independence Recall: A and B independent iff P (A B) = P (A)P (B). This translates to Independent r.v. s X and Y are independent, denoted by X Y, iff f XY (, y) = f X ()f Y (y), y In general, X 1,..., X n are independent iff f( 1,..., n ) = n i=1 f i( i ) for all i R. Analogously: X Y iff F (, y) = F X ()F Y (y) Coin eample: f(0, 0) = 1/ but f X (0)f Y (0) 0.06, so not independent. Uniform eample: f(, y) = 1/240 and f X ()f Y (y) = = for all, y so they are independent.

18 17/17 Summary Three items on random variables today: 1 Normal distribution 2 Transformation formula for continuous r.v. s 3 Joint distributions: Etending to multivariates Net week we finish with etending the probability theory from sets to multivariate random variables, and learn the theorem that leads us to statistics.