Probability- the good parts version. I. Random variables and their distributions; continuous random variables.

Transcription

1 Probability- the good arts version I. Random variables and their distributions; continuous random variables. A random variable (r.v) X is continuous if its distribution is given by a robability density function (df) f(x) that is ositive on an interval. For real numbers a < b, P (a < X < b) = b a f(x)dx. Random variables X and Y are jointly continuous if there s a joint density function f(x, y) such that P (a < X < b, c < Y < d) = b d a c f(x, y)dydx. The marginal density for X is found by integrating out the y and viceversa for Y. X and Y are called indeendent if their joint density is the roduct of their marginals. In this case, it follows that P (a < X < b, c < Y < d) = P (a < X < b)p (c < Y < d). The cumulative distribution function (cdf) of X is the function F, F (x) = P (X x) = x f(t)dt. Given a function g(), the exected value of g(x) = E(g(X)) = g(x)f(x)dx. In articular, the mean of X= E(X)= µ; the variance of X= V (X)= σ 2 = E(X µ) 2 = EX 2 µ 2 ; and the standard deviation σ = σ 2. 1

2 The moment generating function of X is M X (t) = E(e Xt ) = e xt f(x)dx. Proerties of the moment generating function: (1) M (n) X dn (0) = dt M n X (t), evaluated at t=0= EX n. (2) If X 1, X 2,..., X n are indeendent random variables then M X1 +X X n (t) = M X1 (t) M X2 (t) M Xn (t). (3) The m.g.f. secifies the distribution: if M X (t) = M Y (t), then X and Y have the same distribution. (4) M ax+b (t) = e bt M X (at). (1) X N(µ, σ 2 ) The standard class of continuous distributions. density: f(x) = 1 2πσ e (x µ)2 /2σ 2, < x <. mean: E(X) = µ. variance: V (X) = σ 2. mgf: M(t) = e µt+σ2 t 2 /2. If X N(µ, σ 2 ), then (X µ)/σ N(0, 1). If X N(µ X, σx 2 ) and Y N(µ Y, σy 2 ) are indeendent, then ax + by N(aµ X + bµ Y, a 2 σx 2 + b2 σy 2 ). (2) X Chi-Square with n degrees of freedom (X χ 2 (n)) if X Gamma(n/2, 1/2). mean: E(X) = n. variance: V (X) = 2n. mgf: M(t) = ( 1 1 2t )n/2, t < 1/2. 2

3 (3) X Exonential(λ). density: f(x) = λe λx, x 0. mean: E(X) = 1/λ. variance: V (X) = 1/λ 2. mgf: M(t) = ( λ λ t ), t < λ. P (X > x) = e λx, for x > 0. Note: There are 2 common conventions followed for the Exonential(λ): (i) λ is the arameter in the density as above and the mean is 1/λ, and (ii) λ is the mean and the density is f(x) = (1/λ)e x/λ ; the interretation in a articular roblem should be clear from context. (4) X Gamma(α, λ). density: f(x) = λα Γ(α) xα 1 e λx, x 0. mean: E(X) = α/λ. variance: V (X) = α/λ 2. mgf: M(t) = ( λ λ t )α, t < λ. Note : Γ(α) = 0 t α 1 e t dt. Γ(n) = (n 1)! when n is a ositive integer. When X 1, X 2,..., X n Exonential(λ) are indeendent, X 1 + X X n Gamma(n, λ). In articular, Exonential(λ) = Gamma(1, λ). (5) X U[a, b] density: f(x) = 1/(b a) for a < x < b. mean: E(X) = (a + b)/2. variance: V (X) = (b a) 2 /12. mgf: M(t) = ebt e at (b a)t, < t <. 3

4 If [c,d] [a,b], then P (c X d) is Length[c,d]/Length[a,b]. II. Random variables and their distributions; discrete random variables. A discrete rv X takes on a finite or countable number of values. Probabilities are comuted using a frequency function (k) = P (X = k); this is also called a robability density function (df) or robability mass function. P (a < X < b) = k (a,b) (k). Given a function g, the exected value of g(x) = E(g(X)) = k g(k)(k); in articular, the moment generating function of X is M X (t) = E(e Xt ) = k e kt (k). The standard class of discrete distributions. (1) X Bernoulli() frequency function: (1) =, (0) = q = 1. mean: E(X) =. variance: V (X) = q. mgf: M(t) = (q + e t ), < t <. (2) X Binomial(n, ) frequency function: (k) = mean: E(X) = n. variance: V (X) = nq. ( ) n k k q n k, k = 0, 1,..., n. 4

5 mgf: M(t) = (q + e t ) n, < t <. If X is the number of successes in n indeendent Bernoulli trials, then X Binomial(n, ). (3) X Geometric(). There are two definitions for the Geometric() distribution. (I) X is the number of failures required to see the first success in a sequence of Bernoulli trials and (II) X is the number of trials required to see the first success in a sequence of Bernoulli trials. If X and Y reresent have these resective distributions, then Y = X + 1. We give results searately for the two definitions. Case I. frequency function: (k) = q k, k = 0, 1,..., where q = 1. mean: E(X) = q/. variance: V (X) = q/ 2. mgf: M(t) = Case II. 1 qe t, < t < ln(1/q). frequency function: (k) = q k 1, k = 1, 2,..., where q = 1. mean: E(X) = 1/. variance: V (X) = q/ 2. mgf: M(t) = et 1 qe, < t < ln(1/q). t (4) X Negative Binomial(r, ). Again, there are two definitions for the Negative Binomial(r, ) distribution. (I) X is the number of failures before the rth success in a sequence of Bernoulli trials and (II) Y is the number of trials required to see the rth success in a sequence of Bernoulli trials. If X and Y have these resective distributions, then Y = X + r. We give results searately for the two definitions. Case I. 5

6 q = 1. frequency function: X (k) = ( ) k + r 1 r 1 r q k, k = 0, 1,..., where mean: E(X) = rq/. variance: V (X) = rq/ 2. mgf: M(t) = ( 1 qe ) r, < t < ln(1/q). t Case II. frequency function: Y (k) = q = 1. ( ) k 1 r 1 r q k r, k = r, r + 1,..., where mean: E(X) = r/. variance: V (X) = rq/ 2. mgf: M(t) = ( et 1 qe t ) r, < t < ln(1/q). (5) X Poisson(λ) frequency function: (k) = e λ λ k k!, k = 0, 1,.... mean: E(X) = λ. variance: V (X) = λ. mgf: M(t) = e λ(et 1), < t <. III. General Proerties. E(X) E(aX + by ) = ae(x) + be(y ) for any random variables X and Y. E(XY ) = E(X)E(Y ) if X and Y are indeendent. V (X) and Cov(X, Y ) 6

7 V (X) = E[(X µ) 2 ] = E(X 2 ) µ 2. V (ax + b) = a 2 V (X). V (X + Y ) = V (X) + V (Y ) + 2Cov(X, Y ) for any random variables X and Y. V (X + Y ) = V (X) + V (Y ) if X and Y are indeendent. Cov(X, Y ) = E[(X µ X )(Y µ Y )] = E(XY ) E(X)E(Y ). Cov(X, Y ) = 0 if X and Y are indeendent. Cov[X, X) = V (X). Cov(aX, by ) = abcov(x, Y ). Cov(X +Y, U +V ) = Cov(X, U)+Cov(X, V )+Cov(Y, U)+Cov(Y, V ). Samling If {X i } are n indeendent, identically distributed random variables with E(X i ) = µ and V (X i ) = σ 2 and X = 1 n n i=1 X i is the samle mean, then: E( X) = µ and V ( X) = σ 2 /n. Central limit theorem If {X i } are n indeendent, identically distributed random variables with E(X i ) = µ and V (X i ) = σ 2, then n i=1 X i aroximately N(nµ, nσ 2 ) as n. X aroximately N(µ, σ 2 /n) as n. Joint and conditional distributions If X and Y have joint df f X,Y (x, y), then f X (x) = f X,Y (x, y)dy or all y f X,Y (x, y); f Y (y) = f X,Y (x, y)dx or all x f X,Y (x, y); f X Y =y (x) = f X,Y (x, y)/f Y (y); 7

8 f Y X=x (y) = f X,Y (x, y)/f X (x); f X,Y (x, y) = f X (x)f Y (y) if and only if X and Y are indeendent. X max, X min If {X i } are n indeendent, identically distributed random variables with df f X (x), then f Xmax (x) = nf X (x)[f X (x)] n 1 f Xmin (x) = nf X (x)[1 F X (x)] n 1 8