Joint p.d.f. and Independent Random Variables

Transcription

1 1 Joint p.d.f. and Independent Random Variables Let X and Y be two discrete r.v. s and let R be the corresponding space of X and Y. The joint p.d.f. of X = x and Y = y, denoted by f(x, y) = P(X = x, Y = y), has the following properties: (a) 0 f(x, y) 1, f(x, y) 0 for < x, y <. (b) (x,y) R f(x, y) = 1, f(x, y)dxdy = 1. (c) P[(X, Y ) A] = (x,y) A f(x, y) ( A f(x, y)), A R. The marginal p.d.f. of X is defined as f X (x) = y f(x, y) ( f(x, y)dy), x R x. The marginal p.d.f. of Y is defined as f Y (y) = x f(x, y) ( f(x, y)dx), y R y. The random variables X and Y are independent iff f(x, y) f X (x)f Y (y) for x R x, y R y. Example 1. f(x, y) = (x + y)/21, x = 1, 2, 3; y = 1, 2, then X and Y are not independent. Example 2. f(x, y) = (xy 2 )/30, x = 1, 2, 3; y = 1, 2, then X and Y are independent. The collection of n independent and identically distributed random variables X 1, X 2,..., X n, is called a random sample of size n from the common distribution, say, X j N(0, 1), 1 j n.

2 2 Sampling Distribution Theory The collection of n independent and identically distributed random variables X 1, X 2,..., X n, is called a random sample of size n from the common distribution, e.g., X j N(0, 1), 1 j n. Some functions of a random sample, called statistics, are of interest, for examples, mean and variance. Sampling distribution theory refers to the derivation of distributions for functions of a random sample. Theorem 1: Let X 1, X 2,..., X n be n independent r.v. s with respective means {µ i } and variances {σ 2 i }, then Y = n i=1 a i X i has mean µ Y = n i=1 a i µ i and variance σ 2 Y = a 2 i σ2 i, respectively. Theorem 2: Let X 1, X 2,...,X n be n independent r.v. s with respective moment-generating functions {M i (t)}, 1 i n, then the moment-generating function of Y = n i=1 a i X i is M Y (t) = n i=1 M i (a i t). Corollary: If X 1, X 2,...,X n are observations of a random sample from a distribution with moment-generating function M(t), then (a) M Y (t) = n i=1 M(t), where Y = n i=1 X i. (b) M X (t) = n i=1 M(t/n), where X = 1 n X i. Example 1: Let X i b(k, p) be a random sample of size n. Define Y = n i=1 X i, then M Y (t) = n i=1 (q + pe t ) k = (q + pe t ) kn. Example 2: Let X i Gamma(1, θ) be a random sample of size n. Define Y = n i=1 X i, then M Y (t) = n i=1 (1 θt) 1 = 1/(1 θt) n. Exercises:

3 3 Random Functions Associated with Normal Distributions In statistical applications, it is usually assumed that the population from which a sample is taken is N(µ, σ 2 ). Theorem: Let X 1, X 2,...,X n be a random sample of size n from N(µ, σ 2 ). Define X = X i, then X N(µ, σ 2 /n). 1 n Theorem: Let X j χ 2 (r j ), 1 j n. If X 1, X 2,...,X n are independent, then Y = X i χ 2 (r 1 + r r n ). Theorem: Let Z 1, Z 2,...,Z n be a random sample of size n from N(0, 1), then W = Z Z Z2 n χ2 (n). Corollary: Let {X is} be independent random variables from N(µ i, σ 2 i ), respectively, then W = n i=1 (X i µ i ) 2 /σ 2 i is χ 2 (n). Theorem: Let {X is} be observations of a random sample of size n from N(µ, σ 2 ). Define X = 1 X n i and S 2 = 1 (X n 1 i X) 2, then (a) X and S 2 are independent. (b) (n 1)S2 σ 2 = n i=1 (X i X) 2 /σ 2 χ 2 (n 1). Example 1: Let X 1, X 2, X 3, X 4 be a random sample of size 4 from the normal distribution N(76.4, 383). Then (a) U = n i=1 (X i 76.4) 2 /383 χ 2 (4), P(0.711 U 7.779) = =0.85. (b) W = n i=1 (X i X) 2 /383 χ 2 (3), P(0.352 W 6.251) = =0.85. Theorem: Let X i N(µ i, σ 2 i ), 1 i n, be independent. Define Y = n i=1 a i X i, then Y N( n i=1 a i µ i, n i=1 a 2 i σ2 i ). Exercises:

4 4 The Central Limit Theorem Theorem: Let X 1, X 2,...,X n be a random sample of size n from N(µ, σ 2 ). Define X = X i, then X N(µ, σ 2 /n). 1 n Theorem: Let X be the mean of a random sample X 1, X 2,..., X n of size n from a distribution with mean µ and variance σ 2. Define W n = (X µ)/(σ/ n). Then (a) W n = ( n i=1 X i nµ)/( nσ) (b) P(W n w) w 1 2π e z2 /2 dz = Φ(w). (c) W n N(0, 1) as n. Example 1: Let X 1, X 2,..., X n be a random sample of size n from a χ 2 (1). Define Y = X i. Then (a) Y χ 2 (n). (b) (Y n)/ 2n N(0, 1). Example 2: Let X 1, X 2,...,X n be a random sample of size n from a U(0, 1). Define Y = n i=1 X i. Then (Y 0.5n)/ n/12 N(0, 1). Example 3: Let X 1, X 2,...,X n be a random sample of size n from a Bernoulli(p). Define Y = n i=1 X i. Then (a) Y b(n, p). (b) (Y np)/ np(1 p) N(0, 1). Example 4: Let X 1, X 2,..., X n be a random sample of size n from an exponential distribution with mean θ. Define Y = n i=1 X i. Then (a) Y Gamma(n, θ). (b) (Y nθ)/ nθ 2 N(0, 1).

5 5 Approximations for Discrete Distributions Use the normal distribution to approximate probabilities for certain discrete-type distributions. Example 1: Let Y b(10, 1/2). Then P(3 Y < 6) = P(2.5 Y 5.5) = Φ(0.316) Φ( 1.581) = = ( by T able II). (1) Example 2: Let X 1, X 2,...,X n be a random sample of size n from a Poisson(λ). Define Y = n i=1 X i. Then (Y nλ)/ nλ N(0, 1). Example 3: Let Y Poisson(λ = 20). Then Exercises: P(16 < Y 21) = P(16.5 Y 21.5) = P[( )/ 20 (Y 20)/20 ( )/ 20)] = Φ(0.335) Φ( 0.783) = (2)

6 6 Limiting Moment-Generating Functions Theorem: If a sequence of moment-generating functions approaches a certain one, say, M(t), then the limit of the corresponding distribution must be the distribution corresponding to M(t). Example 1: Let Y b(50, 0.04) and let λ = np = = 2. Then P(Y 1) = P(Y 1) by a Poisson approximation. Exercises:

7 7 Box-Muller Transformation Box-Muller Transformation Let {X 1, X 2 } be a random sample from U(0,1), define Z 1 = 2lnX 1 cos(2πx 2 ), and Z 2 = 2lnX 1 sin(2πx 2 ), then The joint p.d.f. of X 1 and X 2 is f(x 1, x 2 ) = 1, 0 < x 1, x 2 < 1, The joint p.d.f. of Z 1 and Z 2 is g(z 1, z 2 ) = 1 2π exp[ (z 1 2 +z 2 2 )/2], < z 1, z 2 <.

8 8 The Beta, Student s t, and F Distributions Random variables whose space are intervals or a union of intervals are said to be of the continuous types. The p.d.f. of a r.v. X of continuous type is an integrable function f(x) satisfying (a) f(x) > 0, x R (b) R f(x)dx = 1 (c) The probability of the event X A is P(A) = A f(x)dx Beta f(x) = Γ(α+β) Γ(α)Γ(β) xα 1 (1 x) β 1, 0 < x < 1; α, β > 0 Student s t Let Z N(0, 1) and V χ 2 (r) be two independent random variables. Define T = Z/ V/r. Then T has a t-distribution with p.d.f. f(t) = Γ[(r+1)/2] πrγ(r/2) 1 (1+t 2 /r) (r+1)/2, < t < F-distribution Let U χ 2 (r 1 ) and V χ 2 (r 2 ) be two independent random variables. Define W = (U/r 1 )/(V/r 2 ). Then W has an F-distribution with p.d.f. f(w) = Γ[(r 1+r 2 )/2](r 1 /r 2 ) r 1 /2 Γ(r 1 /2)Γ(r 2 /2) x (r 1 /2) 1 (1+r 1 w/r 2 ) (r 1 +r 2 )/2, 0 < w < Exercises:

9 9 Expectation and Covariance Matrix Let X 1, X 2,...,X n be random variables such that the expectation, variance, and covariance are defined as follows. µ j = E(X j ), σ 2 j = V ar(x j) = E[(X j µ j ) 2 ] Cov(X i, X j ) = E[(X i µ i )(X j µ j )] = ρ ij σ i σ j Suppose that X = [X 1, X 2,...,X n ] t is a random vector, then the expected mean vector and covariance matrix of X is defined as E(X) = [µ 1, µ 2,...,µ n ] t = µ, Cov(X) = [E((X i µ i )(X j µ j ))]

10 10 Multivariate (Normal) Distributions (Gaussian) Normal Distribution: X N(u, σ 2 ) f X (x) = f(x) = 1 2πσ 2 exp (x u)2 /2σ 2 for < x < mean and variance : E(X) = u, V ar(x) = σ 2 (Gaussian) Normal Distribution: X N(u, C) f X (x) = f(x) = 1 (2π) d/2 [det(c)] 1/2e (x u)t C 1 (x u)/2 for x R d mean vector and covariance matrix : E(X) = u, Cov(X) = C Simulate X N(u, C) (1) C = LL t, where L is lower-. (2) Generate y N(0, I). (3) x = u + L y (4) Repeat Steps (2)and(3) M times. % Simulate N([1 3], [4,2; 2,5]) % n=30; X1=random( normal,0,1,n,1); X2=random( normal,0,1,n,1); Y=[ones(n,1), 3*ones(n,1)]+[X1,X2]*[2 1; 0, 2]; Yhat=mean(Y) % estimated mean vector Chat=cov(Y) % estimated covariance matrix % Z=[X1, X2];

11 11 Plot a 2D standard Gaussian Distribution x=-3.6:0.3:3.6; y=x ; X=ones(length(y),1)*x; Y=y*ones(1,length(x)); Z=exp(-(X.^2+Y.^2)/2+eps)/(2*pi); mesh(z); title( f(x,y)= (1/2\pi)*exp[-(x^2+y^2)/2.0] ) f(x,y)= (1/2π)*exp[ (x 2 +y 2 )/2.0]