Properties of Random Variables 1 Definitions A discrete random variable is defined by a probability distribution that lists each possible outcome and the probability of obtaining that outcome If the random variable x is discrete with n possible outcomes, here is its probablilty distribution Outcome Probability x 1 p 1 x 2 p 2 x n p n The commonly used discrete Binomial random variable shows the number of successes for n independent random trials when the probability of a single success is p Here is the Binomial probability distribution f for obtaining k successes out of n trials: ( n f(k n, p p k (1 p n k k Example: What is the probability distribution for the number of successes for shooting 3 basketball free throws when you are a 80% free throw shooter? Answer: Outcome Probability 0 0008 1 0096 2 0384 3 0512 A continuous random variable is defined by its probability density function To find the probability that a x b for the continuous random variable x, find the area under its probability density p: P (a x b b a p(x dx Note that for any value a, P (x a 0 for a continuous random variable x 1
The most popular continuous distribution is the normal distribution, which is discussed in Section 4 2 Expected Value The expected value of a random variable is the weighted average of its possible values Each value is weighted by the probability that the outcome occurs If a discrete random variable x has outcomes x 1, x 2,, x n, with probabilities p 1, p 2,, p n, respectively, the expected value of x is E(x x i p i If a continuous random variable has the probability density p, its expected value is defined by E(x xp(x dx In the justification of the properties of random variables later in this section, we assume continuous random variables The justifications for discrete random variables are obtained by replacing the integrals with summations For the remainder of this section, the letters x and y represent random variables and the letter c represents a constant Property 1: E(x + y E(x + E(y Let p(x and q(y be the probability density functions of the random variables x and y, respectively Also let r(x, y be the joint density of x and y considered together We have p(x r(x, y dy and q(y r(x, y dx p(x and q(x are called the marginal densities of r(x, y 2
Then E(x + y x (x + yr(x, y dy dx x r(x, y dy dx + x p(x dx + E(x + E(y r(x, y dy dx + y q(x dy y y r(x, ydy dx r(x, ydx dy Property 2: E(c c c can be thought of as a discrete random variable that takes on the value c with probability 1 E(c is then simply the value of c times 1 Property 3: E(cx c E(x variable x Then Let p(x be the probability density function of the random E(cx cx p(x dx c x p(x dx c E(x 2 Independence Two random variables x and y are independent if their joint density r(x, y factors into two one-dimensional densities: r(x, y p(xq(y Property 4: If x and y are independent, E(xy E(x E(y 3
E(xy x p(x xy r(x, y dy dx xy p(xq(y dy dx x p(x dx E(x E(y y q(y dy dx y q(y dy 3 Variance The variance of a random variable x is defined as Var(x E(x E(x 2 Property 5: Var(x E(x 2 E(x 2 Var(x E(x E(x 2 E(x 2 2 E(x E(x + E(x 2 E(x 2 E(x 2 Property 6: If x and y are independent, Var(x + y Var(x + Var(y Var(x + y E((x + y 2 E(x + y 2 E(x 2 + 2xy + y 2 (E(x + E(y 2 E(x 2 + 2 E(xy + E(y 2 (E(x 2 + 2 E(x E(y + E(y 2 E(x 2 + 2 E(x E(y + E(y 2 (E(x 2 + 2 E(x E(y + E(y 2 E(x 2 E(x 2 + 2 E(x E(y 2 E(x E(y + E(y 2 E(y 2 Var(x + Var(y Property 7: Var(cx c 2 Var(x 4
Var(cx E((cx 2 E(cx 2 E(c 2 x 2 (c E(x 2 c 2 E(x 2 c 2 E(x 2 c 2 (E(x 2 E(x 2 c 2 Var(x 4 The Normal Density The normal distribution with mean µ and variance σ 2 is defined by its density ( φ(x 1 x µ 2 σ 2π e 2σ (1 The expression x N(µ, σ 2 (2 means that the random variable x has a normal distribution with mean µ and variance σ 2 Ask the instructor for references if you are interested in a justification of (1 implies (2 A modern justification that E(x µ and Var(x σ 2 for a normal random variable x uses the moment generating function, which is defined as M x (t E(e tx For a N(µ, σ 2 random variable, M x (t e tµ+ 1 2 σ2 t 2 5 The Unbiased Property of the Sample Mean A statistic t is unbiased for a parameter θ if E(t θ Property 8: x N(µ, σ 2 implies x is unbiased for µ Suppose that x i N(µ, σ 2 for each observation x i in the random sample x 1,, x n Then E(x i µ and Var(x i σ 2 Furthermore, ( ( 1 E( x E x i 1 n n E x i 1 E(x i 1 µ n n n n µ µ 5
Property 9: If the observations in the random sample x 1, x 2,, x n are independent and x i N(µ, σ 2 for each observation x i, Var( x Var ( 1 n 1 n 2 Var( x σ2 n ( x i 1 n Var x 2 i 1 Var(x n 2 i σ 2 n n 2 σ2 σ2 n 7 The Unbiased Property of the Sample Variance The sample standard deviation s is defined as s 1 n 1 (x i x 2 The n 1 in the denominator is chosen so that s 2 x will be unbiased for σ 2 x2, as we see in this property: Property 10: s 2 x is unbiased for σ 2 x Assume that we have a random sample x 1,, x n, where all observations are independent, E(x i µ and Var(x i σ 2 for all observations Since Var(x i E(x 2 i E(x i 2 by Property 4, σ 2 E(x i 2 µ 2, so E(x 2 i σ 2 + µ 2 Also, if x i x j, using the fact that x i and x j are independent, by Property 3, E(x i x j E(x i E(x j µµ µ 2 We then 6
have ( E(SSE E (x i x 2 Therefore, ( E x i 1 n ( ( E x 2 i 2 n ( E x 2 i 2 n E(s x E E(x 2 i 2 n E(x 2 i 2 n E(x 2 i 1 n j1 2 x j x i x j + 1 n 2 j1 x i x j + 1 n 2 j1 x j x k k1 E(x i x j + 1 n 2 j1 E(x i x j + n n 2 j1 E(x i x j j1 j1 k1 x j x k j1 k1 n(σ 2 + µ 2 1 n [n(σ2 + µ 2 + n(n 1µ 2 ] nσ 2 + nµ 2 σ 2 + µ 2 + (n 1µ 2 ] (n 1σ 2 j1 k1 E(x j x k E(x j x k ( SSE 1 n 1 n 1 E (SSE 1 n 1 (n 1σ2 σ 2 Random Vectors Property 11: E(v + w E(v + E(w 7
v 1 + w 1 E(v + w E v n + w n E(v 1 + E(w 1 E(v n + E(w n E(v + E(w E(v 1 + w 1 E(v n + w n E(v 1 E(v n + E(w 1 E(w n Property 12: If b is a constant vector, E(b b E(b E b 1 b n E(b 1 E(b n b 1 b n b Property 13: E(Av A E(w 8
Let A be an m n matrix and v be an n 1 vector a 11 a 1n v 1 E(Av E a m1 a mn v n ( a 1i v i E a 1i v i E ( a mi v i E a mi v i A E(v a 1i E(v i a 1i a 1n E(v 1 a n1 a nn E(v n a mi E(v i Property 14: Cov(v + b Cov(v Cov(v + b Var(v 1 + b Cov(v n + b, v 1 + b Cov(v 1 + b, v n + b Var(v n + b Var(v 1 Cov(v n, v 1 Cov(v 1, v n Var(v n Cov(v Property 15: Cov(Av A Cov(vA T 9
Let Cov(v i, v j σ ij Then a 11 a 1n v 1 Cov(Av Cov a n1 a nn v n n a 1iv i Cov n a niv i ( ( Var a 1i v i Cov a 1i v i, a ni v i, ( ( Cov a 1i v i, a ni v i, Var a nn v i j1 j1 a 1i a 1j σ ij a 1i a nj σ ij j1 a ni a 1j σ ij a ni a nj σ ij a 11 a 1n a n1 a nn j1 a 1j σ 1j a nj σ 1j a 1j σ nj a nj σ nj a 11 a 1n σ 11 σ 1n a 11 a n1 a n1 a nn σ n1 σ nn a 1n a nn A Cov(vA T 10