Properties of Summation Operator

Transcription

1 Econ 325 Section 003/004 Notes on Variance, Covariance, and Summation Operator By Hiro Kasahara Properties of Summation Operator For a sequence of the values {x 1, x 2,..., x n, we write the sum of x 1, x 2,..., x n 1, and x n using the summation operator as x 1 + x x n x i. (1 Given a constant c, cx i cx 1 + cx cx n c (x 1 + x x n c x i. (2 For example, consider the case that n 2 with the values of {x 1, x 2 given by x 1 0 and x 2 1. Suppose that c 4. Then, 2 4 x i ( x i. In the special case of x 1 x 2... x n 1, we have n cx i n c 1 c n 1 c ( nc. Consider another sequence {y 1, y 2,..., y m in addition to {x 1, x 2,..., x n. Then, we may consider double summations over possible values of x s and y s. For example, consider the case of n m 2. Then, 2 2 j1 x iy j is equal to x 1 y 1 + x 1 y 2 + x 2 y 1 + x 2 y 2 because x 1 y 1 + x 1 y 2 + x 2 y 1 + x 2 y 2 x 1 (y 1 + y 2 + x 2 (y 1 + y 2 (by factorization x i (y 1 + y 2 (by def. of the summation operator by setting c (y 1 + y 2 in (2 x i j1 y j (because y 1 + y 2 2 j1 y j x i y j (because x 2 i j1 y j x i (y 1 + y 2 (x i y 1 + x i y 2 2 j1 x iy j j1 j1 x i y j. Note that 2 2 j1 x iy j 2 2 j1 x iy j. In general case of {x 1, x 2,..., x n and {y 1, y 2,..., y m, we have n m j1 x iy j m n j1 x iy j. Note that 2 j1 x iy j x i 2 j1 y j using (2 because x i is treated as a constant in the summation operator over j s. Hence, we can write x i y j j1 x i j1 y j y j j1 x i. 1

2 In general, we have j1 x i y j m x i j1 y j y j j1 x i. (3 That is, when we have double summations, we can take x i s out of the summation over j s. Similarly, we can take y j s out of the summation over i s. Expectation, Variance, and Covariance Let X and Y be two discrete random variables. The set of possible values for X is {x 1,..., x n ; and the set of possible values for Y is {y 1,..., y m. The joint probability function is given by P (X x i, Y y j, i 1,... n; j 1,..., m. The marginal probability function of X is p X i P (X x i and the marginal probability function of Y is j1, i 1,... n, p Y j P (Y y j, j 1,... m. 1. If c is a constant, then E[cX] ce[x]. (4 Proof: By definition of the expected value of cx, E[cX] (cx i p X i (by def. of the expected value cx 1 p X 1 + cx 2 p X 2 + cx 3 p X cx n 1 p X n 1 + cx n p X n (by def. of the summation operator c (x 1 p X 1 + x 2 p X 2 + x 3 p X x n 1 p X n 1 + x n p X n (because c is a common factor ( c x i p X i (by def. of the summation operator c E[X] (by def. of the expected value of X ce[x]. 2. E[X + Y ] E[X] + E[Y ]. (5 2

3 Proof: E(X + Y j1 j1 j1 (x i + y j (x i x i + x i j1 + y j j1 + y j (6 ( y j j1 because we can take x i out of m j1 because x i does not depend on j s x i p X i + y j p Y j j1 because p X i m j1 px,y and p Y j n px,y E(X + E(Y Equation (6: To understand n m j1 (x i consider the case of n m 2. Then, j1 (x i + y j +y j n (7 m j1 x i + n m j1 y j, (x y (x y (x y (x y 2 22 (x x x x (y y y y 2 j1 x i + j1 y j. 22 Equation (7: This is a generalization of (3. To understand n m j1 x i ( m, consider the case of n m 2. Then, j1 px,y n x i j1 x i x x x x 2 22 x 1 ( x 2( 21 + x i ( i1 + i2 x i (. j1 22 Similarly, we may show that 2 2 j1 y j 2 j1 y j ( 2 px,y. 3. If a and b are constants, then E[a + bx] a + be[x]. 3

4 Proof: E(a + bx a (a + bx i p X i (ap X i + bx i p X i ap X i + p X i + b bx i p X i (8 x i p X i, (by using (2 a 1 + be(x, where n px i n P (X x i 1 and n x ip X i E(X a + be(x. Equation (8: This is similar to (6. To understand n (apx i + bx i p X i n apx i + n bx ip X i, consider the case of n 2. Then, 2 (apx i + bx i p X i (apx 1 + bx 1p X 1 + (apx 2 + bx 2 p X 2 (apx 1 + apx 2 + (bx 1p X 1 + bx 2p X 2 2 apx i + 2 bx ip X i. 4. If c is a constant, then Cov (X, c 0. Proof: According to the definition of covariance, Cov(X, c E[(X E(X(c E(c]. Since the expectation of a constant is itself, i.e., E(c c, Cov(X, c E[(X E(X(c c] E[(X E(X 0] E[0] 0 p X i Cov (X, X V ar (X. 4

5 Proof: According to the definition of covariance, we can expand Cov(X, X as follows: Cov(X, X E[(X E(X(X E(X] [x i E(X][x i E(X] P (X x i, where E(X [x i E(X][x i E(X] p X i [x i E(X] 2 p X i E[(X E(X 2 ] (by def. of the expected value V ar(x. x i p X i 6. Cov (X, Y Cov (Y, X. Proof: According to the definition of covariance, we can expand Cov(X, Y as follows: Cov(X, Y E[(X E(X(Y E(Y ] [x i E(X][y j E(Y ], where E(X x i p X i and E(Y j1 j1 [y j E(Y ][x i E(X] E[(Y E(Y (X E(X] (by def. of the expected value Cov(Y, X. (by def. of the covariance 7. Cov (a 1 + b 1 X, a 2 + b 2 Y b 1 b 2 Cov (X, Y, where a 1, a 2, b 1, and b 2 are some constants. y j p Y j j1 Proof: Using E(a 1 + b 1 X a 1 + b 1 E(X and E(a 2 + b 2 Y a 2 + b 2 E(Y, we can expand Cov (a 1 + b 1 X, a 2 + b 2 Y as follows: Cov(X, Y E[(a 1 + b 1 X E(a 1 + b 1 X(a 2 + b 2 Y E(a 2 + b 2 Y ] E[(a 1 + b 1 X (a 1 + b 1 E(X(a 2 + b 2 Y (a 2 + b 2 E(Y ] E[(a 1 a 1 + b 1 X b 1 E(X(a 2 a 2 + b 2 Y b 2 E(Y ] E[(b 1 X b 1 E(X(b 2 Y b 2 E(Y ] E[b 1 (X E(X b 2 (Y E(Y ] E[b 1 b 2 (X E(X(Y E(Y ] b 1 b 2 (x i E(X(y j E(Y j1 b 1 b 2 j1 b 1 b 2 Cov(X, Y. [x i E(X][y j E(Y ] (by using (2 5

6 8. If X and Y are independent, then Cov (X, Y 0. Proof: If X and Y are independent, by definition of stochastic independence, P (X x i, Y y j P (X x i P (Y y j p X i py j for any i 1,..., n and j 1,..., m. Then, we may expand Cov (X, Y as follows. Cov(X, Y E[(X E(X(Y E(Y ] [x i E(X][y j E(Y ] P (X x i, Y y j j1 j1 [x i E(X][y j E(Y ]p X i p Y j because X and Y are independent {[x i E(X]p X i {[y j E(Y ]p Y j j1 [x i E(X]p X i [y j E(Y ]p Y j j1 because we can move [x i E(X]p X i outside of m j1 because [x i E(X]p X i does not depend on the index j s { [y j E(Y ]p Y j [x i E(X]p X i j1 { m because we can move j1 [y j E(Y ]p Y j outside of n { m because j1 [y j E(Y ]p Y j does not depend on the index i s { x i p X i E(Xp X i y j p Y j E(Y p Y j j1 j1 { m E(X E(Xp X i E(Y E(Y p Y j j1 by definition of E(X and E(Y { m E(X E(X p X i E(Y E(Y because we can move E(X and E(Y outside of n and m j1, respectively {E(X E(X 1 {E(Y E(Y Equations (9 and (10: This is similar to equations (3 and (7. Please consider the case of n m 2 and convince yourself that (9 and (10 hold. 9. V ar (X + Y V ar (X + V ar (Y + 2Cov (X, Y. j1 p Y j (9 (10 6

7 Proof: By the definition of variance, V ar(x + Y E[(X + Y E(X + Y 2 ]. Then, V ar(x + Y E[(X + Y E(X + Y 2 ] E[((X E(X + (Y E(Y 2 ] E[(X E(X 2 + (Y E(Y 2 + 2(X E(X(Y E(Y ] because for any a and b, (a + b 2 a 2 + b 2 + 2ab E[(X E(X 2 ] + E[(Y E(Y 2 ] + 2E[(X E(X(Y E(Y ] (by using (5 V ar(x + V ar(y + 2Cov(X, Y by definition of variance and covariance 10. V ar (X Y V ar (X + V ar (Y 2Cov (X, Y. Proof: The proof of V ar (X Y V ar (X+V ar (Y 2Cov (X, Y is similar to the proof of V ar (X + Y V ar (X + V ar (Y + 2Cov (X, Y. First, we may show that E(X Y E(X E(Y. Then, V ar(x Y E[(X Y E(X Y 2 ] E[((X E(X (Y E(Y 2 ] E[(X E(X 2 + (Y E(Y 2 2(X E(X(Y E(Y ] E[(X E(X 2 ] + E[(Y E(Y 2 ] 2E[(X E(X(Y E(Y ] (by using (5 V ar(x + V ar(y 2Cov(X, Y 11. Define W (X E(X/ V ar(x and Z (Y E(Y / V ar(y. Show that Cov(W, Z Corr(X, Z. 7

8 Proof: Expanding Cov(W, Z, we have Cov(W, Z E[(W E(W (Z E(Z] E[W Z] (because E[W ] E[Z] 0 { X E(X E Y E(Y V ar(x V ar(y by definition of W and Z { 1 1 E [X E(X]E[Y E(Y ] V ar(x V ar(y 1 1 E {[X E(X]E[Y E(Y ] (by using (2 and (4 V ar(x V ar(y 1 because both and 1 are constant V ar(x V ar(y E {[X E(X]E[Y E(Y ] V ar(x V ar(y Cov(X, Y V ar(x V ar(y (by definition of covariance Corr(X, Y (by definition of correlation coefficient 12. Let {x i : i 1,..., n and {y i : i 1,..., n be two sequences. Define the averages x 1 n ȳ 1 n x i, y i. (a n (x i x 0. Proof: (x i x x i x x i n x because n x x + x x n x n n x i n x n because n x i n n n x i n n x n x 0. because x n x i n n x i n (b n (x i x 2 n x i (x i x. 8

9 Proof: We use the result of 2.(a above. (x i x 2 (x i x (x i x x i (x i x x (x i x x i (x i x x (x i x because x is constant and does not depend on i s because n (x i x 0. as shown above x i (x i x. x i (x i x x 0 (c n (x i x (y i ȳ n y i (x i x n x i (y i ȳ. Proof: The proof is similar to the proof of 2.(b above. Also, (x i x (y i ȳ (x i x (y i ȳ (x i x y i (x i x ȳ (x i x y i ȳ (x i x (x i x y i ȳ 0 y i (x i x. x i (y i ȳ x (y i ȳ x i (y i ȳ x (y i ȳ x i (y i ȳ x 0 x i (y i ȳ. 9