Properties of Summation Operator

Similar documents
Lecture 4: Proofs for Expectation, Variance, and Covariance Formula

More than one variable

Problem Set #5. Econ 103. Solution: By the complement rule p(0) = 1 p q. q, 1 x 0 < 0 1 p, 0 x 0 < 1. Solution: E[X] = 1 q + 0 (1 p q) + p 1 = p q

2 (Statistics) Random variables

Math 510 midterm 3 answers

Lecture 25: Review. Statistics 104. April 23, Colin Rundel

Business Statistics 41000: Homework # 2 Solutions

Econ 371 Problem Set #1 Answer Sheet

Covariance and Correlation

Properties of Random Variables

Expectation and Variance

Probability Theory and Statistics. Peter Jochumzen

UNIVERSITY OF TORONTO SCARBOROUGH Department of Computer and Mathematical Sciences Midterm Test, March 2014

LIST OF FORMULAS FOR STK1100 AND STK1110

Homework 5 Solutions

STOR Lecture 16. Properties of Expectation - I

Lecture 13 (Part 2): Deviation from mean: Markov s inequality, variance and its properties, Chebyshev s inequality

Political Science 6000: Beginnings and Mini Math Boot Camp

Topic 2: Probability & Distributions. Road Map Probability & Distributions. ECO220Y5Y: Quantitative Methods in Economics. Dr.

Random Variables and Expectations

EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)

ECON 5350 Class Notes Review of Probability and Distribution Theory

Lecture Note 1: Probability Theory and Statistics

Probability. Hosung Sohn

STA 2201/442 Assignment 2

Final Exam. Economics 835: Econometrics. Fall 2010

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.

MA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2

Econ 2120: Section 2

Part IA Probability. Theorems. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

f X, Y (x, y)dx (x), where f(x,y) is the joint pdf of X and Y. (x) dx

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Problem Set 8 Fall 2007

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n

(4) Suppose X is a random variable such that E(X) = 5 and Var(X) = 16. Find

ECON Fundamentals of Probability

Problem Y is an exponential random variable with parameter λ = 0.2. Given the event A = {Y < 2},

Multivariate Random Variable

Functions of two random variables. Conditional pairs

Bivariate distributions

Preliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com

01 Probability Theory and Statistics Review

Exam P Review Sheet. for a > 0. ln(a) i=0 ari = a. (1 r) 2. (Note that the A i s form a partition)

Lecture 2: Review of Probability

MATH Notebook 4 Fall 2018/2019

Chapter 4 continued. Chapter 4 sections

MAS113 Introduction to Probability and Statistics. Proofs of theorems

Lecture 16 : Independence, Covariance and Correlation of Discrete Random Variables

NANYANG TECHNOLOGICAL UNIVERSITY SEMESTER II EXAMINATION MAS451/MTH451 Time Series Analysis TIME ALLOWED: 2 HOURS

HW5 Solutions. (a) (8 pts.) Show that if two random variables X and Y are independent, then E[XY ] = E[X]E[Y ] xy p X,Y (x, y)

Chp 4. Expectation and Variance

Intelligent Data Analysis. Principal Component Analysis. School of Computer Science University of Birmingham

Chapter 2. Some Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables

Review of Probability Theory

EXAMINATIONS OF THE HONG KONG STATISTICAL SOCIETY GRADUATE DIPLOMA, Statistical Theory and Methods I. Time Allowed: Three Hours

The Multivariate Normal Distribution 1

STA 256: Statistics and Probability I

Random Vectors 1. STA442/2101 Fall See last slide for copyright information. 1 / 30

Lecture 22: Variance and Covariance

Jointly Distributed Random Variables

ENGG2430A-Homework 2

Covariance. Lecture 20: Covariance / Correlation & General Bivariate Normal. Covariance, cont. Properties of Covariance

Lecture 3. Probability - Part 2. Luigi Freda. ALCOR Lab DIAG University of Rome La Sapienza. October 19, 2016

Introduction to Machine Learning

Regression and Covariance

MATH 38061/MATH48061/MATH68061: MULTIVARIATE STATISTICS Solutions to Problems on Random Vectors and Random Sampling. 1+ x2 +y 2 ) (n+2)/2

Joint probability distributions: Discrete Variables. Two Discrete Random Variables. Example 1. Example 1

1 Basic continuous random variable problems

CS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R

CHAPTER 5. Jointly Probability Mass Function for Two Discrete Distributed Random Variables:

Introduction to Computational Finance and Financial Econometrics Probability Review - Part 2

Final Review: Problem Solving Strategies for Stat 430

STT 441 Final Exam Fall 2013

ECE Lecture #9 Part 2 Overview

STAT/MATH 395 PROBABILITY II

Statistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v }

Practice Examination # 3

Raquel Prado. Name: Department of Applied Mathematics and Statistics AMS-131. Spring 2010

Ch. 5 Joint Probability Distributions and Random Samples

An introduction to multivariate data

Notes on Random Processes

UC Berkeley Department of Electrical Engineering and Computer Science. EE 126: Probablity and Random Processes. Problem Set 9 Fall 2007

Statistics Assignment 2 HET551 Design and Development Project 1

Machine learning - HT Maximum Likelihood

Brandon C. Kelly (Harvard Smithsonian Center for Astrophysics)

Problem Solving. Correlation and Covariance. Yi Lu. Problem Solving. Yi Lu ECE 313 2/51

MATHEMATICS 154, SPRING 2009 PROBABILITY THEORY Outline #11 (Tail-Sum Theorem, Conditional distribution and expectation)

Variances and covariances

Random Variables. Lecture 6: E(X ), Var(X ), & Cov(X, Y ) Random Variables - Vocabulary. Random Variables, cont.

UNIT-2: MULTIPLE RANDOM VARIABLES & OPERATIONS

Multivariate probability distributions and linear regression

Elements of Probability Theory

Some Concepts of Probability (Review) Volker Tresp Summer 2018

EECS 126 Probability and Random Processes University of California, Berkeley: Spring 2015 Abhay Parekh February 17, 2015.

ECE302 Exam 2 Version A April 21, You must show ALL of your work for full credit. Please leave fractions as fractions, but simplify them, etc.

Continuous Random Variables

Basic Concepts in Matrix Algebra

I L L I N O I S UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN

Simple Linear Regression Estimation and Properties

Introduction to Machine Learning

Transcription:

Econ 325 Section 003/004 Notes on Variance, Covariance, and Summation Operator By Hiro Kasahara Properties of Summation Operator For a sequence of the values {x 1, x 2,..., x n, we write the sum of x 1, x 2,..., x n 1, and x n using the summation operator as x 1 + x 2 +... + x n x i. (1 Given a constant c, cx i cx 1 + cx 2 +... + cx n c (x 1 + x 2 +... + x n c x i. (2 For example, consider the case that n 2 with the values of {x 1, x 2 given by x 1 0 and x 2 1. Suppose that c 4. Then, 2 4 x i 4 0 + 4 1 4 (0 + 1 4 2 x i. In the special case of x 1 x 2... x n 1, we have n cx i n c 1 c n 1 c (1 + 1 +... + 1 nc. Consider another sequence {y 1, y 2,..., y m in addition to {x 1, x 2,..., x n. Then, we may consider double summations over possible values of x s and y s. For example, consider the case of n m 2. Then, 2 2 j1 x iy j is equal to x 1 y 1 + x 1 y 2 + x 2 y 1 + x 2 y 2 because x 1 y 1 + x 1 y 2 + x 2 y 1 + x 2 y 2 x 1 (y 1 + y 2 + x 2 (y 1 + y 2 (by factorization x i (y 1 + y 2 (by def. of the summation operator by setting c (y 1 + y 2 in (2 x i j1 y j (because y 1 + y 2 2 j1 y j x i y j (because x 2 i j1 y j x i (y 1 + y 2 (x i y 1 + x i y 2 2 j1 x iy j j1 j1 x i y j. Note that 2 2 j1 x iy j 2 2 j1 x iy j. In general case of {x 1, x 2,..., x n and {y 1, y 2,..., y m, we have n m j1 x iy j m n j1 x iy j. Note that 2 j1 x iy j x i 2 j1 y j using (2 because x i is treated as a constant in the summation operator over j s. Hence, we can write x i y j j1 x i j1 y j y j j1 x i. 1

In general, we have j1 x i y j m x i j1 y j y j j1 x i. (3 That is, when we have double summations, we can take x i s out of the summation over j s. Similarly, we can take y j s out of the summation over i s. Expectation, Variance, and Covariance Let X and Y be two discrete random variables. The set of possible values for X is {x 1,..., x n ; and the set of possible values for Y is {y 1,..., y m. The joint probability function is given by P (X x i, Y y j, i 1,... n; j 1,..., m. The marginal probability function of X is p X i P (X x i and the marginal probability function of Y is j1, i 1,... n, p Y j P (Y y j, j 1,... m. 1. If c is a constant, then E[cX] ce[x]. (4 Proof: By definition of the expected value of cx, E[cX] (cx i p X i (by def. of the expected value cx 1 p X 1 + cx 2 p X 2 + cx 3 p X 3 +... + cx n 1 p X n 1 + cx n p X n (by def. of the summation operator c (x 1 p X 1 + x 2 p X 2 + x 3 p X 3 +... + x n 1 p X n 1 + x n p X n (because c is a common factor ( c x i p X i (by def. of the summation operator c E[X] (by def. of the expected value of X ce[x]. 2. E[X + Y ] E[X] + E[Y ]. (5 2

Proof: E(X + Y j1 j1 j1 (x i + y j (x i x i + x i j1 + y j j1 + y j (6 ( y j j1 because we can take x i out of m j1 because x i does not depend on j s x i p X i + y j p Y j j1 because p X i m j1 px,y and p Y j n px,y E(X + E(Y Equation (6: To understand n m j1 (x i consider the case of n m 2. Then, j1 (x i + y j +y j n (7 m j1 x i + n m j1 y j, (x 1 11 + y 1 11 + (x 1 12 + y 2 12 + (x 2 21 + y 1 21 + (x 2 22 + y 2 22 (x 1 11 + x 1 12 + x 2 21 + x 2 22 + (y 1 11 + y 2 12 + y 1 21 + y 2 j1 x i + j1 y j. 22 Equation (7: This is a generalization of (3. To understand n m j1 x i ( m, consider the case of n m 2. Then, j1 px,y n x i j1 x i x 1 11 + x 1 12 + x 2 21 + x 2 22 x 1 ( 11 + 12 + x 2( 21 + x i ( i1 + i2 x i (. j1 22 Similarly, we may show that 2 2 j1 y j 2 j1 y j ( 2 px,y. 3. If a and b are constants, then E[a + bx] a + be[x]. 3

Proof: E(a + bx a (a + bx i p X i (ap X i + bx i p X i ap X i + p X i + b bx i p X i (8 x i p X i, (by using (2 a 1 + be(x, where n px i n P (X x i 1 and n x ip X i E(X a + be(x. Equation (8: This is similar to (6. To understand n (apx i + bx i p X i n apx i + n bx ip X i, consider the case of n 2. Then, 2 (apx i + bx i p X i (apx 1 + bx 1p X 1 + (apx 2 + bx 2 p X 2 (apx 1 + apx 2 + (bx 1p X 1 + bx 2p X 2 2 apx i + 2 bx ip X i. 4. If c is a constant, then Cov (X, c 0. Proof: According to the definition of covariance, Cov(X, c E[(X E(X(c E(c]. Since the expectation of a constant is itself, i.e., E(c c, Cov(X, c E[(X E(X(c c] E[(X E(X 0] E[0] 0 p X i 0 0 + 0 +... + 0 0 5. Cov (X, X V ar (X. 4

Proof: According to the definition of covariance, we can expand Cov(X, X as follows: Cov(X, X E[(X E(X(X E(X] [x i E(X][x i E(X] P (X x i, where E(X [x i E(X][x i E(X] p X i [x i E(X] 2 p X i E[(X E(X 2 ] (by def. of the expected value V ar(x. x i p X i 6. Cov (X, Y Cov (Y, X. Proof: According to the definition of covariance, we can expand Cov(X, Y as follows: Cov(X, Y E[(X E(X(Y E(Y ] [x i E(X][y j E(Y ], where E(X x i p X i and E(Y j1 j1 [y j E(Y ][x i E(X] E[(Y E(Y (X E(X] (by def. of the expected value Cov(Y, X. (by def. of the covariance 7. Cov (a 1 + b 1 X, a 2 + b 2 Y b 1 b 2 Cov (X, Y, where a 1, a 2, b 1, and b 2 are some constants. y j p Y j j1 Proof: Using E(a 1 + b 1 X a 1 + b 1 E(X and E(a 2 + b 2 Y a 2 + b 2 E(Y, we can expand Cov (a 1 + b 1 X, a 2 + b 2 Y as follows: Cov(X, Y E[(a 1 + b 1 X E(a 1 + b 1 X(a 2 + b 2 Y E(a 2 + b 2 Y ] E[(a 1 + b 1 X (a 1 + b 1 E(X(a 2 + b 2 Y (a 2 + b 2 E(Y ] E[(a 1 a 1 + b 1 X b 1 E(X(a 2 a 2 + b 2 Y b 2 E(Y ] E[(b 1 X b 1 E(X(b 2 Y b 2 E(Y ] E[b 1 (X E(X b 2 (Y E(Y ] E[b 1 b 2 (X E(X(Y E(Y ] b 1 b 2 (x i E(X(y j E(Y j1 b 1 b 2 j1 b 1 b 2 Cov(X, Y. [x i E(X][y j E(Y ] (by using (2 5

8. If X and Y are independent, then Cov (X, Y 0. Proof: If X and Y are independent, by definition of stochastic independence, P (X x i, Y y j P (X x i P (Y y j p X i py j for any i 1,..., n and j 1,..., m. Then, we may expand Cov (X, Y as follows. Cov(X, Y E[(X E(X(Y E(Y ] [x i E(X][y j E(Y ] P (X x i, Y y j j1 j1 [x i E(X][y j E(Y ]p X i p Y j because X and Y are independent {[x i E(X]p X i {[y j E(Y ]p Y j j1 [x i E(X]p X i [y j E(Y ]p Y j j1 because we can move [x i E(X]p X i outside of m j1 because [x i E(X]p X i does not depend on the index j s { [y j E(Y ]p Y j [x i E(X]p X i j1 { m because we can move j1 [y j E(Y ]p Y j outside of n { m because j1 [y j E(Y ]p Y j does not depend on the index i s { x i p X i E(Xp X i y j p Y j E(Y p Y j j1 j1 { m E(X E(Xp X i E(Y E(Y p Y j j1 by definition of E(X and E(Y { m E(X E(X p X i E(Y E(Y because we can move E(X and E(Y outside of n and m j1, respectively {E(X E(X 1 {E(Y E(Y 1 0 0 0. Equations (9 and (10: This is similar to equations (3 and (7. Please consider the case of n m 2 and convince yourself that (9 and (10 hold. 9. V ar (X + Y V ar (X + V ar (Y + 2Cov (X, Y. j1 p Y j (9 (10 6

Proof: By the definition of variance, V ar(x + Y E[(X + Y E(X + Y 2 ]. Then, V ar(x + Y E[(X + Y E(X + Y 2 ] E[((X E(X + (Y E(Y 2 ] E[(X E(X 2 + (Y E(Y 2 + 2(X E(X(Y E(Y ] because for any a and b, (a + b 2 a 2 + b 2 + 2ab E[(X E(X 2 ] + E[(Y E(Y 2 ] + 2E[(X E(X(Y E(Y ] (by using (5 V ar(x + V ar(y + 2Cov(X, Y by definition of variance and covariance 10. V ar (X Y V ar (X + V ar (Y 2Cov (X, Y. Proof: The proof of V ar (X Y V ar (X+V ar (Y 2Cov (X, Y is similar to the proof of V ar (X + Y V ar (X + V ar (Y + 2Cov (X, Y. First, we may show that E(X Y E(X E(Y. Then, V ar(x Y E[(X Y E(X Y 2 ] E[((X E(X (Y E(Y 2 ] E[(X E(X 2 + (Y E(Y 2 2(X E(X(Y E(Y ] E[(X E(X 2 ] + E[(Y E(Y 2 ] 2E[(X E(X(Y E(Y ] (by using (5 V ar(x + V ar(y 2Cov(X, Y 11. Define W (X E(X/ V ar(x and Z (Y E(Y / V ar(y. Show that Cov(W, Z Corr(X, Z. 7

Proof: Expanding Cov(W, Z, we have Cov(W, Z E[(W E(W (Z E(Z] E[W Z] (because E[W ] E[Z] 0 { X E(X E Y E(Y V ar(x V ar(y by definition of W and Z { 1 1 E [X E(X]E[Y E(Y ] V ar(x V ar(y 1 1 E {[X E(X]E[Y E(Y ] (by using (2 and (4 V ar(x V ar(y 1 because both and 1 are constant V ar(x V ar(y E {[X E(X]E[Y E(Y ] V ar(x V ar(y Cov(X, Y V ar(x V ar(y (by definition of covariance Corr(X, Y (by definition of correlation coefficient 12. Let {x i : i 1,..., n and {y i : i 1,..., n be two sequences. Define the averages x 1 n ȳ 1 n x i, y i. (a n (x i x 0. Proof: (x i x x i x x i n x because n x x + x +... + x n x n n x i n x n because n x i n n n x i n n x n x 0. because x n x i n n x i n (b n (x i x 2 n x i (x i x. 8

Proof: We use the result of 2.(a above. (x i x 2 (x i x (x i x x i (x i x x (x i x x i (x i x x (x i x because x is constant and does not depend on i s because n (x i x 0. as shown above x i (x i x. x i (x i x x 0 (c n (x i x (y i ȳ n y i (x i x n x i (y i ȳ. Proof: The proof is similar to the proof of 2.(b above. Also, (x i x (y i ȳ (x i x (y i ȳ (x i x y i (x i x ȳ (x i x y i ȳ (x i x (x i x y i ȳ 0 y i (x i x. x i (y i ȳ x (y i ȳ x i (y i ȳ x (y i ȳ x i (y i ȳ x 0 x i (y i ȳ. 9

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback