Averages of Random Variables. Expected Value. Die Tossing Example. Averages of Random Variables

Transcription

1 Averages of Random Variables Suppose that a random variable U can tae on any one of L random values, say u 1, u 2,...u L. Imagine that we mae n independent observations of U and that the value u is observed n times, 1, 2,...,L. Of course, n 1 + n n L n. The emperical average can be computed by Averages of Random Variables Lecture 5 Ū 1 L L n n u n n u 1 1 The concept of statistical averages extends from this simple concept Spring 2002 Lecture 5 1 Expected Value The expected value of a discrete random variable U is defined by Die Tossing Example E[U] U u P (u ) Note how this definition compares with the empirical average. The empirical average Ū is a random variable. It will have a different value with each set of sample values. The expected value is a single number. It is a parameter of the distribution. How do you thin the expected value and the random variable Ū relate? Lecture 5 2 ss

2 Function of a Discrete Random Variable Example of V g(u) Let U be a discrete random variable and let V g(u) Suppose that we want to compute the average value of V rather than of U. Given that U cantaeonthevaluesu with probabilities P [U u ], 1, 2,...,L, we can compute both the values and probabilities that can be taen on by V. Let v 1,v 2,...,v r be the set of values that can be assumed by V and let P [V v j ],j1, 2,...r be the corresponding probabilities. Then r E[V ] v j P [V v j ] j1 We need to determine the values and the probabilities. E[V ] v 1 P [U u 1 ]+v 2 (P [U u 5 ]+P[U u 6 ]) +v 3 (P [U u 2 ]+P[U u 4 ]+P[U u 7 ]) + v 4 P [U u 3 ] Lecture 5 4 Lecture 5 5 Expected Value of V g(u) By maing use of the relationship v g(u) we can now rewrite this in a form that associates g(u ) with P [U u ]. The result is the nice compact equation L E[V ] g(u )P [U u ] 1 If you were given machine to generate samples of U, how would you go about estimating E[V ]. Continuous Random Variables The expected value of a continuous random variable U is E[U] uf U(u)du The expected value of a function of a random variable V g(u) is E[V ] g(u)f U(u)du It is not necessary to compute f V (v) Lecture 5 6 Lecture 5 7

3 Example Let U be a random variable with an exponential distribution with parameter a. { ae au for u 0 f U (u) 0 for u<0 To find the expected value of U we calculate the integral E[U] 0 uae au du 1 a The parameter a is therefore the reciprocal of the expected value. You should setch the exponential distribution for a few values of a and convince yourself that this is a reasonable result. Example: V U 2 Let U have an exponential distribution with parameter a, and let V U 2. Find E[V ]E[U 2 ]. { ae au for u 0 f U (u) 0 for u<0 E[U 2 ] 0 u2 ae au du 2 a 2 Lecture 5 8 Lecture 5 9 Mean, Variance and Standard Deviation The mean value of a random variable U is µ E[U]. The variance is σ 2 E[(U µ) 2 ]. The standard deviation is σ. It is always true that σ 2 E[U 2 ] µ 2 For the exponential distribution Moments The th moment of a random variable U is defined as E[U ], and can be computed by the following formulas for the discrete and continuous cases, respectively. E[U L ] u i P [U u i] i1 E[U ] u f U (u)du σ 2 2 a 2 1 a 2 1 a 2 For an exponential distribution, σ µ. Lecture 5 10 Lecture 5 11

4 Example: Rayleigh Distribution f R (r) r b e r2 /2b (r 0) r 2 µ R E[R] 0 b e r2 /2b πb dr 2 E[R 2 r 3 ] 0 b e r2 /2b dr 2b var(r) σr 2 πb 2b b The standard deviation is σ R b. By a little calculus you can show that the pea of the distribution occurs at r b, so that the standard deviation is about 65% of the location of the pea. As b is increased the curve shifts to the right and broadens in proportion to b. Multivariate Functions Let U [U 1,U 2,...,U n ]whereu i, i 1,...,n are random variables. Let V g(u). Then V is a function of several random variables. E[V ] g(u 1,u 2,...,u n )f U (u 1,u 2,...u n )du 1 du 2 du n Lecture 5 12 Lecture 5 13 Dartboard Example Consider the problem of throwing darts at a target. Let X and Y be random variables that represent the horizontal and vertical offset of the dart from the center of the target. Note that the center is at (0,0) so that the variables can have both negative and positive values. Suppose that X and Y are statistically independent normal random variables, with f X (x) 1 σ 2π exp [ x2 f Y (y) 1 [ ] σ 2π exp y2 ] Dartboard Example Because the horizontal and vertical offsets are statistically independent, the joint distribution is f X,Y (x, y) f X (x)f Y (y) 1 [ σ 2 2π exp x2 + y 2 ] Find the distance the dart hits from the center. Let R X 2 + Y 2 Substitute r 2 x 2 + y 2 in the probability density function. The volume above a ring of thicness dr and mean radius r is 2πrf X,Y (r)dr. This must be the same value as achieved under the distribution of f R (r) over the interval dr. Therefore, f R (r)dr 2πrf X,Y (r)dr Lecture 5 14 Lecture 5 15

5 Dartboard Example f R (r) r [ ] σ 2exp r2 Replace σ 2 with a parameter b. The result is f R (r) r [ ] b exp r2 2b This is the Rayleigh probability density or Rayleigh distribution with parameter b. Characteristic Functions What is the PDF on the angle θ? Lecture 5 16 Characteristic Function The characteristic function of a random variable X is the expected value of the function e jux where j 1. Continuous Distribution M X (ju) E [ e jux] f X(x)e jux dx Discrete Distribution M X (ju) E [ e jux] i f X (x i )e jux i The similarity of the characteristic function and the Fourier transform is obvious. Characteristic Function Since f X (x) is nonnegative and e jux has unit magnitude for all values of x and u, f X(x)e jux dx f X (x)e jux dx f X(x)dx 1 Hence, the characteristic function always exists and M X (ju) M X (0) 1 If the cf is nown, then one can compute the pdf by f X (x) 1 2π M X(ju)e jux du A major use of the cf is in generating moments of random variables. Lecture 5 17 Lecture 5 18

6 Moment Generation The function e jux can be expanded in a Taylor series: e jux (jux) n (ju) n X n n0 n! n0 n! Hence, (ju) n M X (ju) E[X n (ju) n ] m n n0 n! n0 n! where m n is the n th moment of X about the origin. Moment Generation If we expand the characteristic function in a Taylor series, then m n can be found from the coefficient of u n in the expansion. Specifically, if M X (ju) n u n n0 then m n n! n /j n. This can be seen by comparison with (ju) n M X (ju) m n n0 n! The moment can also be found by taing derivatives. E[X m ] 1 d m M X j m du m u0 Lecture 5 19 Lecture 5 20 Example: Exponential PDF Consider the exponential pdf given by The characteristic function is f X (x) be bx step(x) M X (u) E [ e ju] 0 be (b ju)x dx b b ju 1 1 ju b Exponential PDF - Continued Within the circle u <bwe can expand the term on the right in a power series by using (for t < 1) 1 1 t 1+t + t2 + t 3 + Then the expansion, with t ju/b, is ( ) ju n M X (u) n0 b In this case n (j/b) n so that m n E [X n ] n! j n n n! b n Lecture 5 21 Lecture 5 22

7 Example: Binomial Distribution The probability of successes in n trials is governed by the binomial probability distribution b(n,, p) ( n) p (1 p) n The cf is n M X (ju) b(n,, p)e ju n ( n) p (1 p) n e ju 0 0 n ( n) (pe ju ) (1 p) n (pe ju +1 p) n 0 Here we have made use of the binomial expansion (a + b) n n ( n) a b n 0 Binomial Distribution - continued The mean value is found by differentiation. m 1 E[X] 1 dm npe ju (pe ju +1 p) n 1 u0 np j du u0 Similarly, m 2 E[X 2 ] 1 d 2 M j 2 du 2 u0 n(n 1)p 2 e ju (pe ju +1 p) n 2 + npe ju (pe ju +1 p) n 1 u0 n(n 1)p 2 + np The variance is easily found by σ 2 var[x] E[X 2 ] E 2 [X] n(n 1)p 2 + np n 2 p 2 np(1 p) Lecture 5 23 Lecture 5 24