Expected value of r.v. s

Transcription

1 10 Epected value of r.v. s CDF or PDF are complete (probabilistic) descriptions of the behavior of a random variable. Sometimes we are interested in less information; in a partial characterization. 8 i i different center different spread BEE ECE 5605, Virginia Tech tria l # 19

2 Epected value or MEAN E tf tdt [ ] () E [ ] p ( ) k k defined if integral or sum converges absolutel E = t f () t dt < k k k BEE ECE 5605, Virginia Tech 0 k E = p < there are random variables for which the above do not converge we then sa the mean does not eist E[ ] represents the "center of mass" arithmetic average of large # independent observations of a r.v. will tend to the mean; it s like the average of

3 E 3.9 mean of U[a,b] [ ] E tf () t dt f Uab [, ] = b a 1 t dt b a + = = b a b a b a 1 b a a b b+ a midpoint of interval [a,b] BEE ECE 5605, Virginia Tech 1

4 E 3.31 mean of eponential r.v. Time between customer arrivals at a service station has an eponential PDF with parameter λ. Find the mean inter-arrival time. u dv udv = uv vdu λt λt λt E = t e dt = te + e dt [ ] λ λt λt e = limte 0 + = t λ As λ is customer arrival rate in customers per second, the mean inter-arrival time of λ -1 seconds/customer makes sense! BEE ECE 5605, Virginia Tech λ

5 E 3.3 mean of geometric r.v. N is the number of times a computer polls a terminal until the terminal has a message read for transmission. Assuming the terminal produces messages according to a sequence of independent Bernoulli trials, N has a geometric distribution. Find the mean of N. [ ] E N = k = 1 kpq k = p = 1 q p d d k k 1 = k = k= 0 k= ( ) if probabilit of success is p, then it makes sense that on average it takes p -1 trials to hit success BEE ECE 5605, Virginia Tech 4

6 Epected value of a function of a r.v. [ ] () E tf tdt = g alternative in terms of k h h k [ ] E f( ) h = = k k k k g f ( ) h g f k k k limit h 0 ( d ) equivalent events k BEE ECE 5605, Virginia Tech 5

7 E 3.33 Θ ~ U 0, ( π ] = acos( ωt+θ) Sampling a sinusoid with random phase. Find the epected value of and of,the power of. [ ] = cos( ω +Θ) E E a t π 1 = acos( ωt+ θ) dθ π 0 a π = sin ( ωt + θ) 0 π a = sin ( ωt+ π) sin ( ωt) = 0 π E E a t a E ( ω ) = cos +Θ ( ω t ) = 1 cos + + Θ π a 1 = 1+ cos( ωt θ) dθ + π 0 a = agreement with time-averages: DC value of 0, power a / BEE ECE 5605, Virginia Tech 6

8 Variance of Provides information about a random variable in addition to its mean value regarding its deviations from the mean D E deviation from the mean [ ] VAR E E STD [ ] [ ] [ ] VAR[ ] [ ] = [ ] + [ ] = E E[ ] E[ ] + E [ ] = E E [ ] VAR E E E measures of width/spread variance of r.v. standard deviation of r.v. BEE ECE 5605, Virginia Tech 8

9 E 3.36 variance of ~U[a,b] [ ] E = b + a 1 b+ a VAR[ ] = d b a b a ( b a) ( b a) ( b a) 3 1 = d= = ( b a) b a 3 b a 1 BEE ECE 5605, Virginia Tech 9 b a ( 4 ( ) ) 4+ ( ) U [, 4] E [ ] = = 1; VAR[ ] = = 3 1

10 3.38 variance of Gaussian r.v. ( m) 1 = < < σ π σ f e σ π = d dσ π = e ( m) BEE ECE 5605, Virginia Tech 5 σ d ( m m ) 3 e σ d σ ( m) σ 1 VAR[ ] = ( m) e d = σ σ π

11 Effect of VAR on the PDF of a Gaussian r.v PDF for Gaussian r.v. VAR = 1 VAR = 4 VAR = f BEE ECE 5605, Virginia Tech 6

12 Mean & variance Are the two most important parameters for (partiall) characterizing the PDF of a r.v. Others are sometimes used, e.g. skewness E E 3 σ ( [ ]) 3 which measures the degree of asmmetr about the mean Skewness is zero for a smmetric PDF Each involves n th n n moment of r.v. : E = f ( ) d VAR[ ] = E E [ ] Under certain conditions, a PDF is completel specified if all moments are known (more on that later) [ ] = 0 [ + ] = [ ] [ ] = [ ] VAR c VAR c VAR VAR c c VAR BEE ECE 5605, Virginia Tech 7

13 Multiple r.v. s Several r.v. s at a time Measuring different quantities simultaneousl Engine oil pressure, RPM, generator voltage Repeated measurement of the same quantit Sampling a waveform, such as EEG, speech Joint behavior of two or more r.v. s Independence of sets of r.v. s Correlation if not independent BEE ECE 5605, Virginia Tech

14 E 4.1 Let a random eperiment be the selection of a student s name from an urn: outcome ζ. Define the following three functions: H(ζ) = student s height, in inches W(ζ) = student s weight, in pounds A(ζ) = student s age, in ears (H(ζ), W(ζ), A(ζ) ) is a vector r.v. BEE ECE 5605, Virginia Tech 4

15 Events & probabilities Each event involving an n-dimensional r.v. =( 1,,, n ) has a corresponding region in an n-dimensional real space E.g. -D r.v. =(,) = { + 10} B = { min(, ) 5} A 10 5 { 5} C = BEE ECE 5605, Virginia Tech 6

16 Independence Intuitivel, if r.v. s and are independent, then events that involve onl should be independent of events that involve onl. In other words, if A 1 is an event that involves onl and A is an event that involves onl, then [, ] = [ ] [ ] P A A P A P A 1 1 In general, n r.v. s are independent if [ ] = [ ] [ ] P 1 A1,, n An P 1 A1 P n An where A is an event that involves onl k if r.v. s are independent, knowing the probabilities of the r.v. s in isolation suffices to specif probabilities of joint events BEE ECE 5605, Virginia Tech 11 k

17 Pairs of discrete r.v. s vector r.v. = (, ) joint probabilit mass function: = { = } { = } p P, j, k j k {(, ), 1,,, 1,, } j k S = j = k = P = j, = k j = 1,, k = 1,, gives probabilit of occurrence of pairs ( j, k ) For an event A: P[ A] = p, j, k ( j, k) BEE ECE 5605, Virginia Tech 1 A j= 1 k= 1 [ ] p, = P S = 1, j k

18 Marginal PMF s = (,an ) p p j, j k k = 1 {{ } { }} { } { } j 1 j (, ), j k =,, k j k j= 1 { } = P = = = = = p p p marginal PMF s are insufficient in general - for specifing joint PMF BEE ECE 5605, Virginia Tech

19 E 4.6 tossing loaded dice k joint PMF in units of 1 4 j marginals give no clue BEE ECE 5605, Virginia Tech 14 6

20 Joint CDF of and 1 Basic building block: 1 semi-infinite rectangle:, : {( ) { } { } } 1 1 joint cumulative distribution function of and : F, = P { } { }, [, ] = P 1 1 product-form event long-term proportion of times in which the outcome (,) falls in the blue rectangle, or the probabilit mass in it BEE ECE 5605, Virginia Tech 17

21 properties of the joint CDF i. F, F,,, 1 1, 1 1 ii. F, = F, = 0, 1, iii. F, = 1, non-decreasing in NE direction neither nor takes on values <- and can take on onl values < iv. F F, P, P = ( ) = [ < ] = [ ], F F,, P, P = ( ) = [ < ] = [ ] v. lim F, = F a, a + marginal cumulative distribution functions,, lim F, = F b, b +,, continuous from N & E BEE ECE 5605, Virginia Tech 18

22 E 4.8 marginals for continuous joint r.v. s Given F, = 1 e 1 e u u, ( α)( β) 1 F = lim F, = 1 e u ( α ), 1 1 F = lim F, = 1 e u ( β ), 1 marginal CDFs are eponential distributions with parameters α and β respectivel BEE ECE 5605, Virginia Tech 19

23 Joint CDF in terms of joint PDF & v.v. F, f ', ' dd ' ' f =,,,, = if jointl continuous r.v. s F,, if the CDF is discontinuous - or the partial derivatives are discontinuous - then the joint PDF does not eist BEE ECE 5605, Virginia Tech 3

24 {(, ): ; } A = a< b a< b 1 1 b b 1 [ ] Pa< b, a < b = f ', ' dd ' ' 1 1, a a [ ] P d, d f ', ' d ' d ' 1 + d + d < + < + = f,,, dd joint PDF specifies probabilit of product-form events: < + d < + d { } { } BEE ECE 5605, Virginia Tech 4

25 Marginal PDF s from joint PDF + d + d d d f = F,, = f, ', ' d' d' d d = f,, ' d' d f ( ) = F (, ) = f ( ', ) d',, d concentrating mass on -ais concentrating mass on -ais marginal PDF s are obtained b integrating out the other variable(s) BEE ECE 5605, Virginia Tech 5

26 I 1 0 f, (, ) ( ) [ ] [ ] 1, 0,1 0,1 = 0 ow.. E 4.10 CDF for jointl uniform r.v. s III II 1 V IV = F, f ', ' d ' d ',, ( ) = 0 for, I = 1 d ' d ' = for, II BEE ECE 5605, Virginia Tech = 1 d ' d ' = for, III = 1 d ' d' = for, IV = 1 d ' d ' = 1 for, V 0 0

27 E 4.11 finding marginal PDF s f, (, ) ce e 0 < = 0 ow.. 1 = ce e dd = ce e dd = e 1 c = ce d = c = f 1 0 = e e d = e e < 0 ( ) f = e e d= e e = e 0 < are these PDF s? BEE ECE 5605, Virginia Tech 8

28 E 4.1 Find P[+ 1] in E 4.11 = 1 = [ ] P + 1 = e e dd = e e dd ( 1 ) = e e e d 0 = e + 1 e = 1 e BEE ECE 5605, Virginia Tech 9

29 E 4.13 Find marginal PDFs of jointl Gaussian PDF 1 f,, = e π 1 ρ ( ρ ) ( 1 ρ ) + ( ρ+ 1 ρ ) ( e 1 ρ f ) = e d π 1 ρ = ( 1 ρ ) ( ) ( ρ ρ+ 1 ρ ) ( e 1 ρ ) e d π 1 ρ completing the square ( ρ) e 1 1 ( ρ ) e = e d = = N π 1 ρ π π ( 0,1) BEE ECE 5605, Virginia Tech 30 it s a PDF smmetr = f f

30 Independence of two r.v. s discrete r.v.'s and are independent iff joint PMF is product of marginal PMFs, j k BEE ECE 5605, Virginia Tech 9

31 E 4.15 another look at E 4.6 loaded dice k joint PMF in units of 1 marginals give no clue 4 j BEE ECE 5605, Virginia Tech 10 6 not independent p,, p p, j k j k j k

32 Independence of two r.v. s in general IFF p,, = p p, j k j k j k discrete F F F,, =, f f f,, =, jointl continuous BEE ECE 5605, Virginia Tech 14

33 E 4.17 back to E 4.11 joint f,, e e 0 < = 0 ow.. 0 = marginal PDFs f 1 0 = e e d = e e < 0 ( ) f = e e d= e e = e 0 < product? e e e 1 e e and are not independent BEE ECE 5605, Virginia Tech 15

34 E 4.18 back to E f,, = e π 1 ρ f ( ) = e π f ( ρ ) ( 1 ρ ) + ( ) e = iff ρ = 0 π f f ( ) + e e e = = π π π BEE ECE 5605, Virginia Tech 16

35 Conditional probabilit Man r.v. s are not independent, such as when measuring one variable to learn about another one, or when sampling slowl relative to the rate of change of a signal. We re then interested in the probabilit of an event related to, given the knowledge of = (a measurement) We ll see that E[ =] is of significance also BEE ECE 5605, Virginia Tech 18

36 Conditional probabilit [ ] P A = = discrete [, = ] P[ = ] P A [, = ] k P[ = ] P F = for P = > 0 [ ] k k k if derivative eists f = d F d k k [ ] ( ) P A = = f d k k A independence F = F BEE ECE 5605, Virginia Tech 19 f = f

37 Conditional probabilit [ ] P A = = [, = ] P[ = ] P A and discrete P =, ( ) k = p j k = P j = = k = P = δ functions ~ PMF weights in general (, ) [ ] p, k j = = > P[ ] k p k = p k 0for P[ = ] = p ( ) = 0 for 0 k k [ = ] ( ) k = j k = j P A p p A A j j BEE ECE 5605, Virginia Tech 0 k j if also independent

38 E 4.0 back to E 4.14 is the input and the output of a communication channel. P[<0 =+1]? f ( 1 ) ~ U[ 1,3] [ 0 1] 1 1 P< =+ = d= integrate over event BEE ECE 5605, Virginia Tech 1

39 Conditional CDF of given = F lim F < + h h 0 [, < + ] P[ < + h] P h = = + h BEE ECE 5605, Virginia Tech 3 f + h, ', ' d ' d ' f ' d' h f,, ' d' f,, ' d' = hf f = f ' d' = F if independent

40 Conditional PDF of given = F = f d,, ' ' f d f,, f = F = d f if independent f = f f almost Baes rule f, (, ) dd d = f d BEE ECE 5605, Virginia Tech 4

41 E 4.1 back to E 4.11 f,, ce e 0 < = 0 ow.. 0 = f = e 1 e 0 < ( ) f = e 0 < f (, ) e e f = = = e <, ( ) ; f e f, (, ) e e e f = ;0 f = = 1 e < ( e 1 e ) BEE ECE 5605, Virginia Tech 5