3 Operations on One R.V. - Expectation

Transcription

1 Engineering Dept.-JUST. EE360 Signal Analysis-Electrical - Electrical Engineering Department. EE360-Random Signal Analysis-Electrical Engineering Dept.-JUST. EE360-Random Signal Analysis-Electrical Engineering Department-JUST. EE360-Random EE360 Random Signal analysis Chapter 3: Operations on One R.V - Expectation Operations on One R.V. - Expectation Expectation=Averaging=Mean value=statistical Average X = E[X] = = N i=1 xfx(x)dx continuous xip(xi) discrete If X has pdf that is symmetrical around x = a, i.e. fx(x + a) = fx( x + a), E[X] = a

2 Chapter 3: Operations on One R.V - Expectation - 2 Ex Ninty people have change as follows # of people change in cents Average=(8*18+12*45+28*64+22*72+15*77+5*95)/( )=63.22 Cents. or we can write it as Average=(8/90)*18+(12/90)*45+(28/90)*64+(22/90)*72+(15/90)*77+(5/90)*95=63.22 Cents

3 Chapter 3: Operations on One R.V - Expectation - 3 Example Find X for the continuous, exponential r.v. with f X (x) = 1 b e (x a)/b x > a 0 x < a E[X] = a x 1 b e (x a)/b dx = ea/b xe x/b dx = a + b b a xe ax dx = e ax [ x a 1 a 2 ] from Appendix C

4 Chapter 3: Operations on One R.V - Expectation - 4 Expected value of a function of a R.V. E[g(X)] = = g(x)f X (x)dx continuous N g(x i )P(x i ) discrete i=1 If g(x) = N k=1 g k(x) E(g(X)) = N k=1 E(g k(x))

5 Chapter 3: Operations on One R.V - Expectation - 5 Example V =Rayleigh r.v., a = 0, b = 5. Y = g(v ) = V 2 E[g(V )] = E[V 2 ] = = 0 0 v 2 f V (v)dv = 2v e v /5 dv a v 2 2 b (v a)e (v a)2 /b dv Let ξ = v2 5, dξ = 2vdv 5, substitute E[V 2 ] = 5 0 ξe ξ dξ = 5

6 Chapter 3: Operations on One R.V - Expectation - 6 Ex Average Information (Entropy) X can be one of L symbols, each having probability of P(x i ). Define the information in a symbol x i as log 2 ( 1 P(x i ) ) = log 2(P(x i )) Average entropy of the discrete source = H bits per symbol= H = L P(x i ) log 2 (P(x i )) i=

7 Chapter 3: Operations on One R.V - Expectation - 7 Conditional Expected Value E(X B) = xf X (x B)dx If B = {X b}, < b < and using the results from chapter 2: f X (x) b x < b f X (x X b) = f X(x)dx 0 x b to get E(X X b) = xf X(x)dx b f X(x)dx = b xf X(x)dx b f X(x)dx

8 Chapter 3: Operations on One R.V - Expectation - 8 Moments 1. Moments about the Origin m n = nth moment = E[X n ] = Here m 0 = 1.0, and m 1 = X x n f X (x)dx 2. Central Moments µ n = nth central moment = E[(X X) n ] = Here µ 0 = 1.0, and µ 1 = 0 (x X) n f X (x)dx

9 Chapter 3: Operations on One R.V - Expectation - 9 Variance and Skew σ 2 X = Variance = µ 2 =second central moment = = E[(X X) 2 ] = (x X) 2 f X (x)dx = E[X 2 2 XX + X 2 ] = E[X 2 ] 2 XE[X] + X 2 = E[X 2 ] X 2 = m 2 m Skew of the density function=the third central moment µ 3 = E[(X X) 3 ] is a measure of the asymmetry of f X (x) about x = X = m 1. Skewness or Coefficient of skewness= µ 3 σ 3 X If f X (x) is symmetric about x = X, it has zero skew.

10 Chapter 3: Operations on One R.V - Expectation - 10 Example X is exponentially distributed r.v. E[X] = a x 1 b e (x a)/b dx = a + b done in Example σ 2 X = a (x X) 2 1 b e (x a)/b dx Let ξ = x X, σ 2 X = e ( X a)/b b a X ξ 2 e ξ/b dξ = (a + b X) 2 + b 2 = b

11 Chapter 3: Operations on One R.V - Expectation - 11 Example For the previous example (exponential pdf) compute the skew and coefficient of skewness. E[(X X) 3 ] = E[X 3 3 XX X 2 X X 3 ] = E(X 3 ) 3 XE[X 2 ] + 2 X 3 = E(X 3 ) 3 X(σ X 2 + X 2 ) + 2 X 3 = E(X 3 ) 3 Xσ X 2 X 3 E[X 3 ] = a x 3 and since X = a + b and σ 2 X = b2 we get b e (x a)/b dx = a 3 + 3a 2 b + 6ab 2 + 6b µ 3 = 2b 3 and µ 3 σ 3 X = 2

12 Chapter 3: Operations on One R.V - Expectation - 12 Chebychev s Inequality For any r.v. X having mean= X and variance=σ 2 X, then P { X X ǫ} σ2 X ǫ 2, ǫ > 0 P { X X < ǫ} 1 σ2 X ǫ 2, ǫ > 0 Proof: σ 2 X = ǫ 2 (x X) 2 f X (x)dx x X ǫ x X ǫ f X (x)dx = ǫ 2 P { X X ǫ} (x X) 2 f X (x)dx

13 Chapter 3: Operations on One R.V - Expectation - 13 Chebychev s Inequality - alternate form For any r.v. X having mean= X and variance=σ 2 X, then P { X X kσ X } 1 k 2, k > 0 P { X X < kσ X } > 1 1 k 2, k > 0 Proof: σ 2 X = k 2 σ 2 X (x X) 2 f X (x)dx x X kσ X (x X) 2 f X (x)dx x X kσ X f X (x)dx = k 2 σ 2 XP { X X kσ X } from which P { X X kσ X } 1 k 2, k > 0

14 Chapter 3: Operations on One R.V - Expectation - 14 Example Find the largest value of P { X X 3σ X }. Use Chebychev s inequality with ǫ = 3σ X to get P { X X 3σ X } σ2 X (3σ X ) 2 = 1 9 = 11.1%

15 Chapter 3: Operations on One R.V - Expectation - 15 Markov s Inequality If X is a non-negative r.v., P {X a} E[X] a, a > 0 Proof: E[X] = 0 xf X (x)dx a xf X (x)dx a a f X (x)dx = ap {Y a}

16 Chapter 3: Operations on One R.V - Expectation - 16 Bienayme Inequality In Markov inequality P {Y α} E[Y ] α, α > 0, let Y = X a n 0, and let α = ǫ n P { X a n ǫ n } E[ X a n ] ǫ n P { X a ǫ} E[ X a n ] ǫ n Can derive Chebyshev by setting a = η = X, n = 2, ǫ = kσ and substitute in Bienayme inequality

17 Chapter 3: Operations on One R.V - Expectation - 17 Characteristic Function Characteristic function of a r.v. X is defined as and Φ X (ω) = E[e jωx ], j = 1, < ω < = f X (x) = 1 2π f X (x)e jωx dx Φ X (ω)e jωx dω The n th moment of X is given by 1. Φ X (ω) always exists. m n = ( j) n dn Φ X (ω) dω n ω= Φ X (ω) < Φ X (0) = 1 (Problem 3.3.1)

18 Chapter 3: Operations on One R.V - Expectation - 18 Ex Find the characteristic fcn Φ X (ω) for an exponential r.v. X Φ X (ω) = E[e jωx ] = = e a b b = Its first moment m 1 = ( j) dφ X(ω) dω a e jωa 1 jωb f X (x)e jωx 1 x a dx = a b e b e jωx dx [ ] e ( 1 b jω)x dx = e a b e ( 1 b jω)x b ( 1 b jω) a [ ] ω=0 = ( j)e jωa ja 1 jωb + jb (1 jωb) 2 ω=0 = a+b

19 Chapter 3: Operations on One R.V - Expectation - 19 Moment Generating Function M X (ν) = E[e νx ], where ν is a real number < ν < = f X (x)e νx dx If M X (ν) exists, moments can be evaluated from the relation m n = dn M X (ν) dν n ν=

20 Chapter 3: Operations on One R.V - Expectation - 20 Example Find M X (ν) and its first moment for an exponential r.v. X M X (ν) = E[e νx ] = = e a b b a f X (x)e νx dx = e [ν (1/b)]x dx = eaν 1 bν a 1 (x a) b e b e νx dx its first moment can be evaluated from the relation m 1 = dm X(ν) ν=0 = eaν [a(1 bν) + b] dν (1 bν) 2 ν=0 = a + b

21 Chapter 3: Operations on One R.V - Expectation - 21 Chernoff s Inequality and Bound Ex Chernoff s Inequality: P {X a} e νa M X (ν) a arbitrary real constant, ν > 0. Proof: note that e ν(x a) u(x a) P {X a} = a f X (x)dx = f X (x)u(x a)dx f X (x)e ν(x a) dx = e νa M X (ν) Chernoff s bound: Take the minimum w.r.to ν gives Chernoff s bound.

22 Chapter 3: Operations on One R.V - Expectation - 22 Transformations of a Random Variable Given r.v. X having pdf f X (x) and another r.v. Y that relates to X as Y = T(X), where T(.) is a transformation (mapping) from X to Y. Find f Y (y) =?

23 Chapter 3: Operations on One R.V - Expectation - 23 Monotonic Transformation of a Continuous R.V. y y = y 1 y x 1 x y = y 1 x 1 x Figure 1: Monotonic Transformation: (a) increasing (b) decreasing

24 Chapter 3: Operations on One R.V - Expectation - 24 Monotonic Transformation of a Continuous R.V. y y = y 1 x 1 x all probability measures, i.e. +ve quantities. P {y 1 Y < y 1 + y} = P {x 1 X < x 1 + x} f Y (y) y = f X (x) x from which f Y (y) = f X(x) dy dx

25 Chapter 3: Operations on One R.V - Expectation - 25 Ex Linear transformation T such that Y = T(X) = ax + b, a, b real constants, then X = T 1 (Y ) = (Y b)/a and dx dy = 1 a and 1 If f X (x) = 2πσ 2 X e (x ax) 2 f Y (y) = f X ( y b a ) 1 a (2σ 2 X ), then f Y (y) = 1 e [ y b a a X ]2 2σ 2 X 1 2πσ 2 X a = 1 2πa2 σx 2 e [y (aax+b)] 2 2a 2 σ 2 X which is a gaussian r.v. with mean=a Y = aa X + b and variance=σ 2 Y = a2 σ 2 X

26 Chapter 3: Operations on One R.V - Expectation - 26 Nonmonotonic Transformation of a Cont. R.V. y y = y 1 x 1 x 2 x 3 x all probability measures, i.e. +ve quantities. P {y 1 Y < y 1 + y} = 3 i=1 P {x i X < x i + x} f Y (y 1 ) y = 3 i=1 f X(x i ) x from which f Y (y 1 ) = 3 i=1 f X (x i ) dy dx x=x i

27 Chapter 3: Operations on One R.V - Expectation - 27 Transformation of r.v.s: Example X has pdf = f X (x). Y = T(X) = cx 2, c real > 0. Find distribution of Y? F Y (y) = P {Y y} = P {cx 2 y} = P { y/c X y/c} = F X ( y/c) F X ( y/c) from which its derivative f Y (y) = f X ( y/c) d( y/c) dy = f X( y/c) + f X ( y/c) 2, y 0 cy f X ( y/c) d( y/c) dy

28 Chapter 3: Operations on One R.V - Expectation - 28 Transformation of r.v.s: Example-Ross X u(0, 1). Y = X n. Find distribution of Y for 0 y 1 F Y (y) = P {Y y} = P {X n y} = P {X Y 1 n } = F X (y 1 n ) = y 1 n from which its derivative f Y (y) = ( 1 n y 1 n 1 )(u(y) u(y 1))

29 Chapter 3: Operations on One R.V - Expectation - 29 Transformation of a Discrete R.V.: Example R.V. X have values x = 1, 0, 1, 2 with probabilities 0.1, 0.3, 0.4, 0.2, i.e. f X (x) = 0.1δ(x + 1) + 0.3δ(x) + 0.4δ(x 1) + 0.2δ(x 2). If T(X) = Y = 2 X 2 + (X 3 /3), find f Y (y) =? The corresponding values of y are 2/3, 2, 4/3, and 2/3. Here P {Y = 2/3} = P {X = 1} + P {X = 2} and f Y (y) = 0.3δ(y (2/3)) + 0.4δ(y (4/3)) + 0.3δ(y 2)

30 Chapter 3: Operations on One R.V - Expectation - 30 Computer Generation of a One R.V. Find the transformation T(X) that will create a r.v. Y of prescribed distribution function when X has a uniform distribution on (0,1). Pick x (0, 1). y = T(x), x = T 1 (y). Here we put x as F Y (y), i.e. F Y (y) = x,0 < x < 1. Solve for y = T(x).

31 Chapter 3: Operations on One R.V - Expectation - 31 Computer Generation of a One R.V. - Cont. -1 : Example Generate Rayleigh r.v. with a = 0. x = F Y (y) = 1 e y2 /b, 0 < x < 1. 1 x = e y2 /b y 2 /b = ln(1 x) y = T(x) = b ln(1 x), 0 < x < 1

32 Chapter 3: Operations on One R.V - Expectation - 32 Computer Generation of a One R.V. - Cont. -2 : Example Generate Arcsine r.v. F Y (y) = 0 y a π sin 1 ( y a ) a < y < a 1 a y The solution y = a sin[π(2x 1)/2], 0 < x < 1 = 0, x, 1,

33 Chapter 3: Operations on One R.V - Expectation - 33 Computer Generation of a One R.V. - Cont. -3 : Gaussian from Uniform - Box-Muller s Transformation R 1, R 2 [0, 1]. Generate two Gaussian r.v. s N 1, N 2 both zero mean and unit variance as follows: N 1 = ( 2 lnr 1 ) 1/2 cos(2πr 2 ) and N 2 = ( 2 lnr 1 ) 1/2 sin(2πr 2 ) Note that r 2 = N1 2 + N2 2 = 2 lnr 1 and Φ = 2πR 2 If two gaussian r.v. s with mean m and variance=σ 2, X and Y N(m, σ 2 ) are needed we use the transformations: X = m + σn 1 and Y = m + σn 2

34 Chapter 3: Operations on One R.V - Expectation - 34 Figure 2: A Square-law Transformation

35 Chapter 3: Operations on One R.V - Expectation - 35 Figure 3: Example 3.5.3: Uniform R.V.: histogram and true density function

36 Chapter 3: Operations on One R.V - Expectation - 36 Generation of Rayleigh CPDF and PDF %Rayleigh ditributed r.v. x=rand(10000,1); a=1.5; b=2; %b > 0 y=a+sqrt(-b*log(1-x)); [X,IX]=sort(x); Y=y(IX); plot(y,x); %Distribution function print -deps rayleighcpdf.eps pause N=100 hist(y,n); %plot pdf into N range bins print -deps rayleighpdf.eps

37 Chapter 3: Operations on One R.V - Expectation - 37 Rayleigh Distribution PDF for Rayleigh CPDF for Rayleigh

38 Chapter 3: Operations on One R.V - Expectation - 38 Generation of Gaussian r.v. % Normal distribution Npoints=10000; r1=rand(npoints,1); r2=rand(npoints,1); n1=sqrt(-2*log(r1)).*cos(2*pi*r2); n2=sqrt(-2*log(r1)).*sin(2*pi*r2); Nbins=100; hist(n1,nbins); print -deps gaussianpdf.eps

39 Chapter 3: Operations on One R.V - Expectation - 39 Gaussian PDF PDF for Gaussian

40 Chapter 3: Operations on One R.V - Expectation - 40 Gaussian Distribution Gaussian pdf Gaussian cpdf mean=3, sigma= mean=3, sigma=1 PDF for Gaussian CPDF for Gaussian

41 Chapter 3: Operations on One R.V - Expectation y = x

42 Chapter 3: Operations on One R.V - Expectation - 42 Problem page 102 Find Φ X (ω) for the binomial N Φ X (ω) = E[e jωx ] = P {X = k}e jωx N = N k=0 k N = N k k=0 k=0 = ((1 p) + pe jω ) N p k (1 p) N k e jωk (pe jω ) k (1 p) N k

43 Chapter 3: Operations on One R.V - Expectation - 43 Problem page 99 Find X,σ 2 X for the Binomial N X = E[X] = kp {X = k} k=0 N = k N k k=0 p k (1 p) N k Since (a + b) N = N k=0 N k a k b N k, take a to get N(a + b)n 1 = N k=0 N k ka k 1 b N k,

44 Chapter 3: Operations on One R.V - Expectation - 44 then multiplying both sides by a to get Na(a + b) N 1 = N k=0 N k ka k b N k. Now if we set a = p, b = 1 p we get Np(p + (1 p)) N 1 = N k=0 N k kp k (1 p) N k = Np = X For σ 2 X take the second derivative w.r.to a: N(N 1)a(a + b) N 2 + N(a + b) N 1 = N k=0 N k k 2 a k 1 b N k after multiplying both sides by a and letting a = p, b = 1 p we get N(N 1)p 2 + Np = E[X 2 ] = m 2 = N 2 p 2 Np 2 + Np = N 2 p 2 + Np(1 p) = N 2 p 2 + Np(1 p). σ 2 X = m2 m2 1 = N 2 p 2 + Np(1 p) (Np) 2 = Np(1 p) An alternative solution is to calculate this via Φ X (ω).

45 Chapter 3: Operations on One R.V - Expectation - 45 Problem X,σ 2 X for the Uniform (a,b) f X (x) = 1 b a (u(x a) u(x b)) X = b a x 1 b a dx = b2 a 2 2(b a) = a+b 2 m 2 = b a x2 1 b a dx = b3 a 3 3(b a) = b2 +ab+a 2 3 σx 2 = m 2 m 2 1 = b2 +ab+a 2 3 (a+b)2 4 = b2 2ab+a 2 12 = (b a)2 12

46 Chapter 3: Operations on One R.V - Expectation - 46 Problem page 102 Find Φ X (ω) for the Poisson Φ X (ω) = E[e jωx ] = = k=0 = e b P {X = k}e jωx k=0 b bk e k! ejωk k=0 (be jω ) k k! note that e x = k=0 = e b e bejω = e b+bejω = e b(1 e jω ) x k k!

47 Chapter 3: Operations on One R.V - Expectation - 47 Problem page 100 X,σ 2 X for the Poisson X = E[X] = = k=0 kp {X = k} k=0 b bk ke k! = e b (0 + b 1 + 2b2 2! + 3b3 3! + 4b4 4! + + mbm m! + ) = e b b( b1 1! + b2 2! + b3 3! + + bm 1 (m 1)! + ) = e b be b = b

48 Chapter 3: Operations on One R.V - Expectation - 48 m 2 = E[X 2 ] = = k=0 k 2 P {X = k} k=0 k 2 b bk e k! = e b (0 + (1) b 1 + (22 ) b2 2! + (32 ) b3 3! + (42 ) b4 4! + + (m2 ) bm m! + ) = e b ( b 1 + 2b2 1! + 3b3 2! + 4b4 3! + + m bm (m 1)! + ) = e b b( (1 + 1)b1 1! + (2 + 1)b2 2! = e b b( b1 1! + b2 2! + b3 3! + + bm 1 (m 1)! (3 + 1)b3 3! + + (1 + (m 1)) b m 1 (m 1) (1) b1 1! + (2)b2 2! + (3)b3 3! + + (m 1) b m 1 (m 1)! + )

49 Chapter 3: Operations on One R.V - Expectation - 49 = e b b(e b + be b ) = b + b 2 Hence σ 2 X = m2 m2 1 = b + b 2 (b) 2 = b

50 Chapter 3: Operations on One R.V - Expectation - 50 Gaussian f X (x) = 1 e (x a) 2σ 2 X 2πσ 2 X 2 dφ X (ω) dω = e jωa ω 2 σ 2 X 2 (ja ωσx 2 ) Φ X (ω) = e jωa ω 2 σ 2 X 2 At ω = 0, m 1 = ( j) dφ X(ω) dω = ( j)(j)a = a. In the proof we need the relation that f(t) = e t2 F.I. 2σ 2 F(ω) = 2πσ 2 e σ2 ω 2 2

51 Chapter 3: Operations on One R.V - Expectation - 51 X U(0, 1),Y = X,P {0.1 Y 0.5} =? f Y (y) = f X(x) dy dx = 2 x = 2y(u(y) u(y 1) P {0.1 Y 0.5} = = f Y (y)dy 2ydy = y = = 0.24

52 Chapter 3: Operations on One R.V - Expectation - 52 Monte Carlo Integration-1 Integrate g(x) = x between x 1 = 0, x 2 = 1 % int_mc.m % Monte Carlo Integration % Integrate f(x)=x between 0 and 1. N=input( Enter # of points to use for simulation = ); p=rand(n,2); I=find(p(:,2) >= p(:,1)); area=length(i)/n

53 Chapter 3: Operations on One R.V - Expectation - 53 Monte Carlo Integration-2 % int_mc1.m % Integrate f(x)=xˆ2 between x1 and x2. N=input( # of points to use for simulation = ); p=rand(n,2); x1=2; x2=5; y1=x1*x1; y2=x2*x2; % scale p(:,1) to be from x1 to x2 p(:,1)=p(:,1)*(x2-x1)+x1; % scale p(:,2) to be from y1 to y2 p(:,2)=p(:,2)*(y2-y1)+y1; I=find(p(:,2)<=p(:,1).ˆ2); % y <= xˆ2 ratio=(length(i)/n); area=ratio*(x2-x1)*(y2-y1)+(x2-x1)*y1

54 Chapter 3: Operations on One R.V - Expectation - 54 Monte-Carlo Integration: Check results % as a check first plot the function f=inline( x.ˆ2, x ); ezplot(f) %integrate using quadl area=quadl(@(x) f(x),x1,x2) %=39

55 Chapter 3: Operations on One R.V - Expectation - 55 Solution of the Monte Carlo Integration HW Problem After plotting the curve, we notice that the curve is positive for values of x. (the zero of a function can be computed via the command fzero: fzero( 3*x.ˆ2-7*x.*sin(x.ˆ2-1)+6,[-1,2]) From the plot we find that the range of x = [ 1, 2], y = [ 10, 5] N=1000; X=-1+(2-(-1))*rand(N,1); % scale X to be from Y=2+(18-2)*rand(N,1); %scale Y to be y=3*x.ˆ2-7*x.*sin(x.ˆ2-1)+6; %calculated y for X I=find(Y < y); %find generated (random) below the curve (calc area=(length(i)/n)*(3*16)

56 Chapter 3: Operations on One R.V - Expectation - 56 Summary Items covered were: Expectation was defined in general for a random variable or some function of a random variable. Moments (about the origin, and central) were developed as valuable measures of a random variable s characteristics. Of particular value were mean value, variance, and skew. The characteristic function and moment generating function were given as convenient methods for finding moments. Methods were given to transform one random variable into another, and to find distribution and density functions of the new random variable. The important concepts of how to generate a specified random variable by computer were developed and an example and chapter-end problems were included that are based on use of MATLAB software.