ECE 450 - Lecture #9, Part 1 Pairs of Random Variables, continued - Overview Joint PDF s A Discrete Example (PDF for Pair of Discrete RV s) Jointly Gaussian RV s Conditional Probability Density Functions (Again) Bayes Rules for PDF s Mnemonic Table: Conditional Probabilities vs. Conditional PDF s Communications Example: Signal + Noise 1
Pairs of RV s A Discrete Example (from Ross, A First Course in Probability) Suppose that 15 percent of the families in a certain community have no children, 0% have 1 child, 35% have children, and 30% have 3 children. Also suppose that each child is equally likely to be a boy or girl, independent of family size. Experiment: Choose a family at random from the community. RV s: Let B be the # of boys in the family, and let G be the # of girls. Find the joint pdf, f BG (b, g). Note: both RV s (B & G) are discrete, and each RV can take values:,,, or
Repeating the givens Fraction of families w/0 children:.15 Fraction of families w/1 child:.0 Fraction of families w/ children:.35 Fraction of families w/3 children:.30 Calculating some probabilities: Pr(B = 0, G = 0) = Pr(no children) =.15 Assumes Pr(G) = Pr(B) =.5 Pr(B = 0, G = 1) = Pr(1 girl and total of 1 child) = Pr(1 girl 1 child) Pr(1 child) =.5 (.0) =.10 Pr(B = 1, G = 1) = Pr(1 boy, 1 girl children) Pr( children) = Pr{ (GB or BG children) (.35) =.5 (.35) =.175 Pr(B =, G = 1) = Pr( boys, 1 girl 3 children) Pr(3 children) = Pr(BBG or BGB or GBB 3 children) (.30) = (3/8) *.30 =.115 etc. 3
i (Answers calculated on previous page are in bold.) Table of Joint Probabilities: P(B = i, G = j) j 0 1 3 Row sum = Pr(B = i) 0.15.10.0875.0375.3750 1.10.175.115 0.3875.0875.115 0 0.000 3.0375 0 0 0.0375 Col. Sum = Pr(G = j).3750.3875.000.0375 Verification of the other probabilities (not in bold) will be a homework problem. The joint pdf for B, G would be 16 delta functions in the x-y (or b-g) plane, with areas given by the corresponding probability numbers. The last row and the last column are the marginal probabilities, Pr(G = j) and Pr(B = i). 4
Jointly Gaussian RV s RV s and are said to be jointly Gaussian (or jointly normal) if: f (x, y) = 1 x y 1 1 exp{ (1 x m [( ) x m ( y m )( y m ) ( ) x ) ]} Parameters: Mean of =, Mean of =, Standard Deviation of =, Standard Deviation of = Correlation Coefficient (between and ) = 5
Jointly Gaussian RV s Claim (to be shown for homework): If we let = 0 in the equation for f (x, y) (repeated below), the joint pdf can be factored into the form: f (x, y) = f (x) f (y) f (x, y) = 1 1 1 1 exp{ (1 x m [( ) x m ( y m )( y m ) ( ) x ) ]} This will prove that: If two jointly normal RV s and have correlation coefficient = 0, then and are independent. 6
Jointly Gaussian RV s (aka: Bivariate Gaussians) m = m = 0; = = 1; = 0 m = m = 0; = = 1; =.9 0. 0.4 0.15 0.3 pdf 0.1 pdf 0. 0.05 0.1 0 4 0 y - -4-4 - x 0 4 0 4 0 y - -4-4 - x 0 4 Note Bell-shaped curve with circular cross-sections. Note: Bell is squished when x & y are highly correlated; elliptical cross-sections 7
MATLAB Code for the -Dimensional Gaussian PDF Function % function pdf = Gauss_d(mx, my, varx, vary, r) % Generates plot of -d Gaussian pdf function pdf = Gauss_d(mx, my, varx, vary, r) stdx = sqrt(varx); stdy = sqrt(vary); maxstd = max(stdx, stdy); xend = mx + 4 * maxstd; xstart = mx - 4 * maxstd; xstep = (xend - xstart)/100; yend = my + 4 * maxstd; ystart = my - 4 * maxstd; ystep = (yend - ystart)/100; A = 1/(*pi*stdx*stdy*sqrt(1-r^)); [x,y] = meshgrid(xstart : xstep : xend, ystart : ystep : yend); exparg = -1/(*(1-r^)); trm1 = ((x - mx).^)./varx; trm = *r*(x - mx).*(y - my)./ (stdx * stdy); trm3 = ((y - my).^)./vary; pdf = A * exp(exparg*(trm1 - trm + trm3)); surf(x,y,pdf) % surface plot of pdf(x,y) ECE 650 D. van Alphen 8
Conditional pdf s - Again Recall: f (x M) = (d/dx) F(x M), Special Case: If M = {a < b), then f (x M) = f (x) Pr(a b) Same shape as original pdf over region (a, b) Now: event M will be defined in terms of a nd RV, Example: If M is the event: y, then Pr( x, y) F (x M) = F (x y) = Pr(M) F(x, y) F (y) Similarly: If M is the event: y 1 <, then F (x M) = F (x y 1 < ) = Pr( x,m) Pr(M) F(x, y F (y ) ) F(x, y1) F (y ) 1 9
Conditional pdf s - Again In the previous cases, the event M had non-zero probability, so Pr(M) made sense in the denominator. Problem: How do we handle an event M with probability 0, as in M = { = y}, when is a continuous RV? F (x M) = F (x = y) = Pr( x, y) Pr( y) 0 0? Instead, try: F (x = y) = lim F (x y < y + Dy) Similarly: f f (x (y y) Dy 0 x) f (x, y) f (y) f (x, y) f (x) See Cooper & McGillem, 3 rd ed., p. 15 for proof (subscripts on LHS often omitted) 10
Bayes Rule for PDF s Shortened notation: Sometimes we write f (x =y) as f(x y); and f (y =x) as f(y x) So from the previous page we would write: f (x y) f (x, y) f (y) and f (y x) f (x, y) f (x) f (x, y) f (x y) f (y) f (x, y) f (y x) f (x) Equating the RHS s of the two equations above yields: Alternate form: f (x y) f (x y)f (y) f (y x) f (x) f(x y) f (y) = f(y x) f (x) f (y x)f (x) f (y) 11
Mnemonic: Table of Similar Equations Conditional Probabilities Conditional pdf s Definition (1) Pr(A B) Pr(A,B) Pr(B) f(x y) f(x,y) f () Definition () Pr(B A) Pr(A,B) Pr(A) f(y x) f(x,y) f (x) Total Prob. (1) Pr(B) = Pr(B A i ) Pr(A i ) f (y) = f(y x) f (x) dx Total Prob. () Pr(A) = Pr(A B i ) Pr(B i ) f (x) = f(x y) f (y) dx Bayes Rule (1) Pr(A B) Pr(B A)Pr(A) Pr(B) f(x y) f(y x)f(x) f () Bayes Rule () Pr(B A) Pr(A B)Pr(B) Pr(A) f(y x) f(x y)f(y) f (x) 1
Mixed Form of Bayes Rule (Mixing probabilities of events and pdf s) Let A be an event and let be a RV, with pdf f (x). Then Pr(A = x) = f (x A)Pr(A) f (x) and f (x A) = Pr( A x)f Pr(A) (x) 13
Signal + Noise An Example N Tx + = + N Suppose that signal is tx d over an additive noise channel, so that the received signal is = + N. Here is called the observable. The specific observable (say = y 1 ) is considered a good estimate for the desired value,. Goal: Find f(x y). For Bayes Rule, we need f(y x). But since = + N, and is given (as implied by f(y x), then the randomness (in ) is all in N, and is modeled by f N (n), so: f(y x) = f N (n = y x) = f N (y-x) (*) 14
Signal + Noise Specific Example, MAP Receivers for Binary Communication Consider communicating one of two messages (m 0 or m 1 ) over the additive white noise channel: x m i y i Modulator S (In transmitter) n Rcvr From communication theory, the maximum a posteriori probability (or MAP) decision rule for the receiver (which yields minimum Pr(error) in additive white noise for equally likely signals), based on observable y 0 is: m ^ i Modulator: maps messages or symbols to waveforms Choose m = m 1 iff Pr(m 1 =y 0 ) > Pr(m 0 =y 0 ) 15
Signal + Noise Specific Example, MAP Receivers, continued x m i y i Mod. S rcvr m ^ i n MAP Receiver decision rule is based on observable y: ^ Choose m i = m 1 iff Pr(m 1 = y) > Pr(m 0 = y) f(y m 1 ) Pr(m 1 )/f(y) > f(y m 0 ) Pr(m 0 )/f(y) Assume Pr(m 1 ) = Pr(m 0 ) = ½ f N (y-x 1 ) Pr(m 1 ) > f N (y-x 0 ) Pr(m 0 ) f N (y-x 1 ) > f N (y-x 0 ) 16
MAP Rcvr: Binary Case, Equally Likely Signals ^ So far: choose m i = m 1 iff f N (y-x 1 ) > f N (y-x 0 ) Now suppose that the modulator mapping (from messages to signals) is: For message m 0 : tx signal x 0 = 5 volts For message m 1 : tx signal x 1 = -5 volts And suppose that the noise is Gaussian: N(0, = sqrt()): ^ MAP rule becomes: choose m i = m 1 iff f N (y+5) > f N (y-5) 1 exp{ (y 5) /( )} 1 exp{ (y 5) /( )} > 17
MAP Rcvr: Binary Case, Equally Likely Signals (5, -5) in AWGN So far: Choose m i = m 1 iff: ^ < < < < Receiver implementation, with comparator: m 0 if y m 1 if y 18