Discrete Random Variable

Transcription

1 Discrete Random Variable Outcome of a random experiment need not to be a number. We are generally interested in some measurement or numerical attribute of the outcome, rather than the outcome itself. n tosses of a coin: our interest may be the total number of heads and we are not interested in specific order in which heads and tails occur. A random variable is a function that assigns a real number,, to each outcome in the sample space of a random experiment. x S x Real Line S

2 Discrete Random Variable Example: A coin is tossed three times and the outcomes are noted. S HHH, HHT, HTH, HTT, THH, THT, TTH, TTT Let be the number of heads in the three tosses. assigns each outcome in S a number S 0,,,3. from the set S HHH HHT HTH HTT THH THT TTH TTT 0 3 x S 0,,,3

3 Discrete Random Variable Example: A player pays $.50 to play the following game: A coin is tossed three times and the number of heads is counted. The player receives $ if = and $8 if =3, but nothing otherwise. Let Y be the reward to the player. Y is a function of a random variable and its outcomes can be related back to the sample space of the underlying experiment as follows: : HHH HHT HTH THH HTT THT TTH TTT 3 0 : : Y Example: Let be the number of heads in three independent tosses of a fair coin. Find the probability of the event. Find the probability that the player in example above wins $8. P P HHT, HTH, HHT P HHT P HTH P HHT 3 8 $8 8 P Y P HHH

4 Discrete Random Variable A random variable is called discrete if its range (the set of values that it can take) is finite or at most countably infinite. A discrete random variable is defined as a random variable that assumes values from S x, x, x,. a countable set, that is, 3 A discrete random variable is said to be finite if its range is finite, that is, S x, x, x,, x. 3 n The probabilities of all events involving can be found from the probabilities of the s where Ak : xk. Example: Consider the experiment of choosing a point a from the interval,. The random variable that associate the numerical value a to the outcome a is not discrete. However, the random variable defined as follows is discrete. if a 0, sgna 0 if a0, if a 0, A k

5 Discrete Random Variable: Probability Mass Function The Probability Mass Function (PMF) of a discrete random variable is defined as follows: p x P x P : x for all x p x is a function of x over the real line, and that p x, x, x,. For in, we have p x P A. values 3 The events S x k S k k A, A, form a partition of S as shown below. x can be nonzero only at the A A Ak x x xk x Proof: A A : x and x, j k Disjoint j k j k

6 Discrete Random Variable: Probability Mass Function Proof: (Continuing) Sample space S is the union of the A k s: Every in S is mapped into some Therefore: x k so that every belongs to an event S A A A k in the partition. All events involving the random variable can be expressed as the union of events B x, x, Consider the event in 5 A k s. in : : PA A5 PA PA5 p x p x P B P x x 5 5

7 Discrete Random Variable: Probability Mass Function The PMF p x satisfies three properties that provide all the information required to calculate probabilities for events involving the discrete random variable : ) p x 0 for all x Proof: ) p x p x P A k k xs all k all k P B p x where B S 3) in xb Since the PMF values are defined as a probability p x P x is true. Property follows because the events A x k k, the first property form a partition of S. Any event B involving is the union of the elementary events, so by Axiom III we have P in B P : x P x p x xb xb xb

8 Discrete Random Variable: Probability Mass Function Rule for Calculating of the PMF of Random Variable : For each possible value x of ) Collect all possible outcomes that give rise to the event x ) Add their probabilities to obtain p x Example: A coin with head probability p is tossed three times independently. Let be the number of heads. Find the PMF of. p 0 P 0 P TTT p p P P HTT, THT, TTH 3p p p P P HHT, THH, THH 3p p p P P HHH p p p p p

9 Discrete Random Variable: Probability Mass Function Example: Consider the game where a fair coin is tossed three times, a player receives $ if the number of heads is, $8 if the number of heads is 3, but nothing otherwise. Find the PMF of the reward Y. 8 8 p 0 p p 8 p 0 P TTT, TTH, THT, HTT 4 8 Y p P THH,HTH, HHT 3 8 Y p P HHH Y Y Y Y y

10 Second Roll Discrete Random Variable: Probability Mass Function Example: Consider the two rolls of a four-sided fair die. Let be the maximum of two rolls. Find the PMF of. Sample Space Pair of Rolls p p p p First Roll P P, 6,,,,, 3 6 P 3 3,,,3, 3,,,3, 3,3 5 6 P x 4 4 4,,,4, 4,,,4, 4,3, 3,4, 4,

11 Discrete Random Variable Discrete Uniform Random Variable: The discrete random variable takes on values in a set of consecutive integers S j, j,, j N with equal probability. Thus, PMF of the discrete uniform random variable is p k P k, for k j,, j N N Discrete uniform random variable occurs whenever outcomes are equally likely Toss of a fair coin or a fair die. Selection of numbers from an urn. Example: A random number generator produces an integer number that is equally likely S 0,,,, M. Find the PMF of. to be any element in the set For each k in S, we have p k M. Note that p p p p p M 0 3

12 Discrete Random Variable Bernoulli Random Variable: Let A be an event of interest in some random experiment, e.g., a head in a coin tossing. We say that a success occurs if A occurs when we perform the experiment. The Bernoulli random variable is equal to if A occurs and zero otherwise, and its PMF is defined as follows: p if x p x p if x 0 p p, p 0 p p p 0 Examples: State of telephone at a given time that can be either free or busy. A person who can be either healthy or sick with a certain disease. The preference of a person who can be either for or against a certain political candidate.

13 Discrete Random Variable Binomial Random Variable: A biased coin is tossed independently n times. At each toss, the coin comes up a head with probability p. Let be the number of heads in n-toss sequence. We refer as a Binomial Random Variable with parameters n and p. The probability mass function of is defined as follows by using binomial probabilities: n k nk p k P k p p, k 0,,, n k n k0 n p k k nk p Normalization Property

14 Discrete Random Variable Binomial Random Variable: px(k) px(k) k k n 9, p 0.5 n50, p0.

15 Discrete Random Variable Binomial Random Variable: Example: A binary communication channel introduce a bit error in a transmission with probability p. Let be the number of errors in n independent transmission. Find the PMF of. Find the probability of one or fewer error. takes on values in the set S 0,,,, n no error and a if there is an error. P "" p, P "0" p. Each transmission results in a 0 if there is The probability of k errors in n bit transmissions is equal to sum of the probability of the error patterns those whom have k s and n k 0 s: n k nk p k P k p p, k 0,,, n k

16 Discrete Random Variable Binomial Random Variable: Example: (Continuing) The probability of one or fewer error is given as follows:

17 Discrete Random Variable Geometric Random Variable: Suppose that we repeatedly and independently toss a coin with probability of head equal to p. The Geometric Random Variable is defined as the number of tosses that we need to get a head for the first time. Since the probability of the sequence of k tails followed by a head is p probability mass function of random variable is k p, the k p k P k p p, k,, k k p k p p p p p p k k k0 More generally, geometric random variable can be interpreted as the number of independent trials needs to obtain the first success.

18 Discrete Random Variable Geometric Random Variable: Example: Let be the number of times a message needs to be transmitted until it arrives correctly at its destination. Assume that the probability of error in a single transmission is p. Find the PMF of. Find the probability that is an even number. is a discrete random variable taking on values from S,,3,. The event k occurs if the underlying experiment find k consecutive erroneous transmission ( failures ) followed by a error-free one ( success ): k p k P k P p p, k,, k k P is even p k p p p k k p p

19 Discrete Random Variable The Poisson Random Variable: Poisson random variable is related to the number of occurrence of an event in a certain time period or in a certain region in space. Number of demands for telephone connections Number of defects in a semiconductor chip Number of clutter in a radar return The Poisson random variable has a PMF given by k, 0,,, p x e k k! where lambda is a positive number characterizing the PMF. k0 k 3 e e e e k!! 3!

20 Discrete Random Variable The Poisson Random Variable: px(k) 0.3 px(k) k k 0.5 3

21 Discrete Random Variable: Functions of Random Variables Consider a probability model of today s weather, let the random variable be the temperature in degrees Celsius, and consider the transformation Y =.8 + 3, which gives the temperature in degrees Fahrenheit. In this example, Y is a linear function of, of the form Yg a b where a and b are scalars. Nonlinear function of can also be considered Result: If Y g log Yg is a function of a random variable, then Y is also a random variable, since it provides a numerical value for each outcome. Every outcome in the sample space defines a numerical value x for and hence also the numerical value y g x for Y. If is a discrete random variable with PMF PMF can be calculated using the PMF of. p x, then Y is also discrete and its

22 Discrete Random Variable: Functions of Random Variables Probability mass function of random variable Y can be obtained for any y by summing up g x y: the probabilities of all x such that py y p x x g x Example: Consider the random variable with probability mass function defined as follows: 9 if x is an integer in the range 4,4 p x 0 otherwise Let Y and find the probability mass function of Y. The possible values of Y are 0,,,3,4 To compute values of x such that y y y, i.e., SY 0,,,3,4. p y for some given value of y in this range, we must add p Y x. x over all

23 Discrete Random Variable: Functions of Random Variables Example: (Continuing) p x Y 9 py y x y y 0: y : py 0 p x p 0 py p x p p xx 0 9 xx 9

24 Discrete Random Variable: Functions of Random Variables Example: (Continuing) y : p p x p p Y xx y 3: Y p 3 p x p 3 p 3 Y xx 3 y 4: p 4 p x p 4 p 4 Y xx y,,3,4 p y 9 y 0 0 otherwise

25 Discrete Random Variable: Functions of Random Variables Example: (Continuing) Consider the random variable Z. Probability mass function of Z can be obtained by considering Z either as the square of or as the square of Y. pz z p x py y x x z y y z The possible values of Z would be z 0,,4,9,6 and its PMF is 9 z,4,9,6 pz z 9 z 0 0 otherwise

26 Discrete Random Variable: Expectation, Mean and Variance The PMF of a random variable provides us several numbers, the probabilities of all possible values of. It would be desirable to summarize this information in a single representative number. This could be accomplished by the expectation of, which is a weighted average of the possible values of. 7 6 Yi i Trial Number Figure : 50 repetitions of the experiments yielding and Y

27 Discrete Random Variable: Expectation, Mean and Variance Definition: The expected value (expectation or the mean) of a discrete random variable p x is defined by, with PMF E xp x The expected value E is defined if the above sum converges absolutely, that is x x E x p x There are random variable for which the summation given above does not converge. In such cases, we say that the expected value does not exist. In other words, when we deal with the random variables that take countably infinite number of possible values, the infinite sum xp x must be well defined. x

28 Discrete Random Variable: Expectation, Mean and Variance Example: Find the expected value of the Bernoulli random variable. p, k 0 p k E 0 p p p p, k Example: Consider two independent coin tosses, each with a 3/4 probability of a head, and let be the number of heads obtained. Find the mean of. This is a binomial random variable with parameters n = and p = 3/4. Its PMF is k 0,, k0 6, k 0 p k 6 6, k 9 6, k E kp k kp k

29 Discrete Random Variable: Expectation, Mean and Variance Example: Let be a random variable with PMF expected value of. k0 k0 p k M for k 0, M. Find the M M M M M E kp k k 0 M M M M Example: A player at a fair coin tossing game pays $.50 to toss a coin three times. The player receives $ if the number of heads is, $8 if the number is 3, but nothing otherwise. Find the expected value of the reward Y. What is the expected value of the gain? 4 3 EY 0 py 0 py 8 py The expected gain is EY Let Z Y.5 E Z E Y.5 8 be a new random variable. Show that

30 Discrete Random Variable: Expectation, Mean and Variance Example: Let be the number of bytes in a message, and suppose that has a geometric distribution with parameter p. Find the mean of. can take on arbitrarily large values since,,3, S. The expected value is: k k k k E kp p p k p This expression is readily evaluated by differentiating the series k x x k0 to obtain k kx x k0 Letting x p, we obtain E p p p has a finite mean as long as p 0.

31 Discrete Random Variable: Expectation, Mean and Variance Expected Value of Functions of a Random Variable: Let be a random variable with PMF p x, and let. Then, the expected value of the random variable E g g x p x x g be a real valued function of g is given by Proof: Let Z g be a random variable that takes values from the set z, z,. The expectation rule can be rewritten by grouping the x values which are mapped to each value of z j : g x p x z j p x z j pz z j EZ x z j z, z, x gxz j z j z, z, g x z, which is The sum inside the braces is the probability of all x terms such that j j Z j P Z z p z

32 Discrete Random Variable: Expectation, Mean and Variance Expected Value of Functions of a Random Variable: Example: Let be a noise voltage that is uniformly distributed in S 3,,, 3 with p k 4for k in. Find EZ where Z. S First let us find the PMF of Z and then evaluate the expected value of Z. p Z k k,9 0 otherwise 0 EZ 9 5 Expected value of Z can be calculated by using the direct formula: EZ E k p k k

33 Discrete Random Variable: Expectation, Mean and Variance Expected Value of Functions of a Random Variable: Let be a random variable with PMF follows: p x and a new random variable Z is defined as Z ag bh c where a, b, and c are real numbers and g, and h are real valued functions. E Z E ag bh c ag k bh k c p k ag k p k bh k p k cp k k k k a g k p k b h k p k c p k k k k ae g be h c k

34 Discrete Random Variable: Expectation, Mean and Variance Expected Value of Functions of a Random Variable: Z ag bh c By setting a, b, and/or c to 0 or, one can obtain following expressions: E g h E g E h E Y E E Y E ag ae g E a ae E c E c Ec c

35 Discrete Random Variable: Expectation, Mean and Variance Expected Value of Functions of a Random Variable: Example: Let be a noise voltage that is uniformly distributed in 3,,, 3 p k 4for k in Y. Find EZ where Z Y. S S with. The noise voltage is amplified and shifted to obtain 0 E Z E Y E 0 E E 40E 00

36 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: The most important quantity associated with a random variable, other than the mean, is its variance. The variance of random variable is denoted by var and is defined as the expected value of the random variable E Since E : var E E can only take nonnegative values, the variance is always nonnegative. The variance provides a measure of dispersion of around its mean. Another measure of dispersion is the standard deviation of, which is defined as the square root of the variance and is denoted by : var

37 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Moments of a random variable with PMF power of the random variable. E kp k k k 3 3 E k p k k E k p k n n E k p k k p First moment of Second moment of Third moment of nth moment of x is defined as the expected value of the Variance of the random variable can be expressed in terms of moments as follows: var E E E E E E E E E E E E

38 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: Consider the experiment of tossing a biased coin, which comes up a head with probability p and a tail with probability p, and the Bernoulli random variable is defined with PMF p if k p k p if k 0 Find the variance of. 0 E p p p E p 0 p p var E E p p p p

39 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: (Continuing)

40 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: What is the mean and variance of the roll of a fair six-sided die where the results of rolls are viewed as a random variable? p k 6 if k,,3,4,5,6 0 otherwise E kp k k k k E k p k k k k E E n n n var 3.5 6

41 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: Calculate the mean and variance of a discrete uniform random variable whose PMF is, if k a, a,, b, b, p k ba 0, otherwise, where a and b are two integers with a b. p k ba a b k

42 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: a b E, since the PMF is symmetric around a b (Calculation HW). In order to calculate the variance of, let us define a new random variable as follows: Y a Y is a discrete uniform r.v. in the range,b a var Y var PMF of Y is just a shifted version of the PMF of. Let n b a, then var k. Note that n n n n n n E Y k n n 6 6, since the n n n n n n Y E Y EY 6 4 n b a, which yields Therefore, the desired variance is given by the above formula with var var Y b ab a

43 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: Calculate mean and variance of the Poisson random variable. k, 0,,, p k e k k! k E kp k ke ke, the k 0 term is zero! k! k k0 k0 k k k m = e e, let m k k! m! k m0 Variance of the Poisson random variable can be obtained in a similar way.

44 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: (Continuing) k k! k! k E k p x k e e k k0 k0 k k k k k k k k k e k e k! k! k! k! i0 i! j0 j! i j e e e e var E E

45 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: Find the variance of the binomial random variable with parameters n and p. We know from our earlier knowledge that the mean of the binomial random variable is np. n n n n k n! k n! k E k p p k p p k p p k 0 k k0 k! n k! k k! n k! n! n npk p p np k p p k0 k! n k! k0 k n n k n k n k n np k p p k k n p n n k n k nk nk nk 0 k0 np n p np np p k n p k k p n k var E E np np p np np p k

46 Discrete Random Variable: Expectation, Mean and Variance Variance and Moments: Example: The number N of queries arriving in t seconds at a call center is a Poisson random variable with t where is the average arrival rate in queries/second. Assume that the arrival rate is four queries per minute. Find the probability of more than four queries in 0 seconds. Find the probability of less than five queries in two minutes. 4 queries 60sec queries sec 5 First part of the example is questioning more than four queries in 0 seconds. So we have a Poisson random variable with queries k e k e 4 P N 4 PN k0 k! k0 k! In the second part of the example, the time of interest is t 0, so 50 8 queries k e k e P N k! k! k0 k0

47 Discrete Random Variable: Joint PMFs of Multiple RVs Probabilistic models often involve several random variables of interest. In a medical diagnosis context, the results of several tests may be significant In a networking context, the workloads of several gateway may be of interest All of these random variables are associated with the same experiment, sample space and probability law. Consider two discrete random variable and Y associated with the same experiment. The joint PMF of and Y is defined by p, x, y p x, y P x, Y y Y for all pairs of x, y that and Y can take. Alternative notation for joint probability: P x, Y y P x Y y P x and Y y Y

48 Discrete Random Variable: Joint PMFs of Multiple RVs The joint PMF determines the probability of any event that can be specified in terms of the random variable and Y. For example if A is the set of all pairs xy, that have a certain property, then, Y x, P Y A p y x, y A When the event A is the entire sample space S Y, then P, Y S p x, y p x, y Y Y Y x, y S x y The marginal PMFs of and Y can be calculated by using the formula: Y,,, p x p x y p y p x y Y Y Y y x

49 Discrete Random Variable: Joint PMFs of Multiple RVs Example: A packet switch has two input ports and two output ports. At a given time slot a packet arrives at each input port with probability and is equally likely to be destined to output port or. It is assumed that the each packet arrival and destining are independent. Let and Y be the number of packets destined for output ports and in a given time slot, respectively. Find the PMF of and Y. Let us first define the sample space. The outcome for the input port I j n : no packet arrival a : packet arrival destined for output port a : packet arrival destined for output port j ( j, ) can take the following values: The underlying sample space consists of the pair of input outcomes I, I S n, n, n, a, n, a, a, n, a, a, a, a, a, n, a, a, a, a The mapping between and Y, is nn, na, na, a, n a, a a, a a, n a, a a, a Y, 0,0,0 0,,0,0, 0,, 0,

50 Discrete Random Variable: Joint PMFs of Multiple RVs Example: (Continuing) P n, n p Y 0,0 There is no packet arrival for both output ports 4 P n, a py,0 P a, n 4 P a, n py 0, P n, a 4 P a, a py, One packet arrival for each port P a, a 8 P a, a p Y,0 Two packets arrival for output port 6 P a, a p Y 0, Two packets arrival for output port 6 One packet arrival for output port One packet arrival for output port

51 Discrete Random Variable: Joint PMFs of Multiple RVs Example: (Continuing) py x, y 6 y y y x 0 x x

52 Discrete Random Variable: Joint PMFs of Multiple RVs Example: (Continuing) py x, y y 6 y py y p 3 8 x x 6 x 0 Y Y, p x p x, y p y p x y x 0 0 y Y

53 0 p p p Discrete Random Variable: Joint PMFs of Multiple RVs Example: (Continuing) y x py py py

54 Discrete Random Variable: Joint PMFs of Multiple RVs Example: A random experiment consists of tossing two loaded dice and noting the pair of numbers (, Y) facing up. The joint PMF py j, k for j,,6 and k,,6 is given by the two dimensional table shown in figure below. The jk, entry of the table contains the value p j, k. Find the P Y y Y min, P min, Y 3 p j,3 py 3, k Y j3 k x

55 Discrete Random Variable: Joint PMFs of Multiple RVs Example: Find the marginal PMFs. (Continuing) 6, p j py j k k 6, p p k k 6 Y, p p k k 6 Y 3 3, p p k k 6 Y 4 4, p p k k 6 Y 5 5, p p k k 6 Y 6 6, p p k k Y , p k py j k Y j 6, p p j Y j 6 Y, p p j Y j 6 Y 3,3 p p j Y j 6 Y 4,4 p p j Y j 6 Y 5,5 p p j Y j 6 Y 6,6 p p j j Y

56 Discrete Random Variable: Joint PMFs of Multiple RVs Function of Multiple Random Variables: It is possible to generate new random variables by considering functions involving several random variables. In particular, functions of random variables and Y Z g, Y defines another random variable. Its PMF can be calculated from the joint PMF p x, y according to, pz z py x y x, y gx, y Furthermore, the expected value rule for functions naturally extends and takes the form,,, E Z E g Y g x y py x y z xy, Y

57 Discrete Random Variable: Joint PMFs of Multiple RVs Function of Multiple Random Variables: In special case where g is linear and of the form a by c, where a, b, and c are given scalars More Than Two Random Variable:, E g Y E a by c ae be Y c The Joint PMF of three random variables, Y, and Z is defined as in analogy with previous discussion,,,, for all triplets of numerical values x, yz, pyz x y z P x Y y Z z Corresponding marginal PMFs can be obtained as follows:,,,,,, p x y p x y z p x p x y z Y YZ YZ z y z

58 Discrete Random Variable: Joint PMFs of Multiple RVs Function of Multiple Random Variables: More Than Two Random Variable The expectation rule for functions takes the form,,,,,, E g Y Z g x y z pyz x y z x, y, z If g is linear and of the form a by cz d,, E g Y Z E a by cz d ae be Y ce Z d Furthermore, the above form can be generalized to n random variable case as follows: E a a a a E a E a E n n n n

59 Discrete Random Variable: Joint PMFs of Multiple RVs Function of Multiple Random Variables: More Than Two Random Variable Example: Your probability class has 300 students and each student has probability /3 of getting an A, independently of any other student. What is the mean of, the number of students that get an A? Let i if the ith student gets an A, 0 ortherwise Thus,,, n are Bernoulli random variables with common mean p 3 and variance p p 9. Sum of these random variable would give the number of the students who get an A. n 300 E E 300 i i

60 Discrete Random Variable: Joint PMFs of Multiple RVs Function of Multiple Random Variables: More Than Two Random Variable Example: Suppose that n people throw their hats in a box and then each picks up one hat at random. What is the expected value of, the number of people that get back their own hat? Let i if the ith person selects his/her own hat, 0 ortherwise, 0, and P n P n i i E i n n E E n E i n i n 0 n n n

61 Discrete Random Variable: Conditioning Conditioning a Random Variable on an Event: The conditional PMF of a random variable, conditioned on a particular event A with P A is defined by 0 p x P x A A P x A P A Event x p x A A Event A P x A P A Event x x x

62 Discrete Random Variable: Conditioning Conditioning a Random Variable on an Event: Note that the events x A are disjoint for different values of x, their union is A, and, therefore, Combining the above two formula gives us x P A P x A x p x A Example: Let be the roll of a fair die and let A be the event that the roll is an even number. Then the conditional PMF is P x and is even pa x P x roll is even P roll is even 3 if x,4,6 0 otherwise

63 Discrete Random Variable: Conditioning Conditioning a Random Variable on an Event: Example: The minute hand in a clock is spun and the outcome is the minute where the hand comes to rest. We assume that the hand is equally likely to rest at any of the minutes. Let be the hour where the hand comes to rest. Find the PMF of. Find the conditional PMF of given B first 4 hours ; given D. The sample space of the outcomes is S,,3,,60, so P k 60 for all k in S. 0 S P,,, 5 60 p i x 5 P p x 5 60 p i if x,,, 0 otherwise

64 Discrete Random Variable: Conditioning Conditioning a Random Variable on an Event: Example: (Continuing) Since B,,3,4 : p B x B P x if x,,3, otherwise x,,3,4,,3,4 P x B P P B P

65 Discrete Random Variable: Conditioning Conditioning a Random Variable on an Event: Example: (Continuing) P x D P : x and,, p D x D P D P,, 4 60 P,3, 4,5 4 if x P 6,7,8,9,0 5 if x P if x otherwise

66 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: Let and Y be two random variables associated with the same experiment. If we know that the experimental value of Y is some particular y with py y 0, this provides partial knowledge about the value of. This knowledge is captured by the conditional PMF p defined by py x y P x Y y For some fixed value of y with py y 0, Y Y x y of and Y, which is P x, Y y py x, y P Y y p y This function is a legitimate PMF: It assigns nonnegative values to each possible x. p x y add to. Values of Y py x y has the same shape as py, dividing with y py p x y is a function of x. Y x y except that it is normalized by

67 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: py x, y p x Y 3 y p 3 x Y x 3 x p x y Y 3 x p 3 x Y x 3 x

68 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: The conditional PMF is often convenient for calculation of the joint PMF, using a sequential approach and formula, or p x, y p x p y x p x y p y p x y Y Y Y Example: Professor May B. Right often has her facts wrong, and answers each of her students questions incorrectly with probability /4, independently of other questions. In each lecture May is asked 0,, or questions with equal probability /3. Let and Y be the number of questions May is asked and the number of questions she answers wrong in a given lecture, respectively. Find the joint PMF of and Y. To construct joint PMF py, P x, Y y description of the experiment and the multiplication rule p x, y p y p x y Y x y, we need to calculate all the probabilities for all combinations of values of x and y. This can be done by a sequential Y. Y Y Y

69 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: Example: (Continuing) p p p 48, Y Y p p p p p p p p p p p p 48 6 p 0 py 0 0 py 0,0 48, Y Y 0,0 Y Y, Y Y 0,0 Y Y y py 0 x, y x : Number of questions asked Y: Number of questions answered wrong

70 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: The conditional PMF can be used to calculate the marginal PMFs., p x p x y p y p x y Y Y Y y y The equation given above is identical to Total Probability Theorem Example: Consider a transmitter that is sending messages over a computer network. Let be the travel time of a given message and Y be the length of a given message. PMF of Y is given as follows: 5 6 if y 0 py y 4 6 if y 0 It is given that the travel time of a message depends on both its length Y and the congestion 4 level of the network at the time of transmission. In particular, the travel time is 0 Y secs 3 with probability, 0 Y secs with probability 3, and 0 Y secs with probability 6.

71 Discrete Random Variable: Conditioning Conditioning one Random Variable on Another: Example: (Continuing) if x 0 py x 0 3 if x 0 6 if x if x 4 0 py x 3 if x 0 6 if x 00 PMF of can be obtained by using the total probability formula p 0, p p x p y p x y Y Y y , p p p 0,

72 Discrete Random Variable: Conditioning Conditional Expectation: Let and Y be discrete random variable associated with the same experiment. The conditional expectation of given an event A with P A 0, is defined by For a function g, it is given by E A xpa x A x E g A g x pa x A The conditional expectation of given a value y of Y is defined by x E Y y xp x y x Y

73 Discrete Random Variable: Conditioning The conditional variance of given B is defined as: E B B B E B B B p B x B var E Let,,, n given event expected values: B B B B B be the partition of S, and let p x B i x B be the conditional PMF of B i. Expected value of can be calculated from the conditional n n i Bi i i Bi i x x i i x n E xp x x P B p x B P B xp x B PB E B E i i i i

74 Discrete Random Variable: Conditioning Example: Messages transmitted by a computer in Boston through a data network are destined for New York with probability 0.5, for Chicago with probability 0.3, and for San Francisco with probability 0.. The transit time of a message is random. Its mean is 0.05 secs if it is destined for New York, 0. secs if it is destined for Chicago, and 0.3 secs if it is destined for San Francisco. Find the expected value of. B i,,3 B : Messages are destined for New York Let us define events B : Messages are destined for Chicago i to form a partition of S: B 3: Messages are destined for San Francisco E B 0.05, E B 0., E B n n i Bi i i Bi i x x i i x E xp x x P B p x B P B xp x B n i E B P B secs i i

75 Discrete Random Variable: Independence Independence of a Random Variable from an Event: The random variable is independent of event A if and P x A P x P A p x P A, for all x. which is the same as requiring that the two events choice of x. x and A be independent, for any As long as the P A 0, and using the definition of conditional PMF, independence can be seen as the fulfilment of the condition p x A p x A

76 Discrete Random Variable: Independence Independence of a Random Variable from an Event: Example: Consider two independent tosses of a fair coin. Let be the number of heads and let A be the event that the number of heads is even. The PMF of is and P A 4 if x 0 p x if x 4 if x. The conditional PMF can be obtained from the definition pa x A P x and A P A if x 0 pa x A if x 0 otherwise Clearly, since p x p x A A, and A are not independent.

77 Discrete Random Variable: Independence Independence of a Random Variables: We say that two random variables and Y are independent if, p x y p x p y, for all x, y. Y This is the same as requiring that the two events every x and y. x and Y y be independent for Assume that py y 0 for all y, then the independence can be defined in terms of conditional PMF as follows: p x y p x, for all x and y. Y Intuitively, independence means that the experimental value of Y tells us nothing about the value of.

78 Discrete Random Variable: Independence Independence of a Random Variables: and Y are said to be conditionally independent, given an event A with P A 0, if, p x y A p x A p y A, for all x and y. Y A A Y A Conditional independence may not imply unconditional independence and vice versa. If and Y are independent random variables, then Proof: E EY E Y, E Y xyp x y xp x yp y E E Y Y Y x y x y

79 Discrete Random Variable: Independence Independence of a Random Variables: If and Y are independent random variables, and g. and. h are real valued functions. E g h Y E g E h Y Proof is homework!!! Consider the two independent random variable and Y, and a new random variable defined as their sum Z Y EZ E Y E EY Is this equation true for dependent random variables? var Z var Y E Y E Y E Y E E Y E E E Y E Y Y E Y E E E Y E Y E E Y E Y E E E Y E Y 0

80 Discrete Random Variable: Independence Independence of a Random Variables: var var Z E E E Y E Y var Y Variance of the sum of independent random variables is equal to the sum of their variances. Independence of a Several Random Variables: All of the above results and discussions can be extended to the case of more than two random variables. Three random variables, Y and Z are said to be independent if,, p x y z p x p y p z for all x, y, z. YZ Y Z

81 Discrete Random Variable: Independence Independence of a Several Random Variables: If, Y and Z are independent f,gy and and h.. h Z are also independent for any real valued functions f., g. g Y and hz are also independent for any real valued functions, g. and h. g Y and hz, Y are usually not independent, because they are both effected by Y., If,,, n are independent random variables, then var var var var n n

82 Discrete Random Variable: Independence Independence of a Several Random Variables: Example: We consider n independent coin tosses, with each toss having probability p of coming up a head. For each i, we let to if the ith toss comes up a head, and is 0 otherwise. i be the Bernoulli random variable which is equal Consider a new random variable which is defined as the number of heads in n tosses. is a binomial random variable and can be written in terms of Bernoulli random variables i, i,, n as follows: n By the independence of the coin tosses, the random variables,,, n are independent, and the variance of the binomial random variable can be calculated from the variances of the Bernoulli random variables. n n var var var p p np p n i i i