Probability Foundation for Electrical Engineers Prof. Krishna Jagannathan Department of Electrical Engineering Indian Institute of Technology, Madras

Transcription

1 (Refer Slide Time: 00:23) Probability Foundation for Electrical Engineers Prof. Krishna Jagannathan Department of Electrical Engineering Indian Institute of Technology, Madras Lecture - 22 Independent Random Variables 2 Welcome back. So, we were discussing conditional PMF. We said if X and Y are discrete random variable conditional PMF PX comma Y of X PX given Y of x given y is defined as PX comma Y of x comma y over PY of y, right. This is just probability that X is equal to x, y is equal to y Over probability of y is equal to assuming. So, this solve assume that this is positive. So, X and Y discrete random variable. So, they take some countable set of values there right and you what doing essentially is that your fixing let say a value particular value Y right. And asking what condition on Y equal to Y what is the conditional pm f of X right. So, your just scaling this joint for the given Y divided by. So, if by the probability that Y equal to Y right, and if a given a joint PMF you can find the marginal by this starting over all X right. So, this guide simply some over all X of PXY of XY right. That is the denominator. So, if you given the joint disjoint PMF we can find the conditional PMF and this is this as to be possible to only then is the conditional PMF defined. So, we discuss this last class for the Any questions on this? So, know state the thing let X and Y re discrete random

2 variable then the following are equivalent. (Refer Slide Time: 03:52) Therefore for all XY and R such that, PY of Y is greater than 0; we have PX 1 Y of y equals PX of x. So, this a this theorem is actually fairly simple theorem to proof it this it gives equivalent condition for discrete random variables X and Y being independent. So, we already defend independents of random variables X and Y as just corresponding sigma algebras there in sigma algebras being independent. So, we know at this means and this for the specific case when X and Y are both discrete this is during for their condition for equivalent conditions which are easy are to verify often for discrete random variable. Other than go hunting for sigma X and sigma Y and poof there are independent sigma algebras. The equivalent condition you can verify which i this is enough to say that X and Y are independent. So, the following are equivalent meaning that. So, the 4 statement said are equivalent statements. So, what is that mean. So, so what the first of all if I say to 2 statement are equivalent what to that what is it mean? So, if a stay statement A and statement A are equivalent A implies B and B implies A correct. So, if I say that to statements are equivalent how t proof 2 results this direction and this direction

3 right. So, here and saying 4 statements are equivalent which means I should proof how many statements? Supposed 12. So, 12 statements are proof right is that true? So, I for so I can pick any 4 choose way of picking the statements right 6 statements and have to proof to theorem both ways rights to have to proof 12 theorems that was results in this right. But often you do not have to that is why logically that is what we have to do right; we have to prove that this implies this. This implies this right this implies that this is the third 1 and so on right. We have to exhaust 12 statements, but of an what you can do is you can prove a circular 1 implies 2 implies 3 implies 4 implies 1 right. That kind of a within is also find right often you may be able to get away with verifying a smaller substantive of this 12 statements right. So, in this case actually see is some this a trivial right; the equivalents of 2 and 3 is quite trivial right. So, the we are saying this says that the event X is equal to X and Y equal to Y are independent events. And the equivalents to the PMF factorizing is very obvious right it is all most like from the definition it for all right from the definition right. In fact the equivalents of 2 3 and 4 is very easy. That that I will not do its very easy to verify.

4 (Refer Slide Time: 08:05) Proof: equivalence of 2 3 and 4 the easy, I will not d it youth it is all most a definition. So, if you by the are equivalent how many statements we know now what you have to of proof. It is enough to proof that 1 implies any 1 of these guides right and converse the if assume any 1 of this it implies 1 that would be enough right. Any question? So, let is proof that 1 implies 2. How do you proof 1 implies 2? So, you are assuming that X and Y are independent right and you want prove that these 2 are independent events right. How do you prove that? The sigma algebras if X and Y are independent. So, the sigma algebras if X and Y are independent sigma algebras there rated by X and Y are independence sigma algebras, which means for any Borel sets for any Borel B1 and B2. We have P of X belong in to B1 and Y belong in to B2 is equal to PX belong in to B 1 times PX B Y belong in to B2. So, these are Borel sets in R. So, they are for any Borel sets B1 and B2 in R, you must have this reduced we are independents sigma algebras means correct. Now, what how do you know what to do you next? Take single tone take B1 as the single tone X and B2 as a single tone why right. Now, take B1 as single the X and B2 as single tone Y y right. That is been end of it,

5 because then you will have probably the X is equal to X and Y is equal to Y will be will product out which means that is to are independent events right? See when a right come of a inter section that I already is mention I think alright and that is it process over right. So, 1 this direction is very easy. Now, have to proof that 1 of this rate is assume either 2 or 3 or 4 you are you get 1 right. So, assume that now let say 2 holds right then you have to proof 1 alright. The procedure is follows again B1 B2 are Borel sets an R; Borel sets B1 B2 we have probability of X belongs to B1 and Y belongs to B2. So, this is equal to So, I am going to take, so this can be return as a summation is it not it summation X belongs t B1 Y belongs to B2 probability X. X is equal to x Y equal to y and you summing over all. So, you are looking at the problem X is in B1 and Y is in t B2. So, you are looking at summing over all the masses that at sitting inside B1 and B2 respectively correct. That relation is ok? This is just saying that if I know the joint PMF, I know the joint when is a saying I know the joint law right X at X accept the Borel set here is of the is the Cartesian product of 2 oral sets on R right B1 cross B2. (Refer Slide Time: 14:21) Now, so this is equal to I am assuming to then it So, this equal to. So, this is will be equal to probability of X is equal to x times probability of Y is equal to y, because I am

6 assuming 2 right. Now, this will become summation X belongs to B1 probability X is equal to x times summation Y belongs to B2 probability Y equal to y. How it is work? Because see I now, you have started is right. So, this X summation only the this Y summation only the that right clear and you have what you want right. So, this is equal to probability that, so this X is a discrete random variable now right. So, this is equal to what probability? That is probability that Y belongs to B2 correct. And this is through for any Borel sets B1 and B2 right which means the sigma of a X and sigma of Y are independent sigma algebras which means X and Y are independent. So, this is through for all Borel B1 and B2 implies X Y are independent. So, are there any questions on this? So, what we are said this that if you want to for if you want talk about independents of discrete random variable X and Y. It is good enough t very find that the joint PMF factorizes in to the marginal s right. The equivalent saying X and Y are independent random variables and a conditional PMF is something you can calculate if you are given only the joint. Because if are the joint you can get the marginal and divided on by the error we get the conditional PMF. And this conditions of simply says if your given if the conditional PMF same as the unconditional PMF also saying X and Y are independent. There equivalent the very easy step right; this is as elementary as gets this create random variables are very easy. Now, if you move a 1 to continues random variables. So, in the discrete case, we said if X takes f X x discrete, then a takes only countable value on R with the probability 1. Similarly, Y takes countable set of values and are with the probability 1 then the X comma Y necessarily takes only countable values on R2. That is because Cartesian product of 2 countable sets in necessarily a countable set. However, in that countable case in continues case this is not so simple if X is a continues random variable are Y is a continues random variable. It need not be true that X and Y are jointly continuous first of all have on told you what jointly continuous is which viable.

7 But, what you suspect it will be remember we say that we said X as a continues random variable if the machine PX is absolutely continuous, with respect to lay break less on R. In other word, it is all length 0 sets have 0 probability measure the right. So, if you talking about random variables X and Y mapping to R to mapping omega to R to what should the equivalent definition of a continues jointly continues X Y be exactly right? So, it should they just it should just be that. So, you have 2 measures on are 2 which is your PX comma Y the joint law PX comma Y induce by the random variable X and Y. And you always absolutely have big measures which is area 1 or 2 right. Now, if are every 0 areas at 0 big measure set on are 2 you have the joint law is 0 right. Then you say X comma Y is a jointly continues jointly continues random variables right. But, 1 I am saying this that it is not through if X is continues and Y is continues it is not necessarily through that X comma Y is jointly continues. It is in the discrete case if X and Y are separately discrete there jointly discrete right. So, let me define this. So, you have just remain do you have a probabilities space omega P and you have your are X comma Y mapping omega on to R2. And we said the at if X is X and Y are random variables then all Borel sets on are will have free measures which are rough measurable that we actually be a homework. You have to proof this right, the free measures of Borel sets on are to a is a necessarily F measurable. Now, what I am saying this that a So, there is 2 measures here 1 is the measure of pushed by the random variables which is PXY the other measure is a big.

8 (Refer Slide Time: 21:10) That is called it lambda for the want to better name. So, what we want is the following definition X and Y are jointly continuous if the joint law PXY. Where PXY is absolutely continuous with respect to the league measure on R2; strictly speaking I should say they let like measure on the Borel sigma algebras on to right. Absolutely continuous means that if you take any Borel set on r 2 which as 0 area 0 league measure then the corresponding probability law PXY must also be 0 right is at clear. So, if you give me any Borel set n as that lambda when is 0 then I must start PXY is n is also 0. Any question?

9 (Refer Slide Time: 23:07). If you just an electrician of what we said in the 1 dimension case right. Caution if X and Y are continuous random variables X and Y need not be jointly continuous. So, if X and Y are separately discrete in necessarily means that the joint measure X Y will also take only countable values the rights. So, X and Y jointly discrete, but in the continuous case if X and Y are separately continuous it does not mean that X comma Y is jointly continuous. Can you see why roughly you are right, you are talking about. So, X and Y are separately continuous means that there joint see their marginal laws are absolutely continuous with respect to league measure on R right. But that does not imply that the joint laws absolutely continuous with respect to league measure on R2 right. So, if you we can just take any simple example you want let say. So, you just to lets a X is Gaussian. Remember this Gaussian with new equal to 0 sigma equal to 1 right. So, this is denoted by n 0 1 remember this example. So, this you can take anything you want right this n giving n example. Let say an Y say let Y equal to 2 X x is some standard Gaussian. You can take standard exponential for all equal for all a care you can take any continuous random variable you want an take Y equal to 2XY will also be a continuous random variable. In fact, little be Gaussian with parameters 0 and 4 you can

10 show we will see that later. So, then Y is distributed like n 04 which is also continuous random variable alright, but what you see is that this guide this Y this measure exists this is your sample. But, your measure exist is only 1 the line Y equal to X right all the measure is sitting on their line right. All omegas your necessarily map to that line Y equal to X see Y the X of omega was whatever it is Y of omega because is necessarily twice of X of omega by definition right. So, all the measure will exists on that line with me, but this line has 0 lipase measure it as 0 area right the line as 0 area you can actually show integrally. If you be considered the line as some countable inter section of small rectangles in. So, what and proof that this line as 0 lipase measure. And so you have all the probability measure of 1 is sitting on a set of lipase measure 0 1 R2 right. So, this is an example to show that even if X and Y are separately continuous random variables they need not be jointly continuous according to a definition right. So, saying that X and Y are jointly continuous is a strictly stronger condition then X and Y R. So, a award will show is that X and Y are jointly continuous they have to be marginally continuous as well that a something you will show very shown very soon. So, joint continuity is a stronger condition then being marginally continuous X and Y being marginally continuous. Is that clear? Any question? No, see Y is it true X is some Gaussian and Y is equal 2 X; it also Gaussian Y is also Gaussian we can show. What I am saying is that this is you sample space all omegas necessarily map t this line correct it that is to clear everybody right. Because, if X of omegas whatever it is Y of omega has to be twice X of omega. So, all your measure has to sit on that line which as lipase measure is 0. So, it is not jointly continuous there. So, know suppose that X and Y are. In fact, jointly continuous random variable which means that PXY is absolutely continuous with respect to lambda. Now, what kind you say no you can in work what can in work know you can in work rate on equal in theorem. The rights at rate on theorem holds for very unhold for obituary X spaces sigma finite measure. So, with every general theorem that is what I met to say that to So, PXY can be return as the integral of some non negative measurable function the lambda on R2.

11 (Refer Slide Time: 29:32) So, theorem in theorem only imply the following read on in theorem right that. So, if X comma Y are jointly continuous random variables theorem when imply that there exists a measurable function; non negative measurable function FXY which maps R2 to 0 infinity. Such that, for any Borel b on R2 we have the joint law PXY of B given by the integral lower B FXY d lambda, this is lambda is on measure on the plane R2. Actually if you profile, you can just right this is dx dy in nothing wrong righting it is dx dy it. So, I mean again this is an integral you will not fully understand now this is liable integral again you can guess think of b is being some box right few want some rectangle as. So, that probability of X and Y mapping to a box is integral over that box FXY dx dy. And this is saying there exist such on negative function and in particular taking what d you know you will take generating class right. So, what is the generating class for Borel sigma algebras R2 semi infinite rectangles right. So, taking d equal to minus infinity X cross minus infinity Y minus infinity Y we have. Then PXY of at set like that will be nothing, but what the joint c d f does not it we have f XY of X comma Y which is nothing, but the probability that X is less than equal to XY is less than equal to Y equal to integral minus infinity to X integral minus infinity to y FXY dy dx.

12 It should right may be dfxy of XY FXY of some lets a R comma X something like that because, the same variables FXY of s comma t d tdx that does makes it and not writing dx Dy. Because, I mean that that guise sitting outside right they cannot integrate with. So, let on ride on equivalent theorem upon that the c d f can be return as the integral over the semi infinite rectangles of some non negative function. Again just like in the single dimensional case this as this is called probability density function accept now it is a joint PDF joint probability density function this a this is called joint probability density function. Cross X and Y it some non negative measurable function again, it has know interpretation of probability what? So, have a only if integrity over Borel set do get a probability and also this just like in 1 day measure case this PDF is only uniquely specified up to a set of Lebage measure 0. So, only any questions in this? So, well this is a point will you appreciate later I will just saying that see the c d f or the probability law can be return as the integral of some non negative measurable function. With respect to the Lebage measure all ever saying the that this joint PDF is uniquely specified only up to a set of Lebague measure 0 which is the function on some 1 or 2 points are a countable. Points are some 0 measure point that an really change the integral as we will see later. So, you can in it is any up to a set of measure 0 uniquely specified up to set up measure 0. So, when I mean measurable f X Y; that means, that free images of Borel set an R must be Borel set on R2. I mean when this you will understand later you can only integrate see when we do integration theory we will integrator measurable function; with respect to a measure over measurable set we will do that later. So, which is Y this you put a question mark you will get back to it in about 2 weeks, but you this understand right this guide is twist your independent re mark integral find. In fact, in this you can also it is also through that the order in which integrate will not matter, because this is a non negative function R. Then you can be shown the order in which integrate does not matter you can jolly well just right dx dt in integrate with opposite order now. So, if a give the joint PDF you can integrated and find the joint

13 CDF, but if you given the joint CDF. We can always fine the probability law right which means that a if I specify the joint PDF it is. If complete characterizations of to jointly continuous random variable right again any question on this; what you can show also is that? So, you have that relation right. So, I can now show the if I just considered the probability that X is less than equal to X suppose I only interested in probability that. (Refer Slide Time: 37:49) So, I this want compute this let us say which is the probability that X say X and Y jointly continuous. So, what will we do I know the joint CDF and you know theorem of both the c d f is you sent Y it infinity you get the marginal CDF of the other guy X right. So, if I send Y it infinity in this relation. So, this is this we can this we on show already if you send Y to infinity I will get the joint I will get the marginal this is f X comma Y. I will get the marginal of X right, this is from 1 of are properties of joint CDF is a nothing to do with continuous random variables. But, now I am going to in this relation right now I am going to use the fact that X and Y are jointly continuous. So, then I am going to right. So, I send leave it Y to infinity right.

14 As I said you can just integrated to any order you want right the I will not proof that, but just take it from many that if haven non negative function. You can changes the order of integrations nothing happens if this guide can be positive as well as negative. Then you have to a any careful about inter changing orders of integration; you cannot always integrate in any order you wish, but this guys non negative. So, this theorem know as for been theorem which we will not do in this class, which integrate in any order you wish. So, you can right this as integral right this is infinity all that right FXY of s t dt dx right and this putting Y equal to infinity I am being the little bit use I am sending. So, what I doing I am sending leave it inside 1 of the integrals right all this you can do because F is nonnegative normally you cannot do all this. But, here your here your pushing limit in to the second and first integral now what happens. So, if you look at so let us look at So, my X variable is s right for X variable is s and my first into first integral is hitting t and t is going from minus infinite to infinity. So, if I simply just consider that bit the t integral own right. What remains will be a function of s correct and I am integrating in non negative function over by infinite to infinity. So, what I am us get is an also non negative function. It should be a non negative measurable function if integrate non negative measurable function we another nonnegative measurable function. So, this is some non negative measurable function which means I can right the CDF of X as the integral from minus infinity to X of some nonnegative measurable function of s. Which means that, X is a continuous random variable for its a you can call this use. Let us call that is some let us say let say that g of s right g of s d s right and g of s is non negative and measurable. So, and I am an saying the like an right f X of X as the integral of a non negative measurable function, which means that, X is a continuous random variable right because, in fact, the rade on theorem is a if an only if statement right. If PX is continuous or new is absolutely continuous with respect to you if an only if new of we can be returns as the integral Fd new so if an only if statement.

15 (Refer Slide Time: 42:44). So, this implies x is a continuous random variable because it c d f can be return as the integral of a non negative measurable function this is. After all, I am should not be I am can jolly with calls is fx right. So, I can know that I know it must be the p d f of X right i can just call this fx of s right. So, if I am given the joint probability density function I can get the marginal probability density function by integrating out the other variable from minus infinity. So, may be here as will say there fx of X equal to integral fx of X is integral minus infinity fx comma Y of s comma t dt right. That what I am calling fx of s and because my CDF is in expressible as the integral of fx of s. This must automatically be the probability density function of X the marginal probability density function of X. So, fx and Y are jointly continuous random variables. It is the case, that they are marginally continuous will just proved it right the clear, but off course we know that X and Y of marginally continuous. They need not be jointly continuous that we have we have a counter example. So, being jointly continuous is a strictly stronger condition then the random variables being marginally continuous. Is that clear? Yes, that is correct exactly correct alright any questions. So, for how much time do have I have 5 more minutes. So, next we will

16 discuss the independents of 2 jointly continuous random variables. So, we know that see we know that independents of any 2 random variables is equivalent to the CDF factorizing right that we have establish already. So, which means that, so if fx and Y R jointly continuous and independent we must have fxy of XY must be fx of X times fy of Y. But, this is joint CDF itself can be return as integral right and the marginal CDF can also integral right. So, what we can show is that if X and Y R jointly continuous and there independent. You can show that the joint PDF must necessarily factorize and conversely you can show that if the joint PDF factorize X and Y are independent. That seems perfectly reasonable right, for jointly continuous random variable. So, that a something we will start in the next class, I do not start it now. I will stop here.