9.07 Introduction to Probability and Statistics for Brain and Cognitive Sciences Emery N. Brown

Transcription

1 9.07 Introduction to Probability and Statistics for Brain and Cognitive Sciences Emery N. Brown I. Objectives Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables Understand the concept of a conditional distribution in the discrete and continuous cases. Understand how to derive the distribution of the sum of two random variables. Understand how to compute the distribution for the transformation of two or more random variables. II. Conditional Distributions Just as we used conditional probabilities in Lecture 1 to evaluate the likelihood of one event given another, we develop here the concepts of discrete and continuous conditional distributions and discrete and continuous conditional probability mass functions and probability density functions to evaluate the behavior of one random variable given knowledge of another. A. Discrete Conditional Distributions If X and Y are jointly distributed discrete random variables, the conditional probability that X = x i given Y = y j is Pr( X = x i,y = y j ) Pr( X = x i Y = y j ) = Pr( Y = y j ) pxy ( x i, y j ) = p ( y ) y j (5.1) provided that py( yj) > 0. This is the conditional probability mass function of X given Y = y j. The numerator is a function of x for a fixed value of y. Similarly, we have We also have xy ( i, j ) Pr( = j = i ) p x y Y y X x = (5.) p ( x ) x i pxy ( xi, y j ) = p x ( x i )p y x ( y i x i ) = p y ( y j )p x y ( x i y j ). (5.3) Example 5.1 Melencolia I. The following discrete joint probability mass function is based on a magical square in Albert Dϋrer s engraving Melencolia I.

2 page : Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables X Y (5.4) Find Pr(Y =1 X = 3) and Pr(X = 4 Y = ) Pr( Y = 1I X = 3) Pr( Y = 1 X = 3) = Pr( X = 3) Pr(Y = 1I X = 3) = 4 Pr(Y = y i I X = 3) i=1 9 1 = / ( ) = 9 34 Pr(X = 4 I Y = ) Pr( X = 4 Y = ) = 4 Pr(X = x i I Y = ) i= = / ( ) 15 = 34 (5.5) Example 5. Suppose that each acetylcholine molecule released at the neuromuscular junction of a frog has a probability p of being defective. If the distribution of the number of molecules released in a 10 msec time interval is a Poisson distribution with parameter λ, what is the distribution of the number of defective acetylcholine molecules? Let N be the number of acetylcholine molecules released in 10 msec and let X be the number that is defective. If we assume that one molecule being defective is independent of any other being defective, we have that the conditional pmf of X given N = n is the binomial distribution And we have that n k n k Pr( X = k N = n ) = p (1 p) (5.6) k

3 page 3: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables By the Law of Total Probability (Lecture 1), we have n λ λ e Pr( N = n) =. (5.7) n! Pr( X = k ) = Pr( N = n ) Pr( x = k N = n) n=0 n λ λ e n k n k = p (1 p) n! k n= k n λ λ e n! k n k = p (1 p) n! k!( n k )! n k = (5.8) k λ n k n k λ (λ p) (1 p) = e k! ( n k)! n= k k (λ p) λ λ(1 p) = e e k! (λ p) k λ p = e k! which is a Poisson model with rate parameter λ p. B. Continuous Conditional Distribution If X and Y are continuous random variables with joint probability density f xy (, x y ), then the conditional probability density of Y given X is defined as f yx ( y x ) = f xy ( x, y ) f ( x) x (5.9) for 0 ( )< and 0 otherwise. If we define this in terms of differentials we have < f x x Pr(y < Y y + dy I x X x + dx) Pr(y < Y y + dy x X x + dx) = Pr(x X x + dx) = f xy ( x, y ) dxdy f ( ) x x dx Hence, we have f xy ( x, y) dy =. f ( x) x (5.10)

4 page 4: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables f yx ( y x) = = ( Pr( y < Y y + dy x X x + dx) lim dy dy f xy ( x, y) fx ( x ) The numerator is a function of y for a fixed value of x. Similarly, we can define fxy ( x y) = f xy ( x, y) f xy ( x, y) = f y x (y x) f x (x) f y ( y ) (5.11) and the marginal probability density of Y is fy ( y)= (y x) f (x)dx (5.1) f y x x which is the Law of Total Probability for continuous random variables. Example 4.6 (continued). Figure 4H shows the joint probability density of the two MEG sensors. Consider the distribution of the front sensor (y-axis) given that the value of the back sensor (x-axis) is 0.5. If the bivariate Gaussian probability density is a reasonable model for these data, then we can show that the conditional distribution of Y given X is the Gaussian probability density defined as We have σ y [ y µ y ρ ( x µ x )] 1 σ x f ( x) = [πσ y (1 ρ )] 1 YX y exp (5.13) σ y (1 ρ ) σ y EY ( X ) = µ y + ρ (x µ x ) σ x (5.14) Var( Y X ) = σ y (1 ρ ). Note that if ρ = 0, we have the marginal density of Y is the Gaussian density with EY ( )= µ y and Var( Y)= σ, consistent with the idea that y X and Y would be independent. Notice also that σ y (1 ρ ) σ y showing that knowledge about X reduces the uncertainty in Y.

5 page 5: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables σ y The conditional mean µ yx = µ y + ρ (x µ x ) is called the regression line of y on x. It gives the σ x best prediction of y given x. When we study regression analysis we will make this statement precise. Example 4.5 (continued). In this example we have the joint and marginal densities as λ y fxy ( x,y ) = λ e 0 x y f x = λe λ x (5.15) x ( ) 0 x The conditional densities are for 0 < x< y <, and fy ( y ) = λ y e λ y 0 y f xy ( x, y ) λ e λ y λ( y x) ( y x) = = = λe (5.16) f ( x) λ x λe f yx x λ e λ y fxy x = λ ye λ y for 0 < x y. That is, X is unifo rmly distributed on (0, y]. Note that ( y) = y 1 (5.17) f xy (, x y) = f y x (y x) f x (x) = f x y (x y) f y (y). (5.18) We can simulate data from this joint probability density by either one of the following two algorithms. Algorithm 5.1 1) Draw X from the e xponential density fx( x ). ) Draw Y from the e xponential density f ( x) on the interval [ x, ). yx y Algorithm 5. 1) Draw Y from the ga mma density fy y. ) Draw X from the u niform density on the interval [0,Y ].

6 page 6: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables We will show later in this lecture that Algorithm 5. states that conditional on (or given that) the sum of two exponential random variables which is a gamma random variable, the distribution of the first random variable is uniform. Example 5.3 Dendritic Dynamics (Lee et al., PLoSB 4():e9 (006)). To study the neuronal remodeling that occurs in the brain on a day-to-day basis, Elly Nedivi and colleagues used a multiphoton-based microscopy system for chronic in vivo imaging and reconstruction of entire neurons in the superficial layers of the rodent cerebral cortex. Over a period of months, they observed neurons extending and retracting existing branches, and in rare cases, budding new tips. Thirty-five of 59 non-pyramidal interneuron dendritic tips showed rearrangement and 0 of 14 pyramidal cell dendritic tips showed rearrangement. The objective is to analyze the probability of rearrangement for the two types of neurons. Assume the propensity to change is a probability p i (0,1), where i = 1 is the interneuron propensity and i = is the pyramidal neuron propensity. We will perform Bayesian analysis of this problem. Assume that p i has a beta distribution with parameters α and β. This will represent our knowledge of p i prior to the experiment and is called a prior distribution. Given p i and X i : Binomial ( n i, p i ) we have n i x 1 xi f x p ( x i pi ) = p i i ( 1 p i ) (5.19) i i x i for x n and f p i is the beta density i = 0,1,..., i ( ) Γ(α+ β) α 1 β 1 f( p i ) = p ( 1 p ). (5.0) Γ( α) Γ( β) i i Let s compute f( x i ) and f( p i x i ). The joint distribution is the product of the pmf of a discrete random variable x i and a continuous random variable p i and is defined as f( xi, p i ) = f ( p i ) f( x i p i ) Γ(α + β) α 1 β 1 ni x i ni xi = p i (1 p i ) p i (1 p i ) (5.1) Γ( α) Γ( β) x i Γ(α + β) Γ( n i + 1 ) α + x i 1 +β x 1 i i i = p (1 p ) n Γ( α ) Γ( β) Γ( x i + 1 )Γ(n i x i +1) i i using the fact that if x i is an integer Γ( ) x )! by Factoid 1 from the Addendum to Lecture x i = ( i 1 3 ni n i! Γ( n i +1) = = (5.) x i xi!( ni x i )! Γ(x i +1)Γ(ni xi +1)

7 page 7: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables To compute f( x i ) we consider From Lecture 3 and the Addendum to Lecture 3, we know that 1 f (x i ) = f (x i, p i )dp i. (5.3) 0 1 α+x i 1 n +β x 1 Γ(α + x i )Γ(n i + β x i ) i i p i i (1 p i ) dp i = (5.4) 0 Γ(α+ β + n i ) Hence, Γ(α+ β) Γ(n i +1) Γ( α + x i ) Γ(n i + β x i ) f( x i ) = (5.5) Γ( α) Γ( β ) Γ(x i +1) Γ(n i x i +1 ) Γ( α + β + n i ) for x i = 0,1,,..., n i. If we take f( p i ) to be the beta probability density with α = β = 1, then we have the uniform probability density on [0,1]. This means we are indifferent among all the values of p in [0,1] prior to the start of the experiment and f( x i ) simplifies to Verify this. 1 f( xi ) =. (5.6) ni +1 For the Bayesian analysis we will be interested in f( pi x i ), the most likely values of p i given the data x i for i = 1,. It is f( p i, x i ) f( p i x i ) = f( x i ) Γ(α + β)γ(n i +1 ) α + xi 1 n i + β xi 1 p i (1 p i ) Γ( α) Γ( β )Γ(x +1) Γ(n x i +1) = i i (5.7) Γ(α + β)γ(n i +1) Γ( α + x i )Γ( n i + β x i ) Γ( α) Γ( β )Γ(x i +1) Γ(n i x i +1 )Γ ( α + β + n i ) Γ(α + β + n i ) α + xi 1 n +β x 1 = pi (1 p i ) i i Γ(α + x i )Γ(n i + β x i ) which is a beta probability density with parameters α + x i and n + β x. We have now laid the i i ground work for an analysis we will be performing on these data in a few weeks.

8 page 8: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables III. Functions of Jointly Distributed Random Variables The two functions of jointly distributed random variables X and Y that are most commonly considered are the sum and quotient Z = X + Y Y Z =. X (5.8) We will consider only the sum. The quotient is discussed in Rice, pp A. Sum of Two Random Variables 1. Discrete Random Variables If X and Y are discrete random variables with joint pmf pxy) (, then Z = X + Y is the set of all events Z = z, X = x, Y = z x. Hence, or if X + Y are independent pz ( z) = p(x, z x) (5.9) x= pz ( z)= p x (x) p y (z x). (5.30) x= The sum is called the convolution of p x and p y. Example 5.4. Suppose that the number of channel openings at two distant (independent) sites along the frog neuromuscular junction in a 500 msec interval are Poisson random variables with parameters λ 1 and λ. What is the probability mass function of the total number of channel openings in a 500 msec interval at the two sites? We have X : P( λ 1 ), Y : P( λ ) and let Z = X + Y We want to consider the event n {Z = X + Y = n} = U {X = k I Y = n k}. Hence, we have k =0

9 page 9: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables + = n Pr(X Y n) = Pr(X = k I Y = n k) k=0 n = Pr(X = k ) Pr(Y = n k) k=0 n λ 1 k λ n k e λ1 e λ = k! (n k)! k=0 (5.31) = e n k n k (λ + λ ) 1 1 k=0 ( λ 1 + λ ) n λ λ k!( n k )! e n! k n k = λ λ n! 1 k!(n k)! k=0 (λ λ ) n 1 + ) = 1 + e (λ λ. n! We see that the sum of two independent Poisson random variables is again Poisson with J J parameters λ λ. In general, if Z = X j where X P( ) then Z P( λ j. We will be able 1 + j : λ j : ) j=1 j=1 to establish this latter result rigorously once we develop the concept of the moment generating function in Lecture 6. Z=X+Y Figure 5.A. Set of X and Y values considered in the computation of a convolution of X+Y.. Continuous Random Variables Now we consider in the continuous case Z = X + Y z which is the region below the line x + y = z. Thus, if the joint pdf of X and Y is f(,y x ) we have

10 page 10: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables Fz ( z ) = Pr( X + Y z) = f ( X,Y )dxdy R z z x = f x (,y)dydx (5.3) Let y = v x, then dy = dv and y = z x implies v = y+x = z x + x = z z = f(,v x x)dvdx. (5.33) If f( x, v x) dx is continuous at z, then on differentiating, we have f () z = f(, x z x) dx z (5.34) and if X and Y are independent we obtain fz () z = fx () x fy ( z x) dx (5.35) which is the convolution of f x and f y. X + Y = Z Figure 5B. Convolution of Two Uniform Random Variables Yields a Triangular Distribution. Example 5.5. Suppose X and Y are independent random variables both uniformly distributed on (0,1). Find Z = X + Y (Figure 5B).

11 page 11: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables fx ( x ) = 1 0< x <1 0 otherwise fy ( y ) = 1 0< y <1 0 otherwise (5.36) z (0,). To derive this density we must consider two cases: Case i) 0 < z 1 then x+ y z and f z f(x) f (z x)d = dx = z (5.37) z () = 1 x z 0 0 Case ii) 1 < z < then x+ y = z such that 1 < z < then we must have x (z 1,1) and 1 1 fz ()= z d x= x =1 (z 1 ) = z (5.38) z 1 z 1 f()is z the triangle probability density. X + Y = Z Figure 5C. Convolution of Two Exponential Random Variables Yields a Gamma Distribution. Example 5.4 (continued). Suppose that the length of time that each of the two ion channels is open is an exponential random variable with parameters λ 1 for the first and λ for the second. Because the channels are distant, we assume that they are independent. Find the probability density for the sum of the channel opening times if λ 1 = λ = λ (Figure 5C). Let Z = X + Y, then we have λx λ y f(, x y ) = f () x f () y = λ e e (5.39) x y for x > 0 and y > 0

12 page 1: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables z ()= x y z λx λ(z x λ ) d f z f (x) f (z x)dx = e e x 0 z = λ λz λz e dx = λ ze 0 (5.40) which is the probability density of a gamma random variable with α = and β = λ. Note that the convolution here, as in the case of the uniform random variables, smooths out the probability densities and makes values in the middle more likely (Figure. 5C). We saw the start of a similar phenomenon with the sum of two uniform random variables in Example 5.5 (Figure 5B) having a triangular density. This is a key observation for understanding the Central Limit Theorem in Lecture 7. B. General Transformations of Two Random Variables We state the general case as a Proposition. Proposition 5.1. If X and Y are continuous random variables with joint pdf f(, x y) and Y are mapped onto U and V by the transformation suppose X u v = g 1 ( x, y) = g ( x, y) (5.41) and the transformation can be inverted to obtain x y = h 1 ( u,v) = h ( u,v) (5.4) Assume that g 1 and g have continuous partial derivatives, then the Jacobian is and the absolute value of its determinant is then h 1 h1 u v Ju (,v) = h h u v Ju, ( v ) = h1 h h h 1 (5.43) u v u v

13 page 13: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables for u g x y = x ) for some (, )and 0 elsewhere. = 1 (, ) v g (,y xy fuv (, u v) = f xy (h 1 (, u v), h (, u v)) J( u, v) (5.44) Notice that we used a form of Proposition 5.1 to compute the normalizing constant for the beta distribution in the Addendum to Lecture 3. Proof (Heuristic): The function g is a one-to-one mapping of a region in the ( xy)plane, into a region in the (,)plane. uv It must be the case that the probability for any small region in the ( xy, ) plane must be equal to the probability of the corresponding small region in the (,)plane uv that it maps into under g. That is, f uv ( u, v) d( u, v) = f xy ( x, y) d( x, y) f uv ( u, v) = f xy ( x, y) d( x,y) duv) (, f uv ( u, v) = f xy ( x, y) J ( u, v) f uv ( u, v) = f xy ( h 1 ( u, v), h ( u, v)) J ( u, v) where, by analogy with the univariate change-of-variables in Lecture 4, we define Juv (, ) as the absolute value of the determinant of Juv (, )to insure that fuv (,v) u 0. Example 5.6. If X and Y are independent gamma random variables with parameters α and λ and β and λ respectively, find the joint pdf of U = X + Y and V = X (X + Y ) 1. The joint density of X and Y is given by λ α α 1 λx λ β β 1 λ y f xy, ( x, y) = x e y e Γ( α) Γ( β ) λ α+ β α 1 β 1 λ( x+ y) = x y e Γ( α) Γ( β) (5.45) Now we have 1 u= g 1 (, x y) = x + y, v= g (,y) x = x(x + y), x= h 1 (,v) u = u v, and y= h (, u v) = u( 1 v). The Jacobian is

14 page 14: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables h 1 h = v 1 = u u v h h = 1 v = u u v (5.46) The absolute value of the determinant of the Jacobian is (. Thus, Ju,v) = uv u(1 v) = u f uv ( u, v) = f xy (uv, u( 1 v))u + λ α β α 1 β 1 λu = ( uv ) [u( 1 v)] e u (5.47) Γ( α)γ( β ) λ α+ β α+ β 1 λu α 1 β 1 = u e v (1 v). Γ( α)γ( β ) Multiplying and dividing by Γ(α β yields + ) α+ β λ α+ β 1 λ u Γ( α + β ) α 1 ( β 1 f uv ( u, v) = u e v 1 Γ(α + β) Γ( α)γ(β ) v) = f ( u)f ( v) u v (5.48) We see that fu ( u ) is the probability density of a gamma random variable with parameters α + β and λ, whereas fv ()is v the probability density of a beta random variable with parameters α and β. Therefore, the joint probability density fuv (, u v) is the product of a gamma and a beta density. By Lecture 4, because fuv (,v) u = f u (u) f v (v),we conclude that u and v are independent. Notice that the arguments used in Eqs to 5.48 are essentially the arguments used to compute the normalizing constant for the beta probability density in the Addendum to Lecture 3. Example 5.7. Let X and Y be independent standard Gaussian random variables. Find the joint density of U = X and V = X + Y. We have g x ) x g (,y) = 1 (,y = x x + y (5.49) The Jacobian is x= h 1 (,v) u = u y = h (,v) u = v u

15 page 15: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables h 1 h1 = 1 = 0 u v h h = 1 = 1 u v and the absolute value of its determinant is Ju,v) ( = =1. (5.50) Therefore, f uv ( u,v) = f xy ( u, v u) J = f x (u) f y (v u) J 1 = (π ) 1 exp{ [u + (v u) ]} 1 1 = (π ) 1 exp{ [u + v uv + u ]} 1 = (π ) 1 exp{ [u + v uv]}. (5.51) Now matching up the terms on the right hand side of Eq with the terms in the joint density function of the bivariate Gaussian density in Eq. 4.3 yields µ u = µ v = 0 σσ u v (1 ρ ) 1 = 1 and σ (1 ρ ) = 1 u σ v = (5.5) 1 σ v (1 ρ ) = 1 ρ = or ρ = 1 and σ u = 1. Therefore, U and V have a bivariate Gaussian probability density with 1 µ = µ v = 0,σ u = 1,σ v = and ρ =. u Proposition 5.1 generalizes to n random variables (see Rice pp ). IV. Summary We have shown how to construct conditional pmf s and pdf s. In so doing, we have laid important groundwork for our statistical analyses. Similarly, understanding the convolution process that is necessary to compute the pmf or pdf of the sum of two independent random variables gives us key insig ht into how an important result like the Central Limit Theorem comes about.

16 page 16: Lecture 5: Conditional Distributions and Functions of Jointly Distributed Random Variables Acknowledgments I am grateful to Paymon Hosseini for making the figures, James Mutch for helpful editorial comments and Julie Scott for technical assistance. References Lee W, Huang H, Feng G, Sanes JR, Brown EN, So PT, Nedivi E. Dynamic remodeling of dendritic arbors in non-pyramidal neurons of adult visual cortex. Public Library of Science Biology. 4()e9, 006. Rice JA. Mathematical Statistics and Data Analysis, 3 rd edition. Boston, MA.

17 MIT OpenCourseWare Statistics for Brain and Cognitive Science Fall 016 For information about citing these materials or our Terms of Use, visit: