Let X and Y be two real valued stochastic variables defined on (Ω, F, P). Theorem: If X and Y are independent then. . p.1/21

Transcription

1 Multivariate transformations The remaining part of the probability course is centered around transformations t : R k R m and how they transform probability measures. For instance tx 1,...,x k = x x k convolution tx 1,...,x k = x 1,...,x k, x 1... x k x1 x 1 + x 2 tx 1,...,x k =,,..., x x k 1 x 1 + x 2 x 1 + x 2 + x 3 x x k tx 1, x 2 = x 1 x2 Interpretation Let X and Y be two real valued stochastic variables defined on Ω, F, P. Theorem: If X and Y are independent then X + Y P = XP Y P. Proof: We use the definition and observe that X + Y P = φ X, Y P = φ X, Y P = φ XP Y P = XP Y P The distribution of a sum of two independent stochastic variables is the convolution of their marginal distributions.. p.1/21. p.3/21 Convolution of probability measures Let µ and ν be probability measures on R, B. Consider the functions φ : R 2 R defined as Definition: The image measure is called the convolution of µ and ν. φx, y = x + y for x, y R 2 µ ν = φµ ν Let PrR, B be the set of probability measures on R, B. As an algebraic operation on PrR, B the convolution,, is commutative and associative. Convolutions of discrete measures Theorem: If X and Y are concentrated on Z with point probabilities PX = z = pz, PY = z = rz for all z Z then X + Y is concentrated on Z with point probabilities PX + Y = z = m= pz m rm, for all z Z. Proof: We use a classic; divide and conquer. We split the computation of PX + Y = z according to the value of Y and use σ-additivity. The probability functions is thus: PX + Y = z = m= PX = z m, Y = m= m= pz m rm.. p.2/21. p.4/21

2 Convolution of binomials Example: Let X and Y be independent and X binn, p, Y binm, p Then X + Y is concentrated on N 0 with point probabilities pi = n p i j 1 p n i+j i j m j n m = p i 1 p n+m i i j j n + m = p i 1 p n+m i i p j 1 p m j Convolution from distribution functions Theorem: Let X have distribution µ with distribution function F. Let Y have distribution ν and distribution function G. If X and Y are independent then X + Y have distribution function H given by Hz = Proof: Take z R. Then Fz y dνy = Gz x dµx. Hz = PX + Y z = X, Y P {x, y R 2 x + y z} = µ ν {x, y R 2 x + y z}. Tonellis theorem gives that for all i N 0. That is, X + Y binn + m, p Hz = µ {x R x + y z} dνy= Fz y dνy. p.5/21. p.7/21 Convolution of Poisson distribution Example: If X and Y are independent X poisλ, Y poisµ then X + Y is concentrated on N 0 with point probabilities Convolution from densities Corollary: Let X and Y be independent real stochastic variables. Assume that PX A = fx dx, PY B = gy dy for all A, B B, A B pi = λ i j µj e λ i j! j! e µ = λ + µ i e λ+µ i! = λ + µ i e λ+µ i! = λ + µi i! e λ+µ i λ i j µ j j λ + µ λ + µ λ λ + µ + µ λ + µ for all i N 0. That is, X + Y is Poisson distributed with parameter λ + µ. i. p.6/21 then X + Y has density h w.r.t. m, where hz = f gz := fz ygy dy for z R Proof: Define the measure λ = h m. This measure fulfills that λ, z] = = z z hx dx = z fx y dxgy dy= fx ygy dy dx which is in fact the distribution function for X + Y. Fz y dνy,. p.8/21

3 Convolution of normal distributions Sums of squares Example: If X and Y are independent and if Example: If X 1,...,X n are independent N0, 1-distributed variables then then X Nξ, σ 2, Y Nµ, ν 2 X + Y Nξ + µ, σ 2 + ν 2 X i 2 χ 2, df = n The important content: X + Y has a normal distribution. This holds because X 1 2,...,X n 2 are independent χ 2 -distributed each with 1 degree of freedom. By induction, if X i Nξ i, σ i 2 for i = 1,...,n are independent then n X i N ξ i, σ i 2. That is, they are Γ-distributed with shape parameter 1/2 and scale parameter 2.. p.9/21. p.11/21 Convolution of Γ-distributions Transformation effects Example: If X and Y are independent and if X Γλ, β, Y Γµ, β then X + Y Γλ + µ, β The important content: X + Y is Γ-distributed. A substantial part of probabilistic modeling consists of starting with certain assumptions on independence and the derive via transformations a range of more interesting/complicated distributions. Bold claim: All dependence and every complicated distribution is created by transformations from simple, independent variables. By induction, if X i Γλ i, β for i = 1,...,n are independent then n X i Γ λ i, β.. p.10/21. p.12/21

4 Regression model Tabulation Let Z, Y 1,...,Y n, ǫ 1,...,ǫ n denote 2n + 1 independent, real stochastic variables. Define Observe that X i = Z + βy i + ǫ i. CovX i, X j = V Z, thus the variables are dependent if Z is not degenerate. We will in practice observe the X s, perhaps the Y s but neither the Z nor the ǫ s. The Z represents an unobserved, shared component in all the n measurements from the experiment and the ǫ s the unobserved, random fluctuations noise. Construct the maps t n : {1,...,N} n N 0 N as follows: n t n x 1,...,x n = 1 {1} x i,..., and define Remark: It always holds that 1 {N} x i Y 1,...,Y N = t n X 1,...,X n N Y j = n. j=1 for x 1,...,x n X n. Thus, if we know the distribution of the first N 1 of the Y s we know the last in particular, the Y s can impossibly be independent.. p.13/21. p.15/21 Tabulation Tabulation Let X 1,...,X n be independent, identically distributed stochastic variables with values in {1,...,N}. Assume PX i = j = p j for j = 1,...,N We regard the X s as classification variables. Define Y j = 1 Xi =j for j = 1,...,N, which is simply counting how many of the X s that have the value j. These Y s correspond to a tabulation of the X s. Theorem: Let X = X 1,...,X n be independent identically distributed random variables, defined on Ω, F, P, with values in {1,...,N}. Assume that PX i = j = p j, j = 1,...,N, i = 1,...,n. Let Y = Y 1,...,Y N denote the tabulation of the X s then PY = y = n! N j=1 y j! for y = y 1,...,y N N N 0 with N j=1 y j = n. N j=1 p y j j. Note the Y j Binn, p j. Challenge: What is the joint distribution of Y 1,...,Y N?. p.14/21. p.16/21

5 Tabulation Re-labeling and collapsing Proof: With y = y 1,...,y N we find that PY 1 = y 1,...,Y N = y N = Pt n X = y For any multinomial distribution we can always invent an underlying sequence X 1,...,X n of independent variables with values in {1,...,N} whose tabulation gives this multinomial distribution. If s : {1,...,N} {1,...,K} where t n 1 y = n! N j=1 y j! = n. y 1...y N is surjective, this is a re-labeling and collapsing of the categories and sx 1,...,sX n are independent. Defining q i = p j, i = 1,...,K j:sj=i we observe that this re-labeling and collapsing of a multinomial distribution results in a multinomial distribution with parameters n, q 1,...,q K.. p.17/21. p.19/21 The multinomial distribution Definition: The multinomial distribution on SN, n = {x 1,...,x N N N 0 x x N = n} has point probabilities n N px = x 1...x N j=1 p j x j for x = x 1,...,x N SN, n. From the multinomial formula we have that px = p p N n = 1. x SN,n Convolution Same idea for convolution. If Y 1 and Y 2 are two multinomial distributions with parameters n, p 1,...,p N and n, p 1,...,p N let X 1,...,X n, X n+1,...,x n+n be independent, identically distributed with values in {1,...,N} and px i = j = p j, j = 1,...,N. Then Y 1 has the same distribution as the tabulation of X 1,...,X n and Y 2 has the same distribution as the tabulation of X n+1,...,x n+n. In conclusion, Y 1 + Y 2 has the same distribution as the tabulation of X 1,...,X n+n, thus a multinomial distribution with parameters n + n, p 1,...,p N.. p.18/21. p.20/21

6 Covariance If Y = Y 1,...,Y N has a multinomial distribution with parameters n, p 1,...,p N then using the representation as tabulation without even mentioning it CovY i, Y j = Cov 1 {i} X k, r = k k 1 {j} X r PX k = i, X k = j PX k = ipx k = j = np i p j, for i j where we used independence of the X s for the second equality. For i = j we find V Y i = np i 1 p i. The correlations become np i p j corry i, Y j = npi 1 p i np j 1 p j = p i p j 1 p i 1 p j.. p.21/21