Section 8.1. Vector Notation
|
|
- Gwendoline Blair
- 5 years ago
- Views:
Transcription
1 Section 8.1 Vector Notation
2 Definition 8.1 Random Vector A random vector is a column vector X = [ X 1 ]. X n Each Xi is a random variable.
3 Definition 8.2 Vector Sample Value A sample value of a random vector is a column vector x = [ x 1 x n ]. The ith component, x i, of the vector x is a sample value of a random variable, X i.
4 Random Vectors: Notation Following our convention for random variables, the uppercase X is the random vector and the lowercase x is a sample value of X. However, we also use boldface capitals such as A and B to denote matrices with components that are not random variables. It will be clear from the context whether A is a matrix of numbers, a matrix of random variables, or a random vector.
5 Definition 8.3 Random Vector Probability Functions (a) The CDF of a random vector X is F X (x) = F X1,...,X n (x 1,..., x n ). (b) The PMF of a discrete random vector X is P X (x) = P X1,...,X n (x 1,..., x n ). (c) The PDF of a continuous random vector X is f X (x) = f X1,...,X n (x 1,..., x n ).
6 Definition 8.4 Probability Functions of a Pair of Random Vectors For random vectors X with n components and Y with m components: (a) The joint CDF of X and Y is F X,Y (x, y) = F X1,...,X n,y 1,...,Y m (x 1,..., x n, y 1,..., y m ) ; (b) The joint PMF of discrete random vectors X and Y is P X,Y (x, y) = P X1,...,X n,y 1,...,Y m (x 1,..., x n, y 1,..., y m ) ; (c) The joint PDF of continuous random vectors X and Y is f X,Y (x, y) = f X1,...,X n,y 1,...,Y m (x 1,..., x n, y 1,..., y m ).
7 Example 8.1 Problem Random vector X has PDF f X (x) = 6e a x x 0, 0 otherwise, (8.1) where a = [ ]. What is the CDF of X?
8 Example 8.1 Solution Because a has three components, we infer that X is a three-dimensional random vector. Expanding a x, we write the PDF as a function of the vector components, f X (x) = 6e x 1 2x 2 3x 3 x i 0, 0 otherwise. (8.2) Applying Definition 8.4, we integrate the PDF with respect to the three variables to obtain F X (x) = (1 e x 1)(1 e 2x 2)(1 e 3x 3) x i 0, 0 otherwise. (8.3)
9 Quiz 8.1 Discrete random vectors X = [ ] x 1 x 2 x 3 and Y = [ ] y 1 y 2 y 3 are related by Y = AX. Find the joint PMF P Y (y) if X has joint PMF P X (x) = (1 p)p x 3 x 1 < x 2 < x 3 ; x 1, x 2, x 3 {1, 2,...}, and A = 0 otherwise,
10 Quiz 8.1 Solution By definition of A, Y 1 = X 1, Y 2 = X 2 X 1 and Y 3 = X 3 X 2. Since 0 < X 1 < X 2 < X 3, each Y i must be a strictly positive integer. Thus, for y 1, y 2, y 3 {1, 2,...}, P Y (y) = P [Y 1 = y 1, Y 2 = y 2, Y 3 = y 3 ] X 1 = y 1, = P X 2 X 1 = y 2, X 3 X 2 = y 3 X 1 = y 1, = P X 2 = y 2 + y 1, X 3 = y 3 + y 2 + y 1 = P X (y 1, y 2 + y 1, y 3 + y 2 + y 1 ) = (1 p) 3 p y 1+y 2 +y 3. (1) With a = [ ] and q = 1 p, the joint PMF of Y is P Y (y) = qp a y y 1, y 2, y 3 {1, 2,...}, 0 otherwise.
11 Section 8.2 Independent Random Variables and Random Vectors
12 Definition 8.5 Independent Random Vectors Random vectors X and Y are independent if Discrete: P X,Y (x, y) = P X (x)p Y (y) Continuous: f X,Y (x, y) = f X (x)f Y (y).
13 Example 8.2 Problem As in Example 5.22, random variables Y 1,..., Y 4 have the joint PDF f Y1,...,Y 4 (y 1,..., y 4 ) = 4 0 y 1 y 2 1, 0 y 3 y 4 1, 0 otherwise. (8.4) Let V = [ Y 1 ] [ Y 4 and W = Y2 ]. Y 3 Are V and W independent random vectors?
14 Example 8.2 Solution We first note that the components of V are V 1 = Y 1, and V 2 = Y 4. Also, W 1 = Y 2, and W 2 = Y 3. Therefore, 4 0 v 1 w 1 1; f V,W (v, w) = f Y1,...,Y 4 (v 1, w 1, w 2, v 2 ) = 0 w 2 v 2 1, (8.5) 0 otherwise. Since V = [ ] Y 1 Y 4 and W = [ ], Y 2 Y 3 f V (v) = f Y1,Y 4 (v 1, v 2 ), f W (w) = f Y2,Y 3 (w 1, w 2 ). (8.6) In Example 5.22, we found the marginal PDFs f Y1,Y 4 (y 1, y 4 ) and f Y2,Y 3 (y 2, y 3 ) in Equations (5.78) and (5.80). From these marginal PDFs, we have Therefore, f V (v) = f W (w) = { 4(1 v1 )v 2 0 v 1, v 2 1, 0 otherwise, { 4w1 (1 w 2 ) 0 w 1, w 2 1, 0 otherwise. (8.7) (8.8) { 16(1 v1 )v 2 w 1 (1 w 2 ) 0 v 1, v 2, w 1, w 2 1, f V (v) f W (w) = 0 otherwise, which is not equal to f V,W (v, w). Therefore V and W are not independent. (8.9)
15 Quiz 8.2 Use the components of Y = [ Y 1,..., Y 4 ] in Example 8.2 to construct two independent random vectors V and W. Prove that V and W are independent.
16 Quiz 8.2 Solution In the PDF f Y (y), the components have dependencies as a result of the ordering constraints Y 1 Y 2 and Y 3 Y 4. We can separate these constraints by creating the vectors V = The joint PDF of V and W is [ ] Y1 Y 2 f V,W (v, w) =, W = 4 [ ] Y3 Y 4 0 v 1 v 2 1; 0 w 1 w 2 1, 0 otherwise.. (1) We must verify that V and W are independent. For 0 v 1 v 2 1, f V (v) = = f V,W (v, w) dw 1 dw 2 ( w 1 4 dw 2 ) dw 1 = 1 (2) 0 4(1 w 1) dw 1 = 2. (3) [Continued]
17 Quiz 8.2 Solution (Continued 2) Similarly, for 0 w 1 w 2 1, f W (w) = = f V,W (v, w) dv 1 dv 2 ( 1 1 It follows that V and W have PDFs f V (v) = f W (w) = 0 v 1 4 dv 2 ) 2 0 v 1 v 2 1, 0 otherwise. 2 0 w 1 w 2 1, 0 otherwise. dv 1 = 2. (4) It is easy to verify that f V,W (v, w) = f V (v)f W (w), confirming that V and W are independent vectors. (5) (6)
18 Section 8.3 Functions of Random Vectors
19 Theorem 8.1 For random variable W = g(x), Discrete: P W (w) = P[W = w] = P X (x) Continuous: F W (w) = P[W w] = x:g(x)=w g(x) w f X (x) dx 1 dx n.
20 Example 8.3 Problem Consider an experiment that consists of spinning the pointer on the wheel of circumference 1 meter in Example 4.1 n times and observing Y n meters, the maximum position of the pointer in the n spins. Find the CDF and PDF of Y n.
21 Example 8.3 Solution If X i is the position of the pointer on spin i, then Y n = max{x 1, X 2,..., X n }. As a result, Y n y if and only if each X i y. This implies F Yn (y) = P [Y n y] = P [X 1 y, X 2 y,... X n y]. (8.10) If we assume the spins to be independent, the events {X 1 y}, {X 2 y},..., {X n y} are independent events. Thus F Yn (y) = P [X 1 y] P [X n y] = (P [X y]) n = (F X (y)) n. (8.11) Example 4.2 derives Equation (4.8): F X (x) = 0 x < 0, x 0 x < 1, 1 x 1. (8.12) Equations (8.11) and (8.12) imply that the CDF and corresponding PDF are F Yn (y) = 0 y < 0, y n 0 y 1, 1 y > 1, f Yn (y) = ny n 1 0 y 1, 0 otherwise. (8.13)
22 Theorem 8.2 Let X be a vector of n iid continuous random variables, each with CDF F X (x) and PDF f X (x). (a) The CDF and the PDF of Y = max{x 1,..., X n } are F Y (y) = (F X (y)) n, f Y (y) = n(f X (y)) n 1 f X (y). (b) The CDF and the PDF of W = min{x 1,..., X n } are F W (w) = 1 (1 F X (w)) n, f W (w) = n(1 F X (w)) n 1 f X (w).
23 Proof: Theorem 8.2 By definition, F Y (y) = P[Y y]. Because Y is the maximum value of {X 1,..., X n }, the event {Y y} = {X 1 y, X 2 y,..., X n y}. Because all the random variables X i are iid, {Y y} is the intersection of n independent events. Each of the events {X i y} has probability F X (y). The probability of the intersection is the product of the individual probabilities, which implies the first part of the theorem: F Y (y) = (F X (y)) n. The second part is the result of differentiating F Y (y) with respect to y. The derivations of F W (w) and f W (w) are similar. They begin with the observations that F W (w) = 1 P[W > w] and that the event {W > w} = {X 1 > w, X 2 > w,... X n > w}, which is the intersection of n independent events, each with probability 1 F X (w).
24 Theorem 8.3 For a random vector X, the random variable g(x) has expected value Discrete: E[g(X)] = x 1 S X1 x n S X n g(x)p X (x) Continuous: E[g(X)] = g(x)f X (x) dx 1 dx n.
25 Theorem 8.4 When the components of X are independent random variables, E [g 1 (X 1 )g 2 (X 2 ) g n (X n )] = E [g 1 (X 1 )] E [g 2 (X 2 )] E [g n (X n )].
26 Proof: Theorem 8.4 When X is discrete, independence implies P X (x) = P X1 (x 1 ) P Xn (x n ). This implies E [g 1 (X 1 ) g n (X n )] = = x 1 S X1 ( x n S X n x 1 S X1 g 1 (x 1 )P X1 (x 1 ) g 1 (x 1 ) g n (x n )P X (x) (8.14) ) ( x n S X n g n (x n )P Xn (x n ) ) (8.15) = E [g 1 (X 1 )] E [g 2 (X 2 )] E [g n (X n )]. (8.16) The derivation is similar for independent continuous random variables.
27 Theorem 8.5 Given the continuous random vector X, define the derived random vector Y such that Y k = ax k + b for constants a > 0 and b. The CDF and PDF of Y are ( y1 b F Y (y) = F X ( y1 b a,..., y ) n b, f a Y (y) = 1 a nf X a,..., y ) n b. a
28 Proof: Theorem 8.5 We observe Y has CDF F Y (y) = P[aX 1 + b y 1,..., ax n + b y n ]. Since a > 0, [ F Y (y) = P X 1 y 1 b,..., X n y ] ( n b y1 b = F a a X,..., y ) n b. a a (8.17) Definition 5.13 defines the joint PDF of Y, f Y (y) = n F Y1,...,Y n (y 1,..., y n ) = 1 ( y1 b y 1 y n a nf X,..., y ) n b. (8.18) a a
29 Theorem 8.6 If X is a continuous random vector and A is an invertible matrix, then Y = AX + b has PDF f Y (y) = 1 det (A) f ( X A 1 (y b) )
30 Proof: Theorem 8.6 Let B = {y y ỹ} so that F Y (ỹ) = B f Y (y) dy. Define the vector transformation x = T (y) = A 1 (y b). It follows that Y B if and only if X T (B), where T (B) = {x Ax + b ỹ} is the image of B under transformation T. This implies F Y (ỹ) = P [X T (B)] = By the change-of-variable theorem (Math Fact B.13), F Y (ỹ) = B f X T (B) f X(x) dx (8.19) ( A 1 (y b) ) det ( A 1 ) dy (8.20) where det(a 1 ) is the absolute value of the determinant of A 1. Definition 8.3 for the CDF and PDF of a random vector combined with Theorem 5.23(b) imply that f Y (y) = f X (A 1 (y b)) det(a 1 ). The theorem follows, since det(a 1 ) = 1/ det(a).
31 Quiz 8.3(A) A test of light bulbs produced by a machine has three possible outcomes: L, long life; A, average life; and R, reject. The results of different tests are independent. All tests have the following probability model: P[L] = 0.3, P[A] = 0.6, and P[R] = 0.1. Let X 1, X 2, and X 3 be the number of light bulbs that are L, A, and R respectively in five tests. Find the PMF P X (x); the marginal PMFs P X1 (x 1 ), P X2 (x 2 ), and P X3 (x 3 ); and the PMF of W = max(x 1, X 2, X 3 ).
32 Quiz 8.3(A) Solution Referring to Theorem 2.9, each test is a subexperiment with three possible outcomes: L, A and R. In five trials, the vector X = [ ] X 1 X 2 X 3 indicating the number of outcomes of each subexperiment has the multinomial PMF ( 5 ) P X (x) = 0.3 x 1 x 1, x 2, x 0.6x 2 0.1x 3. 3 We can find the marginal PMF for each X i from the joint PMF P X (x); however it is simpler to just start from first principles and observe that X 1 is the number of occurrences of L in five independent tests. If we view each test as a trial with success probability P[L] = 0.3, we see that X 1 is a binomial (n, p) = (5, 0.3) random variable. Similarly, X 2 is a binomial (5, 0.6) random variable and X 3 is a binomial (5, 0.1) random variable. That is, for p 1 = 0.3, p 2 = 0.6 and p 3 = 0.1, ( 5 P Xi (x) = p x) x i (1 p i) 5 x. (1) [Continued]
33 Quiz 8.3(A) Solution (Continued 2) From the marginal PMFs, we see that X 1, X 2 and X 3 are not independent. Hence, we must use Theorem 8.1 to find the PMF of W. In particular, since X 1 + X 2 + X 3 = 5 and since each X i is non-negative, P W (0) = P W (1) = 0. Furthermore, P W (2) = P X (1, 2, 2) + P X (2, 1, 2) + P X (2, 2, 1) = 5!0.3(0.6)2 (0.1) 2 2!2!1! = !0.32 (0.6)(0.1) 2 2!2!1! + 5!0.32 (0.6) 2 (0.1) 2!2!1! In addition, for w = 3, w = 4, and w = 5, the event W = w occurs if and only if one of the mutually exclusive events X 1 = w, X 2 = w, or X 3 = w occurs. Thus, P W (3) = P W (4) = P W (5) = (2) 3 P Xi (3) = 0.486, (3) i=1 3 P Xi (4) = 0.288, (4) i=1 3 P Xi (5) = (5) i=1
34 Quiz 8.3(B) The random vector X has PDF f X (x) = e x 3 0 x 1 x 2 x 3, 0 otherwise. (8.21) Find the PDF of Y = AX + b. where A = diag[2, 2, 2] and b = [ ].
35 Quiz 8.3(B) Solution Since each Y i = 2X i + 4, we can apply Theorem 8.5 to write f Y (y) = 1 ( y f X 2, y 2 4 2, y ) = (1/8)e (y 3 4)/2 4 y 1 y 2 y 3, 0 otherwise. (1) Note that for other matrices A, the constraints on y resulting from the constraints 0 X 1 X 2 X 3 can be much more complicated.
36 Section 8.4 Expected Value Vector and Correlation Matrix
37 Definition 8.6 Expected Value Vector The expected value of a random vector X is a column vector E [X] = µ X = [ E [X 1 ] E [X 2 ] E [X n ]].
38 Example 8.4 Problem If X = [ X 1 X 2 X 3 ], what are the components of XX?
39 Example 8.4 Solution XX = X 1 X 2 X 3 [ ] X 1 2 X 1 X 2 X 1 X 3 X1 X 2 X 3 = X 2 X 1 X2 2 X 2 X 3. (8.22) X 3 X 1 X 3 X 2 X3 2
40 Definition 8.7 Expected Value of a Random Matrix For a random matrix A with the random variable A ij as its i, jth element, E[A] is a matrix with i, jth element E[A ij ].
41 Definition 8.8 Vector Correlation The correlation of a random vector X is an n n matrix R X with i, jth element R X (i, j) = E[X i X j ]. In vector notation, R X = E [ XX ].
42 Example 8.5 If X = [ X 1 X 2 X 3 ], the correlation matrix of X is R X = E [ X 2 1 ] E [X 2 X 1 ] E [ X 2 2 E [X 1 X 2 ] E [X 1 X 3 ] ] E [X 2 X 3 ] E [X 3 X 1 ] E [X 3 X 2 ] E [ X 2 3 ] = E [ X 2 1 ] r X2,X 1 E [ X 2 2 r X1,X 2 r X1,X 3 ] r X2,X 3 ]. r X3,X 1 r X3,X 2 E [ X 2 3
43 Definition 8.9 Vector Covariance The covariance of a random vector X is an n n matrix C X with components C X (i, j) = Cov[X i, X j ]. In vector notation, C X = E [ (X µ X )(X µ X ) ]
44 Example 8.6 If X = [ X 1 X 2 X 3 ], the covariance matrix of X is C X = Var[X 1 ] Cov [X 1, X 2 ] Cov [X 1, X 3 ] Cov [X 2, X 1 ] Var[X 2 ] Cov [X 2, X 3 ] (8.23) Cov [X 3, X 1 ] Cov [X 3, X 2 ] Var[X 3 ]
45 Theorem 8.7 For a random vector X with correlation matrix R X, covariance matrix C X, and vector expected value µ X, C X = R X µ X µ X.
46 Proof: Theorem 8.7 The proof is essentially the same as the proof of Theorem 5.16(a), with vectors replacing scalars. Cross multiplying inside the expectation of Definition 8.9 yields C X = E [ XX Xµ X µ XX + µ X µ X = E [ XX ] E [ Xµ X] E [ µx X ] + E [ µ X µ X]. (8.24) Since E[X] = µ X is a constant vector, C X = R X E [X] µ X µ X E [ X ] + µ X µ X = R X µ Xµ X. (8.25) ]
47 Example 8.7 Problem Find the expected value E[X], the correlation matrix R X, and the covariance matrix C X of the two-dimensional random vector X with PDF f X (x) = 2 0 x 1 x 2 1, 0 otherwise. (8.26)
48 Example 8.7 Solution The elements of the expected value vector are E [X i ] = x if X (x) dx 1 dx 2 = 1 0 x2 0 2x i dx 1 dx 2, i = 1, 2. (8.27) The integrals are E[X 1 ] = 1/3 and E[X 2 ] = 2/3, so that µ X = E[X] = [ 1/3 2/3 ]. The elements of the correlation matrix are E [ X 2 1 E [ X2 2 E [X 1 X 2 ] = ] = ] = x2 1 f X(x) dx 1 dx 2 = x2 2 f X(x) dx 1 dx 2 = x 1x 2 f X (x) dx 1 dx 2 = x x2 1 dx 1dx 2, (8.28) x x2 2 dx 1dx 2, (8.29) x2 0 2x 1x 2 dx 1 dx 2. (8.30) These integrals are E[X 1 2 ] = 1/6, E[X 2 2 ] = 1/2, and E[X 1 X 2 ] = 1/4. [Continued]
49 Example 8.7 Solution (Continued 2) Therefore, R X = [ ] 1/6 1/4. (8.31) 1/4 1/2 We use Theorem 8.7 to find the elements of the covariance matrix. [ ] [ ] [ ] C X = R X µ X µ 1/6 1/4 1/9 2/9 1/18 1/36 X = =. (8.32) 1/4 1/2 2/9 4/9 1/36 1/18
50 Definition 8.10 Vector Cross-Correlation The cross-correlation of random vectors, X with n components and Y with m components, is an n m matrix R XY with i, jth element R XY (i, j) = E[X i Y j ], or, in vector notation, R XY = E [ XY ].
51 Definition 8.11 Vector Cross-Covariance The cross-covariance of a pair of random vectors X with n components and Y with m components is an n m matrix C XY with i, jth element C XY (i, j) = Cov[X i, Y j ], or, in vector notation, C XY = E [ (X µ X )(Y µ Y ) ].
52 Theorem 8.8 X is an n-dimensional random vector with expected value µ X, correlation R X, and covariance C X. The m-dimensional random vector Y = AX + b, where A is an m n matrix and b is an m-dimensional vector, has expected value µ Y, correlation matrix R Y, and covariance matrix C Y given by µ Y = Aµ X + b, R Y = AR X A + (Aµ X )b + b(aµ X ) + bb, C Y = AC X A.
53 Proof: Theorem 8.8 We derive the formulas for the expected value and covariance of Y. The derivation for the correlation is similar. First, the expected value of Y is µ Y = E [AX + b] = A E [X] + E [b] = Aµ X + b. (8.33) It follows that Y µ Y = A(X µ X ). This implies C Y = E [ (A(X µ X ))(A(X µ X )) ] = E [ A(X µ X ))(X µ X ) A ] = A E [ (X µ X )(X µ X ) ] A = AC X A. (8.34)
54 Example 8.8 Problem Given the expected value µ X, the correlation R X, and the covariance C X of random vector X in Example 8.7, and Y = AX + b, where A = and b = , (8.35) 2 find the expected value µ Y, the correlation R Y, and the covariance C Y.
55 Example 8.8 Solution From the matrix operations of Theorem 8.8, we obtain µ Y = [ 1/3 2 3 ] and R Y = 1/6 13/12 4/3 13/ ; C Y = 4/ /18 5/12 1/3 5/ (8.36) 1/
56 Theorem 8.9 The vectors X and Y = AX + b have cross-correlation R XY and crosscovariance C XY given by R XY = R X A + µ X b, C XY = C X A.
57 Example 8.9 Problem Continuing Example 8.8 for random vectors X and Y = AX+b, calculate (a) The cross-correlation matrix R XY and the cross-covariance matrix C XY. (b) The correlation coefficients ρ Y1,Y 3 and ρ X2,Y 1.
58 Example 8.9 Solution (a) Direct matrix calculation using Theorem 8.9 yields [ ] [ 1/6 13/12 4/3 1/18 5/12 1/3 R XY = ; C 1/4 5/3 29/12 XY = 1/36 1/3 5/12 ]. (8.37) (b) Referring to Definition 5.6 and recognizing that Var[Y i ] = C Y (i, i), we have ρ Y1,Y 3 = Cov [Y 1, Y 3 ] Var[Y 1 ] Var[Y 3 ] Similarly, ρ X2,Y 1 = Cov [X 2, Y 1 ] Var[X 2 ] Var[Y 1 ] = = C Y (1, 3) C Y (1, 1)C Y (3, 3) C XY (2, 1) C X (2, 2)C Y (1, 1) = (8.38) = 1/2. (8.39)
59 Quiz 8.4 The three-dimensional random vector X = [ X 1 X 2 X 3 ] has PDF f X (x) = 6 0 x 1 x 2 x 3 1, 0 otherwise. Find E[X] and the correlation and covariance matrices R X and C X. (8.40)
60 Quiz 8.4 Solution To solve this problem, we need to find the expected values E[X i ] and E[X i X j ] for each I and j. To do this, we need the marginal PDFs f Xi (x i ) and f Xi,X j (x i, x j ). First we note that each marginal PDF is nonzero only if any subset of the x i obeys the ordering contraints 0 x 1 x 2 x 3 1. Within these constraints, we have and and f X1,X 2 (x 1, x 2 ) = f X2,X 3 (x 2, x 3 ) = f X1,X 3 (x 1, x 3 ) = f X (x) dx 3 = 1 f X (x) dx 1 = f X (x) dx 2 = x3 x 2 6 dx 3 = 6(1 x 2 ), (1) x2 0 6 dx 1 = 6x 2, (2) x 1 6 dx 2 = 6(x 3 x 1 ). (3) In particular, we must keep in mind that f X1,X 2 (x 1, x 2 ) = 0 unless 0 x 1 x 2 1, f X2,X 3 (x 2, x 3 ) = 0 unless 0 x 2 x 3 1, and that f X1,X 3 (x 1, x 3 ) = 0 unless 0 x 1 x 3 1. The complete expressions are { 6(1 x2 ) 0 x 1 x 2 1, f X1,X 2 (x 1, x 2 ) = (4) 0 otherwise. { 6x2 0 x 2 x 3 1, f X2,X 3 (x 2, x 3 ) = (5) 0 otherwise. { 6(x3 x 1 ) 0 x 1 x 3 1, f X1,X 3 (x 1, x 3 ) = (6) 0 otherwise. [Continued]
61 Quiz 8.4 Solution (Continued 2) Now we can find the marginal PDFs. When 0 x i 1 for each x i, f X1 (x 1 ) = = 1 f X1,X 2 (x 1, x 2 ) dx 2 x 1 6(1 x 2 ) dx 2 = 3(1 x 1 ) 2. (7) f X2 (x 2 ) = = 1 f X2,X 3 (x 2, x 3 ) dx 3 x 2 6x 2 dx 3 = 6x 2 (1 x 2 ). (8) f X3 (x 3 ) = = x3 0 f X2,X 3 (x 2, x 3 ) dx 2 6x 2 dx 2 = 3x 2 3. (9) [Continued]
62 Quiz 8.4 Solution (Continued 3) The complete expressions are f X1 (x 1 ) = { 3(1 x1 ) 2 0 x 1 1, 0 otherwise. (10) f X2 (x 2 ) = f X3 (x 3 ) = { 6x2 (1 x 2 ) 0 x 2 1, 0 otherwise. { 3x x 3 1, 0 otherwise. (11) (12) Now we can find the components E[X i ] = xf X i (x) dx of µ X. E [X 1 ] = E [X 2 ] = E [X 3 ] = x(1 x) 2 dx = 1/4, (13) 6x 2 (1 x) dx = 1/2, (14) 3x 3 dx = 3/4. (15) [Continued]
63 Quiz 8.4 Solution (Continued 4) To find the correlation matrix R X, we need to find E[X i X j ] for all i and j. with the second moments: Using marginal PDFs, the cross terms are E [X 1 X 2 ] = x 1 x 2 f X1,X 2 (x 1, x 2 ), dx 1 dx 2 1 ( 1 ) = 6x 1 x 2 (1 x 2 ) dx 2 dx 1 = 0 x 1 We start E [ ] 1 X1 2 = 3x 2 (1 x) 2 dx = (16) E [ ] 1 X2 2 = 6x 3 (1 x) dx = (17) 0 E [ ] 1 X3 2 = 3x 4 dx = 3 5. (18) [x 1 3x x4 1 ] dx 1 = (19) E [X 2 X 3 ] = x 2 6x 2 2 x 3 dx 3 dx 2 = 1 0 [3x 2 2 3x4 2 ] dx 2 = 2 5. [Continued]
64 Quiz 8.4 Solution (Continued 5) E [X 1 X 3 ] = = = x 1 x 3 (x 3 x 1 ) dx 3 dx 1 x 1 ( ) (2x 1 x 3 3 3x2 1 x2 3 ) x 3=1 x 3 =x 1 dx 1 [2x 1 3x x4 1 ] dx 1 = 1/5. (20) Summarizing the results, X has correlation matrix 1/10 3/20 1/5 R X = 3/20 3/10 2/5. (21) 1/5 2/5 3/5 Vector X has covariance matrix C X = R X E [X] E [X] = [ ] 3 4 = (22) This problem shows that even for fairly simple joint PDFs, computing the covariance matrix can be time consuming.
65 Section 8.5 Gaussian Random Vectors
66 Definition 8.12 Gaussian Random Vector X is the Gaussian (µ X, C X ) random vector with expected value µ X and covariance C X if and only if ( 1 f X (x) = (2π) n/2 exp 1 ) [det (C X )] 1/2 2 (x µ X ) C 1 X (x µ X ) where det(c X ), the determinant of C X, satisfies det(c X ) > 0.
67 Theorem 8.10 A Gaussian random vector X has independent components if and only if C X is a diagonal matrix.
68 Proof: Theorem 8.10 First, if the components of X are independent, then for i j, X i and X j are independent. By Theorem 5.17(c), Cov[X i, X j ] = 0. Hence the off-diagonal terms of C X are all zero. If C X is diagonal, then C X = σ σ 2 n and C 1 X = 1/σ2 1 It follows that C X has determinant det(c X ) = n i=1 σ2 i and that (x µ X ) C 1 X (x µ X) =... 1/σ 2 n. (8.41) n (X i µ i ) 2. (8.42) From Definition 8.12, we see that ( ) 1 n f X (x) = (2π) n/2 n exp (x i µ i )/2σ i=1 σ2 i 2 i i=1 (8.43) n 1 = exp ( ) (x i µ i ) 2 /2σ 2 2πσ 2 i. i (8.44) i=1 Thus f X (x) = n i=1 f X i (x i ), implying X 1,..., X n are independent. i=1 σ 2 i
69 Example 8.10 Problem Consider the outdoor temperature at a certain weather station. On May 5, the temperature measurements in units of degrees Fahrenheit taken at 6 AM, 12 noon, and 6 PM are all Gaussian random variables, X 1, X 2, X 3, with variance 16 degrees 2. The expected values are 50 degrees, 62 degrees, and 58 degrees respectively. The covariance matrix of the three measurements is C X = (8.45) (a) Write the joint PDF of X 1, X 2 using the algebraic notation of Definition (b) Write the joint PDF of X 1, X 2 using vector notation. (c) Write the joint PDF of X = [ X 1 X 2 X 3 ] using vector notation.
70 Example 8.10 Solution (a) First we note that X 1 and X 2 have expected values µ 1 = 50 and µ 2 = 62, variances σ 2 1 = σ2 2 = 16, and covariance Cov[X 1, X 2 ] = It follows from Definition 5.6 that the correlation coefficient is ρ X1,X 2 = Cov [X 1, X 2 ] = 12.8 σ 1 σ 2 16 From Definition 5.10, the joint PDF is f X1,X 2 (x 1, x 2 ) = exp [ = 0.8. (8.46) (x 1 50) 2 1.6(x 1 50)(x 2 62) + (x 2 62) (b) Let W = [ ] X 1 X 2 denote a vector representation for random variables X 1 and X 2. From the covariance matrix C X, we observe that the 2 2 submatrix in the upper left corner is the covariance matrix of the random vector W. Thus [Continued] ].
71 Example 8.10 Solution (Continued 2) µ W = [ ] 50, C 62 W = [ ] (8.47) We observe that det(c W ) = and det(c W ) 1/2 = 9.6. From Definition 8.12, the joint PDF of W is f W (w) = 1 ( 60.3 exp 1 ) 2 (w µ W )T C 1 W (w µ W ). (8.48) (c) Since µ X = [ ] and det(cx ) 1/2 = , X has PDF f X (x) = 1 ( exp 1 ) 2 (x µ X )T C 1 X (x µ X ). (8.49)
72 Theorem 8.11 Given an n-dimensional Gaussian random vector X with expected value µ X and covariance C X, and an m n matrix A with rank(a) = m, Y = AX + b is an m-dimensional Gaussian random vector with expected value µ Y = Aµ X + b and covariance C Y = AC X A.
73 Proof: Theorem 8.11 The proof of Theorem 8.8 contains the derivations of µ Y and C Y. Our proof that Y has a Gaussian PDF is confined to the special case when m = n and A is an invertible matrix. The case of m < n is addressed in Problem When m = n, we use Theorem 8.6 to write 1 f Y (y) = det (A) f ( X A 1 (y b) ) (8.50) = exp ( 1 2 [A 1 (y b) µ X ] C 1 X [A 1 (y b) µ X ] ) (2π) n/2 det (A) det (C X ) 1/2. (8.51) In the exponent of f Y (y), we observe that A 1 (y b) µ X = A 1 [y (Aµ X + b)] = A 1 (y µ Y ), (8.52) since µ Y = Aµ X + b. [Continued]
74 Proof: Theorem 8.11 (Continued 2) Applying (8.52) to (8.51) yields f Y (y) = exp ( 1 2 [A 1 (y µ Y )] C 1 X [A 1 (y µ Y )] ) (2π) n/2 det (A) det (C X ) 1/2. (8.53) Using the identities det(a) det(c X ) 1/2 = det(ac X A ) 1/2 and (A 1 ) = (A ) 1, we can write f Y (y) = exp ( 1 2 (y µ Y ) (A ) 1 C 1 X A 1 (y µ Y ) ) (2π) n/2 det (AC X A ) 1/2. (8.54) Since (A ) 1 C 1 X A 1 = (AC X A ) 1, we see from Equation (8.54) that Y is a Gaussian vector with expected value µ Y and covariance matrix C Y = AC X A.
75 Example 8.11 Problem Continuing Example 8.10, use the formula Y i = (5/9)(X i 32) to convert the three temperature measurements to degrees Celsius. (a) What is µ Y, the expected value of random vector Y? (b) What is C Y, the covariance of random vector Y? (c) Write the joint PDF of Y = [ Y 1 Y 2 Y 3 ] using vector notation.
76 Example 8.11 Solution (a) In terms of matrices, we observe that Y = AX + b where A = 5/ /9 0, b = / (8.55) 1 (b) Since µ X = [ ], from Theorem 8.11, µ Y = Aµ X + b = 10 50/3 130/9. (8.56) (c) The covariance of Y is C Y = AC X A. We note that A = A = (5/9)I where I is the 3 3 identity matrix. Thus C Y = (5/9) 2 C X and C 1 Y = (9/5)2 C 1 X f Y (y) = exp. The PDF of Y is ( (y µ Y )T C 1 X (y µ Y ) ). (8.57)
77 Definition 8.13 Standard Normal Random Vector The n-dimensional standard normal random vector Z is the n-dimensional Gaussian random vector with E[Z] = 0 and C Z = I.
78 Theorem 8.12 For a Gaussian (µ X, C X ) random vector, let A be an n n matrix with the property AA = C X. The random vector Z = A 1 (X µ X ) is a standard normal random vector.
79 Proof: Theorem 8.12 Applying Theorem 8.11 with A replaced by A 1, and b = A 1 µ X, we have that Z is a Gaussian random vector with expected value and covariance E [Z] = E [ A 1 (X µ X ) ] = A 1 E [X µ X ] = 0 (8.58) C Z = A 1 C X (A 1 ) = A 1 AA (A ) 1 = I. (8.59)
80 Theorem 8.13 Given the n-dimensional standard normal random vector Z, an invertible n n matrix A, and an n-dimensional vector b, X = AZ + b is an n-dimensional Gaussian random vector with expected value µ X = b and covariance matrix C X = AA.
81 Proof: Theorem 8.13 By Theorem 8.11, X is a Gaussian random vector with expected value µ X = E [X] = E [AZ + µ X ] = A E [Z] + b = b. (8.60) The covariance of X is C X = AC Z A = AIA = AA. (8.61)
82 Theorem 8.14 For a Gaussian vector X with covariance C X, there always exists a matrix A such that C X = AA.
83 Proof: Theorem 8.14 To verify this fact, we connect some simple facts: In Problem , we ask you to show that every random vector X has a positive semidefinite covariance matrix C X. By Math Fact B.17, every eigenvalue of C X is nonnegative. The definition of the Gaussian vector PDF requires the existence of C 1 X. Hence, for a Gaussian vector X, all eigenvalues of C X are nonzero. From the previous step, we observe that all eigenvalues of C X must be positive. Since C X is a real symmetric matrix, Math Fact B.15 says it has a singular value decomposition (SVD) C X = UDU where D = diag[d 1,..., d n ] is the diagonal matrix of eigenvalues of C X. Since each d i is positive, we can define D 1/2 = diag[ d 1,..., d n ], and we can write We see that A = UD 1/2. C X = UD 1/2 D 1/2 U = ( UD 1/2) ( UD 1/2). (8.62)
84 Quiz 8.5 Z is the two-dimensional standard normal random vector. The Gaussian random vector X has components X 1 = 2Z 1 + Z and X 2 = Z 1 Z 2. (8.65) Calculate the expected value vector µ X and the covariance matrix C X.
85 Quiz 8.5 Solution We observe that X = AZ + b where [ ] 2 1 A =, b = 1 1 It follows from Theorem 8.13 that µ X = b and that [ ] [ ] [ C X = AA = = [ ] 2. (1) 0 ].
86 Section 8.6 Matlab
87 Example 8.12 Problem Finite random vector X = [ X 1 X 2, X 5 ] has PMF P X (x) = k x x x i { 10, 9,..., 10} ; i = 1, 2,..., 5, 0 otherwise. (8.66) What is the constant k? Find the expected value and standard deviation of X 3.
88 Example 8.12 Solution Summing P X (x) over all possible values of x is the sort of tedious task that Matlab handles easily. Here are the code and corresponding output: %x5.m sx=-10:10; [SX1,SX2,SX3,SX4,SX5]... =ndgrid(sx,sx,sx,sx,sx); P=sqrt(SX1.^2 +SX2.^2+SX3.^2+SX4.^2+SX5.^2); k=1.0/(sum(sum(sum(sum(sum(p)))))) P=k*P; EX3=sum(sum(sum(sum(sum(P.*SX3))))) EX32=sum(sum(sum(sum(sum(P.*(SX3.^2)))))); sigma3=sqrt(ex32-(ex3)^2) >> x5 k = e-008 EX3 = e-017 sigma3 = >> In fact, by symmetry arguments, it should be clear that E[X 3 ] = 0. In adding 11 5 terms, Matlab s finite precision led to a small error on the order of
89 Example 8.13 Problem Write a Matlab function f=gaussvectorpdf(mu,c,x) that calculates f X (x) for a Gaussian (µ, C) random vector.
90 Example 8.13 Solution function f=gaussvectorpdf(mu,c,x) n=length(x); z=x(:)-mu(:); f=exp(-z *inv(c)*z)/... sqrt((2*pi)^n*det(c)); gaussvectorpdf computes the Gaussian vector PDF f X (x) of Definition Of course, Matlab makes the calculation simple by providing operators for matrix inverses and determinants.
91 Quiz 8.6 The daily noon temperature, measured in degrees Fahrenheit, in New Jersey in July can be modeled as a Gaussian random vector T = [ T 1 T 31 ] where T i is the temperature on the ith day of the month. Suppose that E[T i ] = 80 for all i, and that T i and T j have covariance Cov [ T i, T j ] = i j Define the daily average temperature as (8.67) Y = T 1 + T T 31. (8.68) 31 Based on this model, write a Matlab program p=julytemps(t) that calculates P[Y T ], the probability that the daily average temperature is at least T degrees.
92 Quiz 8.6 Solution First, we observe that Y = AT where A = [ 1/31 1/31 1/31 ]. Since T is a Gaussian random vector, Theorem 8.11 tells us that Y is a 1 dimensional Gaussian vector, i.e., just a Gaussian random variable. The expected value of Y is µ Y = µ T = 80. The covariance matrix of Y is 1 1 and is just equal to Var[Y ]. Thus, by Theorem 8.11, Var[Y ] = AC T A. In julytemps.m shown below, the first two lines generate the covariance matrix CT, or C T. Next we calculate Var[Y ]. The final step is to use the Φ( ) function to calculate P[Y < T ]. function p=julytemps(t); [D1 D2]=ndgrid((1:31),(1:31)); CT=36./(1+abs(D1-D2)); A=ones(31,1)/31.0; CY=(A )*CT*A; p=phi((t-80)/sqrt(cy)); [Continued]
93 Quiz 8.6 Solution (Continued 2) Here is the output of julytemps.m: >> julytemps([ ]) ans = Note that P[T 70] is not actually zero and that P[T 90] is not actually Its just that the Matlab s short format output, invoked with the command format short, rounds off those probabilities. The long format output resembles: >> format long >> julytemps([70 75]) ans = >> julytemps([85 90]) ans = The ndgrid function is a useful to way calculate many covariance matrices. However, in this problem, C X has a special structure; the i, jth element is [Continued]
94 Quiz 8.6 Solution (Continued 3) C T (i, j) = c i j = i j. (1) If we write out the elements of the covariance matrix, we see that C T = c 0 c 1 c 30 c 1 c c 1. (2) c 30 c 1 c 0 This covariance matrix is known as a symmetric Toeplitz matrix. Because Toeplitz covariance matrices are quite common, Matlab has a toeplitz function for generating them. The function julytemps2 use the toeplitz to generate the correlation matrix C T. function p=julytemps2(t); c=36./(1+abs(0:30)); CT=toeplitz(c); A=ones(31,1)/31.0; CY=(A )*CT*A; p=phi((t-80)/sqrt(cy));
5. Random Vectors. probabilities. characteristic function. cross correlation, cross covariance. Gaussian random vectors. functions of random vectors
EE401 (Semester 1) 5. Random Vectors Jitkomut Songsiri probabilities characteristic function cross correlation, cross covariance Gaussian random vectors functions of random vectors 5-1 Random vectors we
More information3. Probability and Statistics
FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important
More informationBivariate distributions
Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient
More informationProbability and Distributions
Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated
More information2 Functions of random variables
2 Functions of random variables A basic statistical model for sample data is a collection of random variables X 1,..., X n. The data are summarised in terms of certain sample statistics, calculated as
More informationEEL 5544 Noise in Linear Systems Lecture 30. X (s) = E [ e sx] f X (x)e sx dx. Moments can be found from the Laplace transform as
L30-1 EEL 5544 Noise in Linear Systems Lecture 30 OTHER TRANSFORMS For a continuous, nonnegative RV X, the Laplace transform of X is X (s) = E [ e sx] = 0 f X (x)e sx dx. For a nonnegative RV, the Laplace
More informationMultiple Random Variables
Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An
More informationP (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n
JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are
More informationContinuous Random Variables
1 / 24 Continuous Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay February 27, 2013 2 / 24 Continuous Random Variables
More informationChapter 4. Multivariate Distributions. Obviously, the marginal distributions may be obtained easily from the joint distribution:
4.1 Bivariate Distributions. Chapter 4. Multivariate Distributions For a pair r.v.s (X,Y ), the Joint CDF is defined as F X,Y (x, y ) = P (X x,y y ). Obviously, the marginal distributions may be obtained
More informationThe Multivariate Normal Distribution. In this case according to our theorem
The Multivariate Normal Distribution Defn: Z R 1 N(0, 1) iff f Z (z) = 1 2π e z2 /2. Defn: Z R p MV N p (0, I) if and only if Z = (Z 1,..., Z p ) T with the Z i independent and each Z i N(0, 1). In this
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationIntroduction to Probability and Stocastic Processes - Part I
Introduction to Probability and Stocastic Processes - Part I Lecture 2 Henrik Vie Christensen vie@control.auc.dk Department of Control Engineering Institute of Electronic Systems Aalborg University Denmark
More informationA Probability Review
A Probability Review Outline: A probability review Shorthand notation: RV stands for random variable EE 527, Detection and Estimation Theory, # 0b 1 A Probability Review Reading: Go over handouts 2 5 in
More informationSection 9.1. Expected Values of Sums
Section 9.1 Expected Values of Sums Theorem 9.1 For any set of random variables X 1,..., X n, the sum W n = X 1 + + X n has expected value E [W n ] = E [X 1 ] + E [X 2 ] + + E [X n ]. Proof: Theorem 9.1
More informationRandom Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R
In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample
More informationBASICS OF PROBABILITY
October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,
More information4. Distributions of Functions of Random Variables
4. Distributions of Functions of Random Variables Setup: Consider as given the joint distribution of X 1,..., X n (i.e. consider as given f X1,...,X n and F X1,...,X n ) Consider k functions g 1 : R n
More informationMA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems
MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Introduction to Probability and Statistics Lecture 14: Continuous random variables Purnamrita Sarkar Department of Statistics and Data Science The University of Texas at Austin www.cs.cmu.edu/
More informationSTAT Chapter 5 Continuous Distributions
STAT 270 - Chapter 5 Continuous Distributions June 27, 2012 Shirin Golchi () STAT270 June 27, 2012 1 / 59 Continuous rv s Definition: X is a continuous rv if it takes values in an interval, i.e., range
More informationQuick Tour of Basic Probability Theory and Linear Algebra
Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions
More informationJoint Distributions. (a) Scalar multiplication: k = c d. (b) Product of two matrices: c d. (c) The transpose of a matrix:
Joint Distributions Joint Distributions A bivariate normal distribution generalizes the concept of normal distribution to bivariate random variables It requires a matrix formulation of quadratic forms,
More informationEE4601 Communication Systems
EE4601 Communication Systems Week 2 Review of Probability, Important Distributions 0 c 2011, Georgia Institute of Technology (lect2 1) Conditional Probability Consider a sample space that consists of two
More informationJoint Probability Distributions and Random Samples (Devore Chapter Five)
Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete
More informationFormulas for probability theory and linear models SF2941
Formulas for probability theory and linear models SF2941 These pages + Appendix 2 of Gut) are permitted as assistance at the exam. 11 maj 2008 Selected formulae of probability Bivariate probability Transforms
More informationRandom Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
More informationWe introduce methods that are useful in:
Instructor: Shengyu Zhang Content Derived Distributions Covariance and Correlation Conditional Expectation and Variance Revisited Transforms Sum of a Random Number of Independent Random Variables more
More informationReview of Probability Theory
Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving
More informationReview (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology
Review (probability, linear algebra) CE-717 : Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Some slides have been adopted from Prof. H.R. Rabiee s and also Prof. R. Gutierrez-Osuna
More informationPerhaps the simplest way of modeling two (discrete) random variables is by means of a joint PMF, defined as follows.
Chapter 5 Two Random Variables In a practical engineering problem, there is almost always causal relationship between different events. Some relationships are determined by physical laws, e.g., voltage
More informationREVIEW OF MAIN CONCEPTS AND FORMULAS A B = Ā B. Pr(A B C) = Pr(A) Pr(A B C) =Pr(A) Pr(B A) Pr(C A B)
REVIEW OF MAIN CONCEPTS AND FORMULAS Boolean algebra of events (subsets of a sample space) DeMorgan s formula: A B = Ā B A B = Ā B The notion of conditional probability, and of mutual independence of two
More informationECE 541 Stochastic Signals and Systems Problem Set 9 Solutions
ECE 541 Stochastic Signals and Systems Problem Set 9 Solutions Problem Solutions : Yates and Goodman, 9.5.3 9.1.4 9.2.2 9.2.6 9.3.2 9.4.2 9.4.6 9.4.7 and Problem 9.1.4 Solution The joint PDF of X and Y
More informationFinal Exam # 3. Sta 230: Probability. December 16, 2012
Final Exam # 3 Sta 230: Probability December 16, 2012 This is a closed-book exam so do not refer to your notes, the text, or any other books (please put them on the floor). You may use the extra sheets
More informationMoment Generating Function. STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution
Moment Generating Function STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution T. Linder Queen s University Winter 07 Definition Let X (X,...,X n ) T be a random vector and
More information2 (Statistics) Random variables
2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes
More informationBivariate Distributions
STAT/MATH 395 A - PROBABILITY II UW Winter Quarter 17 Néhémy Lim Bivariate Distributions 1 Distributions of Two Random Variables Definition 1.1. Let X and Y be two rrvs on probability space (Ω, A, P).
More information[POLS 8500] Review of Linear Algebra, Probability and Information Theory
[POLS 8500] Review of Linear Algebra, Probability and Information Theory Professor Jason Anastasopoulos ljanastas@uga.edu January 12, 2017 For today... Basic linear algebra. Basic probability. Programming
More informationLet X and Y denote two random variables. The joint distribution of these random
EE385 Class Notes 9/7/0 John Stensby Chapter 3: Multiple Random Variables Let X and Y denote two random variables. The joint distribution of these random variables is defined as F XY(x,y) = [X x,y y] P.
More informationMAS223 Statistical Inference and Modelling Exercises
MAS223 Statistical Inference and Modelling Exercises The exercises are grouped into sections, corresponding to chapters of the lecture notes Within each section exercises are divided into warm-up questions,
More information1 Random Variable: Topics
Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?
More informationconditional cdf, conditional pdf, total probability theorem?
6 Multiple Random Variables 6.0 INTRODUCTION scalar vs. random variable cdf, pdf transformation of a random variable conditional cdf, conditional pdf, total probability theorem expectation of a random
More informationReview (Probability & Linear Algebra)
Review (Probability & Linear Algebra) CE-725 : Statistical Pattern Recognition Sharif University of Technology Spring 2013 M. Soleymani Outline Axioms of probability theory Conditional probability, Joint
More informationUC Berkeley Department of Electrical Engineering and Computer Sciences. EECS 126: Probability and Random Processes
UC Berkeley Department of Electrical Engineering and Computer Sciences EECS 6: Probability and Random Processes Problem Set 3 Spring 9 Self-Graded Scores Due: February 8, 9 Submit your self-graded scores
More informationEC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)
1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For
More informationProblem Solutions Chapter 4
Problem Solutions Chapter 4 Problem 4.. Solution (a) The probability P [, 3] can be found be evaluating the joint CDF F, (x, y) at x andy 3. This yields P [, 3] F, (, 3) ( e )( e 3 ) () (b) To find the
More informationReview of probability
Review of probability Computer Sciences 760 Spring 2014 http://pages.cs.wisc.edu/~dpage/cs760/ Goals for the lecture you should understand the following concepts definition of probability random variables
More information1 Review of Probability and Distributions
Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote
More informationLecture 11. Probability Theory: an Overveiw
Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the
More informationReview: mostly probability and some statistics
Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random
More informationThe Multivariate Gaussian Distribution
The Multivariate Gaussian Distribution Chuong B. Do October, 8 A vector-valued random variable X = T X X n is said to have a multivariate normal or Gaussian) distribution with mean µ R n and covariance
More informationStatistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v }
Statistics 35 Probability I Fall 6 (63 Final Exam Solutions Instructor: Michael Kozdron (a Solving for X and Y gives X UV and Y V UV, so that the Jacobian of this transformation is x x u v J y y v u v
More informationwhere r n = dn+1 x(t)
Random Variables Overview Probability Random variables Transforms of pdfs Moments and cumulants Useful distributions Random vectors Linear transformations of random vectors The multivariate normal distribution
More informationIntroduction to Machine Learning
What does this mean? Outline Contents Introduction to Machine Learning Introduction to Probabilistic Methods Varun Chandola December 26, 2017 1 Introduction to Probability 1 2 Random Variables 3 3 Bayes
More informationLecture 2: Repetition of probability theory and statistics
Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:
More informationA Probability Primer. A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes.
A Probability Primer A random walk down a probabilistic path leading to some stochastic thoughts on chance events and uncertain outcomes. Are you holding all the cards?? Random Events A random event, E,
More informationProblem Set 1. MAS 622J/1.126J: Pattern Recognition and Analysis. Due: 5:00 p.m. on September 20
Problem Set MAS 6J/.6J: Pattern Recognition and Analysis Due: 5:00 p.m. on September 0 [Note: All instructions to plot data or write a program should be carried out using Matlab. In order to maintain a
More informationPCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities
PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets
More informationStatistics for scientists and engineers
Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3
More informationNotes for Math 324, Part 19
48 Notes for Math 324, Part 9 Chapter 9 Multivariate distributions, covariance Often, we need to consider several random variables at the same time. We have a sample space S and r.v. s X, Y,..., which
More informationCS145: Probability & Computing
CS45: Probability & Computing Lecture 5: Concentration Inequalities, Law of Large Numbers, Central Limit Theorem Instructor: Eli Upfal Brown University Computer Science Figure credits: Bertsekas & Tsitsiklis,
More informationLecture 25: Review. Statistics 104. April 23, Colin Rundel
Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April
More informationProblem Y is an exponential random variable with parameter λ = 0.2. Given the event A = {Y < 2},
ECE32 Spring 25 HW Solutions April 6, 25 Solutions to HW Note: Most of these solutions were generated by R. D. Yates and D. J. Goodman, the authors of our textbook. I have added comments in italics where
More informationBrief Review of Probability
Brief Review of Probability Nuno Vasconcelos (Ken Kreutz-Delgado) ECE Department, UCSD Probability Probability theory is a mathematical language to deal with processes or experiments that are non-deterministic
More informationSpring 2012 Math 541B Exam 1
Spring 2012 Math 541B Exam 1 1. A sample of size n is drawn without replacement from an urn containing N balls, m of which are red and N m are black; the balls are otherwise indistinguishable. Let X denote
More informationStat 206: Sampling theory, sample moments, mahalanobis
Stat 206: Sampling theory, sample moments, mahalanobis topology James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Notation My notation is different from the book s. This is partly because
More informationJointly Distributed Random Variables
Jointly Distributed Random Variables CE 311S What if there is more than one random variable we are interested in? How should you invest the extra money from your summer internship? To simplify matters,
More informationME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 PROBABILITY. Prof. Steven Waslander
ME 597: AUTONOMOUS MOBILE ROBOTICS SECTION 2 Prof. Steven Waslander p(a): Probability that A is true 0 pa ( ) 1 p( True) 1, p( False) 0 p( A B) p( A) p( B) p( A B) A A B B 2 Discrete Random Variable X
More informationStatistics, Data Analysis, and Simulation SS 2015
Statistics, Data Analysis, and Simulation SS 2015 08.128.730 Statistik, Datenanalyse und Simulation Dr. Michael O. Distler Mainz, 27. April 2015 Dr. Michael O. Distler
More informationACM 116: Lectures 3 4
1 ACM 116: Lectures 3 4 Joint distributions The multivariate normal distribution Conditional distributions Independent random variables Conditional distributions and Monte Carlo: Rejection sampling Variance
More informationStatistical Pattern Recognition
Statistical Pattern Recognition A Brief Mathematical Review Hamid R. Rabiee Jafar Muhammadi, Ali Jalali, Alireza Ghasemi Spring 2012 http://ce.sharif.edu/courses/90-91/2/ce725-1/ Agenda Probability theory
More informationEXPECTED VALUE of a RV. corresponds to the average value one would get for the RV when repeating the experiment, =0.
EXPECTED VALUE of a RV corresponds to the average value one would get for the RV when repeating the experiment, independently, infinitely many times. Sample (RIS) of n values of X (e.g. More accurately,
More informationAlgorithms for Uncertainty Quantification
Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example
More informationRandom Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.
Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example
More informationPreliminary statistics
1 Preliminary statistics The solution of a geophysical inverse problem can be obtained by a combination of information from observed data, the theoretical relation between data and earth parameters (models),
More information3d scatterplots. You can also make 3d scatterplots, although these are less common than scatterplot matrices.
3d scatterplots You can also make 3d scatterplots, although these are less common than scatterplot matrices. > library(scatterplot3d) > y par(mfrow=c(2,2)) > scatterplot3d(y,highlight.3d=t,angle=20)
More informationMultivariate Statistics
Multivariate Statistics Chapter 2: Multivariate distributions and inference Pedro Galeano Departamento de Estadística Universidad Carlos III de Madrid pedro.galeano@uc3m.es Course 2016/2017 Master in Mathematical
More information8 - Continuous random vectors
8-1 Continuous random vectors S. Lall, Stanford 2011.01.25.01 8 - Continuous random vectors Mean-square deviation Mean-variance decomposition Gaussian random vectors The Gamma function The χ 2 distribution
More informationSTAT/MATH 395 PROBABILITY II
STAT/MATH 395 PROBABILITY II Bivariate Distributions Néhémy Lim University of Washington Winter 2017 Outline Distributions of Two Random Variables Distributions of Two Discrete Random Variables Distributions
More information4. CONTINUOUS RANDOM VARIABLES
IA Probability Lent Term 4 CONTINUOUS RANDOM VARIABLES 4 Introduction Up to now we have restricted consideration to sample spaces Ω which are finite, or countable; we will now relax that assumption We
More informationECE 650 Lecture 4. Intro to Estimation Theory Random Vectors. ECE 650 D. Van Alphen 1
EE 650 Lecture 4 Intro to Estimation Theory Random Vectors EE 650 D. Van Alphen 1 Lecture Overview: Random Variables & Estimation Theory Functions of RV s (5.9) Introduction to Estimation Theory MMSE Estimation
More informationECE 302 Division 2 Exam 2 Solutions, 11/4/2009.
NAME: ECE 32 Division 2 Exam 2 Solutions, /4/29. You will be required to show your student ID during the exam. This is a closed-book exam. A formula sheet is provided. No calculators are allowed. Total
More informationMultivariate Distributions
IEOR E4602: Quantitative Risk Management Spring 2016 c 2016 by Martin Haugh Multivariate Distributions We will study multivariate distributions in these notes, focusing 1 in particular on multivariate
More informationChapter 3: Random Variables 1
Chapter 3: Random Variables 1 Yunghsiang S. Han Graduate Institute of Communication Engineering, National Taipei University Taiwan E-mail: yshan@mail.ntpu.edu.tw 1 Modified from the lecture notes by Prof.
More information1 Presessional Probability
1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional
More informationE X A M. Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours. Number of pages incl.
E X A M Course code: Course name: Number of pages incl. front page: 6 MA430-G Probability Theory and Stochastic Processes Date: December 13, 2016 Duration: 4 hours Resources allowed: Notes: Pocket calculator,
More informationChapter 2. Probability
2-1 Chapter 2 Probability 2-2 Section 2.1: Basic Ideas Definition: An experiment is a process that results in an outcome that cannot be predicted in advance with certainty. Examples: rolling a die tossing
More informationStat 206: Linear algebra
Stat 206: Linear algebra James Johndrow (adapted from Iain Johnstone s notes) 2016-11-02 Vectors We have already been working with vectors, but let s review a few more concepts. The inner product of two
More information1.1 Review of Probability Theory
1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,
More informationRandom Variables. P(x) = P[X(e)] = P(e). (1)
Random Variables Random variable (discrete or continuous) is used to derive the output statistical properties of a system whose input is a random variable or random in nature. Definition Consider an experiment
More informationBasics on Probability. Jingrui He 09/11/2007
Basics on Probability Jingrui He 09/11/2007 Coin Flips You flip a coin Head with probability 0.5 You flip 100 coins How many heads would you expect Coin Flips cont. You flip a coin Head with probability
More informationRecall that if X 1,...,X n are random variables with finite expectations, then. The X i can be continuous or discrete or of any other type.
Expectations of Sums of Random Variables STAT/MTHE 353: 4 - More on Expectations and Variances T. Linder Queen s University Winter 017 Recall that if X 1,...,X n are random variables with finite expectations,
More informationStat 5101 Notes: Algorithms (thru 2nd midterm)
Stat 5101 Notes: Algorithms (thru 2nd midterm) Charles J. Geyer October 18, 2012 Contents 1 Calculating an Expectation or a Probability 2 1.1 From a PMF........................... 2 1.2 From a PDF...........................
More informationTAMS39 Lecture 2 Multivariate normal distribution
TAMS39 Lecture 2 Multivariate normal distribution Martin Singull Department of Mathematics Mathematical Statistics Linköping University, Sweden Content Lecture Random vectors Multivariate normal distribution
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationGeneral Random Variables
1/65 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Probability A general random variable is discrete, continuous, or mixed. A discrete random variable
More informationRecitation 2: Probability
Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions
More informationMultivariate probability distributions and linear regression
Multivariate probability distributions and linear regression Patrik Hoyer 1 Contents: Random variable, probability distribution Joint distribution Marginal distribution Conditional distribution Independence,
More informationENGG2430A-Homework 2
ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,
More information