Random Variables and Their Distributions

Transcription

1 Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital letters: X, Y or Z and their values by small letters: x, y or z respectively. There are two types of r.vs: discrete and continuous. Random variables that can take a countable number of values are called discrete. Random variables that take values from an interval of real numbers are called continuous. Example 3. Discrete r.v., Brockwell and Davis (2002) Yearly results of the all-star baseball games over years , where X t = { if the National League won in year t, if the Americal League won in year t. In each of the realizations there is some probability p of X t = and probability p of X t =. Example 3.2 Continuous r.v. The r.vs in all the examples given in Chapter are continuous. 33

2 34 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS 3. One Dimensional Random Variables Definition 3.. If E is an experiment having sample space S, and X is a function that assigns a real number X(e) to every outcome e S, then X is called a random variable. Definition 3.2. Let X denote a r.v. and x its particular value from the whole range of all values of X, say R X. The probability of the event (X x) expressed as a function of x: F X (x) = P X (X x) (3.) is called the cumulative distribution function (c.d.f.) of the r.v. X. Properties of cumulative distribution functions 0 F X (x), < x < lim x F X (x) = lim x F X (x) = 0 The function is nondecreasing. That is, if x x 2 then F X (x ) F X (x 2 ). 3.. Discrete Random Variables Values of a discrete r.v. are elements of a countable set {x, x 2,...}. We associate a number p X (x i ) = P X (X = x i ) with each value x i, i =, 2,..., such that:. p X (x i ) 0 for all i 2. i= p X(x i ) = Note that F X (x i ) = P X (X x i ) = x x i p X (x) (3.2) p X (x i ) = F X (x i ) F X (x i ) (3.3) The function p X is called the probability mass function of the random variable X, and the collection of pairs {(x i, p X (x i )), i =, 2,...} (3.4)

3 3.. ONE DIMENSIONAL RANDOM VARIABLES 35 is called the probability distribution of X. The distribution is usually presented in either tabular, graphical or mathematical form. Examples of known p.m.fs Ex. 3.3 The Binomial Distribution (X denotes k successes in n independent trials). The p.m.f. of a binomially distributed r.v. X with parameters n and p is ( ) n P(X = k) = p k ( p) n k, k = 0,, 2,..., n, k where n is a positive integer and 0 p. Ex. 3.4 The Uniform Distribution The p.m.f. of a r.v. X uniformly distributed on {, 2,..., n} is P(X = k) =, k =, 2,..., n, n where n is a positive integer. Ex. 3.5 The Poisson Distribution (X denotes a number of outcomes in a period of time). The p.m.f. of a r.v. X having a Poisson distribution with parameter λ > 0 is P(X = k) = λk k! e λ. Take as an example a r.v. X having the Binomial distribution: X Bin(8, 0.4). That is n = 8 and the probability of success p = 0.4. The distribution, shown in a mathematical, tabular and graphical way and a graph of the c.d.f. of the variable X follow. Mathematical form: Tabular form: {(k, P(X = k) = n C k p k ( p) n k ), k = 0,, 2,..., 8} (3.5) k P(X = k) P(X k)

4 36 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS x x Figure 3.: Graphical representation of the mass function and the cumulative distribution function for X Bin(8, 0.4) Other important discrete distributions are: Bernoulli(p) Geometric(p) Hypogeomeric(n, M, N) 3..2 Continuous Random Variables Values of a continuous r.v. are elements of an uncountable set, for example a real interval. A c.d.f. of a continuous r.v. is a continuous, nondecreasing, differentiable function. An interesting difference from a discrete r.v. is that for δ > 0 P X (X = x) = lim δ 0 (F X (x + δ) F X (x)) = 0. We define the probability density function (p.d.f.) of a continuous r.v. as: Hence f X (x) = d dx F X(x) (3.6) F X (x) = x f X (t)dt (3.7) Similarly to the properties of the probability distribution of a discrete r.v. we have the following properties of the density function:. f X (x) 0 for all x R X 2. R X f X (x)dx =

5 Your text 3.. ONE DIMENSIONAL RANDOM VARIABLES 37 F(x) Density 0.0 cdf a b c 0.0 x Figure 3.2: N(4.5, 2) Distribution Function and Cumulative Distribution Function for Probability of an event (X A), where A is an interval (, a), is expressed as an integral P X ( < X < a) = or for a bounded interval (b, c) P X (b < X < c) = c b a f X (x)dx = F X (a) (3.8) f X (x)dx = F X (c) F X (b). (3.9) Examples of continuous r.vs Ex. 3.6 The normal distribution N(µ, σ 2 ) The density function is given by f X (x) = σ e (x µ) 2 2σ 2. (3.0) 2π There are two parameters which tell us about the position and the shape of the density curve: the expected value µ and the standard deviation σ. Ex. 3.7 The uniform distribution U(a, b) The p.d.f. of a r.v. X uniformly distributed on [a, b] is given by, if a x b, f X (x) = b a 0, otherwise.

6 38 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS pdf cdf x x Figure 3.3: Distribution density function and cumulative distribution function for Exp() Ex. 3.8 The exponential distribution Exp(λ) The p.d.f. of an exponentially distributed r.v. X with parameter λ > 0 is { 0, if x < 0, f X (x) = λe λx, if x 0. The corresponding c.d.f. is { 0, if x < 0, F X (x) = e λx, if x 0. Ex. 3.9 The gamma distribution Gamma(α, λ) with parameters representing shape (α) and scale (λ). The p.d.f. of a gamma distributed r.v. X is { 0, if x < 0, f X (x) = x α λ α e λx /Γ(α), if x 0, where the α > 0, λ > 0 and Γ is the gamma function defined by Γ(n) = 0 x n e x dx, for n > 0. A recursive relationship that may be easily shown integrating the above equation by parts is Γ(n) = (n )Γ(n ). If n is a positive integer, then since Γ() =. Γ(n) = (n )!

7 3.. ONE DIMENSIONAL RANDOM VARIABLES 39 Other important continuous distributions are The chi-squared distribution with ν degrees of freedom, χ 2 ν The F distribution with ν, ν 2 degrees of freedom, F ν,ν 2. Student t-distribution with ν degrees of freedom, t ν. These distributions are functions of normally distributed r.vs. Note that all the distributions depend on some parameters, like p, λ, µ, σ or other. These values are usually unknown, and so their estimation is one of the important problems in statistical analyses Expectation The expectation of a function g of a r.v. X is defined by E(g(X)) = g(x)f(x)dx, for a continuous r.v., g(x j )p(x j ) for a discrete r.v., j=0 (3.) and g is any function such that E g(x) <. Two important special cases of g are: Mean E(X), also denoted by E X, when g(x) = X, Variance E(X E X) 2, when g(x) = (X E X) 2. The following relation is very useful while calculating the variance E(X E X) 2 = E X 2 (E X) 2. (3.2) Example 3.0 Let X be a r.v. such that sin x, for x [0, π], f(x) = 2 0 otherwise. Then the expectation and variance are following

8 40 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS Expectation Variance E X = 2 π 0 0 x sin xdx = π 2. var(x) = EX 2 (E X) 2 = π ( π 2 x 2 sin xdx 2 2) = π2 4. Example 3. The Normal distribution, X N(µ, σ 2 ) We will show that the parameter µ is the expectation of X and the parameter σ 2 is the variance of X. First we show that E(X µ) = 0. Hence E X = µ. E(X µ) = (x µ)f(x)dx = (x µ) σ e (x µ) 2 2σ 2 dx 2π = σ 2 f (x)dx = 0. Similar arguments give E(X µ) 2 = (x µ) 2 f(x)dx = σ 2 (x µ)f (x)dx = σ 2. A useful linearity property of expectation is E[ag(X) + bh(y ) + c] = a E[g(X)] + b E[h(Y )] + c, (3.3) for any real constants a, b and c and for any functions g and h of r.vs X and Y whose expectations exist.

9 3.2. TWO DIMENSIONAL RANDOM VARIABLES Two Dimensional Random Variables Definition 3.3. Let S be a sample space associated with an experiment E, and X, X 2 be functions, each assigning a real number X (e), X 2 (e) to every outcome e E. Then the pair X = (X, X 2 ) is called a two-dimensional random variable. The range space of the two-dimensional random variable is R X = {(x, x 2 ) : x R X, x 2 R X2 } R 2. Definition 3.4. The cumulative distribution function of a two-dimensional r.v. X = (X, X 2 ) is F X (x, x 2 ) = P(X x, X 2 x 2 ) (3.4) 3.2. Discrete Two-Dimensional Random Variables If all values of X = (X, X 2 ) are countable, i.e., the values are in the range R X = {(x i, x 2j ), i =,...,n, j =,...,m} then the variable is discrete. The c.d.f. of a discrete r.v. X = (X, X 2 ) is defined as F X (x, x 2 ) = p(x i, x 2j ) (3.5) x i x x 2j x 2 where p(x i, x 2j ) denotes the joint probability mass function and p(x i, x 2j ) = P(X = x i, X 2 = x 2j ). As in the univariate case, the joint p.m.f. satisfies the following conditions.. p(x i, x 2j ) 0, for all i, j 2. all j all i p(x i, x 2j ) = Example 3.2 Tossing two fair dice. Consider an experiment of tossing two fair dice and noting the outcome on each die. The whole sample space consists of 36 elements, i.e., S = {(i, j) : i, j =,..., 6}.

10 42 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS Now, with each of these 36 elements associate values of two variables, X and X 2, such that X = sum of the outcomes on the two dice, X 2 = difference of the outcomes on the two dice. Then the bivariate r.v. X = (X, X 2 ) has the following joint probability mas function. x x Marginal p.m.fs Theorem 3.. Let X = (X, X 2 ) be a discrete bivariate random variable with joint p.m.f. p(x, x 2 ). Then the marginal p.m.fs of X and X 2, p X and p X2, are given respectively by p X (x ) = P(X = x ) = R X2 p(x, x 2 ) and p X2 (x 2 ) = P(X 2 = x 2 ) = R X p(x, x 2 ). Example 3.2 cont. The marginal distributions of the variables X and X 2 are following. x P(X = x ) x P(X 2 = x 2 )

11 3.2. TWO DIMENSIONAL RANDOM VARIABLES 43 Expectations of functions of bivariate random variables are calculated the same way as the univariate r.vs. Let g(x, x 2 ) be a real valued function defined on R X. Then g(x) = g(x, X 2 ) is a r.v. and its expectation is E[g(X)] = R X g(x, x 2 )p(x, x 2 ). Example 3.2 cont. For g(x, X 2 ) = X X 2 we obtain E[g(X)] = = Continuous Two-Dimensional Random Variables If the values of X = (X, X 2 ) are elements of an uncountable set in the Euclidean plane, then the variable is continuous. For example the values might be in the range R X = {(x, x 2 ) : a x b, c x 2 d} for some real a, b, c, d. The c.d.f. of a continuous r.v. X = (X, X 2 ) is defined as F X (x, x 2 ) = x2 x where f(x, x 2 ) is the probability density function such that. f(x, x 2 ) 0 for all (x, x 2 ) R 2 2. f(x, x 2 )dx dx 2 =. The equation (3.6) implies that Also P(a X b, c X 2 d) = f(t, t 2 )dt dt 2, (3.6) 2 F(x, x 2 ) x x 2 = f(x, x 2 ). (3.7) d b c a f(x, x 2 )dx dx 2. (3.8)

12 44 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS X_2 R (X,X2) 0 X_ Figure 3.4: Domain of the p.d.f. (3.2) The marginal p.d.fs of X and X 2 are defined similarly as in the discrete case, here using integrals. f X (x ) = f(x, x 2 )dx 2, for < x <, (3.9) f X2 (x 2 ) = f(x, x 2 )dx, for < x 2 <. (3.20) Example 3.3 Calculate P[X A], where A = {(x, x 2 ) : x + x 2 } and the joint p.d.f. of X = (X, X 2 ) is defined by { 6x x 2 2 for 0 < x <, 0 < x 2 <, f X (x, x 2 ) = 0 otherwise. (3.2) The probability is a double integral of the p.d.f. over the region A. The region is however limited by the domain in which the p.d.f. is positive, see Figure 3.4.

13 3.2. TWO DIMENSIONAL RANDOM VARIABLES 45 We can write Hence, the probability is P(X A) = A = {(x, x 2 ) : x + x 2, 0 < x <, 0 < x 2 < } = {(x, x 2 ) : x x 2, 0 < x <, 0 < x 2 < } = {(x, x 2 ) : x 2 < x <, 0 < x 2 < }. A f(x, x 2 )dx dx 2 = 0 x 2 6x x 2 2 dx dx 2 = 0.9 Also, using formulae ((3.9)) and ((3.20)) we can calculate marginal p.d.fs. f X (x ) = f X2 (x 2 ) = 0 0 6x x 2 2 dx 2 = 2x x = 2x, 6x x 2 2dx = 3x 2 x 2 2 0= 3x 2 2. These functions allow us to calculate probabilities involving only one variable. For example ( P 4 < X < ) 2 = 2x dx = Similarly to the case of a univariate r.v. the following linear property for the expectation holds. 4 E[ag(X) + bh(x) + c] = a E[g(X)] + b E[h(X)] + c, (3.22) where a, b and c are constants and g and h are some functions of the bivariate r.v. X = (X, X 2 ) Conditional Distributions and Independence Definition 3.5. Let X = (X, X 2 ) denote a discrete bivariate r.v. with joint p.m.f. p X (x, x 2 ) and marginal p.m.fs p X (x ) and p X2 (x 2 ). For any x such that p X (x ) > 0, the conditional p.m.f. of X 2 given that X = x is the function of x 2 defined by p X2 (x 2 x ) = p X(x, x 2 ) p X (x ). (3.23) Analogously, we define the conditional p.m.f. of X given X 2 = x 2 p X (x x 2 ) = p X(x, x 2 ) p X2 (x 2 ). (3.24)

14 46 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS It is easy to check that these functions are indeed p.d.fs. For example, for (3.23) we have R X2 p X (x, x 2 ) p X (x, x 2 ) p X2 (x 2 x ) = = p X (x ) R X2 R X2 p X (x ) = p X (x ) p X (x ) =. Example 3.2 cont. The conditional p.m.f. of X 2 given, for example, X = 5, is x p X2 (x 2 5) Analogously to the conditional distribution for discrete r.vs, we define the conditional distribution for continuous r.vs. Definition 3.6. Let X = (X, X 2 ) denote a continuous bivariate r.v. with joint p.d.f. f X (x, x 2 ) and marginal p.d.fs f X (x ) and f X2 (x 2 ). For any x such that f X (x ) > 0, the conditional p.d.f. of X 2 given that X = x is the function of x 2 defined by f X2 (x 2 x ) = f X(x, x 2 ) f X (x ). (3.25) Analogously, we define the conditional p.d.f. of X given X 2 = x 2 f X (x x 2 ) = f X(x, x 2 ) f X2 (x 2 ). (3.26) Here too, it is easy to verify that these functions are p.d.fs. For example, for the function (3.25) we have f X (x, x 2 ) f X2 (x 2 x )dx 2 = R X2 R X2 f X (x ) dx 2 R X2 f X (x, x 2 )dx 2 = f X (x ) = f X (x ) f X (x ) =.

15 3.2. TWO DIMENSIONAL RANDOM VARIABLES 47 Example 3.3 cont. The conditional p.d.fs are and f X (x x 2 ) = f X(x, x 2 ) f X2 (x 2 ) f X2 (x 2 x ) = f X(x, x 2 ) f X (x ) = 6x x 2 2 3x 2 2 = 2x = 6x x 2 2 2x = 3x 2 2. The conditional p.d.fs make it possible to calculate conditional expectations. The conditional expected value of a function g(x 2 ) given that X = x is defined by g(x 2 )p X2 (x 2 x ) for a discrete r.v., R X2 E[g(X 2 ) x ] = (3.27) g(x 2 )f X2 (x 2 x )dx 2 for a continuous r.v.. R X2 Example 3.3 cont. Equation ((3.27)) allows to calculate conditional mean and variance of a r.v. Here we have and µ X2 x = E(X 2 x ) = σ 2 X 2 x = var(x 2 x ) = E(X 2 2 x ) [E(X 2 x )] 2 = 0 x 2 3x 2 2 dx 2 = 3 4, 0 x 2 23x 2 2dx 2 ( ) 2 3 = Definition 3.7. Let X = (X, X 2 ) denote a continuous bivariate r.v. with joint p.d.f. f X (x, x 2 ) and marginal p.d.fs f X (x ) and f X2 (x 2 ). Then X and X 2 are called independent random variables if, for every x R X and x 2 R X2 f X (x, x 2 ) = f X (x )f X2 (x 2 ). (3.28)

16 48 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS If X and X 2 are independent, then the conditional p.d.f. of X 2 given X = x is f X2 (x 2 x ) = f X(x, x 2 ) f X (x ) = f X (x )f X2 (x 2 ) f X (x ) = f X2 (x 2 ) regardless of the value of x. Analogous property holds for the conditional p.d.f. of X given X 2 = x 2. Example 3.3 cont. It is easy to notice that f X (x, x 2 ) = 6x x 2 2 = 2x 3x 2 2 = f X (x )f X2 (x 2 ). So, the variables X and X 2 are independent. In fact, two r.vs are independent if and only if there exist functions g(x ) and h(x 2 ) such that for every x R X and x 2 R X2, f X (x, x 2 ) = g(x )h(x 2 ). Theorem 3.2. Let X and X 2 be independent random variables. Then. For any A R and B R P(X A, X 2 B) = P(X A)P(X 2 B), that is, the events {X A} and {X 2 B} are independent events. 2. For g(x ), a function of X only, and for h(x 2 ), a function of X 2 only, we have E[g(X )h(x 2 )] = E[g(X )] E[h(X 2 )] Covariance and Correlation Covariance and correlation are two measures of the strength of a relationship between two r.vs.

17 3.2. TWO DIMENSIONAL RANDOM VARIABLES 49 We will use the following notation. E X = µ X E X 2 = µ X2 var(x ) = σ 2 X var(x 2 ) = σ 2 X 2 Also, we assume that σ 2 X and σ 2 X 2 are finite positive values. A simplified notation µ, µ 2, σ 2, σ2 2 notation refers to. will be used when it is clear which r.vs the Definition 3.8. The covariance of X and X 2 is defined by cov(x, X 2 ) = E[(X µ X )(X 2 µ X2 )]. (3.29) Definition 3.9. The correlation of X and X 2 is defined by ρ (X,X 2 ) = corr(x, X 2 ) = cov(x, X 2 ) σ X σ X2. (3.30) Properties of covariance For any r.vs X and X 2, cov(x, X 2 ) = E(X X 2 ) µ X µ X2. If r.vs X and X 2 are independent then cov(x, X 2 ) = 0. Then also ρ (X,X 2 ) = 0. For any r.vs X and X 2 and any constants a and b, var(ax + bx 2 ) = a 2 var(x ) + b 2 var(x 2 ) + 2ab cov(x, X 2 ).

18 50 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS Properties of correlation For any r.vs X and X 2 ρ (X,X 2 ), ρ (X,X 2 ) = iff there exist numbers a 0 and b such that P(X 2 = ax + b) =. If ρ (X,X 2 ) = then a > 0, and if ρ (X,X 2 ) = then a < Bivariate Normal Distribution Here, we use matrix notation. A bivariate r.v. is treated as a random vector ( ) X X =. Let X = (X, X 2 ) T be a bivariate random vector with expectation ( ) ( ) X µ µ = E X = E =. X 2 X 2 µ 2 and the variance-covariance matrix ( var(x ) cov(x V =, X 2 ) cov(x 2, X ) var(x 2 ) ) ( σ 2 = ρσ σ 2 ρσ σ 2 σ2 2 ). Then the joint p.d.f. of the vector r.v. X is given by f X (x) = where x = (x, x 2 ) T. 2π det(v ) exp { 2 (x µ)t V (x µ) }, (3.3) The determinant of V is ( σ 2 det V = det ρσ σ 2 ρσ σ 2 σ2 2 ) = ( ρ 2 )σσ

19 3.3. MULTIVARIATE NORMAL DISTRIBUTION 5 Hence, the inverse of V is V = ( σ2 2 ρσ σ 2 det V ρσ σ 2 σ 2 ) = ( ρ 2 σ 2 ρσ 2 ρσ σ2 σ2 2 ). Then the exponent in formula (3.3) can be written as 2 (x µ)t V (x µ) = ( = 2( ρ 2 ) (x µ, x 2 µ 2 ) ( (x µ ) 2 = 2( ρ 2 ) σ 2 σ 2 ρσ σ2 ρσ σ2 σ2 2 2ρ (x µ )(x 2 µ 2 ) σ σ 2 + (x 2 µ 2 ) 2 σ 2 2 So, the joint p.d.f. of the 2-dimensional vector r.v. X is f X (x) = 2πσ σ 2 ( ρ2 ) { ( (x µ ) 2 exp 2( ρ 2 ) σ 2 ) ( ) x µ x 2 µ ). 2ρ (x µ )(x 2 µ 2 ) + (x )} 2 µ 2 ) 2. σ σ 2 σ2 2 (3.32) Remark 3.. Note that when ρ = 0 the joint p.d.f. (3.32) simplifies to { f X (x) = exp ( (x µ ) 2 + (x )} 2 µ 2 ) 2, 2πσ σ 2 2 σ 2 σ2 2 which can be written as a product of the marginal distributions of X and X 2. Hence, if X = (X, X 2 ) T has a bivariate normal distribution and ρ = 0 then the variables X and X 2 are independent. 3.3 Multivariate Normal Distribution Denote by X = (X,...,X n ) T an n-dimensional random vector whose each component is a random variable. Then all the definitions given for bivariate r.vs extend to the multivariate r.vs. X has a multivariate normal distribution if its p.d.f. can be written as in (3.3), where the mean is µ = (µ,..., µ n ) T,

20 52 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS Figure 3.5: Bivariate Normal pdf

21 3.3. MULTIVARIATE NORMAL DISTRIBUTION 53 and the variance-covariance matrix has the form var(x ) cov(x, X 2 )... cov(x, X n ) cov(x 2, X ) var(x 2 )... cov(x 2, X n ) V = cov(x n, X ) cov(x n, X 2 )... var(x n ) Remark 3.2. If X N n (µ, V ), B is an m n matrix, and a is a real m vector, then the random vector is also multivariate normal with and the variance-covariance matrix, Y = a + BX E(Y ) = a + B E(X) = a + Bµ, V Y = BV B T. Remark 3.3. Taking B = b T, where b is an n dimensional vector and a = 0 we obtain Y = b T X = b X b n X n, and Y N(b T µ, b T V b). Two important properties of multivariate normal random vectors are given in the following proposition. Proposition 3.. Suppose that the normal r.v. is partitioned into two subvectors of dimensions n and n 2 ( ) X () X =, X (2)

22 54 CHAPTER 3. RANDOM VARIABLES AND THEIR DISTRIBUTIONS and, correspondingly, the mean vector and the variance-covariance matrix are partitioned as ( ) ( ) µ () V V µ = µ (2) V = 2, V 2 V 22 where Then µ (i) = E(X (i) ) and V ij = cov(x (i), X (j) ).. X () and X (2) are independent iff the (n n 2 ) dimensional covariance matrix V 2 is a zero matrix. 2. The conditional distribution of X () given X (2) = x (2) is N(µ () + V 2 V 22 (x(2) µ (2) ), V V 2 V 22 V 2). For the bivariate normal r.v. we obtain and E(X X 2 = x 2 ) = µ + ρσ σ 2 (x 2 µ 2 ) (3.33) var(x X 2 = x 2 ) = σ 2 ( ρ2 ). (3.34) Definition 3.0. X t is a Gaussian time series if all its joint distributions are multivariate normal, i.e., if for any collection of integers i,...,i n, the random vector (X i,...,x in ) T has a multivariate normal distribution. For proofs of the results given in this chapter see Castella and Berger (990) and Brockwell and Davis (2002) or other textbooks on probability and statistics.