Probability Theory and Statistics. Peter Jochumzen

Size: px
Start display at page:

Download "Probability Theory and Statistics. Peter Jochumzen"

Transcription

1 Probability Theory and Statistics Peter Jochumzen April 18, 2016

2 Contents 1 Probability Theory And Statistics Experiment, Outcome and Event Probability Rules of Probability Conditional Probabilities and Independent Events Random Variables Discrete and Continuous Random Variables Probability Mass Function Cumulative Distribution Function Cumulative Distribution Function, Discrete Random Variables Cumulative Distribution Function, Continuous Random Variable Probability Density Function Function of a Random Variable Expected Value Variance The Constant Random Variable The Discrete Uniform Distribution The Bernoulli Distribution The Binomial Distribution The Continuous Uniform Distribution The Exponential Distribution The Normal Distribution Two Random Variables Probability Mass Function, Two Random Variables Marginal Probability Mass Function Cumulative Distribution Function, Two Random Variables Probability Density Function, Two Continuous Random Variables Marginal Probability Density Function Conditional Distributions, Discrete Random Variables Conditional Distributions, Continuous Random Variables Independent Random Variables Conditional Expectation Function of Two Random Variables Expected Value and Variance of a Linear Function of Two Random Variables Expected Value of a Product of Two Random Variables Covariance of Two Random Variables Covariance of Two Random Variables, Results Correlation Several Random Variables Random Sample Random Sample of Vectors Statistics Sample Mean Sample Variance Sample Covariance Properties of the Sample Mean

3 1.46 Properties of the Sample Covariance The Chi-square Distribution Properties of the Sample Variance The Standard Error of the Sample Mean The t-distribution The F-distribution Statistical Model Estimator Properties of Estimators Standard Error The Z-statistics Generalizing the Z-statistics The t-statistics Generalizing the t-statistics Critical Values Hypothesis Testing, Theory Hypothesis Tests Involving the Mean Confidence Intervals Asymptotically Unbiased Estimator Consistency plim Rules Convergence in Distribution Central Limit Theorem

4 Chapter 1 Probability Theory And Statistics 1.1 Experiment, Outcome and Event The sample space S of an experiment is the set of all possible outcomes of that experiment. sample space S is the universal set. The Example. Experiment: Toss a dice. Sample space S = {1, 2, 3, 4, 5, 6}. We say that the sample space is finite if it contains a finite number of elements. Elements of the sample space are called outcomes or possible outcomes. An event is a subset of the sample space S. Example (continued). A = {1, 3, 5} is the event toss an odd number. The sample space is called a certain event and an empty set is called an impossible event. An event containing only one outcome is called an elementary event. If A, B are two events then A B, A B, A C (the complement of A in S) and so on are events. Two events A, B are called mutually exclusive if A B = 1.2 Probability If A is an event then P (A) denotes the probability that the experiment will result in an outcome in A Example. Toss a fair dice and let A = {1, 3, 5}, B = {4}. Then P (A) = 1/2, P (B) = 1/6. If S is a finite sample space with n outcomes then we say that each outcome is equally likely if P (A) = 1/n for each elementary event A. Example. Each outcome of a fair dice is equally likely. If the outcomes of an experiment are equally likely then P (A) = k n where A is an arbitrary event with k outcomes and n is the number of outcomes of the sample space. 1.3 Rules of Probability In this section, A, B are arbitrary events in the sample space S. 1. Probabilities are between 0 and 1: 0 P (A) 1 3

5 2. Probabilities of certain and impossible events: P (S) = 1 P ( ) = 0 3. Probabilities of mutually exclusive events: If A, B are mutually exclusive events then P (A B) = P (A) + P (B) (This rule can be extended, in a trivial way, to the case when we have many mutually exclusive events) 4. Probabilities of complements: P (A C ) = 1 P (A) 5. Probabilities of subsets: If A B then P (A) P (B) 6. Probabilities of unions: P (A B) = P (A) + P (B) P (A B) 1.4 Conditional Probabilities and Independent Events In this section, A, B are arbitrary events in the sample space S. If P (B) > 0 we define the conditional probability of the event A given B, denoted by P (A B), as We say that A, B are independent events if P (A B) = P (A B) P (B) P (A B) = P (A)P (B) A, B are independent events and P (B) > 0 if and only if P (A B) = P (A) 1.5 Random Variables Simplified definition: A random variable on a sample space S is a function or a mapping from S to R. If the random variable is denoted by X then X : S R. If c is a constant then X = c is an event defined as the collection of outcomes that X will map to c. If c is a constant then P (X = c) is the probability of the event X = c. Similarly, if a, b are constants, X < a, X b, a < X b and so on are events with probabilities P (X < a), P (X b), P (a < X b) and so on. More generally, If A is an arbitrary subset of R then X A is an event defined as the collection of outcomes that X will map to any number in A and P (X A) is the probability of this event. 1.6 Discrete and Continuous Random Variables If X is a random variable then we define the range of X as the collection of real numbers that X can take. The range of X is typically denoted by R or R X. A finite set is a set with a finite number of elements. The set of natural numbers N = {1, 2, 3, } is a countably infinite set The set of integers Z = {0, ±1, ±2, ±3, } is also a countably infinite set and so is any infinite subset of of Z 4

6 The set of real numbers R is an uncountably infinite set. All intervals such as (a, b), [a, b], (a, b], [a, b), (, b), (a, ) where a < b are real numbers are uncountably infinite sets. If the range of X is finite or countable infinite then we say that X is a discrete random variable. If the range of X is uncountable infinite and P (X = c) for all real numbers c then we say that X is a continuous random variable. 1.7 Probability Mass Function In this section, X is a discrete random variable so that R, the range of X, is finite or countable infinite. If the range of X is finite, we denote the range by R = {x 1, x 2,, x n }. If the range of X is countably infinite, we denote the range by R = {x 1, x 2, }. For an arbitrary real number x, we define as the probability mass function, pmf, of X. f(x) = P (X = x) If x is not in the range of X then X = x is an empty set and f(x) = 0. If x is in the range of X then X = x is a non empty set and 0 f(x) 1. If A is a subset of the range R then the probability that X takes a value in A is given by P (X A) = x A f(x) The pmf must satisfy f(x) = 1 x R If the range is finite, the expression can also be written n f(x i ) = Cumulative Distribution Function In this section, X is an arbitrary random variable. For an arbitrary real number x, we define F (x) = P (X x) as the cumulative distribution function, cdf of X. The cdf F has the following properties: 1. F is an increasing function. 2. F is continuous from the right (but not continuous). 3. lim x F (x) = 0 4. lim x F (x) = 1 i=1 5

7 1.9 Cumulative Distribution Function, Discrete Random Variables In this section, X is a discrete random variable with pmf f and cdf F. We define (See Left hand limits ) We have the following results F (x 0 ) = lim F (x) x x 0 F (x ) = P (X < x) f(x) = F (x) F (x ) F (x) = x i x f(x i ) P (a < X b) = F (b) F (a) P (a X b) = F (b) F (a ) P (a < X < b) = F (b ) F (a) P (a X < b) = F (b ) F (a ) 1.10 Cumulative Distribution Function, Continuous Random Variable In this section, X is a continuous random variable. If X is a continuous random variable then F is a continuous function. F (x ) = F (x) and P (X x) = P (X < x) (remember, P (X = x) = 0) We have the following results P (a < X b) = P (a X b) = P (a < X < b) = P (a X < b) = F (b) F (a) 1.11 Probability Density Function In this section, X is a continuous random variable with cdf F. The probability density function (pdf) of X is defined as (assuming that the derivative exists). f(x) = df (x) dx f(x) is not a probability. If x is not in the range of X then f(x) = 0. If x is in the range of X then f(x) 0 (with no upper limit). If A is a subset of the range R then the probability that X takes a value in A is given by P (X A) = f(x)dx In particular, if A is an interval, A = [a, b], then P (a X b) = x A b a f(x)dx 6

8 The pdf of any continuous random variable must satisfy 1. Non-negative: f(x) 0 for all x. 2. Integrate to 1: f(x)dx = 1 x R Since f(x) = 0 when x is outside the range of X we can also write this as f(x)dx = 1 Calculating the cdf from the pdf: F (x) = x f(t)dt 1.12 Function of a Random Variable If X is a random variable and g is a real-valued function of one real variable with a domain equal to the range of X then Y = g(x) is a new random variable. The range of Y is given by the range of g Expected Value If X is a discrete random variable then we define the expected value of X, denoted by E(X), as E(X) = x R xf(x) where f(x) is the pmf of X. We often use the symbol µ for E(X) or µ X if we need to clarify the name of the random variable. If X is a continuous random variable then we define the expected value of X, also denoted by E(X), as E(X) = xf(x)dx where f(x) is the pdf of X. If X is a discrete random variable and Y = g(x) then the expected value of Y is given by x R E(Y ) = x R g(x)f(x) If X is a continuous random variable and Y = g(x) then the expected value of Y is given by E(Y ) = g(x)f(x)dx x R If X is any random variable and g is a linear function, Y = a + bx then the expected value of Y is given by E(Y ) = a + be(x) 1.14 Variance If X is an arbitrary random variable with expected value µ then we define the variance of X as V ar(x) = E[(X µ) 2 ] = E(X 2 ) µ 2 We often write the right-hand side as E(X µ) 2 with the implicit understanding that this is the expected value of (X µ) 2, not the square of E(X µ). 7

9 We often use the symbol σ 2 for V ar(x) or σx 2 if we need to clarify the name of the random variable. If X is discrete, it follows that V ar(x) = x R(x µ) 2 f(x) If X is continuous, it follows that V ar(x) = (x µ) 2 f(x)dx x R If X is a discrete random variable and Y = g(x) with E(Y ) = µ Y then the variance of Y is given by V ar(y ) = x R(g(x) µ Y ) 2 f(x) If X is a continuous random variable and Y = g(x) with E(Y ) = µ Y by V ar(y ) = (g(x) µ Y ) 2 f(x)dx x R then the variance of Y is given If X is a any random variable and g is a linear function, Y = a + bx then the variance of Y is given by V ar(y ) = b 2 V ar(x) The standard deviation of a random variable X is defined as the square root of the variance, σ = V ar(x) 1.15 The Constant Random Variable A discrete random variable X whose range is a single number c is called a constant random variable or simply a constant and we write X = c. The pmf of a constant random variable is given by f(c) = 1, f(x) = 0 for all x c. We have E(X) = c and V ar(x) = 0; the expected value of a constant is the constant itself and the variance of a constant is zero The Discrete Uniform Distribution A discrete random variable X that takes n different values with equal probability is said to follow a discrete uniform distribution. Formally, the range of X is R = {x 1, x 2, x n } and the pmf is given by f(x i ) = 1/n for i = 1,, n. We have and V ar(x) = 1 n µ = E(X) = 1 n n i=1 x i n (x i µ) 2 = 1 n i=1 n x 2 i µ 2 i=1 8

10 1.17 The Bernoulli Distribution A discrete random variable X with range R = {0, 1} is said to follow a Bernoulli distribution. We typically denote f(1) = P (X = 1) by p such that f(0) = P (X = 0) = 1 p with 0 p 1 and write X Ber(p). We have E(X) = p V ar(x) = p(1 p) 1.18 The Binomial Distribution A discrete random variable X with range R = {0, 1,, n} and pmf ( ) n P (X = k) = f(k) = p k (1 p) n k k where ( ) n n! = k k!(n k)! is said to follow a binomial distribution and we write X B(p) We have E(X) = np V ar(x) = np(1 p) 1.19 The Continuous Uniform Distribution A continuous random variable X with range R = [a, b], b > a and pdf f(x) = 1 b a is said to follow a continuous uniform distribution and we write We have E(X) = a + b 2 X U[a, b] V ar(x) = 1 (b a)2 12 If X U[a, b] and Y = c + dx is a linear function of X with d 0 then Y will also follow a uniform distribution The Exponential Distribution A continuous random variable X with range R = [0, ) and pdf f(x) = 1 λ e x/λ is said to follow an exponential distribution with parameter λ and we write where λ > 0. We have X exp(λ) E(X) = λ V ar(x) = λ 2 The probability density function of an exponential distribution is sometimes written as Written like this, E(X) = λ 1 V ar(x) = λ 2 f(x) = λe λx 9

11 1.21 The Normal Distribution A continuous random variable X with range R = (, ) and pdf ( ) 1 f(x) = exp (x µ)2 2πσ 2 2σ 2 is said to follow an normal distribution with parameters µ, σ 2 and we write X N(µ, σ 2 ) We have E(X) = µ V ar(x) = σ 2 If µ = 0 and σ 2 = 1 we say that X follows a standard normal distribution. A random variable that follows a standard normal distribution is typically denoted by Z and Z N(0, 1) with E(Z) = 0, V ar(z) = 1 and with probability density function f(z) = 1 ) exp ( x2 2π 2 If X N(µ X, σx 2 ) and Y = a + bx is a linear function of X with b 0 then where µ Y = a + bµ X and σ 2 Y = b2 σ 2 X. Y N(µ Y, σ 2 Y ) If X N(µ, σ 2 ) then X µ σ N(0, 1) Excel: If X N with expected value m and standard deviation s then P (X x): NORM.DIST(x,m,s,TRUE) P (X x): 1 NORM.DIST(x,m,s,TRUE) the value x such that P (X x) = p: NORM.INV(p,m,s) If X N(0, 1) then you can use NORM.S.DIST(x,TRUE) and NORM.S.INV(p) 1.22 Two Random Variables Given a sample space S, we can define two functions, X : S R and Y : S R. We then have two random variables. Given two random variables X, Y and two constants x, y, X = x, Y = y is an event. It is the collection of outcomes that X will map to x and Y maps to y. The probability of the event X = x, Y = y is denoted by P (X = x, Y = y). Similarly, expressions such as X x, Y y are events whose probability is P (X x, Y y) X, Y are called discrete random variables if the range of X and the range of Y are both finite or infinite countable. X, Y are called continuous random variables if the range of X and the range of Y are both uncountable and P (X = x, Y = y) = 0 for all x, y. The range of X, Y is the collection of all ordered pairs (x, y) such that there is an outcome in S that is mapped to x by X and to y by Y. 10

12 1.23 Probability Mass Function, Two Random Variables In this section, X, Y are two discrete random variables. For any two real numbers (x, y), we define f(x, y) = P (X = x, Y = y) as the probability mass function, pmf, of X, Y. If (x, y) is not in the range of X, Y then X = x, Y = y is an empty set and f(x, y) = 0. If (x, y) is in the range of X, Y then X = x, Y = y is a non empty set and 0 f(x, y) 1. If A is a subset of the range R then the probability that (X, Y ) takes a value in A is given by P ((X, Y ) A) = f(x, y) (x,y) A The pmf must satisfy (x,y) R f(x, y) = Marginal Probability Mass Function In this section, X, Y are two discrete random variables with range R and pmf f(x, y). The marginal probability mass function of X is defined as f X (x) = y f(x, y) where the sum is over all y such that (x, y) R. The marginal pmf of Y is defined similarly as f Y (y) = x f(x, y) To distinguish f(x, y) from the marginal distributions, we sometimes call it the joint probability mass function or joint pmf and denote it by f X,Y (x, y) Cumulative Distribution Function, Two Random Variables In this section, X, Y are two arbitrary random variables. For two real numbers (x, y), we define F (x, y) = P (X x, Y y) as the cumulative distribution function, cdf of X, Y Probability Density Function, Two Continuous Random Variables In this section, X, Y are two continuous random variable with cdf F (x, y). The probability density function (pdf) of X, Y is defined as (assuming that the derivative exists) f(x, y) = 2 F (x, y) x y 11

13 f(x, y) is not a probability. If (x, y) is not in the range of X, Y then f(x, y) = 0. If (x, y) is in the range of X, Y then f(x, y) 0 (with no upper limit). f(x, y) is sometimes denoted by f X,Y (x, y) 1.27 Marginal Probability Density Function In this section, X, Y are two continuous random variable with pdf f(x, y). The marginal probability density function of X is defined as f X (x) = f(x, y)dy where the integral is over all y such that (x, y) R. Similarly, the marginal pdf of Y is defined as f Y (y) = y x f(x, y)dx f(x, y) is sometimes called the joint probability density function or joint pdf and denoted by f X,Y (x, y) Conditional Distributions, Discrete Random Variables In this section, X, Y are two discrete random variables with joint pmf f X,Y (x, y) and marginal pmf s f X (x) and f Y (y). X = x and Y = y are two separate events. Therefore, if P (Y = y) 0 then P (X = x Y = y) = P (X = x Y = y) P (Y = y) = P (X = x, Y = y) P (Y = y) The probability P (X = x Y = y) is called the conditional probability of X given Y Similarly if P (X = x) 0 then the conditional probability of Y given X is P (Y = y X = x) = P (X = x, Y = y) P (X = x) We denote the conditional probability P (X = x Y = y) by f(x y) or by f X Y (x y) if we need to specify the names of the random variables. P (Y = y X = x) is denoted by f(y x) or by f Y X (y x). f(x y) is called a conditional probability mass function (of x given y) or a conditional pmf Since P (X = x, Y = y) = f X,Y (x, y) we have and whenever the denominators are not zero. f X Y (x y) = f X,Y (x, y) f Y (y) f Y X (y x) = f X,Y (x, y) f X (x) 12

14 1.29 Conditional Distributions, Continuous Random Variables In this section, X, Y are two continuous random variables with joint pdf f X,Y (x, y) and marginal pdf s f X (x) and f Y (y). We define f(x y) = f X,Y (x, y) f Y (y) as a conditional probability density function (of x given y) whenever f Y (y) 0. Similarly, we define whenever f X (x) 0. f(y x) = f X,Y (x, y) f X (x) 1.30 Independent Random Variables Two random variables X, Y are said to be independent if and only if for all (x, y) in the range of X, Y. f(x, y) = f X (x)f Y (y) X, Y are independent if and only if f(x y) = f X (x) for all x, y. Also X, Y are independent if and only if f(y x) = f Y (y) for all x, y Conditional Expectation If X, Y are two discrete random variables then we define the conditional expectation of X given Y = y as the number E(X Y = y) = xf(x y) x E(Y X = x) is defined similarly. If X, Y are two continuous random variables then we define the conditional expectation of X given Y = y as the number E(X Y = y) = xf(x y)dx E(Y X = x) is defined similarly. If X, Y are independent random variables then E(X Y = y) = E(X) and E(Y X = x) = E(Y ). Without specifying a specific value for Y, E(X Y ) is a random variable. Law of iterated expectation: E(E(X Y )) = E(Y ) 1.32 Function of Two Random Variables If X, Y are two random variables and g is a real-valued function of two variables whose domain is equal to the range of X, Y then Z = g(x, Y ) is a new random variable. If g is a linear function, g(x, y) = a + bx + cy then we say that Z = a + bx + cy is a linear function of the random variables X and Y. x 13

15 1.33 Expected Value and Variance of a Linear Function of Two Random Variables If X, Y are two random variables and Z = a + bx + cy is a linear function of X and Y then E(Z) = E(a + bx + cy ) = a + be(x) + ce(y ) If X, Y are two independent random variables and Z = a + bx + cy is a linear function of X and Y then V ar(z) = V ar(a + bx + cy ) = b 2 V ar(x) + c 2 V ar(y ) 1.34 Expected Value of a Product of Two Random Variables If X, Y are two independent random variables and Z = XY is a product of X and Y then E(Z) = E(XY ) = E(X)E(Y ) If, in addition, f and g are two real valued functions of a real variable and Z = f(x)f(y ) is a product of f(x) and f(y ) then E(Z) = E(f(X)f(Y )) = E(f(X))E(f(Y )) 1.35 Covariance of Two Random Variables If X, Y are two random variables with expected values µ X and µ Y covariance of X and Y, denoted by Cov(X, Y ) as the number respectively then we define the Cov(X, Y ) = E[(X µ X )(Y µ Y )] = E(XY ) µ X µ Y Cov(X, Y ) is sometimes denoted by σ X,Y or σ XY 1.36 Covariance of Two Random Variables, Results 1. If X, Y are two independent random variables then Cov(X, Y ) = 0. The opposite is not necessarily true. 2. Cov(X, X) = V ar(x) 3. If c is a constant then Cov(X, c) = 0 4. Covariance of linear functions of random variables (a, b, c, d are constants): Cov(a + bx, c + dy ) = bdcov(x, Y ) 5. Covariance of linear functions of two random variables (a, b, c, d are constants): Cov(aX 1 + bx 2, cy 1 + dy 2 ) = accov(x 1, Y 1 ) + adcov(x 1, Y 2 ) + bccov(x 2, Y 1 ) + bdcov(x 2, Y 2 ) 6. If X, Y are two arbitrary random variables then V ar(a + bx + cy ) = b 2 V ar(x) + c 2 V ar(y ) + 2bcCov(X, Y ) 14

16 1.37 Correlation If X, Y are two random variables then we define the correlation between X and Y, denoted by Corr(X, Y ) as the number Cov(X, Y ) Corr(X, Y ) = V ar(x)v ar(y ) Corr(X, Y ) is sometimes denoted by ρ X,Y For any two random variables X, Y, or ρ XY 1 Corr(X, Y ) 1 We have Corr(X, Y ) = 0 if and only if Cov(X, Y ) = 0. We then say that X and Y are uncorrelated. Corr(X, Y ) > 0 if and only if Cov(X, Y ) > 0. We then say that X and Y are positively correlated. Corr(X, Y ) < 0 if and only if Cov(X, Y ) < 0. We then say that X and Y are negatively correlated Several Random Variables Given a sample space S, we can define n functions, X i : S R for i = 1,, n. We then have n random variables X 1,, X n. Together, they are called a random vector denoted by X = (X 1,, X n ). Given n random variables X 1,, X n and n constants x 1,, x n, X 1 = x 1,, X n = x n is an event. It is the collection of outcomes that X 1 will map to x 1 and X 2 maps to x 2 and so on. The probability of the event X 1 = x 1,, X n = x n is denoted by P (X 1 = x 1,, X n = x n ). Similarly, expressions such as X 1 x 1,, X n x n are events whose probability is P (X 1 x 1,, X n x n ) X 1,, X n are called discrete random variables if the range of all of the X i s are finite or infinite countable. X 1,, X n are called continuous random variables if the range of all of the X i s are uncountable and P (X 1 = x 1,, X n = x n ) = 0 for all x 1,, x n. The range of X 1,, X n is the collection of all ordered n-tuples (x 1,, x n ) such that there is an outcome in S that is mapped to x 1 by X 1 and so on. If (x 1,, x n ) are n real numbers then If X 1,, X n are n arbitrary random variables then we define the cumulative distribution function, cdf of X 1,, X n as F (x 1,, x n ) = P (X 1 x 1,, X n x n ) If X 1,, X n are n discrete random variables then we define the probability mass function, pmf, of X 1,, X n as f(x 1,, x n ) = P (X 1 = x 1,, X n = x n ) If X 1,, X n are n continuous random variables then we define the probability density function, pdf, of X 1,, X n as f(x 1,, x n ) = n F (x 1,, x n ) x 1 x n (assuming that the derivative exists). If X 1,, X n are n discrete random variables then we define the marginal probability mass function of X 1 as f X1 (x 1 ) = f(x 1,, x n ) x 2,,x n where the sum is over all x 2,, x n such that (x 1,, x n ) is in the range of X 1,, X n. The marginal pmf of X 2,, X n are defined similarly. 15

17 If X 1,, X n are n continuous random variables then we define the marginal probability density function of X i similarly by integrating away the remaining random variables. n random variables X 1,, X n are said to be independent if and only if for all ((x 1,, x n ) in the range of X 1,, X n. f(x 1,, x n ) = f X1 (x 1 ) f Xn (x n ) If X 1,, X n are n random variables and g is a real-valued function of n variables whose domain is equal to the range of X 1,, X n then Z = g(x 1,, X n ) is a new random variable. If g is a linear function, g(x 1,, x n ) = c 0 +c 1 x 1 + +c n x n then we say that Z = c 0 +c 1 X 1 + +c n X n is a linear function of the random variables X 1,, X n (c 0, c 1, c n are constants). If X 1,, X n are n random variables and Z = c 0 +c 1 X 1 + +c n X n is a linear function of X 1,, X n then E(Z) = E(c 0 + c 1 X c n X n ) = c 0 + c 1 E(X 1 ) + + c n E(X n ) If X 1,, X n are n independent random variables and Z = c 0 + c 1 X c n X n is a linear function of X 1,, X n then V ar(z) = V ar(c 0 + c 1 X c n X n ) = c 2 1V ar(x 1 ) + + c 2 nv ar(x n ) If X 1,, X n are n arbitrary random variables then V ar(c 0 + c 1 X c n X n ) = 1.39 Random Sample n n c 2 i V ar(x i ) + 2 c i c j Cov(X i, X j ) i=1 i=1 j<i A random sample is defined as an experiment on which we have defined a collection of n random variables X 1,, X n. After the experiment has been performed, we have n observations x 1,, x n viewed as drawings from X 1,, X n. A random sample is an independent random sample if and only if X 1,, X n are independent random variables. A random sample is an identically distributed random sample if and only if the marginal distributions of X 1,, X n are all the same (each random variable in the random sample has the same distribution). In this case, each random variable X i has the same mean and the same variance for i = 1,, n. A random sample is an independent and identically distributed, IID, random sample if and only if it is an independent and identically distributed random sample. Note: In some texts a random sample is always an IID random sample Random Sample of Vectors A random sample of vectors is defined as an experiment on which we have defined a collection of n random vectors X 1,, X n where each X i = (X i,1,, X i,k ) is a random vector with m random variables, i = 1,, n. In total, the sample consists of nm random variables. After the experiment we have nm observations denoted by x 1,, x n where each x i = (x i,1,, x i,k ). Independence and identical distribution is defined the same as for a random sample of random variables. This means that the sample is an IID sample if every X i,k is independent of each X j,k for i j. Random variables within the same random vector may be dependent. Also, the marginal distribution of X i,k must be the X j,k for all i, j = 1,, n but they need not be the same within the random vector. 16

18 1.41 Statistics If X 1,, X n is a random sample and g is a real-values function of n variables with a domain that contains the range of X 1,, X n then Θ = g(x 1,, X n ) is called a statistic. As a function of random variables, a statistic is itself a random variable. Once the experiment has been performed, the outcome of the random variables X 1,, X n is observed and denoted by the numbers x 1,, x n. The outcome of the statistic Θ, the observed value of the statistic, can then be calculated as g(x 1,, x n ) which is now a number that we view as a drawing from the random variable Θ. The definition may be extended in a natural way to a random sample of vectors Sample Mean If X 1,, X n is a random sample then we define a statistic called the sample mean, denoted as X, by X = 1 n The observed value of the sample mean is called the observed sample mean and it is denoted by x Sample Variance If X 1,, X n is a random sample then we defines the sample variance, denoted as S 2, by S 2 = 1 n 1 n i=1 X i n (X i X) 2 The observed value of the sample variance is called the observed sample variance and it is denoted by s 2. S = S 2 is called the sample standard deviation (s is the observed sample standard deviation). Never use the term variance for S 2 or s 2. Variance is a property of a random variable while the sample variance S 2 is a function of the random variables constituting a random sample. Sample variance is to sample mean as variance is to expected value Sample Covariance Consider an IID random sample of size n of random vectors of size 2, X 1,, X n with X i = (X i,1, X i,2 ). i=1 We defines the sample covariance, denoted as S 2 1,2, by S 2 1,2 = 1 n 1 n (X i,1 X 1 )(X i,2 X 2 ) i=1 where X 1 is the sample average of X 1,1,, X n,1 and X 2 is the sample average of X 1,2,, X n,2 Once the experiment has been performed, the outcome of S 2 1,2, sometimes called the observed sample covariance, is denoted by the number s 2 1,2 = 1 n 1 n (x i,1 x 1 )(x i,2 x 2 ) where x 1 and x 2 are observed sample averages of the corresponding sub-samples. i=1 17

19 The matrix ( ) S 2 S = 1 S1,2 2 S2,1 2 S2 2 is called the sample covariance matrix. Here, S 2 1 and S 2 2 are the sample variances of sub-samples 1 and 2. Note that S is symmetric as S 2 2,1 = S 2 1,2. These definitions may be extended to the general case where the random vectors are of size m. The sample covariance matrix will then be an m m matrix Properties of the Sample Mean If X 1,, X n is a random sample and E(X i ) = µ for i = 1,, n then E( X) = µ If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n then E( X) = µ V ar( X) = σ2 n We denote the standard deviation of X by SD( X) and we have SD( X) = V ar( X) = σ n If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n then X N(µ, σ 2 /n) (1.1) 1.46 Properties of the Sample Covariance If X 1,, X n is an IID random sample of size 2 with E(X i,1 ) = µ 1, E(X i,2 ) = µ 2, V ar(x i,1 ) = σ 2 1, V ar(x i,2 ) = σ 2 2 and Cov(X i,1, X i,2 ) = σ 2 1,2 for i = 1,, n then E(S 2 1,2) = σ 2 1, The Chi-square Distribution If Z follows a standard normal distribution then Y = Z 2 is said to follow a chi-square distribution with one degree of freedom and we write Y χ 2 1 If Z 1, Z k are k independent standard normal random variables, then Y = is said to follow a chi-square distribution with k degree of freedom and we write If Y χ 2 k then the range of Y is [0, ) If Y χ 2 k then E(Y ) = k and V ar(y ) = 2k k i=1 Y χ 2 k Z 2 i If Y 1 χ 2 k 1 and Y 2 χ 2 k 2 and Y 1 and Y 2 are independent then Y 1 + Y 2 χ 2 k 1+k 2 Excel: If Y χ 2 k then P (Y y): CHISQ.DIST(y,k,TRUE) P (Y y): CHISQ.DIST.RT(y,k) the value y such that P (Y y) = p: CHISQ.INV(p,k) the value y such that P (Y y) = p: CHISQ.INV.RT(p,k) 18

20 1.48 Properties of the Sample Variance If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n then E(S 2 ) = σ 2 If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n then X and S 2 are independent random variables. If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n then (n 1)S 2 σ 2 χ 2 n The Standard Error of the Sample Mean If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n then we define the standard error of the sample mean, denoted by SE( X), as SE( X) = S 2 n = S n (Remember, SD( X) = σ/ n) The t-distribution If Z follows a standard normal distribution and Y follows a chi-squared distribution with k degrees of freedom and Z and Y are independent, then t = Z is said to follow a t-distribution with k degrees Y/k of freedom and we write If T t k then the range of T is (, ) If T t k then E(T ) = 0 T t k The probability density function of the t-distribution is symmetric around 0, P (T c) = P (T c) for all constants c. It has fatter tails compared to Z, P (T > c) > P (Z > c) for any constant c > 0. Excel: If T t k then P (T t): T.DIST(t,k,TRUE) P (T t): T.DIST.RT(t,k) P ( T t) = P (T t) + P (T t): T.DIST.2T(t,k) the value t such that P (T t) = p: T.INV(p,k) the value t such that P ( T t) = p: T.INV.2T(p,k) 1.51 The F-distribution If X follows a chi-square distribution with k 1 degrees of freedom and Y follows a chi-squared distribution with k 2 degrees of freedom and X and Y are independent random variables, then F = X/k 1 Y/k 2 is said to follow an F-distribution with k 1 and k 2 degrees of freedom and we write F F (k 1, k 2 ) 19

21 If F F (k 1, k 2 ) then the range of F is [0, ) Excel: If F F (k 1, k 2 ) then P (F f): F.DIST(f,k1,k2,TRUE) P (F f): F.DIST.RT(f,k1,k2) the value f such that P (F f) = p: F.INV(p,k1,k2) the value f such that P (F f) = p: F.INV.RT(p,k1,k2) 1.52 Statistical Model If X 1,, X n is a random sample with a given probability density/mass function f(x 1,, x n ) which depends on k unknown parameters θ = (θ 1,, θ k ) then taken together we have a (parametric) statistical model or a data generating process. With a statistical model and given values for θ 1, θ k we can simulate an outcome x 1,, x n for a random sample using a computer. Four common statistical models: 1. X 1,, X n is an independent random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n. This model has two unknown parameters, µ, σ X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n. This is a special case of 1. We write X i IIDN(µ, σ 2 ) i = 1,, n 3. X 1,, X n is an independent random sample with E(X i ) = µ and V ar(x i ) = σ 2 0 for i = 1,, n where σ 2 0 is a known constant. This model has one unknown parameter µ and is a special case of X 1,, X n is an IID random sample with X i N(µ, σ 2 0) for i = 1,, n where σ 2 0 is a known constant. This is a special case of 2 and of 3. We write X i IIDN(µ, σ 2 0) i = 1,, n 1.53 Estimator If Θ = g(x 1,, X n ) is a statistic in a statistical model and we use the outcome of Θ to estimate θ, one of the unknown parameters of the statistical model, then the statistic Θ is called an estimator for θ denoted by ˆθ. The sample mean X is the common estimator of the mean µ in the four common statistical models (see section 1.52). Therefore, X can also be denoted by ˆµ. The sample variance S 2 is the common estimator of the variance σ 2 in the common statistical models when σ 2 is unknown (models 1 and 2). Therefore, S 2 can also be denoted by ˆσ 2. If g is a linear function then Θ = g(x 1,, X n ) is called a linear statistic or a linear estimator if it estimates an unknown parameter. The sample mean is a linear estimator while the sample variance is not Properties of Estimators If θ is an unknown parameter in a statistical model and ˆθ is an estimator of θ then we say that the estimator is unbiased if E(ˆθ) = θ The sample mean X is an unbiased estimator of the mean µ in the four common statistical models. The sample variance S 2 is an unbiased estimator of the variance σ 2 in the common statistical models when σ 2 is unknown (models 1 and 2). 20

22 If θ is an unknown parameter in a statistical model and ˆθ 1 and ˆθ 2 are two unbiased estimators of the same parameter θ then we say that ˆθ 1 is more efficient than ˆθ 2 if V ar(ˆθ 1 ) V ar(ˆθ 2 ) If θ is an unknown parameter in a statistical model and ˆθ is an unbiased estimator which is more efficient than any other unbiased estimator then ˆθ is called a minimum variance unbiased estimator, MVUE or the best unbiased estimator. If X i IIDN(µ, σ 2 ), i = 1,, n (common statistical model number 2) then X is MVUE of µ and S 2 is MVUE of σ 2. If θ is an unknown parameter in a statistical model and ˆθ is a linear unbiased estimator which is more efficient than any other linear unbiased estimator then ˆθ is called a best linear unbiased estimator, BLUE. If X 1,, X n is an independent random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n (common statistical model number 1) then X is BLUE of µ Standard Error Given: ˆθ1,, ˆθ k which are estimators of the unknown parameters θ 1,, θ k in a statistical model. The variance of each estimator, V ar(ˆθ i ), typically depends on some or all of the unknown parameters and is unknown. If each unknown parameter is replaced by its corresponding estimator we get the estimated variance of ˆθ i denoted by V ˆ ar(ˆθ i ). The square root of the estimated variance of an estimator ˆθ is called the standard error of ˆθ, denoted by SE(ˆθ) In the fundamental statistical model, V ar( X) = σ 2 /n and V ˆ ar( X) = S 2 /n and SE( X) = S/ n as defined before The Z-statistics If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ0 2 for i = 1,, n (common statistical model number 3) and µ 0 is an arbitrary constant then we define a statistic called the Z-statistic by If µ 0 = µ then Z = X µ 0 SD( X) = X µ 0 σ 0 / n E(Z) = 0 V ar(z) = 1 If X i N(µ, σ 2 0) for i = 1,, n (common statistical model number 4) and µ 0 = µ then Z N(0, 1) Note that if µ 0 µ then E(Z) 0 and Z cannot follow a standard normal Generalizing the Z-statistics If X 1,, X n is a random sample and Θ is a statistic such that E(Θ) = µ Θ and V ar(θ) = σ 2 Θ then for a given constant µ 0 we define the Z-statistic by If Θ N(µ Θ, σ 2 Θ ) and µ 0 = µ Θ then Z N(0, 1), Z = Θ µ 0 σ Θ Z = Θ µ Θ σ Θ N(0, 1) The Z-statistic defined in section 1.56 when X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 is a special case where Θ = X, µ Θ = µ, σ 2 Θ = σ2 /n. 21

23 1.58 The t-statistics If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n (common statistical model number 1) and µ 0 is an arbitrary constant then we define a statistic called the t-statistic by t = X µ 0 SE( X) = X µ 0 S/ n where S is the square root of the sample variance, see section If X i N(µ, σ 2 ) for i = 1,, n (common statistical model number 2) and µ = µ 0 then the t-statistics follows a t-distribution with n 1 degrees of freedom, t = X µ S/ n t n 1 Note that if µ 0 µ then the t-statistic will not follow a t-distribution Generalizing the t-statistics If X 1,, X n is a random sample and Θ is a statistic such that E(Θ) = µ Θ and V ar(θ) = σθ 2 and if ˆσ Θ 2 is a non-negative estimator of σ2 Θ then for a given constant µ 0 we define the t-statistic by where ˆσ Θ = ˆσ Θ 2 is the standard error of Θ. If µ 0 = µ Θ, Θ N(µ Θ, σ 2 Θ ) pˆσ Θ 2 /σ2 Θ χ2 p Θ and ˆσ Θ 2 are independent random variables t = Θ µ 0 ˆσ Θ then the t-statistics follows a t-distribution with p degrees of freedom, t t p The t-statistic for common statistical model number 2 (see section 1.58) is a special case where Θ = X, µ Θ = µ, σ 2 Θ = σ2 /n, ˆσ 2 Θ = S2 /n and p = n Critical Values If Z N(0, 1) then we define the critical value z α by P (Z > z α ) = α Because of symmetry, P (Z < z α ) = α as well. In Excel, you can calculate z α using NORM.S.INV(1 α) see section If T t n then we define the critical value t α,n by P (T > t α,n ) = α Because of symmetry, P (T < t α,n ) = α as well. In Excel, you can calculate t α,n using T.INV.RT(α, n) see section You can calculate the two-tailed critical value t α/2,n using T.INV.2T(α, n) 22

24 If X χ 2 n then we define the critical value χ 2 α,n by P (X > χ 2 α,n) = α In Excel, you can calculate χ 2 α,n using CHISQ.INV.RT(α, n) see section If F F (n 1, n 2 ) then we define the critical value f α,n1,n 2 by P (F > f α,n1,n 2 ) = α In Excel, you can calculate f α,n1,n 2 using F.INV(α, n 1, n 2 ) see section Hypothesis Testing, Theory Given a statistical model X 1,, X n with unknown parameters θ = (θ 1,, θ k ), a null hypothesis is a subset D 0 of D, where D is the set of all possible values for the unknown parameters. The remaining values, D \ D 0, is called the alternative hypothesis. We say that a null hypothesis is true if the true value of θ is inside D 0. Otherwise we say that it is false. If D 0 contains only a single point, then the null hypothesis is called simple. If at least one of the unknown parameters is restricted to a given value in D 0, then the null hypothesis is called sharp. You test a given hypothesis by defining a test statistic Θ = g(x 1,, X n ) defining a rejection region (or a critical region) which is a subset of the real numbers rejecting the null hypothesis if the outcome of the test statistic falls into the rejection region. If it does not we say that we fail to reject or do not reject the null hypothesis. We never accept a null hypothesis. Rejection of the null hypothesis when it is true is called a Type I error. The probability of committing a type I error is denoted by α. This probability is also called the size of the critical region and the level of significance of the test. Non rejection of the null hypothesis when it is false is called a Type II error. committing a type II error is denoted by β. 1 β is called the power of the test. The probability of It is common to first select the level of significance α and then to choose the rejection region in such a way that the probability of committing a type I error is precisely α. Once the outcome of the test statistic has been calculated, the p-value of the test is the level of significance such that the value of the test statistic is equal to one of the end-points of the critical region such that we are indifferent between rejecting the null hypothesis and not rejecting the null hypothesis. 23

25 1.62 Hypothesis Tests Involving the Mean Throughout this section, X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n where µ is unknown while σ 2 may be known or not known. The null hypothesis is always the subset restricted by µ = µ 0 where µ 0 is a given number. The null hypothesis is written as H 0 : µ = µ 0 If all values for µ are possible then the alternative hypothesis is the subset restricted by µ µ 0 denoted by H 1 : µ µ 0 This is called a two-sided alternative. If restrict µ to µ µ 0 then the alternative hypothesis is the subset restricted by µ < µ 0 denoted by This is called a one-sided alternative. H 1 : µ < µ 0 Similarly, if we restrict µ to µ µ 0 then the one-sided alternative is All tests have a level of significance equal to α. H 1 : µ > µ 0 1. σ 2 is known, two sided alternative H 1 : µ µ 0 Test statistic: The Z statistic Under the null: Z N(0, 1) Critical region: (, z α/2 ) and (z α/2, ) (see section 1.60) 2. σ 2 is known, one sided alternative H 1 : µ < µ 0 Test statistic: The Z statistic Under the null: Z N(0, 1) Critical region: (, z α ) 3. σ 2 is known, one sided alternative H 1 : µ > µ 0 Test statistic: The Z statistic Under the null: Z N(0, 1) Critical region: (z α, ) 4. σ 2 is unknown, two sided alternative H 1 : µ µ 0 Test statistic: The t statistic Under the null: t t n 1 Critical region: (, t α/2,n 1 ) and (t α/2,n 1, ) (see section 1.60) 5. σ 2 is unknown, one sided alternative H 1 : µ < µ 0 Test statistic: The t statistic Under the null: t t n 1 Critical region: (, t α,n 1 ) 6. σ 2 is unknown, one sided alternative H 1 : µ > µ 0 Test statistic: The t statistic Under the null: t t n 1 Critical region: (t α,n 1, ) 24

26 1.63 Confidence Intervals Given a statistical model X 1,, X n where θ is one of the unknown parameters, an interval estimate of θ is an interval of the form Θ 1 < θ < Θ 2 where Θ 1 and Θ 2 are two statistics such that Θ 1 < Θ 2 always holds. If Θ 1 < θ < Θ 2 is an interval estimate of θ such that P r(θ 1 < θ < Θ 2 ) = 1 α then Θ 1 < θ < Θ 2 is a (1 α)100% confidence interval for θ. In general, the smaller the α, the bigger the confidence interval. If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n where σ 2 is known then X z α/2 σ n < µ < X + z α/2 σ n is a (1 α)100% confidence interval for µ. If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n where σ 2 is unknown then X t α/2,n 1 S n < µ < X + t α/2,n 1 S n is a (1 α)100% confidence interval for µ. S 2 is the sample variance, see section If X 1,, X n is an IID random sample with X i N(µ, σ 2 ) for i = 1,, n where σ 2 is known or unknown then the null hypothesis H 0 : µ = µ 0 against the two-sided alternative H 1 : µ µ 0 will be rejected at the level of significance α if and only if µ 0 is outside the (1 α)100% confidence interval for µ Asymptotically Unbiased Estimator X 1,, X n is a random sample of a statistical model with unknown parameters θ = (θ 1,, θ k ). To investigate how a given statistic Θ or a given estimator ˆθ depends on the sample size, we will add a subscript n and write Θ n or ˆθ n If θ is an unknown parameter in a statistical model of sample size n and ˆθ n is an estimator of θ then we say that the estimator is asymptotically unbiased if E(ˆθ n ) θ as n An unbiased estimator is always asymptotically unbiased but the opposite is not necessarily true Consistency If X 1,, X n is a random sample and Θ n is a given statistic then we say that Θ n converges in probability to a constant c if for each ε > 0 P ( Θ n c < ε) 1 as n We then write plim Θ n = c If θ is an unknown parameter in a statistical model of sample size n and ˆθ n is an estimator of θ then we say that the estimator is a consistent estimator of θ if (The opposite is not true in general). plim θ n = θ 25

27 If E(Θ n ) c and V ar(θ n ) 0 as n then Θ n converges in probability to c. If ˆθ n is an asymptotically unbiased estimator of θ and V ar(ˆθ n ) 0 as n then ˆθ n is a consistent estimator of θ. If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n then X is a consistent estimator of µ and S 2 is a consistent estimator of σ plim Rules If X 1,, X n is a random sample and Θ n is a given statistics such that plim Θ n = c then for any arbitrary function f : R R. plim f(θ n ) = f(c) If X 1,, X n is a random sample and Θ 1,, Θ k are k statistics (subscript n removed for clarity) such that plim Θ j = c j for j = 1,, k then for any arbitrary function g : R k R. plim g(θ 1,, Θ k ) = g(c 1,, c k ) In particular, plim Θ 1 Θ 2 = c 1 c 2 plim Θ 1 Θ 2 = c 1 c 2 Keep in mind that the plim-rules do not apply unless the statistics converge to a constant Convergence in Distribution If X 1, X 2 is an infinite sequence of random variables then we say that they converge in distribution to a random variable X if lim n F n(x) = F (x) for every x at which F is continuous. Here, F n is the cdf of X n and F is the cdf of X Central Limit Theorem If X 1,, X n is an IID random sample with E(X i ) = µ and V ar(x i ) = σ 2 for i = 1,, n then the Z-statistic converges in distribution to the standard normal. Formally, if Z n = X n µ σ/ n then Z n N(0, 1) where convergence is in distribution. With the assumptions of this section and with n large enough, X is approximately normally distributed with mean µ and variance σ 2 /n. With the assumptions of this section and with n large enough, the t-statistic t = X n µ S/ n is approximately t-distributed with n 1 degrees of freedom. 26

Lecture 2: Review of Probability

Lecture 2: Review of Probability Lecture 2: Review of Probability Zheng Tian Contents 1 Random Variables and Probability Distributions 2 1.1 Defining probabilities and random variables..................... 2 1.2 Probability distributions................................

More information

Lecture 25: Review. Statistics 104. April 23, Colin Rundel

Lecture 25: Review. Statistics 104. April 23, Colin Rundel Lecture 25: Review Statistics 104 Colin Rundel April 23, 2012 Joint CDF F (x, y) = P [X x, Y y] = P [(X, Y ) lies south-west of the point (x, y)] Y (x,y) X Statistics 104 (Colin Rundel) Lecture 25 April

More information

LIST OF FORMULAS FOR STK1100 AND STK1110

LIST OF FORMULAS FOR STK1100 AND STK1110 LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function

More information

Random Variables and Their Distributions

Random Variables and Their Distributions Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital

More information

Problem Set #5. Econ 103. Solution: By the complement rule p(0) = 1 p q. q, 1 x 0 < 0 1 p, 0 x 0 < 1. Solution: E[X] = 1 q + 0 (1 p q) + p 1 = p q

Problem Set #5. Econ 103. Solution: By the complement rule p(0) = 1 p q. q, 1 x 0 < 0 1 p, 0 x 0 < 1. Solution: E[X] = 1 q + 0 (1 p q) + p 1 = p q Problem Set #5 Econ 103 Part I Problems from the Textbook Chapter 4: 1, 3, 5, 7, 9, 11, 13, 15, 25, 27, 29 Chapter 5: 1, 3, 5, 9, 11, 13, 17 Part II Additional Problems 1. Suppose X is a random variable

More information

Course: ESO-209 Home Work: 1 Instructor: Debasis Kundu

Course: ESO-209 Home Work: 1 Instructor: Debasis Kundu Home Work: 1 1. Describe the sample space when a coin is tossed (a) once, (b) three times, (c) n times, (d) an infinite number of times. 2. A coin is tossed until for the first time the same result appear

More information

1 Probability theory. 2 Random variables and probability theory.

1 Probability theory. 2 Random variables and probability theory. Probability theory Here we summarize some of the probability theory we need. If this is totally unfamiliar to you, you should look at one of the sources given in the readings. In essence, for the major

More information

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems

MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems MA/ST 810 Mathematical-Statistical Modeling and Analysis of Complex Systems Review of Basic Probability The fundamentals, random variables, probability distributions Probability mass/density functions

More information

Review of Probability Theory

Review of Probability Theory Review of Probability Theory Arian Maleki and Tom Do Stanford University Probability theory is the study of uncertainty Through this class, we will be relying on concepts from probability theory for deriving

More information

Stat 5101 Notes: Algorithms

Stat 5101 Notes: Algorithms Stat 5101 Notes: Algorithms Charles J. Geyer January 22, 2016 Contents 1 Calculating an Expectation or a Probability 3 1.1 From a PMF........................... 3 1.2 From a PDF...........................

More information

ECON Fundamentals of Probability

ECON Fundamentals of Probability ECON 351 - Fundamentals of Probability Maggie Jones 1 / 32 Random Variables A random variable is one that takes on numerical values, i.e. numerical summary of a random outcome e.g., prices, total GDP,

More information

Preliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com

Preliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com 1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 3: Probability Models and Distributions (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics,

More information

2 (Statistics) Random variables

2 (Statistics) Random variables 2 (Statistics) Random variables References: DeGroot and Schervish, chapters 3, 4 and 5; Stirzaker, chapters 4, 5 and 6 We will now study the main tools use for modeling experiments with unknown outcomes

More information

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities

PCMI Introduction to Random Matrix Theory Handout # REVIEW OF PROBABILITY THEORY. Chapter 1 - Events and Their Probabilities PCMI 207 - Introduction to Random Matrix Theory Handout #2 06.27.207 REVIEW OF PROBABILITY THEORY Chapter - Events and Their Probabilities.. Events as Sets Definition (σ-field). A collection F of subsets

More information

UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables

UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables UQ, Semester 1, 2017, Companion to STAT2201/CIVL2530 Exam Formulae and Tables To be provided to students with STAT2201 or CIVIL-2530 (Probability and Statistics) Exam Main exam date: Tuesday, 20 June 1

More information

BASICS OF PROBABILITY

BASICS OF PROBABILITY October 10, 2018 BASICS OF PROBABILITY Randomness, sample space and probability Probability is concerned with random experiments. That is, an experiment, the outcome of which cannot be predicted with certainty,

More information

Statistics for scientists and engineers

Statistics for scientists and engineers Statistics for scientists and engineers February 0, 006 Contents Introduction. Motivation - why study statistics?................................... Examples..................................................3

More information

Northwestern University Department of Electrical Engineering and Computer Science

Northwestern University Department of Electrical Engineering and Computer Science Northwestern University Department of Electrical Engineering and Computer Science EECS 454: Modeling and Analysis of Communication Networks Spring 2008 Probability Review As discussed in Lecture 1, probability

More information

Exercises and Answers to Chapter 1

Exercises and Answers to Chapter 1 Exercises and Answers to Chapter The continuous type of random variable X has the following density function: a x, if < x < a, f (x), otherwise. Answer the following questions. () Find a. () Obtain mean

More information

Probability and Distributions

Probability and Distributions Probability and Distributions What is a statistical model? A statistical model is a set of assumptions by which the hypothetical population distribution of data is inferred. It is typically postulated

More information

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R

Random Variables. Random variables. A numerically valued map X of an outcome ω from a sample space Ω to the real line R In probabilistic models, a random variable is a variable whose possible values are numerical outcomes of a random phenomenon. As a function or a map, it maps from an element (or an outcome) of a sample

More information

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n

P (x). all other X j =x j. If X is a continuous random vector (see p.172), then the marginal distributions of X i are: f(x)dx 1 dx n JOINT DENSITIES - RANDOM VECTORS - REVIEW Joint densities describe probability distributions of a random vector X: an n-dimensional vector of random variables, ie, X = (X 1,, X n ), where all X is are

More information

Recitation 2: Probability

Recitation 2: Probability Recitation 2: Probability Colin White, Kenny Marino January 23, 2018 Outline Facts about sets Definitions and facts about probability Random Variables and Joint Distributions Characteristics of distributions

More information

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016

Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 8. For any two events E and F, P (E) = P (E F ) + P (E F c ). Summary of basic probability theory Math 218, Mathematical Statistics D Joyce, Spring 2016 Sample space. A sample space consists of a underlying

More information

1 Review of Probability and Distributions

1 Review of Probability and Distributions Random variables. A numerically valued function X of an outcome ω from a sample space Ω X : Ω R : ω X(ω) is called a random variable (r.v.), and usually determined by an experiment. We conventionally denote

More information

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline.

Random Variables. Cumulative Distribution Function (CDF) Amappingthattransformstheeventstotherealline. Random Variables Amappingthattransformstheeventstotherealline. Example 1. Toss a fair coin. Define a random variable X where X is 1 if head appears and X is if tail appears. P (X =)=1/2 P (X =1)=1/2 Example

More information

ENGG2430A-Homework 2

ENGG2430A-Homework 2 ENGG3A-Homework Due on Feb 9th,. Independence vs correlation a For each of the following cases, compute the marginal pmfs from the joint pmfs. Explain whether the random variables X and Y are independent,

More information

Math 3215 Intro. Probability & Statistics Summer 14. Homework 5: Due 7/3/14

Math 3215 Intro. Probability & Statistics Summer 14. Homework 5: Due 7/3/14 Math 325 Intro. Probability & Statistics Summer Homework 5: Due 7/3/. Let X and Y be continuous random variables with joint/marginal p.d.f. s f(x, y) 2, x y, f (x) 2( x), x, f 2 (y) 2y, y. Find the conditional

More information

EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix)

EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) 1 EC212: Introduction to Econometrics Review Materials (Wooldridge, Appendix) Taisuke Otsu London School of Economics Summer 2018 A.1. Summation operator (Wooldridge, App. A.1) 2 3 Summation operator For

More information

Actuarial Science Exam 1/P

Actuarial Science Exam 1/P Actuarial Science Exam /P Ville A. Satopää December 5, 2009 Contents Review of Algebra and Calculus 2 2 Basic Probability Concepts 3 3 Conditional Probability and Independence 4 4 Combinatorial Principles,

More information

Probability- the good parts version. I. Random variables and their distributions; continuous random variables.

Probability- the good parts version. I. Random variables and their distributions; continuous random variables. Probability- the good arts version I. Random variables and their distributions; continuous random variables. A random variable (r.v) X is continuous if its distribution is given by a robability density

More information

Theory of probability and mathematical statistics

Theory of probability and mathematical statistics Theory of probability and mathematical statistics Tomáš Mrkvička Bibliography [1] J. [2] J. Andďż l: Matematickďż statistika, SNTL/ALFA, Praha 1978 Andďż l: Statistickďż metody, Matfyzpress, Praha 1998

More information

Exam P Review Sheet. for a > 0. ln(a) i=0 ari = a. (1 r) 2. (Note that the A i s form a partition)

Exam P Review Sheet. for a > 0. ln(a) i=0 ari = a. (1 r) 2. (Note that the A i s form a partition) Exam P Review Sheet log b (b x ) = x log b (y k ) = k log b (y) log b (y) = ln(y) ln(b) log b (yz) = log b (y) + log b (z) log b (y/z) = log b (y) log b (z) ln(e x ) = x e ln(y) = y for y > 0. d dx ax

More information

2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y.

2. Variance and Covariance: We will now derive some classic properties of variance and covariance. Assume real-valued random variables X and Y. CS450 Final Review Problems Fall 08 Solutions or worked answers provided Problems -6 are based on the midterm review Identical problems are marked recap] Please consult previous recitations and textbook

More information

STT 441 Final Exam Fall 2013

STT 441 Final Exam Fall 2013 STT 441 Final Exam Fall 2013 (12:45-2:45pm, Thursday, Dec. 12, 2013) NAME: ID: 1. No textbooks or class notes are allowed in this exam. 2. Be sure to show all of your work to receive credit. Credits are

More information

This does not cover everything on the final. Look at the posted practice problems for other topics.

This does not cover everything on the final. Look at the posted practice problems for other topics. Class 7: Review Problems for Final Exam 8.5 Spring 7 This does not cover everything on the final. Look at the posted practice problems for other topics. To save time in class: set up, but do not carry

More information

1.1 Review of Probability Theory

1.1 Review of Probability Theory 1.1 Review of Probability Theory Angela Peace Biomathemtics II MATH 5355 Spring 2017 Lecture notes follow: Allen, Linda JS. An introduction to stochastic processes with applications to biology. CRC Press,

More information

Final Exam # 3. Sta 230: Probability. December 16, 2012

Final Exam # 3. Sta 230: Probability. December 16, 2012 Final Exam # 3 Sta 230: Probability December 16, 2012 This is a closed-book exam so do not refer to your notes, the text, or any other books (please put them on the floor). You may use the extra sheets

More information

Chp 4. Expectation and Variance

Chp 4. Expectation and Variance Chp 4. Expectation and Variance 1 Expectation In this chapter, we will introduce two objectives to directly reflect the properties of a random variable or vector, which are the Expectation and Variance.

More information

01 Probability Theory and Statistics Review

01 Probability Theory and Statistics Review NAVARCH/EECS 568, ROB 530 - Winter 2018 01 Probability Theory and Statistics Review Maani Ghaffari January 08, 2018 Last Time: Bayes Filters Given: Stream of observations z 1:t and action data u 1:t Sensor/measurement

More information

Joint Probability Distributions and Random Samples (Devore Chapter Five)

Joint Probability Distributions and Random Samples (Devore Chapter Five) Joint Probability Distributions and Random Samples (Devore Chapter Five) 1016-345-01: Probability and Statistics for Engineers Spring 2013 Contents 1 Joint Probability Distributions 2 1.1 Two Discrete

More information

1: PROBABILITY REVIEW

1: PROBABILITY REVIEW 1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following

More information

MAS113 Introduction to Probability and Statistics. Proofs of theorems

MAS113 Introduction to Probability and Statistics. Proofs of theorems MAS113 Introduction to Probability and Statistics Proofs of theorems Theorem 1 De Morgan s Laws) See MAS110 Theorem 2 M1 By definition, B and A \ B are disjoint, and their union is A So, because m is a

More information

Stat 5101 Notes: Algorithms (thru 2nd midterm)

Stat 5101 Notes: Algorithms (thru 2nd midterm) Stat 5101 Notes: Algorithms (thru 2nd midterm) Charles J. Geyer October 18, 2012 Contents 1 Calculating an Expectation or a Probability 2 1.1 From a PMF........................... 2 1.2 From a PDF...........................

More information

matrix-free Elements of Probability Theory 1 Random Variables and Distributions Contents Elements of Probability Theory 2

matrix-free Elements of Probability Theory 1 Random Variables and Distributions Contents Elements of Probability Theory 2 Short Guides to Microeconometrics Fall 2018 Kurt Schmidheiny Unversität Basel Elements of Probability Theory 2 1 Random Variables and Distributions Contents Elements of Probability Theory matrix-free 1

More information

1 Review of Probability

1 Review of Probability 1 Review of Probability Random variables are denoted by X, Y, Z, etc. The cumulative distribution function (c.d.f.) of a random variable X is denoted by F (x) = P (X x), < x

More information

Week 12-13: Discrete Probability

Week 12-13: Discrete Probability Week 12-13: Discrete Probability November 21, 2018 1 Probability Space There are many problems about chances or possibilities, called probability in mathematics. When we roll two dice there are possible

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

Math-Stat-491-Fall2014-Notes-I

Math-Stat-491-Fall2014-Notes-I Math-Stat-491-Fall2014-Notes-I Hariharan Narayanan October 2, 2014 1 Introduction This writeup is intended to supplement material in the prescribed texts: Introduction to Probability Models, 10th Edition,

More information

Practice Examination # 3

Practice Examination # 3 Practice Examination # 3 Sta 23: Probability December 13, 212 This is a closed-book exam so do not refer to your notes, the text, or any other books (please put them on the floor). You may use a single

More information

If we want to analyze experimental or simulated data we might encounter the following tasks:

If we want to analyze experimental or simulated data we might encounter the following tasks: Chapter 1 Introduction If we want to analyze experimental or simulated data we might encounter the following tasks: Characterization of the source of the signal and diagnosis Studying dependencies Prediction

More information

Lecture 11. Probability Theory: an Overveiw

Lecture 11. Probability Theory: an Overveiw Math 408 - Mathematical Statistics Lecture 11. Probability Theory: an Overveiw February 11, 2013 Konstantin Zuev (USC) Math 408, Lecture 11 February 11, 2013 1 / 24 The starting point in developing the

More information

Lecture 2: Repetition of probability theory and statistics

Lecture 2: Repetition of probability theory and statistics Algorithms for Uncertainty Quantification SS8, IN2345 Tobias Neckel Scientific Computing in Computer Science TUM Lecture 2: Repetition of probability theory and statistics Concept of Building Block: Prerequisites:

More information

Probability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3

Probability. Paul Schrimpf. January 23, Definitions 2. 2 Properties 3 Probability Paul Schrimpf January 23, 2018 Contents 1 Definitions 2 2 Properties 3 3 Random variables 4 3.1 Discrete........................................... 4 3.2 Continuous.........................................

More information

Topic 2: Probability & Distributions. Road Map Probability & Distributions. ECO220Y5Y: Quantitative Methods in Economics. Dr.

Topic 2: Probability & Distributions. Road Map Probability & Distributions. ECO220Y5Y: Quantitative Methods in Economics. Dr. Topic 2: Probability & Distributions ECO220Y5Y: Quantitative Methods in Economics Dr. Nick Zammit University of Toronto Department of Economics Room KN3272 n.zammit utoronto.ca November 21, 2017 Dr. Nick

More information

1 Presessional Probability

1 Presessional Probability 1 Presessional Probability Probability theory is essential for the development of mathematical models in finance, because of the randomness nature of price fluctuations in the markets. This presessional

More information

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015

Part IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015 Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.

More information

STA1000F Summary. Mitch Myburgh MYBMIT001 May 28, Work Unit 1: Introducing Probability

STA1000F Summary. Mitch Myburgh MYBMIT001 May 28, Work Unit 1: Introducing Probability STA1000F Summary Mitch Myburgh MYBMIT001 May 28, 2015 1 Module 1: Probability 1.1 Work Unit 1: Introducing Probability 1.1.1 Definitions 1. Random Experiment: A procedure whose outcome (result) in a particular

More information

More than one variable

More than one variable Chapter More than one variable.1 Bivariate discrete distributions Suppose that the r.v. s X and Y are discrete and take on the values x j and y j, j 1, respectively. Then the joint p.d.f. of X and Y, to

More information

Multivariate Random Variable

Multivariate Random Variable Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate

More information

SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416)

SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416) SUMMARY OF PROBABILITY CONCEPTS SO FAR (SUPPLEMENT FOR MA416) D. ARAPURA This is a summary of the essential material covered so far. The final will be cumulative. I ve also included some review problems

More information

Quick Tour of Basic Probability Theory and Linear Algebra

Quick Tour of Basic Probability Theory and Linear Algebra Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra CS224w: Social and Information Network Analysis Fall 2011 Quick Tour of and Linear Algebra Quick Tour of and Linear Algebra Outline Definitions

More information

STAT 512 sp 2018 Summary Sheet

STAT 512 sp 2018 Summary Sheet STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}

More information

14.30 Introduction to Statistical Methods in Economics Spring 2009

14.30 Introduction to Statistical Methods in Economics Spring 2009 MIT OpenCourseWare http://ocw.mit.edu 14.30 Introduction to Statistical Methods in Economics Spring 2009 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.

More information

1 Random Variable: Topics

1 Random Variable: Topics Note: Handouts DO NOT replace the book. In most cases, they only provide a guideline on topics and an intuitive feel. 1 Random Variable: Topics Chap 2, 2.1-2.4 and Chap 3, 3.1-3.3 What is a random variable?

More information

September Statistics for MSc Weeks 1-2 Probability and Distribution Theories

September Statistics for MSc Weeks 1-2 Probability and Distribution Theories September Statistics for MSc Weeks 1-2 Probability and Distribution Theories Ali C. Tasiran Department of Economics, Mathematics and Statistics Malet Street, London WC1E 7HX September 2014 Contents 1 Introduction

More information

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable

Lecture Notes 1 Probability and Random Variables. Conditional Probability and Independence. Functions of a Random Variable Lecture Notes 1 Probability and Random Variables Probability Spaces Conditional Probability and Independence Random Variables Functions of a Random Variable Generation of a Random Variable Jointly Distributed

More information

Review of Statistics I

Review of Statistics I Review of Statistics I Hüseyin Taştan 1 1 Department of Economics Yildiz Technical University April 17, 2010 1 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution

More information

Bivariate distributions

Bivariate distributions Bivariate distributions 3 th October 017 lecture based on Hogg Tanis Zimmerman: Probability and Statistical Inference (9th ed.) Bivariate Distributions of the Discrete Type The Correlation Coefficient

More information

3. Probability and Statistics

3. Probability and Statistics FE661 - Statistical Methods for Financial Engineering 3. Probability and Statistics Jitkomut Songsiri definitions, probability measures conditional expectations correlation and covariance some important

More information

1 Random variables and distributions

1 Random variables and distributions Random variables and distributions In this chapter we consider real valued functions, called random variables, defined on the sample space. X : S R X The set of possible values of X is denoted by the set

More information

STAT Chapter 5 Continuous Distributions

STAT Chapter 5 Continuous Distributions STAT 270 - Chapter 5 Continuous Distributions June 27, 2012 Shirin Golchi () STAT270 June 27, 2012 1 / 59 Continuous rv s Definition: X is a continuous rv if it takes values in an interval, i.e., range

More information

Probability Notes. Compiled by Paul J. Hurtado. Last Compiled: September 6, 2017

Probability Notes. Compiled by Paul J. Hurtado. Last Compiled: September 6, 2017 Probability Notes Compiled by Paul J. Hurtado Last Compiled: September 6, 2017 About These Notes These are course notes from a Probability course taught using An Introduction to Mathematical Statistics

More information

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay

Random Variables. Saravanan Vijayakumaran Department of Electrical Engineering Indian Institute of Technology Bombay 1 / 13 Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay August 8, 2013 2 / 13 Random Variable Definition A real-valued

More information

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables.

Probability. Paul Schrimpf. January 23, UBC Economics 326. Probability. Paul Schrimpf. Definitions. Properties. Random variables. Probability UBC Economics 326 January 23, 2018 1 2 3 Wooldridge (2013) appendix B Stock and Watson (2009) chapter 2 Linton (2017) chapters 1-5 Abbring (2001) sections 2.1-2.3 Diez, Barr, and Cetinkaya-Rundel

More information

Multiple Random Variables

Multiple Random Variables Multiple Random Variables Joint Probability Density Let X and Y be two random variables. Their joint distribution function is F ( XY x, y) P X x Y y. F XY ( ) 1, < x

More information

MAS223 Statistical Inference and Modelling Exercises

MAS223 Statistical Inference and Modelling Exercises MAS223 Statistical Inference and Modelling Exercises The exercises are grouped into sections, corresponding to chapters of the lecture notes Within each section exercises are divided into warm-up questions,

More information

f X, Y (x, y)dx (x), where f(x,y) is the joint pdf of X and Y. (x) dx

f X, Y (x, y)dx (x), where f(x,y) is the joint pdf of X and Y. (x) dx INDEPENDENCE, COVARIANCE AND CORRELATION Independence: Intuitive idea of "Y is independent of X": The distribution of Y doesn't depend on the value of X. In terms of the conditional pdf's: "f(y x doesn't

More information

Continuous random variables

Continuous random variables Continuous random variables Can take on an uncountably infinite number of values Any value within an interval over which the variable is definied has some probability of occuring This is different from

More information

Review of Statistics

Review of Statistics Review of Statistics Topics Descriptive Statistics Mean, Variance Probability Union event, joint event Random Variables Discrete and Continuous Distributions, Moments Two Random Variables Covariance and

More information

Problem 1. Problem 2. Problem 3. Problem 4

Problem 1. Problem 2. Problem 3. Problem 4 Problem Let A be the event that the fungus is present, and B the event that the staph-bacteria is present. We have P A = 4, P B = 9, P B A =. We wish to find P AB, to do this we use the multiplication

More information

Lecture 6 Basic Probability

Lecture 6 Basic Probability Lecture 6: Basic Probability 1 of 17 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 6 Basic Probability Probability spaces A mathematical setup behind a probabilistic

More information

Chapter 2: Random Variables

Chapter 2: Random Variables ECE54: Stochastic Signals and Systems Fall 28 Lecture 2 - September 3, 28 Dr. Salim El Rouayheb Scribe: Peiwen Tian, Lu Liu, Ghadir Ayache Chapter 2: Random Variables Example. Tossing a fair coin twice:

More information

1 Probability and Random Variables

1 Probability and Random Variables 1 Probability and Random Variables The models that you have seen thus far are deterministic models. For any time t, there is a unique solution X(t). On the other hand, stochastic models will result in

More information

Multiple Random Variables

Multiple Random Variables Multiple Random Variables This Version: July 30, 2015 Multiple Random Variables 2 Now we consider models with more than one r.v. These are called multivariate models For instance: height and weight An

More information

Elements of Probability Theory

Elements of Probability Theory Short Guides to Microeconometrics Fall 2016 Kurt Schmidheiny Unversität Basel Elements of Probability Theory Contents 1 Random Variables and Distributions 2 1.1 Univariate Random Variables and Distributions......

More information

Review 1: STAT Mark Carpenter, Ph.D. Professor of Statistics Department of Mathematics and Statistics. August 25, 2015

Review 1: STAT Mark Carpenter, Ph.D. Professor of Statistics Department of Mathematics and Statistics. August 25, 2015 Review : STAT 36 Mark Carpenter, Ph.D. Professor of Statistics Department of Mathematics and Statistics August 25, 25 Support of a Random Variable The support of a random variable, which is usually denoted

More information

Preliminary Statistics. Lecture 3: Probability Models and Distributions

Preliminary Statistics. Lecture 3: Probability Models and Distributions Preliminary Statistics Lecture 3: Probability Models and Distributions Rory Macqueen (rm43@soas.ac.uk), September 2015 Outline Revision of Lecture 2 Probability Density Functions Cumulative Distribution

More information

Stat 704 Data Analysis I Probability Review

Stat 704 Data Analysis I Probability Review 1 / 39 Stat 704 Data Analysis I Probability Review Dr. Yen-Yi Ho Department of Statistics, University of South Carolina A.3 Random Variables 2 / 39 def n: A random variable is defined as a function that

More information

Math Review Sheet, Fall 2008

Math Review Sheet, Fall 2008 1 Descriptive Statistics Math 3070-5 Review Sheet, Fall 2008 First we need to know about the relationship among Population Samples Objects The distribution of the population can be given in one of the

More information

Review: mostly probability and some statistics

Review: mostly probability and some statistics Review: mostly probability and some statistics C2 1 Content robability (should know already) Axioms and properties Conditional probability and independence Law of Total probability and Bayes theorem Random

More information

Algorithms for Uncertainty Quantification

Algorithms for Uncertainty Quantification Algorithms for Uncertainty Quantification Tobias Neckel, Ionuț-Gabriel Farcaș Lehrstuhl Informatik V Summer Semester 2017 Lecture 2: Repetition of probability theory and statistics Example: coin flip Example

More information

Properties of Summation Operator

Properties of Summation Operator Econ 325 Section 003/004 Notes on Variance, Covariance, and Summation Operator By Hiro Kasahara Properties of Summation Operator For a sequence of the values {x 1, x 2,..., x n, we write the sum of x 1,

More information

Introduction. The Linear Regression Model One popular model is the linear regression model. It writes as :

Introduction. The Linear Regression Model One popular model is the linear regression model. It writes as : Introduction Definition From Wikipedia : The two main purposes of econometrics are to give empirical content to economic theory and to subject economic theory to potentially falsifying tests. Another popular

More information

CS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro

CS37300 Class Notes. Jennifer Neville, Sebastian Moreno, Bruno Ribeiro CS37300 Class Notes Jennifer Neville, Sebastian Moreno, Bruno Ribeiro 2 Background on Probability and Statistics These are basic definitions, concepts, and equations that should have been covered in your

More information

1 Basic continuous random variable problems

1 Basic continuous random variable problems Name M362K Final Here are problems concerning material from Chapters 5 and 6. To review the other chapters, look over previous practice sheets for the two exams, previous quizzes, previous homeworks and

More information

Definition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R

Definition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R Random Variables Definition: A random variable X is a real valued function that maps a sample space S into the space of real numbers R. X : S R As such, a random variable summarizes the outcome of an experiment

More information

Topics in Probability and Statistics

Topics in Probability and Statistics Topics in Probability and tatistics A Fundamental Construction uppose {, P } is a sample space (with probability P), and suppose X : R is a random variable. The distribution of X is the probability P X

More information

Master s Written Examination

Master s Written Examination Master s Written Examination Option: Statistics and Probability Spring 016 Full points may be obtained for correct answers to eight questions. Each numbered question which may have several parts is worth

More information

Math 494: Mathematical Statistics

Math 494: Mathematical Statistics Math 494: Mathematical Statistics Instructor: Jimin Ding jmding@wustl.edu Department of Mathematics Washington University in St. Louis Class materials are available on course website (www.math.wustl.edu/

More information