Review of Statistics I

Transcription

1 Review of Statistics I Hüseyin Taştan 1 1 Department of Economics Yildiz Technical University April 17,

2 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution functions Probability density function Expectation operator Normal distribution and related distributions: t-distribution, Chi-square distribution, F-distribution 2

3 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution functions Probability density function Expectation operator Normal distribution and related distributions: t-distribution, Chi-square distribution, F-distribution 2

4 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution functions Probability density function Expectation operator Normal distribution and related distributions: t-distribution, Chi-square distribution, F-distribution 2

5 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution functions Probability density function Expectation operator Normal distribution and related distributions: t-distribution, Chi-square distribution, F-distribution 2

6 Review of Distribution Theory Random variables, discrete vs continuous Probability distribution functions Probability density function Expectation operator Normal distribution and related distributions: t-distribution, Chi-square distribution, F-distribution 2

7 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

8 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

9 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

10 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

11 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

12 Random Variables A random variable is a function from the sample space S to the real numbers. We may denote a random variable as X(S) which is a usual, measurable function The value of a random variable cannot be known before the realization of event - uncertainty- Notation: use capital letters X, Y, or Z or any other letter to denote random variables Specific values of random variable will be denoted using letters x, y or z Using this notation we may calculate P(X = x) for discrete random variables Two types of rv: discrete vs. continuous 3

13 Discrete Random Variable Can take a countable number of distinct possible values Whenever all possible values a random variable can assume can be listed (or counted) the random variable is discrete Example: the sum of two numbers on the top when two dices are rolled: x = 2,3,...,12 Example: the number of sales made by a salesperson per week: x = 0,1,2,... Example: the number of errors made by an accountant 4

14 Discrete Random Variable Can take a countable number of distinct possible values Whenever all possible values a random variable can assume can be listed (or counted) the random variable is discrete Example: the sum of two numbers on the top when two dices are rolled: x = 2,3,...,12 Example: the number of sales made by a salesperson per week: x = 0,1,2,... Example: the number of errors made by an accountant 4

15 Discrete Random Variable Can take a countable number of distinct possible values Whenever all possible values a random variable can assume can be listed (or counted) the random variable is discrete Example: the sum of two numbers on the top when two dices are rolled: x = 2,3,...,12 Example: the number of sales made by a salesperson per week: x = 0,1,2,... Example: the number of errors made by an accountant 4

16 Discrete Random Variable Can take a countable number of distinct possible values Whenever all possible values a random variable can assume can be listed (or counted) the random variable is discrete Example: the sum of two numbers on the top when two dices are rolled: x = 2,3,...,12 Example: the number of sales made by a salesperson per week: x = 0,1,2,... Example: the number of errors made by an accountant 4

17 Discrete Random Variable Can take a countable number of distinct possible values Whenever all possible values a random variable can assume can be listed (or counted) the random variable is discrete Example: the sum of two numbers on the top when two dices are rolled: x = 2,3,...,12 Example: the number of sales made by a salesperson per week: x = 0,1,2,... Example: the number of errors made by an accountant 4

18 Continuous Random Variable Can take uncountable number of values within an interval on the real line Many variables from business and economics belong to this category Example: Average disposable income in a city Example: Closing value of a stock exchange index Example: Inflation rate for a given month 5

19 Continuous Random Variable Can take uncountable number of values within an interval on the real line Many variables from business and economics belong to this category Example: Average disposable income in a city Example: Closing value of a stock exchange index Example: Inflation rate for a given month 5

20 Continuous Random Variable Can take uncountable number of values within an interval on the real line Many variables from business and economics belong to this category Example: Average disposable income in a city Example: Closing value of a stock exchange index Example: Inflation rate for a given month 5

21 Continuous Random Variable Can take uncountable number of values within an interval on the real line Many variables from business and economics belong to this category Example: Average disposable income in a city Example: Closing value of a stock exchange index Example: Inflation rate for a given month 5

22 Continuous Random Variable Can take uncountable number of values within an interval on the real line Many variables from business and economics belong to this category Example: Average disposable income in a city Example: Closing value of a stock exchange index Example: Inflation rate for a given month 5

23 Probability Distributions for Discrete R.V. Probability Distribution Function f(x) 0 f(x) = P(X = x) f(x) = 1 x Cumulative Distribution Function P(X x 0 ) = F(x 0 ) = x x 0 f(x) The probability distribution of a discrete random variable is a formula, graph or table that specifies the probability associated with each possible value the random variable can assume 6

24 Example Three coins are tossed. Let X denote the number of Heads Sample space contains 8 points: (HHH), (HHT), (HTH), (THH), (HTT), (THT), (TTH), ve (TTT) These 8 events are mutually exclusive and each has the same probability: 1/8 X can take four values: 0, 1, 2, 3 7

25 Example Three coins are tossed. Let X denote the number of Heads Sample space contains 8 points: (HHH), (HHT), (HTH), (THH), (HTT), (THT), (TTH), ve (TTT) These 8 events are mutually exclusive and each has the same probability: 1/8 X can take four values: 0, 1, 2, 3 7

26 Example Three coins are tossed. Let X denote the number of Heads Sample space contains 8 points: (HHH), (HHT), (HTH), (THH), (HTT), (THT), (TTH), ve (TTT) These 8 events are mutually exclusive and each has the same probability: 1/8 X can take four values: 0, 1, 2, 3 7

27 Example Three coins are tossed. Let X denote the number of Heads Sample space contains 8 points: (HHH), (HHT), (HTH), (THH), (HTT), (THT), (TTH), ve (TTT) These 8 events are mutually exclusive and each has the same probability: 1/8 X can take four values: 0, 1, 2, 3 7

28 Example continued Distribution of X Results x f(x) TTT 0 1/8 TTH 1 THT 1 3/8 HTT 1 THH 2 HTH 2 3/8 HHT 2 HHH 3 1/8 8

29 Example continued Distribution Function of X x f(x) = P(X = x) Note that f(x) = 1 x Find P(X 1) =? P(1 X 3) =? 9

30 Example continued Cumulative Distribution Function of X F(x) = P(X x) = 0, x < 0; 1 8, 0 x < 1; 1 2, 1 x < 2; 7 8, 2 x < 3; 1, x 3. 10

31 Example continued Distribution Function of X 0.4 OLASILIK FONKSIYONU f(x) x 1 BIRIKIMLI OLASILIK FONKSIYONU F(x) x 11

32 Expectations of Discrete R.V. Expected Value of discrete r.v. X E(X) = x xf(x) If g(x) is a function of X then the expected value of g(x) is E(g(X)) = x g(x)f(x) If g(x) = x 2 then E(g(X)) = E(X 2 ) = x x 2 f(x) 12

33 Expectations of Discrete R.V. Expected Value of discrete r.v. X E(X) = x xf(x) If g(x) is a function of X then the expected value of g(x) is E(g(X)) = x g(x)f(x) If g(x) = x 2 then E(g(X)) = E(X 2 ) = x x 2 f(x) 12

34 Expectations of Discrete R.V. Expected Value of discrete r.v. X E(X) = x xf(x) If g(x) is a function of X then the expected value of g(x) is E(g(X)) = x g(x)f(x) If g(x) = x 2 then E(g(X)) = E(X 2 ) = x x 2 f(x) 12

35 Example Let s find the expected value of X in the previous example E(X) = = 3 2 Find the expected value of g(x) = x 2 E(X 2 ) = = 3 Find the expected value of g(x) = 2x+3x 2 13

36 Example Let s find the expected value of X in the previous example E(X) = = 3 2 Find the expected value of g(x) = x 2 E(X 2 ) = = 3 Find the expected value of g(x) = 2x+3x 2 13

37 Example Let s find the expected value of X in the previous example E(X) = = 3 2 Find the expected value of g(x) = x 2 E(X 2 ) = = 3 Find the expected value of g(x) = 2x+3x 2 13

38 Variance of a Discrete RV Definition: Var(X) = E [(X E(X)) 2] = E [ (X 2 2XE(X) +(E(X)) 2 ) ] = E ( X 2) 2E(XE(X)) +E ( (E(X)) 2 ) ) = E ( X 2) 2E(X) 2 +(E(X)) 2 = E ( X 2) E(X) 2 Let µ x = E(X) then the variance can be written as Var(X) = E ( X 2) µ 2 x 14

39 Variance of a Discrete RV Definition: Var(X) = E [(X E(X)) 2] = E [ (X 2 2XE(X) +(E(X)) 2 ) ] = E ( X 2) 2E(XE(X)) +E ( (E(X)) 2 ) ) = E ( X 2) 2E(X) 2 +(E(X)) 2 = E ( X 2) E(X) 2 Let µ x = E(X) then the variance can be written as Var(X) = E ( X 2) µ 2 x 14

40 Moments of a Discrete RV Definiton: kth moment of a discrete rv is defined as µ k = E(X k ) = x x k f(x) k = 0,1,2, moment µ 1 = E(X) = population mean 2. moment µ 2 = E(X 2 ) = Var(X)+µ moment µ 3 = E(X 3 ) 4. moment µ 4 = E(X 4 ) 15

41 Central Moments of a Discrete RV Definiton: kth central moment of a discrete rv is defined as m k = E((X µ 1 ) k ) = x (x µ 1 ) k f(x) k = 0,1,2, central moment m 1 = 0 2. central moment m 2 = E((X µ 1 ) 2 ) = Var(X) 3. central moment m 3 = E((X µ 1 ) 3 ) 4. central moment m 4 = E((X µ 1 ) 4 ) 16

42 Standard Moments of a Discrete RV Definiton: kth standard moment of a discrete rv is defined as γ k = m k σ k k = 0,1,2,... σ is the population standard deviation: σ = Var(X) = E [(X µ 1 ) 2] 1. standart moment γ 1 = 0 2. standart moment γ 2 = 1 why? 3. standart moment γ 3 = m 3 skewness σ 3 4. standart moment γ 4 = m 4 kurtosis σ 4 17

43 Some Discrete Distributions Bernoulli Binomial Hypergeometric Poisson 18

44 Some Discrete Distributions Bernoulli Binomial Hypergeometric Poisson 18

45 Some Discrete Distributions Bernoulli Binomial Hypergeometric Poisson 18

46 Some Discrete Distributions Bernoulli Binomial Hypergeometric Poisson 18

47 Bernoulli(p) Distribution Expected Value: f(x) = { p, if X = 1 1 p, if X = 0 E(X) = x xf(x) = p 1+(1 p) 0 = p Second Moment: E(X 2 ) = x x 2 f(x) = p 1+(1 p) 0 = p Variance (second central moment): Var(X) = E(X 2 ) (E(X)) 2 = p p 2 = p(1 p) 19

48 Binomial Distribution Let X be the number of successes in an n independent trials of Bernoulli random experiment. If Y is distributed as Bernoulli(p) then X = Y has a Binomial(n,p) distribution. X denotes total number of successes. f(x) = ( n x ) p x (1 p) n x = n! x!(n x)! px (1 p) n x, x = 0,1,2,...,n E(X) = np Var(X) = np(1 p) 20

49 Hypergeometric Distribution If Bernoulli trials are not independent then the total number of successes does not have a Binomial distribution but follows hypergeometric distribution. In an N-element population containing B successes total number of successes X in a random sample n has the following distribution f(x) = ( B x )( ) N B n x ( ) N n Here x can take integer values between max(0,n (N B)) and min(n, B) E(X) = np, Var(X) = N n N 1 np(1 p), p = B N 21

50 Poisson Distribution Describes the distribution of the number of realizations of a certain event in a given period of time Notation: X P oisson(λ) Probability DF: f(x,λ) = λx e λ, x = 0,1,2,... x! E(X) = λ Var(X) = λ skewness = 1 λ excess kurtosis = 1 λ 22

51 Joint Distributions of Discrete RV Let X and Y be two discrete rv. Then the joint pd is f(x,y) = P(X = x Y = y) More generally, joint pd of k discrete rv denoted X 1,X 2,...,X k is f(x 1,x 2,...,x k ) = P(X 1 = x 1 X 2 = x 2,,..., X k = x k ) 23

52 Example X: number of customers waiting in line for Counter 1 in a bank, Y: number of customers waiting in line for Counter 2 in a bank. Joint pd for these two rv is given by Joint probability distribution of X and Y y \x Total Total

53 Marginal Distribution Function Marginal DF for X: f(x) = y f(x,y) Marginal DF Y: f(y) = x f(x,y) 25

54 Conditional Distribution Function Conditional DF for X given Y = y: f(x y) = f(x,y) f(y) Conditional DF for Y given X = x: f(y x) = f(x,y) f(x) 26

55 Statistical Independence Discrete rvs X and Y are said to be independent if and only if f(x,y) = f(x)f(y) or, in other words and f(x y) = f(x), f(y x) = f(y) 27

56 Covariance between two discrete rv Let g(x,y) be a function of X and Y. The expected value of this function is: E[g(X,Y)] = g(x,y)f(x,y) x y Now let g(x,y) = (X µ x )(Y µ y ). The expected value of this function is called covariance: Cov(X,Y) = (x µ x )(y µ y )f(x,y) x y It measures the linear association between two random variables. Statistical independence implies that the covariance is zero but the reverse is not true. 28

57 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

58 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

59 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

60 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

61 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

62 Probability Distributions for Continuous Random Variables A continuous rv can assume any value within some interval. Probability distribution for a continuous rv is a smooth function f(x) called probability density function (pdf). The probability associated with a certain value of X is 0. Properties of pdf f(x): f(x) 0 f(x)dx = 1 Pr(a < X < b) = b a f(x)dx 29

63 A Probability Density Function f(x) P(a<X<b) = b a f(x)dx 0 a b x 30

64 Cumulative Density Function Cumulative density function (cdf), denoted F(x), shows the probability that a random variable does not exceed a given value x Definition: F(x) = P(X x) = x f(t)dt From this definition cdf and pdf are related with f(x) = df(x) dx 31

65 Cumulative Density Function Cumulative density function (cdf), denoted F(x), shows the probability that a random variable does not exceed a given value x Definition: F(x) = P(X x) = x f(t)dt From this definition cdf and pdf are related with f(x) = df(x) dx 31

66 Cumulative Density Function Cumulative density function (cdf), denoted F(x), shows the probability that a random variable does not exceed a given value x Definition: F(x) = P(X x) = x f(t)dt From this definition cdf and pdf are related with f(x) = df(x) dx 31

67 Properties of CDF F( ) = 0, F(+ ) = 1 F(x) is a nondecreasing function of x: x 1 x 2, F(x 1 ) F(x 2 ). P(a < X < b) = F(b) F(a) = b a f(x)dx P( < X < + ) = P( < X < a)+p(a < X < b)+p(b < X < + ) a f(x)dx = f(x)dx + b a f(x)dx + + b f(x)dx F(+ ) F( ) = [F(a) F( )]+P(a < X < b)+[f(+ ) F(b)] 1 = F(a) 0+P(a < X < b)+1 F(b) P(a < X < b) = F(b) F(a) 32

68 Properties of CDF F( ) = 0, F(+ ) = 1 F(x) is a nondecreasing function of x: x 1 x 2, F(x 1 ) F(x 2 ). P(a < X < b) = F(b) F(a) = b a f(x)dx P( < X < + ) = P( < X < a)+p(a < X < b)+p(b < X < + ) a f(x)dx = f(x)dx + b a f(x)dx + + b f(x)dx F(+ ) F( ) = [F(a) F( )]+P(a < X < b)+[f(+ ) F(b)] 1 = F(a) 0+P(a < X < b)+1 F(b) P(a < X < b) = F(b) F(a) 32

69 Properties of CDF F( ) = 0, F(+ ) = 1 F(x) is a nondecreasing function of x: x 1 x 2, F(x 1 ) F(x 2 ). P(a < X < b) = F(b) F(a) = b a f(x)dx P( < X < + ) = P( < X < a)+p(a < X < b)+p(b < X < + ) a f(x)dx = f(x)dx + b a f(x)dx + + b f(x)dx F(+ ) F( ) = [F(a) F( )]+P(a < X < b)+[f(+ ) F(b)] 1 = F(a) 0+P(a < X < b)+1 F(b) P(a < X < b) = F(b) F(a) 32

70 Properties of CDF F( ) = 0, F(+ ) = 1 F(x) is a nondecreasing function of x: x 1 x 2, F(x 1 ) F(x 2 ). P(a < X < b) = F(b) F(a) = b a f(x)dx P( < X < + ) = P( < X < a)+p(a < X < b)+p(b < X < + ) a f(x)dx = f(x)dx + b a f(x)dx + + b f(x)dx F(+ ) F( ) = [F(a) F( )]+P(a < X < b)+[f(+ ) F(b)] 1 = F(a) 0+P(a < X < b)+1 F(b) P(a < X < b) = F(b) F(a) 32

71 CDF f(x) F(+ ) F(b) = 1 F(b) F(a) F( ) = F(a) b a f(x)dx = F(b) F(a) 0 a b x 33

72 Expectations of Continuous Random Variables E(X) µ x = If g(x) is a continuous function of X, then E(g(X)) = Var(X) = xf(x)dx g(x)f(x)dx (x µ x ) 2 f(x)dx 34

73 Variance of Continuous RV Var(X) = E [ (X E(X)) 2] = = = E(X 2 ) µ 2 x x 2 f(x)dx+µ 2 x ( x 2 f(x)dx Note that we used f(x)dx = 1 and xf(x)dx = E(X) µ x (x µ x ) 2 f(x)dx f(x)dx 2µ x xf(x)dx xf(x)dx ) 2 35

74 Moments of Continuous RV Definition: kth moment of a continuous rv X is given by µ k = E(X k ) = x k f(x)dx k = 0,1,2,... x X 1. moment µ 1 = E(X) = population mean 2. moment µ 2 = E(X 2 ) = Var(X)+µ moment µ 3 = E(X 3 ) 4. moment µ 4 = E(X 4 ) 36

75 Central Moments of Continuous RV Definition: kth central moment of X is m k = E((X µ 1 ) k ) = (x µ 1 ) k f(x)dx k = 0,1,2,... x X 1. central moment m 1 = 0 2. central moment m 2 = E((X µ 1 ) 2 ) = Var(X) 3. central moment m 3 = E((X µ 1 ) 3 ) 4. central moment m 4 = E((X µ 1 ) 4 ) 37

76 Properties of Expectation Operator Linearity: Let Y = a+bx then the expected value of Y is: E[Y] = E[a+bX] = a+be(x) More generally let X 1,X 2,...,X n be n continuous rvs and let Y be a linear combination of these rvs: Y = b 1 X 1 +b n X b n X n Expected value of Y is given by: E[Y] = b 1 E[X 1 ]+b 2 E[X 2 ]+...+b n E[X n ] or more compactly ( n ) E(Y) = E b i X i = i=1 n b i E(X i ) i=1 38

77 Properties of Expectation Operator Linearity: Let Y = a+bx then the expected value of Y is: E[Y] = E[a+bX] = a+be(x) More generally let X 1,X 2,...,X n be n continuous rvs and let Y be a linear combination of these rvs: Y = b 1 X 1 +b n X b n X n Expected value of Y is given by: E[Y] = b 1 E[X 1 ]+b 2 E[X 2 ]+...+b n E[X n ] or more compactly ( n ) E(Y) = E b i X i = i=1 n b i E(X i ) i=1 38

78 Properties of Expectation Operator For a nonlinear function of X we have (in general) E[h(X)] h(e(x)) For example, E(X 2 ) (E(X)) 2, E(ln(X)) ln(e(x)) For two continuous rvs X and Y ( ) X E E(X) Y E(Y) 39

79 Properties of Expectation Operator For a nonlinear function of X we have (in general) E[h(X)] h(e(x)) For example, E(X 2 ) (E(X)) 2, E(ln(X)) ln(e(x)) For two continuous rvs X and Y ( ) X E E(X) Y E(Y) 39

80 Properties of Expectation Operator For a nonlinear function of X we have (in general) E[h(X)] h(e(x)) For example, E(X 2 ) (E(X)) 2, E(ln(X)) ln(e(x)) For two continuous rvs X and Y ( ) X E E(X) Y E(Y) 39

81 Properties of Variance Let c be any constant then Var(c) = 0 Variance of Y = bx where b is a constant Var(Y) = Var(bX) = b 2 Var(X) Variance of Y = a+bx Var(Y) = Var(a+bX) = b 2 Var(X) For two independent continuous rvs X and Y Var(X +Y) = Var(X)+Var(Y) Var(X Y) = Var(X)+Var(Y) This can easily be generalized to n independent rvs. 40

82 Properties of Variance Let c be any constant then Var(c) = 0 Variance of Y = bx where b is a constant Var(Y) = Var(bX) = b 2 Var(X) Variance of Y = a+bx Var(Y) = Var(a+bX) = b 2 Var(X) For two independent continuous rvs X and Y Var(X +Y) = Var(X)+Var(Y) Var(X Y) = Var(X)+Var(Y) This can easily be generalized to n independent rvs. 40

83 Properties of Variance Let c be any constant then Var(c) = 0 Variance of Y = bx where b is a constant Var(Y) = Var(bX) = b 2 Var(X) Variance of Y = a+bx Var(Y) = Var(a+bX) = b 2 Var(X) For two independent continuous rvs X and Y Var(X +Y) = Var(X)+Var(Y) Var(X Y) = Var(X)+Var(Y) This can easily be generalized to n independent rvs. 40

84 Properties of Variance Let c be any constant then Var(c) = 0 Variance of Y = bx where b is a constant Var(Y) = Var(bX) = b 2 Var(X) Variance of Y = a+bx Var(Y) = Var(a+bX) = b 2 Var(X) For two independent continuous rvs X and Y Var(X +Y) = Var(X)+Var(Y) Var(X Y) = Var(X)+Var(Y) This can easily be generalized to n independent rvs. 40

85 Continuous Standard Uniform Distribution Notation: X U(0,1), pdf: { 1, if 0 < x < 1, f(x) = 0, otherwise. E(X) = 1 2 Var(X) =

86 General Uniform Distribution Notation: X U(a,b), pdf: f(x;a,b) = { 1 b a if a < x < b, 0, otherwise. E(X) = b+a 2 Median = b+a 2 Var(X) = (b a)2 12 Skewness = 0 Excess kurtosis =

87 General Uniform Distribution Expectation of X U(a,b): b x E(X) = a b a dx [ 1 b 2 a 2 ] = b a 2 = (b a)(b+a) 2(b a) = a+b 2 43

88 General Uniform Distribution X U(a,b) Expected value of g(x) = x 2 b E[g(x)] = x 2 1 a b a = b3 a 3 3(b a) = (b a)(b2 +ab+a 2 ) 3(b a) = a2 +ab+b 2 = E[X 2 ]. 3 44

89 General Uniform Distribution Variance of X U(a,b): Var(X) = E[(X E(X)) 2 ] = E(X 2 ) [E(X)] 2 = (a2 +ab+b 2 ) 3 = (b a)2 12 (a+b)2 4 45

90 General Uniform Distribution Cumulative density function (cdf) of U (a,b) : F(x) = P(X x) x 1 = a b a dt t x = b a a = x a b a, for a x b 0, for x < a; x a F(x) = b a, for a x b; 1, for x > b. 46

91 Uniform Distribution f(x) F(x) 1 b a 1 0 a b x 0 a b x 47

92 Example { e f(x) = x, 0 < x < ise; 0, deilse. 1 Show that this function is a pdf. 2 Draw the graph of this function and mark the area for X > 1. 3 Calculate probability: P(X > 1). 4 Find cdf. 48

93 Example { e f(x) = x, 0 < x < ise; 0, deilse. 1 Show that this function is a pdf. 2 Draw the graph of this function and mark the area for X > 1. 3 Calculate probability: P(X > 1). 4 Find cdf. 48

94 Example { e f(x) = x, 0 < x < ise; 0, deilse. 1 Show that this function is a pdf. 2 Draw the graph of this function and mark the area for X > 1. 3 Calculate probability: P(X > 1). 4 Find cdf. 48

95 Example { e f(x) = x, 0 < x < ise; 0, deilse. 1 Show that this function is a pdf. 2 Draw the graph of this function and mark the area for X > 1. 3 Calculate probability: P(X > 1). 4 Find cdf. 48

96 Example continued 1 Let us see if the conditions for pdf are satisfied: 1 (i) First of all, f(x) 0 condition is satisfied for all values within the interval 0 < x <. 2 (ii) Should integrate to 1: e = lim x e x = 0 0 e x dx = 1 e x 0 = 1 e ( e 0 ) = = 1 49

97 Example continued 1 Let us see if the conditions for pdf are satisfied: 1 (i) First of all, f(x) 0 condition is satisfied for all values within the interval 0 < x <. 2 (ii) Should integrate to 1: e = lim x e x = 0 0 e x dx = 1 e x 0 = 1 e ( e 0 ) = = 1 49

98 Example continued 1 Let us see if the conditions for pdf are satisfied: 1 (i) First of all, f(x) 0 condition is satisfied for all values within the interval 0 < x <. 2 (ii) Should integrate to 1: e = lim x e x = 0 0 e x dx = 1 e x 0 = 1 e ( e 0 ) = = 1 49

99 Example continued 1 P(X > 1) area shown on the figure. 2 P(X > 1) = 1 e x dx = e x 1 = e F(x) = x 0 e t dt = e t x 0 = e x +e 0 = 1 e x 50

100 Example continued 1 P(X > 1) area shown on the figure. 2 P(X > 1) = 1 e x dx = e x 1 = e F(x) = x 0 e t dt = e t x 0 = e x +e 0 = 1 e x 50

101 Example continued 1 P(X > 1) area shown on the figure. 2 P(X > 1) = 1 e x dx = e x 1 = e F(x) = x 0 e t dt = e t x 0 = e x +e 0 = 1 e x 50

102 Example continued CDF is F(x) = { 0, x < 0; 1 e x, 0 < x <. f ( 1x ) oyf: f( x) =e x F ( 1 x ) bof: f( x) =1 e x P( X > 1) = 1 e x dx x x 51

103 Joint Probability Density Function Let X and Y be two continuous random variables defined within the intervals < X < + and < Y < +. The joint pdf for X and Y, denoted f(x,y) is defined as f(x,y) 0, Pr(a < X < b,c < Y < d) = f(x,y)dxdy = 1, d b c a f(x, y)dxdy. 52

104 Joint Probability Density Function: Example Graph of joint density f(x,y) = xye (x2 +y 2), x > 0, y > 0, f(x,y) y x

105 Example { k(x+y), 0 < x < 1, 0 < y < 2 ise; f(x,y) = 0, deilse. Find the constant k Find P ( 0 < X < 1 2, 1 < Y < 2). 54

106 Example continued First for f(x,y) > 0 we must have k > 0. From the second condition we have k(x+y)dxdy = 1 2 ( ) 1 = k 0 2 +y dy = k = 3k = 1 ( ) 1 2 y + y Thus k = 1 3. The joint pdf is { 1 f(x,y) = 3 (x+y), 0 < x < 1, 0 < y < 2 ise; 0, deilse. 55

107 Example continued Probability is found as a volume measure: P (0 < X < 12 ), 1 < Y < 2 = = 1 3 = = (x+y)dxdy ( ) 2 y dy ) 2 ( 1 8 y + y

108 Marginal Density Function Definition: marginal pdf of X: f(x) = f(x,y)dy Bounds on the interval is the interval of Y. Marginal pdf of Y: f(y) = f(x,y)dx Bounds on the interval is the interval of X. 57

109 Example Using the joint pdf from the previous example find the marginal pdfs of X and Y. 2 1 f(x) = 0 3 (x+y)dy = 1 ) (xy + y = 2 3 (x+1) 0 58

110 Example continued Marginal pdf of X : Marginal pdf of Y: f(x) = g(y) = { 2 3 (x+1), 0 < x < 1 ; 0, otherwise. { 1 3 (y ), 0 < y < 2 ; 0, otherwise. 59

111 Conditional Density Function Given Y = y the conditional density of X is defined as : f(x y) = f(x,y) f(y) Similarly given X = x the conditional density of Y is defined as f(y x) = f(x,y) f(x) 60

112 Independence Recall that two events A and B are independent if only if: P(A B) = P(A)P(B) Similarly X and Y are independent if and only if the following condition is satisfied: f(x,y) = f(x)f(y) In other words if the joint density function can be written as the product of marginal densities then the two random variables are statistically independent. 61

113 Independence More generally let X 1,X 2,...,X n be n continuous random variables. If the joint density can be written as the product of marginal densities then these random variables are independent: f(x 1,x 2,...,x n ) = f 1 (x 1 ) f 2 (x 2 ),..., f n (x n ) n = f j (x j ) j=1 This property is useful for obtaining Maximum Likelihood estimators for certain parameters. 62

114 Independence: Example Determine if the random variables are independent in the previous example. f(x)g(x) = 2 3 (x+1)1 3 (y ) f(x,y) Thus X and Y are not independent. 63

115 Independence: Another Example Determine if X and Y are independent using the following joint pdf: { 1 f(x,y) = 9, for 1 < x < 4, 1 < y < 4; 0, otherwise. Marginal densities are Thus f(x) = g(y) = dy = dx = 1 3 f(x,y) = 1 9 = f(x)g(y) = ( 1 3 Therefore X and Y are statistically independent. )( )

116 Normal Distribution Notation: X N(µ,σ 2 ). pdf is given by ( 1 2σ 2(x µ)2 f(x;µ,σ 2 ) = 1 σ 2π exp E(X) = µ Var(X) = σ 2 skewness = 0 kurtosis = 3 ), < x < 65

117 Normal Distribution, σ 2 = 1 Different location parameters 0.4 Normal Dagilim, σ 2 = µ = 2 µ = 0 µ = 2 µ = µ = 5 φ(x) x 66

118 Normal Distribution, µ = o Different scale parameters 0.4 Normal Dagilim, µ= σ 2 = 1, µ = φ(x) 0.2 σ 2 = 2, µ = σ 2 = 3, µ = x 67

119 Standard Normal Distribution Define Z = X µ σ, pdf is given by φ(z) = 1 2π exp ( 12 z2 ), < z < E(Z) = 0 Var(Z) = 1 CDF is: Φ(z) = P(Z z) = z 1 exp ( 12 ) t2 dt 2π 68

120 Standard Normal Distribution 0.4 STANDART NORMAL DAGILIM φ(z) 1 STANDART NORMAL DAGILIM Φ(z)

121 Calculating Normal Distribution Probabilities Let X N(µ,σ 2 ). We want to calculate the following probability: P(a < X < b) = b a ( 1 σ 2π exp 1 ) 2σ 2(x µ)2 dx There is no closed-form solution to this integral. It can only be calculated using numerical methods. This requires using computational methods each time we need to evaluate a probability. Instead of this cumbersome method we can use normal distribution tables. Let us write the desired probability as follows: ( a µ P σ < X µ σ < b µ σ ) ( a µ = P σ < Z < b µ σ ) 70

122 Calculating Normal Distribution Probabilities ( a µ P σ < Z < b µ ) = Φ σ ( ) ( ) b µ a µ Φ σ σ Here Φ(z) = P(Z z) is the value of standard normal cdf at z. This can easily found by using standard normal tables. 71

123 Calculating Normal Distribution Probabilities Since the standard normal distribution is symmetric only positive values are listed in the tables. Using the symmetry property one can find the probabilities involving negative values by: e.g.: Φ( z) = P(Z z) = P(Z z) = 1 P(Z z) = 1 Φ(z) P(Z 1.25) = Φ( 1.25) = 1 Φ(1.25) = =

124 Calculating Normal Distribution Probabilities

125 Calculating Normal Distribution Probabilities P( 1.25<Z<1.25) = Φ(1.25) Φ( 1.25) = Φ(1.25) (1 Φ(1.25)) = = Φ( 1.25) = Φ(1.25) =

126 CENTRAL LIMIT THEOREM - CLT Let X 1,X 2,...,X n be a random sample of n iid random variables each having the same mean µ and variance σ 2. More compactly: X i i.i.d (µ,σ 2 ), i = 1,2,...,n iid means: identically and independently distributed Note that we do not mention the name of their distribution. Expected value and variance of the sum of these n iid variables will be: E[X 1 +X X n ] = E[X 1 ]+E[X 2 ]+...+E[X n ] = nµ Var[X 1 +X X n ] = Var[X 1 ]+Var[X 2 ]+...+Var[X n ] = nσ 2 75

127 CENTRAL LIMIT THEOREM - CLT Let X denote the sum of these rvs: X = X 1 +X X n, then Z = X E(X) Var(X) = X nµ nσ 2 = X n µ n 1/2 n σ = X µ σ/ n N(0,1) According to CLT as n, the expression above converges to standard normal distribution: Z N(0,1) 76

128 CENTRAL LIMIT THEOREM - CLT n= 1 n= n= n= n= n= n= n= n=

129 Law of Large Numbers - LLN According to LLN, the sample mean of n iid random variables converges to the population mean as the sample size n increases. Let X n = 1 n (X 1 +X X n ) be the sample mean then the LLN says that n, X n µ In other words, for a positive number ɛ that we can arbitrarily choose as small as possible we can write: lim P [ X n µ < ɛ ] = 1 n 78

130 CLT - Example Let X 1,X 2,...,X 12 be an iid random sample each distributed as U (0,b), b > 0. Using CLT show that the probability P( b 4 < X < 3b 4 ) is approximately Answer: These 12 iid rvs come from the same population. So we need to find the population mean and variance first. For Uniform(a, b) distribution these quantities are Hence in our example µ x = b+a 2, σ2 x = (b a)2 12 µ x = b 2, σ2 x = b

131 CLT - Example Let X 1,X 2,...,X 12 be an iid random sample each distributed as U (0,b), b > 0. Using CLT show that the probability P( b 4 < X < 3b 4 ) is approximately Answer: These 12 iid rvs come from the same population. So we need to find the population mean and variance first. For Uniform(a, b) distribution these quantities are Hence in our example µ x = b+a 2, σ2 x = (b a)2 12 µ x = b 2, σ2 x = b

132 CLT - Example, cont d Using CLT we have: P ( b 4 < X < 3b ) 4 Var(X) = σ2 x n = b2 144 ( b 4 = P b 2 < X µ 3b x b 12 σ 2 x /n < 4 ) b 2 b 12 = P( 3 < Z < 3) = Φ(3) (1 Φ(3)) = ( ) =

133 Chi-square Distribution Square of a standard normal random variable is distributed as chi-square with 1 degrees of freedom: If Z N(0,1) then Z 2 χ 2 1 Sum of squares of n independent standard normal variables is distributed as a chi-squared with n degrees of freedom: n If Z i i.i.d. N(0,1) i = 1,2,...,n then Zi 2 χ 2 n Chi-square distribution has one parameter: ν (nu) degrees of freedom. For χ 2 ν the pdf is 1 f(x) = 2 ν/2 Γ(ν/2) xν/2 1 e x/2, x > 0, ν > 0 where Γ is the gamma distribution: Γ(α) = 0 x α 1 e x dx, α > 0 81 i=1

134 Chi-square Distribution Square of a standard normal random variable is distributed as chi-square with 1 degrees of freedom: If Z N(0,1) then Z 2 χ 2 1 Sum of squares of n independent standard normal variables is distributed as a chi-squared with n degrees of freedom: n If Z i i.i.d. N(0,1) i = 1,2,...,n then Zi 2 χ 2 n Chi-square distribution has one parameter: ν (nu) degrees of freedom. For χ 2 ν the pdf is 1 f(x) = 2 ν/2 Γ(ν/2) xν/2 1 e x/2, x > 0, ν > 0 where Γ is the gamma distribution: Γ(α) = 0 x α 1 e x dx, α > 0 81 i=1

135 Chi-square Distribution Square of a standard normal random variable is distributed as chi-square with 1 degrees of freedom: If Z N(0,1) then Z 2 χ 2 1 Sum of squares of n independent standard normal variables is distributed as a chi-squared with n degrees of freedom: n If Z i i.i.d. N(0,1) i = 1,2,...,n then Zi 2 χ 2 n Chi-square distribution has one parameter: ν (nu) degrees of freedom. For χ 2 ν the pdf is 1 f(x) = 2 ν/2 Γ(ν/2) xν/2 1 e x/2, x > 0, ν > 0 where Γ is the gamma distribution: Γ(α) = 0 x α 1 e x dx, α > 0 81 i=1

136 Chi-square Distribution Let χ 2 ν be a chi-square random variable with ν degrees of freedom. Then the expected value and variance are: E(χ 2 ν ) = ν ve Var(χ2 ν ) = 2ν If the population is normal then the ratio of the sum of squared deviations from the sample mean to population variance has a chi-square distribution with n 1 degrees of freedom: (n 1)s 2 n i=1 σ 2 = (X i X) 2 σ 2 χ 2 n 1 Meaning of dof: the knowledge of n quantities (X i X) is equivalent to (n 1) mathematically independent components. Since we first estimate the sample mean from the observation set there are n 1 mathematically independent terms. 82

137 Chi-square Distribution Let χ 2 ν be a chi-square random variable with ν degrees of freedom. Then the expected value and variance are: E(χ 2 ν ) = ν ve Var(χ2 ν ) = 2ν If the population is normal then the ratio of the sum of squared deviations from the sample mean to population variance has a chi-square distribution with n 1 degrees of freedom: (n 1)s 2 n i=1 σ 2 = (X i X) 2 σ 2 χ 2 n 1 Meaning of dof: the knowledge of n quantities (X i X) is equivalent to (n 1) mathematically independent components. Since we first estimate the sample mean from the observation set there are n 1 mathematically independent terms. 82

138 Chi-square Distribution Let χ 2 ν be a chi-square random variable with ν degrees of freedom. Then the expected value and variance are: E(χ 2 ν ) = ν ve Var(χ2 ν ) = 2ν If the population is normal then the ratio of the sum of squared deviations from the sample mean to population variance has a chi-square distribution with n 1 degrees of freedom: (n 1)s 2 n i=1 σ 2 = (X i X) 2 σ 2 χ 2 n 1 Meaning of dof: the knowledge of n quantities (X i X) is equivalent to (n 1) mathematically independent components. Since we first estimate the sample mean from the observation set there are n 1 mathematically independent terms. 82

139 Chi-square Distribution Skewed to right, i.e., right tail is longer than the left. This means that the 3rd standard moment is positive. As the degrees of freedom parameter ν gets bigger distribution becomes more symmetric. In the limit, it converges to the normal distribution. Chi-square probabilities can easily be calculated using appropriate tables. 83

140 Chi-square Distribution Skewed to right, i.e., right tail is longer than the left. This means that the 3rd standard moment is positive. As the degrees of freedom parameter ν gets bigger distribution becomes more symmetric. In the limit, it converges to the normal distribution. Chi-square probabilities can easily be calculated using appropriate tables. 83

141 Chi-square Distribution Skewed to right, i.e., right tail is longer than the left. This means that the 3rd standard moment is positive. As the degrees of freedom parameter ν gets bigger distribution becomes more symmetric. In the limit, it converges to the normal distribution. Chi-square probabilities can easily be calculated using appropriate tables. 83

142 Chi-square Distribution 0.5 ν serbestlik dereceli ki kare dagilimi ν = ν = ν = ν =

143 Chi-square Distribution 0.1 ν=10 serbestlik dereceli ki kare dagiliminin olasilik yog.fonks f(χ 10 ) P(χ >15.99) = χ 2 (10) 85

144 Student t Distribution Let Z and Y be two rvs defined as follows: Z N(0,1), and Y χ 2 ν. The random variable defined below follows a t distribution with ν degrees of freedom: t ν = Z Y/ν t ν The rv t ν has a t distribution with ν dof defined in the denominator. PDF is given by: f(t) = Γ( ) ν+1 2 Γ(ν/2) πν 1 (1+(t 2 /ν)) (ν+1)/2, < t < It has one parameter (ν) and it has symmetric shape. For E(t ν ) = 0 and ν 3 Var(t ν ) = ν/(ν 2) ν, t ν N(0,1) 86

145 Student t Distribution Let Z and Y be two rvs defined as follows: Z N(0,1), and Y χ 2 ν. The random variable defined below follows a t distribution with ν degrees of freedom: t ν = Z Y/ν t ν The rv t ν has a t distribution with ν dof defined in the denominator. PDF is given by: f(t) = Γ( ) ν+1 2 Γ(ν/2) πν 1 (1+(t 2 /ν)) (ν+1)/2, < t < It has one parameter (ν) and it has symmetric shape. For E(t ν ) = 0 and ν 3 Var(t ν ) = ν/(ν 2) ν, t ν N(0,1) 86

146 Student t Distribution Let Z and Y be two rvs defined as follows: Z N(0,1), and Y χ 2 ν. The random variable defined below follows a t distribution with ν degrees of freedom: t ν = Z Y/ν t ν The rv t ν has a t distribution with ν dof defined in the denominator. PDF is given by: f(t) = Γ( ) ν+1 2 Γ(ν/2) πν 1 (1+(t 2 /ν)) (ν+1)/2, < t < It has one parameter (ν) and it has symmetric shape. For E(t ν ) = 0 and ν 3 Var(t ν ) = ν/(ν 2) ν, t ν N(0,1) 86

147 Student t Distribution 0.4 Standart Normal t ν=1 ν

148 Student t Distribution Standart Normal t ν ν=1 t ν ν=

149 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=

150 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=3 t ν ν=

151 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=3 t ν ν=4 0.3 t ν ν=

152 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=3 t ν ν=4 0.3 t ν ν=5 t ν ν=

153 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=3 t ν ν=4 0.3 t ν ν=5 t ν ν=10 t ν ν=

154 Student t Distribution 0.4 Standart Normal t ν ν= t ν ν=2 t ν ν=3 t ν ν=4 0.3 t ν ν=5 t ν ν=10 t ν ν= t ν ν=

155 Calculating Student t Probabilities s.d. Student t dagilimi α P(t 5 >2.015)=α= α t 5 95

156 Calculating Student t Probabilities s.d. t dagilimi ve standard normal dagilim Std Normal t

157 Calculating Student t Probabilities P( < t 10 < 2.228) = = s.d. Student t dagilimi P( 2.228<t 10 <2.228) = 1 α 0.05 α = α = 0.95 α =

158 F Distribution Let X 1 χ 2 k 1 and X 2 χ 2 k 2 be two independent chi-square random variables. Then the following random variable has an F distribution F = X 1/k 1 X 2 /k 2 F(k 1,k 2 ) F distribution has two parameters: k 1 is the numerator degrees of freedom k 2 is denominator degrees of freedom 98

159 F Distribution Let X 1 χ 2 k 1 and X 2 χ 2 k 2 be two independent chi-square random variables. Then the following random variable has an F distribution F = X 1/k 1 X 2 /k 2 F(k 1,k 2 ) F distribution has two parameters: k 1 is the numerator degrees of freedom k 2 is denominator degrees of freedom 98

160 F Distribution Let X 1 χ 2 k 1 and X 2 χ 2 k 2 be two independent chi-square random variables. Then the following random variable has an F distribution F = X 1/k 1 X 2 /k 2 F(k 1,k 2 ) F distribution has two parameters: k 1 is the numerator degrees of freedom k 2 is denominator degrees of freedom 98

161 F Distribution Let X 1 χ 2 k 1 and X 2 χ 2 k 2 be two independent chi-square random variables. Then the following random variable has an F distribution F = X 1/k 1 X 2 /k 2 F(k 1,k 2 ) F distribution has two parameters: k 1 is the numerator degrees of freedom k 2 is denominator degrees of freedom 98

162 F Distribution Let X 1 χ 2 k 1 and X 2 χ 2 k 2 be two independent chi-square random variables. Then the following random variable has an F distribution F = X 1/k 1 X 2 /k 2 F(k 1,k 2 ) F distribution has two parameters: k 1 is the numerator degrees of freedom k 2 is denominator degrees of freedom 98

163 F Distribution F(2,8) F(6,8) F(6,20) F(10,30)