Stat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables.

Size: px

Start display at page:

Download "Stat Lecture 20. Last class we introduced the covariance and correlation between two jointly distributed random variables."

Hilary Parrish
5 years ago
Views:

1 Stat Lecture 20 Recap of Last Class Last class we introduced the covariance and correlation between two jointly distributed random variables. Today: We will introduce the idea of a statistic and the sampling distribution of a statistic.

2 Samples and Statistics Consider observing n data values x 1,.., x n. These could be say, the test scores of n randomly selected students in a class. Before collecting and observing these data, there is uncertainty regarding the value of each x i, i = 1,..., n. As a result of this uncertainty, before the data becomes available, we view each observation as a random variable and denote the sample (before observation) by X 1,..., X N a collection of random variables.

3 Since X 1,..., X n are considered random variables, any function of these is also a random variable. For example the sample mean X = 1 n or the sample variance S 2 1 n 1 n i=1 n X i i=1 (X i X) 2 are considered random variables before the data are actually observed and the likelihood of X or S 2 taking on any particular value is governed by their probability distributions. More generally, a statistic is any quantity whose value can be calculated from sample data. Prior to observing the data there is uncertainty as to what value of any particular statistic will result a statistic is a random variable.

4 We will denote all statistics with uppercase letters when we are refereing to random variables. We will use lowercase letters to represent the calculated or observed value of a statistic. Example: 1. X is a random variable before we observe data 2. x is the calculate value after we observe x 1,..., x n. So this is just a number. Since any statistic S(X 1,..., X n ) is a random variable (before data are observed) it will have an associated probability distribution.

5 The probability distribution of any statistic is referred to as its sampling distribution. The sampling distribution describes how the statistic varies across all samples of data X 1,..., X n that might be selected. Definition: The random variable X 1,..., X n form a simple random sample of size n if 1. The X i s are independent random variables 2. Every X i has the same probability distribution. In this case we say that the X i s are independent and identically distributed (iid).

6 There are two general methods for obtaining information about the sampling distribution of a statistic. The first is based on analytically deriving the distribution based on probability rules. The second involves conducting a simulation experiment through the use of a computer. Example (analytic derivation): Suppose X 1,..., X n are a random sample from a Bernoulli distribution with parameter p X i iid Bern(p), i = 1,..., n and consider the statistic S(X 1,..., X n ) = n X i i=1 what is the sampling distribution of S(X 1,..., X n )?

7 Since each X i is Bernoulli with parameter p, we can think of each as the outcome of a dichotomous experiment yielding either a success (X i = 1) or a failure (X i = 0). In this case S(X 1,..., X n ) = n i=1 X i is the total number of successes in n independent trials each having probability p of success. S(X 1,..., X n ) Bin(n, p) In this case it was easy to derive the sampling distribution of the statistic analytically. In more complicated situations this is often not possible in which case we can use a simulation experiment to investigate the sampling distribution.

8 To conduct such a simulation experiment we need to specify the following: 1. The statistic of interest S(X 1,..., X n ) 2. The probability distribution of X 1,..., X n 3. The sample size n 4. The number of simulation replications k Given this information we 1. Use a computer to obtain k different random samples each of size n 2. For each simulated sample we calculate the value of the statistic 3. Construct a histogram of the k calculated values.

9 The histogram of the simulated values will give the approximate sampling distribution of the statistic. Example: Consider a sample of size n = 500 with X i iid Poisson(0.1) and suppose our statistic of interest was S(X 1,..., X 500 ) = X X2 500 n There is no easy way to analytically determine (based on what we have learned so far)the sampling distribution of S here. Instead we can simulate 1000 random samples each of size 500 with X i iid Poisson(0.1), i = 1,..., 500 and for each sample, calculate the value of S(X 1,..., X n ).

10 From this we will then have 1000 simulated value of the statistic: S 1,..., S The histogram of these simulated values gives us a rough idea of what the pmf of S(X 1,..., X 500 ) looks like:

11 Histogram of S from 1000 Simulations Frequency s

12 In this case the histogram of the simulated value looked normal... next class we will discuss the theory behind this observation.

13 Summary of Today s Class We introduced the idea of a statistic and its sampling distribution. Homework: Problem set 20

Bernoulli and Binomial Distributions. Notes. Bernoulli Trials. Bernoulli/Binomial Random Variables Bernoulli and Binomial Distributions.

Bernoulli and Binomial Distributions. Notes. Bernoulli Trials. Bernoulli/Binomial Random Variables Bernoulli and Binomial Distributions. Lecture 11 Text: A Course in Probability by Weiss 5.3 STAT 225 Introduction to Probability Models February 16, 2014 Whitney Huang Purdue University 11.1 Agenda 1 2 11.2 Bernoulli trials Many problems in