Chapter 1 Describing Data

Transcription

1 Chapter 1 Describing Data

2 Variable Basics Def: A Variable is any characteristic of an individual. Def: Individuals are the objects described by data. Note the term individual is somewhat flawed. Sometimes groups of things constitute an individual.

3 Variable Basics Broad categories of variables Categorical (AKA Qualitative) Groups or categories Quantitative (AKA Numerical) variables for which arithmetic operations make sense (summing, averaging, etc.) Continuous: No theoretical end to the ability to subdivide gradations Discrete: Variables with finite possible values. Often things we count (baskets made, failures experienced, dots on a die)

4 Levels of Measurement* Remember NOIR Nominal ( Names ) Ordinal ( Order of things ) Interval ( Same gaps between marks on a scale ) Ratio ( Twice is nice There s a true zero, and doubling really means doubling!) * Not officially part of the AP curriculum

5 Variability Variables vary! That s why we call them variables. Variability is how far from the middle. (Much more on this later) A Distribution of a variable is some kind of display that tells us: What values the variable takes on How often those values occur, or how probable they are to occur.

6 Graphs involving Categorical Variables Bar Graphs Bars don t touch Horizontal axis is a categorical variable Vertical axis is a numerical variable (Usually a count or a percent) See p. 8 for How to construct a bar graph.

7 Graphs involving Categorical Variables Pie Charts Must account for 100% of the data. Can use an other category if necessary. Only trick to constructing a pie chart is to equate percent with central angle Θ = (%) x 100 x 360

8 Displays of quantitative variables Two proto-histograms 1. Dotplot (see p. 11) Easy! Dotplots preserve your data it s all still there. 2. Stemplot All data is preserved. Final digits are the leaves Must have a key: 5 4 = 54 Apples Use a back-to-back stemplot to compare two distributions (see p. 58 for a good example) Space digits the same laterally. Then, when you re done, you can tilt your head to the side and see a sort of histogram. This shows the shape of the distribution.

9 Describing Data SOCS/U Shape Outliers Center Spread Unusual Features

10 Measures of Center (AKA Measures of Central Tendency ) Mean Median Mode

11 Side Trip The 5-number summary Min Q1 (The Median of the lower half) Median (AKA Q2) Q3 (The Median of the upper half) Max (AKA Q4)

12 The -ile suffix the number at or below which are fraction of the data E.g: The 75 th percentile is the number at or below which are 75 percent of all values in the data. E.g.: The 3 rd Quartile is the number at or below which are 3 quarters of all data.

13 Hot Calculator Move! 1-var stats (Stat Calc 1-var stats) Gives you summary info for your data, including the mean, standard deviation and the 5- number summary

14 Histograms (See text) Consider home prices in a nice neighborhood 2, 3, 3, 4, 5, 8, 10, 13 Task: Build a 4-class histogram

15 Histograms 2, 3, 3, 4, 5, 8, 10, 13 Class width: Range divided by number of classes, then round up to the next integer. (13-2)/4 = 2 +. Round up to 3.

16 In which class is the 40 th percentile? n=13 n=17 n=5 n=1 n=0 n=2 n=4 n=0 n=1

17 Measures of Spread (AKA Measures of Dispersion ) Range (Max Min = Range) Interquartile Range (Q3 Q1) = IQR Standard Deviation (More to follow)

18 Histograms Warmup Consider the following data regarding muffins produced by students for a class 4, 5, 6, 7, 8, 10, 12, 18, 22, 24, 36, 30, 31 16, 17, 19, 20, 38, 39, 45, 50 Task: Build a 5-class histogram with the smallest class starting at 4 First-Establish the width of each class algebraically (Here I shifted to the whiteboard for a demonstration)

19 Histograms Warmup Consider the following data regarding muffins produced by students for a class 4, 5, 6, 7, 8, 10, 12, 18, 22, 24, 36, 30, 31 16, 17, 19, 20, 38, 39, 45, 50 Task: Build a 5-class histogram with the smallest class starting at 4 First-Establish the width of each class algebraically After you build a histogram, build a cumulative relative frequency ogive for the same data.

20 Outliers! Defined: Any observation that s not part of the overall pattern. The 1.5 IQR Rule On the high side: A number is an outlier if it is greater than Q IQR On the low side: A number is an outlier if it is lower than Q1 1.5 IQR

21 Outliers Data: 4, 3, 6, 8, 2, 1, 16, 7 Task: Determine whether 16 is an outlier. Establish the 5-number summary 1.5 IQR = Q IQR = So, 16 an outlier Yes/ No is / is not Min = Q1 = Med = Q3 = Max =

22 Outliers Data: -1, 4, 6, 8, 7, 6, 9, 5 Task: Determine whether -1 is an outlier. Establish the 5-number summary 1.5 IQR = Q1-1.5 IQR = So, -1 an outlier Yes/ No is / is not Min = Q1 = Med = Q3 = Max =

23 While we re at it: Boxplots Four elements to a successful (fully creditworthy) boxplot: A scale distinct from the plot itself The boxplot itself Labels along the top of the boxplot (Min, Q1, Median, Q3, Max) Values along the bottom. Whiteboard demo.

24 Variance, then Standard Deviation Standard Deviation: The most common measure of spread Appropriate when you ve chosen mean as your measure of center. This implies an approximately symmetric distribution Variance: The waypoint on the way to calculating standard deviation

25 Variance For a Sample: s 2 s 2 ( xx) n1 ( 2n 1) ( xx) 2 2 ( xx) n 2 For a Population 2 s 2 2 ( xx) n1 ( N ) 2 ( x) ( xx) n 2 2

26 Standard Deviation For a Sample: s ( xx) 2 ( n1) For a Population: ( 2x ) s 2 ( N) ( xx) n1 2 2 ( xx) n 2

27 Why n-1 for a sample Edward Sapir(?) Possibly the most frequently asked and least frequently answered question is why does the definition of the standard deviation involve division by -1, when might seem the obvious choice. This is a question which perplexes introductory statistics students and calculator manufacturers alike. The explanations given in calculator manuals tend to range from obscure to fanciful, and both options are given on the keypad, usually labelled s-1 and s to add to the confusion. (The symbol s is reserved for the standard deviation of a random variable or a population/distribution, an entity which lecturers valiantly try but usually fail to keep distinct from its sample counterpart.) Australian students first meet the standard deviation in secondary school, where the definition given does indeed involve division by. This definition is preferred to avoid the question about the -1 being raised it would seem. Secondary school teachers have a hard enough life as it is. And it must be remembered that the difference between the two definitions is largely academic for all but the smallest of sample sizes (say, less than 10 observations). So, don t get too agitated by the revamped definition. The truth can be told but the telling usually quells the desire to know. If the fire still burns in your belly, read on. The most widely accepted explanation involves the concept of unbiasedness, and I see some have stopped reading already. If you fire arrows at a target and consistently hit a mark 5 cm to the left of the bullseye, there is something wrong with your aim. It shows a bias. The definition of the sample which involves division by has this flaw. It consistently underestimates the variance of the population/distribution from which the sample was drawn. The -1 formula fixes the astigmatism. Compelling and relatively simple as this argument is, it doesn t quite ring true. Both definitions of the sample standard deviation produce biased estimates of the of the population/distribution, although the -1 alternative is less biased. If you re after unbiasedness, why not use a definition which gives you unbiasedness where it s needed - on the original scale of measurement, rather than on the squared scale. Such a contender exists, but it involves gamma functions in the definition, and I see quite a few more people have drifted away. (Gamma functions extend the concept of factorials to non-integers.) The real reason is a simple housekeeping issue. If you deal with the -1 straight away in the definition of the standard deviation, it doesn t keep popping up in every subsequent procedure involving the standard deviation, to the increasing annoyance of all concerned. The subsequent procedures in question involve the definition of the and c2 distributions where the issue of degrees of freedom arises. Degrees of freedom means what it says - in how many independent directions can you move at once. If you re a point moving on a page, you are moving in two dimensions and you have correspondingly two degrees of freedom. The freedom to move up the page and the freedom to move across it. Any motion on a page can be described in terms of these two independent motions. Now consider a sample of size. It inhabits an dimensional space. There are degrees of freedom in total. Each sample member is free to take any value it likes, independently from all the others. If however, you fix the sample mean, then the sample values are constrained to have a fixed sum. You can let -1 of them roam free, but the value of the remaining sample value is determined by the fixed sum. The space inhabited by the deviations from the sample mean is thus -1 dimensional rather than dimensional, since the deviations must sum to zero. The sum of the squared deviations, although looking like a sum of things is actually a sum of only -1 independent things, and its natural divisor - its degrees of freedom - is also -1. But wait, there s more. Degrees of freedom will return to haunt you if and when you do (ANOVA to its friends). You will be ahead of the game if you grasp the concept now. Degrees of freedom can neither be created nor destroyed. You start off with, the sample size. You use up a few trying to estimate the structure of the mean. For example, the mean could be a straight line, as in simple linear regression. You need two degrees of freedom to estimate the two characteristics possessed by all straight lines - a slope and an intercept. These characteristics are called parameters. So, two degrees of freedom have gone into the mean. This is the signal. Everything else in this model is noise. The remaining -2 degrees of freedom go into estimating the one parameter which describes the noise - the variance. If you re not part of the solution (the signal or mean) you re part of the problem (the noise). The simplest model is the one which says the mean is a single constant, ably estimated by the sample mean. Everything else is just inexplicable variation about that constant. That s -1 degrees of freedom s worth of noise, all kindly donated to the sample variance. Kim Anderson, mayor of Naples, Florida in Servant Leadership by Robert Greenleaf

28 Why n-1 for a sample? In other words It produces an unbiased estimator of the population SD, even though you only have a sample. It relates to the number of degrees of freedom the formula enjoys. If mean is fixed in place, only n-1 of the values could be filled in freely. That last value would have to respond to all those others. DON T WORRY ABOUT IT!

29 Important-to-know stuff about variance and standard deviation The sum of all deviations will always = 0, so if we merely averaged them, the average would be 0. (How boring!) This is why it makes sense to square the deviations before averaging them. However, squaring makes the units of variance the squares of the data s units. Hmmm. Dollars squared, grams squared it s wacky. After squaring the deviations and averaging them, the final square root unsquares the result. This returns the units of standard deviation back to the same units as the data. This alone is a good reason to use standard deviation instead of variance as our measure of spread.

30 Linear Transformations (Adding a constant) Exercise (All hands ) Take the numbers 2, 5, 7, 6, 4 (a population) Calculate mean: and Std Dev: Now add 2 to each value Calculate mean: and Std Dev: Conclusion:

31 Linear Transformations 1 (Adding a constant) Exercise (All hands ) Take the numbers 2, 5, 7, 6, 4 Calculate mean: and Std Dev: Now add 2 to each value Calculate mean: and Std Dev: Conclusion: Adding a number raises mean by that amount, but standard deviation remains unchanged.

32 Linear Transformations 1 (Multiplying by a constant) Exercise (All hands ) Now take the same numbers: 2, 5, 7, 6, 4 Originally, mean is 4.8, std dev (pop) is 1.72 Multiply each value by 3. Calculate mean: and Std Dev: How does this compare to the original mean and standard deviation?

33 Linear Transformations (Multiplying by a constant) Exercise (All hands ) Now take the same numbers: 2, 5, 7, 6, 4 Originally, mean is 4.8, std dev (pop) is 1.72 Multiply each value by 3. Calculate mean: and Std Dev: Both Mean and Standard Deviation went up by a factor of 3.

34 Linear Transformation Example 1 You are performing a physics experiment in which you measure temperatures in degrees Celsius. The mean temperature is 47, with a standard deviation of 13. Another physicist is interested in your research but wants all values in degrees Kelvin. What are the new mean and standard deviation?

35 Linear Transformation Example 1 You are performing a physics experiment in which you measure temperatures in degrees Celsius. The mean temperature is 47, with a standard deviation of 13. K = C +273, so add 273 to mean, and leave standard deviation alone. μ = 320, and σ = 13

36 Linear Transformation Example 2 Same setup: You are performing a physics experiment in which you measure temperatures in degrees Celsius. The mean temperature is 47, with a standard deviation of 13. Now, a different physicist wants results in degrees Fahrenheit. Conversion: F = (9/5)C +32

37 Linear Transformation Example 2 Conversion: F = (9/5)C +32 First, take care of the multiplication step: Mean = (9/5)47 = 84.6 degrees F Standard Deviation = (9/5) 13 = 23.5 degrees F Next, take care of the + 32 Add 32 to the mean: = Leave Standard Deviation unchanged = 23.5

38 Describing a set of Data Example: Lengths of spiders The distribution of spider lengths is approximately symmetric with a mean of 1.7 centimeters and a standard deviation of.3 CM. There are two outliers among the data at 3.3 and 3.7 CM, possibly accounted for by the fact that these two were found in a different area. There is a gap in the data between.8 and 1.0 CM which cannot be accounted for by any known flaw in sampling.

39 Example: Problem 12 from HW 1-1 The distribution of percents of those aged 65 and older in each state is approximately symmetric, with a mean of 12.7 percent and a standard deviation of 1.94 percentage points. There are two outliers one at the high end and one at the low end. Alaska has only 5.5 percent of its population 65 or over, and Florida has 18.3% in that age group. On further review, Median and 5-Num Summary may be more approriate than Mean/Std Dev.There are two gaps in the data. No state has a percent 65 and over between 6 and 8%, or between 16 and 18 percent, so with the exception of the outliers, states percent 65 and over are from about 9 to 15% of their populations.

40 When Describing: Address each element of SOCS/U Not necessarily in SOCS/U order Speak to context. Ex: The distribution of spider lengths is Not simply, the distribution is Write in English, not bulletized grunts.