Introduce Exploration! Before we go on, notice one more thing. We'll come back to the derivation if we have time.

Transcription

1 Introduce Exploration! Before we go on, notice one more thing. We'll come back to the derivation if we have time. Simplifying the calculation of variance Notice that we can rewrite the calculation of a sample variance as shown below. This generally makes the calculation simpler. 1/17HW & Variance calc 1

2 20F.1: #1,3,4 7 (Variance & SD Intro) 20F.2: #1 2 (Samples vs.population) 20F.3: #1,3,5 (SD for data classes) QB: #3,7 (IB Practice) Take questions as needed Go through these quickly Stat Unit Test Monday, 1/28 Did you learn anything from problems 5 & 6? Adding a constant to all the data points increases μ by the constant but does not affect σ Multiplying all the data points by k increases μ and σ by a factor of k. Before we go, notice one more thing...we'll come back to the derivation if we have time. An alternative calculation for the variance can be shown to be: It can be a handy shortcut and can give insight into how the mean and the data values contribute to the variance. Up to now we've been looking at statistics of one variable measures, distributions, etc. We not turn our attention to questions involving two variables and the relationship between them. 1. Gain an intuitive understanding of correlation. 21A: #2 4 (Correlation) 21B: #1,2,4,6 (Finding r) Correlation the relationship or association between two variables. It involves several factors: Direction Linearity Strength Outliers Causation We'll discuss each of these ideas in turn. In the graphs below, each dot represents a measurement of two values that may be related in some way. Direction An upward trend. As one variable increases, the other one also increases. Positive correlation Examples? A downward trend. As one variable increases, the other one decreases. Negative correlation Examples? No obvious trend. Zero correlation Examples? Linearity Is the pattern a line? A relationship may be linear for some values, but then change. So be careful. Gradually changing relationships can appear linear. Watch the scale. Strength We will learn how to calculate numbers that represent the strength of a correlation. Outliers Values that fall "far" outside the general trend. There are other mathematical definitions of what constitutes an outlier similar to what we did with box and whiskers. Causation Just because two variables are correlated does not mean that a change in one variable causes the change in the other variable. Some examples: The liklihood of snow is correlated to the number of hours of daylight. Does daylight cause snow? Many social arguments hinge on the question of causation. 1. Consider the arguments around global warming. There is no argument around whether the climate is changing in a way that is correlated to changes in CO 2 levels in the atmosphere. Some people will argue, however, that the change is part of a natural cycle and not caused by the increase in CO 2 in the atmosphere. 2. Test scores are correlated with income. Does having more money make a person smarter? You can do scatter plots on your calculator. Put x values in one list (L 1) and y values in another list (L 2). Use 2nd/STATPLOT to set up a scatter plot, adjusting the window to see the full set of points. 1/17 21A Correlation 2

3 1. Understand the origins of Pearson's Correlation coefficient 2. Know how to calculate and interpret r 2 How can we assign a number to the correlation of two variables? The most common approach was made popular by Carl Pearson, an English mathematician in about Actually, it was his teacher, Sir Francis Galton, who provided the ideas. Galton was a cousin of Charles Darwin and was a promoter of eugenics, the science of improving humans by selective breeding of those with "desirable" traits (who chooses what desirable is?). Very interesting article at: Let's look: Correlation Coefficient This looks complex (it's not that bad) and is very tedious to calculate by hand. What's important here is to understand what the result means! Interpretating Peason's Correlation Coefficient r ranges between 1 and 1 r > 0 means positive correlation as x increases, so does y (in general). r < 0 means negative correlation as x increases, y generally decreases. r close to 0 means weak correlation r close to 1 means strong correlation Let's look at some visual examples: Guess the value of r To help you understand this, we'll do one by hand. This may be the only time in your life when you get to do this, so enjoy it! (You'll do it a couple times in the HW too, to reinforce the meaning of r) The numbers are nice, don't use a calc until the end. This is our target, but we need to break it down. Writing things in an orderly fashion is key! = 3 = 6 Now a table will help All right, now let's use the calculator Fortunately, you live in 2016 where a calculator will do much of this work for you. To begin, 1. STAT/EDIT: Enter x data into L 1 and y data into L 2 (you can use other lists) To draw a scatter plot, 2 nd STATPLOT and set things up. To determine the correlation coefficient, we will do a Linear Regression (more on that later basically, we're finding the "best" line that goes through the data). 1. MODE: Scroll down and turn Stat Diagnostics on! 2. STAT/CALC/LinReg(ax + b) (Option 4) 3. Use L 1 and L 2 (or the lists you entered your data in) 4. Use a FreqList if certain points occurred more than once and you've entered them into a list. 5. Leave Store RegEQ alone for now. You will see several numbers, the last of which is r. We will discuss the other values later. r = There is a fairly strong, negative correlation between the amount of chemical and the # of surviving beetles. The chemical seems to work. If we square r the value we get has an useful meaning. Coefficient of Determination = r 2 If x and y are causally related, then r 2 as a percent is the amount of variation in y that can be explained by the variation in x. So, in our example above let's assume that the chemical is, in fact, responsible for killing beetles. In that case, since r 2 = , we can conclude that about 73.8% of the change in number of surviving beetles is determined by the change in the amount of chemical added. The rest, about 26.2%, must be explained by other factors. Excel Regression Calculator Use your calculator for 21A: #2 4 (Correlation) 21A #3 & 4 and 21B #4 & 6 21B: #1,2,4,6 (Finding r) Stat Unit Test Monday, 1/28 1/17 21B Correlation Coefficient 3

4 21A: #2 4 (Scatter Plots) Questions? 21B: #1,2,4,6 (Correlation) Discuss #1, 2(Results),6 TED.MonaChalabi.BadStatistics 3 ways to Spot bad stats 1. Understand the origins of a line of best fit 2. Know how to find and interpret a linear regression with a calculator 21C: #2,3 (Line of best fit) 21D: #3,4 (Least Squares line) QB #1e,2,4,5 (2 Var stats) Remember our goal here is to describe the pattern of the data with a mathematical function, in this case, a line. We can estimate such a line by eye. A best fit line should have at least two properties: 1. It should go through the "middle" 2. It should have about the same number of points above it as below it. Stat Unit Test: next Monday, 1/28 Concentration of Benzene (ppb) The mean point is given by: Days after oil spill Once we have a line that "describes" the data, we can guess (estimate) what the value of y would be for values of x that we didn't measure! 1/22 21C Line of best fit by eye 4

5 1. Understand the origins of a line of best fit 2. Know how to find and interpret a linear regression with a calculator Let's get more precise about finding the best fit line (or, more generally, the best fit curve). Consider the following. Each data point differs from the given line of best fit by some amount. The residual or error is the vertical distance between a data point and the graph of a regression equation. Notice that residuals can be positive or negative (or zero!) So now we can see that we can put a number on the quality of the fit by summing up the sizes of the residuals. A regression equation that results in a small total residual will represent a better fit. Can we just add the residuals? NO! Because some are positive and some are negative, they would cancel if we did that. So instead we sum the squares of the residuals. This has two effects: All the residuals contribute a positive amount to the total. Larger residuals are weighted more heavily that smaller ones (since they are squared) The result is a value called the total squared error. Using summation notation, the total squared error is given by where f(x) is the regression line in question. One way to define the "best" fit regression equation is to find the one that has the least total squared error. That is: Least Squares Regression The function f(x) that results in the least total squared error is called the least squares regression function. For linear equations we sometimes call this the least square line. For a linear fit, the slope of the least squares regression line can be calculated from the data values using a rather complex formula: Slope of the Least Squares Regression Line The slope of the least squares regression line for a set of points is given by: This looks complex and a thorough development is beyond this course. But S x is the variance of x and S xy is the covariance of x and y. Covariance The covariance of two variables is a measure of the strength of the correlation between them and is given by: But back to the line of best fit. We now have a formula for its slope and we previously found the mean point so we can use point slope form to create... Equation of the Least Squares Regression Line The equation of the least squares regression line for a set of points is given by: Where and Just for the heck of it, let's see this in its full glory in slope intercept form. The least squares regression line for a set of points is given by: The good news, is that all of this can be calculated on your TI 84: Finding Best Fit Curves on TI Enter x values in L 1 and y values in L Set CATALOG/DiagOn if you want r values 3. Use STAT/CALC and select the kind of curve you desire: > LinReg (ax + b) Linear > QuadReg Quadratic > CubicReg Cubic > QuartReg Quartic > LinReg (a + bx) Linear > LnReg Logarithmic > ExpReg Exponential (y = ab x ) > PwrReg Power function (y = ax b ) > Logistic Logistic (growth that levels out) > SinReg Sinusoidal 4. The parameters of the best fit curve will be stored in VARS/STATISTICS/EQ > RegEQ has the equation itself which you can paste into a Y= function to graph it. We now understand how to find the equation of best fit for lines or other functions. Understand what the calculator is doing, but let it do the work for you. 21C: #2,3 (Line of best fit) 21D: #3,4 (Least Squares line) QB #1e,2,4,5 (2 Var stats) Stat Unit Test Monday, 1/28 1/22 21D Least squares regression 5

6 Return Investigations 21C: #2,3 (Line of best fit) Questions as needed 21D: #3,4 (Least Squares line) QB #1e,2,4,5 (2 Var stats) 1. Understand when and how to use interpolation and extrapolation using linear regression lines. 21E: #2,4,6 (Interpolation & Extrapolation) QB: #6 10 read only (2 Var Stats review) Stat Unit Test Monday! Note: The QB problems for this unit are coming from Math Studies because it's a new topic for SL. If you understand them well, you will do fine on this portion of your exam. What average attendance might the Pittsburgh Pirates (PIT) expect if they could raise their winning percentage to 50%? The first question is an example of interpolation, using a line of best fit to estimate points between measured points. If the Yankees (NYY) won 70% of their games, what should the capacity of their new stadium be to hold the average number of attendees? The second question is an example of extrapolation, using a line of best fit to estimate points outside the range of measured points. Some vocabulary: 21E: #2,4,6 (Interpolation & Extrapolation) QB: #6 10 read only (2 Var Stats review) Stat Unit Test Monday! 1/24 21E Interpolation & Extrapolation 6