Statistics Terminology

Transcription

1 <run>:\the\world Machine Learning Camp Day 1: Exploring Data with Excel Statistics Terminology Variance Variance is a measure of how spread out a set of data or numbers is from its mean. For a set of numbers x 1, x 2,..., x n, their variance is given by σ 2 x, where σx 2 = (x 1 x) 2 + (x 2 x) (x n x) 2, n and x is the mean of x 1, x 2,..., x n. Since each x i is some distance from the mean, x, we can think of variance as the average of these distances. Here, we measure this distance as always positive, giving greater weight to numbers further away from the mean. 1. Before calculating any variances, lets establish some intuition about variance. What interval of numbers can variance possibly range between? 2. Suppose that five darts are thrown at each dartboard below, so that the mean sample distance from the bullseye is zero. On the left dartboard, draw a distribution of darts with high variance, and on the right dartboard draw a distribution of darts with low variance. 3. Calculate the variance of {28, 29, 30, 31, 32}.

2 4. Calculate the variance of {10, 20, 30, 40, 50}. What do you observe about the variances of this set of numbers and the previous one? 5. In Excel, we can calculate variance using the command var.p. We will calculate the variances of the petal length and petal width variables of the iris data. Select the empty cell next to Variance, problem 5 in the Statistics sheet. Type in =var.p(. Now, navigate to the Iris sheet and click and drag over all the cells in one of the columns of data - for now, just use the data for Iris versicolor. The cell location should populate in the formula - you can see this in the formula bar above the sheet. Then close out the formula by typing ), and press enter. You can do the same for the other column of data to find its variance. 6. Using one of the datasets your explored before, calculate the variance of one of the measures. Does this data have higher or lower variance than the iris data? What can you conclude, if anything, from this?

3 Covariance Covariance is a measure of how two datasets vary with each other. For a set of numbers X = {x 1, x 2,..., x n }, and Y = {y 1, y 2,..., y n }, their covariance is given by cov(x, Y ) = (x 1 x)(y 1 ȳ) + (x 2 x)(y 2 ȳ) + + (x n x)(y n ȳ), n where x and ȳ are the means of X and Y, respectively. If we want to relate this back to the idea of variance, notice that if X = Y, then this precisely gives us the variance. Variance is a measure of how a dataset varies with itself, rather than with another set of data. 1. If large values of one set of data correspond with large values of another set of data, and similarly with small values, then covariance is positive. However, if large values of one dataset correspond with small values of another dataset, then covariance can be negative. What is the possible interval of values that the covariance of two samples of data can range between? 2. On the axes below, sketch a scatter plot of two datasets (one on the x axis and one on the y axis) with (a) positive covariance, (b) negative covariance, and (c) zero covariance. (a) (b) (c) 3. In Excel, we can calculate the covariance of two sets of data by typing =covariance.p( into the empty cell next to Covariance, problem 3. Then select one column of data, type a comma, and select a second column of data. For example, use Iris versicolor petal length and width. Close off the formula with a parenthesis and press enter. What do you observe? 4. Now, calculate the covariance of sepal length and width for Iris versicolor. Is anything different? What about petal width and sepal width for Iris setosa?

4 5. What are some possible drawbacks to using covariance as a statistical measure? To explore this question, consider the following two data sets: (a) What are the variances of A 1 and A 2? A A (b) Use Excel to generate two data sets, B 1 and B 2, so that A 1 = 2B 1 and A 2 = 2B 2. These two pairs of A i and B i have the same relationship to each other, so one might expect them to vary together identically. Calculate their covariance. (c) Why do you think the covariances are so different?

5 Correlation Correlation is a measure of how strongly related two samples of data are. For a set of numbers X = {x 1, x 2,..., x n }, and Y = {y 1, y 2,..., y n }, we can measure their correlation using the sample correlation coefficient, r, which is given by r = cov(x, Y ) (x 1 x)(y 1 ȳ) + (x 2 x)(y 2 ȳ) + + (x n x)(y n ȳ) = σ x σ y (x1 x) 2 + (x 2 x) (x n x) 2 (y 1 ȳ) 2 + (y 2 ȳ) (y n ȳ). 2 As before, x and ȳ are the means of the sets X and Y, respectively. Correlation is similar to covariance, but instead of describing how two data sets vary with each other using all real numbers, correlation normalizes the covariance, in a sense, by dividing the covariance of two data sets by the square root of the variance of each. If X = Y, then we are precisely dividing by the variance. 1. Correlation varies between 1 and 1. Using the definition given above, explain why. 2. The idea of correlation has frequent use in everyday life. List five ideas that should be highly correlated, and five ideas that should have low correlation, or none. Below is an example of each. Feel free to share ideas with your neighbor! High Correlation Your eye color and your relatives eye colors. Low Correlation A dog s name and the type of dog treats they prefer. 3. We can calculate correlation in Excel by typing =correl(, selecting two columns as with covariance, closing the formula, and pressing enter. Can you identify a pairing of the data sets in the iris data that has positive correlation? How about one with no correlation? (There isn t one with negative correlation!)

6 4. Think of the non-iris data you graphed earlier. Were there any pairs of variables with positive, negative, or no correlation? Go back and calculate the correlation for these variables. We ll have a contest to see who can find the variables with the highest correlation. 5. Warning: you may be familiar with the turn of phrase, Correlation does not imply causation. It is tempting, knowing that two data sets are correlated, to conclude that one may imply the other. Here s a funny example, courtesy of tylervigen.com: The correlation coefficient for these two data sets is r 0.952! Think of some examples of things that may be correlated, but a causal relationship would be absurd. Discuss with your neighbors! For example, what about a child s reading ability and shoe size? Kaitlin Hill (hillk@umn.edu)