Simple Linear Regression

Transcription

1 Simple Linear Regression Up until this point, the quantitative methods we have been studying have been designed to help us understand a single random variable In many cases, we are interested in examining the relationships between multiple variables In geography, this is usually in an effort to explain the spatial pattern of one set of values in terms of other co-located spatial patterns The simplest instances of these approaches are bivariate analyses, where we study the relationship between two variables

2 Two Sorts of Bivariate Relationships Generally, we can classify the nature of the relationship between a pair of variables into two types: A bivariate relationship can be deterministic, where knowledge of one of the variables entails a perfect knowledge of the other OR A bivariate relationship can be probabilistic, where knowledge of one of the variables can allow you to estimate the value of the other variable, but not with absolute accuracy and/or certainty

3 A Deterministic Relationship Suppose we are traveling from one place to another on the Interstate, and we travel at a constant speed There is a deterministic relationship between the time spent driving and the distance traveled that we can express graphically, or using an equation: distance (s) intercept (s 0 ) time (t) slope (v) s = s 0 + vt s: distance traveled s 0 : initial distance v: speed t: time traveled Unfortunately, few relationships are truly deterministic

4 A Probabilistic Relationship More often, we find relationships between two variables that have a probabilistic nature For example, suppose we compare the ages and heights of a sample of young people between 2 and 20 years old: height (meters) age (years) Here, we cannot predict height from age as we could distance from time in the previous example There is a relationship here, but there is an element of unpredictability or error contained in this model

5 Simple Linear Regression A means that we can use to characterize a probabilistic relationship like the one we saw in the previous slide is using simple linear regression, a linear model with the following characteristics: y = a + bx + ε y (dependent) a error: ε b x (independent) x is the independent variable y is the dependent variable b is the slope of the fitted line a is the intercept of the fitted line ε is the error term

6 Fitting a Line to a Set of Points When we have a data set consisting of an independent and a dependent variable, and we plot these using a scatterplot, to construct our model between the relationship between the variables, we need to select a line that represents the relationship: y (dependent) x (independent) We can choose a line that fits best using a least squares method The least squares line is the line that minimizes the vertical distances between the points and the line, i.e. it minimizes the error term ε when it is considered for all points in the data set

7 Least Squares Method The least squares method operates mathematically, minimizing the error term ε for all points We can describe the line of best fit we will find using the equation ŷ = a + bx, and you ll recall that from a previous slide that the formula for our linear model was expressed using y = a + bx + ε y ŷ We use the value ŷ on the line to estimate the true value, y (y - ŷ) The difference between the two is (y - ŷ) ŷ = a + bx This difference is positive for points above the line, and negative for points below it

8 Minimizing the Error Term ε In a linear model, the error in estimating the true value of the dependent variable y is expressed by the difference between the true value and the estimated value ŷ, ε = (y - ŷ) Sometimes this difference will be positive (when the line underestimates the value of y) and sometimes it will be negative (when the line overestimates the value of y), because there will be points above and below the line If we were to simply sum these error terms, the positive and negative values would cancel out Instead, we can square the differences and then sum them up to create a useful estimate of the overall error

9 Error Sum of Squares By squaring the differences between y and ŷ, and summing these values for all points in the data set, we calculate the error sum of squares (usually denoted by SSE): n SSE = Σ (y - ŷ) 2 i = 1 The least squares method of selecting a line of best fit functions by finding the parameters of a line (intercept a and slope b) that minimizes the error sum of squares, i.e. it is known as the least squares method because it finds the line that makes the SSE as small as it can possibly be, minimizing the vertical distances between the line and the points

10 Finding Regression Coefficients The equations used to find the values for the slope (b) and intercept (a) of the line of best fit using the least squares method are: b = n Σ (x i - x) (y i -y) i = 1 a = y - bx n Σ (x i -x) 2 i = 1 Where: x i is the i th independent variable value y i is the i th dependent variable value x is the mean value of all the x i values y is the mean value of all the y i values

11 Interpreting Slope (b) The slope of the line (b), gives the change in y (dependent variable) due to a unit change in x (independent variable): b > 0 b < 0 Positive relationship As the values of x increase, the values of y increase too Negative (a.k.a. inverse) relationship As values of x increase, the values of y decrease

12 The Strength of Relationships Source: Earickson, RJ, and Harlin, JM Geographic Measurement and Quantitative Analysis. USA: Macmillan College Publishing Co., p. 209.

13 The Strength of Relationships By finding the regression line using the least squares method, we have an equation that describes the relationship between the independent and dependent variables We can examine the slope of the regression line to indicate the nature of that relationship (positive or inverse) We also need a means to characterize the strength of the relationship, or to put this another way, a means by which we can express how effectively the regression estimates the true values of the dependent variable from the independent variable

14 Coefficient of Determination (R 2 ) For example, suppose we have two datasets, and we fit a regression line to each using the least squares method: (a) (b) y y x While the same approach (the least squares method) has been used to select the line of best fit for both data sets, the relationship between x and y is clearly stronger in (a) than in (b), because the points are closer to the line We have a numerical measure to express the strength of the relationship; the coefficient of determination (R 2 ) x

15 Coefficient of Determination (R 2 ) y ŷ y If we use y to estimate y, the error is (y - y) If we use ŷ to estimate y, the error is (y - ŷ) Thus, (ŷ - y) is the improvement in our model To account for the total improvement for the model, we can calculate this distance and sum it for all points in the data set, first taking the square of the difference (ŷ -y)

16 Coefficient of Determination (R 2 ) The regression sum of squares (SSR) expresses the improvement made in estimating y by using the regression line: n y ŷ y SSR = Σ (ŷ i -y) 2 i = 1 The total sum of squares (SST) expresses the overall variation between the values of y and their mean y: n SST = Σ (y i -y) 2 i = 1 The coefficient of determination (R 2 ) expresses the amount of variation in y explained by the regression line (the strength of the relationship): R 2 = SSR SST

17 Pond Branch Catchment Control Color Infrared Digital Orthophotography

18 Soil Moisture Sampling Method 25 samples taken using a random walk within a 5 meter circle ThetaProbe Soil Moisture Sensor - measures the impedance of the sensing rod array, a f(x) of the soil s moisture content 5 meter diameter

19 Pond Branch Catchment Control Topographic Index Example

20 Comparing Soil Moisture and TMI Sites Theta TMI Compare Vol. Soil Moisture (V/V) Pond Branch - 6/26/02 - Average TMI

21 Pond Branch - PG 11.25m DEM Wet May 29/30 Avg. June 26/28 Dry August 22 Theta R 2 = TMI Theta R 2 = TMI Theta R 2 = TMI Glyndon LIDAR 0.5m DEM 11x11 Theta R 2 = TMI Theta R 2 = TMI Theta R 2 = TMI