II. Descriptive Statistics D. Liear Correlatio ad Regressio I this sectio Liear Correlatio Cause ad Effect Liear Regressio 1. Liear Correlatio Quatifyig Liear Correlatio The Pearso product-momet correlatio coefficiet, deoted as r, describes a liear relatioship betwee two quatitative variables. It is importat to otice that whe lookig at this value, it oly idicates the liear relatioship. You could have aother kid of relatioship preset i the data. The r value idicates both the stregth of the liear associatio ad its directio. You will ot eed to calculate a r formula is below: value by had, but i case you are iterested the SXY r SSX SSY where SSX x x SSY SXY x y y y x x y y x y xy x y You will be required to iterpret what a r directio of the relatioship. value tells you. We will start with the Positive r suggests large values of X ad Y occur together ad that small values of X ad Y occur together. This meas that the slope of the lie that best fits the poits is positive. A example would be Experiece ad Salary. People with lower levels of experiece ted to have lower salaries ad people with more experiece ted to have higher salaries. Negative r suggests large values of oe variable ted to occur with small values of the other variable. This meas that the slope of the lie that best fits the poits is egative. A example would be Weight of a car ad Gas mileage. Light cars ted to have higher gas mileage ad heavier cars ted to have lower gas mileage. 1
So the sig of the r value tells us the directio of the relatioship. Stregth of the relatioship is measured by the actual value of the umber. By the term stregth, we mea how close are the poits to a lie? The closer the poits are to a lie, the stroger the relatioship. Because of the setup of r, the maximum value for r (i terms of absolute value) is 1. Below are some useful thigs to keep i mid. 1 r 1 If the there is perfect positive liear correlatio all data are exactly o a lie with positive slope If the there is perfect egative liear correlatio all data are exactly o a lie with egative slope If the there is o liear relatioship (keep i mid there could be aother type of correlatio) r 1 r 1 r The stroger the liear relatioship, the larger Geerally, we will say there is a strog relatioship if r (the closer to 1 this value will be). r.75 Lookig back to our example from the last sectio whe itroducig scatterplots we had X = Dosage of Drug ad Y = Reductio i Blood Pressure, what do you thik the r value will be? Remember based o the scatterplot that the poits had a strog positive liear relatioship. Not perfect but pretty close meaig the r value should be close to 1. If you calculate this value, you will get r. 9978. This should seem reasoable as it supports what we idetified i the graph. Aother measure, you will sometimes see reported is the R-squared value. It is commo for computer software to give you a R-squared value istead of r. This value represets the percet of variatio i Y explaied by the model. It measures the stregth of the relatioship ad i the liear case is simply calculated by squarig the r value. The higher R-squared is, the better the model. % R 1% For the Drug example r. 9978 R.9978.995 99.5 %
. Cause ad Effect Causal Research Whe the objective is to determie if a variable causes a certai behavior (whether there is a cause ad effect relatioship betwee variables) Note: it is ever possible to prove causality just based o the relatioship betwee two variables There is a strog statistical correlatio over the moths of the year betwee ice cream cosumptio ad the umber of assaults i the U.S. The r value for this data is above.9. Does this mea ice cream maufacturers are resposible for crime? No! The correlatio occurs statistically because the hot temperatures of summer icreases both ice cream cosumptio ad assaults (High values occur at the same time ad low values occur at the same time) Thus, correlatio does NOT imply causatio. This is oe of the biggest mistakes that I see i the iterpretatio of a correlatio. You should always keep i mid that other factors besides cause ad effect ca create a observed correlatio. To establish whether two variables are causally related you must establish all of the followig: 1) Time order - the cause must have occurred before the effect ) Co-variatio (statistical associatio) the correlatio coefficiet ad graph must show a strog relatioship betwee the depedet ad idepedet variable 3) Ratioale - there must be a logical ad compellig explaatio for why oe variable causes the other 4) No-spuriousess - it must be established that the idepedet variable X, ad oly X, was the cause of chages i the depedet variable Y; rival explaatios must be ruled out The first three of these ca be easily established i may cases. It is the fourth criteria which is hard ad ca rarely be show. To help idetify a relatioship as cause ad effect, a study should be performed may times. The study should yield the same results every time it is coducted. Give that the outside variables will differ from situatio to situatio, this helps rule out rival explaatios. Causal research is very complex ad the researcher ca rarely be certai that other factors are ot ifluecig a relatioship. 3
3. Liear Regressio Determiistic View This is the idea that Y is caused by X or that oce X has happeed, Y will follow. I this situatio, the exact value of Y is kow. The determiistic view is studied i a typical algebra class. However, a determiistic view whe applied to the behavior of may variables is ot possible. Regressio A techique used to predict variables (typically difficult to measure variables) based o a set of other variables (typically easier to measure variables). Liear Regressio Used to predict the value of Y (the respose variable), based o X (the explaatory variable) usig a liear equatio. Predict reactio time based o blood alcohol level. Reactio time is difficult to measure so istead we predict it with blood alcohol level which is easy to measure. The liear regressio model expresses Y as a fuctio of X plus radom error. Radom error reflects variatio i Y values. Keep i mid we are goig to measure X, so assumig we get a good measure there is o error i the X variable. However, whe we go to use X to predict Y, the predictio will ot be exact. Therefore, there is error i the Y variable. Graphically this error is represeted by the vertical distace betwee the poits ad the lie. The liear regressio model is: b b x Y 1 where b is the y-itercept b 1 is the slope The above formula is the same format as what you should be used to from a algebra class. However, the way we deote the relatioship is differet. It is importat you become familiar with this otatio. I order to use liear regressio, we must first make sure the model is reasoable. The scatter plot ad r should idicate a strog relatioship. If the model is ot reasoable, do ot fit a lie. It may still be possible to do regressio with a more complicated model. However, if there is o relatioship betwee the variables the regressio caot be used. I this class we will ot worry about more complicated models, but you should uderstad that a simple liear model is just oe of the may optios available. 4
Whe usig a liear regressio model, we eed the lie that is the best fit for our data. Sice our purpose will be to predict, we will wat to pick the lie that will miimize the error i the predictio. To accomplish this we will use the method of least-squares. Method of Least-Squares says that the sum of the squares of the vertical distaces from the poits to the lie is miimized. Remember it is the vertical distace that represets the error. To calculate the best fit lie you ca use the followig formulas. You do ot have to do this by had i this class. I show you the formulas i case you are iterested. SXY SSX ( x x)( y y) ( x x) xy x x y x b1 b y b1 x At the begiig of this sectio whe lookig at correlatio for the Dosage of drug ad Reductio i blood pressure example we idetified r. 9978 which idicates a high positive liear correlatio. This fact alog with the scatterplot supports the use of liear regressio i this case. With the above formulas, you ca calculate b 1. 118 ad b 3. 4. Therefore, the regressio model i this case is: y b b x y 3.4.118x 1 As I stated earlier, you will ot have to calculate the formula by had. Istead, I will provide computer output ad you eed to be able to aswer questios based o the output. The computer output (a regressio plot) for this example follows. 5
Regressio Plot Y = -3.4 +.118X R-Sq = 99.5 % 6 5 Pressure 4 3 1 1 3 4 5 Drug I this output, the equatio ad the R-squared value are give. If you look above the graph, you will see this iformatio. Notice the R-squared value for this example is exactly what we stated previously i this sectio of material. You eed to be able to get the r value based o the R-squared that is give i the output. All you have to do is take the square root of the R-squared value. The thig you have to be careful of is the directio of the relatioship. Remember that if the slope of the lie is positive the r is positive ad if the slope is egative the r is egative. Therefore, you must look at the slope i order to decide if r is positive or egative. I terms of the equatio, you eed to be able to use it for predictio. This is a pretty direct process as we will always be predictig Y based o X. Therefore, you will plug i for X ad solve for Y. For our example, predict the Reductio i Blood Pressure if 5 is the Dosage of Drug. y 3.4.118x y 3.4.118(5) y 6.1 6
Cosider the followig data ad software output, which give the weight (i thousads of pouds), X, ad gasolie mileage (miles per gallo), Y, for te differet automobiles. X.5 3. 4. 3.5.7 4.5 3.8.9 5.. Y 4 43 3 35 4 19 3 39 15 44 Regressio Plot Y = 7.158-1.6175 X S = 3.46 R-Sq = 9. % R-Sq(adj) = 91. % 45 35 Y 5 15 3 X 4 5 1. Calculate r.. Based o r ad the scatterplot is liear regressio justified i this case. 3. Predict the gas mileage for a car weighig 4. (4,) pouds. Aswers 1. r R.9. 959 This is egative because of the egative slope. Yes, liear regressio is justified, r. 75 ad the poits are spread reasoably about the lie o the scatterplot Y 7.158 1.6175X 7.158 1.6175 4. 5. 3. 51 7
Cautios with regressio There are two commo mistakes with regressio. You must be aware of the problems with extrapolatio ad extreme values. Iterpolatio predictig Y values for X values that are withi the rage of the scatter plot (this is what regressio should be used for) Extrapolatio predictig Y values for X values beyod the rage of the observatios (this should ot be doe with a basic regressio model, it is a complex problem) If our X variable rages from 1 to 5 as it does i the Dosage of drug ad Reductio i blood pressure example the it is reasoable to predict withi that rage. However, if you try ad predict for a X of 1 the you have o data idicatig that this relatioship holds at that value. It is quite possible that the relatioship chages beyod the rage of the data. There is o way to kow this without collectig data cosistet with the X values you wat to predict. The least-squares lie ad the r value ca be affected greatly by extreme data poits. I order to illustrate this we will look at some computer output. 6 Regressio Plot Y = 4.4338 + 1.61E-X R-Sq =. % 5 4 C1 3 1-1 1 C 3 4 5 Calculate the r value for the above data. r With a r of, we kow that there is o liear relatioship betwee X ad Y. 8
Regressio Plot Y = 3.99359 +.869956X R-Sq = 77.9 % 3 C1 1 1 C 3 Calculate r for the above data. r. 779.883 With a r of.883, which is bigger tha the criteria of.75 it seems like we have a strog relatioship. With further ivestigatio via the scatterplot, you will see that all of the data is i the bottom left of the graph except oe data poit which is extreme. What I actually did was take the data from the previous graph with a r value of ad add oe extreme value. Notice the extreme value makes the other data poits appear close together. They also appear umerically close sice the oe value is so extreme. Therefore, the r value is high because the poits are close to the lie. I this case liear regressio is ot justified. If you have a extreme value i a plot like i this case, you should remove the extreme value ad see if the relatioship still exists. I this case it does ot so liear regressio will ot work for this data. 9