Regression Analysis for Undergraduates. By Dr. Anju Gupta. Ms. Shweta Jain

Transcription

1 Regression Analysis for Undergraduates By Dr. Anju Gupta Director, CWEB, University of Delhi Ms. Shweta Jain Assistant Professor, CWEB, University of Delhi Introduction The statistical tool correlation determines the degree and direction of relationship between two or more variables. We are interested in estimating the value of an unknown variable on the basis of a known variable. For example, if we know that the income and consumption expenditure of a person are correlated, then we can find out the consumption expenditure of that person for a fixed income. Hence, the statistical technique with the help of which we can predict or estimate the unknown variable from known variable is called regression. The literal meaning of the term Regression is the act of moving back or stepping back or returning to average value. Sir Francis Galton was the first person who used this term regression as a statistical concept in He studied the relationship between the height of fathers and their sons. Very interesting relationships were revealed by his study which are described below: i. All tall fathers tend to have tall sons and all short fathers have short sons. ii. The average height of the sons of short fathers is more than the average height of their fathers. iii. The average height of the sons of tall fathers is less than the average height of their fathers. The line describing this tendency of going back is called regression Line. These days modern writers are using the term estimating line instead of regression line as the term estimating line is more clear in character. Some of the important definitions of regression analysis are given below: 1. Regression is the measure of the average relationship between two or more variables in terms of the original units of the data. _ Blair 2. Regression analysis attempts to establish the nature of the relationship between variables that is, to study the functional relationship between the variables and thereby provide a mechanism for prediction. Or forecasting. _Ya-Lun- Chou 3. The term regression analysis refers to the methods by which estimates are made of the values of a variable from a knowledge of the values of one or more other variables and to the measurement of the errors involved in this estimation process. _ Morris Hamburg 1

2 4. One of the most frequently used technique in economics and business research, to find a relation between two or more variables that are related casually, is regression analysis. _ Taro Yamane It is clear from the above definitions that the statistical tool regression helps to estimate the value of an unknown variable on the basis of a known variable. The variable which is used to predict the values of other variable is known as independent variable and the variable which we are trying to predict is known as dependent variable. The analysis used is called simple linear regression analysis simple because there is only one independent variable, and linear because of the assumption of linear relationship between the dependent and the independent variables. Uses of Regression Analysis 1. Regression analysis helps in estimating the values of dependent variable from the values of the independent variable. For this estimation, regression line is used. 2. The second use of regression analysis is to get a measure of the error which is involved in using the regression line as a basis for estimation. The standard error of estimate is calculated for this purpose. 3. The regression coefficients are also useful in calculating the correlation coefficient. Difference between Correlation and Regression 1. Degree and nature of relationship: The correlation coefficient measures the degree of covariability between two variables whereas regression analysis is used to study the nature of relationship between these two variables so that we can estimate the value of one variable on the basis of other variable. The closer the relationship between two variables, the greater the belief that may be placed in the estimates. 2. Cause and Effect relationship: Correlation is a tool which can only be used to determine the degree of relationship between two or more variables. It does not establish cause and effect relationship between these two variables. A high degree of correlation between income and consumption expenditure at a particular point of time may not suggest which is the cause and which is the effect. However, in regression analysis, this cause and effect relationship is clearly explained as one variable is taken as independent and the other is taken as dependent. Lines of Regression Principle of Least Squares Regression analysis expresses the relationship between a dependent variable with one or more independent variables. Such relationship is expressed by a line of regression drawn by the method of the least squares. This line of regression can be derived algebraically with the help of regression equations or it can also be drawn graphically. According to the least square criterion, the line of best fit is the one that minimizes the sum of the squares of the vertical distances from the observed points to the line i.e., the sum of the squares of the deviations of the observed y values from the fitted line shall be minimum. The line of best satisfies the following conditions: i. The sum of deviations below the line and above the line is equal to zero. (x -x c ) = and (y -y c ) = 2

3 Where x c and y c are the values that are derived with the help of regression technique. ii. The sum of the squares of the deviations of the observed y values from the fitted line is less than the sum of the squares of the deviations from any other line. (x -x c ) 2 is smaller than (x A) 2 and (y -y c ) 2 is smaller than (y A) 2 Where A is any other line. iii. The line of best fit intersects at the mean values of the variables i.e., Xand Y. Regression Equations These regression equations are algebraic expressions of the regression lines. They are also known as estimating equations Since there are two regression lines, there are two regression equations : Y on X which is used to describe the variations in the values of Y for given changes in X X on Y which is used to describe the variations in the values of X for given changes in Y Regression equation of Yon X The regression equation of Y on X is given as follows: Y c = a + bx In the above equation, a and b are fixed numerical values (constants) which determine the position of the line completely. These constants are called the parameters of the line. When the values of these parameters change, another line is determined. The parameter a indicates the intercept of the line i.e., the value of Y variable(dependent variable) when X variable(independent variable) is Zero. The parameter b refers to the slope of the line, i.e., the change in Y per unit change in X.Y c refers to the value of Y calculated from the value of X. When we obtain the values of these parameters ( a and b), the line is completely determined and the method of least squares helps in obtaining the values of these parameters. With a little algebra and differential calculus it can be shown that the following two equations, when solved simultaneously, will provide the values of the parameters a and b such that the least squares requirement is fulfilled: ΣY = a + bσx ΣXY = aσx + bσx 2 These equations are called the normal equations. Regression equation of X on Y The regression equation of X on Y is given as follows: X c = a + by 3

4 In order to determine the values of parameters a and b, the following two normal equatins are to be solved simultaneously: ΣX = a + bσy ΣXY = aσy + bσy 2 Y Regression line of X on Y Regression Line of Y on X X Illustration: Obtain the two regression equations from the following data: X : Y : Solution: Computation of Regression equations X Y XY X 2 Y Σx =3 Σy =4 Σxy =214 Σ x 2 =22 Σ y 2 =34 Regression line of Y on X is expressed by the equation: Y c = a + bx In order to determine the values of a and b, the following two normal equations are solved ΣY = a + bσx ΣXY = aσx + bσx 2 By substituting the values, we get 4 = 5a + 3b.....(i) 214 = 3a + 22b (ii) Multiplying the equation (i) by 6, we get 24 = 3a +18b...(iii) 214 = 3a + 22b (iv) Deduct equation (iv) from (iii) -4b = +26 b = -.65 Substituting the value of b in equation (i) 4

5 4 = 5a + 3(-.65) 5a = or a = 11. Substitute the value of a and b in the equation Regression line of Y on X is Y c = x Regression line of X on Y X c = a + by The corresponding normal equations are ΣX = a + bσy ΣXY = aσy + bσy 2 Substituting the values 3 = 5a + 4b..(i) 214 = 4a +34b.(ii) Multiply equation (i) by 8 24 = 4a + 32b (iii) 214 = 4a + 34b....(iv) Deduct equation (iv) from (iii) -2b =26 or b= -1.3 Substitute the value of b in equation (i) 3 =5a + 4 (-1.3) 5a =3 +52 or a = 16.4 Substitute the value of a and b in the equation Regression line of X on Y is X c = y Deviations taken from Arithmetic Means of X and Y: The above method of finding regression equations involves tedious calculations. These calculations can be simplified if deviations of X and y series are taken from their respective means instead of dealing with the actual values of X and Y. In such a case, the two regression equations are as follows: i. Regression equation of X on Y X-X =bxy (Y- Y) Where X is the mean of series X; Y is the mean of series Y and bxy is regression coefficient of X on Y When Deviations taken from Arithmetic Means of X and Y, the regression coefficient of X on Y is obtained as follows: bxy = r σx σy Σxy bxy = X σx σxσy σy = Σxy σyy 2 5

6 where x = X- X and y = Y - Y bxy = Σxy Σy2 ii. Regression equation of Y on X (Y - Y) = byx (X - X) byx = r σy σx byx = Σxy σxσy X σy σx = Σxy σxx 2 where x = X- X and y = Y - Y byx = Σxy Σxx 2 Illustration: Calculate the regression coefficients from the following data; Series x Series y Average 2 18 Standard Deviation 4 5 r =. Solution: The coefficient of regression of y on x is bxy = r σx σy =.(4/5) = +.72 The coefficient of regression of y on x is byx = r σy σx =.(5/4) =1.125 Illustration: Calculate the following: (i) The two regression equations (ii) The coefficient of correlation (iii) The most likely marks in Statistics when the marks in maths are 4 Marks in Maths : Marks in Statistics : Solution: 6

7 Marks in Maths (X) Marks in (X-X)=x x 2 Stats (Y) (Y-Y)=y y 2 xy ΣX =32 Σx= Σ x 2 =14 ΣY=38 Σy = Σy 2 =38 Σxy =-3 Regression equation X on Y X-X =bxy (Y- Y) bxy = Σxy Σyy 2 = 3/U38 = X = ΣX = 32 = 32 and Y = ΣY = 38 =38 1 By Substituting the values X 32 = (Y-38) X 32 = -.234Y Or X = Y Regression equation Y on X (Y - Y) = byx (X - X) = -3 = byx = Σxy Σxx 2 X = 32, Y = 38, byx=

8 Therefore, Y-38 = (X 32) Y-38 = -.664X Y = X (b) Correlation Coefficient (r) = bxy * byx = -.23 x = -.34 Since both the regression coefficients have negative values, value of r must also be negative. (c) Likely marks in Statistics when marks in Maths are 4 Y= -.664X where X = 4 Y= (-.664 x 4) = or 33 Illustration: By using the following data, find both the regression equations: Marks in Maths(X) Marks in Statistics (Y) Mean 1 Standard Deviation 3 12 Correlation Coefficient.8 (i) Calculate regression equations (ii) Estimate the value of X when Y = 12 Solution: We are given the following information: X = 1, Y=, σ x = 3, σ y = 12 and r=.8 Therefore, byx = r σy σx =.8(12/3) = 3.2 =.8(3/12) =.2 Regression line of Y on X: Y- Y =byx (X - X) Y- = 3.2 (X - 1) Y = 3.2X + 58 bxy = r σx σy Regression line of X on Y: X- X = bxy (Y-Y) X- 1 =.2 (Y- ) X =.2Y 8 Therefore, value of X when Y= 12 is given by: X =.2 x 12 8 = 16 Deviations taken from Assumed mean: 8

9 In order to avoid difficult calculations when mean of both the variables X and Y are in fraction, deviations are taken from assumed mean to calculate coefficient of regression. In this case, the values of bxy and byx are given by : Regression Coefficient : X on Y where dx = (X- A) and dy = (Y-A) Regression Coefficient : Y on X bxy = Σdxdy {(Σdx )x (Σdy )} Σdyy 2 (Σdy )2 byx = Σdxdy {(Σdx )x (Σdy )} Σdxx 2 (Σdx )2 Illustration: Compute the two regression equations by method of least squares Income : Expenditure : Solution: Incom e X (X-76) Expenditure dx dx 2 Y (Y-12) dy dy 2 dxdy Σdx= -1 Σ dx 2 =225 Σdy=-5 Σ dy 2 =24 Σdxdy=183

10 X = A + Σdx = 76 1 = 75 and Y = A + Σdy = 12 5 = Regression Coefficient : X on Y 183 (-1) x (-5) = 1_ = 1753 = (-5) Regression Coefficient : Y on X bxy = Σdxdy {(Σdx )x (Σdy )} Σdyy 2 (Σdy )2 byx = Σdxdy {(Σdx )x (Σdy )} Σdxx 2 (Σdx )2 183 (-1) x (-5) = 1_ = 1753 = (-1) Regression Equation : X on Y X- X = bxy (Y-Y) X 75 =.652 (Y- 115) X =.652Y +.2 Regression Equation : Y on X Y- Y =byx (X - X) Y- 115 =.78(X - 75) Y =.78X Illustration: Calculate the following: i. Two regression coefficients ii. Coefficient of correlation 1

11 iii. The two regression equations from the following data =1 ΣX=35 ΣY= 31 Σ(X-35) 2 =162 Σ(Y-31) 2 = 222 Σ(X-35)(Y-31)=2 Solution: (i) X = ΣX = 35 = 35 and Y =ΣY =31 = We are given that: Σ(X-35) 2 =162 Σ(Y-31) 2 = 222 Σ(X-35)(Y-31)=2 Or, Σx 2 = 162 Σy 2 = 222 Σxy= 2 Regression coefficient of X on Y: bxy = Σxy= 2 =.414 Σy Regression coefficient of Y on X: byx = Σxy= 2 =.568 Σx (ii) Coefficient of correlation is given by r = bxy * byx =.414 x.568 =.485 (iii) Regression Equations: Regression Equations of X on Y is: X- X = bxy (Y-Y) X 35 =.414 (Y- 31) X =.414Y Regression Equations of Yon X is: Y- Y =byx (X - X) Y- 31 =.568(X - 35) Y =.568X