Regression. X (b) Line of Regression. (a) Curve of Regression. Figure Regression lines

Transcription

1 Regression 1. The term regression was used b ir Frances Galton in connection with the studies he made on the statures fathers and sons.. It is a technique which determines a relationship between two variables to estimate one of the variables (dependent) for a given value of the other variable (independent). 3. The variable whose value is to be estimated is called dependent variable () whereas the variable whose value is given is called independent variable (x). 4. Examples of dependent and independent variables are: Independent Price Rainfall Credit sales Volume of production Dependent Demand Yield Bad debts Manufacturing expenses 5. The values of the independent variable are assumed to be fixed. Hence it is not a random variable. On the other hand, the dependent variable, whose values are determined on the basis of the independent variable, is a random variable. 6. If x is the independent variable and is the dependent variable then the relationship between x and, described b a straight line ( = a + bx), is called linear relationship. Regression Lines: 1. If we plot the paired observations (X 1 Y 1 ), (X Y ),.., (X n Y n ) on a graph, the resulting set of points is called a scatter diagram.. A scatter diagram indicates a relationship between the variables X and Y and the dots of the scatter diagram tend to cluster around a curve or a line. uch a curve or line is known as curve of regression or line of regression. Y Y (a) Curve of Regression X (b) Line of Regression X Figure Regression lines

2 Linear Regression Model: 1. For a fixed value of independent variable x, if the value of dependent variable is observed a large number of times, different values are possible each time because of the random error involved in the measurement process. The mean of these -values is called the conditional mean of given x and is denoted b.. The linear relationship between equation of on x : and x is called a population regression x x / x x Where α and β are the parameters of the equation. 3. An observation i is the sum of a population mean Random Error () (read as epsilon ). i i / x or x and a component called x This equation is called a linear regression model of on x and is the random variable with mean is equal to zero and variance. 1 = α + βx + e Y μ /x = α + βx μ /x 3 μ /x 1 0 x 1 x x 3 x 4 X Figure Linear Regression Model 4. In the above diagram, the line represents the line of regression of Y on X. The parameter α, which is the expected value of Y when X = 0, is called Y-intercept. The parameter β is slope of the population regression line and is known as the

3 population regression coefficient. When the line slopes downward to the right, the value of β will be negative; it then represents the amount of decrease in Y for each unit increase in X. 5. In practice, the population regression line is unknown. ince the regression is defined b the Y-intercept α and the slope β, therefore, the task of estimating the population regression line involves obtaining the estimates of α and β (based on sample data). Thus the population regression line (μ /x = α + βx) is estimated b the sample regression line or sample regression equation : ˆ a bx (i) The problem of estimating the regression parameters α and β can be considered as fitting the best model on the scatter diagram. One method for this purpose is the method of least squares. Method of Least quares: 1. According to the principle of least squares, a line or a curve is best fitted if the sum of squares of the deviations of estimated values of from the observed values of is minimum. uch line or a curve is called the least square curve or least square line. And the sum of squares is called the Error um of quares (E). Therefore, the E is to be minimised and is represented b: E = i ŷ Where E : Error sum of squares i : observed values ŷ : estimated values, i.e., ( ˆ a bx) It is further elaborated as: E = Σ( i a bx). As we know that the statistic b is an estimator of β, is known as sample regression coefficient. It measures changes in per unit change in x. Therefore, it represents the slope of regression line. Mathematicall it is represented as below: b n x x n x x (ii)(a) x x x x b (ii)(b) x x

4 3. The statistic a is the estimator of α, is called the sample regression constant, and it measures the -intercept of the sample regression line: a bx (iii) 4. Now assume to be independent and x to be dependent. The regression equation of x on is as follows: xˆ c d (i) d d x n x (ii)(a) n x x x (ii)(b) c x d (iii) Example: A sample of paired observations is given as below: X Y Required: (a) Fit a line of regression to the data in the above table. (b) Construct a scatter diagram and graph the fitted line on the scatter diagram, and (c) Calculate error sum of squares. olution: (a): Regression Line of Y on X x x x ŷ ( ˆ ) ( ˆ) ˆ a bx (i)

5 x n x b n x x (44 ) (49)(50) (40 ) (49) (ii) a bx (iii) ˆ x For x =, ˆ () x = 4, ˆ (4). 594 x = 6, ˆ (6) x =, ˆ (). 14 x = 9, ˆ (9) x = 10, ˆ (10) x = 11, ˆ (11) (b): Y Estimated points Observed points X catter Diagram

6 (c) Error um of quares (E): E = i ŷ = Coefficient of Determination: 1. A measure of variation in a sample of n values is given b the sample variance: n n 1 nn 1 It measures the variation in about the sample mean. The term called Total um of quares (T). is. Another measure of variance in a sample of n paired values is called variance of estimate : / x n It measures the variation in about the estimated regression line. The term is called the Error um of quares (E) : E T 3. The Regression um of quares (R) is the difference or excess of T over E: R = T E Therefore, the T is partitioned into two components, i.e., E and R: T = R + E 4. R is the variation in reduced (or explained) b the regression equation and the E is the variation which remains (or unexplained) in when regression line is filled. Thus, the total variation is divided into two, i.e., explained variation and unexplained variation. 5. R is used as a measure of reliabilit of the estimate obtained b the filled regression line. For this purpose the proportion of variation explained b the regression equation, called Coefficient of Determination denoted b r, is calculated as:

7 r R T T E T 1 E T 1 ˆ or ˆ Note that the minimum value of r is zero (when R = 0 and E = T), and the maximum value of r is +1 (when R = T and E = 0); therefore, r lies between 0 to 1: 6. Another formula is: 0 r 1 r a b x n n. Coefficient of determination of two regression equations: r = b d Example: Take the previous example, and calculate the coefficient of determination. olution: Coefficient of Determination x x x

8 96.1% or (.143) 510 (.143) (44) (50) 3.4 n n x b a r