LINEAR REGRESSION ANALYSIS

Transcription

1 LINEAR REGRESSION ANALYSIS MODULE V Lecture - 2 Correcting Model Inadequacies Through Transformation and Weighting Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technolog Kanpur

2 2 The graphical methods help in detecting the violation of basic assumptions in regression analsis. Now we consider the methods and procedures for building the models through data transformation when some of the assumptions are violated. Variance stabilizing transformations In regression analsis, it is assumed that the variance of disturbances is constant, i.e., Var i n 2 ( εi ) = σ, = 1,2,...,. Suppose this assumption is violated. A common reason for such isolation is that the stud variable follows a probabilit distribution in which the variance is functionall related to mean. For eample, if stud variable () in the model is Poisson random variable in a simple linear regression model, then its variance is same as mean. Since mean of is related to eplanator variable so the variance of will be proportional to. In such cases, variance stabilizing transformations are useful. In another eample, if is proportion, i.e., i 1 then in such cases the variance of is proportional to E( )[1 E( )]. In such case, the variance stabilizing transformation is useful.

3 3 Some commonl used variance-stabilizing transformations in the order of their strength are as follows: After making the suitable transformation, use * as a stud variable in respective case. The strength of a transformation depends on the amount of curvature present in the curve between stud and eplanator variable. The transformation mentioned here range from relativel mild to relativel strong. The square root transformation is relativel mild and reciprocal transformation is relativel strong.

4 4 In general, a mild transformation applied when the minimum and maimum values do not range much (e.g. ma / min < 2,3) and such transformation has little effect on the curvature. On the other hand when the minimum and maimum var much then, a strong transformation is needed that will have a strong effect on the analsis. In the presence of nonconstant variance, the OLSE will remain unbiased but will loose the minimum variance propert. When the stud variable has been transformed as *, then the predicted values are in the transformed scale. It is often necessar to convert the predicted values back to the original units (). When inverse transformation is directl applied to the original values, then it gives an estimate of the median of the distribution of stud variable instead of mean. So one needs to be careful while doing so. Confidence interval and prediction interval ma be directl converted from one metric to another. Reason being that the interval estimates are percentile of a distribution and percentiles are unaffected b transformation. One ma note that the resulting intervals ma or ma not remain the shortest possible intervals.

5 5 Transformations to linearize the model The basic assumption in linear regression analsis is that the relationship between stud variable and eplanator variables is linear. Suppose this assumption is violated. Such violation can be checked b scatter plot matri, scatter diagrams, partial regression plots, lack of fit test etc. In some cases, a nonlinear model can be linearized b using a suitable transformation. Such nonlinear models are called intrinsicall or transformabl linear. The advantage of transforming the nonlinear function into linear function is that the statistical tools are developed for the case of linear regression model. For eample, eact tests for test of hpothesis, confidence interval estimation etc. are developed for the case of linear regression model. Once the nonlinear function is transformed to a linear function, all such tools can be readil applied and there is no need to develop them separatel. Some linearizable functions are as follows:

6 1. If the curve between and is like as follows: 6 = β ( β >, β >, > ) β1 = β ( β >, >, β < ) β1 β 1 > 1 β 1 = 1 β 1 = -1 β 1 < - 1 β < β 1 < 1-1< β 1 < β 1 1 then the possible linearizable function is of the form = β 1 β. Using the transformation log = log β + β log * = ln, * = ln, i.e., b taking log on both sides, the model becomes or * = β + β * β * * = log β β log β where and the model becomes a linear model. Note that the parameter changes to in the transformed model.

7 2. If the curve between and is like as follows 7 = β e β ( β > ) 1 = β e β ( β < ) 1 β e β /e β / β 1 1 / β 1 then the possible linearizable function is of the form = β ep ( β ) Taking log e (ln) on both sides, where So is the transformation needed in this case. The intercept term becomes in the transformed model. or * ln ln = ln β + β * = β + β * * = ln and β = ln β. * = β ln β

8 8 3. If the curve between and is like as follows = β + β log ( β > ) 1 = β + β log ( β > ) 1 then the possible linearizable function is of the form = β + β log which can be written as = β + β1 * using the transformation * = log.

9 9 4. If the curve between and is like as follows = ( β1 ) β β > = ( β1 ) β β < 1/β 1/β β 1 / β β 1 /β then the possible linearizable function is of the form which can be written as or 1 β1 = β = β β * = β + β * which becomes a linear modelb using the transformation 1 1 * =, * =.

10 1 With the observed behaviour of the plots, one can choose an such curve and use the linearized form of the function. When such transformations are used, man times the form of = β ep( β ) ε ε also gets changed. For eample, in case of or ln = ln β + β + ln ε * = β + β + ε. * * This implies that the multiplicative error in original model is log normall distributed in the transformed model. Man times, we ignore this aspect and continue to assume that the random errors are still normall distributed. In such cases, the residuals from the transformed model should be checked for the validit of the assumptions. When such transformations are used, the OLSE has the desired properties with respect to the transformed data and not the original data.