Chaper 6 The Simple Linear Regression Model: Reporing he Resuls and Choosing he Funcional Form To complee he analysis of he simple linear regression model, in his chaper we will consider how o measure he variaion in y, explained by he model how o repor he resuls of a regression analysis, some alernaive funcional forms ha may be used o represen possible relaionships beween y and x. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.1
6.1 The Coefficien of Deerminaion Two major reasons for analyzing he model y = β+β x + e (6.1.1) 1 are 1. o explain how he dependen variable (y ) changes as he independen variable (x ) changes, and. o predic y 0 given an x 0. Closely allied wih he predicion problem is he desire o use x o explain as much of he variaion in he dependen variable y as possible. In (6.1.1) we inroduce he explanaory variable x in hope ha is variaion will explain he variaion in y. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.
To develop a measure of he variaion in y ha is explained by he model, we begin by separaing y ino is explainable and unexplainable componens. y = E( y ) + e (6.1.) E( y ) =β 1+β x is he explainable, sysemaic componen of y, and e is he random, unsysemaic, unexplainable noise componen of y. We can esimae he unknown parameers β 1 and β and decompose he value of y ino y = y垐 + e (6.1.3) where y垐 = b + b x and e = y y?. 1 [Figure 6.1 goes here] Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.3
Subrac he sample mean y from boh sides of he equaion o obain y y = ( y垐 y) + e (6.1.4) The difference beween y and is mean value y consiss of a par ha is explained by he regression model, yˆ y, and a par ha is unexplained, e. ˆ A measure of he oal variaion y is o square he differences beween y and is mean value y and sum over he enire sample. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.4
( y y) = [( y垐 y) + e ] y垐垐 y e y y) e = ( ) + + ( (6.1.5) = ( y垐 y) + e The cross-produc erm ( y垐 y) e=0 and drops ou. 1. ( y y) = oal sum of squares = SST: a measure of oal variaion in y abou is sample mean.. ( yˆ y) = explained sum of squares = SSR: ha par of oal variaion in y abou is sample mean ha is explained by he regression. 3. eˆ = error sum of squares = SSE: ha par of oal variaion in y abou is mean ha is no explained by he regression. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.5
Thus, SST = SSR + SSE (6.1.6) This decomposiion is usually presened in wha is called an Analysis of Variance able wih general forma Table 6.1 Analysis of Variance Table Source of Sum of Mean Variaion DF Squares Square Explained 1 SSR SSR/1 Unexplained T SSE SSE/(T ) [ = σ ˆ ] Toal T 1 SST Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.6
The degrees of freedom (DF) for hese sums of squares are: 1. df = 1 for SSR (he number of explanaory variables oher han he inercep);. df = T for SSE (he number of observaions minus he number of parameers in he model); 3. df = T 1 for SST (he number of observaions minus 1, which is he number of parameers in a model conaining only β 1.) In he column labeled Mean Square are (i) he raio of SSR o is degrees of freedom, SSR/1, and (ii) he raio of SSE o is degrees of freedom, SSE/(T ) = ˆσ. The mean square error is our unbiased esimae of he error variance. One widespread use of he informaion in he Analysis of Variance able is o define a measure of he proporion of variaion in y explained by x wihin he regression model: Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.7
R = SSR SSE 1 SST = SST (6.1.7) The measure R is called he coefficien of deerminaion. The closer R is o one, he If beer he job we have done in explaining he variaion in y wih y = b + b x ; and he ˆ 1 greaer is he predicive abiliy of our model over all he sample observaions. R =1, hen all he sample daa fall exacly on he fied leas squares line, so SSE=0, and he model fis he daa perfecly. If he sample daa for y and x are uncorrelaed and show no linear associaion, hen he leas squares fied line is horizonal, and idenical o y, so ha SSR=0 and R =0. When 0 < R < 1, i is inerpreed as he percenage of he variaion in y abou is mean ha is explained by he regression model. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.8
Remark: R is a descripive measure. By iself i does no measure he qualiy of he regression model. I is no he objecive of regression analysis o find he model wih he highes maximizing R is no a good idea. R. Following a regression sraegy focused solely on Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.9
6.1.1 Analysis of Variance Table and R for Food Expendiure Example The compuer oupu usually conains he Analysis of Variance, Table 6.1. For he food expendiure daa i is: Table 6.3 Analysis of Variance Table Sum of Mean Source DF Squares Square Explained 1 51.9 51.9 Unexplained 38 54311.3314 149.455 Toal 39 7953.5544 R-square 0.3171 Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.10
From his able we find ha: SST = SSR = ( y y) = 7953. ( y ˆ y) = 51. SSE = eˆ = 54311. R = SSR SSE 1 SST = SST = 0.317 SSE/(T ) = ˆσ = 149.455 Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.11
6.1. Correlaion Analysis The correlaion coefficien ρ beween X and Y is cov( XY, ) ρ= (6.1.8) var( X ) var( Y ) Given a sample of daa pairs (x,y ), =1,...,T, he sample correlaion coefficien is obained by replacing he covariance and variances in (6.1.8) by heir sample analogues: r cov( ˆ XY, ) = (6.1.9) var( 垐 X ) var( Y ) where Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.1
T cov( ˆ XY, ) = ( x x)( y y) /( T 1) (6.1.10a) = 1 var( ˆ X) T ( x x) /( T 1) = 1 The sample variance of Y is defined like var( ˆ X ). The sample correlaion coefficien r is = (6.1.10b) r = T = 1 ( x x)( y y) T T ( x x) ( y y) = 1 = 1 (6.1.11) The sample correlaion coefficien r has a value beween 1 and 1, and i measures he srengh of he linear associaion beween observed values of X and Y. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.13
6.1.3 Correlaion Analysis and R There are wo ineresing relaionships beween R and r in he simple linear regression model. 1. The firs is ha r = R. Tha is, he square of he sample correlaion coefficien beween he sample daa values x and y is algebraically equal o. R can also be compued as he square of he sample correlaion coefficien beween y and y = b + b x. As such i measures he linear associaion, or goodness of fi, ˆ 1 beween he sample daa and heir prediced values. Consequenly R is someimes called a measure of goodness of fi. R Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.14
6. Reporing he Resuls of a Regression Analysis One way o summarize he regression resuls is in he form of a fied regression equaion: yˆ =40.7676 + 0.183 x R = 0.317 (s.e.) (.1387)(0.0305) (R6.6) The value b 1 = 40.7676 esimaes he weekly food expendiure by a household wih no income; b =0.183 implies ha given a $1 increase in weekly income we expec expendiure on food o increase by $.13; or, in more reasonable unis of measuremen, if income increases by $100 we expec food expendiure o rise by $1.83. The R =0.317 says ha abou 3% of he variaion in food expendiure abou is mean is explained by variaions in income. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.15
The numbers in parenheses underneah he esimaed coefficiens are he sandard errors of he leas squares esimaes. Apar from criical values from he -disribuion, (R6.6) conains all he informaion ha is required o consruc inerval esimaes for β 1 or β or o es hypoheses abou β 1 or β. Anoher convenional way o repor resuls is o replace he sandard errors wih he values These values arise when esing H 0 : β 1 = 0 agains H 1 : β 1 0 and H 0 : β = 0 agains H 1 : β 0. Using hese -values we can repor he regression resuls as yˆ = 40.7676 + 0.183 x R = 0.317 ( ) (1.84) (4.0) (6..) Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.16
6..1 The Effecs of Scaling he Daa Daa we obain are no always in a convenien form for presenaion in a able or use in a regression analysis. When he scale of he daa is no convenien i can be alered wihou changing any of he real underlying relaionships beween variables. For example, suppose we are ineresed in he variable x = U.S. oal real disposable personal income. In 1999 he value of x = $93,491,400,000,000. We migh divide he variable x by 1 rillion and use insead he scaled variable * x = x/1,000,000,000,000= $93.4914 rillion dollars. Consider he food expendiure model. We inerpre he leas squares esimae b = 0.183 as he expeced increase in food expendiure, in dollars, given a $1 increase in weekly income. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.17
I may be more convenien o discuss increases in weekly income of $100. Such a change in he unis of measuremen is called scaling he daa. The choice of he scale is made by he invesigaor so as o make inerpreaion meaningful and convenien. The choice of he scale does no affec he measuremen of he underlying relaionship, bu i does affec he inerpreaion of he coefficien esimaes and some summary measures. Le us summarize he possibiliies: 1. Changing he scale of x: yˆ =40.77 + 0.183x x =40.77+ ( 100 0.183 ) (R6.8) 100 =40.77 + 1.83x * In he food expendiure model b =0.183 measures he effec of a change in income of $1 while 100b =$1.83 measures he effec of a change in income of $100. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.18
When he scale of x is alered he only oher change occurs in he sandard error of he regression coefficien, bu i changes by he same muliplicaive facor as he coefficien, so ha heir raio, he -saisic, is unaffeced. All oher regression saisics are unchanged.. Changing he scale of y: x 100 yˆ = ( 100 40.77) + ( 100 0.183 ) 100 (R6.9) * yˆ =4077 + 1.83 In his rescaled model measures he change we expec in in x. x * * β given a 1 uni change Because he error erm is scaled in his process he leas squares residuals will also be scaled. This will affec he sandard errors of he regression coefficiens, bu i will no affec saisics or R. y Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.19
3. If he scale of y and he scale of x are changed by he same facor, hen here will be no change in he repored regression resuls for b, bu he esimaed inercep and residuals will change; -saisics and R are unaffeced. The inerpreaion of he parameers is made relaive o he new unis of measuremen. 6.3 Choosing a Funcional Form In he household food expendiure funcion he dependen variable, household food expendiure, has been assumed o be a linear funcion of household income. Wha if he relaionship beween y and x is no linear? Remark: The erm linear in simple linear regression model means no a linear relaionship beween he variables, bu a model in which he parameers ener in a linear way. Tha is, he model is linear in he parameers, bu i is no, necessarily, linear in he variables. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.0
By linear in he parameers we mean ha he parameers are no muliplied ogeher, divided, squared, cubed, ec. The variables, however, can be ransformed in any convenien way, as long as he resuling model saisfies assumpions SR1-SR5 of he simple linear regression model. In he food expendiure model we do no expec ha as household income rises ha food expendiures will coninue o rise indefiniely a he same consan rae. Insead, as income rises we expec food expendiures o rise, bu we expec such expendiures o increase a a decreasing rae. y x Figure 6. A Nonlinear Relaionship beween Food Expendiure and Income Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.1
6.3.1 Some Commonly Used Funcional Forms The variable ransformaions ha we begin wih are: 1. The naural logarihm: if x is a variable hen is naural logarihm is ln(x).. The reciprocal: if x is a variable hen is reciprocal is 1/x. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.
Type Saisical Model Slope Elasiciy 1. Linear y = β+β 1 x + e β β x y. Reciprocal y 1 = β+β 1 + e x x 1 β β 1 xy 3. Log-Log ln( y) = β+β 1 ln( x) + e β y x β 4. Log-Linear (Exponenial) ln( y ) = β+β 1 x + e β y βx 5. Linear-Log (Semi-log) y = β+β 1 ln( x ) + e 1 β β x 1 y 6. Log-inverse ln( y ) 1 = β β 1 + e β β x x y 1 x Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.3
[Figure 6.3 goes here] 1. The model ha is linear in he variables describes fiing a sraigh line o he original daa, wih slope β and poin elasiciy β x y. The slope of he relaionship is consan bu he elasiciy changes a each poin.. The reciprocal model akes shapes shown in Figure 6.3(a). As x increases y approaches he inercep, is asympoe, from above or below depending on he sign of β. The slope of his curve changes, and flaens ou, as x increases. The elasiciy also changes a each poin and is opposie in sign o / β. In Figure 6.3(a), when β >0, he relaionship beween x and y is an inverse one and he elasiciy is negaive: a 1% increase in x leads o a reducion in y of β /( xy) %. 3. The log-log model is a very popular one. The name log-log comes from he fac ha he logarihm appears on boh sides of he equaion. In order o use his model all values of y and x mus be posiive. The shapes ha his equaion can ake are shown in Figures 6.3(b) and 6.3(c). Figure 6.3(b) shows cases in which > 0, and Figure 6.3(c) shows β Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.4
cases when < 0. The slopes of hese curves change a every poin, bu he elasiciy is β consan and equal o β. This consan elasiciy model is very convenien for economiss, since we like o alk abou elasicies and are familiar wih heir meaning. 4. The log-linear model ( log on he lef-hand-side of he equaion and linear on he righ) can ake he shapes shown in Figure 6.3(d). Boh is slope and elasiciy change a each poin and are he same sign as. β 5. The linear-log model has shapes shown in Figure 6.3(e). I is an increasing or decreasing funcion depending upon he sign of. β 6. The log-inverse model ( log on he lef-hand-side of he equaion and a reciprocal on he righ) has a shape shown in Figure 6.3(f). I has he characerisic ha near he origin i increases a an increasing rae (convex) and hen, afer a poin, increases a a decreasing rae (concave). Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.5
Remark: Given his array of models, some of which have similar shapes, wha are some guidelines for choosing a funcional form? We mus cerainly choose a funcional form ha is sufficienly flexible o fi he daa. Choosing a saisfacory funcional form helps preserve he model assumpions. Tha is, a major objecive of choosing a funcional form, or ransforming he variables, is o creae a model in which he error erm has he following properies; 1. E(e )=0. var(e )=σ 3. cov(e i,e j )=0 4. e ~N(0, σ ) If hese assumpions hold hen he leas squares esimaors have good saisical properies and we can use he procedures for saisical inference ha we have developed in Chapers 4 and 5. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.6
6.3. Examples Using Alernaive Funcional Forms In his secion we will examine an array of economic examples and possible choices for he funcional form. 6.3.a The Food Expendiure Model From he array of shapes in Figure 6.3 wo possible choices ha are similar in some aspecs o Figure 6. are he reciprocal model and he linear-log model. The reciprocal model is y 1 = β+β 1 + e (6.3.) x For he food expendiure model we migh assume ha β > 0 and β 1 < 0. If his is he case, hen as income increases, household consumpion of food increases a a decreasing rae and reaches an upper bound β 1. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.7
This model is linear in he parameers bu i is nonlinear in he variables. If he error erm e saisfies our usual assumpions, hen he unknown parameers can be esimaed by leas squares, and inferences can be made in he usual way. Anoher propery of he reciprocal model, ignoring he error erm, is ha when x < β /β 1 he model predics expendiure on food o be negaive. This is unrealisic and implies ha his funcional form is inappropriae for small values of x. When choosing a funcional form one pracical guideline is o consider how he dependen variable changes wih he independen variable. In he reciprocal model he slope of he relaionship beween y and x is dy dx = β If he parameer β < 0 hen here is a posiive relaionship beween food expendiure and income, and, as income increases his marginal propensiy o spend on food diminishes, as economic heory predics. 1 x Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.8
For he food expendiure relaionship an alernaive o he reciprocal model is he linear-log model y = β+β 1 ln( x) + e (6.3.3) which is shown in Figure 6.3(e). For β > 0 his funcion is increasing, bu a a decreasing rae. As x increases he slope β /x decreases. Similarly, he greaer he amoun of food expendiure y he smaller he elasiciy, β /y. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.9
6.3.b Some Oher Economic Models and Funcional Forms 1. Demand Models: models of he relaionship beween quaniy demanded (y d ) and price (x) are very frequenly aken o be linear in he variables, creaing a linear demand curve, as so ofen depiced in exbooks. Alernaively, he log-log form of he d model, ln( y ) =β +β ln( x) + e, is very convenien in his siuaion because of is 1 consan elasiciy propery. Consider Figure 6.3(c) where several log-log models are shown for several values of β < 0. They are negaively sloped, as is appropriae for demand curves, and he price-elasiciy of demand is he consan β.. Supply Models: if y s is he quaniy supplied, hen is relaionship o price is ofen assumed o be linear, creaing a linear supply curve. Alernaively he log-log, consan s elasiciy form, ln( y ) = β+β ln( x) + e, can be used 1 Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.30
3. Producion Funcions: One of he assumpions of producion heory is ha diminishing reurns hold; he marginal-physical produc of he variable inpu declines as more is used. To permi a decreasing marginal produc, he relaion beween oupu (y) and inpu (x) is ofen modeled as a log-log model, wih β < 1. This relaionship is shown in Figure 6.3(b). I has he propery ha he marginal produc, which is he slope of he oal produc curve, is diminishing, as required. 4. Cos Funcions: a family of cos curves, which can be esimaed using he simple linear regression model, is based on a quadraic oal cos curve. Suppose ha you wish o esimae he oal cos (y) of producing oupu (x); hen a poenial model is given by y = β+β x + e (6.3.4) 1 If we wish o esimae he average cos (y/x) of producing oupu x hen we migh divide boh sides of equaion 6.3.4 by x and use Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.31
( / ) = β / +β + / (6.3.5) y x 1 x x e x which is consisen wih he quadraic oal cos curve. 5. The Phillips Curve: If we le w be he wage rae in ime, hen he percenage change in he wage rae is w w Δ w = (6.3.6) 1 % w 1 If we assume ha % Δ wrie w is proporional o he excess demand for labor d, we may % Δw =γ d (6.3.7) Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.3
where γ is an economic parameer. Since he unemploymen rae u is inversely relaed o he excess demand for labor, we could wrie his using a reciprocal funcion as d 1 = α+η (6.3.8) u where α and η are economic parameers. Given equaion 6.3.7 we can subsiue for d, and rearrange, o obain Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.33
% Δ w 1 =γ α+η u 1 =γα+γη u This model is nonlinear in he parameers and nonlinear in he variables. Undergraduae Economerics, nd Ediion-Chaper 6 Slide 6.34