ECONOMICS 5* -- NOTE (Summary) ECON 5* -- NOTE The Multple Classcal Lnear Regresson Model (CLRM): Specfcaton and Assumptons. Introducton CLRM stands for the Classcal Lnear Regresson Model. The CLRM s also known as the standard lnear regresson model. Three sets of assumptons defne the multple CLRM -- essentally the same three sets of assumptons that defned the smple CLRM, wth one modfcaton to assumpton A8.. Assumptons respectng the formulaton of the populaton regresson equaton, or PRE. Assumpton A. Assumptons respectng the statstcal propertes of the random error term and the dependent varable. Assumptons A-A4 Assumpton A: The Assumpton of Zero Condtonal Mean Error Assumpton A: The Assumpton of Constant Error Varances Assumpton A4: The Assumpton of Zero Error Covarances. Assumptons respectng the propertes of the sample data. Assumptons A5-A8 Assumpton A5: The Assumpton of Independent Random Samplng Assumpton A6: The Assumpton of Suffcent Sample Data (N > k) Assumpton A7: The Assumpton of Nonconstant Regressors Assumpton A8: The Assumpton of No Perfect Multcollnearty ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary). Formulaton of the Populaton Regresson Equaton (PRE) Assumpton A: The populaton regresson equaton, or PRE, takes the form Y = β + β + β + L+ β + u = β + β + u (A) k k As n the smple CLRM, the PRE (A) ncorporates three dstnct assumptons. A.: Assumpton of an Addtve Random Error Term. k j= j j The random error term u enters the PRE addtvely. Y u = for all ( ). A.: Assumpton of Lnearty-n-Parameters or Lnearty-n-Coeffcents. The PRE s lnear n the populaton regresson coeffcents β j (j =,..., k). Let x = [ L k ] be the (k ) vector of regressor values for observaton. Y β j = f j (x ) where f x ) contans no unknown parameters, j =,..., k. j ( A.: Assumpton of Parameter or Coeffcent Constancy. The populaton regresson coeffcents β j (j =,,..., k) are (unknown) constants that do not vary across observatons. β j = β j = a constant (j =,,..., k). ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary). Propertes of the Random Error Term Assumpton A: The Assumpton of Zero Condtonal Mean Error The condtonal mean, or condtonal expectaton, of the random error terms u for any gven values j of the regressors j s equal to zero: E ( u,,, ) = E( u x ) = 0 K (A) k where x = [ L k ] denotes the (k ) vector of regressor values for a partcular observaton, namely observaton. Implcatons of Assumpton A Implcaton of A. Assumpton A mples that the uncondtonal mean of the populaton values of the random error term u equals zero: E ( u x ) 0 = ( u ) 0 E =. (A-) Implcaton of A: the Orthogonalty Condton. Assumpton A also mples that the populaton values j of the regressor j and u of the random error term u have zero covarance --.e., the populaton values of j and u are uncorrelated: E ( u x ) = 0 Cov ( j, u ) E( j u ) = 0 =, j =,,, k (A-) Note that zero covarance between j and u mples zero correlaton between j and u, snce the smple correlaton coeffcent between j and u, denoted as ρ( j, u ), s defned as ρ( (, ), u Cov j u j ) = Var( ) Var( u ) j Cov( j, u ) sd( ) sd( u ). j From ths defnton of ρ( j, u ), t s obvous that Cov ( j, u ) = 0 ρ( j u ), = 0. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary) Implcaton of A. Assumpton A mples that the condtonal mean of the populaton Y values correspondng to gven values j of the regressors j (j =,, k) equals the populaton regresson functon (PRF): E ( u x ) = 0 E( Y x ) = f (x) = β + β + β + L + βkk = β k + β jj j=. (A-) Meanng of the Zero Condtonal Mean Error Assumpton A: Each set of regressor values x [ L ] = k dentfes a segment or subset of the relevant populaton, specfcally the segment that has those partcular values of the regressors. For each of these populaton segments or subsets, assumpton A says that the mean of the random error u s zero. Assumpton A rules out both lnear dependence and nonlnear dependence between each j and u; that s, t requres that j and u be statstcally ndependent for all j =,, k. Volatons of the Zero Condtonal Mean Error Assumpton A Remember that the random error term u represents all the unobservable, unmeasured and unknown varables other than the regressors j, j =,, k that determne the populaton values of the dependent varable Y. Anythng that causes the random error u to be correlated wth one or more of the regressors j (j =,, k) wll volate assumpton A: Cov( j, u) 0 or ρ (, u) 0 ( u x) 0 j If j and u are correlated, then ( u x) zero. Note that the converse s not true: E. E must depend on j and so cannot be Cov( j, u) = 0 or ρ ( j, u) = 0 for all j does not mply that ( u x) 0 E =. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 4 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Common causes of correlaton or dependence between the j and u --.e., common causes of volatons of assumpton A.. Incorrect specfcaton of the functonal form of the relatonshp between Y and the j, j =,, k. Examples: Usng Y as the dependent varable when the true model has ln(y) as the dependent varable. Or usng j as the ndependent varable when the true model has ln( j ) as the ndependent varable.. Omsson of relevant varables that are correlated wth one or more of the ncluded regressors j, j =,, k.. Measurement errors n the regressors j, j =,, k. 4. Jont determnaton of one or more j and Y. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 5 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Assumpton A: The Assumpton of Constant Error Varances The Assumpton of Homoskedastc Errors The Assumpton of Homoskedastcty The condtonal varances of the random error terms u are dentcal for all observatons --.e., for all sets of regressor values x = [ L k ] -- and equal the same fnte postve constant σ for all : Var ( u x ) E( u x ) = σ > 0 = (A) where σ s a fnte postve (unknown) constant and x = [ L ] s the (k ) vector of regressor values for observaton. Implcaton of A: Assumpton A mples that the uncondtonal varance of the random error u s also equal to σ : Var [ ] = E( u ) = σ ( u ) E ( u E(u )) where ( u ) E( ) =. u Var = because E(u ) = 0 by A-. Implcaton of A: Assumpton A mples that the condtonal varance of the regressand Y correspondng to gven set of regressor values x = L ] equals the condtonal error varance σ : [ k k or Var Var ( u x) = σ > 0 Var( Y x) σ > 0. = (A-) ( u x ) = σ > 0 Var( Y x ) σ > 0. = (A-) ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 6 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Meanng of the Homoskedastcty Assumpton A For each set of regressor values, there s a condtonal dstrbuton of random errors, and a correspondng condtonal dstrbuton of populaton Y values. Assumpton A says that the varance of the random errors for any partcular set of regressor values x = [ L k ] s the same as the varance of the random errors for any other set of regressor values x = L ] (for all xs x ). s [ s s ks In other words, the varances of the condtonal random error dstrbutons correspondng to each set of regressor values n the relevant populaton are all equal to the same fnte postve constant σ. Var ( u x ) Var( u x ) = σ 0 = for all xs x. s s > Implcaton A- says that the varance of the populaton Y values for x = x = [ L k ] s the same as the varance of the populaton Y values for any other set of regressor values x = xs = [ s s L ks ] (for all xs x ). The condtonal dstrbutons of the populaton Y values around the PRF have the same constant varance σ for all sets of regressor values. Var ( Y x ) Var( Y x ) = σ 0 = for all xs x. s s > ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 7 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Assumpton A4: The Assumpton of Zero Error Covarances The Assumpton of Nonautoregressve Errors The Assumpton of Nonautocorrelated Errors Consder any par of dstnct random error terms u and u s ( s) correspondng to two dfferent sets (or vectors) of values of the regressors x x s. Ths assumpton states that u and u s have zero covarance: Cov ( u, u x, x ) E( u u x, x ) = 0 s. s s = (A4) s s Implcaton of A4: Assumpton A4 mples that the condtonal covarance of any two dstnct values of the regressand, say Y and Y s where s, s equal to zero: ( s s) Cov u, u x, x = s 0 Cov ( s s) Y, Y x, x = 0 s. Meanng of A4: Assumpton A4 means that there s no systematc lnear assocaton between u and u s, or between Y and Y s, where and s correspond to dfferent observatons (or dfferent sets of regressor values x ). x s. Each random error term u has zero covarance wth, or s uncorrelated wth, each and every other random error term u s (s ).. Equvalently, each regressand value Y has zero covarance wth, or s uncorrelated wth, each and every other regressand value Y s (s ). The assumpton of zero covarance, or zero correlaton, between each par of dstnct observatons s weaker than the assumpton of ndependent random samplng A5 from an underlyng populaton. The assumpton of ndependent random samplng mples that the sample observatons are statstcally ndependent. The assumpton of statstcally ndependent observatons s suffcent for the assumpton of zero covarance between observatons, but s stronger than necessary. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 8 of 9 pages
ECONOMICS 5* -- NOTE (Summary) 4. Propertes of the Sample Data Assumpton A5: Random Samplng or Independent Random Samplng The sample data consst of N randomly selected observatons on the regressand Y and the regressors j (j =,..., k), the observable varables n the PRE descrbed by A. These N randomly selected observatons can be wrtten as N row vectors: Sample data [(Y, x), (Y, x ), K, (YN, x N )] ( Y,,,, K, ) (Y, x ) k =, K, N =, K, N. Implcatons of the Random Samplng Assumpton A5 The assumpton of random samplng mples that the sample observatons are statstcally ndependent.. It thus means that the error terms u and u s are statstcally ndependent, and hence have zero covarance, for any two observatons and s. Random samplng ( u,u x, x ) Cov = ( u, ) s s Cov = 0 s.. It also means that the dependent varable values Y and Y s are statstcally ndependent, and hence have zero covarance, for any two observatons and s. Random samplng ( Y, Y x, x ) s s u s Cov = ( Y, ) Cov = 0 s. The assumpton of random samplng s therefore suffcent for assumpton A4 of zero covarance between observatons, but s stronger than necessary. Y s When s the Random Samplng Assumpton A5 Approprate? The random samplng assumpton s often approprate for cross-sectonal regresson models, but s hardly ever approprate for tme-seres regresson models. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 9 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Assumpton A6: The number of sample observatons N s greater than the number of unknown parameters k: number of sample observatons > number of unknown parameters N > k. (A6) Meanng of A6: Unless ths assumpton s satsfed, t s not possble to compute from a gven sample of N observatons estmates of all the unknown parameters n the model. Assumpton A7: Nonconstant Regressors The sample values j of each regressor j (j =,, k) n a gven sample (and hence n the populaton) are not all equal to a constant: j c j =,..., N where the c j are constants (j =,..., k). (A7) Techncal Form of A7: Assumpton A7 requres that the sample varances of all k non-constant regressors j (j =,..., k) must be fnte postve numbers for any sample sze N;.e., sample varance of j Var( j ) = ( ) N j j = s where > 0 are fnte postve numbers for all j =,..., k. s j j > 0, Meanng of A7: Assumpton A7 requres that each nonconstant regressor j (j =,, k) takes at least two dfferent values n any gven sample. Unless ths assumpton s satsfed, t s not possble to compute from the sample data an estmate of the effect on the regressand Y of changes n the value of the regressor j. In other words, to calculate the effect of changes n j on Y, the sample values j of the regressor j must vary across observatons n any gven sample. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 0 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Assumpton A8: No Perfect Multcollnearty The sample values of the regressors j (j =,..., k) n a multple regresson model do not exhbt perfect or exact multcollnearty. Ths assumpton s relevant only n multple regresson models that contan two or more non-constant regressors. Ths assumpton s the only new assumpton requred for the multple lnear regresson model. Statement of Assumpton A8: The absence of perfect multcollnearty means that there exsts no exact lnear relatonshp among the sample values of the non-constant regressors j (j =,..., k). An exact lnear relatonshp exsts among the sample values of the nonconstant regressors f the sample values of the regressors j (j =,..., k) satsfy a lnear relatonshp of the form λ + λ + λ + L + λ = =,, K, N. k k 0 () where the λ ( j = K,,, k) are fxed constants, not all of whch equal zero. j Assumpton A8 -- the absence of perfect multcollnearty -- means that there exsts no relatonshp of the form () among the sample values j of the regressors j (j =,..., k). Meanng of Assumpton A8: Each non-constant regressor j (j =,..., k) must exhbt some ndependent lnear varaton n the sample data. Otherwse, t s not possble to estmate the separate lnear effect of each and every non-constant regressor on the regressand Y. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary) Example of Perfect Multcollnearty Consder the followng multple lnear regresson model: Y = β + β + β + u ( =,...,N). () Suppose that the sample values of the regressors and satsfy the followng lnear equalty for all sample observatons: = or = 0 =,...,N. () The exact lnear relatonshp () can be wrtten n the general form ().. For the lnear regresson model gven by PRE (), equaton () takes the form λ + λ + λ = 0 =,, K, N.. Set λ = 0, λ =, and λ = n the above equaton: = 0 =,, K, N. (dentcal to equaton () above.) ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary) Consequences of Perfect Multcollnearty. Substtute for n PRE () the equvalent expresson = : Y = β = β = β = β = β + β + β + β + ( ) ( β + β ) + α + β + β + u + β + u + u + u + u where α = β + β. (4a) It s possble to estmate from the sample data the regresson coeffcents β and α. But from the estmate of α t s not possble to compute estmates of the coeffcents β and β. Reason: The equaton α = β + β s one equaton contanng two unknowns, namely β and β. Result: It s not possble to compute from the sample data estmates of both β and β, the separate lnear effects of and on the regressand Y. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page of 9 pages
ECONOMICS 5* -- NOTE (Summary). Alternatvely, substtute for n PRE () the equvalent expresson = : Y = β + β + β + u = β + β + β β = β + β + + u β = β + β + + u + = β + α + u where α = β + u β. (4b) It s possble to estmate from the sample data the regresson coeffcents β and α. But from the estmate of α t s not possble to compute estmates of the coeffcents β and β. Reason: The equaton α = β β + s one equaton contanng two unknowns, namely β and β. Result: Agan, t s not possble to compute from the sample data estmates of both β and β, the separate lnear effects of and on the regressand Y. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 4 of 9 pages
ECONOMICS 5* -- NOTE (Summary) 5. Interpretng Slope Coeffcents n Multple Lnear Regresson Models Consder the multple lnear regresson model gven by the followng populaton regresson equaton (PRE): Y = β + β + β + β + u (5) 4, and 4 are three dstnct ndependent or explanatory varables that determne the populaton values of Y. 4 Because regresson equaton (5) contans more than one regressor, t s called a multple lnear regresson model. The populaton regresson functon (PRF) correspondng to PRE (5) s: E ( Y x ) E( Y,, 4 ) = β + β + β + β44 = (6) where x s the 4 row vector of regressors: x = ( ). Interpretng the Slope Coeffcents n Multple Regresson Model (5) Each slope coeffcent β j s the margnal effect of the correspondng explanatory varable j on the condtonal mean of Y. Formally, the slope coeffcents {β j : j =,, 4} are the partal dervatves of the populaton regresson functon (PRF) wth respect to the explanatory varables { j : j =,, 4}: E ( Y x ) E( Y,, K, ) j = j k = β j 4 j =,, 4 (7) ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 5 of 9 pages
ECONOMICS 5* -- NOTE (Summary) For example, for j = n multple regresson model (5): E ( Y,, ) 4 = ( β + β + β + β 4 4 ) = β (8) Interpretaton: A partal dervatve solates the margnal effect on the condtonal mean of Y of small varatons n one of the explanatory varables, whle holdng constant the values of the other explanatory varables n the PRF. Example: In multple regresson model (5) Y = β + β + β + β + u (5) wth populaton regresson functon E ( Y,, 4 ) β + β + β + β44 4 4 = (6) the slope coeffcents β, β and β 4 are nterpreted as follows: β = the partal margnal effect of on the condtonal mean of Y holdng constant the values of the other regressors and 4. β = the partal margnal effect of on the condtonal mean of Y holdng constant the values of the other regressors and 4. β 4 = the partal margnal effect of 4 on the condtonal mean of Y holdng constant the values of the other regressors and. Includng and 4 n the regresson functon allows us to estmate the partal E Y,, whle margnal effect of on ( ) 4 holdng constant the values of and 4 controllng for the effects on Y of and 4 condtonng on and 4. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 6 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Interpretng the Slope Coeffcent β n Multple Regresson Model (5) Y = β + β + β + β + u (5) E ( Y,, 4 ) β + β + β + β44 4 4 = (6) Denote the ntal values of the explanatory varables, and 4 as 0, 0 and 40. The ntal value of the populaton regresson functon for Y for the ntal values of, and 4 s: E ( Y 0, 0, 40 ) β + β0 + β0 + β440 = (9) Now change the value of the explanatory varable by, whle holdng constant the values of the other two explanatory varables and 4 at ther ntal values 0 and 40. The new value of s therefore = 0 + The change n the value of s thus = 0 The new value of the populaton regresson functon for Y at the new value of the explanatory varable s: ( Y,, ) E = β + β + β0 + β440 0 40 = β + β( 0 + ) + β0 + β440 = + β0 + β + β0 + β440 β (0) ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 7 of 9 pages
ECONOMICS 5* -- NOTE (Summary) The change n the condtonal mean value of Y assocated wth the change n the value of s obtaned by subtractng the ntal value of the populaton regresson functon gven by (9) from the new value of the populaton regresson functon gven by (0): E( Y,, ) = E ( Y,, ) E ( Y,, ) 4 Solve for β n (): 0 40 0 0 40 = β + β0 + β + β0 + β440 ( β + β0 + β0 + β440 ) = β + β0 + β + β0 + β440 β β0 β0 β440 = β () E(Y,, 4) β = = = 0, = 0 4 E ( Y,, ) 4 β = the partal margnal effect of on the condtonal mean of Y holdng constant the values of the other regressors and 4. Comparng Slope Coeffcents n Smple and Multple Regresson Models Compare the multple lnear regresson model Y = β + β + β + β + u (5) wth the smple lnear regresson model 4 4 Y = β + β + u () Queston: What s the dfference between the slope coeffcent β n these two regresson models? ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 8 of 9 pages
ECONOMICS 5* -- NOTE (Summary) Answer: Compare the populaton regresson functons for these two models. For the multple regresson model (5), the populaton regresson functon s ( Y,, ) E 4 = β + β + β + β44 As we have seen, the slope coeffcent β n multple regresson model (5) s β n model (5) = E(Y 4 =,, ) = 0, = 0 4 E ( Y,, ) 4 For the smple regresson model (), the populaton regresson functon s ( Y ) E = β + β The slope coeffcent β n smple regresson model () s β n model () = E(Y ) = d E ( Y ) d Compare β n model (5) wth β n model () β n multple regresson model (5) controls for -- or accounts for -- the effects of and 4 on the condtonal mean value of the dependent varable Y. β n multple regresson model (5) s therefore referred to as the adjusted margnal effect of on Y. β n smple regresson model () does not control for -- or account for -- the effects of and 4 on the condtonal mean value of the dependent varable Y. β n smple regresson model () s therefore referred to as the unadjusted margnal effect of on Y. ECON 5* -- Note ovrnot.doc: Multple Regresson Models Page 9 of 9 pages