Common Errors: How to (and Not to) Control for Unobserved Heterogeneity *

Transcription

1 Common Errors: How to (and ot to) Control or Unobserved Heterogeneity * Todd A. Gormley and David A. Matsa June 6 0 Abstract Controlling or unobserved heterogeneity (or common errors ) such as industry-speciic shocks is a undamental challenge in empirical research as ailing to do so can introduce omitted variables biases and preclude causal inerence. This paper discusses limitations o two approaches commonly used to control or unobserved group-level heterogeneity in inance research demeaning the dependent variable with respect to the group (e.g. industry-adusting ) and adding the group s mean as a control. We show that these techniques which are used widely in both asset pricing and corporate inance research typically provide inconsistent coeicients and can lead researchers to incorrect inerences. In contrast the ixed eects estimator is consistent and should be used instead. We also explain how to estimate the ixed eects model when traditional methods are computationally ineasible. (JEL G G G3 C0 C3) Keywords: unobserved heterogeneity group ixed eects industry-adust bias * For helpul comments we thank Michael Anderson Joshua Angrist Christian Hansen Alexander Lungqvist Mitchell Petersen agpurnanand R. Prabhala Michael Roberts ick Souleles Michael R. Wagner and Jerey Wooldridge as well as the seminar participants at Wharton and MIT. Matthew Denes Christine Dobridge Jingling Guan and Kanis Saengchote provided helpul research assistance. Gormley thanks the Rodney L. White Center or Financial Research Brandywine Global Investment Management Research Fellowship and the Cynthia and Bennett Golub Endowed Faculty Scholar Award or inancial support. The Wharton School University o Pennsylvania 360 Locust Walk Suite 400 Philadelphia PA 904. Phone: (5) Fax: (5) tgormley@wharton.upenn.edu Kellogg School o Management orthwestern University 00 Sheridan Road Evanston IL Phone: (847) Fax: (847) dmatsa@kellogg.northwestern.edu

2 Controlling or unobserved heterogeneity is a undamental challenge in empirical inance research because most corporate policies including inancing and investment depend on actors that are unobservable to the econometrician. I these actors are correlated with the variables o interest then without proper treatment omitted variables bias inects the estimated parameters and precludes causal inerence. In many settings important sources o unobserved heterogeneity are common within groups o observations creating a component o the regression errors that is common across observations. For example unobserved actors like investment opportunities are oten common across irms in an industry and aect many corporate decisions. Failing to control or such actors can cause serious identiication challenges. Although the existing literature uses various estimation strategies to control or unobserved group-level heterogeneity there is little understanding o how these approaches dier and under which circumstances each provides consistent estimates. Our paper examines this question and shows that some commonly used approaches typically lead to inconsistent estimates and can distort inerences. We ocus on two popular estimation strategies. The irst which we reer to as adusted-y (AdY) demeans the dependent variable with respect to the group beore estimating the model with ordinary least squares (OLS). A common example is when researchers industry-adust their dependent variable so as to remove common industry actors in a irm-level analysis. A second approach which we reer to as average eects (AvgE) uses the group s mean as a control in the OLS speciication. A common implementation o AvgE uses observations state-year mean to control or time-varying dierences in local economic environments. Both AdY and AvgE are widely used in empirical inance research. Articles published in top inance ournals including the Journal o Finance Journal o Financial Economics and Review o Financial Studies have used both approaches since at least the late 980s and they continue to be used today. Among articles published in these three ournals in we ound over 60 articles that Potential unobserved actors abound: Unobserved dierences in local economic environments management quality and the cost o capital to name a ew can also pose diicult identiication problems. The exact origin o the two estimators in inance is unclear; we suspect they were inspired by the event studies literature in which stock returns are regressed on market-average returns. AdY may have been inspired by analyses o market-adusted returns and AvgE by estimations o the market model.

3 employed at least one o the two techniques. The techniques are used to study a variety o inance topics including asset pricing banking capital structure corporate boards governance executive compensation and corporate control. Articles using these estimation methods have also been published in the American Economic Review Journal o Political Economy and Quarterly Journal o Economics. Our paper shows that despite their popularity the AdY and AvgE estimators rarely provide consistent estimates o models with unobserved group-level heterogeneity; both estimators can exhibit severe biases. In the presence o such heterogeneity the AdY estimator suers rom an omitted variable bias i there is any within-group correlation across observations either among or across independent variables in the model. Such correlations are likely in the presence o any unobserved group-level variation. The AvgE estimator suers rom a measurement error bias because even in the absence o any within-group correlations the group s sample mean measures the true unobserved heterogeneity with error. For both estimators the bias can be large and complicated; trying to predict even the sign o the bias is typically impractical because it depends on numerous correlations. 3 The shortcomings o the AdY and AvgE estimators stand in stark contrast to the ixed eects (FE) estimator another approach available to control or unobserved group-level heterogeneity. The FE estimator which instead adds group indicator variables to the OLS estimation is consistent in the presence o unobserved group-level heterogeneity. When there is only one source o unobserved grouplevel heterogeneity the FE estimator is equivalent to demeaning all o the dependent and independent variables with respect to the group and then estimating using OLS. The dierences between the estimators are important because the AdY and AvgE estimators can lead researchers to make incorrect inerences. We show that AdY and AvgE estimates can be more biased than OLS and even yield estimates with the opposite sign o the true coeicient. 4 AdY and AvgE can also be inconsistent even in circumstances in which the original OLS estimates would be consistent. 3 Our analysis ocuses on settings where the underlying data structure exhibits unobserved group-level heterogeneity; we do not analyze the perormance o AdY and AvgE estimators in other settings. ote however that even when the underlying data structure exactly matches the AdY or AvgE speciications (implying there are peer eects) both estimators are still be biased. See Section.4 or more details. 4 Our analysis compares the consistency o estimators as the number o observations goes to ininity. Any reerences in this paper to bias do not reer to the inite-sample properties o the estimator but rather the dierence between the probability limit o the estimate and the true parameter as goes to ininity.

4 When estimating a ew textbook inance models using each o the dierent techniques to control or unobserved heterogeneity we ind large dierences between the AdY AvgE and FE estimates and conirm that AdY and AvgE can exhibit larger biases than OLS and can yield coeicients o the opposite sign as FE. These dierences conirm the presence o unobserved group-level heterogeneity in these settings and o correlations within these commonly used data structures that cause the AdY and AvgE estimators to be inconsistent and potentially quite misleading in practice. Based on these indings we argue that AdY and AvgE and related estimators should not be used to control or unobserved group-level heterogeneity. The same is also true o related techniques used in the literature to remove unobserved heterogeneity. Any estimation that transorms the dependent variable but not the independent variables will typically yield inconsistent estimates. For example subtracting the group median or the mean or median o a comparable set o irms rom the dependent variable will yield inconsistent estimates by ailing to account or correlations across independent variables within these groups. The method o characteristically adusting stock returns in asset pricing which subtracts the return o a benchmark portolio containing stocks with similar characteristics beore comparing these stock returns across subsamples is another transormation that can be problematic because it does not control or how the benchmark characteristics vary across these subsamples. Regression analyses o conglomerates diversiication discount are also subect to a similar critique. Even a simple comparison o industry-adusted means beore and ater events as is common in analyses o corporate control transactions stock issues and other sets o 0/ events does not reveal the true causal eect o the events because the comparison ails to adust the implicit independent variable the event indicator to control or the share o irms in the aected industry. 5 FE estimators should be used instead o these other approaches. FE estimators are consistent because they are equivalent to transorming both the dependent and independent variables so as to remove the unobserved heterogeneity. For any AdY or AvgE estimator there is a corresponding FE estimator that properly accounts or correlations in both the dependent and independent variables. For example 5 Put dierently by demeaning the data using an industry mean that includes both aected and unaected irms the simple comparison incorrectly removes part o the events eect rom the estimate. 3

5 rather than industry-adusting a dependent variable or controlling or the dependent variable mean researchers should instead estimate a model with industry ixed eects. Likewise rather than correlating benchmark-portolio-adusted stock returns with explanatory variables o interest a researcher should instead estimate a model with ixed eects or each benchmark portolio. FE estimation still transorms the stock returns using the average returns or the benchmark portolios but also accounts or correlations between the benchmark portolios and the explanatory variables o interest. The FE estimator however also has limitations. Although the FE estimator controls or unobserved actors that vary across groups it is unable to control or unobserved within-group heterogeneities. FE estimation also cannot identiy the eect o independent variables that do not vary within groups and is subect to severe attenuation bias in the presence o measurement error. We discuss these limitations and provide guidance on when FE estimation is appropriate. We also describe how researchers can use instrumental variable (IV) techniques within the FE ramework to address concerns about attenuation bias and to identiy the eects o independent variables that do not vary within groups. We also address another limitation o FE that has motivated some researchers to use AdY or AvgE rather than FE computational diiculties that arise when trying to estimate FE models with multiple types o unobserved heterogeneity. As the size and detail o datasets has increased researchers are increasingly interested in controlling or multiple sources o unobserved heterogeneity. 6 When there are multiple sources o unobserved group heterogeneity in an unbalanced panel demeaning the data multiple times is not equivalent to ixed eects. FE estimation o such models requires a large number o indicator variables which can pose computational problems. The computer memory required to estimate these models can exceed the resources available to most researchers. We discuss techniques that provide consistent estimates or models with multiple highdimensional group eects while avoiding the computational constraints o a standard FE estimator. One 6 For example in analyses o executive compensation there is concern about unobserved heterogeneity both across managers and across irms (Graham Li and Qui 0; Coles and Li 0a). Controlling or both irm- and manager-level ixed eects may also be important in other contexts (Coles and Li 0b). Likewise researchers who use irm-level data are increasingly concerned about time-varying heterogeneity across industries such as industry-level shocks to demand. Such heterogeneity can be accounted or by using industry-by-time ixed eects (e.g. Matsa 00). 4

6 approach is to interact all values o the multiple group eects to create a large set o ixed eects in one dimension that can be removed by transorming the data. A second approach which helps to avoid potential attenuation biases and allows the researcher to estimate a larger set o parameters is to maintain the multidimensional structure but to make estimation easible by reducing the amount o inormation that needs to be stored in memory. This can be accomplished by using the properties o sparse matrices and/or by employing iterative algorithms. We discuss the relative advantages o each approach and how these techniques can be implemented easily in the widely used statistical package Stata. 7 Overall our paper provides practical guidance on empirical estimation in the presence o unobserved group-level heterogeneity a pervasive identiication challenge in empirical inance research. A small but impactul set o recent articles have addressed other challenges researchers ace. For example Bertrand Dulo and Mullainathan (004) and Petersen (009) recommend methods to account or correlation across residuals in computing standard errors; Almeida Campello and Galvao (00) and Erickson and Whited (0) compare methods used to account or measurement error in investment regressions; and Fee Hadlock and Pierce (0) evaluate the use o F-tests on indicator variables in managerial style regressions. The remainder o this paper is organized as ollows. Section describes the underlying identiication concern o estimating a model with unobserved group-level heterogeneity and describes why AdY and AvgE provide inconsistent estimates. Section contrasts the AdY and AvgE estimators with the FE estimator and discusses why other related estimation techniques which are commonly used in the literature will yield inconsistent estimates. Section 3 shows that dierences between the estimation techniques can be important in practice. Section 4 discusses limitations o the FE estimator and provides guidance on when its use is appropriate. Section 5 concludes.. Estimation Techniques Used to Control or Unobserved Group-Level Heterogeneity We consider the case in which an independent variable o interest has a causal eect on a dependent variable y which also has unobserved group-level heterogeneity that is correlated with. 7 To help interested researchers we have also posted code and additional resources on our website to show how common implementations o AdY and AvgE can be transormed into consistent FE estimators. 5

7 Assume the data exhibits the ollowing structure: y i i i var( ) 0 var( ) var( ) co rr( ) 0 i co rr( ) co rr( ) 0 i i i i co rr( ) 0 i () where the unit o analysis is i (e.g. irm) these units are organized into larger groups (e.g. industry) and there is some constant unobserved group eect or each group. There are a total o observations or the unit o analysis i with observations in each group. The model in Equation () can be easily augmented to relect more complicated sources o unobserved heterogeneity without aecting our subsequent analysis. Time-varying omitted actors (such as industry shocks that vary over time) can be captured by adding an additional subscript t to each variable including the unobserved heterogeneity. A model with two types o unobserved heterogeneity such as irm and year group eects in panel data can be captured by adding a second type o unobserved heterogeneity to Equation (). Because there exists nonzero correlation between the unobserved group term and the independent variable o interest ailing to account or the unobserved heterogeneity causes an omitted variable problem. We assume the unobserved group term is the only omitted variable that is and the independent variable o interest do not covary with the residual. For ease o exposition we urther assume that both the group term and independent variable are mean zero (i.e. μ = μ = 0). 8 It is well known that using ordinary least squares (OLS) to estimate the causal eect o on y when the data exhibit the structure speciied in Equation () will yield an inconsistent estimate. 9 OLS 8 This assumption only simpliies the analysis and has no eect on the estimate o β under the dierent estimation techniques we analyze in Section. Assuming nonzero means aects the estimate o the constant α. 9 Throughout the paper we use the standard large-sample approach to determine an estimate s consistency by taking the number o observations to ininity while holding group size constant. In our later analysis o average eects we discuss what happens when the number o observations per group also goes to ininity. The adusted-y estimates are not aected by group size. 6

8 estimates the ollowing speciication: y u. () OLS OLS OLS i i i And the OLS estimate is ˆ OLS. (3) The OLS estimate o is inconsistent when the correlation between the group term and the independent variable o interest is nonzero because o an omitted variable problem. By ailing to control or the group term OLS will relect the causal eects o both and on the dependent variable y. Because OLS is inconsistent in the presence o unobserved heterogeneity researchers must rely on other estimation techniques. Two popular approaches are adusted-y (AdY) which demeans the dependent variable with respect to the group beore estimating the model with OLS and average eects (AvgE) which uses the group s mean o the dependent variable as a control in an OLS speciication. In this section we describe the AdY and AvgE estimates and discuss why they typically lead to inconsistent estimates o the coeicient o interest. The source o the bias extends to models with more complicated data structures.. Adusted-Y estimation The AdY estimator attempts to remove the inluence o the group term on the dependent variable o interest by demeaning the dependent variable within each group. Adusted-Y estimation is applied or example at the industry level in irm-level panel datasets by subtracting the industry-mean rom the dependent variable o interest. When this adustment is applied at the industry or industry-year level researchers typically reer to the dependent variable as being industry-adusted. Although there are a variety o approaches used in practice a common implementation o AdY is to demean the dependent variable using the sample group s mean excluding the observation at hand. 0 More speciically the researcher calculates the group mean y i as 0 In some cases the authors do not exclude the observation at hand and in other cases the median is used. Both o these alternative approaches yield similarly inconsistent estimates and proo o this in the case when the mean o all observations is used is provided in the Appendix A6. 7

9 y i k k kgroup ki (4) and estimates the ollowing model using OLS: y y u. (5) AdY AdY AdY i i i i The adusted-y estimation however does not generally provide a consistent estimate o in the presence o unobserved group-level heterogeneity because the estimation suers rom an omitted variable problem. To see this it is helpul to re-express the sample group mean as where r y r (6) i i i k k kgroup ki. (7) In the presence o unobserved group-level heterogeneity as in Equation () the dependent variable in AdY estimation can be written as y y r. (8) i i i i i Comparing Equations (5) and (8) we see that the adusted-y estimation ails to control or r i leading to a biased estimate or i r i is correlated with the independent variable. In other words the AdY covariance between i and the estimation error u i is not zero when the correlation between r i and i is nonzero. But because i r includes the mean o the other observations in the group i it will be correlated with i i the s are correlated within groups. Such a correlation is likely when the group eect is correlated with i. Letting i represent the within-group correlation between s we can derive the sign and i magnitude o the potential bias in the AdY estimate o Equation (). Proposition. In the presence o unobserved group-level heterogeneity as in Equation () the adusted-y estimator yields an inconsistent estimate or β. Speciically ˆ AdY i. i 8

10 The proo o Proposition is provided in the Appendix. As with any omitted variable bias the direction and magnitude o the bias depend on the correlation o and the omitted variable which in this case is the within-group correlation between s. I the s are positively correlated within groups such that (0] i then the adusted-y estimate i or exhibits an attenuation bias. And vice versa i the s are negatively correlated within groups the adusted-y estimate o is biased away rom zero. In practice such within-group correlations are commonplace leading to inconsistent AdY estimates. For example consider a standard irm-level capital structure estimation where leverage is regressed onto multiple independent variables such as return on assets bankruptcy risk and market-tobook ratio. Because irms in the same industry are subect to common demand and technology shocks their return on assets bankruptcy risk and the other regressors exhibit positive within-industry correlation. Within-industry correlation in leverage the dependent variable results as well. By industry-adusting only leverage the AdY approach treats the correlation in the dependent variable but ails to address the correlation in the regressors. By ailing to control or the correlations in return on assets bankruptcy risk and the other regressors the AdY (or industry-adusting approach to removing the underlying industry-level heterogeneity) is inconsistent. The bias in the adusted-y estimation is present even with very large groups and even when standard OLS estimates are consistent. Because the AdY estimator suers rom an omitted variable bias increasing group size does not lessen the identiication problem the estimation s error term will always contain the summation o the other within-group s which are correlated with the independent variable i when i i 0. This is also why AdY is inconsistent even when the correlation between and is zero and the OLS estimate is consistent. In this case AdY introduces a new omitted variable problem in its attempt to control or a nonexistent omitted variable problem in the original OLS speciication. The bias o the AdY estimator becomes considerably more complex when there is more than one In practice the bias on β is typically attenuating because it is unusual or the s to be negatively correlated within groups when there is also a group component that has nonzero correlation with (as in Equation ()). 9

11 independent variable o interest. Suppose the true model is given by the ollowing: y Z i i i i var( ) 0 var( ) var( ) var( Z) Z co rr( ) 0 i co rr( ) co rr( ) 0 i i i i co rr( Z ) co rr( Z ) 0 i i i i co rr( ) i co rr( Z ) i i Z co rr( Z ). i Z (9) There is still ust one type o unobserved group-level heterogeneity but there are two independent variables o interest and Z. As beore assume that the independent variables i and Z i and the unobserved group-level eect are uncorrelated with the errors. The two independent variables however are correlated with each other and with the unobserved group-level heterogeneity. For ease o exposition again assume without loss o generality that both the group term and independent variables are mean zero (i.e. μ = μ Z = μ = 0). Proposition. In the presence o unobserved group-level heterogeneity and two independent variables as in Equation (9) the adusted-y estimator yields inconsistent estimates or both β and. Speciically Z i Z i i i Z Zi Z i i Z i ˆ AdY Z Z Z Z i Z i Zi Z AdY i Z i i i Z i ˆ Z Z Z where Zi Z and i i represent the within-group correlations among the independent i variables and i Z represents the within-group correlations across these variables. i The proo o Proposition is provided in the Appendix. 0

12 The direction and magnitude o the bias in the AdY estimator is ambiguous when there are two or more independent variables. The sign o ˆ AdY may not even match the sign o the true. The expression or ˆ AdY is more complicated in Proposition than in Proposition because the omitted variables includes both i and Z i which may each be correlated with the independent variables i and Z i. Even a within-group correlation across independent variables contributes to the inconsistency o AdY. Such correlations are again commonplace in practice. For example in the capital structure regression or industry-adusted leverage discussed earlier the independent variables bankruptcy risk and return on assets are likely to be positively correlated within industries because o common demand and technology shocks.. Average eects estimation Average eects estimation approaches the problem o unobserved heterogeneity a dierent way. Instead o adusting the dependent variable AvgE uses a proxy the group s sample mean y i to control or the unobserved variation. A common implementation o AvgE uses observations stateyear mean to control or time-varying dierences in local economic environments. It is again common to exclude the current observation when calculating the group sample mean or each observation. When there is one independent variable o interest the average eects approach estimates the ollowing regression: AvgE AvgE AvgE AvgE yi i yi ui. (0) In the presence o unobserved group heterogeneity as in Equation () average eects estimation is inconsistent. The underlying problem is that AvgE suers rom a measurement error bias. As seen in Equation (6) y i r i the groups sample means y i measure the unobserved variation with error r i. Such measurement error does not only bias the coeicient on the mismeasured variable; it also biases coeicients or other variables in the estimated equation. As shown in Appendix A6 not excluding the observation at hand also yields inconsistent estimates.

13 Proposition 3. In the presence o unobserved group-level heterogeneity as in Equation () the average eects estimator yields an inconsistent estimate or β. Speciically ˆ AvgE i i i i i i i i i i i i. The proo o Proposition 3 is provided in the Appendix. The sign and magnitude o the bias or ˆ AvgE is considerably more complicated than or ˆ AdY. The bias is complicated because the measurement error r i is correlated with both the mismeasured variable y i and with i when the within-group correlation among the s is nonzero. Put another way the measurement error can be viewed as an omitted variable in the AvgE speciication that is not ust correlated with i (as is the omitted variable in AdY) but is also correlated with y i the proxy or the unobserved variation. The AvgE estimator is likely to be inconsistent in practice. Similar to AdY the AvgE estimator will be inconsistent when there is correlation among independent variables within groups. For example in a irm-level AvgE estimation where a researcher uses the state-year mean to control or time-varying dierences in local economic environments any correlation among the independent variables o irms within a state such as return on assets would cause AvgE to be inconsistent. Such correlations are likely to exist because irms located in the same state are subect to similar local demand shocks and investment opportunities. Moreover even when 0 (and OLS is consistent) and 0 i (and AdY is i consistent) AvgE will still be inconsistent because o the measurement error. Likewise large group size does not eliminate the bias when either 0 or i i 0. Although increasing reduces the noise o the measurement error it does not disappear completely; as seen in Proposition 3 AvgE is still asymptotically inconsistent even when the number o observations per group goes to ininity. 3 3 The variance o the measurement error r i is given by [( ( ) ) / ( )] i i i i and only goes to zero i β = 0 and also goes to ininity.

14 .3 Relative perormance o OLS AdY and AvgE estimators Type error. The potential or incorrect inerences diers across the three approaches. When 0 both the AdY and AvgE estimation techniques provide less-biased coeicients than does OLS when there is unobserved group-level heterogeneity. Proposition 4. Assume 0 in Equation (). The adusted-y estimator or β is consistent. The average-eects estimator or β is inconsistent when but is closer than the ordinary least squares estimator to the true β. The proo o Proposition 4 is provided in the Appendix. AdY entirely avoids Type errors because AdY s bias is multiplicative. 4 AvgE however can incorrectly reect the null hypothesis because the bias in AvgE is also additive similar to the bias in OLS. When 0 the bias or AvgE however is less than that or OLS and unlike OLS the bias approaches zero when the number o observations per group increases. In act the AvgE estimator is asymptotically consistent when both and go to ininity and β = 0. Type error. Although both AdY and AvgE are less biased than OLS when 0 this is not necessarily the case when 0. In some cases AvgE and AdY will provide a less-biased estimate o than does OLS and in other cases they will be more biased and possibly even have the wrong sign. Some examples o this are provided in Table. Under certain parameters OLS will actually be less biased then both AdY and AvgE. But as shown in Table there are also cases in which OLS is less biased than AvgE but more biased then AdY and other cases in which OLS is less biased than AdY but more biased than AvgE. There are also cases in which the AvgE estimate actually has the opposite (and incorrect) sign. 5 The correlation between the independent variable o interest and the unobserved group-level 4 This is not true however when there is more than one independent variable. As shown in Proposition the bias in the AdY estimate is more complicated when there are two independent variables. With two independent variables as in Equation (9) AdY only avoids Type errors i both β = 0 and γ = 0. 5 In the case o ust one independent variable as in Table the AdY estimator cannot incorrectly lip the sign o β. But as seen in Proposition (and our later applications in Section 3) this is no longer true when there is more than one independent variable. In this case both the AdY and AvgE estimators can return an inconsistent estimate with the incorrect sign while the OLS estimate (also inconsistent) has the correct sign. 3

15 heterogeneity has a large eect on the relative perormance o each estimator. Figure graphs OLS AdY and AvgE estimates o Equation () as unctions o various parameter values when β =. Each igure panel shows the eect o varying a speciic parameter in the data structure while holding the rest constant. When not being varied the deault parameters values are 0.5 ; 0.5 ; 0 ; / ; and /. Panel A plots the impact o the correlation between the independent variable and the unobserved group-level heterogeneity on each estimator. AdY is less biased than the OLS only when the absolute magnitude o the correlation between and is large. This is because the AdY bias is unaected by whereas the magnitude o the OLS bias increases linearly in the absolute value o. The AvgE bias in contrast is nonlinear in. Under these parameters AvgE is extremely biased and can even reverse the sign o the coeicient or low values o whereas or high values o the AvgE is less biased than both OLS and AdY. The correlation between observations within a group also aects the relative perormance o the AdY and AvgE estimators. As shown in Panel B the OLS estimate is unaected by i i i i i i whereas the sign and magnitude o the bias or both AvgE and AdY depend on. As shown earlier both the omitted variable in AdY and the measurement error in AvgE include i and are correlated i i with i when 0 but the omitted variable in OLS is not. Because o this both AdY and AvgE are i i positively biased when is suiciently low (when 0 in the case o AdY) and are negatively biased when is high. i i i i i i An increase in the number o observations per group does not necessarily improve the perormance o the various estimators. As shown in Panel C the OLS and AdY bias are unaected by the number o observations per group because both estimators suer rom an omitted variable bias that is independent o. Bias in the AvgE estimate however is aected by ; but under the parameters displayed in Figure an increase in observations per group actually increases the bias. Although increasing reduces the measurement error s noise the measurement error is still correlated with i. On net the reduction in noise leads the AvgE estimator to asymptote to a biased estimate. The same 4

16 occurs when the variance o the error approaches zero. As shown in Panel D the AvgE estimate asymptotes to the same biased coeicient as the noise rom measurement error declines. For high values o the AvgE estimate asymptotes to a dierent biased coeicient which under these parameters has a bias o the opposite sign as when is low. The relative variation o the unobserved group-level heterogeneity to that o the independent variable / also has dierent implications or the various estimators. As shown in Panel E the AdY estimate is unaected whereas the magnitude o the bias is increasing with / or both OLS and AvgE albeit in opposite directions. In sum whether AdY or AvgE provides an improvement over OLS when β 0 depends on the exact parameter values but regardless all three estimates are inconsistent.. Fixed Eects Estimation and General Implications Comparing the AdY and AvgE estimators to the well-known ixed eects (FE) estimator provides urther insight into why AdY and AvgE are inconsistent. The comparison also highlights why other related estimation techniques commonly used in the literature yield inconsistent estimates.. Fixed eects estimation Although the OLS AdY and AvgE estimates are all inconsistent in the presence o unobserved heterogeneity the FE estimator is consistent. FE estimation inserts an indicator variable or each group directly into the OLS equation thereby allowing the predicted mean o the dependent variable to vary across each group. This estimation which is also reerred to as least squares dummy variable (LSDV) estimation is consistent because it controls directly or the unobserved group-level heterogeneity in Equation (). The FE estimate is consistent even when 0 and the original OLS estimate is consistent. Equivalently the FE estimator can be implemented by transorming the data to remove the unobserved group-level heterogeneity. This is implemented by demeaning all o the variables both the dependent and independent variables with respect to the group and then estimating OLS on the transormed data. Speciically ixed eects (FE) estimates 5

17 FE FE FE i i i y y u () where y k k kgroup kgroup k. () Even in the presence o unobserved heterogeneity as in Equation() the dependent variable in the FE speciication can be written as y y (3) i i i where kgroup k. (4) Because the s covariance with is zero as assumed in Equation () their covariance with the last term in Equation (3) is also zero and the FE estimate o will be consistent. 6 Although the estimates are consistent the standard errors must be appropriately adusted to account or the degrees o reedom. Typically the degrees o reedom is adusted downward (i.e. the estimated standard errors are increased) to account or the number o ixed eects removed in the within transormation. However when estimating cluster-robust standard errors (which allows or heteroscedasticity and within-group correlations) this adustment is not required so long as the ixed eects swept away by the within-group transormation are nested within clusters (meaning all the observations or any given group are in the same cluster) as is commonly the case (e.g. irm ixed eects are nested within irm industry or state clusters). Statistical sotware programs that estimate FE speciications make these adustments automatically. 7 See Wooldridge (00 Chapters 0 and 0) Arellano (987) and Stock and Watson (008) or more details. 6 In panel datasets in which the unit o analysis i is time (i.e. the group is a set o observations over time) irst dierencing can also be used to remove the unobserved group-level heterogeneity. Although both are consistent in this case the irst-dierence and ixed-eects estimators dier in their assumptions about the idiosyncratic errors. See Wooldridge (00 pg. 3) or details on the relative eiciency o the two estimators. 7 In the current version o Stata or example these standard errors are reported by the TREG command. The AREG command however reports larger cluster-robust standard errors that include a ull degrees o reedom adustment. More inormation on dierences between AREG and TREG cluster-robust standard errors is available on our website. 6

18 . Understanding AdY and AvgE in context o FE The within-group transormation o the FE estimator highlights the key problem o the AdY and AvgE estimators: They ail to control or the unobserved group-level heterogeneity across the independent variables. Comparing the FE estimation in Equation () with the true underlying structure o the demeaned dependent variable in Equation (3) we see that the FE estimator correctly controls or the independent variable mean and restricts its coeicient to equal β. The AdY estimator however implicitly assumes this coeicient is zero (see Equation (5)). The AvgE estimator makes the same mistake and also ails to restrict the coeicient on y i to equal one (see Equation (0)). Another way to understand why AdY and AvgE provide inconsistent estimates is to compare them to a regression o Y onto two independent variables and Z. As is well known a researcher interested in the eect o on Y controlling or Z can identiy this eect by regressing the residuals rom a regression o Y on Z onto the residuals rom a regression o on Z. Partialing out the eect o Z rom both and Y beore regressing Y on is equivalent to regressing Y on controlling or Z (see Greene 000 pp or more detail). This is the same reason why the within-group transormation implementation o the FE estimator is equivalent to least squares dummy variable estimation. The withingroup transormation is simply the result o partialing out the collection o indicator variables (the Z in this case) rom both the independent and dependent variables. The AdY estimation however is equivalent to partialing out the eect o Z rom only the dependent variable Y which is not equivalent to regressing Y on controlling or Z. The AvgE approach is equivalent to regressing Y on and the itted values rom a regression o Y on Z which is also not the same as regressing Y on and Z. By ailing to transorm the independent variable both estimators ail to control or the variation in which is correlated with the unobserved group-level heterogeneity..3 Other related estimation techniques are also inconsistent Using the same logic other ways o demeaning the dependent variable in AdY estimation such as subtracting a group median or subtracting a value-weighted group-level mean similarly result in inconsistent estimates. More complicated adustments to the dependent variable suer rom a similar 7

19 problem. In this section we discuss three related estimation techniques that are inconsistent or a similar reason and explain how to obtain consistent estimates in these cases..3. Conglomerate diversiication discount Subtracting the mean or median o a matched control sample (or a variable constructed based on a set o matched controls) presents a similar issue to AdY estimation. Demeaning the dependent variable using a matched or constructed control but not accounting or the demeaned component o the independent variable creates an omitted variable bias whenever this demeaned component is correlated with a group member s own independent variable. Studies examining conglomerates diversiication discount provide an example o this issue. In this literature it is common to regress conglomerates market value o equity onto various independent variables. But irst the conglomerate s market value is adusted using the market value o a constructed control o nonconglomerate irms rom similar industries. This estimation however will suer an omitted variable bias i the independent variables o the constructed control are correlated with the independent variables o the conglomerate. For example consider when the independent variable is investment. Assuming investment aects the market value o equity the investment o the constructed control will aect the adusted market value o equity and i the investment o the conglomerate and matched controls are correlated there will be an omitted variable bias. To obtain consistent estimates the researcher can do one o two things. The irst is to adust all the variables in the estimation both the dependent and independent variables using the constructed controls. The second and equivalent approach would be to estimate the model with both sets o observations the conglomerates and their constructed controls and include ixed eects or each conglomerate-control pairing..3. Characteristically adusted stock returns AdY-type estimators also can be ound in many asset pricing articles. One approach commonly used to remove the inluence o common risk actors is to adust irms stock returns using the average return o comparable irms. This method was irst proposed by Daniel Grinblatt Titman and Wermers (997) and has since been adopted widely in diverse settings. A researcher irst sorts the data into various 8

20 benchmark portolios based on irm-level characteristics such as size book-to-market ratios and momentum and then adusts the individual stock returns by demeaning them using the average return o other irms in the same benchmark portolio. From an econometric perspective there is nothing wrong with using the adusted return as a measure o stocks perormance as proposed by Daniel et al. (997); it accurately summarizes a portolio s perormance relative to a benchmark return. Problems arise however i the adusted return is then correlated with other (unadusted) stock or irm characteristics. For example it is common or researchers to calculate adusted stock returns sort them into portolios based on an independent variable that is thought to aect stock returns and then compare the adusted returns across the top and bottom portolios as a test o whether the independent variable aects stock returns. This sort however is equivalent to AdY estimation where the adusted returns are regressed onto indicators or each independent variable portolio while excluding the constant term rom the regression. Similar to other AdY estimators this approach typically provides inconsistent estimates because the speciication ails to control or how the average independent variable o other irms in the portolio inluences the adusted stock return. As an example consider analyses o R&D intensity and stock returns. Although R&D and returns are positively correlated it is possible that dierences in irm size conound this relationship. Larger irms are associated with lower stock returns (or reasons presumably unrelated to R&D intensity). I R&D intensity is also correlated with irm size then the correlation between R&D and returns may be attributable to irm size rather than a causal relation. Using size-matched benchmark portolios to adust returns but not adust R&D does not adequately control or size because it does not account or average dierences in R&D intensity across the benchmark portolios. The average R&D o irms in a stock s benchmark portolio aects its adusted stock return but this act is overlooked when one compares adusted returns across portolios sorted on irms unadusted R&D intensity. To control or unobserved risk actors across portolios appropriately one needs to adust the independent variable being analyzed in addition to adusting returns. One way to accomplish this is to double-sort returns on the benchmark portolio and the independent variable and then to compare returns across the independent variable within each benchmark portolio. In the R&D example this within 9

21 benchmark portolio comparison properly controls or dierences in average R&D intensity across irm size. The average dierence between the top and bottom independent variable portolios across all o the benchmark portolios summarizes that average eect o the independent variable on stock returns while properly controlling or the characteristics used to construct benchmark portolios. This quantity is equivalent to a ixed eects estimator; the identical estimate can be obtained by regressing returns onto indicators or each independent variable portolio (excluding the bottom portolio) and benchmark portolio ixed eects. 8 In Section 3.4 we provide an example o this alternative ixed eects approach and show that the more conventional characteristically adusted stock returns estimator is inconsistent..3.3 Dierence-in-dierences estimators Using AdY in the context o a dierence-in-dierences estimator also yields inconsistent estimates. A properly speciied dierence-in-dierences estimator compares the dierences in means between treated and untreated observations across pre- and post-treatment periods. In many applications in inance however such as studies o mergers and acquisitions and leveraged buyouts researchers instead compare the means o an industry-adusted dependent variable or treated irms across the two periods. This AdY-type comparison does not reveal the true causal eect o the event being analyzed because AdY removes a weighted-average o treated and untreated irms in the industry whereas the dierence-in-dierences estimator removes the mean o ust the untreated irms. Proposition 5. An AdY-type comparison o pre- versus post-treatment periods means or treated irms does not reveal the causal eect o the treatment. Speciically ˆ AdY = where is the average raction o untreated irms in a treated irm s group. The proo o Proposition 5 is provided in the Appendix. The AdY-type comparison suers an attenuation bias because it incorrectly removes part o the treatment s eect on the dependent variable when it demeans the data using a weighted average o treated 8 In practice the double-sort is cumbersome to report when there are many benchmark portolios (e.g. 5 size x 5 book-to-market x 5 momentum = 5 portolios) and systematic patterns may be diicult to eyeball. The FE estimator accurately summarizes these patterns and is easy to implement. 0

22 and untreated irms. The source o this bias is the same as the more general AdY bias described previously. In regression orm the AdY approach to dierence-in-dierences suers an omitted variable problem in that it ails to control or the share o irms in each industry that are treated (see the Appendix or more details)..4 AdY and AvgE are also inconsistent under other data structures Even i the true underlying data structure did not exhibit unobserved group-level heterogeneity and exactly matched the AdY and AvgE speciications both estimators would still be biased. Suppose the underlying data exhibited the ollowing structure: y y. (5) i i i i This data structure contains a peer eect such that each observation within a group inluences the other observations in the group. However as shown in Manski (993) and Leary and Roberts (00) OLS estimation o the peer eects model does not reveal causal eects because o a relection problem. Because the dependent variable has a causal eect on the dependent variable o other group members using the group mean as an independent variable in OLS introduces an endogeneity problem Comparing Approaches in Common Finance Applications In this section we examine how important the dierences between the various approaches actually are in practice. We estimate standard empirical inance models using each o the various estimators OLS FE AdY and AvgE and compare the resulting estimates. 3. Capital structure and unobserved heterogeneity across irms We start by estimating a standard capital structure OLS regression: ( D/ A) β' (6) it it it where (D/A) it is book leverage (debt divided by assets) or irm i in year t and it is a vector o variables thought to aect leverage. We use data or the period o rom Compustat and we include ive independent variables in it : ixed assets / total assets Ln(sales) return on assets modiied Altman-Z score and market-to-book ratio. All variables are winsorized at the % level and the standard 9 For the same reason both OLS and FE would also yield inconsistent estimates o the peer eects model.

23 errors are adusted or clustering at the irm level. To account or possible unobserved irm-speciic actors in the residual it that covary with it we estimate the model using the our dierent techniques analyzed here: OLS AdY AvgE and FE. The estimates are reported in Table. The various estimation techniques lead to very dierent estimates or β conirming the importance o within-irm correlations that cause OLS AdY and AvgE to yield inconsistent estimates. The OLS estimates reported in Column () o Table dier considerably rom the FE estimates in Column (4); this suggests the presence o an unobserved irm characteristic like the irm s cost o capital or investment opportunities that is correlated with both the dependent variable (leverage) and independent variables (e.g. ROA). Such unobserved heterogeneities cause the OLS and AvgE estimates to be inconsistent. The AdY estimates (Column ()) also dier rom the FE estimates. Based on Propositions and the large dierences between AdY and FE imply that there is considerable correlation among independent variables within groups; or example both ixed assets/total assets and Ln(sales) are serially correlated across time or irms. Such correlations cause both AdY and AvgE to be inconsistent. The estimates in Table illustrate that these within-irm correlations can lead to severe biases and incorrect inerences. For example or the coeicient on the proportion o ixed assets the AdY and AvgE estimates are 73% and 58% smaller in magnitude than the FE estimate respectively; or the z-score the AdY and AvgE are smaller in magnitude than the FE estimate by about 40%. A researcher using either AdY or AvgE might thereore iner that the role o collateral or bankruptcy risk on leverage is considerably smaller than the truth. AdY and AvgE can even yield an estimate that has the opposite sign as OLS and FE. As reported in Table the OLS estimate or return on assets is Relying on the AdY and AvgE estimates o 0.05 and respectively one might conclude that unobserved irm-level heterogeneity imposes a large downward bias on the OLS coeicient. But the FE estimate o almost twice the OLS estimate suggests that the bias actually works the other way. Trying to understand (or predict) the sign o the bias o AdY and AvgE is typically impractical. As seen in Propositions and 3 the AvgE bias is very complicated even with ust one independent variable and AdY is similarly complicated with ust two independent variables. When there are even