Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches

Transcription

1 September, 2005 Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches Mitchell A. Petersen Kellogg School of Management, orthwestern University and BER Abstract In both corporate finance and asset pricing empirical work, researchers are often confronted with panel data. In these data sets, the residuals may be correlated across firms and across time, and OLS standard errors can be biased. Historically, the two literatures have used different solutions to this problem. Corporate finance has relied on clustered standard errors, while asset pricing has used the Fama-MacBeth procedure to estimate standard errors. his paper examines the different methods used in the literature and explains when the different methods yield the same (and correct) standard errors and when they diverge. he intent is to provide intuition as to why the different approaches sometimes give different answers and give researchers guidance for their use. I thank the Center for Financial Institutions and Markets at orthwestern University s Kellogg School for support. In writing this paper, I have benefitted greatly from discussions with oby Daglish, Kent Daniel, Michael Faulkender, Wayne Ferson, Mariassunta Giannetti, John Graham, Chris Hansen, Wei Jiang, oby Moskowitz, Joshua Rauh, Michael Roberts, Paola Sapienza, Georgios Skoulakis, Doug Staiger, and Annette Vissing-Jorgensen as well as the comments of seminar participants at the Federal Reserve Bank of Chicago, orthwestern University, Stanford University, and the Universities of California at Berkeley, Chicago, Columbia, and Iowa. he research assistance of Sungoon Park, ick Halpern, Casey Liang, and Amit Patel is greatly appreciated.

2 I) Introduction It is well known that OLS standard errors are unbiased when the residuals are independent and identically distributed. When the residuals are correlated across observations, OLS standard errors can be biased and either over or underestimate the true variability of the coefficient estimates. Although the use of panel data sets (e.g. data sets that contain observations on multiple firms in multiple years) is common in finance, the ways that researchers have addressed possible biases in the standard errors varies widely and in many cases is incorrect. In recently published finance papers which include a regression on panel data, forty-two percent of the papers did not adust the standard errors for possible dependence in the residuals. 1 Approaches for estimating the coefficients and standard errors in the presence of within cluster correlation varied among the remaining papers. hirty-four percent of the papers estimated both the coefficients and the standard errors using the Fama-MacBeth procedure (Fama-MacBeth, 1973). wenty-nine percent of the papers included dummy variables for each cluster (e.g. fixed effects). he next two most common methods used OLS (or an analogous method) to estimate the coefficients but reported standard errors adusted for correlation within a cluster. Seven percent of the papers adusted the standard errors using the 1 I searched papers published in the Journal of Finance, the Journal of Financial Economics, and the Review of Financial Studies in the years for a description of how the coefficients and standard errors were estimated in a panel data set. I included both linear regressions as well as non-linear techniques such as logits and tobits in my survey. Panel data sets are data sets where observations can be grouped into clusters (e.g. multiple observations per firm, per industry, per year, or per country). I included only papers which report at least five observations in each dimension (e.g. firms and years). 207 papers met the selection criteria. Papers which did not report the method for estimating the standard errors, or reported correcting the standard errors only for heteroscedasticity (i.e. White standard errors which are not robust to within cluster dependence), were coded as not having corrected the standard errors for within cluster dependence. Where the paper s description was ambiguous, I contacted the authors. Although White/OLS standard errors may sometimes be correct, many of the published papers report regressions where I would expect the residuals to be correlated across observations on the same firm in different years (e.g. bid-ask spread regressed on exchange dummies, stock price, volatility, and average daily volume or leverage regressed on the market to book ratio and firm size) or correlated across observations on different firms in the same year (e.g. equity returns regresses on earnings surprises). In these cases, the bias in the standard errors can be quite large. See Section VI for two illustrations. 1

3 ewey-west procedure (ewey and West, 1987) modified for use in a panel data set, while 23 percent of the papers reported clustered standard errors (Williams, 2000, Rogers, 1993, Andrews, 1991, Moulton, 1990, Arellano, 1987, Moulton, 1986) which are White standard errors adusted to account for possible correlation within a cluster. hese are also called Rogers standard errors. Although the literature has used a diversity of methods to estimate standard errors in panel data sets, the chosen method is often incorrect and the literature provides little guidance to researchers as to which method should be used. Since the methods sometimes produce incorrect estimates, it is important to understand how the methods compare and how to select the correct one. hat is this paper s obective. here are two general forms of dependence which are most common in finance applications. hey will serve as the basis for the analysis. he residuals of a given firm may be correlated across years (time series dependence) for a given firm. I will call this a firm effect. Alternatively, the residuals of a given year may be correlated across firms (cross-sectional dependence). I will call this a time effect. I will simulate panel data with both forms of dependence, first individually and then ointly. With the simulated data, I can estimate the coefficients and standard errors using each of the methods and compare their relative performance. Section II contains the standard error estimates in the presence of a fixed firm effect. My results show that both OLS and the Fama-MacBeth standard errors are biased downward. he ewey-west standard errors, as modified for panel data, are also biased but the bias is small. Of the most common approaches used in the literature and examined in this paper, only clustered standard errors are unbiased as they account for the residual dependence created by the firm effect. In Section III, the same analysis is conducted with a time effect instead of a firm effect. Since 2

4 the Fama-MacBeth procedure is designed to address a time effect, not a firm effect, the Fama- MacBeth standard errors are unbiased. he intuition of these first two sections carries over to Section IV, were I simulate data with both a firm and a time effect. hus far, I have specified the firm effect as a constant (e.g. it does not decay over time). In practice, the firm effect may decay and so the correlation between residuals declines as the time between them grows. In Section V, I simulate data with a more general correlation structure. his allows me to compare OLS, clustered, and Fama-MacBeth standard errors in a more general setting. I show that OLS and Fama-MacBeth standard errors are biased and clustered standard errors are unbiased. Simulating the temporary firm effect also allows me to examine the relative accuracy of two additional methods for adusting standard errors: fixed effects (firm dummies) and an adusted Fama-MacBeth standard error whose use is becoming more popular. I show that including fixed effects eliminates the bias in OLS standard errors only when the firm effect is fixed. I also show that even after adusting Fama-MacBeth standard errors, as suggested by some authors, they are still biased (Cochrane, 2001). Most papers do not report standard errors estimated by multiple methods. hus in Section VI, I apply the various estimation techniques to two real data sets and compare their relative performance. his serves two purposes. First, it demonstrates that the methods used in some published papers produce biases in the standard errors and t-statistics which are significant. his is why using the correct method to estimate standard errors is important. Examining actual data also allows me to show how differences in standard error estimates (e.g. White versus clustered standard error) can provide information about the deficiency in a model and directions for improving them. 3

5 II) Estimating Standard Errors in the Presence of a Fixed Firm Effect. A) Clustered Standard Error Estimates. o provide intuition on why the standard errors produced by OLS are incorrect and how clustered standard errors correct this problem, it is helpful to very briefly review the expression for the variance of the estimated coefficients. he standard regression for a panel data set is: Y it ' X it β % ε it (1) where we have observations on firms (i) across years (t). X and ε are assumed to be independent of each other and to have a zero mean. he zero mean is without loss of generality and allows us to calculate variances as sums of the squares of the variable. he estimated coefficient is: ˆβ OLS ' ' β % X it Y it X 2 it ' X it ε it X 2 it X it (X it β % ε it ) X 2 it (2) and, taking the regressors as fixed, the variance of the coefficient is: Var [ ˆβ OLS & β ] ' E ' E 2 X it ε it X 2 it ε 2 it X 2 it X 2 it &2 &2 (3) ' σ 2 X σ2 ε (σ2 X )&2 ' σ 2 ε σ 2 X 4

6 his is the standard OLS formula and is based on the assumption that the errors are independent and identically distributed (Green, 2000). he independence assumption is used to move from the first to the second line in equation (3) (i.e., the covariance between residuals is zero). he assumption of an identical distribution (e.g., homoscedastic errors) is used to move from the second to the third line. 2 he independence assumption is often violated in panel data and this is the focus of the paper. In relaxing the assumption of independent errors, I initially assume the data has a fixed firm effect. hus the residuals consist of a firm specific component (γ i ) as well as a component which is unique to each observation (η it ). he residuals can be specified as: ε it ' γ i % η it (4) Assume that the independent variable X also has a firm specific component. X it ' µ i % ν it (5) Each of the components of X (µ and ν) and ε (γ and η) are independent of each other. his is necessary for the coefficient estimates to be consistent. 3 Both the independent variable and the residual are correlated across observations of the same firm, but are independent across firms. 2 Clustered standard errors are robust to heteroscedasticity. Since this is not my focus, I assume the errors are homoscedastic. I use White standard errors as my baseline estimates when analyzing actual data in Section VI, since the residuals are not homoscedastic in those data sets (White, 1984). 3 I am assuming that the model is correctly specified. I do this to focus on estimating the standard errors. In actual data sets, this assumption does not necessarily hold and would need to be tested. 5

7 corr( X it,x s ) ' 1 ' ρ X ' σ 2 µ / σ2 X ' 0 corr( ε it, ε s ) ' 1 ' ρ ε ' σ 2 γ / σ2 ε ' 0 for i' and t ' s for i' andallt s for all i for i' and t ' s for i' andallt s for all i (6) Given this data structure [equations (1), (4), and (5)], I can calculate the true standard error of the OLS coefficient. Since the residuals are no longer independent within cluster, the square of the summed residuals is not equal to the sum of the squared residuals. he same statement can be made about the independent variable. he co-variances must be included as well. he variance of the OLS coefficient estimate is: Var [ ˆβ OLS & β ] ' E ' E ' E 2 X it ε it 2 X it ε it X 2 &1 it ε2 it % 2 s't%1 X 2 it X 2 it &2 &2 X it X is ε it ε is ' σ 2 X σ2 e % (&1) ρ x σ2 X ρ e σ2 e σ 2 X &2 X 2 it &2 (7) ' σ 2 ε σ 2 X 1 % (&1) ρ X ρ ε I use the assumption that residuals are independent across firms in deriving the second line. Given the assumed data structure, the within cluster correlations of both X and ε are positive and are equal to the fraction of the variance which is attributable to the fixed firm effect. When the data have a fixed firm effect, the OLS standard errors will always understate the true standard error if and 6

8 only if both ρ X or ρ ε are non-zero. 4 he magnitude of the error is also increasing in the number of years in the data (see Bertrand, Duflo, and Mullainathan, 2004). o understand this intuition, consider the extreme case where the independent variables and residuals are perfectly correlated across time (i.e. ρ X =1 and ρ ε =1). In this case, each additional year provides no additional information and will have no effect on the true standard error. However, the OLS standard errors will assume each additional year provides additional observations and the estimated standard error will shrink accordingly and incorrectly. he correlation of the residuals within cluster is the problem the clustered standard errors (White standard errors adusted for clustering) are designed to correct. 5 By squaring the sum of X it ε it within each cluster, the covariance between residuals within cluster is estimated (see Figure 1). his correlation can be of any form; no parametric structure is assumed. However, the squared sum of X it ε it is assumed to have the same distribution across the clusters. hus these standard errors are consistent as the number of clusters grows (Donald and Lang, 2001; and Wooldridge, 2002). I return 4 If the firm effect is not fixed, the variance of the coefficient estimate is a weighted sum of the correlations between ε t and ε t-k times the correlation between X t and X t-k, for all k< and is equal to: Var [ ˆβ OLS & β ] ' σ 2 ε σ 2 X 1 % 2 k'1 (&k) ρ x,k ρ ε,k he auto-correlations can be positive or negative. It is thus possible for the OLS standard error to under or over-estimate the true standard error. I will address auto-correlations which decline as the lag length (k) increases in Section V. If the panel is unbalanced (different for each i), the true standard error and the bias in the OLS standard errors is even larger than equation (7) (see Moulton, 1986). 5 he exact formula for the clustered standard error is: S 2 ( β ) ' (& 1) ( & k) ( & 1) 2 X it ε it X 2 it 2 7

9 to this issue in Section III. B) esting the Standard Error Estimates by Simulation. I simulated a panel data set and then estimated the slope coefficient and its standard error. By doing this multiple times we can observe the true standard error as well as the average estimated standard errors. 6 In the first version of the simulation, I included a fixed firm effect but no time effect in the independent variable and the residual. hus the data are simulated as described in equations (4) and (5). Across simulations I assumed that the standard deviation of the independent variable and the residual are both constant at one and two respectively. his will produce an R 2 of 20 percent. Across different simulations, I altered the fraction of the variance in the independent variable which is due to the firm effect. his fraction ranges from zero to seventy-five percent in twenty-five percent increments (see able 1). I did the same for the residual. his allows me to demonstrate how the magnitude of the bias in the OLS standard errors varies with the strength of the firm effect in both the independent variable and the residual. he results of the simulations are reported in able 1. he first two entries in each cell are the average value of the slope coefficient and the standard deviation of the coefficient estimate. he standard deviation is the true standard error of the coefficient and ideally the estimated standard error will be close to this number. he average standard error estimated by OLS is the third entry in each cell and is the same as the true standard error in the first row of the table. When there is no firm effect in the residual (i.e. the residuals are independent across observations), the standard error 6 Each simulated data set contains 5,000 observations (500 firms and 10 years per firm). he components of the independent variable (µ ν) and the residual (γ η) are independent of each other and normally distributed with zero means. For each data set, I estimated the coefficients and standard errors using each method described below. he reported means and standard deviations reported in the tables are based on 5,000 simulations. he basic program which I used to simulate the data and estimate the coefficients and standard errors is posted on my web site. I have also posted the code for estimating the different standard errors which are discussed in this paper. 8

10 estimated by OLS is correct (see able 1, row 1). When there is no firm effect in the independent variable (i.e. the independent variable is independent across observations), the standard errors estimated by OLS are also unbiased, even if the residuals are highly correlated (see able 1, column 1). his follows from the intuition in equation (7). he bias in the OLS standard errors is a product of the dependence in the independent variable (ρ X ) and the residual (ρ ε ). When either correlation is zero, OLS standard errors are unbiased. When there is a firm effect in both the independent variable and the residual, then the OLS standard errors underestimate the true standard errors, and the magnitude of the underestimation can be large. For example, when fifty percent of the variability in both the residual and the independent variable is due to the fixed firm effect (ρ X = ρ ε = 0.50), the OLS estimated standard error is one half of the true standard error (0.557 = /0.0508). 7 he standard errors estimated by OLS do not rise as the firm effect increases across either the columns (i.e. in the independent variable) or across the rows (i.e. in the residual). he true standard error does rise. When I estimate the standard error of the coefficient using clustered standard errors, the estimates are very close to the true standard error. hese estimates rise along with the true standard error as the fraction of variability arising from the firm effect increases. he clustered standard errors correctly account for the dependence in the data common in a panel data set (Rogers, 1993, Williams, 2000) and produce unbiased estimates. 7 All of the regressions contained a constant whose true value is zero. he above intuition carries over to the intercept estimation. he estimated slope coefficient averages with a standard deviation of , when ρ X = ρ ε = he OLS standard errors are biased (0.0283) and the clustered standard errors are unbiased (0.0663). he simulated residuals are homoscedastic, so calculating standard errors which are robust to heteroscedasticity is not necessary. When I estimated White standard errors in the simulation they have the same bias as the OLS standard errors. For example, the average White standard error was compared to the OLS estimate of and a true standard error of when ρ X = ρ ε =

11 An alternative way to examine the magnitude of the bias is to examine the empirical distribution of the simulated t-statistics. he t-statistics based on the OLS standard errors are too large in absolute value (see Figure 2-A) percent of the OLS t-statistics are statistically significant at the 1 percent level (i.e. greater than 2.58). his is the intuition we saw in ables 1. he clustered standard errors are unbiased (see able 1), and the empirical distribution of the t-statistics is also correct (see Figure 2-B). 0.9 percent of the clustered t-statistics are significant at the one percent level. he reason the t-statistics give us the same intuition as the standard errors is because the standard errors are estimated very precisely. For example, the mean OLS standard error is with a standard deviation of and the mean clustered standard error is with a standard deviation of (when ρ X =ρ ε =0.50). 8 he bias in OLS standard errors is highly sensitive to the number of time periods (years) used in the estimation as well. As the number of years doubles, OLS assumes a doubling of the information. However if the independent variable and the residual are correlated within the cluster, the amount of information (independent variation) increases by less than a factor of two. he bias rises from about 30 percent when there are five years of data per firm to 73 percent when there are 50 years (when ρ X =ρ ε =0.50, see Figure 3). he robust standard errors are consistently close to the true standard errors independent of the number of time periods (see Figure 3). Most of the simulations in the paper are based on linear regressions. o evaluate the performance of the standard error estimates in a non-linear setting, I simulated data according to equations 4 and 5. I took y as the latent variable and either censored the bottom 25% of the data (y<- 8 I do not report the MSE of the standard error estimates, since they add no additional information beyond what is reported in ables 1 and 2. Since the variances of the standard error estimates is extremely small, the MSE are essentially equal to the bias squared. ables of MSE are available from the author. 10

12 1.5) or created a dummy variable (equal to one if y is positive, and zero otherwise). With this data I estimated a tobit and a probit model. he usual ( OLS ) standard errors are too small and the clustered standard errors are unbiased. he magnitude of the underestimate is the same as reported in able 1 for the tobit model and slightly smaller in the probit model. he results are available from the author. C) Fama-MacBeth Standard Errors: he Equations An alternative way to estimate the regression coefficients and standard errors when the residuals are not independent is the Fama-MacBeth approach (Fama and MacBeth, 1973). In this approach, the researcher runs cross sectional regressions. he average of the estimates is the coefficient estimate. ˆβ FM ' ' 1 ˆβ t i't i't X it Y it X 2 it ' β % 1 i't i't X it ε it X 2 it (8) and the estimated variance of the Fama-MacBeth estimate is calculated as: S 2 ( ˆβ FM ) ' 1 ( ˆβ t & ˆβ FM ) 2 & 1 (9) he variance formula, however, assumes that the yearly estimates of the coefficient (β t ) are independent of each other. his is only correct if X it ε it is uncorrelated with X is ε is for t s. As discussed above, this is not true when there is a firm effect in the data (i.e. ρ X ρ ε 0). hus, Fama- MacBeth variance estimate is too small in the presence of a firm effect. In the presence of a firm 11

13 effect, the true variance of the Fama-MacBeth estimate is: Var( ˆβ FM ) ' 1 Var( 2 ' Var( ˆβ t ) ' Var( ˆβ t ) % ˆβ t ) 2 &1 s't%1 Cov( ˆβ t, ˆβ s ) 2 % (&1) 2 Cov( ˆβ t, ˆβ s ) (10) Given our specification of the data structure (equations 4 and 5), the covariance between the coefficient estimates of different years is independent of t-s (which ustifies the simplification in the last line of equation 10) and can be calculated as follows for t s: Cov( ˆβ t, ˆβ s ) ' E X 2 it ' (σ 2 X )&2 E ' (σ 2 X )&2 E &1 X it ε it X it ε it X it X is ε it ε is X is ε is X is ε is X 2 is &1 (11) ' (σ 2 X )&2 ρ X σ 2 X ρ ε σ2 ε ' ρ X ρ ε σ ε σ 2 X Combining equations (10) and (11) gives us an expression for the true variance of the Fama- MacBeth coefficient estimates. 12

14 Var( ˆβ FM ) ' Var( ˆβ t ) ' 1 σ 2 ε σ 2 X % (&1) 2 Cov( ˆβ t, ˆβ s ) % (&1) 2 2 ρ X ρ ε σε σ 2 X (12) ' σ 2 ε σ 2 X 1 % (&1)ρ X ρ ε his is same as our expression for the variance of the OLS coefficient (see equation 7). he Fama-MacBeth standard error are biased in exactly the same way as the OLS estimates. In both cases, the magnitude of the bias is a function of the serial correlation of both the independent variable and the residual within a cluster and the number of time periods per firm. D) Simulating Fama-MacBeth Standard Errors. o document the bias of the Fama-MacBeth standard error estimates, I calculate the Fama- MacBeth estimate of the slope coefficient and the standard error in each of the 5,000 simulated data sets which were used in able 1. he results are reported in able 2. he Fama-MacBeth estimates are consistent and as efficient as OLS (the correlation between the two is consistently above 0.99). he standard deviation of the two coefficient estimates is also the same (compare the second entry in each cell of able 1 and 2). hese results demonstrate that both OLS and Fama-MacBeth standard errors are biased downward (see able 2). However the Fama-MacBeth standard errors have a larger bias than the OLS standard errors. For example, when both ρ X and ρ ε are equal to 75 percent, the OLS standard error has a bias of 60% (0.595 = /0.0698, see able I) and the Fama- MacBeth standard error has a bias of 74 percent (0.738 = /0.0183, see able II). Moving down the diagonal of able 2 from upper left to bottom right, the true standard error increases but the standard error estimated by Fama-MacBeth shrinks. As the firm effect becomes larger (ρ X ρ ε 13

15 increases), the OLS bias grows, and the Fama-MacBeth bias grows even faster. 9 he incremental bias of the Fama-MacBeth standard errors is due to the way in which the estimated variance is calculated. o see this we need to expand the expression of the estimated variance (equation 9). Var[ β FM ] ' ' 1 (&1) 1 (&1) X it ε it X 2 it & 1 (µ i %ν it )(γ i %η it ) i't (µ i % ν it ) 2 & 1 X it ε it X 2 it 2 (µ i %ν it )(γ i %η it ) (µ i % ν it ) 2 2 (13) he true variance of the Fama-MacBeth coefficients is a measure of how far each yearly coefficient estimate deviates from the true coefficient (one in our simulations). he estimated variance, however, measures how far each yearly estimate deviates from the sample average. Since the firm effect influences both the yearly coefficient estimate and the sample average of the yearly coefficient estimates, it does not appear in the estimated variance. hus increases in the firm effect (increases in ρ X ρ ε ) actually reduce the estimated Fama-MacBeth standard error at the same time it increases the true standard error of the estimated coefficients. o make this concrete, take the extreme example where ρ X ρ ε is equal to one; the true standard error is (σ ε /σ X ) ½ while the estimated Fama- MacBeth standard error is zero. his additional source of bias shrinks as the number of years increases since the estimated slope coefficient will converge to the true coefficient (see Figure 3). Although I have ust demonstrated that the Fama-MacBeth standard errors are biased in the 9 he distribution of empirical t-statistics is even wider for the Fama-MacBeth than for OLS (see Figures 2-A and 2-C). 25 percent of the Fama-MacBeth t-statistics are statistically significant at the 1 percent level compared to 15 percent of the OLS t-statistics (when ρ X = ρ ε = 0.50). 14

16 presence of a firm effect, they are often used to measure statistical significance in published papers when the underlying regression contains a firm effect. As part of my literature survey, I looked for papers which regressed a persistent firm characteristics on other persistent firm characteristics. his is the type of data that will likely contain a firm effect. Since I am not able to replicate each of the studies, I will discuss several examples of where Fama-MacBeth standard errors have been used with such data. he first example is a logit estimate of whether a firm pays a dividend (a highly persistent variable) on firm characteristics such as the firm s market to book ratio, the earnings to assets ratio, and relative firm size (Fama and French, 2001). Another example are the papers which examine how the market values firms by regressing a firm s market to book ratio on firm characteristics such as the firm s age, a dummy for whether it pays a dividend, leverage, and firm size (Pastor and Veronesi, 2003, and Kemsley and issim, 2002). 10 A third example are the papers which run capital structure regressions. In these papers, the authors try to explain a firm s use of leverage by regressing the firm s debt to asset ratio on firm characteristics such as the firm s market to book ratio, the ratio of property, plant, and equipment to total assets, the earnings to assets, depreciation to asset ratio, R&D to assets ratio, and firm size (see for example Baker and Wurgler, 2002, Fama and French, 2002, and Johnson, 2003). 11 Since both the left and right hand side variables in these 10 Both papers correct the Fama-MacBeth standard errors for the first order auto-correlation of the estimated slopes. Pastor and Veronesi (2003) report that this does not change their answer. I will show in Section V-C that this correction still produces biased standard errors and this may explain Pastor and Veronesi s finding that the adustment has little effect on their estimated standard errors. 11 Baker and Wurgler (2002) estimate both White and Fama-MacBeth standard errors but do not report the Fama-MacBeth standard errors since they are the same as the White standard errors. Fama and French (2002) acknowledge that Fama-MacBeth standard errors may understate the true standard errors and so report adusted Fama- MacBeth standard errors which I will discuss in Section V-C ( We use a less formal approach. We assume the standard errors of the average slopes... should be inflated by a factor of 2.5"). 15

17 three regressions are highly persistent, this is the kind of data which likely contains a significant firm effect. In Section VI-B, I will estimate a capital structure regression and show that the magnitude of the bias is indeed large. Despite the presence of a firm effect and the resulting bias of Fama- MacBeth standard errors documented above, they are still used in the literature. he literature is a teaching tool. Authors read published papers to learn which econometric methods are appropriate in which situations. hus when readers see published papers using the Fama-MacBeth standard errors in the kinds of regressions I have listed, they believe (incorrectly) that this approach is correct. he problem is actually worse. he finance literature also contains (incorrect) advice that the Fama-MacBeth approach corrects the standard errors for the residual correlation in the presence of a firm effect (e.g. ρ X 0 and ρ ε 0). For example, Wu (2004) uses...the Fama and MacBeth (1973) method to account for the lack of independence because of multiple yearly observations per company. Denis, Denis, and Yost (2002) argue that the...pooling of cross-sectional and time-series data in our tests creates a lack of independence in the regression models. his results in the deflated standard errors and, therefore, inflated t-statistics. o address the importance of this bias we estimate the regression model separately for each of the 14 calendar years in our sample... he coefficients and statistical significance of the other control variables are similar to those in the pooled cross-sectional time series data. In the presence of a firm effect, Fama-MacBeth and OLS standard errors are both biased, and as discussed above the estimates can be quite close to each other even when the bias is large (compare equations 7 and 12). he problem isn t with the Fama-MacBeth method, per se, only with its use. It was developed to account for correlation between observations on different firms in the same year, not to account for correlation between observations on the same firm in different years. 16

18 It is now being used and recommended in cases where it produces biased estimates and overstated significance levels. Given the Fama-MacBeth approach was designed to deal with time effects in a panel data set, not firm effects, I turn to this data structure in the next section. E) ewey-west Standard Errors. An alternative approach for addressing the correlation of errors across observation is the ewey-west procedure (ewey and West, 1987). his procedure was initially designed to account for serial correlation of unknown form in the residuals of a single time series. It has been modified for use in a panel data set by estimating only correlations between lagged residuals in the same cluster (see Bertrand, Duflo, and Mullainathan, 2004, Doidge, 2004, MacKay, 2003, Brockman and Chung, 2001). he problem of choosing a lag length is simplified in a panel data set, since the maximum lag length is one less than the maximum number of years per firm. 12 o examine the relative performance of the ewey-west, I simulated 5,000 data sets where the fixed firm effect is assumed to account for twenty-five percent of the variability of both the independent variable and the residual. he standard error estimated by the ewey-west is an increasing function of the lag length in the simulation. When the lag length is set to zero, the estimated standard error is numerically identical to the White standard error, which is only robust to heteroscedasticity (White, 1984). his is the same as the OLS standard error in my simulation. ot surprisingly, this estimate significantly underestimates the true standard error (see Figure 4). As the lag length is increased from 0 to 9, the 12 In the standard application of ewey-west, a lag length of M implies that the correlation between ε t and ε t-k are included for k running from -M to M. When ewey-west has been applied to panel data sets, correlations between lagged and leaded values are only included when they are drawn from the same cluster. hus a cluster which contains years of data per firm uses a maximum lag length of -1 and would include t-1 lags and -t leads for the t th observation where t runs from 1 to. 17

19 standard error estimated by the ewey-west rises from the OLS/White estimate of to when the lag length is 9 (see Figure 4). In the presence of a fixed firm effect, an observation of a given firm is correlated with all observations for the same firm no matter how far apart in time the observations are spaced. hus having a lag length of less than the maximum (-1), will cause the ewey-west standard errors to underestimate the true standard error when the firm effect is fixed. However, even with the maximum lag length of 9, the ewey-west estimates have a small bias underestimating the true standard error by 8% [0.084 = /0.0358]. As the simulation demonstrates, the ewey-west approach to estimating standard errors, as applied to panel data, does not yield the unbiased estimates produced by the clustered standard errors. he difference between the two estimates is due to the weighting function used by ewey West. When estimating the standard errors, ewey-west multiplies the covariance of lag (e.g. ε t ε t- ) by the weight [1-/(M+1)], where M is the specified maximum lag. If I set the maximum lag equal to -1, then the central matrix in the variance equation of the ewey-west standard error is: 2 X it ε it ' ' ' X 2 X 2 X 2 &1 it ε2 it % 2 &1 it ε2 it % 2 &1 it ε2 it % 2 s't%1 &t '1 &t '1 w(t&s)x it X is ε it ε is w() X it X it& ε it ε it& 1& X it X it& ε it ε it& (14) his is identical to the clustered standard error formula (see footnote 5) except for the weighting function [w()]. he clustered standard errors use a weighting function of one for all co-variances. he ewey-west procedure was originally designed for a single time-series and the weighting function was necessary to make the estimate of this matrix positive semi-definite. For fixed the 18

20 weight w() approaches 1 as the maximum lag length (M) grows. ewey and West show that if M is allowed to grow at the correct rate with the sample size (), then their estimate is consistent. However, in the panel data setting, the number of time periods is often small. he consistency of the clustered standard error is based on the number of clusters () being large, opposed to the number of time periods (). hus the ewey-west weighting function is unnecessary and leads to standard error estimates which have a small bias in a panel data setting. 13 III) Estimating Standard Errors in the Presence of a ime Effect. o demonstrate how the techniques work in the presence of a time effect I generated data sets which contain only a time effect (observations on different firms within the same year are correlated). his is the data structure for which the Fama-MacBeth approach was designed (see Fama-MacBeth, 1973). If I assume that the panel data structure contains only a time effect, the equations I derived above are essentially unchanged. he expressions for the standard errors in the presence of only a time effect are correct once I exchange and. A) Clustered Standard Error Estimates. Simulating the data with only a time effect means the dependent variable will still be specified by equation (1), but now the error term and independent variable are specified as: ε it ' δ t % η it X it ' ζ t % ν it (15) 13 Although the bootstrap method of estimating standard errors was rarely used in the articles which I surveyed, it is an alternative way to estimate the standard errors in a panel data set (see for example Kayhan and itman, 2004 and Efron and ibshirani, 1986). o test its relative performance, I drew 100 samples with replacement and re-estimated the regression for each simulated data set. When I drew observations independently (e.g. I drew 5,000 firm-year), the estimated standard errors are the same as the OLS standard errors reported in able I (e.g for the bootstrap versus for OLS when ρ X = ρ ε = 0.50). When I drew observations as a cluster (e.g. I drew 500 firms with replacement and took all 10 years for any firm which was drawn), the estimated standard errors are the same as the clustered standard errors (e.g for bootstrap versus for clustered). he results were essentially the same when I drew 1,000 samples for each simulated data set. 19

21 As before, I simulated 5,000 data sets of 5,000 observations each. I allowed the fraction of variability in both the residual and the independent variable which is due to the time effect to range from zero to seventy-five percent in twenty-five percent increments. he OLS coefficient, the true standard error, as well as the OLS and clustered standard errors are reported in able 3. here are several interesting findings to note. First, as with the firm effect results, the OLS standard errors are correct when there is no time effect in either the independent variable (σ(ζ)=0) or the residual (σ(δ)=0). As the time effect in the independent variable and the residual rise, so does the magnitude by which the OLS standard errors underestimate the true standard errors. When half of the variability in both comes from the time effect, the true standard error is eleven times the OLS estimate [11.0 = /0.0282, see able 3]. he clustered standard errors are much more accurate, but unlike our results with the firm effect, they underestimate the true standard error. he magnitude of the underestimate is small, ranging from 13 percent [ /0.1490] when the time effect accounts for 25 percent of the variability to 19 percent [ /0.4927] when the time effect accounts for 75 percent of the variability. he problem arises due to the limited number of clusters (e.g. years). When I estimated the standard errors in the presence of the firm effects, I had 500 firms (clusters). When I estimated the standard errors in the presence of a time effect, I have only 10 years (clusters). Since the clustered standard error places no restriction on the correlation structure of the residuals within a cluster, its consistency depends upon having a sufficient number of clusters. Based on these results, 10 clusters is too small and 500 is sufficient (see Kezdi, 2004, and Bertrand, Duflo, and Mullainathan, 2004, Hansen, 2005). o explore this issue, I simulated data sets of 5,000 observations with the number of years 20

22 (or clusters) ranging from 10 to 100. In all of the simulations, 25 percent of the variability in both the independent variable and the residual is due to the time effect [i.e. ρ X = ρ ε = 0.25]. he bias in the clustered standard error estimates declines with the number of clusters, dropping from 13 percent when there are 10 years (or clusters) to 4 percent when there are 40 years to under 1 percent when there are 100 years (see Figure 5). he bias in the clustered standard error estimates is a product of the small number of clusters. Since panel data sets of 10 or 20 years are not uncommon in finance, however, this may be a concern in practice. B) Fama-MacBeth Standard Errors. When there is only a time effect, the correlation of the estimated slope coefficients across years is zero and the standard errors estimated by Fama-MacBeth are unbiased (see equation 9 and 12). his is what I find in the simulation (see able 4). he estimated standard errors are extremely close to the true standard errors and the confidence intervals are the correct size. In addition to producing unbiased standard error estimates, Fama-MacBeth also produces more efficient estimates than OLS. For example, when 25 percent of the variability of both the independent variable and the residual is due to the time effect, the standard deviation of the Fama-MacBeth coefficient estimate is 81 percent [ /0.1490] smaller than the standard deviation of the OLS estimate (compare able 3 and 4). he improvement in efficiency arises from the way in which Fama-MacBeth accounts for the time effect. By running cross sectional regressions for each year the intercept absorbs the time effect. Since the variability due to the time effect is no longer in the residual, the residual variability is significantly smaller. he lower residual variance leads to less variable coefficient estimates and greater efficiency. I will revisit this issue in the next section when I consider the presence of both a firm and a time effect. 21

23 IV) Estimating Standard Errors in the Presence of a Fixed Firm Effect and ime Effect. According to the simulation results thus far, the best method for estimating the coefficient and standard errors in a panel data set depends upon the source of the dependence in the data. If the panel data only contains a firm effect, the standard errors clustered by firm are superior as they produce unbiased standard errors. If the data has only a time effect, the Fama-MacBeth estimates perform better than standard errors clustered by time when there are few years (clusters) and equally well when the number of years (clusters) is sufficiently large. Although the above results are instructive, they are unlikely to be completely descriptive of actual data confronted by empirical financial researchers. Panel data sets may include both a firm effect and a time effect. hus to provide guidance on which method to use I need to assess their relative performance when both effects are present. In this section, I will simulate a data set where both the independent variable and the residual have both a firm and a time effect. A conceptual problem with using these techniques (Clustered or Fama-MacBeth standard errors) is neither is designed to deal with correlation in two dimensions (e.g. across firms and across time). 14 he clustered standard error approach allows us to be agnostic about the form of the correlation within a cluster. However, the cost of this is the residuals must be uncorrelated across clusters. hus if we cluster by firm, we must assume there is no correlation between residuals of different firms in the same year. In practice, empirical researchers account for one dimension of the cross observation correlation by including dummy variables and account for the other dimension by 14 It is possible to estimate standard errors accounting for clustering in multiple dimensions, but only if there are a sufficient number of observations within each cluster. For example, if a researcher has observations on firms in industries across multiple years, she could cluster by industry and year (i.e. each cluster would be a specific year and industry, see Lipson and Mortal, 2004). In this case, since there are multiple firms in a given industry in each year, clustering would be possible. If clustering was done by firm and year, since there is only one observation within each cluster, this is numerically identical to White standard errors. 22

24 clustering on that dimension. Since most panel data sets have more firms than years, the most common approach is to include dummy variables for each year (to absorb the time effect) and then cluster by firm (Anderson and Reeb, 2004, Gross and Souleles, 2004, Petersen and Faulkender, 2005, Sapienza, 2004, and Lamont and Polk, 2001). I use this approach in my simulations. A) Clustered Standard Error Estimates. o test the relative performance of the two methods, 5,000 data sets were simulated with both a firm and a time effect. Across the simulations, the fixed firm effect accounts for either 25 or 50 percent of the variability. he fraction of the variability due to the time effect is also assumed to be 25 or 50 percent of the total variability. his gives us three possible scenarios for the independent variable [(25,25),(25,50), and (50,25)]. he same three scenarios were used for simulating the residual, for a total of nine simulations (see able 5). he results in the presence of both a firm and time effect (able 5) are qualitatively similar to what we found in the presence of only a fixed firm effect (able 1). he OLS standard errors underestimate the true standard errors whereas the standard errors clustered by firm are unbiased independent of how I specify the firm and time effects. As we saw above, the bias in the OLS standard errors increases as the firm effect becomes larger. B) Fama-MacBeth Estimates he statistical properties of the OLS and Fama-MacBeth coefficient estimates are quite similar. he means and the standard deviations of the estimates are almost identical (see able 5 and able 6), and the correlation between the two estimates is never less than in any of the simulations. Once I include a set of time dummies in the OLS regression, the difference in efficiency I found in ables 3 and 4 disappears. he OLS estimates are now as efficient as the Fama-MacBeth, 23

25 even in the presence of a time effect. he standard errors estimated by Fama-MacBeth, however, are once again too small, ust as I found in the absence of a firm effect (able 2). When 50 percent of the variability is from the firm effect and 25 percent comes from a time effect, the true standard error is three times the Fama-MacBeth estimate [3.1 = /0.0206]. Most of the intuition from the earlier tables carries over. In the presence of a fixed firm effect both OLS and Fama-MacBeth standard error estimates are biased down significantly. Clustered standard errors which account for clustering by firm produce estimates which are unbiased. he presence of a fixed time effect, if it is controlled for with dummy variables, does not alter these results, except for accentuating the magnitude of the firm effect and thus making the bias in the OLS and Fama-MacBeth standard errors larger. V) Estimating Standard Errors in the Presence of a emporary Firm Effect he analysis thus far has assumed that the firm effect is fixed. Although this is common in the literature, it may not always be true. he dependence between residuals may decay as the time between them increases (i.e. ρ(ε t, ε t-k ) may decline with k). In a panel with a short time series, distinguishing between a permanent and a temporary firm effect may be impossible. However, as the number of years in the panel increases it may be feasible to empirically identify the permanence of the firm effect. In addition, if the performance of the different standard error estimators depends upon the permanence of the firm effect, researchers need to know this. A) emporary Firm Effects: Specifying the Data Structure. o explore the performance of the different standard error estimates in a more general context, I simulated a data structure which includes both a permanent component (a fixed firm effect) and a temporary component (non-fixed firm effect) which I assume is a first order auto 24

26 regressive process. his allows the firm effect to die away at a rate between a first order autoregressive decay and zero. o construct the data, I assumed that non-firm effect portion of the residual (η i t from equation 4) is specified as η it ' ς it if t ' 1 ' φη it&1 % 1 & φ 2 ς it if t > 1 (16) hus φ is the first order auto correlation between η it and η it-1, and the correlation between η i t and η i,t-k is φ k. 15 Combining this term with the fixed firm effect (γ i in equation 4), means the serial correlation of the residuals dies off over time, but more slowly than implied by a first order auto-regressive and asymptotes to ρ ε (from equation 6). By choosing the relative magnitude of the fixed firm effect (ρ ε ) and the first order auto correlation (φ), I can alter the pattern of auto correlations in the residual. he correlation of lag length k is: Corr(ε i,t, ε i,t&k ) ' Cov( γ i % η i,t,γ i % η i,t&k ) Var( γ i % η i,t )Var(γ i % η i,t&k ) ' σ2 γ % φk σ 2 η σ 2 γ % σ2 η ' ρ ε % (1& ρ ε ) φ k (17) An analogous data structure is specified for the independent variable. he correlation for lags one 15 I multiply the ς term by %&1&- &φ& 2 to make the residuals homoscedastic. From equation (16), Var( η it ) ' σ 2 ς if t ' 1 ' φ 2 σ 2 ς % (1 & φ2 ) σ 2 ς ' σ2 ς if t > 1 where the last step is by recursion (if it is true for t=m, it is true for t=m+1). Assuming homoscedastic residuals is not necessary since the Fama-MacBeth and clustered standard errors are robust to heteroscedasticity (Jagannathan and Wang, 1998, & Rogers, 1993). However, assuming homoscedasticity makes the interpretation of the results simpler. If I assume the residuals are homoscedastic, then any difference in the standard errors I find is due to the dependence of observations within a cluster not heteroscedasticity. 25