Faculty & Research. Working Paper Series. Estimation in the Presence of Unobserved Fixed Factors: Consistency or Efficiency?

Transcription

1 Faculty & Research Estimation in the Presence of Unobserved Fixed Factors: Consistency or Efficiency? by M. Christen and H. Gatignon 003/5/MKT Working Paper Series

2 Estimation in the Presence of Unobserved Fixed Factors: Consistency or Efficiency? Markus Christen and Hubert Gatignon* January 13, 003 * Markus Christen is Assistant Professor and Hubert Gatignon is the Claude Janssen Chaired Professor of Business Administration and Professor of Marketing at INSEAD, France and Singapore. Address: Prof. Markus Christen; INSEAD, Boulevard de Constance, Fontainebleau-Cedex, France; ph: , fax: , markus.christen@insead.edu.

3 Estimation in the Presence of Unobserved Fixed Factors: Consistency or Efficiency? Abstract The recent literature examining the impact of strategic factors on business performance suggests the use of estimation methods that control for unobserved fixed firm factors. This recommendation is driven by the consideration of unbiasedness as the exclusive criterion for estimator selection, even though this may lead to inefficient estimators resulting in the inability to detect significant effects. The concern for unbiasedness is also reflected in instrumental estimation. The selection of instruments is typically focuses on the independence assumption of instruments and error term at the expense of the explanatory power of instruments. In this study, we examine the implications of making the statistical tradeoffs between the bias and the variance of estimators by considering the mean square error as the criterion for estimator selection. More specifically, we identify three measures that determine the selection of the estimator among the consistent but potentially inefficient fixed-effects estimator, the inconsistent but efficient random-effects estimators, and IVestimation: (i) the within cross-sections variance relative to total sample variance of the independent variable, (ii) the correlation between the firm specific factor and the independent variable, and (iii) the quality of the instrumental variable. Through analytical evaluations of the estimators and extensive Monte-Carlo simulations, we determine conditions for selecting the appropriate estimator. Because the correlation between firm specific effects and the independent variables are unknown, we use simulation data and take advantage of the structural constraints imposed on this correlation to predict that correlation based on measurable factors. We illustrate the value of the methodology by applying these criteria to the study of the effect of market share on profitability. We demonstrate that, using the PIMS data, the key determinants are within a range where the fixed-effects model is not necessarily best in terms of MSE and we conclude that, if one is willing to trade off some bias for efficiency, the evidence tends to corroborate earlier work for a significant market share impact on firm performance.

4 1. Introduction Without properly controlling for unobserved factors, results from empirical analyses of strategic actions on business performance may be questionable because one cannot exclude the possibility that an observed relationship between action and performance is caused by other factors such as good management capabilities or luck. Thus, the results can be spurious (Boulding 1990; Jacobson 1990a). One of the best-known examples of this problem is the effect of market share on business profitability. Research shows that the apparent effect of market share on ROI goes away after controlling for various unobserved effects (e.g., Ailawadi et al. 1999; Boulding and Staelin 1993; Jacobson and Aaker 1985). Boulding and Staelin (1995) advocate an estimation approach that is capable of controlling for the potentially biasing effect of the following three types of unobserved factors: (1) firm-specific factors that do not change over the time of analysis; () contemporaneous random shocks; and (3) dynamic factors whose influence dissipates over time. They suggest the use of lagged variables as instruments to control for contemporaneous shocks, followed by a differencing procedure to remove first-order autocorrelated effects (see also Erickson and Jacobson 199; Jacobson 1990a). Unbiased estimates are then obtained with fixed-effects estimation (Hsiao 1986). The debate about the appropriate estimation approach has largely focused on obtaining unbiased estimates and paid little attention to the potential costs of achieving this objective. These costs can be quite significant. First, the approach advocated by Boulding and Staelin (1995) implies that empirical studies of firm performance cannot be done with cross-sectional data alone. This eliminates typical survey data unless appropriate instruments can be found, that is, variables that are themselves free of any correlation with unobserved factors. Collecting panel data imposes a significant cost on researchers, yet it is not clear to what extent a second or third observation per cross-section is sufficient to obtain unbiased 1

5 estimates with a reasonable level of efficiency. In fact, the procedure proposed by Boulding and Staelin (1995) requires at least four periods (e.g., years) of observations per cross-section. Second, the recommended statistical procedures often lead to large variances of the parameter estimates. Fixed-effects estimation removes all variance between cross-sections and relies solely on the variance that remains within a cross-section, i.e., the variance over time for a cross-section. In most cases, persisting differences between firms are much more extensive than changes over time within a particular firm. This holds particularly true for strategic firm decisions and the time horizon used in most empirical studies. In the extreme case when the strategic variable of interest does not change over time, fixed-effects estimation cannot be applied at all. What happens when the strategic variable changes but very little over time? How much change is needed for fixed-effects estimation to be suitable? Is it possible that a biased but more efficient random-effects estimate is statistically closer to the true effect than an unbiased fixed-effects estimate? Instrumental-variable (IV) estimation offers another approach to obtain unbiased results. Hausman and Taylor (1981) developed an IV-estimator that controls for unobserved fixed factors and does not suffer from inefficiency when there is little within-variance. In fact, it can even be used to estimate the effects of variables that do not change at all over time. The problem with this approach is finding appropriate instruments. Instrument selection has traditionally focused more on the independence assumption of instruments and error term and less on the explanatory power of instruments. However, weak instruments, i.e., instruments that do not explain much of the explanatory variables, can increase rather than decrease the bias when the instrument is not exactly exogenous (Bound, Jaeger and Baker 1995; Staiger and Stock 1997). This problem occurs in addition to the reduction of efficiency due to the instrumentation. To assess the relevance of instruments, some diagnostic tools have been proposed (Hahn and Hausman 00; Hall et al. 1996; Shea 1997). However, they do not

6 indicate when instruments are sufficiently strong to also overcome potential inefficiency problems. Again, is it possible that a biased random-effects estimate or an inefficient fixedeffects estimate is closer to the true effect than an IV-estimate? The objective of this paper is to systematically examine the tradeoff between bias and efficiency for different estimators in the presence of unobserved factors that are correlated with explanatory variables. In particular, we focus on the problem of controlling for fixed unobserved factors. The importance of fixed factors is, among other things, driven by the resource-based view of the firm, which suggests that a substantial part of differences in firm performance must be attributed to unique firm resources (Wernerfelt 1984). 1 Moreover, controlling for contemporaneous and/or dynamic shocks is easier when longitudinal data are available and typically does not reduce the efficiency as much as removing fixed-effects. The likely problem of focusing solely on obtaining unbiased estimates can be illustrated with the results in Ailawadi et al. (1999). This study, using PIMS data, reports a strong and positive effect of market share on ROI after controlling for contemporaneous shocks and measurement error (b = 0.498, s.e. = 0.05). However, when also controlling for unobserved fixed factors, the effect of market share is no longer significantly different from zero (b = -0.7, s.e. = 0.553). At the same time, the standard error increases more than twenty-fold. The data transformation required by fixed-effects estimation removes over 97 percent of the variance in the market share variable. It is thus unclear whether the nonsignificant result implies that the observed market share effect is caused by unobserved factors and thus spurious or whether it reflects the inefficiency of fixed-effects estimation. 1 Boulding (1990) and Jacobson (1990b) disagree about the importance of time-invariant unobserved factors. Jacobson argues that the presence of such factors implies that abnormal returns persist indefinitely. Since most empirical studies cover only a few years, unobserved factors like advertising, branding, pioneering, which decay only slowly are typically treated as time-invariant. 3

7 This issue is at the basis of the controversy surrounding the impact of strategic factors on profitability. If one is only interested in directional results, the interpretation of results obtained by following the procedure advocated by Boulding and Staelin (1995) is clear as long as it yields significant parameter estimates. However, the pattern of the market share- ROI estimates is not unusual. Of the various market share effects tested by Ailawadi et al. (1999), only 1 of five significant estimates remains significant after controlling for fixed factors. Estimating the effect of a market orientation provides another example. Narver and Slater (1990), using cross-sectional data, find a positive effect of market orientation on firm performance. A follow-up study by Narver, Jacobson and Slater (1993), which controls for different unobserved factors, finds mostly insignificant effects. If one is also interested in the magnitude of an effect, efficiency becomes even more important. The difference between the random-effects and fixed-effects estimators disappears as the number of observations within cross-sections approaches infinity. However, for samples of a limited size, the magnitude of the advantages and disadvantages of different estimation approaches is generally not well understood. Taylor (1980) compares the performance of the random-effects and fixed-effects estimators in small samples when the factors of interest are independent of unobserved factors and thus, both approaches yield consistent estimates. He found that except for very small panel datasets the random-effects estimator is preferred even when the variance components are unknown and must be estimated. When both estimators yield unbiased estimates, a comparison of efficiency is sufficient for an assessment of the two approaches. When the comparison includes a potentially biased (random-effects) estimator, the appropriate measure, from a decision-analytic point of view, is its mean square error (MSE), which is defined as the sum of the square of the bias and the variance of the estimate (Bass and Wittink 1975; 1978; Judge et al. 1985; Wallace 197). This measure captures the tradeoff between bias and inefficiency. 4

8 In sum, there is no dominant estimation approach when unobserved fixed factors are present with sample sizes typical of marketing studies. Thus, we address the following questions in this paper: How does a biased random-effects estimator compare to a fixed-effects estimator under different conditions in terms of mean square error (MSE)? How much withinvariance relative to the total variance is needed in an explanatory variable for fixedeffects estimation to be suitable? How strong do instruments have to be for IV-estimation to be preferred to randomeffects estimation or fixed-effects estimation in terms of MSE? How well does a cross-sectional analysis perform in terms of MSE and by how much does a second or third observation per cross-section improve the MSE? How can sample statistics, specification tests and estimation results be used to select the estimation approach with the lowest MSE in a given situation? How sensitive is the Hausman specification test to detect the presence of endogenous fixed factors when the within-variance component of the data is limited? The paper is organized as follows. In the next section we introduce a simple model on which we base our study and provide some analytic results and approximations to assess the tradeoff between bias and efficiency. Since exact analytic expressions do not exist when the explanatory variables are not independent of the error term, we use a Monte-Carlo simulation to address these research questions. We then describe our Monte-Carlo simulation and present the results. We illustrate the insights from the simulation results by examining the effect of market share on different performance measures using PIMS data. We conclude the paper with several specific recommendations relevant to empirical research.. Model and Analytic Results For our analysis we use the following one-factor linear model, which we write as (1) y it = x it + u it, and u it = i + it, i = 1 N, t = 1 T, where i and it are independent, normally distributed random variables with mean zero and variance and, respectively. Grouping the data by individuals (cross-sections), the 5

9 covariance matrix of the combined unobservables, u it, is = I N, = I T + j T j T, where j T denotes a T-vector of ones. Our analysis focuses on obtaining an estimate of 1. Through appropriate transformation of the data, the independent and dependent variables can be partitioned into two sub-samples, namely into N observations with crosssection means over time and N(T 1) observations of deviations from these cross-section means (Taylor 1980), y B XB u B () β, y W X W u W with X B = [j N x ], X W = [j N(T-1) x ], 1 x i. x it and x x x, where x T t is a vector of length N and x a vector of length N(T-1) (y B, y W, u B and u W are defined similarly). This orthogonal partition splits the variance of an independent variable, x, into the variance between cross-sections and the variance within cross-sections, i.e., x x B x W. We denote the ratio of the within-variance to the total variance as x (3) W W. x Estimators and Their Problems This partition forms the basis for three different estimators of, (1) the between-groups estimator (BT), β ˆ B 1 XB XB XB y B, () the within-group or fixed-effects estimator (FE), βˆ W 1 X W X W XW y W, and (3) the random-effects or Gauss-Markov estimator (GM), which is a weighted average of the first two estimators, i.e., GM B W βˆ W varβˆ var W βˆ B β ˆ δβˆ 1 δ βˆ, where δ var (Bass and Wittink 1975; Hsiao 1986; Taylor 1980). The feasible random-effects estimator (FGM) requires that the variance components and We use bold face-type and capital letters to indicate matrices. 6

10 to calculate be replaced by the respective estimators s and s. The between-groups estimator is used in the traditional PIMS studies (e.g., Buzzell and Gale 1987), while the fixed-effects estimator is advocated by Boulding and Staelin (1995). If the unobservables, u it, are uncorrelated with the independent variable, x it, in equation (1), all three estimators are unbiased but the random-effects estimator is most efficient (Hsiao 1986). When the number of cross-sections is large, the random-effects estimator dominates the within or fixed-effects estimator (Taylor 1980). Only for samples with few cross-sections (N k 10, where k is the number of regressors) is there any ambiguity in estimator choice. In this case, the tradeoff between the two estimators is influenced by the efficiency of the estimates of the variance components, which are needed for the feasible random-effects estimator. With cross-sectional data only, i.e., when T = 1, one can only choose between simple OLS and instrumental variable (IV) estimation. With panel data, i.e., when T > 1, it is also possible, in addition to the three estimators already mentioned, to use the simple OLS estimator and a two-stage IV-estimator with instruments, z, for the explanatory variable, x, followed by random-effects estimation. When the independent variable, x, is fixed over time, it is the only estimator that can yield a consistent estimates of 0 and 1 (Hausman and Taylor 1981). Table 1 summarizes these different estimators and their potential problems. INSERT TABLE 1 ABOUT HERE Several assumptions underlying the various estimators can be tested. Breusch and Pagan (1979) developed a Lagrange multiplier test for the assumption that = 0. This test enables a researcher to choose between a simple OLS estimator and a random-effects estimator. When is large enough and thus the error covariance matrix no longer diagonal, i.e., E[uu] I NT, the random-effects estimator is more efficient because it appropriately 7

11 weighs the between-variance and the within-variance of the data (Hsiao 1986). When is high enough, the specification test by Hausman (1978) can then be used to test the assumption that the fixed effects,, and the independent variable, x, are uncorrelated, i.e., E[x ] = 0. 3 The Hausman specification test (Hausman 1978) can be used to test the independence assumption of random-effects estimation and thus select between random-effects and fixedeffects estimation. A statistically significant difference between the two estimators is used as an indication of the presence of unobserved fixed factors that are correlated with explanatory variables. However, this test does not consider efficiency differences. It is also susceptible to other problems. On the one hand, a significant test result could indicate a model misspecification (Jacobson 1990b; Schmalensee 1987). On the other hand, an insignificant difference could be the result of insufficient power, a problem that plagues this test (Holly 198). For the data used by Ailawadi et al. (1999) the specification test does not reject the random-effect estimate of the effect of market share. For an IV-estimator, the appropriateness of instruments can also be tested. An instrument, z, that is only weakly correlated with the independent variable, x, can lead to even more biased estimation results when E[z u] is not exactly zero (Bound, Jaeger and Baker 1995; Staiger and Stock 1997). In the case with a single independent variable and one instrument the sample correlation can be used, which is identical to using the R of the firststage regression. The -test statistic is LR = -NTlog(1 - xz ), where xz is the correlation between the independent variable, x, and its instrument, z. Otherwise more complex tests must be performed (Hahn and Hausman 00; Hall et al. 1996; Shea 1997). With crosssectional data, the independence of the instrument, i.e., E[z ] = 0, can only be assured through theoretical arguments. With panel data, the independence of the instrument(s) can be 3 It requires, however, that the independent variable, x, is independent of the random error component,. For details about these two tests see, for example, Greene (1991). 8

12 tested using a specification test developed by Hausman and Taylor (1981), which is very similar to the standard Hausman specification test. Boulding and Christen (00) provide an application of this test and IV-estimation procedure in marketing. In this paper we are interested in the performance of these different estimators when the unobserved fixed effect,, is correlated with the explanatory variable, x, i.e., (4) E[x ] = x x 0 Bias, Efficiency and MSE The process of unobserved fixed effects can lead to a biased estimator, where the bias is xα α (5) β x x Ex α 0 E βˆ σ x The exact bias for each estimator depends on the data transformation (indicated in Table 1) that is required by the estimator. The first column in Table summarizes the expected bias for each estimator in terms of the basic parameters, x, x, xz, and W. ρ While the bias of βˆ 1 can be determined analytically, the exact expression of the σ. variance of the estimator βˆ 1 when E Xu 0 cannot be deduced since 1 1 E βˆ β E XX XuuX XX σ XX 1 V βˆ As a result, there is no simple analytic expression of the mean square error of each estimator, MSE ˆ E ˆ variance of β 1 V ˆ. To gain some insights, we use analytic approximations for the E ee n k ˆ based on 1 V ˆ XX, where n is the number of observations, k the number of explanatory variables, and E[ee] = E[uM 0 u] = E[M 0 ] + E[M 0 ] with M 0 = I X(XX) -1 X. With the approximation E[X(XX) -1 X] E[X]E[(XX) -1 ]E[X], we obtain. 9

13 (6) E[M 0 ] = E[] E[X(XX) -1 X] 1 n x k x. This approximation forms the basis for the results presented in columns and 3 of Table. For the feasible random-effects estimator, the analysis is complicated by the fact that the weighting factor,, must be estimated, which requires an estimate of. But since this estimate is itself biased, the variance of 1 and thus the MSE cannot be approximated. INSERT TABLE ABOUT HERE The results in Table highlight the problems of different estimators. The bias of the between-group estimator is always larger than the bias of the random-effects estimator and the OLS estimator as long as W > 0. 4 When W = 0, the three estimators are equally biased. The results also show that the variance of the fixed-effects estimator becomes very large when the fraction of within variance, W, approaches zero. The between-group estimator suffers from the same problem when W approaches 1. In contrast, the random-effects estimator is not adversely affected by extreme values of W since it uses both within and between variance components of x. Finally, the results illustrate the problem of the IV-estimator when the quality of instruments is low, i.e., as xz approaches zero. In addition to the lack of efficiency, the MSE can become very large because of a bias when z is not exactly zero. These analytic approximations allow us to compare the MSE of these estimators for different conditions. Figure 1 compares the MSE of the fixed-effects estimator with the approximated MSE of the random-effects estimator. The horizontal axis indicates the correlation x of the explanatory variable, x, with the fixed factor,, and the vertical axis the fraction of within cross-section variance, W. For a given size of cross-sections N, the solid lines show the locus of points for which the fixed-effects MSE and the random-effects MSE 4 This implies that the use of 4-year averages in initial PIMS studies potentially exacerbated the biasing effect of unobserved fixed effects. 10

14 are equal. Above the curves in Figure 1 and within the shaded area the fixed-effects estimator has a lower MSE; below the solid curves the random-effects estimator has a lower MSE. The shaded area indicates the feasible within-variance for a given value of x, where the maximum value is given by (7) W max = 1 x. INSERT FIGURE 1 ABOUT HERE Figure 1 indicates that the random-effects estimator dominates the fixed-effects estimator for all values of W, when x = 0 and the number of cross-sections, N, is reasonably large. This is the finding of Taylor (1980). The tradeoff also depends on the relative magnitudes of the component variances, and respectively. When is much smaller than, the random-effects estimator tends to be preferred even for relatively high values of x (Figure 1a). However, as increases, the fixed-effects estimator quickly starts to dominate even at relatively small values of W, unless x is small (Figure 1b). With few cross sections and <<, the random-effects estimator tends to dominate even for relatively large values of x (see Figure 1a). The situation when x approaches 1 is particularly interesting. In this case the random-effects estimator starts to dominate again because W cannot be sufficiently large due to the relationship that constrains W and x together (as per equation (7)). 5 Figure compares the fixed-effects estimator with the IV-estimator. The horizontal axis now indicates the correlation xz of the instrument, z, with the explanatory variable, x. The larger this correlation, the better is the quality of the instrument. This correlation is also constrained by the correlation x. When x approaches 1, the instrument, z, cannot, at the 5 When the error components need to be estimated, the relative performance of the fixed-effects estimator improves, but the conclusions remain the same. 11

15 same time, be highly correlated with the explanatory variable, x, yet uncorrelated with the fixed factor,. When z = 0, then the upper limit for the quality of an instrument is (8). max xz 1 x INSERT FIGURE ABOUT HERE As in Figure 1, the solid curves indicate the locus of points where the MSE of the two estimators (fixed-effects and IV) are equal. The fixed-effects estimator has a lower MSE when the fraction of within-variance, W, is high enough, i.e., above the indifference curves in Figure. The different shaded areas indicate the feasible data range for different values of x. Figure a assumes that the instruments are perfectly exogenous, i.e., z = 0. As before, fixed-effects estimation is more likely preferred the higher relative to. Figure b shows that the MSE of the IV-estimation is quite sensitive to the exogeneity assumption. Even for small correlations z of the instrument z with the fixed factors, the fixed-effects estimator dominates except for small values of W. The sample size does not have a large effect on the tradeoff between the two estimators. A larger sample size tends to make fixedeffects estimation more likely preferred. Figures 1 and indicate that fixed-effects estimation performs quite well even for relatively small values of W, which lends support to the procedure advocated by Boulding and Staelin (1995). More importantly though, they indicate that it is very difficult to obtain unbiased estimates that are also highly efficient. When x approaches 1 and thus the need is greatest for an alternative to random-effects estimation due to the large bias, the performance of fixed-effects and IV estimation is weakest because of the natural constraints placed on the highest possible fraction of within-variance, W, and the best possible quality of instruments, xz, respectively. This makes it particularly important to understand the advantages and disadvantages of these different estimation approaches in limited samples. 1

16 The comparisons presented so far are based on approximations and on knowing the true variances of the error components. Feasible versions of the random-effects and the IV estimators require an estimation of the variance components, and, respectively, which reduces the efficiency. Moreover, any estimate of is biased when x 0. For example, the typical estimator that relies on the variance from the between estimator is biased since E T. In fact, it yields an estimate of that is too small, which in turn leads B to an underestimation of, the factor used to transform the error covariance matrix to be of the form I NT (see column in Table 1). Because of these issues, we next use a Monte- Carlo simulation to test the analytic findings and gain further insights. 3. Monte-Carlo Simulation Description of Simulation We simulate observations for x it, i, it and z it and then calculate y it using equation (1). For simplicity and without loss of generality we set 0 = 1 = 1. The data generation function is controlled by two multivariate normal distributions, one for the time-fixed part and one for the time-variable part of the data, respectively. More specifically, we assume that the variables have zero mean and the following respective covariance matrices: xf xv (9) F xfzf zf and. xf 0 V xvzv zv 0 0 The fixed and time-variant distributions are independent, i.e., x = T xf + xv, z = T zf + zv, and u = T +. The ratio of variances for the fixed and time-variant components of x and z are captured by the parameters W = T xf / x and W z = T zf / z, respectively, while the three non-zero covariance terms in equations (9) are controlled by the correlation parameters x, z and xz, respectively. In addition to distributional parameters, 13

17 we also vary the size of the data set through N and T. The different simulation parameters are drawn from discrete or uniform distributions as summarized in Table 3. The last column of this Table also indicates the actual ranges in the simulated data sets. INSERT TABLE 3 ABOUT HERE The data are constructed in the following way: (1) a fixed part for each of the N firms is drawn from N[0, F ]; () T variable parts for each firm are drawn from N[0, V ]; (3) the actual variables are determined as x = x F + x V, z = z F + z V and u = + ; and (4) the dependent variable is calculated as y = 1 + x + u. We then apply the different estimation approaches summarized in Table 1 to each simulated data set and record the results as well as relevant sample and test statistics. Overall we generated more than 6,000 different datasets of which we used 3,996 for our analysis. 6 Simulation Results: Fixed-Effects vs. Random-Effects Estimation First, we examine the performance of fixed-effects and random-effects estimators. Table 4 shows the average bias and MSE results for twelve different conditions 4 levels of W and 3 levels of x. In addition to the results for the fixed-effects estimator (FE) and the randomeffects or Gauss-Markov estimator (GM) along with its feasible version (FGM), we also include the results for another version of the feasible random-effects estimator, which tries to reduce the problem of the underestimated variance,. Of course, the correlation factor, x, to fully compensate for this underestimation is unknown. However, the results from simple OLS and fixed-effects estimations can be used to obtain an estimate of the bias, B OLS, and thus x. Using as an estimate of the bias and and to calcualte the B OLS OLS W two variance components, we can recalculate the variance,, as s OLS s FE 6 We eliminated observations leading to negative values of s in equation (10), those with virtually no withinvariance (W < 0.001), and those with very poor quality of instruments (LR < 0.1). 14

18 (10) ˆ s s B. OLS FE This new estimate of can then be used to reestimate. We call the resulting estimator adjusted feasible Gauss-Markov estimator (AFGM). INSERT TABLE 4 ABOUT HERE As expected, random-effects estimation (GM, FGM, AFGM) yields a lower MSE than fixed-effects estimation (FE) when x is limited despite a small bias (see first set of rows in Table 4 where 0 < x < 0.33). These conditions essentially correspond to the case analyzed by Taylor (1980). However, even for moderately high values of x, fixed-effects estimation dominates random-effects estimation, except for small values of W. The MSE for all estimators is very high when x is large and W small (see third condition of x > 0.67 in Table 4). In this situation, it is possible that even a biased random-effects estimator can be preferred, in terms of MSE, to a fixed-effects estimator, especially when the dataset is relatively small. The effect of the size of a dataset is illustrated in Figure 3. It compares the MSE of the fixed-effects (FE) and the feasible Gauss-Markov estimator (FGM) estimator for different values of W across all values of x. Figure 3 also shows that the performance of the two estimators in terms of MSE converges as W becomes large. INSERT FIGURE 3 ABOUT HERE A small fraction of within-variance, W, not only leads to highly inefficient fixedeffects estimates, it also makes these estimates much more sensitive to empirical deviations from the assumption that the independent variable, x, is uncorrelated with the contemporaneous error term, it, i.e., x = 0 since the bias is B FE = x /W x. (This problem is equivalent to the weak instrument problem.) This effect is shown in Figure 4, which depicts the estimation bias for the fixed-effects and the Gauss-Markov estimators as a function of x for a random sample of 100 observations with W < The bias for the OLS x 15

19 Gauss-Markov estimator is shown net of the bias induced by x 0. The greater dispersion of the fixed-effects estimates reflects the inefficiency caused by the small values of W. However, this figure also indicates that the bias for the fixed-effects estimator increases faster than the bias for the Gauss-Markov estimator as x increases. The mean-difference for this random sample is 0.6 (t =.40). As discussed earlier, the need to estimate the variance component,, adds a second biasing effect to the random-effects estimators in the presence of unobserved fixed factors. The results in Table 4 show that the adjusted Gauss-Markov estimator (AFGM), which uses the recalculated variance component (equation (10)), always has a lower bias and a lower MSE than the feasible random-effects estimator (FGM) for values of x > Moreover, the performance in terms of bias and MSE is statistically identical to the Gauss-Markov estimator, which uses the true variance components, with one exception. When both x and W are small, the efficiency of the adjusted estimator suffers because of the inefficiency of the fixed-effects estimator, which is used to obtain an estimate of the bias (as per equation (10)). Simulation Results: IV-Estimation We next turn our attention to the question regarding the performance of IV-estimation (IVFGM). Table 5 presents the results for the IV-estimator for twelve different conditions. They include the same levels of x as before (see Table 4). To examine the effect of the quality of instruments, we break the data into four groups based on the likelihood ratio value, LR = -NTlog(1 - xz ), which is used to test the significance of an instrument. The previous results indicate that an alternative to fixed-effects estimation is needed most when the fraction of within-variance, W, is small. Thus, we focus our analysis on observations with W < Table 5 shows bias and MSE for the fixed-effects, feasible Gauss-Markov and IV estimators. INSERT TABLE 5 ABOUT HERE 16

20 Table 5 demonstrates that using weak instruments (when LR < 4 and thus the correlation xz is not significant at a 0.05-level), not only leads to inefficient estimates but also severely biased estimates when x is even moderately high. However, when the fraction of within-variance W is low, an instrument of even moderate quality can already improve the estimation. The simulation results also confirm the conclusion from Figure in the previous section that for values W > 0., the fixed-effects estimator is always preferred. For values 0.05 < W < 0.0, relatively high quality instruments are needed for the IVestimator to outperform the fixed-effects estimator. These findings are illustrated in Figure 5. INSERT FIGURE 5 ABOUT HERE Simulation Results: Cross-Sectional Analysis The results regarding the performance of a cross-sectional analysis and the value of additional observations for a given cross section are presented in Table 6. This Table shows the bias and MSE for a number of different estimators for the same twelve conditions of correlation x and within-variance W as Table 4. These results confirm the analytic finding that the between-group estimator (BT) performs poorly. A simple cross-sectional analysis (OLS1), which uses only T = 1 observation per cross-section, leads to similar or even better results in terms of bias and MSE. INSERT TABLE 6 ABOUT HERE A comparison of the simple OLS estimator (OLS) with the feasible Gauss-Markov estimator (FGM) shows that the two estimators yield similar bias results, but the latter is, as expected, more efficient. This difference increases as the fraction of within-variance, W, increases. When x > 0.33, the results in Table 6 show that a cross-sectional analysis (OLS1) leads to very poor MSE results (as does simple OLS and between-group estimation with the entire panel). As long as W is not too small, there is significant value in collecting a second 17

21 or third observation. Fixed-effects estimation with only T = observations (FE) removes the bias and leads to a lower MSE. Adding a third observation (FE3) further improves the performance of the fixed-effects estimator. When the data set contains only a small number of cross-sections (N < 0), either a third observation or a value of W > 0.1 is needed for fixedeffects estimation to yield a lower MSE than cross-sectional analysis. As expected, when W is small adding few observations per cross-section does not help. Finally, we show in Table 6 the performance of the IV-estimator with cross-sectional data (IV1). The results indicate that a statistically significant instrument (LR>4) yields superior estimates in the presence of large unobserved fixed factors and low values of W. However, the exogeneity assumption of the instrumental variable, i.e., z = 0, cannot be tested statistically. Simulation Results: Value of Hausman-Specification Test Given the estimation problems with small values of W, we examine to what extent the Hausman specification-test can help with the selection between fixed-effects and randomeffects estimator. The results for two different levels of significance (5% and 0%, respectively) are also shown in Table 4 (H05 and H0). In this case, the fixed-effects estimator is used whenever the test-statistic is statistically significant at the respective level of significance. The bias, as expected, is between the biases of the two underlying estimators. More interesting though is the finding that a selection between fixed and random-effects estimators based on the Hausman specification test does not yield estimates with a lower MSE than both underlying estimators. When W is small, the feasible Gauss-Markov estimator yields a lower average MSE. Conversely, when W is sufficiently large, the fixedeffect estimator yields a lower average MSE. This indicates that one can obtain an estimate that, on average, has a lower MSE simply by measuring W and using an appropriate cutoff value to decide between the two estimators. 18

22 This result is a consequence of the low power of the Hausman-specification test when W is small. The results in Table 7 show this lack of power. It indicates the fraction of observations with a significant Hausman specification test for different levels of x. The first three columns show the findings for small levels of x, the last three columns for high levels of x. The results in the first three columns of Table 7 indicate that the test yields a type-1 error consistent with the level of significance. The results in the last three columns of Table 7 show the lack of power to detect even relatively high values of x when the fraction of within-variance, W, is small. The power to detect a value of x > 0.7 for a test with a 0.05-significance level is 0.5. When reducing the level of significance to 0.0, the power of the test increases to In other words, the test accepts too many biased random-effects estimates. Conversely, the power of the test is about 0.80 or higher when the within variance is sufficiently large (W > 0.05). However, in this case the fixed-effects estimation yields results that often are at least as good in terms of MSE. INSERT TABLE 7 ABOUT HERE 4. Selection of Estimator The simulation results strongly confirm that there is no single best estimator that yields the lowest MSE across all kinds of data conditions. They further indicate that the Hausman specification test, especially in the most critical situation when x is large, is usually not very diagnostic. This raises the question about how to select among the various estimators for a given situation. Based on the analytical and simulation results described above, we develop a selection procedure in two steps that achieves good performance. In the first step, this procedure requires the following two pieces of information: 1) the fraction of within-variance, W, and ) 19

23 the likelihood-ratio test-statistic for the instrumental variable, LR. The suggested choices for different conditions are shown in Figure 6. INSERT FIGURE 6 ABOUT HERE The results in Table 4 suggest that a biased random-effects estimator (FGM) yields, on average, estimates with a lower MSE than the fixed-effects or the IV-estimator when the fraction of within-variance is very small (W < 0.05) and no significant instruments are available (LR < 4). When statistically significant instruments are available, i.e., LR > 4, it is advisable to use an instrumental variable estimator (IVFGM). When the fraction of withinvariance, W, is between 0.05 and 0.05, random-effects estimation yields about the same value of MSE than fixed-effects estimation. However, since it is potentially biased, we suggest that random-effects estimation be no longer used, even when the Hausman specification test cannot reject the hypothesis, x = 0. So the choice of estimator is between the fixed-effects estimator and the IV-estimator. As the fraction of within-variance, W, increases, instruments of higher and higher quality are needed for the IV estimator to yield estimates with lower MSE. Note that both estimators yield unbiased and consistent estimates. The cutoff values for the test statistic, LR, as determined from our simulation results, are indicated in Figure 6. When W > 0.0, fixed-effects estimation can be used regardless of the quality of an available instrument. This procedure ignores the key problem factor, the correlation x, which of course is unobserved. In a second step, using our simulated data, we calibrate a prediction model that links observable data sample information and different estimation results to the correlation x. This model can then be used to estimate x. The final model contains information about the size of the dataset, the variances of the independent variable and the error components and estimation results from simple OLS, fixed-effects and random-effects 0

24 estimation. The exact components of this model including the calibration results obtained from half of the dataset are shown in Table 8. 7 INSERT TABLE 8 ABOUT HERE The estimate of the correlation x is now used to refine the selection procedure described in Figure 6. Specifically, following the analytic results of Figure 1, we use a feasible Gauss-Markov estimator whenever the estimate is small, i.e., ˆ x < Otherwise, we follow the selection procedure based on values of W and LR as described in Figure 6. Table 9 provides summary statistics about the bias and the MSE of these two selection procedures and a number of other estimators and selection procedures for the holdout sample. The selection procedure denoted Z in Table 9 applies the procedure from Figure 6, while the selection procedure denoted Y also uses the estimation of the correlation x. To benchmark the various results, we also include the results that would be obtained if the estimator with the lowest MSE were selected in every condition (BEST and BEST, which does not include the results from simple OLS estimation). 8 The results show that (i) the two proposed selection procedures yield, on average, estimates with a significantly lower MSE than any of the estimators or procedures based on specification tests and (ii) using the estimated correlation coefficient ˆ further reduces the MSE, albeit at the cost of a somewhat higher bias. INSERT TABLE 9 ABOUT HERE x 7 We experimented with a number of model formulations, including a multinomial logit model for the choice of optimal estimator. 8 We also considered the diagnostic value of the estimated variance components, ˆ (see 10) and ˆ, the sample size, NT, the test statistics of the Hausman specification test and the Breusch-Pagan test, which tests the assumption = 0, and the R - value of the estimation. None of these factors significantly improved upon the results of the simple selection procedure. The sample size, NT, did make a difference in case of very small sample sizes, i.e., NT < 60. In an additional analysis we found that in this case, it is better to consider using fixed-effects estimation only when W >

25 To further assess the performance of the two selection procedures, we calculate the following measure for different estimators and selection procedures denoted by a subscript (11) Q MSE MSE Best, MSE H05 MSE Best where MSE Best is the lowest MSE of all estimators for a given dataset. As a benchmark we use the MSE that is achieved by using the Hausman specification test with a 5%-level of significance to decide between the fixed and the feasible Gauss-Markov estimator. A value for Q of less than 1 indicates that the respective estimator yields a lower MSE than the benchmark based on the Hausman specification test, while a value of Q higher than 1 indicates a higher MSE than the benchmark. A value of 0 indicates the best possible estimator for a particular data condition. Thus, Q allows us to compare the performance in terms of MSE across very different conditions. The results of this analysis for five different estimators and the two proposed selection procedures are shown in Table 10 for different values of within-variance, W, and the LR test statistic (quality of instruments). Again, these results are based on the holdout sample. The most important finding is that the selection procedures, denoted Z and Y, yield estimation results with lower MSE than any other estimator and selection procedure based on typical statistical tests. Importantly, using the estimate of x further reduces the MSE in virtually all conditions. INSERT TABLE 10 ABOUT HERE The first set of results for W < 0.05 illustrates the problem with fixed-effects estimation when the within-variance is very small. Even simple OLS yields on average better estimates than either fixed-effects estimation or a choice of estimator based on the Hausman specification test. The results also show the good performance of the adjusted feasible Gauss-

26 Markov estimator. 9 For larger values of within-variance, i.e., W > 0.075, the only alternative estimator that can potentially match the performance of fixed-effects estimation is IVestimation with appropriate instruments. We can conclude this section with the following points. First, the proposed selection procedure is always better than the benchmark of using either a fixed-effects or a randomeffects estimator based on Hausman test with a 5%-significance level (or other levels of significance). Second, simply using a fixed-effects estimator is preferred to using a Hausman test when W > Third, a 5%-significance level for testing the usefulness of an instrument is not restrictive enough when W > 0.10 for small datasets (NT < 00) and W > 0.05 for large datasets (NT > 400). Fourth, collecting a second data point per cross-section in order to use a fixed-effects estimator leads to a lower MSE than a purely cross-sectional analysis in the following situations: (1) for large datasets (NT > 80) when W > 0.05 and () for medium size data sets (NT > 40) when W > 0.1. For small datasets (NT < 0), a second data point is, in most cases, not sufficient. Finally, applying our proposed model to estimate the correlation coefficient x leads to estimates with lower MSE. We illustrate the usefulness of this procedure, in the context of a number of typical strategic relationships using different estimators. 5. Empirical Application We specifically address the controversial question of the impact of Market Share on Profitability, as investigated by a number of authors using the PIMS data (Boulding 1990; Buzzell and Gale 1987; Jacobson 1988; 1990a). This is a particularly relevant example because it typifies the strategic questions researched through cross-sectional databases where, in the best of cases, a short time-series is available. It is also an exemplar of the variability of 9 The selection procedures rely on the feasible Gauss-Markov estimator (FGM). Their performance could be further improved by using the adjusted feasible Gauss-Markov estimator (AFGM) instead. 3

27 the strategic variables over time; in this particular case, market share at the SBU level does not change very much from one year to the next. In fact, depending on the business sector (consumer durables, consumer non-durables, capital goods, raw material, components and supplies), the value of W varies between 0.0 and 0.09, values for which we have shown that the gains of a fixed-effects estimation in terms of Mean Squared Error is not obvious. In order to focus on firm fixed effects, we control for random processes over time. We first remove contemporaneous correlation by using lagged instruments. We also remove first-order serial correlation with the usual -differencing method. We then apply each of the methods that we compared analytically and through the simulation, i.e., OLS, fixed-effects estimator and random-effect estimator. The parameter estimates of the market share coefficients are shown in Table 11 for each business sector and when all the industries are combined. The sample sizes (combining cross-sections and time-series) and the values of W are shown also for each sector. The chi-squared value corresponds to Hausman s specification test comparing the fixed-effect vs. the random-effects models. A lack of significance of the chi-squared indicates that the random-effects model should be selected. INSERT TABLE 11 ABOUT HERE First, one notes that the estimation approach has a significant impact on the parameter estimates, as observed in prior research with, for example, for the ROI model, only one significant coefficient (at = 0.05) among the fixed-effects estimates (for consumer nondurables) and only one being insignificant among the random-effects estimates (for raw material). Although the coefficients tend to be more significant across estimation methods for the ROS model, similar results are obtained for the Net Income model (three significant coefficients for the fixed-effects model and one insignificant for the random-effects model). This clearly demonstrates that the lack of support for a market share effect can be due to the inefficiency of the estimation approach. Table 11 also shows that the specification test rejects 4