Testing for categorical moderating effects: factor scores or factor-based scores

Transcription

1 Asia Pacific Management Review (2003) 8(4), Testing for categorical moderating effects: factor scores or factor-based scores Mei-Fang Chen and Cherng G. Ding ( received April 2003; revision received June 2003; accepted August 2003 ) To test for categorical moderating effects on the relationships between constructs, the moderated regression analysis (MRA) can be used, but the adequacy of measurement needs to be ensured. The factor-based scores are often used as input for MRA for the sake of simplicity. However, the corresponding power may be insufficient. The factor scores derived from CFA instead of factor-based scores are recommended. Power comparisons between the scoring appr - oaches were made by using Monte Carlo simulation for dichotomous and trichotomous moderators. The results indicated that the factor scores derived from CFA can lead to substantially higher power, and therefore are recommended as input for MRA. Keywords: Factor-based scores; Moderators; Factor scores derived from CFA; MRA; Monte Carlo simulation; Power 1. Introduction In behavioral studies, socioeconomic variables are often introduced as moderators. Moderated regression analysis (MRA) has long been applied for evaluating the moderating effects. When the MRA approach is applied to examine moderating effects on the relationships between unobservable latent constructs, these constructs need to be first scored before moderating effects are examined. However, such a procedure is inherently indeterminate [17]. The power issue of the MRA approach for detecting moderating effects on the construct relationship is rarely addressed in the management literature. It is not uncommon to find insufficient statistical power or not to refer to the issue of statistical power in applied psychology and management research [23]. Insufficient statistical power often leads to false decision. To examine the moderating effects on the relationship between latent constructs, many researchers used to adopt factor-based scores to perform MRA for the sake of simplicity. However, the corresponding power may be insufficient. Since the use of factor scores derived from confirmatory factor analysis (CFA) Corresponding author. Institute of Business and Management, National Chiao Tung University, and Department of Business Management, Tatung University, 40 Chung-Shan N. Rd, Section 3, Taipei, Taiwan; mfchen@ttu.edu.tw. Institute of Business and Management, National Chiao Tung University, 4F, 114 Chung Hsiao W. Rd., Section 1, Taipei, Taiwan; cding@mail.nctu.edu.tw. 483

2 could reduce the distorting effect from the measurement errors on coefficient estimates [4, 9], they may lead to higher power of the test for the moderating effects. The purpose of this article is to make power comparisons between the scoring approaches (factor-based scores versus factor scores derived from CFA) in MRA by using Monte Carlo simulation for dichotomous and trichotomous moderators, and then provide suggestions, based on our findings, for empirical researchers. 2. Literature Review 2.1 Moderated Regression Analysis Moderators often appear in behavioral science research. It affects the form or strength of a relationship between an independent variable and a dependent variable [7, 32]. In other words, the independent variable and the moderator interact to reflect that the effect of the independent variable on the dependent variable depends on the level of the moderator [21]. Traditionally, two approaches are available to test for moderating effects. One is moderated regression analysis (MRA), and the other is subgroup analysis [27]. MRA, developed by Saunders [30], has been increasingly applied and is widely recognized valid in search of moderator variables. In MRA, we test for the product terms to determine whether moderating effects are statistically significant. For example, for a simple MRA model with potential moderator Z, Y = ß 0 + ß 1 X + ß 2 Z + ß 3 XZ +, the null hypothesis devoid of the moderating effects is H 0 : ß 3 = 0. If this hypothesis is rejected, then the estimated value of ß 3 provides the information concerning the moderating nature. For potential categorical moderators, we introduce k 1 dummy variables for a moderator with k levels and their product terms with predictors into the model, and then test for the product terms to examine moderating effects. This approach is easy to perform and interpret. In behavioral studies, independent and dependent variables are sometimes constructs such as satisfaction, attitude, and so on, which can not be directly measured. CFA, a case of structural equation model (SEM), is often used to deal with the measurement problems of constructs. Before the MRA approach can be applied to examine moderating effects for structural models with latent constructs, the constructs need to be scored in advance [e.g., 5, 33, 34, 35]. The scoring results are inherently indeterminate [17]. 2.2 Factor Score Indeterminacy Latent factors are often scored before conducting subsequent analysis [e.g., 14, 16, 18, 33, 34, 35]. However, factor scores are known to be charac- 484

3 terized by indeterminacy [17], which means that, for the same observed data, different scoring approaches could lead to different factor scores. Some researchers use factor scores obtained by the regression method. Other methods of factor score estimation are also available [22, 29]. However, different methods produce highly correlated estimates. It should be noted that the scores for latent constructs cannot totally remove measurement errors [9]. From the simulation results of Acito and Anderson [1], we learn that the indeterminacy depends on the sizes of the factor loadings. If the measures with strong relationships to the latent constructs are selected, factor score indeterminacy can be reduced. Furthermore, using more indicators per factor will decrease indeterminacy, though the degree of improvement is not as great as that from increasing factor loadings. The method used to extract the initial factors is found to have no significant effect on the accuracy of factor scores. With the factor-based scoring approach, two or more indicators measuring a common underlying construct are summed up and divided by the number of the items. To test for moderating effects on the relationships between constructs, MRA is usually used, with factor-based scores as input after ensuring that the measurement for the constructs is adequate [e.g., 5, 8, 25, 34, 35]. Since the use of factor scores derived from CFA could reduce the distorting effects from the measurement errors on coefficient estimates [4, 9], some researchers have used factor scores for subsequent analyses [e.g., 13, 28, 31]. However, how the factor scoring approaches influence the power of a test for moderating effects is rarely seen in the literature. 2.3 Statistical Power Power analysis is used for assessing a statistical test. Statistical power comprises three co-determinants [10]: the significance level α, which is the research s long-term probability of erroneously rejecting the null hypotheses, the sample size, and the effect size, which reflects the magnitude of a phenomenon under the alternative hypothesis. Insufficient statistical power is not an uncommon finding across diverse fields of the social sciences [23]. The decision quality will be improved when statistical power is increased. For the simple MRA model mentioned earlier, Y = ß 0 + ß 1 X + ß 2 Z + ß 3 XZ + we test H 0 : ß 3 = 0 versus H 1 : ß 3 0 for examining the moderating effects. Power analysis requires the evaluations of Pr (reject H 0 H 0 ) and Pr (reject H 0 H 1 ). Despite its popularity, as 485

4 numerous researchers have noted, MRA has insufficient statistical power in detecting the hypothesized moderating effects [2, 11, 12, 24] and is therefore inappropriate for conventional moderator analyses [26]. Furthermore, little research has addressed the power analysis of MRA when it is applied to test for the moderating effects on the relationships between constructs. The purpose of this study is to investigate, by using Monte Carlo simulation, if factor scores derived from CFA can lead to higher power in testing for moderating effects on the relationships between constructs than factor-based scores. 3. Methodology Anderson and Gerbing [6] recommended that the overall measurement should be examined before further analysis is performed. CFA is often used to make sure that the measurement model is adequate. To facilitate comparison, we consider the case with two exogenous latent constructs F 1, F 2, one endogenous latent construct F 3, and one qualitative variable M 1 (dichotomous) or M 2 (trichotomous) moderating the relationship between F 2 and F 3. Each latent construct was measured by three indicators separately, with X 1 ~ X 3 to measure F 1, X 4 ~ X 6 to measure F 2, and Y 1 ~ Y 3 to measure F 3 (see Figure 1). X 1 X 2 F 1 X 3 F 3 Y 1 Y 2 X 4 F 2 Y 3 X 5 X 6 M 1 or M 2 Figure 1 The Path Diagram Showing the Moderating Effect of One Qualitative 486

5 Variable M 1 (dichotomous) or M 2 (trichotomous) on the relationship between exogenous construct F 2 and endogenous construct F 3, controlling for another exogenous construct F 1. Dummy variable D was introduced to represent M 1 ; that is, D = 0 for the first level of M 1 and D = 1 for the second level. Dummy variables K 1 and K 2 were introduced to represent M 2 ; that is, K 1 = 0, K 2 = 0 for the first level of M 2, K 1 = 1, K 2 = 0 for the second level of M 2, and K 1 = 0, K 2 = 1 for the third. Case 1: Considering dichotomous moderator M 1. The MRA model is as follows: F 3 = ß 0 + ß 1 F 1 + ß 2 F 2 + ß 3 D + ß 4 D*F 2 + It follows that, controlling for F 1, (a) for D = 0, E (F 3 ) = ß 0 + ß 1 F 1 + ß 2 F 2 ; (b) for D = 1, E (F 3 ) = (ß 0 + ß 3 ) + ß 1 F 1 + (ß 2 + ß 4 ) F 2. If ß 4 = 0, then the effects of F 2 on F 3 are identical for both levels of M 1, indicating that there are no moderating effects. Therefore, moderating effects can be examined by testing H 01 : ß 4 = 0. The t-test (t = estimate / standard error of the estimate) can be used. Case 2: Considering trichotomous moderator M 2. The MRA model is as follows: F 3 = ß 0 + ß 1 F 1 + ß 2 F 2 + ß 3 K 1 + ß 4 K 2 + ß 5 K 1 *F 2 + ß 6 K 2 *F 2 + It follows that, controlling for F 1, (a) for K 1 = 0, K 2 = 0, E (F 3 ) = ß 0 + ß 1 F 1 + ß 2 F 2 ; (b) for K 1 = 1, K 2 = 0, E (F 3 ) = (ß 0 + ß 3 ) + ß 1 F 1 + (ß 2 + ß 5 ) F 2 ; (c) for K 1 = 0, K 2 = 1, E (F 3 ) = (ß 0 + ß 4 ) + ß 1 F 1 + (ß 2 + ß 6 ) F 2. To investigate whether M 2 moderates the relationship between F 2 and F 3, we simply test H 02 : ß 5 = 0, ß 6 = 0 (The regression coefficients associated with F 2 for each group of M 2 are equal). The above joint test can be conducted by using the well-known F-test. If H 02 is rejected, then further investigation is needed to ascertain the nature of moderating effects. 487

6 3.1 Monte Carlo Simulation Design In this study, attempts were made to compare the power of the tests for categorical moderating effects on the relationships between constructs based on different factor scoring approaches. Aguinis and Stone-Romero [3] performed Monte Carlo simulations to examine the degree to which the statistical power of MRA, whose function is to detect the effects of a dichotomous moderator variable, was affected by the main and interaction effects of: (1) predictor variable range restrictions, (2) total sample size, (3) sample sizes for 2 moderator variable-based subgroups, (4) predictor variable intercorrelation, and (5) magnitude of the moderating effect. Considering the similar factors mentioned above, we conducted Monte Carlo simulation with SAS to compare the difference of power stemming from factor scores and factorbased scores. Normally distributed random numbers were generated using the RANNOR function, with a randomly selected seed value used for each sample generated. The simulation study took into account the following factors, presumed to influence power: 1. Total sample size: To meet the requirement of CFA, the sample size should be 150 or more. Larger samples are always preferable [19]. The samples of 200 and 300 are used. 2. Proportion of each subgroup based on the moderator s level: For a dichotomous moderator variable, the total number of observations was randomly assigned to each of the moderator subgroups. The proportion of observations in subgroup 1 for case 1 was set at values of 0.3 and 0.5 so as to represent the reasonable range of proportions for social science studies. For a trichotomous moderating variable in case 2, we divided the total number of observations equally for convenience. 3. Correlation between predictor variables: Since the study include- ed two exogenous constructs with multiple-indicators, we set the correlation at 0.2 to reflect the practical phenomena. High correlation was not used in an attempt to avoid the lack of discriminant validity. 4. Factor loadings: To meet the requirement of reliability, the factor loadings should be higher than 0.7. This study set the factor loadings at 0.7, 0.8, and 0.9 for the three indicators of each latent construct. 5. Regression coefficients: Without loss of generality, either 0.5 or 1.0 was set as the value for the regression coefficients [20]. In case 1, there 488

7 were 8 combinations for F 1, F 2 and D. As can be seen later form the results for case 1, the magnitude of the regression coefficient associated with the dummy variable showed little influence on the power. Therefore, in case 2, the value setup was for the regression coefficients associated with F 1 and F 2 only. There were four combinations. 6. Magnitude of moderating effects: We started the simulation from no moderating effects (where the moderating effect size, denoted by ES, was 0 and the corresponding power was the significance level α) till the effect size with power close to 1. It follows that, for case 1, powers of the tests based on factor scores and factor-based scores were estimated and compared for (2 2 8) cells. For case 2, there were 4 cells only. The simulation process for assessing the power, for each cell, is shown by the following steps (referring to Figure 1): 1. Generating data 1 A value of F 1 was randomly generated from the standard normal distribution (denoted by N (0,1)), and the values of X 1 ~ X 3 were gen- erated using the factor analysis model. Under the standardized situation, 1 = Var (X i ) = communality + specific variance (communality = loading 2 ). Given the factor loadings of 0.9, 0.8, and 0.7, the values of X 1 ~ X 3 were generated as follows: X 1 = 0.9 * F * Z 1 X 2 = 0.8 * F * Z 2 X 3 = 0.7 * F * Z 3 where Z 1, Z 2, Z 3 were independent N (0,1) random variates. 2 Although the exogenous constructs are allowed to be correlated, high correlation will damage the discriminant validity. As a result, the correlation coefficient between F 1 and F 2 was set as 0.2. A value of F 2 could then be generated as follows [15]: F 2 = 0.2 * F 1 + SQRT (1 (0.2) 2 ) * Z where Z was an independent N (0,1) random variate. F 2 was also an N (0,1) random variate. Values of X 4 ~ X 6 were generated through F 2 as follows: 489

8 X 4 = 0.9 * F * Z 4 X 5 = 0.8 * F * Z 5 X 6 = 0.7 * F * Z 6 where Z 4, Z 5, Z 6 were independent N (0,1) random variates. 3 A value of F 3 was obtained through the regression model given below: (The regression coefficients combinations have been mentioned earlier) F 3 = ß 1 F 1 + ß 2 F 2 + ß 3 D + ß 4 D*F 2 4 Values of Y 1 ~ Y 3 were generated through F 3 as follows: Y 1 = 0.9 * F * Z 7 Y 2 = 0.8 * F * Z 8 Y 3 = 0.7 * F * Z 9 where Z 7, Z 8, Z 9 were independent N (0,1) random variates. Repeat Step (1) through Step (4) n times to obtain n observations for X 1 ~ X 6 and Y 1 ~ Y 3. For case 1, n = 200 and 300; for case 2, n = Estimating scores for latent constructs 1 Factor-based scores: the factor-based scores for a particular construct were obtained by summing up its corresponding indicators and dividing the sums by the number of indicators (3). 2 Factor scores derived from CFA: factor scores are computed based on the regression approach. They are easy to obtain by the PLATCOV command under PROC CALIS in SAS. 3. Performing MRA Factor-based scores and factor scores derived from CFA were put in MRA separately to carry out the moderating effect test. Under the 0.05 significance, we test H 01 : ß 4 = 0 (Case 1) with the t-test, and test H 02 : ß 5 = 0, ß 6 = 0 (Case 2) with the F-test. 4. Repeating Step 1 through Step times The power is assessed by computing Pˆ r (reject H 0 ), which equals the number of rejections / Pˆ r (reject H 0 H 0 ) should be close to α (significance level); the higher the Pˆ r (reject H 0 H 1 ), the better. 490

9 Table 1. Comparisons of the Power of the Tests for Moderating Effects Between Factor-Based Scores and Factor Scores for Case 1 1. F3=0.5*F1+0.5*F2+0.5*D+ES*D*F2+ ; 2. F3=0.5*F1+0.5*F2+1.0*D+ES*D*F2+ ; Sample Size Proportio n ES FB FS FB FS FB FS FB FS FB FS FB FS FB FS FB FS % 5.02% 5.44% 5.82% 5.00% 5.02% 5.54% 6.14% 5.18% 5.54% 5.58% 5.84% 5.12% 5.14% 5.10% 5.32% % 29.94% 19.84% 24.44% 32.88% 42.08% 28.88% 34.48% 23.58% 29.42% 20.20% 24.88% 32.96% 41.30% 29.02% 36.12% % 54.34% 37.40% 46.28% 62.22% 73.68% 52.28% 64.20% 44.04% 54.22% 37.16% 46.44% 61.54% 73.96% 52.86% 63.80% % 78.22% 57.96% 68.64% 83.82% 91.88% 76.72% 85.26% 67.60% 78.92% 57.78% 70.66% 85.20% 92.74% 76.70% 86.02% % 91.98% 76.60% 85.64% 94.78% 98.44% 91.04% 96.34% 85.18% 92.70% 76.26% 86.54% 95.22% 98.74% 90.94% 96.32% % 97.64% 87.08% 93.18% 99.32% 99.88% 97.20% 99.26% 93.58% 97.82% 88.90% 94.62% 99.14% 99.80% 97.32% 99.28% % 99.94% 98.22% 99.40% % % 99.88% 99.98% 99.50% % 98.14% 99.70% % % 99.94% % 3. F3=0.5*F1+1.0*F2+0.5*D+ES*D*F2+ ; 4. F3=1.0*F1+0.5*F2+0.5*D+ES*D*F2+ ; Sample Size Proportio n ES FB FS FB FS FB FS FB FS FB FS FB FS FB FS FB FS % 5.22% 5.28% 5.58% 5.38% 5.38% 4.50% 5.08% 5.50% 5.38% 5.16% 5.12% 4.94% 4.86% 5.50% 5.32% % 20.94% 15.26% 19.08% 22.78% 29.38% 19.88% 25.20% 18.64% 20.64% 15.50% 18.74% 23.58% 29.32% 21.16% 24.64% % 40.52% 27.46% 33.48% 45.76% 55.98% 39.48% 47.80% 32.62% 39.62% 29.06% 35.54% 46.24% 55.72% 40.04% 47.90% % 61.52% 42.14% 52.12% 68.42% 80.20% 58.98% 71.14% 51.62% 61.64% 45.00% 53.16% 69.64% 78.98% 61.96% 71.98% % 80.50% 57.84% 68.96% 84.88% 93.26% 78.32% 87.40% 70.44% 79.44% 61.66% 71.28% 85.88% 92.94% 80.36% 87.68% % 91.04% 73.30% 84.42% 95.02% 98.66% 89.22% 95.22% 84.32% 91.44% 76.36% 83.70% 95.54% 98.46% 91.08% 95.48% % 98.98% 92.12% 95.52% 99.44% 99.94% 98.78% 99.48% 97.28% 98.98% 93.80% 97.06% 99.62% 99.96% 98.98% 99.52% 5. F3=1.0*F1+1.0*F2+0.5*D+ES*D*F2 + ; 6. F3=0.5*F1+1.0*F2+1.0*D+ES*D*F2; + ; Sample Size Proportio n ES FB FS FB FS FB FS FB FS FB FS FB FS FB FS FB FS % 5.04% 4.72% 4.84% 4.96% 4.96% 4.80% 4.92% 5.46% 5.06% 5.18% 4.74% 4.74% 4.96% 5.08% 4.58% % 18.64% 13.36% 15.88% 19.28% 23.46% 17.44% 20.00% 16.48% 21.26% 14.94% 18.82% 23.84% 30.92% 20.64% 27.28% % 32.66% 22.12% 27.18% 36.98% 45.46% 31.32% 38.06% 33.32% 42.60% 26.42% 35.10% 45.28% 57.26% 39.12% 49.98% % 52.06% 34.62% 42.24% 57.42% 68.26% 50.78% 60.08% 50.92% 63.90% 44.10% 56.16% 68.44% 80.94% 58.48% 71.80% % 69.36% 50.18% 60.28% 74.92% 85.58% 66.92% 77.10% 66.92% 80.40% 59.02% 71.74% 84.90% 94.12% 77.30% 88.22% % 83.08% 63.90% 72.72% 88.30% 94.68% 81.98% 89.48% 83.56% 92.88% 73.50% 84.84% 94.82% 98.74% 89.08% 95.42% % 96.00% 86.08% 92.20% 98.46% 99.56% 96.16% 98.26% 95.82% 99.16% 92.22% 97.30% 99.62% 99.94% 98.86% 99.82% 7. F3=1.0*F1+0.5*F2+1.0*D+ES*D*F2+ ; 8. F3=1.0*F1+1.0*F2+1.0*D+ES*D*F2; + ; Sample Size Proportio n ES FB FS FB FS FB FS FB FS FB FS FB FS FB FS FB FS % 5.34% 5.70% 5.92% 5.36% 4.64% 5.28% 5.56% 4.80% 5.10% 4.86% 4.56% 5.12% 4.74% 5.08% 4.66% % 19.42% 16.12% 19.18% 24.50% 29.70% 21.02% 25.00% 14.54% 17.54% 12.90% 16.04% 19.22% 24.30% 15.68% 20.32% % 40.64% 27.94% 34.12% 46.46% 57.58% 39.98% 47.86% 26.24% 32.56% 22.84% 28.68% 37.38% 46.66% 31.66% 40.48% % 62.12% 44.98% 53.38% 68.78% 79.58% 60.14% 70.62% 42.32% 51.76% 35.78% 45.56% 59.34% 70.28% 48.96% 60.70% % 81.20% 61.04% 71.68% 86.02% 93.00% 79.68% 88.40% 57.40% 70.04% 48.28% 60.46% 75.94% 87.32% 66.44% 78.24% % 92.10% 76.36% 85.78% 95.46% 98.80% 92.02% 96.52% 71.36% 83.10% 62.54% 73.86% 89.08% 95.42% 81.62% 90.58% % 99.14% 93.58% 97.42% 99.76% 99.98% 99.04% 99.72% 91.34% 97.02% 86.28% 93.02% 98.46% 99.62% 95.66% 98.82% % 99.44% 95.38% 97.92% 99.84% % 99.32% 99.90% Notes: ES: the moderating effect size β 4 ; FB: factor-based scores; FS: factor scores derived from CFA 4. Results and Discussion The results from the Monte Carlo simulation for case 1 and case 2 are shown in Table 1 and Table 2, respectively. Table 1 reveals that the power increases with the increase of the sample size, the dichotomous proportion, and the moderating effect size, consistent with our expectation by statistical theory. In Table 1 we find that the statistical power at ES = 0 (where there is 491

10 no moderating effect on the relationships between constructs) is approximately equal to α = 0.05 for all combinations. It means that the probabilities of making Type I error from these two factor scores do not differ. However, the power under the alternative hypothesis based on factor scores derived from CFA outperforms that based on factor-based scores, especially when the effect size ranges from 0.1 to 0.3. In Table 2, the power is about 0.05 at ß 5 (ES1) = ß 6 (ES2) = 0. When ß 5 is fixed, power is an increasing function of Table 2. Comparisons of the Power Between the Two Scoring Approaches for Case 2 1 a 2 b 3 c 4 d ES1 ES2 FB FS FB FS FB FS FB FS (ES2) (ES1) % 4.78% 4.76% 4.58% 5.16% 5.24% 5.34% 4.96% % 19.70% 12.48% 14.69% 12.47% 14.89% 10.68% 11.31% % 29.88% 17.68% 22.98% 19.18% 22.58% 14.22% 17.38% % 48.82% 27.80% 35.52% 28.96% 34.54% 22.02% 27.94% % 70.36% 41.68% 51.88% 43.42% 52.86% 33.02% 40.34% % 77.42% 57.10% 69.24% 58.62% 69.48% 46.38% 57.44% % 94.44% 68.24% 81.44% 73.32% 83.24% 59.58% 70.70% % 98.52% 81.08% 92.10% 83.50% 91.60% 69.84% 82.28% % 98.72% 89.20% 96.08% % 63.96% 36.59% 46.55% 37.31% 45.44% 28.51% 35.39% % 73.34% 44.86% 55.38% 46.32% 54.96% 35.42% 43.42% % 83.86% 56.20% 68.30% 57.16% 68.16% 46.30% 55.92% % 92.32% 68.18% 80.46% 70.14% 79.50% 56.36% 66.34% % 97.14% 77.86% 88.30% 80.38% 89.16% 66.88% 77.74% % 97.88% 93.48% 97.60% 85.12% 93.16% % 92.37% 67.13% 79.40% 70.58% 80.09% 56.62% 67.60% % 98.10% 81.30% 89.86% 82.64% 90.24% 71.46% 80.78% % 97.24% 94.12% 97.90% 85.46% 92.38% % 99.44% 89.08% 96.30% 91.98% 96.26% 80.74% 90.76% Notes: ES1: the moderating effect size β 5 ; ES2: the moderating effect size β 6 ; FB: factor-based scores. FS: factor scores derived from CFA a F 3 = 0.5*F *F *K *K 2 + ES 1 *K 1 *F 2 + ES 2 *K 2 *F 2 + ; b F 3 = 0.5*F *F *K *K 2 + ES 1 *K 1 *F 2 + ES 2 *K 2 *F 2 + ; c F 3 = 1.0*F *F *K *K 2 + ES 1 *K 1 *F 2 + ES 2 *K 2 *F 2 + ; d F 3 = 1.0*F *F *K *K 2 + ES 1 *K 1 *F 2 + ES 2 *K 2 *F 2 + ; 492

11 ß 6 ; when ß 6 is fixed, power is an increasing function of ß 5. Power resulting from factor scores performs better for all combinations. When the effect size reaches 0.4 or 0.5 in both tables, the power based on the two approaches come close to equal because the effect size is too huge to make a difference. The results indicated that the traditional MRA, which adopts factorbased scores, can hardly detect trivial effect sizes. The MRA espousing factor scores derived from CFA shows significant improvement on examining trivial effect sizes. In addition, factor scores could reach the same power level as factor-based scores with a smaller sample size. Thus, the factor scores can be more effective. This finding does improve the quality of decision-making. Regarding the influence of the factor loadings, although not shown in the tables, we have additionally conducted the power comparison for several different sets of factor loadings such as (0.9, 0.8, 0.8), (0.6, 0.7, 0.8), and (0.9, 0.6, 0.5), all leading to the Cronbach α values at least 70%, as well as their cross setups for the constructs, and obtained the same conclusion as above. Moreover, if each of the indicators of the underlying construct has the same factor loading (such as (0.8, 0.8, 0.8)), then the power based on the two scoring approaches is almost the same. The reason is that, identical factor loading will lead to approximately equal scoring coefficients for the indicators and for factor-based scores, the indicators are assigned equal weights (1/3) for measuring their corresponding constructs. It appears that factorbased scores are actually a special case of scoring coefficients. For the situations of different factor loadings, as usually seen in empirical studies, the indicator with the highest factor loading will dominate (having the greatest weight) the evaluation of factor scores. The corresponding power is substantially higher. Although the idea seems simple and obvious, it has not received much attention in empirical research. In this study, it has been shown that the power based on factor scores derived from CFA does outperform that based on factor-based scores, and the magnitudes of power differences have been specifically evaluated. 5. Conclusion In behavioral science studies, latent constructs appear frequently. Socioeconomic variables are often introduced as categorical moderators. Researchers, for convenience, use the MRA approach to investigate the moderating effects on the relationships between constructs after ensuring that the measurement part is adequate. However, the factor scores estimation procedure is inherently indeterminate, which may influence the power to detect moder- 493

12 ating effects. In this study, we have used Monte Carlo simulation to compare the power of tests for moderating effects on the relationships between constructs based on different scoring approaches. The statistical power shows no difference between the two scoring approaches when there is no moderating effect (both of the power are close to significance level α). However, when there is a moderating effect, the power based on factor scores is substantially higher than that based on factor-based scores. The finding is true for all designed situations. According to the simulation results, we strongly suggest that researchers use factor scores derived from CFA instead of factor-based scores as input for MRA to test for moderating effects on the relationships between constructs. Acknowledgements The authors thank two anonymous referees for helpful comments. This work was partially supported by a grant from the National Science Council, Republic of China (NSC H ). References [1]. Acito, F., R. D. Anderson A simulation study of factor score indeterminacy. Journal of Marketing Research [2]. Aguinis, H Statistical power problems with moderated multiple regression in management research. Journal of Management [3]., E. F. Stone-Romero Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology [4]. Allison, P Survival Analysis Using the SAS System, SAS Institute Inc., Cary, NC. [5]. Andaleeb, S. S., A. K. Basu Technical complexity and consumer knowledge as moderators of service quality evaluation in the automobile service industry. Journal of Retailing [6]. Anderson, J. C., D. W. Gerbing Structural equation modeling in practice: A review and recommended two-step approach. Psychological Bulletin [7]. Baron, R. M., D. A. Kenny The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical consideration. Journal of Personality and Social Psychology [8]. Barrett, H., J. L. Balloun, A. Weinstein Marketing mix factors 494

13 as moderators of the corporate entrepreneurship-business performance relationship- a multistage, multivariate analysis. Journal of Marketing Theory and Practice [9]. Bollen, K Structural Equations with Latent Variables, Wiley, New York. [10]. Cohen, J Statistical Power for the Behavioral Sciences (2 nd ed.), Lawrence Erlbaum, Hillsdale, NJ. [11]., P. Cohen Applied Multiple Regression/Correlation Analysis for the Behavior Sciences (2 nd ed), Lawrence Erlbaum, Hillsdale, NJ. [12]. Evans, M. G A Monte Carlo study of the effects of correlated method variance in moderated multiple regression analysis. Organizational Behavior and Human Decision Processes [13]. Hom, P. W., A. J. Kinicki Toward a greater understanding of how dissatisfaction drives employee turnover. Academy of Management Journal [14]. Kamaruddin, A. R., S. Mokhlis, M. N. Othman Ethnocentrism orientation and choice decisions of Malaysian consumers: the effects of socio-cultural and demographic factors. Asia Pacific Management Review [15]. Kuan, C.M Statistics: Concepts and Methods, Hwatai, Taipei, Taiwan. [16]. Kuei, C. H., C. N. Madu Identifying critical success factors for supply chain quality management (SCQM). Asia Pacific Management Review [17]. Lawley, D. N., A. E. Maxwell Factor Analysis as a Statistical Method. American Elsevier, New York. [18]. Lin, B. W., C. J. Chen The effects of formation motives and interfirm diversity on the performance of strategic alliance. Asia Pacific Management Review [19]. MacCallum, R. C., M. Roznowski, L. B. Necowitz Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin [20]., C. M. Mar Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin [21]. McClelland, G. H., C. M. Judd Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin [22]. McDonald, R. P., E. J. A. Burr A comparison of four methods of 495

14 constructing factor scores. Psychometrika [23]. Mone, M. A., G. C. Mueller, W. Mauland The perceptions and usage of statistical power in applied psychology and management research. Personnel Psychology [24]. Morris, J. H., J. Sherman, E. R. Mansfield Failures to detect moderating effects with ordinary least squares moderated-regression: Some reasons and a remedy. Psychological Bulletin [25]. Patterson, P. G A contingency approach to modeling satisfaction with management consulting services. Journal of Service Research [26]. Paunonen, S. V., D. N. Jackson Type I error rates for moderated multiple regression analysis. Journal of Applied Psychology [27]. Peters, W. S., J. E. Champoux The role and analysis of moderator variables in organizational research, in: R. T. Mowday & R. M. Steers (Ed.). Research in Organizations Issues and Controversies, Goodyear Publishing Company, Inc., Santa Monica, CA, [28]. Romney, D. M., C. D. Jenkins, J. M. Bynner A structural analysis of health-related quality of life dimensions. Human Relations [29]. Saris, W. E., W. M. de Pijper, J. Mulder Optimal procedures for estimation of factor scores. Sociological Methods and Research [30]. Saunders, D. R Moderator variables in prediction. Educational and Psychological Measurement [31]. Shane, S., L. Kolvereid National environment, strategy, and new venture performance: a three country study. Journal of Small Business Management [32]. Sharma, S., R. D. Durand, O. Gur-Arie Identification and analysis of moderator variables. Journal of Marketing Research [33]. Troy, L. C., D. M. Szymanski, P. R. Varadarajan Generating new product ideas: an initial investigation of the role of market information and organizational characteristics. Journal of the Academy of Marketing Science [34]. Waldman, D. A., G. G. Ramirez, R. J. House, P. Puranam Does leadership matter? CEO leadership attributes and profitability under conditions of perceived environmental uncertainty. Academy of Management Journal [35]. Witt, L. A., M. C. Andrews, K. M. Kacmar The role of particip- 496

15 ation in decision-making in the organizational politics-job satisfaction relationship. Human Relations Y 1 Y 2 Y 3 497