Estimating and Testing Cross-Sectional Asset Pricing Models: A Robust IV Econometric Technique

Transcription

1 Estmatng and Testng Cross-Sectonal Asset Prcng Models: A Robust IV Econometrc Technque December 17, 29 (Prelmnary) Abstract In ths paper, we ntroduce a new technque for estmatng and testng crosssectonal asset prcng models that s desgned to deal wth weak dentfcaton, whch can arse due to model msspecfcaton, small betas on putatve rsk factors, or a varety of other problems. We conduct smulatons to assess the mportance of weak dentfcaton for emprcal work on cross-sectonal asset prcng models and to compare the propertes of the robust IV technque wth other technques, such as Fama-MacBeth. Our smulatons are based on two popular asset prcng models one uncondtonal (Fama- French three-factor) and one condtonal (Lettau-Ludvgson CCAPM). When weak dentfcaton s not a problem, both the Fama-MacBeth-based procedures and robust IV have good sze propertes. In the smulatons, the robust IV procedure has better power n tests of some ndvdual parameters and better power than the Shanken F test to reject a msspecfed model. In smulatons of a msspecfed emprcal model, Fama-MacBeth has substantal sze dstortons. Robust IV does better. Kan and Zhang (1999) show that there can be mportant problems wth nference when betas on a factor are small. Followng Klebergen (29), we fnd small betas n the Lettau-Ludvgson model. Smulatons wth small betas show mportant sze dstortons wth Fama-MacBeth but not wth robust IV. Robust IV has two further, useful propertes. Frst, t can provde a warnng when weak dentfcaton may affect the sze propertes of other estmators. Second, model rejectons under robust IV can provde nformaton that may be useful n mprovng a gven asset prcng model;.e., model rejectons can be nformatve. The paper also ncludes emprcal applcatons that further llustrate the propertes of the estmators. Keywords: asset prcng models, asset prcng tests, msspecfcaton, weak dentfcaton, CAPM, Fama-French three-factor model, Lettau-Ludvgson model JEL codes: G12, G11, C12, C13 Lynda Khalaf (Department of Economcs, Carleton Unversty, 1125 Colonel By Dr., Ottawa, Ontaro, Canada K1S 5B6; (613) x 8697; Lynda_Khalaf@carleton.ca) Huntley Schaller (Department of Economcs, Carleton Unversty, 1125 Colonel By Dr., Ottawa, Ontaro, Canada K1S 5B6; (613) , schaller@ccs.carleton.ca) We would lke to thank Raymond Kan for provdng software for the Kan, Robott and Shanken (29) procedures, Martn Lettau for provdng the Lettau and Ludvgson (21) data, Mark Blanchette for outstandng research assstance, and the Canada Research Char Program, the Insttut de Fnance Mathématque de Montréal (IFM2), the Canadan Network of Centres of Excellence (program on Mathematcs of Informaton Technology and Complex Systems [MITACS]), SSHRC, and the Fonds de Recherche sur la Socété et la Culture (Québec) for fnancal support.

2 1. Introducton In ths paper, we ntroduce a new technque for estmatng and testng crosssectonal asset prcng models, whch s desgned to address problems of weak dentfcaton. Examples of how weak dentfcaton can arse nclude msspecfed emprcal models and the problem of small betas orgnally analyzed by Kan and Zhang (1999). 1 Our technque comes out of a broader lterature n econometrcs, from whch a number of lessons have emerged. Frst, standard asymptotc results often provde poor approxmatons to the dstrbutons of estmators and test statstcs. The actual sze of a test (probablty of rejectng a true model) can be dramatcally dfferent than the nomnal sze (e.g.,.5). The power of a test can be low, so that the test fals to reject a false parameter value or a false model. Both the sze and power problems can be serous even when the sample sze s large. Second, standard correcton technques ncludng many bootstraps may fal. 2 The standard methodology n emprcal tests of cross-sectonal asset prcng models s based on a two-pass approach. The frst step s to estmate the betas. The second step s to estmate a cross-sectonal regresson of expected returns on the betas. The coeffcents on the betas are the estmated rsk prces assocated wth the factors. Weak dentfcaton has the potental to create serous problems for tests of asset prcng models. The broader econometrc lterature has shown that many estmaton technques wll yeld pont estmates wth seemngly tght standard errors, even when the data contan lttle nformaton about the true parameter. The usual confdence ntervals may be tghtly centered on the wrong value of the parameter. Weak dentfcaton can therefore lead to poor coverage. 3 1 Measurement error n the betas s another potental source of weak dentfcaton. For a concse revew of the errors-n-varables problem wth estmated betas, see secton 5.8 of Campbell, Lo, and MacKnlay (1997). Ltzenberger and Ramaswamy (1979) developed an approach to correctng the standard errors, whch was further refned by Shanken (1992). See also, e.g., Ferson (1995), Kan and Zhang (1999b), and Kan and Chen (24). See also Shanken and Wensten (26), Shanken and Zhou (27), Kan, Robott, and Shanken (29), and Km (1995). 2 For nsghts on these econometrc ssues and assocated research drectons, see, e.g., Dufour (1997, 23), Stager and Stock (27), Wand and Zvot (1998), Stock and Wrght (2), Stock, Wrght and Yogo (22), Klebergen (22, 25). The relablty of bootstrappng procedures depends on regularty condtons. These regularty condtons may fal (or hold only weakly) n regressons where the ndependent varables are generated regressors. For econometrc analyss of these ssues, see, e.g., Dufour (1997), Andrews (2), and Bolduc, Khalaf, and Yelou (27). 3 In other words, the estmated confdence nterval may not contan (cover) the true value of the parameter. 1

3 Research on the mplcatons of the weak dentfcaton problem for asset prcng tests s lmted. 4 Our paper s one of the frst that addresses cross-sectonal asset prcng models. We ntroduce a new technque for estmatng and testng cross-sectonal asset prcng models that s robust to the problems that arse due to weak dentfcaton. Our method works as follows. We conduct a statstcal assessment of prcng errors vewed as functons of the parameters of the model. (In ths respect, our approach s smlar to GMM.) On ths bass, we defne a formal measure of model ft (n the sprt of the J statstc n GMM). Ths measure (the p max statstc) can be nterpreted as the p-value for the rejecton of the model. We then derve the set of model parameters that are statstcally compatble, gven the data, wth the p max statstc. For example, for the standard 5% sze, our technque selects the set of parameters that yeld a p max statstc greater than or equal to.5. Ths set s guaranteed to cover the true (unknown) parameters wth the desred sze. We conduct smulatons to assess the mportance of weak dentfcaton for emprcal work on cross-sectonal asset prcng models and to compare the propertes of the robust IV technque wth other technques, such as Fama-MacBeth. Our smulatons are based on two popular asset prcng models one uncondtonal (Fama-French threefactor) and one condtonal (Lettau-Ludvgson CCAPM). Weak dentfcaton s not a problem n smulatons of the Fama-French threefactor model. In these smulatons, Fama-MacBeth has good sze propertes, as do two other Fama-MacBeth-based procedures the Shanken (1992) correcton and the Kan, Robott, and Shanken (29) msspecfcaton-robust procedure. The robust IV method also yelds the correct sze for tests of ndvdual parameters. In general, we don't know the true model, so t s possble that the asset prcng model may be msspecfed. For mportant parameters, msspecfcaton of the emprcal model can lead to substantal sze dstortons. For example, f the Fama-French threefactor model s msspecfed by omttng one factor, the actual Fama-MacBeth sze s.69 (for a nomnal sze of.5) for the rsk prce on the market factor. The robust IV method tends to do better. 4 See, e.g., Burnsde (27) or Klebergen (29). 2

4 Our smulatons consder two aspects of power -- the power to reject ncorrect parameter values and the power to reject an ncorrect model. In smulatons of the Fama- French three-factor model, the robust IV method dsplays consderable power to reject ncorrect parameter values. The power of the robust IV approach compares favorably wth Fama-MacBeth. For example, the power of Fama-MacBeth to reject a value of the HML rsk prce 4 steps away from the true value (where a step s equal to the OLS standard error) s less than.1. The robust IV power s greater than.8. Lewellen, Nagel, and Shanken (29) have argued that the power to reject ncorrect models s an mportant ssue n the cross-sectonal asset prcng lterature. The Monte Carlo smulatons show that the p max statstc has consderable power to reject ncorrect models. For example, n smulatons where returns are generated by the Fama- French three-factor model, the power of the p max statstc to reject the CAPM s 1.. As a pont of reference, the power of the Shanken F statstc s.3. Kan and Zhang (1999) and Klebergen (29) provde evdence on another source of weak dentfcaton. They show that sze dstortons can arse when the betas on a factor are zero (or, more generally, small). Klebergen (29) provdes evdence that ths stuaton s emprcally relevant for the Lettau-Ludvgson condtonal CCAPM. In our smulatons of the Lettau-Ludvgson model, weak dentfcaton due to small betas leads to sze dstortons for all parameters wth Fama-MacBeth estmaton. Robust IV does not suffer from sze dstortons n these smulatons. The smulatons of the Lettau-Ludvgson model also examne the ablty of the robust IV procedure to detect weak dentfcaton. In the presence of weak dentfcaton, the robust IV procedure wll produce unbounded confdence sets. The smulatons show that the robust IV method has a detecton rate for weak dentfcaton of more than 9%. The robust IV procedure wll reject a model when the nstruments are correlated wth the prcng errors. Ths means that model rejectons can be nformatve: By consderng dfferent nstruments sets, the researcher can get useful hnts about the weaknesses of an asset prcng model and the promsng drectons for mprovement. We explore ths property n smulatons of the Fama-French three-factor model, showng that a false CAPM emprcal specfcaton tends to be rejected when the nstrument set ncludes varables that are correlated wth the omtted factor loadngs. 3

5 We consder three emprcal applcatons the CAPM, Fama-French three-factor model, and Lettau-Ludvgson condtonal CCAPM. We focus on the 25 sze and book-tomarket portfolos. For the Black form of the CAPM, the Fama-MacBeth-based procedures and robust IV reach the same concluson: Market beta s nsgnfcant n explanng cross-sectonal returns. The effcency of Fama-MacBeth and robust IV are about the same. Both the pmax and Shanken F statstcs strongly reject the CAPM. For the Sharpe-Lntner form, the robust IV approach s about fve tmes as effcent as Fama- MacBeth n estmatng the market rsk prce. For the Fama-French three-factor model, the pont estmates of the SMB and HML rsk prces from Fama-MacBeth and robust IV are smlar and the effcency of the two procedures s about the same. Both procedures reach the same concluson about the market rsk prce: It s nsgnfcantly dfferent from zero. Varaton n the nstrument sets for robust IV provdes some hnts about what mght be mssng from the model. Both the Fama-MacBeth and Shanken (1992) procedures fnd that the key parameter n the Lettau-Ludvgson model ( c cay ) s sgnfcantly dfferent from zero, as Lettau and Ludvgson (21) report. The robust IV procedure, however, gves a warnng of weak dentfcaton (n the form of unbounded confdence sets) usng the most natural nstrument set. An advantage of the robust IV procedure s that t allows the researcher to add addtonal nformaton (by addng varables to the nstrument set). By dong so, t s possble to retan the sze propertes of the robust IV procedures and, wth a good choce of nstruments, ncrease effcency. Usng ths approach, we fnd that the data fal to reject the null hypothess that c cay. Ths s the same concluson that s reached by the Kan, Robott, and Shanken (29) msspecfcaton-robust procedure. The paper s organzed as follows. Secton 2 descrbes our econometrc procedure. Secton 3 presents the Monte Carlo smulaton results. Secton 4 reports emprcal results for the CAPM. Secton 5 reports emprcal results for the Fama-French three-factor model. Secton 6 reports emprcal results for the Lettau-Ludvgson condtonal CCAPM. 4

6 2. Econometrc Procedure We begn by wrtng the asset prcng model n a form that separates out the exogenous varables that enter the model lnearly: ' (, ), 1,..., n, (1) where s a k1 -dmensonal vector of exogenous varables and s the correspondng vector of parameters,, s a (possbly nonlnear) functon of observed varables, s a matrx that can nclude both endogenous and exogenous varables, s an m 1 vector of unknown parameters of nterest, and s a dsturbance wth mean zero. To llustrate our econometrc technque, consder the hypothess :. ' If (1) holds, then clearly,. Defne as a k2 1 vector of exogenous or predetermned varables such that k2 m. Snce both and are exogenous or predetermned (.e., orthogonal to ), f we regress (, ) on and, the coeffcents on n the resultng artfcal regresson ' ', (2) should be statstcally zero. Hence, can be tested by assessng n regresson (2). nclude exogenous varables n : (3) can be vewed as a vector of extra nstruments, whch may. The advantage of reformulatng the asset prcng model as we have done n equaton (2) s that we can readly deal wth the sources of weak dentfcaton that may be present n equaton (1). Ths has some mportant benefts for testng. Frst, the sze of our tests (e.g., 5%) wll be correct, even though the betas are estmated n the frst stage of the usual two-pass procedure. Second, the sze of the test wll not be affected by the qualty of the nstruments. Thrd, the sze of the test wll be correct regardless of whether we use all 5

7 relevant nstruments or not. 5 Fourth, we can test the null hypothess correctly wthout assumng that the parameters are dentfed. We denote the desred sze of the test by ; e.g.,.5. Usng a standard F test statstc, we can calculate the p-value under the null hypothess, whch we denote as pˆ,. To obtan a confdence set for wth the desrable statstcal propertes descrbed above, we can nvert the test of the null hypothess. In other words, to obtan a jont confdence regon of the desred sze for the elements of the vector need to collect all, we values that are not rejected by the test at the sgnfcance level. Then, usng projecton methods, we can obtan confdence sets for each element of. 6 The pont estmates of the parameters are obtaned dfferently n our approach than n the standard approach. In the standard approach, equaton (1) s estmated drectly (e.g., by OLS or GMM). In our approach, equaton (1) s mapped nto the artfcal regresson (2) for all values of the structural parameters of nterest, whch allows us to test each of these values. We then choose the parameter vector that yelds the hghest value of p ˆ, the least-rejected models., as a pont estmate. Pont estmates so defned correspond to The robust IV method also provdes a natural test of the overall ft of the model, a counterpart to the 2 R n a least squares regresson or the J statstc n GMM estmaton. For a partcular value of the parameter vector p ˆ,. Defne: max pˆ, max, our procedure provdes a p-value p. (4) In other words, p max s the p-value for the parameters of the asset prcng model that best fts the data. It therefore has a natural nterpretaton as the p-value for the overall rejecton of the model. 5 More formally, we can ensure that the nomnal sze of the test s equal to the asymptotc sze. Ths wll also hold even f underlyng factors have small betas. See Kan and Zhang (1999a,b) and Klebergen (29). 6 More generally, we can obtan confdence sets for any scalar functon of the form where s a nonzero m-dmensonal vector. For example, f m = 2, so the asset prcng model has two parameters, and 1, 1, we can obtan a test and confdence nterval for the null hypothess that the dfference between the two parameters s. 6

8 Indeed, testng may be nterpreted n the rsk-return framework as assessng whether addtonal explanatory varables hold further nformaton on the left hand sde returns that are not explaned by the betas. The excluson restrctons n may also be vewed as reflectng orthogonalty condtons consstent wth the GMM prncple. As an example of the applcaton of the robust IV method, the cross-sectonal regresson estmated n the Fama-French three-factor model s R ˆ ˆ ˆ e (9) e EMR, EMR SMB, SMB HML, HML where the bar denotes the tme-seres mean and R R R denotes the sample average e rf smple excess return on asset. The ntercept term n the regresson can be represented by the ' term n equaton (1), snce estmaton of the ntercept nvolves only exogenous varables and enters the asset prcng model lnearly. Specfcally: where 1 and s. The prcng error functon The varables are the factor loadngs: The parameter vector ' (1) R ˆ ˆ ˆ,,,. (11) EMR EMR SMB SMB HML HML, EMR, SMB, HML ˆ, ˆ, ˆ (12) conssts of the rsk prces:,,. (13) EMR SMB HML The above dscusson assumed that exogenous nstruments exst. To address ths problem emprcally, we proceed followng recommendatons from the weak nstruments lterature and rely on a splt-sample method [see e.g. Dufour & Jasak (21) or Angrst & Krueger (1994)]. Specfcally, we construct nstruments usng avalable data from tme 1 to T. We then construct prcng errors from the remander of the sample s (2) (2) (2) jj j1 U R ˆ, 1,..., n, where the superscrpt 2 denotes the second subsample, portfolo, (2) U s the prcng error for (2) R s mean returns for portfolo, s the expected return on the zero-beta 7

9 portfolo, portfolo. j s the rsk prce of factor j, and (2) ˆj s the estmated beta on factor j for Interestngly, and n contrast wth tradtonal splt-sample methods ncludng Dufour & Jasak (21), splttng the sample here does not cost degrees of freedom, snce the dstrbutonal result we use depends on the number of portfolos used n the secondpass cross-sectonal regresson, rather than on the tme seres dmenson of the data. 7 We set T such that TT.3. In other words, the frst 3% of the sample s used to create the nstruments and the remanng 7% of the sample s used for estmaton. The above defned GMM-type procedure s vald for any exogenous nstrument set even f ts explanatory power s weak, n the sense that test rejectons are compellng (are not spurous). In other words, the values of not retaned n our confdence set can be safely rejected at the consdered sgnfcance level. However, the power of the nverted test and the tghtness of the assocated confdence set depend on the choce of nstruments and on ther explanatory power. the In our emprcal settng, estmates of the betas from the frst subsample [formally, (1) ˆj ] provde natural nstruments. We also add nstruments that may capture mssng rsk measures. As orgnally argued n Fama and MacBeth (1973), ths may provde useful clues about the drecton n whch a gven asset prcng model should be modfed. 8 In most cases, sample perod s December 1962 through December 21, nclusve. The data s standard data that s wdely used n the cross-sectonal asset prcng lterature and s drawn prmarly from CRSP and the Ken French webste. (Where we estmate the Campbell and Vuolteenaho (24) model, we use data downloaded from the Amercan Economc Revew webste.) Martn Lettau kndly provded the Lettau and Ludvgson (21) data. In order to mantan consstency wth ther paper, the sample perod for the Lettau-Ludvgson data s 1963:Q3 through 1998:Q3, nclusve. 7 For an OLS-based splt-sample cross-sectonal regresson method, see also Beauleu, Gagnon & Khalaf (28). 8 Ths descrpton of our econometrc procedure n ths secton s ntended to be nformal. Techncal detals are provded n the Appendx. 8

10 3. Monte Carlo Smulatons 3.a. Sze n a correctly specfed model We begn wth a Monte Carlo smulaton n whch returns are generated by the Fama-French three-factor model: 3 R f U t j jt j t 1 T 1 f, j 1,...,3, j j jt T t1 U,..., U are.. d. N,, U U,..., U ', t 1,..., T 1 T t 1t nt, as n Shanken & Zhou (27), where f 1 s the market factor (EMR), f 2 s SMB, f 3 s HML, j s the factor loadng (beta) of portfolo j on factor, j s the rsk prce of factor j and U t s a random dsturbance to the returns of portfolo n perod t. The factors are set equal to the actual tme seres for our sample (so the factors are fxed n repeated samples). We set, where j j s the OLS estmate of the beta of portfolo j on factor from the frst-pass tme seres regresson for our sample. Smlarly, we set, where s the OLS estmate of the rsk prce for factor from the second-pass cross-sectonal regresson. The varance-covarance matrx of the dsturbances to returns s set equal to the estmated condtonal varance-covarance matrx. The portfolos are the 25 sze and book-to-market portfolos. Table 1 presents results on sze for several dfferent econometrc technques. For each econometrc technque, the frst-pass tme seres regresson of returns on the three Fama-French factors (and a constant) s estmated by OLS on each of the 25 sze and book-to-market portfolos. We begn by dscussng the second column, n whch the second-pass cross-sectonal regresson of returns on the betas s estmated by OLS (and n whch no correcton s made to the OLS standard errors). Ths column llustrates how easly sze dstortons can arse n estmatng cross-sectonal asset prcng models. Even though there are no specal problems n ths smulaton, OLS estmaton leads to sze dstortons. For a nomnal sze of.5, the smulatons show that the actual sze for the market rsk prce s.19. 9

11 Of course, OLS s not typcally used n estmatng cross-sectonal asset prcng models. A more common technque s Fama-MacBeth, n whch a cross-sectonal regresson s estmated for each tme perod, and nference s based on the tme seres mean of the estmated rsk prces for each perod. Ltzenberger and Ramaswamy (1979) dscuss a problem wth ths approach: the true betas are unknown, so there s a generated regressor problem when the estmated betas are used n the second-pass cross-sectonal regresson, leadng to based estmates of the standard errors. The column labeled Shanken employs the correcton to the varance-covarance matrx suggested by Shanken (1992). Kan, Robott, and Shanken (29) emphasze that "all models are, at best, approxmatons of realty and are lkely to be subject to a certan degree of msspecfcaton." But standard approaches to statstcal nference, such as Fama- MacBeth, are based on the assumpton that the models are correctly specfed. Kan, Robott, and Shanken (29) propose a procedure for computng msspecfcaton-robust asymptotc standard errors. The column labeled KRS employs ther msspecfcatonrobust procedure. Table 1 shows that, n our baselne smulaton, Fama-MacBeth, the Shanken (1992) correcton, and the KRS msspecfcaton-robust procedure all have good sze propertes. The robust IV procedure has about the same sze propertes. Table 1 llustrates a good feature of all of the estmators n ths smulaton (even OLS): they all provde unbased pont estmates of the parameters. 3.b. Sze n a msspecfed model One potental source of weak dentfcaton s a msspecfed emprcal model. In our second smulaton, returns are stll generated by the Fama-French three-factor model, but the emprcal specfcaton omts one of the factors (HML). The smulatons suggest that OLS can be problematc wth a msspecfed model. For example, as shown n Table 2, the sze for the rsk prce of the market factor s 1.. Ths means that OLS rejects the true value of the market rsk prce parameter 1% of the tme n the smulaton. Fama-MacBeth s better: t has a sze of.69 for the market rsk prce. Although the Shanken (1992) correcton was not desgned to deal wth 1

12 msspecfcaton, t provdes an mprovement on Fama-MacBeth wth a sze of.58. The KRS approach, whch was desgned to deal wth msspecfcaton, does much better than Fama-MacBeth wth a sze of.31. Robust IV s not mmune to sze dstorton, but the dstorton s moderate. Robust IV produces a sze of.9 for the market rsk prce. Table 2 reveals part of the reason why OLS provde such poor coverage: n the msspecfed model, the estmate of the market rsk prce s based. The medan dstance between the OLS estmate and the true value of the parameter s But the wdth of the medan confdence nterval s.68. As a result, the confdence ntervals n the smulatons never contan the true value of the parameter. Ths s an example of an ssue mentoned n the ntroducton. When there s weak dentfcaton (e.g., due to msspecfcaton), t s possble for the confdence nterval to be tghtly centered on an ncorrect parameter value. Fama-MacBeth and related approaches do better. In the smulatons, they yeld the same (based) parameter estmate, but the confdence nterval s much wder, yeldng better coverage. The KRS procedure performs the best n ths class of procedures because the medan wdth of the KRS confdence nterval s almost three tmes as wde as the OLS confdence nterval. A wde confdence nterval s helpful here because t approprately reflects uncertanty about the true model. The robust IV procedure yelds better sze propertes than Fama-MacBeth n ths smulaton for two reasons. Frst, t produces a medan confdence nterval that s about the same as KRS. Second, t yelds a less based pont estmate of the market rsk prce, as shown n Table 2. 3.c. Power to reject ncorrect parameter values To assess the power of the dfferent econometrc procedures to reject ncorrect parameter values, we agan generate returns based on the Fama-French three-factor model. The only dfference from the smulatons n Secton 3.a s that we now set step SE, where SE s the OLS standard error of. Table 3 reports power results for null hypotheses of the form. Fama-MacBeth has farly good power for the market factor, rejectng the null hypothess about half the tme for step 11

13 equal to -2 or +2 and more than 97% of the tme for step equal to -4 or +4. However, Fama-MacBeth has been very low power for SMB and HML. For step equal to -2 or +2, the rejecton rate for ether parameter s. For step equal to -4 or +4, the rejecton rate s for SMB and less than 8% for HML. Usng the Shanken (1992) correcton or the KRS msspecfcaton-robust approach yelds smlar results for power. The robust IV approach has more power for SMB and HML. For step equal to -2, the rejecton rate s about one-thrd. For step equal to -4 or +4, the rejecton rate s more than 8%. The robust IV approach has somewhat less power for the market factor (although the rejecton rate for step equal to -4 or +4 s stll greater than two-thrds). 3.d. Power to reject ncorrect models Lewellen, and Nagel, and Shanken (29) argue that power s low for some procedures that are commonly used to evaluate cross-sectonal asset prcng models. They provde evdence that a hgh 2 R statstc n the cross-sectonal regresson for sze and book-to-market portfolos "s smply not a suffcently hgh hurdle by whch to evaluate a model." In ther emprcal applcaton, they employ the Shanken (1985) 2 T statstc, whch s smlar to the Gbbons, Ross, and Shanken (1989) statstc except that t s based on the prcng errors n the cross-sectonal regresson. The Shanken F statstc replaces the asymptotc dstrbuton. 2 dstrbuton of the Shanken (1985) 2 T statstc wth a fnte-sample F In addton to the Shanken F statstc, we report the p max statstc, whch s the p- value for the parameters of the asset prcng model that best ft the data. It s also based on the prcng errors and bears some smlarty to the GMM J statstc, snce t arses from choosng the parameters that lead to the least rejecton of the null hypothess that the prcng errors are orthogonal to the nstruments. The p max statstc has a natural nterpretaton as the p-value for the overall rejecton of the model. The frst row of Table 4 consders the power of the Shanken F and p max statstcs to reject the CAPM n smulatons where returns are generated by the Fama-French threefactor model. The CAPM mposes two restrctons on the Fama-French three-factor 12

14 model -- that the rsk prces of SMB and HML are zero. As shown n the frst row of Table 4, the Shanken F statstc s able to reject the CAPM restrctons n 26% of the smulatons. The p max statstc rejects the CAPM restrctons n 1% of the smulatons. Arguably, the frst row of Table 4 does not provde a tough case for assessng the power of a msspecfcaton test, snce t s well known that the CAPM does not do a good job of explanng returns on sze and book-to-market portfolos (and, as noted above, our smulatons are desgned to be emprcally realstc). The second row of the table provdes a tougher test. Here, returns are stll generated by the Fama-French threefactor model and the emprcal specfcaton excludes only the SMB factor. In ths case, the rejecton rate for the Shanken F statstc s 15%, whle the p max statstc has a rejecton rate of 95%. We consder one further case. In the thrd row of the table, returns are generated by the Fama-French three-factor model, but the market factor has a zero rsk prce ( EMR ) n the smulatons. In the emprcal specfcaton, HML s excluded but no restrcton s placed on the rsk prce for the market factor. The Shanken F rejects the msspecfed emprcal model n 34% of the smulatons. The rejecton rate for the p max statstc s 57%. 3.e. Weak dentfcaton: small betas Kan and Zhang (1999) show that the presence of useless factors n an asset prcng model can dstort the sze of test statstcs. The true betas of useless factors are zero, but the researcher may not know ths. Analytcal and smulaton results show that standard test statstcs wll suffer sze dstortons n the presence of useless factors. Tests wll show that a useless factor has a nonzero rsk prce more frequently than the nomnal sze of the test ndcates. Nor s ths purely a problem of small sample sze. Kan and Zhang (1999) show that the probablty of rejectng the null hypothess of a zero rsk premum for a useless factor approaches one as the tme seres dmenson of the sample approaches nfnty. Klebergen (29) shows that the problem orgnally analyzed by Kan and Zhang (1999) s more general. Fama-MacBeth test statstcs for the rsk prces are senstve to 13

15 collnearty of the betas, so sze dstortons can arse f: 1) the betas on a factor are zero for a subset of portfolos (not necessarly all portfolos, as n Kan and Zhang (1999); or 2) the betas on a factor are small (not necessarly zero, as n Kan and Zhang (1999). Klebergen (29) fnds evdence that small betas affect nference about the key parameter n the Lettau-Ludvgson condtonal CCAPM. Two of the varables n the frstpass tme seres regresson -- the factor consumpton) and the product of c (the change n the log of aggregate c and cay (the rato of consumpton to aggregate wealth, obtaned as a contegratng regresson resdual) -- have small betas. 9 In Table 5, we therefore use smulatons of the Lettau-Ludvgson model to explore the effect of small betas on the sze propertes of the three Fama-MacBeth-based estmators and robust IV. As the smulatons show, the Fama-MacBeth test statstcs suffer from sze dstortons. The actual sze of the test statstc for c s.144, almost three tmes the nomnal sze of.5. For c cay, actual sze s.229, more than four tmes the nomnal sze. Although t was not specfcally desgned to deal wth the problem of small betas, the Shanken (1992) correcton helps. For c, the actual sze s.112, compared wth the Fama-MacBeth sze of.229. Smlarly, although the KRS test statstc was desgned to deal wth msspecfcaton, rather than small betas, t does well here, wth an actual sze of.76. The last row of Table 5 shows that the small beta problem can also affect model tests, such as the Shanken F statstc. In ths smulaton, the rejecton rate for the Shanken F s 1.6% (or slghtly more than double the nomnal sze of the test). As the frst column of the table shows, the robust IV procedure does not suffer from sze dstortons n smulatons of an asset prcng model where small betas are a problem. The smulatons reveal another appealng feature of the robust IV procedure. When a model s weakly dentfed (whether due to msspecfcaton, small betas, or some other problem), the robust IV procedure produces unbounded confdence sets. In the smulatons of the Lettau-Ludvgson model, there s a strong warnng of weak 9 We verfy ths pont usng the Hotellng test proposed by Beauleu, Dufour, and Khalaf (29). 14

16 dentfcaton: 94.2% of the smulatons produce unbounded confdence sets (not shown n the table). 3.f. Informatveness of model rejectons usng robust IV The robust IV procedure wll reject a model when the nstruments are correlated wth the prcng errors. One way that such a correlaton could arse s f an nstrument s correlated wth a varable that belongs n the asset prcng model. More generally, a model wll be rejected when there s addtonal nformaton n the nstrument set that s useful for cross-sectonal asset prcng. Ths means that model rejectons can be nformatve: By consderng dfferent nstruments sets, the researcher can get useful hnts about the weaknesses of an asset prcng model and the promsng drectons for mprovement. To llustrate ths, we conduct smulatons n whch returns are generated by the Fama-French three-factor model. We then consder the Sharpe-Lntner CAPM as an emprcal specfcaton. We estmate the CAPM on the frst 3% of each tme seres of smulated data. We use the estmated CAPM beta from ths frst subsample ( (1) ˆCAPM the basc nstrument set. We then add one more nstrument and calculate the resultng power of the p max statstc to reject the CAPM. To obtan an addtonal nstrument, we estmate the Fama-French three-factor model on the frst 3% of each tme seres of smulated data. We then use the estmated beta on SMB from ths frst subsample as an addtonal nstrument when we estmate the CAPM on the second 7% of each tme seres of smulated data. The frst column of Table 6 shows that ths nstrument set has excellent power to reject the CAPM n the smulatons: The rejecton rate s 98.1%. Ths result makes sense. In these smulatons, the cross-secton of returns depends on the market, SMB, and HML betas. When we estmate the CAPM on the smulated data, the prcng error contans the SMB and HML betas (multpled by ther rsk prces). As a result, the estmated SMB beta (from the frst subsample) has excellent power to reject the CAPM. The second column of Table 6 shows that a smlar result holds when the estmated HML beta (from the frst subsample) s the addtonal nstrument. In ths case, the power to reject the CAPM s even greater: The rejecton rate s 1%. ) as 15

17 If we use an nstrument based on an rrelevant competng model, the test wll have weaker power. For example, we can estmate the Fama-French fve-factor model on the frst 3% of each tme seres of smulated data. We can then use the estmated TERM beta (from the frst subsample) as the addtonal nstrument. The power to reject the CAPM wth ths nstrument set s very low (wth a 1.4% rejecton rate). Agan, ths makes sense. The prcng errors that result from the CAPM emprcal specfcaton are not related to the TERM beta. The table shows that the power results are smlar f the estmated DEF beta from the Fama-French fve-factor model or the estmated MOM beta from the Carhart model s used as the addtonal nstrument. Of course, the robust IV procedure wll not always pont the way to the true asset prcng model n such a straghtforward way. For example, prevous emprcal research has shown that the Campbell-Vuolteenaho model s an effectve compettor to the Fama- French three-factor model n the sense that both do a reasonable job of accountng for returns on the sze and book-to-market portfolos. Suppose we estmate the Campbell- Vuolteenaho model on the frst 3% of each tme seres of smulated data. We can then use the estmated Campbell-Vuolteenaho dscount rate beta from the frst subsample as the addtonal nstrument. Ths has very good power to reject the CAPM (wth a rejecton rate of 94.%). Ths s the correct concluson -- the CAPM s not generatng the returns n the smulaton -- but t would be premature to conclude that Campbell-Vuolteenaho s the asset prcng model that s generatng returns. There s a good reason, however, why the Campbell-Vuolteenaho dscount rate beta has such good power to reject the CAPM n the smulaton. Its correlaton wth the SMB beta s Even wth the caveat n the prevous paragraph, the smulatons show that rejectons usng the robust IV procedure are dfferent than those from some other model tests, whch smply reject or fal to reject a model. By varyng the nstrument set, the researcher at least has the possblty of usng p max statstc rejectons to obtan clues about a better asset prcng model. 16

18 4. Emprcal Results: CAPM To see how the robust IV estmator performs n famlar emprcal applcatons -- and to compare t wth other estmators -- we begn wth the CAPM. Panel A of Table 7 reports results for the Black form. For the Fama-MacBeth pont estmates, we report confdence ntervals based on the conventonal Fama-MacBeth standard errors (n parentheses), the Shanken (1992) correcton (n brackets), and the Kan, Robott, and Shanken (29) procedure (n braces). In the smulatons n Table 1, the nstruments for robust IV are a constant and the own-model betas estmated over the frst subsample (as dscussed n the secton on econometrc procedure). The robust IV estmates n the second column of Table 7 also use as nstruments a constant and the own-model beta estmated over the frst subsample ( ). (1) ˆCAPM Both the Fama-MacBeth-based procedures and robust IV reach the same concluson: CAPM beta s nsgnfcant n explanng returns on the sze and book-tomarket portfolos. Unlke the Fama-MacBeth-based procedures, the robust IV confdence nterval s not calculated by multplyng the estmated standard error by a fxed number (e.g., 1.96). Instead, the robust IV confdence sets reflect the amount of nformaton about a gven parameter that s contaned n the data. Ths s llustrated by the confdence nterval for the market rsk prce CAPM. The lower bound of the robust IV confdence nterval (-.63) s closer to the pont estmate (.2) than the upper bound (.11). Ths reflects the fact that the data allow us to rule out more canddate values n the regon to the left of the pont estmate than n the regon to the rght of the pont estmate. Wth ths feature of the robust IV procedure n mnd, we use the wdth of the confdence nterval to evaluate the effcency of each procedure. On ths bass, the effcency of three of the procedures (Fama-MacBeth, Shanken, and robust IV) n estmatng the market rsk prce s about the same. Because the KRS confdence nterval s wder than the conventonal Fama-MacBeth or Shanken confdence ntervals, robust IV s more effcent than KRS n ths example, but the dfference s small. 17

19 The Black form of the CAPM has two parameters ( and CAPM ) and the robust IV estmates n the second column use two nstruments, so the model s exactly dentfed. The p max statstc can only be calculated for an overdentfed model. Followng the approach used n the nformatveness smulatons (specfcally, the frst column of Table 6), we augment the nstrument set by addng (1) ˆSMB. As n the smulatons, the p max statstc strongly rejects the CAPM. Panel B of Table 7 provdes estmates of the Sharpe-Lntner form of the CAPM. The pont estmates from Fama-MacBeth and robust IV are smlar. So s the concluson about the market rsk prce: Both approaches fnd that t s sgnfcantly postve. In ths example, robust IV s substantally more effcent. The conventonal Fama-MacBeth confdence nterval s more than fve tmes as wde as the robust IV confdence nterval. 5. Emprcal Results: Fama-French Three-Factor Model Table 8 presents results for the Fama-French three-factor model. The pont estmates of the SMB and HML rsk prces from Fama-MacBeth and robust IV are smlar. Robust IV s equally (or perhaps slghtly more) effcent n estmatng these rsk prces. Both the Fama-MacBeth and robust IV reach the same concluson about the market rsk prce: It s nsgnfcantly dfferent from zero. The robust IV confdence ntervals for and CAPM are much wder than the correspondng confdence ntervals for any of the three Fama-MacBeth-based procedures. There s a deep econometrc reason for ths. The submatrx of the estmated beta matrx comprsed of the constant and the market betas s almost of reduced rank n the Fama- French three-factor model. The reason for ths s collnearty between the constant and the market betas that are estmated n the frst-pass Fama-French tme seres regresson. Note that, n the Black form of the CAPM, robust IV s about as effcent as Fama-MacBeth. Ths s because there s consderable varaton n the estmated market betas. When SMB and HML are added to the frst-past tme seres regresson, the varaton n market betas largely dsappears. The market betas are tghtly clustered around 1, as shown n Table 9. Fama and French (1993) notced the close connecton between the estmate of and the ncluson of estmated market betas n the second-pass cross-sectonal regresson. When 18

20 they estmate a specfcaton that ncludes only SMB and HML, they fnd that the estmate of s unrealstcally large. Includng estmated market betas substantally reduces the estmate of. As Table 1 shows, the fact that a submatrx of the beta matrx s almost of reduced rank leads to large sze dstortons wth OLS. The smulatons suggest that, n ths case, Fama-MacBeth largely solves the problem, producng (at most) mnor sze dstortons for. In other cases, however, t s possble that the sze dstorton could be mportant. Ths suggests a use for the robust IV procedure that s complementary to Fama- MacBeth. Klebergen (29) shows that, f the beta matrx s of almost reduced rank: 1) estmates of the rsk prces wll be based; and 2) Fama-MacBeth-based confdence sets wll tend to overstate precson. If the researcher uses both Fama-MacBeth and robust IV and fnds that the robust IV confdence ntervals for some parameters are much wder than the correspondng Fama-MacBeth confdence ntervals, the researcher may want to exercse cauton n nterpretng both the pont estmates and the precson mpled by the Fama-MacBeth confdence ntervals. 1 The smulatons on nformatveness show that robust IV s more lkely to reject the model when the nstrument set ncludes varables that are correlated wth the prcng errors. Ths can happen, for example, f there s an omtted varable n the estmated asset prcng model. Despte the tght factor structure of the sze and book-to-market portfolos 11, the thrd and fourth columns of Table 8 provde examples of nstrument sets that lead to the rejecton of the Fama-French three-factor model. These nstruments sets nclude a beta from a competng asset prcng model (Campbell-Vuolteenaho (24)) and characterstcs (mean sze and book-to-market rato of the portfolos over the frst subsample). 12 The results n the thrd and fourth columns of Table 8 are certanly not ntended as a comprehensve search for nstruments that mght gude research on mproved asset prcng models. But they gve an emprcal example of how varaton n 1 For example, a Monte Carlo smulaton could be used to check the propertes of the Fama-MacBeth estmator. 11 Lewellen, Nagel, and Shanken (29) provde evdence on the tght factor structure of sze and book-tomarket portfolos and dscuss ts mplcatons for tests of asset prcng models. 12 Instrument sets that nclude another characterstc (the mean of the French momentum varable over the frst subsample) also lead to rejecton of the Fama-French three-factor model. 19

21 the robust IV nstrument sets may provde hnts about frutful modfcatons of exstng models. 6. Emprcal Results: Lettau-Ludvgson Condtonal CCAPM In the smulatons of the Lettau-Ludvgson model, we saw that small betas can lead to sze dstortons wth Fama-MacBeth and that the robust IV procedure provdes a warnng of weak dentfcaton n the form of unbounded confdence sets. Table 1 reports estmates of the Lettau-Ludvgson model on actual data. Based on the Fama- MacBeth confdence nterval, t s possble to reject the null hypothess that the key parameter of the model ( c cay ) s equal to zero. Ths corresponds to what Lettau and Ludvgson (21) fnd. The Shanken (1992) correcton leads to a wder confdence nterval, but t s stll possble to reject the null hypothess that c cay s zero. Agan, ths corresponds wth Lettau and Ludvgson's results. The robust IV procedure, however, produces a warnng of weak dentfcaton (n the form of unbounded confdence sets). Ths s useful because the smulatons n Table 5 show that the Fama-MacBeth test statstcs for c cay are substantally overszed. Accordng to the smulatons, they wll reject the null hypothess that c cay more than four tmes as often as they should. The smulatons show that the KRS msspecfcaton-robust test statstcs have better sze propertes than the other two Fama-MacBeth-based procedures. As Table 1 shows, the KRS confdence nterval for c cay ncludes. Thus, based on the KRS procedure, one would fal to reject the null hypothess that c cay. In the thrd column of Table 1, we add an addtonal nstrument to the nstrument set for robust IV (mean sze over the frst subsample). Addng nstruments can ncrease the effcency of the robust IV estmator. Ths s llustrated n the thrd column of the table. Wth the addton of ths nstrument, robust IV yelds a bounded confdence nterval. The fourth column reports robust IV estmates based on an alternatve nstrument set, ths tme ncludng own-model betas, sze, and the portfolo betas on squared excess returns from the three-moment model of Kraus and Ltzenberger (1976). 2

22 The confdence ntervals based on robust IV lead to the same concluson as the KRS procedure: They do not reject the null hypothess that c cay. 21

23 References Anderson, T. W. & Rubn, H. (1949), Estmaton of the parameters of a sngle equaton n a complete system of stochastc equatons, Annals of Mathematcal Statstcs 2, Andrews D. W. K. (2). Inconsstency of the Bootstrap When a Parameter s on the Boundary of the Parameter Space, Econometrca 68, Black, F. (1972), Captal Market Equlbrum wth Restrcted Borrowng, Journal of Busness 45, Bolduc D., Khalaf L. and C. Yélou (27), Identfcaton Robust Confdence Sets Methods for Inference on Parameter Ratos wth Applcaton to Dscrete Choce Models, Journal of Econometrcs, forthcomng. Bolduc, D., Khalaf, L. & Moyneur, E. (28), Identfcaton-robust smulaton-based nference n jont dscrete/contnuous models for energy markets, Computatonal Statstcs and Data Analyss forthcomng. Burnsde A. C. (27). Emprcal Asset Prcng and Statstcal Power n the Presence of Weak Rsk Factors, NBER Workng Papers Campbell, J. Y. (1993), Intertemporal Asset Prcng wthout Consumpton Data, Amercan Economc Revew 83(3), Campbell, J. Y. & Vuolteenaho, T. (24), Bad beta, good beta, Amercan Economc Revew 94, Campbell, J.Y., Lo, A. & MacKnlay, A.C. (1997), The Econometrcs of Fnancal Markets, Prnceton Unversty Press, Prnceton. Davdson, R. & MacKnnon, J. G. (1993), Estmaton and Inference n Econometrcs, Oxford Unversty Press, New York. Dufour, J.-M. (1997), Some mpossblty theorems n econometrcs, wth applcatons to structural and dynamc models, Econometrca 65, Dufour, J.-M. (23), Identfcaton, weak nstruments and statstcal nference n econometrcs, Canadan Journal of Economcs 36(4), Dufour, J.-M. & Jasak, J. (21), Fnte sample lmted nformaton nference methods for structural equatons and models wth generated regressors, Internatonal Economc Revew 42,

24 Dufour, J.-M., Khalaf, L. & Kchan, M. (26), Inflaton dynamcs and the new Keynesan phllps curve: an dentfcaton robust econometrc analyss, Journal of Economc Dynamcs and Control 3, Dufour, J.-M. & Taamout, M. (25), Projecton-based statstcal nference n lnear structural models wth possbly weak nstruments, Econometrca 73, Dufour, J.-M. & Taamout, M. (26), Pont-optmal nstruments and generalzed Anderson-Rubn procedures for nonlnear models, Techncal report, C.R.D.E., Unversté de Montréal. Dufour, J.-M. & Taamout, M. (27), Further results on projecton-based nference n IV regressons wth weak, collnear ormssng nstruments, Journal of Econometrcs 139, Epsten, L.G. and S. E. Zn (1989), Substtuton, Rsk Averson and the Temporal Behavour of Consumpton and Asset Returns: A Theoretcal Framework, Econometrca 57(4), Epsten, L.G. and S. E. Zn (1991), Substtuton, Rsk Averson and the Temporal Behavour of Consumpton and Asset Returns: An Emprcal Analyss, Journal of Poltcal Economy 99(2), Fama, E. F. & French, K. R. (1992), The cross-secton of expected stock returns, Journal of Fnance 47, Fama, E. F. & and French, K. R. (1993), Common Rsk Factors n the Returns on Stocks and Bonds, Journal of Fnancal Economcs 33, Fama, E. F. & French,K. R. (1995), Sze and book-to-market factors n earnngs and returns, The Journal of Fnance 5, Fama, E. F. & MacBeth, J. (1973), Rsk, return, and equlbrum: Emprcal tests, Journal of Poltcal Economy 71, Ferson, W. E. (1995). Theory and Emprcal Testng of Asset Prcng Models, Chapter 5 n Fnance, Handbooks n Operatons Research and Management Scence, by Jarrow, Maksmovc and Zemba (edtors), Elsever, Ferson W. E. and S. Foerster (1994), Fnte Sample Propertes of the Generalzed Method of Moments n Tests of Condtonal Asset Prcng Models, Journal of Fnancal Economcs 36, Ferson W. E. and A. F. Segel (21), The effcent use of Condtonng Informaton n Portfolos, Journal of Fnance 56,

25 Ferson, W. E and A. F. Segel (26), Testng Portfolo Effcency wth Condtonng Informaton, Workng Paper, Boston College and Unversty of Washngton. Ghysels E. (1998). On Stable Factor Structures n the Prcng of Rsk: do Tme-Varyng Betas Help or Hurt?, Journal of Fnance 53, Hodges, Jr, J. L. & Lehmann, E. L. (1963), Estmates of locaton based on rank tests, The Annals of Mathematcal Statstcs 34, Hodges, Jr, J. L. & Lehmann, E. L. (1983), Hodges-Lehmann estmators, n N. L. Johnson, S. Kotz & C. Read, eds, Encyclopeda of Statstcal Scences, Volume 3, John Wley & Sons, New York, pp Kan R. and R. Chen (24). Fnte Sample Analyss of Two-Pass Cross-Sectonal Regressons, Workng Paper, Unversty of Toronto. Kan, R., Robott, C. & Shanken, J. (28), Two-pass cross-sectonal regressons under potentally msspecfed models, Techncal report, Rotman School of Management, Unversty of Toronto, Toronto, Canada. Kan R. and C. Zhang (1999b). Two-Pass tests of Asset Prcng Models wth Useless Factors, Journal of Fnance 54, Km D. (1995). The errors n the varables problem n the cross-secton of expected stock returns. Journal of Fnance 5, Klebergen, F., (22). Pvotal Statstcs for Testng Structural Parameters n Instrumental Varables Regresson, Econometrca 7, Klebergen, F., (25). Testng Parameters n GMM wthout Assumng that they are Identfed, Econometrca 73, Klebergen, F., (29). Tests of Rsk Prema n Lnear Factor Models, Journal of Econometrcs 149, Kraus, Alan, and Robert Ltzenberger, 1976, Skewness Preference and the Valuaton of rsk assets, Journal of Fnance 31, Lewellen, J., S. Nagel and J. Shanken (27), A Skeptcal Apprasal of Asset-Prcng Tests, Unpublshed Workng Paper. Ltzenberger, R. and K. Ramaswamy (1979), The Effect of Personal Taxes and Dvdends on Captal Asset Prces: Theory and Evdence, Journal of Fnancal Economcs 7,