Predicting economic turning points by professional forecasters -Are they useful?

Transcription

1 Predicting economic turning points by professional forecasters -Are they useful? 21 Feb 2013 Nobuo Iizuka KANAGAWA University nobuo iizuka u.ac.jp

2 Motivation Evaluating Economic forecasting has long history Debated vigorously about the best measure of inaccuracy Directional analysis is used in many papers recently Ash et al(1998) : Forecasts by OECD Pons(2000) : Comparing forecasts by IMF and OECD Ashiya(2003) : Forecasts by IMF Ashiya(2006) : Forecasts by Japanese institutional forecasters Sinclair et al (2010) : Forecasts by FRB Tsuchiya(2013) : Comparing forecasts by IMF and Japanese Government

3 Motivation On the other hand, not so many literature analyzed the accuracy of economic turning points prediction by professional economist Since 2004, ESPF has been gathering predictions of economic turning points, comparable with ESRI s Index of Business Condition Using ESPF data, this article evaluates accuracy of economic turning point predictions by professional forecasters in Japan through directional analysis

4 Objective Question1: Do professional forecasters have accurate predictions for the economic turning points? Question2: Are those predictions consistent with other predictions, such as real GDP growth?

5 Results Question1 ESPF s consensus Business Condition forecast is useful for 0~4 months ahead prediction More useful than Leading Index in ESRI s Index of Business Condition Only a few forecaster s have accurate predictions for 5 months ahead As useful as ESPF s GDP Forecast Consensus Business Condition forecast is slightly more useful than GDP forecast Looking into Individual forecaster, GDP consensus may be more useful

6 Results Question2 Most of Business Condition forecasts are consistent with GDP growth forecasts Business Condition and GDP growth forecast move in the same direction Both consensus and most of individual forecasters Forecasters who are good at Business condition forecast may be different from who are good at GDP forecast Individual DI forecast s accuracy rates vary widely than GDP forecast s

7 Outline Dataset Stylized Facts Methodology Results Conclusion

8 Dataset Actual data ESRI "Index of Business Condition Coincident Index, Leading Index Diffusion Index, Quarterly averaged Both historical data and real time data ESRI "Quarterly Estimates of GDP Real GDP Growth Quarter to quarter, Seasonally adjusted, annualized Both historical data and real time data

9 % Ⅱ 2004Ⅰ 2004Ⅱ 2004Ⅲ 2004Ⅳ 2005Ⅰ 2005Ⅱ 2005Ⅲ 2005Ⅳ 2006Ⅰ 2006Ⅱ 2006Ⅲ 2006Ⅳ 2007Ⅰ 2007Ⅱ 2007Ⅲ 2007Ⅳ 2008Ⅰ 2008Ⅱ 2008Ⅲ 2008Ⅳ 2009Ⅰ 2009Ⅱ 2009Ⅲ 2009Ⅳ 2010Ⅰ 2010Ⅱ 2010Ⅲ 2010Ⅳ 2011Ⅰ 2011Ⅱ 2011Ⅲ 2011Ⅳ 2012Ⅰ 2012Ⅲ Business Condition and GDP Growth Rate (Historical data) Business Condition Index (coincident DI) GDP growth rate (annualized,rhs) DI>50 23 GDP growth>0 22 DI=50 0 GDP growth=0 1 DI<50 12 GDP growth<0 12

10 % Ⅰ 2004Ⅱ 2004Ⅲ 2004Ⅳ 2005Ⅰ 2005Ⅱ 2005Ⅲ 2005Ⅳ 2006Ⅰ 2006Ⅱ 2006Ⅲ 2006Ⅳ 2007Ⅰ 2007Ⅱ 2007Ⅲ 2007Ⅳ 2008Ⅰ 2008Ⅱ 2008Ⅲ 2008Ⅳ 2009Ⅰ 2009Ⅱ 2009Ⅲ 2009Ⅳ 2010Ⅰ 2010Ⅱ 2010Ⅲ 2010Ⅳ 2011Ⅰ 2011Ⅱ 2011Ⅲ 2011Ⅳ 2012Ⅰ 2012Ⅱ 2012Ⅲ Business Condition and GDP Growth Rate (Real time data) Business Condition Index (Coincident DI) GDP growth rate (annualized,rhs) DI>50 23 GDP growth>0 25 DI=50 0 GDP growth=0 0 DI<50 12 GDP growth<0 10 Business Condition and GDP growth tend to move in the same direction!

11 Forecast Data JCER "ESP Forecast Dataset Forecasting Business Condition Choose from 3 options Expansion= Coincident DI will be over 50 Flat= Coincident DI will be (around) 50 or difficult to judge Contraction= Coincident DI will be below 50 Consensus= % of expansion+% of flat*0.5 Forecasting Real GDP Growth Quarter to quarter, Seasonally adjusted, annualized

12 Forecast Data Sample Dataset From 2004:1 to 2012:3, 35 quarters 12 forecasts for each quarter Forecast Horizon is 11 ~0 months ahead 32 forecasters Exclude 19 forecasters whose participate rate is fewer than 70% (concerning Business Condition forecast)

13 Stylized Facts Forecasting Business Condition Consensus Tend to forecast Expansion 83.2% of all forecasts ( 327 in 393 ) Over 90% in ft 6 ~ f t 11 Accuracy may not be a proper gauge for Usefulness Larger than 65.7% in Actual( 23 in 35 quarters ) Forecasting may be affected by actual Coincident DI Contraction prediction began in Feb08 Real time Coincident DI of Nov07(released Jan09) was below 50 But Contraction prediction ended in Apr09 Real time Coincident DI began to be over 50 in Jul09(released Sep09)

14 Stylized Facts Forecasting Business Condition (cont.) Consensus Turning Point prediction Contraction from 08Q1 Forecast for 08Q1= below 50 from (Feb08) Forecast for 08Q2=below 50 from f t 4 (APR08) Turning Point predicted in APR08 4 months Earlier than judgment by Index of Business Condition ( coincident CI ) Expansion from 09Q2 f t 3 Forecast for 09Q2=over 50 from (May09) Forecast for 09Q3=over 50 from f t 6 (May09) Turning Point predicted in May09 7 months Earlier than judgment by Index of Business Condition ( coincident CI) f t 3

15 Stylized Facts Forecasting Business Condition(cont.) Individual Forecasters Tend to forecast Flat Average ratio in predictions is 20% ~ 30% Max ratio is around 50 % and Over Also tend to forecast Expansion Turning Point Prediction Contraction from 08Q1 Need to care for flat answers Forecast for 08Q1 =Contraction Ratio around 50% from f t 3 (Mar08) Forecast for 08Q2 =Contraction Ratio over 50% from f t 2 (Jul08) Expansion from 09Q2 Forecast for 09Q2 f t 3 =Expansion Ratio over 50% from (Jun09) Forecast for 09Q3 =Expansion Ratio over 50% from f t 6 (Jun09)

16 Distribution of Forecasts (Business Condition (coincident DI )) Consensus Individual Forecaster

17 Turning Point Prediction(contraction) <Individual Forecaster> Forecast for 2008Q1 Forecast for 2008Q2 100% 100% 90% 90% 80% 80% 70% 70% 60% 60% 50% contraction flat 50% contraction flat 40% Expansion 40% Expansion 30% 30% 20% 20% 10% 10% 0% t 11 t 10 t 9 t 8 t 7 t 6 t 5 t 4 t 3 t 2 t 1 No data in t 0% t 11 t 10 t 9 t 8 t 7 t 6 t 5 t 4 t 3 t 2 t 1 t

18 Turning Point Prediction(Expansion) <Individual Forecaster> Forecast for 2009Q2 Forecast for 2009Q3 100% 100% 90% 90% 80% 80% 70% 70% 60% 60% 50% contraction flat 50% contraction flat 40% Expansion 40% Expansion 30% 30% 20% 20% 10% 10% 0% t 11 t 10 t 9 t 8 t 7 t 6 t 5 t 4 t 3 t 2 t 1 t 0% t 11 t 10 t 9 t 8 t 7 t 6 t 5 t 4 t 3 t 2 t 1 No data in t 1

19 Methodology Directional Analysis The null hypothesis The direction of change in a forecast and that in the actual are independent Two type test Fisher's Exact Test Pesaran and Timmermann s(1992) Test Sample excluding 50 or flat answer is also tested Because there is no just 50 in Actual data but many 50 or flat answer in Forecast data

20 Results Forecast Performance ESPF Business Condition All Sample Consensus 0 ~4 months ahead forecasts are Useful No significant difference between historical and realtime data Individual Forecasters Results are mixed and hard to interpret No significant difference between historical and realtime data Partly because there is no just 50 in actual DI data but many flat answer in forecast DI data

21 Table 1 1 Results(Consensus) Historical data Real time data Forecast Horizon (months) a>50 f>50 a>50 f 50 a 50 f>50 a 50 f 50 Accuracy rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft * 7.44 ** ft ft ft ft ft ft ft Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT Forecast Horizon (months) a>50 f>50 a>50 f 50 a 50 f>50 a 50 f 50 Accuracy rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft * 6.17 * ft ft ft ft ft ft ft Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT

22 Table1 2 Results(Individual Forecasters) Historical data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance Real time data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance

23 Results Forecast Performance ESPF Business Condition Sample excluding F=50 or flat Consensus 0 ~4 months ahead forecasts are useful No significant difference between historical and realtime data Same as all sample analysis Individual Forecasters Most of 0~3 months ahead forecast are useful Around half of 4 months ahead forecast are useful Only 1 or 2 forecaster s 5 months ahead forecast are useful

24 Table 1 3 Results(consensus) excluding f=50 or flat answer Historical data Real time data Forecast Horizon (months) a>50 f>50 a>50 f<50 a<50 f>50 a<50 f<50 f=50 Accuracy rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft * 7.44 ** ft ft ft ft ft ft ft Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT Forecast Horizon (months) a>50 f>50 a>50 f<50 a<50 f>50 a<50 f<50 f=50 Accuracy rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft * 6.17 * ft ft ft ft ft ft ft Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT

25 Table1 4 Results(Individual Forecasters) excluding f=50 or flat answer Historical data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance Real time data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance

26 Comparing Forecast Performance vs ESPF GDP Consensus ESPF Business condition forecast may be slightly more useful than ESPF GDP Individual Forecaster ESPF GDP is as useful as ESPF Business condition forecast vs Business Condition Index s Leading DI ESPF Business condition forecast is more useful than Leading DI

27 Results Forecast Performance ESPF GDP(Historical data and Real time data) Consensus 0 ~3 months ahead forecasts are Useful All sample and Sample excluding F=0 and A=0 Individual Forecasters Most of 0~3 months ahead forecasts are useful one third of 4 months ahead forecasts are useful Also some forecaster s 5 months ahead forecasts are useful Results of sample excluding actual=0 and forecast=0 is almost the same See the appendix

28 Table 2 1 Results(Consensus) Historical data Real time data Forecast Horizon (months) a>0 f>0 a>0 f 0 a 0 f>0 a 0 f 0 Accuracy rates(%) ft ** ** ft ** ** ft * ** ft * 7.54 ** ft ft ft ft ft ft ft ft NA Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT Forecast Horizon (months) a>0 f>0 a>0 f 0 a 0 f>0 a 0 f 0 Accuracy rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft * ft ft ft ft ft ft ft NA Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT

29 Table 2 2 Results(Individual Forecasters) Historical data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance Real time data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance

30 Results Forecast Performance Business Condition Index Leading DI Leading DI are processed as below 1 month ahead forecast=(t 1+t 2+t 3)/3 2 month ahead forecast=(t 2+t 3+t 4)/3 12 month ahead forecast=(t 12+t 13+t 14)/3 Only 1 2 months ahead forecast are useful

31 Table3 Results(Leading Index) Forecast Horizon (months) a>50 l>50 a>50 l 50 a 50 l>50 a 50 l 50 Accuracy rates(%) ** ** ** ** Notes: a= actual data,l=leading Index,FE=Fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT

32 Results Consistency Comparing directions of Business condition forecasts and GDP growth forecasts by directional analysis Most of consensus DI forecasts are consistent with GDP forecasts Most of individual DI forecasts are also consistent with individual GDP forecasts As for 6 ~ 11 months ahead forecast, some forecast s PT test static couldn t be calculated

33 Table 4 1 Results(Consensus) All sample Excluding F=50(F=0) Forecast Horizon (months) d>50 g>0 d>50 g 0 d<50 g>0 d<50 g 0 Coincide rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft ** ** ft ** ft ** ** ft ** ** ft * 9.17 ** ft ** ft * ** ft NA Notes: d= DI forecast data, g=gdp forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT Forecast Horizon (months) d>50 g>0 d>50 g<0 d<50 g>0 d<50 g<0 Coincide rates(%) ft ** ** ft ** ** ft ** ** ft ** ** ft ** ** ft ** ft ** ** ft ** ** ft * ** ft ** ft * ** ft NA Notes: d= DI forecast data, g=gdp forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT

34 Table4 2 Results(Individual Forecasters) All sample Forecast Horizon (months) Coincide rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance Excluding F=50(F=0) Forecast Horizon (months) Conincide rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance

35 Results consistency Comparing Accuracy rates of individual forecasters As for 0~4 months forecast 13 forecasters are useful in GDP growth forecast 11 forecasters are useful in Business condition forecast Only 7 forecasters are useful in both Business condition and GDP growth forecast As for 0~5 months forecast Only 1 forecaster is useful in Business condition forecast 3 forecasters are useful in GDP growth forecast No forecaster is useful in both Business condition and GDP growth forecast Most of GDP forecast accuracy rates are better than DI forecast s in each forecasting period Business Condition forecast accuracy rates vary widely

36 Table 4 3 Results (Individual Forecasters, Real time data, excluding F=50(F=0) ) Categorized individual forecasters Whether each individual forecasts are useful in all 0~4 months ahead or not DI GDP sum sum

37 Comparing Accuracy (real time data, excluding F=50(F=0)) f t f t Accyracy rate of GDP forecast(%) Accuracy rate of DI forecast (%) Accyracy rate of GDP forecast(%) Accuracy rate of DI forecast (%)

38 Comparing Accuracy (real time data, excluding F=50(F=0)) f t 2 f t Accyracy rate of GDP forecast(%) Accuracy rate of DI forecast (%) Accyracy rate of GDP forecast(%) Accuracy rate of DI forecast (%)

39 Comparing Accuracy (real time data, excluding F=50(F=0)) f t 4 f t Accyracy rate of GDP forecast(%) Accyracy rate of GDP forecast(%) Accuracy rate of DI forecast (%) Accuracy rate of DI forecast (%)

40 Conclusion Do professional forecasters have accurate predictions for the economic turning points? Yes for 0~4 months ahead prediction Both Consensus and Individual forecasters Only a few individual forecasters are useful for 5 months ahead More useful than Business Condition Index s Leading DI As useful as ESPF s GDP Forecast

41 Conclusion Are those predictions consistent with other predictions, such as real GDP growth? Most of Business Condition forecasts move in the same direction with GDP forecasts Both Consensus and Individual forecasters Forecasters who are good at DI forecast may be different from who are good at GDP forecast Individual Business Condition forecasts accuracy rates vary widely than GDP forecasts

42 Future Research Treating Flat answers in Business Condition Forecast In this article, treat as their answer mean difficult to judge The way of calculation for ESPF s consensus appropriate? Consensus= % of expansion+% of flat*0.5 The reason for varying in individual forecaster s accuracy rates for Business Condition forecast Especially in f t Are Business condition forecast helpful for GDP growth forecast?, or vice versa

43 Appendix

44 Consensus Forecast for Business condition coincident DI

45 Target, Horizon and Release month (ESPF DI forecast)

46 Consensus Forecast for Business condition coincident DI

47 Target, Horizon and Release month (ESPF DI forecast)

48 Stylized Facts Forecasting GDP growth rate Consensus Accuracy may not be a proper 89.7% of all prediction ( 357 in 398 ) gauge for Over 90% in from ft 5 to f t 11 Usefulness Larger than 62.9% in Actual(Historical: 22 in 35 Qtr) Larger than 71.4% in Actual(Real time: 25 in 35 Qtr ) Larger than Business Condition forecast Tend to forecast Expansion Individual Forecasters Tend to forecast Expansion From ft 5 to f t 11, some forecasters predict no negative growth Not necessarily the same forecaster

49 Consensus Distribution of Predictions (GDP Growth rate) f f f f f f t t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 t 10 t 11 f f f f f f Sum f> f< f= sum Individual Forecaster % ft ft 1 ft 2 ft 3 ft 4 ft 5 ft 6 ft 7 ft 8 ft 9 ft 10 ft 11 mean f>0 max min mean f<0 max min mean f=0 max min

50 Table 2 3 Results(consensus) GDP growth forecast excluding F=0 and A=0 Historical data Real time data Forecast Horizon (months) a>0 f>0 a>0 f<0 a<0 f>0 a<0 f<0 f=0 a=0 Accuracy rates(%) ft ** ** ft ** ** ft * 7.21 ** ft * 8.21 ** ft ft ft ft ft ft ft ft NA Notes: a= actual data,f=forecast data,fe=fisher's Exact Test PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance FE PT No 0% data exists in Realtime GDP growth data

51 Table 2 4 Results(Individual Forecasters) GDP growth forecast excluding F=0 and A=0 Historical data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance Real time data Forecast Horizon (months) Accuracy rates(%) PT (Number of forecasters) Mean Max Min ** * ft ft ft ft ft ft ft ft ft ft ft ft Notes: PT=Pesaran and Timmermann's(1992)test * indicates rejection at 5% significance ** indicates rejection at 1% significance