Nowcasting GDP directional change with an application to French business survey data

Transcription

1 Nowcasting GDP directional change with an application to French business survey data Matthieu Cornec joint work with Fanny Mikol INSEE 18 November 2011 Cornec (INSEE) GDP direction forecasts 18 November / 31

2 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

3 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

4 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

5 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

6 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

7 Introduction For their GDP flash estimates nowcasting exercise, economists have mainly focused on point forecasts : +0, 5%, +0, 2%,... but Economic outlook analysis is mainly qualitative : economic forecasters describe their baseline scenario in terms of acceleration or deceleration : production accelerates, activity slows down, business climate remains stable Example In the Note de Conjoncture of june 2010, 7 times accelerate, and more than 20 times increase Cornec (INSEE) GDP direction forecasts 18 November / 31

8 Introduction For their GDP flash estimates nowcasting exercise, economists have mainly focused on point forecasts : +0, 5%, +0, 2%,... but Economic outlook analysis is mainly qualitative : economic forecasters describe their baseline scenario in terms of acceleration or deceleration : production accelerates, activity slows down, business climate remains stable Example In the Note de Conjoncture of june 2010, 7 times accelerate, and more than 20 times increase Cornec (INSEE) GDP direction forecasts 18 November / 31

9 Introduction For their GDP flash estimates nowcasting exercise, economists have mainly focused on point forecasts : +0, 5%, +0, 2%,... but Economic outlook analysis is mainly qualitative : economic forecasters describe their baseline scenario in terms of acceleration or deceleration : production accelerates, activity slows down, business climate remains stable Example In the Note de Conjoncture of june 2010, 7 times accelerate, and more than 20 times increase Cornec (INSEE) GDP direction forecasts 18 November / 31

10 Problem set up Usual quantitative problem : flash estimate point forecast. New qualitative question : can we forecast GDP growth rate directional change? Will it be acceleration or deceleration? Cornec (INSEE) GDP direction forecasts 18 November / 31

11 Problem set up Usual quantitative problem : flash estimate point forecast. New qualitative question : can we forecast GDP growth rate directional change? Will it be acceleration or deceleration? Cornec (INSEE) GDP direction forecasts 18 November / 31

12 Outline 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

13 Directional change definition Definition The GDP directional change between quarters q 1 and q is defined as : ε q := 1 {y q y q 1 }. with y q the flash estimate of q-o-q GDP growth, Remark We have the following notations : { εq = 1 acceleration ε q =0 deceleration Cornec (INSEE) GDP direction forecasts 18 November / 31

14 Directional change definition Definition The GDP directional change between quarters q 1 and q is defined as : ε q := 1 {y q y q 1 }. with y q the flash estimate of q-o-q GDP growth, Remark We have the following notations : { εq = 1 acceleration ε q =0 deceleration Cornec (INSEE) GDP direction forecasts 18 November / 31

15 Directional changes time series Qtr1 Qtr2 Qtr3 Qtr Cornec (INSEE) GDP direction forecasts 18 November / 31

16 Directional forecasing strategies The economic forecaster is thus asked to guess the next outcome ε q of a sequence of binary outcomes ({0, 1}) ε 1,..., ε q 1 with the knowledge of the past ε q 1 := (ε 1,..., ε q 1 ) and the side economic information x q := (x 1,..., x q ) Definition A forecasting strategy is defined as a family of predictors (φ q ) q : ε q := φ q (x q, ε q 1 ). Remark In this study, the economic information included in x q, apart from past GDP observations, will be business surveys. Cornec (INSEE) GDP direction forecasts 18 November / 31

17 Directional forecasing strategies The economic forecaster is thus asked to guess the next outcome ε q of a sequence of binary outcomes ({0, 1}) ε 1,..., ε q 1 with the knowledge of the past ε q 1 := (ε 1,..., ε q 1 ) and the side economic information x q := (x 1,..., x q ) Definition A forecasting strategy is defined as a family of predictors (φ q ) q : ε q := φ q (x q, ε q 1 ). Remark In this study, the economic information included in x q, apart from past GDP observations, will be business surveys. Cornec (INSEE) GDP direction forecasts 18 November / 31

18 Directional forecasing strategies The economic forecaster is thus asked to guess the next outcome ε q of a sequence of binary outcomes ({0, 1}) ε 1,..., ε q 1 with the knowledge of the past ε q 1 := (ε 1,..., ε q 1 ) and the side economic information x q := (x 1,..., x q ) Definition A forecasting strategy is defined as a family of predictors (φ q ) q : ε q := φ q (x q, ε q 1 ). Remark In this study, the economic information included in x q, apart from past GDP observations, will be business surveys. Cornec (INSEE) GDP direction forecasts 18 November / 31

19 Strategy cost Definition At quarter q, the (normalized) cumulative loss related to the strategy (φ q ) is defined as : L q (φ q ) := 1 q 1(ε t ˆε t ). q t=1 It corresponds to the average error rate. Remark We conduct an empirical study of error rates since Cornec (INSEE) GDP direction forecasts 18 November / 31

20 Strategy cost Definition At quarter q, the (normalized) cumulative loss related to the strategy (φ q ) is defined as : L q (φ q ) := 1 q 1(ε t ˆε t ). q t=1 It corresponds to the average error rate. Remark We conduct an empirical study of error rates since Cornec (INSEE) GDP direction forecasts 18 November / 31

21 Outline 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

22 Uninformed agent The uninformed agent forecasts the next sign change by simply drawing a random variable from a Bernouilli with parameter 1/2. ε q := u q. With probability greater than 95% the average error rate since 1997 would have bee greater than L q (Φ B ) 35%. Cornec (INSEE) GDP direction forecasts 18 November / 31

23 Uninformed agent The uninformed agent forecasts the next sign change by simply drawing a random variable from a Bernouilli with parameter 1/2. ε q := u q. With probability greater than 95% the average error rate since 1997 would have bee greater than L q (Φ B ) 35%. Cornec (INSEE) GDP direction forecasts 18 November / 31

24 Regression model based forecast Here we derive a profile forecast from a quantitative GDP growth forecast : ŷ PR q = β 0 + β 1 y PR q 1 + β 2 F q + β 3 F q F q with F q the French climate indicator. Directional forecast is then given by : ε q := 1 {ŷ PR q Average error rate since 1997 is 18%. } yq 1. PR Cornec (INSEE) GDP direction forecasts 18 November / 31

25 Regression model based forecast Here we derive a profile forecast from a quantitative GDP growth forecast : ŷ PR q = β 0 + β 1 y PR q 1 + β 2 F q + β 3 F q F q with F q the French climate indicator. Directional forecast is then given by : ε q := 1 {ŷ PR q Average error rate since 1997 is 18%. } yq 1. PR Cornec (INSEE) GDP direction forecasts 18 November / 31

26 Regression model based forecast Here we derive a profile forecast from a quantitative GDP growth forecast : ŷ PR q = β 0 + β 1 y PR q 1 + β 2 F q + β 3 F q F q with F q the French climate indicator. Directional forecast is then given by : ε q := 1 {ŷ PR q Average error rate since 1997 is 18%. } yq 1. PR Cornec (INSEE) GDP direction forecasts 18 November / 31

27 Classification versus regression Setting up the problem as a classification problem allows the use of a wide range of methods. Methods designed for classification : LDA, QDA, Probit, SVM, Arbre de classification,... In theory, they can outperform regression. Cornec (INSEE) GDP direction forecasts 18 November / 31

28 Classification versus regression Setting up the problem as a classification problem allows the use of a wide range of methods. Methods designed for classification : LDA, QDA, Probit, SVM, Arbre de classification,... In theory, they can outperform regression. Cornec (INSEE) GDP direction forecasts 18 November / 31

29 Classification versus regression Setting up the problem as a classification problem allows the use of a wide range of methods. Methods designed for classification : LDA, QDA, Probit, SVM, Arbre de classification,... In theory, they can outperform regression. Cornec (INSEE) GDP direction forecasts 18 November / 31

30 Probit forecast This model states that P(ε q = 1 X q = x q ) = F (βx q ) with F is the cumulative normal distribution. Sign forecasts are then given by : { } ε q := φ prob q (x q, ε q 1 ) = 1 F ( ˆβx q ) 0, 5 Probit average error rate since 1997 is 16%. Cornec (INSEE) GDP direction forecasts 18 November / 31

31 Probit forecast This model states that P(ε q = 1 X q = x q ) = F (βx q ) with F is the cumulative normal distribution. Sign forecasts are then given by : { } ε q := φ prob q (x q, ε q 1 ) = 1 F ( ˆβx q ) 0, 5 Probit average error rate since 1997 is 16%. Cornec (INSEE) GDP direction forecasts 18 November / 31

32 Probit forecast This model states that P(ε q = 1 X q = x q ) = F (βx q ) with F is the cumulative normal distribution. Sign forecasts are then given by : { } ε q := φ prob q (x q, ε q 1 ) = 1 F ( ˆβx q ) 0, 5 Probit average error rate since 1997 is 16%. Cornec (INSEE) GDP direction forecasts 18 November / 31

33 Linear Discriminant Analysis Notice that with two classes there is a simple correspondence between LDA and classification by linear least squares. Estimate a linear model ε q β 0 + β 1 yq 1 PR + β 2F q + β 3 F q F q, the sign forecast is then given by : { } ε q := 1 ˆβ 0 + ˆβ 1 yq 1 PR + ˆβ 2 F q + ˆβ 3 F q F q 0, 5. LDA average error rate since 1997 is 12%. Cornec (INSEE) GDP direction forecasts 18 November / 31

34 Linear Discriminant Analysis Notice that with two classes there is a simple correspondence between LDA and classification by linear least squares. Estimate a linear model ε q β 0 + β 1 yq 1 PR + β 2F q + β 3 F q F q, the sign forecast is then given by : { } ε q := 1 ˆβ 0 + ˆβ 1 yq 1 PR + ˆβ 2 F q + ˆβ 3 F q F q 0, 5. LDA average error rate since 1997 is 12%. Cornec (INSEE) GDP direction forecasts 18 November / 31

35 Linear Discriminant Analysis Notice that with two classes there is a simple correspondence between LDA and classification by linear least squares. Estimate a linear model ε q β 0 + β 1 yq 1 PR + β 2F q + β 3 F q F q, the sign forecast is then given by : { } ε q := 1 ˆβ 0 + ˆβ 1 yq 1 PR + ˆβ 2 F q + ˆβ 3 F q F q 0, 5. LDA average error rate since 1997 is 12%. Cornec (INSEE) GDP direction forecasts 18 November / 31

36 Scores for strategies Definition We define the p-value of a strategy φ q as p q := P(L q (φ random q ) L q (φ q )). By convention, we say that a strategy is significant at level α if p q α. Remark If p q α, it means that a random forecast has less than α% chances to get an error rate lower than the error rate L q (φ q ) of the strategy φ q. Cornec (INSEE) GDP direction forecasts 18 November / 31

37 Performances Summary Strategy Error since 1997 p-value Random forecast 0,50 - Opposite of the previous value 0,36 1, Opposite of the long-term mean 0,39 5, Markov forecast 0,36 1, Regression model (core variables) 0,18 8, Regression model (manufacturing variables) 0,18 8, Probit (core variables) 0,14 3, Probit (manufacturing variables) 0,16 1, LDA (core variables) 0,12 6, QDA (core variables) 0,14 3, RPART (core variables) 0,25 9, SVM (core variables) 0,16 1, Cornec (INSEE) GDP direction forecasts 18 November / 31

38 Outline 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

39 Error rate of future sign forecasts Average error rate for the next Q = 8 directional LDA forecasts (2 years : [0; 0, 5] (90% confidence interval). Average error rate for the next Q = 60 (15 years) LDA forecasts : [0; 0, 33] (intervalle de confiance à 90%). For an uninformed agent, 90% confidence interval for error rate on the next Q = 8 forecasts (2 y) : [0, 2; 0, 8]. Cornec (INSEE) GDP direction forecasts 18 November / 31

40 Error rate of future sign forecasts Average error rate for the next Q = 8 directional LDA forecasts (2 years : [0; 0, 5] (90% confidence interval). Average error rate for the next Q = 60 (15 years) LDA forecasts : [0; 0, 33] (intervalle de confiance à 90%). For an uninformed agent, 90% confidence interval for error rate on the next Q = 8 forecasts (2 y) : [0, 2; 0, 8]. Cornec (INSEE) GDP direction forecasts 18 November / 31

41 Error rate of future sign forecasts Average error rate for the next Q = 8 directional LDA forecasts (2 years : [0; 0, 5] (90% confidence interval). Average error rate for the next Q = 60 (15 years) LDA forecasts : [0; 0, 33] (intervalle de confiance à 90%). For an uninformed agent, 90% confidence interval for error rate on the next Q = 8 forecasts (2 y) : [0, 2; 0, 8]. Cornec (INSEE) GDP direction forecasts 18 November / 31

42 Error rate of future sign forecasts Average error rate for the next Q = 8 directional LDA forecasts (2 years : [0; 0, 5] (90% confidence interval). Average error rate for the next Q = 60 (15 years) LDA forecasts : [0; 0, 33] (intervalle de confiance à 90%). For an uninformed agent, 90% confidence interval for error rate on the next Q = 8 forecasts (2 y) : [0, 2; 0, 8]. Cornec (INSEE) GDP direction forecasts 18 November / 31

43 Limits of unconditional approach For practical purposes, economic forecasters may consider those intervals too large, i.e. unconditional probability of success not close enough from 1 : P(ˆε q = ε q ) << 1 Since P(ˆε q = ε q ) = E xq P(ˆε q = ε q x q ), conditional probabilities could reduce uncertainty for some quarters. Goal : for a given quarter q, estimate the conditional probability of success for our sign forecast : P(ˆε q = ε q x q ). Cornec (INSEE) GDP direction forecasts 18 November / 31

44 Limits of unconditional approach For practical purposes, economic forecasters may consider those intervals too large, i.e. unconditional probability of success not close enough from 1 : P(ˆε q = ε q ) << 1 Since P(ˆε q = ε q ) = E xq P(ˆε q = ε q x q ), conditional probabilities could reduce uncertainty for some quarters. Goal : for a given quarter q, estimate the conditional probability of success for our sign forecast : P(ˆε q = ε q x q ). Cornec (INSEE) GDP direction forecasts 18 November / 31

45 Outline 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

46 Directional risk index We derive a directional risk index I q in real time, given business surveys such that : its values lie in [ 1, +1]. Close to +1 (resp. 1), it gives strong signal of acceleration (resp. of deceleration). Between 0, 5 and +0, 5, it displays a growing uncertainty. Cornec (INSEE) GDP direction forecasts 18 November / 31

47 Directional risk index We derive a directional risk index I q in real time, given business surveys such that : its values lie in [ 1, +1]. Close to +1 (resp. 1), it gives strong signal of acceleration (resp. of deceleration). Between 0, 5 and +0, 5, it displays a growing uncertainty. Cornec (INSEE) GDP direction forecasts 18 November / 31

48 Directional risk index We derive a directional risk index I q in real time, given business surveys such that : its values lie in [ 1, +1]. Close to +1 (resp. 1), it gives strong signal of acceleration (resp. of deceleration). Between 0, 5 and +0, 5, it displays a growing uncertainty. Cornec (INSEE) GDP direction forecasts 18 November / 31

49 Definition Directional risk index Definition The directional risk index is defined as : I q := 2(ˆP q (ε q = +1) 1/2) with ˆP q the conditional probability knowing the business surveys at quarter q. Cornec (INSEE) GDP direction forecasts 18 November / 31

50 Computing Directional risk index ˆP q (ε q = +1) can be derived from different methods. For example : ˆP q (ε q = +1) = ˆP q (ŷ q PR yq 1 PR ) with ŷ q PR is the point forecast obtained through regression method. ˆP q (ε q = +1) = F ( ˆβx q ) with ˆβ is the coefficient estimates of the Probit method. Cornec (INSEE) GDP direction forecasts 18 November / 31

51 Computing Directional risk index ˆP q (ε q = +1) can be derived from different methods. For example : ˆP q (ε q = +1) = ˆP q (ŷ q PR yq 1 PR ) with ŷ q PR is the point forecast obtained through regression method. ˆP q (ε q = +1) = F ( ˆβx q ) with ˆβ is the coefficient estimates of the Probit method. Cornec (INSEE) GDP direction forecasts 18 November / 31

52 Computing Directional risk index ˆP q (ε q = +1) can be derived from different methods. For example : ˆP q (ε q = +1) = ˆP q (ŷ q PR yq 1 PR ) with ŷ q PR is the point forecast obtained through regression method. ˆP q (ε q = +1) = F ( ˆβx q ) with ˆβ is the coefficient estimates of the Probit method. Cornec (INSEE) GDP direction forecasts 18 November / 31

53 Directional risk index (graph) Cornec (INSEE) GDP direction forecasts 18 November / 31

54 Numerical results Empirical out-sample study : Over our historical sample, the directional risk index falled in the reliable area around 66% of the time. Inside that area, the average error rate obtained through any of our best strategies (LDA, probit, regression...) falls below 4%. Cornec (INSEE) GDP direction forecasts 18 November / 31

55 Numerical results Empirical out-sample study : Over our historical sample, the directional risk index falled in the reliable area around 66% of the time. Inside that area, the average error rate obtained through any of our best strategies (LDA, probit, regression...) falls below 4%. Cornec (INSEE) GDP direction forecasts 18 November / 31

56 Summary New problem : forecast directional changes. Average success rate of our best strategies between 80% et 90% Average success rate for LDA during the next 8 quarters : [50%, 100%] ( 90% confidence interval), a lot better than an uninformed agent [20%, 80%] Construction of a directional risk index that specifies the uncertainty inherent to the directional forecast. Cornec (INSEE) GDP direction forecasts 18 November / 31

57 Summary New problem : forecast directional changes. Average success rate of our best strategies between 80% et 90% Average success rate for LDA during the next 8 quarters : [50%, 100%] ( 90% confidence interval), a lot better than an uninformed agent [20%, 80%] Construction of a directional risk index that specifies the uncertainty inherent to the directional forecast. Cornec (INSEE) GDP direction forecasts 18 November / 31

58 Summary New problem : forecast directional changes. Average success rate of our best strategies between 80% et 90% Average success rate for LDA during the next 8 quarters : [50%, 100%] ( 90% confidence interval), a lot better than an uninformed agent [20%, 80%] Construction of a directional risk index that specifies the uncertainty inherent to the directional forecast. Cornec (INSEE) GDP direction forecasts 18 November / 31

59 Summary New problem : forecast directional changes. Average success rate of our best strategies between 80% et 90% Average success rate for LDA during the next 8 quarters : [50%, 100%] ( 90% confidence interval), a lot better than an uninformed agent [20%, 80%] Construction of a directional risk index that specifies the uncertainty inherent to the directional forecast. Cornec (INSEE) GDP direction forecasts 18 November / 31

60 Outline 1 Introduction 2 Notations, definitions 3 Benchmark strategies 4 Error rate of future sign forecasts 5 Directional Risk Index Cornec (INSEE) GDP direction forecasts 18 November / 31

61 Test of independence Proposition Assuming a Markov chain of order 1, an asymptotic test for the null hypothesis H 0 := {Error independence} = {Cov(ε t, ε t 1 ) = 0} with Z i := ε i 1 ε i ε i ε i 1 { Q Ĉov(ε t, ε t 1 ) ˆλ ˆΣˆλ } q 1 α/2, ˆλ := (1, Ê(Z 3 t ), Ê(Z 2 t )) et ˆΣ := Var(Z 1 ) + 2 [ĉov(z 1, Z 2 ) + ĉov(z 1, Z 3 )] Cornec (INSEE) GDP direction forecasts 18 November / 31

62 Asymptotic confidence interval for the error rate of future forecasts Proposition Assuming the independence of errors, an asymptotic confidence interval for the error rate of future sign forecasts 1 Q ε i p 1 ± ˆσq 1 α/4 1 Q + i=1 Q }{{}}{{} Moyenne des futures error Forecast error 1 N }{{} Estimation error with N the size of the estimation set, and Q the number of future sign forecasts. Cornec (INSEE) GDP direction forecasts 18 November / 31

63 Non asymptotic confidence intervalle for the error rate Proposition Assuming the independence of errors, an non-asymptotic confidence interval for the error rate of future sign forecasts 1 δ with probability greater than 1 δ : 1 ( 1 ε q p 1 ˆσ N 2 ln(8/δ) + 1 ) + 7 ln(8/δ) Q q Q N 3(N 1) + ln(8/δ) 3Q + 2 ln(8/δ) Q(N 1) }{{}}{{} Asymptotic Error Approximation error with N the size of the estimation set, and Q the number of future sign forecasts. Cornec (INSEE) GDP direction forecasts 18 November / 31