On the importance of the long-term seasonal component in day-ahead electricity price forecasting. Part II - Probabilistic forecasting

Transcription

1 On the importance of the long-term seasonal component in day-ahead electricity price forecasting. Part II - Probabilistic forecasting Rafał Weron Department of Operations Research Wrocław University of Science and Technology (PWr), Poland Based on work with Grzegorz Marcjasz and Bartosz Uniejewski Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 1 / 31

2 Introduction LTSC and short-term price forecasting Can the long-term trend-seasonal component (LTSC) impact short-term (day-ahead) electricity price forecasts? Log(price) LTSC Log(price) - LTSC Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 2 / 31

3 Introduction Point forecasting: Yes, it can! Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 3 / 31

4 Introduction What about probabilistic forecasts? t 4 t 5 P t 2 t 3 t t 1 Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 4 / 31

5 Agenda 1 Introduction 2 The Seasonal Component (SC) approach Wavelets and the HP-filter SCAR/SCANN models 3 Study design Datasets Probabilistic forecasts and the pinball score 4 Results Combining SCAR models across LTSCs Combining SCANN models across runs Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 5 / 31

6 The Seasonal Component (SC) approach Wavelets and the HP-filter Wavelets Decomposition of a signal into an approximation (a smoother) and a sequence of detail series: A J + D J + D J D 1 ): 1 Original signal Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 6 / 31

7 The Seasonal Component (SC) approach Wavelets and the HP-filter Wavelets Decomposition of a signal into an approximation (a smoother) and a sequence of detail series: A J + D J + D J D 1 ): 1 Original signal 1 Approximation 1 level S Details 1 level D Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 6 / 31

8 The Seasonal Component (SC) approach Wavelets and the HP-filter Wavelets Decomposition of a signal into an approximation (a smoother) and a sequence of detail series: A J + D J + D J D 1 ): 1 Original signal 1 Approximation 1 level S 1 6 Approximation 7 level S Details 1 level D Details 7 level D Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 6 / 31

9 The Seasonal Component (SC) approach Wavelets and the HP-filter The Hodrick-Prescott (198, 1997) filter A simple alternative to wavelets Originally proposed for decomposing GDP into a long-term growth component and a cyclical component Returns a smoothed series τ t for a noisy input series y t : min τ t { T (y t τ t ) 2 + λ t=1 T 1 t=2 Punish for: deviating from the original series roughness of the smoothed series [ ] } 2 (τ t+1 τ t ) (τ t τ t 1 ), Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 7 / 31

10 The Seasonal Component (SC) approach Wavelets and the HP-filter Wavelet and HP-filter based LTSCs GEFCom214 log-price p d,h S 6 S 9 S Nord Pool log-price p d,h HP 1 8 HP HP Wavelet filters (-S J ): S 5, S 6,..., S 14, ranging from daily smoothing (S hours) up to biannual (S hours) HP-filters (-HP λ ): with λ = 1 8, 5 1 8, 1 9,..., Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 8 / 31

11 The Seasonal Component (SC) approach SCAR/SCANN models A multivariate framework (Ziel & Weron, 218, ENEECO) Price at hour h = Price at hour h = Price at hour h = Price at hour h = D-6 D-5 D-4 D-3 D-2 D-1 D D Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 9 / 31

12 The Seasonal Component (SC) approach SCAR/SCANN models The ARX model For the log-price, i.e., p d,h = log(p d,h ), the model is given by: p d,h = β h,1 p d 1,h + β h,2 p d 2,h + β h,3 p d 7,h + β h,4 p d 1,min }{{}}{{} autoregressive effects non-linear effect + β h,5 z d,h + 3 }{{} β i=1 h,i+5d i +ε d,h (1) }{{} load forecast Mon, Sat, Sun dummies p d 1,min is yesterday s minimum hourly price z t is the logarithm of system load/consumption Dummy variables D 1, D 2 and D 3 refer to Monday, Saturday and Sunday, respectively Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 1 / 31

13 The Seasonal Component (SC) approach SCAR/SCANN models A nonlinear alternative: The ANN model p d 1,h Input layer Hidden layer Ouput layer p d 2,h p d 7,h p min d 1 z d,h p d,h D 1 D 2 D 3 Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 11 / 31

14 The Seasonal Component (SC) approach SCAR/SCANN models The SCAR/SCANN modeling framework Nowotarski-Weron, 216, ENEECO; Marcjasz et al., 218, IJF; Uniejewski et al., 218, ENEECO The Seasonal Component AutoRegressive (SCAR) / Artificial Neural Network (SCANN) framework consists of the following steps: 1 (a) Decompose the (log-)price in the calibration window into the LTSC T d,h and the stochastic component q d,h (b) Decompose the exogenous series in the calibration window using the same type of LTSC as for prices 2 Calibrate the ARX or ANN model to q t and compute forecasts for the 24 hours of the next day (24 separate series) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 12 / 31

15 The Seasonal Component (SC) approach SCAR/SCANN models The SCAR/SCANN modeling framework cont log(price) LTSC LTSC forecast Add stochastic component forecasts ˆq d+1,h to persistent forecasts ˆT d+1,h of the LTSC to yield log-price forecasts ˆp d+1,h 4 Convert them into price forecasts of the SCAR or SCANN model, i.e., ˆP d+1,h = exp (ˆp d+1,h ) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 13 / 31

16 Agenda 1 Introduction 2 The Seasonal Component (SC) approach Wavelets and the HP-filter SCAR/SCANN models 3 Study design Datasets Probabilistic forecasts and the pinball score 4 Results Combining SCAR models across LTSCs Combining SCANN models across runs Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 14 / 31

17 Study design and results Datasets GEFCom214 ( ) From the Global Energy Forecasting Competition LMP [USD/MWh] Initial calibration window for point forecasts Initial calibration window for probabilistic forecasts Test period for point forecasts Test period for probabilistic forecasts System load [GWh] Hours [ ] Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 15 / 31

18 Study design and results Datasets Nord Pool ( ) Initial calibration window for point forecasts Test period for point forecasts Test period for probabilistic forecasts Price [EUR/MWh] Initial calibration window for probabilistic forecasts 3 Consumption [GWh] Hours [ ] Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 16 / 31

19 Study design and results Probabilistic forecasts and the pinball score Three methods of constructing PIs 1 Historical simulation (H), which consists of computing sample quantiles of the empirical distribution of ε d,h s 2 Bootstrapping (B), which first generates pseudo-prices recursively using sampled normalized residuals, then computes desired quantiles of the bootstrapped prices Takes into account not only historical forecast errors but also parameter uncertainty 3 Quantile Regression Averaging (Q) Note: All require that one-day ahead point prediction errors are available in the calibration window for probabilistic forecasts Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 17 / 31

20 Individual point forecasts Study design and results Probabilistic forecasts and the pinball score Quantile Regression Averaging (QRA): The idea (Nowotarski & Weron, 215, COST) y 1,t y 2,t Quantile regression: min β q t q 1 yt <X t β q y t X t β q y t L, y t U y m,t X t = 1, y 1,t,, y m,t β q - vector of parameters Combined interval forecast (e.g. for q=.5 &.95) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 18 / 31

21 Study design and results Probabilistic forecasts and the pinball score Combining probabilistic forecasts Average probability forecast: F-Ave n 1 n ni=1 ˆFi (x) a vertical average of predictive distributions Average quantile forecast: Q-Ave n ˆQ 1 (x) with ˆQ(x) = 1 ni=1 ˆQ n i (x) and quantile forecast ˆQ i (x) = ˆF i 1 (x) a horizontal average = H, B or Q denotes the method of constructing PIs 1 Averaging probabilities 1 Averaging quantiles 1 Comparison CDF.6.4 SCARX Q HP5e1 CDF.6.4 SCARX Q HP5e1 CDF SCARX Q S6 Q F-Ave 2.2 SCARX Q S6 Q-Ave 2 Q.2 F-Ave 2 Q Q-Ave 2 Q System price (EUR/MWh) System price (EUR/MWh) System price (EUR/MWh) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 19 / 31

22 Study design and results Probabilistic forecasts and the pinball score Sharpness and the pinball loss Pinball ( ˆQPt (q), P t, q ) (1 q) ( ) ˆQ Pt (q) P t, for Pt < ˆQ Pt (q), = q ( P t ˆQ Pt (q) ), for P t ˆQ Pt (q), ˆQ Pt (q) is the price forecast at the q-th quantile P t is the actually observed price To provide an aggregate score we average: across all hours in the test period across 99 percentiles q=5% q=25% q=5% Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 2 / 31

23 Study design and results Probabilistic forecasts and the pinball score Diebold-Mariano (DM) tests Define the multivariate loss differential series in the 1 -norm as: X,Y,d = π X,d 1 π Y,d 1 where π X,d = (π X,d,1,..., π X,d,24 ) is the vector of pinball scores for model X and day d π X,d 1 = 24 h=1 π X,d,h is the average across the 24 hours As in the standard DM test, we assume that the loss differential series is covariance stationary For each model pair we compute two one-sided DM tests: 1 H : E( X,Y,d ) X yields better forecasts 2 H R : E( X,Y,d ) Y yields better forecasts Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 21 / 31

24 Agenda 1 Introduction 2 The Seasonal Component (SC) approach Wavelets and the HP-filter SCAR/SCANN models 3 Study design Datasets Probabilistic forecasts and the pinball score 4 Results Combining SCAR models across LTSCs Combining SCANN models across runs Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 22 / 31

25 Results Combining SCAR models across LTSCs Combining SCAR models across LTSCs (Uniejewski et al., 218, ENEECO) We present results for 14 selected ARX-type models: Both naive benchmarks Naive H, Naive Q All three ARX benchmarks ARX H, ARX B, ARX Q The best ex-post SCARX H, SCARX B and SCARX Q models Q-Ave H n, Q-Ave B n and Q-Ave Q n F-Ave H n, F-Ave B n and F-Ave Q n average quantile forecasts average probability forecasts Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 23 / 31

26 Results Combining SCAR models across LTSCs p-values of the DM test across 99 percentiles Naive H GEFCom214, Pinball across all 99 percentiles.1 Naive H Nord Pool, Pinball across all 99 percentiles.1 Naive Q.9 Naive Q.9 ARX H ARX B ARX Q.8.7 ARX H ARX B ARX Q.8.7 SCARX H S9.6 SCARX H S9.6 SCARX B S9.5 SCARX B S9.5 SCARX Q S9 SCARX Q S9 Q-Ave H 18.4 Q-Ave H 7.4 Q-Ave B 18.3 Q-Ave B 19.3 Q-Ave Q 16 Q-Ave Q 19 F-Ave H 18.2 F-Ave H 7.2 F-Ave B 18.1 F-Ave B 19.1 F-Ave Q 16 Naive H Naive Q ARX H ARX B ARX Q SCARX H S9 SCARX B S9 SCARX Q S9 Q-Ave H 18 Q-Ave B 18 Q-Ave Q 16 F-Ave H 18 F-Ave B 18 F-Ave Q 16 F-Ave Q 19 Naive H Naive Q ARX H ARX B ARX Q SCARX H S9 SCARX B S9 SCARX Q S9 Q-Ave H 7 Q-Ave B 19 Q-Ave Q 19 F-Ave H 7 F-Ave B 19 F-Ave Q 19 We use a heat map to indicate the range of the p-values the closer they are to zero ( dark green) the more significant is the difference between the forecasts of a model on the X-axis (better) and the forecasts of a model on the Y-axis (worse) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 24 / 31

27 Results Combining SCAR models across LTSCs Main findings Probabilistic SCARX models (nearly always) significantly outperform the Naive and ARX benchmarks SCARX Q models (nearly always) significantly outperform SCARX H and SCARX B Both averaging schemes generally significantly outperform the benchmarks and the non-combined SCARX models Averaging over probabilities (F-Ave n) generally yields better probabilistic EPFs than averaging over quantiles (Q-Ave n) In contrast to typically encountered economic forecasting problems (Lichtendahl et al., 213, Mgnt Sci.) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 25 / 31

28 Results Combining SCANN models across runs The SCANN modeling framework (Marcjasz et al., 218, IJF; Marcjasz et al., 218, WP) x(t) 5 y(t) 1 W,1,6 W Hidden b 5 W b Output 1 y(t+1) 1 One hidden layer with 5 neurons and sigmoid activation functions NARX inputs identical as in the ARX model Trained using Matlab s trainlm function, utilizing the Levenberg-Marquardt algorithm for supervised learning Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 26 / 31

29 Results Combining SCANN models across runs Sample gains from using committee machines Average WMAE (in %) GEFCom: ANN n (SC)ANN n (SC)ARX n GEFCom: SCANN n-hp n NP: ANN n n NP: SCANN n-s n Forecast errors roughly scale as a power-law function of the number of networks (runs) in a committee machine We should use as large committee machines as we can or use a more efficient training algorithm FANN library, see Marcjasz, Uniejewski & Weron (218, WP) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 27 / 31

30 Results Combining SCANN models across runs Combine point or probabilistic forecasts? Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 28 / 31

31 Results Combining SCANN models across runs p-values of the DM test across 99 percentiles (Marcjasz, Uniejewski & Weron, 218, WP) Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 29 / 31

32 Results Combining SCANN models across runs Main findings Point forecasts (SC)ANN models outperform (SC)ARX for every LTSC Probabilistic forecasts The QRA and QRM models are much better than Hist QRA(2) is the best performer, but much slower than QRM(N) It is always beneficial to average forecasts (vertical is better) Times needed to produce one week of hourly forecasts: QRM(N) QRA(2) QRA(3) QRA(4) QRA(5) 25 s 68 s s s 248 s Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 3 / 31

33 Wrap up Point forecasts (SC)ANN models outperform (SC)ARX... if properly designed Probabilistic forecasts (SC)ANN models offer opportunities that (SC)ARX do not However: Rafał Weron (Wrocław, PL) LTSC in probabilistic EPF , CEMA, Rome 31 / 31