Verification of wind forecasts of ramping events

Transcription

1 Verification of wind forecasts of ramping events Matt Pocernich Research Application Laboratory - NCAR pocernic@ucar.edu Thanks to Brice Lambi, Seth Linden and Gregory Roux

2 Key Points A single verification statistic cannot address all concerns. Different users have different concerns errors need to be reduced across all time scales. Statistics such as rmse, mae, bias and correlation are directly related, so each can serve as a surrogate for one another. For example, without further information one would guess that a forecast with less bias has less rmse, a forecast with lower rmse is more correlated with obs, a forecast with less mae times events better. Some statistics are more appropriate or correct for evaluating different aspects of a forecast. For forecasting ramping events in wind speeds, we should examine changes in forecast and observed wind speeds.

3 Example of long lead forecasts ( 1-24 hrs)

4 Qualitative vs. Quantitative Analysis Quantitative Example 8% improvement RMSE = 2.4 m/s MAE = 3.1 m/s Bias = 0.67 m/s Qualitative = Pictures Graphic presentation of forecasts and obs Diagnostics Can reinforce information convey by quantitative summaries

5 Identifying Ramping Events Identify ramps by filtering (Pocernich, NCAR, Ela NREL) A sequence of 15 minute increases greater than 2% node capacity Interrupted by either decreases for 2 periods (30 minutes) OR a drop of 20% ramp total. ( Smaller drop criteria creates more events.) These assumptions should reflect user s requirements BPA Wind Plant Tracking System 1 hour window for 0 to 6 hour forecast, 3 hour window for 7-36 hour forecast. Hit is defined as within a specified MAE% within the window. Contingency type scores discussed. Rife, D. C. Davis, J. Knievel, Temporal Changes in Wind as Objects for Evaluating Mesoscale Numerical Weather Prediction, Weather and Forecasting; 24: Fixed, 1,2,3,6 hour changes in observed and forecast winds. Independently calculated, but spatially summarized.

6 Diagnostic approaches to verification Classic forecast v. observation graph Δ forecast v. Δ observation (3 hours)

7 Single example

8 Transforming scatter plot of changes into a contingency table

9 3 by 3 contingency table Illustrated Forecast Up Ramp Observed Up Ramp Neutral Down Ramp Neutral Down Ramp

10 Wind speed example 0 lead, 3 hour duration Forecast Up Ramp Observed Up Ramp Neutral Down Ramp 19% 9% 3% Neutral 9% 14% 12% Down Ramp 5% 9% 19% Percent Correct 59% Gerrity Skill Score 0.52

11 Changes in power forecast by short term forecast from in the first 3 hours Forecast Up Ramp Observed Up Ramp Neutral Down Ramp Neutral Down Ramp Percent Correct 42% Gerrity Skill Score 0.24

12 Contingency table-type graph for wind speeds.

13 QQ Plots for delta values.

14 Timing Timing errors are not explicitly addressed in the contingency table approach. However; Perfect forecasts at the shortest temporal resolution will result in the a perfect forecast of changes across longer intervals. A perfect forecasted change across a longer interval can be achieved in multiple ways. This allows and implicit way to evaluate timing errors.

15 New areas of research Wavelets decompose forecast and observed values by a range of time scales. Identify skill levels at each time scale Spectral decomposition Principal component analysis