Presented at the PIRP Workshop Folsom, CA April 16, 2007 PIRP Forecast Performance John W. Zack AWS Truewind LLC Albany, New York jzack@awstruewind.com
Overview PIRP Forecast Performance Forecast Performance Factors Bias Correction Algorithm Other factors Forecasting the Future of Forecasting Efforts to improve PIRP forecasts
PIRP Forecast Performance What are the typical levels of performance?
PIRP Forecast Performance Specifications Next Operating Hour Definition: Hour starting 2 hr 45 min after forecast delivery Penalty Monthly MAE > 12% of installed capacity Monthly Bias > 0.6% of monthly production Bonus Monthly MAE < 10% of installed capacity Monthly Bias < 0.1% of monthly production Next Day No performance criteria 15 10 5-5 -10-15 Month to Date Foreacst Bias and MAE Bias Corrected vs Uncorrected Forecasts PIRP wind plant: June 2005 Bias (Corrected) MAE (Corrected) Bias (Uncorrected) MAE (Uncorrected) 0 6/1 6/4 6/7 6/10 6/13 6/16 6/19 6/22 6/25 6/28 7/1 Date
Typical Range of Absolute Forecast Accuracy Chart depicts composite of annual MAEs for many AWST forecast sites in North America Month to month variability at one site is typically greater than site to site variability of annual MAE 25% 20% 15% 10% 5% 0% 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 Forecast Time Horizon (Hours)
Forecast Performance Factors What factors are responsible for variations in forecast performance?
Forecast performance factors (assuming same metric and forecast system) How forecasts are optimized Quality of generation & met data from the plant Forecast time horizon (especially for short-term) Distribution of wind speeds relative to the power curve Shape of the plant-scale power curve Amount of variability in the wind resource Meteorological scales of wind variability Meteorological processes generating the winds Sensitivity of a forecast to initialization error Plus other factors...
Forecast Performance Factors: How Forecasts are Optimized PIRP power production forecasts are optimized to minimize the monthly net deviation (I.e. the monthly forecast bias) External correction procedure is used Net Deviation (ND) is calculated from start of month: curren t hour ND = ( F i O i ) i=first hr of month Bias adjustment is calculated from ND for each forecast hour: F biasadj = F 0 C * ND Adjustment phased in between 6th and 10th of month C = 0 from 1st to 5th of month C linearly increases to max value from 6th to 10th C remains at max value from 11th to end of month Hourly adjustment limited to the magnitude of the MAE
Forecast Performance Factors: How Forecasts are Optimized Impact of Low-bias Optimization on GMC Impact of Bias Correction on GMC for PIRP Participants - 2006 Bias Coorected Uncorrected $1,000,000 $800,000 $600,000 $400,000 $200,000 $- $919,914 Bias Coorected $880,516 Uncorrected Removing bias correction results in 4.3% reduction in total estimated GMC
Forecast Performance Factors: Amount and Qualify of Data from the Wind Plant Percent of Power Production Data Available PIRP Participants 2006 1 2 3 4 5 6 7 8 9 10 11 12 13 100% Plant Eight Available Data by Month Available data 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 Plant 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 11 12 Month Data availability tends to be bimodal - one group has annual availability over 90%; the other group is near 80% Near 80% group typically has one or more significant outages
Forecast Performance Factors: Plant-Scale Power Curve Shape Experiment: Assume all hours over a 1-year period have a +/- 2 m/s wind speed error and have the same wind speed distribution 100 90 80 70 60 50 40 30 20 10 0 Plant-scale Power Curves Solano Tehachapi San Gorgonio 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Wind Speed (m/s) 18% 15% 12% 9% 6% 3% 0% Simulated Annual Forecast Error - 2005 Assumptions:+/- 2 m/s Wind Speed Error; Same Wind Distribution Solano Tehachapi San Gorgonio 13.8% 12.6% 9.9% Solano Tehachapi San Gorgonio Wind Plant Slope of plant-scale power curve varies (related to correlations in wind speeds among turbines); therefore the sensitivity to wind speed forecast error varies
Forecast Performance Factors: Wind Speed Distribution Experiment: Assume all hours over a 1-yr period have the same +/- 2 m/s error and the same plant-scale power curve Horly Average Wind Speed Distribution 2005 Solano Tehachapi San Gorgonio Simulated Annual Forecast Error - 2005 Assumptions: +/- 2 m/s Wind Speed Error; Same Power Curve Solano Tehachapi San Gorgonio 14% 18% 12% 10% 8% 6% 15% 12% 9% 13.8% 12.6% 10.1% 4% 6% 2% 3% 0% 1 3 5 7 9 11 13 15 17 19 21 Wind Speed Bin Upper Boundary (m/s) 23 25 0% Solano Tehachapi San Gorgonio Wind Plant Slope of plant-scale power curve varies (related to correlations in wind speeds among turbines); therefore the sensitivity to wind speed forecast error varies
Forecasting the Future of Forecasting What is being (can be) done to improve forecast performance?
How will forecasts be improved? (Top Three List) (3) Improved physics-based/statistical models Improved physics-based modeling of sub-grid and surface processes Better data assimilation techniques for physics-based models Learning theory advances: how to extract more relevant info from data (2) More effective use of models Enabled by more computational power Higher resolution, more frequent physics-based model runs More sophisticated use of ensemble forecasting Use of more advanced statistical models and training methods (1) More/better data Expanded availability and use of off-site data in the vicinity of wind plants, especially from remote sensors A leap in quality/quantity of satellite-based sensor data
Efforts to Improve PIRP Forecasts Identify and exploit offsite useful predictor information Use more advanced physics-based and statistical modeling techniques Lower cost computing resources Larger PIRP data samples Explore use of new remote sensing technology Low-cost, low-power Doppler radars New satellite-based sensors
Offsite Forecast Correlations Use physics-based models to identify locations and parameters that have significant time-lagged correlations to wind speed at the forecast site Find (or install) measurement sites near best locations Verify relationships and put into operational use Correlation between next 2-hour wind speed at WHM with last 2-hr wind direction over the simulation domain