Validation of operational NWP forecasts for global, diffuse and direct solar exposure over Australia

Transcription

1 Validation of operational NWP forecasts for global, diffuse and direct solar exposure over Australia Lawrie Rikus, Paul Gregory, Zhian Sun, Tomas Glowacki Bureau of Meteorology Research Branch, 15 June 2015

2 Motivation: why am I here? The Background: Model evaluation Need to compare model variables with observational data not included as input to DA. Surface solar radiation is an essential variable for the model forecast process NWP solar radiation forecasts are potentially a basis for solar power forecasts Solar power stations could be a source of additional validation data The Question: How well do the raw NWP surface solar radiation fields agree with the observations? Compare raw NWP fields with the Bureau s surface solar measurements Hourly accumulations available from all operational models Limitations: Full radiation calculation is done at most each hour Cloud fixed over the hour Solar zenith angle corrected at each time step

3 The ACCESS NWP Systems Australian ACCESS-NWP Community Climate (APS1 Earth-System - Domains) Simulator Based on MetOffice Unified Model and 4DVar data assimilation system APS0: Operational 2Q2010 N144 global (~80 km) ACCESS-R 40 km ACCESS-A 11 km ACCESS-C 5 km L60 APS1: Implemented 3Q2013 N320 global (~40 km) ACCESS-R 11 km ACCESS-C 4 km L70 APS2: Implemented around now N512 global (~40 km) ACCESS-R 11 km ACCESS-C 1.5 km L70

4 Station name Start End Years The Validation sites Adelaide 1994 open 15 Alice Springs 1993 open 21 Broome 1996 open 18 Cairns Cape Grim 1998 open 16 Cobar Cocos Island 2004 open 9 Darwin 1993 open 20 Geraldton Airport Geraldton Airport Comparison Kalgoorlie-Boulder Learmonth Longreach Aero 2012 open 1 Melbourne Airport 1999 open 15 Mildura Mount Gambier Rockhampton Aero 1996 open 19 Tennant Creek Airport Townsville Aero Wagga Wagga 1997 open 18 Woomera minute high quality data available

5 The relationship between the observations and the model domains 4 domains have one long-term site DN BN AD SY VT has 3 long-term sites PH has no long-term sites

6 Documentation o Legacy (pre-access) 12 km model o Gregory, P. A., L. J. Rikus, and J. D. Kepert, 2012: Testing and Diagnosing the Ability of the Bureau of Meteorology s Numerical Weather Prediction Systems to Support Prediction of Solar Energy Production. J. Appl. Meteor. Climatol, 51, o APS0 12km model (ACCESS-A) o Gregory, P. A. and L. J. Rikus: Validation of Bureau of Meteorology s Global, Diffuse and Direct Solar Exposure Forecasts using the ACCESS Numerical Weather Prediction Systems, submitted to J. Appl. Meteor. Climatol The 1-minute site data were aggregated into the relevant hour spanned by the model s forecasts Hourly accumulated global, direct and diffuse solar irradiance at the surface processed

7 Forecast metrics Solar variability is predominantly due to cloud cover and solar position Variation in solar position is completely deterministic Variation in cloud cover is mostly stochastic Need to de-couple these two factors, otherwise you inflate the skill of the NWP model. A clear sky model (Ineichen and Perez (2002)) was used to normalise forecast and observed data. Standard statistical metrics used for validation RMSE, correlation, multiplicative bias Metrics developed by Espinar et al. (2009) Integrate the absolute difference between the observed and forecast empirical cumulative distribution functions (CDFs)

8 Standard forecast metrics Paul Gregory developed the scripts to implement the validation process for ACCESS We can now apply them easily to the model archive for any period (since late 2013).

9 Validation of ACCESS-A hourly data Hourly results for January 2012 at Adelaide Global exposure Diffuse exposure Direct exposure Bias MAE RAE (%) Bias MAE RAE (%) Bias MAE RAE (%) All sky Clear sky Low cloud

10 Annual ACCESS-A Clear Sky Results Adelaide Alice Springs Broome Cape Grim Darwin Melbourne Rockhampton Wagga Wagga Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Overall day 1 better than day 2 except for Darwin, Cape Grim Correlation ~ 0.6 for global and direct < 0.5 for diffuse

11 Annual ACCESS-A All Sky Results Adelaide Alice Springs Broome Cape Grim Darwin Melbourne Rockhampton Wagga Wagga Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Direct generally under-predicted Diffuse generally over-predicted But not always! The results are site dependent.

12 Global exposure bias and RMS as function of CSI and SZA Clear Sky Index created by dividing observed exposure by clear-sky model exposure

13 Discussion of direct and diffuse irradiance Model tends to over-estimate direct and under-estimate diffuse Parameterization is tuned for global irradiance at the surface and TOA and atmospheric heating rate. Global and direct are calculated separately and differenced to produce diffuse. The two stream approach makes approximations for angular integration. Large number of different approximations in the literature Optimised for different cloud properties Can we try a different two stream scheme? Schemes which give same global radiation should not effect NWP forecast skill. Easier to implement in operational suite. (Would possibly effect surface parameterization scheme)

14 Unscaled direct two-stream approximation GHI DNI Scaled DNI Unscaled Work by Zhian Sun

15 ACCESS-C2 model experiments 0UTC Results for December 2014 Expt 1: Control Expt 2: unscaled direct Expt 3: PC2 T2m 00Z + 24h Exp1 Exp2 Exp3 AD Bias Err St Dev RMS Error BN 95 DN 35 PH 174 SY 153 VT 266 Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error D2m 00Z + 24h Exp1 Exp2 Exp3 AD Bias Err St Dev RMS Error BN 62 DN 32 PH 49 SY 69 VT 153 Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error Bias Err St Dev RMS Error Courtesy: Tomas Glowacki

16 ACCESS-C2 model experiments Solar All Sky Results for December and 12UTC runs AD AdelaideBN RockhamDN Darwin SY Wagga_VT CapeGr VT3 Melb_ai VT4 Wagga_ Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Count Global Direct Diffuse Midl Cld Bias Low Cld Bias High Cld Bias Little change in global Direct increased/diffuse decreased in Exp 2

17 ACCESS-R model experiments Solar Results Results for December and 12UTC runs Adelaide Alice Springs Broome Cape_Grim Cocos_Island Darwin Melb_airport Rockhampton Wagga_Wagga R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Count Global Direct Diffuse Midl Cld Bias Low Cld Bias High CldBias

18 Wagga-wagga all models SY VT4 A R2 R1 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER SY and VT same model but different domains (1.5 km) Wagga is close to the boundary of SY Diffuse Bias RMSE Correlation KSI OVER Midl Cld Bias Low Cld Bias High Cld Bias

19 Melbourne all models VT A R2 R1 Global Bias RMSE Correlation KSI OVER Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Midl Cld Bias Low Cld Bias High Cld Bias

20 Darwin all models DN A R2 R1 Global Bias RMSE Correlation KSI OVER DN is 1.5 km resolution and the only model which is convection permitting Direct Bias RMSE Correlation KSI OVER Diffuse Bias RMSE Correlation KSI OVER Midl Cld Bias Low Cld Bias High Cld Bias

21 APS Upgrade Plans G APS1 APS2 APS3 (~2017/2018) APS4 (~2020) 40km L70, 4dVAR Mar-2012 (Op) 25km L70, 4dVAR (2 x 240FC + 2 x 78FC) 12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC) 12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC) R 12km L70, 4dVAR Mar-2013 (Op) 12km L70, 4dVAR (4 x 72FC) 8km, L85?, 4dVAR / Hybrid? (4 x 72FC) 5km, L85?, 4dVAR / Hybrid? (4 x 72FC) C 4km L70, FC-only Mar-2013 (Op) 1.5km L70, FC-only {6 X C1} 1.5km(V) L85? 4dVAR (Rad), LHN (4 x 36FC + 4 x 18FC + 16 x 9FC ) Unchanged On Demand 1.5km L70, FC-only 1.5km(V) L85? DS + M * (3dVAR (Rad), LHN), 4 domains max (4 x 36FC + 4 x 18FC + 16 x 9FC ) Unchanged En-G 60km L70, M24 30km L85?, M24 (2 x 240FC) 30km L85?, M32 (2 x 240FC) En-C 2.2km(V) L85, M6 En-C-1 (4 X 24FC, 4 X 36FC ) 1.5km(V) L85?, M12? En-C-1 (4 X 24FC, 4 X 36FC)

22 Rapid update cycle model FDP

23 The RUC and times D0H23 D0H23 VALID TIME D3H11 BASE TIME D1H22 Daylight Daylight Possible ensemble applications? 10 output for wind, screen variables, precip, etc High frequency solar requires fast surface scheme (e.g.sunflux)

24 SUNFLUX: A fast surface radiation parameterization Zhian Sun's work Radiative transfer is expensive Hourly is 30% of model run time Clouds, SZA change but assumed constant SUNFLUX Fast but accurate calculation of surface irradiance Efficient enough to run every time step Accounts for cloud, SZA changes Could be implemented in APS3

25 Conclusions There is a scatter in the metrics with variations from site to site Different synoptics Cloud frequencies Cloud properties Accuracy of radiative transfer assumptions to different cloud regimes Aerosol not accounted for in model The comparisons all show a scatter in the metrics for individual sites Is that significant? If so which do we prefer?

26 Further work Extend evaluation to all operational models for all archive times Establish statistical significance for the different metrics Partition hourly results in terms of solar zenith angle and time of year (suggestion by John Boland) Implement fast surface radiation scheme to produce 10 minute forecasts in Model (SUNFLUX) Find more data for validation Global model Satellite derived fields

27 Lawrie Rikus Phone: Web: Thank you The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology