Validation of operational NWP forecasts for global, diffuse and direct solar exposure over Australia

Validation of operational NWP forecasts for global, diffuse and direct solar exposure over Australia www.bom.gov.au Lawrie Rikus, Paul Gregory, Zhian Sun, Tomas Glowacki Bureau of Meteorology Research Branch, 15 June 2015

Motivation: why am I here? The Background: Model evaluation Need to compare model variables with observational data not included as input to DA. Surface solar radiation is an essential variable for the model forecast process NWP solar radiation forecasts are potentially a basis for solar power forecasts Solar power stations could be a source of additional validation data The Question: How well do the raw NWP surface solar radiation fields agree with the observations? Compare raw NWP fields with the Bureau s surface solar measurements Hourly accumulations available from all operational models Limitations: Full radiation calculation is done at most each hour Cloud fixed over the hour Solar zenith angle corrected at each time step

The ACCESS NWP Systems Australian ACCESS-NWP Community Climate (APS1 Earth-System - Domains) Simulator Based on MetOffice Unified Model and 4DVar data assimilation system APS0: Operational 2Q2010 N144 global (~80 km) ACCESS-R 40 km ACCESS-A 11 km ACCESS-C 5 km L60 APS1: Implemented 3Q2013 N320 global (~40 km) ACCESS-R 11 km ACCESS-C 4 km L70 APS2: Implemented around now N512 global (~40 km) ACCESS-R 11 km ACCESS-C 1.5 km L70

Station name Start End Years The Validation sites Adelaide 1994 open 15 Alice Springs 1993 open 21 Broome 1996 open 18 Cairns 1997 2004 6 Cape Grim 1998 open 16 Cobar 2012 2014 1 Cocos Island 2004 open 9 Darwin 1993 open 20 Geraldton Airport 2012 2014 1 Geraldton Airport Comparison 1996 2006 9 Kalgoorlie-Boulder 1998 2013 9 Learmonth 1996 2013 10 Longreach Aero 2012 open 1 Melbourne Airport 1999 open 15 Mildura 1996 2013 10 Mount Gambier 1993 2006 11 Rockhampton Aero 1996 open 19 Tennant Creek Airport 1996 2006 10 Townsville Aero 2012 2014 1 Wagga Wagga 1997 open 18 Woomera 2012 2013 1 http://www.bom.gov.au/climate/data/oneminsolar/stations.shtml 1 minute high quality data available

The relationship between the observations and the model domains 4 domains have one long-term site DN BN AD SY VT has 3 long-term sites PH has no long-term sites

Documentation o Legacy (pre-access) 12 km model o Gregory, P. A., L. J. Rikus, and J. D. Kepert, 2012: Testing and Diagnosing the Ability of the Bureau of Meteorology s Numerical Weather Prediction Systems to Support Prediction of Solar Energy Production. J. Appl. Meteor. Climatol, 51, 1577 1601. o APS0 12km model (ACCESS-A) o Gregory, P. A. and L. J. Rikus: Validation of Bureau of Meteorology s Global, Diffuse and Direct Solar Exposure Forecasts using the ACCESS Numerical Weather Prediction Systems, submitted to J. Appl. Meteor. Climatol The 1-minute site data were aggregated into the relevant hour spanned by the model s forecasts Hourly accumulated global, direct and diffuse solar irradiance at the surface processed

Forecast metrics Solar variability is predominantly due to cloud cover and solar position Variation in solar position is completely deterministic Variation in cloud cover is mostly stochastic Need to de-couple these two factors, otherwise you inflate the skill of the NWP model. A clear sky model (Ineichen and Perez (2002)) was used to normalise forecast and observed data. Standard statistical metrics used for validation RMSE, correlation, multiplicative bias Metrics developed by Espinar et al. (2009) Integrate the absolute difference between the observed and forecast empirical cumulative distribution functions (CDFs)

Standard forecast metrics Paul Gregory developed the scripts to implement the validation process for ACCESS We can now apply them easily to the model archive for any period (since late 2013).

Validation of ACCESS-A hourly data Hourly results for January 2012 at Adelaide Global exposure Diffuse exposure Direct exposure Bias MAE RAE (%) Bias MAE RAE (%) Bias MAE RAE (%) All sky 1.01 0.28 16.38 0.95 0.21 46.05 1.03 0.44 33.54 Clear sky 1.00 0.16 8.11 1.01 0.13 40.11 1.00 0.26 14.96 Low cloud 0.88 0.05 18.89 0.84 0.05 23.17 4.55 0.01 80.21

Annual ACCESS-A Clear Sky Results Adelaide Alice Springs Broome Cape Grim Darwin Melbourne Rockhampton Wagga Wagga Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Global Bias 1.01 1.00 0.99 0.99 0.97 0.97 1.00 0.98 0.93 0.93 1.00 0.99 0.99 0.98 1.01 1.00 RMSE 0.23 0.25 0.28 0.25 0.28 0.28 0.34 0.35 0.39 0.39 0.33 0.29 0.24 0.27 0.27 0.27 Correlation 0.62 0.62 0.56 0.58 0.59 0.59 0.58 0.46 0.61 0.59 0.58 0.63 0.59 0.55 0.64 0.66 KSI 14.31 11.65 31.61 27.71 59.19 67.59 12.58 13.97 93.98 80.40 9.79 13.23 21.57 24.15 15.00 22.67 OVER 0.00 0.00 0.25 0.00 21.29 38.36 0.00 0.00 66.48 51.96 0.00 0.00 0.00 0.00 0.00 0.00 Direct Bias 1.02 1.01 1.02 1.02 0.97 0.96 0.93 0.94 0.87 0.88 0.97 0.91 1.00 0.97 1.05 1.02 RMSE 0.39 0.40 0.42 0.41 0.46 0.48 0.63 0.57 0.58 0.56 0.50 0.63 0.41 0.46 0.43 0.44 Correlation 0.64 0.65 0.59 0.59 0.57 0.56 0.53 0.51 0.64 0.62 0.56 0.50 0.60 0.56 0.65 0.63 KSI 23.58 24.60 34.28 31.97 47.91 62.14 35.86 35.30 150.10 125.10 15.35 49.43 26.51 31.95 40.25 24.84 OVER 0.00 0.00 8.18 9.67 16.54 23.85 0.00 0.00 135.50 106.90 0.00 0.00 0.00 1.92 0.00 0.00 Diffuse Bias 0.91 0.94 0.82 0.84 0.93 0.94 1.26 1.23 1.22 1.18 1.12 1.28 0.94 1.00 0.82 0.89 RMSE 0.21 0.21 0.20 0.21 0.19 0.19 0.34 0.29 0.23 0.22 0.30 0.42 0.20 0.23 0.20 0.21 Correlation 0.45 0.45 0.39 0.37 0.41 0.41 0.27 0.41 0.47 0.47 0.28 0.18 0.41 0.39 0.46 0.39 KSI 102.20 108.10 237.90 384.40 164.90 191.80 78.38 71.33 168.80 129.30 100.10 131.80 134.90 182.90 138.40 177.40 OVER 65.66 78.52 213.30 365.30 164.90 191.80 52.71 40.33 147.90 113.00 67.48 121.90 119.50 162.20 109.20 153.50 Overall day 1 better than day 2 except for Darwin, Cape Grim Correlation ~ 0.6 for global and direct < 0.5 for diffuse

Annual ACCESS-A All Sky Results Adelaide Alice Springs Broome Cape Grim Darwin Melbourne Rockhampton Wagga Wagga Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Global Bias 1.02 1.01 0.97 0.97 0.96 0.96 1.03 1.01 0.91 0.92 0.97 0.97 1.00 0.98 1.01 1.00 RMSE 0.50 0.50 0.47 0.48 0.52 0.49 0.50 0.55 0.60 0.62 0.48 0.54 0.44 0.49 0.43 0.47 Correlation 0.62 0.60 0.66 0.62 0.61 0.63 0.64 0.56 0.59 0.55 0.67 0.60 0.74 0.68 0.73 0.70 KSI 26.51 19.82 56.44 59.19 84.05 86.11 38.45 18.54 181.60 157.20 39.18 34.81 17.96 34.72 12.80 15.58 OVER 0.00 0.00 4.35 13.81 59.18 62.25 0.76 0.00 176.40 150.90 0.00 0.00 0.00 0.00 0.00 0.00 Direct Bias 0.99 0.98 0.98 0.97 0.94 0.93 0.90 0.88 0.79 0.83 0.84 0.84 0.94 0.91 1.02 1.01 RMSE 0.65 0.68 0.63 0.66 0.71 0.69 0.62 0.66 0.78 0.78 0.62 0.67 0.60 0.66 0.58 0.63 Correlation 0.63 0.60 0.68 0.63 0.63 0.65 0.61 0.56 0.61 0.58 0.64 0.58 0.72 0.67 0.71 0.67 KSI 87.32 83.29 65.25 71.38 106.90 110.50 82.71 103.60 328.90 262.30 148.90 152.20 79.25 113.70 70.88 57.25 OVER 0.00 46.38 38.87 42.01 82.82 85.45 55.80 79.13 326.20 258.60 136.30 137.10 55.49 93.46 24.04 21.28 Diffuse Bias 1.08 1.08 0.92 0.94 0.98 0.99 1.19 1.19 1.21 1.14 1.18 1.19 1.11 1.12 0.96 0.99 RMSE 0.34 0.33 0.30 0.30 0.30 0.29 0.33 0.33 0.33 0.34 0.33 0.34 0.32 0.33 0.29 0.31 Correlation 0.36 0.34 0.44 0.38 0.38 0.40 0.35 0.30 0.34 0.30 0.34 0.28 0.40 0.37 0.42 0.34 KSI 148.30 154.70 205.70 220.60 152.80 154.10 197.10 195.30 234.60 173.90 167.90 199.00 195.20 209.60 127.40 140.40 OVER 0.00 136.20 182.40 201.70 126.80 126.90 181.90 181.80 217.40 157.50 148.70 182.90 184.90 196.70 85.02 116.40 Direct generally under-predicted Diffuse generally over-predicted But not always! The results are site dependent.

Global exposure bias and RMS as function of CSI and SZA Clear Sky Index created by dividing observed exposure by clear-sky model exposure

Discussion of direct and diffuse irradiance Model tends to over-estimate direct and under-estimate diffuse Parameterization is tuned for global irradiance at the surface and TOA and atmospheric heating rate. Global and direct are calculated separately and differenced to produce diffuse. The two stream approach makes approximations for angular integration. Large number of different approximations in the literature Optimised for different cloud properties Can we try a different two stream scheme? Schemes which give same global radiation should not effect NWP forecast skill. Easier to implement in operational suite. (Would possibly effect surface parameterization scheme)

Unscaled direct two-stream approximation GHI DNI Scaled DNI Unscaled Work by Zhian Sun

ACCESS-C2 model experiments 0UTC Results for December 2014 Expt 1: Control Expt 2: unscaled direct Expt 3: PC2 T2m 00Z + 24h Exp1 Exp2 Exp3 AD Bias -0.6819-0.6762-0.7619 89 Err St Dev 1.4285 1.4248 1.8738 RMS Error 1.6964 1.6926 2.1304 BN 95 DN 35 PH 174 SY 153 VT 266 Bias -0.7933-0.7956-1.1621 Err St Dev 1.4054 1.4042 1.7572 RMS Error 1.7144 1.7138 2.1924 Bias -0.3850-0.3823-0.3091 Err St Dev 1.3537 1.3495 2.4741 RMS Error 1.5807 1.5776 2.6110 Bias -0.4630-0.4575 0.0019 Err St Dev 1.6652 1.6623 2.3227 RMS Error 1.8254 1.8220 2.4602 Bias -0.5869-0.5915-0.6702 Err St Dev 1.5021 1.4996 1.7590 RMS Error 1.7593 1.7584 2.0149 Bias -0.6153-0.6148-0.5816 Err St Dev 1.5300 1.5328 1.8318 RMS Error 1.7759 1.7771 2.0474 D2m 00Z + 24h Exp1 Exp2 Exp3 AD Bias -0.4438-0.4571-0.1234 68 Err St Dev 1.8676 1.8735 2.0035 RMS Error 2.2411 2.2509 2.3543 BN 62 DN 32 PH 49 SY 69 VT 153 Bias -0.0376-0.0334 0.2782 Err St Dev 1.8704 1.8684 2.0874 RMS Error 2.0611 2.0580 2.3316 Bias -0.5316-0.5290-0.1444 Err St Dev 2.1853 2.1920 2.7055 RMS Error 2.5135 2.5167 3.1528 Bias 0.0634 0.0595 0.5265 Err St Dev 1.7709 1.7609 2.0151 RMS Error 1.9622 1.9488 2.2516 Bias -0.2387-0.2339 0.0161 Err St Dev 1.7616 1.7592 1.9189 RMS Error 1.9366 1.9351 2.0821 Bias -0.4536-0.4569-0.2859 Err St Dev 1.6792 1.6778 1.8901 RMS Error 1.9321 1.9331 2.1025 Courtesy: Tomas Glowacki

ACCESS-C2 model experiments Solar All Sky Results for December 2014-0 and 12UTC runs AD AdelaideBN RockhamDN Darwin SY Wagga_VT CapeGr VT3 Melb_ai VT4 Wagga_ Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Global Bias 1.03 1.03 0.99 0.99 0.93 0.93 1.02 1.02 1.00 1.00 1.06 1.06 1.03 1.03 RMSE 0.47 0.46 0.46 0.44 0.31 0.31 0.40 0.40 0.47 0.45 0.53 0.54 0.38 0.37 Correlation 0.64 0.64 0.76 0.77 0.80 0.81 0.73 0.73 0.64 0.65 0.60 0.60 0.75 0.76 KSI 21.14 21.81 15.94 16.16 47.16 44.26 19.07 74.79 16.78 15.00 28.22 30.22 21.51 20.09 OVER 0 0 0.00 0.00 16.29 15.54 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Direct Bias 0.98 1.13 0.98 1.11 0.87 0.95 1.04 1.18 0.88 1.07 1.00 1.18 1.07 1.21 RMSE 0.63 0.62 0.61 0.60 0.48 0.41 0.45 0.47 0.62 0.58 0.59 0.59 0.40 0.44 Correlation 0.64 0.65 0.73 0.74 0.77 0.78 0.79 0.80 0.58 0.58 0.63 0.63 0.83 0.83 KSI 21.41 55.25 29.54 61.14 81.15 32.44 24.71 74.79 39.03 29.20 37.42 68.25 34.65 87.32 OVER 0.00 11.11 0.00 8.02 50.05 0.97 0.00 29.97 0.90 0.00 4.24 8.91 0.00 47.27 Diffuse Bias 1.12 0.85 0.96 0.75 1.21 0.88 0.97 0.74 1.21 0.92 1.12 0.91 0.95 0.72 RMSE 0.32 0.33 0.28 0.34 0.20 0.19 0.25 0.31 0.30 0.32 0.31 0.35 0.24 0.32 Correlation 0.39 0.35 0.54 0.49 0.56 0.54 0.44 0.31 0.31 0.22 0.33 0.19 0.53 0.41 KSI 80.40 79.27 60.92 101.40 91.46 89.92 88.98 99.70 79.37 37.78 70.90 43.62 72.86 94.85 OVER 43.73 60.57 33.62 79.58 68.37 68.49 51.63 86.90 64.72 18.59 33.63 27.26 41.78 80.00 Count Global 283 283 318 318 328 328 313 313 290 290 319 319 313 313 Direct 283 283 318 318 328 328 313 313 290 290 319 319 313 313 Diffuse 283 283 318 318 328 328 313 313 290 290 319 319 313 313 Midl Cld Bias 80.78 74.32 155.10 151.30 0.00 0.00 105.70 105.50 0.00 0.00 109.50 108.10 98.94 100.80 Low Cld Bias 437.10 404.20 262.50 254.60 571.30 500.00 340.30 349.40 0.00 0.00 264.50 271.10 288.50 280.20 High Cld Bias 101.00 97.52 99.64 89.69 1829.00 1822.00 34.01 33.69 0.00 0.00 83.81 84.77 31.46 31.52 Little change in global Direct increased/diffuse decreased in Exp 2

ACCESS-R model experiments Solar Results Results for December 2014-0 and 12UTC runs Adelaide Alice Springs Broome Cape_Grim Cocos_Island Darwin Melb_airport Rockhampton Wagga_Wagga R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 Global Bias 1.05 1.09 1.05 1.04 0.97 0.97 0.97 0.96 0.97 1.05 1.05 1.12 1.16 1.20 1.29 0.96 1.10 1.02 1.09 1.07 1.03 1.14 1.10 1.05 1.02 0.99 0.96 RMSE 0.60 0.57 0.65 0.58 0.59 0.67 0.50 0.51 0.50 0.65 0.62 0.73 0.65 0.71 0.87 0.83 0.84 0.78 0.55 0.57 0.65 0.63 0.68 0.69 0.52 0.53 0.58 Correlation 0.77 0.79 0.73 0.61 0.62 0.62 0.54 0.51 0.50 0.68 0.70 0.61 0.61 0.59 0.47 0.25 0.26 0.25 0.79 0.78 0.71 0.65 0.60 0.54 0.72 0.72 0.68 KSI 48.86 49.93 33.89 20.8 27.8 17.5 37.1 31.3 35.7 18.8 30.0 52.7 59.7 88.3 120.2 75.1 78.4 65.7 22.5 33.4 20.9 52.4 55.4 28.9 23.0 21.7 23.0 OVER 0.0 0.0 0.0 0.00 0.00 0.00 0.0 0.0 0.0 0.0 0.0 6.4 30.1 56.5 100.3 30.8 42.4 5.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Direct Bias 1.06 1.28 1.01 0.95 0.90 0.73 0.83 1.07 0.87 0.94 1.16 0.96 1.26 1.96 1.27 0.85 1.25 0.86 1.05 1.25 0.95 1.17 1.25 0.98 1.03 1.14 0.87 RMSE 0.80 0.82 0.82 1.03 0.94 1.06 0.76 0.79 0.76 0.73 0.65 0.76 0.82 0.97 0.95 0.82 0.82 0.89 0.62 0.67 0.73 0.76 0.81 0.87 0.67 0.66 0.77 Correlation 0.74 0.75 0.71 0.46 0.53 0.60 0.57 0.51 0.53 0.66 0.69 0.62 0.48 0.40 0.30 0.30 0.32 0.19 0.78 0.76 0.71 0.63 0.61 0.51 0.72 0.74 0.68 KSI 54.45 102.80 19.47 86.2 62.5 131.3 74.2 56.6 69.2 35.6 45.7 16.7 91.7 173.1 83.9 59.5 91.2 58.7 22.9 78.2 24.7 46.4 97.0 33.3 54.6 64.7 60.2 OVER 9.0 79.3 0.0 56.1 36.4 118.3 52.5 17.2 53.9 15.2 1.6 1.9 62.7 165.0 53.3 29.2 58.4 43.9 0.7 12.0 0.0 0.7 74.2 6.9 1.4 32.8 5.7 Diffuse Bias 1.02 0.75 1.18 0.90 0.84 1.58 1.32 0.77 1.32 1.19 0.94 1.32 1.03 0.83 1.29 1.16 0.96 1.27 1.15 0.86 1.19 1.09 0.89 1.18 1.00 0.75 1.27 RMSE 0.37 0.44 0.40 0.47 0.50 0.51 0.36 0.45 0.38 0.38 0.37 0.42 0.38 0.48 0.50 0.35 0.37 0.43 0.31 0.39 0.40 0.32 0.36 0.40 0.29 0.36 0.34 Correlation 0.47 0.38 0.43 0.20 0.23 0.38 0.46 0.40 0.43 0.51 0.52 0.48 0.20 0.12 0.09 0.12-0.01-0.02 0.49 0.35 0.35 0.43 0.30 0.23 0.52 0.51 0.50 KSI 54.28 101.30 46.30 85.7 99.0 137.5 86.4 105.7 118.6 85.3 34.1 125.1 48.1 80.6 106.7 118.2 39.2 115.3 54.3 62.4 73.7 65.7 61.8 74.4 63.4 99.4 81.0 OVER 23.9 83.6 24.1 49.5 71.2 126.4 50.2 91.5 92.3 68.0 19.1 110.7 8.4 64.3 81.6 103.8 0.0 96.1 4.5 35.1 33.1 31.5 42.7 29.3 33.7 75.3 43.6 Count Global 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392 Direct 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392 Diffuse 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392 Midl Cld Bias 179.40 179.40 165.40 289.8 271.4 195.8 835.4 685.3 638.7 0.0 0.0 0.0 66.7 58.5 43.3 3078.0 507.2 602.6 90.2 85.4 91.5 163.9 171.3 246.3 115.5 95.5 98.4 Low Cld Bias 482.10 412.80 334.20 199.1 1125.0 362.7 145.3 135.5 169.7 0.0 0.0 0.0 286.8 236.1 278.7 291.9 269.3 305.6 297.2 271.9 263.8 226.8 268.6 255.8 250.0 318.6 257.8 High CldBias 100.10 75.60 66.80 45.6 28.1 30.8 445.3 417.2 397.9 0.0 0.0 0.0 123.7 106.4 139.1 619.4 449.1 609.9 36.6 44.2 50.7 136.5 150.9 161.8 79.9 54.2 61.9

Wagga-wagga all models SY VT4 A R2 R1 Global Bias 1.02 1.03 1.01 1.02 0.96 RMSE 0.40 0.38 0.43 0.52 0.58 Correlation 0.73 0.75 0.73 0.72 0.68 KSI 19.07 21.51 12.80 23.0 23.0 OVER 0.00 0.00 0.00 0.0 0.0 Direct Bias 1.04 1.07 1.02 1.03 0.87 RMSE 0.45 0.40 0.58 0.67 0.77 Correlation 0.79 0.83 0.71 0.72 0.68 KSI 24.71 34.65 70.88 54.6 60.2 OVER 0.00 0.00 24.04 1.4 5.7 SY and VT same model but different domains (1.5 km) Wagga is close to the boundary of SY Diffuse Bias 0.97 0.95 0.96 1.00 1.27 RMSE 0.25 0.24 0.29 0.29 0.34 Correlation 0.44 0.53 0.42 0.52 0.50 KSI 88.98 72.86 127.40 63.4 81.0 OVER 51.63 41.78 85.02 33.7 43.6 Midl Cld Bias 105.70 98.94 115.5 98.4 Low Cld Bias 340.30 288.50 250.0 257.8 High Cld Bias 34.01 31.46 79.9 61.9

Melbourne all models VT A R2 R1 Global Bias 1.06 0.97 1.09 1.03 RMSE 0.53 0.48 0.55 0.65 Correlation 0.60 0.67 0.79 0.71 KSI 28.22 39.18 22.5 20.9 OVER 0.00 0.00 0.0 0.0 Direct Bias 1.00 0.84 1.05 0.95 RMSE 0.59 0.62 0.62 0.73 Correlation 0.63 0.64 0.78 0.71 KSI 37.42 148.90 22.9 24.7 OVER 4.24 136.30 0.7 0.0 Diffuse Bias 1.12 1.18 1.15 1.19 RMSE 0.31 0.33 0.31 0.40 Correlation 0.33 0.34 0.49 0.35 KSI 70.90 167.90 54.3 73.7 OVER 33.63 148.70 4.5 33.1 Midl Cld Bias 109.50 90.2 91.5 Low Cld Bias 264.50 297.2 263.8 High Cld Bias 83.81 36.6 50.7

Darwin all models DN A R2 R1 Global Bias 0.93 0.91 0.96 1.02 RMSE 0.31 0.60 0.83 0.78 Correlation 0.80 0.59 0.25 0.25 KSI 47.16 181.60 75.1 65.7 OVER 16.29 176.40 30.8 5.2 DN is 1.5 km resolution and the only model which is convection permitting Direct Bias 0.87 0.79 0.85 0.86 RMSE 0.48 0.78 0.82 0.89 Correlation 0.77 0.61 0.30 0.19 KSI 81.15 328.90 59.5 58.7 OVER 50.05 326.20 29.2 43.9 Diffuse Bias 1.21 1.21 1.16 1.27 RMSE 0.20 0.33 0.35 0.43 Correlation 0.56 0.34 0.12-0.02 KSI 91.46 234.60 118.2 115.3 OVER 68.37 217.40 103.8 96.1 Midl Cld Bias 0.00 3078.0 602.6 Low Cld Bias 571.30 291.9 305.6 High Cld Bias 1829.00 619.4 609.9

APS Upgrade Plans G APS1 APS2 APS3 (~2017/2018) APS4 (~2020) 40km L70, 4dVAR Mar-2012 (Op) 25km L70, 4dVAR (2 x 240FC + 2 x 78FC) 12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC) 12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC) R 12km L70, 4dVAR Mar-2013 (Op) 12km L70, 4dVAR (4 x 72FC) 8km, L85?, 4dVAR / Hybrid? (4 x 72FC) 5km, L85?, 4dVAR / Hybrid? (4 x 72FC) C 4km L70, FC-only Mar-2013 (Op) 1.5km L70, FC-only {6 X C1} 1.5km(V) L85? 4dVAR (Rad), LHN (4 x 36FC + 4 x 18FC + 16 x 9FC ) Unchanged On Demand 1.5km L70, FC-only 1.5km(V) L85? DS + M * (3dVAR (Rad), LHN), 4 domains max (4 x 36FC + 4 x 18FC + 16 x 9FC ) Unchanged En-G 60km L70, M24 30km L85?, M24 (2 x 240FC) 30km L85?, M32 (2 x 240FC) En-C 2.2km(V) L85, M6 En-C-1 (4 X 24FC, 4 X 36FC ) 1.5km(V) L85?, M12? En-C-1 (4 X 24FC, 4 X 36FC)

Rapid update cycle model FDP

The RUC and times D0H23 D0H23 VALID TIME D3H11 BASE TIME D1H22 Daylight Daylight Possible ensemble applications? 10 output for wind, screen variables, precip, etc High frequency solar requires fast surface scheme (e.g.sunflux)

SUNFLUX: A fast surface radiation parameterization Zhian Sun's work Radiative transfer is expensive Hourly is 30% of model run time Clouds, SZA change but assumed constant SUNFLUX Fast but accurate calculation of surface irradiance Efficient enough to run every time step Accounts for cloud, SZA changes Could be implemented in APS3

Conclusions There is a scatter in the metrics with variations from site to site Different synoptics Cloud frequencies Cloud properties Accuracy of radiative transfer assumptions to different cloud regimes Aerosol not accounted for in model The comparisons all show a scatter in the metrics for individual sites Is that significant? If so which do we prefer?

Further work Extend evaluation to all operational models for all archive times Establish statistical significance for the different metrics Partition hourly results in terms of solar zenith angle and time of year (suggestion by John Boland) Implement fast surface radiation scheme to produce 10 minute forecasts in Model (SUNFLUX) Find more data for validation Global model Satellite derived fields

Lawrie Rikus Phone: 03 9669 4452 Email: l.rikus@bom.gov.au Web: www.bom.gov.au Thank you The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology