DNICast Direct Normal Irradiance Nowcasting methods for optimized operation of concentrating solar technologies THEME [ENERGY.2013.2.9.2] [Methods for the estimation of the Direct Normal Irradiation (DNI)] Grant agreement no: 608623 Deliverable Nr.: 4.1 Deliverable title: Validation of nowcasted spatial DNI maps Project coordinator: OME WP leader: DLR Name of the organization: Authors: P. Kuhn, S. Wilbert, C. Prahl, A. Kazantzidis, L. Ramirez, L. Zarzalejo, L. Vuilleumier, P. Blanc, R. Pitz-Paal Submission date: April, 14, 2017 Version Nr.: 2 Disclaimer: The information and views set out in this report are those of the author(s) and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.
ABSTRACT To enable efficient plant operation, Direct Normal Irradiance (DNI) irradiance maps in high spatial and temporal resolutions for up to 30 minutes ahead are of interest. Ground-based All Sky Imagers (ASI) can be used to detect, track and predict 3D positions of clouds possibly shading the plant. Within the framework of the DNICast project, an ASI-based nowcasting method was developed and applied to data measured on Plataforma Solar de Almería (PSA). The ASI derived nowcasts are validated in this report with special focus on the effects of temporal and spatial aggregation on the deviations found comparing the nowcasts to ground measurements. In most publications, nowcasted irradiance maps are only validated against singular ground measurements and commonly only for one-minute averages. These validation metrics are of minor relevance for Concentrating Solar Power (CSP) and most Concentrating Photovoltaics (CPV) plants as spatially aggregated irradiances (e.g. over a whole subfield) and temporally aggregated irradiance are more important. This is due to the thermal inertia of the heat transfer fluid and the interconnection between several CPV collectors. Therefore, spatial and temporal aggregation effects are investigated in this publication in detail. Keywords: Direct normal irradiance, forecasting, nowcasting, all sky imager, concentrating solar power technology, concentrating photovoltaic technology, validation ii
Table of Contents ABSTRACT... ii Table of Contents... 3 1. Validation of DNI nowcasting results... 4 1.1 Methodology... 4 1.2 Investigating the origins of deviations: Fixed lead time mode... 6 1.3 Focusing on industrial applications: Operator mode... 8 1.4 Considering deviations for solar fields... 10 1.5 Linking spatial and temporal aggregation... 12 1.6 Discussion of Results... 14 2. Conclusion... 15 3. References... 16 4. Appendix... 17 4.1 Effects of spatial aggregation (2015-09-18)... 17 4.2 Effects of spatial and temporal aggregation (2015-09-18)... 18 3
1. Validation of DNI nowcasting results The following chapters introduce the validation methodologies and the benchmarking framework, which is used to benchmark and compare different nowcasting approaches. In Kuhn et al., 2016 and Schüler et al., 2016, a first validation of the nowcasting system described in Blanc et al., 2016, and developed by University of Patras and MINES ParisTech was conducted. This validation is extended with especial focus on industrial applications. The validation is conducted on nine days (2015-09-15, 2015-09-18, 2015-09-19, 2015-10-01, 2015-10-02, 2015-10-04, 2015-10-17, 2015-10-18, 2015-11-22) and presented here considering four different approaches. The first and the second approach called fixed lead time mode and operator mode investigate the effects of temporal aggregations on the deviations. Temporal aggregation is inherent within CSP (Concentrating Solar Power) plants due to thermal inertia of the heat transfer fluid. In Concentrating Photovoltaics (CPV) plants batteries might be used which also correspond to temporal aggregation. Thus, it is important to understand how accurate nowcasts are for relevant temporal aggregations as this behavior defines the required size of e.g. steam buffers or batteries. The third approach considers the effects of spatial aggregations on the deviations found for the nowcasts. This is of special interest for CSP and CPV plants as the spatially aggregated irradiance over subfields is more relevant for operations than singular point measurements/predictions. The fourth approach links the effects of spatial and temporal aggregation. Considering these two effects, which are present within CSP and CPV plants, the deviations of the nowcasting system drop significantly. 1.1 Methodology In sections 1.2 and 1.3, measurements from pyrheliometers are compared to DNI values of the corresponding pixels in the nowcasts for different lead times and time averages. Bias, relative bias, mean absolute error (MAE), relative MAE, root mean square error (RMSE), relative RMSE, standard deviation (std) and relative std are calculated for each station and each day. The formulas are briefly explained in the following with o i as the DNI values from the observations in the reference data set and p i as the predicted DNI values from the nowcasted data set at timestamp i. N equals the total number of timestamps included in the evaluated validation interval. Relative values are only derived for single days and O m is set to the daily mean observed DNI (see Gueymard, 2014). The bias is the arithmetic average of the values of the differences: 4
N 1 bias p i o i N i 1 The relative bias is given by: N 100 relative bias pi oi N Om i 1 The Mean Absolute Error (MAE) is: N 1 MAE p i o i N i 1 The relative MAE is defined as follows: N 100 relative MAE pi oi N Om i 1 The Root Mean Square Error (RMSE) is N ( ) 2 RMSE pi oi / N i 1 The relative RMSE is defined as follows: 100 relative RMSE O m The standard deviation (std) is: std 1 N 1 N i 1 N i 1 1 ( pi oi ) N The relative std is defined as follows: 1/ 2 ) 2 ( pi oi / N N j 1 ( p std relative std 100 O m j 1/ 2 2 o ) j For sections 1.4 and 1.5, a novel shadow camera system (Kuhn et. al., 2017) is applied to derive spatially resolved reference irradiance maps. The shadow camera system has six cameras taking photos of the ground from the top of an 87 m high solar tower (CESA1). Six concurrent images are combined to one so-called orthoimage, which is a geometrically corrected image (Figure 1a). Using the orthoimage of the investigated timestamp and two reference orthoimages, taken when no shadow fell on the PSA and when the PSA was completely shaded, as well as corresponding DNI measurements, a spatially resolved DNI map is calculated. An example of an orthoimage for Sept. 19th 2015, 12:57 UTC+1, is shown in Figure 1a. Shadows are clearly visible. The positions of the radiometers and cameras are marked. The derived reference DNI map for this timestamp is shown in Figure 1b. A detailed validation and further explanations of the reference system are given in Kuhn et al., 2017. With the spatially resolved reference irradiance maps provided by the shadow camera system and the spatially resolved predicted irradiance 1/2 5
maps of a nowcasting system, the deviations of the nowcasts can be studied for various field sizes. As the average irradiance on the whole solar power plant is the most relevant figure for plant operations, understanding spatial aggregation effects is crucial. Figure 1. Instrument setup at PSA. Black pixels are excluded from the evaluation. Red circles indicate the position of the ASIs. The red square is at the position of all six shadow cameras. The green stars represent two-axis trackers with pyrheliometers and pyranometers. The blue circles mark Si-pyranometers. Reference DNI map generated from the image on the left, reference images and ground data. 1.2 Investigating the origins of deviations: Fixed lead time mode ASI based nowcasting systems consist of several algorithmic modules conducting different operations: For instance, clouds must be detected in the raw images, their forms and heights must be derived and their movements must be predicted. All these modules can add deviations. To distinguish the effects caused by the singular modules, the so called fixed lead time mode is applied. For the fixed lead time mode, in order to look at the effects of cloud tracking, all nowcasted irradiance maps which are temporally averaged have the same lead time. For instance, considering a lead time of 20 minutes and a time average of 10 minutes at timestamp T, the temporal average is taken from nowcasts having the lead time of 20 min up to 10 min prior to timestamp T. Figure 2 to Figure 5 display the results of the mean of the nine daily error metrics (RMSE, MAE, std and bias) as calculated with the fixed lead time mode. For lead times of 0, 5, 10, 15, 20 and 30 minutes, temporal aggregation is conducted with time averages between 1 and 15 minutes. The operator mode, which will be presented in section 1.3, calculates the time averages differently and in a way more relevant to plant operations. 6
Figure 2. Mean of the daily RMSE for DNI predictions for nine days with different weather situations derived with the fixed lead time mode. Mean of the daily relative RMSE on nine days. Figure 3. Mean of the daily MAE for DNI predictions for nine days with different weather situations evaluated with the fixed lead time mode. Mean of the daily relative MAE on nine days. Figure 4. Mean of the daily std for DNI predictions for nine days with different weather situations evaluated with the fixed lead time mode. Mean of the daily relative std on nine days. 7
Figure 5. Mean of the daily bias for DNI predictions for nine days with different weather situations derived with the fixed lead time mode. Mean of the daily relative bias on nine days. As can be seen in Figure 2-Figure 5, the current situation (lead time zero) is predicted with small deviations from the ground measurements. The deviations increase with increasing lead times and are highest for lead time 30 min. For ASI based nowcasting systems, which rely on predicting the movements of clouds, 30 min lead time cannot be achieved with good accuracy in many situations. This is due to clouds at low altitudes and with high velocities exceeding the range of vision of the cameras for such long lead times. The effects of temporal aggregations are depicted on the y-axis in the figures. As expected, temporal aggregations generally reduce the deviations. However, contrary to this, the deviations for lead time 0 min are smaller for 7 min temporal averages than for 15 min temporal averages. This might be caused by taking the average over the daily deviations on days with various weather situations. The effects of temporal aggregation will also be investigated in the next section ( 1.3). 1.3 Focusing on industrial applications: Operator mode In section 1.2, the effects of temporal aggregation were investigated with time averages being built within nowcasts of the same lead time (fixed lead time mode). The fixed lead time mode was applied to separate the origins of deviations, but it is difficult to apply uncertainties derived from this mode in reality if temporal averages are of interest. For example, the operator of a solar plant wants to know the uncertainty of the nowcasted DNI average for the upcoming 5 minutes. This information is provided in the so-called operator mode. For the operator mode, the deviations are averaged between different lead times for each timestamp. The deviations found e.g. for the next 20 min ahead are averaged amongst the four current predictions for lead times for 8
0 min, 5 min, 10 min, 15 min and 20 min for each prediction timestamp. Thus the operator mode analyses the accuracy of the temporal averages DNI that are predicted from now up to a given lead time. The current situations (lead time 0 min) are not averaged. This averaging represents the situation in a solar power plant if the operators are interested in the general behavior of the average irradiance for the next minutes ahead. The deviation found for this operator mode are presented in Figure 6. Comparing Figure 6 and Figure 7 depicts the effects of temporal aggregation. In Figure 7 (published in Kuhn et al., 2016), the nowcasted DNI maps are compared to ground measurements for one-minute averages without additional temporal averaging. In Figure 6, time averages are calculated with the operator mode. Without temporal aggregations, the deviations are significantly larger. Figure 6. Mean of the absolute daily error metrics of DNI predictions on nine days with different weather situations calculated with the operator mode. Mean of the relative daily error metrics on nine days. Figure 7. Deviations (RMSE, MAE) found for one-minute averages of the nowcasting system (ASI) in comparison to persistence forecasts on nine days. Corresponding standard deviation and bias. Further explanations are given in Kuhn et al., 2016. 9
1.4 Considering deviations for solar fields For solar power plants, the most important piece of information is the irradiance on the whole field. In sharp contrast to this, most ASI derived nowcasts are validated only against singular ground measurements. This way, the effects of spatial aggregation are not considered and must be estimated. In this section, the effects of spatial aggregation on the deviations found for the ASI derived nowcasts are investigated. The effects are studied via a novel shadow camera system, which provides spatially resolved irradiance maps. The system is briefly explained in section 1.1 and validated in Kuhn et al., 2017. Previously, three field sizes relevant for solar power plants were considered (Kuhn et al., 2016). In this publication, various field sizes are considered. The validation is conducted on two exemplary days (see following figures and section 4.1 in the appendix). Preliminary validations were presented in Kuhn et al., 2016. In Figure 8-Figure 11, the effects of spatial aggregation are displayed for one-minute temporal averages. The joined effects of both spatial and temporal aggregation are displayed in section 1.5. As can be seen, spatial aggregation significantly reduces deviations. In industry, spatial aggregation occurs e.g. in the subfields of a parabolic trough solar power plant. Figure 8. RMSE found on 2015-09-19 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages. Relative RMSE found on 2015-09-19. 10
Figure 9. MAE found on 2015-09-19 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages. Relative MAE found on 2015-09-19. Figure 10. Standard deviations found on 2015-09-19 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and oneminute averages. Relative std found on 2015-09-19. Figure 11. Bias found on 2015-09-19 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages. Relative bias found on 2015-09-19. 11
1.5 Linking spatial and temporal aggregation In industrial CSP and CPV solar power plants, both temporal and spatial aggregations are in effect. Temporal aggregation is related to the thermal inertia of the heat transfer fluid or batteries and can be increased by adding buffers. Spatial aggregation occurs e.g. in the in the subfields of a parabolic trough plant. Having discussed both the effects of temporal or spatial aggregations separately above, we will look in this section at the effects of both temporal and spatial aggregation in this section. For an example day (2015-09-19, another day is presented in section 4.2 in the appendix) and various spatially aggregated field sizes, the effects of spatial and temporal aggregation on RMSE are displayed in Figure 12 for time averages derived from the operator mode (see section 1.3). Figure 13 depicts the corresponding MAE deviations. Standard deviations and bias are shown in Figure 14 and Figure 15. In comparison to Figure 8 and Figure 9, the combined effects of temporal and spatial aggregations further reduce the deviations. Figure 12. RMSE found on 2015-09-19 for averages over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative RMSE found on 2015-09-19. 12
Figure 13. Relative MAE found on 2015-09-19 for averages over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative MAE found on 2015-09-19. Figure 14. Standard deviations found on 2015-09-19 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative std found on 2015-09-19. Figure 15. Bias found on 2015-09-19 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative bias found on 2015-09-19. 13
1.6 Discussion of Results The results are in alignment with publications on other ASI systems: RMSE values between 155 W/m 2 and 200 W/m 2 for 10 min GHI forecasts are reported in Bernecker et al., 2014 and RMSE values also for GHI forecasts up to 250 W/m 2 (depending on specific weather situations) for 23 min lead time in Schmidt et al., 2016. Note that these values hold for GHI. A hybrid ASI/stochastic learning approach presented in Chu et al., 2013, achieved RMSE values between 55 W/m 2 and 140 W/m 2 for DNI forecasts and a lead time of 10 min. In Fu et al., 2013, RMSE values for DNI forecasts between 169 W/m 2 for 5 min lead time and 191 W/m 2 for 15 min lead time as well as MAE values between 139 W/m 2 (5 min) and 152 W/m 2 (15 min) are demonstrated. As the specifics of the selected days and the atmospheric conditions of the validation site heavily influence the error metrics, a large variety of conditions or the same data set must be included for detailed comparisons of these error metrics. To the best of our knowledge, no error metrics are provided in the literature for other ASI nowcasting systems for spatially and temporally averaged data. The temporal and spatial averaging must be considered for a comparison among ASI systems and also when ASI systems are compared to satellite or NWP based nowcasts. Considering the effects of spatial aggregation, we found that the deviations drop sharply to 20.8 % RMSE and 13.6 % MAE for lead time 0 min, oneminute temporal averages and spatially aggregated field sizes of 4 km 2. Temporal aggregation (compare Figure 6 and Figure 7) also significantly reduces deviations. As within CSP and CPV plants, both temporal and spatial aggregations effects occur, the deviations found for the DNICast nowcasting system relevant for industrial applications are encouragingly low. Therefore, although the results are far from being perfect, the validations of the DNICast nowcasting system revealed promising potential. 14
2. Conclusion The methodology to validate and benchmark nowcasting systems providing DNI maps with high temporal and spatial resolutions has been developed, significantly enhanced and presented using an exemplary data set. The nowcasts deviate from the reference system similarly to previous findings reported in the literature. Previous nowcasting systems were only evaluated using ground measurement stations. Nevertheless, comparing nowcasts to singular radiometers is not relevant for industrial applications without taking the spatio-temporal aggregation effects into account. To the best of our knowledge, this is the first time that a detailed study of both temporal and spatial aggregation effects on the accuracy of ASI derived nowcasting systems is published. Results show a significant reduction in the deviations for both temporal and spatial aggregations. In CSP plants and CPV plants with batteries, adapted temporal and spatial aggregations take place, leading to accurate forecasts. These forecasts can then be used to optimize plant operations, especially for variable (partially cloudy) situations. The optimum ASI based nowcasting configuration has to be further investigated. Different algorithmic approaches, the optimal number and relative positions of ASI cameras and the benefits of additional sensors such as ceilometers are currently benchmarked. At PSA, a benchmarking framework for nowcasting systems is operational. With this reference system, large-scale comparisons for different nowcasting configurations can be performed. This way, the development of the optimum nowcasting system is supported. Optimized nowcasting systems support both the integration of solar power into electricity grids and operations inside solar power plants. 15
3. References Bernecker, D.; Riess, C.; Angelopoulou, E.; Hornegger, J.; Continuous shortterm irradiance forecasts using sky images, Solar Energy, 110, 303 315 (2014). Blanc, P.; Massip, P.; Kazantzidis, A.; Tzoumanikas, P.; Kuhn, P.; Wilbert, S.; Schüler, D.; Prahl, C.; Short-term forecasting of high resolution local DNI maps with multiple fish-eye cameras in stereoscopic mode, SolarPACES conference, 2016; Accepted for AIP Conference Proceedings, 2017. Chu, Y.; Pedro, H. T.C.; Coimbra, C. F.M.; Hybrid intra-hour DNI forecasts with sky image processing enhanced by stochastic learning, Solar Energy, Volume 98, Part C, December 2013, Pages 592-603, ISSN 0038-092X. Fu, C.-L.; Cheng, H.-Y.; "Predicting solar irradiance with all-sky image features via regression", Solar Energy 97 (2013): 537-550. Gueymard, C.; A review of validation methodologies and statistical performance indicators for modelled solar radiation data: Towards a better bankability of solar projects, Renewable and Sustainable Energy Reviews, 39 (2014), 1024-1034. Kuhn, P.; Wilbert, S.; Schüler, D.; Prahl, C.; Haase, T.; Ramirez, L.; Zarzalejo, L.; Meyer, A.; Vuilleumier, L.; Blanc, P.; Dubrana, J.; Kazantzidis, A.; Schroedter-Homscheidt, M.; Hirsch, T.; Pitz-Paal, R.; Validation of Spatially Resolved All Sky Imager Derived DNI Nowcasts, SolarPACES conference, 2016; Also accepted for AIP Conference Proceedings, 2017. Kuhn, P.; Wilbert, S.; Prahl, C.; Schüler, D.; Haase, T.; Hirsch, T.; Wittmann, M.; Pitz-Paal, R., Blanc, P.; Ramirez, L.; Zarzalejo, L.; Meyer, A.; Vuilleumier, L.; Shadow camera system for the generation of solar irradiance maps, Publication accepted with minor revisions, Journal of Solar Energy, April 2017. Schmidt T.; Kalisch, J.; Lorenz, E.; Heinemann, D.; Evaluating the Spatio- Temporal Performance of Sky Imager Based Solar Irradiance Analysis and Forecasts, 2016, Atmospheric Chemistry and Physics, 16, 3399-3412. Schüler, D.; Kuhn, P.; Wilbert, S.; Prahl, C.; Kazantzidis, A.; Meyer, A.; Ramirez, L.; Zarzalejo, L.; Hirsch, T.; Pitz Paal, R.; Report on the operation of sky imagers and preliminary validation of results including method uncertainties, DNICast report Deliverable 3.4, 2016. 16
4. Appendix 4.1 Effects of spatial aggregation (2015-09-18) Besides 2015-09-19, a day with many transient clouds, which is presented in section 1.4, the deviations found for 2015-09-18 are displayed here. 2015-09-18 is a day with very little transient clouds. Such days are not too challenging for ASI based nowcasting systems. Thus, the absolute and relative deviations are far smaller than for 2015-09-19. The effects of spatial aggregation are visible in Figure 16 to Figure 19 from an evaluation in the fixed lead time mode, but due to the generally small deviations, the expected increase of the errors for longer lead times is not prominent. The same holds for the common rise in the deviations with longer lead times: As the overall errors are very small, this effect is only partially visible. Figure 16. RMSE found on 2015-09-18 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages evaluated in fixed lead time mode. Relative RMSE found on 2015-09-18. Figure 17. MAE found on 2015-09-18 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages fixed lead time mode. Relative MAE found on 2015-09-18. 17
Figure 18. Standard deviations found on 2015-09-18 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and oneminute averages. Relative std found on 2015-09-18. Figure 19. Bias found on 2015-09-18 for lead times of 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and one-minute averages. Relative bias found on 2015-09-18. 4.2 Effects of spatial and temporal aggregation (2015-09-18) As 2015-09-19 is presented for both spatial and temporal aggregations (see section 1.5), 2015-09-18, a day with few transient clouds, is shortly presented in this section. In Figure 20, Figure 21, Figure 22 and Figure 23 the deviations are depicted, which are calculated with the operator mode (see section 1.3). Due to the generally small deviations on this day, both temporal and spatial aggregations do not significantly improve the results. 18
Figure 20. RMSE found on 2015-09-18 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative RMSE found on 2015-09-18. Figure 21. MAE found on 2015-09-18 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative MAE found on 2015-09-18. Figure 22. Standard deviations found on 2015-09-18 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative std found on 2015-09-18. 19
Figure 23. Bias found on 2015-09-18 for the average over the next 0 min, 5 min, 10 min, 15 min, 20 min and 30 min for various spatially aggregated field sizes and time averages of the operator mode. Relative bias found on 2015-09-18. 20