DNICast Direct Normal Irradiance Nowcasting Methods for Optimized Operation of Concentrating Solar Technologies

Transcription

1 DNICast Direct Normal Irradiance Nowcasting Methods for Optimized Operation of Concentrating Solar Technologies THEME [ENERGY ] [Methods for the estimation of the Direct Normal Irradiation (DNI)] Grant agreement no: Deliverable Nr.: 3.6 Project coordinator: OME Name of the organization: Submission date: Deliverable title: Report on satellite-based nowcasting methods WP leader: DLR Authors: T. Landelius, M. Lindskog, H. Körnich, S. Müller, T. Sirch, M. Schroedter- Homscheidt Version nr.: Disclaimer: The information and views set out in this report are those of the author(s) and do not necessarily reflect the official opinion of the European Union. Neither the European Union institutions and bodies nor any person acting on their behalf may be held responsible for the use which may be made of the information contained therein.

2 Abstract The development of nowcasting methods based on satellite imagery is reported here. As the field of view of ground based systems is limited to a few kilometers, only a short-term forecast of up to 30 minutes is possible, depending on the cloud velocity. Satellite imagery based nowcasting methods allow the extension of the forecast window to several hours. For the nowcasting, the instrument SEVIRI from Meteosat Second Generation (MSG) is utilized. Different centers have developed different methods for Direct Normal Irradiance (DNI) nowcasting. Meteotest determines the Clearness-Index (CI) from MSG and advects the information with winds from a Numerical Weather Prediction model at a single height. Different conversions from CI to DNI are tested with different background aerosol estimates. Local observations of DNI, if available, are used to adjust the final forecast. DLR-PA employs two algorithms to separate optical thickness for ice and water clouds from MSG. The advection velocity is determined with an optical flow method from consecutive images. The development of convective cells is treated separately. DNI is then calculated directly from the optical thickness. Using MSG Rapid-Scan-Modus it was demonstrated that the more frequent update improves the DNI nowcasting, while the advection velocity shows no clear improvement by the more frequent sampling. DLR-DFD introduces a receptor-like approach, where the derived optical thickness for ice and water/mixed phase clouds from MSG is analyzed in several sectors around a point of interest. The advection velocity is determined from consecutive images. Furthermore, in order to assess variations of DNI, new variability classes for DNI were defined in dependence of the mean DNI and of the number of direction changes in DNI within the hour. The classification was derived for ground-based station data, but it is also applied to satellite data. In order to utilize MSG/SEVIRI in numerical weather prediction, SMHI has implemented two different aspects of the satellite information in the data assimilation scheme of its forecasting model. Firstly, the radiances of two infrared channels of SEVIRI are assimilated for cloud-free conditions. Secondly, the cloud mask together with cloud top temperature and cloud base height are calculated using the EUMETSAT Satellite Application Facility to NoWCasting & Very Short Range Forecasting (SAFNWC) MSG software. This cloud information is then used for an innovative cloud initialization procedure. The impact of these aspects will be assessed in sensitivity experiments. ii

3 Keywords: Nowcasting, Satellite, DNI, Data Assimilation iii

4 Table of Contents Abstract... ii Table of Contents... 4 List of Figures Introduction Nowcasting methods based on satellite data Meteotest method using GFS based Weather Research and Forecasting model for wind fields DLR-PA method using Meteosat Rapid-Scan-Modus HRV channel Cloud Remote Sensing with MSG/SEVIRI Forecast Algorithm Ideal Exploitation of the temporal resolution of SEVIRI Ideal Exploitation of the spatial resolution of SEVIRI DLR-DFD method using a sectoral method based on Meteosat Second Generation imagery Method for cloud mask nowcasting Method of DNI nowcasting Method of DNI variability nowcasting Satellite radiances and NWC SAF products for data assimilation The HARMONIE system for Numerical Weather Prediction Assimilation of SEVIRI radiances Cloud initialization with MSG data References Appendix Nomenclature

5 5

6 List of Figures Figure 1: MSG SEVIRI Instrument channels Of these 12 spectral channels, 11 provide measurements with a resolution of 3 km at the subsatellite point with the 12th, the High Resolution Visible (HRV) channel, providing measurements with a resolution of 1 km Figure 2: Forecast scheme of the Meteotest satellite based nowcasting method. Clearness index maps are derived from satellite images and each pixel is propagated forward by wind vectors from a numerical weather model. Measurements are included in the post processing step to account for offsets Figure 3: Domain area of Meteotest s operational WRF data used in this work Figure 4: Example of the U component of the wind field. GFS model data are used outside the WRF area. The blue dots represent the forecast locations Figure 5: Scheme of the image trajectory movement. Each pixel is propagated along the vector components u and v. In a second step the pixels are interpolated from the new locations to the original grid Figure 6: From the initial satellite image (left side) to a set of forecasted CI maps in 5 minute time steps Figure 7: Direct CI-to-DNI conversion. Relationship of DNI and CI during the two test periods from data from the PSA (left and center) and the transfer function used in the calculation (right) Figure 8: Images A and B (a,b) display a pair of squares. (c) The final disparity vector field V is plotted on A' = A( P - V ) with (d) the remaining difference field A' -B after processing on all pyramid levels Figure 9: Ice optical thickness derived by COCS for the real situation (middle) compared to the object-based forecast (left) and the forecast with a decrease in optical thickness for decaying cells (right) Figure 10: Ice optical thickness derived by COCS for 7 April 2013, (top left) 13:00 UTC and (top right) 13:15 UTC with the calculated disparity vector field. A 1h-forecast is created (bottom left) and can be compared to the real scene for this time (bottom right) Figure 11: Calculated direct normal irradiance in W/m2 for 7 April 2013, 13:

7 Figure 12: Comparison of the forecast quality for 4 different time intervals and the persistence method Figure 13: Cloud masks for low (left) and high (right) resolution Figure 14: Nowcasting scheme as applied to APOLLO cloud masks red colored clouds are tracked as they are those arriving at the power plant location Figure 15: Sectoral approach with a cloud mask (blue, orange, red depending on cloud height) and cloud free areas (green) distributed over an area of 150 x 150 km Figure 16: Evaluation of all sectors color lines show the existence of clouds along the sector lines. Relevant sectors are marked with orange circles Figure 17: Temporal variability scales as investigated in existing studies Figure 18: Density scatterplots of variability indices for GHI after Skartveit, Stein and Coimbra for the station BSRN Carpentras and daytime hours on variable days in 2012 (Skartveit vs. Stein upper left, Coimbra vs. Stein upper right, and Coimbra vs. Skartveit lower left, color bar indicates number of cases) Figure 19: Density scatterplots of variability indices for DNI after Skartveit, Stein and Coimbra for the station BSRN Carpentras and daytime hours on variable days in 2012 (Skartveit vs. Stein upper left, Coimbra vs. Stein upper right, and Coimbra vs. Skartveit lower left, color bar indicates number of cases) Figure 20: Arbitrarily chosen examples of the variability cloud classes 1 to 8. Hours being classified in one of the classes are marked by a red box. For some classes, the red box extends the range of a single hour and illustrates several hours being included in the reference database. Minute values (yellow), 10 min moving averages (black) and McClear clear sky values (thin) are given Figure 21: Typical box dimensions of different cloud masks, source: S. Glas (2014) Figure 22: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 1 related cases in the reference data base Figure 23: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 2 related cases in the reference data base

8 Figure 24: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 3 related cases in the reference data base Figure 25: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 4 related cases in the reference data base Figure 26: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 5 related cases in the reference data base Figure 27: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 6 related cases in the reference data base Figure 28: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 7 related cases in the reference data base Figure 29: Box-Whisker plot for the distribution of normalized satellitebased cloud parameters as found for class 8 related cases in the reference data base Figure 30: Median values of each satellite-based cloud parameter for all classes Figure 31: Comparison of automatic classification versus visual classification results for 2012, BSRN Carpentras, and hourly variability classes 1 to Figure 32: Observed SEVIRI wv73 brightness temperatures (left,unit:k) for 2013/09/30, 00 UTC. Corresponding observation minus HARMONIE model state equivalents (right, unit K) Figure 33: Vertical profiles of humidity (left) and temperature (right) valid at 2013/03/20, 12 UTC in the grid-point (2 o E, 38 o N). Red curves are before MSG cloud-based initialization and blue curves after. Also marked are the saturation humidity profiles (labeled saturated) and the humidity profiles to which saturation is decreased when cloud free (labeled saturated*ccc)

9 1. Introduction The purpose of DNICast is to improve the use of short term forecasts (commonly called nowcasts) of direct normal irradiance (DNI) from a combination of numerical forecast products and observations. Reliable forecasts of DNI can increase efficiency of concentrating solar technologies (CST) if used as a decision-making tool in operation management. Figure 1: MSG SEVIRI Instrument channels Of these 12 spectral channels, 11 provide measurements with a resolution of 3 km at the sub-satellite point with the 12th, the High Resolution Visible (HRV) channel, providing measurements with a resolution of 1 km. The development of nowcasting methods based on satellite imagery is described in this report. As the field of view of ground based systems is limited to a few kilometers, only a short-term forecast of up to 30 minutes is possible, depending on the cloud velocity. Satellite imagery based nowcasting methods allow the extension of the forecast window to several hours. 9

10 Meteosat Second Generation (MSG) is a series of geostationary satellites operated by EUMETSAT. Their primary mission is the continuous observation of the earth s full disk for the observation and forecasting of weather phenomena. For this purpose the 12-channel imager SEVIRI (Spinning Enhanced Visible and Infrared Imager) has been developed. Additionally a broadband high resolution visible (HRV) channel is integrated, which covers only a part of the earth s full disk with a higher spatial resolution of 1 km at the sub-satellite point. The repetition rate is usually 15 min, but can be reduced to 5 min in rapid-scan-mode for a part of the disk. An example for the output from MSG SEVIRI is displayed in Figure 1. This information is then utilized for nowcasting. In section 2, three methods for satellite-based DNI nowcasting are presented. The Meteotest method is described in section 2.1. Here a Clearness-Index (CI) is derived from MSG and the information is advected with winds from a Numerical Weather Prediction model at a single height. DLR-PA employs two algorithms to separate optical thickness for ice and water clouds from MSG. The advection velocity is determined with an optical flow method from consecutive images. This method is described in section 2.2. In section 2.3, the method by DLR-DFD is introduced. It is a receptor-like approach, where the derived optical thickness for ice and water/mixed phase clouds from MSG is analyzed in several sectors around a point of interest and the advection velocity is determined from consecutive images. In order to utilize MSG/SEVIRI in numerical weather prediction, SMHI has implemented two different aspects of the satellite information in the data assimilation scheme of its forecasting model. These two methods are described in section 3. Firstly, the radiances of two infrared channels of SEVIRI are assimilated for cloud-free conditions. Secondly, the cloud mask together with cloud top temperature and cloud base height are calculated using the EUMETSAT Satellite Application Facility to NoWCasting & Very Short Range Forecasting (SAFNWC) MSG software. 2. Nowcasting methods based on satellite data 2.1 Meteotest method using GFS based Weather Research and Forecasting model for wind fields The two main input sources of the nowcasting method used by Meteotest are satellite images from MSG/SEVIRI with 15 minute time resolution and numerical weather prediction (NWP) models. Satellite images were processed to a clearness index (CI) map. From the NWP model the wind vector field is taken and the CI map pixels are propagated forward along the 10

11 wind trajectories for the next 4 hours. Ground measurements are included for post processing. The forecast scheme is visualized in Figure 2: Figure 2: Forecast scheme of the Meteotest satellite based nowcasting method. Clearness index maps are derived from satellite images and each pixel is propagated forward by wind vectors from a numerical weather model. Measurements are included in the post processing step to account for offsets. Satellite image processing to get the CI map is done using the empirical Heliosat 2 method (Rigollier et al., 2004). CI is used as the division of global horizontal radiation and clearsky global radiation. We use the HRV channel for the cloud detection with 1 km resolution at the sub-satellite point and around 3 km in the Mediterranean region. Furthermore a combination of the visible and infrared channels (VIS06, IR16, IR108) is used for the detection of snow cover at the ground. This implies clear sky conditions for satellite image pixels when snow cover is detected. For the NWP wind field we use the WRF output, from the operational WRF run at Meteotest, with GFS as boundary conditions every 6 hours. The outermost domain (domain 1) covers Western and Southern Europe as well as the Northern part of Africa (see Figure 3:). Details of the WRF configuration are given in the table below. Outside the WRF domain GFS output with 0.5 degree resolution is used directly. 11

12 Parameter Value WRF Version ARW core Resolution Domain 1: 15 km MP Physics Lin et al. Scheme PBL Physics Yonsei University Scheme (YSU) CU Physics Kain Fritsch Scheme RA LW Physics RRTM Longwave Scheme RA SW Physics Dudhia Shortwave Scheme Figure 3: Domain area of Meteotest s operational WRF data used in this work. Wind vectors given by its u and v components are taken from the NWP models WRF and GFS (see example in Figure 4:) at an altitude level of 4 km to calculate trajectories for each satellite image pixel with time integration steps of 5 minutes. The fix altitude level of 4 km for wind vector calculations was chosen based on previous tests by groups of Meteotest and University of Oldenburg (personal communication). Mixing different altitude levels to account for multiple cloud layers as well as different fixed heights have been evaluated. Optimal results were obtained using fixed altitude levels of 3-4 km above sea level. Each image pixel is propagated forward along the trajectory lines (Figure 5:). To avoid empty gaps in the moved image, we apply an interpolation on the original grid. This produces a set of forecasted CI maps in 5 minute interval (Figure 6:). Cloud formation and cloud 12

13 dissolving is not considered. The site specific forecast is then extracted from that set of forecast maps. Figure 4: Example of the U component of the wind field. GFS model data are used outside the WRF area. The blue dots represent the forecast locations. Figure 5: Scheme of the image trajectory movement. Each pixel is propagated along the vector components u and v. In a second step the pixels are interpolated from the new locations to the original grid. 13

14 Figure 6: From the initial satellite image (left side) to a set of forecasted CI maps in 5 minute time steps. DNI is calculated based on CI. For this step, we tested three approaches. These were using either a direct CI-to-DNI conversion or an indirect CI-to- DNI conversion method, different clear sky models and different aerosol sources. The direct CI-to-DNI conversion follows the formulation of (Hammer, 2009) where DNI is calculated as a function of CI and direct beam radiation under clear sky conditions (Figure 7:). With the indirect CI-to-DNI conversion, we calculate GHI first, followed by splitting GHI into the diffuse and direct beam radiation component using the BRL model (Ridley et al., 2010). To model the clear sky conditions two clear sky models were tested. One is the clear sky model of the European Solar Radiation Atlas ESRA (Rigollier et al., 2000) and the second is the simplified version of the SOLIS clear sky model from (Ineichen, 2008). Aerosols are a main component for modeling clear sky radiation. Two sources were evaluated. A climatology of Aerosol Optical Depth (AOD) and Linke turbidity (Remund et al., 2015) and AOD from the MACC (Monitoring Atmospheric Composition & Climate) project ( 14

15 Figure 7: Direct CI-to-DNI conversion. Relationship of DNI and CI during the two test periods from data from the PSA (left and center) and the transfer function used in the calculation (right). With the different models and data sources from the list above, three versions of model chains were defined (see list below). Nr DNI method Aerosol Clear sky model M1 Indirect, BRL Climatology (Remund) ESRA M2 Direct, Hammer Climatology (Remund) ESRA M3 Direct, Hammer MACC Simple SOLIS If local DNI measurements are available, they can be included in the post processing scheme and the actual forecast is adapted. We used a simple offset correction, which can be induced by e.g. incorrect aerosol input (Müller et al., 2013). The post processing uses the last hour of measurements and the starting points of the last forecasts. If the forecasts are constantly lower or constantly higher than the measurements during that past hour, then the average difference in CI is calculated and added to the actual forecast. The adaption is not applied in case of variable conditions. 2.2 DLR-PA method using Meteosat Rapid-Scan-Modus HRV channel To overcome the limitations of current nowcasting methods using satellite data, which mostly evaluate cloud motion without discrimination of cloud type or cloud height based on a low image repetition rate, the potential of the Meteosat Rapid-Scan-Modus (5 instead of 15 minutes repetition rate) in combination with the HRV-channel (High Resolution Visible, 1km instead of 3km maximum spatial resolution) will be investigated. Using different channels of the satellite data additional information on the cloud type and the corresponding attenuation of the solar radiation can be provided. The work will be based on (Breitkreuz, 2009). A method using satellite data to provide forecasts of cloudiness or DNI maps up to several hours ahead is 15

16 presented to allow for the consideration of differential cloud motion in separate cloud layers which improves the nowcast capabilities considerably Cloud Remote Sensing with MSG/SEVIRI Meteosat Second Generation (MSG) carries the 12-channel imager SEVIRI (Spinning Enhanced Visible and Infrared Imager) and a broadband high resolution visible (HRV) channel that provide continuous observation of the Earth s full disk. Optical, micro- and macrophysical properties of clouds are important parameters for the modelling of radiation-cloud interactions. Thus, their determination plays an important role for the computation of surface radiation. The algorithms used in this work are presented in the following sections COCS For ice clouds "The Cirrus Optical properties derived from CALIOP and SEVIRI during day and night" (COCS, Kox et al., 2014) algorithm is used. It is based on the depolarization-lidar CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) aboard the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) mission and on SEVIRI aboard MSG. Utilizing a backpropagation neural network, which is trained with collocated measurements of CALIOP and SEVIRI, the algorithm provides optical thickness and cloud top height for ice clouds. In a validation study against airborne High Spectral Resolution Lidar measurements, COCS detected 80% of the cirrus clouds with optical thickness 0.2 and its detection efficiency increased for higher optical thicknesses. For optical thickness 0.1 COCS detected already 50% of the cirrus clouds. The false alarm rate amounted to 2.6% for all measured cirrus clouds. It is very robust for small optical thicknesses above 0.1 up to a maximum value of 2.5, because of the failing ability of CALIOP to penetrate through deeper cloud layers (Winker et al., 2010). As COCS works with the thermal SEVIRI channels it can be applied during day and night. Furthermore COCS provides information only about the highest ice cloud layer APICS For water clouds the APICS ("Algorithm for the Physical Investigation of Clouds with SEVIRI", Bugliaro et al., 2011) cloud retrieval is used. The cloud detection is based on two groups of threshold tests consisting in reflectance tests and spatial coherence tests applied to the solar SEVIRI channels. A pixel is cloudy if at least one test gives a positive result. 16

17 In a second step, cloud optical thickness and effective radius are derived from two solar channels based on the method by Nakajima and King (1990) and Nakajima and Nakajima (1995). The SEVIRI channels centered at 0.6 and 1.6 m are used: in the first spectral range clouds mainly scatter radiation, while in the second channel clouds both absorb and scatter the incoming solar radiation. Since absorption is a function of effective radius while scattering depends on optical thickness, a simultaneous retrieval of optical thickness and effective radius can be performed by minimizing the difference between measured and computed reflectivities. The look-up tables required for this application have been computed with the radiative transfer model libradtran (Mayer and Kylling, 2005) using a mid-latitude standard atmosphere (Anderson et al., 1986), a typical continental aerosol load (a rural type aerosol in the boundary layer, background aerosol above 2 km, spring-summer conditions and a visibility of 50 km, Shettle (1989)). Surface albedo is taken from the temporally appropriate white sky MODIS albedo product MCD43C1 (Schaaf et al., 2002). APICS is furthermore capable of detecting thick ice clouds and deriving their optical thickness by using the parameterization by Baum et al. (2005). The retrieval is not as accurate as COCS but essential for the determination of multi-phase clouds, which will be described in the following section Cb-TRAM Because of their rapid development convective clouds exhibit a large change in their properties in a short period of time, which increases the error in the forecast. Therefore it is reasonable to treat convective and advective clouds separately and to investigate the development of convective cells, which is a challenging task. The main difficulty lies in the small spatial and temporal scales as well as the dynamic and microphysical state of convective clouds. With a developing time of less than one hour and a spatial extent of only a few kilometers, it is very difficult for the models to predict the onset and path of individual thunderstorms. Similarly, satellite based nowcasting methods are not able to predict the location and time of convective initiation. However, nowcasting methods are better suited to forecast the development of convective clouds. For the detection of convective clouds, detection methods of the Cb-TRAM algorithm (CumulonimBus TRacking And Monitoring, Zinner et al. (2008, 2013)) have been examined, which divides convection into three stages: convection initiation (stage 1) rapid cooling (stage 2) mature thunderstorm cells (stage 3) 17

18 For the detection of convection initiation (stage 1), a combination of the evolution of the reflectance in the HRV and the cooling rate in the thermal spectral range is used. Stage 2 is issued for rapid vertical developments detectable in the water vapor channels at 6.2 μm. The detection of mature thunderstorm cells (stage 3) is bound to areas with a strong spatial roughness of the HRV, determined by the local standard deviation, combined with the brightness temperature difference of 6.2 μm and 10.8 μm (T(6.2) T(10.8) > 0K). This is only valid for daytime. During the night the reflectivity of the HRV is replaced by the brightness temperature at 6.2 μm. As for stage 1 and 2 the further growth of the cells is mainly characterized by vertical movement and microphysical processes, the forecast algorithm is not able to predict the development. Moreover, not all detected cells evolve to mature thunderstorms; this is especially valid for stage 1. So only stage 3 is applied in the algorithm Pyramidal Matcher The forecast rests upon an optical flow method determining a motion vector field from two consecutive images (Zinner et al., 2008)). Unlike feature based approaches this method is area based: instead of vectors only for interesting cloud patterns a disparity vector field V defined at each pixel position P is derived. Movements in the atmosphere take place on different scales reaching from microscale (few centimeters) to global scale (10,000 km). These large-scale flows overlay the smallscale movements, so that the determination of the disparity vector field for all scales is challenging. In order to take this into account the disparity vector fields are successively derived on different scales, starting from low resolution down to high resolution - a pyramidal scheme. The number of subsampling levels N can be varied according to the application. A detailed example is shown in Figure 8 for two images A and B (a,b) with a size of nx=100 x ny=100 pixels displaying two squares (10 x 10 pixels). Initially the size of the images is increased to a multiple of 2^N. Then the resolution is reduced to dimensions of a multiple of 2^N to allow for the stepwise processing. For a pyramid with 3 levels (N=2) we get dimensions of nx/4 x ny/4 (25 x 25 pixels) on the topmost level. The large scale disparity vector field V is determined by shifting every pixel P in image A to A' = A(P + K (i,j)) by K(i,j) = {i, j}, i,j ϵ {0,1,2}, in both dimensions and then comparing it to image B. In order to get the best fit (minimizing the difference between A' and B) two quality criteria can be applied: 18

19 the localized correlation coefficient is maximized the squared difference of the intensities in a surrounding of each pixel is minimized Which criterion is selected depends on the application. For details see Zinner et al. (2008). Figure 8: Images A and B (a,b) display a pair of squares. (c) The final disparity vector field V is plotted on A' = A( P - V ) with (d) the remaining difference field A' -B after processing on all pyramid levels. With the disparity vector field V the image A is warped to A' = A( P - V ) with the size of 25 x 25 pixel (nx/4 x ny/4) for this topmost level. A' is now replacing A and then used for the next level. Successively the refined disparity vector field V in all levels is determined and the resulting disparities from each level are added to V until the final disparity vector field V in full resolution is achieved, plotted on A and A' (Figure 8c). The displacement of image A onto B shows good results as the final remaining difference field (Figure 8d) A' - B exhibits only small differences on the edges of the squares caused by the Gaussian filter used for smoothing. 19

20 2.2.2 Forecast Algorithm Input Data For the application of the forecast algorithm on SEVIRI we use the optical thickness derived by COCS and APICS (Sect , ). There are two reasons for the utilization of the optical thickness: first, it is the quantity which is needed for the calculation of the DNI. Additionally the algorithm works better for a mask of one property than for a distribution of it over the whole scene. The derived optical thickness attains values from 0 to 100 for lower water clouds and from 0 to 2.5 for ice clouds. Since the forecast algorithm performs better for similar values for neighboring pixels due to technical reasons the optical thickness for water clouds is normalized to [0; 1]. To this end the derived optical thickness is divided by its maximum. As APICS works with solar channels, also thick ice clouds can be detected (Sect ) and in case of a thin cirrus cloud with an underlying water cloud, the signal of the water cloud leads to a positive result in APICS. Additionally the total optical thickness of multi-layer and multi-phase cloud systems like Cb (Cumulonimbus) can be derived, where COCS only assigns small optical thickness for the icy tops. Thus, a cloud top phase mask is produced, where clouds that are detected by APICS alone are considered to be liquid water clouds. But these clouds detected by both APICS and COCS are assumed to be ice clouds, as the algorithm has not the skill to determine if the detected signal comes from the ice cloud or a water cloud below. For this an approximation is needed to detect the cases of multi-phase clouds. For potential multi-layer clouds the cloud retrieval in APICS is done with the parameterization by Baum et al. (2005) and compared to COCS. For a derived optical thickness τ(apics) up to 2.5 the cloud phase is defined as ice. In case of thicker clouds the difference between τ(apics) and τ(cocs) is examined and pixels with τ(apics) - τ(cocs) > 2.3 are classified as multiphase clouds with COCS for the upper ice clouds and APICS for the clouds below. For pixels with τ(apics) - τ(cocs) < 2.3 the cloud phase is defined as ice with the optical thickness COCS. With this we get two cloud classes: 1) the ice clouds, detected by COCS, with an optical thickness from 0 to 2.5 and 2) the water clouds, with an optical thickness, derived by APICS, ranging from 0 to 100. This is a quite simple method with partially high errors in the optical thickness for water clouds. But as there is no information about the clouds below the ice clouds, the knowledge about the multi-phase clouds leads to 20

21 an improvement in cloud detection and in particular for the calculation of the DNI. Instead of an attenuation of the DNI only by ice clouds we have an additional reduction, which is especially important for thick clouds with zero DNI at ground level Adaption of Vector Field To get a forecast F image B is warped with the derived disparity vector field to F = B( P - V ) (1) as described in Sect For a time interval T = 15min between the two consecutive startup images a 15-min forecast is created, but also forecasts with other timesteps can be performed if required and reasonable. For this a disparity vector field V according to the length of the timestep S is needed, which means either the time interval T of the two startup images is equal to S or the disparity vector field V has to be multiplied by a factor d = S/T (2) For example if a forecast with a timestep of S = 5min is required and T = 15 min (repetition rate of MSG), the resulting disparity vector field V has to be multiplied by a factor d = 1/3 according to Eq. 2. As it is for the cloud detection, a separation of upper and lower clouds for the forecast is also necessary, since their motion vectors differ in most cases because of the varying dynamics in the atmospheric layers. In particular, the wind speed in the troposphere usually exhibits very strong variations with altitude. But this so-called pixel-based forecast is quite messy, because of small-scale motion vectors, which may differ widely in direction and absolute value for neighboring pixels. As a consequence, cloud patterns dissolve into small patches within a short period of time, which does not correspond to reality. The clouds are therefore classified as objects with similar properties (optical thickness) and all motion vectors within these objects are averaged. Hence the whole object is shifted with this mean motion vector. As the motion vectors are derived from the visible cloud movements, the disparity vector field in the area between the clouds is zero. In case that cloud objects move into these regions, they stop moving. Thus, a method must be developed to estimate the cloud motion vectors in these cloud free areas, which are in the following referred to as the background. To this end a weighted triangular interpolation of the motion vectors between the clouds is applied. 21

22 Intensity Correction for Decaying Convective Cells The atmosphere is a variable system and in a continuous progress of spatial and microphysical change, which should be considered in the forecast. The pyramidal matcher can only predict the movement of the features in the images, but not the evolution of their properties, in our case the cloud optical thickness, without further information. Therefore, the change of optical thickness was examined to take this into account. We concentrated on convection and found that under certain conditions the evolution of optical thickness can be predicted for decaying convective cells. As we do not know the start time or velocity of this process, the start of the decay phase and the further evolution cannot be forecasted for cells in the mature stage. But as soon as the cell reaches the decaying stage, where the lower parts of the Cb dissolve with only the anvil as remnant, it is possible to estimate the further development. For this the detection method of Cb-TRAM for stage 3 for nighttime is used, because it detects not only the core of the cells but also most parts of the anvil. The development between two consecutive images is examined by eye to find out if the cells decay has started and two criteria for the determination of decaying cells has been found: change in optical thickness < divergence > 0.1 The decay of a cell comes along with a decrease in cloud optical thickness. As for passive imagers only the upper part of a Cb can be investigated, a decrease of optical thickness of the anvil is one criterion for a decaying cell. The other criterion results from the reduction of the spatial extent of the cell detectable by the examination of the divergence of the motion vectors in this area. A convergence of the motion vectors indicates decay. The further development will be forecasted in the following way. For the displacement of the cell the disparity vector field with the object classification is used. Additionally the development in the ice optical thickness is forecasted originating from the change in optical thickness in the two startup images. One example of a decaying cell is shown in Figure 9. The uncorrected object-based forecast (left) predicts a larger ice optical thickness for the cell than it is in reality (middle), which is the case for all the investigated decaying cells. To resolve this deviation an additional and more realistic decrease in ice optical thickness is applied (Figure 9, right). 22

23 Figure 9: Ice optical thickness derived by COCS for the real situation (middle) compared to the object-based forecast (left) and the forecast with a decrease in optical thickness for decaying cells (right) Final Forecast After applying all modifications to the initial forecast algorithm the final forecast is created. Figure 10 shows as an example of the ice optical thickness derived by COCS for 13:00 UTC (top left) and 13:15 UTC (top right). The disparity vector field on top of this second startup image (top right) is then applied to it to get a forecast of 15 min. By multiplication of V with d=4 (Eq. 2) a 1h-forecast is created (Figure 10, bottom left) and can be compared to the real scene for this time (bottom right). Figure 10: Ice optical thickness derived by COCS for 7 April 2013, (top left) 13:00 UTC and (top right) 13:15 UTC with the calculated disparity vector field. A 1h-forecast is created (bottom left) and can be compared to the real scene for this time (bottom right). 23

24 Calculation of Direct Normal Irradiance The direct irradiance received on a plane normal to the sun over the total solar spectrum is called Direct Normal Irradiance (DNI). Blanc et al. (2014) gathered the multiple definitions of DNI used in literature for different scientific fields. For the calculation of the DNI we apply the strict definition for numerical modeling of radiative transfer, which refers to photons that do not interact with the atmosphere. So no circum-solar radiation is taken into account and we get DNI = E₀ exp( τ(sza)) (3) with the solar constant E₀ and the optical thickness of the atmosphere with contributions of clouds, trace gases (based on a standard atmosphere) and depending on the solar zenith angle sza. Figure 11 depicts the computed DNI for the same scene as in Figure 10. The values range from 0 W/m2 for areas with thick clouds (black) to around 900 W/m2 for cloud free areas. Thin clouds reduce the DNI according to Eq. 3 as can be seen in the lower right corner. 24

25 Figure 11: Calculated direct normal irradiance in W/m2 for 7 April 2013, 13: Ideal Exploitation of the temporal resolution of SEVIRI With the aim of determining the ideal time interval for every timestep in the forecast horizon the potential of the better temporal resolution of the Meteosat Rapid-Scan-Modus (5 instead of 15 minutes repetition rate) has been investigated. Therefore for the period from March to June 2013 forecasts up to 2 hours with a time interval of 5 min, 10 min, 15 min, 30 min and timesteps of 5 min have been created. The forecasts have been started every hour during daytime. Figure 12: Comparison of the forecast quality for 4 different time intervals and the persistence method. In order to quantitatively assess the performance of the forecast for the four time intervals, we evaluate the capability of the algorithm to predict cloud amount by examining the errors of the cloud mask. The errors are misses (cloudy pixel observed but not forecasted) and false alarms (cloudy pixel forecasted but not observed) and the results are shown in Figure

26 The differences between the single time intervals are very small and it can be stated that the forecast accuracy does not depend on the length of the time intervals. The main advantage of the higher temporal resolution is therefore the higher frequency of observations allowing a more frequent start of the forecasts Ideal Exploitation of the spatial resolution of SEVIRI For a better spatial resolution the HRV-channel (High Resolution Visible, 1km instead of 3km maximum spatial resolution at subsatellite point) has been integrated into the forecast. To detect clouds the difference of the measured reflectivity from SEVIRI and the clear-sky reflectivity calculated from surface albedo (MODIS) with LUT has been used. This was done for land, sea and coast separately because of different absolute values and gradients in surface albedo. A pre-requisite for this work was an adjustment of the geolocation accuracy for MSG and MODIS by shifting the MODIS data and quantifying the correlation between both data sets. In case of the high resolution finer structures of the clouds and also small clouds are visible as can be seen in Figure 13, which shows for comparison cloud masks for low (left) and high (right) resolution. For the determination of the optical thickness a radiative transfer model is used, which calculates the optical thickness in dependence of sun and satellite geometry, albedo and effective radius. Figure 13: Cloud masks for low (left) and high (right) resolution. 2.3 DLR-DFD method using a sectoral method based on Meteosat Second Generation imagery Method for cloud mask nowcasting The successful integration of solar electricity from concentrating solar power plants into the existing electricity supply requires an electricity production forecast for 48 hours to cover the current day and the day 26

27 ahead. For concentrating solar power plants the electricity production forecast is driven mainly by the direct normal irradiance forecast and - with lower importance - by temperature and wind speed forecasts as well as plant status and operation strategy of the storage and additional heaters. For an optimized operation of the power plant any improvement of the state of the art numerical weather prediction s accuracy over the next upcoming hours is welcome. This study applies a receptor-like approach tracking only clouds which are coming closer to the power plant (red color in Figure 14) and discriminating between thin cirrus and other cloud types. Meteosat Second Generation (MSG) with its Spinning Enhanced Visible and Infrared Radiometer (SEVIRI) instrument scans the Earth within 15 minutes and provides satellite imagery with several km spatial resolution (3 km at nadir). The AVHRR Processing scheme Over clouds Land and Ocean (APOLLO) has been adapted to the MSG SEVIRI instrument. It discretizes all pixels into four different groups called cloud-free, fully cloudy, partially cloudy (i.e. neither cloud-free nor fully cloudy) and snow/ice-contaminated, before deriving physical properties (Saunders et al., 1998, Kriebel et al., 1989, Gesell, 1989, Kriebel et al., 2003). Within APOLLO, clouds are categorized into three layers according to their top temperature (low, medium, high). The layer boundaries are set to 700 hpa and 400 hpa. The associated temperatures are derived from standard atmospheres. Further, each fully cloudy pixel is checked to see whether it is a thick or thin cloud, depending on its 11 μm and 12 μm brightness temperatures and, during daytime, its channel 0.6 μm and 0.8 μm reflectances. Optically thin clouds (with no thick clouds underneath) are taken as ice clouds, i.e. cirrus, whereas thick clouds are treated as water clouds. 27

28 Figure 14: Nowcasting scheme as applied to APOLLO cloud masks red colored clouds are tracked as they are those arriving at the power plant location. Figure 15: Sectoral approach with a cloud mask (blue, orange, red depending on cloud height) and cloud free areas (green) distributed over an area of 150 x 150 km. Figure 16: Evaluation of all sectors color lines show the existence of clouds along the sector lines. Relevant sectors are marked with orange circles. From the MSG/APOLLO-based cloud physical parameters especially the cloud mask and the cloud type discrimination into water/mixed phase and 28

29 thin ice phase clouds are used in a pixel-wise resolution. Additionally, the cloud optical depth (COD) is used. The receptor model looks for the movement of a cloud towards the power plant. It performs the following steps: Satellite pixels in a 29 x 29 pixel neighborhood are remapped from a latitude-longitude grid (SEVIRI satellite projection) into a x-y kilometer polar coordinate system with the location of interest in the center. The algorithm uses two separate cloud masks one consists of thin ice phase clouds only, the other consists of all pixels being classified as water or mixed phase cloud. Typically, the thin ice clouds are cirrus-like clouds in higher altitudes. They often have different directions of movement, are larger scale features with small COD and a large variability of COD. A low pass filtering of the cloud mask is performed. Small clouds of just a few pixels size are difficult to be tracked and most likely will disappear as individual clouds through a merging process with neighboring clouds or will increase or shrink over the nowcast horizon in an unpredictable manner. Therefore, they only enhance the noise in the signal without adding to the method s accuracy. The filter takes 5x5 pixels into account. If there are less than 5 cloudy pixels in this window, the central pixel is set from cloudy to cloud free in the cloud mask. This is done separately for both cloud masks. The nowcast is being initialized with the situation being cloudy or not cloudy at the pixel at the start of the nowcast. Clouds but also cloud gaps moving towards the power plant are tracked within each sector over 3 time slots of satellite imagery (Figure 16). The surroundings of the location of interest are separated into 32 sectors with equal angular distribution (Figure 15). Values of 8 sectors and multiples of 16 have been tested, but 32 sectors performed best during the development phase. All pixels having a polar angle within the sector are mapped onto the bisecting line inside the sector at the distance to the central pixel. By that a vector of cloud mask values (cloudy, cloud free) is created along each sector s bisecting line and for both cloud masks separately. The more detailed structure inside the two-dimensional sector is reduced to a one-dimensional vector by this approach. This reduces the spatial resolution, but also increases the clearness of the signal itself. Within this one-dimensional vector again a low pass filter is performed. Cloud gaps up to 7 km along each bisecting line are set to cloudy as well to avoid small gaps being created by the mapping to the bisecting line process. Clouds being only 1 pixel long are set to cloud free on the other hand. For each bisecting line vector a search of sign changes between 0 and 1 (cloud free and cloudy) is performed. They represent the borders of cloud systems. Their movement towards and away from the central pixel is 29

30 monitored. Only if a sign change is found in both images this sector is used for a cloud movement vector derivation in this sector. Sectors with only one sign change or differing sign changes are excluded as they do not contribute a clear signal of movement towards the central pixel. Also, sectors with two strongly differing vectors between image 1 and 2 and image 2 and 3 are excluded. The criterion set is three times the smallest vector length. If the second vector is longer than this, again the sector is excluded as providing no clear signal. This is frequently seen if clouds are moving far away from the central pixel with a significant component normal to the bisection line. They are unlikely to move to the central pixel, but create a strongly varying signal of the movement component along the section line. As the movement normal to the section line is not monitored explicitly, this is used as an indicator to detect such cases. For each valid sector a movement vector is calculated along the bisection line and for each sign change. According to the vector length, the time of reaching the central pixel is marked in the nowcasting vector describing the cloud mask situation at the central pixel as evolution over time. Four cases are therefore tracked separately: Arrival of a thin cirrus type cloud and end of such a cloud period, as well as arrival of a thick water/mixed phase cloud and the end of such a cloud period. By that a vector of upcoming thin and thick cloud situations over time is created. The nowcasting vector s temporal resolution is set to 15 minutes as the satellite-based input data itself. Tuning parameters like number of sectors, gap lengths, maximum allowed differences between vector lengths, and smoothing filter for the cloud fields have been optimized with Andasol solar power plant location ground measurements for 2007 for a nowcasting start slot of 10 UTC and applied for all other times of the day and now for all stations presented in this study Method of DNI nowcasting Up to now, hourly DNI is calculated from the clear sky model value multiplied by the part of the hour being cloudy and multiplied by a cloud extinction factor set currently to 0.4. This value has also been derived at the Andasol location for This value is under further investigation at the moment. This approach is being motivated by the grid integration use case for which the algorithm has been used so far. It is aimed to replace this by using the actual COD being nowcasted for the final version of this algorithm and applying the Heliosat-4 parameterization of radiative transfer. For DNI this simply results in applying the Bouguer- Lambert-Beer law. Only for GHI, it requires the full Heliosat-4 method providing look up tables as a function of clear sky parameters as well. Within the method development phase, a 1 min resolved output has been integrated. It makes use of 1 minute resolved clear sky irradiances as 30

31 provided by the MACC-RAD service (MACC-RAD, 2015; Lefèvre et al., 2013). This will allow generating 5 min, 15 min or 1 min resolved output in the future by making use of the variability information as described below Method of DNI variability nowcasting Based on the first Advisory Board meeting, the issue of 1 minute variability has been further investigated. This has been done additionally to the description of work originally foreseen for DLR. Answering to the needs being expressed, this chapter first reviews existing variability indices and suggests a definition of generic variability classes Variability indices Several indices describing the variability of GHI have been published already, but they refer to various time scales in the original studies (Figure 17). 20 sec 1 min. 5 min. 15 min. 1 hour Figure 17: Temporal variability scales as investigated in existing studies. Skartveit et al. (1998) introduced a variability index σ Sk based on the clear sky index of the actual hour t versus the previous (t-1) and the following hour (t+1). The differences are squared, averaged and a square root is taken. σ = (kt(t) kt(t 1))2 + (kt(t) kt(t + 1)) 2 The clear sky index k c is defined as the ratio between actually measured GHI and theoretically expected GHI in the cloud free case taken only Rayleigh scattering, trace gas absorption and aerosol extinction into account. 2 31

32 This index has been derived by using hourly pyranometer observations at the location of Bergen, Norway for the snow-free months April to October in all years from 1965 to Hours with small solar zenith angles below are excluded. The index is dimensionless, has very small values in nonvariable conditions and reaches typically 0.5 to 1.0 in broken cloud conditions. In our study we apply this index for 1 minute resolved observations. The index is calculated for three consecutive minutes. Overall, this results in 58 individual index values within an hour, which are averaged finally to obtain a single value per hour. Stein et al. (2012) define a dimensionless variability index VI inside a time period as the ratio of the sum of n differences of sequential GHI values versus the sum of n differences of sequential clear sky GHI (CSI) values over a selected time period. The time interval Δt between two consecutive GHI values is taken into account as well. VI = n k=2 (GHI(k) GHI(k 1))2 + t 2 n (CSI(k) CSI(k 1)) 2 + t 2 k=2 In cloud-free, non-variable conditions the VI reaches values around 1, while fully cloudy days with small GHI values are characterized by values close to zero. In more variable conditions the VI can reach very high values clearly above 1. In their study, they used minute resolved GHI observations during a day and derived variability indices for each day. We apply the same formula on minute-wise measurements (Δt = 1), but sum only differences inside an hour to get an hourly VI. Coimbra et al. (2013) define a variability index V as the standard deviation of temporal differences of k c in a defined time interval Δt. Their development has been made with respect to forecast verification on various time scales. V = 1 N N ( I(t) Iclear(t) t=1 = 1 N N ( kt)2 t=1 I(t 1) Iclear (t 1) ) 2 The index V is dimensionless. The more variable the irradiance is, the larger is the index value. They recommend calculating k c differences from either GHI or DNI (I) at intervals of 5 minutes or smaller to avoid a dependency on 32

33 sun geometry. Nevertheless, for consistency in our study we apply this equation for all 1 minute measurements inside an hour. All indices described so far make use of clear sky indices. They have been developed independently from each other. They have been used for different temporal resolutions and had different applications as a motivation. In order to understand their relationship to each other for our purpose, Figure 18 presents density scatterplots for all available daytime hours in 2012 at Carpentras for GHI and Figure 19 the same plots, but for DNI. Days with low variability are excluded as well in order to avoid a large number of cases with small index values making the scatterplots unreadable. The excluded days have been defined by a GHI between 10 and 14 UTC above Figure 18: Density scatterplots of variability indices for GHI after Skartveit, Stein and Coimbra for the station BSRN Carpentras and daytime hours on variable days in 2012 (Skartveit vs. Stein upper left, Coimbra vs. Stein upper right, and Coimbra vs. Skartveit lower left, color bar indicates number of cases). 33

34 Figure 19: Density scatterplots of variability indices for DNI after Skartveit, Stein and Coimbra for the station BSRN Carpentras and daytime hours on variable days in 2012 (Skartveit vs. Stein upper left, Coimbra vs. Stein upper right, and Coimbra vs. Skartveit lower left, color bar indicates number of cases). A potential linear dependence has been quantified by using rank correlation coefficients after Spearman (R). Comparing GHI based indices of Stein and Skartveit as well as comparing the GHI based indices of Coimbra vs. Stein result in rank correlation coefficients around 0.8. For GHI based indices of Skartveit vs. Coimbra, a rank correlation coefficient of 0.99 is found. Therefore, either Skartveit or Coimbra s index can be omitted in the further study and Skartveit s index is chosen to be excluded as the rank correlation between Skartveit and Stein is slightly higher than between Coimbra and Stein. The same result is found for DNI. Overall, the indices reach larger values as the DNI is more sensitive to clouds and can also reach zero values. DNI based indices of Stein and Skartveit result in a rank correlation coefficient of 0.88, indices of Coimbra vs. Stein result in 0.84, and indices of Skartveit vs Coimbra reach a value of This also justifies the further use of the indices after Stein and Coimbra while the index of Skartveit is omitted in the further study. Perez et al. (2011) describe variability inside an hour by the mean Δk c_mean, the standard deviation Δk c_σ and the maximum Δk c_max of all differences in k c within consecutive time instants. In their study they used temporal differences of k c based on 20 seconds, 1 minute, 5 minute and 15 minute 34

35 intervals as they are interested in the high temporal variability of photovoltaics for grid integration purposes. The mean is justified as the expected mean change of the irradiance from one time step to the other. The standard deviation describes the width of the distribution of all irradiance changes and indicates whether all changes are comparable to each other or not. The maximum of all differences is a worst case estimate for the individual hour. In our study we apply these indices for the one minute resolved DNI observations within one hour as ΔDNI _mean, the standard deviation ΔDNI _σ and the maximum ΔDNI _max and also for the k c values as mean Δk c_mean, the standard deviation Δk c_σ and the maximum Δk c_max. The indices described so far have been developed for GHI and most of them with respect to the quantification of ramp events in photovoltaic electricity production. Kraas et al. (2013) count the direction changes (DCH) in DNI in a day on the basis of hourly DNI observations. They apply this index to assess the expected quality of day ahead numerical weather prediction forecasts for an electricity trading application by assuming that variable days are related to a lower forecast accuracy. Days with low variability typically have DCH = 1 due to the noon maximum, while days with large variability show DCH between 5 and 8 in hourly DNI. In our study, we apply the same strategy but for all local minima and maxima in the minute-wise observations inside each hour. In order to avoid any domination by only small scale variations, cases with only very small differences in consecutive extreme values should not be counted as a direction change. Therefore, only extrema being different from the next extremum by more than 15% of the clear sky irradiance value are treated. Overshooting irradiances are defined as irradiances in cloudy conditions which reach larger values than the theoretical clear sky irradiance. Zehner et al. (2011, 2012) report that the power of a photovoltaic system in situations with many overshootings can be even larger than in cloud free conditions. In our study we define an overshooting if the GHI in a minute is enhanced more than 5% above the clear sky condition irradiance value. As an index the number of overshooting minute values in an hour (NOVER) is counted. Finally, this study suggests a new group of indices describing the upper and lower envelope of the irradiance values. The difference between upper and lower envelope is a measure about the intensity of the irradiance variation. It describes an area which represents the energy losses due to variability. The time series is first split into four minute long pieces. Within those, the local minima and maxima are defined. In case of non-existence of any extreme value, the subset of the time series is extended by another time 35

36 instant step by step until an extreme value can be identified, either a minimum or a maximum. Two new upper and lower envelope time series consisting only of minima and maxima are defined. In case they do not include the first and last minute value of the original time series, these are added to ensure a comparable time extension of the envelope time series. This approach of local windows in the original time series may result in the identification of a local minimum in the vicinity of another local, but deeper minimum in the neighboring window. Therefore, a further quality control is performed: Between each 2 minima being node points of the envelope a line is assumed. In case that there are any individual values of the original time series below this line, they are added in the minimum envelope time series. The same is done with the maximum envelope. Having defined the upper and lower envelope time series, the integral INT_UML (upper minus lower) between the two is calculated for each hour. Any larger integral value for two cases with the same average k c is caused by either more or deeper ramps occurring in the hour. Another integral is calculated between the upper envelope and the clear sky time series (INT_UMC, upper minus clear). It describes the upper border of the ramps occurring in an hour. This value is small if the atmosphere is clear. It gets further negative if clouds occur and the upper envelope has smaller irradiance values than the clear sky time series. Please note, that the integral indices are applied for DNI which has no overshooting effects like GHI. A third integral value for each hour is defined as the area between the lower envelope and the abscissa axis (INT_LMA, lower minus abscissa). It describes the depth of the ramps occurring in an hour Variability classes For this study eight classes have been selected due to their varying direct normal irradiation and the number of fluctuations. DNI is chosen as it is more sensitive to variability induced by clouds, while GHI variability is always damped by the diffuse fraction and never reaches the zero value during daytime. Additionally, DNI is already normalized by the cosine of the sun zenith angle. 36

37 Figure 20: Arbitrarily chosen examples of the variability cloud classes 1 to 8. Hours being classified in one of the classes are marked by a red box. For some classes, the red box extends the range of a single hour and illustrates several hours being included in the reference database. Minute values (yellow), 10 min moving averages (black) and McClear clear sky values (thin) are given. 37

38 The selection is derived by a visual interpretation by three different scientists of a full year s time series. It also reflects the information of variability structures as being rated as important to know in various personal communications by power plant operators, by storage developers, by solar project planners and electricity grid operators. The selection has been made on the basis of 1 min temporally resolved DNI observations. Figure 20 provides example hours (marked by red boxes) which have been attributed to one of the eight classes. Yellow lines indicate the 1 min DNI observations, while the black line represents a 10 min moving DNI average. Dashed lines indicate the clear sky DNI. Class 1 (VHD_LDCH) consists of cloud-free sky cases where the DNI follows the clear sky DNI. Class 2 (HD_LDCH) consists of cases with nearly clear sky values in the 10 min moving averages and nearly no difference in the 10 min moving averages and the individual 1 min values reflecting a small variability from minute to minute. Class 3(HD_MDCH) also shows nearly clear sky values in the 10 min moving averages, but has a much stronger variability from minute to minute. Minute values may reach 30 to 50% of the clear sky DNI which is already a strong reduction in few minutes inside the hour. Class 4 (HD_HDCH) has both a large variability from minute to minute and among the consecutive 10 min moving averages. Individual minute values even reach DNI close to zero. Class 5 (MD_MDCH) has significantly lower 10 min moving averages than the clear sky values, mean k cdni (clear sky index, but derived for DNI) of 0.66 are observed. Nevertheless, the additional variability from 1 minute to another minute is small. Class 6 (MD_HDCH) also has medium level DNI values like class 5, but additionally, the minute to minute ramps are very high. They may reach from zero to clear sky DNI values and provide the largest individual ramps. Class 7 (LD_MDCH) has very low medium k cdni, but still some large ramps from minute to minute. Finally, class 8 (LD_LDCH) consists of cases with zero DNI and now variability from minute to minute. Overall, the classes are sorted from largest k cdni to the smallest k cdni. Classes 1, 2, and 8 have a small number of DCH, classes 3, 5, and 7 a medium number of DCH and classes 4 and 6 have a large number of DCH. The naming of classes is a combination of low, medium, high or very high DNI (LD, MD, HD, or VHD) with a low, medium and high number of DCH (LDCH, MDCH, or HDCH). Based on visual interpretation, a number of cases for each variability class was selected out of the hours with high sun elevation between 9 and 14 UTC in 2012 and for the measurements at the station BSRN-Carpentras. Hours with varying conditions and changing their related variability class inside the hour have been excluded. Such transition situations are frequent in real time series, but are not suited for the inclusion in our reference database. The database shall contain only those hours which can be 38

39 unambiguously attributed to a single variability class. The class characteristics can be quantified by their typical k cdni values and the number of direction changes (Tab. 1). Table 1: Variability class characterisation by k cdni, the number of direction changes in DNI within the hour, and the number of cases in the reference database. Class name mean k cdni no. of DCH (#DCH) in DNI no. of cases 1 very high DNI, low #DCH high DNI, low #DCH , mean high DNI, medium #DCH , mean high DNI, high #DCH , mean medium DNI and #DCH , mean medium DNI, high #DCH , mean low DNI, medium #DCH , mean low DNI, low #DCH It has already been mentioned that the class definition is done with respect to DNI, but nevertheless, the class characterization can also be done with respect to GHI and the variability indices applied to the 1 minute resolved GHI observations. The number of direction changes in GHI, the number of overshootings above 5% (NOVER _5) of the clear sky value and the number of overshootings (NOVER _10) being larger than 10% of the clear sky value. For classes 2, 5, 6, 7, and 8, the mean k c is larger than the k cdni values reflecting the diffuse contribution which generally dampens the ramp height. For both k c and k cdni the value is below 1.0 indicating that the MACC McClear model is slightly overestimating the Carpentras conditions. The numbers of DCH are slightly different, but the general view remains the same. The overshooting effect is only visible in the GHI (as expected). Classes 4 and 6 show the largest number of DCH and a similar pattern for the NOVER 5%, but differentiate in the NOVER 10% with class 4 having less overshootings than class 6. 39

40 Table 2: Variability class characterisation with respect to GHI mean k c, the number of direction changes in GHI within the hour, NOVER 5%, and NOVER 10%. Class short name mean k c no. of DCH in GHI NOVER 5% NOVER 10% 1 VHD_LDCH HD_LDCH , mean , mean , mean 1 3 HD_MDCH , mean , mean 4 0-9, mean 1 4 HD_HDCH , mean , mean , mean 5 5 MD_MDCH , mean , mean , mean 2 6 MD_HDCH , mean , mean , mean 8 7 LD_MDCH , mean , mean , mean 1 8 LD_LDCH , mean Satellite-based variability indicators The APOLLO methodology uses multiple spectral channels of the Meteosat Second Generation satellite (MSG) to discriminate between different cloud types. The APOLLO methodology delivers cloud mask, cloud optical depth, liquid and ice water path, and cloud top temperature as cloud parameter products for each MSG SEVIRI pixel in a temporal resolution of 15 minutes during daytime, for the period (10 years). The covered zone is [60 N,60 S,60 E,60W], with a resolution of 3x3 km 2 at the nadir of the satellite [0, 0 ]. The resolution in Europe is about 4x5 km 2 to 5x6 km 2. The following parameters are computed and stored: Cloud mask and snow Cloud coverage (0-100%) Cloud type (low, medium, high water/mixed water/ice phase clouds; optically thin ice clouds) Cloud optical depth Cloud top temperature The analysis is extended further in a 29x29 pixel window around the location of interest. In this window, several values are computed, mostly the number of cloud elements, the number of gradients in a binary cloud mask changing from cloud to non-cloud, the window cloud fraction, and the cloud shape complexity from the fractal box counting dimension. This fractal box counting dimension represents the complexity of the clouds 40

41 shape and evolves from zero (a point) through one (a line) to two (an area). In most cases, it lies between one and two if the cloudy pixels clusters in the window are several pixels wide. Figure 21 shows typical examples of these situations. Figure 21: Typical box dimensions of different cloud masks, source: S. Glas (2014). Additionally to the single pixel results, the surrounding 29x29 pixel window is evaluated in order to understand the medium scale cloud situation and to discriminate between overcast/broken, scattered and isolated cloud fields (cloud area type). If there are more than 10 individual cloud elements in the surroundings, the situation is classified as scattered unless the total cloud fraction in the surroundings is above 80%, which classifies the case as broken/overcast. If there are less than 10 cloud elements, the situation is classified as broken/overcast unless there is a high number of more than 175 cloud/no cloud changes from pixel to pixel in any direction, which again results in a scattered case. Also, any cloud/no cloud change from the central pixel to the direct neighbors always results in a scattered classification Satellite-based automatic variability classification Based on the indicators as defined above, the same approach can be applied. Each variability class is described by a distribution of occurring APOLLO based parameters (Figures 22 to 29). 41

42 Figure 22: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 1 related cases in the reference data base. Figure 23: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 2 related cases in the reference data base. 42

43 Figure 24: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 3 related cases in the reference data base. Figure 25: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 4 related cases in the reference data base. 43

44 Figure 26: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 5 related cases in the reference data base. Figure 27: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 6 related cases in the reference data base. 44

45 Figure 28: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 7 related cases in the reference data base. Figure 29: Box-Whisker plot for the distribution of normalized satellite-based cloud parameters as found for class 8 related cases in the reference data base. 45

46 Combining the findings from Figure 22 to 29 into a single graph, Figure 30 provides the median of each distribution for all satellite-based cloud parameters and each class. Figure 30: Median values of each satellite-based cloud parameter for all classes. Applying a distance criterion, the best fitting variability class is selected. The sum of all distances between the actual cloud parameter value for a satellite image and the median of the n-th class is calculated. For all 8 classes, the minimum distance is chosen and the respective class is allocated to the satellite image time instant. Finally, a comparison of the satellite-based automatic classification versus the visual class interpretation of all cases of the reference database (section ) is performed (Figure 31). 46

47 Figure 31: Comparison of automatic classification versus visual classification results for 2012, BSRN Carpentras, and hourly variability classes 1 to 8. Overall, around 50% of all cases are identified correctly by the satellite-only approach. If a mis-classification by a single class is accepted (e.g. class 4 is mis-classified as class 5), a detection rate of 80% is found Outlook on using the variability classes for nowcasting Having derived variability classes based on satellite-based cloud parameters, the variability class itself can be provided as an additional output every 15 minutes based on the observations. It is valid at least for the hour after the observation s time instant. Based on the class characteristics, an artificial, but representative 1 minute resolved time series can be created as nowcasting result. For this nowcasted hour, the 1 min resolved values cannot be expected to be valid exactly, but the distribution of values can be assessed e.g. by a cumulative Kolmogorov-Smirnov integral approach. 3. Satellite radiances and NWC SAF products for data assimilation This task focuses on the connection between the satellite based nowcasting and numerical weather prediction (NWP) models. Beyond state-of-the-art methods will be developed for usage of high-resolution (in time and space) satellite radiances for general sky conditions and for the use of SAFNWC products when generating the NWP initial state. 47

48 3.1 The HARMONIE system for Numerical Weather Prediction Data assimilation in Numerical Weather Prediction (NWP) optimally blends observations with atmospheric model data in order to obtain the best possible initial state for an atmospheric model prediction. It was early realized (Lorenz, 1965) that the forecast quality is strongly dependent on an accurate description of the initial state and hence on the abilities of the data assimilation system and the observations used. Different data assimilation methods exist. One method well suited to handle meteorological observations that are non-linearly related to the model state variables, such as is the case for satellite radiances, is variational data assimilation. Here, the HARMONIE NWP system is used to produce shortrange forecasts. The HARMONIE NWP model system is a non-hydrostatic meso-scale forecasting system developed within the HIRLAM-ALADIN consortia. Presently HARMONIE consists of initial condition generation through data assimilation for the upper air, and the AROME forecast model with two model components (Seity et al., 2011). The current HARMONIE data assimilation system consists of surface data assimilation (Giard and Bazile, 2000) based on optimal interpolation and upper-air 3-dimensional variational data assimilation (3D-Var), with climatological background error statistics (Seity et al., 2011). 3.2 Assimilation of SEVIRI radiances The SEVIRI instrument (Schmetz et al., 2002), on board the geostationary Meteosat Second Generation weather satellites, is an optical imaging spectrometer measuring top-of-the-atmosphere (TOA) radiances in 12 spectral channels. The spatial resolution of the measurements is about 3 km over the HARMONIE Iberian Peninsula DNICAST domain. The instrument scans the atmosphere every 15 minutes. Here, two of eight IR channels were used, which are spectrally located around 6.25 and 7.35 μm (also referred to as water-vapor channels). To assure that only radiances not affected by clouds are used we also incorporate SEVIRI cloud-retrieval products processed with a software package (Le Gleau and Derrien, 2002) developed in the framework of the EUMETSAT Satellite Application Facility on Nowcasting & Very Short Range Forecasting. An adaptive so-called variational bias correction (Dee, 2005) is applied to handle systematic errors in the SEVIRI radiances. To mitigate the impact of spatially correlated observation errors a spatial thinning of SEVIRI radiances is applied. The horizontal thinning distance is on average 35 km and with a minimum value of 15 km. 48

49 The data assimilation aims at finding the model state, x, and bias coefficients, ß, minimizing the following penalty function, J: J T 1 T 1 x, β = x x B x x + β β B β β b T 1 Hx+b x, β y R Hx+b x, β b 1 2 where x b denotes the model state, y are the observations with estimated biases b. H is the observation operator projecting the model state on the observed quantities. Furthermore B, B ß and R represents the error covariance matrices for the model background, the background for the bias coefficients and for the observations, respectively. The biases, b are represented using a linear predictor of the following form: b N x, β = β p x i= 1 i i where N is the number of predictors, p, with associated coefficients ß. For our SEVIRI bias corrections we have chosen to use one predictor only, representing observation bias through an offset value. A possible future extension is to include also one or more of the predictors hpa thickness, hpa thickness and total column water. In Figure 32 observed SEVIRI wv73 brightness temperatures (left) and observed values minus HARMONIE model equivalents (right) are illustrated for one particular case, 2013/09/30, 00 UTC. The observation minus model state equivalents shown are after application of cloud mask and bias correction, but before spatial thinning. It can be seen that in some areas the observed brightness temperatures are higher than the corresponding model equivalents, whereas in other areas the observed values are smaller than the corresponding model equivalents. Figure 32 is an example of investigations made to ensure the functionality of the SEVIRI observation handling in the HARMONIE system. b β y b 49

50 Figure 32: Observed SEVIRI wv73 brightness temperatures (left,unit:k) for 2013/09/30, 00 UTC. Corresponding observation minus HARMONIE model state equivalents (right, unit K). To evaluate the impact of assimilation of SEVIRI radiances on the forecast initial state and on the following forecasts parallel data assimilation and forecast experiment are set-up. There are two parallel experiments run for a one-month period over the HARMONIE DNICast Iberian Peninsula domain. In one of the parallel experiments only a baseline set of meteorological observation types will be used in the 3-dimensional variational data assimilation for the atmosphere. The other parallel experiment will be similar, except for that also SEVIRI radiances are used in the 3-dimensional variational data assimilation for the atmosphere. The time period of the parallel experiments is from 1 to 30 of April The assimilation period is preceded by a two-week period during which the SEVIRI variational bias correction coefficients are spun up. Surface initial conditions and upper air initial states for the beginning of the spin-up period were taken from one of the extended standard HARMONIE DNICast runs. In the evaluation of the parallel experiments we will focus on the forecast quality in terms of humidity, clouds and DNI. 3.3 Cloud initialization with MSG data In addition to assimilation of satellite radiances an approach based on ideas of van der Veen (2013) is examined to use SAFNWC products for improving the NWP initial state and the following short-range cloud forecasts. Twodimensional fields of cloud mask, cloud top height and cloud base are used to modify the HARMONIE model 3-dimensional initial state. Not only the model cloud fields will be modified in accordance with the satellite based products, but also the temperature and humidity fields are modified. The humidity modifications are based on model relations between cloud amount and humidity. In order to preserve the same buoyancy after the 50