MAIN ATTRIBUTES OF THE PRECIPITATION PRODUCTS DEVELOPED BY THE HYDROLOGY SAF PROJECT RESULTS OF THE VALIDATION IN HUNGARY

MAIN ATTRIBUTES OF THE PRECIPITATION PRODUCTS DEVELOPED BY THE HYDROLOGY SAF PROJECT RESULTS OF THE VALIDATION IN HUNGARY Eszter Lábó OMSZ-Hungarian Meteorological Service, Budapest, Hungary labo.e@met.hu H-1525 Budapest, P. O. Box 38, Hungary Abstract The OMSZ-Hungarian Meteorological Service participates in the validation activity of the Precipitation Products within the HSAF project. So far, five precipitation products (three instantaneous and two accumulated rainfall products) were validated, on a monthly basis. The reference data for the validation is the radar data of the three meteorological radars operated by the Hungarian Meteorological Service. Besides the accuracy statistics, we regularly prepare case studies incorporating the weather situation that describes the meteorological conditions to compare the products with radar data and other satellite-derived information. The case studies are performed through visual validation of several kinds of precipitation-related information. The visualization software used for validation purposes in Hungary is the home-developed HAWK (Hungarian Advanced WorKstation) displaying system. Hereby we present the accuracy and multi-categorical statistical results of the validation activity for the last 2-and-half years. This is for the first time that longer time-scale attributes of the validation results are investigated and presented since 2 and a half-year long time-series have not been available until this year. This enables us to examine the seasonal features of the products. Some case studies are also shown, with typical weather situations in Hungary, especially convective events with large amount of rainfall. INTRODUCTION The OMSZ-Hungarian Meteorological Service is a consortium partner in the youngest satellite application facility of EUMETSAT, the Hydrology SAF project. The institute participates in the validation activity of the precipitation products. Since October 2006 when the first precipitation products were released, the OMSZ has developed validation techniques in collaboration with the HSAF precipitation products developers and validation group for characterizing the error structure and reliability of the products (Lábó et al., 2008). These methods allow us to compare the precipitation rates based on satellite measurements with Hungarian ground truth data. We calculate monthly statistics for the H01, H02 instantaneous products retrieved from microwave satellite measurements; and for the H03 instantaneous products retrieved from combined microwave and infrared satellite measurements since January 2008 onward. The accumulated H05 and H06 products were validated from January 2009 onward. The current paper presents monthly results over the last 2 and half years for H01, H02, H03 products, and for 1 and half year for H05 and H06 product till August 2010. The HSAF precipitation products The H01 product is a precipitation rate estimated from the measurements of the SSMI and SSMI/S passive microwave instruments onboard of the polar orbiting DSMP satellites. The algorithm used is a retrieval of precipitation profiles, based on a Cloud Dynamical and Radiation Database generated by a Cloud Resolving Model. Microwave radiation is sensible to water- and cloud droplet distributions due to scattering and absorption processes that occur. The H02 product is calculated from the microwave measurements of the AMSU-A and AMSU-B instruments, onboard of the NOAA satellites; and AMSU-A on the MetOp satellite. The channels on these instruments are sensible to humidity- and temperature profiles in the atmosphere. These profiles

are related to the cloud particle characteristics and thus, to precipitation rate. The latter is calculated by the application of neural networks in the case of the H02 product. This is contrary to the H01 where the retrieval of cloud microphysical properties is a physically-based method (based on a Cloud Resolving Model). The H03 product is a precipitation rate at ground based on the infrared measurements of the MSG geostationary satellite, which is supported by MW measurements from H02 product of the H-SAF project. The channel of the SEVIRI instrument that is used is the 10.8µm channel which is an atmospheric window (the absorption in this channel is of little account). For details on the HSAF precipitation products, see ATDD in References (H-SAF Consortium, ATDD, 2010). Reference data The goal of validation is to compare the satellite-derived precipitation fields with ground measurements. In Hungary we have chosen the radar measurements as reference fields. We consider that the point-like rain-gauge measurements are less representing the precipitation fields of the satellite raster images than radar measurements which are raster images as well. However, rain gauges data is important, and it is used in the correction of the 12-hourly and 24-hourly accumulated radar precipitation fields. The Hungarian radar network consists of 3 Doppler radars. Every 15 minutes a composite image is calculated from the 3 radar measurements with a resolution of app. 2 km x 2km. The rain intensity is derived from the (column maximum) radar reflectivity using the most widely-spread Marshall-Palmer formula. RESULTS FOR THE INSTANTANEOUS PRODUCTS Continuous and multi-categorical scores for the H01 product The continuous statistical results of the instantaneous products are presented in three rain rate categories. The goal of this classification is to separate the ability of the satellite-retrieved estimates to detect low rainfall rates from high values of precipitation which occur in convective weather situations. So we have defined three categories according to the radar values: rain rate smaller than 1 mm/h (indicated as <1 mm/h in the followings); rain rate between 1 and 10 mm/h (1-10mm/h), and rain rate higher than 10 mm/h (>10 mm/h). The results for the <1 mm/h and 1-10 mm/h are plotted in the same diagram; whereas the results for the >10 mm/h is plotted in a separate diagram, due to the larger scales that these results require. For the Correlation Coefficients (CC), only months were plotted when the number of radar rain rate values fallen into the category was larger than 2. Multi-categorical Scores such as Probability of Detection (POD), False Alarm rate (FAR), and Critical Success Index (CSI) were also calculated on a monthly basis. The upper right graph shows the variation of these scores over the months. Figure 1: Mean, Mean Error, RMSE, Standard Deviation and Correlation Coefficient values; and POD, FAR, CSI values for the H01 product validation over Hungary over the period from January 2008 till August 2010 Overview of H01 statistics

We can conclude from the graphs in Figure 1 that there is considerable underestimation of large rainfalls in the > 10 mm/h category. It is an underestimation by -10 mm/h or more. For 1-10 mm/h rain rates, usually there is underestimation, but in May 2009, and August 2010 there is overestimation (Mean Error is 1.03mm/h and 1.5mm/h). The Standard Deviation is higher in summer months than in other seasons for all categories. However, we can say that Standard Deviation is relatively stable in case of H01, although it has increased slightly since 2008 in categories <1 mm/h and 1-10 mm/h. The correlation is the best/highest in September, October 2009 (0.34 and 0.36); whereas it is very low in winters, going under zero in December 2008, February 2009 and February 2010. There is an improvement in Probability of Detection in summer months of 2010 compared to the summer in 2008. But, indeed, the False Alarm rate is constantly very high (around 0.8) since October 2008. Continuous and multi-categorical scores for the H02 product The same scores were evaluated for the H02 product as for the H01 product. We can see the same graphs as in Figure 1 calculated for the H02 product in Figure 2. Figure 2: Mean, Mean Error, RMSE, Standard Deviation and Correlation Coefficient values; and POD, FAR, CSI values for the H02 product validation over Hungary over the period from January 2008 till August 2010 Overview of H02 statistics The correlation values for the 1-10 mm/h category are around 0.4. The correlation is also good for the >10mm/h category where only August 2008 and August 2009 have negative CC. The Mean Error has decreased and become stable since 2008 (for 1-10mm/h, it stays around - 0.5mm/h, and for the category <1mm/h, ~-0.2mm/h in 2010). We can see also in the upper left graph that the ME is not changing significantly, except for June-July-August 2008. In these three months, it had large ME, RMSE and SD values for all three categories. For elaborating the cause for this, we investigated the number of satellite pixels fallen into the category > 10mm/h; and compared it to the number of radar pixels which fell into this category. The number of satellite pixels with value higher than 10 mm/h was 2134, 6390, and 716, respectively; whereas the number of radar pixels in these months fallen into this category was only 58, 33, 33, respectively. So in this period, the H02 satellite products had unrealistically high values, which distorted the result of the statistical comparisons. However, it can be stated that this huge overestimation was removed from the product and since then, in summer 2009 and summer 2010, the H02 product had stable Mean Error around 0.1-0.2 mm/h for the category < 1mm/h. It is important to note that the ME is positive in summer months, whereas in other periods of the year, it has negative value (this is true for the light and the moderate rain rates as well see upper left panel in Figure 2). This seasonal feature of the performance of the H02 product can be also depicted in the Multicategorical statistics: the Probability of Detection (POD) is very good (0.733 in Aug 2010) in summer months, low (~0.1-0.2) in winter months (see upper right panel in Figure 2). Here, we can also see in that there is decreasing tendency in the False Alarm Rate (FAR) since 2008.

Continuous and multi-categorical scores for the H03 product The same scores were evaluated for the H03 product as for the H01 and H02 product. We can see the same graphs as in Figure 1 and Figure 2 calculated for the H03 product in Figure 3. Figure 3: Mean, Mean Error, RMSE, Standard Deviation and Correlation Coefficient values; and POD, FAR, CSI values for the H03 product validation over Hungary over the period from January 2008 till August 2010 As the MSG satellite is a geostationary satellite which takes measures every 15 minutes, the H03 product is available every 15 minutes, with a spatial resolution of 4 km in average. The total number of cases for each month grows rapidly, compared to the two Microwave products: while for H01, the average total number of cases in one month was 5340; in case of H02 it was 9400; in case of H03, it is 1.911.250. This means that the statistical values represent a dataset 200 times more populated than in case of H02, so the statistical values are much more reliable. Overview of H03 statistics There is a slight underestimation for low rain rates around -0.2-.0.3 mm/h, but for the months May- June-July-August there is slight overestimation. However, there is underestimation for higher rain rates for all seasons, between -15mm/h and -20 mm/h. Among the instantaneous products, the absolute values of the Mean Error are the highest for H03. Sometimes there are large values of Standard Deviation for July 2008 (22mm/h for category <1 mm/h, 1-10mm/h), Feb and Nov 2009 (~100mm/h for category >10mm/h). For July 2008, we know that this large SD is inherited from the H02 product, which had also high values of SD in this month. The large SD values in case of category > 10 mm/h in Feb and Nov 2009 are associated with unusually high MEAN (radar) values for these months as well (see upper left graph in Figure 3). As no special even of rainfall was reported for these two months, we exclude the Feb and Nov 2009 from the further considerations on the statistical results for H03. In the right lower panel in Figure 3, we see low correlation values: the highest is 0.2 for moderate rain; for higher rain rates, there is almost no correlation between the satellite products and the ground truth. The Correlation Coefficients do not depend on the month. The FAR is high (~0.8) and it does not vary with the months. Conversely, the Probability of Detection changes with the season: during winter, ~0.2; in summer, we can find much better with values 0.5-0.6. This underpins the ability of infrared measurements to detect the rainfall in summer, when there are a lot of convection with high-level cloud tops and small ice particles on cloud tops. Case studies for 11 th June 2009 for the H01, H02 and H03 products Case studies are evaluated through the visualization of satellite and radar products through the HAWK displaying system at the OMSZ-Hungarian Meteorological Service. In each of the precipitation events analyzed, we have visualized the Cloud Type (CT) image (also drawn from the MSG satellite by the help of the NWCSAF software package, run operationally at OMSZ). One selected event is presented

in the followings, which is characterized by convective rainfall. More case studies for convective and also stratiform rainfall events can be found in (H-SAF Consortium, PVD, 2010). The CT image can be found in the lower left panel of each image that represents one precipitation event. The radar data at the closest time slot is shown in the panel next to the CT. The panel above the radar is also showing the radar data, but it is up-scaled to the resolution of the satellite image which can be found in the upper left panel of each images. The case selected for convective rainfall is the 11 th June 2009 15:34 UTC for H01, 15:41 UTC for H02, and 12:10 UTC for the H03 product. We can see the case-study panels for H01 and H02 in Figure 4. The stormy cells can be well depicted on the radar images at the original radar resolutions (dark red spots represent high rain rates, above 30 mm/h which is the sign of heavy rainfall from convection). The CT images also show high-level (brownish color), and very high level clouds (white color) clouds; it is obvious from the cloud structure that these clouds are separate ones (green color means clear pixels). Figure 4: The case-study analysis of the H01 and H02 products (left and right panels, respectively) using the radar (right side in each panel) and Cloud Type images (lower left image in each panel). We can see in Figure 4 that the H01 detects well the convective spots. It overestimates the rain intensity in most of the pixels. The area of precipitation is well detected. Similarly, H02 captures the area of rainfall. Intensities are also well described in this case. Even small cells are detected in the middle of the country. For the H03-analysis in Figure 5, we have put the Cloud Type images in the right upper image of the 4-window panels: the H03 product is in the upper left image; under the H03, the H02 product can be seen. The original radar image can be found in the lower left image; however, it is colored by the same color scale that we use for the H02 and H03 products. In Figure 5 we can see that H03 underestimates the rain intensity. It is a general feature for the H03 product. H02 gives back the structure of the rain intensity, but H03 does not. H03 reflects well the cloud shapes in the Cloud Type image. There is a high correlation between the Cloud Type and the H03 product. This is also a general feature. Figure 5: The case-study analysis of the H03 product using the radar (right side in the panel) and Cloud Type images (lower left image in the panel).

RESULTS FOR THE ACCUMULATED PRODUCTS Continuous and multi-categorical scores for the H05 and H06 product The HSAF project has two accumulated rainfall products: the H05 and the H06. The H05 is based on the H03 product (which is a merge between infrared and microwave satellite measurements); whereas the H06 product is a numerical weather model output for accumulated rain amounts. These are two, completely different types of rainfall estimations. We will see the different nature of the estimations in the results as well. For all two products, we have evaluated the scores for the 3-, 6-, 12-, and 24-hourly accumulations. We have calculated the continuous scores as the Mean Error, the Standard Deviation, the Root Mean Square Error, and the Correlation coefficients. Furthermore, multi-categorical Scores such as POD, FAR, and CSI were calculated on a monthly basis. Figure 6, 7 and 9 show the variation of continuous statistical scores over the months; whereas Figure 8 and 10 show the variation of the multi-categorical Scores over the months. Figure 6: Mean, Mean Error, RMSE, Standard Deviation and Correlation Coefficient values for the H05 product of four different time lengths from validation over Hungary over the period from January 2009 till August 2010 Figure 7: Correlation coefficients for H05 and H06 accumulations for four different time lengths. Overview of H05 statistics From Figure 7, we can see that there is an underestimation of rain amount in case of 3,6,12-hourly products (Mean Error is negative) in all months. While, the 24-hourly product has negative ME values only in convective seasons (April-September), and positive ME values in winter months (~4 mm/h). For all the accumulation periods the absolute value of the ME has decreased in May-August 2010. The tendency over the months of the ME, SD is similar in case of 3,6,12-hourly products; but not for the 24-hourly accumulated product. The correlation coefficients have increased since January 2009; they are ~0.3 in summer 2010 (Figure 7). Also, the Probability of Detection has increased for summer 2010 compared to summer 2009 (Figure 8).

Figure 8: POD, FAR, and CSI values for the H05 product of four different time lengths from validation over Hungary over the period from January 2009 till August 2010. Figure 9: Mean, Mean Error, RMSE, Standard Deviation and Correlation Coefficient values for the H06 product of four different time lengths from validation over Hungary over the period from January 2009 till August 2010 Figure 10: POD, FAR, and CSI values for the H06 product of four different time lengths from validation over Hungary over the period from January 2009 till August 2010.

Overview of H06 statistics There is slight underestimation of rain amount in case of 3-hourly products (Mean Error ~ -2mm/h). In case of 6-,12-,24-hourly products, the ME only for convective seasons (April-September) is negative. However, high overestimation can be seen in winter months, similarly to H05 24-hourly product. For all the products, the ME has increased in absolute value in May-August 2010, contrary to the H05 product for which the absolute value of Mean Error is the lowest for this period. The ME, SD monthly variations are similar in case of 3-,6-,12-,24-hourly products. CCs are better in winter (Figure 7), in summer 2010 they have become lower (<0.1) for all the four products. FAR is very high (~0.8), POD decreased in summer 2010 to low values. CONCLUSION: We have validated the precipitation products of the HSAF project over long time periods. They have proved to be stable in their performance. However, there is a need for continuous monitoring of quality and validation to detect the improvements and possible problems over different weather situations. From the validation results of H01 over the period investigated, it seems that its correlation to radar rain rates is the best in seasons with non-convective, but heavy rainfall events (autumn and spring). The correlation values are higher for H02 than for H01: for moderate rain rates (1-10 mm/h), they are around 0.4; and the correlation is also better for the highest rain rates. The Critical Success Index is also better for the H02 than for H01, which shows the higher ability of the product to correctly detect rainfall. We can also see that the microwave measurements are more capable to detect the rain in summer seasons than in winter months. The precipitation estimating ability of the H03 product is very poor (Correlation Coefficient is low), and there is a general underestimation, with Mean Errors of large rainfalls reaching the highest value of the instantaneous products. The results show that the infrared measurements detect more properly precipitation events on in convective situations. We have evaluated the statistical scores for 3-, 6-, 12-, and 24-hourly accumulated rain for the H05. The monthly characteristics vary similarly in case of the first three products; while the 24-hourly accumulated products have different, but better performance. Among the accumulated products, there is a decrease in the detection of rain in case of the H06 products in summer 2010, while the H05 product seems to perform better in summer 2010 than previously. ACKNOWLEDGEMENTS The author would like to thank EUMETSAT for the funding of the project. Also, I would like to thank the cooperation of all the members of the Precipitation Product s Validation Working Group, and especially to the coordinator of the work, Silvia Puca. REFERENCES H-SAF Consortium (2010): Satellite Application Facility on Support to Operational Hydrology and Water Management (H-SAF) ALGORITHM THEORETICAL DEFINITION DOCUMENT (ATDD), 31 August 2010, available on H-SAF webpage. http://www.meteoam.it/modules.php?name=hsaf H-SAF Consortium (2010): Satellite Application Facility on Support to Operational Hydrology and Water Management (H-SAF) Product Validation Report (PVR), 31 August 2010, available on H-SAF webpage. http://www.meteoam.it/modules.php?name=hsaf Lábó, E., Kerényi, J., Puca, S. (2008): Methods used for the validation of the satellite-derived precipitation products, part of the Hydrology SAF in Hungary, Proceedings of the 2008 Meteorological Satellite Data Users' Conference, EUMETSAT P. 52