Way Forward in Using Radar Data for Climate Monitoring 28 February 2017

Transcription

1 WORLD METEOROLOGICAL ORGANIZATION INTERGOVERNMENTAL OCEANOGRAPHIC COMMISSION Way Forward in Using Radar Data for Climate Monitoring 28 February 2017 GCOS-xxx UNITED NATIONS ENVIRONMENT PROGRAMME INTERNATIONAL COUNCIL FOR SCIENCE

2 Table of Contents 1. INTRODUCTION WHAT TO ARCHIVE Meteorological phenomena of interest Radar variables to be saved Known datasets and centres Challenges Synergies Proposed way forward In a nutshell : Detailed description References Acronyms

3 PREFACE EXECUTIVE SUMMARY This document describes the suggestion of the international radar community for way forward in using radar data for climate monitoring. A. Assess the present situation a. Ensure all key players are involved b. Collect updated requirements of the climate monitoring c. Assess existing international archives, their extent and quality d. Assess existing national archives, their extent and quality B. Suggest procedures for old data a. Suggest national level efforts b. Suggest international efforts C. Define recommendations for future activities a. Data acquisition b. Quality assessment c. Archiving d. Metadata - 3 -

4 1. INTRODUCTION The Task team was summoned by inviting volunteers from the Expert team of EUMETNET s European Radar project OPERA and WMO s Inter-Programme Expert Team on Operational Weather Radars IPET- OWR. The following people volunteered: Elena Saltikoff, Finland, FMI (chair) Laurent Delobbe, Belgium, RMI Norman Donaldson, Canada ECCC Bernard Urban, Meteo France Demetris Charalambous, Cyprus (Mike Dixon, USA joining later if still needed) For this paper, valuable comments were also received from Paul Joe and Vincenzo Levizzani, as well as several members of the above mentioned expert teams and their colleagues. The team concludes that harmonization is important, data volumes are large, but these difficulties should not discourage this endeavor, given its importance and potential impact. Especially we want to emphasize its impact for future generations. Dataset from WMO normal term may be patchy, but we are the people who can affect quality of the dataset from normal period WHAT TO ARCHIVE 2.1 Meteorological phenomena of interest In the 20 th century, weather radars were mainly seen as rainfall radars. However, there are also several variables related to severe convection, which may be of interest in mesoscale climatology. Some national climatologies of hail occurrence have been published, but small scale wind phenomena (tornadoes, downbursts) can also be considered (see Becker 2013, Brimelow 2004, Burcea 2016, Cintineo 2012, Chen 2012, Grams 2012, Lukach 2017, Nisi 2015, Punge 2016) In our experience, the added value of radar for climate monitoring is particularly high for convective precipitation with high spatial and temporal variations, which cannot correctly be captured by rain gauge networks. Precipitation statistics for short durations (10 minutes for example) associated with very local precipitation requires very long gauge records. The hope is that such statistics could be obtained with shorter records of radar observations (see Berg 2016, Devasthale 2014, Eggert 2015, Fairman 2015, Goudenhoofdt & Delobbe 2013, 2016, Junghnel 2015, Kronenberg 2015, Tabary 2007, Thorndahl 2014).. For reliable continent-wide precipitation estimates, the pure rainrate is not enough: we also need to assess the snowfall. In addition to hydrological applications, there are areas where locating dry areas is relevant, such as agriculture and forest fires. This sets its own requirements for quality assessment, especially separating between two zeroes: no rain and no data. 2.2 Radar variables to be saved In addition to the parameter actually used for climate monitoring (rainrate or dbz and or KdP for precipitation, Doppler velocity for wind phenomena) it is useful to save enough metadata. This can be split in two: describing metadata (ODIM parameter HOW ) which tells about measurement details and postprocessing. But another part is technical metadata: Parameters called SQI, SNR, CSR, RhoHV. It is quite likely the future generations will develop even more advanced quality assessment methods, and ideally we should save input for those. 3. KNOWN DATASETS AND CENTRES In Europe, OPERA data centre Odyssey has European composites of rainrate and max reflectivity since They are available through a DCPC user interface. Single radar volumes are also archived, and the user interface will become available in In the United States the NCEI (former NCDC) archives NEXRAD base data, called Level-II, and derived products, called Level-III. Level-II data include the original three meteorological base data quantities: reflectivity, mean radial velocity, and spectrum width, as well as the dual-polarization base data of differential reflectivity, correlation coefficient, and differential phase. The archives go back to 1994.(see NOAA NCDC) NEXRAD data is available in Amazon archive. 270+TB compressed, 1PB uncompressed. Microsoft and google have copied this to the cloud. The KNMI in Netherlands has a portal with advanced user interface, including realtime visualization with tools like h52gif

5 Many other countries have their national archives, too, and some are also archiving data they acquire in bilateral exchange. Typically, all these data centres have their own standard for data model and data format. In some cases, the data model and format is not consistent for all the years and or all the radars even in the same archive. An example of rather large database is the Meteo France radar rainfall reanalysis product Comephore (Tabary 2012) 4. CHALLENGES In a nutshell: Quality, homogeneity, traceability and IP issues. Traditionally, hydrology people have been most worried about radar echoes not related to precipitation, such as hills, birds and windfarms, and favoured aggressive data cleaning methods. However, for drynessapplications, false removal of precipitating echoes is equally dangerous. National archives usually keep one or several of the following data types: Polar volumes in manufacturer s native data format Polar volumes in international data format (HDF5, BUFR, often national dialect ) Cartesian composites in image format (png, jpg) For single radars, if the time series seem to be long, there has been upgrades of hardware and software and several changes of the measurement, calibration and quality control. Documentation of these is poor, heterogeneous and inconsistent. For national and international composites, the list of participating radars changes both systematically and randomly. The same pixel can be usually covered by good quality data from nearby radar, but during its blackouts by another radar further away. Even if a radar time series is not used for assessing trends, it is important to know about systematic effects on the data that might affect e.g. the precipitation intensity distribution. For single radar accumulation precipitation, there are methods to compensate for a few missing images. But if a single radar is missing from composites, the compensation methodologies are not usually in place. Quality flags must be thought separately for such products. Homogenization is a key issue. There is a very abundant literature on the homogenization of long term records but it generally concerns point measurements. The Berg et al. (2016) paper suggests a method that homogenizes radar data by relating it to a gridded station dataset at a coarser time resolution. The problem has much in common with the one that arises when using satellite data. In particular, with respect to the correction of systematic errors the methods used by e.g. Dee et al. could be useful. Data availability is also a question of intellectual property: who owns the data and who can get access to it. Especially if the party which measured the data, the party which performed the corrections and the party which makes the data available in a portal are not the same institute or service, problems may arise. 5. SYNERGIES Satellite-based radars such as GPM (Global Precipitation measurement) have probably already thought about many of the issues we struggle with. They could also benefit of a homogenous dataset for validation. Global Precipitation Climatology Centre is a natural partner in many aspects. However, radars do provide information about other phenomena than just rainfall. Recently, many research publications have started to ask for the dataset to be available for other researchers. If a CDR could harbor such data, it would probably rise interest among researchers. 6. PROPOSED WAY FORWARD 6.1 In a nutshell : A. Assess the present situation a. Ensure all key players are involved b. Updated requirements of the climate monitoring c. Existing international archives, their extent and quality d. Existing national archives, their extent and quality B. Suggest procedures for old data a. National level efforts b. International efforts C. Define recommendations for future activities - 5 -

6 a. Data acquisition b. Quality assessment c. Archiving d. Metadata 6.2 Detailed description A. Assess the present situation a. Ensure all key players are involved i. Both climate monitoring and weather radars are quite diverse communities, and their members work with different boundary conditions. It is very important to think, who should be involved and in which phase. b. Updated requirements of the climate monitoring i. Already this short discussion among radar experts has brought up issues, where qualitative educated guesses of the climate experts would be beneficial. This dialog suits well for the AOPC. The key question is What are we talking about when we talk about radar data. Images or data? Measured (dbz, ZDR) or derived (mm, KdP) parameters? Fields or points? Single site or composites? Derived products ( level 3 ) such as VIL or MAX? We may be approaching a change of paradigm, and the question is, how do we overcome the hinge with least damage. In other words: homogenous time series or best possible data for each moment? How much missing data is acceptable? Can anyone define the requirements for time and spatial resolution for climate, knowing that there are already quite different need? What is the relationship between data from surface-based radars and data from satellite missions such as TRMM and GPM? c. Existing international archives, their extent and quality i. Consolidate documentation of NEXRAD, OPERA and WMO (Turkey and OSCAR). Document data model, data format, process to extract data from archive. How many years, how many radars? ii. Results of the work done in task (b) may help planning the survey (c) better. d. Existing national archives, their extent and quality i. As a joint effort of radar and climate experts, agree on short web-based survey (people are tired of all these surveys). Use the WMO radar database Turkey contact to reach all operators. Publish first version, name and shame to get the missing replies. Include an IP question in survey. B. Suggest procedures for old data a. National level efforts i. How to document the existing data, especially knowing that metadata has been changing during the years (to the level that what was a good pixel in 2005 may now contain a wind farm)? Those who archive derived products have typically changed the data acquisition and processing during the years (from 15 min to 5 min intervals, from dbz- based to KdP-based rainrates, ) How to document missing data? How to keep the elementary metadata cascading to derived products? ii. Shall we suggest conversion to international formats? iii. Shall we suggest re-processing with new algorithms? iv. Shall we suggest quality masks / pixel-level flags / other ways to summarize all the challenges in data sets? b. International efforts i. Consider data centres: expanding existing ones or create a new one? ii. Support services? If we want to recommend procedures listed above (especially if we want to collect the data in one place) many WMO members will need holding hands to produce useful data. Even though WMO/OPERA MoU covers some support for ODIM conversion, such support does not appear pro bono. iii. Funding? Copernicus? H2020 Big Data? This project has many essential buzz words which would support project proposal for research funding, but where could we get the resources for writing such a proposal? - 6 -

7 iv. If (and task team feeling is when) we recommend reprocessing archived volume data, it means that we need to examine and agree which processing is needed, which algorithms should be applied (clutter filtering, beam blockage, VPR, hail, and so on). C. Define recommendations for future activities a. Data acquisition i. Assess existing guidelines to be in line with other WMO efforts (related to WIGOS) ii. Review the metadata needs from climate point of view (again, related to WIGOS). Probably a dialog with climate monitoring people, IPET-OWR and WIGOS is needed. b. Quality assessment i. Assess existing guidelines to be in line with other WMO efforts ii. Review the metadata needs from climate point of view. Define best practices - Pixel-level, radar-level, network-level? - Every day, year, per measurement? - Binary, quantitative, labeling, how to document corrections? c. Archiving i. Recommendation for national processes and their documentation ii. Pros and cons, costs and benefits analysis for an international archive d. Metadata i. What to document: most of the parameters must follow the data cheek to cheek (typically in file headers), some data can be collected externally. ii. Where to publish: When a reasonable set of metadata parameters has been defined, they need to be integrated to existing data models, and that is a task for IPET-OWR. For external collection, should it be made available as an extension of WMO radar database or OSCAR? Other publication channels? iii. Dictionary of definitions ( raw, CSR, ) Especially now when approaches are changing, a lot of definitions are needed. 7. REFERENCES NOAA NCDC Radar data description Becker, A, 2013: Review of the climate requirements for Weather Radar data. CBS Workshop on radar data exchange, CBS/OPAG-IOS/WxR_EXCHANGE/2.4 Berg, P et al: 2016: Creation of a high resolution precipitation data set by merging gridded gauge data and radar observations for Sweden. Journal of Hydrology 541 (2016): Berg P et al, (2013). "Strong increase in convective precipitation in response to higher temperatures." Nature Geoscience 6.3: Brimelow, et al: 2004: A radar-based methodology for preparing a severe thunderstorm climatology in central Alberta. Atmosphere-Ocean. Burcea et al 2016: Hail Climatology and Trends in Romania: Mon Wea Rev. Cintineo et al 2012: An Objective High-Resolution Hail Climatology of the Contiguous United States. Wea. Forec. Chen et al 2012: Diurnal variations in convective storm activity over contiguous North China during the warm season based on radar mosaic climatology. JGR Dee, D. P., and S. Uppala "Variational bias correction of satellite radiance data in the ERA Interim reanalysis." QJRMS (2009): Dee, D P "Bias and data assimilation." QJRMS (2005):

8 Devasthale, A.; Norin, L. 2014: The large-scale spatio-temporal variability of precipitation over Sweden observed from the weather radar network. Atm. Meas. Tech Eggert, B., et al. 2015: "Temporal and spatial scaling impacts on extreme precipitation." Atmospheric Chemistry and Physics (2015): Fairman et al. 2015: A radar-based rainfall climatology of Great Britain and Ireland. The Weather. DOI: /wea.2486 Goudenhoofdt, E. and L. Delobbe (2016): Generation and Verification of Rainfall Estimates from 10-Yr Volumetric Weather Radar Measurements. J. Hydrometeorology. Goudenhoofdt, E. and L. Delobbe (2013): Statistical Characteristics of Convective Storms in Belgium Derived from Volumetric Weather Radar Observations. J. Appl.Met.Clim Grams et al, 2012: A Climatology and Comparison of Parameters for Significant Tornado Events in the United States. Wea.Forec. Hanel & Buihand (2010) On the value of hourly precipitation extremes in regional climate model simulations. J. Hydrology Junghnel T. et al, 2015 Towards a Radar-Based Precipitation Climatology for Germany - First Results and Future Perspectives. AMS 37th Conference on Radar Meteorology Kendon et al, 2017: Do Convection-Permitting Regional Climate Models Improve Projections of Future Precipitation Change? Bull.Met.Soc. Kronenberg R. and C Bernhofer (2015): A method to adapt radar-derived precipitation fieldsfor climatological applications. Met. Appl. DOI: /met.1498 Lukach, M., L. Foresti, O. Giot and L. Delobbe (2017): Estimating Occurrence and Severity of Hail Based on 10-years Observations From Weather Radar in Belgium. Meteor. Appl. Moseley, C et al, 2013: "Probing the precipitation life cycle by iterative rain cell tracking." J. Geophys.Res.: Atmospheres ( Nisi et al, 2015: Spatial and temporal distribution of hailstorms in the Alpine region: a long-term, high resolution, radar-based analysis. QJRMS Overeem et al,(2009) Derivation of a 10-Year Radar-Based Climatology of Rainfall. J.Appl.Met.Clim. Punge et Kunz (2016) Hail observations and hailstorm characteristics in Europe: A review. Atm.Res. Tabary et al (2007): The new French operational radar rainfall product. Wea.Forec. Tabary, P. et al. (2012): A 10-year ( ) reanalysis of Quantitative Precipitation Estimation over France: methodology and first results. Weather Radar and Hydrology. IAHS Publ. 351, Thorndahl et al, 2014, Bias adjustment and advection interpolation of long-term high resolution radar rainfall series Journal of Hydrology - 8 -

9 8. ACRONYMS AOPC BUFR dbz DWD ECV GCOS GPM GRUAN HDF5 ICM IPET-OWR IP LC NEXRAD NMS NCDC NOAA NRT NWP ODIM OPERA OSCAR SQI SNR SMART TRMM VIL WG-GRUAN WIGOS WMO Z ZDR Atmospheric Observations Panel for Climate Binary Universal Form for the Representation of meteorological data (logarithmic unit of) radar reflectivity factor Deutscher Wetterdienst Essential Climate Variable Global Climate Observing System Global Precipitation Measurement (satellite campaign) GCOS Reference Upper Air Network a Hierarchical Data Format Implementation and Coordination Meeting Inter-Programme Extert Team of Operational Weather Radars Implementation Plan Lead Centre Next-Generation Radar, radar network in US National Meteorological Service National Climatic Data Center (USA) US National Oceanic and Atmospheric Administration Near Real Time Numerical Weather Prediction OPERA Data Information Model Operational European Radar project Observing Systems Capability Analysis and Review Tool Signal Quality index Signal-to-noise ratio Specific, Measurable, Actionable, Realistic, and Timebound Tropical Rainfall Measuring Mission Vertically Integrated liquid (a derived radar product) Working Group on GRUAN WMO Integrated Global Observing System World Meteorological Organization radar reflectivity factor Differential Reflectivity - 9 -