New rainfall and climate quality control systems used for land surface observation Monday, 30 October 2017 Clément Hutin - Climate Database Developer clement.hutin@metoffice.gov.uk
Overview I. Importance of good quality data II. Met Office surface observation network III. What could go wrong? IV. Performing quality checks on data I. Spatial Observations Data Assessment (SODA) II. AREAL Climate QC
Importance of good quality data Climate monitoring Placing extreme events in historical context Drought and flood management Initialising numerical weather prediction models Reliably inform the public and partners. Kendon et al., 2017 Met Office, 2017 NASA/GFSC, 2010 Public Weather Media Service, 2017
Land surface observation network Voluntary observer climate sites 164 Automatic (MMS) sites 268 Daily rain gauges 2640 Met Office college, 2016 Monthly rain gauges 372 Land Networks Maps, 2017
What could go wrong with rainfall measurements? Understanding the source of possible errors is essential to develop appropriate and effective quality control tools. Manual observing errors Time of read Irregular/ missing readings Unindicated accumulations Wrong day entry Decimal point / tens errors / miss reads and resets Automatic errors Blocked rain gauge Snow melt in rain gauge Equipment fault (reed switch) Comms failures Change in environment Sheep in instrument enclosure, Eskdalemuir Rebecca Anderson, 2017 Met Office college, 2016 Wildlife
Spatial Observations Data Assessment (SODA) Web-based software used to quality control daily and monthly rainfall totals; Replaces Daily Rainfall Assessment Method (DRAM) developed in 1990 s; Written in Java, Python and PL/SQL; SODA - operational since August 2016. DRAM - Operational up-until September 2016 SODA
Features of SODA - Map Values plotted on high resolution map with detailed topography; Hourly RADAR images available; ATDnet location (lightning detection); Identify suspect values; Suggest source of error. Example of the passage of a cold front on the 3 rd of August 2016 at 0300 to the north-east of Stirling (UK) with embedded thunderstorms. Denotes thunder detected by ATDnet
Features of SODA - Values Plan value calculated for each day based on observations at neighbour stations; Substitute for certain missing values; Calculates the Average Annual Rainfall % for each site; Re-apportion accumulated values. PLAN = i wt i dist wt (d ) wt dist,i i (d ) wt i qual,i R qual,i i R: rainfall value wt dist : distance weighting function d: distance of neighbouring gauge wt qual, i : perceived quality weighted function i: neighbouring gauge index
Case study - Llanidloes 08/07/16 24 Hour Rainfall totals 8 th July 0900 to 0900 on 9th 8 th and 9 th flagged as EARLY2.4mm/15.1mm Low compared to neighbours
Case study - Llanidloes 09/07 -- 0500Z 0800Z 0900Z 09/07 0600Z 0700Z Hourly radar shows front moving north-east with light rain becoming mod/heavy 07-09Z; Rain card Gauge Read 0730Z ; Values reapportioned using SODA.
Case study - Llanidloes Values reapportioned over these two days
DRAM vs. SODA Research done by Rebecca Anderson; Compared calculated plan value from DRAM and SODA using 6 sites; mean difference between observations and planned value: SODA more accurate than DRAM; mean difference between SODA and observation < 1mm; Improvements due to near real time data ingest and choice of neighbouring gauges.
AREAL Climate QC Incoming data from 164 voluntary observers: Initial QC consistence and range checks Monthly AREAL QC Previously every value was checked Now suspect values are flagged In-depth spatial and temporal checks WOW WeatherObservationsWebsite Increased the efficiency in QC process WOW e-mail Paper 3208 form 4% 18% 78% Data transfer
AREAL Climate QC checks SQL-based package developed by Philip Hawkins and Climate QC team; Automatically flags suspect data suggests errors; Display values from 10 near neighbours with station information; Estimates calculated using spatial weighted regression You et al., 2008 Source or errors Non-reset Instrument malfunction 2 day value Range Step Reads late/early Missing reading Grass min 1 C over min Temp conflict between dry bulb & min/max or grass min. UK Climate Observations (min temp C) Aberfeldy 26/10/2016 miss-read
Summary SODA Rainfall QC Visualise and spatially compare rainfall values; Overlay radar images and thunder activity; Automatically calculate plan value and multi-day reapportion.
Summary AREAL Climate QC Visualise and compare observations spatially; Automatically flags suspect values; Suggests error source and correction estimate; Greatly increased efficiency in QC process.
Thank you for listening. For further detail please contact: clement.hutin@metoffice.gov.uk philip.hawkins@metoffice.gov.uk
References Kendon, M., McCarthy, M., Jevrejeva, S. and Legg, T. 2017, State of the UK Climate 2016, Met Office, Exeter, UK. Land Networks Maps, 2017, Met Office, available at https://metnet2.metoffice.gov.uk/content/land-networks-maps Public Weather Media Service, 2017, Barometer, Met Office, available at http:///barometer/features/public-weather-media-service NASA/GFSC, 2010, MODIS Rapid Response, available at: https://earthdata.nasa.gov/earth-observation-data/near-real-time/rapid-response You, J., Hubbard, K. G., Goddard, S., 2008, Comparison of methods for spatially estimating station temperatures in a quality control system, International Journal of Climatology, 28, 777-787.