Traceable estimates of uncertainty for climate data records

Transcription

1 ESA Climate Change Initiative Phase-II Sea Surface Temperature (SST) Traceable estimates of uncertainty for climate data records Chris Merchant

2 Viewpoint of this talk Earth Observation community can improve as a measurement science, particularly in our approach to uncertainty information We should do this by taking advantage of the hard-won insights, vocabulary and methods from metrology Metrology is the science of measurement We are making measurements We should apply the principles of the science of measurement Benefits: logical rigour, reduction in ambiguity, better communication, More informed use of data generated some satellite SSTs may be usable as reference data for calibration of models and re-analysis systems but a present, potential users have little idea about the relative qualities of alternative datasets 2

3 etrology Worldwide trade/manufacturing Public health and safety Scientific Research Requires data that are: Stable over time so that scales and references aren t changing Insensitive to the method of measurement so the result doesn t depend on how you make the measurement Uniform worldwide so you can build the wings in France and the fuselage in Spain Based on references that can improve methods will improve over time as new technologies are available harmonisation should not be at the expense of improvements

4 Earth Climate is difficult Observation to measure. Requires data that is: particularly for multi-decadal applications Stable over time so data can be compared across decades meaningfully Insensitive to the method of measurement so data from different sensors (and techniques) can be combined Uniform worldwide so data from different space agencies can be combined Based on references that can improve methods will improve over time as new technologies are available harmonisation should not be at the expense of improvements

5 Definitions Error Concept: How different is the measured value from the (unknown) true value of the measurand? WRONGNESS Uncertainty Concept: Given the measured value, what range of values is it reasonable to attribute to the measurand? DOUBTFULNESS These are the correct scientific definitions and also match the usage of normal people Only scientists say things like the error in this value is +/-X and think it makes sense Standard uncertainty usual quantification of uncertainty as standard deviation of the estimated distribution from which errors are drawn

6 Not just random and systematic Random effect a source of errors that are uncorrelated between repeated measured values note: errors can be random (uncorrelated); uncertainty cannot be random (or systematic) Systematic effect a source of correlated errors that you could correct for if you understood it note: this is NOT the same as a bias In EO data and in NWP fields there are effects in between these limits, e.g., locally systematic and example of a structured random effect 6

7 Uncertainty cascade detector, amplifier, digi0sa0on, non-linearity calibra0on (targets and model), geoloca0on ambiguity of inversion, defini0onal uncertainty spa0o-temporal sampling extra-/interpola0on, smoothing L0 ê L1b ê L2 ê L3 ê L4

8 For CDRs, consider all types of uncertainty If you compare two SSTs on different space-0me scales the dominant sources of uncertainty in that difference change. I think this is generally true for other ECVs See blog ar0cle hfp://

9 20 o S to 20 o N mean The differences in simulated tropospheric temperature trends between model runs [forced with] HadISST1 and Hurrell [NOAA OI v2] are significant compared to the trends themselves, focusing attention on the uncertainty in the SST datasets. Flanaghan et al., 2014 Journal of Geophysical Research: Atmospheres Volume 119, Issue 23, pages 13,327-13,337, 11 DEC 2014 DOI: /2014JD Merchant: A certain uncertainty 9

10 How uncertainties are communicated in SST CCI products

11 How uncertainties are communicated in SST CCI products

12 How uncertainties are communicated in SST CCI products

13 How uncertainties are communicated in SST CCI products

14 Random ( Noise in SST) Propagation of NEDT (scene dependent and instrument state dependent) through retrieval process (context dependent)

15 Random Systematic (large scale)

16 Random ( Noise in SST) Locally systematic (limitations of retr.) Optical Radiometry for Ocean Climate Measurements (EMPS Vol 47) Chapter 4.3

17 Random Systematic (large scale) Locally systematic Total uncertainty (skin SST)

18 Dual view 3 and 2 channel estimates have different uncertainty distributions D3 D2

19 How to validate uncertainty? If we had perfect validation data (in situ), then

20 How to validate uncertainty? But drifting buoys have calibration uncertainty ~0.2 K

21 AATSR D2 SST at 0.1 deg lat-lon based on coefficients SST uncertainty estimate / ck

22 SST CCI vs. FIDUCEO SST CCI ATSRs and AVHRRs FIDUCEO AVHRRs L0 No effort L0 Model all instrument error effects L1 Calibration harmonisation L1 Model calibration system errors O/P uncertainty for each effect L2 Assume noise and propagate Estimate retrieval uncertainty L2 Propagate uncertainty from L1 L3 Propagate from L2 Sampling uncertainty model L3 Propagate from L2 Sampling uncertainty model L4 Use L2/L3 uncertainty L4 Use L2/L3 uncertainty 22

23 How to go further? Working with NPL to transfer all applicable metrological principles and techniques Every transforma4on from detec4on of radiance to geophysical product at L3 to be analysed in terms of how it introduces and propagates error fully traceable uncertainty es0mates Tools that embody rigorous metrology in usable form analy0c propaga0on law especially Monte Carlo propaga0on of error distribu0ons through complex non-linear transforma0ons

24 Simulate L0 è L2 transformations Use real orbit data for loca0ons/angles/0mings Clear-sky ocean data for this run Need to represent the fact there is a real world, and we don t perfectly know it SatZA distribu0on of simulated data

25 TRUE WORLD Simulation system for SST AVAILABLE WORLD Reynolds SST NOAA GFS SST GBCS Emissivity Model RTTOV ECMWF ERA- Interim Profiles CRTM Emissivity Model CRTM NCEP Profiles AVHRR Instrument Model SST Algorithm Interpolate to posi0on Observed BT Observed BT Observed BT realisa4ons with different Monte Carlo errors Observed BT Observed BT Retrieved SST Simulated BT/ Tangent linears

26 AVHRR Model AVHRR Data Reference Data Colloca0on with Reference Correct pre-launch to in-orbit Fit Calibra0on Parameters Calibra0on parameters ICT PRT Temperatures ICT Gradient Correc0on Detector Earth Radiance Detector Instrument Model Instrument Gain Instrument Thermal State Calibrated Radiances

27 Distribution of random BT errors 230K 300K Error / K Error / K Accounts for: digi0za0on and noise effects in on-board calibra0on Not convolved with error distribu0on for systema0c calibra0on effects

28 Uncertainty cascade detector, amplifier, digi0sa0on, non-linearity calibra0on (targets and model), geoloca0on ambiguity of inversion, defini0onal uncertainty spa0o-temporal sampling extra-/interpola0on, smoothing L0 ê L1b ê L2 ê L3 ê L4

29 FIDUCEO approach easy-fcdr, all main components of uncertainty FIDUCEO CDRs uncertainty traceable to FCDR FIDUCEO L3 uncertainty traceable to L2 CDR L0 ê L1 ê L2 ê L3 ê L4

30 FIDUCEO FCDRs DATASET NATURE POSSIBLE USES AVHRR FCDR HIRS FCDR MW Sounder FCDR Meteosat VIS FCDR Harmonised infra-red radiances and best available reflectance radiances, Harmonised infra-red radiances, Harmonised microwave BTs for AMSU-B and equivalent channels, Improved visible spectral response func0ons and radiance 1982 to 2016 SST, LSWT, aerosol, LST, phenology, cloud proper0es, surface reflectance Atmospheric humidity, NWP re-analysis, stratospheric aerosol Atmospheric humidity, NWP re-analysis Albedo, aerosol, NWP reanalysis, cloud, wind mo0on vectors,

31 FIDUCEO CDRs DATASET NATURE USE Surface Temperature CDRs Ensemble SST and lake surface water temperature UTH CDR From HIRS and MW, Most of climate science model evalua0on, reanalysis, derived/synthesis products.. Sensi0ve climate change metric, re-analysis Albedo and aerosol CDRs From M5 7 ( ) Climate forcing and change, health Aerosol CDR aerosol for Europe and Africa from AVHRR Climate forcing and change, health

32 What will be new about FIDUCEO data? Uncertainty-quan4fied FCDR At all data set scales (from pixel level in product through to mul4-annual stability) there is adequate quan4fica4on of error distribu4ons to propagate uncertainty across all data transforma4ons (especially to CDR) accoun4ng for error correla4on structures, in a traceable, metrologically defensible manner Uncertainty-quan0fied CDR Uncertainty informa0on in product that (i) discriminates more and less certain data, (ii) is validated as being realis0c in magnitude, (iii) is traceable back to the FCDR uncertainty informa4on

33 How should uncertainty be presented? Just give me one number (60%) Total uncertainty (35%) Confidence interval (25%) Separate out main components of uncertainty (20%) A probability distribu0on of error would be nice. (15%) Ensemble, please. (5%) Result of SST CCI User Survey, 2010

34 How should uncertainty be presented? Ensemble, please. Significant request from major users at SST CCI user consultation in 2014 (after discussing uncertainty concepts and issues for two days)

35 Opportunities for CCI-FIDUCEO links Interviews to understand FCDR user requirements FCDR/CDR users would you like to be a trail blazer? early access to easy-fcdr datasets and beta tools in exchange for feedback FIDUCEO workshops September 2017 Autumn 2018 Website, mailing-lists, blog etc coming Proposal from Gilles Larnicol Cross-ECV study on improving uncertainty representation in CCI+ Part of this could be exploitation of FIDUCEO-generated tools and cookbooks across CCI programme