Oct National Academy of Sciences Extreme Attribution

Size: px

Start display at page:

Download "Oct National Academy of Sciences Extreme Attribution"

Judith Fletcher
6 years ago
Views:

1 Extreme Attribution: Confidence and Fidelity CATS London School of Economics Pembroke College, Oxford October 21, 2015 National Academy of Science Washington My ask today: Clear complete well distinguished definitions Distinguish In principle/in practice challenges Construct a Necessary Condition List for each package Restriction to specific extreme weather events (SoW) Pro: Risk Based Approaches

Key Hurdles for any approach to weather attribution General Necessary Conditions Notion of Causality in systems of PDEs or ODEs If then Nature of relationship between theory and DGM (target system)

.. intermittency, homoclinicity, torus doubling Newton's laws: Neptune.

2 Key Hurdles for any approach to weather attribution General Necessary Conditions Notion of Causality in systems of PDEs or ODEs If then Nature of relationship between theory and DGM (target system) If then Nature of the relationship between our model (class) and target system Properties of physical models do not always (?rarely?) map onto the system Annulus: onset of cells, waves... intermittency, homoclinicity, torus doubling Newton's laws: Neptune... Vulcan Mullin, Chaos, S02 (Either/or Design of Higgs Detection Experiment) In silico Analysis Cases of Equifinality given nature of system & nature of obs & timescales Beven Cases of Equidismality : implications of low fidelity both in replication and in climate Diversity of our model runs does not reflect uncertainty in the world. In Practice: Statistical Analysis: Basic definitions in a transient, rapidly mixing system Counting statistics of extreme events in a poorly observed rapidly evolving system Epidemiology with a population of One. (Not a general criticism of Causal Inference) RCP, Chaos, S02

3 Clarity of Definitions: Aims and Interpretation There are at least four relevant attribution in play. Individual Events vs Types of Event IQ TQ Attribution IQ 40% of these events this event was 40% percent more Attribution ID this event much more likely as routinely employed this event is what we expect to become Attribution TQ Attribution TD 40% of the events in this group 40% increase of events of this type ID TD quite possible potential price tag undoubtedly exacerbated odds have increases at least a partial explanation why a major role Quantitative vs Descriptive

4 Clarity of Definitions: Aims and Interpretation There are at least four relevant attribution in play. Individual Events vs Types of Event IQ TQ Attribution IQ 40% of these events this event was 40% percent more Attribution ID this event much more likely as routinely employed this event is what we expect to become Attribution TQ Attribution TD 40% of the events in this group 40% increase of events of this type 6:30 GMT this Morning ID TD quite possible potential price tag undoubtedly exacerbated odds have increases at least a partial explanation why a major role Quantitative vs Descriptive

5 Clarity of Definitions: Aims and Interpretation There are at least four relevant attribution in play. Individual Events vs Types of Event IQ TQ Attribution IQ 40% or these events this event was 40% percent more Attribution ID Attribution TQ Attribution TD 40% of the events in this group 40% increase of events of this type 6:30 GMT this Morning ID TD this event much more likely as routinely employed this event is what we expect to become quite possible potential price tag undoubtedly exacerbated odds have increases at least a partial explanation why a major role Quantitative vs Descriptive

6 Wooly Terms (I) The definitions we adopt determine which questions make sense and which aims are achievable. What exactly is intended by frequency is doubled? Half the events would not have happened & the remaining half would happen as they did All change: None of these events would have happened, but instead a totally different set of events with only half the number Other Natural Variability widens : Confuses properties of models with that of the target system, the system includes natural/internal variability. The fact that a model simulates poorly reflects the fidelity of the model. The predictability of the system bounds all forecast systems this should always be clearly distinguished from bounds due to model inadequacy. Relevance of each proposed X Index to Fidelity of Simulation

7 Wooly Terms (II) Signal to Noise Ratio How is this term so differently interpreted in the climate context (when it has nothing to do with observational noise on a precise physical signal) How might it be applied to rare events in an rapidly evolving system? Internal Variability, Natural Variability, Natural Internal Variability Natural variability need not reduce skill relative to a surrogate (no physical simulation) model, it may or may not reduce skill against a perfect model, that is not our problem. It is a mistake to say natural variability widens a distribution the target trajectory displays change, the model distributions are too narrow to capture this If a simulation model has low skill relative to physics free statistical models this suggests low fidelity of the simulation model (reducing its utility in attribution all and in prediction) Model inadequacy can result in low fidelity simulation and poorly simulation of real world variability this : Reduces skill of the simulation model at all lead times Leads to model irrelevance in long range due to mis drivenfeedbacks Seriously degrades relevance any counterfactual reference class from model distributions

8 Common misperceptions Current literature is confused/inconsistent regarding properties of simulation and predictability. chaotic dynamics prevents... longer/shorter term statements Internal Variability, Natural Variability, Natural Internal Variability Confusion of DGM (System) with the target(s) of a Model(s) influences external to the atmosphere Properties of Mixing Systems, in particularly as parameters evolve. In principle challenges of model inadequacy. TFS, S02

9 Simulation Diversity is not Real world Uncertainty

10 AIM: Necessary Condition List(s) Can we construct a list of necessary conditions that allows us to discard packages of unviable approaches/aims quickly. For a given event (class), for a given technique, the following necessary conditions must be met for (this given approach to) Attribution i,j,k (hereafter: package) to fulfil its promise: 1) 2) 3)... 16) Indications of failure (in practice) of this package include: 1) 2)... 4) Not Are our models good enough? but rather What is required for a model to be good enough? and how could we test this in practice?

11 Common, nontrivial, systematic errors This map shows what is missing in HadCM3 Including 2km walls of rock (Andes). All 2014 GCMs suffer from related inadequacies: Common Technology

12 NAG Weather Aug 13 Aug 14 Aug 15 Aug 16 Aug 17 and so on 8 Oct 2015 National escience Symposium Doing escience in the Dark

What exactly does my distribution of 1024 golf balls tell us about the one (and only) red rubber ball?

13 Diversity is not Uncertainty In the NAG board, probability forecasting corresponds to predicting with a collection (ensemble) of golf balls Ensembles inform us of uncertainty growth within our model(s)! But reality is not a golf ball reality is a red rubber ball. What exactly does my distribution of 1024 golf balls tell us about the one (and only) red rubber ball? While we never see similar initial states, we can still learn from our mistakes!(in this weather-like case) 8 Oct 2015 National escience Symposium Doing escience in the Dark

14 Challenges Reference Class Problem (RCP) Hajek No separation of causes: Fully coupled rapidly mixing System with unknowable drivers (light cone) Transient trajectory/pathway with no sense of either repeatability or a population. In principle Challenges: The levels of similarity of physics and reality Mullin Even hi fidelity models of laboratory systems do not show like with like behaviour Uniqueness of time/trajectory Beven in a system best modelled as nonlinear, rapidly mixing and undergoing directed parameter change Structural Stability of nonlinear dynamical systems (in our case, the models) S02 In practice challenges: Low fidelity of current simulation models Observational constraints regarding rare events in a unique rapidly evolving system. Application of Causal Inference to populations of One (with no high fidelity model). The desire for best available without regard for adequacy for purpose. (leading to the perpetual motion machine defence )

15 The connection between our models and our target. The definitions we adopt determine which questions make sense and which aims are achievable. Distinguishing the (four+) relevant attribution in play. Criticisms of causal Inference only in context of systems best modelled nonlinear, rapidly mixing, and currently at low fidelity. Current literature is confused regarding properties of simulation and predictability Can we construct a list of necessary conditions that allows us to discard unviable approaches/aims quickly.

16 References A03: Allen M (2003) Nature HvS: Von Storch H (2014) Climate Research HPONG: Hannart et al (in press) BAMS L&D: James R et al (2014) Nature Cliamte Change Mullin: Mullin T (1991) IMA J Appl Maths Pearl: Pearl J (2015) An Introduction to Causal Inference DOI: / RCP: Hajek A et al (2007) Synthese S02: Smith, LA (2002) PNAS Chaos: Smith LA (2007) A Very Short Introduction to Chaos OUP. Solow: Solow AR (2015) Science TFS: Trenberth K et al (2015) NCC UoP: Beven K (2000) Hydro and Earth Systems, Hydro Process (2011) Tag lines The plumage don t enter into it. (Discussion of nice irrelevant properties) Epidemiology given a population of one. Uniqueness of time/trajectory Model Fidelity and Probabilistic Adequacy (of even excellent physical models.) Causality in systems best modelled as rapidly mixing and nonlinear Confusing the properties of classes of models with that of the target system [light cone] Considering distributions which exist only in (classes of) models, not in the target system. Arguing for best available independent of evidence of adequate for purpose.

Abstract: Queuing the Wrong U

Abstract: Queuing the Wrong U Just as there are many different types of uncertainty, there are many different types of models. The best technique for quantifying and communicating uncertainty will depend