Model parameters of chaotic dynamics: metrics for comparing trajectories
|
|
- Allen Simpson
- 5 years ago
- Views:
Transcription
1 Model parameters of chaotic dynamics: metrics for comparing trajectories H. Haario 1,2, L.Kalachev 3, J. Hakkarainen 2, A. Bibov 1, FMI people June 25, Lappeenranta University of Technology, Finland 2 Finnish Meteorological Institute (FMI) 3 University of Montana, USA
2 Contents No standard likelihood due to chaoticity Likelihoods based on Summary statistics State space filtering Metrics based on fractal dimension concepts Examples: Lorenz, Shallow Water
3 Chaotic Systems: construction of the likelihood (cost function)? state variable time Small changes in the initial values (or solver settings) can lead to unpredictable deviations from the observations.
4 Likelihood: 1st attempts, summary statistics Observations and simulations are transformed to summary statistics: s = s(y 1:n ), s θ = s(z θ,1:n ) Likelihood is formulated for s, which yields the posterior p (θ s) p(θ)p(s θ)
5 Likelihood: 1st attempts, summary statistics Observations and simulations are transformed to summary statistics: s = s(y 1:n ), s θ = s(z θ,1:n ) Likelihood is formulated for s, which yields the posterior p (θ s) p(θ)p(s θ) The approach was implemented for the ECHAM5 climate model, using likelihood based on monthly global and zonal net radiation averages. MCMC was used to estimate four parameters related to cloud formation and precipitation
6 Climate model MCMC results CMFCTOP.4.2 CAULOC.15 CMFCTOP CPRCON.1.5 ENTRSCV 6 x CPRCON
7 Climate model MCMC results CMFCTOP.4.2 CAULOC.15 CMFCTOP CPRCON.1.5 ENTRSCV 6 x CPRCON The summary statistics do not identify the parameters
8 6.2. Individual Climate Variables Chapter 6. Performance Metrics Applied Nonsmooth cost function log(cmfctop) log(cmfctop) J1 with t log(cauloc) J3 with t J2 with t log(cauloc) log(cauloc) (a) the large perturbations of the parameters log(cmfctop) log(cmfctop) log(cauloc) J4 with t x J1 with t5 1.2 x 1 5 J2 with t5 1 5 fault 5 1 fault 5
9 6.2. Individual Climate Variables Chapter 6. Performance Metrics Applied Nonsmooth cost function log(cmfctop) log(cmfctop) J1 with t log(cauloc) J3 with t J2 with t log(cauloc) log(cauloc) (a) the large perturbations of the parameters log(cmfctop) log(cmfctop) log(cauloc) J4 with t x 1 5 x 1 5 J1 with t5 J2 with t projections: 5 helps but does 1 bot 5 solve the problem Data Analysis for summary cost functions to get informative fault fault
10 EKF Likelihood: by State Space filtering Tame chaos by filtering out the state space. Use state estimation methods to keep the model close to the data. Filtering: integrate out the state space, what remains gives a likelihood for parameters. Standard way for linear time series (DLM, Dynamical Linear Models) and SDE (stochastic differential equations) systems. Less standard for chaotic dynamics, but can be implemented with EKF. 1 Lorenz95 system Not assimilated Assimilated Observations 5 State variable Time
11 The toy model: parameterized Lorenz dx k dt NATURE: = x k 1 (x k 2 x k+1) x k + F hc b Jk j=j(k 1)+1 dy j dt = cbyj+1 (yj+2 yj 1) cyj + c hc Fy + b b x 1+ j 1 J y j FORECAST MODEL: dx k = x k 1 (x k 2 x k+1) x k + F g(x k, θ) dt We use a polynomial parameterization: g(x k, θ) = d i=1 θ ix (i 1) k
12 The toy model: parameterized Lorenz forecast model slow state fast state 6 4 state variable time
13 Results: filter likelihood θ.12 θ θ 1 log(σ 2 ) Scattering plot of parameter pairs from MCMC runs using 1 (blue), 2 (red), 5 (black) and 5 (green) day simulations.
14 Comments, so far The summary statistics approach have problems in properly identifying the parameters of chaotic dynamical systems The likelihood can be computed by integrating out the uncertain model states using filtering methods. Each filter algorithm has built-in tuning parameters (model error covariance, linearization...). The amount of bias introduced by them? Filtering rather avoids the problem than solves it.
15 Metric based on fractal dimension concepts Several concepts exist to define the fractal dimension of a chaotic trajectory, such as the Hausdorff dimension or box-counting. We employ the concept of Correlation Dimension (CD), as it is simple for computations. Recall first the Correlation Integral:
16 Denote by s i, i = 1, 2,..., N points of a trajectory vector s R n, evaluated at time points t i. For R > set C(R, N) = 1/N 2 i,j #( s i s j < R) and define then the correlation integral as the limit C(R) = lim N C(R, N). So we take the total number of points closer than R, normalize by the number of pairs N 2 and take the limit. Not that for each N we have 1/N C(R, N) 1. If ν is the dimension of the trajectory, we should have C(R) R ν and the Correlation Dimension ν is defined as the limit ν = lim R log C(R)/ log(r).
17 Numerical estimation of Correlation Dimension In numerical practice, we have a finite time interval [, T ] the trajectory vector s i is evaluated on a finite number of time instants t i, i = 1, 2,..., N. The above limit may be approximated by a log-log plot obtained by computing log C(R) at various values of R. A few constants to be selected first: The number of points, N. The maximum radius R, large enough for each ball B(s i, R) to contain all the points s j, j i The set of smaller radii by R k = b k R, with k = 1, 2,..., M. Select M and the base b. Typical defaults M = 1, b = 2. Then: For each k, compute the C(R k, N) Create the log-log curve log(r k ) vs log C(R k, N), k = 1, 2,...N. Estimate ν from the linear part of the slope.
18 CD for distance between trajectories The above CD is a standard method to calculate the dimension of a (chaotic) trajectory. Here, we want to modify it to get a measure for the distance between two model trajectories, as given, e.g., with different model parameters. Due to chaoticity, even small differences in initial values or numerical solvers change the trajectories. We want to separate this variability with fixed model parameters from that due to different model parameters.
19 Distance via a generalized correlation sum Fix again the numerical tuning factors (T,N,R,b,M) to cover the range of the trajectory s. The generalized correlation sum between trajectories s = s(θ, x) and s = s( θ, x) is then defined as C(R, N, θ, x, θ, x) = 1/N 2 i,j #( s i s j < R), (1) where θ, θ denote the respective model parameters and x, x the initial values. For θ = θ, x = x the formula reduced to the original definition of the correlation sum.
20 Correlation Curve variability with fixed model parameter First, characterize the within variability of a chaotic dynamical system with fixed model parameter vector: 1. Repeatedly simulate the trajectory, with varying initial values (and solver tolerances), but fixed model parameter θ. 2. Compute the distance matrix between (all) different trajectory pairs, to get the values C(R, N, θ, x, θ, x). An example for Lorenz3, with a log-scale for R:
21 Cost function for parameter estimation We treat the above vectors y = C(R k, N, θ, x, θ, x), k = 1,..., M as measurements of the variability of a chaotic trajectory with a given fixed model parameter. Construct the respective likelihood: 1. Empirically estimate the statistics of y = C(R, N, θ, x, θ, x) from repeated simulations. 2. Create the empirical likelihood function. 3. For any trajectory s(θ) compute the distance matrix from the reference trajectory, and the respective C(R k, N, θ, x, θ, x). Evaluate the likelihood.
22 Example: Likelihood for 3D Lorenz dx dt = σ(y X), dy dt = X(ρ Z) Y, dz dt = XY βz. (2) TIME Figure: Observation samples of 3D Lorenz.
23 Example: Likelihood for 3D Lorenz The values y k = C(R k, N, θ, x, θ, x), k = 1,..., M are averages of distances between state vectors. In analogy with the Central Limit Theorem, test a Gaussian distribution for the vector y: calculate the mean value µ and covariance matrix Σ of the training set. Compute the statistics of the expression (µ y)σ 1 (µ y), should obey the χ 2 distribution for a Gaussian y, (µ y)σ 1 (µ y) χ 2 M (3)
24 Example: Likelihood for 3D Lorenz.12 χ 2 distribution, dof: 1.35 χ 2 distribution, dof: Figure: Normality check of the correlation integral vector by the χ 2 test for the Lorenz 63 system. Left: with 1 radius values used. Right: with 92 radius values
25 Inference as a pseudo-marginal MCMC algorithm Due to chaoticity and randomised x the likelihood is non-deterministic. But sampling from can be interpreted as sampling from the joint distribution of the initial values and model parameters. Denote the likelihood function of y, evaluated for an arbitrary θ by T θ (θ, x). The target distribution for for θ is given as π(θ) = T θ (θ, x)λ(x)dx, where λ(x) is the distribution of the initial values x. In our situation, T θ (θ, x) is unknown, but an empirical approximation can be created as above. The method we implement is a bivariate Markov chain: (θ n, T n ) n, where T n are auxiliary variables that are non-negative, unbiased estimators of the underlying intractable target density π(θ n ). In other words, the method is a pseudo-marginal algorithm targeting π.
26 Pseudo-marginal MCMC Start from a pair (θ, T ) and iterate the following steps for n : 1. Propose θ = θ n + Z, where Z is sampled from a Gaussian proposal distribution. 2. Propose x λ and calculate T = T θ (θ, x ). 3. With probability min { 1, T /T n } accept and set (θ n+1, T n+1 ) = (θ, T ); otherwise reject and set (θ n+1, T n+1 ) = (θ n, T n ). In our cases T is non-negative (Gaussian), and the conditional expectation of T given θ is π(θ ). Therefore, the method provides correct simulation in the sense that the ergodic averages n 1 n k=1 f(θ k) converge to π(f) almost surely given minimal irreducibility and aperiodicity assumptions
27 Example: 3D Lorenz β 1.5 σ σ ρ Figure: Marginal distributions of model parameters obtained using MCMC simulations for the three dimensional Lorenz system are very close to Gaussian.
28 Example: L95 2 Lorenz 95 MCMC results (no splitting) h F The sampled values for the parameters (F, h) (only, keeping other model parameters fixed) of L95 when the correlation integral vector is computed from the whole system s = (x k, y j ), i.e., the slow and fast, weakly coupled subsystems together.
29 Example: L95 Lorenz 95 MCMC results h F The sampled values for the parameters (F, h) of L95 when the correlation integral vector calculations are splitted, separately computed for the slow and fast subsystems, and both vectors used as data for the sampling cost function.
30 High dimension: Shallow water model Given as h t + (hu) x + (hv) y =, (hu) t + ( hu gh2) x + (huv) y = ghb x, (hv t ) + (huv) x + ( hu gh2) y = ghb y. Here h denotes water elevation, u and v are horizontal and vertical velocity components, B x and B y denote gradient direction of the surface implementing topography, and g is acceleration of gravity. It is possible to account for additional phenomena (e.g. wind stresses, friction etc.) by playing with the right-hand-side part of the equations
31 Discretization by finite volumes Numerics: Kurganov-Petrova second-order well-balanced positivity preserving central-upwind scheme The problem is solved for a huge set of discretization cells that form a staggered grid. Each cell describes solution by 5 components, where each component contains related data for velocity field and water elevation
32 Introduction to CUDA Idea: move computations to the GPU side Each GPU computes information for billions of screen pixels almost independently GPUs are highly parallel Modern GPUs comprise thousands of universal computation cores GPUs can be arranged into arrays forming a multi-gpu node Simple application interfaces that allow general purpose GPU programming exist (e.g. OpenCL, OpenGL Compute Shaders, CUDA etc.)
33 CPU vs CUDA GPU implementation Both implementations run at resolution of 256-by-256 grid cells Time step is.6 units of model time Time cost of a single model step for CPU implementation:.365 sec Time cost of a single model step for GPU implementation:,5 sec
34 SWE: can you distinguish change of flow pattern? A stone in river, with Van Karman vortex shedding. Dimension of the state (h, u, v) around 2.. Two slightly different cases. Example:
35 SWE: can you distinguish change of flow pattern? Snapshots at 1 time points, repeat simulations 5 times (producing a training set of 1225 pairs). Split the simulated vectors in two parts, use 1/2 as the training set, 1/2 as test set:.5 KHI2 NORMALITY TEST OF TRAINING VECTORS KHI2 SIMILARITY TEST FOR NEW CASES
36 Where is the real data? No measured data is directly used for parameter estimation. Instead, assume basic model parameters given, and want to determine the posterior of parameters that would produce essentially the same chaotic dynamics. A real example: reanalysis studies of weather and climate models (e.g., the ERA-4 data and ECHAM5), that combine past real data and model predictions to achieve the best understanding of the systems. The aim here: characterize the parameter distributions of the reanalyzed models, that fit the climatology of long time runs of a given climate model. Further use them to quantify the uncertainty of model predictions with respect to the given parameters, by parameter ensemble simulations under various scenarios, such as increased CO 2 levels.
37 Summary, next The Correlation distance a promising way to characterize distances between chaotic trajectories. Quite insensitive with respect to varying initial values, solver numerics, etc., in small systems Next: more applications to high dimensional systems. No technical obstacles, in principle: Only L2 norms between vectors computed after model simulation. Can be done in parallel, during the simulation. From K simulations get K(K 1)/2 trajectory pairs to create the empirical distributions: moderate K enough. Problems/modifications expected for multi fractal situations, integration times: Careful with rare but large outliers in the training set. Distinguish, generally, a difference from normal behaviour: use for various classification, pattern recognition problems?
38 References Järvinen, H., Räisänen, P., Laine, M., Tamminen, J., Ilin, A., Oja, E., Solonen, A., and Haario, H.: Estimation of ECHAM5 climate model closure parameters with adaptive MCMC, Atmos. Chem. Phys., Vol. 1, nro. 2, , 21. J. Hakkarainen, A. Ilin, S. Solonen, M. Laine, H. Haario, J. Tamminen, E. Oja, H. Järvinen: On Closure Parameter Estimation in Chaotic Systems. Nonlin. Processes Geophys., 19, , 212. A. Solonen, P. Ollinaho, M. Laine, H. Haario, J. Tamminen, H. Järvinen: Efficient MCMC for Climate model Parameter Estimation: Parallel Adaptive chains and Early Rejection. Bayesian Analysis, 7, Number 2, pp 1 22, 212. Janne Hakkarainen, Antti Solonen, Alexander Ilin, Jouni Susiluoto, Marko Laine, Heikki Haario and Heikki Järvinen. A dilemma on the uniqueness of weather and climate model closure parameters. Tellus A 213, 65, Ollinaho P, Laine M, Solonen A,Haario H, Järvinen H.:NWP model forecast skill optimization via closure parameter variations. Q. J. R. Meteorol. Soc., 139, 675, pp , 213. Ollinaho, P., Bechtold, P., Leutbecher, M., Laine, M., Solonen, A., Haario, H., and Järvinen, H.: Parameter variations in prediction skill optimization at ECMWF, Nonlin. Processes Geophys., 2, 6,11-11, 213. Heikki Haario, Leonid Kalachev, Janne Hakkarainen Generalized Correlation Integral Vectors: A New Distance Concept for Chaotic Dynamical Systems. UM Tech.Rep 8/214. ( Generalized Correlation integral vectors: A distance concept for chaotic dynamical systems. Chaos: An Interdisciplinary Journal of Nonlinear Science 25, 6312, 215; doi: 1.163/
Advanced uncertainty evaluation of climate models by Monte Carlo methods
Advanced uncertainty evaluation of climate models by Monte Carlo methods Marko Laine marko.laine@fmi.fi Pirkka Ollinaho, Janne Hakkarainen, Johanna Tamminen, Heikki Järvinen (FMI) Antti Solonen, Heikki
More informationCorrelation integral likelihood for stochastic differential equations
Correlation integral likelihood for stochastic differential equations Heikki Haario, Janne Hakkarainen, Ramona Maraia, Sebastian Springer Abstract A new approach was recently introduced for the task of
More informationJanne Hakkarainen ON STATE AND PARAMETER ESTIMATION IN CHAOTIC SYSTEMS. Acta Universitatis Lappeenrantaensis 545
Janne Hakkarainen ON STATE AND PARAMETER ESTIMATION IN CHAOTIC SYSTEMS Thesis for the degree of Doctor of Philosophy to be presented with due permission for public examination and criticism in the Auditorium
More informationNWP model forecast skill optimization via closure parameter variations
Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 139: 1520 1532, July 2013 B NWP model forecast skill optimization via closure parameter variations Pirkka Ollinaho, a,b * Marko
More informationStability of Ensemble Kalman Filters
Stability of Ensemble Kalman Filters Idrissa S. Amour, Zubeda Mussa, Alexander Bibov, Antti Solonen, John Bardsley, Heikki Haario and Tuomo Kauranne Lappeenranta University of Technology University of
More informationOn closure parameter estimation in chaotic systems
Nonlin. Processes Geophys., 19, 17 143, 1 www.nonlin-processes-geophys.net/19/17/1/ doi:1.5194/npg-19-17-1 Author(s) 1. CC Attribution 3. License. Nonlinear Processes in Geophysics On closure parameter
More informationEMPIRICAL EVALUATION OF BAYESIAN OPTIMIZATION IN PARAMETRIC TUNING OF CHAOTIC SYSTEMS
Pre-print submitted to the International Journal for Uncertainty Quantification, 6 EMPIRICAL EVALUATION OF BAYESIAN OPTIMIZATION IN PARAMETRIC TUNING OF CHAOTIC SYSTEMS Mudassar Abbas,, Alexander Ilin,
More informationA dilemma of the uniqueness of weather and climate model closure parameters
https://helda.helsinki.fi A dilemma of the uniqueness of weather and climate model closure parameters Hakkarainen, Janne 2013 Hakkarainen, J, Solonen, A, Ilin, A, Susiluoto, J, Laine, M, Haario, H & Järvinen,
More informationStochastic Collocation Methods for Polynomial Chaos: Analysis and Applications
Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Dongbin Xiu Department of Mathematics, Purdue University Support: AFOSR FA955-8-1-353 (Computational Math) SF CAREER DMS-64535
More informationExploring and extending the limits of weather predictability? Antje Weisheimer
Exploring and extending the limits of weather predictability? Antje Weisheimer Arnt Eliassen s legacy for NWP ECMWF is an independent intergovernmental organisation supported by 34 states. ECMWF produces
More informationParameter variations in prediction skill optimization at ECMWF
https://helda.helsinki.fi Parameter variations in prediction skill optimization at ECMWF Ollinaho, P. 03 Ollinaho, P, Bechtold, P, Leutbecher, M, Laine, M, Solonen, A, Haario, H & Järvinen, H 03, ' Parameter
More informationEstimation of ECHAM5 climate model closure parameters with adaptive MCMC
doi:10.5194/acp-10-9993-2010 Author(s) 2010. CC Attribution 3.0 License. Atmospheric Chemistry and Physics Estimation of ECHAM5 climate model closure parameters with adaptive MCMC H. Järvinen 1, P. Räisänen
More informationInformation geometry for bivariate distribution control
Information geometry for bivariate distribution control C.T.J.Dodson + Hong Wang Mathematics + Control Systems Centre, University of Manchester Institute of Science and Technology Optimal control of stochastic
More informationShort tutorial on data assimilation
Mitglied der Helmholtz-Gemeinschaft Short tutorial on data assimilation 23 June 2015 Wolfgang Kurtz & Harrie-Jan Hendricks Franssen Institute of Bio- and Geosciences IBG-3 (Agrosphere), Forschungszentrum
More informationSmoothers: Types and Benchmarks
Smoothers: Types and Benchmarks Patrick N. Raanes Oxford University, NERSC 8th International EnKF Workshop May 27, 2013 Chris Farmer, Irene Moroz Laurent Bertino NERSC Geir Evensen Abstract Talk builds
More informationMarkov chain Monte Carlo methods in atmospheric remote sensing
1 / 45 Markov chain Monte Carlo methods in atmospheric remote sensing Johanna Tamminen johanna.tamminen@fmi.fi ESA Summer School on Earth System Monitoring and Modeling July 3 Aug 11, 212, Frascati July,
More informationA Note on the Particle Filter with Posterior Gaussian Resampling
Tellus (6), 8A, 46 46 Copyright C Blackwell Munksgaard, 6 Printed in Singapore. All rights reserved TELLUS A Note on the Particle Filter with Posterior Gaussian Resampling By X. XIONG 1,I.M.NAVON 1,2 and
More informationSTA 294: Stochastic Processes & Bayesian Nonparametrics
MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a
More informationDevelopment of Stochastic Artificial Neural Networks for Hydrological Prediction
Development of Stochastic Artificial Neural Networks for Hydrological Prediction G. B. Kingston, M. F. Lambert and H. R. Maier Centre for Applied Modelling in Water Engineering, School of Civil and Environmental
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationProf. Stephen G. Penny University of Maryland NOAA/NCEP, RIKEN AICS, ECMWF US CLIVAR Summit, 9 August 2017
COUPLED DATA ASSIMILATION: What we need from observations and modellers to make coupled data assimilation the new standard for prediction and reanalysis. Prof. Stephen G. Penny University of Maryland NOAA/NCEP,
More informationOrganization. I MCMC discussion. I project talks. I Lecture.
Organization I MCMC discussion I project talks. I Lecture. Content I Uncertainty Propagation Overview I Forward-Backward with an Ensemble I Model Reduction (Intro) Uncertainty Propagation in Causal Systems
More informationESTIMATING THE ATTRACTOR DIMENSION OF THE EQUATORIAL WEATHER SYSTEM M. Leok B.T.
This paper was awarded in the I International Competition (99/9) First Step to Nobel Prize in Physics and published in the competition proceedings (Acta Phys. Pol. A 8 Supplement, S- (99)). The paper is
More informationChapter 6: Ensemble Forecasting and Atmospheric Predictability. Introduction
Chapter 6: Ensemble Forecasting and Atmospheric Predictability Introduction Deterministic Chaos (what!?) In 1951 Charney indicated that forecast skill would break down, but he attributed it to model errors
More informationEnsemble forecasting and flow-dependent estimates of initial uncertainty. Martin Leutbecher
Ensemble forecasting and flow-dependent estimates of initial uncertainty Martin Leutbecher acknowledgements: Roberto Buizza, Lars Isaksen Flow-dependent aspects of data assimilation, ECMWF 11 13 June 2007
More informationApproximate Bayesian Computation and Particle Filters
Approximate Bayesian Computation and Particle Filters Dennis Prangle Reading University 5th February 2014 Introduction Talk is mostly a literature review A few comments on my own ongoing research See Jasra
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
More informationEdward Lorenz: Predictability
Edward Lorenz: Predictability Master Literature Seminar, speaker: Josef Schröttle Edward Lorenz in 1994, Northern Hemisphere, Lorenz Attractor I) Lorenz, E.N.: Deterministic Nonperiodic Flow (JAS, 1963)
More informationErgodicity in data assimilation methods
Ergodicity in data assimilation methods David Kelly Andy Majda Xin Tong Courant Institute New York University New York NY www.dtbkelly.com April 15, 2016 ETH Zurich David Kelly (CIMS) Data assimilation
More informationA Global Atmospheric Model. Joe Tribbia NCAR Turbulence Summer School July 2008
A Global Atmospheric Model Joe Tribbia NCAR Turbulence Summer School July 2008 Outline Broad overview of what is in a global climate/weather model of the atmosphere Spectral dynamical core Some results-climate
More informationAn introduction to Bayesian statistics and model calibration and a host of related topics
An introduction to Bayesian statistics and model calibration and a host of related topics Derek Bingham Statistics and Actuarial Science Simon Fraser University Cast of thousands have participated in the
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationIntroduction to Restricted Boltzmann Machines
Introduction to Restricted Boltzmann Machines Ilija Bogunovic and Edo Collins EPFL {ilija.bogunovic,edo.collins}@epfl.ch October 13, 2014 Introduction Ingredients: 1. Probabilistic graphical models (undirected,
More informationAdaptive Sampling of Clouds with a Fleet of UAVs: Improving Gaussian Process Regression by Including Prior Knowledge
Master s Thesis Presentation Adaptive Sampling of Clouds with a Fleet of UAVs: Improving Gaussian Process Regression by Including Prior Knowledge Diego Selle (RIS @ LAAS-CNRS, RT-TUM) Master s Thesis Presentation
More informationForecasting Wind Ramps
Forecasting Wind Ramps Erin Summers and Anand Subramanian Jan 5, 20 Introduction The recent increase in the number of wind power producers has necessitated changes in the methods power system operators
More informationBayesian Networks: Construction, Inference, Learning and Causal Interpretation. Volker Tresp Summer 2016
Bayesian Networks: Construction, Inference, Learning and Causal Interpretation Volker Tresp Summer 2016 1 Introduction So far we were mostly concerned with supervised learning: we predicted one or several
More informationMonte Carlo in Bayesian Statistics
Monte Carlo in Bayesian Statistics Matthew Thomas SAMBa - University of Bath m.l.thomas@bath.ac.uk December 4, 2014 Matthew Thomas (SAMBa) Monte Carlo in Bayesian Statistics December 4, 2014 1 / 16 Overview
More informationA graph contains a set of nodes (vertices) connected by links (edges or arcs)
BOLTZMANN MACHINES Generative Models Graphical Models A graph contains a set of nodes (vertices) connected by links (edges or arcs) In a probabilistic graphical model, each node represents a random variable,
More informationComputer Practical: Metropolis-Hastings-based MCMC
Computer Practical: Metropolis-Hastings-based MCMC Andrea Arnold and Franz Hamilton North Carolina State University July 30, 2016 A. Arnold / F. Hamilton (NCSU) MH-based MCMC July 30, 2016 1 / 19 Markov
More informationBayesian Calibration of Simulators with Structured Discretization Uncertainty
Bayesian Calibration of Simulators with Structured Discretization Uncertainty Oksana A. Chkrebtii Department of Statistics, The Ohio State University Joint work with Matthew T. Pratola (Statistics, The
More informationStochastic methods for representing atmospheric model uncertainties in ECMWF's IFS model
Stochastic methods for representing atmospheric model uncertainties in ECMWF's IFS model Sarah-Jane Lock Model Uncertainty, Research Department, ECMWF With thanks to Martin Leutbecher, Simon Lang, Pirkka
More informationBayesian rules of probability as principles of logic [Cox] Notation: pr(x I) is the probability (or pdf) of x being true given information I
Bayesian rules of probability as principles of logic [Cox] Notation: pr(x I) is the probability (or pdf) of x being true given information I 1 Sum rule: If set {x i } is exhaustive and exclusive, pr(x
More informationOPTIMIZATION-BASED SAMPLING IN ENSEMBLE KALMAN FILTERING
International Journal for Uncertainty Quantification, 1(1):xxx xxx, 2013 OPTIMIZATION-BASED SAMPLING IN ENSEMBLE KALMAN FILTERING Antti Solonen, 1,3, Alexander Bibov, 1 Johnathan M. Bardsley, 2 Haario
More informationThe Planetary Boundary Layer and Uncertainty in Lower Boundary Conditions
The Planetary Boundary Layer and Uncertainty in Lower Boundary Conditions Joshua Hacker National Center for Atmospheric Research hacker@ucar.edu Topics The closure problem and physical parameterizations
More informationMCMC Sampling for Bayesian Inference using L1-type Priors
MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling
More information5. General Circulation Models
5. General Circulation Models I. 3-D Climate Models (General Circulation Models) To include the full three-dimensional aspect of climate, including the calculation of the dynamical transports, requires
More informationStable Semi-Discrete Schemes for the 2D Incompressible Euler Equations
Stable Semi-Discrete Schemes for the D Incompressible Euler Equations Doron Levy Department of Mathematics Stanford University dlevy@math.stanford.edu http://math.stanford.edu/ dlevy Maryland, May 004
More informationWeb Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.
Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we
More informationEstimating model evidence using data assimilation
Estimating model evidence using data assimilation Application to the attribution of climate-related events Marc Bocquet 1, Alberto Carrassi 2, Alexis Hannart 3 & Michael Ghil 4,5 and the DADA Team (1 CEREA,
More informationDrought forecasting methods Blaz Kurnik DESERT Action JRC
Ljubljana on 24 September 2009 1 st DMCSEE JRC Workshop on Drought Monitoring 1 Drought forecasting methods Blaz Kurnik DESERT Action JRC Motivations for drought forecasting Ljubljana on 24 September 2009
More informationNumerical Solutions of ODEs by Gaussian (Kalman) Filtering
Numerical Solutions of ODEs by Gaussian (Kalman) Filtering Hans Kersting joint work with Michael Schober, Philipp Hennig, Tim Sullivan and Han C. Lie SIAM CSE, Atlanta March 1, 2017 Emmy Noether Group
More informationWe honor Ed Lorenz ( ) who started the whole new science of predictability
Atmospheric Predictability: From Basic Theory to Forecasting Practice. Eugenia Kalnay Alghero, May 2008, Lecture 1 We honor Ed Lorenz (1917-2008) who started the whole new science of predictability Ed
More informationSequential Monte Carlo Samplers for Applications in High Dimensions
Sequential Monte Carlo Samplers for Applications in High Dimensions Alexandros Beskos National University of Singapore KAUST, 26th February 2014 Joint work with: Dan Crisan, Ajay Jasra, Nik Kantas, Alex
More informationLagrangian Data Assimilation and Its Application to Geophysical Fluid Flows
Lagrangian Data Assimilation and Its Application to Geophysical Fluid Flows Laura Slivinski June, 3 Laura Slivinski (Brown University) Lagrangian Data Assimilation June, 3 / 3 Data Assimilation Setup:
More informationMechanisms of Chaos: Stable Instability
Mechanisms of Chaos: Stable Instability Reading for this lecture: NDAC, Sec. 2.-2.3, 9.3, and.5. Unpredictability: Orbit complicated: difficult to follow Repeatedly convergent and divergent Net amplification
More informationModel Uncertainty Quantification for Data Assimilation in partially observed Lorenz 96
Model Uncertainty Quantification for Data Assimilation in partially observed Lorenz 96 Sahani Pathiraja, Peter Jan Van Leeuwen Institut für Mathematik Universität Potsdam With thanks: Sebastian Reich,
More information5.3 METABOLIC NETWORKS 193. P (x i P a (x i )) (5.30) i=1
5.3 METABOLIC NETWORKS 193 5.3 Metabolic Networks 5.4 Bayesian Networks Let G = (V, E) be a directed acyclic graph. We assume that the vertices i V (1 i n) represent for example genes and correspond to
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationData assimilation in high dimensions
Data assimilation in high dimensions David Kelly Courant Institute New York University New York NY www.dtbkelly.com February 12, 2015 Graduate seminar, CIMS David Kelly (CIMS) Data assimilation February
More informationMesoscale Predictability of Terrain Induced Flows
Mesoscale Predictability of Terrain Induced Flows Dale R. Durran University of Washington Dept. of Atmospheric Sciences Box 3516 Seattle, WA 98195 phone: (206) 543-74 fax: (206) 543-0308 email: durrand@atmos.washington.edu
More informationData assimilation in high dimensions
Data assimilation in high dimensions David Kelly Kody Law Andy Majda Andrew Stuart Xin Tong Courant Institute New York University New York NY www.dtbkelly.com February 3, 2016 DPMMS, University of Cambridge
More informationHierarchical Bayesian Modeling and Analysis: Why and How
Hierarchical Bayesian Modeling and Analysis: Why and How Mark Berliner Department of Statistics The Ohio State University IMA June 10, 2011 Outline I. Bayesian Hierarchical Modeling II. Examples: Glacial
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationApplications of Hurst Coefficient Analysis to Chaotic Response of ODE Systems: Part 1a, The Original Lorenz System of 1963
Applications of Hurst Coefficient Analysis to Chaotic Response of ODE Systems: Part 1a, The Original Lorenz System of 1963 Dan Hughes December 2008 Abstract I have coded the process for calculating Hurst
More informationAn ABC interpretation of the multiple auxiliary variable method
School of Mathematical and Physical Sciences Department of Mathematics and Statistics Preprint MPS-2016-07 27 April 2016 An ABC interpretation of the multiple auxiliary variable method by Dennis Prangle
More informationan introduction to bayesian inference
with an application to network analysis http://jakehofman.com january 13, 2010 motivation would like models that: provide predictive and explanatory power are complex enough to describe observed phenomena
More informationLagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model. David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC
Lagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC Background Data Assimilation Iterative process Forecast Analysis Background
More informationFundamentals of Data Assimilation
National Center for Atmospheric Research, Boulder, CO USA GSI Data Assimilation Tutorial - June 28-30, 2010 Acknowledgments and References WRFDA Overview (WRF Tutorial Lectures, H. Huang and D. Barker)
More informationEstimation of State Noise for the Ensemble Kalman filter algorithm for 2D shallow water equations.
Estimation of State Noise for the Ensemble Kalman filter algorithm for 2D shallow water equations. May 6, 2009 Motivation Constitutive Equations EnKF algorithm Some results Method Navier Stokes equations
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Centre for Computational Statistics and Machine Learning University College London c.archambeau@cs.ucl.ac.uk CSML
More informationObservability, a Problem in Data Assimilation
Observability, Data Assimilation with the Extended Kalman Filter 1 Observability, a Problem in Data Assimilation Chris Danforth Department of Applied Mathematics and Scientific Computation, UMD March 10,
More informationStatistical Data Mining and Machine Learning Hilary Term 2016
Statistical Data Mining and Machine Learning Hilary Term 2016 Dino Sejdinovic Department of Statistics Oxford Slides and other materials available at: http://www.stats.ox.ac.uk/~sejdinov/sdmml Naïve Bayes
More informationStatistical Methods in Particle Physics Lecture 1: Bayesian methods
Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationA Search and Jump Algorithm for Markov Chain Monte Carlo Sampling. Christopher Jennison. Adriana Ibrahim. Seminar at University of Kuwait
A Search and Jump Algorithm for Markov Chain Monte Carlo Sampling Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Adriana Ibrahim Institute
More informationBred vectors: theory and applications in operational forecasting. Eugenia Kalnay Lecture 3 Alghero, May 2008
Bred vectors: theory and applications in operational forecasting. Eugenia Kalnay Lecture 3 Alghero, May 2008 ca. 1974 Central theorem of chaos (Lorenz, 1960s): a) Unstable systems have finite predictability
More informationSeminar: Data Assimilation
Seminar: Data Assimilation Jonas Latz, Elisabeth Ullmann Chair of Numerical Mathematics (M2) Technical University of Munich Jonas Latz, Elisabeth Ullmann (TUM) Data Assimilation 1 / 28 Prerequisites Bachelor:
More information(Regional) Climate Model Validation
(Regional) Climate Model Validation Francis W. Zwiers Canadian Centre for Climate Modelling and Analysis Atmospheric Environment Service Victoria, BC Outline - three questions What sophisticated validation
More informationDRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING. Inversion basics. Erkki Kyrölä Finnish Meteorological Institute
Inversion basics y = Kx + ε x ˆ = (K T K) 1 K T y Erkki Kyrölä Finnish Meteorological Institute Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 1 Contents 1. Introduction: Measurements, models, inversion
More informationA Spectral Approach to Linear Bayesian Updating
A Spectral Approach to Linear Bayesian Updating Oliver Pajonk 1,2, Bojana V. Rosic 1, Alexander Litvinenko 1, and Hermann G. Matthies 1 1 Institute of Scientific Computing, TU Braunschweig, Germany 2 SPT
More informationDATA ASSIMILATION FOR FLOOD FORECASTING
DATA ASSIMILATION FOR FLOOD FORECASTING Arnold Heemin Delft University of Technology 09/16/14 1 Data assimilation is the incorporation of measurement into a numerical model to improve the model results
More informationBayesian Methods with Monte Carlo Markov Chains II
Bayesian Methods with Monte Carlo Markov Chains II Henry Horng-Shing Lu Institute of Statistics National Chiao Tung University hslu@stat.nctu.edu.tw http://tigpbp.iis.sinica.edu.tw/courses.htm 1 Part 3
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More informationIntroduction to Data Assimilation. Reima Eresmaa Finnish Meteorological Institute
Introduction to Data Assimilation Reima Eresmaa Finnish Meteorological Institute 15 June 2006 Outline 1) The purpose of data assimilation 2) The inputs for data assimilation 3) Analysis methods Theoretical
More informationMethods of Data Assimilation and Comparisons for Lagrangian Data
Methods of Data Assimilation and Comparisons for Lagrangian Data Chris Jones, Warwick and UNC-CH Kayo Ide, UCLA Andrew Stuart, Jochen Voss, Warwick Guillaume Vernieres, UNC-CH Amarjit Budiraja, UNC-CH
More informationComputer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
More informationLECTURE 15 Markov chain Monte Carlo
LECTURE 15 Markov chain Monte Carlo There are many settings when posterior computation is a challenge in that one does not have a closed form expression for the posterior distribution. Markov chain Monte
More informationDaniel J. Jacob, Models of Atmospheric Transport and Chemistry, 2007.
1 0. CHEMICAL TRACER MODELS: AN INTRODUCTION Concentrations of chemicals in the atmosphere are affected by four general types of processes: transport, chemistry, emissions, and deposition. 3-D numerical
More informationBayesian Inference and the Symbolic Dynamics of Deterministic Chaos. Christopher C. Strelioff 1,2 Dr. James P. Crutchfield 2
How Random Bayesian Inference and the Symbolic Dynamics of Deterministic Chaos Christopher C. Strelioff 1,2 Dr. James P. Crutchfield 2 1 Center for Complex Systems Research and Department of Physics University
More informationOn Reparametrization and the Gibbs Sampler
On Reparametrization and the Gibbs Sampler Jorge Carlos Román Department of Mathematics Vanderbilt University James P. Hobert Department of Statistics University of Florida March 2014 Brett Presnell Department
More information2.5 Shallow water equations, quasigeostrophic filtering, and filtering of inertia-gravity waves
Chapter. The continuous equations φ=gh Φ=gH φ s =gh s Fig..5: Schematic of the shallow water model, a hydrostatic, incompressible fluid with a rigid bottom h s (x,y), a free surface h(x,y,t), and horizontal
More informationEnsemble Data Assimilation and Uncertainty Quantification
Ensemble Data Assimilation and Uncertainty Quantification Jeff Anderson National Center for Atmospheric Research pg 1 What is Data Assimilation? Observations combined with a Model forecast + to produce
More informationThe Canadian approach to ensemble prediction
The Canadian approach to ensemble prediction ECMWF 2017 Annual seminar: Ensemble prediction : past, present and future. Pieter Houtekamer Montreal, Canada Overview. The Canadian approach. What are the
More informationCalibration and selection of stochastic palaeoclimate models
Calibration and selection of stochastic palaeoclimate models Michel Crucifix, Jake Carson, Simon Preston, Richard Wilkinson, Jonathan Rougier IRM - Internal Seminar - Uccle 2 April 2014 Posing the problem
More informationDETERMINATION OF MODEL VALID PREDICTION PERIOD USING THE BACKWARD FOKKER-PLANCK EQUATION
.4 DETERMINATION OF MODEL VALID PREDICTION PERIOD USING THE BACKWARD FOKKER-PLANCK EQUATION Peter C. Chu, Leonid M. Ivanov, and C.W. Fan Department of Oceanography Naval Postgraduate School Monterey, California.
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationState and Parameter Estimation in Stochastic Dynamical Models
State and Parameter Estimation in Stochastic Dynamical Models Timothy DelSole George Mason University, Fairfax, Va and Center for Ocean-Land-Atmosphere Studies, Calverton, MD June 21, 2011 1 1 collaboration
More informationExploring stochastic model uncertainty representations
Exploring stochastic model uncertainty representations with relevance to the greyzone Sarah-Jane Lock, Martin Leutbecher, Peter Bechtold, Richard Forbes Research Department, ECMWF ECMWF November 15, 2017
More informationManifold Monte Carlo Methods
Manifold Monte Carlo Methods Mark Girolami Department of Statistical Science University College London Joint work with Ben Calderhead Research Section Ordinary Meeting The Royal Statistical Society October
More informationThe priority program SPP1167 Quantitative Precipitation Forecast PQP and the stochastic view of weather forecasting
The priority program SPP1167 Quantitative Precipitation Forecast PQP and the stochastic view of weather forecasting Andreas Hense 9. November 2007 Overview The priority program SPP1167: mission and structure
More informationSCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA. Sistemi di Elaborazione dell Informazione. Regressione. Ruggero Donida Labati
SCUOLA DI SPECIALIZZAZIONE IN FISICA MEDICA Sistemi di Elaborazione dell Informazione Regressione Ruggero Donida Labati Dipartimento di Informatica via Bramante 65, 26013 Crema (CR), Italy http://homes.di.unimi.it/donida
More information