Adaptive Data Assimilation and Multi-Model Fusion
|
|
- Ethan Simon
- 5 years ago
- Views:
Transcription
1 Adaptive Data Assimilation and Multi-Model Fusion Pierre F.J. Lermusiaux, Oleg G. Logoutov and Patrick J. Haley Jr. Mechanical Engineering and Ocean Science and Engineering, MIT We thank: Allan R. Robinson Wayne G. Leslie AOSN-II and MB06 teams ONR
2 Adaptive Data Assimilation Here, we review and illustrate several of the ESSE adaptable components (see Lermusiaux, Physica D, 2007) Other Adaptive DA references (Blanchet et al, 1997; Menemenlis and Chechelnistky, 2000; etc) Most DA schemes used for realistic studies approximate fundamental principles These DA schemes involve parameters, options and heuristic algorithms whose specifics impact results In the Error Subspace Statistical Estimation system, specifics Vary with each application and with users inputs Are adapted with time, as a function of the available data, regional dynamics or other considerations Data assimilation is said to be adaptive when parameters, functional or schemes used for DA are a (quantitative) function of the measurements: the DA learns from data
3 1) Adaptive Error Covariance Estimation: Adaptive Learning of the Dominant Errors in ESSE
4 Real-time 1996 example See Lermusiaux, DAO (1999)
5 2) Adaptive Error Scaling in ESSE Dominant Error covariance estimate can be scaled by a block diagonal matrix Γ T B=ΓEΠE Γ Each block corresponds to a state variable and is defined by one scaling factor (in atmos., Γ set to a scalar, cov. inflation) Scaling used in the error initialization Presently, scaling is tuned by trial and error (shooting) At t 0, values in Γ are usually set within 0.3 to 0.7 (when EΠE T is set to variability, e.g. Lermusiaux, Anderson and Lozano, 2000). At t 0 and t k s, the tuning of Γ is done by batch. Successive batches are compared to data-model misfits and the initial error increased/decreased accordingly For mesoscale coastal ocean, Γ stabilizes to I after 2-7 DA cycles (days to a week)
6 3) Adaptive Parameterization of the Truncated Errors Uncertainties not represented by the error subspace j modeled by random noise n k For each state variable v, random noise is sum of additive and multiplicative noise: ( j α ) k ( j ε ) k v v ( n j j j j ) k = αk + εkxk : White noise reddened by Shapiro Filter : Non-dimensional white noise factor of small amplitude (1 to 5%) Presently, scaling (increase/decrease) of these parameters by trial and error (shooting) on future data-model misfits v
7 4) Adaptive Ensemble Size, Error Subspace Rank and Stochastic Forcing Size of ensemble controlled in real-time by quantitative criteria Error Subspace rank selected based on Sing. Val. Stochastic forcing parameters should be function of data-model misfits d
8 Sensitivity of T. error correlation estimates to error subspace rank Ens. Size of 500 Ens. Size of 500, Subspace of Rank 100 Ens. Size of 500, Subspace of Rank 300 Ens. Size of 500, Subspace of Rank 20
9 Sensitivity of T. err. cor. estimates to ensemble size and subspace rank Ens. Size of 100 Ens. Size of 500, Subspace of Rank 100 Ens. Size of 100, Subspace of Rank 20 Ens. Size of 500, Subspace of Rank 20
10 Stochastic Primitive Equation Model See Lermusiaux, JCP-2006 The diagonal of time-decorrelations: are here The diagonal of noise variances are chosen function of z only, of amplitude set to: ε * geostrophy
11 Sensitivity of T. err. cor. estimates to stochastic forcing (and error subspace rank) Ens. Size of 500 Ens. Size of 500, with stochastic forcing Ens. Size of 500, Subspace of Rank 100 Ens. Size of 500, stochas. frc., Rank 100
12 Percentage of Variance Explained Normalized, Cumulative Error Variance Red dash-dotted: without stochastic forcing Blue: with stochastic forcing
13 5) Effect of (Adaptive) Schur product of ESSE covariances with a matrix whose values decay with distances T and S profiles assimilated on Aug 28, 2003 ESSE error standard deviation prediction for surface T Reduction (prior posterior) of error standard deviation due to DA of T and S, no tapering by Schur product Error reduction, with tapering by Schur product
14 DA increments (surface T), No tapering DA increments (surface T), With tapering Error Reduction no tapering by Schur product Error reduction, with tapering by Schur product
15 Multi-Model Fusion for Ocean Prediction based on Adaptive Uncertainty Estimation A Methodology for Multi-Model Forecast Fusion Adaptive Uncertainty Estimation Schemes Bias Correction followed by Error Variance Estimation Capable of operating with observational data that are limited and sparse (in space and in time) with respect to the dominant ocean scales Adaptive/sequential, using the small samples of error estimation events (possibly 1 event)
16 Bayesian Multi-Model Fusion Approach And Assumptions Errors = systematic + random components Markovian behavior (past errors are at least partially relevant to future errors) 1) Estimate and correct the biases of each model 2) Estimate the error (co)-variances of each bias-corrected model 3) Optimally combine the states or forecasts Sequential/Adaptive Schemes Uncertainty Estimation from Incomplete Data-Model Misfits 1) Linear sequential bias estimator (of mimimum error variance) 2) Error Variance estimator of minimum mean square error Multi-Model Fusion based on Minimum Error Variance
17 Bayesian Multi-Model Fusion (at any fixed time) Seek multi-model (central) forecast as a linear combination of the individual forecasts, with spatially varying weights (Interpolated on same grid) Weight matrices D found to ensure that central forecast has minimum error variance and is unbiased (For uncorrelated models errors and to ensure unbiased estimator) (Gorokhov and Stoica, IEEE Trans. Sig. Proc., 2000) Logutov, O. G. Multi-Model Fusion and Uncertainty Estimation for Ocean Prediction. Ph.D. dissertation, Harvard University, 2007
18 Employing Bayesian Multi-Model Fusion for Integration of Multiple Models Into a Single Ocean Prediction System Example: 24 hour HOPS/ROMS SST forecast, valid Aug 28, 2003 Consists of combining the individual forecasts based on their relative error variances (defined here as uncertainty) minimizes error variance of multi-model (central) forecast spatially varying diagonal weights have clear interpretation of Bayes factors associated with the individual models
19 Optimal Multi-Model Fusion We need: Unbiased forecasts Forecast error variances (main diagonal of ) B (i)k tt
20 Characteristics of Coastal Ocean Data Assimilation and Prediction: Observational data are sparse in space and in time Data are collected at different locations for different validation events Volume of data changes with time Data Model Misfits [ o C] z = 10 m, 8/7/ Data Model Misfits [ o C] z = 10 m, 8/11/ Data Model Misfits [ o C] z = 10 m, 8/24/ o N 37 o N 37 o N o N 2 36 o N 2 36 o N o W o W o W o W o W o W 40 3 Sequence of validation events => Sequential/Adaptive schemes
21 Sequential/Adaptive Bias Estimation Model-data misfits consist of the bias and of the random forecast and observational errors (m data pts) Practical Bias Model: sequential, level averaged, linear misfit update Using the above misfit definition, the error variance of the bias is: Weights w are chosen to minimize the error variance of the bias estimate Define: with the unbiased estimator constraint:
22 Sequential/Adaptive Bias Estimation (continued) Solution to this constrained minimum error variance minimization is (see Gorokhov and Stoica, IEEE Trans. Sig. Proc., 2000): Optimal Bias Model is: with w =
23 Example of bias estimation for MREA 2003 Exercises (Ligurian Sea) Bias corrected 24-hour forecast profiles Bias correction from one validating event Bias correction from three validating events
24 Bias estimation for AOSN-2 Z=10 m Z=150 m HOPS ROMS
25 Error (co)-variance Estimation Given q realizations of (random) forecast error the unconstrained Maximum-Likelihood error covariance estimate has a Wishart distribution of order q This classic estimator has a large variance for q small Mean-Squared Error (MSE) of any estimator of B
26 Error (co)-variance Estimator Look for error (co)-variance estimate as a linear combination of the classic (ML) unconstrained estimate and of a spatially constrained estimate Unconstrained estimate is asymptotically unbiased but can have larger estimation error variance Constrained (e.g. constant on fixed depth/density levels) estimate has bias coming from structural assumption, but smaller estimation error variance MSE of can be expressed via expectation and variance of quadratic forms in normal variables Given consider a quadratic form in x Expectation and variance of y are given by:
27 Presently: Error variance Estimator Look for uncertainty estimate in the form From three validation events Optimal lambdas are found from where are simple expressions in terms of trace of Example of Uncertainty Estimate for AOSN-2
28 Example of Central Forecast for AOSN-2 Central Forecast HOPS ROMS 24-hour T forecast for Aug 14, 2003
29 RMSE of HOPS, ROMS, and Two-model (Central) 24-hour Temperature Forecasts Z=10 m Z=150 m
30 Combine uncertainty estimation from data-model misfits with the adaptive ensemble-based ESSE uncertainty modeling By analyzing expectation and variance of quadratic forms we can compute error variance of uncertainty estimates generated from data-model misfits (uncertainty of uncertainty)
31 CONCLUSIONS Even though much more research on Adaptive DA is needed, results indicate that error estimates, ensemble sizes, error subspace ranks, covariance tapering parameters and stochastic error models can/should be calibrated by quantitative adaptation to observational data New Bayesian-based fusion of multiple model estimates based on Estimation of uncertainties (Bias + Variance) of ocean models based on the comparison of past model estimates to measurements Subsequent sum of model estimates with optimum error variance ased weights Much work remains, including Combinations of Adaptive DA and multi-model fusion schemes Infer improvements needed in models (adaptive modeling)
32
33 Conclusions The formalisms of Bayesian multi-model fusion, sequential bias estimation, and forecast uncertainty estimation, suited for ocean prediction, provide the methodology for integrating multiple ocean models into a single ocean prediction system Multi-model fusion consists of combining the individual forecasts based on their relative uncertainties minimizes error variance of central forecast spatially varying weights have clear interpretation of Bayes factors associated with the individual models Sequential bias estimation is different from Dee et al. type of algorithms since we explicitly compute the error variance of bias estimate and use that error variance once new data become available. Adaptiveness of the algorithm is controlled through the prior estimate error variance which determines the effect of the prior data on the current bias estimate Uncertainty estimation: by analyzing expectation and variance of quadratic forms we can compute error variance of uncertainty estimates generated from data-model misfits. Therefore, estimates can be combined with model-propagated uncertainties using Bayesian principle
34 Error Subspace Statistical Estimation (ESSE) Uncertainty forecasts (with dynamic error subspace, error learning) Ensemble-based (with nonlinear and stochastic primitive eq. model (HOPS) Multivariate, non-homogeneous and non-isotropic Data Assimilation (DA) Consistent DA and adaptive sampling schemes Software: not tied to any model, but specifics currently tailored to HOPS
35 STOCHASTIC FORCING MODEL: Sub-grid-scales
36
37 Example of bias estimation for MREA 2003 Exercises Bias estimation from a single validation profile 24-hour forecast profiles in the 2 nd half of experiment with bias model trained on the 1 st half
38 Tidal Inversion New HU code implemented in Matlab Shallow water equations in the frequency domain open boundary forcing: where Inverse solution found as: where Adjoint of dynamics Dynamic error covariance Observational error covariance Reference: Egbert G.D. and S. Erofeeva (2002). Efficient Inverse Modeling of Barotropic Ocean Tides. J.Atm.Oc.Tech., Vol. 19, pp
39
Harvard University AMS Annual Meeting, Seattle, WA, January 13, 2004
Application of the ESSE System to Real-Time Error Forecasting, Data Assimilation and Adaptive Sampling off the Central California Coast during AOSN-II: Pierre F.J. Lermusiaux, Wayne G. Leslie, Constantinos
More informationReal-time Forecasting of Fields and Uncertainties, Data Assimilation, Adaptive Sampling and Multi-Scale Ocean Dynamics for DART-05
Real-time Forecasting of Fields and Uncertainties, Data Assimilation, Adaptive Sampling and Multi-Scale Ocean Dynamics for DART-05 P.F.J. Lermusiaux, A.R. Robinson, P.J. Haley, W.G. Leslie, O. Logoutov,
More informationHarvard University. ASLO, February 21, 2004
Real-Time Error Forecasting, Data Assimilation and Adaptive Sampling in Monterey Bay during AOSN-II using the Error Subspace Statistical Estimation System Wayne G. Leslie, Pierre F.J. Lermusiaux, Constantinos
More informationHarvard Research and Plans: Adaptive Sampling and Prediction (ASAP) Persistent Littoral Undersea Surveillance Network (PLUSNet)
Harvard Research and Plans: Adaptive Sampling and Prediction (ASAP) Persistent Littoral Undersea Surveillance Network (PLUSNet) P.F.J. Lermusiaux, P.J. Haley, O. Logoutov, W.G. Leslie and A.R. Robinson
More informationAdaptive modeling, adaptive data assimilation and adaptive sampling
Physica D 230 (2007) 172 196 www.elsevier.com/locate/physd Adaptive modeling, adaptive data assimilation and adaptive sampling Pierre F.J. Lermusiaux Massachusetts Institute of Technology, Department of
More informationAOSN-II in Monterey Bay and the California Current System: Modeling and Predicting Multiple Scales and Processes Allan R. Robinson and Pierre F.J. Lermusiaux Division of Engineering and Applied Sciences
More informationOcean Prediction Systems: Advanced Concepts and Research Issues
Ocean Prediction Systems: Advanced Concepts and Research Issues Allan R. Robinson Harvard University Division of Engineering and Applied Sciences Department of Earth and Planetary Sciences System Concepts
More informationInterdisciplinary Modeling and Dynamics of Archipelago Straits:
Interdisciplinary Modeling and Dynamics of Archipelago Straits: Pierre F.J. Lermusiaux, Patrick J. Haley and Oleg Logutov. Mechanical Engineering, Ocean Science and Engineering, MIT Contributors: HU, MURI-ASAP,
More informationOcean Prediction Systems: Concepts and Advanced Research Issues
Ocean Prediction Systems: Concepts and Advanced Research Issues Allan R. Robinson Harvard University Division of Engineering and Applied Sciences Department of Earth and Planetary Sciences Ocean Prediction
More informationEnsemble Data Assimilation and Uncertainty Quantification
Ensemble Data Assimilation and Uncertainty Quantification Jeff Anderson National Center for Atmospheric Research pg 1 What is Data Assimilation? Observations combined with a Model forecast + to produce
More informationGaussian Filtering Strategies for Nonlinear Systems
Gaussian Filtering Strategies for Nonlinear Systems Canonical Nonlinear Filtering Problem ~u m+1 = ~ f (~u m )+~ m+1 ~v m+1 = ~g(~u m+1 )+~ o m+1 I ~ f and ~g are nonlinear & deterministic I Noise/Errors
More informationA new Hierarchical Bayes approach to ensemble-variational data assimilation
A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander
More informationHierarchical Bayes Ensemble Kalman Filter
Hierarchical Bayes Ensemble Kalman Filter M Tsyrulnikov and A Rakitko HydroMetCenter of Russia Wrocław, 7 Sep 2015 M Tsyrulnikov and A Rakitko (HMC) Hierarchical Bayes Ensemble Kalman Filter Wrocław, 7
More informationFOUR-DIMENSIONAL DATA ASSIMILATION FOR COUPLED PHYSICAL-ACOUSTICAL FIELDS
FOUR-DIMENSIONAL DATA ASSIMILATION FOR COUPLED PHYSICAL-ACOUSTICAL FIELDS P.F.J. LERMUSIAUX 1 AND C.S. CHIU 2 1 Harvard University, DEAS, Pierce Hall G2A, 29 Oxford Street, Cambridge MA 02318, USA E-mail:
More informationRelative Merits of 4D-Var and Ensemble Kalman Filter
Relative Merits of 4D-Var and Ensemble Kalman Filter Andrew Lorenc Met Office, Exeter International summer school on Atmospheric and Oceanic Sciences (ISSAOS) "Atmospheric Data Assimilation". August 29
More informationConsider the joint probability, P(x,y), shown as the contours in the figure above. P(x) is given by the integral of P(x,y) over all values of y.
ATMO/OPTI 656b Spring 009 Bayesian Retrievals Note: This follows the discussion in Chapter of Rogers (000) As we have seen, the problem with the nadir viewing emission measurements is they do not contain
More informationThe Kalman Filter. Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience. Sarah Dance
The Kalman Filter Data Assimilation & Inverse Problems from Weather Forecasting to Neuroscience Sarah Dance School of Mathematical and Physical Sciences, University of Reading s.l.dance@reading.ac.uk July
More informationFundamentals of Data Assimila1on
014 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO July 14-16, 014 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
More informationParameter Estimation and Adaptive Modeling Studies in Ocean Mixing. Eric Vincent Heubel
Multidisciplinary Simulation, Estimation, and Assimilation Systems Reports in Ocean Science and Engineering MSEAS-1 Parameter Estimation and Adaptive Modeling Studies in Ocean Mixing by Eric Vincent Heubel
More informationVerification and training of real-time forecasting of multi-scale ocean dynamics for maritime rapid environmental assessment
Available online at www.sciencedirect.com Journal of Marine Systems 69 (2008) 3 16 www.elsevier.com/locate/jmarsys Verification and training of real-time forecasting of multi-scale ocean dynamics for maritime
More informationAspects of the practical application of ensemble-based Kalman filters
Aspects of the practical application of ensemble-based Kalman filters Lars Nerger Alfred Wegener Institute for Polar and Marine Research Bremerhaven, Germany and Bremen Supercomputing Competence Center
More informationIntroduction to Data Assimilation
Introduction to Data Assimilation Alan O Neill Data Assimilation Research Centre University of Reading What is data assimilation? Data assimilation is the technique whereby observational data are combined
More informationPrediction Systems With Data Assimilation For Coupled Ocean Science And Ocean Acoustics
Prediction Systems With Data Assimilation For Coupled Ocean Science And Ocean Acoustics Allan R. Robinson Pierre F.J. Lermusiaux Division of Engineering and Applied Sciences Department of Earth and Planetary
More informationFundamentals of Data Assimila1on
2015 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO August 11-14, 2015 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
More informationBayesian Statistics and Data Assimilation. Jonathan Stroud. Department of Statistics The George Washington University
Bayesian Statistics and Data Assimilation Jonathan Stroud Department of Statistics The George Washington University 1 Outline Motivation Bayesian Statistics Parameter Estimation in Data Assimilation Combined
More informationKalman Filter and Ensemble Kalman Filter
Kalman Filter and Ensemble Kalman Filter 1 Motivation Ensemble forecasting : Provides flow-dependent estimate of uncertainty of the forecast. Data assimilation : requires information about uncertainty
More informationTRANSFER OF UNCERTAINTIES THROUGH PHYSICAL-ACOUSTICAL-SONAR END-TO-END SYSTEMS: A CONCEPTUAL BASIS
TRANSFER OF UNCERTAINTIES THROUGH PHYSICAL-ACOUSTICAL-SONAR END-TO-END SYSTEMS: A CONCEPTUAL BASIS A.R. ROBINSON 1, P. ABBOT 2, P.F.J. LERMUSIAUX 1 AND L. DILLMAN 2 1 Harvard University, Cambridge, MA
More informationSmoothers: Types and Benchmarks
Smoothers: Types and Benchmarks Patrick N. Raanes Oxford University, NERSC 8th International EnKF Workshop May 27, 2013 Chris Farmer, Irene Moroz Laurent Bertino NERSC Geir Evensen Abstract Talk builds
More informationAlexander Barth, Aida Alvera-Azc. Azcárate, Robert H. Weisberg, University of South Florida. George Halliwell RSMAS, University of Miami
Ensemble-based based Assimilation of HF-Radar Surface Currents in a West Florida Shelf ROMS Nested into HYCOM and filtering of spurious surface gravity waves. Alexander Barth, Aida Alvera-Azc Azcárate,
More informationApplication of the Ensemble Kalman Filter to History Matching
Application of the Ensemble Kalman Filter to History Matching Presented at Texas A&M, November 16,2010 Outline Philosophy EnKF for Data Assimilation Field History Match Using EnKF with Covariance Localization
More informationDATA ASSIMILATION FOR FLOOD FORECASTING
DATA ASSIMILATION FOR FLOOD FORECASTING Arnold Heemin Delft University of Technology 09/16/14 1 Data assimilation is the incorporation of measurement into a numerical model to improve the model results
More informationGaussian Models
Gaussian Models ddebarr@uw.edu 2016-04-28 Agenda Introduction Gaussian Discriminant Analysis Inference Linear Gaussian Systems The Wishart Distribution Inferring Parameters Introduction Gaussian Density
More informationIntroduction to Ensemble Kalman Filters and the Data Assimilation Research Testbed
Introduction to Ensemble Kalman Filters and the Data Assimilation Research Testbed Jeffrey Anderson, Tim Hoar, Nancy Collins NCAR Institute for Math Applied to Geophysics pg 1 What is Data Assimilation?
More informationTRANSFER OF UNCERTAINTIES THROUGH PHYSICAL-ACOUSTICAL-SONAR END-TO-END SYSTEMS: A CONCEPTUAL BASIS
TRANSFER OF UNCERTAINTIES THROUGH PHYSICAL-ACOUSTICAL-SONAR END-TO-END SYSTEMS: A CONCEPTUAL BASIS A. R. ROBINSON 1, P. ABBOT 2, P.F.J LERMUSIAUX 1, L. DILLMAN 2 1 Harvard University, Cambridge, MA 02138
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More information4. DATA ASSIMILATION FUNDAMENTALS
4. DATA ASSIMILATION FUNDAMENTALS... [the atmosphere] "is a chaotic system in which errors introduced into the system can grow with time... As a consequence, data assimilation is a struggle between chaotic
More informationData assimilation in high dimensions
Data assimilation in high dimensions David Kelly Courant Institute New York University New York NY www.dtbkelly.com February 12, 2015 Graduate seminar, CIMS David Kelly (CIMS) Data assimilation February
More informationEnsemble square-root filters
Ensemble square-root filters MICHAEL K. TIPPETT International Research Institute for climate prediction, Palisades, New Yor JEFFREY L. ANDERSON GFDL, Princeton, New Jersy CRAIG H. BISHOP Naval Research
More informationProf. Stephen G. Penny University of Maryland NOAA/NCEP, RIKEN AICS, ECMWF US CLIVAR Summit, 9 August 2017
COUPLED DATA ASSIMILATION: What we need from observations and modellers to make coupled data assimilation the new standard for prediction and reanalysis. Prof. Stephen G. Penny University of Maryland NOAA/NCEP,
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
More informationBayesian Estimation of Input Output Tables for Russia
Bayesian Estimation of Input Output Tables for Russia Oleg Lugovoy (EDF, RANE) Andrey Polbin (RANE) Vladimir Potashnikov (RANE) WIOD Conference April 24, 2012 Groningen Outline Motivation Objectives Bayesian
More informationThe Ensemble Kalman Filter:
p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen 23, Ocean Dynamics, Vol 53, No 4 p.2 The Ensemble
More informationForecasting and data assimilation
Supported by the National Science Foundation DMS Forecasting and data assimilation Outline Numerical models Kalman Filter Ensembles Douglas Nychka, Thomas Bengtsson, Chris Snyder Geophysical Statistics
More informationFundamentals of Data Assimilation
National Center for Atmospheric Research, Boulder, CO USA GSI Data Assimilation Tutorial - June 28-30, 2010 Acknowledgments and References WRFDA Overview (WRF Tutorial Lectures, H. Huang and D. Barker)
More informationDART_LAB Tutorial Section 5: Adaptive Inflation
DART_LAB Tutorial Section 5: Adaptive Inflation UCAR 14 The National Center for Atmospheric Research is sponsored by the National Science Foundation. Any opinions, findings and conclusions or recommendations
More informationState and Parameter Estimation in Stochastic Dynamical Models
State and Parameter Estimation in Stochastic Dynamical Models Timothy DelSole George Mason University, Fairfax, Va and Center for Ocean-Land-Atmosphere Studies, Calverton, MD June 21, 2011 1 1 collaboration
More informationOrganization. I MCMC discussion. I project talks. I Lecture.
Organization I MCMC discussion I project talks. I Lecture. Content I Uncertainty Propagation Overview I Forward-Backward with an Ensemble I Model Reduction (Intro) Uncertainty Propagation in Causal Systems
More informationOptimising Group Sequential Designs. Decision Theory, Dynamic Programming. and Optimal Stopping
: Decision Theory, Dynamic Programming and Optimal Stopping Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj InSPiRe Conference on Methodology
More informationLearning Gaussian Process Models from Uncertain Data
Learning Gaussian Process Models from Uncertain Data Patrick Dallaire, Camille Besse, and Brahim Chaib-draa DAMAS Laboratory, Computer Science & Software Engineering Department, Laval University, Canada
More informationNumerical Weather prediction at the European Centre for Medium-Range Weather Forecasts (2)
Numerical Weather prediction at the European Centre for Medium-Range Weather Forecasts (2) Time series curves 500hPa geopotential Correlation coefficent of forecast anomaly N Hemisphere Lat 20.0 to 90.0
More informationNonlinear State Estimation! Particle, Sigma-Points Filters!
Nonlinear State Estimation! Particle, Sigma-Points Filters! Robert Stengel! Optimal Control and Estimation, MAE 546! Princeton University, 2017!! Particle filter!! Sigma-Points Unscented Kalman ) filter!!
More informationData assimilation in high dimensions
Data assimilation in high dimensions David Kelly Kody Law Andy Majda Andrew Stuart Xin Tong Courant Institute New York University New York NY www.dtbkelly.com February 3, 2016 DPMMS, University of Cambridge
More informationErgodicity in data assimilation methods
Ergodicity in data assimilation methods David Kelly Andy Majda Xin Tong Courant Institute New York University New York NY www.dtbkelly.com April 15, 2016 ETH Zurich David Kelly (CIMS) Data assimilation
More informationStability of Ensemble Kalman Filters
Stability of Ensemble Kalman Filters Idrissa S. Amour, Zubeda Mussa, Alexander Bibov, Antti Solonen, John Bardsley, Heikki Haario and Tuomo Kauranne Lappeenranta University of Technology University of
More informationM.Sc. in Meteorology. Numerical Weather Prediction
M.Sc. in Meteorology UCD Numerical Weather Prediction Prof Peter Lynch Meteorology & Climate Cehtre School of Mathematical Sciences University College Dublin Second Semester, 2005 2006. Text for the Course
More informationAdaptive Coupled Physical and Biogeochemical Ocean Predictions: A Conceptual Basis
Adaptive Coupled Physical and Biogeochemical Ocean Predictions: A Conceptual Basis P.F.J. Lermusiaux 1, C. Evangelinos 2, R. Tian 1, P.J. Haley 1, J.J. McCarthy 1, N.M. Patrikalakis 2, A.R. Robinson 1,
More informationOptimal Interpolation ( 5.4) We now generalize the least squares method to obtain the OI equations for vectors of observations and background fields.
Optimal Interpolation ( 5.4) We now generalize the least squares method to obtain the OI equations for vectors of observations and background fields. Optimal Interpolation ( 5.4) We now generalize the
More informationInterpretation of two error statistics estimation methods: 1 - the Derozier s method 2 the NMC method (lagged forecast)
Interpretation of two error statistics estimation methods: 1 - the Derozier s method 2 the NMC method (lagged forecast) Richard Ménard, Yan Yang and Yves Rochon Air Quality Research Division Environment
More informationData Assimilation with the Ensemble Kalman Filter and the SEIK Filter applied to a Finite Element Model of the North Atlantic
Data Assimilation with the Ensemble Kalman Filter and the SEIK Filter applied to a Finite Element Model of the North Atlantic L. Nerger S. Danilov, G. Kivman, W. Hiller, and J. Schröter Alfred Wegener
More informationcovariance function, 174 probability structure of; Yule-Walker equations, 174 Moving average process, fluctuations, 5-6, 175 probability structure of
Index* The Statistical Analysis of Time Series by T. W. Anderson Copyright 1971 John Wiley & Sons, Inc. Aliasing, 387-388 Autoregressive {continued) Amplitude, 4, 94 case of first-order, 174 Associated
More informationThe ECMWF Hybrid 4D-Var and Ensemble of Data Assimilations
The Hybrid 4D-Var and Ensemble of Data Assimilations Lars Isaksen, Massimo Bonavita and Elias Holm Data Assimilation Section lars.isaksen@ecmwf.int Acknowledgements to: Mike Fisher and Marta Janiskova
More informationData Assimilation: Finding the Initial Conditions in Large Dynamical Systems. Eric Kostelich Data Mining Seminar, Feb. 6, 2006
Data Assimilation: Finding the Initial Conditions in Large Dynamical Systems Eric Kostelich Data Mining Seminar, Feb. 6, 2006 kostelich@asu.edu Co-Workers Istvan Szunyogh, Gyorgyi Gyarmati, Ed Ott, Brian
More informationData assimilation concepts and methods March 1999
Data assimilation concepts and methods March 1999 By F. Bouttier and P. Courtier Abstract These training course lecture notes are an advanced and comprehensive presentation of most data assimilation methods
More informationBayesian Nonlinear Assimilation of Eulerian and Lagrangian Coastal Flow Data
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. Bayesian Nonlinear Assimilation of Eulerian and Lagrangian Coastal Flow Data Dr. Pierre F.J. Lermusiaux Department of Mechanical
More informationIntroduction to data assimilation and least squares methods
Introduction to data assimilation and least squares methods Eugenia Kalnay and many friends University of Maryland October 008 (part 1 Contents (1 Forecasting the weather - we are really getting better!
More informationThe impact of the assimilation of SLA along track Observation with high-frequency signal in IBI system
The impact of the assimilation of SLA along track Observation with high-frequency signal in IBI system M. Benkiran, C.Dufau (CLS) and Mercator Ocean Team http://www.mercator-ocean.fr mbenkiran@cls.fr Context
More informationEnvironment Canada s Regional Ensemble Kalman Filter
Environment Canada s Regional Ensemble Kalman Filter May 19, 2014 Seung-Jong Baek, Luc Fillion, Kao-Shen Chung, and Peter Houtekamer Meteorological Research Division, Environment Canada, Dorval, Quebec
More informationModel error and parameter estimation
Model error and parameter estimation Chiara Piccolo and Mike Cullen ECMWF Annual Seminar, 11 September 2018 Summary The application of interest is atmospheric data assimilation focus on EDA; A good ensemble
More informationLINEAR MODELS FOR CLASSIFICATION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
LINEAR MODELS FOR CLASSIFICATION Classification: Problem Statement 2 In regression, we are modeling the relationship between a continuous input variable x and a continuous target variable t. In classification,
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationParametric Models. Dr. Shuang LIANG. School of Software Engineering TongJi University Fall, 2012
Parametric Models Dr. Shuang LIANG School of Software Engineering TongJi University Fall, 2012 Today s Topics Maximum Likelihood Estimation Bayesian Density Estimation Today s Topics Maximum Likelihood
More informationstatistical methods for tailoring seasonal climate forecasts Andrew W. Robertson, IRI
statistical methods for tailoring seasonal climate forecasts Andrew W. Robertson, IRI tailored seasonal forecasts why do we make probabilistic forecasts? to reduce our uncertainty about the (unknown) future
More informationComparison of 3D-Var and LETKF in an Atmospheric GCM: SPEEDY
Comparison of 3D-Var and LEKF in an Atmospheric GCM: SPEEDY Catherine Sabol Kayo Ide Eugenia Kalnay, akemasa Miyoshi Weather Chaos, UMD 9 April 2012 Outline SPEEDY Formulation Single Observation Eperiments
More informationLagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model. David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC
Lagrangian Data Assimilation and Manifold Detection for a Point-Vortex Model David Darmon, AMSC Kayo Ide, AOSC, IPST, CSCAMM, ESSIC Background Data Assimilation Iterative process Forecast Analysis Background
More informationIndependent Component Analysis. Contents
Contents Preface xvii 1 Introduction 1 1.1 Linear representation of multivariate data 1 1.1.1 The general statistical setting 1 1.1.2 Dimension reduction methods 2 1.1.3 Independence as a guiding principle
More informationOcean data assimilation for reanalysis
Ocean data assimilation for reanalysis Matt Martin. ERA-CLIM2 Symposium, University of Bern, 14 th December 2017. Contents Introduction. On-going developments to improve ocean data assimilation for reanalysis.
More informationSensor Tasking and Control
Sensor Tasking and Control Sensing Networking Leonidas Guibas Stanford University Computation CS428 Sensor systems are about sensing, after all... System State Continuous and Discrete Variables The quantities
More informationAOSN Monterey Bay Experiment: Creating Sustainable Ocean Observation/Prediction Systems
AOSN Monterey Bay Experiment: Creating Sustainable Ocean Observation/Prediction Systems AOSN II Objectives: To design and build an adaptive coupled observation/modeling system. The system should be sustainable
More information+ + ( + ) = Linear recurrent networks. Simpler, much more amenable to analytic treatment E.g. by choosing
Linear recurrent networks Simpler, much more amenable to analytic treatment E.g. by choosing + ( + ) = Firing rates can be negative Approximates dynamics around fixed point Approximation often reasonable
More informationBayesian Inverse problem, Data assimilation and Localization
Bayesian Inverse problem, Data assimilation and Localization Xin T Tong National University of Singapore ICIP, Singapore 2018 X.Tong Localization 1 / 37 Content What is Bayesian inverse problem? What is
More informationMathematical Concepts of Data Assimilation
Mathematical Concepts of Data Assimilation N.K. Nichols 1 Introduction Environmental systems can be realistically described by mathematical and numerical models of the system dynamics. These models can
More informationEfficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter
Efficient Data Assimilation for Spatiotemporal Chaos: a Local Ensemble Transform Kalman Filter arxiv:physics/0511236 v1 28 Nov 2005 Brian R. Hunt Institute for Physical Science and Technology and Department
More informationThe Canadian approach to ensemble prediction
The Canadian approach to ensemble prediction ECMWF 2017 Annual seminar: Ensemble prediction : past, present and future. Pieter Houtekamer Montreal, Canada Overview. The Canadian approach. What are the
More informationOPTIMAL ESTIMATION of DYNAMIC SYSTEMS
CHAPMAN & HALL/CRC APPLIED MATHEMATICS -. AND NONLINEAR SCIENCE SERIES OPTIMAL ESTIMATION of DYNAMIC SYSTEMS John L Crassidis and John L. Junkins CHAPMAN & HALL/CRC A CRC Press Company Boca Raton London
More informationLocalization in the ensemble Kalman Filter
Department of Meteorology Localization in the ensemble Kalman Filter Ruth Elizabeth Petrie A dissertation submitted in partial fulfilment of the requirement for the degree of MSc. Atmosphere, Ocean and
More informationSystematic strategies for real time filtering of turbulent signals in complex systems
Systematic strategies for real time filtering of turbulent signals in complex systems Statistical inversion theory for Gaussian random variables The Kalman Filter for Vector Systems: Reduced Filters and
More informationCovariance Matrix Simplification For Efficient Uncertainty Management
PASEO MaxEnt 2007 Covariance Matrix Simplification For Efficient Uncertainty Management André Jalobeanu, Jorge A. Gutiérrez PASEO Research Group LSIIT (CNRS/ Univ. Strasbourg) - Illkirch, France *part
More informationFeb 21 and 25: Local weighted least squares: Quadratic loess smoother
Feb 1 and 5: Local weighted least squares: Quadratic loess smoother An example of weighted least squares fitting of data to a simple model for the purposes of simultaneous smoothing and interpolation is
More informationA Comparison of Error Subspace Kalman Filters
Tellus 000, 000 000 (0000) Printed 4 February 2005 (Tellus LATEX style file v2.2) A Comparison of Error Subspace Kalman Filters By LARS NERGER, WOLFGANG HILLER and JENS SCHRÖTER Alfred Wegener Institute
More informationStochastic Analogues to Deterministic Optimizers
Stochastic Analogues to Deterministic Optimizers ISMP 2018 Bordeaux, France Vivak Patel Presented by: Mihai Anitescu July 6, 2018 1 Apology I apologize for not being here to give this talk myself. I injured
More informationLagrangian data assimilation for point vortex systems
JOT J OURNAL OF TURBULENCE http://jot.iop.org/ Lagrangian data assimilation for point vortex systems Kayo Ide 1, Leonid Kuznetsov 2 and Christopher KRTJones 2 1 Department of Atmospheric Sciences and Institute
More informationEnsembles and Particle Filters for Ocean Data Assimilation
DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. Ensembles and Particle Filters for Ocean Data Assimilation Robert N. Miller College of Oceanic and Atmospheric Sciences
More informationParameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets
Parameter Estimation in the Spatio-Temporal Mixed Effects Model Analysis of Massive Spatio-Temporal Data Sets Matthias Katzfuß Advisor: Dr. Noel Cressie Department of Statistics The Ohio State University
More informationDemonstration and Comparison of of Sequential Approaches for Altimeter Data Assimilation in in HYCOM
Demonstration and Comparison of of Sequential Approaches for Altimeter Data Assimilation in in HYCOM A. Srinivasan, E. P. Chassignet, O. M. Smedstad, C. Thacker, L. Bertino, P. Brasseur, T. M. Chin,, F.
More informationRobust Ensemble Filtering With Improved Storm Surge Forecasting
Robust Ensemble Filtering With Improved Storm Surge Forecasting U. Altaf, T. Buttler, X. Luo, C. Dawson, T. Mao, I.Hoteit Meteo France, Toulouse, Nov 13, 2012 Project Ensemble data assimilation for storm
More informationReview of Covariance Localization in Ensemble Filters
NOAA Earth System Research Laboratory Review of Covariance Localization in Ensemble Filters Tom Hamill NOAA Earth System Research Lab, Boulder, CO tom.hamill@noaa.gov Canonical ensemble Kalman filter update
More information(Extended) Kalman Filter
(Extended) Kalman Filter Brian Hunt 7 June 2013 Goals of Data Assimilation (DA) Estimate the state of a system based on both current and all past observations of the system, using a model for the system
More informationA6523 Modeling, Inference, and Mining Jim Cordes, Cornell University
A6523 Modeling, Inference, and Mining Jim Cordes, Cornell University Lecture 19 Modeling Topics plan: Modeling (linear/non- linear least squares) Bayesian inference Bayesian approaches to spectral esbmabon;
More informationThe Ensemble Kalman Filter:
p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen, Ocean Dynamics, Vol 5, No p. The Ensemble
More informationSupport Vector Regression (SVR) Descriptions of SVR in this discussion follow that in Refs. (2, 6, 7, 8, 9). The literature
Support Vector Regression (SVR) Descriptions of SVR in this discussion follow that in Refs. (2, 6, 7, 8, 9). The literature suggests the design variables should be normalized to a range of [-1,1] or [0,1].
More information