An Efficient Ensemble Data Assimilation Approach To Deal With Range Limited Observation

Size: px
Start display at page:

Download "An Efficient Ensemble Data Assimilation Approach To Deal With Range Limited Observation"

Transcription

1 An Efficient Ensemble Data Assimilation Approach To Deal With Range Limited Observation A. Shah 1,2, M. E. Gharamti 1, L. Bertino 1 1 Nansen Environmental and Remote Sensing Center 2 University of Bergen 11 th International EnKF Workshop June 2, 216 1

2 Outline Motivation Range Limited Observations (RLO): Methodology and Algorithm Numerical Experiments Conclusion 2

3 Motivation Many available measurement in environmental systems are defined within certain interval. To use the qualitative information available from the range limited observations. Very few studies carried out dealing this issue Borup et. al., (215) 3

4 Range limited Observations Photo : Earth Imaging Journal 4

5 Methodology and Algorithm Bayesian Rule p( x y) = p ( x )p( y x) p( y) Borup et. al., (215): Ensemble Partial Updating (EnPU) for RLO EnPU will allow us to use qualitative information about data i.e., the posterior will be p( x k y quant, y ) qual where y quant and y qual are quantitative and qualitative observation respectively 5

6 Figure : (Borup et. al., 215) With and without partial updating when the measurement gauge has lower observation limit 6

7 Partial Ensemble Kalman Filter (PEnKF) OR-observation Create virtual observation at threshold limit Data likelihood for perturbing observations Two Piece Gaussian distribution (Fechner s Kollektivmasslehre, 1897) One of the observation variance in 2-piece Gaussian s or = p* ( Hx f ) where p is positive real number 7

8 Probability Probability Cont.. Posterior when the prior is in-range ï ( ) µ p(x k)p(y quant x k ) í p x k y quant, y qualit ì îï p(x k )p(y qualit x k ) Bayesian Representation Prior Obs. Likelihood Prior Obs. Prior Likelihood Posterior Threshold State value 8

9 Check if y isout of range No Update x i f as it isdoneinenkf Yes Check if Hx i f is within range No Updatethe X i f settinginnovationtozero Yes UpdateX i f with perturb observation generated from 2-piece Gaussian Repeat from the step 2 for all all the members Repeat from the step 2 for all the members 9

10 Numerical Experiments EnKF, PEnKF and EnKF-ignored DA methods are tested under the framework of twin experiment. Model Lorenz 96 with configuration as below Number of Ensemble 1 dt -.5 (~6 hours) Total time of integration is 5 Years Model error introduced by using wrong forcing s obs =1 1

11 Cont.. Experiments for Sensitivity to Number of observation Observation frequency Threshold limit Model error Diagnostics tools: Root Mean Square Error Average Ensemble Spread Observation Influence Rank Histograms 11

12 RMSE RMSE and Avg. Ensemble Spread RMSE-Analysis for EnKF, PEnKF and EnKF-Ignored with full obs. network EnKF PEnKF EnKF-ignored % of observations are out of range on an average for total time of integration12

13 RMSE Avg. RMSE for EnKF, PEnKF and EnKF-Ignored for different observation frequency 2.5 EnKF PEnKF EnKF-Ignored Observation frequency (Time step) 13

14 RMS and AESP AESP Average Ensemble Spread Analysis for PEnKF and EnKF-(Ignored) Snap-shot of time series of RMSE and AESP of Analysis AESP-PEnKF AESP-EnKF(Ign) RMSE-PEnKF AESP-PEnKF RMSE-EnKF(Ign) AESP-EnKF(Ign) Time Time 14

15 tr(kh) T /No.of obs Observation Influence.45.4 Average Observation influence in PEnkF All Observation OR-Observation Time 15

16 P(rank) P(rank) P(rank) P(rank) P(rank) P(rank) Rank Histogram(Reliability).6 PEnKF-all observation.1 PEnKF- Half observation.6 PEnKF- 13 observations Rank Rank Rank.6 EnKF(Ignored) all observation.1 EnKF(Ignored) - Half observation.6 EnKF(Ign)-13 observations Rank Rank Rank 16

17 RMSE Sensitivity-Model Error 3.5 RMSE Analysis for strong model error PEnKF EnKF-Ignored Time 17

18 Conclusion and future work Adding qualitative information with PEnKF Improve quality of forecast Reduce uncertainty Improves reliability of forecast Adding strong model error deteriorates the performance of the proposed DA scheme Implementation with some real world model and data set To investigate further for some possible improvement if possible 18

19 19

Ensemble Kalman Filter

Ensemble Kalman Filter Ensemble Kalman Filter Geir Evensen and Laurent Bertino Hydro Research Centre, Bergen, Norway, Nansen Environmental and Remote Sensing Center, Bergen, Norway The Ensemble Kalman Filter (EnKF) Represents

More information

A data-driven method for improving the correlation estimation in serial ensemble Kalman filter

A data-driven method for improving the correlation estimation in serial ensemble Kalman filter A data-driven method for improving the correlation estimation in serial ensemble Kalman filter Michèle De La Chevrotière, 1 John Harlim 2 1 Department of Mathematics, Penn State University, 2 Department

More information

Lagrangian Data Assimilation and Its Application to Geophysical Fluid Flows

Lagrangian Data Assimilation and Its Application to Geophysical Fluid Flows Lagrangian Data Assimilation and Its Application to Geophysical Fluid Flows Laura Slivinski June, 3 Laura Slivinski (Brown University) Lagrangian Data Assimilation June, 3 / 3 Data Assimilation Setup:

More information

Data Assimilation Research Testbed Tutorial

Data Assimilation Research Testbed Tutorial Data Assimilation Research Testbed Tutorial Section 2: How should observations of a state variable impact an unobserved state variable? Multivariate assimilation. Single observed variable, single unobserved

More information

Data assimilation with and without a model

Data assimilation with and without a model Data assimilation with and without a model Tim Sauer George Mason University Parameter estimation and UQ U. Pittsburgh Mar. 5, 2017 Partially supported by NSF Most of this work is due to: Tyrus Berry,

More information

DART_LAB Tutorial Section 2: How should observations impact an unobserved state variable? Multivariate assimilation.

DART_LAB Tutorial Section 2: How should observations impact an unobserved state variable? Multivariate assimilation. DART_LAB Tutorial Section 2: How should observations impact an unobserved state variable? Multivariate assimilation. UCAR 2014 The National Center for Atmospheric Research is sponsored by the National

More information

Hierarchical Bayes Ensemble Kalman Filter

Hierarchical Bayes Ensemble Kalman Filter Hierarchical Bayes Ensemble Kalman Filter M Tsyrulnikov and A Rakitko HydroMetCenter of Russia Wrocław, 7 Sep 2015 M Tsyrulnikov and A Rakitko (HMC) Hierarchical Bayes Ensemble Kalman Filter Wrocław, 7

More information

A new Hierarchical Bayes approach to ensemble-variational data assimilation

A new Hierarchical Bayes approach to ensemble-variational data assimilation A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander

More information

(Extended) Kalman Filter

(Extended) Kalman Filter (Extended) Kalman Filter Brian Hunt 7 June 2013 Goals of Data Assimilation (DA) Estimate the state of a system based on both current and all past observations of the system, using a model for the system

More information

EnKF-based particle filters

EnKF-based particle filters EnKF-based particle filters Jana de Wiljes, Sebastian Reich, Wilhelm Stannat, Walter Acevedo June 20, 2017 Filtering Problem Signal dx t = f (X t )dt + 2CdW t Observations dy t = h(x t )dt + R 1/2 dv t.

More information

Fundamentals of Data Assimila1on

Fundamentals of Data Assimila1on 014 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO July 14-16, 014 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University

More information

Ergodicity in data assimilation methods

Ergodicity in data assimilation methods Ergodicity in data assimilation methods David Kelly Andy Majda Xin Tong Courant Institute New York University New York NY www.dtbkelly.com April 15, 2016 ETH Zurich David Kelly (CIMS) Data assimilation

More information

Ensemble forecasting and flow-dependent estimates of initial uncertainty. Martin Leutbecher

Ensemble forecasting and flow-dependent estimates of initial uncertainty. Martin Leutbecher Ensemble forecasting and flow-dependent estimates of initial uncertainty Martin Leutbecher acknowledgements: Roberto Buizza, Lars Isaksen Flow-dependent aspects of data assimilation, ECMWF 11 13 June 2007

More information

Smoothers: Types and Benchmarks

Smoothers: Types and Benchmarks Smoothers: Types and Benchmarks Patrick N. Raanes Oxford University, NERSC 8th International EnKF Workshop May 27, 2013 Chris Farmer, Irene Moroz Laurent Bertino NERSC Geir Evensen Abstract Talk builds

More information

Ensemble Data Assimilation and Uncertainty Quantification

Ensemble Data Assimilation and Uncertainty Quantification Ensemble Data Assimilation and Uncertainty Quantification Jeff Anderson National Center for Atmospheric Research pg 1 What is Data Assimilation? Observations combined with a Model forecast + to produce

More information

Improved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter

Improved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter Improved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter Hong Li, Junjie Liu, and Elana Fertig E. Kalnay I. Szunyogh, E. J. Kostelich Weather and Chaos Group

More information

Data assimilation with and without a model

Data assimilation with and without a model Data assimilation with and without a model Tyrus Berry George Mason University NJIT Feb. 28, 2017 Postdoc supported by NSF This work is in collaboration with: Tim Sauer, GMU Franz Hamilton, Postdoc, NCSU

More information

Correcting biased observation model error in data assimilation

Correcting biased observation model error in data assimilation Correcting biased observation model error in data assimilation Tyrus Berry Dept. of Mathematical Sciences, GMU PSU-UMD DA Workshop June 27, 217 Joint work with John Harlim, PSU BIAS IN OBSERVATION MODELS

More information

Enhancing information transfer from observations to unobserved state variables for mesoscale radar data assimilation

Enhancing information transfer from observations to unobserved state variables for mesoscale radar data assimilation Enhancing information transfer from observations to unobserved state variables for mesoscale radar data assimilation Weiguang Chang and Isztar Zawadzki Department of Atmospheric and Oceanic Sciences Faculty

More information

The Ensemble Kalman Filter:

The Ensemble Kalman Filter: p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen 23, Ocean Dynamics, Vol 53, No 4 p.2 The Ensemble

More information

Convective-scale data assimilation in the Weather Research and Forecasting model using a nonlinear ensemble filter

Convective-scale data assimilation in the Weather Research and Forecasting model using a nonlinear ensemble filter Convective-scale data assimilation in the Weather Research and Forecasting model using a nonlinear ensemble filter Jon Poterjoy, Ryan Sobash, and Jeffrey Anderson National Center for Atmospheric Research

More information

Recap on Data Assimilation

Recap on Data Assimilation Concluding Thoughts Recap on Data Assimilation FORECAST ANALYSIS Kalman Filter Forecast Analysis Analytical projection of the ANALYSIS mean and cov from t-1 to the FORECAST mean and cov for t Update FORECAST

More information

Short tutorial on data assimilation

Short tutorial on data assimilation Mitglied der Helmholtz-Gemeinschaft Short tutorial on data assimilation 23 June 2015 Wolfgang Kurtz & Harrie-Jan Hendricks Franssen Institute of Bio- and Geosciences IBG-3 (Agrosphere), Forschungszentrum

More information

Challenges in Inverse Modeling and Data Assimilation of Atmospheric Constituents

Challenges in Inverse Modeling and Data Assimilation of Atmospheric Constituents Challenges in Inverse Modeling and Data Assimilation of Atmospheric Constituents Ave Arellano Atmospheric Chemistry Division National Center for Atmospheric Research NCAR is sponsored by the National Science

More information

Prospects for river discharge and depth estimation through assimilation of swath altimetry into a raster-based hydraulics model

Prospects for river discharge and depth estimation through assimilation of swath altimetry into a raster-based hydraulics model Prospects for river discharge and depth estimation through assimilation of swath altimetry into a raster-based hydraulics model Kostas Andreadis 1, Elizabeth Clark 2, Dennis Lettenmaier 1, and Doug Alsdorf

More information

Juli I. Rubin. NRC Postdoctoral Research Associate Naval Research Laboratory, Monterey, CA

Juli I. Rubin. NRC Postdoctoral Research Associate Naval Research Laboratory, Monterey, CA Development of the Ensemble Navy Aerosol Analysis Prediction System and its application of the Data Assimilation Research Testbed in Support of Aerosol Forecasting Juli I. Rubin NRC Postdoctoral Research

More information

Asynchronous data assimilation

Asynchronous data assimilation Ensemble Kalman Filter, lecture 2 Asynchronous data assimilation Pavel Sakov Nansen Environmental and Remote Sensing Center, Norway This talk has been prepared in the course of evita-enkf project funded

More information

Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model

Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model Rahul Mahajan NOAA / NCEP / EMC - IMSG College Park, MD 20740, USA Acknowledgements: Ron Gelaro, Ricardo

More information

Par$cle Filters Part I: Theory. Peter Jan van Leeuwen Data- Assimila$on Research Centre DARC University of Reading

Par$cle Filters Part I: Theory. Peter Jan van Leeuwen Data- Assimila$on Research Centre DARC University of Reading Par$cle Filters Part I: Theory Peter Jan van Leeuwen Data- Assimila$on Research Centre DARC University of Reading Reading July 2013 Why Data Assimila$on Predic$on Model improvement: - Parameter es$ma$on

More information

Fundamentals of Data Assimila1on

Fundamentals of Data Assimila1on 2015 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO August 11-14, 2015 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University

More information

Data assimilation in high dimensions

Data assimilation in high dimensions Data assimilation in high dimensions David Kelly Courant Institute New York University New York NY www.dtbkelly.com February 12, 2015 Graduate seminar, CIMS David Kelly (CIMS) Data assimilation February

More information

Radiance Data Assimilation with an EnKF

Radiance Data Assimilation with an EnKF Radiance Data Assimilation with an EnKF Zhiquan Liu, Craig Schwartz, Xiangyu Huang (NCAR/MMM) Yongsheng Chen (York University) 4/7/2010 4th EnKF Workshop 1 Outline Radiance Assimilation Methodology Apply

More information

COMPARATIVE EVALUATION OF ENKF AND MLEF FOR ASSIMILATION OF STREAMFLOW DATA INTO NWS OPERATIONAL HYDROLOGIC MODELS

COMPARATIVE EVALUATION OF ENKF AND MLEF FOR ASSIMILATION OF STREAMFLOW DATA INTO NWS OPERATIONAL HYDROLOGIC MODELS COMPARATIVE EVALUATION OF ENKF AND MLEF FOR ASSIMILATION OF STREAMFLOW DATA INTO NWS OPERATIONAL HYDROLOGIC MODELS Arezoo Raieei Nasab 1, Dong-Jun Seo 1, Haksu Lee 2,3, Sunghee Kim 1 1 Dept. o Civil Eng.,

More information

A Spectral Approach to Linear Bayesian Updating

A Spectral Approach to Linear Bayesian Updating A Spectral Approach to Linear Bayesian Updating Oliver Pajonk 1,2, Bojana V. Rosic 1, Alexander Litvinenko 1, and Hermann G. Matthies 1 1 Institute of Scientific Computing, TU Braunschweig, Germany 2 SPT

More information

A Note on the Particle Filter with Posterior Gaussian Resampling

A Note on the Particle Filter with Posterior Gaussian Resampling Tellus (6), 8A, 46 46 Copyright C Blackwell Munksgaard, 6 Printed in Singapore. All rights reserved TELLUS A Note on the Particle Filter with Posterior Gaussian Resampling By X. XIONG 1,I.M.NAVON 1,2 and

More information

State and Parameter Estimation in Stochastic Dynamical Models

State and Parameter Estimation in Stochastic Dynamical Models State and Parameter Estimation in Stochastic Dynamical Models Timothy DelSole George Mason University, Fairfax, Va and Center for Ocean-Land-Atmosphere Studies, Calverton, MD June 21, 2011 1 1 collaboration

More information

UNDERSTANDING DATA ASSIMILATION APPLICATIONS TO HIGH-LATITUDE IONOSPHERIC ELECTRODYNAMICS

UNDERSTANDING DATA ASSIMILATION APPLICATIONS TO HIGH-LATITUDE IONOSPHERIC ELECTRODYNAMICS Monday, June 27, 2011 CEDAR-GEM joint workshop: DA tutorial 1 UNDERSTANDING DATA ASSIMILATION APPLICATIONS TO HIGH-LATITUDE IONOSPHERIC ELECTRODYNAMICS Tomoko Matsuo University of Colorado, Boulder Space

More information

Multimodel Ensemble forecasts

Multimodel Ensemble forecasts Multimodel Ensemble forecasts Calibrated methods Michael K. Tippett International Research Institute for Climate and Society The Earth Institute, Columbia University ERFS Climate Predictability Tool Training

More information

Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF

Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF Eugenia Kalnay and Shu-Chih Yang with Alberto Carrasi, Matteo Corazza and Takemasa Miyoshi 4th EnKF Workshop, April 2010 Relationship

More information

The Ensemble Kalman Filter:

The Ensemble Kalman Filter: p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen, Ocean Dynamics, Vol 5, No p. The Ensemble

More information

Application of the Ensemble Kalman Filter to History Matching

Application of the Ensemble Kalman Filter to History Matching Application of the Ensemble Kalman Filter to History Matching Presented at Texas A&M, November 16,2010 Outline Philosophy EnKF for Data Assimilation Field History Match Using EnKF with Covariance Localization

More information

An Ensemble based Reliable Storm Surge Forecasting for Gulf of Mexico

An Ensemble based Reliable Storm Surge Forecasting for Gulf of Mexico An Ensemble based Reliable Storm Surge Forecasting for Gulf of Mexico Umer Altaf Delft University of Technology, Delft ICES, University of Texas at Austin, USA KAUST, Saudi Arabia JONSMOD 2012, Ifremer,

More information

Gaussian Filtering Strategies for Nonlinear Systems

Gaussian Filtering Strategies for Nonlinear Systems Gaussian Filtering Strategies for Nonlinear Systems Canonical Nonlinear Filtering Problem ~u m+1 = ~ f (~u m )+~ m+1 ~v m+1 = ~g(~u m+1 )+~ o m+1 I ~ f and ~g are nonlinear & deterministic I Noise/Errors

More information

Parameter Estimation in Reservoir Engineering Models via Data Assimilation Techniques

Parameter Estimation in Reservoir Engineering Models via Data Assimilation Techniques Parameter Estimation in Reservoir Engineering Models via Data Assimilation Techniques Mariya V. Krymskaya TU Delft July 6, 2007 Ensemble Kalman Filter (EnKF) Iterative Ensemble Kalman Filter (IEnKF) State

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond Department of Biomedical Engineering and Computational Science Aalto University January 26, 2012 Contents 1 Batch and Recursive Estimation

More information

Adaptive ensemble Kalman filtering of nonlinear systems

Adaptive ensemble Kalman filtering of nonlinear systems Adaptive ensemble Kalman filtering of nonlinear systems Tyrus Berry George Mason University June 12, 213 : Problem Setup We consider a system of the form: x k+1 = f (x k ) + ω k+1 ω N (, Q) y k+1 = h(x

More information

An Iterative EnKF for Strongly Nonlinear Systems

An Iterative EnKF for Strongly Nonlinear Systems 1988 M O N T H L Y W E A T H E R R E V I E W VOLUME 140 An Iterative EnKF for Strongly Nonlinear Systems PAVEL SAKOV Nansen Environmental and Remote Sensing Center, Bergen, Norway DEAN S. OLIVER Uni Centre

More information

Cross-validation methods for quality control, cloud screening, etc.

Cross-validation methods for quality control, cloud screening, etc. Cross-validation methods for quality control, cloud screening, etc. Olaf Stiller, Deutscher Wetterdienst Are observations consistent Sensitivity functions with the other observations? given the background

More information

DART Tutorial Section 18: Lost in Phase Space: The Challenge of Not Knowing the Truth.

DART Tutorial Section 18: Lost in Phase Space: The Challenge of Not Knowing the Truth. DART Tutorial Section 18: Lost in Phase Space: The Challenge of Not Knowing the Truth. UCAR The National Center for Atmospheric Research is sponsored by the National Science Foundation. Any opinions, findings

More information

Towards a probabilistic hydrological forecasting and data assimilation system. Henrik Madsen DHI, Denmark

Towards a probabilistic hydrological forecasting and data assimilation system. Henrik Madsen DHI, Denmark Towards a probabilistic hydrological forecasting and data assimilation system Henrik Madsen DHI, Denmark Outline Hydrological forecasting Data assimilation framework Data assimilation experiments Concluding

More information

A Stochastic Collocation based. for Data Assimilation

A Stochastic Collocation based. for Data Assimilation A Stochastic Collocation based Kalman Filter (SCKF) for Data Assimilation Lingzao Zeng and Dongxiao Zhang University of Southern California August 11, 2009 Los Angeles Outline Introduction SCKF Algorithm

More information

Bayesian Inverse problem, Data assimilation and Localization

Bayesian Inverse problem, Data assimilation and Localization Bayesian Inverse problem, Data assimilation and Localization Xin T Tong National University of Singapore ICIP, Singapore 2018 X.Tong Localization 1 / 37 Content What is Bayesian inverse problem? What is

More information

Fundamentals of Data Assimilation

Fundamentals of Data Assimilation National Center for Atmospheric Research, Boulder, CO USA GSI Data Assimilation Tutorial - June 28-30, 2010 Acknowledgments and References WRFDA Overview (WRF Tutorial Lectures, H. Huang and D. Barker)

More information

Finite-size ensemble Kalman filters (EnKF-N)

Finite-size ensemble Kalman filters (EnKF-N) Finite-size ensemble Kalman filters (EnKF-N) Marc Bocquet (1,2), Pavel Sakov (3,4) (1) Université Paris-Est, CEREA, joint lab École des Ponts ParisTech and EdF R&D, France (2) INRIA, Paris-Rocquencourt

More information

Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications

Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Dongbin Xiu Department of Mathematics, Purdue University Support: AFOSR FA955-8-1-353 (Computational Math) SF CAREER DMS-64535

More information

Application of Mean Recentering Scheme to Improve the Typhoon Track Forecast: A Case Study of Typhoon Nanmadol (2011) Chih-Chien Chang, Shu-Chih Yang

Application of Mean Recentering Scheme to Improve the Typhoon Track Forecast: A Case Study of Typhoon Nanmadol (2011) Chih-Chien Chang, Shu-Chih Yang 6 th EnKF workshop Application of Mean Recentering Scheme to Improve the Typhoon Track Forecast: A Case Study of Typhoon Nanmadol (2011) Chih-Chien Chang, Shu-Chih Yang National Central University, Taiwan

More information

Forecasting and data assimilation

Forecasting and data assimilation Supported by the National Science Foundation DMS Forecasting and data assimilation Outline Numerical models Kalman Filter Ensembles Douglas Nychka, Thomas Bengtsson, Chris Snyder Geophysical Statistics

More information

Lecture 2: From Linear Regression to Kalman Filter and Beyond

Lecture 2: From Linear Regression to Kalman Filter and Beyond Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing

More information

Analysis Scheme in the Ensemble Kalman Filter

Analysis Scheme in the Ensemble Kalman Filter JUNE 1998 BURGERS ET AL. 1719 Analysis Scheme in the Ensemble Kalman Filter GERRIT BURGERS Royal Netherlands Meteorological Institute, De Bilt, the Netherlands PETER JAN VAN LEEUWEN Institute or Marine

More information

Data assimilation in high dimensions

Data assimilation in high dimensions Data assimilation in high dimensions David Kelly Kody Law Andy Majda Andrew Stuart Xin Tong Courant Institute New York University New York NY www.dtbkelly.com February 3, 2016 DPMMS, University of Cambridge

More information

A Data-driven Approach for Remaining Useful Life Prediction of Critical Components

A Data-driven Approach for Remaining Useful Life Prediction of Critical Components GT S3 : Sûreté, Surveillance, Supervision Meeting GdR Modélisation, Analyse et Conduite des Systèmes Dynamiques (MACS) January 28 th, 2014 A Data-driven Approach for Remaining Useful Life Prediction of

More information

Advances and Challenges in Ensemblebased Data Assimilation in Meteorology. Takemasa Miyoshi

Advances and Challenges in Ensemblebased Data Assimilation in Meteorology. Takemasa Miyoshi January 18, 2013, DA Workshop, Tachikawa, Japan Advances and Challenges in Ensemblebased Data Assimilation in Meteorology Takemasa Miyoshi RIKEN Advanced Institute for Computational Science Takemasa.Miyoshi@riken.jp

More information

Lagrangian Data Assimilation and its Applications to Geophysical Fluid Flows

Lagrangian Data Assimilation and its Applications to Geophysical Fluid Flows Lagrangian Data Assimilation and its Applications to Geophysical Fluid Flows by Laura Slivinski B.S., University of Maryland; College Park, MD, 2009 Sc.M, Brown University; Providence, RI, 2010 A dissertation

More information

Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model

Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model Estimating Observation Impact in a Hybrid Data Assimilation System: Experiments with a Simple Model NOAA / NCEP / EMC College Park, MD 20740, USA 10 February 2014 Overview Goal Sensitivity Theory Adjoint

More information

DART Tutorial Part IV: Other Updates for an Observed Variable

DART Tutorial Part IV: Other Updates for an Observed Variable DART Tutorial Part IV: Other Updates for an Observed Variable UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons

More information

Practical Aspects of Ensemble-based Kalman Filters

Practical Aspects of Ensemble-based Kalman Filters Practical Aspects of Ensemble-based Kalman Filters Lars Nerger Alfred Wegener Institute for Polar and Marine Research Bremerhaven, Germany and Bremen Supercomputing Competence Center BremHLR Bremen, Germany

More information

Relative Merits of 4D-Var and Ensemble Kalman Filter

Relative Merits of 4D-Var and Ensemble Kalman Filter Relative Merits of 4D-Var and Ensemble Kalman Filter Andrew Lorenc Met Office, Exeter International summer school on Atmospheric and Oceanic Sciences (ISSAOS) "Atmospheric Data Assimilation". August 29

More information

Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit

Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Evan Kwiatkowski, Jan Mandel University of Colorado Denver December 11, 2014 OUTLINE 2 Data Assimilation Bayesian Estimation

More information

Error (E) Covariance Diagnostics. in Ensemble Data Assimilation. Kayo Ide, Daryl Kleist, Adam Kubaryk University of Maryland

Error (E) Covariance Diagnostics. in Ensemble Data Assimilation. Kayo Ide, Daryl Kleist, Adam Kubaryk University of Maryland Error (E) Covariance Diagnostics in Ensemble Data Assimilation Kayo Ide, Daryl Kleist, Adam Kubaryk University of Maryland 7 th EnKF Data Assimilation Workshop May 2016 Outline Part I. Background: E-diagnostics

More information

An ETKF approach for initial state and parameter estimation in ice sheet modelling

An ETKF approach for initial state and parameter estimation in ice sheet modelling An ETKF approach for initial state and parameter estimation in ice sheet modelling Bertrand Bonan University of Reading DARC Seminar, Reading October 30, 2013 DARC Seminar (Reading) ETKF for ice sheet

More information

State Space Models for Wind Forecast Correction

State Space Models for Wind Forecast Correction for Wind Forecast Correction Valérie 1 Pierre Ailliot 2 Anne Cuzol 1 1 Université de Bretagne Sud 2 Université de Brest MAS - 2008/28/08 Outline 1 2 Linear Model : an adaptive bias correction Non Linear

More information

Bayesian Statistics and Data Assimilation. Jonathan Stroud. Department of Statistics The George Washington University

Bayesian Statistics and Data Assimilation. Jonathan Stroud. Department of Statistics The George Washington University Bayesian Statistics and Data Assimilation Jonathan Stroud Department of Statistics The George Washington University 1 Outline Motivation Bayesian Statistics Parameter Estimation in Data Assimilation Combined

More information

Methods of Data Assimilation and Comparisons for Lagrangian Data

Methods of Data Assimilation and Comparisons for Lagrangian Data Methods of Data Assimilation and Comparisons for Lagrangian Data Chris Jones, Warwick and UNC-CH Kayo Ide, UCLA Andrew Stuart, Jochen Voss, Warwick Guillaume Vernieres, UNC-CH Amarjit Budiraja, UNC-CH

More information

Jidong Gao and David Stensrud. NOAA/National Severe Storm Laboratory Norman, Oklahoma

Jidong Gao and David Stensrud. NOAA/National Severe Storm Laboratory Norman, Oklahoma Assimilation of Reflectivity and Radial Velocity in a Convective-Scale, Cycled 3DVAR Framework with Hydrometeor Classification Jidong Gao and David Stensrud NOAA/National Severe Storm Laboratory Norman,

More information

Model error and parameter estimation

Model error and parameter estimation Model error and parameter estimation Chiara Piccolo and Mike Cullen ECMWF Annual Seminar, 11 September 2018 Summary The application of interest is atmospheric data assimilation focus on EDA; A good ensemble

More information

6 Sequential Data Assimilation for Nonlinear Dynamics: The Ensemble Kalman Filter

6 Sequential Data Assimilation for Nonlinear Dynamics: The Ensemble Kalman Filter 6 Sequential Data Assimilation for Nonlinear Dynamics: The Ensemble Kalman Filter GEIR EVENSEN Nansen Environmental and Remote Sensing Center, Bergen, Norway 6.1 Introduction Sequential data assimilation

More information

CIS 390 Fall 2016 Robotics: Planning and Perception Final Review Questions

CIS 390 Fall 2016 Robotics: Planning and Perception Final Review Questions CIS 390 Fall 2016 Robotics: Planning and Perception Final Review Questions December 14, 2016 Questions Throughout the following questions we will assume that x t is the state vector at time t, z t is the

More information

Toward improved initial conditions for NCAR s real-time convection-allowing ensemble. Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell

Toward improved initial conditions for NCAR s real-time convection-allowing ensemble. Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell Toward improved initial conditions for NCAR s real-time convection-allowing ensemble Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell Storm-scale ensemble design Can an EnKF be used to initialize

More information

4. DATA ASSIMILATION FUNDAMENTALS

4. DATA ASSIMILATION FUNDAMENTALS 4. DATA ASSIMILATION FUNDAMENTALS... [the atmosphere] "is a chaotic system in which errors introduced into the system can grow with time... As a consequence, data assimilation is a struggle between chaotic

More information

A data assimilation approach for reconstructing sea ice volume in the Southern Hemisphere

A data assimilation approach for reconstructing sea ice volume in the Southern Hemisphere Harmony on Ice 2 meeting Paris, 28-29 Nov. 2011 A data assimilation approach for reconstructing sea ice volume in the Southern Hemisphere F. Massonnet, P. Mathiot, T. Fichefet, H. Goosse, C. König Beatty,

More information

EnKF Review. P.L. Houtekamer 7th EnKF workshop Introduction to the EnKF. Challenges. The ultimate global EnKF algorithm

EnKF Review. P.L. Houtekamer 7th EnKF workshop Introduction to the EnKF. Challenges. The ultimate global EnKF algorithm Overview 1 2 3 Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation 6th EnKF Purpose EnKF equations localization After the 6th EnKF (2014), I decided with Prof. Zhang to summarize progress

More information

Data Assimilation in Geosciences: A highly multidisciplinary enterprise

Data Assimilation in Geosciences: A highly multidisciplinary enterprise Data Assimilation in Geosciences: A highly multidisciplinary enterprise Adrian Sandu Computational Science Laboratory Department of Computer Science Virginia Tech Data assimilation fuses information from

More information

Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF

Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF Eugenia Kalnay and Shu-Chih Yang with Alberto Carrasi, Matteo Corazza and Takemasa Miyoshi ECODYC10, Dresden 28 January 2010 Relationship

More information

Gaussian Process Approximations of Stochastic Differential Equations

Gaussian Process Approximations of Stochastic Differential Equations Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run

More information

Aspects of the practical application of ensemble-based Kalman filters

Aspects of the practical application of ensemble-based Kalman filters Aspects of the practical application of ensemble-based Kalman filters Lars Nerger Alfred Wegener Institute for Polar and Marine Research Bremerhaven, Germany and Bremen Supercomputing Competence Center

More information

Data assimilation; comparison of 4D-Var and LETKF smoothers

Data assimilation; comparison of 4D-Var and LETKF smoothers Data assimilation; comparison of 4D-Var and LETKF smoothers Eugenia Kalnay and many friends University of Maryland CSCAMM DAS13 June 2013 Contents First part: Forecasting the weather - we are really getting

More information

Ensemble Data Assimila.on and Uncertainty Quan.fica.on

Ensemble Data Assimila.on and Uncertainty Quan.fica.on Ensemble Data Assimila.on and Uncertainty Quan.fica.on Jeffrey Anderson, Alicia Karspeck, Tim Hoar, Nancy Collins, Kevin Raeder, Steve Yeager Na.onal Center for Atmospheric Research Ocean Sciences Mee.ng

More information

Adaptive Inflation for Ensemble Assimilation

Adaptive Inflation for Ensemble Assimilation Adaptive Inflation for Ensemble Assimilation Jeffrey Anderson IMAGe Data Assimilation Research Section (DAReS) Thanks to Ryan Torn, Nancy Collins, Tim Hoar, Hui Liu, Kevin Raeder, Xuguang Wang Anderson:

More information

Systematic strategies for real time filtering of turbulent signals in complex systems

Systematic strategies for real time filtering of turbulent signals in complex systems Systematic strategies for real time filtering of turbulent signals in complex systems Statistical inversion theory for Gaussian random variables The Kalman Filter for Vector Systems: Reduced Filters and

More information

Robust Ensemble Filtering With Improved Storm Surge Forecasting

Robust Ensemble Filtering With Improved Storm Surge Forecasting Robust Ensemble Filtering With Improved Storm Surge Forecasting U. Altaf, T. Buttler, X. Luo, C. Dawson, T. Mao, I.Hoteit Meteo France, Toulouse, Nov 13, 2012 Project Ensemble data assimilation for storm

More information

Organization. I MCMC discussion. I project talks. I Lecture.

Organization. I MCMC discussion. I project talks. I Lecture. Organization I MCMC discussion I project talks. I Lecture. Content I Uncertainty Propagation Overview I Forward-Backward with an Ensemble I Model Reduction (Intro) Uncertainty Propagation in Causal Systems

More information

DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING. Inversion basics. Erkki Kyrölä Finnish Meteorological Institute

DRAGON ADVANCED TRAINING COURSE IN ATMOSPHERE REMOTE SENSING. Inversion basics. Erkki Kyrölä Finnish Meteorological Institute Inversion basics y = Kx + ε x ˆ = (K T K) 1 K T y Erkki Kyrölä Finnish Meteorological Institute Day 3 Lecture 1 Retrieval techniques - Erkki Kyrölä 1 Contents 1. Introduction: Measurements, models, inversion

More information

New Fast Kalman filter method

New Fast Kalman filter method New Fast Kalman filter method Hojat Ghorbanidehno, Hee Sun Lee 1. Introduction Data assimilation methods combine dynamical models of a system with typically noisy observations to obtain estimates of the

More information

Variational ensemble DA at Météo-France Cliquez pour modifier le style des sous-titres du masque

Variational ensemble DA at Météo-France Cliquez pour modifier le style des sous-titres du masque Cliquez pour modifier le style du titre Variational ensemble DA at Météo-France Cliquez pour modifier le style des sous-titres du masque L. Berre, G. Desroziers, H. Varella, L. Raynaud, C. Labadie and

More information

DART_LAB Tutorial Section 5: Adaptive Inflation

DART_LAB Tutorial Section 5: Adaptive Inflation DART_LAB Tutorial Section 5: Adaptive Inflation UCAR 14 The National Center for Atmospheric Research is sponsored by the National Science Foundation. Any opinions, findings and conclusions or recommendations

More information

Estimation of positive sum-to-one constrained parameters with ensemble-based Kalman filters: application to an ocean ecosystem model

Estimation of positive sum-to-one constrained parameters with ensemble-based Kalman filters: application to an ocean ecosystem model Estimation of positive sum-to-one constrained parameters with ensemble-based Kalman filters: application to an ocean ecosystem model Ehouarn Simon 1, Annette Samuelsen 1, Laurent Bertino 1 Dany Dumont

More information

Using Observations at Different Spatial. Scales in Data Assimilation for. Environmental Prediction. Joanne A. Waller

Using Observations at Different Spatial. Scales in Data Assimilation for. Environmental Prediction. Joanne A. Waller UNIVERSITY OF READING DEPARTMENT OF MATHEMATICS AND STATISTICS Using Observations at Different Spatial Scales in Data Assimilation for Environmental Prediction Joanne A. Waller Thesis submitted for the

More information

Review of Covariance Localization in Ensemble Filters

Review of Covariance Localization in Ensemble Filters NOAA Earth System Research Laboratory Review of Covariance Localization in Ensemble Filters Tom Hamill NOAA Earth System Research Lab, Boulder, CO tom.hamill@noaa.gov Canonical ensemble Kalman filter update

More information

P 1.86 A COMPARISON OF THE HYBRID ENSEMBLE TRANSFORM KALMAN FILTER (ETKF)- 3DVAR AND THE PURE ENSEMBLE SQUARE ROOT FILTER (EnSRF) ANALYSIS SCHEMES

P 1.86 A COMPARISON OF THE HYBRID ENSEMBLE TRANSFORM KALMAN FILTER (ETKF)- 3DVAR AND THE PURE ENSEMBLE SQUARE ROOT FILTER (EnSRF) ANALYSIS SCHEMES P 1.86 A COMPARISON OF THE HYBRID ENSEMBLE TRANSFORM KALMAN FILTER (ETKF)- 3DVAR AND THE PURE ENSEMBLE SQUARE ROOT FILTER (EnSRF) ANALYSIS SCHEMES Xuguang Wang*, Thomas M. Hamill, Jeffrey S. Whitaker NOAA/CIRES

More information

4DEnVar. Four-Dimensional Ensemble-Variational Data Assimilation. Colloque National sur l'assimilation de données

4DEnVar. Four-Dimensional Ensemble-Variational Data Assimilation. Colloque National sur l'assimilation de données Four-Dimensional Ensemble-Variational Data Assimilation 4DEnVar Colloque National sur l'assimilation de données Andrew Lorenc, Toulouse France. 1-3 décembre 2014 Crown copyright Met Office 4DEnVar: Topics

More information