Data Assimilation Research Testbed Tutorial
|
|
- Dayna Warren
- 5 years ago
- Views:
Transcription
1 Data Assimilation Research Testbed Tutorial Section 3: Hierarchical Group Filters and Localization Version 2.: September, 26 Anderson: Ensemble Tutorial 9//6
2 Ways to deal with regression sampling error: 3. Use additional a priori information about relation between observations and state variables. Regression Weight Distance from Observation (Km?) Can use other functions to weight regression. Unclear what distance means for some obs./state variable pairs. Referred to as LOCALIZATION. Anderson: Ensemble Tutorial 2 9//6
3 Localization is function of expected correlation between obs and state. Often, don t know much about this. Horizontal distance between same type of variable may be okay. What is expected correlation for co-located temperature and pressure? What about vertical localization? Looks pretty complex. What about complicated forward operators: Expected correlation of satellite radiance and wind component? Note: DART does allow vertical localization for more complex models. Anderson: Ensemble Tutorial 3 9//6
4 Ways to deal with regression sampling error: 4. Try to determine the amount of sampling error and correct for it: A. Could weight regressions based on sample correlation. Limited success in tests. For small true correlations, can still get large sample correl. B. Do bootstrap with sample correlation to measure sampling error. Limited success. Repeatedly compute sample correlation with a sample removed. C. Use hierarchical Monte Carlo. Have a sample of samples. Compute expected error in regression coefficients and weight. Anderson: Ensemble Tutorial 4 9//6
5 Ways to deal with regression sampling error: 4C. Use hierarchical Monte Carlo: ensemble of ensembles. t k * * t k * * H H H H H M independent N-member Ensembles H y y β y t k+2 Regression Confidence Factor, α β Μ y t k+2 M groups of N-member ensembles. Compute obs. increments for each group. For given obs. / state pair:. Have M samples of regression coefficient, β. 2. Uncertainty in β implies state variable increments should be reduced. 3. Compute regression confidence factor, α. Anderson: Ensemble Tutorial 5 9//6
6 4C. Use hierarchical Monte Carlo: ensemble of ensembles. Split ensemble into M independent groups. For instance, 8 ensemble members becomes 4 groups of 2. With M groups get M estimates of regression coefficient, β i. Find regression confidence factor α (weight) that minimizes: M j = M [ αβ i β j ] 2 i =, i j Minimizes RMS error in the regression (and state increments). Anderson: Ensemble Tutorial 6 9//6
7 4C. Use hierarchical Monte Carlo: ensemble of ensembles. Regression Confidence Factor, α Group Size 2 Group Size 4 Group Size 8 Group Size Q: Ratio of sample standard deviation to mean Weight regression by α. If one has repeated observations, can generate sample mean or median statistics for α. Mean α can be used in subsequent assimilations as a localization. α is function of M and Q = Σ β β (sample SD / sample mean regression) Anderson: Ensemble Tutorial 7 9//6
8 4C. Use hierarchical Monte Carlo: ensemble of ensembles. Hierarchical filter controlled by setting number of groups, M. num_groups in filter_nml. If we don t know how to localize to start with, can use groups to help. Retrieve original values for input.nml by getting from archive copy (we ll have to determine where this is for this workshop). Try splitting 8 ensemble members into 4 groups for Lorenz-96. (num_groups=4, ens_size=8, 4 groups of 2 each). Use adaptive inflation (.5 lower bounds) to make things nice. (inf_flavor=3, inf_sd_initial=.5, inf_sd_lower_bound=.5) Anderson: Ensemble Tutorial 8 9//6
9 4C. Use hierarchical Monte Carlo: ensemble of ensembles. Turn on regression factor diagnostics, save_reg_diagnostics=.true. After running the 8 by 4 group filter, look at plots of α. Essentially an estimate of a good localization for a given observation. Use plot_reg_factor in matlab. Select default input file name. Only observations, 2, 3, and 4 are available: Located at:.39,.7,.64,.86 Think about value of time median vs. time mean. Could use time mean or median as prior localization functions Play around with model error again. What happens to localization? Anderson: Ensemble Tutorial 9 9//6
10 Lorenz 96 Experimental Design Initial ensemble members random draws from climatology Observations every time step 4 step assimilations, results shown from second 2 steps Covariance inflation tuned for minimum RMS 4 groups of ensembles used unless otherwise noted EXPERIMENT SET : 4 Randomly located observations Error variance -7 (SMALL!) ERROR comes almost entirely from degeneracy of ensemble covariance All errors shown are prior ensemble mean estimates Anderson: Ensemble Tutorial 9//6
11 Time Mean Regression Confidence Envelopes: Small Error Limit.9 Regression Confidence Factor Most results are for observation at State Variable Location Location of 4 randomly located observations Anderson: Ensemble Tutorial 9//6
12 Time Mean Regression Confidence Envelopes: Small Error Limit.9 OBS ENS. 3 Regression Confidence Factor State Variable Location Envelope (localization) for 3 member ensemble (barely degenerate) Anderson: Ensemble Tutorial 2 9//6
13 Time Mean Regression Confidence Envelopes: Small Error Limit Regression Confidence Factor OBS ENS. 3 ENS State Variable Location Envelope for 8 member ensemble Anderson: Ensemble Tutorial 3 9//6
14 Time Mean Regression Confidence Envelopes: Small Error Limit Regression Confidence Factor OBS ENS. 3 ENS. 8 ENS State Variable Location Envelope for 5 member ensemble Anderson: Ensemble Tutorial 4 9//6
15 Time Mean Regression Confidence Envelopes: Small Error Limit Regression Confidence Factor OBS ENS. 3 ENS. 8 ENS. 5 GC State Variable Location Compare to Gaspari Cohn with half-width.2 Anderson: Ensemble Tutorial 5 9//6
16 Time Mean Regression Confidence Envelopes: Small Error Limit Regression Confidence Factor OBS ENS. 3 ENS. 8 ENS. 5 GC.2 ENS. > State Variable Location Ensemble sizes > 3 require NO LOCALIZATION (no significant error) Anderson: Ensemble Tutorial 6 9//6
17 Time Median Regression Confidence Envelopes: Small Error Limit Regression Confidence Factor ENS. 3 ENS. 8 ENS. 5 GC.2 ENS. > State Variable Location Median reduces noise for small expected correlations Anderson: Ensemble Tutorial 7 9//6
18 Additional Experiments: Experimental Design. Base case: plain ensemble, optimized Gaspari Cohn localization half-width 2. Time mean case: plain ensemble but with localization using time mean regression confidence envelope from group assimilation 3. Time median case: as above but with time median from group All Start from group ensemble at time step 2 Covariance inflation tuned independently for each case Anderson: Ensemble Tutorial 8 9//6
19 Time Mean global mean RMS Error: Small Error Results Error grows with reduced ensemble size base 4 group 4 mean 4 median 8 group 8 mean 8 median 8 Group filters always best Time mean and median close to tuned base case Ensemble 3 Ensemble 8 Ensemble 5 Mean/median cost same as base case Anderson: Ensemble Tutorial 9 9//6
20 Experimental Design: Varying Observational Error Variance Observation set as before Observation error variance -5, -3,.,.,., 7 4 member ensembles; not degenerate Error source is now from observation limitations, etc. Claim: behavior is similar, error source is irrelevant for correction Understand degeneracy and other error sources as sampling error Anderson: Ensemble Tutorial 2 9//6
21 Time Median Envelopes: Varying Obs. Error Variance.9 Regression Confidence Factor e State Variable Location Small error implies no need for localization Anderson: Ensemble Tutorial 2 9//6
22 Time Median Envelopes: Varying Obs. Error Variance.9 Regression Confidence Factor e 5 e State Variable Location Anderson: Ensemble Tutorial 22 9//6
23 Time Median Envelopes: Varying Obs. Error Variance Regression Confidence Factor e 5. e 3 e State Variable Location Increasing error implies increasing localization Anderson: Ensemble Tutorial 23 9//6
24 Time Median Envelopes: Varying Obs. Error Variance.9 Regression Confidence Factor e 5. e 3 e State Variable Location Anderson: Ensemble Tutorial 24 9//6
25 Time Median Envelopes: Varying Obs. Error Variance Regression Confidence Factor e 5 e 3. e State Variable Location Single Gaspari Cohn half-width can t deal with this range of errors Anderson: Ensemble Tutorial 25 9//6
26 Time Median Envelopes: Varying Obs. Error Variance Regression Confidence Factor e 5.2 e 3 e... e State Variable Location Climatological case is unique: Looks like time mean coherence Anderson: Ensemble Tutorial 26 9//6
27 Time Median Envelopes: Varying Obs. Error Variance Base 4 4x4 Mean Median Error scaled by obs. error standard deviation As error grows, filter becomes less efficient Things are more nonlinear for large errors.25.2 E 5 E 3... Group, mean and median compare favorably with tuned base for all cases! Anderson: Ensemble Tutorial 27 9//6
28 Regression Confidence Factor Sensitivity of results to group size 2 Groups 4 Groups 8 Groups 6 Groups State Variable Location Obs. error variance., 4 member ensemble case Anderson: Ensemble Tutorial 28 9//6
29 groups 4 groups 8 groups 6 groups Sensitivity of results to group size Gradual improvement for increased group size Time mean/median improve more slowly Important that small groups give good results (for cost) Group better than time mean implies some time varying information (time means should reduce noise) Group Time Mean Time Median Obs. error variance., 4 member ensemble case Anderson: Ensemble Tutorial 29 9//6
30 5 Time variation of regression confidence factor Time 25.5 Time State Variable Location State Variable Location 6 groups 2 groups Some consistent time variation; noise dominates far from obs. Notice behavior around step 33 for instance.2 Anderson: Ensemble Tutorial 3 9//6
31 Time Mean Error Challenging Traditional Localization: Varying spatial obs. density Groups Base Filter Time Mean State Variable Location 39 Dense obs isolated ob. Group filter does better in sparse region Time mean filter performance in between Time Mean Prior RMS Error as function of spatial location Anderson: Ensemble Tutorial 3 9//6
32 Challenging Traditional Localization: Varying spatial obs. density.9 Regression Confidence Factor OB OB 9 OB 39 OB State Variable Location Time median envelopes vary lots with spatial location of obs. Anderson: Ensemble Tutorial 32 9//6
33 Regression Confidence Factor Challenging traditional localization: Spatial-mean obs. Forward observation operator Time Mean Time Median is fifteen grid point mean State Variable Location Envelope has become less Gaussian In one-d L96 domain, everything stays pretty Gaussian But..., group filter does eliminate need for tuning localization! Anderson: Ensemble Tutorial 33 9//6
34 Assimilating observations at times different from state estimate Ensemble smoothers: use future observations Targeted observations: examine impact of obs. in past Real-time assimilation: use of late arriving observations in forecast Expect correlations to diminish as time separation increases Need a localization in time, too Group filter can provide this Anderson: Ensemble Tutorial 34 9//6
35 Time localization : Experimental design 4 group, 4 ensemble member filter 4 random obs. with. error variance additional observation at location.642 The additional observation is from a prior time step Time mean regression confidence envelope as function of time lag Anderson: Ensemble Tutorial 35 9//6
36 Regression confidence factor as function of obs. lag time.9 Time Lag State Variable Location Moves with group velocity (approximately); dies off with lead Anderson: Ensemble Tutorial 36 9//6
37 Assimilation in Idealized AGCM: GFDL FMS B-Grid Dynamical Core (Havana) Held-Suarez Configuration (no zonal variation, fixed forcing) Low-Resolution (6 lons, 3 lats, 5 levels); Timestep hour T at Level 3 (Middle of Atmosphere) Cross Section of Zonal Mean U Has Baroclinic Instability Anderson: Ensemble Tutorial 37 9//6
38 Surface Pressure: Day39 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 38 9//6
39 Surface Pressure: Day392 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 39 9//6
40 Surface Pressure: Day393 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 4 9//6
41 Surface Pressure: Day394 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 4 9//6
42 Surface Pressure: Day395 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 42 9//6
43 Surface Pressure: Day396 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 43 9//6
44 Surface Pressure: Day397 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 44 9//6
45 Surface Pressure: Day398 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 45 9//6
46 Surface Pressure: Day399 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 46 9//6
47 Surface Pressure: Day4 x Temperature, Level 3: Day Anderson: Ensemble Tutorial 47 9//6
48 Experimental Design Details: Bgrid AGCM Results for 4x2 group filter Assimilation for 4 days; starting from climatological distribution Summary results are from last 2 days No covariance inflation 8 randomly located surface pressure stations observe once every 24 hours Observational error variance is mb Anderson: Ensemble Tutorial 48 9//6
49 Baseline Case: 8 PS Obs every 24 hours Largest error in midlatitudes, synoptic scales after 4 days Min = Max = RMS ERROR = HECTO PASCALS RMS ENSEMBLE MEAN ERROR 45 ENSEMBLE SPREAD PS Error Reduces by aboutx Asymptotes after about 5 days DAYS Ensemble spread is approximately correlated with RMS error Anderson: Ensemble Tutorial 49 9//6
50 Largest T error in tropics for interior levels (level 3, day 4 shown) DEGREES K Baseline Case: 8 PS Obs every 24 hours Min =.92 Max = RMS ERROR = LEVEL (Top) LEVEL2 LEVEL3 LEVEL4 LEVEL5 (Bottom) 5 5 DAYS T Error also reduced by about x Asymptotes about day Final error about.25 K for interior levels Anderson: Ensemble Tutorial 5 9//6
51 Hierarchical Filter Regression Confidence Factors: PS Obs. at 2N, 6E ps mean factor cross section at row PS to PS ps median factor cross section at column Anderson: Ensemble Tutorial 5 9//6
52 Hierarchical Filter Regression Confidence Factors: PS Obs. at 2N, 6E t mean factor level t mean factor level t mean factor level PS to T t mean factor level t mean factor level cross section at row Anderson: Ensemble Tutorial 52 9//6
53 Hierarchical Filter Regression Confidence Factors: PS Obs. at 2N, 6E u mean factor level u mean factor level u mean factor level PS to U u mean factor level u mean factor level cross section at row Anderson: Ensemble Tutorial 53 9//6
54 Hierarchical Filter Regression Confidence Factors: PS Obs. at 2N, 6E v mean factor level v mean factor level v mean factor level PS to V v mean factor level v mean factor level cross section at row Anderson: Ensemble Tutorial 54 9//6
55 Running the bgrid dynamical core Go to models/bgrid_solo/work and run workshop_setup.csh This assimilates hourly observations over a two-hour period Observations are 6 randomly located column observations Each column observes surface pressure, plus T, u and v at each level Use diagnostic tools to examine output Useful to compute and examine innovation netcdf field Try ncdiff Prior_Diag.nc Posterior_Diag.nc Innov.nc Anderson: Ensemble Tutorial 55 9//6
Localization and Correlation in Ensemble Kalman Filters
Localization and Correlation in Ensemble Kalman Filters Jeff Anderson NCAR Data Assimilation Research Section The National Center for Atmospheric Research is sponsored by the National Science Foundation.
More informationReducing the Impact of Sampling Errors in Ensemble Filters
Reducing the Impact of Sampling Errors in Ensemble Filters Jeffrey Anderson NCAR Data Assimilation Research Section The National Center for Atmospheric Research is sponsored by the National Science Foundation.
More informationEnsemble Data Assimilation and Uncertainty Quantification
Ensemble Data Assimilation and Uncertainty Quantification Jeff Anderson National Center for Atmospheric Research pg 1 What is Data Assimilation? Observations combined with a Model forecast + to produce
More informationEnKF Review. P.L. Houtekamer 7th EnKF workshop Introduction to the EnKF. Challenges. The ultimate global EnKF algorithm
Overview 1 2 3 Review of the Ensemble Kalman Filter for Atmospheric Data Assimilation 6th EnKF Purpose EnKF equations localization After the 6th EnKF (2014), I decided with Prof. Zhang to summarize progress
More informationData Assimilation Research Testbed Tutorial
Data Assimilation Research Testbed Tutorial Section 2: How should observations of a state variable impact an unobserved state variable? Multivariate assimilation. Single observed variable, single unobserved
More informationDART_LAB Tutorial Section 5: Adaptive Inflation
DART_LAB Tutorial Section 5: Adaptive Inflation UCAR 14 The National Center for Atmospheric Research is sponsored by the National Science Foundation. Any opinions, findings and conclusions or recommendations
More informationData Assimilation Research Testbed Tutorial. Section 7: Some Additional Low-Order Models
Data Assimilation Research Testbed Tutorial Section 7: Some Additional Low-Order Models Version.: September, 6 /home/jla/dart_tutorial/dart/tutorial/section7/tut_section7.fm 1 7/13/7 Low-order models in
More informationDART Tutorial Section 18: Lost in Phase Space: The Challenge of Not Knowing the Truth.
DART Tutorial Section 18: Lost in Phase Space: The Challenge of Not Knowing the Truth. UCAR The National Center for Atmospheric Research is sponsored by the National Science Foundation. Any opinions, findings
More informationAdaptive Inflation for Ensemble Assimilation
Adaptive Inflation for Ensemble Assimilation Jeffrey Anderson IMAGe Data Assimilation Research Section (DAReS) Thanks to Ryan Torn, Nancy Collins, Tim Hoar, Hui Liu, Kevin Raeder, Xuguang Wang Anderson:
More informationDART Tutorial Sec'on 9: More on Dealing with Error: Infla'on
DART Tutorial Sec'on 9: More on Dealing with Error: Infla'on UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons
More informationRelative Merits of 4D-Var and Ensemble Kalman Filter
Relative Merits of 4D-Var and Ensemble Kalman Filter Andrew Lorenc Met Office, Exeter International summer school on Atmospheric and Oceanic Sciences (ISSAOS) "Atmospheric Data Assimilation". August 29
More informationDART Tutorial Sec'on 5: Comprehensive Filtering Theory: Non-Iden'ty Observa'ons and the Joint Phase Space
DART Tutorial Sec'on 5: Comprehensive Filtering Theory: Non-Iden'ty Observa'ons and the Joint Phase Space UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on.
More informationDART_LAB Tutorial Section 2: How should observations impact an unobserved state variable? Multivariate assimilation.
DART_LAB Tutorial Section 2: How should observations impact an unobserved state variable? Multivariate assimilation. UCAR 2014 The National Center for Atmospheric Research is sponsored by the National
More informationA data-driven method for improving the correlation estimation in serial ensemble Kalman filter
A data-driven method for improving the correlation estimation in serial ensemble Kalman filter Michèle De La Chevrotière, 1 John Harlim 2 1 Department of Mathematics, Penn State University, 2 Department
More informationIntroduction to Ensemble Kalman Filters and the Data Assimilation Research Testbed
Introduction to Ensemble Kalman Filters and the Data Assimilation Research Testbed Jeffrey Anderson, Tim Hoar, Nancy Collins NCAR Institute for Math Applied to Geophysics pg 1 What is Data Assimilation?
More informationDART Tutorial Sec'on 1: Filtering For a One Variable System
DART Tutorial Sec'on 1: Filtering For a One Variable System UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons
More informationImproved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter
Improved analyses and forecasts with AIRS retrievals using the Local Ensemble Transform Kalman Filter Hong Li, Junjie Liu, and Elana Fertig E. Kalnay I. Szunyogh, E. J. Kostelich Weather and Chaos Group
More informationData Assimilation: Finding the Initial Conditions in Large Dynamical Systems. Eric Kostelich Data Mining Seminar, Feb. 6, 2006
Data Assimilation: Finding the Initial Conditions in Large Dynamical Systems Eric Kostelich Data Mining Seminar, Feb. 6, 2006 kostelich@asu.edu Co-Workers Istvan Szunyogh, Gyorgyi Gyarmati, Ed Ott, Brian
More informationReview of Covariance Localization in Ensemble Filters
NOAA Earth System Research Laboratory Review of Covariance Localization in Ensemble Filters Tom Hamill NOAA Earth System Research Lab, Boulder, CO tom.hamill@noaa.gov Canonical ensemble Kalman filter update
More informationGaussian Filtering Strategies for Nonlinear Systems
Gaussian Filtering Strategies for Nonlinear Systems Canonical Nonlinear Filtering Problem ~u m+1 = ~ f (~u m )+~ m+1 ~v m+1 = ~g(~u m+1 )+~ o m+1 I ~ f and ~g are nonlinear & deterministic I Noise/Errors
More informationAccepted in Tellus A 2 October, *Correspondence
1 An Adaptive Covariance Inflation Error Correction Algorithm for Ensemble Filters Jeffrey L. Anderson * NCAR Data Assimilation Research Section P.O. Box 3000 Boulder, CO 80307-3000 USA Accepted in Tellus
More informationBrian J. Etherton University of North Carolina
Brian J. Etherton University of North Carolina The next 90 minutes of your life Data Assimilation Introit Different methodologies Barnes Analysis in IDV NWP Error Sources 1. Intrinsic Predictability Limitations
More informationDART Ini)al Condi)ons for a Refined Grid CAM- SE Forecast of Hurricane Katrina. Kevin Raeder (IMAGe) Colin Zarzycki (ASP)
DART Ini)al Condi)ons for a Refined Grid CAM- SE Forecast of Hurricane Katrina Kevin Raeder (IMAGe) Colin Zarzycki (ASP) 1 Mo)va)on Thousands of processors on current supercomputers. - > new CAM dynamical
More informationKalman Filter and Ensemble Kalman Filter
Kalman Filter and Ensemble Kalman Filter 1 Motivation Ensemble forecasting : Provides flow-dependent estimate of uncertainty of the forecast. Data assimilation : requires information about uncertainty
More informationA Note on the Particle Filter with Posterior Gaussian Resampling
Tellus (6), 8A, 46 46 Copyright C Blackwell Munksgaard, 6 Printed in Singapore. All rights reserved TELLUS A Note on the Particle Filter with Posterior Gaussian Resampling By X. XIONG 1,I.M.NAVON 1,2 and
More informationEnsemble Kalman Filters for WRF-ARW. Chris Snyder MMM and IMAGe National Center for Atmospheric Research
Ensemble Kalman Filters for WRF-ARW Chris Snyder MMM and IMAGe National Center for Atmospheric Research Preliminaries Notation: x = modelʼs state w.r.t. some discrete basis, e.g. grid-pt values y = Hx
More informationRepresentation of inhomogeneous, non-separable covariances by sparse wavelet-transformed matrices
Representation of inhomogeneous, non-separable covariances by sparse wavelet-transformed matrices Andreas Rhodin, Harald Anlauf German Weather Service (DWD) Workshop on Flow-dependent aspects of data assimilation,
More informationFundamentals of Data Assimilation
National Center for Atmospheric Research, Boulder, CO USA GSI Data Assimilation Tutorial - June 28-30, 2010 Acknowledgments and References WRFDA Overview (WRF Tutorial Lectures, H. Huang and D. Barker)
More informationModel error and parameter estimation
Model error and parameter estimation Chiara Piccolo and Mike Cullen ECMWF Annual Seminar, 11 September 2018 Summary The application of interest is atmospheric data assimilation focus on EDA; A good ensemble
More information(Toward) Scale-dependent weighting and localization for the NCEP GFS hybrid 4DEnVar Scheme
(Toward) Scale-dependent weighting and localization for the NCEP GFS hybrid 4DEnVar Scheme Daryl Kleist 1, Kayo Ide 1, Rahul Mahajan 2, Deng-Shun Chen 3 1 University of Maryland - Dept. of Atmospheric
More informationDART Tutorial Part IV: Other Updates for an Observed Variable
DART Tutorial Part IV: Other Updates for an Observed Variable UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, findings and conclusions or recommenda'ons
More informationGSI Tutorial Background and Observation Errors: Estimation and Tuning. Daryl Kleist NCEP/EMC June 2011 GSI Tutorial
GSI Tutorial 2011 Background and Observation Errors: Estimation and Tuning Daryl Kleist NCEP/EMC 29-30 June 2011 GSI Tutorial 1 Background Errors 1. Background error covariance 2. Multivariate relationships
More informationShort tutorial on data assimilation
Mitglied der Helmholtz-Gemeinschaft Short tutorial on data assimilation 23 June 2015 Wolfgang Kurtz & Harrie-Jan Hendricks Franssen Institute of Bio- and Geosciences IBG-3 (Agrosphere), Forschungszentrum
More informationBackground Error, Observation Error, and GSI Hybrid Analysis
2015 GSI Community Tutorial August 11-13, 2013, NCAR, Boulder Background Error, Observation Error, and GSI Hybrid Analysis Ming Hu Developmental Testbed Center Outlines GSI fundamentals (1): Setup and
More informationIntroduction to GSI Background Error Covariance (BE)
23 Beijing GSI Tutorial May 29, 23 Beijing, China Introduction to GSI Background Error Covariance (BE) Ming Hu Developmental Testbed Center NCAR-NOAA/GSD BE related Tutorial lectures GSI Tutorial 2 Background
More informationSome Applications of WRF/DART
Some Applications of WRF/DART Chris Snyder, National Center for Atmospheric Research Mesoscale and Microscale Meteorology Division (MMM), and Institue for Mathematics Applied to Geoscience (IMAGe) WRF/DART
More informationThunderstorm-Scale EnKF Analyses Verified with Dual-Polarization, Dual-Doppler Radar Data
Thunderstorm-Scale EnKF Analyses Verified with Dual-Polarization, Dual-Doppler Radar Data David Dowell and Wiebke Deierling National Center for Atmospheric Research, Boulder, CO Ensemble Data Assimilation
More informationHybrid variational-ensemble data assimilation. Daryl T. Kleist. Kayo Ide, Dave Parrish, John Derber, Jeff Whitaker
Hybrid variational-ensemble data assimilation Daryl T. Kleist Kayo Ide, Dave Parrish, John Derber, Jeff Whitaker Weather and Chaos Group Meeting 07 March 20 Variational Data Assimilation J Var J 2 2 T
More informationObjective localization of ensemble covariances: theory and applications
Institutionnel Grand Public Objective localization of ensemble covariances: theory and applications Yann Michel1, B. Me ne trier2 and T. Montmerle1 Professionnel (1) Me te o-france & CNRS, Toulouse, France
More informationJuli I. Rubin. NRC Postdoctoral Research Associate Naval Research Laboratory, Monterey, CA
Development of the Ensemble Navy Aerosol Analysis Prediction System and its application of the Data Assimilation Research Testbed in Support of Aerosol Forecasting Juli I. Rubin NRC Postdoctoral Research
More informationForecasting and data assimilation
Supported by the National Science Foundation DMS Forecasting and data assimilation Outline Numerical models Kalman Filter Ensembles Douglas Nychka, Thomas Bengtsson, Chris Snyder Geophysical Statistics
More informationReport on the Joint SRNWP workshop on DA-EPS Bologna, March. Nils Gustafsson Alex Deckmyn.
Report on the Joint SRNWP workshop on DA-EPS Bologna, 22-24 March Nils Gustafsson Alex Deckmyn http://www.smr.arpa.emr.it/srnwp/ Purpose of the workshop On the one hand, data assimilation techniques require
More informationGSI Tutorial Background and Observation Error Estimation and Tuning. 8/6/2013 Wan-Shu Wu 1
GSI Tutorial 2013 Background and Observation Error Estimation and Tuning 8/6/2013 Wan-Shu Wu 1 Analysis system produces an analysis through the minimization of an objective function given by J = x T B
More informationMarkov chain Monte Carlo methods in atmospheric remote sensing
1 / 45 Markov chain Monte Carlo methods in atmospheric remote sensing Johanna Tamminen johanna.tamminen@fmi.fi ESA Summer School on Earth System Monitoring and Modeling July 3 Aug 11, 212, Frascati July,
More informationNumerical Weather Prediction: Data assimilation. Steven Cavallo
Numerical Weather Prediction: Data assimilation Steven Cavallo Data assimilation (DA) is the process estimating the true state of a system given observations of the system and a background estimate. Observations
More informationF denotes cumulative density. denotes probability density function; (.)
BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models
More informationA Non-Gaussian Ensemble Filter Update for Data Assimilation
4186 M O N T H L Y W E A T H E R R E V I E W VOLUME 138 A Non-Gaussian Ensemble Filter Update for Data Assimilation JEFFREY L. ANDERSON NCAR Data Assimilation Research Section,* Boulder, Colorado (Manuscript
More informationParticle Filters. Outline
Particle Filters M. Sami Fadali Professor of EE University of Nevada Outline Monte Carlo integration. Particle filter. Importance sampling. Degeneracy Resampling Example. 1 2 Monte Carlo Integration Numerical
More informationThe University of Texas at Austin, Jackson School of Geosciences, Austin, Texas 2. The National Center for Atmospheric Research, Boulder, Colorado 3
Assimilation of MODIS Snow Cover and GRACE Terrestrial Water Storage Data through DART/CLM4 Yong-Fei Zhang 1, Zong-Liang Yang 1, Tim J. Hoar 2, Hua Su 1, Jeffrey L. Anderson 2, Ally M. Toure 3,4, and Matthew
More informationP 1.86 A COMPARISON OF THE HYBRID ENSEMBLE TRANSFORM KALMAN FILTER (ETKF)- 3DVAR AND THE PURE ENSEMBLE SQUARE ROOT FILTER (EnSRF) ANALYSIS SCHEMES
P 1.86 A COMPARISON OF THE HYBRID ENSEMBLE TRANSFORM KALMAN FILTER (ETKF)- 3DVAR AND THE PURE ENSEMBLE SQUARE ROOT FILTER (EnSRF) ANALYSIS SCHEMES Xuguang Wang*, Thomas M. Hamill, Jeffrey S. Whitaker NOAA/CIRES
More informationComparison of 3D-Var and LETKF in an Atmospheric GCM: SPEEDY
Comparison of 3D-Var and LEKF in an Atmospheric GCM: SPEEDY Catherine Sabol Kayo Ide Eugenia Kalnay, akemasa Miyoshi Weather Chaos, UMD 9 April 2012 Outline SPEEDY Formulation Single Observation Eperiments
More informationToward improved initial conditions for NCAR s real-time convection-allowing ensemble. Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell
Toward improved initial conditions for NCAR s real-time convection-allowing ensemble Ryan Sobash, Glen Romine, Craig Schwartz, and Kate Fossell Storm-scale ensemble design Can an EnKF be used to initialize
More informationAspects of the practical application of ensemble-based Kalman filters
Aspects of the practical application of ensemble-based Kalman filters Lars Nerger Alfred Wegener Institute for Polar and Marine Research Bremerhaven, Germany and Bremen Supercomputing Competence Center
More information5.3 TESTING AND EVALUATION OF THE GSI DATA ASSIMILATION SYSTEM
5.3 TESTING AND EVALUATION OF THE GSI DATA ASSIMILATION SYSTEM Kathryn M Newman*, C. Zhou, H. Shao, X.Y. Huang, M. Hu National Center for Atmospheric Research, Boulder, CO Developmental Testbed Center
More informationBayesian Inverse problem, Data assimilation and Localization
Bayesian Inverse problem, Data assimilation and Localization Xin T Tong National University of Singapore ICIP, Singapore 2018 X.Tong Localization 1 / 37 Content What is Bayesian inverse problem? What is
More informationAdvances and Challenges in Ensemblebased Data Assimilation in Meteorology. Takemasa Miyoshi
January 18, 2013, DA Workshop, Tachikawa, Japan Advances and Challenges in Ensemblebased Data Assimilation in Meteorology Takemasa Miyoshi RIKEN Advanced Institute for Computational Science Takemasa.Miyoshi@riken.jp
More informationA Hybrid ETKF 3DVAR Data Assimilation Scheme for the WRF Model. Part I: Observing System Simulation Experiment
5116 M O N T H L Y W E A T H E R R E V I E W VOLUME 136 A Hybrid ETKF 3DVAR Data Assimilation Scheme for the WRF Model. Part I: Observing System Simulation Experiment XUGUANG WANG Cooperative Institute
More informationEstimation of Surface Fluxes of Carbon, Heat, Moisture and Momentum from Atmospheric Data Assimilation
AICS Data Assimilation Workshop February 27, 2013 Estimation of Surface Fluxes of Carbon, Heat, Moisture and Momentum from Atmospheric Data Assimilation Ji-Sun Kang (KIAPS), Eugenia Kalnay (Univ. of Maryland,
More informationImproving GFS 4DEnVar Hybrid Data Assimilation System Using Time-lagged Ensembles
Improving GFS 4DEnVar Hybrid Data Assimilation System Using Time-lagged Ensembles Bo Huang and Xuguang Wang School of Meteorology University of Oklahoma, Norman, OK, USA Acknowledgement: Junkyung Kay (OU);
More informationSatellite Observations of Greenhouse Gases
Satellite Observations of Greenhouse Gases Richard Engelen European Centre for Medium-Range Weather Forecasts Outline Introduction Data assimilation vs. retrievals 4D-Var data assimilation Observations
More informationThe hybrid ETKF- Variational data assimilation scheme in HIRLAM
The hybrid ETKF- Variational data assimilation scheme in HIRLAM (current status, problems and further developments) The Hungarian Meteorological Service, Budapest, 24.01.2011 Nils Gustafsson, Jelena Bojarova
More informationEnsemble Data Assimila.on for Climate System Component Models
Ensemble Data Assimila.on for Climate System Component Models Jeffrey Anderson Na.onal Center for Atmospheric Research In collabora.on with: Alicia Karspeck, Kevin Raeder, Tim Hoar, Nancy Collins IMA 11
More informationEnsemble Data Assimila.on and Uncertainty Quan.fica.on
Ensemble Data Assimila.on and Uncertainty Quan.fica.on Jeffrey Anderson, Alicia Karspeck, Tim Hoar, Nancy Collins, Kevin Raeder, Steve Yeager Na.onal Center for Atmospheric Research Ocean Sciences Mee.ng
More informationRosemary Munro*, Graeme Kelly, Michael Rohn* and Roger Saunders
ASSIMILATION OF METEOSAT RADIANCE DATA WITHIN THE 4DVAR SYSTEM AT ECMWF Rosemary Munro*, Graeme Kelly, Michael Rohn* and Roger Saunders European Centre for Medium Range Weather Forecasts Shinfield Park,
More informationSignal Modeling, Statistical Inference and Data Mining in Astrophysics
ASTRONOMY 6523 Spring 2013 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Course Approach The philosophy of the course reflects that of the instructor, who takes a dualistic view
More informationBoundary Conditions for Limited-Area Ensemble Kalman Filters
2490 M O N T H L Y W E A T H E R R E V I E W VOLUME 134 Boundary Conditions for Limited-Area Ensemble Kalman Filters RYAN D. TORN AND GREGORY J. HAKIM University of Washington, Seattle, Washington CHRIS
More informationError (E) Covariance Diagnostics. in Ensemble Data Assimilation. Kayo Ide, Daryl Kleist, Adam Kubaryk University of Maryland
Error (E) Covariance Diagnostics in Ensemble Data Assimilation Kayo Ide, Daryl Kleist, Adam Kubaryk University of Maryland 7 th EnKF Data Assimilation Workshop May 2016 Outline Part I. Background: E-diagnostics
More informationComparisons between 4DEnVar and 4DVar on the Met Office global model
Comparisons between 4DEnVar and 4DVar on the Met Office global model David Fairbairn University of Surrey/Met Office 26 th June 2013 Joint project by David Fairbairn, Stephen Pring, Andrew Lorenc, Neill
More informationJ5.8 ESTIMATES OF BOUNDARY LAYER PROFILES BY MEANS OF ENSEMBLE-FILTER ASSIMILATION OF NEAR SURFACE OBSERVATIONS IN A PARAMETERIZED PBL
J5.8 ESTIMATES OF BOUNDARY LAYER PROFILES BY MEANS OF ENSEMBLE-FILTER ASSIMILATION OF NEAR SURFACE OBSERVATIONS IN A PARAMETERIZED PBL Dorita Rostkier-Edelstein 1 and Joshua P. Hacker The National Center
More informationUpdates to Land DA at the Met Office
Updates to Land DA at the Met Office Brett Candy & Keir Bovis Satellite Soil Moisture Workshop, Amsterdam, 10th July 2014 Land DA - The Story So Far - Kalman Filter 18 months in operations - Analyses of
More informationEvaluation of a non-local observation operator in assimilation of. CHAMP radio occultation refractivity with WRF
Evaluation of a non-local observation operator in assimilation of CHAMP radio occultation refractivity with WRF Hui Liu, Jeffrey Anderson, Ying-Hwa Kuo, Chris Snyder, and Alain Caya National Center for
More informationTangent-linear and adjoint models in data assimilation
Tangent-linear and adjoint models in data assimilation Marta Janisková and Philippe Lopez ECMWF Thanks to: F. Váňa, M.Fielding 2018 Annual Seminar: Earth system assimilation 10-13 September 2018 Tangent-linear
More informationEnhancing information transfer from observations to unobserved state variables for mesoscale radar data assimilation
Enhancing information transfer from observations to unobserved state variables for mesoscale radar data assimilation Weiguang Chang and Isztar Zawadzki Department of Atmospheric and Oceanic Sciences Faculty
More informationThe Australian Operational Daily Rain Gauge Analysis
The Australian Operational Daily Rain Gauge Analysis Beth Ebert and Gary Weymouth Bureau of Meteorology Research Centre, Melbourne, Australia e.ebert@bom.gov.au Daily rainfall data and analysis procedure
More informationSatellite Retrieval Assimilation (a modified observation operator)
Satellite Retrieval Assimilation (a modified observation operator) David D. Kuhl With Istvan Szunyogh and Brad Pierce Sept. 21 2009 Weather Chaos Group Meeting Overview Introduction Derivation of retrieval
More informationJi-Sun Kang. Pr. Eugenia Kalnay (Chair/Advisor) Pr. Ning Zeng (Co-Chair) Pr. Brian Hunt (Dean s representative) Pr. Kayo Ide Pr.
Carbon Cycle Data Assimilation Using a Coupled Atmosphere-Vegetation Model and the LETKF Ji-Sun Kang Committee in charge: Pr. Eugenia Kalnay (Chair/Advisor) Pr. Ning Zeng (Co-Chair) Pr. Brian Hunt (Dean
More informationAssimilation of cloud/precipitation data at regional scales
Assimilation of cloud/precipitation data at regional scales Thomas Auligné National Center for Atmospheric Research auligne@ucar.edu Acknowledgments to: Steven Cavallo, David Dowell, Aimé Fournier, Hans
More informationGMAO Analyses and Forecasts for YOTC
GMAO Analyses and Forecasts for YOTC Michele Rienecker and Max Suarez Arlindo da Silva Global Modeling and Assimilation Office NASA/Goddard Space Flight Center YOTC Implementation Planning Meeting Hawaii,
More informationInter-comparison of 4D-Var and EnKF systems for operational deterministic NWP
Inter-comparison of 4D-Var and EnKF systems for operational deterministic NWP Project eam: Mark Buehner Cecilien Charette Bin He Peter Houtekamer Herschel Mitchell WWRP/HORPEX Workshop on 4D-VAR and Ensemble
More informationData Assimilation Development for the FV3GFSv2
Data Assimilation Development for the FV3GFSv2 Catherine Thomas 1, 2, Rahul Mahajan 1, 2, Daryl Kleist 2, Emily Liu 3,2, Yanqiu Zhu 1, 2, John Derber 2, Andrew Collard 1, 2, Russ Treadon 2, Jeff Whitaker
More information4DEnVar. Four-Dimensional Ensemble-Variational Data Assimilation. Colloque National sur l'assimilation de données
Four-Dimensional Ensemble-Variational Data Assimilation 4DEnVar Colloque National sur l'assimilation de données Andrew Lorenc, Toulouse France. 1-3 décembre 2014 Crown copyright Met Office 4DEnVar: Topics
More informationIntroduction to data assimilation and least squares methods
Introduction to data assimilation and least squares methods Eugenia Kalnay and many friends University of Maryland October 008 (part 1 Contents (1 Forecasting the weather - we are really getting better!
More informationIntroduction to Data Assimilation. Saroja Polavarapu Meteorological Service of Canada University of Toronto
Introduction to Data Assimilation Saroja Polavarapu Meteorological Service of Canada University of Toronto GCC Summer School, Banff. May 22-28, 2004 Outline of lectures General idea Numerical weather prediction
More informationAssimilation in the PBL
Assimilation in the PBL Joshua Hacker hacker@ucar.edu National Center for Atmospheric Research, Research Applications Program Data Assimilation Initiative review, Sept 2004 p.1/17 Outline DAI in my world
More informationEnsemble Kalman Filter
Ensemble Kalman Filter Geir Evensen and Laurent Bertino Hydro Research Centre, Bergen, Norway, Nansen Environmental and Remote Sensing Center, Bergen, Norway The Ensemble Kalman Filter (EnKF) Represents
More informationGenerating climatological forecast error covariance for Variational DAs with ensemble perturbations: comparison with the NMC method
Generating climatological forecast error covariance for Variational DAs with ensemble perturbations: comparison with the NMC method Matthew Wespetal Advisor: Dr. Eugenia Kalnay UMD, AOSC Department March
More informationA new Hierarchical Bayes approach to ensemble-variational data assimilation
A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander
More informationA variance limiting Kalman filter for data assimilation: I. Sparse observational grids II. Model error
A variance limiting Kalman filter for data assimilation: I. Sparse observational grids II. Model error Georg Gottwald, Lewis Mitchell, Sebastian Reich* University of Sydney, *Universität Potsdam Durham,
More informationThe Ensemble Kalman Filter:
p.1 The Ensemble Kalman Filter: Theoretical formulation and practical implementation Geir Evensen Norsk Hydro Research Centre, Bergen, Norway Based on Evensen 23, Ocean Dynamics, Vol 53, No 4 p.2 The Ensemble
More informationEnKF Localization Techniques and Balance
EnKF Localization Techniques and Balance Steven Greybush Eugenia Kalnay, Kayo Ide, Takemasa Miyoshi, and Brian Hunt Weather Chaos Meeting September 21, 2009 Data Assimilation Equation Scalar form: x a
More informationThe Local Ensemble Transform Kalman Filter (LETKF) Eric Kostelich. Main topics
The Local Ensemble Transform Kalman Filter (LETKF) Eric Kostelich Arizona State University Co-workers: Istvan Szunyogh, Brian Hunt, Ed Ott, Eugenia Kalnay, Jim Yorke, and many others http://www.weatherchaos.umd.edu
More informationUsing DART Tools for CAM Development
Using DART Tools for CAM Development Kevin Raeder, For DART: Jeff Anderson, Tim Hoar, Nancy Collins, Johnny Hendricks CSEG: Alice Bertini, Mariana Vertenstein, Steve Goldhaber, Jim Edwards And: Nick Pedatella
More information(Statistical Forecasting: with NWP). Notes from Kalnay (2003), appendix C Postprocessing of Numerical Model Output to Obtain Station Weather Forecasts
35 (Statistical Forecasting: with NWP). Notes from Kalnay (2003), appendix C Postprocessing of Numerical Model Output to Obtain Station Weather Forecasts If the numerical model forecasts are skillful,
More informationUpdate on the KENDA project
Christoph Schraff Deutscher Wetterdienst, Offenbach, Germany and many colleagues from CH, D, I, ROM, RU Km-scale ENsemble-based Data Assimilation : COSMO priority project Local Ensemble Transform Kalman
More informationBackground error modelling: climatological flow-dependence
Background error modelling: climatological flow-dependence Yann MICHEL NCAR/MMM/B Meeting 16 th April 2009 1 Introduction 2 A new estimate of lengthscales 3 Climatological flow-dependence Yann MICHEL B
More informationPerformance of ensemble Kalman filters with small ensembles
Performance of ensemble Kalman filters with small ensembles Xin T Tong Joint work with Andrew J. Majda National University of Singapore Sunday 28 th May, 2017 X.Tong EnKF performance 1 / 31 Content Ensemble
More informationCan hybrid-4denvar match hybrid-4dvar?
Comparing ensemble-variational assimilation methods for NWP: Can hybrid-4denvar match hybrid-4dvar? WWOSC, Montreal, August 2014. Andrew Lorenc, Neill Bowler, Adam Clayton, David Fairbairn and Stephen
More informationRelationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF
Relationship between Singular Vectors, Bred Vectors, 4D-Var and EnKF Eugenia Kalnay and Shu-Chih Yang with Alberto Carrasi, Matteo Corazza and Takemasa Miyoshi 4th EnKF Workshop, April 2010 Relationship
More informationEvaluating Methods to Account for System Errors in Ensemble Data Assimilation
3078 M O N T H L Y W E A T H E R R E V I E W VOLUME 140 Evaluating Methods to Account for System Errors in Ensemble Data Assimilation JEFFREY S. WHITAKER AND THOMAS M. HAMILL NOAA/Earth System Research
More informationData assimilation in high dimensions
Data assimilation in high dimensions David Kelly Courant Institute New York University New York NY www.dtbkelly.com February 12, 2015 Graduate seminar, CIMS David Kelly (CIMS) Data assimilation February
More information