Hybrid variational-ensemble data assimilation Daryl T. Kleist Kayo Ide, Dave Parrish, John Derber, Jeff Whitaker Weather and Chaos Group Meeting 07 March 20
Variational Data Assimilation J Var J 2 2 T T B Var y o H R y o H c J : Penalty (Fit to background + Fit to observations + Constraints) : Analysis increment ( a b ) ; where b is a background B Var : Background error covariance H : Observations (forward) operator R : Observation error covariance (Instrument + representativeness) y o : Observation innovations J c : Constraints (physical quantities, balance/noise, etc.) B is typically static and estimated a-priori/offline 2
Motivation from Var Current background error covariance (applied operationally) in VAR Isotropic recursive filters Poor handle on cross-variable covariance Minimal flow-dependence added Implicit flow-dependence through linearization in normal mode constraint (Kleist et al. 2009) Flow-dependent variances (only for wind, temperature, and pressure) based on background tendencies Tuned NMC-based estimate (lagged forecast pairs) 3
Kalman Filter in Var Setting Forecast Step Analysis K B KF a A B KF b b KF M MA K H T KF a M H I KH T b y T RHB H KF B KF Q Etended Kalman Filter Analysis step in variational framework (cost function) J KF 2 T T B y H R y H KF B KF : Time evolving background error covariance A KF : Inverse [Hessian of J KF ( )] 2 o o 4
Motivation from KF Problem: dimensions of A KF and B KF are huge, making this practically impossible for large systems (GFS for eample). Solution: sample and update using an ensemble instead of evolving A KF /B KF eplicitly Forecast Step: Analysis Step: a b X X b a X X B A KF KF X K X K b a X X b T a T Ensemble Perturbations 5
Why Hybrid? Benefit from use of flow dependent ensemble covariance instead of static B VAR (3D, 4D) EnKF Hybrid References Hamill and Snyder 2000; Wang et al. 2007b,2008ab, 2009b, Wang 20; Buehner et al. 200ab Robust for small ensemble Wang et al. 2007b, 2009b; Buehner et al. 200b Better localization for integrated measure, e.g. satellite radiance Easy framework to add various constraints Framework to treat non- Gaussianity Use of various eisting capabilities in VAR Campbell et al. 2009 6
7 Hybrid Variational-Ensemble Incorporate ensemble perturbations directly into variational cost function through etended control variable Lorenc (2003), Buehner (2005), Wang et. al. (2007), etc. e f t o T t o T e f T f f f 2 2 2 H y R H y L B, J K k k k e f t f & e : weighting coefficients for fied and ensemble covariance respectively t : (total increment) sum of increment from fied/static B ( f ) and ensemble B k : etended control variable; :ensemble perturbation L: correlation matri [localization on ensemble perturbations] e k
Importance of Ensemble Generation Method? GEFS (already operational) 80 cycled members ETR Virtually no computational cost Uses analysis error mask derived for 500 mb streamfunction Tuned for medium range forecast spread and fast error growth T90L28 version of the GFS model Viable for hybrid paradigm? EnKF 80 cycled members Perturbations specifically designed to represent analysis and background errors T254L64 version of the GFS Etra computational costs worth it for hybrid? 8
EnKF/ETR Comparison EnKF (green) versus ETR (red) spread/standard deviation for surface pressure (mb) valid 200032 Surface Pressure spread normalized difference (ETR has much less spread, ecept poleward of 70N) 9
EnKF/ETR Comparison EnKF zonal wind (m/s) ensemble standard deviation valid 200032 ETR zonal wind (m/s) ensemble standard deviation valid 200032 0
EnKF/NMC B Compare EnKF zonal wind (m/s) ensemble standard deviation valid 200032 Zonal Wind standard deviation (m/s) from NMC-method
Hybrid with (global) GSI Control variable has been implemented into GSI 3DVAR* Full B preconditioning Working on etensions to B /2 preconditioned minimization options Spectral filter for horizontal part of A Eventually replace with (anisotropic) recursive filters Recursive filter used for vertical Dual resolution capability Ensemble can be from different resolution than background/analysis (vertical levels are the eception) Various localization options for A Grid units or scale height Level dependent (plans to epand) Option to apply TLNMC (Kleist et al. 2009) to analysis increment C f K k k e k *Acknowledgement: Dave Parrish for original implementation of etended control variable 2
Single Observation Single 850mb Tv observation (K O-F, K error) 3
Single Observation Single 850mb zonal wind observation (3 m/s O-F, m/s error) in Hurricane Ike circulation 4
Dual-Res Coupled Hybrid member forecast member 2 forecast member 3 forecast EnKF member update recenter analysis ensemble member analysis member 2 analysis member 3 analysis high res forecast GSI Hybrid Ens/Var high res analysis Previous Cycle Current Update Cycle
Hybrid Var-EnKF GFS eperiment Model GFS deterministic (T574L64; post July 200 version current operational version) GFS ensemble (T254L64) 80 ensemble members, EnKF update, GSI for observation operators Observations All operationally available observations (including radiances) Includes early (GFS) and late (GDAS/cycled) cycles as in production Dual-resolution/Coupled High resolution control/deterministic component Includes TC Relocation on guess Ensemble is recentered every cycle about hybrid analysis Discard ensemble mean analysis Satellite bias corrections Coefficients come from GSI/VAR Parameter settings /3 static B, 2/3 ensemble Fied localization: 800km &.5 scale heights Test Period 5 July 200 5 October 200 (first two weeks ignored for spin-up ) 6
500 hpa Anom.Corr. Northern Hemisphere Southern Hemisphere 7
AC Frequency Distributions Northern Hemisphere Southern Hemisphere 8
Geopotential Height RMSE Northern Hemisphere Southern Hemisphere Significant reduction in mean height errors 9
Stratospheric Fits Improved fits to stratospheric observations 20
Forecast Fits to Obs (Tropical Winds) Forecasts from hybrid analyses fit observation much better. 2
22
23
24
HVEDAS (3D) for GDAS/GFS Prototype dual-resolution, two-way coupled hybrid Var/EnKF system outperforms standard 3DVAR in GFS eperiments 200 Hurricane Season (August 5 through October 3 200) run offsite Emphasis on AC, RMSE, TC Tracks Plan underway to implement into GDAS/GFS operationally Target: Spring 202 (subject to many potential issues) Porting of codes/scripts back to IBM P6 Cost analysis (will everything fit in production suite?) More thorough (pre-implementation) testing and evaluation More test periods (including NH winter) Other/more verification metrics Potential moratorium associated with move to new NCEP facility Issues Weighting between ensemble and static B Localization How should EnKF be used within ensemble forecasting paradigm? 25
HVEDAS Etensions and Improvements Epand hybrid to 4D Hybrid within traditional 4DVAR (with adjoint) Pure ensemble 4DVAR (non-adjoint) Ensemble 4DVAR with static B supplement (non-adjoint)* EnKF improvements Eplore alternatives such as LETKF Adaptive localization and inflation Non-GFS applications in development NASA GEOS-5 (GMAO) NAM (Dave Parrish, others) Hurricanes/HWRF (Mingjing Tong, HFIP, many collaborators) Storm-scale initialization (Jacob Carley, collaborators) RR (Xuguang Wang, Ming Xue, Stan Benjamin, Jeff Whitaker, Steve Weygandt, others) NCEP strives to have single DA system to develop, maintain, and run operationally (global, mesoscale, severe weather, hurricanes, etc.) GSI (including hybrid development) is community code supported through DTC EnKF used for GFS-based hybrid being epanded for use with other applications 26