Data assimilation in mesoscale modeling and numerical weather prediction Nils Gustafsson Croatian USA Workshop on Mesometeorology June 2012
Perspective: What are the important issues for development of operational data assimilation when we are in the move: From hydrostatic models with a grid resolution of 5-20 km (HIRLAM) To non-hydrostatic models with a grid resolution of 1-3 km (HARMONIE) Taking also the needs for probabilistic forecasting into account?? + work in a wider community (HIRLAM, ALADIN, Meteo-France + use of the IFS-code (ECMWF)
Outline of talk Relative importance of different model variables in the initial conditions Candidate assimilation methods Host model coupling issues Moisture assimilation Radar data assimilation Initialization Surface assimilation
Importance of different model variables in the initial conditions convection permitting scales OUTCRY: There are too many types of data, too many new ones appearing every year, and there are too few people to work on their assimilation! Could there be a way to prioritize data assimilation efforts?
Fabry & Sun: Questions (for mesoscale foreasting of convection in this study): What is the relative importance of different model variables in the initial conditions? What is the time-scale for how perturbations in one model variable influence other model variables? (Tadjust) What is the time-scale for non-linearities to have important effects for different model variables? (Tnl) What is the signal strength of different instruments measuring different variables? Important: Tadjust < assimilation window < Tnl
Fabry & Sun: Approach and tools Identical twin model simulation experiments Perturb initial data of different variables, check how perturbations influence other variables WRF, 4 km grid resolution Use different amplitudes of initial perturbations, check for how long time linearity holds
Fabry & Sun: Linear, non-linear and contradictory regimes
Fabry & Sun: Rainfall accumulation from 16 control cases
Fabry & Sun: Horizontal spectra and variances of simulated initial errors
Experimentation strategy Fabry & Sun
Fabry & Sun: RESULTS error growth
Fabry & Sun: Error spread to different model variables 3 hours 15 minutes
Fabry & Sun: Non-linearity index Contradictory index
What can we learn from Fabry & Sun? Exept for low-level moisture and soil properties, most initial condition errors propagate quickly to other variables. Uncertainties in mid-level moisture caused the greatest uncertainties in the forecasts. For mesoscale forecasts longer than 6h, the uncertainties in all other variables had comparable effects. For forecasts of a few hours, uncertainties in temperatures, low-level moisture and mid-level winds stood out as the greatest cause of forecast uncertainties after mid-level moisture. There are limits on the assimilation interval that are different for different variables, in particular condensates have shorter predictability and can be used with shorter windows only. Remember: Identical twin experiment!!!
The assimilation method Mesoscale balance constraints, in particular involving moisture? Analytical or statistical constraint? Utilization of ensembles? How to handle inhomogeneities and an-isotropy? Flow-dependent structure functions? 4D-Var, ensemble techniques or synthesis? How to handle model errors?
Candidate data assimilation techniques for the mesoscale Variational techniques (3D-Var and 4D-Var) Ensemble Kalman Filters Hybrid variational ensemble methods Nudging (will not be discussed) Warping or alignment to correct for phase errors
Basic idea of 4D-Var: Fit model trajectory to observations over a time window
4D-Var Possibilities: Handle weak nonlinearities via outer loops Provide flowdependencies within assimilation window Problems: Strong non-linearities Static influence of observations at the start of the window Maintain tangent linear and adjoint codes Scalability on parallell computers
Comparison of HIRLAM 3D-Var and 4D-Var for the stormy month of December 1999 (Gustafsson et al. 2012)
The Danish storm, 3 December 1999 18 UTC 3D-Var +30h 3D-Var analysis 4D-Var +30 h Gustafsson et al. 2012) 4D-Var analysis
Why does 4D-Var make a better job than 3D-Var in storm cases? Improved implicit flow-dependent structure functions. Better quality control decisions due to a better background and better implicit structure functions Use of more observation? In general, it is difficult to point to a specific reason for improvements, the improvements generally occur gradually through the assimilation cycles
Assimilation increments 3D-Var 06UTC 4D-Var 06UTC 3D-Var 12UTC 4D-Var 12UTC
Surface pressure increments for the Danish mesoscale storm (Single simulated observation experiment) 3D-Var 4D-Var, spectral TL prop. of incr 4D-Var; gp model prop. of incr.
Mesoscale data assimilation and the use of ensembles Will assumptions on weak non-linearities at 10 km resolution break down at the km scale? Most likely yes! Can ensembles provide information on uncertainties and balances at the mesoscale? Most likely Yes! Is it too risky to put all our efforts on 4D-Var, that works on the 10 km scale but may fail at the km scale? Yes! Should we drop our investments in variational data assimilation and start with ensemble Kalman Filters or even particle filters? No, EnsKF and particle filters still have weaknesses! Can we develop hybrid data assimilation schemes that combine the best of the two worlds? We believe so!
Synoptic variation of covariances as calculated from 3 days of 12h minus 36 h forecast differences 2006011620060118 2006011620060118 500 hpa geopotential height Wavelet representation of 500 hpa temperature covariances From Landelius
Ensemble Kalman Filters Basic idea: Estimate forecast error covariances from an ensemble of forecasts Ensemble generation: Perturbed observations Model physics perturbations Model tendency perturbations Advantages: Flow-dependencies Natural together with EPS Problems: 50-100 members needed; Costly Low rank of covariance matrix (ensemble size) makes some type of localization necessary
From Whitaker
HIRLAM first approach to use ensembles in 3D-Var and 4D-Var Use the ETKF algorithm for re-scaling of a 6h forecast ensemble to an analysis ensemble (estimation of the analysis error covariance). Use ensemble of 6h forecasts to estimate the background error covariance and blend it with the static background error covariance.
First version of HIRLAM implementation Ensemble weights a have horizontal variations only (horizontal localization) and is controlled in spectral space with the assumption of isotropy this is equivalent to a horizontal localization of covariances with a Schur product based on a horizontal correlation function. The localization is applied for vorticity, divergence, temperature, surface pressure and specific humidity for better balancing.
Forecast verification scores for a first winter case (12 ensemble members only) 3D-Var (red curve) Hybrid (green curve) Hybrid + tuning of obs. error s.dev. (blue curve) Difficult to show the same improvement for the hybrid over 4D-Var!
Example of assimilation increments (temperature and wind) Model level around 800 hpa Notice increments along a front
(Landelius et al., 2011)
Landelius et al. 2011)
Host model coupling issues Large scales are poorly assimilated on a small regional domain one may wish to handle these large scales differently In case lateral boundary conditions are coming from an earlier host model run, one may wish to update also the inner domain when new LBC conditions are available One may also wish to control lateral boundary conditions in a mesoscale 4D-Var
Large scale error constraint in 3-4D-Var (from Dahlgren and Gustafsson, 2012) Add large-scale constraint to 4D-Var cost function: Large scales of vorticity are constrained by a short range host model forecast at the start of the 4D-Var assimilation window.
Impact of Jk on surface pressure forecast verification scores (Dahlgren and Gustafsson, 2012)
Control of lateral boundary conditions WHY? +0 h +6 h In case an object that we want to assimilate is outside the lateral boundaries at the start of the assimilation window, but described by observations inside the lateral boundaries later during the assimilation window, we must control it by lateral boundary conditions. Let it come in! Similarly observations may tell us to let objects go out even if this is not described by the host model LBCs.
Control of Lateral Boundary Conditions (1) Introduce the LBCs at the end of the data assimilation window as assimilation control variables (full model state = double size control vector) (2) Introduce the adjoints of the Davies LBC relaxation scheme and the time interpolation of the LBCs (3) Introduce a smoothing and balancing constraint for the LBCs into the cost function to be minimized J = Jb + Jo + Jc + Jlbc where Jlbc = (Xlbc- (Xlbc)b)T B-1 (Xlbc- (Xlbc)b) And, as a first approximation, B is taken as identical to B for the background constraint
Single simulated observation impact study With control of lbc Simulated observation SW 9 m/s at 32N 12W 3Dec 1999 11 UTC 3 December 1999 Assimilation window 06UTC 11UTC Strong SW inflow in the background field (Gustafsson, 2012) 06 UTC 09 UTC 11 UTC
Specific problems in assimilation of moisture Non-Gaussian behavour of water vapour close to zero humidity and saturation Flow-dependent structures, for example in rain and no-rain regimes Sharp vertical gradients (inversions, PBL top) Handling of model errors (biases)
Effects of a model dry bias (700 hpa RH)
A new moisture control variable (Elias Holm) Pseudo-relative humidity: Normaliztion with a standard-deviation depending also on the increment (non-linear transform):
Effect on PDF of new moisture control variable
Standard deviation of relative humidity forecast differences as a function of background relative humidity
Effect on assimilation increments near saturation Q control variable RH* control variable RH* control variable + relinearization BUT, no effect on forecast verification scores!
Simulated observation experiments Shown to be very important for assimilation of radar data in a mesoscale model (AROME)! From Montmerle & Berre
Smoothing effect of climatological vertical correlations Single observation experiment with HIRLAM 4D-Var Cloud-affected SEVIRI IR data (water vaour channel) Observation operator includes diagnosis of clouds (ECMWF simplified physics) and radiative transfer Elias Holm moisture control variable From Stengel et al. (2012) in preparation
Cloud layer Jacobians Increrments Without SP With SP q T
Impact of clear SEVIRI radiances on forcast scores (relative RMS reduction)
Additional impact of cloud-affected radiances
Assimilation of radar data As an example, the approach to assimilation of radar reflectivity data within the HIRLAM and ALADIN communities are briefly described. Meteo-France has been the leading partner (Caumont, Ducrocq, Wattrelot et al.). HIRLAM and ALADIN have ongoing trials to collect, pre-process and assimilate radar data from an extended European network. The slides have been borrowed from a presentation by Caumont et al. (2006) and Mahfouf (2011)
From Caumont et al. (2006)
From Mahfouf (2011)
From Mafouf (2011)
From Mahfouf (2011)
Weak digital filter constraint Try to come close to the slow manifold at the same time as minimization of Jo helps us to come close to the observations! Note the un-known coefficient γdf.
Does this help to prevent noise in the non-linear model integration? Horizontal average of the absolute value of the surface pressure tendency.
Scaleselective DFI Also: Scale-selective DFI => less noise! (Termonia, 2008)
Validation and tuning of HARMONIE surface assimilation 1. Validate the present OI scheme using precalculated coeffficients for the soil variables 2. Compare the performance of the new EKF scheme for soil variables with the OI scheme check that everything is correctly implemented 3. Tune if needed the EKF scheme (beat the OI scheme!) 4. Use of satellite data (ASCATT and SMOS)? Work started with 1-3.
Example of time-series in SYNOP stations Gardermoen Without OI With OI Improved daily cycle! Improved RH2 Reduced wg2
Verification T2 and RH2
Verification PMSL!!
Summary Priority areas for HIRLAM, ALADIN in mesoscale DA Use of high-resolution observations: - Radar reflectivity and radial winds; - Cloudy radiances; - Ground-based GPS; Low-level winds (10 met, scattereometer) Flow-dependencies: - Heterogenious structure functions (rain-no rain, cloud-no cloud); Hybrids; 4D Ens Var; 4DVar (as reference); Alignment, warping and dynamical transform; Balances: - Flow-dependent structure functions; - Mesoscale DF; Spinup (consistency between DA and physics) Improve large scales: Jk & Control of LBC in 4D-Var Surface assimilation: - EKF (=> flow and model dependency); satellite observations; snow, lakes; interpolation for heterogenious surfaces
Solution for long term perspective powerful LAM EPS realistic LAM perturbations efficient DA scheme skillful control forecast Transversary issue: interaction of initialisation/dynamics/physics/model error Meteorologisk institutt met.no Greetings from my student Lena Bojarova, 2012