Numerical Weather prediction at the European Centre for Medium-Range Weather Forecasts Time series curves 500hPa geopotential Correlation coefficent of forecast anomaly N Hemisphere Lat 20.0 to 90.0 Lon -180.0 to 180.0 Reaches (60) oper_an od oper 00UTC,12UTC,beginning Score reaches 60 (MA 12m of ccaf) reaching 60% day 10 9.5 9 8.5 8 Massimo Bonavita Data Assimilation Section Massimo.Bonavita@ecmwf.int www.ecmwf.int 7.5 7 6.5 6 5.5 5 198619871988198919901991199219931994199519961997199819992000200120022003200420052006200720082009 Slide 1
Who are we and what do we do? European Centre Medium-Range Weather Forecasts We are an independent international organisation funded by 34 States Up to fifteen days ahead. Today our products also include monthly and seasonal forecasts and we collect and store meteorological data. We produce world-wide weather forecasts What do we have to achieve this? People Equipment Budget Experience About 260 staff, specialists and contractors State-of-the-art supercomputers and data handling systems 50 million per year 37 years Slide 2
ECMWF An independent intergovernmental organisation established in 1975 with 20 Member States 14 Co-operating States Slide 3
Why medium-range? Slide 4
Three essential components Observations Surface (weather stations, balloons, ships, buoys, ) Space (satellites) Mathematical model Modelling software developed by the Research Department Very large computer (supercomputer) Was central to the establishment of ECMWF Today, a single 10-day forecast requires 6,500,000,000,000,000 calculations Slide 5
Data sources for the ECMWF Meteorological Operational System Slide 6
Instrument usage Slide 7
Slide 8
How ECMWF was established Start of operational activities 1978 Installation of first computer system (CRAY 1-A) 1979 Start of operations N48 grid point model 200km Slide 9
Supercomputers at ECMWF ECMWF has a long history of using High Performance Computing in NWP 1978 1996: Cray (Cray-1A, XMP, YMP, C90, T3D) 1996 2002: Fujitsu (VPP700, VPP700E, VPP5000) 2002 today: IBM (Power4, Power5, Power6,Power7) ECMWF has currently the 44 th machine of the TOP 500 list in terms of computing capacity Slide 10
The ECMWF Archive statistics In Spring 2013 the DHS archive held more than 40 PB = 40,000,000,000,000,000 bytes This data is stored on cartridges, each one containing 5 TB = 5,000,000,000,000 bytes If this data was stored on CDs, the CD pile would be 72 km high This is more than the distance from Reading to London! If this data was printed, there would be enough books to reach the moon 8.1 times! 2,595,097 km Slide 11 ECMWF
The operational forecasting system High resolution deterministic forecast: twice per day 16 km 137-level, to 10 days ahead Ensemble forecast (EPS): twice daily 51 members, 30/60 km 62-level, to 15 days ahead 11 Aug 2011 Monthly forecast EPS extension: twice a week (Mon/Thursdays) 51 members, 30/60 km 62 levels, to 1 month ahead Seasonal forecast: once a month (coupled to ocean model) 41 members, 125 km 62 levels, to 7 months ahead Slide 12
T L 799 Previous operational resolution 634.0 600 550 500 450 400 350 300 250 200 150 100 T L 1279 Current operational resolution (since January 2010) 684.1 650 600 550 500 450 400 350 300 250 200 150 100 50 N 50 N Slide 13 25 km grid-spacing ( 843,490 grid-points in the reduced Gaussian grid) 0 50 10 50 N 50 N 16 km grid-spacing ( 2,140,704 grid-points in the reduced Gaussian grid) 0 50 10
Increasing Resolution ~210km ~125km ~63km ~39km ~25km ~16km Slide 14
Slide 15
Evolution of ECMWF scores comparison northern and southern hemispheres Courtesy of ECMWF. Adapted and extended from Simmons & Hollingsworth (2002) Slide 16
Evolution of ECMWF and other Centers scores Error reduction ECMWF ERA reforecasts UKMO NCEP Slide 17
Hurricane Sandy 22-30 Oct. 2012 Largest Atlantic hurricane on record 2 nd costliest hurricane in US history Slide 18
Hurricane Sandy 22-30 Oct. 2012 Mslp Analysis 30/10/2012 00UTC Mslp t+204h Forecast started on 22/10/2012 00UTC Slide 19
Hurricane Sandy 22-30 Oct. 2012 Slide 20
Data assimilation Slide 21
Data assimilation Slide 22
Conventional observations SYNOP/METAR/SHIP: Surf. Press., 10m wind, RH BUOY: Surf. Press., 10m wind RADIOSONDE: wind, temperature, humidity PILOT/Profilers: wind Slide 23 AIRCRAFT: wind, temperature, (humidity) SYNOP/METAR/SHIP 120000 BUOY 15000 RADIOSONDE 650 PILOT/Profilers 4000 AIRCRAFT 180000
Satellite observations T/Hum. Sounders: AMSU-A/B, MHS, HIRS, AIRS, IASI BUOY: Imagers: Surf. Press., SSMI, 10m SSMIS, windamsr-e, TMI GPS radio occultations Geostationary+MODIS: IR and AMV Scatterometer ocean low-level winds Slide 24 IR and MW Sounders 12000000 Imagers 500000 GEO + MODIS IR/AMV 200000 GPS 200000 SCATT 130000
Observations Satellite Obs. make up 95% of total used observations Conventional Obs. are still very important in the North Hem. and for anchoring the bias correction of the satellite radiances Slide 25
Quality control of Observations Check out duplicate reports Ship tracks check Hydrostatic check Blacklisting Data skipped due to systematic bad performance or due to different considerations (e.g. data being assessed in passive mode) Departures and flags available for all data for further assessment Thinning Some data is not used to avoid over-sampling and correlated errors Departures and flags are still calculated for further assessment Slide 26 Analysis
4DVar analysis All observations within a 12-hour period (~13,000,000) are used simultaneously in one global (iterative) estimation problem Observation model values are computed at the observation time at high resolution: 16 km 4D-Var finds the 12-hour forecast that take account of the observations in a dynamically consistent way. Based on a tangent linear and adjoint forecast models, used in the minimization process. 80,000,000 model variables (surface pressure, temperature, wind, specific humidity and ozone) are adjusted Slide 27
4DVar analysis n 1 T 1 1 T 1 J ( x0 xb) B ( x0 xb) HiMix0 yi Ri HiM ix0 yi 2 2 i0 J b J o B background error covariance matrix, R observation error covariance matrix M forward nonlinear forecast model (time evolution of the model state, index i) H observation operator (model spaceobservation space) Note that the generalised observation operators are assumed to be perfect: y it = H i M i [x 0t ] To efficiently find the minimum of this cost function we need to compute its gradient w.r.t. the initial state: n 1 T ' T 1 x B x 0 0 xb M i 0 H i R i i i x0 yi i 0 min J J ( ) [ t,t ] H M 0 where H T = adjoint of observation operator and M T = adjoint of forecast model. Slide 28
4DVar analysis n 1 T 1 1 T 1 J ( x0 xb) B ( x0 xb) HiMix0 yi Ri HiM ix0 yi 2 2 i0 J b J o n 1 T ' T 1 x B x 0 0 xb M i 0 H i R i i i x0 yi i 0 min J J ( ) [ t,t ] H M 0 The cost of the 4DVar analysis is still prohibitive: 1. A typical minimization will require between 10 and 100 evaluations of the gradient 2. The cost of the adjoint model is typically 3 times that of the forward model. 3. The analysis window in the ECMWF system is 12-hours Hence, the cost of the analysis is roughly equivalent to between 20 and 200 days of model integration. Slide 29
Incremental 4DVar We introduce a linearization state x t so that at any time i over the assimilation window: x i x ti x i, xti M ( t0, ti ) xt 0 The 4DVar cost function can then be written in terms of the increments xi M[ t0, ti] x0 and approximated by the quadratic function: n 1 T 1 1 ' T 1 ' J ( x ) x0 B x0 ( HiM '[ t0, ti ] x0 di ) R ( H im '[ t0, ti ] x0 2 2 0 i i0 where d i =y i -H i (x ti ) are the departures/innovations computed using the non linear model and observation operator. This ensures the highest possible accuracy in the computation of the departures, which are the primary input to the assimilation! To reduce the computational cost we make a further approx. and evaluate the quadratic cost function at lower resolution d ) Slide 30
Incremental 4DVar T1279L137 T159L137 T255L137 T255L137 Slide 31
Incremental 4DVar Slide 32
Incremental 4DVar Analysis increments for vorticity at the start of the assimilation window, model level 64, 2012/09/30 21UTC 1 st Outer Loop T159L137 2 nd Outer Loop T255L137 Slide 33 3 rd Outer Loop T255L137
Incremental 4DVar Another agreeable property of 4DVar is that it implicitly evolves background error covariances over the length of the assimilation window (Thepaut et al.,1996) with the tangent linear dynamics: B(t) M B(t 0 )M T Slide 34
Incremental 4DVar MSLP (contours) and 500 hpa height (shaded) background Temperature analysis increments for a single temperature observation at the start of the assimilation window: x a (t)-x b (t) MBM T H T (y-hx)/(σ b2 + σ o2 ) t=+0h t=+3h t=+6h Slide 35
Incremental 4DVar The 4D-Var solution implicitly evolves background error covariances over the assimilation window (Thepaut et al.,1996) with the tangent linear dynamics: B(t) M B(t 0 )M T But it does not propagate error information from one assimilation cycle to the next: 4DVar behaves like a Kalman Filter where B(t 0 ) is reset to a climatological, stationary estimate at the beginning of each assimilation window. This is suboptimal. What to do? Slide 36
Improving 4DVar: The non-sequential approach It can be shown (e.g. Menard and Daley, 1996) that, for a linear system the state at the end of the analysis window in 4D-Var is identical to that produced by a Kalman Filter, given the same states and covariances. If we run 4D-Var with a long enough window, the state and covariance at the start of the window (x b and B) will have little influence on the analysis at the end of the window. How long should this window be then? Slide 37
Improving 4DVar: The non-sequential approach How long should this window be then? Time series curves 500hPa Geopotential Root mean square error forecast S.hem Lat -90.0 to -20.0 Lon -180.0 to 180.0 T+120 all obs all obs 120 110 100 90 80 70 60 50 40 Slide 38 30 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 AUGUST 2005
Slide 39 Improving 4DVar: The non-sequential approach For long assimilation windows (> 12h) it is no longer accurate to assume the model to be perfect. For long windows we have to add a model error term to our cost function (Weak-constraint 4D-Var): Where Q t is the covariance of the model error = This is valid for Gaussian, temporally uncorrelated model errors Note also that it is no longer possible to reduce the minimization problem to a function of the initial state only, i.e. J(x 0 ) 1 1 1 1 0 1 1 1 0 1 1 0,...,, t t t t T T t t t t t t t t t T T t t t t o b T o b T M M H H J x x Q x x x y R x y x x B x x x x x
Improving 4DVar: The non-sequential approach In the non-sequential approach problem is shifted from the estimation of B to the estimation of Q: this is not any easier! It is difficult in the 4D-Var framework to produce good estimates of the analysis errors, which are fundamental for the initialization of an ensemble prediction system Long window, weak constrain 4DVar has been proven in simplified systems, but not in full operational setups yet To make progress we clearly need another approach Slide 40
Thank you for your attention Slide 41