Numerical Weather Prediction Chaos, Predictability, and Data Assimilation

Transcription

1 July 23, 2013, DA summer school, Reading, UK Numerical Weather Prediction Chaos, Predictability, and Data Assimilation Takemasa Miyoshi RIKEN Advanced Institute for Computational Science With many thanks to UMD Weather-Chaos group, Data Assimilation Research Team,

2 Who am I? RIKEN AICS We are hiring!

3 Which is real, or simulation? A B Courtesy of H. Miura

4 State of the art

5 What do we compute? 1. Discretize the atmosphere Each grid box has the meteorological variables: winds, temperature, humidity, clouds, pressure

6 What do we compute? 1. Discretize the atmosphere Each grid box has the meteorological variables: winds, temperature, humidity, clouds, pressure 2. Solve the equations of atmospheric dynamics and physics Initial condition winds, temperature, humidity, clouds, pressure NWP MODEL (dynamics, physics) Boundary conditions Forecast

7 Discretization NICAM model (courtesy of H. Miura) dx~480 km dx~240 km dx~120 km dx~60 km dx~30 km dx~15 km dx~7.5 km dx~3.75 km

8 Outline Introduction to data assimilation in NWP Chaos and predictability Chaos synchronization Ensemble Kalman Filter Recent research Some flavor of cutting-edge research topics Future perspective Towards Big Data Assimilation

9 Dynamical simulations Model = time-advancing operator Simulation = state evolution time

10 Let s think about predictions. What kind of forecast is reliable? 1. Thunderstorm 2. Seasonal forecast (warm summer this year?) 3. Ocean tide 4. Solar eclipse 5. Stock price 6. Who you will marry What characterizes the reliability of forecast?

11 Sensitivity to initial conditions Perturb the initial conditions and run the multiple forecasts (a.k.a. ensemble forecasts) P P Less certain More certain T=t0 T=t1 T=t0 T=t1

12

13 Deterministic chaos = sensitivity to the initial conditions Model solutions in phase space Uncertain initial states Local instabilities Diverging predictions

14 Predictability is about uncertainties. More predictable Less predictable Very unpredictable

15 Predictability is about uncertainties. More predictable Less predictable We have uncertain initial estimates, and uncertain predictions. Very unpredictable

16 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

17 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

18 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

19 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

20 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

21 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

22 Weather system is chaotic. Chaotic dynamical system has intrinsic limit to predictability.

23 What can we do? (frontier research) Obtain more accurate initial conditions More observations Better data assimilation methods Understand the error growth Better understand the dynamics and physics Predict the predictability Let users know how (un)certain the forecasts.

24 Data Assimilation (DA) Observations Numerical models Data Assimilation Vaisala Data assimilation best combines observations and a model, and brings synergy.

25 DA as Chaos Synchronization (Yang et al. 2006) Master (drive) system Slave (response) system Nature Observation Transferring information Simulation

26 Chaos synchronization problem We would like to synchronize the model simulations with reality. (Questions:) Under what conditions do they synchronize? How easy/difficult is the synchronization? (Answer:) Synchronization depends on the coupling strength and system s instabilities. Observing network (quality and quantity of transferring information) Accuracy of the simulation model Optimality of the data assimilation method

27 Numerical Weather Prediction An example of synchronizing chaos Forecast Forecast Model Simulation Analysis Observation Analysis Analysis Observation True atmosphere (Unknown) time

28 Global Observing System Radar Aircraft Satellite Weather balloon Ship Buoy Surface station

29 Collecting the data World s effort! (no border in the atmosphere)

30 Collecting the data

31 Data Assimilation DA corrects forecast fields to fit better with observations. DA produces the best estimate of the current atmospheric state, which is used as the initial condition for NWP. Geopotential height at upper atmosphere is basically parallel to winds.

32 A simple example: two thermometers [C] p Temperature of this room A B Best estimate * *, B A B A B A B A A B T T T σ σ σ σ σ σ σ σ σ + = + + = More accurate analysis is obtained by combining two independent information ) ( exp ) ( A A A T T T p σ ) ( exp ) ( B B B T T T p σ = exp 2 ) ( 2 ) ( exp ) ( ) ( ) ( B A B A A B B A B A B B A A B A B A T T T T T T T T p T p T p σ σ σ σ σ σ σ σ σ σ

33 Multidimensional generalization )] ( ) ( exp[ ) ( f T f f p x x B x x x )] ( ) ( exp[ ) ( o T o o H H p y x R y x x )}] ( ) ( ) ( ) {( exp[ ) ( ) ( ) ( o T o f T f o f o f H H p p p y x R y x x x B x x x x x + = Generalizing to a multidimensional variable Background PDF Observation PDF Joint distribution Analysis is given by the maximizer x (maximum likelihood). Background error covariance Observation error covariance

34 Flow-dependence Analysis equation: xx aa = xx bb + KK(yy HHxx bb ) innovation B determines the analysis increments, the correction made by the observations. Traditionally, OI (Optimal Interpolation) and 3D-Var (3-dimensional variational) methods use the flow-independent B. 4D-Var and EnKF (Ensemble Kalman Filter) uses flow-dependent B. These methods are known as advanced data assimilation methods.

35 Difference between EnKF and 3D-Var R a x Flow-dependent errors expand in low-dimensional subspace Uniform error structure B Errors of the day y o x f Analysis without flow-dependent error structure (e.g., 3D-Var)

36 Flow-dependent error structure We use constant B in 3DVAR In Kalman filtering, we forecast B An example of using the flow-dependent B There is a cold front With constant B With flow-dependent B Kalman filtering can consider the flow-dependent error structure.

37 Flow-dependent analysis increment A B L H : innovation (observation minus background) Question: Which is 3D-Var or 4D-Var?

38 Kalman Filter (KF) Analysis w/ errors R OBS w/ errors Analysis w/ errors T=t0 FCST w/ errors T=t1 Direct application to high-dimensional systems is prohibitive. P = M P M f a t1 a x t0 x a t 0 t 0 T

39 Ensemble Kalman Filter (EnKF) Analysis ensemble mean R Obs. Analysis w/ errors An approximation to KF with ensemble representations f f f δxt1( δxt P 1) t1 m 1 FCST ensemble mean T=t0 T=t1 T=t2 T

40 LETKF (Local Ensemble Transform Kalman Filter) Analysis is given by a linear combination of forecast ensemble: X a = x f +δx f T T = Ensemble Transform Matrix (ETKF, Bishop et al. 2001; LETKF, Hunt et al. 2007) ~ a T 1 o f ~ a P ( δy) R ( y H ( x ensemble mean update )) + [( m 1) P ] 1/ 2 uncertainty update ~ a T 1 1 P = [( m 1) I / ρ + ( δy) R δy] Analysis error covariance in the ensemble subspace

41 4D-LETKF (Ensemble Kalman Smoother) t n-1 x a (t n 1 ) = x a (t n 1 ) + X a (t n 1 )w a (t n ) X a (t n 1 ) = X a (t n 1 )W a (t n ) t n w a = P a Y b T R 1 (y H(x )); W a = [(K 1) P a ] 1 2 4D-LETKF can treat observations within a time window. Including future observations (smoother) Better treating frequent observations (satellites, radars, etc.)

42 Effective data assimilation xx 1 (tt) xx 1 (tt + 1) xx 2 (tt) xx 2 (tt + 1) Question: Which mode would we like to correct?

43 Effective data assimilation xx 1 (tt) xx 1 (tt + 1) xx 2 (tt) xx 2 (tt + 1) EnKF corrects growing errors effectively.

44 Dynamical adjustment and Balance Geostrophic adjustment suggests that winds adjust to pressure evolution at larger scales (more precisely, larger than Rossby deformation scale), and that pressure adjusts to wind evolution at smaller scales. It is essential to correct pressure fields in synoptic scale weather forecasts. If we observe winds, it is essential to correct pressure fields based on balance relationship in error correlations.

45 DA has an impact. SV w/ 4D-Var JMA operational system LETKF under development OBS FCST OBS FCST Miyoshi and Sato (2007) Using the same NWP model and observations. DA matters!

46 DA is important in NWP. RMSE(m) T213L30 JMA( 日 ) ECMWF( 欧 ) NCEP( 米 ) UKMO( 英 ) Revision of cumulus parameterization T213L40 Revision of cumulus parameterization QuikSCAT 3D-Var ATOVS BT 500 hpa Geopotential Height RMSE (NH) MODIS AMVs TL319L40 4D-Var Revision of cloud Revision of radiation SSM/I and TMI VarBC TL959L60 Reduced Grid TL959L CSR Revision of VarBC Revision of RTM SSMIS, ASCAT Courtesy of Y. Sato (JMA)

47 DA gives feedback about observations. Estimated impact of observations (from NCEP Global Forecasting System, Y. Ota 2012) AMSU-A (Satellite) Improving Degrading RAOB (In-situ) Degrading Improving (Courtesy of Y. Ota) two-way Vaisala

48 Impact of WC-130J dropsondes Kunii and Miyoshi (2012) Degrading Improving

49 DA can find optimal model parameters. Sensitivity to the model parameters (a real TC case) Ruiz and Miyoshi (2012) Less sensitive More sensitive Sensible heat flux parameter Latent heat flux parameter Find optimal parameters using observations

50 DA can find optimal model parameters. Sensitivity to the model parameters (a real TC case) Ruiz and Miyoshi (2012) Less sensitive More sensitive Sensible heat flux parameter Latent heat flux parameter Find optimal parameters using observations Parameter estimation with an EnKF (idealized experiments) Bad initial values : true value Time-varying parameters Accurate and stable estimates after spin-up

51 A challenge: better use of satellite data CTRL AIRS: Atmospheric Infrared Sounder Conventional (NCEP PREPBUFR) Conv. + AIRS retrievals (AIRX2RET - T, q) Larger inflation is estimated due to the AIRS data. Adaptive inflation method was newly developed (Miyoshi 2011).

52 AIRS impact on TC forecasts ~28 samples Too deep to resolve by 60-km WRF TC track forecasts for Typhoon Sinlaku (2008) were significantly better, particularly in longer leads.

53 A challenge: multi-scale localization Localization plays an essential role in an EnKF to cope with limited ensemble size. No localization Higher resolution requires more localization, limiting the use of observations. Localized We look for better use of observations by separating the scales. Analysis increment from a single profile observation (20 members)

54 An idea of merging two scales Motivated by Buehner (2012), we construct analysis increments at high (h) and low (l) resolutions separately. δδxx = δδxx h + δδxx ll δδxx h δδxx ll Miyoshi and Kondo (2013)

55 Results are promising. Experiments with the T30L7 SPEEDY model (Molteni, 2003) Global-average RMSE Regular localization (700 km) Dual localization ( km) Mid-level U Low-level T Near-surface Q Surface pressure

56 Improved almost everywhere

57 Challenges with higher resolutions Algorithmic design with arbitrary grid structures is a challenge. 60-km analysis 60/20-km 2-way nested analysis Miyoshi and Kunii (2012)

58 Challenges of DA On the observations Advanced observing systems Next-generation geostationary satellites, Phased array radars, etc. More effective use of observations E.g., High-resolution model Better use of high-resolution observations Exploring new types of data Use of under-utilized data (e.g., satellite sensors, dual-pol radar, surface obs) New data source (e.g., live camera images?) On the simulations Improved resolutions Multi-component integration (ocean, land, aerosols, etc.) On the data assimilation methods Higher-order statistics (non-gaussian) Multi-scale Multi-component covariance Model errors (parameter estimation, multi-model ensembles) Efficient algorithms (parallel efficiency, search algorithms, matrix manipulation) Applications (sensitivity analysis, observation impact, observing system design)

59 Phased array radar (courtesy of NICT) Conventional Radar ~15 scan angles Every 5-10 minutes Phased Array Radar ~100 scan angles Every seconds

60 New data: can we use live-camera images? 1. Assimilation of reduced/extracted information (e.g., weather type, visibility) (challenge) Automated image processing technology 2. Simulating images from model outputs (i.e., having observation operators of live cameras) Direct assimilation (challenge) precise 3-dimensional radiation model

61 Next-generation observing systems High-frequency, high-resolution large data volume Radar Currently, 5-min volume scan 10 seconds! Geostationary satellite Currently, 30-min Full Disk Image 5 minutes! Currently 2.62Mbps raw data rate ~100Mbps Diversity Diverse sources, formats

62 Big Data Assimilation Era High-resolution simulation Big Data Assimilation Interdisciplinary development Better simulation Evaluating model errors, parameter estimation Next-generation observations

63 Toward next 20 years of DA Computational requirement In addition to FLOPS, IO throughput is essential. Big Data Assimilation Era Throughput ~10 Exabytes/day Exploding data Big Data Enabling effective use Big Data High-resolution simulation More computational power High-resolution obs Advanced obs technology

64 Summary Observations Numerical models Data Assimilation Vaisala DA as a bridge between real-world data and simulation Tackling predictability of chaotic dynamical systems Optimizing observation systems and model parameters Getting the most from both simulation and data DA methods are based on statistical mathematics Sharing experience among wide applications

65 Expanding collaborations AFES JMA GSM Atmosphere CFES JMA MSM Ocean WRF-ROMS WRF OFES ROMS LETKF CPTEC Brazil GFS SPEEDY CO2 MOM JAMSTEC Chem Mars GCM :Existing U.Tokyo Aerosol :Possible future expansion MRI Chem CAM Chemistry

66 I would welcome new collaborations! AFES JMA GSM Atmosphere CFES JMA MSM Ocean WRF-ROMS WRF OFES ROMS 1 Toy models Thank (e.g., Lorenz model) 2you Intermediate very AGCMmuch for (SPEEDY model, Molteni 2003) Real systems 3 your kind attention!! (e.g., operational models) CPTEC Brazil GFS SPEEDY CO2 MOM JAMSTEC Chem Mars GCM :Existing U.Tokyo Aerosol :Possible future expansion MRI Chem CAM Chemistry

67 We are hiring researchers. Please feel free to contact me for details.