Evolution and Advancement of Satellite Data Assimilation

Transcription

1 Overview of JCSDA Technical Activities and Plans Evolution and Advancement of Satellite Data Assimilation Presented by Tom Auligné, Director, JCSDA With Contributions from JCSDA Scientists and Executive Team members

2 Introduction Source: Met Office

3 Introduction Source: Met Office

4 Introduction Source: Met Office

5 Introduction Source: Met Office

6 Introduction Data assimilation systems usually combine together information from a set of observations, a short term forecast, and possibly other information to estimate the most probable state of atmosphere. x b y o Courtesy: Ménétrier

7 Introduction Hypotheses: observation and model background have errors that are uncorrelated, unbiased, normally distributed, with known covariances Method: Bayesian statistical framework combined with dynamical constraints Outcome: best estimate of current state (maximum likelihood, minimum RMSE)

8 Model Background Error Covariances Horizontal autocorrelations Vertical auto-correlation RH (mid-troposphere)

9 Mini-4DVar (10min) Wang, Sun, Zhang, Huang and Auligné (MWR 2013)

10 OSE Activities / Impact Assessment An extensive assessment of the global observing system impact on NOAA forecast system has been undertaken. The impact assessment was done wrt satellite data (collectively &individually: microwave AMSU, MHS, GPS, hyperspectral IR, AMVs, etc) as well as conventional data. Satellite data as a group, had a very significant impact which surpasses the conventional data impact (by a wide margin), especially in the southern hemisphere. The impacts of individual classes of sensors did not add up to the significant impact above. Source: Jung (2012) Skill improvement ~ 2.4h/y estimated socio-economic benefit ~$500M/y Source: Riishojgaard 10

11 Description of the JCSDA Vision: An interagency partnership working to become a world leader in applying satellite data and research to operational goals in environmental analysis and prediction NOAA NESDIS NASA GSFC U.S. Navy JCSDA NOAA NWS U.S. Air Force NOAA OAR Mission: to accelerate and improve the quantitative use of research and operational satellite data in weather, ocean, climate and environmental analysis and prediction models. 11

12 JCSDA Science Priorities Overarching goal: Help the operational services improve the quality of their prediction products via improved and accelerated use of satellite data and related research Radiative Transfer Modeling (CRTM) Preparation for assimilation of data from new instruments Clouds and precipitation Assimilation of land surface observations Assimilation of ocean surface observations Atmospheric composition; chemistry and aerosol Approved by the Science Steering Committee 12

13 JCSDA Mode of operation JCSDA Mode of Operation Internal Research External Research Visiting Scientist Program Coordination, Education & Outreach Systems Support - Directed by JCSDA affiliated scientists - Research undertaken independently by partners, overlapping with JCSDA priorities -Carried out by the partners - Mixture of new and leveraged funding - Short-term ROI expected - Grants and/or contracts awarded - Administered alternately by NOAA, NASA - Open to the broader research community - Funding awarded competitively - Near-term ROI expected - Open to scientists from everywhere -Short-term (a few weeks/months) - Requires to Identify host at partner institution; work on JCSDArelevant topic - Mid-term ROI expected - Workshops, - Colloquium, Seminar series, Quarterly newsletter, website - Publications - Meetings (ET, MOB, AA, SSC) - Coordination: planning and reports=ing - Management, admin support - Offers O2R environment - Open to External scientists, even not funded by JCSDA - Goal: funnel efforts toward helping the JCSDA partners (incl. NOAA) - Access to HPC & operational systems in R&D - S4 and JIBB 13

14 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 14

15 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 15

16 Data Being Assimilated/Assessed New Sensors Data Assimilation: (new QC, error optimization, impact assessment on NOAA forecast systems) HIMAWARI-8 AHI (Dry run for GOES-R ABI) GPM /GMI Megha-Tropiques SAPHIR (WV Sounder) ISS-RAPIDSCAT (Scatterometer) GCOMW AMSR2 Existing Sensors optimization: (QC, Surface-sensitive channels assimilation, pre-processing, dynamic emissivity, etc) ATMS SSMIS AMSU MHS 16

17 SAPHIR Data Assimilation SAPHIR Preliminary Forecast Impacts The NOAA GDAS has been extended to assimilate brightness temperatures from SAPHIR L1A2 data 100hPa RH RMSE Day 3 time series suggests improvement when SAPHIR is assimilated Preliminary results Indicate that the assimilation of SAPHIR has a neutral impact on the global forecast, with noticeable improvement in moisture at upper levels Assimilation is done in clear-sky conditions over ocean. Algorithms have been developed to retrieve cloud, ice, and/or emissivity for filtering and quality control 100hPa RH RMSE 42-day mean: GFS forecast verified against ECMWF analysis Improvement (reduction in RMSE) in upper level RH when SAPHIR is assimilated, pronounced for short-mid range forecast times

18 AMVs Data Assimilation Assessment Project Objective: Compare assimilation strategies employed by JCSDA partners (NRL, GMAO, NOAA) and optimize the AMV filtering/qc/thinning as well as observation errors NOAA SATWND BUFR are compared to NRL super-obs Red: Superobs worse than SATWND BUFR Green: Superobs better than SATWND BUFR Some positive impact Mid-level wind RMS, biases. Negative impacts some day 1. In general impact of super-observations is mostly neutral or mixed.

19 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 19

20 CRTM CRTM Mission Satellite radiance simulation and assimilation for passive MW, IR, & Visible sensors of NOAA,NASA,DoD satellites, and others (200 sensors) Simulation of clear/cloudy/precipitating scenes, globally CRTM Applications Data assimilation in supporting of weather forecasting Physical retrieval algorithm for products Stability and accuracy monitoring of satellite observations Research: reanalysis, climate studies, air quality forecasting, and a radiative tool for students CRTM On-going Development New sensors: Acquarius, SMOS, SMAP,.. New Cloud Optical Table RT Solver Optimization Multiple Aerosol Models for GOCART and CMAQ apps Comparison of CrIS unapodized measurement, CRTM-OSS and CRTM-ODPS simulations The CRTM-OSS allows for: Unapodized radiance simulation for better spectral sampling sensitivity, Monochromatic scattering to avoid optical path scaling problem because of polychromatic issue. 20

21 The Community Active Sensor Model (CASM) Purpose: The Community Active Sensor Module (CASM) will enable the Community Radiative Transfer Model (CRTM) to simulate active microwave and active visible remote sensing observations. CASM facilitates the assimilation of all-weather active-sensor observations, such as the Global Precipitation Measurement mission (GPM) Dual-frequency Precipitation Radar (DPR), either directly or through preprocessing using (MIIDAPS). CASM CASM forward modeled radar reflectivities (Black line), compared to GPM-DPR observations (Red line).

22 Benefits: Quality control based on non-convergence Detection of rain and ice contamination, Coast contamination, RFI for imagers Dynamically-retrieved emissivity to allow assimilation of surfacesensitive channels Provide sounding products in cloudy/rainy conditions Allows background adjustment to fit obs JOINT CENTER FOR SATELLITE DATA ASSIMILATION Universal QC & Pre-Processing Tool: Satellite Pre-Processing (MIIDAPS) MIIDAPS valid for MW, IR (Geo, Polar) Extension to Active sensors Uses CRTM and CASM as forward operators Emissivity ATMS- Liquid Path ATMS- Goal is to have a community QC tool for satellite data assimilation quality-control & pre-processing: T and Q sounding CrIS/ATMS- Cloud CrIS 22

23 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 23

24 Observation Error Covariances AIRS Diagnostic R Matrix Correlated errors (esp. for moisture channels) At least partly due to representativeness error (Waller et al. 2014) Source: Weston (2011)

25 First Guess JOINT CENTER FOR SATELLITE DATA ASSIMILATION Second Guess Third Guess Observation Update of model q cloud, q ice First Guess Second Guess Third Guess AIRS Window Channel #787 Observations

26 Displacement Analysis (Grid Warping) Observation Innovation Model Hurricane Katrina OSSE Synthetic observations (TPW)

27 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 27

28 CNTRL 3POLAR 2POLAR 3PGPS Current Remove satellite quasiredundant assimilated data as operationally. satellite data. Keep Remove 1 PM satellite polar GPSRO data in data each (SNPP) with no future to PRIMARY simulate mission or JPSS orbit Data uncertain Gap. Polar Coverage funding. JOINT CENTER FOR SATELLITE DATA ASSIMILATION Observing System Experiments Assimilated Denied GPSRO Coverage *MODIS IR winds are a proxy for SNPP VIIRS 28

29 500 hpa Anomaly Correlation (vs CNTRL) 500mb Height AC vs Forecast Time MEAN AC SCORE CNTRL 3POLAR 2POLAR 3PGPS NH (top) NH SH (bottom) Significantly Worse Distribution of DAY 5 Height 500mb Anomaly Correlation More Good Forecasts NH More Good Forecasts SH SH Significantly Worse More Bad Forecasts More Bad Forecasts Impacts on 500 mb Height Forecast Anomaly Correlation (Day 5) Timeseries of AC shows few dropouts for all experiments, with 2POLAR having much lower mean AC than other experiments. 3POLAR slightly degraded AC in NH, neutral in SH. 3PGPS significantly degraded in NH and slightly at Day 3-4 in SH 2POLAR significantly degraded in NH and SH for Day 1-7 forecast. 2POLAR exhibits more frequent low AC score. 29

30 Forecast Sensitivity Observation Impact (FSOI) from Gelaro et al. 2009

31 Forecast Sensitivity Observation Impact (FSOI) from Gelaro 2009

32 Strategic Goals 1. Expand capabilities in assimilating satellite sensors 2. Spearhead a community data assimilation initiative 3. Address scientific frontiers to optimize the use of satellite data 4. Deliver new and improved tools to support observing system impact assessments 5. Foster improved organizational management, interagency coordination and outreach strategies 32

33 JCSDA Education & Outreach Activities Summer colloquium on satellite DA (3-year cycle). Annual JCSDA Science Workshop Joint Workshops with Programs and International Partners Dec. 2015: 3 rd Joint JCSDA-ECMWF Workshop (clouds) Jan 2016: JCSDA Annual Meeting March 2016: Joint NCAR-JCSDA Workshop (next-gen DA) Jul 2016: Joint JCSDA-DTC GSI/EnKF Tutorial Monthly Seminar Series on DA: remote access available JCSDA Newsletters (quarterly), active web site External Research funding: NOAA FFO NASA ROSES 33

34 Goal: Accelerates the use of satellite data in NWP centers Ensures resources are in place to achieve successful transition: supercomputer, a software integration team, etc. Ensures consistency and demonstrates benefits with operational systems like: Hybrid GDAS system (global), HWRF (Hurricane forecast). JOINT CENTER FOR SATELLITE DATA ASSIMILATION JCSDA R2O Concept Scientific efforts in satellite DA in academia, CIs Select projects for R2O Transition Scientific efforts in satellite DA in research community Scientific efforts in satellite DA in NOAA/NESDIS (funded by GOES-R PGs, JPSS PGs, etc) JCSDA s own DA Activities Scientific efforts in satellite DA in NOAA/OAR Diverse R&D activities Products, techniques, improvements, with direct and immediate relevance to NWP Operational Models (both global and regional) Made consistent with Operational Systems Ongoing Baseline Improvement in NWP centers NWP Operational Centers Objective: Improvements in Forecast skills

35 Looking into the future

36 Major Trends in Global Observing System An explosion of new sensors and data volume have occurred and will continue to occur in the near future New technologies are allowing more measurements to be made, more frequently, better Overall, more nations are building and launching satellitebased Earth Observing sensors Clearly, we might be in the middle of a golden era of satellite-based earth observation sensors

37 Multiple Converging Applications with DA Perfect Situational Awareness Nowcasting Forecast Skill NWP Blending Seasonal Numerical Models Little Forecast Lead Time (hours) Extrapolation Adapted from Sai Ravela (MIT)

38 Use of NOAA DA (GSI) in NESDIS as Data Fusion Tool (Satellite, Conventional, ground based, Airborne, etc) SDRs (Polar) Metop, N19, NPP, DMSP IR, MW SDRs (Geo) GOES, GOES-R, MSG, Ground- Based Data Radar Conventional Data Airborne Data GPS Data Merge data across platforms, agencies, and sources to provide integrated product sets to users SA Mode (In NESDIS): - Data Fusion of all sensors, -Every ½ hour globally SA Environment Analysis Geophysical products (Data Fusion) Common Data Assimilation & Data Fusion Tool - Combine DA and RS Expertise - Highly flexible to serve as - Platform for O2R/R2O - Complete Analysis (atmosphere, cryosphere, ocean, land, hydrometeors, trace gases etc) NG Environment Analysis Geophysical products (Data Fusion) NG Mode (In NWS): - Closely tied to Forecast Model, - Every 6 hours Forecaster This project aims to merge Remote Sensing and DA Expertise for both Data Fusion and DA Purposes 38

39 NCEP Coupled Hybrid Data Assimilation and Forecast System NEMS NEMS OCEAN SEA-ICE WAVE LAND AERO ATMOS ATMOS AERO LAND WAVE SEA-ICE OCEAN Coupled Model Ensemble Forecast Coupled Ensemble Forecast (N members) Data Assimilation Coupled Model Ensemble Forecast Ensemble Analysis (N Members) INPUT OUTPUT Courtesy: Suru

40 Joint Effort for Data assimilation Integration (JEDI) A new DA code architecture Modular, flexible, object-oriented code Model-agnostic components Shareable, interchangeable, plug-n-play objects Minimize code redundancies Overarching goals Collaborative community operation & research effort Accelerate development of new features and sensors Facilitate strongly coupled DA Improve readability, maintenance and testing Simplify code optimization on new platforms

41 Conclusion JOINT CENTER FOR SATELLITE DATA ASSIMILATION Source: Will McCarty (NASA/GMAO)

42 Satellite Data Assimilation Click to edit Master text styles Second level Third level Questions? Fourth level» Fifth level Nov 11 th 23, Annual 2015 NOAA/NESDIS CoRP Science Symposium College Park, MD. Sept. 16 th, 2015.

43 splitting supercell thunderstorms Cold-pools from isolated storms ahead of the cold front 3 km global MPAS Source: simulation MPAS. Bill Skamarock

44 Major Milestones (over-)simplified 1990 s: Variational DA, Assimilation of radiances 2000 s: 4-dimensional DA algorithm (4DVar), VarBC 2010 s: Ensemble Covariances, Cloud-affected radiances Observation Model simulated Source: ECMWF

45 Introduction hpa RMSE Magnusson and Källen 2013 de/dt=(αe+β)(1-e/e ) Chaotic ~ Model error contribution α ~ 0.4/day β ~ 2.4m/day E ~ 115m

46 GMI Data Assimilation Coordination with GMAO JCSDA began optimizing the assimilation of GMI data in GDAS. A new QC subroutine has been developed to filter out cloud and precipitation contaminated observations from GMI data for clear sky data assimilation. Bias correction, observation errors, and QC routines continue to be optimized, and forecast impacts are being assessed. (Preliminary but Encouraging Results) Anomaly Correlation 850 hpa Wind, N. Hemisphere: Satellite data assimilated. Forecast Preliminary assessment shows the assimilation of GMI data changing forecast hurricane tracks and the skill scores of forecast variables. Results are mixed, sample sizes thus far have been small. Work is ongoing to isolate and qualify forecast impacts. Control Forecast Hurricane Tracks, Hurricane Julio, with and without GMI Assimilated: No satellite data assimilated. Forecast period Experiment GMI Assimilated

47

48 Big Data Paradigm Meteorology has faced massive data issues for some time Supercomputers do not compensate for time constraints Big data is good: robustness, anchoring via redundancy Volume, Velocity, Variety, Variability, Complexity Source: Météo-France

49 Bias Correction for SSMIS F18 O-B Before Bias Correction O-B Before Bias Correction Global Using Met Office SSMIS Bias Correction Predictors Dsc Asc Unbias & Bias Corrected O-B T (K) T (K) O-B After Bias Correction O-B After Bias Correction Global Ascending Node Descending Node Latitude Asc Dsc Courtesy: Andrew Collard T (K)