Observing System Simulation Experiments (OSSEs) with Radio Occultation observations Lidia Cucurull Deputy Director, NOAA Quantitative Observing System Assessment Program (QOSAP) NOAA OAR Principal Investigator for GNSS-RO NOAA OAR Atlantic Oceanographic and Meteorological Laboratory (AOML) Robert Atlas Director, NOAA OAR Atlantic Oceanographic and Meteorological Laboratory (AOML) Director, NOAA Quantitative Observing System Assessment Program (QOSAP) Joseph Pica COSMIC/IROWG, Estes Park, 21-27 September 2017 Director, NOAA NWS Office of Observations 1
Motivation The COSMIC Radio Occultation (RO) constellation (launched in April 2006) is already past the end of its nominal lifetime and the number of soundings are rapidly declining. The U.S. Government, in partnership with Taiwan, has planned for the COSMIC-2 mission. - COSMIC-2A: six satellites in a low-inclination orbit for dense equatorial coverage (Q3FY18 estimated launch, additional ionospheric payloads) - COSMIC-2B: six satellites in a high-inclination orbit for global coverage Request for OSSEs to determine the potential value of proposed RO constellations (including COSMIC-2A and COSMIC-2B) for current operational numerical weather prediction systems requested by NWS, NESDIS, and the U.S. Congress. 2
Outline NOAA conducted comprehensive RO experiments with a previously existing global OSSE system and a newly developed, rigorous OSSE system results with both OSSE systems will be presented in this talk - earlier OSSE system (T511 ECMWF nature run and older DA system). - state-of-the-art OSSE system (NASA/GMAO G5NR nature run and a more current DA system). Work conducted under the auspices of the NOAA Quantitative Observing System Assessment Program (QOSAP). 3
Primary Objectives QOSAP Primary Objectives QOSAP s primary objective is to improve quantitative and objective assessment capabilities to evaluate operational and future observation system impacts and trade-offs to assess and to prioritize NOAA s observing system architecture. Increase NOAA s capacity to conduct quantitative observing system assessments. Develop and use appropriate quantitative assessment methodologies. Inform major decisions on the design and implementation of optimal composite observing systems. 4
QOSAP functions QOSAP Functions QOSAP coordinates the assessment of the impact of current and new observations across the different NOAA Line Offices (JPL is now a partner as well). QOSAP uses observing system experiments (OSEs) and observing system simulation experiments (OSSEs) as effective techniques to evaluate the impact of the different observation types. These studies can help NOAA management prioritize mission designs in a cost-effective way by analyzing tradeoffs in the design of proposed observing systems. QOSAP maintains an inventory of OSSEs and OSEs. 5
Earlier OSSE system: experiment design T511 ECWMF Nature Run (~ 40 km, 91 vertical levels). T382 NCEP s global data assimilation system (~50 km, 64 vertical levels) with non-hybrid GSI analysis (2012 configuration version). RO observation (and forward operator) is refractivity, assimilation stops at 30 km. Assimilation time window is 6 hours (±3 hours from the analysis time). All observations are assumed error-free except for satellite radiances, which have biases added on, using geographical locations from July-August 2012. GPS and GLONASS transmitter satellites are considered total of ~12,000 COSMIC-2 profiles/day (6,000 from COSMIC-2A and 6,000 from COSMIC-2B). Experiments ran from 1 July 2005 to 30 August 2005, verification period is 20 July to 30 August 2005. Two sets of COSMIC-2 impact experiments: - Adding RO to a poor baseline configuration that assimilates surface pressure observations only impact isolated from other observing systems - Adding RO to a simplified version of an operational data assimilation system impact of added value beyond the conventional and satellite data Investigate sensitivity of our results to the number of COSMIC-2 soundings assimilated in both sets of experiments (not shown in this presentation). 6
Baseline configuration: Positive impact from COSMIC-2B is much larger than COSMIC-2A BASE: surface pressure observations BC2EQ: BASE + COSMIC-2A BC2PO: BASE + COSMIC-2B NH SH BC2: BASE + COSMIC-2A + COSMIC-2B NR: perfect observations of temperature, moisture and wind Cucurull, Li, Peevey 2017 (MWR) 7
Simplified operational configuration: Positive impact from COSMIC-2B is larger than COSMIC-2A CTL: operational satellite radiances and conventional observations C2EQ: CTL + COSMIC-2A C2PO: CTL + COSMIC-2B NH SH C2: CTL + COSMIC-2A + COSMIC-2B NR: perfect observations of temperature, moisture and wind Cucurull, Li, Peevey 2017 (MWR) 8
Simplified operational configuration: Positive impact from COSMIC-2B is larger than COSMIC-2A Cucurull, Li, Peevey 2017 (MWR) 9
Main conclusions utilizing earlier OSSE system In agreement with earlier studies using real RO observations, the benefits from assimilating COSMIC-2 observations were found to be most significant in the Southern Hemisphere extra-tropics. With the model resolution and data assimilation configuration used in this study, and for the verification metrics investigated here, no or very little gain in global forecast skill was found by adding COSMIC-2A to COSMIC-2B, indicating that the global coverage of the polar orbiting COSMIC-2B is more impactful for global terrestrial weather forecasting than COSMIC-2A which is focused on the tropics. As COSMIC-2 soundings were added to the assimilation data stream, no saturation in skill was observed, indicating that further improvements might be obtained with the addition of new RO and/or other observations, such as wind profiles from space. 10
Weather Bill Law HR353 May 18, 2017 11
State-of-the-art OSSE system: experiment design NASA GMAO G5NR Nature Run (non-hydrostatic, ~ 7 km, 72 vertical levels). NCEP s model configuration used in our experiments very similar to what was operational in 2015 (Q1FY15, Jan 2015 implementation) but at a lower resolution of T670/T254 (~ 27 km/50 km). RO observation (and forward operator) is refractivity, assimilation stops at 30 km. Assimilation time window is 6 hours (±3 hours from the analysis time). Observations were simulated from the NASA GMAO G5NR NR and using the 2014 global observing system configuration. Errors were added for all simulated observations based on realistic error characterization and appropriately tuned for the OSSE system. The RO observations used in these experiments were simulated with the geographic sampling expected from the Constellation Observing Satellites for Meteorology, Ionosphere, and Climate-2 (COSMIC-2) system, with 6 equatorial (total of ~6,000 soundings/day) and 6 polar (total of ~6,000 soundings/day) receiver satellites. Time period: 1 August 30 September 2006 (1-31 Aug for model spin up; 1-30 Sep for verification of results). 12
Anomaly Correlation: 500-hPa Heights NHX (20N 80N) SHX (20S 80S) CTL_NORO: operational satellite radiances and conventional observations C2REF_ERR: CTL_NORO + constellation of 12 RO satellites (COSMIC-2 orbit configuration) 13
RMSE: 200-hPa Wind NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 14
RMSE: 850-hPa Wind NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 15
Main conclusions utilizing the state-of-the-art OSSE system In general, increasing the number of assimilated RO to 12 COSMIC-2 receiving satellites improves global weather forecasts. The improvement is greatest in the Southern Hemisphere extra-tropics and tropics. RO (COSMIC-2) observations increased the length of the reliable forecast by 0.6 hours in the Northern Hemisphere extra-tropics (a negligible 0.4% improvement), 5.9 hours in the Southern Hemisphere extratropics (a significant 4.0 % improvement) and 12.1 hours in the tropics (a substantial 28.4% improvement). [Note: improvement in the extra-tropics is reported as an increase in the forecast time to reach AC=0.8 for the 500 ha geopotential heights. Improvement in tropics is reported as an increase in the forecast time to reach RMSE= 6 m/s for the 200 hpa winds]. Assimilation of RO data into global and regional hurricane numerical weather prediction systems can lead to meaningful improvement in some hurricane track forecasts (not shown). No improvement of hurricane intensity and precipitation forecasts through global assimilation of RO (COSMIC-2) data was demonstrated for the cases studied. The impact on severe local storm indices was mixed. Future planned experiments will examine other cases and the impact of regional assimilation of RO data on hurricane intensity and severe local storm forecasts. Results are posted on the QOSAP web site (http://www.aoml.noaa.gov/qosap/). Two manuscripts in preparation: Cucurull et al. 2017 (MWR), Mueller et al. 2017 (MWR). 16
Additional RO experiments requested by NWS 1. OSSE I: evaluate impact of COSMIC-2A (6 satellites) 2. OSSE II: evaluate impact of COSMIC-2A (6 satellites) and COSMIC-2B (6 satellites) 3. OSSE III: evaluate impact of COSMIC-2A (6 satellites) and COSMIC-2B (4 satellites) 4. OSSE IV: evaluate impact of COSMIC-2A (6 satellites) and COSMIC-2B (6 satellites), but with COSMIC-2B at a lower (COSMIC) level of accuracy 17
List of experiments CONTROL: assimilates all observations assimilated in operations in 2014, but does not assimilate RO observations COSMIC2A: CONTROL with COSMIC-2A (6 satellites) COSMIC2: CONTROL with COSMIC-2A (6 satellites) and COSMIC-2B (6 satellites) COSMIC2_4PO: CONTROL with COSMIC-2A (6 satellites) and COSMIC-2B (4 satellites) COSMIC2_C1PO: CONTROL with COSMIC-2A (6 satellites) and a lower (COSMIC) level accuracy of COSMIC-2B (6 satellites) Note: Quality control algorithms were modified in COSMIC-2 experiments to allow deeper penetration in the lower troposphere, according to expected performance. The impact of lower instrument errors in COSMIC-2 is negligible when compared to the total observation error (instrument + retrieval + representativeness) used in the assimilation algorithms. 18
Anomaly Correlation: 500-hPa Heights OSSE I & OSSE II NHX (20N 80N) SHX (20S 80S) 19
RMSE: 200-hPa Wind OSSE I & OSSE II NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 20
RMSE: 850-hPa Wind OSSE I & OSSE II NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 21
Anomaly Correlation: 500-hPa Heights OSSE III & OSSE IV NHX (20N 80N) SHX (20S 80S) 22
RMSE: 200-hPa Wind OSSE III & OSSE IV NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 23
RMSE: 850-hPa Wind OSSE III & OSSE IV NHX (20N 80N) TRO (20N 20S) SHX (20S 80S) 24
Main conclusions utilizing the state-of-the-art OSSE system (COSMIC-2 suite of experiments) The largest benefit from COSMIC2-A assimilation is to improve tropical winds. Increasing the number of assimilated RO receiving satellites and spatial coverage from COSMIC-2A to COSMIC- 2 improves global weather forecasts. The improvement is greatest in the Southern Hemisphere extra-tropics and Tropics. The largest impact from reducing COSMIC-2B from 6 to 4 satellites is to slightly degrade weather forecast skill in the Northern Hemisphere extra-tropics. The impact of degrading COSMIC-2B to COSMIC level of accuracy on weather forecast skill is neutral in the extra-tropics. The lower level of accuracy appears to slightly improve tropical winds, perhaps due to oversaturation of the system with the larger number of observations in the tropical latitudes with COSMIC-2A and COSMIC-2B level of accuracy, and the lower resolution of the DA system used in the experiments. Benefits from COSMIC-2A or COSMIC-2B on regional and hurricane models have not yet been evaluated. The impact from COSMIC-2A and COSMIC-2B have not been quantified for Space Weather applications. 25
Summary NOAA conducted comprehensive experiments with a previously existing global OSSE system to determine the potential value of RO constellations for current operational numerical weather prediction systems. Results have been extended to a newly developed OSSE system. Further development of the new state-of-the art OSSE system for weather forecasting applications is underway. Existing ongoing and planned experiments to evaluate the impact of different global observing systems in NOAA s operational models, as requested by NWS, U.S. Congress and NESDIS. As the new OSSE system is being refined, we are improving the simulation of the RO observations (e.g. bending angle) and the characterization of COSMIC-1 versus COSMIC- 2 level of accuracy, and we plan to repeat the experiments with the new state-of-the-art OSSE system. 26