ATOVS Level 2 Product Guide

Transcription

1 EUMETSAT Doc.No. : Am Kavalleriesand 31, D Darmstadt, Germany Tel: Issue : v2 Fax: Telex: metsat d Date : 22 July

2

3 Welcome to the. As a potential user of ATOVS Level 2 products, you will find here information to familiarise yourself with the ATOVS instruments, the data processing, end-product contents and format, and potential usage and applications. A supplement of appendices applicable to all the Product Guides is also available. This contains a product summary and details of generic data, as well as information on the Metop operational orbit, and a list of acronyms and abbreviations. The supplement is accessible under Document Reference: EUM/OPS-EPS/MAN/08/0034 or electronically via the following Hummingbird link: DOCSLIB-# Common Appendices for EPS Product Guides Page 3 of 76

4 Document Change Record Version/Date Section Description of change v1 17/12/2004 Full document First issue of the document v2 22/07/2009 Full document Update to reflect start of Metop/ ATOVS Level 2: Sec. 2: Added references RD31, RD32, RD33, RD47 and RD48. Sec. 3: Revised to reflect configuration history. Sec. 5: EPSView description replaced by brief text on generic tools. (Also minor associated updates to Sec. 2 & 6.) Sec. 6: - Provided more details of WMO and HDF formats. - Sec deleted because reduced products in BUFR format no longer applicable. Sec. 7: Rewritten to provide more details. Sec. 11 (formerly Annex 7): Tables added summarising record contents for each product format version; separate versions (3 & 4) for MDR record tables; enhanced description of FLG_RETBOU bitfield. Sec. 12 BUFR Descriptor Sequences (formerly Annex 8) deleted. Other general textual & layout improvements and typo corrections. Page 4 of 76

5 Table of Contents 1 Introduction Reference Documents EPS programme documents SAF documents Papers, reports and other technical documentation ATOVS Level 2 Products Configuration History ATOVS Level 2 Products Overview The ATOVS and AVHRR/3 instrument suite Instruments technical description and spectral characteristics Scanning characteristics Instruments calibration Overview of the ground processing Input Data Vector Preparation Function ATOVS Level 2 retrieval ATOVS Level 2 product characteristics and use General characteristics Quality information in the products Summary of ATOVS Level 2 product current and potential applications Data Viewing and Reading ATOVS Level 2 Product Formats and Dissemination EPS products available dissemination means Satellite Direct Broadcast Service EUMETCast GTS/RMDCN UMARF ATOVS Level 2 products dissemination Near-real-time dissemination Archive retrieval ATOVS EPS native product formats The EPS native formats The (Full) ATOVS Level 2 product format The HDF formats The WMO formats ATOVS Level 2 Product Processing Algorithms ATOVS Level 2 processing details Data reception Cloud mask Retrieval Cloud Top Pressure Temperature and water vapour Water calculation Partly cloudy retrieval AMSU-A stand-alone Quality control ATOVS Level 2 Products Validation ATOVS Level 2 Products Routine Monitoring ATOVS, IASI and AVHRR Processing Chain Inter-dependencies Record Description of the ATOVS Level 2 Product MPHR ( name 'mphr', class 1, subclass 0, version 2 ) GIADR (name 'giadr-levels', class 5, subclass 1, version 3 ) GIADR ( name 'giadr-config', class 5, subclass 2, version 2 ) MDR ( name 'mdr-2', class 8, subclass 1, version 3 ) MDR ( name 'mdr-2', class 8, subclass 1, version 4 )...56 Page 5 of 76

6 1 INTRODUCTION This user guide is intended for users of EPS ATOVS Level 2 products. It provides information about the products available, how to access them, how to extract and interpret the data, and it also aims to help the user in choosing a product for a particular application. In Appendix A, a full list of EPS products generated at EUMETSAT is given. The products that will be addressed in this guide are: ATOVS Level 2 full product: o Temperature Profiles o Humidity Profiles o Surface Temperature o Surface Emissivity o Cloud Top Temperature o Cloud Top Pressure o Cloud Liquid Water Content o Total Column Precipitable Water The above products will be generated by the EPS CGS from instruments on board the Metop and the NOAA platforms. Note that EUMETSAT generates Level 2 from ATOVS Level 1b products supported by AVHRR cloud/scenes analysis, which are also generated and distributed to users. If you are interested in ATOVS radiance products, their processing, formats, dissemination means and applications, you might want to check the ATOVS Level 1b Product Guide instead. An essential part of the ATOVS Level 2 processing is the RTTOV7 Fast Radiative Transfer Model, developed by the Numerical Weather Prediction Satellite Application Facility (NWP SAF). For further questions not addressed in this guide, on these or other EPS products, users are welcome to access the EUMETSAT Polar System pages on our website or to contact directly the EUMETSAT User Services Helpdesk. These pages should be the main interface for information on access to all EPS products. Comprehensive information on the NWP SAF and their products and activities can also be found on the EUMETSAT website, and on the NWP-SAF webpage Page 6 of 76

7 2 REFERENCE DOCUMENTS The following documents have been used to compile the information in this guide. Some of them are referenced within the text, others are provided here for further reading. 2.1 EPS programme documents [RD11] EPS Generic Product Format Specification EPS/GGS/SPE/96167 [RD12] ATOVS Level 2 Product Format Specification EPS/MIS/SPE/ [RD13] ATOVS Level 2 Product Generation EUM.EPS.SYS.SPE Specification [RD18] ATOVS Calibration and Validation Plan EUM.EPS.SYS.PLN [RD19] EPS Programme Calibration and Validation EUM.EPS.SYS.PLN Overall Plan [RD21] U-MARF LEO Format Descriptions EUM/OPS/USR/06/1855 [RD22] EUMETCast Technical Description EUM TD 15 [RD23] AVHRR Level 1 Product Generation EUM.EPS.SYS.SPE Specification [RD24] AMSU-A Level 1 Product Generation EUM.EPS.SYS.SPE Specification [RD25] MHS Level 1 Product Generation Specification EUM.EPS.SYS.SPE [RD26] HIRS Level 1 Product Generation Specification EPS.SYS.SPE [RD27] AMSU-A Level 1 Product Format Specification EPS.MIS.SPE [RD28] MHS Level 1 Product Format Specification EPS/MIS/SPE/97229 [RD29] HIRS Level 1 Product Format Specification EPS/MIS/SPE/97230 [RD30] AVHRR Level 1 Product Format Specification EPS.MIS.SPE [RD31] EPS Product file naming for EUMETCast EUM/OPS-EPS-TEN/07/0012 [RD32] EPS CGS Quality Control Facility Detailed EPS-ACS-DD-0011 Design Document, Volume 3: EPS Algorithms [RD33] Metop Space to Ground Interface Specification MO-IF-MMT-SY0001 Future versions of this document will be available on the EUMETSAT UMARF webpages. On EUMETSAT Metop - AHRPT documentation webpage. See for more information on the project. 2.2 SAF documents See for more information on the NWP SAF project. Page 7 of 76

8 2.3 Papers, reports and other technical documentation [RD41] NOAA KLM User s Guide [RD42] The ATOVS and AVHRR Processing Facility for EPS [RD43] Automatic Cloud Detection Applied to NOAA-11/AVHRR Imagery [RD44] Inversion of cloudy satellite sounding radiances by nonlinear optimal estimation. I. Theory and simulation for TOVS and II. Application to ATOVS data [RD45] Manual on the Global Telecommunication System [RD46] World Meteorological Organization Manual on Codes [RD47] Inverse Methods for Atmospheric Sounding [RD48] Determination of precipitable water and cloud liquid water over oceans from the NOAA 15 advanced microwave sounding unit www2.ncdc.noaa.gov/docs/klm Klaes, K.D., Ackermann, J, Schraidt, R, Patterson, T, Schlüssel, P, Phillips, P, Arriaga, A, and Grandell, J. Technical Proceedings of the 13th International TOVS Study Conference, St. Adele, Canada, 28 Oct. - 5 Nov Derrien, M., Farki, B., Hrang, L., LeGleau, H., Noyalet, A., Pohic, D., Sairouni, A. Remote Sens. Environ. 46, , Eyre, J.R. Q.J.R. Meteorolog. Soc.115, , WMO - No. 386 WMO - No. 306 Rodgers, C. D. World Scientific, Singapore, Grody, N.C. et al. J. Geophys. Res. 106, , Page 8 of 76

9 3 ATOVS LEVEL 2 PRODUCTS CONFIGURATION HISTORY In the following table the current version on the operational Ground Segment is shown on a white background. Date introduced Product format version PFS version PGS version Major number Minor number [RD12] [RD13] 19/10/ /08/ /7B 5.3/6 Comments Table 3-1: ATOVS Level 2 document versions Page 9 of 76

10 4 ATOVS LEVEL 2 PRODUCTS OVERVIEW 4.1 The ATOVS and AVHRR/3 instrument suite The ATOVS (Advanced TIROS (Television and Infrared Observational Satellite) Operational Vertical Sounder) is a sounding instrument package first flown on the NOAA-KLM satellite series. It was originally composed of the Advanced Microwave Sounding Units A and B (AMSU-A, AMSU-B), complemented by the High Resolution Infrared Radiation Sounder (HIRS/3). For the Metop and the NOAA N-N' satellite series, the AMSU-B sounder has been replaced by the Microwave Humidity Sounder (MHS), and the infrared sounder has been upgraded to HIRS/4, though HIRS/4 will not be flown on Metop-C. Although not considered formally part of the ATOVS package, the Advanced Very High Resolution Radiometer (AVHRR/3) is an imager also flying on Metop, NOAA-18 and NOAA-19, which supports the ATOVS Level 1b processing. A separate AVHRR Level 1b Product Guide is provided and AVHRR/3 will be discussed here only in the context of its support to the ATOVS products processing. A detailed account of the ATOVS instruments technical characteristics is given in the various ATOVS L1 Product Generation Specifications ([RD24], [RD25], [RD26]), the ATOVS Level 1b Product Guide and in the NOAA KLM Users s Guide [RD41]. We will provide in this guide the basic information necessary for the product understanding and usage Instruments technical description and spectral characteristics The AMSU-A is a fifteen-channel microwave radiometer that is used for measuring global atmospheric temperature profiles and providing information on atmospheric water in all of its forms. AMSU-A measures in 15 spectral bands, where the temperature sounding mainly exploits the oxygen band at 50 GHz. The MHS is the follow-on instrument for the Advanced Microwave Sounding Unit-B (AMSU-B) flying as a part of ATOVS on the NOAA-18, NOAA-19 and Metop satellites. MHS is a 5-channel microwave radiometer, which complements the Advanced Microwave Sounding Unit-A (AMSU-A) channels. It is planned to derive from these frequencies humidity profiles, cloud liquid water content and precipitation. Additionally, its sensitivity to large water droplets in precipitating clouds can provide a qualitative estimate of precipitation rates. It is technically similar to the AMSU-B instrument, except for channel 5, where the AMSU-B side-band at GHz is missing. The HIRS/4 instrument is an infrared sounder that measures the incident radiation primarily in the infrared region of the spectrum in 19 channels. It also has one channel in the visible. Its purpose is to measure temperature and water vapour profiles under clear-sky or cloudy conditions. The AVHRR/3 is a multipurpose imaging instrument used for global monitoring of cloud cover, sea surface temperature, ice, snow and vegetation cover characteristics and is currently flying on the Metop-A satellite and the NOAA POES satellites (NOAA-15 to NOAA-19). The AVHRR/3 is a 6-channel scanning radiometer providing three solar channels in the visible-near infrared region and three thermal infrared channels. However, only five channels are transmitted to the ground at any given time. On Metop-A, channel 3a (1.6 µm) is operated Page 10 of 76

11 during the daytime portion of the orbit and channel 3b (3.7 µm) during the night-time portion. On the NOAA-18 and NOAA-19 satellites, the AVHRR/3 product used is the GAC product, and channel 3b is operated at all times Scanning characteristics All ATOVS and AVHRR/3 instruments are cross-track scanning radiometers. The ATOVS instruments scan from West to East for ascending passes, and the AVHRR/3 scans instead from East to West. The IFOV and sampling interval for the different instruments is different, as summarised in the following tables. In Figure 4-1 the spot of AMSU-A (red), MHS (green), HIRS/4 (blue) and IASI (yellow) is illustrated for a near nadir location. The spot s for instruments at the edge of an AMSU-A scan line are shown in Figure 4-2. Instrument IFOV type IFOV (deg) Sampling interval (acrosstrack) (deg) IFOV (nadir) (km) Samples per scan line Scan separation (km) Swath width (km) Scan type AMSU-A circular ± step and stare HIRS/4 circular ± step and stare MHS circular ± continuous Table 4-1: Summary of ATOVS instruments scanning characteristics Page 11 of 76

12 Figure 4-1: Simulated earth-surface footprints for AMSU-A (red), MHS (green), HIRS/4 (blue) and IASI (yellow) near nadir. Distances are given in kilometres. Figure 4-2: Simulated earth-surface footprints for AMSU-A (red), MHS (green), HIRS/4 (blue) and IASI (yellow) at the edge of an AMSU-A scan line. Distances are given in kilometres. Page 12 of 76

13 Instrument (processing chain) AVHRR/3 (Metop) AVHRR/3 (NOAA GAC) IFOV type IFOV (deg) IFOV sampling (deg) Scan separation (km) Samples per scan line IFOV (nadir) (km) Sampling grid (nadir) (km) Swath width (km) Scan type square x x 1.1 ± continuous square (3 x ) (acrosstrack) x (alongtrack) 3 x 1.1 (only every third scan line) (acrosstrack) x 1.1 (alongtrack) 5.5 (acrosstrack) x 3.3 (alongtrack) ± continuous Table 4-2: Summary of AVHRR/3 instruments (Metop and NOAA) scanning characteristics The NOAA-GAC frame rate is only one third of the AVHRR/3 frame rate. The NOAA-GAC processing of the AVHRR/3 data makes the frame rates directly compatible by only using the data from every third AVHRR/3 scan. A further reduction is achieved by averaging the value of four adjacent samples and skipping one sample of each channel of AVHRR data across each scan line used. The GAC output is not available for direct readout users. The NOAA GAC footprint and overlap of the full resolution Metop AVHRR/3 footprints are shown for five scan lines near nadir in Figure 4-3. Figure 4-3: Simulated earth-surface footprints for AVHRR/3. The GAC coverage is shown in blue. The full resolution AVHRR/3 footprints are shown by black lines. The overlay of the AVHRR/3 field of views across track is indicated by the reddish coloured footprints. Distances are given in kilometres. Page 13 of 76

14 4.1.3 Instruments calibration Concerning the calibration of the infrared and microwave channels, all sensors rely on a selfcalibrating system, providing regular views of an internal warm target (of a temperature monitored by several platinum resistance thermometers) as well as of cold space. Visible and near-infrared channels are calibrated on ground prior to launch and no on-board self calibration is made. In the case of AVHRR/3 though, a vicarious calibration against stable surface regions and against other satellite measurements is foreseen as part of the validation activities. 4.2 Overview of the ground processing The objective of the ATOVS Level 2 ground processing is the collocation and further processing of ATOVS and AVHRR/3 calibrated radiances for the purpose of the generation of higher level geophysical parameters, namely: Temperature Profiles Humidity Profiles Surface Temperature Cloud Top Temperature Cloud Top Pressure Effective Cloud Amount Cloud Liquid Water Content Total Column Precipitable Water Input to the ATOVS Level 2 processor is: The ATOVS and AVHRR/3 Level 1b data flows The results of the AVHRR/3 scenes analysis within the AVHRR/3 Level 1 processing chain Numerical Weather Prediction forecast information Other auxiliary and configuration data In Section 10, the context of the ATOVS, IASI and AVHRR processing chain interactions is provided for information. The ATOVS Level 2 ground processing at EUMETSAT is applied to data from the ATOVS instruments on both Metop and NOAA satellites. The following processing steps are performed Input Data Vector Preparation Function The ATOVS instruments (AMSU-A, HIRS/4, MHS) and AVHRR/3 have different sampling geometries (see Figure 4-1). For that reason, a first step of the ATOVS L2 processing is the collocation and mapping of the Level 1 data. The default reference grid onto which all the data are mapped is the HIRS/4 sampling grid. The vertical grid is determined from the available input satellite data and all other required input data are mapped onto it. Further data preparation includes, in the case of the microwave instruments AMSU-A and MHS, the detection of contamination effects through ice cloud particles and precipitation, and the detection of the surface type. In the case of the visible and infrared instruments, AVHRR/3 and HIRS/4, the data preparation includes a cloud analysis, performing an Page 14 of 76

15 AVHRR/3 scenes analysis to specify a cloud mask and to characterise the surface. The scenes analysis results are mapped to the HIRS/4 field of views. (Note that the AVHRR scenes analysis is performed during AVHRR, not ATOVS L2, processing.) The AMSU-A, MHS and HIRS/4 data preparation function includes the calculation of brightness temperatures using the Planck function. A scan angle-dependent bias correction is further applied to the measurements. The detection of contamination effects on the microwave instruments is based on the fact that the signatures of rain and ice particles are provided through absorption for frequencies lower than 60 GHz (AMSU-A channels 1 to 14) and through scattering processes for higher frequencies (AMSU-A channel 15 and MHS channels 1 to 5). Scattering index calculation and precipitation tests are carried out for that purpose, supported by a surface analysis function based on AMSU-A and MHS data co-registered onto the AMSU-A sampling grid using a spatial weighted average interpolation method. Additional checks for contamination are applied to MHS data by running a median filter to detect potential spikes, and to AMSU A data by examining the range of MHS brightness temperature values within an AMSU-A FOV. In the case of the infrared and visible instruments, the scenes analysis results from AVHRR/3 Level 1b processing are examined for consistency. If this is not available from AVHRR, a HIRS/4-only cloud analysis function is triggered, based on different threshold tests on single or multi-channel combinations. Additionally, determination of the land/sea distribution for all instruments based on an Earth topography data set is performed, all data and flags are mapped to the retrieval grid (i.e., HIRS/4 sampling grid) and the Retrieval Input Vector for the retrieval process is created. The mapping method onto the HIRS/4 sampling grid is different for different instruments, and chosen according to the individual instrument resolutions and spatial samplings. For AMSU-A, bilinear interpolation is used. For MHS, a spatial weighted average is used. For AVHRR/3, the same collocation method used for the HIRS/4 Level 1b cloud analysis is used (see the ATOVS Level 1b Product Guide for more details) ATOVS Level 2 retrieval The retrieval step performs first retrieval-method-dependent data preparation and corrections. The input vector is examined and a decision is taken whether synergistic retrieval can be performed using all sounding instrument data. Then, the complete retrieval input is examined for cloud coverage and a decision is taken whether the retrieval will be performed in cloudy or partly cloudy cases. Finally, an initialisation step for the retrieval follows, based on NWP forecast data. The retrieval is an iterative process which includes radiative transfer and error covariance calculations, and is based on solving a Fredholm integral equation of the first kind [RD44]. The Fast Radiative Transfer Model (FRTM) used follows RTTOV7 (see the web page of the NWP SAF for access to full RTTOV7 documentation). Page 15 of 76

16 4.3 ATOVS Level 2 product characteristics and use General characteristics Table 4-3 summarises the main characteristics of ATOVS Level 2 products available to users. All products contain quality control and other information about the retrieval and their use, which are important to know when you choose the product needed for your application. Two different types of Level 2 products are generated, from Metop and from NOAA data. Instrument Product Main geophysical parameter Accuracy Grid spacing Vertical sampling Swath width Coverage Generated ATOVS and AVHRR/3 Full ATOVS Level 2 from Metop and Full ATOVS Level 2 from NOAA Atmospheric Temperature Atmospheric Water Vapour Surface Temperature Surface Emissivity 1.7 K Troposphere, 2 K Stratosphere 20% HIRS/4 instrument horizontal sampling grid: 56 views per scan and 0.6 K km scan n/a (not a retrieved parameter) separation Typical number of pressure levels: 40 Typical number of pressure levels: 15 n/a n/a ±1080 km Global and continuous EPS CGS Fractional Cloud Cover 5-10% n/a Cloud Top Temperature 1-2 K n/a Cloud Top Pressure 50 hpa n/a Tropopause Height 50 hpa n/a Cloud Liquid Water Content 0.04 mm Typical number of levels : 1 Total Column Precipitable Water 5% n/a ATOVS and AVHRR/3 Reduced ATOVS Level 2 from Atmospheric Temperature 1.7 K Troposphere, 2 K Stratosphere Every 4th HIRS/4 FOV and Pressure levels for atmospheric temperature Page 16 of 76

17 Metop and Reduced ATOVS Level 2 from NOAA Atmospheric Water Vapour Surface Temperature 20% every 2nd HIRS/4 scan line 0.6 K and water vapour profiles: every 4th level of the full product Table 4-3: Summary of the main characteristics of ATOVS Level 2 products (Note that the figures on accuracy are, for this issue of the, pre-launch estimations. For subsequent post-launch information, please refer to relevant reports on the EUMETSAT Product Validation Reports webpage.) Apart from the main geophysical parameters given per each HIRS/4 field of view (FOV), navigation information per scan line and geolocation information for every FOV is given, as well as angular relations for every navigation point. The mapped brightness temperatures for all microwave and IR channels are also given, as well as the HIRS/4 channel 20 radiance. Finally, an important piece of information included in the product is the associated retrieval error, in the form of a compressed error covariance matrix Quality information in the products A number of quality flags are generated during the Level 2 processing, associated with individual scan lines. The following are the most relevant with respect to data use. All quality flags are mapped onto the retrieval grid. A full list and detailed explanation of all flags is later given in the ATOVS Level 2 products content and format description in Section 11. Retrieval rejection flag, indicating whether the inversion was OK or otherwise rejected. Surface type Flags indicating the results of contamination assessment (scattering index, precipitation probability, MHS median test) Flag indicating the results of the HIRS/4 stand-alone cloud detection Flag indicating if either cloudy or clear retrieval was performed 4.4 Summary of ATOVS Level 2 product current and potential applications Not available for this issue of the. Page 17 of 76

18 5 DATA VIEWING AND READING Readers for the native EPS format ATOVS Level 2 products are available online at the EUMETSAT website on the Useful Programs & Tools page. Tools to read HDF formats are TBD, but it is intended that the products can be read using standard HDF libraries. For more information on HDF5 formats in general, see the HDF5 webpages. Software capable of reading the WMO formats is available from a variety of sources, including ECMWF. Page 18 of 76

19 6 ATOVS LEVEL 2 PRODUCT FORMATS AND DISSEMINATION A description of the dissemination means for EPS products and formats is provided in the following paragraphs, focusing down on ATOVS Level 2 products and their formats. 6.1 EPS products available dissemination means Note that this section about dissemination means of EPS products in general could be removed when that info is available on the EPS website Satellite Direct Broadcast Service Instrument and ancillary data acquired by the Metop satellites will be broadcast and received by authorised users in real-time via: Advanced High Resolution Picture Transmission (AHRPT) - transmission of data from all Metop instruments in full resolution. The data will be received by local reception stations. It is the responsibility of the user to procure and install a local reception station. Specification documentation for a EUMETSATbased HRPT Reference User Station is available for information on the EUMETSAT webpage Metop AHRPT. The output format of the EUMETSAT HRPT Reference User Station is Level 0 products in the EPS Native format [RD11], [RD33]. The broadcast data are encrypted. To get authorisation to access the data, users need to register with the EUMETSAT User Services and will receive the data decryption information. Data from the NOAA payload are also broadcast and received by local users via the HRPT mechanism. For details on the NOAA HRPT system, the reader is referred to the NOAA KLM User s Guide [RD41] EUMETCast Global EPS products at different levels will be distributed in near real-time via EUMETSAT s Data Distribution System (EUMETCast) in Ku-band. EUMETCast utilises the services of a satellite operator and telecommunications provider to distribute data files using Digital Video Broadcast (DVB) to a wide audience located within the geographical coverage zone which includes most of Europe and certain areas in Africa. Within the current EUMETCast configuration, the multicast system is based upon a client/server system with the server side implemented at the EUMETCast uplink site (Usingen, Germany) and the client side installed on the individual EUMETCast reception stations. The telecommunications suppliers provide the DVB multicast distribution mechanism. Data/product files are transferred via a dedicated communications line from EUMETSAT to the uplink facility. These files are encoded and transmitted to a geostationary communications satellite for broadcast to user receiving stations. Each receiving station decodes the signal and recreates the data/products according to a defined directory and file name structure. A single reception station can receive any combination of the provided services. Page 19 of 76

20 A typical EUMETCast reception station comprises a standard PC with DVB card inserted and a satellite off-set antenna fitted with a digital universal V/H LNB. In addition, users require the multicast client software, which can be obtained via the EUMETSAT User Services. More detailed information on this service can be found in the EUMETSAT webpage EUMETCast Dissemination Scheme. Products distributed on EUMETCast can be formatted in a variety of formats, including EPS native format and the WMO formats (BUFR and GRIB) GTS/RMDCN A subset of EPS products will be disseminated additionally in near real-time via the Global Telecommunication System (GTS). GTS is the World Meteorological Organization integrated network of point-to-point circuits, and multi-point circuits which interconnect meteorological telecommunication centres. Its purpose is to enable an efficient exchange of meteorological data and products in a timely and reliable way to meet the needs of World, Regional and National Meteorological Centres. The circuits of the GTS are composed of a combination of terrestrial and satellite telecommunication links. Meteorological Telecommunication Centres are responsible for receiving data and relaying them selectively on GTS circuits. The GTS is organised on a three-level basis, namely: The Main Telecommunication Network, linking together 3 World meteorological centres and 15 regional telecommunication hubs. The Regional Meteorological Telecommunication Networks, consisting of an integrated network of circuits interconnecting meteorological centres in a region, which are complemented by radio broadcasts where necessary. In Europe, the GTS network is supported by the Regional Meteorological Data Communication Network (RMDCN). The National Meteorological Telecommunication Networks, which extend the GTS network down to national level. More detailed information on this service can be found on the WMO website Products distributed on the GTS are in official WMO formats, namely BUFR or GRIB UMARF All EPS products and auxiliary data are normally archived and made available to users from the EUMETSAT Unified Meteorological Archive and Retrieval Facility (UMARF) upon request. The UMARF can be accessed through the EUMETSAT webpage Archive Services. Access is through a web interface through which the users are able to browse and order products, manage their user profile, retrieve products, documentation and software libraries, get help, etc. UMARF features include geographical and time sub-setting and image preview. EPS products archived in the UMARF can be accessed in a variety of formats, including EPS native format and HDF5. Page 20 of 76

21 6.2 ATOVS Level 2 products dissemination Table 6-1 summarises the different dissemination means and formats for all ATOVS Level 2 products that are available to users. Format EPS native format Real-Time Direct Broadcast Near-Real-Time dissemination on EUMETCast (timeliness) Near-Real-Time dissemination on GTS (timeliness) UMARF retrieval (timeliness) ATOVS Level 2 from Metop and NOAA (8-9 h) HDF ATOVS Level 2 and Reduced Level 2 from Metop and NOAA (8-9 h) WMO (BUFR) -- ATOVS Level 2 from Metop and NOAA (3 h) ATOVS Level 2 from Metop and NOAA (3 h) Timeliness refers to the elapsed time between sensing and dissemination. Table 6-1: Summary of dissemination means and formats for ATOVS Level 2 products Near-real-time dissemination The ATOVS Level 2 products disseminated to users in near real-time are: Full ATOVS Level 2 BUFR products from Metop with a timeliness of 3 h from sensing on EUMETCast and GTS Full ATOVS Level 2 BUFR products from NOAA with a timeliness of 3 h from sensing on EUMETCast and GTS The dissemination granularity of the data is 3 minutes Archive retrieval The ATOVS Level 2 products available from the UMARF are: Full ATOVS Level 2 products from Metop in EPS native format or HDF5 Full ATOVS Level 2 products from NOAA in EPS native format or HDF5 The products are archived as full-dump products, but sub-setting capabilities are provided by the UMARF to the user in the retrieval step. The products are available for the users in the UMARF 8 to 9 hours after sensing. Page 21 of 76

22 6.3 ATOVS EPS native product formats The EPS native formats General overview of the EPS generic product format All products in EPS native format are structured and defined according to an EPS Generic Product Format. This format is not ATOVS specific. The general product section breakdown is given, and the following sections will focus on how this generic format is further applied to ATOVS products. This description is not aimed at supporting the writing of reader software for the ATOVS or other EPS products, because readers and product extraction tools are already available (see Section 5). The intention of this and the following sections is to provide enough information to be able to use such available tools and to interpret the retrieved information. For users interested in writing their own product readers for one or several ATOVS products in EPS native format, we refer them to the detailed format specifications provided in [RD11] and [RD12]. The general structure of the products is broken down in sections, which contain one or more records of different classes. Every single record is accompanied by a Generic Record Header (GRH), which contains the metadata necessary to uniquely identify the record type and occurrence within the product. The following general structure is followed by all EPS products, where all the sections occur always in the given order. Header Section, containing metadata applicable to the entire product. The header section may contain two records, the Main Product Header Record (MPHR) and the Secondary Product Header Record (SPHR). This is the only section that contains ASCII records; the rest of the product is in binary. Pointer Section, containing pointer information to navigate within the product. It consists of a series of Internal Pointer Records (IPR), which include pointers to records within the Global Auxiliary Data, Variable Auxiliary Data and Body Sections that follow. Global Auxiliary Data Section, containing information on the auxiliary data that have been used or produced during the process of the product and applies to the whole length of the product. There can be zero or more records in this section, and they can be of two classes: Global External Auxiliary Data Record (GEADR), containing an ASCII pointer to the source of the auxiliary data used, and Global Internal Auxiliary Data Record (GIADR), containing the auxiliary data used itself. Variable Auxiliary Data Section, containing information on the auxiliary data that have been used or produced during the process of the product and may vary within a product, but with a frequency in any case less than the measurement data itself. There can be zero or more records in this section, and they can be of two classes: Variable External Auxiliary Data Record (VEADR), containing an ASCII pointer to the source of the auxiliary data used, and Variable Internal Auxiliary Data Record (VIADR), containing the auxiliary data used itself. Body Section, which is usually the main bulk of the product and contains the raw or processed instrument data and associated information. This section contains time-ordered Measurement Data Records (MDR). A particular type of MDR can occur to indicate the Page 22 of 76

23 location of an unexpected data gap within any product, the Dummy Measurement Data Record (DMDR). The format of the MPHR, IPRs, GEADR, VEADR and DMDRs is common to all products, while the other records can be of different formats and contents, and identified as of different sub-classes for different products. Every record consists of a series of fields, which can have different data types. See Appendix C for all possible data types. It is important to note that GEADR and VEADR records are included in the products to support processing configuration control for EUMETSAT at product level. They point to the name of auxiliary data files used in the processing, but they are not of any interest or use to the end-user for the utilisation of the products. Two types of records deserve special description, because they are key to navigating within the products, namely the GRH and the IPR. Their format and the meaning of their fields are detailed in Appendix D. In particular, IPRs can be used to skip through VEADRs and GEADRs and get to the measurement data of interest to the user. Table 6-2 gives an example of the general structure of the Generic Product Format. Section RECORD CLASS RECORD SUBCLASS START TIME STOP TIME HEADER SECTION MAIN PRODUCT HEADER RECORD SECONDARY PRODUCT HEADER RECORD T1 T1 T6 T6 INTERNAL INTERNAL POINTER RECORD (GEADR Subclass A) T1 T6 POINTER INTERNAL POINTER RECORD (GEADR Subclass B) T1 T6 SECTION INTERNAL POINTER RECORD (GIADR Subclass A) T1 T6 INTERNAL POINTER RECORD (GIADR Subclass B) T1 T6 INTERNAL POINTER RECORD (GIADR Subclass C) T1 T6 INTERNAL POINTER RECORD (VEADR Subclass A) T1 T6 INTERNAL POINTER RECORD (VEADR Subclass B) T1 T6 INTERNAL POINTER RECORD (VEADR Subclass C) T1 T6 INTERNAL POINTER RECORD (VIADR Subclass A) T1 T6 INTERNAL POINTER RECORD (VIADR Subclass B) T1 T6 INTERNAL POINTER RECORD (VIADR Subclass C) T1 T6 INTERNAL POINTER RECORD (MDR Subclass A) T1 T6 INTERNAL POINTER RECORD (MDR Subclass B) T1 T6 INTERNAL POINTER RECORD (MDR DUMMY) T1 T6 INTERNAL POINTER RECORD (MDR Subclass A) T1 T6 INTERNAL POINTER RECORD (MDR Subclass B) T1 T6 GLOBAL GLOBAL INTERNAL AUXILIARY DATA RECORD SUBCLASS A T1 T6 AUXILIARY GLOBAL INTERNAL AUXILIARY DATA RECORD SUBCLASS B T1 T6 DATA SECTION GLOBAL INTERNAL AUXILIARY DATA RECORD SUBCLASS A T1 T6 GLOBAL INTERNAL AUXILIARY DATA RECORD SUBCLASS B T1 T6 GLOBAL INTERNAL AUXILIARY DATA RECORD SUBCLASS C T1 T6 VARIABLE VARIABLE INTERNAL AUXILIARY DATA RECORD SUBCLASS A T1 T6 Page 23 of 76

24 AUXILIARY DATA SECTION VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD VARIABLE INTERNAL AUXILIARY DATA RECORD SUBCLASS B SUBCLASS B SUBCLASS C SUBCLASS C SUBCLASS A SUBCLASS A SUBCLASS A SUBCLASS B SUBCLASS C T1 T3 T1 T5 T1 T2 T4 T1 T1 T3 T6 T5 T6 T2 T4 T6 T6 T6 BODY SECTION MEASUREMENT DATA RECORD MEASUREMENT DATA RECORD MEASUREMENT DATA RECORD MEASUREMENT DATA RECORD MEASUREMENT DATA RECORD SUBCLASS A SUBCLASS B DUMMY SUBCLASS A SUBCLASS B T1 T2 T3 T4 T5 T2 T3 T4 T5 T6 Table 6-2: Generalised schematic of the generic product format Granularity of the EPS products The Full EPS product is produced by processing a dump of data. This is the product used to archive in the UMARF. In addition, the Regional EPS product is a full product that has been passed through a geographical filter. This may happen, for example, during the retrieval of the product from the UMARF. Finally, a Product Dissemination Unit (PDU) is the near-real-time dissemination of the full product, and it is typically of 3 minutes. A PDU is often referred to as product granule. The EPS Generic Product Format has been defined to apply to any length of sensing. That means that the same generic format described above applies to a 3-minute duration granule, half an orbit or a full dump of data. The length in time of the product is contained in the MPHR Product format version control Every record class and sub-class has an associated record version number contained in its corresponding GRH. In addition, each product has a format version number, which is stored in the MPHR Product naming convention File naming convention for EPS products in EPS native format provides a product name that uniquely identifies any product and provides a summary of its contents. The field contents in a product name correspond to those in the MPHR. <INSTRUMENT_ID>_<PRODUCT_TYPE>_<PROCESSING_LEVEL>_<SPACECRAFT_ID> <SENSING_START>_<SENSING_END>_<PROCESSING_MODE>_<DISPOSITION_MODE> <PROCESSING_ TIME> Page 24 of 76

25 Product Name Field / MPHR Field Description Size in Characters INSTRUMENT_ID Instrument identification 4 PRODUCT_TYPE Product Type 3 PROCESSING_LEVEL Processing Level Identification 2 SPACECRAFT_IUD Spacecraft identification 3 SENSING_START UTC Time of start of Sensing Data 15 SENSING_END UTC Time of end of Sensing Data 15 PROCESSING_MODE Identification of the mode of processing 1 DISPOSITION_MODE Identification of the type of processing 1 PROCESSING_TIME UTC time at start of processing for the product 15 Table 6-3: EPS product name fields and their correspondence with MPHR fields For the ATOVS Level 2 products, the resulting product file names are as follows: Product Product name Full ATOVS Level L2 from Metop ATOV_SND_02_Mnn_<...> Full ATOVS Level L2 from NOAA ATOV_SND_02_Nnn_<...> Table 6-4: Generic ATOVS Level 2 product names The (Full) ATOVS Level 2 product format Records to be found in the ATOVS Level 2 product are: Record Name Description Usage Subclass ID MPHR Main Product Header Record Main product identification details 0 Necessary IPRs GEADRs Internal Product Record Pointers Pointers to global auxiliary data file names used in the processing (*) Necessary to access directly different records in the product Not relevant for end-user 1, 2,... not available Page 25 of 76

26 GIADR LEVELS GIADR CONFIG MDR-2 Pressure levels Necessary to interpret the main product contents Processor configuration data Not relevant for end-user 2 Same record format for products from Metop and NOAA Level 2 main product contents (see below for more details) 1 1 Table 6-5: Record types in ATOVS Level 2 product (*) A full list of GEADRs for the ATOVS Level 2 product is not available at the time of writing this version of the ATOVS Level 2 product guide. IPRs as found in the products can however be used to skip over GEADRs, as those records do not contain any information relevant for the end-user. These products are organised as successive lines of pixels along track, according to the HIRS/4 measurement horizontal grid, referenced by the orbit time that corresponds to that line of pixels. The START/STOP times indicated in the MPHR and the corresponding VIADRs, are also referenced with respect to that time. Each MDR contains data corresponding to one scan line of pixels. Pixels in one scan line are given in the direction of scanning: left to right (e.g., West to East for Northbound satellite direction). Data included in each MDR are of several types: Measurement data, including, for all 56 FOVs, the following scalar values output of the retrieval process: o surface temperature o cloud top temperature o cloud top pressure o tropopause height o total column of precipitable water and the following retrieved profile information: o atmospheric temperature o atmospheric water vapour o cloud liquid water content An additional parameter included as part of the measurement data is the surface emissivity values used in the retrieval. Navigation data corresponding to that line and geolocation of every FOV. Mapped brightness temperatures for all IR and NIR channels used, as well as the radiance value for HIRS/4 channel 20. Appended data to help interpret the results, which contain typically the retrieval Input Data Vector, i.e., among others: o surface type o land mask o percentage of AVHRR cloudy fields of view o availability of input channels o whether a clear or cloudy retrieval was performed o the scattering index Page 26 of 76

27 o precipitation probability o daytime/night-time flag Several flags and quality information. In particular, a retrieval rejection flag, indicating whether the inversion was OK The retrieval error data Concerning the dimensioning of some of those fields, note that for the profile information, the number and value of pressure levels used in the retrieval can be read in the GIADR LEVELS record, and those levels can in principle be different for temperature, humidity and cloud liquid water content profiles. The number and value of pressure levels in the FRTM is also included in the GIADR-LEVEL record for information. Additionally, the number and value of wavelengths used to derive the value of the surface emissivity is also included in the GIADR-LEVELS record. Concerning the contents of the retrieval error data, those are steered by the flag FLG_STER in the MDR, which can contain the following values: 0 when no error data is appended 1 or 2 when variances are included for the retrieval state vector 4 when the diagonal values of the inverted original covariance matrix and wavelet values are included for the retrieval state vector and for the number of wavelet coefficients used. The number of soundings in an ATOVS Level 2 product is 56 and there is one line of soundings for every HIRS/4 scan line. The swath is continuous and no interruptions are expected in nominal operation. To summarise, the occurrence of the different records in the ATOVS Level 2 product is as follows: Record MPHR IPRs GEADRs GIADR-LEVELS GIADR-LEVELS MDR-2 Occurrence Once per product Once each per product Once each per product Once per product Once per product Once every HIRS/4 scan line Table 6-6: Occurrence of records in ATOVS Level 2 product See Section 11 for more details on the contents and format of the ATOVS Level 2 product. 6.4 The HDF formats The contents and formats of the individual fields of the ATOVS Level 2 HDF5 products are the same as for the EPS native format. The organisation of the data is different. Typically, the EPS native format presents each scan and corresponding parameters as one complete Page 27 of 76

28 sequence, stored in a Measurement Data Record (MDR), which is successively repeated until the whole swath is completed. In conversion to HDF5 the measurement values and associated parameters are grouped into separate arrays. Detailed format descriptions are provided in [RD21]. The products retrieved from the UMARF have the same name as the original EPS formatted ones, with the extension appended:.h5 for HDF5 formatted products,.nat for products in the native EPS format. Tools to read HDF formats are TBD, but it is intended that the products can be read using standard HDF libraries. For more information on HDF5 formats in general, see the HDF5 webpages. 6.5 The WMO formats The ATOVS Level 2 products available in WMO (BUFR) format are summarised in the table below. Product Bulletin header Originating station Descriptor sequence ATOVS Level 2 TBD EUMP See [RD46] Table 6-7: ATOVS products available in WMO (BUFR) format The full format description of these products is available in the WMO Manual on Codes [RD46]. The names of the ATOVS Level 2 products distributed on EUMETCast are specified in [RD31]. They follow the pattern: atovs_yyyymmdd_hhmmss_metopa_nnnnn_eps_o.l2_bufr where: yyyymmdd stands for the UTC year, month, day of the data start sensing time hhmmss stands for the UTC hour, minute, second of the data start sensing time nnnnn is the orbit number Page 28 of 76

29 7 ATOVS LEVEL 2 PRODUCT PROCESSING ALGORITHMS The data calibration and retrieval algorithms are documented in the Product Generation Specification (PGS) document [RD13]. The operational processing algorithms are summarised below. 7.1 ATOVS Level 2 processing details Data reception Level 2 processing begins with a Level 1 data reception: 1) An antenna correction is applied to AMSU-A and MHS radiances to correct for the efficiency with which the instruments look at the Earth, the spacecraft and space for each scan position. Coefficients are determined pre-launch, but can still be adjusted post-launch. They were determined by calculating the sum of the Earth, cold space and spacecraft radiances within the antenna side lobes, weighted with the corresponding efficiency. Note that the antenna correction is currently applied to the AMSU-A and MHS radiances in the EPS Level 1b products already, so that this step is disabled in the Level 2 processing. 2) Band corrections are applied to generate brightness temperatures from AMSU-A, MHS and HIRS (channels 1-19) radiances and a single Planck function calculation for each channel. This is a computational technique analogous to that employed for black body radiance calculation. For AMSU-A, the default coefficients imply no effective bandwidth correction. 3) Provision has been made for limb correction and correction for surface emissivity effects of HIRS, MHS and AMSU-A brightness temperatures. Currently this is not envisaged to be applied. The altitude of the weighting function peak changes with scan angle. Rather than adjusting the measured radiances directly, this limb effect is accounted for by including the scan angle effect in the theoretical radiative transfer calculations (and first guess libraries). 4) The primary retrieval takes place on the HIRS field of view (FOV). (A number of contingency cases exist for retrieval, but only the major ones are discussed in this overview.) Other instrument data are mapped to this grid. MHS data are first mapped to the AMSU-A FOV (twice). Mapping can be based upon bilinear interpolation (AMSU-A to HIRS field of view), nearest neighbour selection or spatial weighted averaging (MHS to AMSU A). The different mappings were chosen to reflect the relative overlap of the instrument fields of view. 5) Precipitation detection is different over ocean and over land. Multiple tests are applied in sequence. Ocean only: o Brightness temperatures are compared at different AMSU-A frequencies that have a varying sensitivity to scattering and absorption. A scattering index, based upon the difference between predicted (using channels 1-3) and measured channel 15 brightness temperatures, is calculated for this purpose. Page 29 of 76

30 This test is carried out both before and after mapping AMSU-A data onto the HIRS and again onto the MHS FOV. In the latter case, MHS channel 1 is used in place of AMSU-A channel 15. Ocean and land: o A regression test is applied based upon the relative scattering properties of hydrometeors as a function of AMSU-A frequency (the test after Crosby et al. as discussed in the PGS [RD13]). o Strong absorptions can be detected by predicting AMSU-A channel-1 brightness temperatures from channel 2 (the Grody light rainfall test, as described in the PGS). This test is carried out both before and after mapping AMSU-A data onto the HIRS FOV. 6) The underlying surface type is determined using a minimum variance technique. This relies upon a pre-computed set of AMSU-A mean brightness temperatures and associated covariance matrices. A minimum variance exceeding a threshold is used to flag potentially cloudy scenes. The radiative transfer calculations to produce these values are carried out for a variety of surface types, without cloud liquid water. This test is carried out both before and after mapping AMSU-A data onto the HIRS FOV. Only certain underlying surface types are suitable for subsequent microwave surface emissivity determination with the current model: - new sea ice - multi-year sea ice - sea - wet snow For other surface types, default values are instead assumed. Contaminated scenes are identified: o A median filter is applied to MHS data, flagging (as contaminated) any scene whose channel 1 brightness temperature differs by more than a threshold value from the median value for nearby pixels (user-configured number of adjacent pixels to check). o If the range of MHS 89 GHz channel associated temperatures for the pixels within a given AMSU-A FOV exceeds a threshold, that FOV is classified as being contaminated. o Both of these tests fail with inhomogeneous backgrounds Cloud mask 1) The HIRS Level 1 cloud mask derived from AVHRR measurements is used. 2) HIRS has a stand-alone cloud determination scheme applied at the Level 1 > 2 processing stage if AVHRR is not available. Temperatures and temperature differences are tested against threshold values to detect unreasonably cold surfaces, low spatial coherence, or inconsistencies between different frequency-windows. This is the same principle employed for AVHRR cloud determination. Different tests are applied daytime and night-time. Page 30 of 76