AVHRR Global Winds Product: Validation Report Doc.No. : EUM/TSS/REP/14/751801 Issue : v1d Date : 25 February 2015 WBS : EUMETSAT Eumetsat-Allee 1, D-64295 Darmstadt, Germany Tel: +49 6151 807-7 Fax: +49 6151 807 555 http://www.eumetsat.int EUMETSAT The copyright of this document is the property of EUMETSAT.
Document Change Record Issue / Revision Date DCN. No Changed Pages / Paragraphs v1 21 March 2014 Initial version. v1a 15 April 2014 Summarizes last changes in version 2.2.4. v1b 10 Dec 2014 Correction of biases between Metop A/B and Metop B/A products. v1c 17 Feb 2014 Document completed for review of the PVRB and for release as operational status. v1d 25 Feb 2015 Document revision following operational status. Table of Contents 1 Introduction... 4 1.1 Purpose of this Document... 4 1.2 Algorithms and Products Involved... 4 1.3 Structure of the Document... 4 1.4 Applicable Documents... 5 1.5 Acronyms and Abbreviations Used in this Document... 5 2 Description of Changes in version 2.4.1... 6 2.1 Anomalies... 6 2.2 Fixes... 6 2.3 Changes... 7 2.4 New features... 7 3 Validation Data Set Parameters... 8 4 Validation Results for Global winds product extracted from METOP A AND METOP B... 9 4.1 Summary of Results... 9 4.2 Impact of the IASI L2 Version 6 processor... 9 4.3 Bias between Metop A/B and Metop B/A products removed... 11 4.4 Statistics represented in time series... 14 4.5 Intercomparison with other AMV products... 20 5 conclusions... 27 5.1 Summary... 27 5.2 Recommendation... 27 6 Preliminary monitoring by ECMWF... 28 6.1 Plots Used... 28 6.2 Summary... 28 Page 2 of 36
Table of Figures Figure 1: Comparison of the use of IASI L2 v5 and IASI L2 v6 product in the AMV PPF.... 10 Figure 2: AVHRR global winds extracted 5 January 2015 over the globe..... 11 Figure 3: AVHRR global wind speed extracted 5 January 2015 over the globe..... 12 Figure 4: AVHRR global corresponding wind height extracted 5 January 2015 over the globe.... 12 Figure 5: AVHRR global corresponding speed bias against forecast extracted 5 January 2015 over the globe..... 13 Figure 6: Zonal plots of AVHRR global wind speeds extracted on 25 January 2015 using EBBT (left) and IASI (right) to set the altitude.... 13 Figure 7: Zonal plots of AVHRR global wind biases extracted 25 January 2015 using EBBT (left) and IASI (right) to set the altitude.... 14 Figure 8: Zonal plots of AVHRR global wind RMSV extracted 25 January 2015 using EBBT (left) and IASI (right) to set the altitude.... 14 Figure 9: Time series: product timeliness, number of processed PDUs and processing time for Metop A/B and Metop B/A products for the period 1-20 January 2015.... 15 Figure 10: Time series: total count of winds, count of good winds and percentage of good winds for Metop A/B and Metop B/A products over North Pole for the period 1-20 January 2015.... 16 Figure 11: Time series: winds height distribution, IASI height percentage, quality index and bias and RMS against forecast fields, for Metop A/B and Metop B/A products over North Pole during the period 1-20 January 2015.... 17 Figure 12: Time series: total count of winds, count of good winds and percentage of good winds for Metop A/B and Metop B/A products over South Pole during the period 1-20 January 2015.... 18 Figure 13: Time series: winds height distribution, IASI height percentage, quality index and bias of RMS against forecast fields for Metop A/B and Metop B/A products over South Pole during the period 1-20 January 2015.... 19 Figure 14 Geographic frequency distribution of METOP/validation CMV pairs on a 2 2 grid for spatial and temporal collocation thresholds of less than or equal to 150 km and less than or equal to 90 minutes. The geostationary CMVs, from west to east, are from GOES-15, GOES-13, METEOSAT-10, METEOSAT-7, and MTSAT-2, while the polar CMVs are from MODIS Terra.... 20 Figure 15: AVHRR global wind speed validation statistics averaged for all pressure levels and height assignment techniques: (a) bias and (b) RMSD, separately for the globe, tropics (25 S-25 N), northern hemisphere (NH, 25 N-90 N), and southern hemisphere (SH, 25 S-90 S). Standard collocation thresholds and comparison winds with the maximum QI were used.... 21 Figure 16: Vertical variation of AVHRR global wind speed bias (left) and RMSD (right) for standard collocation and maximum QI wind.... 22 Figure 17: Scatter plots of CMV speed (left column) and pressure (right column) for METOP versus GOES- 15 (top), GOES-13 (middle), and MTSAT-2 (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency... 23 Figure 18: Scatter plots of AMV speed (left column) and pressure (right column) for METOP versus METEOSAT-10 (top) and METEOSAT-7 (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency... 24 Figure 19: Scatter plots of AMV speed (left column) and pressure (right column) for METOP versus MODIS Terra (top) and MISR Terra (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency.... 25 Figure 20: Scatter plots of AVHRR global Winds speeds (top left), directions (top right), and pressures (bottom) against collocated Meteosat 10 AMVs for the period November 2013 to January 2014.26 Page 3 of 36
1 INTRODUCTION 1.1 Purpose of this Document The purpose of this document is to summarise the results of the validation of the latest version (version 2.4.1) of the AVHRR Level 2 Atmospheric Motion Vectors (AMV) Product Processing Facility (PPF). This version derives global coverage AVHRR winds products. The products are derived from data produced by the Advanced Very High Resolution Radiometer (AVHRR/3) on board Metop A and Metop B, and processed in the EPS (EUMETSAT Polar System) Ground Segment (GS). Based on the findings described herein, a dedicated Product Validation Review Board (PVRB) will decide on the full dissemination of the version 2.4 AVHRR global winds product to the user community. The product will be extracted from Metop A and Metop B. The purpose of this document is to present the validation results of AVHRR global winds products extracted from version v2.4.1 of the AMV PPF, with the goal of changing the status of this product from pre-operational to operational. A previous PVRB on 28 August 2014 requested these validation criteria be fulfilled before changing the status of the product to operational: a test of the impact of the latest version 6 of the IASI L2 product used in the AMV PPF an investigation of the bias between Metop A/B and Metop B/A products noted by several users monitoring events a receipt of the final report of the validation study done by TROPOS (Leipzig) (EUM-ITT207600) 1.2 Algorithms and Products Involved The algorithm developed to extract AVHRR global winds from consecutive data taken by the two Metop satellites is exactly the same than the algorithm initially developed to extract polar winds from a single Metop satellite. The results of the previously-validated versions of the AVHRR Level 2 AMV Product PPF over polar areas are reported in RD1 and in the previous version 1 and 1a of the present document. This last version described there (version 2.4.1) corrects the bias which existed between the Metop A/B and Metop B/A products in the previous versions. The polar winds derived from Level 1B data of AVHRR/3 on board Metop A and Metop B are processed in the EPS Ground Segment according to the description in [RD 2]. The description is being updated for this release. The AVHRR Level 2 PFS [RD 3] describes the format of the product, including the derived wind vectors. The AVHRR AMV native product changed according to the last version s Product Format Specification. 1.3 Structure of the Document The document is organised in the following sections: Section 1 Section 2 Section 3 Section 4 Section 5 Introduction Descriptions of Version 2.4. New changes to processing are detailed. Explanation of the Data Sets. Product validation results Conclusions and recommendations Page 4 of 36
1.4 Applicable Documents The documents listed here are referenced in the text of this document. All the documents listed are available on the EUMETSAT Technical Documents web page. Ref Title EUMETSAT Reference RD 1 Metop-A and Metop-B polar winds validation report EUM/RSP/REP/13/696404 RD 2 AVHRR Level 2 Polar Winds Product Generation Specification EUM/OPS-EPS/SPE/08/0346 RD 3 AVHRR Level 2 Polar Winds Product Format Specification EUM/OPS-EPS/SPE/08/0338 RD 4 AVHRR global Winds Intercomparison Study EUM-TT207600 Satwind Final Report/769487 1.5 Acronyms and Abbreviations Used in this Document Acronym AMV CTH CTT EBBT ECMWF FC GS IASI QI RMSV RMSVD Meaning Atmospheric Motion Vectors Cloud Top Height Cloud Top Temperature Effective/Equivalent Black Body Temperature European Centre for Medium-Range Weather Forecasts Forecast Ground Station Infrared Atmospheric Sounding Interferometer Quality Index Root Mean Square Value Root Mean Square Value Deviation Page 5 of 36
2 DESCRIPTION OF CHANGES IN VERSION 2.4.1 The version 2.4.1 of the algorithm that extracts AVHRR global winds is very similar to the operational version 2.4 that extracts single Metop winds over polar areas. The main difference is the use of two consecutive AVHRR images taken by two different Metop satellites (Metop A and Metop B; or Metop B and Metop A) instead of only one. The overlapping area seen consecutively by the two Metop satellites represents, at minimum, half of the AVHRR swath width (~1000 km) at low latitude as shown in Figure 1. This is an area large enough to extract wind information from consecutive AVHRR images over the whole globe. Two complementary products are then extracted, one that uses Metop A as the first image and Metop B as the second image of the pair this is named Metop A/Metop B in the following document. Another one uses Metop B as the first image and Metop A as the second image of the pair This is named Metop B/Metop A in this document. The temporal gap between the two consecutive images is equal to 49 50 minutes for the Metop A/Metop B result, and equal to 51 52 minutes for the Metop B/Metop A result. The following changes (respective to v.2.2.4) have been introduced in the current version 2.4.1: 2.1 Anomalies EUMETSAT AR EPS/AR/15112 EPS/AR/15250 EPS/AR/15330 EPS/AR/15830 Meaning 1. Time difference between the two images in Dual Mode: Fixes the time difference used for first guess and speed computations 2. Does not apply anymore the parallax correction before looking for a IASI pixel since both instruments consider a null altitude in the pixel georeferencing. In some cases, the backtrack corresponding to the forward vector fails. In these cases, the QI were very low but the second intermediate component is undefined and could lead to BUFR encoding error. In v. 2.3.1, the winds records are not saved anymore. The reading of the new IASI SND L2 products might crash due to stack mangling. Problem was fixed and the extraction of the IASI temperature profile was disabled since it is not used. The orbit drift since launch introduced a bias between the two dual Metop AMV products. Version 2.4.1 fixes the bias and the configuration settings were updated to the actual phase separation. 2.2 Fixes EUMETSAT AR EPS/AR/15830 Meaning The orbit drift since launch introduced a bias between the two dual Metop AMV products. The v2.4.1 fixes the bias and the configuration settings were updated to the actual phase separation. Page 6 of 36
2.3 Changes EUMETSAT AR EPS/AR/15830 Meaning 1. Parallax correction is only applied at the end of the process. 2. The vector displacement is computed using the centres of the reference and search windows instead of the barycentre of the CCC pixels mask. 3. Adaptative search window size: depending on the time lag between the two images and on the forecast wind speed. This change was a solution to get a direct comparison between single and dual AMVs quality. 2.4 New features Meaning 1. The intermediate components stored: the backward vector corrected for parallax (= final vector) in the first position and the forward vector corrected for parallax (used only for QI computation) in the second position. 2. The second satellite zenith angle (previous image) is now stored in the product. 3. The IASI L2 processor v6 (ECPD-566) introduces a new product format specification. In this specification, format v. 11 is read by this version but the compatibility with v. 10 remains. 4. Intermediate results in JSON format can be output on request. Page 7 of 36
3 VALIDATION DATA SET PARAMETERS The test of the Impact of the new IASI L2 v6 processor has been done for the period 21 September 2014 to 30 September 2014 using version 2.4.0 of the algorithm. Version 2.4.1, which fixed the issue of the bias in AVHRR global wind products was installed on GS1 on 2 December 2014. The overall statistics on AVHRR global Metop A/B and Metop B/A winds products are presented for the period of 1-20 January 2015. Page 8 of 36
4 VALIDATION RESULTS FOR GLOBAL WINDS PRODUCT EXTRACTED FROM METOP A AND METOP B This section summarises the results of the actual version of the PPF processor applied to Metop A and Metop B data to derive AVHRR global winds. 4.1 Summary of Results Item New IASI L2 product (version 6) running on the AMV PPF Bias observed between Metop A/B and Metop B/A products disappeared in Version 2.4.1. Comment No impact or negligible impact on data. Statistical results of the two complementary products are now similar. Note: The Leibniz Institute for Tropospheric Research e.v (TROPOS) provided EUMETSAT with the AVHRR global Winds Intercomparison Study. 4.2 Impact of the IASI L2 Version 6 processor The AMV PPF uses IASI cloud top height to set wind altitudes when the wind is collocated with an IASI footprint. The complete IASI L2 processor was released at the end of September 2014, when version 6 of the PPF was made operational. Figure 1 compares the results for the AVHRR global winds product using IASI L2 v.5 and IASI L2 v.6 to set the wind altitudes. In the top panel are shown the scatter plots for collocated AVHRR global winds altitudes, speed, direction and quality index excluding forecast check (QIx). These were extracted using IASI L2 v.6 (Y axis) as a function of IASI L2 v5 (X axis). The color scale is logarithmic; and the two sets of data are very similar and there are very few outliers. The middle panel represents the distribution function for the same four parameters: pressure, speed, direction and QIx for all the collocated AMVs, using IASI data in the height assignment process. The bottom panel shows the dependency of the speed bias against the different parameters for the two experiments, using IASI L2 v.5 and IASI L2 v.6. Taken together, these plots show very similar statistical behaviour for the two validation runs. This means that the use of Version 6 of IASI L2 processor does not affect the AVHRR global wind product in any way. Page 9 of 36
Figure 1: Comparison of the use of IASI L2 v5 and IASI L2 v6 product in the AMV PPF. Page 10 of 36
4.3 Bias between Metop A/B and Metop B/A products removed In order to show this removal of bias, we have prepared a series of plots as a comparison: the number of winds (Figure 2) the corresponding wind speeds (Figure 3) wind height (Figure 4) bias against forecast fields (Figure 5) for AVHRR global winds extracted using the last version (v.2.4.1) of the AMV PPF on 5 January 2015. In these plots, the results for the North Pole are in the top left panel, results for the South Pole are at top right. Global coverage of the winds is in the bottom panel. Figure 6 through Figure 8 show the corresponding zonal plots plots of statistics as function of pressure and latitude of wind speeds, wind speed bias against forecast and Root Mean Square Velocity (RMSV). Jet streams are indicated by high wind speeds and can be seen in Figure 3 and Figure 6. The large positive bias at high levels observed in tropical areas can also be seen in Version 2.4.1. See Figure 5 and Figure 7. Note that this large bias in tropical areas is also found with geostationary winds, as described in Section 4.5. So, this is not specific to the AVHRR global winds product. Figure 2: AVHRR global winds extracted 5 January 2015 over the globe. North Pole is at upper left, South Pole is at upper right and global coverage is on the lover panel. Page 11 of 36
EUM/TSS/REP/14/751801 Figure 3: AVHRR global wind speed extracted 5 January 2015 over the globe. North Pole is at upper left, South Pole is at upper right, and global coverage in the lower panel. Figure 4: AVHRR global corresponding wind height extracted 5 January 2015 over the globe. North Pole is at upper left, South Pole is at upper right, and global coverage is on the lover panel. Page 12 of 36
EUM/TSS/REP/14/751801 Figure 5: AVHRR global corresponding speed bias against forecast extracted 5 January 2015 over the globe. North Pole is at upper left, South Pole is at upper right and global coverage is on the lower panel. Figure 6: Zonal plots of AVHRR global wind speeds extracted on 25 January 2015 using EBBT (left) and IASI (right) to set the altitude. Page 13 of 36
Figure 7: Zonal plots of AVHRR global wind biases extracted 25 January 2015 using EBBT (left) and IASI (right) to set the altitude. Figure 8: Zonal plots of AVHRR global wind RMSV extracted 25 January 2015 using EBBT (left) and IASI (right) to set the altitude. 4.4 Statistics represented in time series The figures in this section (Figure 8 through Figure 12) show time series statistics of Metop A/B and Metop B/A AVHRR global wind products extracted using Version 2.4.1 of the AMV PPF between 1 and 20 January 2015. Metop A/B is plotted in red. Metop B/A is plotted in blue. Statistics are shown for total orbit in Figure 9, the North Pole in Figure 10 and Figure 11and for the South Pole in Figure 12 and Figure 13. Several statistical parameters are plotted. These show a very solid agreement, between Metop A/B and Metop B/A products. If you look carefully at Figure 12 and Figure 13, you see that the biases against forecast and RMS are now similar for the two complementary products both using Version 2.4.1. This result is further confirmed by the passive monitoring of this product done at ECMWF, which is presented in Section 6 of this document. Page 14 of 36
Figure 9: Time series: product timeliness, number of processed PDUs and processing time for Metop A/B and Metop B/A products for the period 1-20 January 2015. Page 15 of 36
Figure 10: Time series: total count of winds, count of good winds and percentage of good winds for Metop A/B and Metop B/A products over North Pole for the period 1-20 January 2015. Page 16 of 36
Figure 11: Time series: winds height distribution, IASI height percentage, quality index and bias and RMS against forecast fields, for Metop A/B and Metop B/A products over North Pole during the period 1-20 January 2015. Page 17 of 36
Figure 12: Time series: total count of winds, count of good winds and percentage of good winds for Metop A/B and Metop B/A products over South Pole during the period 1-20 January 2015. Page 18 of 36
Figure 13: Time series: winds height distribution, IASI height percentage, quality index and bias of RMS against forecast fields for Metop A/B and Metop B/A products over South Pole during the period 1-20 January 2015. Page 19 of 36
4.5 Intercomparison with other AMV products The global coverage of the AVHRR global Winds product allows a direct comparison with Atmospheric Motion Vectors (AMV) derived from other satellites. For this validation, we assembled a comparison data set made up of the following geostationary Cloud Motion Vector (CMV) data: GOES-15 at 135 W GOES-13 at 75 W METEOSAT-10 at 0 METEOSAT-7 at 57.3 E MTSAT-2 at 145 E Additionally, polar-orbiter CMVs from MODIS and MISR on the NASA spacecraft Terra were used. See Figure 14. This study was conducted by TROPOS for a validation period lasting from 20 October 2013 to 31 January 2014. Results are presented in the study s final report. See Section 6. Figure 14: Geographic frequency distribution of METOP/validation CMV pairs on a 2 2 grid for spatial and temporal collocation thresholds of less than or equal to 150 km and less than or equal to 90 minutes. The geostationary CMVs, from west to east, are from GOES-15, GOES-13, METEOSAT-10, METEOSAT-7, and MTSAT-2, while the polar CMVs are from MODIS Terra. Winds at all pressure levels with QI greater than or equal to 80 are considered, irrespective of height assignment method. Page 20 of 36
Figure 15 shows the overall statistical plots for the intercomparison data. The mean speed difference (bias) between METOP and the geostationary/modis Terra winds ranges between 2 m/s -1 and +1 m/s -1. Most sensors and regions have negative biases, that is, slower METOP winds. The Root Mean Square Deviation (RMSD) varied mostly between 8 m/s -1 and 9 m/s -1. The overall agreement was best in the tropics, with the smallest bias and RMSD about 6 m/s -1. Here, we must emphasize that the AVHRR global Wind product agrees closely with other wind observations in the tropics exactly where the bias against forecast is the largest. See Figure 5 and Figure 7 above. This proves that the large speed bias found in the tropics is not specifically linked to the AVHRR global wind product, but is more probably linked to the atmospheric physics like large convective cells, wind representativity in this area. The validation results with the Multi-angle Imaging Spectral Radiometer (MISR) on Terra were outliers, especially outside the tropics. In a comparison of the METOP and MISR RMSD bias, correlation varied between 2 m/s -1 to 6 m/s -1, 8 m/s -1 to 12 m/s -1, and 0.65 m/s -1 to 0.75 m/s -1, respectively. So, on average, METOP winds were faster than MISR ones. This was due to the fact that a significant number of horizontally/temporally collocated METOP MISR wind pairs corresponded to cases where METOP tracked faster-moving mid-level to high-level clouds and MISR tracked slower-moving low-level clouds. This, in turn, stems from the different channels used for tracking (IR versus visible) and the design of the MISR stereo matcher that is optimized to lock in on the highest-contrast low-level cloud structures. To be a consistent data source, the AVHRR global wind product could be used as a reference in the comparison to Geostationary AMV products. For instance, GOES-13 and GOES-15 show a noticeably different behaviour. This might be worthy of investigation and analysis. a) b) Figure 15: AVHRR global wind speed validation statistics averaged for all pressure levels and height assignment techniques: (a) bias and (b) RMSD, separately for the globe, tropics (25 S-25 N), northern hemisphere (NH, 25 N-90 N), and southern hemisphere (SH, 25 S-90 S). Standard collocation thresholds and comparison winds with the maximum QI were used. Page 21 of 36
Figure 16: Vertical variation of AVHRR global wind speed bias (left) and RMSD (right) for standard collocation and maximum QI wind. Page 22 of 36
Figure 17: Scatter plots of CMV speed (left column) and pressure (right column) for METOP versus GOES-15 (top), GOES-13 (middle), and MTSAT-2 (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency. Page 23 of 36
Figure 18: Scatter plots of AMV speed (left column) and pressure (right column) for METOP versus METEOSAT-10 (top) and METEOSAT-7 (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency. Page 24 of 36
Figure 19: Scatter plots of AMV speed (left column) and pressure (right column) for METOP versus MODIS Terra (top) and MISR Terra (bottom). Standard collocation thresholds and comparison winds with the maximum QI were used, and both height assignment methods were considered. The colour code ranges from blue (less) to red (more) showing increasing number frequency. Page 25 of 36
Figure 20: Scatter plots of AVHRR global Winds speeds (top left), directions (top right), and pressures (bottom) against collocated Meteosat 10 AMVs for the period November 2013 to January 2014. Page 26 of 36
5 CONCLUSIONS 5.1 Summary Results of validation for the AVHRR global Winds products as detailed in this document support the following: The use of Version 6 IASI L2 product in the AMV PPF has no impact on AVHRR global Winds product quality. The bias between the two complementary products Metop B/A and Metop A/B in previous versions of the AMV PPF has disappeared in Version 2.4.1. This result is confirmed by the passive monitoring received by ECMWF. See Section 6. For support, a complete validation study was undertaken to compare the AVHRR global winds against several other winds observations extracted especially from GOES, MTSAT, Meteosat, MODIS, and MISR. The study was carried out by the Leibniz Institute for Tropospheric Research (TROPOS). The complete results are available in [RD 4]. The results showed a good agreement between the AVHRR global winds product and the other satellite AMV products, even in the tropics. 5.2 Recommendation Based on the results presented in this document, we propose to move the AVHRR global Winds product to operational status and to disseminate it to all users. Page 27 of 36
6 PRELIMINARY MONITORING BY ECMWF 6.1 Plots Used Plots in this section show OmB and RMSVD for the three usual vertical levels. Upper panels: Show passive monitoring for 9.5 weeks from 31 July 2014 to 5 October 2014. Lower panels: Show passive monitoring for 4.5 weeks from 5 December 2014 to 4 January 2015. Note: The complete ECMWF feedback is in this document. Order it from the EUMETSAT help desk: Title Internal EUMETSAT reference AVHRR global winds-ecmwf Monitoring 051214 040115 790746 6.2 Summary There were similar statistics for A/B and B/A after the change on 4 December 2014. Bias characteristics at mid levels have changed when compared to the old monitoring period. There are new areas of negative bias over NH. There are noticeable changes in the Root Mean Square Value Deviation (RMSVD) maps. Page 28 of 36
Page 29 of 36
Page 30 of 36
Page 31 of 36
Page 32 of 36
Page 33 of 36
Page 34 of 36
Page 35 of 36
Page 36 of 36