Validation and Monitoring of the OSI SAF Low Resolution Sea Ice Drift Product

Transcription

1 Ocean & Sea Ice SAF Validation and Monitoring of the OSI SAF Low Resolution Sea Ice Drift Product GBL LR SID OSI-405-c Example ice drift vector fields as processed at the EUMETSAT OSI SAF. Left panel displays displacement vectors and length (color shades) in the Arctic Ocean in April 2010, while right panel displays vectors and displacement length in the Ross Sea in August Both examples are from the multisensor merged product. Version 5 July 2016 Thomas Lavergne

2 This page is left blank intentionally.

3 SAF/OSI/CDOP/Met.no/T&V/RP/131 3 Documentation Change Record: Version Date Author Description v TL Initial version, before review v TL Amended after PCR comments v TL Include more in-situ data sources, change the (spatial) collocation method and extend validation time period. v TL Include more in-situ data sources, introduce SAR ice drift as ground truth over SH sea ice, change (temporal) collocation method. v TL Include more in-situ data sources. Validation of SSMIS (F17) and AMSR2 (GCOM-W1) products. No update of SH validation (not enough validation data). v TL Validation of OSI-405-c in NH and SH. Validation of summer drift vectors. Assessment of uncertainties.

4 SAF/OSI/CDOP/Met.no/T&V/RP/131 4 Table of contents Table of contents 4 List of Figures 5 1 Introduction The EUMETSAT Ocean and Sea Ice SAF Scope and Overview Validation exercise version history Glossary Validation dataset In-situ trajectories Remarks on the validation dataset Acknowledgements Spatial and Temporal overview of the validation dataset Validation methodology Variables to be validated Reformatting of the validation dataset Collocation strategies Representativity error Graphs and statistical measures Results and discussions Validation over the NH area Validation over the SH area Validation using SSMIS F18 and ASCAT Metop-B Assessment of the uncertainties Conclusion 32 References 34

5 SAF/OSI/CDOP/Met.no/T&V/RP/131 5 List of Figures 0.1 Example OSI-405 products Trajectories of NH validation drifters Trajectories of SH validation drifters NH validation graphs for single-sensor products NH validation graphs for multi-sensor product NH map of mismatch between products and validation data Example validation map of multi-oi product in Beaufort Sea SH validation graphs for single-sensor products SH validation graphs for multi-sensor product SH map of mismatch between products and validation data NH and SH uncertainty assessment for multi-sensor product

6 SAF/OSI/CDOP/Met.no/T&V/RP/ Introduction 1.1 The EUMETSAT Ocean and Sea Ice SAF For complementing its Central Facilities capability in Darmstadt and taking more benefit from specialized expertise in Member States, EUMETSAT created Satellite Application Facilities (SAFs), based on co-operation between several institutes and hosted by a National Meteorological Service. More on SAFs can be read from The Ocean and Sea Ice Satellite Application Facility (OSI SAF) is producing on an operational basis a range of air-sea interface products, namely: wind, sea ice characteristics, Sea Surface Temperatures (SST), Surface Solar Irradiance (SSI) and Downward Longwave Irradiance (DLI). The sea ice products include sea ice concentration, the sea ice emissivity, sea ice edge, sea ice type and sea ice drift and sea ice surface temperature (from mid 2013). The OSI SAF consortium is hosted by Météo-France. The sea ice processing is performed at the High Latitude processing facility (HL centre), operated jointly by the Norwegian and Danish Meteorological Institutes. Note: The ownership and copyrights of the data set belong to EUMETSAT. The data is distributed freely, but EUMETSAT must be acknowledged when using the data. EUMETSAT s copyright credit must be shown by displaying the words copyright (year) EUMETSAT on each of the products used. User feedback to the OSI SAF project team is highly valued. The comments we get from our users is important argumentation when defining development activities and updates. We welcome anyone to use the data and provide feedback. 1.2 Scope and Overview Sea ice drift products for Northern and Southern Hemisphere are processed at the High Latitude center of the Ocean & Sea Ice Satellite Application Facility (EUMETSAT OSI SAF). Those datasets are introduced and documented in a dedicated Product User s Manual ((PUM), Lavergne (2016b)) and in an Algorithm Theoretical Basis Document ((ATBD), Lavergne (2016a)) that can both be found on See for real time examples of the products as well as updated information. The latest version of this document can also be found there. General information about the OSI SAF is given at This Validation and Monitoring report only deals with the OSI-405 series of low resolution sea ice drift products. The medium resolution ice drift product based on AVHRR imagery, the OSI-407 series, is documented in a dedicated report. The aims of this report are several: 1. To document the level of agreement between the OSI SAF low resolution sea ice drift product and ground-based truth. Various graphs displaying the match between the satel-

7 SAF/OSI/CDOP/Met.no/T&V/RP/131 7 lite product and the in-situ datasets should give (qualitative) confidence in using the product. 2. To report quantitative estimates of errors and uncertainties in the product. Particularly, the bias and uncertainty covariance matrix is computed. It is important that each singlesensor and the multi-sensor products are validated separately so that users can have error estimates for the product they choose to use. 3. To match these quantitative estimates with the accuracy thresholds defined in the Product Requirements Document ((PRD), CDOP2 PRD). Chapter 2 presents the datasets used as validation data while chapter 3 documents the validation strategy and, particularly, the way collocation is handled. Chapter 4 provides detailed, graphical and quantitative analysis of the validation results. We conclude in chapter 5. Note that the OSI SAF low resolution sea ice drift product will not be introduced at any depth in this report. Refer to the (PUM) (ATBD), and for information on the algorithms, processing schemes and data format. Let us nonetheless remind that the OSI SAF low resolution ice drift product comes as daily vector fields obtained by processing low-resolution satellite signal from, among others, AMSR2, SSMIS and ASCAT. It is computed on a Northern Hemisphere (NH) and a Southern Hemisphere (SH) grid. It is a 2 days (48 hours) ice drift product on a 62.5 km resolution polar stereographic grid. Both single and multi sensor products are distributed. Starting with OSI- 405-c, the multi-sensor products is processed and delivered all year round. Earlier versions were not processed during the summer season in both hemisphere. 1.3 Validation exercise version history This report is updated regularly when new validation results need to be documented, particularly when the OSI SAF low resolution ice drift product undergoes reviews in view of operational upgrades. This section aims at shortly introducing the rational for each new version Version 1 Fall 2009 Version 1 of the validation exercise takes place during the Product Consolidation Review, where the potential accuracy of the ice motion algorithms is assessed Version 2 Spring 2010 Version 2 of the validation exercise extends previous version by introducing more buoy data. Only the Northern Hemisphere (NH) product is available (and validated here) Version 3 Winter 2012 Version 3 of the validation exercise is prompted by the need to document the accuracy of the product in the Southern Hemisphere (SH), in view of starting operational distribution of SH drift product. Due to the insufficient number of buoys available, this is mainly achieved by an comparison to high-accuracy ice motion vectors processed from Synthetic Apperture Radar images available from DTU/PolarView/MyOcean.

8 SAF/OSI/CDOP/Met.no/T&V/RP/ Version 4 Spring 2015 Version 4 of this validation exercise is prompted by operationalization of OSI-405-b. It documents the accuracy obtained when processing ice drift vectors from JAXA s AMSR2 on-board GCOM-W1. There is no change to the processing algorithm. At the same time, the opportunity is taken to also update validation results for SSMIS (F17) and ASCAT (Metop-A) Version 5 Summer 2016 Version 5 of this validation exercise documents the accuracy obtained with OSI-405-c. OSI- 405-c introduces (1) ice drift vectors during summer (from JAXA s AMSR GHz imagery), (2) switch ASCAT from Metop-A to -B, (3) switch SSMIS from DMSP F17 to F18, and (4) maps of uncertainties. All these aspects are validated and documented in Version 5 of the report. The validation runs over more than 3 years (Jan 2013 to Apr 2016) and covers both NH and SH.

9 SAF/OSI/CDOP/Met.no/T&V/RP/ Glossary AARI Arctic and Antarctic Research Institute ASCAT Advanced SCATterometer AVHRR Advanced Very High Resolution Radiometer AMSR2 Advanced Microwave Scanning Radiometer - 2 AMSR-E Advanced Microwave Scanning Radiometer for EOS AWI Alfred Wegener Institute CDOP Continuous Development and Operations Phase CF Climate and Forecast CRREL Cold Regions Research & Engineering Laboratory (US Army) DMI Danish Meteorological Institute DMSP Defense Meteorological Satellite Program DTU Danish Technical University IABP International Arctic Buoy Program ITP Ice Tethered Profiler IMB Ice Mass Balance GCOM-W Global Change Observation Mission for Water ( Shizuku ) GPS Global Positioning System HL High Latitudes JAXA Japan Aerospace Exploration Agency MET Norway Norwegian Meteorological Institute NetCDF network Common Data Format NH Northern Hemisphere NP North Pole SAF Satellite Application Facility SAR Synthetic Apperture Radar SAMS Scottish Association for Marine Research SH Sourthern Hemisphere SIMBA Sea-Ice Mass BAlance SIP AWI s SSM/I Special Sensor Microwave/Imager SSMIS Special Sensor Microwave Imager Sounder Tb Brightness Temperature TOA Top Of Atmosphere WSM Wide Swath Mode

10 SAF/OSI/CDOP/Met.no/T&V/RP/ Validation dataset In this section, we introduce the ice motion datasets that constitute our best estimate of the ground truth and that is used as reference to validate the OSI SAF low resolution sea ice drift product. Several data sources are available for validating an ice drift product and they can be sorted into three groups: 1. Trajectories of in-situ ice drifters. Historically, this is the main validation data source. A fair number of buoys are indeed deployed in the ice covered ocean to measure atmospheric, cryospheric or oceanic variables (e.g. Mean Sea Level Pressure, ice thickness or temperature and salinity profiles of the ocean). Of interest to us is the fact that they regularly and automatically report their position via the Argos system or by transmitting GPS positions as part as their data stream. Drifting ships (like the Tara) or manned stations (NP-35, NP-36, etc...) also constitute good opportunities to get ice trajectory data, sometimes in near-real-time. 2. High resolution satellite based ice drift datasets. Those are processed from high resolution satellite images (e.g. ENVISAT SAR or AVHRR). Those products are not ground truth but are assumed to present much less deviations to truth than this low resolution ice drift datasets. 3. Moored Doppler-based velocity measures from under the ice. This source of data presents three major disavantages. Firstly, they are Eulerian measures of instantaneous velocity, a quantity that is not directly comparable to satellite-based ice displacement vectors. Second, they do not transmit data in near-real-time and are thus not suitable for daily monitoring of a product. Finally, they are often located in coastal areas where the retrieval of sea ice drift from low resolution sensors is challenged by the proximity to land. For the validation exercise reported in this document only in-situ buoy trajectories are used. 2.1 In-situ trajectories Ice Tethered Profilers The Ice Tethered Profilers (ITP) platforms are advanced autonomous drifting instrument that are designed to measure temperature and salinity profiles in the ocean under sea ice. As part of its daily data stream, each ITP transfers hourly unfiltered GPS locations. As of early 2015, there are about 9 active (plus additional 80 completed) ITPs in the Arctic Ocean that form a high quality validation dataset, especially for the Beaufort Sea, Canadian Basin and Fram Strait. It is noteworthy that the primary objective of ITPs is to sample the water column and, thus, require to be deployed in location with enough water depth. This excludes all shelf area like Laptev or Chuckchi Sea and limits the spatial sampling of the Arctic Ocean. The ASCII formatted level 1 raw data position files (itpnrawlocs.dat) for all ITPs were downloaded from the FTP server at Woods Hole Oceanographic Institution and processed to extract ice drift vectors.

11 SAF/OSI/CDOP/Met.no/T&V/RP/ Russian manned polar stations GPS trajectory logs for the Russian manned stations NP-35 to NP-40 were made available by the Arctic and Antarctic Research Institute (AARI, The drift start and end dates are in table 2.1. Station Begin End Duration [days] NP NP NP NP NP NP Table 2.1: Drift trajectories of AARI s North Pole stations SAMS SIMBA buoys The Scottish Association for Marine Science (SAMS) designs robust thermistor-based Sea- Ice Mass Balance (SIMBA) buoys for monitoring the snow and sea ice thickness. Part of the data stream sent via Iridium are the sub-hourly GPS locations of the platforms. In recent years, SIMBA buoys were deployed by several teams, in various research projects such as EU-ACCESS, etc... We access the unfiltered Iridium messages direcly from the SAMS servers, and apply our own quality control CRREL IMB buoys The Cold Regions Research & Engineering Laboratory (CRREL) of the US Army also designs thermistor-based Ice Mass Balance (IMB). The earliest versions were already deployed in As of early 2015, there are 6 active platforms, and 97 archived trajectories, available from AWI s buoys The Alfred Wegener Instistute (AWI) deploys sea ice buoys in Arctic and Antarctic regions. They are collected and made available in near-real-time from the Data Portal of seaiceportal.de. First deployments were in 2012, with increasing number of buoys until today. This is our only source for SH buoy trajectories. 2.2 Remarks on the validation dataset Although we tried to use as many good quality drifters as possible, entire regions covered by the OSI SAF ice drift product grid are not sampled by our validation dataset. See, for reference, figure 2.1. In the Northern Hemisphere, two such regions are the Baffin and Hudson Bay for which it was not possible to obtain trajectories in the validation period we have been covering. Some

12 SAF/OSI/CDOP/Met.no/T&V/RP/ buoys are released every year in the Nares Strait. However, most of them sink due to unstable ice conditions in the strait or Baffin Bay at that period. A major validation data gap is also over the Antarctic sea ice. Thanks to the recent efforts at AWI (see section 2.1.5), the situation is improving, and one can access a fair amount of GPS trajectories since 2012, all in the Weddell Sea. The Ross Sea is mostly empty for buoy data (or we have no access to them). 2.3 Acknowledgements The Ice-Tethered Profiler data were collected and made available by the Ice-Tethered Profiler Program based at the Woods Hole Oceanographic Institution GPS-located data from Russsian stations were kindly provided by the Arctic and Antarctic Research Institute (AARI, of Roshydromet, PIs Vladimir Sokolov and Vasily Smolyanitsky. The trajectories for SIMBA buoys are provided by Phil Hwang from the Scottish Association for Maring Science (SAMS). The trajectories of CRREL IMBs are from Perovich et al. (2014) Autonomous sea ice measurements (position) from November 2012 to April 2016 were obtained from (grant: REKLIM ). 2.4 Spatial and Temporal overview of the validation dataset Northern Hemisphere Figure 2.1 displays a graphical overview of the in-situ trajectories that were used in the validation period for NH area. All the color-coded position records are obtained with GPS accuracy. It is not seldom that, e.g., both an ITP and a CRREL buoys are deployed by the same crew, on the same ice flow. This is the reason why some trajectories in the left panel of Figure 2.1 switch color. Note that we do not mix positions of an ITP and CRREL buoys when extracting the validation drift vector. The right panel displays the same positions, but the color now varies with the time of the position record from 1 st January 2013 (dark blue) to 31 st December 2015 (dark red). This helps visualizing the direction of the drift, but also the time of the year when each area was mostly validated Southern Hemisphere Figure 2.2 illustrates the spatial and time coverage of the buoy validation dataset over SH sea ice. All the buoys are in the Weddell Sea, and are distributed by

13 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 2.1: Trajectories of validation drifters between 1 st January 2013 and 31 st December Left panel: the colour codes the buoy programme or data provider (SIP stands for AWI s Right panel: The colour codes the timestamp of the position record. Figure 2.2: Same as right panel of figure 2.1 but for SH region. All the buoys are from

14 SAF/OSI/CDOP/Met.no/T&V/RP/ Validation methodology The validation strategy is introduced in this chapter. It covers the re-formatting of the trajectories and the collocation with the sea ice drift product. We also present the graphs and statistical metrics that will be displayed and commented upon in chapter Variables to be validated As introduced in the sea ice drift (PUM), an ice drift vector is fully described with 6 values: the geographical position of the start point (lat 0 and lon 0 ), the start time of the drift (t 0 ), the position of the end point of the drift (lat 1 and lon 1 ) as well as the end time of the drift (t 1 ). However, the primary variables the ice drift processing software optimizes are dx and dy, the components of the displacement vector along the X and Y axes of the product grid. Those are thus the two variables we are aiming at validating. 3.2 Reformatting of the validation dataset In any validation exercise, especially if making use of a broad range of data sources, one is confronted to new and varying formats. Most of the times, trajectories from in-situ drifters are available in an ASCII format, proposing one position and time stamp per line. The various formats for the time information, in particular, as well as the ordering of columns make it challenging to design a unique software package to read all those files. A first step of this validation effort has thus been the design of dedicated software routines to read the observation files, extract the portion of the trajectory that fits the time span of the OSI SAF ice drift product file and dump the validation data in a NetCDF formatted file. 3.3 Collocation strategies In order to compare the OSI SAF sea ice drift product with the validation trajectories, they need to be collocated one with the other. Collocation is the act of selecting or transforming one or both datasets so that they represent the same quantity, at the same time and at the same geographical location. Because the OSI SAF ice drift product comes with two flavours of time information (refer to (PUM), section Time information), two validation exercises are conducted: One using a 2D collocation, in which the satellite product is considered representing a drift from day D@1200 UTC to (D+2)@1200 UTC, uniformly over the grid. One with a 3D collocation, in which the position dependent start and end times are used (found in datasets dt0 and dt1, in the product file). The reason for having those two validation strategies is that some users might wish (or have to) ignore the accurate timing information provided with each vector. Using only the central

15 SAF/OSI/CDOP/Met.no/T&V/RP/ times, these users need to know if the uncertainty estimates are to be enlarged and, if so, by which amount D collocation The in-situ drift vector is defined by first selecting the start and end position record in their trajectory. Those are the ones closest (in time) to 1200 UTC, on both dates. From those 2 positions, dx ref and dy ref are computed. The product dx prod and dy prod are those of the nearest-neighbour in the product grid D collocation The in-situ start (end) point is searched for along the trajectory: each record is remapped into the product grid where a product start (end) time is computed by bilinear interpolation from the 4 encompassing grid points. Because the records are ordered chronologically, it is possible to stop searching as soon as both start and end in-situ records are selected. As in the 2D collocation, they define the truth displacement components: dx ref and dy ref. The components for the product (dx prod and dy prod ) are selected as those of the nearest-neighbour in the product grid Additional remarks In version v1 of this validation exercise, the spatial collocation was achieved by bi-linear interpolation of the 4 neighbouring vectors from the product grid to the position of the reference vector. Further investigations confirmed that this method could lead to artificially good validation statistics, since part of the noise in the product was averaged out in this process. From v2 of this report onwards, spatial collocation relies on nearest-neighbour selection. The distance to the nearest neighbour must be inside 40 km radius from the start of reference vector. In versions v1 and v2 of this validation report, matchup pairs were allowed in the validation dataset if the time difference of both the start and stop records were less than 3 hours. This choice conducted to potentially allowing differences in drift durations of up to 6 hours. Further investigation of the v1 and v2 validation pairs, however, demonstrated that the difference in drift durations was mostly in the range [-1 h:+1 h]. This was thanks to the hourly (sometimes subhourly) sampling of most of the buoy trajectories, allowing for accurate temporal registration. This excellent temporal sampling is also why changing the time collocation criteria did not lead to significant changes in results in (Hwang and Lavergne, 2010b, section 4.1). From version v3 of the validation excercise, the temporal collocation was changed to allow collocation pairs only if the time difference at the start data record is within [-3 h:+ 3h], and if the difference in drift duration is within [-1 h:+1 h]. As noted above, this change has very little impact on the selection of buoy matchups since they mostly report hourly sampled trajectories. This change does not invalidate earlier validation results documented in v2 of this report, nor those in Lavergne et al. (2010) or Hwang and Lavergne (2010b). To clear the temporal and spatial collocation criteria above is not enough for entering the validation dataset. Additional constraints are imposed to obtain a more controlled validation: A validation vector must be surrounded by 4 valid OSI SAF vectors. Although only the nearest of these 4 is considered in the statistics, this constraint is introduced to avoid validation data in the outer edges of the vector field, like in the marginal ice zone, or in

16 SAF/OSI/CDOP/Met.no/T&V/RP/ coastal regions (land-fast ice). Since some of the ice-tethered buoys we use (e.g. ITPs) are designed to continue floating when sea ice melts, this constraint is also an effective way of not collocating ocean drift measurements with our product. Any two validation vectors for a given product date must be separated by at least = km. This constraint is introduced so that all data pairs entering the validation dataset are independent from each others. Two neighbouring OSI SAF sea ice drift vectors indeed exhibit correlated uncertainties due to the overlap of their image-matching windows (e.g. (Lavergne et al., 2010, section 3.2)). This correlation of neighbouring vector should ideally be taken into account when computing validation statistics. However, thanks to the excellent coverage of the OSI SAF product, we can be strict about the correlations and steer away from them by rejecting validation pairs that are too close to each others on a daily grid. 3.4 Representativity error Although we only use high quality buoy position data and although the collocation metods and parameters are quite stringents, a possibly high and mostly uncontrolled source of error resides in the representativity mismatch between the scales sampled by the buoy and the satellite product. A buoy indeed samples the motion of the ice floe it was deployed over. Although investigators in field campaigns tend to choose rather large floes for limiting the risk of the buoy disappearing too rapidly, the size and shape of the floe will change with time through collision or breaking events. On the other hand, the satellite ice drift product samples the motion of a much larger area of the sea ice surface that is close to km 2. The mismatch between the two scales of motion contributes to part of the error budget and it is not possible to separate this representativity error from the measurement error of the satellite product with such a simple two-way statistical analysis. See (Hwang and Lavergne, 2010a, section 3.3) for a discussion on the representativity error budget of comparing a satellite drift vector to one recorded by a buoy. 3.5 Graphs and statistical measures As introduced in section 3.1, this report is mostly interested in validating drift components dx and dy. We concentrate on two comparison exercises for reporting validation results for those variables Product vs Reference In this type of graph, the x-axis is the truth and the y-axis is the estimate given by the product. In an ideal comparison, all (truth,product) pairs are aligned on the 1-to-1 line. The spread around this ideal line can be expressed by the statistical correlation coefficient ρ(reference, Product), noted ρ(r, P ). If, at the same time, this correlation is close to 1 and the parameters of the regression line are close to 1 (α, slope) and 0 (β, intercept), then the match between the truth and the product is satisfactory. In this report, a unique graph (and statistical values) is produced for dx and dy at the same time. This means that the pairs appearing on the graphs are both (dx ref,dx prod ) and

17 SAF/OSI/CDOP/Met.no/T&V/RP/ (dy ref,dy prod ). This also implies that errors in dx and dy are considered globally independent, an assumption that is validated using the graphs introduced in the next section Error in dy vs error in dx In this type of graph, the x-axis is the error in dx, that is dx prod dx ref (noted ε(dx)) and the y-axis is the error in dy, that is dy prod dy ref, noted ε(dy ). This graph is a more interesting approach for presenting the validation data than the one in the previous section. Indeed, such a graph permits giving quantitative estimates for: the statistical bias 1 in both components: ε(dx) and ε(dy ) ; the statistical standard deviation of the errors in both components: σ(dx) and σ(dy ); the statistical correlation between the errors in both components: ρ(εdx, εdy ). The last three quantities enter the error covariance matrix C obs which is of prime importance to any data assimilation scheme. It is important to note the difference between the correlation coefficient introduced in this section and the one from section ρ(εdx, εdy ) assesses if the errors in the two components of the drift vector are correlated or not. ρ(r, P ) assesses if the product (seen as a sample) is close to a linear scaling of the reference dataset (seen as a sample too). In any case, those are statistical measures of the errors. They can only give average uncertainties estimates and result in a unique set of numbers (those populating C obs ) to be used for an extended period of time (all distribution year round) and for the whole extent of the Northern or Southern Hemisphere grid. In addition, the statistics include an unknown amount of representativity error (section 3.4). 1 x is the average of x.

18 SAF/OSI/CDOP/Met.no/T&V/RP/ Results and discussions Validation of the NH and SH products are addressed separately in this chapter, with the following sub-sections: section 4.1: results from a 3 years long ( ) validation exercise against NH buoys and using AMSR2 (GCOM-W1), SSMIS (F17), and ASCAT (METOP-A); section 4.2: same as section 4.1 but against SH buoy trajectories; section 4.3: results from a shorter validation exercise to assess that SSMIS (F18) and ASCAT (METOP-B) can be used instead of the same instruments on board F17 and METOP-A; section 4.4: an assessment of the uncertainties provided with the products. 4.1 Validation over the NH area The validation results presented in section section exclude the Fram Strait and East Greenland Sea regions. These are addressed in section Graphs and analysis Figure 4.1 and figure 4.2 introduce selected validation graphs for four single-sensor OSI SAF sea ice drift products, as well as for the multi-sensor dataset. For all plots, the geographical region being validated is the Northern Hemisphere (excluding Fram Strait south of 82N) and the validation period includes all product files whose start date is between 1 st January 2013 and 31 st December In the scatterplots, the solid black line is the regression line for both dx and dy. Its coefficients are entered as labels in the plot area, together with a number of statistical results including bias and standard deviation of mismatch, as well as correlation coefficient between the product (Y-axis) and the buoys (X-axis). The right panel in figure 4.2 is a time-series plot of the monthly validation statistics for the multi-sensor product. The solid lines are for the standard deviation of the mismatch, and dashed lines are for the bias against the reference data. Figure 4.1 and figure 4.2 are a simple and effective way of presenting the validation results and get a good impression of the quality of each product. First, it can be noted that all products are mostly non biased. The magnitudes of the biases ε(dx), ε(dy ) are indeed small (maximum a couple of 100 metres) in comparison to the standard deviations (a couple of 1000 metres). The multi-sensor product is the one having largest bias, and the time-series plot seems to locate the bias during summer, when the AMSR2 GW GHz single-sensor product is in-practice the only source for drift vectors. It also clearly appears from an analysis of figure 4.1 and figure 4.2 that the algorithms implemented in the OSI SAF chain results in limited uncertainty. Displacement errors (in terms

19 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 4.1: Selected validation graphs for single-sensor products from AMSR2 GW GHz channels (top left), AMSR2 GW GHz channels (top right), SSMIS F17 91 GHz channels (bottom left) and ASCAT Metop-A σ 0 (bottom right). All pertain to NH area, to the 3D collocation setup and 1 st January 2013 and 31 st December 2015 period. N is the number of validation pairs. Note that only the top-right panel (AMSR2 GW GHz product) reports validation on an all-year-round basis, the other three are for winter cases only (Oct-April).

20 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 4.2: NH validation results for the multi-sensor MULTI-OI product, that blends all of the single-sensor products featured in figure 4.1. The statistics reported on the left-hand-side panel are for all-year-round matchups, thus including all summer months. of standard deviation) are small (about 2-3 km) when averaged over the winter periods. During summer, the standard deviations are higher and up to 6 km in June-July-August. This is still a large improvement wrt to earlier versions of the OSI SAF product, that did not process the 18.7 GHz channels of the AMSR2 GW1 instrument and did not distribute any drift vector in the period May-September. The analysis conducted so far indicates that the error distribution (when spatially and temporally averaged) can be approximated by a 0 mean, uncorrelated, bivariate Gaussian probability model. Only the standard deviations σ(dx) and σ(dy ) are to be adapted when choosing from the set of single-sensor and multi-sensor ice drift products. It is noteworthy that only the AMSR2 GW GHz product (top-right panel on figure 4.1) reports validation on an all-year-round basis, the other three are for winter cases only (Oct- April). The sea ice drift product retrieved from AMSR2 GCOM-W1 (36.5 GHz and 18.7 GHz channels) presents, by far, the smallest values for both σ(dx) and σ(dy ) (First row in figure 4.1). This limited range of errors also translates in the high correlation coefficient (ρ = and ρ = 0.954) and good regression line for these single-sensor products. This can also be visualized by looking at the vector field itself which, most of the times, looks less noisy than the ones from other instruments. This higher quality might be explained by several factors, including the smaller footprint/spacing of the two 36.4 GHz channels of AMSR2 (see the (PUM)) and the better temporal stability of their intensity patterns (compared to, e.g., those at 91GHz on SSMIS). In any case, the ice drift product from AMSR GHz channels allows statistical standard deviations of 2.29 km (2.32 km) in winter months and is the product comparing best to the reference dataset. The validation statistics obtained from the 18.7 GHz channels are slightly worse (3.7 km standard deviation in both components), but are for all-year-round conditions (thus include matchups during all summer months). This is very much in line with the results obtained with the AMSR-E (Aqua) sensor (June October 2011) as documented in earlier version of this report. The AMSR2 (GCOM-

21 SAF/OSI/CDOP/Met.no/T&V/RP/ W1) instrument is in many respect the follow-on mission from JAXA, and the good accuracy was expected. In addition, the data aquisition and downlink stream seems more reliable with AMSR2 than it was with AMSR-E, so that we can expect excellent ice motion coverage (fewer days with missing data). Ice drift estimates from SSMIS (F17) have roughly the same accuracy as was documented earlier for SSM/I (F15), with standard deviation of mismatch between satellite product and validation data of 3.95 km (3.63 km) and correlation coefficient of 0.95 (bottom-left panel in figure 4.1). As documented in earlier version of the report, ASCAT (Metop-A) estimates are those with relatively worse accuracy with standard deviations of 4.32 km (4.48 km) (bottom-left panel in figure 4.1). As is expected from a multi-sensor analysis, the multi-oi product (bottom row in figure 4.2) achieves a good (but not best ) accuracy (all-year-round standard deviations of 3.80 km and 3.77 km) and more importantly a better spatial coverage. The purpose of the MULTI-OI is exactly to provide a more consistent spatial coverage, with least possible missing vectors Impact of the collocation strategy All the validation results discussed so far were obtained using the 3D collocation scheme (section 3.3.2), that is a collocation both in space and time. The validation results obtained with the 2D (section 3.3.1) collocation scheme are slightly worse, as summarized in table 4.1. Product 3D coll. 2D coll. σ(dx) σ(dy ) σ(dx) σ(dy ) amsr2-gw1-bt amsr2-gw1-bt ssmis-f ascat-metopa Table 4.1: Validation statistics for single-sensor products in NH, obtained using 3D and 2D collocation. The columns report the standard deviation of the mismatch between satellite and validation datasets for dx and dy components of the drift vector (unit km). The statistical uncertainties slightly increase (about 200 m) from the 3D to 2D collocation for all single-sensor products. This confirms that the start t 0 and stop t 1 time information provided with the product file is relevant to use when comparing (or assimilating) the motion vectors. This is although the motion vectors are computed from daily averaged maps in this OSI SAF product. Note that this enlargement of the error statistics is here dampened by the high level of averaging occuring in our validation exercise, spatially and temporally. On a single case basis, like when a circular motion pattern is induced by a moving atmospheric low pressure, the t 0 and t 1 are quite significant and should not be neglected, as illustrated in Lavergne et al. (2008). The MULTI-OI product has all its start and stop times at D@1200 UTC and the two collocation strategies result in the same matchups and statistics.

22 SAF/OSI/CDOP/Met.no/T&V/RP/ Geographical analysis Basin-scale view The four panels in figure 4.3 show the location and magnitude of mismatch in length of 48 hour drift. The colorbar spans [ 15 : +15] km, and is thus selected to show outliers. As expected from the statistics documented above, the mismatch are generally larger for the ssmis-f17 and ascat-metopa maps than for amsr2-gw1-bt37 or multi-oi. No clear geographical pattern is seen from the maps corresponding to the single-sensor products, although the strongest colors seem to reside in the outskirt of the basin, for example along the northern coast of Canada (multi-oi product), and the Fram Strait and East Greenland Sea (most products). Summer statistics in the Trans-Polar Drift Sumata et al. (2015a) report that AMSR-E sea ice drift data from Kimura and Wakatsuchi (2000) are slightly low biased during summer season, and with enlarged uncertainty, especially in the region of the Trans-Polar Drift. We document that the OSI-405-c product also seems slightly low biased during summer in that region, with much larger uncertainties. In the Trans-Polar Drift, the mean bias (standard deviation) against buoys during the three winter seasons (Oct-Apr 2013, 2014, 2015) is 100 m (2.35 km), while it raises to -330 m (5.22 km) during the three summer seasons (May-Sept 2013, 2014, 2015). These values are not easily compared with those obtained by Sumata et al. (2015a) since they have been comparing monthly averaged velocities, not 48 hours displacements. Users of the OSI-405-c product should be aware of the potential low bias of during summer in this region. Yearly statistics in the Fram Strait and East Greenland Sea Another region of the Northern Hemisphere with very dynamic sea ice drift is the Fram Strait and East Greenland Sea, the main outflow gateway of the Arctic Ocean. In this region, the relatively coarse resolution OSI SAF drift product can be challenged by strong deformations that disable pattern recognition in the satellite images, vicinity of land, vicinity of the ice edge, strong longitudinal gradients in the motion field (bathymetry-driven East Greenland current). With these challenges in mind, table 4.2 summarizes the validation results obtained with the single-sensor and multi-sensor products in the Fram Strait and East Greenland Sea (south of 82N). The statistical bias in the first 2 columns is much larger (and negative) than in the rest of the Arctic Ocean (table 4.3). The standard deviations are also larger (but note the quite limited number of samples N). The slope α of the linear fit is also much smaller than one, even as low as 0.60 for the multi-oi product. Note that there are about 50 additional matchups for the multioi product than for the single-sensor products, which points to a fair number of interpolated vectors.

23 SAF/OSI/CDOP/Met.no/T&V/RP/ amsr2-gw1-bt37 ssmis-f17 ascat-metopa multi-oi Figure 4.3: Map showing the location and magnitude of mismatch of 48 h drift length. Blue (red) shades indicate that the satellite estimates shorter (longer) displacement than observed by the buoy.

24 SAF/OSI/CDOP/Met.no/T&V/RP/ Product ε(dx) ε(dy ) σ(dx) σ(dy ) α β ρ(r, P ) N amsr2-gw1-bt amsr2-gw1-bt ssmis-f ascat-metopa multi-oi Table 4.2: Statistical results for validation of the ice drift products in Fram Strait (south of 82N) for years 2013, 2014, and Case study: Beaufort Sea in early March 2013 As an example, a series of strong underestimations by the multi-oi product (dark blue symbols along the northern coast of Canada) are all against the same ITP41, in early March The situation for 5 th to 7 th March is illustrated on figure 4.4. A weak Beaufort Gyre is visible in the drift field which is otherwise characterized by rapid westerly drift along the canadian coast. This strong drift holds in several days and results in openings between the ice pack and the coast (shear deformation), and more open ice cover (shades of blue in the background of figure 4.4 are for ice concentration classes on March 7 th ). ITP41 (pink arrow) is in this highly dynamic region and drifts at 0.5 m/s (averaged over these two days). On the other hand, the satellite retrievals are challenged by the proximity to coast and an open ice cover, and do not allow nominal quality (black vectors) retrievals close to ITP41 (minimum distance of 70 km towards the ocean). The collocation was actually performed in that case against a interpolation estimate (violet vector closest to ITP41). The simple spatial interpolation scheme implemented in the multi-sensor algorithm does not allow to reproduce the higher coastal speeds, since in that case the interpolation really is an extrapolation. Figure 4.4 is also an opportunity to illustrate how well the OSI SAF multi-oi product matches with the drift vectors from all other drifters when these are located farther from coastal dynamical drift regions Comparison to other datasets Kwok et al. (1998), for example, report standard deviations of 8.9 km (10.8) and 9.9 (11.2) for SSM/I1 85 GHz V. pol. dx (dy ) and H. pol. dx (dy ) products, respectively. This is for a 3 days product in the central Arctic. For their 1 day dataset in the Fram Strait and Baffin Bay, those values are 5.3 km (4.3) and 6.0 (4.7) respectively. Those values are extracted from Table 2, p To be fair, one should mention that the validation exercise in Kwok et al. (1998) was performed against IABP buoys and using a 2D-type collocation (see our section 3.3.1). IABP buoys were mainly tracked with Argos positioning, which are less accurate. Kwok et al. (1998) do not state that they provide the accurate t 0 and t 1 time information which are needed for using their product in a 3D collocation strategy. Error statistics reported for the various IFREMER datasets (QuikSCAT-SSM/I merged and AMSR-E 89 Ghz) as well as by Haarpaintner (2006) are not obviously compared with our values as they are computed for the North-South and East-West components of the drift vectors. Those components exhibit non linear, latitude dependent relationships to the dx and dy we are validating. Note, however, that only the AMSR-E (89 GHz) from IFREMER and the QuikSCAT

25 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 4.4: Ice motion vectors (multi-oi) from 5 th to 7 th March 2013 superimposed with validation vectors of several drifters in the Beaufort Sea. Satellite vectors are in black (nominal quality) and violet (interpolation). In-situ vectors use the same colors as in figure 2.1. The ice mask used in background is that of March 7 th. ITP41 is the pink vector close to the Canadian coast. Note the other six in-situ vectors. product of Haarpaintner (2006) have a time span of 2 days like the OSI SAF product. The merged SSM/I and QuikSCAT dataset delivered by IFREMER is a 3 days ice drift product. An intercomparison of the IFREMER/Cersat, OSI-405 and other products was conducted by Hwang (2013), which addressed to some extent the caveats of intercomparing drift products with varying time spans. Sumata et al. (2014), Sumata et al. (2015b), and Sumata et al. (2015a) are studies dedicated to the inter-comparison and error-characterization of several ice drift dataset in the Arctic Ocean. The OSI SAF product is featured in these three studies, and performs rather well in winter conditions. Since we only now introduce summer ice drift vectors, these are not evaluated elsewhere Discussion and conclusion The validation statistics for all the OSI SAF low resolution ice drift products over NH area (including Fram Strait and all status flag values) are summarized in table 4.3. In table 4.3, the bias in the dy component of the drift is usually larger than for the dxcomponent. Besides, it is quite consistently negative. Kwok et al. (1998) (section 3.2, p. 8203) wrote a de-

26 SAF/OSI/CDOP/Met.no/T&V/RP/ Product ε(dx) ε(dy ) σ(dx) σ(dy ) ρ X,Y α β ρ(r, P ) N amsr2-bt amsr2-bt19(*) ssmis-f ascat-a multi-oi(*) amsre-aqua(**) ? ssmi-f15(**) ? Table 4.3: Statistical results for validation of the ice drift products in the whole Northern Hemisphere for years All status flag values are included and the 3D collocation scheme is used. ρ X,Y is a shortened notation for ρ(σ(dx), σ(dy )), the averaged correlation between the uncertainties on dxand dy. (*) amsr2-gw1-bt19 and multi-oi are all-year-round statitics, others are only for Oct-Apr. (**) The last two lines (AMSR-E AQUA and SSM/I F15 ) are reproduced from v3 of this report, and thus correspond to Oct-Apr ρ X,Y values was not available for these, but were also tiny numbers. tailed investigation of a similar bias in their ice drift product. Since such a limited negative bias was not observed when excluding Fram Strait matchups (section 4.1), we conclude that it is mostly driven by the more pronounced dy low bias in this region. The last two lines in table 4.3 are reproduced from v3 of the report (Table (1)) and correspond to Oct-April , and thus not the same validation dataset, although the ITPs from WHOI were already the prominent data source. These last two lines are reproduced here for confirming that, since the satellite sensors are so similar, the new AMSR2 (SSMIS) and previous AMSR-E (SSM/I) ice motion products have very similar accuracies. Transition from one to the other should be rather straightforward for users. 4.2 Validation over the SH area Figure 4.5 and figure 4.6 are equivalent to figure 4.1 and figure 4.2 but pertain to the Southern Hemisphere sea ice. In practice, due to limited buoy data availability, the only region covered is the Weddell Sea (see figure 2.2). On figure 4.5, similar validation statistics are obtained than in the NH cases, except that the ASCAT Metop-A σ 0 product (bottom right) seems of better accuracy than SSMIS F17 91 GHz channels (bottom left). In the NH cases, the ASCAT product was the worst of our single-sensor product. This relative better accuracy is however not be caused by better ice motion detection capabilities, but SH sea ice is at much lower latitudes than NH sea ice. Only the the Southern parts of the Weddell Sea is imaged on a daily basis by ASCAT, and there are no drift vectors processed in the outskirts of the Sea, where the most dynamical situation occur. This is further illustrated in figure 4.7. The monthly validation statistics for the multi-sensor product (right-hand-side panel in figure 4.6) document that the SH sea ice drift product is also more accurate during the summer period, with monthly standard deviations of mismatch peaking at 7 km. The levels during winter

27 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 4.5: Selected validation graphs for single-sensor products from AMSR2 GW GHz channels (top left), AMSR2 GW GHz channels (top right), SSMIS F17 91 GHz channels (bottom left) and ASCAT Metop-A σ 0 (bottom right). All pertain to SH area, to the 3D collocation setup and 1 st January 2013 and 31 st December 2015 period. N is the number of validation pairs. Note that only the top-right panel (AMSR2 GW GHz product) reports validation on an all-year-round basis, the other three are for winter cases only (April-Oct).

28 SAF/OSI/CDOP/Met.no/T&V/RP/ Figure 4.6: SH validation results for the multi-sensor MULTI-OI product, that blends all of the single-sensor products featured in figure 4.5. The statistics reported on the left-hand-side panel are for all-year-round matchups, thus including all summer months. are also slightly higher than in the NH. Note the much different scale for the number of validation data N: the statistics are not as robust and only for the Weddell Sea (figure 4.7). Still, this is the first time we access sufficient number of buoys to present monthly statistics over the SH sea ice. The validation statistics for all the OSI SAF low resolution ice drift products over SH area (including all status flag values) are summarized in table 4.4. Product ε(dx) ε(dy ) σ(dx) σ(dy ) ρ X,Y α β ρ(r, P ) N amsr2-bt amsr2-bt19(*) ssmis-f ascat-a multi-oi(*) Table 4.4: Statistical results for validation of the ice drift products in the whole Southern Hemisphere for years All status flag values are included and the 3D collocation scheme is used. ρ X,Y is a shortened notation for ρ(σ(dx), σ(dy )), the averaged correlation between the uncertainties on dxand dy. (*) amsr2-gw1-bt19 and multi-oi are all-year-round statitics, others are only for Apr-Oct. 4.3 Validation using SSMIS F18 and ASCAT Metop-B The validation results presented above were for a test product using SSMIS F17 and ASCAT METOP-A in addition to AMSR2 GW1. For ensuring continuity and reliabiity in operations, OSI-405-c now uses F18 instead of F17, and METOP-B instead of METOP-A. Since the switch