CLOUD MASKING FOR THE O&SI SAF GLOBAL METOP/AVHRR SST PRODUCT

Transcription

1 CLOUD MASKING FOR THE O&SI SAF GLOBAL METOP/AVHRR SST PRODUCT A. Dybbroe #, Sara Hörnquist #, Lydie Lavanant, Philippe Marguinaud #: SMHI Folkborgsvägen 1, SE Norrköping, Sweden : Météo-France, CMS, Lannion ABSTRACT The Eumetsat Satellite Application Facility on Ocean and Sea Ice (OSISAF) is heading towards the end of its Initial Operations Phase (IOP) and a proposal for the Continuous Development and Operations Phase (CDOP) is currently being finalised. During the IOP a proposal for a new product was endorsed and development of algorithms for the Global Metop/AVHRR SST started in The Global Metop/AVHRR SST product is expected to be in operation and available to users in Preoperational production should, however, start in 2007 after the end of the Metop commissioning phase. This paper focuses on the development and validation of the cloud mask algorithm for the Global Metop/AVHRR SST product. The cloud mask algorithm is technically founded in the MAIA-v3 algorithm, which is coherent with and has strongly inherited from the NWCSAF/MSG Cloud Mask version 1.2. However, scientifically the new Metop/AVHRR MAIA cloud mask seeks to inherit from both MAIA-v3 and the NWCSAF/PPS scheme. Where MAIA was originally developed for global cloud masking for the HIRS FOV, the PPS Cloud Mask was developed for local AVHRR processing and Nowcasting and for use in the high-latitude OSISAF production. The PPS cloud mask was not originally developed with global processing in mind, but rather it has been developed to cope as well as possible with the generally difficult situations at high latitudes. This project attempts to make the best possible cloud mask over sea from the two different, but still very similar, thresholding schemes, the MAIA-v3 and the PPS. Here we present and discuss recent global inter-comparison and validation results, and provide a summary conclusion for the Global Metop/AVHRR SST product. 1. INTRODUCTION During the Initial Operations Phase (IOP) of the Eumetsat Satellite Application Facility on Ocean and Sea Ice (O&SI SAF) a proposal for the development of Global Metop/AVHRR SST product was endorsed by Eumetsat. The Global Metop/AVHRR SST development started in 2005 and is led by Météo-France, with the participation of the Norwegian Meteorological Institute (met.no), the Danish Meteorological Institute (DMI), and the Swedish Meteorological and Hydrological Institute (SMHI). The project ends on April 1 st 2008, and the outcome is a real-time operational production chain of global SST products from Metop data installed and running at the Centre Météorologie Spatiale (CMS) in Lannion, France. Some milestones are depicted in Table 1.

2 Date / Time period Event / Work package January 2005 Start of Global Metop/AVHRR SST project Jul. 1, 2005 Apr. 30, 2006 Cloudmask intercomparison and convergence 1 May, Feb., 2007 Development and implementation of cloud mask processing chain 1 Mar., Mar., 2008 SST processing chain setup: Prepare, implement, document, test August 2007 Operational processing chain in place at Météo- France Table 1: Main milestones and sub-phases of the Global Metop/AVHRR SST project. As for every SST retrieval a crucial part of the Global Metop/AVHRR SST product is of course the cloud clearing step. Two options for the selection of a suitable cloud mask algorithm for this purpose was discussed at an early stage of the definition of this project, namely the MAIA and the Nowcasting SAF (NWCSAF) Polar Platform System (PPS) cloudmask algorithms. The MAIA cloudmask was selected, mainly for technical and practical reasons. The MAIA scheme is developed at Météo-France, and is already being run on NOAA data in a similar environment as planned for the Global Metop/AVHRR SST production. Former versions are implemented in the NWPSAF AAPP package and this release will be part of AAPPv6. The PPS cloudmask was first of all developed in the framework of the NWCSAF for Nowcasting applications in Europe, for the high latitude applications of the OSISAF, and in the Climate Monitoring SAF (CMSAF) for cloud climate parameters at high latitudes. Recently the scope of the NWCSAF/PPS has been widened to the global scale, and the PPS cloudmask version 2.0 to be released in early 2007 will run on full swath data (satellite projection). In order to both secure a high quality cloudmasking from the MAIA scheme, and in order to guarantee comparable quality from both the MAIA and PPS cloudmasks, both now becoming used in various SAF s at the same time, a work package aimed at intercomparing and validating the two cloudmasks on a global scale was initiated. First an intercomparison of the two schemes was performed as they were at the start of this project, then efforts to improve on both schemes were carried out, and finally both schemes were intercompared and validated. Here we present the results of this intercomparison and validation. Section 2 presents a short overview of the MAIA cloudmask, the data and results are presented in section 3 and 4 respectively, and in section 5 we briefly discuss the results and provide a summary conclusion. 2. THE MAIA CLOUDMASK The MAIA cloud mask is an independent library which can be invoked by different applications, provided the input arguments of the processed situation are correctly filled (vector of the 5 AVHRR channels, viewing geometry and position, local spatial texture). Its aim is to determine if the input situation is clear or cloudy and to classify the cloud. The MAIA v3 package is interfaced in input to AAPP level1b data (HRPT local acquisition stations) and to PFS leve1b data (global EUMETCast data). The output format is the AAPP AVHRR level1c format (full resolution, satellite projection). An additional output routine allows characterizing the clouds inside the sounder (HIRS, IASI) FOV for its implementation in the AAPP package. Soon a release will allow processing the AVHRR radiance analysis classes available at a global scale in the IASI level1c files. The software is written in fortran90 and installs under Unix and Linux. The cloud detection algorithm is a succession of thresholds tests applied to every AVHRR situation to various combinations of the AVHRR channels. It follows the scientific algorithm developed in Derrien and LeGleau, A situation is said to be cloudy if one test is not satisfied (so a pixel is said to be clear if all tests are satisfied). The series of tests applied depend on the surface type (land, sea or coast) and the solar zenith angle which determines the period of the day (day, twilight or night) and on the presence of specular reflection during the daytime (sunglint). Daytime period is defined by solar zenith angle less than 83 degrees, night-time period by solar zenith angle higher than 90 degrees and twilight if the angle is between 83 and 90 degrees.

3 The surface type and elevation are input optional arguments of the library, and if not provided, the MAIA scheme makes their determination from the situation position using ancillary datasets. Depending on the surface type, daytime period and presence of sunglint, different series of tests and threshold values are invoked. Successions of tests for each marine case are described in table 2. Daytime Sunglint Twilight Nighttime Ice detection Ice detection Ice detection SST SST SST SST T3.7μm-T10.8μm R0.8μm T10.8μm-T12.0μm T10.8μm-T12.0μm T10.8μm-T3.7μm R1.6μm R0.8μm T10.8μm-T3.7μm T12.0μm-T3.7μm T10.8μm-T12.0μm Low Clouds in sunglint T12.0μm-T3.7μm T10.8μm-T12.0μm T10.8μm-T3.7μm T10.8μm-T3.7μm T3.7μm-T10.8μm Marine texture T3.7μm-T10.8μm Marine texture R0.8μm Local Spatial Texture Marine texture Local Spatial texture R1.6μm Local Spatial texture Marine texture Local Spatial Texture Table 2: MAIA test sequence over sea The tests are done on different combination of channels, in brightness temperature, (ex: μm, μm, μm...), on local spatial texture of channels 1,2,4,3-4 computed on a 3*3 box centered on each AVHRR pixel. R0.6μm, R0.8μm and R1.6μm stand for VIS/NIR bi-directional top of atmosphere reflectances at 0.6, 0.8 and 1.6 μm normalised for solar illumination. SST is the splitwindow (used for SST calculation in the O&SISAF) computed from T10.8μm and T12.0μm measurements and compared to the climatological SST. Low Clouds in Sunglint is a specific module for low clouds identification in sunglint areas. Marine texture is a relaxed local spatial texture on channel 10.8μm which reduces the false detection of the ocean fronts as clouds. Most thresholds are determined in-line from satellite-dependent look-up tables using as input the viewing geometry (sun and satellite viewing angles), NWP forecast fields (surface temperature over land and total atmospheric water vapour content land and sea) and ancillary data (elevation, climatological minimum SST over sea and maximum reflectance over land). To save computing time and in coherence with the external information spatial resolution, the thresholds are computed at a spatial resolution defined by the user as a number of AVHRR pixels and lines (often 16x16). The lookup tables are computed using the RTTOV fast forward model (Saunders, 1999) for IR channels over the ECMWF climatological profiles dataset (Chevallier, 1999) and 6S (Vermote, 1997) for the visible channels. They are recomputed for each new satellite. Some thresholds are set empirically or satellite-dependent constants, determined from a long experience over the Europe acquisition area and from global training and validation targets data. Two sets of monthly global climatologies at a 0.1 degree resolution are provided with the software for the sea surface temperature over sea and the R0.8μm visible reflectance over land. A specific humidity climatology at a 5 degree resolution is also available but only used when the forecast is lacking. Elevation and surface type datasets are provided with the package at a 0.1 degree resolution. The extracted forecast values at the pixel position are the surface air temperature (only used over land), the total water vapour content (TWVC), the surface pressure, the air temperature at three standard levels (used for the cloud classification) and the geometric altitude. Two formats for reading forecasts are available: GRIB (WMO standard format) and ASCII. Over sea, the TWVC can be directly determined from co-registered AMSU-A regression (optional input argument) and the forecast value is not used. The quality of the cloud detection process is assessed. A test is then applied to cloud contaminated pixels to check whether the cloud cover is opaque and completely fills the FOV. When a situation is flagged cloudy, a further process is done to determine its

4 cloud type. The input AVHRR channels vector goes through a classification tests sequence governed by its illumination (day, night, dawn), with the same philosophy for computing the thresholds. Ten cloud categories (identical to those defined in the NWCSAF) are defined: five opaque cloud classes according to their altitude: very low, low, medium, high and very high three semi-transparent classes according to their thickness: thick, mean and thin one class of semi-transparent clouds above lower clouds one fractional clouds class 3. DATA For the intercomparison of the MAIA and PPS cloudmask schemes over sea we chose to use AVHRR and MODIS data collected and classified by experienced nephanalysists, and where possible also guided by objective ground truth as provided by ground based cloud radar and lidar. In total five different oceanic datasets with interactively collected AVHRR (and AVHRR like) training and validation targets was selected. The concept of these training and validation target data are described in Derrien and LeGleau, 2005, Lavanant, 2002 and Dybbroe et al., The datasets are named SMHI European (figure 1), CMS European (figure 2), SMHI Arctic (figure 3), CMS Global (figure 4), CMS Modis (figure 5), explaining both the coverage and the origin. Clear situations are in blue, cloudy targets in green and sea-ice in red on the figures. Figure 1: Dataset of targets collected at SMHI using the HRPT station in Norrköping, Sweden. Figure 2: Dataset of AVHRR targets collected at CMS using the HRPT station at Lannion, France. Figure 3: Dataset of NOAA LAC data collected at SMHI with targets over the SHEBA area and near the ARM site at Barrow, Alaska. During nighttime target collection was guided by data from ground based lidar/radar at the ARM site.

5 Figure 4: Global dataset collected at CMS, using various local HRPT stations around the world Figure 5: Dataset of MODIS targets collected at CMS. MODIS data taken from the EOS archive in USA In addition to the training target datasets we also looked at full HRPT NOAA imagery from both low/mid and high latitudes. We chose 6 full HRPT scenes from the acquisition station at La Reunion in the Indian Ocean, and 6 full HRPT scenes from the acquisition station in Kangerlussuaq, Greenland (Courtesy DMI). 4. MAIA-PPS CLOUDMASK INTERCOMPARISON RESULTS Contingency tables have been computed using results of the MAIA and PPS cloud masks applied to the AVHRR observations from the target datasets. Cloud detected Clear detected Cloud observed n a n b Clear observed n c n d Statistical indicators derived from the contingency tables have been computed: PC= (n a +n d )/(n a +n b +n c +n d ) is the percentage of correct detections. It should be as large as possible. (1.-POD Cloud )= n b /(n a +n b ) is the rate of missed cloud observations, i.e. targets classified as cloudfree but observed cloudy. It should be as low as possible mainly for SST application. (1.-POD Clear )= n c /(n c +n d ) is the rate of missed clear observations, i.e. targets classified as cloudy but observed clear. It should be as low as possible. Statistical indicators are associated with changes in latitudes conditions (Nordic for 55N-90N and 90S- 55S, Mid-latitude for 20N-55N and 55S-20S, Low-latitude for 20S-20N), scene background (sea or sea-ice) and illumination conditions (day, night, twilight, sunglint). Each target is a 5x5 pixels extraction in a full scene but the performances are analysed at the center pixel because of the local texture definition (on 5x5 pixels for PPS and 3x3 for MAIA). Tables 3 to 6 present the statistical indicators after updating both MAIA and PPS for the 5 target datasets together.

6 The environmental category conventions (ex: viewing angle for the transition between night and twilight, sunglint geometry) are different in MAIA and PPS. Tables 3 and 4 present the statistics with their own conventions introducing slightly different number of situations in both software categories. However, this has no real impact on the different results and for an easier comparison, tables 5 and 6 show the statistics using the MAIA convention. Figures 6 and 7 show two examples of visual comparison of full HRPT acquisitions, on polar (Kangerlussuaq) and tropical (La Réunion ) area. 5. DISCUSSION AND CONCLUSION MAIA and PPS cloudmasks proved very similar over sea. Most of the problems identified at the start of this project have been solved for both masks: About 93-94% of clear+cloudy situations are correctly classified in all categories Only about 6% of clouds and 7% of clear situations are missed on average. In the details, more clear situations are missed at nighttime conditions (about 10-11%) and in contrary; more clouds are missed in twilight, mainly in 1.6μm conditions (about 15-18%). MAIA cloud detection is slightly more severe compared to PPS at nighttime and PPS tends to slightly miss more clouds than MAIA. One reason for this can be found in the more strict texture thresholding used in MAIA for this study. However, in MAIA an additional test uses a relaxed threshold on the local texture which allows O&SI SAF to monitor sea surface thermal fronts. This study pointed to some remaining detection problems over sea: Ice-detection using 3b still not satisfactory for both schemes, but more data are needed. Semi-transparent cirrus cloud detection over sea ice is difficult for both schemes The problems related to cloud detection during the polar night are not sufficiently challenged with this dataset. The visual inspection of HRPT passes indicates that many Very-Low clouds in MAIA are classified fractional in PPS (see figure 7) but it is difficult to judge which is more correct. Over snow and ice, PPS better classifies the cirrus clouds as semi-transparent when MAIA tends to classify them as opaque clouds. MAIA correctly detects sea-ice around Greenland when PPS classifies cloudy (see figure 6). Following this intercomparison stage, the updated MAIA scheme has been implemented in the Global Metop/AVHRR SST processing chain and a new validation effort of the cloudmask will be conducted as soon as Metop AVHRR data are available. 6. REFERENCES Chevalier, F., Sampled databases of 60-level atmospheric profiles from the ECMWF analyses. NWP SAF research report no4. Derrien, M. and Le Gléau, H., MSG/SEVIRI cloud mask and type from SAFNWC, Int. J. of Remote Sensing, Vol. 26, No. 21, pp Dybbroe, A., Karlsson, K.-G. and Thoss, A., NWCSAF AVHRR cloud detection and analysis using dynamic thresholds and radiative transfer modelling - part one: Algorithm description, J. Appl. Meteor., Vol 44, No 1, pp Lavanant, L., MAIA v3 AVHRR cloud mask and classification. EUMETSAT contract documentation. Available at Saunders,R., M. Matricardi, and P. Brunel, 1999: An improved fast radiative transfer model for assimilation of satellite radiances observations. Quart. J. Meteor. Soc., 125, Vermote, E., Tanré D., Deuzé L., Herman M. and J.J. Morcrette, Second simulation of the satellite signal in the solar spectrum, 6S : an overview. IEEE Trans. Geosci. Remote Sens., 35, pp

7 MAIA all daytime nighttime twilight glint ice all high mid low all high mid low all high mid low all all nb PC % '1' % '2' % table 3: MAIA performances on all targets. Both channels 3a and 3b displayed together. PC stands for percentage of correct detections. '1' stands for (1.-POD Cloud ) the rate of missed cloud observations, and '2' stands for (1.-POD Clear ), the rate of missed clear observations. Values are the nearest integer in percent. For ice category, '1' stands for 'ice observed but classified cloudy', '2' stands for 'ice observed but classified clear'. PPS all daytime nighttime twilight glint ice all high mid low all high mid low all high mid low all all nb PC % '1' % '2' % table 4: PPS performances on all targets. Channels 3a and 3b together. See table3 definition. all daytime nighttime twilight glint ice all high mid low all high mid low all high mid low all all nb PC % MAIA PPS '1' % MAIA PPS '2' % MAIA PPS table 5: MAIA and PPS performances on all targets.channel 3a. See table3 definition. all daytime nighttime twilight glint ice all high mid low all high mid low all high mid low all all nb PC % MAIA PPS '1' % MAIA PPS '2' % MAIA PPS table 6: MAIA and PPS performances on all targets.channel 3b. See table3 definition.

8 Unprocessed Cloudfree land Cloudfree sea Snow covere Sea ice Very low clouds Low clouds Medium level clouds High clouds Very high clouds Very thin cirrus Thin cirrus Thick cirrus Cirrus above low/medium Fractional clouds Unclassified Figure 7: Visual inspection on HRPT pass. Kangerlussuaq: NOAA :18 UTC. Left figure is the RGB multi-composite image 1,2 and 4. Middle: PPS mask. Right: MAIA mask. Figure 8: Visual inspection on HRPT pass. La Réunion: NOAA :16 UTC. Left figure is the RGB multicomposite image 1,3a and 4. Middle: PPS mask. Right: MAIA mask.