Read-me-first note for the release of the SMOS level 2 Soil Moisture data products

Transcription

1 Read-me-first note for the release of the SMOS level 2 Soil Moisture data products Processor version Release date by ESA Authors Further information Contact for helpline Comments to ESL Level 2 soil moisture team Level 2 Soil Moisture v 5.51 (OPER data set) Level 2 Soil Moisture v 5.51 (REPR data set) 07 June 2012 for the operational (OPER) data set 09 August 2013 for the reprocessed (REPR) data set: catch-up soil moisture reprocessing campaign Expert Support Laboratory Level 2 Soil Moisture + ARRAY Details on the processing algorithms can be found in the Algorithm Theoretical Baseline Document (ATBD, version 3.6), and on the L2SM products structure in the SMOS Level 2 and Auxiliary Data Products Specifications (SO-TN-IDR- GS-0006, Issue ), both available from ARRAY ( or the CESBIO blog ( or ESA ( For all issues related to data access, formats and read/write, processors contact ESA s HelpDesk on eohelp@esa.int. The Level 2 soil moisture team would like to get your feedback on the product, either directly (yann.kerr@cesbio.cnes.fr, ali@array.ca) or through the BLOG ( ) where you can also find the latest news! Document history This note was first issued in October 2011 after the deployment of the Level 2 Soil Moisture Operational Processor (L2SM) V5.00 in the SMOS processing. L2SM v5.00 implemented substantial corrections and improvements as result of the evolution work done in the commissioning phase as well as an improved RFI detection algorithm which considers a temperature threshold linked to the surface expected emissivity rather than a fixed 340 K threshold. The note was updated in March 2012 to announce the delivery of the first soil moisture mission reprocessed data set for the period 12 January 2010 to 28 November 2011 generated with the L2SM V5.01, which includes the multithread version of the L2SM V5.00 processor (otherwise identical to V5.00) A further update was issued in April 2012 following the deployment of the L2SM V5.51 processor in the SMOS processing and again in June 2012 to better characterize the changes introduced through the L2SM V5.51. The main difference between the V5.51 and V5.01 (V5.00) processors was the change of the dielectric constant model from the Dobson model with Peplinski's modification (all processor version prior to v5.51) to the Mironov formulation (from v5.51 onward).

2 This release concerns a so called catch-up soil moisture reprocessing, i.e. aligning the entire soil moisture data archive to the currently used L2SM processor baseline V5.51. The catchup reprocessing period hence spans from 12 January 2010 to 27 April The precise start and stop acquisition time of the catch-up reprocessed data set are: T and T The level 1C dataset used as input for this catch-up soil moisture reprocessing campaign is detailed in the following Table-1: Table-1 Input L1C dataset used for the L2SM catch-up reprocessing campaign Level 2 SM catch-up reprocessing campaign (REPR dataset) Input L1C used Time format is yymmddthhmmss From: T To: T REPR (V5.05) From: T To: T OPER (V5.05) Note that the L1C REPR V5.05 is expected to be slightly better calibrated as it corresponds to a reprocessing period where we can use an optimal calibration scheme (in retrospect). The impact in the brightness temperature measurements is minimal and the soils moisture retrieval is only marginally impacted. Product description SMOS was launched on November 2 nd The commissioning phase ended in May Since then SMOS has been in routine operations. This note describes the improvements and known limitations in the quality of the SMOS Level 2 Soil Moisture data products generated by V5.51 of the Level 2 Operational Processor. Version 5.51 of the Soil Moisture (SM) level 2 processor, which includes substantial corrections and improvements as result of the evolution work done since the Commissioning Phase, is now ready to deliver the best soil moisture products available, although still not reaching the expected quality. Studies are currently under-way to improve RFI detection, localisation and flagging. But we are not there yet. Vegetation opacities are still not fully compatible with what we would expect from NDVI-like measurements. This is currently under investigation and any feedback from users and CalVal team is most appreciated. The usage of the Mironov formulation for the dielectric constant model in the L2SM V5.51 improves soil moisture estimates and, increases the number of better and more successful retrievals over dry warm surfaces, and reduces extreme values of soil moisture as shown in Figure 1. First analysis of the data shows that the new formulation increases significantly (more than 0.04 m 3 /m 3 globally) the average retrieved soil moisture as well as the spatial distribution of those differences. The ESLs are investigating this further but the very first analysis seems to indicate that Mironov s formulation is better. GREAT CARE should be taken when studying temporal evolution of soil moisture when mixing L2SM V5.01 and L2SM V5.51.

3 With the catch-up reprocessing campaign data now available, we strongly encourage users to get the reprocessed data set V5.51 and the operational data set V5.51 for their research activities. It offers a consistent and stable algorithms and products version for the last 3 years and half of SMOS observations. Figure 1 Retrievals over Australia using the same input data and approach but the Dobson model (top) and the Mironov model (bottom) Several issues are still under investigation by the ESLs and ARRAY s development team and progress is under way. In this respect, help and feedback from users are strongly appreciated. The following comments have to be taken into account for a proper understanding, interpretation and assessment of the present SM products. 1 Overall remarks The products released are far from perfect: As they are not final, in case of problems, please provide us with feedback on flaws and issues you may identify.

4 Note that the SMOS SM products are not produced as are many other similar products, so read carefully the ATBD to see exactly what is being done (as it is not always standard practice) so as to take advantage of SMOS characteristics The final SMOS L1 product (L1c) consists of brightness temperatures reconstructed from interferometric data in the reference frame of the SMOS antenna plane. Hence these radiometric data are associated to upwelling Stokes parameters through a transformation combining the Faraday effect and geometrical factors. SMOS brightness temperatures are NOT TBH, TBV, ST3 or ST4 and CANNOT be compared directly with Earth surface observation or modelling. Soil moisture retrieval efficiency: at L2, the SMOS soil moisture retrieval is based on matching measured and modelled (surface emission) brightness temperatures, with the modelled values varying as a function of the incidence angle and depending on soil moisture as well as other physical parameters. There is definitely, for a very large fraction of nodes of the SMOS grid, a robust ability of the radiative model to match the angular signature of brightness temperature while producing realistic values of retrieved parameters. Polarisation mode: According to the End-of-Commissioning review decisions, the full polarization acquisition mode has been selected for the operational phase, accounting for the potential information provided by this mode. However, until full understanding of Stokes 3 & 4 parameters has been achieved, pseudo dual pol" will be used for retrievals over land surfaces; i.e. only the antenna level brightness temperatures corresponding to Stokes 1 and 2 will be used in the retrieval. Note that Stokes 3 and 4 are not yet up to expectations. The level 2 algorithm does not us the cross pol terms in the retrieval yet (too noisy). Note that brightness temperature polarisations are always given in the antenna and can only be transformed into ground values (i.e., H and V) through a transformation related to the instantaneous view angles. Only points located on the satellite subtract are (almost) in H and V. Note that a matlab tool is provided on the blog ( to perform this task

5 2 Brightness temperatures Two main areas where progress is needed concern the correction of biases and the detection/mitigation of spurious signals. 2.1 Biases Beyond the average biases which are brought down to satisfactory levels through calibration procedures, there are still imperfectly corrected biases which depend on the location within the instantaneous field of view. Such errors may be generated during the reconstruction process in case the detailed instrument properties (e.g. element antenna pattern) are not perfectly known. Several such problems (across swath spurious trends, pixel biases, so called land contamination) have been detected, identified, and sometimes empirically mitigated by the ocean team; although these errors are less prominent over lands due to the much larger sensitivity of brightness temperatures to physical properties, they are present all the same. A clue to their presence is that the quality of fit is not down to expectations, as illustrated by the fact that the normalized cost function (χ 2 -CHI square) is substantially higher than the theoretically expected value of Spurious signals On one hand, no significant impact of sun contamination (glint) has been detected, so far, over land. On the other hand, spurious signals due to radiofrequency interference (RFI) are much larger and more frequent than expected when knowing that the SMOS bandwidth is - in theory - fully protected. The most efficient mitigation technique consists in requesting trespassers to stop transmitting in the protected MHz band. Indeed this is under way and meets encouraging results. Meanwhile, other mitigation techniques are explored. Several detection algorithms are being used in the L2 codes. The most robust have been used to build maps which depict, for each SMOS grid node, the measured frequency of occurrence of RFI detection over a sizeable SMOS observation period. The following map shows this probability for descending orbits. Unfortunately, small RFI contributions (for example 20K or less) often go undetected over land.

6 Figure 1.a RFI probability for 15-8 to with V4.00 RFI detection algorithm Figure 3.b RFI probability for 15-8 to with V5.00/V5.01 RFI detection algorithm

7 Figure 3.c. Difference between figures 1b and 1a. Figure 2: example of RFI of low amplitude Note: UDP has a new field which is RFI_Prob. It corresponds to the RFI Probability computed from the previous day AUX_DGGRFI (see last version of the ATBD). /!\ => to those having developed their own hard coded reader tools need to update it. This added field is coded on one byte.

8 3. L2 SM algorithm 3.1 Primary decision tree As described in the ATBD, the L2 algorithm is somewhat complicated. This is due to the land surface being very inhomogeneous. Some basic information is given below. For each node of the SMOS grid, the concerned area (the working area: 123 km x 123 km box when including the minor contributions) is described as including several fractions: vegetated soil of course, but also ice, forest, open water and the like. Then, the first decision tree (17 branches) selects which of the fraction(s) will carry the retrieval. This depends on the respective weights of the various land use types, including also consideration on topography. Let us for example assume part of the scene consists of vegetated soil while another fraction consists of open sea. Since the sea fraction is not relevant for retrieving soil moisture, the radiometric contribution due to sea is computed using auxiliary data and it is considered as a default contribution. This is only an example; indeed, the forward modelled brightness temperatures used in the retrieval will integrate most of the time default contributions. Note that the retrieved soil moisture corresponds only to the area where the dominant land use is present. 3.2 Radiative models There are basically 3 radiative models in the L2 algorithm, depending on how the dielectric constant of the surface is computed. They can be used either in the retrieval iterative loop, or simply in order to build default contributions. The nominal model is the standard soil vegetation radiative transfer model and includes soil moisture; The water surface model is used for sea in coastal pixels, wetland and lakes The so-called cardioid model is used for retrieval whenever it can only aim at providing information about the dielectric constant itself (e.g ice, barren surfaces). The nominal model will be used over vegetated soil and forest. While these cases are the only ones of direct interest as far as soil moisture is concerned, it may be mentioned that preliminary results using the cardioids model suggest there is indeed physical meaning in them. It is important to note that a value of -999 for a geophysical parameter and its associated DQX in the UDP implies either the retrieval for that parameter failed or was not attempted. When all retrieval attempts fail for a node the FL_NOPROD is set to 1, otherwise it is set to 0.

9 3.3 Secondary decision tree Depending on the content of the working area, one of 3 radiative models is used for retrieval. For each of them, one must then define which parameters are to be retrieved, and what are the constraints assigned to the initial values. The secondary decision tree lists, for each of the 3 models, 3 options depending on the expected vegetation optical thickness and 3 options depending on the "information richness" expected from the data, which is estimated from the incidence angle coverage. As expected the commissioning phase made clear that this scheme with 27 options was too complicated and it will be simplified during the operational phase. The original idea was to cover all possible cases and adapt and reduce them after the commissioning phase as a function of what the real data looks and behaves. 4 Quality of the results Some caveats are already identified and work is in progress (but help is appreciated): 1. The quality figures DQX (theoretical retrieval uncertainty) and GQX (overall quality) can only be considered as indicative so far. 2. Actually these quality figures do not reflect the poor quality of the fit (see possible explanations for this above) resulting in χ 2 too large. 3. Concerning the radiative nominal model, some weaknesses have been detected. The behaviour of the model for very small soil moisture values produces too optimistic DQX in these regions. 4. Moreover, the retrieval performance over the forests is poor (about 40% instead of about 75% elsewhere); what happens is that the model tends to converge on a secondary minimum for negative SM values, which are finally rejected. Efforts are underway to improve the forest parameterizations and increase retrieval efficiency. The results will appear in the later updates of the L2 algorithm. 5. In too many cases the retrieved soil moisture is negative. Several causes are identified and under consideration: i) The dielectric constant model used for soil is not adapted to dry sandy soils; ii) the contributions from the areas surrounding the area of interest where the retrieval is done are estimated using ECMWF soil moisture which very often over estimates soil moisture. With the Mironov dielectric model implemented in V5.51 results will improve over dry, sandy soil. The ECMWF overestimates will be improved by matching the ECMWF range to the real one, to appear in future updates. 6. Information about the results of the Systematic Product Quality Control analysis for the L2SM catch-up reprocessed data set is available on the ESA SMOS web-page: under the section Reprocessing reports. The user shall consider this information in the usage of the level 2 soil moisture catch-up reprocessed dataset. 5 Conclusions The data is now yours to evaluate. Please let us know the issues encountered (on scientific points!) so that we can try to fix them for the next general - release.

10 We are looking forward to collaborating with you on these topics and make SMOS products even better... Yann, Philippe and Philippe, Ali and Ali, Ahmad, Arnaud, Cecilia, Claire, Delphine, Elsa, François Jean-Claude, Jean Pierre, Jennifer, Nathalie, Ning, Olivier, Paolo, Rachid, Silvia, Steven, Susanne, Tim, Yan