Status and further development of CryoClim global Snow Cover Extent product

Transcription

1 Status and further development of CryoClim global Snow Cover Extent product Rune Solberg 1, Øystein Rudjord 1, Arnt-Børre Salberg 1 and Mari Anne Killie 2 1) Norwegian Computing Center (NR), P.O. Box 114 Blindern, NO-0314 Oslo, Norway 2) Norwegian Meteorological Institute (MET Norway), P.O. Box 43 Blindern, NO-0313 Oslo Presentation at the WMO GCW 2 nd Snow Watch Team Meeting, Columbus, Ohio, USA, June 2016

2 Snow Cover Extent (SCE) product A sub-service of CryoClim ( Binary product based on three component algorithms developed in the CryoClim project: SCE from PMR (10 km) based on SMMR ( ) and SSM/I (1987-present) SCE from optical (5 km) based on AVHRR GAC (1982-present) SCE multi-sensor/temporal (5 km), a time-series combination of optical and PMR Global product (NH + SH) time series 1982 present Aggregation levels: Day, month, year Projection/files: EASE-Grid, NetCDF CF, Northern & Southern Hemisphere Climate-change indicator products: Snow season length, first and last day of snow

3 CryoClim Snow Phase 2 objectives 1. Mitigate weaknesses in the Version 1.0 single sensor components of the algorithm (optical and passive microwave radiometers) and multi-sensor/multitemporal data fusion to further increase the accuracy and robustness of the product. 2. Extend the product with uncertainty estimates at the product and per-pixel levels. 3. Advance the algorithms and processing chains with the inclusion of Sentinel-3 OLCI and SLSTR data. 4. Perform more extensive validation of the product in space and time, including focus on inter-sensor issues in the time series. 5. Include the results in the CryoClim processing chain for snow and advance the operational level of the processing. Project period

4 The PMR SCE algorithm is based on an estimate of the probability of snow PP(SS kk xx 1, xx 2,, xx nn )= SMMR Snow classes: Snow & no snow Features: x1=t18v-t37v x2=t18h-t37h PP xx 1 SS kk PP xx 2 SS kk PP(xx nn SS kk ) PP(SS kk ) CC mm=1 PP xx 1 SS mm PP xx 2 SS mm PP(xx nn SS mm ) PP(SS mm ) SSM/I Snow classes: Dry snow, wet snow, no snow & no snow with a large portion of water Features: x1=t37v-t37h x3=t22v-t85v x2=t19v-t37v x5=t22v x4=(1.95 T19v-0.95 T19h)/0.95 SMMR SSM/I F SSM/I F SSM/I F15

5 The optical AVHRR GAC SCE algorithm is based on an estimate of probability of snow PP(SS kk xx 1, xx 2,, xx nn )= PP xx 1 SS kk PP xx 2 SS kk PP(xx nn SS kk ) PP(SS kk ) PP xx 1 SS mm PP xx 2 SS mm PP(xx nn SS mm ) PP(SS mm ) CC mm=1 prob. for snow prob. for cloud prob. for bare ground

6 A state model based on fusion of single-sensor state models Optical Multi-sensor PMR

7 Implemented the model applying the Hidden Markov Model framework In HMM we observe a system assumed to evolve through a series of different states States: Observables: QQ = SS 1, SS 2,, SS vv XX TT = XX 1, XX 2,, XX TT Transitions from one state to another happen with certain probabilities While in a given state the system will produce observables with a certain probability density Note that a there is one state model per grid cell Prob. distr.: Transition probabilities.: pp XX tt EE tt = SS ii, ii = 1, 2,, vv pp EE tt = SS ii EE tt 1 = SS jj, ii, jj = 1, 2,, vv Initial conditions: pp(ee 1 = SS ii ), ii = 1, 2,, vv

8 Training the algorithm Pr(S) p(snow O ) 0 p(snow O ) 1 p(bare ground snow) 0 p(snow O ) 0.5 Time Climatological probability of snow Per grid cell daily climatological probability of snow computed from Savitzky-Golay smoothed PMR snow probabilities Used to estimate transition probabilities

9 Using the Viterbi algorithm to determine the model sequence best explaining the temporal observations The Viterbi algorithm is a dynamic-programming algorithm for finding the most likely sequence of hidden states (the Viterbi path) that results in a sequence of the observables The algorithm requires as input the state probability density functions, the transition probabilities between the different states and the initial probability of each state VV tt,kk = pp XX tt kk max ii VV 1,kk = pp XX 1 kk pp(ee 1 = SS kk ) pp EE tt = SS ii EE tt 1 = SS jj VV tt 1,kk Final state model chosen

10 Snow season 2005

11 Mitigation of product errors Scattered snow in the summer. Corrected by increasing run-in time from 7 to 15 days Processing errors creating false snow corrected Errors in metadata and static data fixed: Missing pixels in lat/lon grid around poles Area outside map marked as water Land mask error around date line Misc. errors in metadata fixed Uncorrected (left) and corrected (right) product, Asia, July 25, 2002 North America, 30 Aug

12 Remaining known issues Too much snow in Himalaya and surrounding regions in summer: Originates from training of the PMR snow cover product Related to resampling of land cover data Some permanently snowfree shorelines Seems to be a water-land mixture problem Himalaya, 15 July 2005 Systematic lack of snow in some coastal regions 12

13 Optical: Overall performance Overall very good performance Snow cover underestimated in spring 13

14 Improvements optical: Solar angle influences the PDFs No snow The spectral signatures applied are relatively independent of solar angle, but change character for low solar angles High probability for snow Current version: static PDFs. Can give false snow cover for low solar angles OSI SAF SST work

15 Uncertainty estimate, optical: Approach Optical snow cover product: Step 1: AVHRR swaths are processed Step 2: Swath products from one day are combined Goal: Retrieve per pixel uncertainty estimate for aggregated product. First step: uncertainty estimate per swath product pixel Plan: First approach is to combine U = 1 P and distance to nearest «other class» Tune this until the uncertainty matches the hit rate when comparing with synop snow observations 15

16 Uncertainty estimate, optical: Mixed spectral signatures MetOp-01 May 27th 2016, 09:37 UTC For the «signature» Ch3/Ch1, cloud is between snow and snow free. land snow cloud

17 Uncertainty estimate, optical: Aggregated, daily observations Daily product May 27th 2016 Elements to be included in swath product uncertainty estimate typically «matches» troubled area U = 1 P Distance to nearest «other class» Snow free Snow Cloud Aggregated product is not so much affected

18 Uncertainty estimate, multi-sensor: Multi-sensor uncertainty challenging Very difficult to estimate the snow cover uncertainty due to the complexity of the finite state hidden Markov model More or less impossible using classical uncertainty estimation methods like the delta method. Methods like bootstrapping is not practical since the computational load of the multi-sensor multi-temporal snow cover method is high. Moreover, bootstrapping on the training data will not capture all uncertainties. Principle of estimating the uncertainty from the likelihood The figure shows two probability density functions corresponding to two classes (red and green). The x-axis denotes the observed data value The dashed line is the decision boundary. The further to the right the observed data is along the x-axis, the more certain we are that the red class is the correct one. 18

19 Uncertainty estimate, multi-sensor: Cumulative log likelihood The cumulative log-likelihood of the state ωω tt, given the data XX tt = xx 1,, xx tt may be written as LLLL ωω tt XX tt = LLLL ωω tt 1 XX tt 1 + log pp xx tt ωω tt + log pp(ωω tt ωω tt 1 ) data probability density The cumulative likelihood of each state provides a measure of how likely a given state is at a given time instant. However, the sequence backtracking performed by the Viterbi algorithm may results in that the selected state does not correspond to the largest cumulative likelihood. We first estimate the probability of a given state ss from the log likelihood exp LLLL(ss XX) pp LLLL (ss) = ii exp (LLLL(ii XX)) transition probability We may also estimate the data probability corresponding to state ss at time tt pp dd (ss) = exp log pp xx tt ss ii exp (log pp xx tt ii ) 19

20 Uncertainty estimate, multi-sensor: The solution data probability When analysing the probability corresponding to the cumulative likelihood of the selected states we observe that the probability is more or less binary with a larger fraction of 1s and a smaller fraction of 0s (see figure). Probability values between 0 and 1 barely exist. This binary behaviour of the probability means that it is not suited as a proxy for the uncertainty. Histogram of the probability of selected states When we consider the data probability of the selected states we observe that the data probability is not binary. We therefore further investigate the data probability as a proxy for the uncertainty Histogram of the data probability of selected states 20

21 Uncertainty estimate, multi-sensor: Estimated from data probability Here we divide the data probability into five equal bins, and count how many times the estimated snow cover agrees with the ground truth snow-cover observations (met. stations) We observe that the accuracy decreases as the data probability decreases. The line is a calibrated estimate of the snow cover uncertainty High data probability corresponds to high accuracy Low data probability corresponds to low accuracy 21

22 Uncertainty estimate, multi-sensor: Example Northern Hemisphere White and red areas corresponds to certain snow cover estimates (accuracy close to 1) Violet areas corresponds to uncertain snow cover estimates (accuracy close to 0.5; pure guess ) Black areas are water bodies

23 Uncertainty estimate, multi-sensor: Example Europe White and red areas corresponds to certain snow cover estimates (accuracy close to 1) Violet areas corresponds to uncertain snow cover estimates (accuracy close to 0.5; pure guess ) Black areas are water bodies

24 Validation Datasets for validation: Snow depth from the Global Historical Climatology Network Daily (GHCN-D) SYNOP database applied in Phase 1 of product development Daily snow depth observations, including recent years Historical Soviet Daily Snow Dataset (HSDSD) Daily measurements of snow depth and snow cover from meteorological stations in former Soviet Union Former Soviet Union Hydrological Snow Surveys (FSUHSS) Snow transects from meteorological stations in former Soviet Union Russian meteorological stations (RIHMI) preliminary test confirms overall accuracy Ideally, should include a newer dataset covering more recent years and areas outside former Soviet Union, e.g.: more GHCN-D 24

25 Global Historical Climatology Network Daily (GHCN-D) Data set applied in the first phase of snow product development Daily snow depth observations from 2005 Filtering out obvious errors: Like stations with suspicious behaviour related to that zero snow depth not reported explicitly Validation results (2005): Very high accuracy in summer Somewhat lower accuracy in November-January Slightly lower accuracy in April Month Accuracy Number of samples January February March April May June July August September October November December Total

26 Historical Soviet Daily Snow Dataset (HSDSD) Around 280 stations in former Soviet Union Daily snow depth and snow cover measurements Until 1995 (currently 3.5 years of overlap) Filtering and assumptions: Using only stations with coordinates in GHCND data (more accurate). Consider less than 50% snow cover as bare ground. Remove anything slightly suspicious. All fields flagged as humidity measurements are assumed to be snow free. Validation results for the period 1 Aug Dec 1995: High accuracy in summer Lower accuracy in October-November Slightly lower accuracy in April Month Accuracy Number of samples January February March April May June July August September October November December Total

27 Former Soviet Union Hydrological Snow Surveys (FSUHSS) Large number of snow transects in former Soviet Union measured until 1996 (1345 stations in total, not all in recent years) km long transects, usually three per month Several parameters measured, including snow cover and snow depth. Filtering: Using only stations with coordinates in GHCND data (more accurate). Contains no observations of bare ground! Validation results for the period 1 Aug Dec 1996: High accuracy in winter Lower accuracy in spring and autumn No data in summer Month Accuracy Number of samples January February March April May June July August September October November December Total

28 SnowPEx inter-comparison project 28

29 SnowPEx: Landsat inter-comparison Dozier Klein Salomonson TMSCAG TMSCAGca 30m 1 km 5 km CRCLIM 5km JXM10 5 km JXAM5 5 km GLSSE 1km M10C05 500m PATHF 5 km 29

30 Conclusions and way forward Status: Way forward: Most know misclassification errors mitigated Retrieval algorithm for optical component being improved Development of per-pixel uncertainty estimate ongoing Validation results from Phase 1 using GHCN-D confirmed in Phase 2 with HSDSD, FSUHSS and RIHMI data: Overall accuracy 93% Improvements to the PMR component: Reducing diurnal effects from multiple swaths Atmospheric correction Porting optical algorithm to Sentinel-3 More comprehensive validation Additional met. station dataset outside former Soviet Union SnowPEx results feeding into improvement work cryoclim@cryoclim.net 30

31 Snow Watch contributions from CryoClim Long time series, 1982 present Snow maps covering all land area every day, independent of clouds and daylight Uncertainty estimate under development, according to 1 st Snow Watch meeting recommendation Assuring updates for decades into the future based on operational Sentinel-3 data and PMR (SMMR, SMMI, ) 31

32 Further ideas and suggestions SnowPEx: Very valuable contribution to the snow community. However, intercomparison is not absolute validation Suggests a follow-on to SnowPEx doing absolute validation (for snow cover) WorldView-3 s 16 bands allow use of advanced retrieval algorithms, which make it possible to study absolute accuracy of most current retrieval algorithms QuickBird VHR image from CryoLand validation WorldView-3 spectral coverage 32