Multi-Model Calibrated Probabilistic Seasonal Forecasts of Regional Arctic Sea Ice Coverage 2018 Polar Prediction Workshop Arlan Dirkson, William Merryfield, Bertrand Denis Thanks to Woosung Lee and Adam Monahan Support: CanSISE and FRAMS Département des sciences de la Terre et de l Atmosphére Université du Québec à Montréal May 8, 2018 1 / 13
Sea Ice Probability, SIO 2017 https://www.arcus.org/sipn/sea-ice-outlook 2 / 13
Sea Ice Probability, SIO 2017 https://www.arcus.org/sipn/sea-ice-outlook 2 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). Estimate by fitting appropriate distribution to forecast ensemble (Wilks, 2002). 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). Estimate by fitting appropriate distribution to forecast ensemble (Wilks, 2002). Model errors can be large, and need to be corrected. 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). Estimate by fitting appropriate distribution to forecast ensemble (Wilks, 2002). Model errors can be large, and need to be corrected. Multi-model averaging (cancellation of biases) 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). Estimate by fitting appropriate distribution to forecast ensemble (Wilks, 2002). Model errors can be large, and need to be corrected. Multi-model averaging (cancellation of biases) Calibration (based on past forecasts and observations) 3 / 13
Motivation Sea ice forecasts on seasonal and sub-seasonal timescales are uncertain uncertainty should be quantified. Sea ice probability (SIP) metric for Sea Ice Outlook (Stroeve, 2015; Wrigglesworth et al. 2017). Forecast ensembles are (generally) small and can inadequately described the forecast probability distribution (Richardson, 2001). Estimate by fitting appropriate distribution to forecast ensemble (Wilks, 2002). Model errors can be large, and need to be corrected. Multi-model averaging (cancellation of biases) Calibration (based on past forecasts and observations) Combination of both 3 / 13
Multi-Model Forecast Calibration (in a nutshell) Model 1 Observations Model 2 4 / 13
Multi-Model Forecast Calibration (in a nutshell) Model 1 Y = g(x 1 ) Observations Model 2 Y = h(x 2 ) 4 / 13
Multi-Model Forecast Calibration (in a nutshell) Model 1 Y = g(x 1 ) Observations Model 2 Y = h(x 2 ) 4 / 13
Multi-Model Forecast Calibration (in a nutshell) Model 1 Y = g(x 1 ) g Observations Model 2 Y = h(x 2 ) 4 / 13
Multi-Model Forecast Calibration (in a nutshell) Model 1 Y = g(x 1 ) g Observations Model 2 h Y = h(x 2 ) 4 / 13
Overview Experiment Details 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 Initialization Months: June, July, August 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 Initialization Months: June, July, August Multi-Model Calibration 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 Initialization Months: June, July, August Multi-Model Calibration Performed on the sea ice concentration (SIC) variable per model and per grid point. 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 Initialization Months: June, July, August Multi-Model Calibration Performed on the sea ice concentration (SIC) variable per model and per grid point. Utilizes a parametric probability distribution suitable for SIC, the zero- and one- inflated beta (BEINF) distribution (Ospina and Ferrari, 2010). 5 / 13
Overview Experiment Details Models: CanCM3 and CanCM4 (Canadian Seasonal to Interannual Prediction System; CanSIPS) Hindcasts: September, 2000-2017 Initialization Months: June, July, August Multi-Model Calibration Performed on the sea ice concentration (SIC) variable per model and per grid point. Utilizes a parametric probability distribution suitable for SIC, the zero- and one- inflated beta (BEINF) distribution (Ospina and Ferrari, 2010). Trend-adjusted quantile mapping (TAQM) designed for the BEINF distribution and accounts for trends (Dirkson et al., 2018, Jclim (submitted)). 5 / 13
Multi-Model TAQM: September 2017, June-init Example for grid cell in East-Siberian Sea Sea Ice Concentration Sea Ice Concentration Sea Ice Concentration 1.0 0.8 0.6 0.4 Step 1. Adjust Historical Data for Trend CanCM3 t=2017 0.2 Original Trend-Adjusted 0.0 1980 1985 1990 1995 2000 CanCM4 2005 2010 2015 1.0 Original 0.8 Trend-Adjusted 0.6 0.4 0.2 0.0 1980 1985 1990 1995 2000 2005 Observations 2010 2015 1.0 0.8 0.6 0.4 0.2 Original Trend-Adjusted 0.0 1980 1985 1990 1995 2000 2005 2010 2015 Year t=2017 t=2017 6 / 13
Multi-Model TAQM: September 2017, June-init Example for grid cell in East-Siberian Sea Sea Ice Concentration Sea Ice Concentration Sea Ice Concentration 1.0 0.8 0.6 0.4 Step 2. Fit Historical Data to BEINF Distribution CanCM3 t=2017 0.2 Original Trend-Adjusted 0.0 1980 1985 1990 1995 2000 CanCM4 2005 2010 2015 1.0 Original 0.8 Trend-Adjusted 0.6 0.4 0.2 0.0 1980 1985 1990 1995 2000 2005 Observations 2010 2015 1.0 0.8 0.6 0.4 0.2 Original Trend-Adjusted 0.0 1980 1985 1990 1995 2000 2005 2010 2015 Year t=2017 t=2017 Probability Density Probability Density Probability Density 4 3 2 1 CanCM3 Original Trend-Adjusted 0 0.0 0.2 0.4 0.6 0.8 1.0 CanCM4 4 3 2 1 Original Trend-Adjusted 0 0.0 0.2 0.4 0.6 0.8 1.0 Observations 6 5 4 3 2 1 Original Trend-Adjusted 0 0.0 0.2 0.4 0.6 0.8 1.0 Sea Ice Concentration 7 / 13
Multi-Model TAQM: September 2017, June-init Example for grid cell in East-Siberian Sea Step 3. Calibrate Quantile map fcst values 0 < x t < 1: ˆx t = F 1 o,beta [F m,beta(x t )] Correct mean bias in P(x t = 0) and P(x t = 1) 1.0 CanCM3 Historical: 1981-2016 1.0 CanCM3 Forecast: 2017 Fbeta, Bernoulli Masses 0.8 0.6 0.4 0.2 CanCM3, x Obs, y Fbeta, Bernoulli Masses 0.8 0.6 0.4 0.2 uncal, xt taqm, xt 0.0 0.0 0.2 0.4 0.6 0.8 1.0 CanCM4 Historical: 1981-2016 1.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 CanCM4 Forecast: 2017 1.0 Fbeta, Bernoulli Masses 0.8 0.6 0.4 0.2 CanCM4, x Obs, y Fbeta, Bernoulli Masses 0.8 0.6 0.4 0.2 uncal, xt taqm, xt 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Sea Ice Concentration 0.0 0.0 0.2 0.4 0.6 0.8 1.0 Sea Ice Concentration 8 / 13
Multi-Model TAQM: September 2017, June-init June-init 2017 September SIP P(SIC>0.15) CanCM3 (raw) BS = 0.217 CanCM4 (raw) BS = 0.129 observed ice edge CanCM3+CanCM4 (raw) BS = 0.136 1.0 0.8 0.6 CanCM3 (calibrated) CanCM4 (calibrated) BS = 0.096 BS = 0.091 CanCM3+CanCM4 (calibrated) BS = 0.087 0.4 0.2 0.0 9 / 13
Probabilistic Hindcast Skill: September 2000-2017 red=skill blue=no skill 2000-2017 September Hindcast Skill CRPSS = 1 CRPS fcst /CRPS climo June-init July-init August-init CanCM3+CanCM4 (raw) CanCM3+CanCM4 (raw) CanCM3+CanCM4 (raw) 0.5 0.25 0.0 CanCM3+CanCM4 (calibrated) CanCM3+CanCM4 (calibrated) CanCM3+CanCM4 (calibrated) -0.25-0.5-0.75-1.0 10 / 13
Early Forecast Contribution: Route Access May-init 2018 September Probability of OW Access 1.0 0.8 0.6 0.4 0.2 0.0 Low probability for access via both the NSR and NWP (< 20% for both) 11 / 13
Early-Consensus Forecast Contribution: Regional SIA Low 81% Low 75% Arctic Basin Greenland Sea Extreme 6% Low 5% Extreme 8% High High 13% Extreme Low 8% High Kara Sea Low Extreme High 19% Low 7% 19% 54% Extreme High East Siberian Sea Extreme Low Low 34% 14% High 39% 14% Extreme High Beaufort Sea Extreme Low Low 30% 29% 38% High Baffin Bay/Labrador Sea Ice Free Barents Sea Low Extreme 20% Low 20% 60% High Laptev Sea Low 22% 41% Extreme Low 27% 10% Extreme High High Chuckchi Sea Extreme Low Low 29% 29% 12% 30% High Extreme High Canadian Archipelago Low 35% Extreme Low 30% 32% High Arctic Basin: Low (very confident) Canadian Archipelago: Low (somewhat uncertain) Greenland Sea: Low (very confident) Barents/Kara Sea: High or Extreme High (somewhat confident) Laptev/East-Siberian Sea: High (somewhat uncertain) Beaufort/Chukchi Sea: Equal Prob (very uncertain) Baffin Bay/Labrador Sea: Ice Free (very confident) 12 / 13
Methods are open source! :) https://adirkson.github.io/sic-probability arlan.dirkson@gmail.com Thank you for listening! 13 / 13