Downscaling in Time Andrew W. Robertson, IRI Advanced Training Institute on Climate Variability and Food Security, 12 July 2002
Preliminaries Crop yields are driven by daily weather variations! Current seasonal climate predictions (made with either GCM or empirical models) address only seasonal (3-month) averages Temporal downscaling: deriving weather from climate
Weather and climate We observe current weather Try to predict climate: a probabilistic description of weather How often is it likely to rain, when is the rainy season likely to begin, how long are dry spells likely to be? Weather: a particular daily sequence drawn from the population of weather sequences (climate) Probabilistic description is central because weather is unpredictable more than 2 weeks ahead
Kenya s Long Rains An example of differentiating weather and climate Northward seasonal migration of precipitation during March May Monthly mean satellite-derived precipitation
Daily precipitation variance Daily standard deviation exceeds the monthly mean Sub-monthly r.m.s. satellitederived precipitation
Mean atmospheric circulation Monsoonal evolution from dry northeasterlies to wet southeasterlies Monthly-mean wind vectors at 850 hpa
A probabilistic description of monsoon evolution Hidden Markov Model Rainfall occurrence depends only on the weather state P(R t S t ) R t 1 S t 1 R t R t+1 S t S t+1 daily rainfall occurrence time series: 0 0 1 1 1 0 Markov chain of hidden rainfall states P(S t S t -1 ) 2 3 1 1 2 2
Daily rainfall states in Kenya 3-state HMM trained on 29 seasons of daily rainfall at 7 stations 10 0 a) State 1 (830 days) b) State 2 (1083 days) c) State 3 (755 days) KEY: p=1 p=0.5-10 0 10 20 30 40 50 60 Rainfall occurrence probabilities P(R t S t )
Estimated state sequence S t 1 S t S t+1 2 3 1 1 2 2 March April May 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0 12333333333333333222211112222222333222233221111111 111111111111111111111222222222222222221111 22222222223332113333311111112211122222222221111111 111111222222333322222211122221111111111111 33333333333333333333332222223333333221111111111122 222222211122221111111222222212211112333333 33333312333333333333333333333321111111111111111111 111111111111222222211222222211111111111111 22222222222222111111122233322222222222222222222233 322111111113333332221122113333322211111111 33333333333333333222211211122222222222222211111111 112222222221111122222222222221111111111111 33322333333333333222222212333333211222221111222222 222222222222332222211111111122211111111111 33333333222333332233333333333333333222222222211122 222211111122211222222111112111112233222332 33222222233333322112233331111111111122222211111111 122222222222222222211111222222222222111222 33333333333333333322222112222222222222222222222222 222222211122222222222222222222111111111111 33333333333322233333333333333333333333222222222211 111122211111111111111111111111111111111111 33333333333333333333333333333331112211111111111111 112222222222222222222221112211111211111111 33333333333333333322233333322222221112211111122222 222222222222222222222222222222211112222333 33333333311111112222222222222222222222222222222221 111111111112222222222222222111111111111111 33333333333333333211111111111111112222222222222223 333222222222222222222233333322111111111111 33333333333333333333332223333333333333222221121112 112222221111111111111111122111223333111111 33323333333333332221111111111221111111112222222222 222222333332211111111222222221111111122333 33333333333333333333333333333333333333333322112222 222222111111111111111111111111121111122333 33333333333333333312333333333333333333333221111111 111111111223222222211122212222222222211111 33333333333333333333331111111113333322222222222222 222222222223222111222112222222221111133333 33333322223332211112222222222222222222222222221111 222222221111122222222222111111111122222222 33333333333333333323222221111123333331111111111111 111111111112222222222222221111111111111122 22233333333333333333333333322222222233322222222221 111122211222112222222222222333222332222211 12211111111122222222222233221111111123332222222222 222222222332222111111111111111133331111222 33333333333333333333332223322111111111111222221111 222222222222221122211221111122233333321111 22233333333333333311111222333331111222222112222222 222223333222222233322222211111111111111111 11133333333322222333333333111111111111111122222222 222222222222222222222222222222223322222333 33333333333333322222222333333333332222222222222111 112211122222222222222222222221113333333333 33333333333333333322222333333333111222222211111221 111111111222222222222222333333333321111111 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 Day in Season - dry state (#3, yellow) tends to occur in March - wet states (#1, green), (#2, blue) tend to occur in April May To get rainfall sequence: P(R t S t )
Atmospheric circulation state-composites Vectors: low-level winds Contours: mid-tropospheric descent
Pentad analysis of wet/dry spells Okoola (1999) - 700hPa wind vectors during March May Wet Pentads Wet minus Dry Year Dry Pentads
Summarizing over Kenya The climatological-mean smooth seasonal evolution is not realized, but rather Erratic switches between flow regimes characterizes monsoon evolution Temporal downscaling is concerned with predicting interannual variations in daily character of the rains and their northward progression
Temporal downscaling of Seasonal climate predictions Weather can t be predicted more than two weeks in advance (sensitivity to initial conditions), so we can t hope to be accurate on any particular day For this reason, the date of monsoon onset is highly unpredictable Aim to forecast changes in the probability distribution of daily weather sequences and to quantify their uncertainty Signal-to-noise ratio: Signal: Predictable change (shift?) in the distribution of weather Noise: unpredictable spread in the distribution of weather
Some statistics we need to get right 1. Precipitation occurrence Probability of rain Wet/dry spell length Spatial correlations between stations Log-odds ratio (odds of rain at one station vs. rain at another) 2. Precipitation amount Daily histogram
Daily Precipitation Occurrence Probabilities Seven Kenyan Daily Precip stations Occurrence during Probability March May March-May 0.5 Simulated by HMM 0.4 0.3 0.2 0.1 Lodwar 0 0 0.1 0.2 0.3 0.4 0.5 Observed
Wet/Dry Spell Durations 1 Wet/Dry Spell Durations at Lodwar March-May Probability{spell >=T} 0.1 0.01 WET DRY Solid - Observed Dashed - HMM 0.001 0 10 20 30 40 50 60 Spell Duration T (days)
Approaches to temporal downscaling 1. Historical analog techniques Use various subsets of past years based on a seasonalmean predictor(s) K-nearest neighbors 2. Stochastic weather generators Parameters estimated from seasonal (or monthly) GCM predictions 3. Statistical transformation of daily GCM output Usually includes a stochastic element Regression based Weather state models, e.g. HMM
Historical Analogs Simplest & most widely used approach Take daily sequences of weather observed during past events as possible scenarios for a predicted event An event can be defined according to the threshold of an index, such as Niño-3 SST, or a GCM-predicted seasonal-mean quantity (e.g. regional precip.)
Advantage: multivariate structure (both spatial and between-variables) is preserved Disadvantage: may not be many events in the historical record and every event is different (sampling problems)
K-Nearest Neighbors Refinement of the analog approach, retaining its advantages and partially solves the sampling problem Past years daily sequences D t are again selected from the historical record according to the value of some (seasonal-mean) predictor x * but here the past year t is resampled according to the distance x t - x *
So we select the k nearest neighbors of x * in the historical record, estimate appropriate weights to assign to each, and resample D t accordingly The resulting superensemble of years (each is repeated many times) can then be fed to a crop model
Approaches to temporal downscaling 1. Historical analog techniques Use various subsets of past years based on a seasonal-mean predictor(s) K-nearest neighbors 2. Stochastic weather generators Parameters estimated from seasonal (or monthly) GCM predictions 3. Statistical transformation of daily GCM output Usually includes a stochastic element Regression based Weather state models, e.g. HMM
Weather generators Use concept of Monte Carlo stochastic simulation Let computer generate a large number of daily sequences using a stochastic model Honor the statistical properties of the historical data of the same weather variables at the site Precipitation frequency and amount; dry-spell length; monsoon onset and end; monsoon break probabilities Daily max and min temperatures, solar radiation Cast seasonal prediction in terms of changes in these statistical properties
History Probabilistic modeling of precipitation predates numerical weather prediction, but seminal work in 1960s Markov chain model introduced by Gabriel and Neumann (1962) By 1980s, models extended to be able to treat a suite of variables and could be used in agric. apps. ( WGEN Richardson 1981, ) Large literature: many extensions to the basic model
Example of a Weather Generator PREC variable model parameters occurrence Markov chain (order=1) p11, p01 [monthly] (precipitation) amount Gamma distribution _, _ [monthly] SRAD (solar radiation) TMAX (max. temperature) TMIN (min. temperature) AR model: x*(t+1) = Ax*(t) + Be A,B: 3x3 matrices + 3 (wet/dry) (avg s/std's) x m( x) x* = s( x) where: m(x), s(x) depend on day of the year and precipitation occurrence Dubovsky et al. (2001) Need to consider the statistical dependence of the weather variables with each other on the same day, as well as their persistence Markov process used to model persistence Solar radiation and Tmax are likely to be lower on wet days, so precipitation is usually chosen as the driving variable with others conditioned on it Daily-total precipitation is often exactly zero, so most weather generators treat occurrence and amount separately
Precipitation occurrence fully defined by the two conditional probabilities ( transition probabilities ) p 01 = Pr{wet day t dry day t-1} (dry --> wet) p 11 = Pr{wet day t wet day t-1} (wet --> wet) p 10 = 1-p 11 p 00 = 1-p 01 (wet --> dry) (dry --> dry) These lag-1 autocorrelations are estimated from historical data
This simple model is able to capture persistence, and many statistics of interest can be derived from its transition probabilities: p Frequency of wet days: π = 01 1+ p 01 p 11 p 01 < π < p 11, so simulations yield sequences of wet and dry days that are more persistent than independent draws according to the climatological probability π. Wet- and dry-spell lengths follow a geometric distribution Pr{spell = T} = p(1 p) T 1, T =1,2,3,... The mean and variance of the no. of wet days contained within a string of T consecutive days can also be computed
Precipitation Amount Distribution of daily amount is strongly skewed to the right Exponential is simplest reasonable model. It requires only one parameter µ, yet reproduces the strong positive skewness Two-parameter gamma distribution is most popular choice (shape α and scale β) α=1 yields exp distrib, while the extra flexibility improves the fit
Simulation Draw a string of random numbers u [0,1]. If day t-1 was dry, then day t is simulated to be wet if u < p 01.. Wilks & Wilby (1999)
Multi-site extension Run a series of WG s in parallel Use spatially correlated random numbers Wilks (1998)
WGs for downscaling seasonal forecasts Use climatological parameters Then rescale the generated daily values such that their monthly means exactly match the monthly GCM prediction (Hansen) Additive offsets for temperatures & insolation parameters Multiplicative adjustment for precip., with repeated (stochastic) generation to match target
Realizations of daily weather in forecast seasonal climate IRI forecasts are in terms of tercile probabilities {p B, p N, p A } Want to make WG parameters depend on these probabilities
Forecast parameters can be estimated from the historical record (Briggs & Wilks 1996, Wilks 2002) by: 1. computing their values averaged over the years in each tercile of the local precip record {π (B ), π ( N), π (A ) } 2. and weighting according to the forecast {p B, p N, p A } π (p B, p N, p A, ) = p B π (B) + p N π (N ) +p A π (A ) Similar approach can be taken to condition WG parameters on ENSO phase (Woolhiser et al. 1993)
Approaches to temporal downscaling 1. Historical analog techniques Use various subsets of past years based on a seasonal-mean predictor(s) K-nearest neighbors 2. Stochastic weather generators Parameters estimated from seasonal (or monthly) GCM predictions 3. Statistical transformation of daily GCM output Usually includes a stochastic element Regression based Weather state models, e.g. HMM
Statistical transformation of daily GCM output (1) Regression based Make an empirical model by regressing daily station variables (T max, T min, precip.) against daily grid-scale atmospheric observations Use resulting empirical relationships in conjunction with GCM predictions perfect-prog. approach which needs to assume that the GCM perfectly simulates the grid-scale
E.g. from Wilby et al. (1999), wet day probability for a given day i is downscaled using 3 grid-box predictor variables: surface specific humidity (SH), SLP, 500hPa geopotential height (H), and lag-1 autocorrelation O i = α 0 + α Oi 1 + α SH SH i + α slp slp i + α H H i The α s are estimated using linear least squares regression For a given site and day, a wet day is returned if uniformly-distributed random number o i Similar approach used for precip amount (exp), Tmin, Tmax
(2) Weather state models Assume GCM can simulate the large-scale weather state ( perfect prog ) Weather state models attempt to relate local weather to large-scale atmospheric controls Advantage of increased physicality E.g., the non-homogeneous HMM
The HMM revisited A non-homogeneous Hidden Markov Model can link weather state-occurrence to atmospheric controls Rainfall occurrence is conditionally dependent on the weather state Transition probabilities modulated by X S t 1 R t 1 X t 1 R t R t+1 S t S t+1 X t X t+1 daily rainfall occurrence time series: 0 0 1 1 1 0 Markov chain of hidden rainfall states 2 3 1 1 2 2 Daily time series of atmospheric predictor
Downscaling with a NHMM A Markov chain of discrete atmospheric states (rather than precip occurrence as in the WG) Extra state layer is identified with physical atmospheric states Dynamical controls influence these states Then we simply take the predicted daily sequences of X from an ensemble of GCM predictions
HMM Downscaling over SW Australia Charles, Bates & Hughes (2000) NHMM fitted to 15 yrs (1978-92) May Oct. daily gauge precip & Reanalysis X X: area-average SLP, north-south SLP gradient, 850hPa dew-point temperature depression 10 yrs of GCM (~500km resolution) & limited area model (LAM) nested within it (125km res) Simulated X from GCM & LAM used to drive the NHMM to downscale precip occurrence at 30 stations
Six NHMM weather states Charles, Bates & Hughes (2000) Daily ppt occurrence probabilities & SLP averaged over all classified days in each state
Downscaled vs. Obs precip probabilities & log-odds ratio Charles, Bates & Hughes (2000)
Downscaled rainfall amounts vs Observations Charles, Bates & Hughes (2000)
Rank correlations between stations: Downscaled vs Obs Charles, Bates & Hughes (2000)
Issues for future work While many methodologies are well developed, the application to seasonal prediction is very recent These methods have never been systematically compared in the context of seasonal climate prediction! Many rely on empirical cross-scale relationships (perfect prog.), yet in the seasonal prediction context MOS-type corrections should be possible Need to test in conjunction with crop models