Evaluating global reanalysis datasets for provision of boundary conditions in regional climate modelling

Transcription

1 Clim Dyn DOI /s x Evaluating global reanalysis datasets for provision of boundary conditions in regional climate modelling Ditiro B. Moalafhi 1 Jason P. Evans 2 Ashish Sharma 1 Received: 24 August 2015 / Accepted: 15 January 2016 Springer-Verlag Berlin Heidelberg 2016 Abstract Regional climate modelling studies often begin by downscaling a reanalysis dataset in order to simulate the observed climate, allowing the investigation of regional climate processes and quantification of the errors associated with the regional model. To date choice of reanalysis to perform such downscaling has been made based either on convenience or on performance of the reanalyses within the regional domain for relevant variables such as near-surface air temperature and precipitation. However, the only information passed from the reanalysis to the regional model are the atmospheric temperature, moisture and winds at the location of the boundaries of the regional domain. Here we present a methodology to evaluate reanalyses derived lateral boundary conditions for an example domain over southern Africa using satellite data. This study focusses on atmospheric temperature and moisture which are easily available. Five commonly used global reanalyses (NCEP1, NCEP2, ERA-I, 20CRv2, and MERRA) are evaluated against the Atmospheric Infrared Sounder satellite temperature and relative humidity over boundaries of two domains centred on southern Africa for the years inclusive. The study reveals that MERRA is the most suitable for climate mean with NCEP1 the next most suitable. For Electronic supplementary material The online version of this article (doi: /s x) contains supplementary material, which is available to authorized users. * Ashish Sharma a.sharma@unsw.edu.au 1 2 School of Civil and Environmental Engineering, The University of New South Wales, Sydney, NSW 2052, Australia Climate Change Research Centre, The University of New South Wales, Sydney, NSW 2052, Australia climate variability, ERA-I is the best followed by MERRA. Overall, MERRA is preferred for generating lateral boundary conditions for this domain, followed by ERA-I. While a better LBC specification is not the sole precursor to an improved downscaling outcome, any reduction in uncertainty associated with the specification of LBCs is a step in the right direction. Keywords Reanalyses AIRS Lateral boundary conditions RCM Southern Africa 1 Introduction With the inherently complex interplay of atmospheric processes, the need for studying the possible atmospheric mechanisms underlying both regional and global climate has necessitated a need for consistent data sets pertaining to atmospheric circulation. The most convenient data sets in this regard are in the form of global reanalyses. In most cases, the products of reanalyses are freely available and have gained wide usage (Bromwich and Fogt 2004). Reanalysis is described as a climate or weather model simulation of the past that includes data assimilation of historical observations (Bengtsson et al. 2007). The use of reanalysis products has become widespread in the field of climate research due, in part, to the lack of globally and temporally complete direct observations (Qian et al. 2006; Zhang et al. 2013). For this reason reanalysis products have been used to drive land surface models, study the climate system and provide lateral boundary forcing for regional climate models (Decker et al. 2012). We present here a 4-D framework for assessing the relative accuracy of alternate reanalysis products in reference to their use for specifying

2 D. B. Moalafhi et al. atmospheric lateral boundary conditions (LBCs) over a domain prior to use in regional climate modelling. Although reanalyses use some common station data, the products differ in various ways. They use different vertical and horizontal resolutions, data assimilation methods, physical parameterizations and sea surface temperature prescriptions for boundary conditions. While reanalyses continue to contribute significantly to a more detailed and comprehensive understanding of the dynamics of the earth s atmosphere, the various reanalysis products show deficiencies and inconsistencies especially at regional scales. Bromwich et al. (2007) compared several reanalysis datasets including National Centre for Environmental Prediction/National Centre for Atmospheric Research Reanalysis (NCEP1), NCEP2, European Centre for Medium- Range Forecasts Reanalysis (ERA-40), ERA-15 and Japanese 25-year Reanalysis (JRA-25) over high latitude regions and concluded that unreliability of reanalyses during non-summer months over Antarctica was due to limited quantities of satellite data for assimilation prior to the modern satellite era. From this, it was inferred that performance would be worse for the data scarce southern hemisphere. Over the Amazon basin, European Centre for Medium- Range Forecasts Reanalysis-Interim (ERA-I) is cold-biased compared to in situ station data (Betts et al. 2009). The cold bias of ERA-I was also found by Brands et al. (2013) in evaluating reanalyses uncertainty over tropical African regions. In yet another study mostly over tropical Africa and also covering large portions of the Atlantic ocean and the western Indian ocean by Druyan and Fulakeza (2013), it was revealed that though ERA-I has an edge in forcing a regional atmospheric model (RM3) for precipitation simulation, NCEP2 lateral boundary conditions produced better simulations of some features. As more reanalyses became available to the climate research community, evaluation of reanalysis products has increased over time (Bosilovich et al. 2008) but it is still worth noting that these evaluations have different scientific focus, precluding a common conclusion about which reanalyses is superior (Zhang et al. 2013). It should also be noted that attempts have been made to modify lateral boundaries for biased GCM simulations (Rocheta et al. 2014) resulting in dynamical inconsistencies based on the bias correction adopted. Approaches that attempt to correct bias across the entire lateral boundary variable field (instead of one at a time) are considered more appropriate for this purpose (Mehrotra and Sharma 2015). One of the very few studies in which reanalyses were evaluated at high temporal resolution (i.e., daily), focusing on surface variables, was by Pitman and Perkins (2009). In that study, evaluations were made on ERA-40, NCEP2 and JRA-25 over five regions including Africa. Bosilovich et al. (2008) also evaluated surface products of ERA-40, JRA-25, NCEP1, and NCEP2 at global and regional scales and concluded that there are inconsistencies in quality of reanalysis products and improvements to a reanalysis dataset may be limited to the availability (in both space and time) and quality of the assimilated data. Decker et al. (2012) evaluated the performance of Climate Forecast System Reanalysis (CFSR), ERA-40, ERA-I, and Modern- Era Retrospective Analysis for Research and Applications (MERRA) products against flux tower observations of temperature, wind speed, precipitation, downward shortwave radiation, net surface radiation, and latent and sensible heat fluxes at 33 different locations in the Northern Hemisphere. The evaluations, which were made at 6-hourly and monthly time-scales, gave mixed results and thus no reanalyses was identified as the best across all the variables. Over southern Africa, Zhang et al. (2013) evaluated ERA-40, ERAinterim, JRA-25, MERRA, CFSR, NCEP-R1, NCEP-R2 and 20CRv2 based on seasonal cycle in precipitation and concluded that 20CRv2 and CFSR performed better than the rest. It is worth noting that even when reanalyses are evaluated for provision of LBCs for regional climate modelling, such evaluations have only been done at the surface. When running regional climate models (RCMs), lateral boundary conditions (LBCs) that extend vertically through the atmosphere need to be created. Other than the initial conditions which are discarded along with several months of spin-up time for climate simulations, these LBCs are the dominant input to RCM simulations and as such they are assumed to be accurate and reliable though to date this has rarely been tested. If one wants to downscale the climate of the recent past with a RCM then LBCs are created from an available reanalysis. The choice of which reanalyses to use in this situation may be informed by studies such as those above that evaluated reanalyses performance based on surface variables. However, variables such as near surface air temperature and precipitation within the regional domain are dependent on both parameterizations in the reanalysis system (e.g. for the land surface or cumulus convection) that differ from the regional model, and on data that is assimilated within the regional domain. Thus the information in these fields only partially reflects information present in the LBCs for RCMs and the results of such evaluations may be only partially relevant. For the first time, this study evaluates 4-D variables directly relevant to RCM LBCs, atmospheric temperature and humidity, at the boundaries of an example domain to quantify errors associated with these fields and the potential for using such an evaluation to determine which reanalysis can provide the most robust LBCs for long term climate simulations over Southern Africa. Ideally such an evaluation would address all atmospheric LBCs (temperature, moisture and winds) however a spatially continuous wind dataset does not currently exist limiting this evaluation to temperature and

3 Evaluating global reanalysis datasets for provision of boundary conditions in regional moisture. The reanalyses wind fields are compared to each other at the lateral boundary locations to assess how much variability between reanalyses exists. While the methodology is general, the application is specific to the location of the lateral boundaries and the evaluation results may differ for other regional domains. Here the technique is evaluated for its robustness to small domain variations by considering an additional domain with lateral boundaries offset from the original domain. The technique described herein can be applied to any region, and can be evaluated directly at the location of the intended RCM boundaries. This should assist with the choice of reanalysis to use to provide boundary conditions to the RCM so that the most accurate LBCs possible are used for long climate simulations. While good modelling outcomes may occur for a number of reasons, using the best set of inputs (in our case LBCs) is a necessary first step in any model application exercise. The rest of the paper is as follows. Datasets which include Atmospheric Infrared Sounder (AIRS) satellite data and global reanalyses data are described in Sect. 2. Descriptions of statistical measures employed in evaluations of reanalyses against AIRS satellite data are presented as methodology in Sect. 3 followed by Sect. 4 where results are presented and discussed. Conclusions follow in Sect Datasets 2.1 Observations Atmospheric Infrared Sounder (AIRS) satellite data version 6 was chosen as it provides 4-D global data for both temperature (24 levels) and relative humidity (12 levels). AIRS was launched on EOS Aqua on 4 May 2002, together with AMSU A and HSB, to form a next generation polar orbiting infrared and microwave atmospheric sounding system (Susskind et al. 2006). The instrument suite is designed to support climate research and improve weather forecasting. The data, which is at 1 1 grid resolution, is available from National Aeronautics and Space Administration (NASA) at Aqua_AIRS_Level3/. The data used are level 3 standard products of both descending and ascending overhead passes at 1:30 and 13:30 h local time respectively referenced by 0 longitude every day. AIRS is the first of a new generation of advanced satellite-based atmospheric sounders that produces daily, high accuracy, high vertical and horizontal resolution profiles of temperature, water vapour and minor gases, along with cloud and surface properties over most of the Earth s surface (Aumann et al. 2003; Tobin et al. 2006). The highaccuracy retrieval goals of AIRS of 1 K Root Mean Squared errors in 1 km layers below 100 hpa for air temperature, 10 % Root Mean Squared errors in 2 km layers below 100 hpa for water vapour concentration, combined with the large temporal and spatial variability of the atmosphere and difficulties in making accurate measurements of the atmospheric state, necessitated continuous careful and detailed validation using well-characterized ground-based sites (Tobin et al. 2006). In this regard the continuing efforts to validate and give quality assurance to AIRS products, has always been positive (Fetzer 2006; Divakarla et al. 2006). One of the insurance factors in use of AIRS products, for example, is that temperature profile accuracy in the troposphere matches that achieved by radiosondes. Temperature and water vapour retrievals from AIRS are in good agreement with global radiosonde measurements over both land and ocean (Divakarla et al. 2006). With the latest version (version 6), significant improvement in the ability to obtain both accurate temperature profiles and surface skin temperatures under partial cloud cover conditions has been achieved (Susskind et al. 2014). 2.2 Reanalysis products The reanalysis products which were used in the study are the European Centre for Medium-Range Forecasts Interim Reanalysis (ERA-I), National Centre for Environmental Prediction/National Centre for Atmospheric Research Reanalysis (NCEP1), National Centre for Environmental Prediction-Department of Energy reanalysis (NCEP2), Modern Era Retrospective-analysis for Research and Applications (MERRA), and Twentieth Century Reanalysis Version 2 (20CRv2). The Climate Forecast System Reanalysis (CFSR) could not be considered as it does not cover the entire period of interest up to the end of NCEP1, NCEP2 and 20CRv2 data were available from Earth System Research Laboratory (ESRL) at noaa.gov/psd/data/while ERA-I data were sourced from European Centre for Medium-Range Weather Forecasts (ECMWF) at MERRA data were available from National Aeronautics and Space Administration (NASA) at daac-bin/dataholdings.pl. Attributes of these reanalysis are summarized in Table 1. While all reanalyses assimilate some of the same data there are differences between them. NCEP1 and NCEP2 use the lowest resolution, followed by 20CRv2 and MERRA with ERA-I using the highest resolution. 20CRv2 does not assimilate any observations in the upper atmosphere, and hence is less constrained there than the other reanalyses. ERA-I and MERRA are forced by the same SST dataset (OISST Reynolds et al. 2007) and hence are likely to be more similar near the surface over the ocean. Ultimately, differences between the reanalyses are caused

4 D. B. Moalafhi et al. Table 1 Summary details of the global reanalyses that were used in the study Name Organization Temporal coverage Horizontal resolution Vertical levels SST & sea-ice forcing Assimilation ERA-I ECMWF 1979 present T255 (80 km) 60 Daily OISST (from 4DVAR 2002) NCEP-R1 NCEP/NCAR 1948 present T62 (210 km) 28 GISST 3DVAR NCEP-R2 NCEP/DOE 1979 present T62 (210 km) 28 SST AMIP-2 3DVAR MERRA NASA GMAO 1979 present 1.5 by 1.5 (167 km) 72 Weekly OISST 3DVAR 20CRv2 NOAA/ESRL PSD 1871 present T62 (210 km) 28 HadlSST1.1 Ensemble Kalman Filter Source: Zhang et al. (2013) latitude 30 towards the south and some areas northwards around longitude and latitude Processing of AIRS observational data Fig. 1 Elevation in metres above mean sea level (mamsl) showing the range over both the main and secondary domain by many interacting processes within these extremely large and complicated model/data systems. 3 Methodology Evaluations were made at the boundaries of the main domain (North, South, East and West) and stretching inwards by 4, as shown in Fig. 1. ERA-I was available at , NCEP1 and NCEP2 at , 20CRv2 at , and MERRA at grid resolutions. Picking the boundary panels at 4 was mostly deemed accommodative of the grid resolutions of all the reanalysis products and also gives some flexibility in variations of the boundaries of the domain. The resultant boundary panels of the domain were thus demarcated as North (8.5 to 50.5 E, 0.5 to 2.5 S), South (8.5 to 50.5 E, 37.5 to 40.5 S), East (50.5 to 53.5 E, 0.5 to 40.5 S), and West (5.5 to 8.5 E, 0.5 to 40.5 S). Elevation over the domain varies between 0 and 3500 m. The highest areas are mostly the mountainous area of Lesotho at around longitude 29 and Level 3 AIRS satellite data version 6 ( ) were available globally at 1 1 grid resolution. Both the descending (1:30 am) and ascending (1:30 pm) overpass times are combined to produce a time series of data every 12 h. These were then sub-setted to the four (4) boundary panels but maintaining the original 1 1 grid resolution and taking pressure levels of 50 hpa to 1000 hpa inclusive (50, 70, 100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000 hpa) for temperature and 100 hpa to 1000 hpa inclusive (100, 150, 200, 250, 300, 400, 500, 600, 700, 850, 925, 1000 hpa) for relative humidity. For derivation of summary statistics for relative humidity, data is used for pressure levels of 300 hpa to 1000 hpa since NCEP1 s relative humidity covers only levels of 300 hpa to 1000 hpa. 3.2 Processing of reanalyses data Reanalyses temperature and relative humidity were available at 6 hourly intervals and 3 hourly intervals for MERRA. The products were sub-setted to the four different boundary panels and re-gridded using linear interpolation to the 1 1 grid resolution of AIRS data and the chosen set of pressure levels. 3.3 Evaluation of reanalyses against AIRS observations Since the AIRS data are available every 12 h, reanalyses data were linearly shifted to the over-head pass times of the AIRS data. The 1-h lag from coordinated universal time (UTC) for every 15 longitude across the domain was also taken into consideration. The time shifted reanalysis were then re-sampled to the same time resolutions of AIRS (i.e., 12 hourly). Mean bias, root mean squared error (RMSE), mean absolute error (MAE) and temporal correlation were derived over the individual boundary panels to compare the time series of reanalyses variables with corresponding time

5 Evaluating global reanalysis datasets for provision of boundary conditions in regional series from AIRS at each of the four lateral boundaries. Mean Bias here is the simple difference of the means given by M j Ō j where M j is reanalysis averaged temperature or relative humidity over an jth boundary panel and Ō j is averaged AIRS (observations) temperature or relative humidity over the jth panel. Along with the bias, the RMSE is calculated for each panel to determine which of the reanalyses is closest to the AIRS data. ( n ) RMSE = 1 (M i O i ) 2 (2) n i=1 where n is the number of time steps (i.e., 7306), O i and M i are the AIRS observations and reanalysis product values respectively for i = 1,,n. Here i is an ith time step. Average magnitude of the errors without considering their direction was computed through MAE. MAE = 1 n n M i O i i=1 where M is the reanalysis product, O is the AIRS observation, i is the ith time step and n is the number of time steps. The temporal correlation coefficient is used to evaluate the goodness-of-fit by performing linear regression between reanalyses and AIRS observations. ni=1 o i m i r = ni=1 ni=1 (4) o 2 i mi 2 where n is the number of time steps (7306), o i = Ō i Ō i, m i = M i M, O i and M i are the ith time step AIRS observation and ith time step reanalysis product for i = 1,,n, and Ō and M are mean values of AIRS observations and reanalysis products respectively. The reanalyses performances are also ranked to choose the most suitable reanalysis for downscaling over the domain. For the smallest error statistic under a panel when considering a field, a reanalysis gets a score of 1. Equal error statistics are given equal rank among the reanalyses. For correlation, the best performance (i.e., rank 1) is for the highest temporal correlation coefficient. These scores based on all error statistics are added for all the panels for both temperature and relative humidity individually and collectively. The reanalysis that has the smallest overall score is considered the overall best. There are different interests in downscaling reanalyses which usually influence the error statistics used to choose the most suitable reanalysis. The different interests (1) (3) can broadly be grouped into climate mean and climate variability (temporal). To give an idea on which reanalysis could be most suitable under the broad categories, reanalyses ranking are further made for climate mean (mean bias) and climate variability (temporal correlation) individually. Although we focused on 12-hourly time steps of the AIRS fields, an attempt was also made to give an idea on whether the comparisons based on the error statistics are valid at weekly, monthly and annual time scales. 4 Results and discussion Based on the four boundary panels, mean Bias, RMSE, MAE and temporal correlation were determined for the different reanalyses based on comparisons with AIRS observations and averaged over all pressure levels. The evaluations are made over the whole study period ( inclusive), the driest year (2005) and the wettest year (2012) at 12-hourly time intervals based on the timescale of AIRS satellite data. Results for all reanalyses are displayed in tables. Vertical profiles and cross-sections are also examined. Spatial patterns of the temperature and relative humidity fields are not directly assessed in this study. This assessment is used to arrive at an overall performance associated with each of the datasets, across all the variables analysed. Results from these assessments are presented in the following sub-sections, as well as the supplementary material associated with the article. 4.1 Mean bias Temperature Based on Table 2, all reanalyses overestimate temperature over all the boundary panels with the lowest bias over the northern boundary panel (except for 20CRv2 and MERRA). Among the reanalyses, ERA-I has the lowest bias over all the boundary panels. The reanalyses performances are ranked (Table 5) and ERA-I has the best ranking for temperature mean bias over all panels. Over the vertical levels of the atmosphere (Fig. 2), reanalyses have some variations in mean bias. Generally reanalyses under-estimate temperature at 700 hpa over all the boundary panels. Other under-estimations are near the top of the atmosphere (i.e., at 50 hpa). From the 700 hpa pressure level across all the boundary panels, temperature is over-estimated towards the lower atmosphere (i.e., 1000 hpa). All reanalyses are within 1.5 K of each other through most of the atmosphere except at 100 hpa, 70 hpa and 50 hpa where 20CRv2 s over-estimation of temperature is clear deviation from the rest of reanalysis.

6 D. B. Moalafhi et al. Table 2 Error statistics for the individual boundary panels NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Temperature (K) S N W E Relative humidity (%) S N W E Bold values = best Based on Fig. 3 (see also Supplemental figure S1), temperature is under-estimated at 700 hpa across all the boundary panels over latitude. This under-estimation of temperature at 700 hpa was also revealed by Fig. 2. ERA-I also underestimates temperature just above the tropopause while for the rest of reanalyses, temperature is overestimated at that level. Temperature over-estimations for NCEP1, NCEP2, ERA-I and MERRA at the lower levels of the atmosphere are more pronounced at high latitudes Relative humidity It is evident that all reanalyses overestimate relative humidity over all the boundary panels except NCEP2 for the northern boundary panel (Table 2). Generally ERA-I has the largest positive bias over the boundary panels. Among the reanalyses, NCEP1 has the smallest bias for the northern and western boundary panels. For the southern boundary panel, MERRA has the lowest bias while NCEP2 has the lowest bias for the eastern boundary panel. Ranking the reanalyses reveals that NCEP1 is the best in relative humidity mean bias over the panels (Table 5). Plots of relative humidity over levels of the atmosphere (Fig. 2) reveal that all reanalyses are generally within 20 % of each other throughout the atmosphere with bias ranging between 40 % (NCEP2 at 100 hpa over the northern panel) and +30 % (ERA-I at 150 hpa over the northern panel). NCEP1 and MERRA under-estimate relative humidity at 850 hpa over the northern (also 20CRv2) and southern boundary panels. Reanalyses have largest overestimations of relative humidity at the tropopause level (i.e., at hpa) over all the boundary panels. Based on Fig. 4, ERA-I, MERRA and 20CRv2 have the largest tropopause error near the equator while NCEP2 has it at high latitudes. 20CRv2 has the largest error at both the equator and high latitudes. Apart from these, ERA-I and MERRA have biases fairly evenly spread over latitude. 4.2 Root mean squared error (RMSE) Temperature As from Table 2, all reanalyses generally struggle at higher latitudes (i.e., at the southern boundary panel) with best performances over the northern boundary panel. MERRA has the lowest RMSE s over the western and eastern boundary panels while NCEP1 and NCEP2 do best for the southern and northern boundary panels respectively. Ranking of the reanalyses based on their performances reveals that NCEP1 is generally the best over all four boundary panels for temperature RMSE (Table 5). Based on RMSE plots over levels of the atmosphere (Fig. 5) the following are revealed.

7 Evaluating global reanalysis datasets for provision of boundary conditions in regional Fig. 2 Mean Bias for Temperature (K) and Relative humidity (%) over the levels of the atmosphere for the boundary panels All reanalyses are within 0.5 K of each other through most of the atmosphere, though MERRA is consistently worse across the mid troposphere over the southern boundary panel. MERRA also performs badly in the lower troposphere over the northern boundary panel. 20CRv2 is consistently worse near the top of the atmosphere (i.e., between pressure levels of 100 hpa and 50 hpa). For the southern boundary panel, reanalyses struggle at the tropopause level (i.e., 150 hpa to 200 hpa). RMSEs vertical contour plots at the respective panels (Fig. 6 and Supplemental figure S3) reveal the following. Reanalyses perform worse at high latitudes. MERRA performs badly at high latitudes (i.e., over the southern boundary) and also struggles in the lower troposphere near the equator (i.e., over the northern boundary panel). Reanalyses have over-estimations just above the tropopause level. These errors are more pronounced for 20CRv2 near the equator. At higher latitudes (over the southern boundary panel), reanalyses perform badly near the tropopause (near hpa) Relative humidity Based on Table 2, ERA-I is the best over the eastern and western boundary panels while NCEP1 does best over the northern and eastern panels. The ranks of reanalyses performances indicate that ERA-I and NCEP1 are joint best performers compared to the rest (Table 5). Examining relative humidity RMSE plots over the levels of the atmosphere (Fig. 5), the following are key findings.

8 D. B. Moalafhi et al. Fig. 3 Vertical contour plots of Temperature mean bias (K) for the eastern and western boundary panels Fig. 4 Vertical contour plots of relative humidity mean bias (%) for the eastern and western boundary panels

9 Evaluating global reanalysis datasets for provision of boundary conditions in regional ERA-I and MERRA consistently do better through mid-troposphere for the western, eastern and southern boundary panels. All reanalyses perform similar and are closer to each other for the southern and northern boundary panels than for the eastern and western boundary panels through mid-troposphere. Generally, reanalyses perform better near the surface (i.e., at 1000 hpa) over the eastern, western and northern boundary panels while for the southern boundary panel, best performances are also near the top of the atmosphere (i.e., at 100 hpa). Based on Fig. 7 (also see Supplemental figure S4) on relative humidity RMSE vertical contour plots over latitude (and longitude), the following are the key findings that support findings arrived at from Fig. 5. ERA-I and MERRA consistently do better through mid-troposphere for the western, eastern and southern boundary panels which agrees with Fig. 5. ERA-I and MERRA also do better through mid-troposphere for the northern panel at high longitudes. ERA-I and MERRA have the largest over-estimations of relative humidity near the tropopause. Best performances are generally at the lower levels of the troposphere which reinforces findings through Fig Temporal correlation Temperature Reanalyses are better correlated over time with AIRS at high latitudes (Table 2). NCEP1 is the best when considering all boundary panels (Table 5). Fig. 5 Root mean squared error (RMSE) for temperature (K) and relative humidity (%) over the atmosphere for the boundary panels

10 D. B. Moalafhi et al. Fig. 6 Vertical contour plots of temperature (K), root mean squared error (RMSE) for the eastern and western boundary panels Fig. 7 Vertical contour plots of relative humidity (%), root mean squared error (RMSE) for the eastern and western boundary panels

11 Evaluating global reanalysis datasets for provision of boundary conditions in regional Based on Fig. 8 of temporal correlations over the troposphere, 20CRv2 is consistently the least correlated across all the boundary panels. It is noticed that 20CRv2 has the lowest time correlations particularly at 50 hpa across all the boundary panels. Except for 20CRv2, the rest of the reanalyses perform very similar to each other throughout the troposphere with highest temporal correlations notably at 100 hpa. Vertical contour plots reveal that reanalyses temporal correlations with AIRS are highest at high latitudes (Fig. 9). NCEP1 and NCEP2 have notable high temporal correlations at the top of the tropopause over the eastern, western and northern boundary panels (Fig. 9, see also Supplemental figure S10). Worst performances are generally noticeable around 600 hpa to 700 hpa except for the southern boundary panel Relative humidity ERA-I has the highest temporal correlations over the northern, western and eastern boundary panels while MERRA is the best for the southern boundary panel (Table 2). For all the reanalyses across all the panels, worst performances are generally at the lower layers of the troposphere (Fig. 8). Based on the vertical contour plots, there is an inclination of reanalyses to be better correlated over time with AIRS at high latitudes (Fig. 10, also see Supplemental figure S11). Since 20CRv2 uses Ensemble Kalman filter system data assimilation (Table 1), which only assimilates surface-pressure observations, this could partly explain its particularly poor performance especially around tropopause level. This reanalysis also has relatively coarser vertical and horizontal resolutions. MERRA and ERA-I have the finest vertical and horizontal resolutions compared to the rest of the reanalyses considered in this study (Table 1). This could contribute to their relatively better performances due to fewer interpolation errors. Although MERRA uses 3D-variational data assimilation while ERA-I uses the 4D-variational approach, the two reanalyses performances are similar, which might also be as a result of their common use of Optimum Interpolation Sea Surface Temperature (OISST) Fig. 8 Temporal correlation coefficient for temperature and relative humidity (%) over the levels of the atmosphere for the boundary panels

12 D. B. Moalafhi et al. Fig. 9 Vertical contour plots of Temperature temporal correlation coefficient for the eastern and western boundary panels Fig. 10 Vertical contour plots of relative humidity temporal correlation coefficient for the eastern and western boundary panels

13 Evaluating global reanalysis datasets for provision of boundary conditions in regional forcing (Reynolds et al. 2007). This SST forcing is contributing to the superiority of the MERRA and ERA-I products for the East, West and South boundaries Performance in extreme wet and dry years Further, evaluation was made for the driest and wettest years of the study period. The driest and wettest years were ascertained based on Global Precipitation Climatology Project (GPCP) rainfall data and these are identified as 2005 and 2012 respectively. For the driest year (i.e., 2005), and considering both temperature and relative humidity, MERRA gives the best performance across all the panels as shown in Table 3 and subsequent ranks (Table 6). The rank score for MERRA is 42 followed by ERA-I with 53. Even for the wettest year (i.e., 2012), MERRA has the best rank score of 41 followed by ERA-I with rank score of 57 across all the panels as shown in Table 4 (also see Table 7). ERA- I s over-estimation of relative humidity is more evident for the driest year. It is also revealed that ERA-I underestimates temperature across all boundary panels for both years. Since MAE results are similar to those of RMSE, the former was left out in computing the rankings of reanalyses to avoid giving more weight to one type of error statistics. Considering all the ranks for mean bias, RMSE and temporal correlation for both temperature and relative humidity over the study period ( ) and for the driest (2005) and wettest (2012) years, MERRA has the overall best rank of 140 followed by ERA-I with rank score of 170 (Tables 5, 6, 7) over all the boundary panels. Although ERA-I is the second best reanalysis for the domain, it over-estimates relative humidity and thus using it to provide boundary conditions for the domain could likely result in too much precipitation over the region. The performance of ERA-I in mean bias are consistent with the findings of Zhang et al. (2013) and Kalognomou et al. (2013) in which ERA-I manifests a very strong moisture convergence over the eastern equatorial Atlantic and consequently resulting in strong precipitation over the southern African region. Ratna et al. (2013) also revealed that forcing WRF with ERA-I over southern Africa results in positive rainfall biases over the region irrespective of convective parameterization schemes used. The cold-bias of ERA-I over all the boundary panels is also echoed by a study over the Amazon basin (Betts et al. 2009) in which ERA-I is cold-biased compared to in situ station data. This cold bias of ERA-I were also found by Brands et al. (2013) in evaluating reanalyses uncertainty over tropical African regions. The summary of discussions on the reanalyses performances considering the whole study period ( ), the driest (2005) and wettest (20,120 years is also given in Table 8. Table 3 Error statistics over the individual boundary panels for the driest year (2005) of the study period NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Temperature (K) S N W E Relative humidity (%) S N W E Bold values = best

14 D. B. Moalafhi et al. Table 4 Error statistics over the individual boundary panels for the wettest year (2012) of the study period NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Bias RMSE MAE r Temperature (K) S N W E Relative humidity (%) S N W E Bold values = best Reanalyses performances are ranked vis a vis climate mean and climate variability to suggest which are most suitable for downscaling in each category (Table 9). The ranking reveals that MERRA reanalysis is the most suitable (rank score 49) when consideration is based on climate mean. Alternatively, NCEP1 is the next most suitable (rank score 58). If choice of reanalysis is sought for climate variability downscaling undertakings, ERA-I (rank score 41) is most suited followed by MERRA (rank score 44). It is also noted that MERRA and ERA-I are the most suitable through RMSE which combines both magnitude and variability. To test the reanalyses performances over different time scales, RMSEs were computed at weekly and monthly time scales for the whole study period ( ), the driest year (2005) and the wettest year (2012) for the respective boundary panels. The results (not presented here) in terms of RMSEs magnitudes and ranking of the reanalysis are consistent with those from the daily results presented, suggesting that persistence and related biases are of the same order across all the reanalyses datasets considered. Even results of evaluations for time correlation at monthly time scale (for the study period and the rainfall extreme years) and annual time scale (for the study period) are also consistent with those at daily time scale Wind field comparison between reanalyses Due to the lack of reliable observations of wind that span the atmosphere in space and time, it is not possible to perform the same evaluation at the RCM lateral boundaries that has been performed for temperature and humidity. Here the differences in wind fields between the reanalyses at the RCM lateral boundaries are investigated to provide an indication of the variance in this aspect of the RCM LBCs. We arbitrarily choose ERA-I as the reference dataset, and calculate the bias, RMSE, MAE and temporal correlation for each remaining reanalyses against this reference, over each of the boundary panels for the whole study period ( ), driest (2005) and wettest (2012) years (see supplemental tables S1 to S3, and figures S12 to S15). The statistics over the entire period examined suggests that the MERRA is most similar to ERA-I with the lowest MAEs, highest temporal correlations and amongst the lowest biases. It is also notable that NCEP1 consistently produces the lowest RMSEs. While the meridional wind biases are generally quite small for MERRA, the zonal wind biases are consistently higher. These differences are, however, very small. Also, while MERRA frequently has the highest temporal correlation, this correlation can be as low as 0.64 for meridional wind on the northern boundary. Thus, while the wind fields of MERRA are more similar to ERA-I than any of the other reanalyses, the temporal sequence of winds can differ between them and hence is only weakly constrained by the assimilated observations.

15 Evaluating global reanalysis datasets for provision of boundary conditions in regional Table 5 Reanalyses ranking based on error statistics of Table 2 (whole period: ) NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Temperature (K) S N W E Total Relative humidity (%) S N W E Total Table 6 Reanalyses ranking based on error statistics of Table 3 (driest year: 2005) NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Temperature (K) S N W E Total Relative humidity (%) S N W E Total Examination of the wind field statistics associated with extreme dry and wet years reveals MERRA to be much more similar to ERA-I than any other reanalysis. In fact MERRA wind fields are more similar to ERA-I wind fields during these extreme years than during the record as a whole. 20CRv2 wind fields are least similar to ERA-I during these extreme years with the lowest temporal correlations, highest RMSEs and MAEs, and biases that reach over 1 m/s for zonal wind on the southern boundary. While these results do not evaluate the performance of any particular reanalysis, they do show that ERA-I and MERRA have the most similar wind fields at the lateral boundary locations. Hence the wind field differences between these reanalyses are expected to produce small differences inside the RCM domain compared to the other reanalyses. That is, the relative ranking of ERA-I and MERRA based on temperature and relative humidity at the lateral boundaries as an indicator of preferred RCM LBCs is more robust than that of 20CRv2 whose large wind field differences may dominate the RCMs internal climate Application over a different domain The robustness of the approach was evaluated through doing the same assessment with a different (smaller) secondary domain (Fig. 1) stretching between longitude 9.5

16 D. B. Moalafhi et al. Table 7 Reanalyses ranking based on error statistics of Table 4 (wettest year: 2012) NCEP1 NCEP2 ERA-I 20CRv2 MERRA Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Temperature (K) S N W E Total Relative humidity (%) S N W E Total Table 8 Summary reanalyses performances for temperature and relative humidity Whole period: Driest year: 2005 Wettest year: 2012 Temperature (K) Best: NCEP1 Best: MERRA Best: MERRA Second best: MERRA Second best: ERA-I Second best: ERA-I Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2 Relative humidity (%) Best: MERRA Best: MERRA Best: ERA-I Second best: ERA-I Second best: ERA-I Second best: MERRA Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2 Overall Best: MERRA Best: MERRA Best: MERRA Second best: NCEP1 Second best: ERA-I Second best: ERA-I Worst: 20CRv2 Worst: 20CRv2 Worst: 20CRv2 Conclusion MERRA best followed by ERA-I Table 9 Summary of reanalyses performances under fitness of magnitude (mean bias) and variability (temporal correlation) NCEP1 NCEP2 ERA-I 20CRv2 MERRA Magnitude Variability Magnitude Variability Magnitude Variability Magnitude Variability Magnitude Variability Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Bias RMSE r Whole period ( ) Driest year (2005) Wettest year (2012) Totals 58(2) 67 (3) 75 (3) 68 (3) 86 (4) 80 (4) 80 (4) 49 (2) 41 (1) 100 (5) 110 (5) 102 (5) 49 (1) 47 (1) 44 (2) Rankings are shown in brackets

17 Evaluating global reanalysis datasets for provision of boundary conditions in regional to 38.5 and latitude 17.5 to Evaluations based on mean bias, RMSE, MAE and temporal correlation were made over four (4) 4 boundaries of the domain. The outcomes from this assessment are similar to those for the main domain. In terms of overall rankings, MERRA comes out to be the best and followed by ERA-I. In fact all the five (5) reanalyses still rank the same as with the main domain. The results of such are summarized in supplemental tables S4 to S9 with contour plots also shown through supplemental figures S16 through S31. 5 Conclusions Four dimensional (4-D) atmospheric fields (temperature and relative humidity) used in regional climate model LBCs from five global reanalyses that have been used widely and that have given mixed performances both globally and regionally, have been evaluated against AIRS observations over southern Africa. This analysis is aimed at identifying the reanalysis that can provide the most accurate LBCs for climate simulations over the region. This study is the first of its kind to undertake 4-D evaluation of relevant LBC variables from global reanalyses at the intended boundaries of a domain of interest. By considering the vertical extent of the boundaries, from the surface through the troposphere, we ensure that the choice of reanalysis to provide LBCs to the RCM is as objective as possible and subsequently the most accurate LBCs possible are used for climate simulations. AIRS products at a grid resolution of 1 1 and at selected pressure levels were sub-setted to four (4) panels. These panels are located at the boundaries of the domain and extend by 4 inwards. Reanalyses were subsequently similarly sub-setted and interpolated as per the AIRS s chosen pressure levels and also temporally resampled to match the AIRS satellite data. The study reveals that reanalyses generally over-estimate mean temperature over all the boundary panels, especially from 700 hpa level towards the surface. Although 20CRv2 s over-estimation of mean temperature at the top is a clear deviation from the rest, all reanalyses perform similarly to each other throughout most of the troposphere. Considering all the boundary panels together, the best performance for temperature mean bias goes to ERA-I. Mean relative humidity is also over-estimated by reanalyses, except for NCEP2 over the northern boundary. ERA-I generally performs worst with largest positive bias over all boundary panels. Notable mean relative humidity overestimations by all reanalyses are at the tropopause level over all boundary panels. When all the boundary panels are considered, NCEP1 has the best performance in relative humidity mean bias. Analysis through RMSE reveals that reanalyses performances are worse at higher latitudes as revealed through the southern boundary panel especially at the tropopause level. 20CRv2 performs consistently worse near the top of the atmosphere over all boundary panels. When looking at all the boundary panels, NCEP1 has the best performance in temperature RMSE. ERA-I and NCEP1 have joint best performances in relative humidity RMSE over all the boundary panels. Generally reanalyses perform best for relative humidity near the surface. Generally reanalyses are more correlated in time with AIRS at high latitudes and NCEP1 is the best in this regard when considering all the boundary panels. 20CRv2 has consistently the lowest temporal correlations particularly around the tropopause. Inclinations towards high temporal correlations for high latitudes are also evident for relative humidity though not that well pronounced. ERA-I and MERRA are best performers in the mid-troposphere. Based on mean bias, RMSE, and temporal correlation evaluations for both temperature and relative humidity over all the boundary panels, MERRA is the best followed by NCEP1 and ERA-I. Further analysis using the driest (2005) and wettest (2012) years reveals that MERRA still ranks the best followed by ERA-I for both temperature and relative humidity mean bias, RMSE and temporal correlation over all the boundary panels. It is noted that ERA-I s overestimation of relative humidity is more evident for the driest year and it also under-estimates temperature across all the boundary panels for both the driest and wettest years. Based on all the results of the whole study period ( ), the driest year (2005) and the wettest year (2012), MERRA and ERA-I appear to be the best overall reanalyses for provision of temperature and humidity LBCs for the study domain from the African equator down to the southern-most coast of South Africa including Madagascar. ERA-I has the best performance in terms of capturing the time variability of the atmospheric temperature and humidity fields as shown by the RMSE. When considering temporal correlation, ERA-I also has an edge over MERRA. However, MERRA has the best overall performance being slightly worse than ERA-I at capturing the variability but considerably better in terms of the biases present. When considering choice of reanalysis for climate mean, MERRA is the most suited over the domain. Alternatively if the choice is for climate temporal variability studies, ERA-I appears to be the most suitable. It is revealed that RMSEs (at weekly and monthly time scales) and temporal correlation coefficient (at monthly and annual time scales) are consistent with those from the daily results presented. The robustness of the approach was demonstrated over a different domain in the same region in which the reanalyses rankings came out the same with MERRA the overall best