Non-stationary Cross-Covariance Models for Multivariate Processes on a Globe

Transcription

1 Scandinavian Journal of Statistics, Vol. 38: , 2011 doi: /j x Published by Blackwell Publishing Ltd. Non-stationary Cross-Covariance Models for Multivariate Processes on a Globe MIKYOUNG JUN Department of Statistics, Texas A&M University ABSTRACT. In geophysical and environmental problems, it is common to have multiple variables of interest measured at the same location and time. These multiple variables typically have dependence over space (and/or time). As a consequence, there is a growing interest in developing models for multivariate spatial processes, in particular, the cross-covariance models. On the other hand, many data sets these days cover a large portion of the Earth such as satellite data, which require valid covariance models on a globe. We present a class of parametric covariance models for multivariate processes on a globe. The covariance models are flexible in capturing non-stationarity in the data yet computationally feasible and require moderate numbers of parameters. We apply our covariance model to surface temperature and precipitation data from an NCAR climate model output. We compare our model to the multivariate version of the Matérn cross-covariance function and models based on coregionalization and demonstrate the superior performance of our model in terms of AIC (and/or maximum loglikelihood values) and predictive skill. We also present some challenges in modelling the cross-covariance structure of the temperature and precipitation data. Based on the fitted results using full data, we give the estimated cross-correlation structure between the two variables. Key words: cross-covariance model, linear model of coregionalization, multivariate process, non-stationary process, process on a globe 1. Introduction Geophysical or environmental problems routinely involve multiple variables measured at the same spatial location and time point. Often, the main interest is to study the relationships between the multiple variables, which would include an accounting of any spatial and/or temporal correlations. On the other hand, with the advance of science and technology, it is common to have data with global coverage. One good example is the study about the relationship between surface temperature and precipitation (Trenberth & Shea, 2005; Tebaldi & Lobell, 2008; Tebaldi & Sansó, 2009). As stated in Tebaldi & Lobell (2008), from the climate impact research point of view, studying the joint distribution of surface temperature and precipitation is more interesting than studying each variable separately. Trenberth & Shea (2005) estimate the empirical (spatial) cross-correlation between surface temperature and precipitation using the numerical model outputs from the Community Climate System Model version 3 (CCSM3) developed by the National Center for Atmospheric Research (NCAR). However, their estimates are based on sample correlations and, as several authors point out including Bishop & Hodyss (2007), sample correlations often give spurious correlations when the dimension of the system is much larger than the sample size. Therefore, it is essential to develop joint models for the variables that can account for not only the marginal but also the cross-covariance structures and that are valid over the whole globe. A number of authors have developed cross-covariance models for multivariate spatial processes. One of the most traditional methods is the linear model of coregionalization (LMC) (Goulard & Voltz, 1992; Wackernagel, 2003) and the key idea is to represent each process as a linear combination of latent, independent and stationary (often isotropic) processes.

2 Scand J Statist 38 Non-stationary cross-covariance models 727 Schmidt & Gelfand (2003) present a Bayesian stationary cross-covariance model based on the idea of the LMC. Gelfand et al. (2004) provide a good review of the history of the methods for multivariate processes and Schmidt & Gelfand (2003) extend the model by using a spatially varying LMC to account for non-stationarity. Majumdar & Gelfand (2007) present an approach to modelling stationary processes based on convolving covariance functions. This model is then extended to non-stationary processes in Majumdar et al. (2010). A semiparametric approach to modelling multivariate spatial processes is proposed in Reich & Fuentes (2007). Their covariance model is non-stationary but has a separable structure; the cross-covariance is factored into a multivariate component and a spatial component, which may be limiting in some situations. Choi et al. (2009) use a spatio-temporal version of the LMC model to deal with speciated fine particles over the US with separable covariance functions. In terms of developing parametric classes of covariance models for multivariate spatial processes, other than those based on LMC, there has only been a few papers. Apanasovich & Genton (2010) propose using latent dimensions to create a valid covariance model for multivariate processes from a covariance model for univariate process. They present the model for spatio-temporal processes. Their method is convenient to produce valid covariance models for multivariate processes but they assume stationarity (although in principle their method can be used for non-stationary processes). They introduce a concept of distance between different processes, which is different from the usual spatial distances or temporal lags and it is not clear what this distance actually means and how it compares with physical distances in the space and time domains. Under their setting, one needs to estimate this distance along with covariance parameters. Gneiting et al. (2010) present a Matérn type covariance model for multivariate processes. Their model is for isotropic processes only. One of the nice features of their model is it allows different smoothness for different processes in the multivariate setting, which can be useful for some data. Note though that their cross-covariance model is symmetric. That is, if we consider a bivariate process, (Z 1, Z 2 ), on locations s 1 and s 2, their model implies cov{z 1 (s 1 ), Z 2 (s 2 )} = cov{z 2 (s 1 ), Z 1 (s 2 )}, for all s 1 and s 2, which may not be the case in many geophysical and environmental data sets. The co-located correlation parameter ρ, which controls the strength of cross-correlation between the two variables, is constant over the entire domain and this may be too restrictive for some data sets. As demonstrated in section 4, for the data set that we consider in this article, this limitation leads to an estimate of ˆρ 0, even though there is a clear dependence between the two variables. Furthermore, the above covariance functions are designed for stationary (or isotropic) processes and none are for processes on a globe. To the best of our knowledge, there is no such flexible non-stationary cross-covariance function for spatial processes on a sphere. Our focus in this article is to develop cross-covariance functions for multivariate processes on a globe. Moreover, the covariance model is flexible enough to capture non-stationarity in the data and other complex covariance patterns. Our covariance models require moderate numbers of covariance parameters and are thus computationally feasible. The remainder of the article is organized as follows. In section 2, we discuss some properties of non-stationary covariance structure. Section 3 presents the construction of our covariance model and discusses some computational issues. The application to the joint modelling of global surface temperature and precipitation data is presented in section 4. Section 4 shows a comparison of our model with some models proposed in Gneiting et al. (2010) and the LMC models. We also show our estimated cross-correlation between the surface temperature and precipitation and we compare it with the result of Trenberth & Shea (2005). We conclude the article with some discussion in section 5.

3 728 M. Jun Scand J Statist Non-stationary covariance structure In this section, we explore some prevalent features of non-stationarity in the covariance structure of geophysical processes on a globe. We discuss those properties for both marginal and cross-covariances in parallel and show some empirical figures from our data (for details of the data, see section 4.1). Throughout the section, we denote the variable of interest as (Z i (L, l), i = 1, 2,..., N), a multivariate process on the surface of a globe S 2 (the surface of a sphere in R 3 with radius R) and we illustrate for the case when N = 2. Note that L and l denote latitude and longitude, respectively Dependence on latitude It is common for geophysical processes on a globe to have covariance structure depending on latitude (Stein, 2007). In particular, the local variation of the process usually changes with latitude and in fact, Jun & Stein (2008) show that variances of several linear combinations of total column ozone level exhibit strong dependence on latitude. Surface temperature and precipitation data that we consider in this article possess this kind of non-stationarity for both marginal and cross-covariances. Figures 1 3 in Trenberth & Shea (2005) show that the standard deviations for both variables as well as their cross-correlations have patterns depending on latitude. Figure 1 of this article displays the standard deviations and cross-correlations of surface temperature and precipitation, averaged over November to March each year and then averaged over 1970 to Figures (A), (C) and (E) give each quantity with respect to latitude, that is, at each latitude, the standard deviation or cross-correlation of the data across all longitude values is calculated. Figures (B), (D) and (F) give the standard deviation or crosscorrelation of the data calculated across all latitude values, at each longitude. See Appendix for more details on how these values are calculated. Notice that although we see some dependence of standard deviation and cross-correlation with respect to longitude, the dependence is more obvious with respect to latitude. This may suggest the processes are reasonably modelled as axially symmetric (Jones, 1963) both marginally and jointly; the covariance structure is stationary with respect to longitude and non-stationary with respect to latitude Longitudinal reversibility We say a univariate process Z 1 is longitudinally reversible if cov{z 1 (L 1, l 1 ), Z 1 (L 2, l 2 )} = cov{z 1 (L 1, l 2 ), Z 1 (L 2, l 1 )} for all L 1, L 2, l 1, l 2 (Stein, 2007). Stein (2007) and Jun & Stein (2008) show that the total column ozone process is longitudinally irreversible; we find that it is also the case for both temperature and precipitation data marginally (not shown). This concept can be applied to cross-covariances as well. Call the cross-covariance of the two processes Z 1 and Z 2 longitudinally reversible if cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} = cov{z 1 (L 1, l 2 ), Z 2 (L 2, l 1 )} for all L 1, L 2, l 1, l 2. For some data sets, however, longitudinally irreversible cross-covariances may be hard to estimate. Figure 2(A) shows the empirical estimate of the difference of the cross-correlations at a latitude band, cor{z 1 (L, l), Z 2 (L, l + Δ)} cor{z 1 (L, l + Δ), Z 2 (L, l)}, against latitude (x-axis) and longitude lag, Δ (y-axis). The empirical estimate for the above quantity is based on temporally averaged data, a 30-year average (see section 4.1 for details on how we aggregate the data temporally). To get these empirical estimates, first we bin the

4 Scand J Statist 38 Non-stationary cross-covariance models 729 A Standard deviation of Z 1 B Standard deviation of Z 1 Standard deviation Standard deviation C Standard deviation E Latitude Longitude Standard deviation of Z 2 D Standard deviation of Z 2 Standard deviation Latitude Longitude Cross correlation of Z 1 & Z 2 F Cross correlation of Z 1 & Z 2 Correlation Correlation Latitude Longitude Fig. 1. Comparison of empirical covariances/correlations (dots) and corresponding fitted values (solid lines) from the model NMG2. Fitted values are calculated using the covariance parameter estimates from section 4.4. Temperature process is denoted by Z 1 and precipitation process is denoted by Z 2. For details on how the empirical and fitted values are calculated, see Appendix. latitude with the bin size roughly 7. Then within each bin, for each Δ, we calculate the correlations between the two variables at the longitude lag of Δ. To assess the uncertainty of the empirical longitudinal irreversibility, we also split the total 30-year period into 30 intervals of one year and instead of calculating irreversibility using a 30-year average, we calculate the irreversibility based on 30 annual averages. The mean and the standard deviation of these irreversibility quantities based on these 30 data points (annual averages) are given in (B) and (C), respectively. Note (A) and (B) give quite similar patterns although the range of irreversibility

5 730 M. Jun Scand J Statist 38 A B C Longitude lag D Latitude Latitude E Latitude Longitude lag Latitude Latitude Fig. 2. (A):Empiricalestimateofalongitudinalirreversibility,cor{Z 1 (L, l), Z 2 (L, l + Δ)} cor{z 1 (L, l+δ), Z 2 (L, l)}, based on a 30 year average; (B): similar quantity as (A) except that here longitudinal irreversibility is based on 30 annual averages and the mean of these 30 irreversibility quantities is displayed; (C): standard deviation of the 30 irreversibility quantities; (D): Fitted longitudinal irreversibility using OLS; and (E): Fitted longitudinal irreversibility using MLE. Here Z 1 is the temperature process and Z 2 is the precipitation process. The units for both axes are degrees and the fitted values in (E) are from the estimates given in Table 5. For details about how we get the values in (A) (C), see section 2.2. For details about how we get the OLS estimates, see section 4.4. in (B) is narrower. Although there seems to be some strong longitudinal irreversibility in the cross-correlation near the poles and mid latitude of Southern Hemisphere at large longitudinal lags, the uncertainty associated with it (especially near the poles) is high. It might be hard to fit the pattern with any smooth function of latitude and longitude lags due to the complex nature of the empirical irreversibility surface. See section 4 for more discussion on this issue Asymmetry We now define a general concept of asymmetry for multivariate spatial processes. We call the cross-covariance of Z 1 and Z 2 symmetric if cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} = cov{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )}

6 Scand J Statist 38 Non-stationary cross-covariance models 731 A b 1 = 1 b 2 = 0.1 B b 1 = 1 b 2 = 0.1 C b 1 = 1 b 2 = 0.1 D b 1 = 1 b 2 = 0.1 Irreversibility a 2 = 0 a 2 = 1 a 2 = 0.1 a 2 = Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) E b 1 = 1 b 2 = 0.1 F b 1 = 1 b 2 = 0.1 G b 1 = 1 b 2 = 0.1 H b 1 = 1 b 2 = 0.1 Irreversibility Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Fig. 3. Longitudinal irreversibility in the correlation scale, cor{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cor{z 1 (L 1, l 2 ), Z 2 (L 2, l 1 )}, against l 2 l 1.WesetL 1 = 0, L 2 = 10 N, α = 1, β = 2000 (Km) and a 1 = 1. Solid line gives the irreversibility value when a 2 0 and dotted line gives the value when a 2 < 0 (same colour gives the same magnitude of a 2 ). Top row (Figs (A) (D)) gives the value when ν = 5 and the bottom row (Figs (E) (H)) gives the value when ν = 1.5. Grey line gives the horizontal zero line. for all L 1, L 2, l 1, l 2. If the cross-covariance structure is asymmetric, then the cross-covariance matrix of some set of observations will generally be asymmetric. Note that the model proposed in Gneiting et al. (2010) is always symmetric. The model from the linear model of coregionalization is also symmetric unless the coefficients vary spatially. Apanasovich & Genton (2010) present cross-covariance models that is asymmetric in space-time domain, which is somewhat different from the asymmetry discussed in this article. 3. Methodology In this section, we develop joint covariance models that can exhibit the non-stationary properties discussed in section 2 for not only marginal but also cross-covariance structure. We also discuss computational methods that enable us to compute full likelihoods efficiently when we have large global data sets on a regular grid for multivariate processes Model Jun & Stein (2007) proposed an approach to produce non-stationary covariance models for a univariate process on a globe to capture space-time asymmetry, which is commonly found in environmental data (Gneiting, 2002; Jun & Stein, 2004; Li et al., 2008). The key idea is to apply differential operators with respect to latitude, longitude and time to an isotropic spatio-temporal process and section 4 of Jun & Stein (2007) demonstrates the effectiveness of the approach in capturing such space-time asymmetry. Jun & Stein (2008) further explore the idea of applying differential operators with respect to latitude and longitude to an isotropic spatial process to represent various non-stationary properties of univariate process on a globe. They demonstrate that their model captures small scale variation in the process for a univariate process well. The key in the model is the flexibility, resulting from the products of first order differential operators with respect to latitude and longitude, applied to an underlying process. We extend this idea for multivariate spatial processes on a globe. Now we show how the idea of applying differential operators to the processes can be applied to multivariate isotropic spatial processes to create non-stationary cross-covariance

7 732 M. Jun Scand J Statist 38 structure, in particular, asymmetry and longitudinal irreversibility, which depends on latitude in a flexible way. Suppose we have a multivariate spatial process (Z 1 (L, l),..., Z N (L, l)), defined on a globe, S 2 and we are interested in modelling the joint distribution of Z i s. We will focus on the case that N = 2 here; for the case that N > 2, the method extends in a natural way. Let us assume (Z 1, Z 2 ) is a bivariate process with mean zero. We also assume that Z i s are axially symmetric both marginally and jointly. Let us write Y = G(α, β, ν) if the process Y defined on S 2 has mean zero and its covariance is given by a Matérn covariance function: cov{y (L 1, l 1 ), Y (L 2, l 2 )} = K(L 1, L 2, l 1 l 2 ) = α ( d β ) ν K ν ( d β ). (1) Here, K denotes the covariance function of Y, L i s are latitude values and l i s are longitude values (i = 1, 2). The parameters, α, β, ν > 0, are the sill, spatial range, and the smoothness parameters for a Matérn class, respectively, K ν is the modified Bessel function, and { ( ) ( )} 1/2 d = d(l 1, L 2, l 1 l 2 ) = 2R sin 2 L1 L 2 + cos L 1 cos L 2 sin 2 l1 l 2 (2) 2 2 denotes the chordal distance between the two locations, (L 1, l 1 ) and (L 2, l 2 ). To ensure the positive definiteness of (1) on S 2, we need to use chordal distance as a spatial metric instead of a geodesic distance (see Jun & Stein (2007) for a detailed discussion). Let us first consider a simple setting: δz i (L, l) = a i {Y (L + δ, l) Y (L, l)} + b i {Y (L, l + δ) Y (L, l)}, i = 1, 2, (3) with a i, b i constants and δ > 0. When δ 0, (3) is essentially equivalent to the model with differential operators with respect to latitude and longitude applied to the process Y in the L 2 sense (instead of taking differences). It is easy to see from (3) that when δ 0, the cross-covariance of Z 1 and Z 2 can be written as cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} = 2 a 1 a 2 K(L 1, L 2, l) b 1 b 2 L 1 L 2 l K(L 1, L 2 2, l) a 1 b 2 L 1 l K(L 1, L 2, l) + b 1 a 2 L 2 l K(L 1, L 2, l), (4) with l = l 1 l 2. Note that the second order partial derivatives of K in (4) originate from the limits of the second order differences of the covariance K. For example, 1 K(L 1, L 2, l) = lim L 1 L 2 δ 0 δ {K(L δ, L 2 + δ, l) K(L 1 + δ, L 2, l) K(L 1, L 2 + δ, l) + K(L 1, L 2, l)}. Furthermore, to have the limit properly defined, we need to have ν > 1. For more details on this condition, see section 2 of Stein (1999) and the result in Jun & Stein (2007). Now let us consider the longitudinal irreversibility and the asymmetry discussed in sections 2.2 and 2.3. It is straightforward from (4) that when δ 0, the longitudinal irreversibility is given by cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cov{z 1 (L 1, l 2 ), Z 2 (L 2, l 1 )} = 2a 1 b 2 L 1 l K(L 1, L 2, l) + 2b 1 a 2 L 2 l K(L 1, L 2, l), (5) and the asymmetry is given by

8 Scand J Statist 38 Non-stationary cross-covariance models 733 cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cov{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )} { = ( a 1 b 2 + b 1 a 2 ) L 1 l K(L 1, L 2, l) + } L 2 l K(L 1, L 2, l). (6) From (5) and (6), it is clear that the types of non-stationary in the cross-covariance discussed in sections 2.2 and 2.3 are achieved mainly from the interactions between the first and second order differences of Y in (3). If a 1 = a 2 and b 1 = b 2, the asymmetry in (6) reduces to zero, although the longitudinal irreversibility in (5) may not be zero. We show some plots of the cross-covariance structure for various values of a i s and b i sto further demonstrate the behaviour of the proposed covariance model in a simple setting. We work with correlation scale instead of the covariance scale. We fix α = 1, a 1 = 1 and β = 2000 (Km). We vary a 2, b 1, b 2 and ν to explore the non-stationarities of the covariance model of (3) when δ 0. Figure 3 gives the longitudinal irreversibility in the cross-correlation structure given in (5). The irreversibility in the covariance scale should have the same shape except the scale. Notice that different ν values give different shapes of the irreversibility. From (5), the irreversibility depends on a i s and b i s through a 1 b 2 and b 1 a 2. If the signs of a 1 b 2 and b 1 a 2 change together, the sign of irreversibility should also change. Therefore, we see that several sets of the a i s and b i s give either the same irreversibility or the same magnitude of irreversibility with different signs. For example, when a 2 = 0, the pairs, (A) and (B), (C) and (D), (E) and (F), and (G) and (H), give the same irreversibility. Moreover, when a 2 = 0, the irreversibilities in (A) and (B) and those in (C) and (D) have same magnitudes but different signs. When a 2 = 0, the pairs, (A) and (D), (B) and (C), (E) and (H), and (F) and (G), give the same magnitude of the irreversibility with different signs. It may then appear that there are some identifiability problems in a i s and b i s since some sets of these coefficients give the same irreversibility curves. However, this is not the case for the covariance structure of the bivariate process. As long as we fix the sign of only one of the four coefficients (a i s and b i s), we can avoid the identifiability problem (see (4)). Figure 4 gives the asymmetry against longitudinal lags in the cross-correlation structure given in (6). Unlike the longitudinal irreversibility, the asymmetry in the covariance scale may have different shape than the asymmetry in the correlation scale. This is because the asymmetry in the correlation scale is cor{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cor{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )} cov{z = 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} var{z1 (L 1, l 1 )}var{z 2 (L 2, l 2 )} cov{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )} var{z1 (L 2, l 2 )}var{z 2 (L 1, l 1 )}, (7) and the terms in each denominator are in general not the same except the case such as b 2 = a 2 b 1 (a 2 = 0), for any L 1, L 2, l 1, l 2. Therefore, the asymmetries in the correlation scale in general do not simply depend on the coefficients through a 1 b 2 b 1 a 2 but in a more complex manner. For example, when a 2 = 0, even if (A) and (B) have the same b 2 value, their asymmetries are different. When a 2 = ±0.1 and b 2 = a 2 b 1, the asymmetry is zero. When a 2 = ±0.1 and b 2 = a 2 b 1, then we may get the same magnitude of the irreversibility but with different signs. For example, the pairs, (B) and (C) with a 2 = 0.1 and (A) and (D) with a 2 = 0.1, give the same magnitude of the irreversibility with different signs. It is interesting to note that even if l 1 l 2 = 0, the asymmetry for some combinations of the coefficients is not zero (because L 1 = L 2 ). Figure 5 (A) (D) display the asymmetry against latitude. Note the asymmetries are zero when L 2 = 0 and the pairs, (A) and (D) or (B) and (C), give symmetric asymmetry values around L 2 = 0. The fact that the asymmetries are zero when L 2 = 0 can also be easily

9 734 M. Jun Scand J Statist 38 A b 1 = 1 b 2 = 0.1 B b 1 = 1 b 2 = 0.1 C b 1 = 1 b 2 = 0.1 D b 1 = 1 b 2 = 0.1 Asymmetry a 2 = 0 a 2 = 1 a 2 = 0.1 a 2 = Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) E b 1 = 1 b 2 = 0.1 F b 1 = 1 b 2 = 0.1 G b 1 = 1 b 2 = 0.1 H b 1 = 1 b 2 = 0.1 Asymmetry Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Longitudinal lag (degree) Fig. 4. Asymmetry in the correlation scale, cor{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cor{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )}, against l 2 l 1.WesetL 1 = 0, L 2 = 50, α = 1, β = 2000 (Km) and a 1 = 1. Solid line gives the asymmetry value when a 2 0 and dotted line gives the value when a 2 < 0 (same colour gives the same magnitude of a 2 ). Top row (Figs (A) (D)) gives the value when ν = 5 and the bottom row (Figs (E) (H)) gives the value when ν = 1.5. Grey line gives the horizontal zero line. explained by (11) (we will discuss this further when we introduce (11) later). Note that the asymmetries are not necessarily symmetric against the equator, which is realistic for most real data sets. We now generalize the model in (3) in the sense that the coefficients are functions of latitude values. That is, we write: Z i (L, l) = n k = 1 { A i,k (L) L + B i,k(l) l } Y k (L.l) + C i (L)Y 0 (L, l). (8) Here, the partial derivatives are defined in the L 2 sense and Y k = G(α k, β k, ν k )(α k, β k > 0, k = 0,..., n, ν 0 > 0 and ν k > 1, k = 1,..., n). If ν k 1fork = 1,..., n, then the mean square derivatives of Y k is not properly defined. We assume the Y k s (k = 0,..., n) are independent of each other. A useful simplification is to assume Y k share the same covariance parameters for k = 1,..., n, but we generally let Y 0 have different covariance parameters than Y 1,..., Y n to allow sufficient flexibility in the local behaviour of the model. Note it is not necessary to include Y 0 in (8). We may then let C i = 0 for parsimony. The functions A i,k, B i,k and C i in (8) are non-random functions and we model these functions as linear combinations of Legendre polynomials. For instance, we let m A i,k (L) = a ikj P j (sin L), (9) j = 0 where P j denotes the Legendre polynomial of order j. Then a ikj R are additional covariance parameters to be estimated along with other covariance parameters. The maximum order of Legendre polynomials used here, m, is chosen arbitrarily and we expect a modest number of m should be able to produce flexible covariance functions. The values of m for A i,k, B i,k and C i may be different. Larger m will obviously give more flexibility to the covariance structure and we may let m = 0 for a parsimonious model. We compare the different possibilities discussed here in section 4. Although the Y k s are independent of each other, the Z i s have non-zero cross-covariance and we can get explicit expressions for the cross-covariance of the Z i s. For instance, suppose n = 1, Y 1 = G(1, β, ν) (ν > 1) and C i = 0(i = 1, 2). Set h = h(l 1, L 2, l 1 l 2 ) = (d/β) 2 for d defined

10 Scand J Statist 38 Non-stationary cross-covariance models 735 A b 1 = 1 b 2 = 0.1 B b 1 = 1 b 2 = 0.1 C b 1 = 1 b 2 = 0.1 D b 1 = 1 b 2 = 0.1 Asymmetry a 2 = 0 a2 = 0.1 a 2 = 1 a2 = 10 L 2 (degree) L 2 (degree) L 2 (degree) L 2 (degree) E b 1 = 1 b 20 = 0.1 F b 1 = 1 b 20 = 0.1 G b 1 = 1 b 20 = 0.1 H b 1 = 1 b 20 = 0.1 Asymmetry L 2 (degree) L 2 (degree) L 2 (degree) L 2 (degree) Fig. 5. (A) (D):Asymmetryinthecorrelationscale,cor{Z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} cor{z 1 (L 2, l 2 ), Z 2 (L 1, l 1 )}, against L 2, when L 1 = 0, l 1 = 0 and l 2 = 30. We set α = 1, β = 2000 (Km), ν = 5 and a 1 = 1. Solid line gives the asymmetry value when a 2 0 and dotted line gives the value when a 2 < 0 (same colour gives the same magnitude of a 2 ). (E) (H): Same as (A) (D) except that we now set b 2 = b 2 (L) = b 20 (5+ 5P 2 (sin L)) for latitude L. Here P 2 denotes the Legendre polynomial of order 2. Grey line gives the horizontal zero line. in (2), h p = h p (L 1, L 2, l 1 l 2 ) = h/ x p and h pq = h pq (L 1, L 2, l 1 l 2 ) = 2 h/ x p x q, where x 1 = L 1, x 2 = L 2 and x 3 = l 1 l 2 (see appendix A of Jun & Stein (2007) for explicit expressions for h p and h pq ). Also let M ν (x) = x ν K ν (x). Then the cross-covariance function of Z 1 and Z 2 is given by, cov{z 1 (L 1, l 1 ), Z 2 (L 2, l 2 )} = Γ 1 M ν 2 ( h) + Γ 2 M ν 1 ( h), (10) where Γ 1 and Γ 2 are Γ 1 = 1 4 {A 1,1(L 1 )A 2,1 (L 2 )h 1 h 2 B 1,1 (L 1 )B 2,1 (L 2 )h 2 3 A 1, 1 (L 1 )B 2,1 (L 2 )h 1 h 3 and + B 1,1 (L 1 )A 2,1 (L 2 )h 2 h 3 }, Γ 2 = 1 2 {A 1,1(L 1 )A 2,1 (L 2 )h 12 B 1,1 (L 1 )B 2,1 (L 2 )h 33 A 1, 1 (L 1 )B 2,1 (L 2 )h 13 + B 1,1 (L 1 )A 2,1 (L 2 )h 23 }. The cross product terms of A 1,k or B 1,k and A 2,k or B 2,k in Γ 1 and Γ 2 come from the covariance of processes with / L and / l applied in (8) and through the linear combination terms for A i,k and B i,k as in (9), the resulting cross-covariance model in (10) achieves great flexibility and can capture complex non-stationary structure in the data. In particular, it can be easily shown that for any latitude L, longitude l and longitudinal lag Δ, cov{z 1 (L, l), Z 2 (L, l + Δ)} cov{z 1 (L, l + Δ), Z 2 (L, l)} ( Δ ) ( Δ ) ={B 1,1 (L)A 2,1 (L) A 1,1 (L)B 2,1 (L)} 4β 2 R 2 sin L cos L sin cos 2 2 { 1 2 M ν 2( ( h)4β 2 R 2 cos 2 L sin 2 Δ ) M ν 1 ( } h), (11) 2 where h = 4β 2 R 2 cos 2 L sin 2 (Δ/2). Therefore, by letting A i,k and B i,k functions depend on the latitude, L, the proposed covariance model can produce flexible longitudinal irreversibility. Equation (11) can also be used to prove the fact that the asymmetries are zero when L 2 = 0in Fig. 5 (note that when L 1 = L 2, the asymmetry reduces to the longitudinal irreversibility).

11 736 M. Jun Scand J Statist 38 In fact, under the current covariance model, longitudinal irreversibility at the equator is always zero due to the term sin L in (11). Figure 5 (E) (F) display the asymmetry when A 1,1 (L) = 1, A 2,1 (L) = a 2, B 1,1 (L) = b 1, B 2,1 (L) = b 2 (L) = b 20 (5+ 5P 2 (sin L)). These plots demonstrate that by allowing the coefficients A i,k and B i,k to depend on the latitude, we get more flexibility in the resulting covariance structure. It is shown in Jun & Stein (2008) that the marginal correlation from the model in (8) (with C i = 0) can be as small as 1. For crosscorrelations, we also achieve the range of 1 to 1. For an extreme example, in (8), suppose n = 1, A 1,1 = A 2,1 = 1, B 1,1 = B 2,1 = 0 and C 1 = C 2 = 0. Then it is easy to see that the crosscorrelation between Z 1 and Z 2 is 1 everywhere. The proposed method here has a similar spirit as the LMC model in the sense that each process Z i is modelled as a linear combination of latent processes. However, the model proposed in this article has several significantly different aspects compared to those of LMC. In the model, cross-covariance structure is characterized by the first term of (8) and even when n = 1 with C i = 0, we do achieve fairly flexible cross-covariance models that we cannot with the LMC models with more covariance parameters (see section 4.3). The expression in the summation of (8) may appear to be a linear combination of latent processes, but in fact the differential operators are defined in the L 2 sense. The variations of LMC models that give non-stationary covariance models such as in Gelfand et al. (2004) achieve the nonstationarity quite differently from the way the models in (8) achieve it. One of the fundamental differences between the approach proposed in this article and the LMC model is that the differential operators with respect to latitude and longitude are applied to the same process (Y k in (8)). Suppose we have n = 1 in (8). In the LMC models, they consider linear combinations of independent processes, but the differential operators in (8) effectively evaluate covariances of differences of the same process, Y 1, with small latitudinal or longitudinal lags (see (3)). Hence, the non-stationarity with respect to latitude not only come from the coefficients of partial differential operators, A i,k and B i,k, but also from the differential operators, / L and / l, and the covariance between the processes with each differential operators applied. We choose to use Legendre polynomials in modelling the coefficients of the differential operators. It is not clear how we could get empirical estimates of these coefficients from the data and thus we instead model these coefficients through some orthogonal polynomials of the latitude. Legendre polynomials in that sense are natural choice since they are orthogonal over the interval [ 1, 1] and thus P j (sin L)s for 90 L 90 are orthogonal. There are possible limitations of the model in (8). The first limitation is that each Z i smay have the same spatial range and smoothness parameter since Z i s consist of the same processes (Y k s). One easy fix of this problem is either by letting Y 0 have different covariance parameter than Y k (k > 0) s or by adding more terms (processes) in (8) and let these terms have different covariance parameters. We will explore this issue further for the climatological application in section Computational issues It is common to estimate the covariance parameters as well as the mean parameters using maximum likelihood estimation and for that purpose, we from now on assume that the process is multivariate Gaussian. Many spatial data sets these days are of large dimension and often it can be quite challenging to efficiently compute the full likelihood. For the case of regularly spaced data, which is usually the case for satellite data and the numerical model outputs, however, the computation of the exact likelihood can be quite efficient. Jun & Stein (2008) demonstrate such a method using the Discrete Fourier transform (DFT) for univariate spatial process. The key idea is the following: since the covariance model is axially symmetric

12 Scand J Statist 38 Non-stationary cross-covariance models 737 and we have regularly spaced longitude values covering full range, the resulting covariance matrix can be written in a block circulant form. Then using the fact that a block circulant matrix can be diagonalized by applying the DFT, we can calculate the inverse and the determinant of the covariance matrix efficiently (see Jun & Stein (2008) for more details on how this works). The same idea can be applied for the cross-covariance matrix. As long as the multiple spatial processes are on the same longitudinal grids, cover the full longitude range, and the cross-covariance structure is axially symmetric (note the model in (8) does give an axially symmetric cross-covariance structure), both the marginal covariance matrix for each process and the cross-covariance matrix can be diagonalized by applying the DFT. Note Chan & Wood (1999) consider a multivariate stationary Gaussian random field defined on a rectangular grid in R d and they apply circulant embedding of a block Toeplitz matrix (Toeplitz structure comes from the stationarity of the random field) to create a block circulant covariance matrix. Then they perform the DFT to block diagonalize the covariance matrix. Since in our domain, the block circulant structure of the covariance matrix is naturally given through regularly spaced longitudinal points with 360 coverage, we do not need the step of circulant embedding. Suppose we consider a bivariate process (Z 1, Z 2 ) observed on a regular grid with p latitude points and q longitude points (longitudinal points must be equally spaced over the full longitude range). We denote Z i (L j ) = {Z i (L j, l 1 ),..., Z i (L j, l q )} T for j = 1,..., p and FZ i (L j )is the DFT (with respect to longitude) of Z i (L j ). Then it is well known that the corresponding covariance matrix of the complex normal vector, FZ i (L j ), is a diagonal matrix. Although FZ i (L) is a complex normal random variable, the likelihood of it can be obtained simply by calculating as if it is a real normal random variable with the appropriate covariance matrix (Wooding, 1956). Therefore, if we denote Z * i = {Z* i,1,1,..., Z * i,p,1, Z * i,1,2,..., Z * i,p,2,..., Z * i,p,q} T where Z i,j,k * is the kth element of FZ i(l j ), then the covariance matrix of {Z * 1T, Z * T 2 } T can be written as ( ) D1 D Σ = 12 D *, 12 D 2 where D 1, D 2 and D 12 are complex block diagonal matrices with p p block diagonals and D * 12 is the conjugate transpose of D 12. The determinant of the matrix Σ can be calculated using det(σ) = det(d 1 D 12 D 1 2 D* 12 ) det(d 2) and the quadratic form in the likelihood can be efficiently calculated using the fact that ( (D Σ 1 1 D 12 D 1 2 = D* 12 ) 1 D 1 1 D 12(D 2 D * 12 D 1 1 D ) 12) 1 D 1 2 D* 12 (D 1 D 12 D 1 2 D* 12 ) 1 (D 2 D * 12 D 1 2 D. 12) 1 Note that the lower off diagonal matrix is the conjugate transpose of the upper off diagonal matrix and the inverses of D 1 and D 2 can be calculated efficiently since they are block diagonal matrices with block size p p. In our application, we have p = Application 4.1. Data As noted in section 1, the relationship between precipitation and surface temperature has received a lot of attention by scientists and it is important in the climate impact research area. We apply our covariance functions developed here to build a joint model between surface temperature and precipitation; the data originates from one of the numerical model outputs

13 738 M. Jun Scand J Statist 38 used in Trenberth & Shea (2005), the NCAR CCSM3. The NCAR CCSM3 is one of the climate models developed by NCAR. Jun et al. (2008) give a more detailed background on this and other climate models. We look at the 5 months average for Northern winter (November to March) and we take averages of these over (we call it NDJFM from now on). The temperature output from this model, CCSM3, has also been analysed by Jun et al. (2008) but they consider the differences between the observations and numerical model outputs and they only look at the latitude range of 50 S 50 N on a coarser grid resolution (of 5 5 ). We use the numerical model output only (no observations) for the entire globe (full longitude and latitude ranges) in the original resolution of ( in both longitude and latitude). It is common in climate studies to use numerical model outputs rather than observations since observations usually have a large fraction of missing observations (especially near the poles). For example, Trenberth & Shea (2005) used numerical model outputs only, to study the relationship between the temperature and precipitation. Note the unit for temperature is K and the unit for precipitation is Kg/(m 2 s) (Kilogram per squared meter per second). Tebaldi & Sansó (2009) deal with multiple numerical model outputs along with observations to build a joint model between surface temperature and precipitation, but their approach is relatively simple in terms of modelling the cross-covariance structure of the two variables. In their approach, cross-correlations between the two variables only come from the mean of the processes, that is, they let the mean of the precipitation be a linear function of the surface temperature. They consider spatio-temporal processes, but in this work, we focus on the spatial component of the process Model Model for the mean. The first row of Fig. 6 gives the NDJFM average of temperature and precipitation data. Since the order of precipitation data is 10 5, from now on, we multiply 10 5 to the precipitation data to make it comparable to the surface temperature data. For temperature, it is clear that the mean structure of the field mainly depends on the latitude. For precipitation, such dependence is not as strong as temperature data and there are places with large amount of precipitation around the equator. We first filter out the spatial mean structure using spherical harmonics and work with the residuals. Specifically we use spherical harmonics up to order r = 12 and regress each variable (surface temperature and precipitation) on the spherical harmonics, {Yr s (sin L, l) r = 0, 1, 2,..., s = r,..., r} for r = 12, separately. The second row of Fig. 6 gives the estimated mean structure and the third row gives the residuals. Overall, the estimated mean field removes most of the large-scale spatial patterns in the data. Model for the covariance. Since our main interest in this article is estimating covariance structure, we focus on fitting the covariance models using the residuals. We fit several covariance models to the data. We consider a Matérn model in Gneiting et al. (2010), a version of the LMC model, and a couple of variations of our covariance model developed in section 3.1. Here, Z 1 denotes the surface temperature process and Z 2 denotes the precipitation process. Note that these processes are the residuals after filtering out the mean as explained in section (i) Matérn model (MAT): we use the parsimonious bivariate Matérn model in Gneiting et al. (2010). In particular, we let Z i = G(α i, β, ν i ), i = 1, 2. The parameter ν 3 gives the smoothness for the cross-covariance and by construction, ν 3 = (ν 1 + ν 2 )/2. We also have the co-located correlation coefficient ρ. (ii) LMC model (LMC): we use a version of the LMC model, that is, we set Z i (L, l) = a i W 1 (L, l) + b i W 2 (L, l) + c i U i (L, l), where a i, b i and c i are constants, W j = G(1, β, ν j ) ( j = 1, 2) and U i = G(1, β, ω i ). We also assume W j s are independent, U i s are independent,

14 Scand J Statist 38 Non-stationary cross-covariance models 739 Temperature (NDJFM) Precipitation (NDJFM) Mean Mean Residual Residual Fig. 6. Surface temperature and precipitation data used in the analysis. The first row gives original NDJFM averages, the second row gives the estimated means, and the third row gives the residuals. The unit for temperature is K and the unit for precipitation is Kg/(m 2 s). Note that we multiplied 10 5 to the original data for the precipitation to make the scale comparable to that of temperature. (iii) and W j s and U i s are independent of each other. Therefore U i does not contribute to the cross-covariance structure of the Z i s. Our covariance model (Non-stationary Multivariate Global model): (a) NMG1: we set { } Z i (L, l) = a i L + b i Y (L, l) + c i U i (L, l). l

15 740 M. Jun Scand J Statist 38 Here, Y = G(1, β, ν), U i = G(1, β, ω i ), and we assume the U i s are independent of Y. Note that ν > 1. All of the above models have the property that the processes Z 1 and Z 2 have the same spatial range parameter. Each of the model s properties discussed in section 2 are summarized in Table 1. These models are intentionally set to be relatively simple since they will be used in section 4.3 with the data over a subregion. We use the following more complex model to fit the full data in section 4.4. (b) NMG2: we let { Z i (L, l) = A i (L) L + B i(l) } Y (L, l) + C i (L)W (L, l) + d i U i (L, l) l with A i, B i and C i being defined as in (9) (for instance, A i (L) = m j = 0 a ijp j (sin L)). We let Y = G(1, β 1, ν 1 )(ν 1 > 1), W = G(1, β 2, ν 2 ) and U 1 = G(1, β 3, ν 3 ) (note ν 2, ν 3 > 0). For the choices of m and d i, see section 4.4. For the models LMC, NMG1 and NMG2, we may have an identifiability problem if we let all the parameters of the coefficients, a i, b i, c i, and the coefficients of the linear combinations in A i, B i and C i vary in R. To avoid the problem, for LMC model, we take the signs of a 1, b 1, c 1 and c 2 positive. For NMG1 model, we take the signs of a 1, c 1 and c 2 positive. For NMG2 model, we take the sign of b 10, c 20 and d 1 positive Fit over subdomains Even if we use the computational technique through DFT described in section 3.2, using the full data set for both variables to estimate the covariance structure takes quite some time (the total data size is = 65,536). Therefore, we first choose a subregion over some parts of North America and fit several covariance models for a quick comparison in terms of likelihood and prediction accuracy. We perform the prediction on a region disjoint with the estimation sites, over North America. Figure 7 shows the locations of estimation sites and the prediction sites. Note that there are 774 estimation sites and 172 prediction sites. For the fit in this section, since we have a manageable size of the data, we do not use the technique using DFT. In fact it is not possible since the technique through DFT requires that the data should cover the entire longitude range. We compare the covariance models listed in section (1, 2 and 3(a)) and estimate each covariance parameter using the maximum likelihood estimation method. We used numerical optimization (using nlm and optim functions in R and for optim, we use the default Nelder Mead algorithm) and tried several starting points. The optimization procedures reached to the same maximum point for all of the different starting points that were tried. Table 2 gives the estimated covariance parameter values along with their asymptotic standard errors. Asymptotic standard errors are obtained from the inverse of the Hessian matrix. The maximized loglikelihood values for each model and the corresponding AIC values are also Table 1. Some covariance properties for models fitted in section 4.3 MAT LMC NMG1 Isotropic Y Y N Covariance independent on latitude Y Y N Longitudinally reversible Y Y N Symmetric Y Y N Same smoothness N N N

16 Scand J Statist 38 Non-stationary cross-covariance models 741 Estimation Prediction Fig. 7. Locations of estimation sites and prediction sites for the fit over North America in section 4.3. Table 2. MLE estimates along with their asymptotic standard errors for some covariance parameters from the fit over North America in section 4.3. The unit for spatial range parameter (β) is Km. The estimate for the co-located correlation parameter for MAT is ˆρ = 5.5e 06 MAT LMC NMG1 β (14.73) (12.30) (11.94) ν 2.07 (0.064) ν (0.058) 0.93 (0.078) ν (0.073) 2.27 (0.13) ω ( ) 1.07 (0.045) ω (0.20) 2.43 (0.14) a (0.11) 0.79 (6.05e-03) 6.47e-04 (3.41e-05) a (0.11) (3.13e-03) (2.01e-03) b (7.46e-05) (2.06e-03) b (0.11) (2.21e-03) c (0.052) 1.57 (0.11) c (0.36) 0.17 (0.027) Loglik AIC given. First thing to note is that the NMG1 model gives significantly larger loglikelihood values compared to the other models given comparable number of covariance parameters (the LMC model has the most covariance parameters). The AIC value for the NMG1 model is the smallest among the three. The fact that the LMC model, despite having the most covariance parameters, gives a much smaller loglikelihood value than NMG1 may be a sign that the non-stationarity, in particular, the dependence of covariance structure on latitudes, longitudinal irreversibility, and the asymmetry in the data (for marginal and/or crosscovariance structure) are rather strong and the differential operator term in (8) helps to explain these properties better. In terms of different smoothness in the two variables, it seems that the precipitation process is smoother than the temperature process. From the LMC model, the smallest smoothness parameter value, ν 1 is shared by the two processes, temperature and precipitation but the

17 742 M. Jun Scand J Statist 38 coefficient, a 1, is much larger in magnitude than the coefficient, a 2, and also b 2 is larger in magnitude than b 1. Therefore, the roughest process, W 1, mostly contributes to the temperature process and smoother process, W 2, mostly contributes to the precipitation process. From the NMG1 model, first of all, w 1 is smaller than w 2. Note that the effective smoothness of the process Y is ν 1 = 1.07, and thus Y has similar amount of smoothness to the process U 1. In that sense it is not clear whether precipitation process is smoother than the temperature process or not. Nevertheless the result in Tebaldi & Sansó (2009) shows that the precipitation process is smoother than the temperature process. For the LMC model, the estimate of ω 1 reached near the upper boundary of the parameter space and thus we could not obtain its asymptotic standard errors. It is a common practice to set the range for the smoothness parameter to be (0, 2.5) since the covariance model is not valid for zero or negative values and for large values we often run into the numerical instability problem. This poor fit may imply that the data do not provide enough information on this parameter for the particular model of LMC. The model NMG1 does not have this problem. The estimate for the co-located correlation parameter of the MAT model is almost zero ( ˆρ = 5.5e-06), while the empirical cross-covariance estimate is around 0.3. Figure 8 shows the prediction errors for the covariance models MAT, LMC and NMG1 at the prediction sites. We display the difference between the true and the predicted values at the prediction sites against latitude. Note that the three models do not show much difference in terms of prediction accuracy for the temperature variable, although we see significant difference for the precipitation. The superiority of the NMG1 model is apparent for the prediction of precipitation. For the precipitation variable, the predictive skills of the MAT and LMC models are similar. To make a fair comparison of the predictive performances of the three models, we now repeat the above procedure over 24 disjoint subdomains, S 1,..., S 24, that cover most of globe altogether. That is, S 2n is a subdomain in the Northern Hemisphere with latitude range 0 60 N and S 2n 1 is a subdomain in the Southern Hemisphere with latitude range 0 60 Sforn = 1,..., 12. For each n, S 2n and S 2n 1 cover the longitude range 30(n 1) to 30n and for S 2n and S 2n 1, we set aside the data in the longitude range {30(n 1) + 10} to {30(n 1) + 15} for the validation of prediction. Table 3 shows the maximum loglikelihood values for the three models from the fits over the 24 subdomains. Except S 12, S 16 and S 24, NMG1 achieves the largest maximum loglikelihood values and the differences between the loglikelihood values of NMG1 and the other two models are significantly large in most of the subdomains. Table 4 gives the summary of prediction performance of the three models over the 24 subdomains. It provides the median, mean and maximum values of Mean Squared Errors (MSEs) from the prediction over the 24 subdomains. Except the mean for temperature and maximum for the precipitation, NMG1 gives the smallest MSE values for all the summary statistics. Along with the result in Fig. 8, this result demonstrates that NMG1 model indeed outperforms the other two models not only in terms of the maximum loglikelihood values (and AIC) but also the predictive performance Fit over full domain We now fit the full data set through the computational technique described in section 3.2. From the fitted results in section 4.3, it is clear that there is a strong non-stationarity in the data and the covariance models listed in section (except NMG2) are not flexible enough to capture such non-stationarity. On the other hand, from Tables 2 4 and Fig. 8, it is clear that the NMG1 model outperforms the MAT and LMC models given comparable number of covariance parameters. Hence, we fit the full data to estimate cross-correlation between the