Effect of correlated observation error on parameters, predictions, and uncertainty

Transcription

1 WATER RESOURCES RESEARCH, VOL. 49, , doi: /wrcr.20499, 2013 Effect of correlated observation error on parameters, predictions, and uncertainty Claire R. Tiedeman 1 and Christopher T. Green 1 Received 27 December 2012; revised 14 August 2013; accepted 23 August 2013; published 7 October [1] Correlations among observation errors are typically omitted when calculating observation weights for model calibration by inverse methods. We explore the effects of omitting these correlations on estimates of parameters, predictions, and uncertainties. First, we develop a new analytical expression for the difference in parameter variance estimated with and without error correlations for a simple one-parameter two-observation inverse model. Results indicate that omitting error correlations from both the weight matrix and the variance calculation can either increase or decrease the parameter variance, depending on the values of error correlation () and the ratio of dimensionless scaled sensitivities (r dss ). For small, the difference in variance is always small, but for large, the difference varies widely depending on the sign and magnitude of r dss. Next, we consider a groundwater reactive transport model of denitrification with four parameters and correlated geochemical observation errors that are computed by an error-propagation approach that is new for hydrogeologic studies. We compare parameter estimates, predictions, and uncertainties obtained with and without the error correlations. Omitting the correlations modestly to substantially changes parameter estimates, and causes both increases and decreases of parameter variances, consistent with the analytical expression. Differences in predictions for the models calibrated with and without error correlations can be greater than parameter differences when both are considered relative to their respective confidence intervals. These results indicate that including observation error correlations in weighting for nonlinear regression can have important effects on parameter estimates, predictions, and their respective uncertainties. Citation: Tiedeman, C. R., and C. T. Green (2013), Effect of correlated observation error on parameters, predictions, and uncertainty, Water Resour. Res., 49, , doi: /wrcr Introduction [2] A diagonal weight matrix commonly is used to represent system-state observation errors in inverse models of groundwater systems, with the weights calculated as the inverse of observation error variances [e.g., Hill and Tiedeman, 2007; Singh et al., 2008; James et al., 2009; Liu and Kitanidis, 2011; Majdalani and Ackerer, 2011; Kowalski et al., 2012; Yoon and McKenna, 2012]. This representation and other common simplifications assume there are no observation error correlations. The necessary methods for including error correlations in inverse modeling have been available for decades [e.g., Neuman and Yakowitz, 1979; Cooley, 1982; Carrera and Neuman, 1986; McLaughlin and Townley, 1996; Hill and Tiedeman, 2007] and can be Additional supporting information may be found in the online version of this article. 1 Water Resources Discipline, U.S. Geological Survey, Menlo Park, California, USA. Corresponding author: C. R. Tiedeman, Water Resources Discipline, U.S. Geological Survey, 345 Middlefield Rd., Menlo Park, CA 94025, USA. (tiedeman@usgs.gov) American Geophysical Union. All Rights Reserved /13/ /wrcr implemented, for example, using the inverse modeling software PEST [Doherty, 2008, 2010] and UCODE_2005 [Poeter et al., 2005]. The methods include the correlations by allowing for a full observation variance-covariance matrix to represent observation errors; this matrix is inverted to obtain the full weight matrix. However, despite availability of these methods, difficulties with quantifying the terms that characterize observation error correlations often lead to their omission and to using a diagonal observation weight matrix for convenience. [3] Various types of error correlations are widespread in hydrologic models. Error correlations can result from phenomena such as barometric pumping of wells [Weeks, 1979] and entrapped air in the unsaturated zone [Healy and Cook, 2002] that create spatially and temporally correlated anomalies in groundwater levels. Correlations in errors arise also from use of multiple observations that derive from a single direct measurement, a common situation in hydrologic studies. For example, the water table elevation at a monitoring well is usually calculated from measurements of well elevation and depth of water, so error in the well elevation propagates to all head observations over time at that well. Streamflow observations are usually estimated as nonlinear functions of water depth and empirical rating curve constants. For both stream and groundwater depth, estimates often depend on the nonlinear equations 6339

2 and empirical constants used to estimate pressure from the voltage and temperature at a pressure transducer [Freeman et al., 2004]. Additional examples include multiple observations of temporal changes in hydraulic heads that depend on an instantaneous head measurement [e.g., Hill et al., 2000] and multiple observations of flow-change between stream gauging stations that depend on a single flow estimate. Although error correlations in hydrologic models have not been extensively characterized, these and other examples indicate that error correlations are potentially widespread. [4] In this work, we explore the effect of system-state observation error correlations on parameter estimates, predictions, and uncertainty measures. We use both an analytical expression and a groundwater reactive transport model of denitrification to compare results obtained with and without error correlations. For the transport model, geochemical observation errors are correlated because selected direct measurements are used to calculate more than one calibration observation. The correlations are calculated by propagation of measurement error, a method that has precedence for geochemical data [e.g., Ballentine and Hall, 1999; Aeschbach-Hertig et al., 1999, 2000; Peeters et al., 2002] but, to our knowledge, has not been used previously to calculate error correlations in observation weight matrices for hydrogeologic investigations. Hill [1992] and Christensen et al. [1998] use a less general approach applied to streamflow gains and losses. In the context of inverse modeling, full weight matrices can be used to represent observation, model, and parameter error. [5] Few groundwater studies have considered correlated observation errors. Christensen et al. [1998] used a full weight matrix to represent correlated error in base flows, by expanding the method proposed by Hill [1992] to derive the error covariance terms for a system of branching streams. Christensen et al. [1998] did not compare results using a full and a diagonal weight matrix, but Foglia et al. [2009] reported that an unpublished follow-up comparison found that base flow error correlations had a small effect on the parameter estimates but a larger effect on parameter uncertainty. [6] Cooley [2004] accounted for both observation and model errors in the weight matrix, by summing the variance-covariance matrices of the two error types. Model error was formulated as the difference between stochastic representations of the true and the spatially averaged parameter distributions. Cooley and Christensen [2006] and Christensen and Doherty [2008] used synthetic models to examine the consequences of using a diagonal instead of a correct full weight matrix calculated by the method of Cooley [2004]. Their weight matrices were dominated by model error. Cooley and Christensen [2006] reported that the variance of head prediction residuals was larger in a model calibrated with a full weight matrix compared to the same model calibrated with a diagonal weight matrix. They also found that confidence intervals on predictions calculated with a diagonal weight matrix were much too small whereas those calculated with a full weight matrix were nearly correct. Their results underscore the importance of using the correct full weight matrix when observation and model error correlations exist. Christensen and Doherty [2008] found that inversion with a full weight matrix produced less accurate predictions, in contrast to expected results based on Cooley [2004]. They noted that their result was most likely related to difficulties with generating (by a Monte Carlo method) and inverting the variancecovariance matrix of total error. Lu et al. [2013] considered correlations between total errors in the context of weighting for model averaging. They found that using the variancecovariance matrix of total errors to calculate the weights resulted in better predictive performance for models of both synthetic and experimental uranium transport, compared to using the variance-covariance matrix of observation errors. [7] A full weight matrix commonly is used for parameter errors. For example, when pilot points and regularization are used to estimate the spatial variability of a parameter field, a full variance-covariance matrix for prior information error often is used to represent the spatial correlation of these errors [e.g., Bentley, 1997; Alcolea et al., 2006; Singh et al., 2008; Hendricks-Franssen et al., 2009]. To our knowledge, there are no studies that evaluate the effect of including versus excluding these correlated errors. [8] The implications of observation error correlations extend also to Bayesian methods in hydrologic modeling. In rainfall-runoff modeling, streamflows are the primary system-state observations used for calibration. Observation errors largely stem from using stage-discharge rating curves to determine the flows [e.g., McMillan et al., 2010] and these errors can be correlated as suggested by Foglia et al. [2009]. Recent rainfall-runoff modeling research has comprehensively examined methods for characterizing both model and observation error and its effect on prediction uncertainty [e.g., Thyer et al., 2009; Schoups and Vrugt, 2010; Renard et al., 2010, 2011]. In these papers, error models are developed for the different error sources, and the parameters of these models are estimated together with rainfall-runoff model parameters. The results show that error correlations can be pronounced and that accurate estimates of observation uncertainty are critical for predictive capabilities and for the decomposition of input and structural errors [Thyer et al., 2009; Renard et al., 2010]. [9] In this paper, we first present methods for model calibration, error propagation, and calculation of uncertainty. We next consider parameter uncertainty in a simple oneparameter, two-observation inverse model, and derive a new analytical expression for the errors in uncertainty estimates when observation error correlations are omitted. For this model, we also compare uncertainty estimates that are typically obtained in practice with those derived from theoretical calculation of the variances. A reactive transport model of denitrification is then introduced, calibrated using both full and diagonal weight matrices, and used to predict future nitrate concentrations and uncertainty. Model error is accounted for by considering multiple realizations of the geology. The derived analytical expression helps to explain the differences in reactive transport model parameter uncertainty in the calibrations with and without error correlations and provides some general guidance about the importance of using a full weight matrix in inverse models for which observation errors are correlated. Results are expected to have broad relevance, as most of the methods and analyses presented here for exploring the effects of correlated errors apply regardless of the source of these errors. 6340

3 2. Methods 2.1. Nonlinear Regression and Weighting [10] In this work, parameters are estimated by weighted least squares nonlinear regression using the Gauss- Marquardt-Levenberg method implemented in PEST [Doherty, 2008]. This method minimizes the following objective function, which is a measure of model fit to the observations [Hill and Tiedeman 2007, p. 28]: S ¼ e T xe ¼ ½y y 0 ðþ b Š T x½y y 0 ðþ b Š ð1þ where e ¼ y y 0 (b) is the residual vector of length nd, y is a vector of nd observed values, y 0 (b) is a vector of nd simulated equivalents, b is a vector of np parameter values, and x is the nd nd observation weight matrix that can be diagonal or full. Individual residuals are e i ¼ y i y 0 iðþ. b [11] If the observation weight matrix x is defined as being proportional to the inverse of the observation error variance-covariance matrix, then the parameters estimated by weighted linear regression will have the smallest possible variance [Hill and Tiedeman, 2007, p. 34]. Cooley [2004, pp ] showed that this statement also is valid for nonlinear regression. When observation errors are correlated, a full weight matrix x full is needed: x full ¼ V full ðþ 1 where V full () is the full variance-covariance matrix of the observation errors, containing off-diagonal elements that quantify the correlations. In this paper, the term full denotes a matrix containing at least one nonzero offdiagonal term. When observation errors are uncorrelated or when correlations are omitted, a diagonal weight matrix x diag is used: x diag ¼ V diag ðþ 1 where V diag () is the diagonal observation variancecovariance matrix, with the diagonal elements equal to: ð2þ ð3þ! ii ¼ 1= 2 y i ; i ¼ 1; nd ð4þ where! ii is the weight for observation y i and 2 y i is the observation error variance. Hill and Tiedeman [2007, Appendix A.3] discuss the assumptions required for diagonal weighting to be correct Calculating Observation Error Variance- Covariance Matrix by First-Order Error Propagation [12] When calibration observations are derived from multiple direct measurements, the observation error variances and covariances can be estimated by propagating the measurement error (Figure 1) [e.g., Sherman, 1989; Tellinghuisen, 2001; Feldman et al., 2008]. As discussed in section 1, derived observations commonly are used to calibrate hydrologic models. Issues of measurement error propagation are, therefore, widely applicable to hydrologic models and are likely to become increasingly important as geochemical data and estimates of groundwater ages are used more frequently to calibrate models. [13] Let observations y i and y k be expressed as linear or nonlinear functions of nm independent measurements, u 1... u nm : y i ¼ f i ðu 1...u nm Þ y k ¼ f k ðu 1...u nm Þ The first-order, second-moment error-propagation equations for calculating the error variance of observation y i and the error covariance of observations y i and y k, derived using a first-order Taylor series expansion, are: 2 y i ¼ Xnm ¼1 yiy k ¼ Xnm 2 i @f 2 u [Meyer, 1992, pp. 40 and 45] where 2 u is the error variance of measurement u l, 2 y i is the error variance of observation y i, and yi y k is the error covariance for observations y i and y k. Equation (6) is used to calculate the diagonal elements of both V diag () and V full (). Equation (7) is used to calculate the off-diagonal elements of V full () and is ð5þ ð6þ ð7þ Figure 1. Relationships between input measurements and output observations for this study, and in general terms. 6341

4 nonzero when the same measurement u l is used to calculate more than one observation. The equations are evaluated at the expected values of the measurements used to estimate a specific observation. Multiple measurements of the same type can be averaged to provide a single expected value, depending on the type of observation (e.g., time averaged or temporally varying). Additional terms containing measurement covariances are present in both equations if the errors of the measured quantities u 1... u nm are correlated [Meyer, 1992, pp. 40 and 45]. Compared to these additional terms, the measurement error variance terms shown in equations (6) and (7) tend to dominate the uncertainty calculated by the error-propagation equations [Bevington and Robinson, 1992, p. 43]. Investigation of measurement error correlations is a topic for possible future research. [14] Observation error correlations are often more intuitively understandable than the covariances in equation (7), and are calculated using elements of V full (): ik ¼ y iy k yi yk Correlations can range from 1.0 to 1.0, with larger absolute values indicating a greater degree of correlation Parameter and Prediction Uncertainty [15] In this study, we compare two scenarios to obtain insight into the effects on parameter and prediction uncertainty of omitting observation error correlations. In the first scenario, the error correlations are known, have been correctly quantified, and are included in the observation weighting (i.e., a full matrix is used) and in the calculation of parameter uncertainty. In the second scenario, the error correlations are omitted in both the weighting (i.e., a diagonal matrix is used) and the calculation of uncertainty, as is common in practice. For both these scenarios, we estimate parameter uncertainty using the parameter variancecovariance matrix V(b) calculated as: ð8þ Vb ðþ¼s 2 X T 1 xx ð9aþ where s 2 ¼ S/(nd np) is the calculated error variance of the regression and X is a nd np matrix of observation 0 iðþ=@b b j. For nonlinear models, X generally differs for different sets of parameter values. Estimated parameter variances 2 b j are the diagonal terms of V(b). [16] In the first scenario of this study, with the full weight matrix calculated as the inverse of the full observation error variance-covariance matrix, equation (9a) is the theoretically correct expression for V(b). It is produced by solving for the variance-covariance matrix of the parameter vector and defining the observation weight matrix as the inverse of the error variance-covariance matrix [Hill and Tiedeman, 2007, pp ]. In the second scenario (and in common practice), we use equation (9a) under the assumption that the error correlations are not known, or are assumed to be negligible, and thus are absent from both x and V(). That is, a diagonal weight matrix is assumed to arise from a diagonal V(), as in equation (3). Our approach presumes that if the error correlations have been quantified, resulting in a known full V(), then a full weight matrix will be used that accounts for them, rather than a diagonal weight matrix that does not. [17] When error correlations exist but are omitted from the weight matrix for the reasons given above, equation (9a) is not the theoretically correct expression for the variance-covariance matrix. The correct expression is obtained by using a diagonal weight matrix but a full V() [Cooley, 2004; Cooley and Christensen, 2006]. Under that condition, the derivation of V(b) for a diagonal weight matrix does not simplify to equation (9a), instead yielding a more complex expression that includes V(): VðÞ¼s b 2 X T 1X xx T xvðþxx X T 1 xx ð9bþ [Hill and Tiedeman, 2007, equation (C.20)]. This equation is generally not usable in practice when the error covariances are not known, and therefore it is not a primary focus of this study. However, it serves as a useful point of comparison with equation (9a) to evaluate differences between theoretical and practical estimates of parameter variances when the weight matrix is diagonal. [18] Other measures of parameter uncertainty used here are confidence intervals and coefficients of variation. The individual, linear, 95% confidence interval for parameter b j is calculated as [Hill and Tiedeman, 2007, p. 138]: b j 6tnd ð np; 0:025Þ bj ð10þ where t(nd np, 0.025) is the Student t-statistic for nd np degrees of freedom and a significance level of 0.95 and 2 b j is calculated with equation (9a). Linear confidence intervals are used in this study as an efficient basis for comparing the different estimates of parameters and their uncertainties, and assume that observation errors are normally distributed. The coefficient of variation for b j is: cv j ¼ bj =jb j j ð11þ where 2 b j is calculated with equation (9a). [19] Prediction uncertainty is calculated by a first-order second-moment method [Hill and Tiedeman, 2007, p. 159]: 2 z ¼ x z VðÞx b T z ð12þ where 2 z is the variance of prediction z and x z is the vector of =@b j. Individual, linear 95% confidence intervals are used to express prediction uncertainty, and are calculated in an analogous manner to equation (10). In this study, all calculations with equation (12) use equation (9a) to calculate V(b) Effect of Observation Error Correlation on Uncertainty for a Simple Inverse Model Effect on Parameter Variance [20] To derive a general expression for the effects of observation error correlations on parameter uncertainty, equation (9a) is used with the assumption that the model is linear with respect to the parameters. The parameter variance-covariance matrix in equation (9a) is a function of 6342

5 model fit s 2, sensitivities X, and the weighting x. When computed for models calibrated with x full instead of x diag, both x and s 2 differ in the two calculations. For a linear model, the sensitivities are independent of parameter values and thus are the same in the two calibrations. Under these conditions, the variance-covariance matrices calculated with the full and diagonal weight matrices are: [23] To obtain insight into the effect of observation error correlations on 2 b 1 ;diag =2 b 1 ;full, we expand equation (16) for a simple linear inverse model with one parameter b 1 and two observations y 1 and y 2. This yields: b 1;diag =2 b ¼ y 1 2 y 2 X y 2 þ X y 1 2X 11 X 21 y1y 2 1;full 2 V full ðþ¼s b 2 full X T 1 x full X ð13þ 2 y 1 2 y 2 2 y 1y 2 X y 2 þ X y 1 ð17aþ V diag ðþ¼s b 2 diag X T 1 x diag X ð14þ where s 2 full and s 2 diag are the regression error variances for the models calibrated with x full and x diag, respectively. Use of these equations is consistent with our approach for calculating parameter uncertainty discussed in section 2.3. The diagonals of V full ðþare b the variances calculated with observation error correlations, 2 b j ;full, and the diagonals of V diag ðþare b estimates calculated without error correlations, 2 b j ;diag. The ratio 2 b j ;diag =2 b j ;full is a metric of the change in parameter variance when a diagonal instead of a full weight matrix is used; that is, when error correlations are excluded instead of included. Values of 2 b j ;diag =2 b j ;full > 1.0 indicate the parameter variance is larger for the calibration with the diagonal weight matrix; values <1.0 indicate the variance is smaller for the diagonal weight matrix. [21] We derive an analytical expression for 2 b j ;diag = 2 b j ;full that provides insight into how interaction between the sensitivities and the observation error correlations affect parameter uncertainty estimates. First the s 2 term is expanded using equation (1): s 2 ¼ ðnd npþ 1 ½y y 0 ðþ b Š T x½y y 0 ðþ b Š ð15aþ For a linear model, y 0 ðþ¼xb. b Furthermore, using the solution to the weighted least squares regression normal equations, by which b ¼ X T 1X xx T xy [Draper and Smith, 1998, p. 222], the simulated values can be expressed as y 0 ðþ¼x b X T 1X xx T xy. Substituting this into equation (15a) and performing matrix algebra yields: h s 2 ¼ ðnd npþ 1 y T xy y T xx X T i 1X xx T xy ð15bþ [22] Formulating equation (15b) for both a full and a diagonal weight matrix, substituting these expressions into equations (13) and (14), and forming the ratio of the diagonals yields: 2 b j;diag =2 b j;full h i y T x diag y y T x diag X X T 1X x diag X T x diag y X T 1 x diag X jj ¼ h i y T x full y y T x full X X T 1X x full X T x full y X T 1 x full X jj ð16þ where jj denotes the jth diagonal of a variance-covariance matrix. where 2 y 1 and 2 y 2 are the error variances for observations y 1 and y 2, respectively, y1 y 2 is the error covariance between the two observations, X =@b 1, and X =@b 1. Details of the derivation are provided in supporting information. [24] Equation (17a) can be expressed in terms of just two variables: (equation (8)), the correlation between errors in observations y 1 and y 2 ; and r dss, a ratio composed of sensitivities and variances: 2 b 1;diag =2 b ¼ 1 þ r2 dss 2r 2 dss 1;full 1 þ rdss 2 2 ð17bþ ð 1 2Þ For the special case in which one observation is insensitive to the parameter, equation (17b) becomes: 2 b 1;diag =2 b 1;full ¼ 1= 1 2 ; for X11 ¼ 0orX 21 ¼ 0 ð17cþ In equation (17b), r dss is defined as: r dss ¼ X 21= y2 X 11 = y1 ¼ dss 21 dss 11 ð18þ where the dimensionless scaled sensitivity (dss) is a measure of the information an observation provides about a parameter [Hill and Tiedeman, 2007, p. 48]: dss ij i jb j j! 1=2 j ð19þ Thus, r dss is the ratio of the information that observations y 1 and y 2 provide about parameter b 1, as measured by dss calculated using diagonal weighting. This form of dss is used because it allows the effects of sensitivity and error correlations to be distinguished in equation (17b). [25] To explain the analytical results in equation (17b), it is useful to return to equations (13) and (14) h and express i the variance ratio as 2 b 1 ;diag =2 b 1 ;full ¼ s2 diag =s2 full h i ~ 2 b 1;diag =~ 2 b 1;full where s 2 diag =s2 full ¼ 1 þ r2 dss 2r dss 1 þ r 2 dss ð20aþ 6343

6 ~ 2 b 1;diag =~ 2 b 1;full X T x diag X s 2 diag =s2 full ¼ 1; for X 11 ¼ 0orX 21 ¼ 0 1= X T x full X ð20bþ 1 1þr 2 ¼ dss 2r dss 1þrdss 2 ð 1 2Þ ð21aþ ~ 2 b 1;diag =~2 b 1;full ¼ 1= 1 2 for X11 ¼0orX 21 ¼0 ð21bþ Derivations of equations (20a) and (21a) are provided in supporting information. Equation (20a) also can be derived from equation 50 in Cooley and Christensen (2006). In equation (21) we have defined ~ 2 b 1;diag ¼ X T 1 x diag X and ~ 2 b 1;full ¼ X T 1, x full X and we denote ~ 2 and ~2 b1;diag b 1;full to be scaled parameter variances. [26] The s 2 diag =s2 full term of 2 b 1 ;diag =2 b 1 ;full measures how the observation error correlations and sensitivities affect the difference in fit of models calibrated with and without the correlations. If the model is correct and the weights reflect the accuracy of the observations, s 2 is expected to be 1[Hill and Tiedeman, 2007, p. 96]. For the simple oneparameter two-observation example, the underlying model is assumed to be correct, and x full correctly reflects the accuracy of the observations, so s 2 is expected to be 1 for the calibration with x full. Therefore, s 2 diag =s2 full > 1 can be interpreted as the fit being worse than expected for the model calibrated with x diag, and s 2 diag =s2 full < 1 can be interpreted as the fit being better than expected for this model. The X T 1 xx term of a parameter variance-covariance matrix measures how the interaction of weighting and sensitivities affects the uncertainty of individual parameters. For the one-parameter two-observation example, the ~ 2 b 1 ;diag =~2 b 1 ;full term of 2 b 1 ;diag =2 b 1 ;full reflects how this interaction affects the difference in uncertainty of b 1 when calculated with and without the observation error correlations. [27] Graphs of the ratios in equations (17b), (20a), and (21a) are shown in Figure 2. Different families of curves are produced depending on whether the sign of the product r dss is negative or positive. Cooley [2004, pp. 50 and 54] also found that differences between uncertainties calculated without and with observation and model error correlations were dependent on the signs and magnitudes of sensitivities and correlations. Figure 2 shows that the values of all three ratios depend on both and r dss when jr dss j is between about 0.1 and 10. The ratios for a given approach constant values when jr dss j < 0.1 and jr dss j > 10; that is, when the two dss differ by more than a factor of 10. The constant values equal the right-hand sides of equations (17c), (20b), and (21b), which represent the special case in which one observation is insensitive to the parameter. In this case, Figure 2. Ratios showing the difference in parameter variance (equation (17b)), model fit (equation (20a)), and scaled parameter variance (equation (21a)) when calculated with a diagonal instead of a full observation weight matrix, for (a, b, c) r dss < 0 and (d, e, f) r dss > 0, where is observation error correlation, r dss ¼ dss 21 /dss 11, and dss are dimensionless scaled sensitivities. These ratios assume that when the diagonal weight matrix is used, the observation error correlations are unknown and not included in the calculation of parameter variance. 6344

7 excluding the observation error correlations always overestimates the parameter variance, because the right side of (17c) is always >1. This effect on the variance is caused entirely by the scaled variance term (equation (21b)), because the model fit term is the same in the models with full and diagonal weighting (equation (20b)). To illustrate this result, consider that observation y 2 is insensitive for the simple example. Despite this insensitivity, y 2 provides information that reduces the uncertainty of parameter b 1 through the correlation of its error with that of observation y 1, reducing 2 b 1 ;full in comparison to 2 b 1 ;diag. As the correlation becomes larger, 2 b 1 ;diag =2 b 1 ;full of equation (17c) becomes larger, and the effect of omitting the error correlations in the model with diagonal weighting is more pronounced. While an inverse model with one sensitive and one insensitive observation will rarely occur in practice, the results for this case can be generalized to help explain the effect of error correlations on parameter uncertainty for inverse models that have some insensitive observation types, as illustrated in section [28] For a given, the difference in parameter uncertainty computed with and without observation error correlations is greatest when jr dss j¼1 (Figures 2a and 2d). That is, the largest differences occur when each observation contributes the same amount of information about parameter b 1. [29] For 0.1 < jr dss j<10 and r dss < 0, excluding the error correlations always increases the parameter variance 2 b 1 ( 2 b 1 ;diag >2 b 1 ;full ) as well as s2 and ~ 2 b 1 (Figures 2a 2c). Negative r dss occurs when (1) is negative and the dss each have the same sign or (2) is positive and the dss have opposite signs. For case (1), negative indicates that one observation is expected to be larger than its mean and the other is expected to be smaller than its mean, and samesigned dss indicates that the simulated values both increase, or both decrease, in response to a change in b 1. That is, a change in b 1 affects the simulated values in the opposite manner from how the error correlations affect the observed values; a similar lack of agreement between the sensitivities and correlations occurs for case (2). The consequence is that the diagonal model tends to achieve a fit that is worse than expected (s 2 diag > s2 full, Figure 2b), given its weighting that does not include the observation error correlations. This lack of agreement also contributes to the scaled variance term being larger for the model with diagonal weighting (~ 2 b 1 ;diag > ~2 b 1 ;full, Figure 2c). The effect of excluding the correlations has a greater impact on this uncertainty term than on the model fit term, as shown by the ratios in Figure 2c being larger than those in Figure 2b. [30] For r dss > 0 and jr dss j close to 1, excluding the error correlations decreases the estimated parameter variance 2 b 1 ( 2 b 1 ;diag <2 b 1 ;full ), s2, and the scaled parameter variance ~ 2 b 1 (Figures 2d 2f). Positive r dss occurs when (1) is positive and the dss have the same sign or (2) is negative and the dss have opposite signs. In both these cases, a change in b 1 changes the simulated values in a manner consistent with how the observations jointly vary about their means. This consistency between the sensitivities and correlations means that the model with diagonal weighting fits the observations better than expected (Figure 2e). Similarly, it causes b 1 in the model with diagonal weighting to have a smaller uncertainty as measured by the scaled variance term. When jr dss j is close to 1, there is a pronounced effect on 2 b 1 of the sensitivities and correlations being consistent (Figure 2d). In contrast, when jr dss j is further from 1, and one observation has a smaller magnitude of sensitivity than the other, this effect is less pronounced or absent. Figure 2d also shows that even if observation error correlations are large, there are values of jr dss j for which the parameter variance calculated with the full weight matrix is the same as that calculated using the diagonal weight matrix. [31] To evaluate the effect of using the theoretically correct parameter variance for a diagonal weight matrix (which assumes the error correlations are known), we compare the results in equation (17b) with an alternative parameter variance ratio: _ 2 b1;diag =2 b ¼ 1 þ 2 r2 dss 4 2 rdss 2 1;full 1 þ rdss 2 2 ð 1 2Þ ð22þ where _ 2 b1 ;diag is the variance of parameter b 1 for the diagonal weight matrix, calculated with equation (9b). The derivation is provided in supporting information. The ratio _ 2 b1 ;diag =2 b 1 ;full measures the difference in parameter variance calculated with the full and diagonal weight matrices assuming that a known full V() is available for both calculations. [32] Equation (22) and Figure 3 illustrate that _ 2 b1 ;diag = 2 b 1 ;full is independent of the signs of or r dss, and is always greater than or equal to 1, in contrast to the ratios of 2 b 1 ;diag =2 b 1 ;full shown in Figure 2. The results for jr dss j¼0.01 or 100 are nearly identical to those in Figures 2a and 2d, because these jr dss j approach the case where one observation is insensitive, and under that condition the right side of equation (22) reduces to that of equation (17c). When each observation provides the same amount of Figure 3. Ratios showing the difference in parameter variance, computed using equation (22), when calculated with a diagonal instead of a full observation weight matrix. These ratios assume that when the diagonal weight matrix is used, the observation error correlations are known and included in the calculation of parameter variance. 6345

8 information about the parameter (jr dss j¼1), the results in Figure 3 differ substantially from those in Figures 2a and 2d. In Figure 3, the variance ratio equals 1, indicating the parameter variances calculated with the full and diagonal weight matrices are the same. In Figure 2, the differences in the two variances are maximized when jr dss j¼ 1. Thus, for this one-parameter, two-observation model, the parameter variance typically calculated in practice for a diagonal weight matrix ( 2 b 1 ;diag) often is less accurate than the theoretically correct (yet typically unobtainable in practice) parameter variance for a diagonal weight matrix ( _ 2 b1 ;diag ), where accuracy is measured relative to the variance calculated for the full weight matrix Effect on Prediction Variance [33] Often in groundwater modeling, predictions and their uncertainty are of greater interest than parameter uncertainty. Prediction uncertainty is indirectly affected by the observation error correlations through their effect on parameter uncertainty, as shown in equation (12). Applying this equation to the one-parameter two-observation inverse model yields 2 z ;diag ¼ X z 2 V diag ðþ, b the variance of prediction z calculated with observation error correlations excluded, and 2 z ;full ¼ X z 2 V full ðþ, b the variance calculated with the correlations included. Here Xz 2 is a scalar because there is only one parameter, and it is the same in both equations because the model is assumed linear. The ratio of prediction variances is: 2 z ;diag =2 z ¼ V ;full diagðþ=v b full ðþ¼ b 2 b 1;diag =2 b 1;full ð23þ [34] Thus, for a one-parameter, two-observation model, the effects of observation error correlations and sensitivities on the prediction variance ratio are the same as the effects on the parameter variance ratio. 3. Application: Reactive Transport Model [35] A reactive transport model of denitrification is used to illustrate differences in parameter estimates and uncertainty from calibrations with and without observation error correlations included in the weighting. Predicted future nitrate concentrations and their uncertainty also are compared for different calibrations Model, Parameters, and Calibration Observations [36] The numerical reactive transport model was developed by Green et al. [2010] as part of an investigation of mixing effects on estimates of reaction parameters, using field data from an agricultural setting in the San Joaquin Valley, California (Figure 4). It simulates the reactions of O 2 and NO 3 in water that recharges a shallow alluvial aquifer and migrates toward a river. Steady-state flow is simulated using MODFLOW-2000 [Harbaugh et al., 2000]. Advection and hydrodynamic dispersion are simulated using a random-walk particle tracking code, RWHet [LaBolle et al., 2000], with backward-tracking. Solute concentrations in each well sample are estimated with a program module that calculates, for every particle in the sample, the concentration as a result of the input history of solute and the reactions occurring in the aquifer (see supporting information). This study explores the effect of observation error correlations on reaction parameter estimates and uncertainty, with the hydraulic and transport parameters kept constant. The comparisons in this study of results with and without error correlations are, therefore, not sensitive to dispersion or noise in the random walk solution. [37] The model has four reaction parameters, including the nitrogen isotope fractionation parameter (" N ), firstorder denitrification rate (k N ), first-order O 2 decay rate (k O ), and concentration of O 2 above which denitrification does not occur ([O 2 ] cut ). Green et al. [2010] calibrated the model for several realizations of the heterogeneous sedimentary deposits; five such realizations are considered here (realizations 1, 45, 124, 131, and 136 in Green et al., 2010). The previous calibrations were rerun with minor modifications for the purposes of this study including removing upper limits of the [O 2 ] cut parameter (previously <0.05 mmol L 1 ). Five realizations are used for consistency with the approach of Green et al. [2010] and to allow Figure 4. Example geologic realization and site map showing observation well locations (modified from Green et al. [2010], Figure 2). 6346

9 Table 1. Direct Measurement Types, Values, and Estimated Errors Measurement Type Median (Range) of Measured Value Median (Range) of Measurement Error Standard Deviation a Also an Observation Type for Reactive Transport Model Calibration? [O 2 ] (mmol L 1 ) ( ) ( ) yes [NO 3 ] (mmol L 1 ) 1.0 ( ) 0.29 ( ) yes 15 N[NO 3 ](%) 13.5 ( ) 0.54 ( ) yes 15 N[N 2 ](%) 0.21 ( 1.5 to 2.3) 0.20 ( ) no T( C) 19.2 ( ) 0.95 ( ) no P (mm Hg) ( ) 4.1 ( ) no Ar (mmol L 1 ) ( ) ( ) no [SF 6 ] (pptv) 1.2 ( ) 0.69 ( ) no [N 2 ] (mmol L 1 ) 0.86 ( ) ( ) no a Standard deviations are squared to obtain measurement error variances 2 u. evaluation of the effects of the error correlations relative to the effects of geological uncertainty as represented by the multiple realizations of heterogeneity. [38] The modeling techniques described above were used to obtain simulated equivalents for six types of observations. Three of the types are directly-measured observations (Table 1). The concentration of NO 3, and the stable isotope ratio 15 NofNO 3 (15 N[NO 3 ]) were determined by laboratory analysis of field samples, and the concentration of O 2 was measured in the field. The other three observation types are each derived from five or more directlymeasured values, including O 2, NO 3, and 15 N[NO 3 ] (Table 2). These types include the apparent O 2 decay rate (k O,app ), fraction NO 3 remaining (f N), and apparent isotope fractionation factor (" N,app ). Details of these calculations are provided in supporting information. Because of the shared dependencies of some observation types on one or more direct measurements (Figure 1 and Table 2, supporting information), errors in the observations are correlated. For all observation types except " N,app, observed values are available from 14 piezometers at various depths in five well clusters (Figure 4). Calculated values of " N,app are available from 6 piezometers. [39] Note that there is an oxygen decay rate model parameter (k O ) and an apparent oxygen decay rate observation type (k O,app ). Similarly, there is an isotope fractionation parameter (" N ) and an apparent isotope fractionation observation type (" N,app ). The parameters k O and " N are considered the intrinsic field values of the oxygen decay rate and isotope fractionation, respectively, and are estimated using inverse modeling with reactions applied to individual particles. The observed values k O,app and " N,app are the oxygen decay rate and isotope fractionation, respectively, calculated as described above and in supporting information and using bulk sample concentrations in place of individual particle concentrations. They are considered apparent values that are commonly estimated for field studies [e.g., Böhlke, 2002; Green et al., 2008; Tesoriero and Puckett, 2011;Liao et al., 2012] and differ from the intrinsic values because of the effects of mixing during dispersive transport and field collection of a groundwater sample Observation Error Correlations and Weight Matrices [40] The equations in supporting information and estimates of measurement error variances 2 u (Table 1) were used in equations (6) and (7) to propagate measurement error to observation error. For NO 3, T, and P, samples were available for multiple dates at each well during the year of the study. For each of these measurements, the value of 2 u in equations (6) and (7) was set equal to the annual-average of those samples to provide a single estimated observation consistent with the annual-average value estimated by the transport model at each well. The values of 2 y i for O 2,NO 3, and 15 N[NO 3 ] (Table 1) and the calculated values of 2 y i for k O,app, f N, and " N,app (Table 2) were then used to populate V diag (). These values of 2 y i and the values of yi y k calculated with equation (7) were used to populate V full (). Finally, V diag () and V full () were inverted to obtain x diag and x full, respectively. [41] To screen for possible effects of nonlinearity and non-gaussianity, estimates of observation variances from equation (6) were compared to a Monte Carlo simulation with 10,000 realizations of measurement errors for a single set of input measurements with standard deviations set equal to the medians of estimated errors in Table 1. The Monte Carlo simulated observation error distributions (Figure 5) are approximately normal, and the first-order estimates of standard deviations are within 20% of the actual values, a small difference in comparison to the orders-of- Table 2. Derived Observation Types, Values, Estimated Errors, and Relevant Equations Observation Type Measurements Used to Derive Median (Range) of Observed Value Median (Range) of Observation Error Standard Deviation a Equations b k O,app (yr 1 ) [O 2 ], T, P, [Ar], [SF 6 ] 0.16 ( ) 0.09 ( ) S22, S35 S43 f N [NO 3 ], T, P, [Ar], [N 2] 0.77 ( ) 0.09 ( ) S23 S24, S34, S36 S43 " N,app (%) [NO 3 ], 15 N[NO 3 ], 15 N[N 2 ], 15 ( 20 to 3) 3.6 ( ) S23 S24, S26, S31 S34, S36 S43 T, P, [Ar], [N 2 ] a Standard deviations are squared to obtain observation error variances 2 yi. b See supporting information. 6347

10 Figure 5. Probability density functions of derived observations calculated by Monte Carlo analysis using 10,000 realizations of errors in direct measurements, and equations in supporting information for deriving observations from measurements. Normalized observations for each of the derived observation types are calculated as (y MC y m )/ m where y m and m are, respectively, the mean and the standard deviation of all the simulated observations for a particular observation type, and y MC is a single Monte Carlo generated observation. magnitude variability among the estimated standard deviations for a given observation type (Table 2). These effects of nonlinearity and non-gaussianity are, therefore, unlikely to strongly affect the results of this study. [42] There are seven pairs of observation types with correlated errors (Table 3). For each of these pairs, the correlation for an individual observation pair is nonzero only if both observations are associated with the same monitoring well. For example, observations of [O 2 ] and k O,app from the same well always have correlated errors, but [O 2 ] errors at one well are not correlated with k O,app errors at any other well. Thus, because not all pairs of observation types have correlated errors, and because for those that are correlated only a few of the individual observation pairs have correlated errors, V full () is sparse. Of the 5776 entries in this matrix, 76 entries are the variances on the diagonals, and only 126 of the 5700 off-diagonal entries have nonzero covariances. This matrix is symmetric, and so only 63 observation pairs have correlated errors. In general, observation error variance-covariance matrices are likely to be sparse when error correlations stem from a set of measurements at a given location being used to calculate more than one calibration observation for that location alone. [43] For each of the seven pairs of observation types with correlated errors, Table 3 presents the average fraction of the covariance that is produced by each input measurement type. The entries for the first four pairs listed in the table show that because there is only one shared input measurement in the calculations of the observation types in the pair, 100% of the covariance between the two types is produced by that single shared measurement. Entries for observation pairs f N and k O,app, and k O,app and " N,app, show that the covariances stem almost entirely from the strong dependency of these observation types on [Ar] measurements. The covariance between f N and " N,app is not as strongly dependent on one particular input measurement Effect of Error Correlations on Parameter Estimates and Uncertainty [44] The reactive transport model was calibrated twice for each of the five geologic realizations, once without observation error correlations included (using x diag ) and once with the correlations included (using x full ). The vector of parameters estimated using x diag is denoted b diag and that estimated using x full is denoted b full. Individual parameters are b j,diag and b j,full. The dss indicate that for each parameter, there are at least two observation types with moderate to large sensitivities (Figure 6), so all four parameters were estimated in the regression. Parameter estimates and their linear confidence intervals for the 10 calibrations are shown in Table 4 and Figure 7. Most of the estimated values fall within previously observed ranges, for example, 10 to 30% for " N [Green et al., 2010], yr 1 for k N [Green et al., 2008; Tesoriero and Puckett, 2011], yr 1 for k O [Tesoriero and Puckett, 2011], and mmol L 1 for [O 2 ] cut [Green et al., 2010]. When x diag is used and the observation error correlations are omitted, estimates of parameters " N, k N, and k O are smaller for all realizations and estimates of [O 2 ] cut are larger for four of the five realizations, compared to when x full is used. Median differences range from 30% for " N to 81% for [O 2 ] cut (Table 4). Table 3. Observation Error Correlations and Average Fraction of Covariance Produced by Individual Direct Measurements Pair of Observation Types Error Correlation Average Fraction of Covariance a Produced by Direct Measurements y i y k Median (min to max) [O 2 ] [NO 3 ] 15 N[NO 3 ] T P [Ar] [N 2] [O 2 ] k O,app 0.87 ( 0.97 to 0.21) 1 f N [NO 3 ] 0.59 ( 0.97 to 0.27) 1 " N,app [NO 3 ] 0.33 ( 0.88 to 0.08) 1 15 N[NO 3 ] " N,app 0.27 ( 0.41 to 0.2) 1 f N k O,app 0.01 ( 0.08 to 0.0) k O,app " N,app 0.01 ( 0.01 to 0.01) f N " N,app 0.7 ( 0.69 to 0.93) b X i 2 u, X ns X k a Calculated with yiyk = ¼ j¼1 j¼1 ¼1 yiyk ns ns where, is an input measurement (nm 7) used to calculate model observations y i and y k, j is an individual sample (ns 14) for which input measurements were available, and f i and f k are the systems of equations used to derive model observations y i and y k from input measurement values u. b All pairs of f N and " N,app observations have positive error correlations except for one pair with a correlation of

11 Figure 6. Dimensionless scaled sensitivities (dss) calculated for the model calibrated with a diagonal weight matrix. Box shows interquartile range and whiskers show minimum and maximum values. For each parameter and observation type, these summary statistics are calculated using all individual dss over all five realizations. For observation types with no box and whiskers, all dss ¼ 0. [45] For " N, k N, and [O 2 ] cut, the differences in parameter estimates for the calibrations using x diag and x full are generally small compared to the size of the confidence intervals (Figure 7). For these parameters, the intervals for b j,diag and for b j,full fully overlap for most of the realizations. In addition, the differences in b j,diag or b j,full across the five realizations tend to be larger than the differences between b j,diag and b j,full for a given realization. The variability among realizations largely stems from differences of sediment distributions that affect paths and travel times between the water table and well screen, and therefore affect the travel time distributions of samples and the apparent reaction rates. Thus, for the error correlations considered here, the structural errors related to geological uncertainty and the uncorrelated observation errors affect the estimates of " N, k N, and [O 2 ] cut more strongly than does the omission of error correlations. [46] In contrast, differences between b j,diag and b j,full for parameter k O are much larger compared to their confidence intervals, and the intervals for b j,diag and b j,full overlap less, particularly for realizations 1, 131, and 136. The estimates and intervals for " N in realizations 45 and 124 and for k N in realization 45 also have these characteristics. These larger differences between b j,diag and b j,full relative to the confidence intervals occur because parameter uncertainty is smaller (Figure 8). The coefficients of variation (cv, equation (11)) for k O in all realizations, and for " N and k N in realizations 45 and 124, are substantially smaller than those of most other parameters. This suggests that the importance of including observation error correlations in the weighting increases as parameter uncertainty decreases. With smaller uncertainty, there is greater confidence in the differences between the parameter values estimated with and without these correlations. Also, for parameter k O, the differences Table 4. Parameter Estimates From Model Calibrations With the Full and the Diagonal Observation Weight Matrices " N (%) k N (yr 1 ) k O (yr 1 ) [O 2 ] cut (mmol L 1 ) Estimate Estimate Estimate Estimate Realization Full Diag Pct diff a full diag Pct diff full diag Pct diff Full diag Pct diff Median a Percent difference, defined as 100 (b j,diag b j,full )/jb j,full j. 6349