MULTIMODEL BAYESIAN ANALYSIS OF DATA-WORTH APPLIED TO UNSATURATED FRACTURED TUFFS

Transcription

1 MULTIMOEL BAYESIAN ANALYSIS OF ATA-WORTH APPLIE TO UNSATURATE FRATURE TUFFS M. Ye. Lu S.P. Neuman 2 and L. Xue 2 epartment of Scientific omputing Florida State University Tallahassee Florida USA s: mye@fsu.edu dl7f@fsu.edu 2 epartment of Hydrology and Water Resources University of Arizona Tucson Arizona USA s: neuman@hwr.arizona.edu xueliang983@gmail.com Abstract The management of contaminated groundwater systems requires an understanding of their response to alternative remediation strategies. Such understanding requires the collection of suitable data to help characterize the system and monitor its response to existing and future cleanup and/or containment options. It also requires incorporating such data in suitable models of water flow and contaminant transport. As the acquisition of subsurface characterization and monitoring data is costly it is imperative that the design of corresponding data collection schemes be cost-effective i.e. that the expected benefit of new information exceed its cost. A major benefit of new data is its potential to help improve one's understanding of the system in large part through a reduction in model predictive uncertainty. Traditionally value-of-information or data-worth analyses have relied on a single conceptualmathematical model of site hydrology. Yet there is a growing recognition that analyses and predictions based on a single hydrologic concept are prone to statistical bias and underestimation of uncertainty. This has led to a recent emphasis on conducting hydrologic analyses and rendering corresponding predictions by means of multiple models. Here we adopt a multimodel approach to optimum value-of-information or data-worth analyses based on model averaging within a Bayesian framewor (Neuman et al. 2). The Bayesian model averaging (BMA) approach is compatible with both deterministic and stochastic models in its maximum lielihood version (BMA) (Neuman 23) it is additionally consistent with current statistical methods of hydrologic model calibration. Implementation entails either Monte arlo simulation or lead order approximations. We implement BMA computationally on pneumatic permeability data from an unsaturated fractured tuff site near Superior Arizona USA.. INTROUTION Hydrogeologists face a daunting challenge helping to insure that contaminants in the subsurface do not pose unacceptable future riss to humans and the environment. To quantify and manage these riss one must understand their relationships to alternative pollution remediation schemes. Such understanding requires the collection of suitable data to help characterize the subsurface and monitor its response to existing and future scenarios. It also requires incorporating such data in suitable models of subsurface flow and contaminant transport. As noted by Bac (27) three strategies have traditionally been used to determine the magnitude of a data collection effort: minimizing cost for a specific level of accuracy or precision minimizing uncertainty for a given budget or responding to regulatory demands on data quantity and quality. Various combinations of these strategies have also been described such as a fitness-for-purpose approach [2]. Many today prefer a fourth approach based on value-of-information or data-worth analysis. Here the decision to collect additional data or the design of a data collection program is based on costeffectiveness. A program is considered cost-effective if the expected benefit from the new information exceeds the cost. A major benefit of new data is its potential to help improve one's understanding of the system in large part through a reduction in model predictive uncertainty. This benefit however is worth

2 the cost only if it has the potential to impact decisions concerning management of the water resource system. Value-of-information or data-worth analyses incorporating statistical decision theory have been applied to various water-related problems in the 97s and to groundwater problems in the 98s. More recent applications to groundwater resource and contamination issues have been reported among others in (Russell and Rabideau 2 ausman et al. 2). James and Freeze (993) proposed a Bayesian decision-maing framewor to evaluate the worth of data in the context of contaminated groundwater that has been widely cited in the subsequent literature. A comprehensive review focusing on health ris assessment can be found in (Yoota and Thompson 24). Additional recent publications of relevance include (Feyen and Gorelic 25 Norberg and Rosen 26). A major limitation of many existing approaches is that they rely on a single conceptual-mathematical model of the subsurface and flow/transport processes therein. Yet the subsurface is open and complex rendering its characteristics and corresponding processes prone to multiple interpretations and mathematical descriptions including parameterizations. This is true regardless of the quantity and quality of available data. Predictions and analyses of uncertainty based on a single hydrologic concept are prone to statistical bias (by committing a Type II error through reliance on an inadequate model) and underestimation of uncertainty (by committing a Type I error through under sampling of the relevant model space). Analyses of environmental data-worth which explore how different sets of conditioning data impact the predictive uncertainty of multiple models in a Bayesian context include (Freer et al. 996 Rojas et al. 2) whereas Freer et al. (996) employ Generalized Lielihood Uncertainty Estimation (GLUE) (Beven and Freer 2) Rojas et al. (2) combine GLUE with Bayesian Model Averaging (BMA) (Hoeting et al. 999). Nowa et al. (2) introduce a Bayesian approach to data worth analysis when flow and transport tae place in a random log hydraulic conductivity field. Whereas flow and transport are described by a single (linearized) model each having nown parameters other than those describing spatial variations in log hydraulic conductivity the latter is characterized by a single drift model and a continuous family of variogram models having uncertain parameters. In a similar spirit we adopt in this paper a multimodel approach to optimum value-of-information or dataworth analyses that is based on model averaging within a Bayesian framewor (Neuman et al. 2). Our approach is general in that it considers multiple models of any ind all having uncertain parameters whereas parameterizing models in the manner of (Nowa et al. 2) is elegant and computationally efficient it is unfortunately limited to a narrow range of variogram models and does not generally apply to other models such as those of flow and transport. We prefer BMA over GLUE because it (a) rests on rigorous statistical theory (b) is compatible with deterministic as well as stochastic models and (c) in its maximum lielihood () version (BMA) (Neuman 23) is consistent with current methods of hydrologic model calibration. Whereas BMA (lie the closely related approach in Nowa et al. 2) relies heavily on prior parameter statistics BMA can do without such statistics or otherwise update them on the basis of potential new data both before (during the so-called preposterior stage) and after (during the posterior stage) they are collected. We describe BMA and BMA theoretically outline the approach of Neuman et al. (2) to implement BMA computationally and illustrate the latter on pneumatic permeability data from an unsaturated fractured tuff site near Superior Arizona USA. Our methodology should be of help in designing the collection of hydrologic characterization and monitoring data in a cost-effective manner by maximizing their benefit under given cost constraints. The benefit would accrue from optimum gain in information or reduction in predictive uncertainty upon considering jointly not only traditional sources of uncertainty such as those affecting model parameters and the reliability of data but also most importantly lac of certainty about the conceptual-mathematical models that underlie the analysis and the scenarios under which the system would operate in the future. The methodology should apply to a broad range of models representing natural processes in ubiquitously open and complex earth and environmental systems.

3 2. BAKGROUN Bayesian ecision Analysis Framewor One way to cast the data-worth issue would be within a Bayesian ris-cost-benefit decision framewor such as that of Freeze et al. (99 992). Suppose without loss of generality that the data are intended to help one decide whether or not a contaminated site should be remediated. The problem can be cast in terms of a decision tree (Bac 27) containing objective functions defined for each decision alternative i 2 as i B i i Pf i i where B i is the benefit and i the investment cost ris being expressed as the product Pf i i of a ris aversion factor the probability of failure P i and the cost of failure f i. ollecting additional information generally causes the ris term to decrease due to a decrease in the probability of failure. The corresponding increase in i is the expected value (worth) of the new data. The final outcome of the analysis depends on the choice of decision rule one adopts for example maximizing would result in the largest benefit and lowest cost. i A reduction in the probability of failure comes about through a reduction in uncertainty about the expected system state present or future. The impact of hydrologic data on this expectation and the associated uncertainty are often evaluated by means of a hydrologic model. ommonly the model is considered to be certain while its parameters (and in some cases its forcing terms such as sources and boundary conditions) are treated as being uncertain due to insufficient and error-prone data. As already noted we now of only one wor that considers the impact of data on model predictive uncertainty within a Bayesian framewor by considering the model itself to be uncertain (Freer et al. 996) and one other wor that parameterizes this uncertainty (Nowa et al. 2). Below we provide bacground about Bayesian model averaging and its maximum lielihood version which we employ for this same purpose. Bayesian Model Averaging (BMA) onsider a random vector Δ the multivariate statistics of which are to be predicted with a set M of K mutually independent models M each characterized by a vector of parameters θ conditional on a discrete set of data (the case of correlated models has recently been considered in Sain and Furrer 2). In analogy to the case of a scalar (Hoeting et al. 999) we write the joint posterior (conditional) K distribution of Δ as pδ pδ MpM i.e. as the average over all models of the joint M posterior distributions pδ associated with individual models weighted by the model posterior probabilities pm. These weights are given by Bayes rule in the form K pm p MpM/ p MlpMl where l integrated lielihood of model M p M parameters pθ M the prior density of θ under model M and p M p M θ p θ M dθ is the θ being the joint lielihood of this model and its p M the prior probability of M. All probabilities are implicitly conditional on the choice of models entering into the set M. The posterior mean and covariance of Δ are given by EΔ EΔ MpM and ovδ K Δ Δ. We also consider the trace Tr ov E ov M ov E M M M scalar measure of the posterior variance of Δ. Δ which provides a

4 Maximum Lielihood Bayesian Model Averaging (BMA) BMA can be approximated through replacement of θ by an estimate θ ˆ which maximizes the lielihood p M θ. Obtaining such maximum lielihood () estimates entails calibrating each model against (conditioning on) the data using well-established statistical inverse methods. Approximating pδ M by pδ M where the subscript indicates estimation of θ was shown to be useful (for a scalar ) in the statistical literature. Neuman (23) proposed evaluating the weights pm based on a result due to Kashyap (982) according to (Ye et al. 24) K pm exp KI pm / exp KI l l p M l where KI KI min 2 2 KI ˆ 2ln 2ln ˆ N KI p M θ p θ M N ln ln F M KI being the socalled Kashyap model selection (or information) criterion for model M 2 KI min its minimum value over 2ln p M θ 2ln p θ M a negative log lielihood incorporating all candidate models and prior measurements of the parameters (if available) evaluated at θ ˆ. Here N is the dimension of θ (number of adjustable parameters associated with model M ) N is the dimension of (number of discrete data points which may include measured parameter values) and F is a normalized (by N ) observed Fisher information matrix. In the limit of large N / N KI reduces asymptotically to the 2ln ˆ BI p M θ N ln N (Ye et al. 28). Bayesian selection (or information) criterion 3. EFFET OF ATA AUGMENTATION ON UNERTAINTY We now as the question how would an augmented data base where is a potential new dataset not presently available (and thus unnown) impact the above BMA uncertainty analysis. We address the question through BMA prediction of denoted by and assessment of the corresponding predictive uncertainty. Our proposed analysis entails the following steps (for additional details see Neuman et al. 2):. Postulate a set M of K mutually independent models M with parameters θ for the desired output vector Δ 2. Obtain estimates θ ˆ of θ by calibrating each minimization of the log lielihood 2ln p M 2ln p M M against available data through θ θ then compute a corresponding parameter estimation covariance Γ and KI 3. ompute p M K exp / exp l l l 2 KI p M 2 KI p M where the subscript designates the estimation process in step 2 4. For each model M estimate EΔ M and ovδ M either through linearization or via Monte arlo simulation (we use the latter approach below): a. raw random samples (realizations) of θ from a multivariate Gaussian distribution with mean θ ˆ and covariance Γ b. Estimate EΔ M θ and ov M Δ θ for each realization of θ

5 c. Average over all realizations of ov Δ M E Δ M and θ to obtain sample estimates of 5. ompute E Δ ovδ and/or Tr ov Δ 6. Postulate a set P of I alternative geostatistical statistical or stochastic models P i with parameters π i for a potential (presently unavailable) data set the models P i may be independent of M may form extensions of M or may coincide with the latter as in the computational examples described below 7. Predict multivariate statistics of conditional on via BMA by means of the model set P using a procedure paralleling that described for Δ in steps Estimate E Δ and ovδ where the subscript designates the estimation process in step 2 with respect to an augmented data set either through linearization followed by step or via Monte arlo simulation by using the statistics of from step 7 to generate random realizations of (we use the second option below) for each realization and for each model M : a. Obtain estimates θ ˆ of θ by minimizing this negative log lielihood with respect to θ then compute the corresponding estimation covariance Γ and KI p M b. ompute c. For each model M estimate EΔ M and ovδ M via Monte arlo simulation: i. raw random samples (realizations) of θ from a multivariate Gaussian distribution with mean ˆ θ and covariance Γ E M ov Δ M θ for each ii. Estimate Δ θ and realization of θ iii. Average over all realizations of E M Δ and ov Δ M θ to obtain sample estimates of d. ompute E Δ ovδ and/or Tr ov Δ ov Δ 9. Average over all realizations of to obtain sample estimates of E Δ and/or Tr ov Δ. Repeat steps 6-9 for different sets 2 3 of potential data and select that set which maximizes the difference Tr ov E Δ Tr ov Δ Tr E ov Δ between the trace conditional on and the expected trace conditional on and. 4. APPLIATION TO AIR PERMEABILITIES OF FRATURE TUFF We apply the above approach to log air permeability data (log ) from unsaturated fractured tuff near Superior Arizona. Spatially distributed log data were obtained by Guzman et al. (996) based on a steady state interpretation of 84 pneumatic injection tests in -m-length intervals along 6 boreholes at the site (Figure ). Five of the boreholes (V2 W2A X2 Y2 Z2) are 3-m long and one (Y3) has a length of 45 m five (W2A X2 Y2 Y3 Z2) are inclined at 45 o and one (V2) is vertical. We consider two crossvalidation cases: V I in which log measured in W2a Y3 and Z2 play the role of existing data

6 Z (m) boreholes X2 and Y2 are the sites of potential new data and 2 the goal being to predict log Δ along V2 V II in which log measured in W2a X2 and Y2 play the role of boreholes V2 and Z2 are the sites of potential new data and 2 the goal being to predict log Δ along Y3 (true potential data are denoted by and their estimates by ). Given that in each case one has funds to measure log in only one borehole which among boreholes X2 and Y2 should one conduct such measurements in case V I and which among V2 and Z2 in case V II? The issue is compounded by uncertainty about the correct geostatistical model and parameters to be employed in each case. To address it we consider three alternative variogram models: exponential (Exp) spherical (Sph) and power (Pow). etails about these models their calibration and estimation of model probabilities are given in Ye et al. (24 28). Figure 2 shows how the three variogram models calibrated against and 2 compare with sample variograms in V I and V II. The corresponding posterior probabilities are listed in Table showing a preference for the exponential model in all cases. -5 W2a V2 X2 Y2 Y Z Y (m) X (m) 4 3 Figure. Spatial locations of 84 -m-scale log data at ALRS. V I (a) Exp.2 Sph Pow (c) Exp.2 Sph Pow V II.8 (b).6.4 Exp.2 Sph Pow (d).6.4 Exp.2 Sph Pow (e).4 Exp.2 Sph Pow (f).4 Exp.2 Sph Pow Figure 2. Sample and fitted variograms based on (a-b) (c-d) and (e-f). 2 Figure 3 compares measured log Δ values along borehole V2 with predicted values and their 95% confidence intervals obtained with each variogram model and with BMA based on and

7 in V I. The predictions are contained within the 95% confidence intervals which are seen to 2 narrow down slightly with data augmentation. orresponding values of Var Var and Var Var 2 are depicted in Figure 4. Augmenting the sample size from 5 () to 35 ( ) and 33 ( ) is seen to reduce predictive variance along borehole V2. As Δ Δ = 2.3 and 2 Tr ov Tr ov Tr ov Δ Tr ov Δ = 2. collecting new data along borehole X2 is seen to yield greater uncertainty reduction than doing so along borehole Y2. Table. Posterior probabilities (%) based on and. 2 V I V II Model Pow Exp Sph Pow Exp Sph p(m ) p(m ' ) p(m ' 2 ) (a) Pow - -2 (b) Exp ata index ata index - (c) Sph ata index - (d) BMA ata index Figure 3. Measured log (+) predicted (solid) and 95% confidence intervals (dashed) for (a) Pow (b) Exp (c) Sph and (d) BMA on V2 based on (blac) (blue) and (green) in V I. Since in reality the real data randomly generated data sets 2 E of 2 are unnown we predict them through the mean r. Figure 5 shows E the 2 r realizations and their 95% confidence interval in V I. E is seen to represent a smoothed version of some values of which lie outside the confidence interval suggesting a slight bias in the generated values. Variations of Var Δ E Var Δ and Var E Δ along borehole V2 are shown in Figure 6. A comparison between Figures 6 and 4 reveals that although Tr ov Δ 3.64 and Tr 3.78 ov Δ exceed Tr ov Δ by a slight amount the spatial patterns in the two figures are similar. Liewise though E VarΔ

8 2 2 in Figure 4 these too have near identical spatial patterns. One therefore expects the estimated variance reduction Var E Δ to exhibit a pattern similar to that of the true variance reduction and E VarΔ in Figure 6 tends to exceed Var 2 Δ and Var Δ 2 2 Var Δ Var Δ. The predicted variance reduction measure Tr ov E( ) Δ =.59 is seen in Table 2 to be smaller than its true counterpart ' Tr ov Tr ov ' Δ Δ = 2.3 the same applying to Tr ov E( ) Δ = Δ Δ =.. Selecting borehole X2 as the '.42 in comparison to Tr ov Tr ov ' target for collecting additional data in V I is thus seen in retrospect to have been a correct choice. Though space does not allow us to present all details of analyzing V II we nevertheless summarize the final results in Table 2. In this case the analysis indicates and validates the choice of borehole V2 as the optimum target for data collection. Variance.8.7 (a) Var( ).3 Var( ).2 Var( ) - Var( ) ata index Variance Var( ) Var( 2 ) Var( ) - Var( 2 ) ata index (b) Var Var along borehole V2 based on (a) and (b) in V I. Figure 4. Variations of Var Var and (a) ata index - -2 (b) ata index Figure 5. E (solid blue) 2 realizations r (gray) and 95% confidence interval of (red) all 2 realizations (dashed blue) in V I.

9 Variance Var( ) E Var( ) Var E( ) ata index (a) Variance Var( ) E 2 Var( 2 ) 2 Var 2 E( 2 ) ata index (b) Figure 6. Variation of Var E Var Δ and Var E Δ along predicted borehole V2 based on (a) + and (b) + 2 in V I. Table 2. Predicted and actual trace variance reductions in V I and V II. V I V II 2 2 Predicted Actual ONLUSIONS. A multimodel approach to optimum value-of-information or data-worth analyses has been proposed based on a Bayesian model averaging (BMA) framewor. We have focused on a maximum lielihood (BMA) variant of BMA that (a) is compatible with both deterministic and stochastic models (b) admits but does not require prior information about the parameters (c) is consistent with modern statistical methods of hydrologic model calibration (d) allows approximating lead predictive moments of any model by linearization and (e) updates model posterior probabilities as well as parameter estimates on the basis of potential new data both before and after such data become actually available. 2. The proposed approach should be of help to hydrogeologists in designing the collection of characterization and monitoring data at contaminated sites in a cost-effective manner by maximizing their benefit under given cost constraints. Benefits would accrue from optimum gain in information or reduction in predictive uncertainty (and ris) upon considering jointly not only traditional sources of uncertainty such as those affecting model parameters and the reliability of data but also lac of certainty about the underlying models. 6. AKNOWLEGEMENT This research was supported in part through a contract between the University of Arizona and Vanderbilt University under the onsortium for Ris Evaluation with Staeholder Participation (RESP) III funded by the U.S. epartment of Energy. The first and second authors were supported in part by NSF-EAR grant 974 and OE-ERSP grant E-S2687.

10 7. REFERENING Aaie H. (974) A new loo at statistical model identification. IEEE Transactions on Automatic ontrol A Bac P.-E. (27) A model for estimating the value of sampling programs and the optimal number of samples for contaminated soil. Environ. Geol. doi:.7/s Beven K.J. and J. Freer (2) Equifinality data assimilation and uncertainty estimation in mechanistic modelling of complex environmental systems using the GLUE methodology. Jour. Hydrology 249: 29. ausman A.M. J. oherty.. Langevin and M.. Suop (2) Quantifying data worth toward reducing predictive uncertainty. Ground Water 48(5) eutsch.v. and A.G. Journel (998) GSLIB: Geostatistical Software Library and User s Guide 2nd ed. Oxford Univ. Press New Yor. i Federico V. and S.P. Neuman (997) Scaling of random fields by means of truncated power variograms and associated spectra. Water Resour. Res. 33(5) Feyen L. and S.M. Gorelic (25) Framewor to evaluate the worth of hydraulic conductivity data for optimal groundwater resources management in ecologically sensitive areas. Water Resour. Res. 4 W39 doi:.29/23wr29. Freer J. K. Beven and B. Ambroise (996) Bayesian estimation of uncertainty in runoff predictioin and the value of data: An application of the GLUE approach. Water Resour. Res. 32(7) Freeze R.A. J. Massman L. Smith T. Sperling and B. James (99) Hydrogeological decision analysis:. A framewor. Ground Water 28(5) Freeze R.A. J. Brude J. Massman T. Sperling and L. Smith (992) Hydrogeological decision analysis: 4. The concept of data worth and its use in the development of site investigation strategies. Ground Water 3(4) Guzman A. G. A.M. Geddis M.J. Henrich.F. Lohrstorfer and S.P. Neuman (996) Summary of Air Permeability ata From Single-Hole Injection Tests in Unsaturated Fractured Tuffs at the Apache Leap Research Site: Results of Steady-State Test Interpretation. Rep. NUREG/R-636 prepared for U.S. Nuclear Regulatory ommission Washington.. Hoeting J.A.. Madigan A.E. Raftery and.t. Volinsy (999) Bayesian model averaging: A tutorial. Statist. Sci. 4(4) Hurvich.M. and.-l. Tsai (989) Regression and time series model selection in small simple. Biometria 76(2) James B.R. and R.A. Freeze (993) The worth of data in predicting aquitard continuity in hydrogeological design. Water Resour. Res. 29(7) Kashyap R.L. (982) Optimal choice of AR and MA parts in autoregressive moving average models. IEEE Transactions on Pattern Analysis and Machine Intelligence 4(2) Neuman S.P. (23) Maximum lielihood Bayesian averaging of alternative conceptual-mathematical models. Stoch. Environmental Research and Ris Assessment 7(5) OI:.7/s Neuman S.P. L.Xue M. Ye and. Lu (2) Bayesian analysis of data-worth considering model and parameter uncertainties Advances in Water Resources in press. Norberg T. and L. ROSEN (26) alculating the optimal number of contaminant samples by means of data worth analysis. Environmetrics Nowa W. F.P.J. e Barros and Y. Rubin (2) Bayesian geostatistical design: Tas-driven site investigation when the geostatistical model is uncertain. Water Resour. Res. 46 W35535 doi:.29/29wr832. Rojas R. L. Feyen O. Batelaan and A. assargues (2) On the value of conditioning data to reduce conceptual model uncertainty in groundwater modeling. Water Resour. Res. doi:.29/29wr8822 in press. Russell K.T. and A.J. Rabideau (2) ecision analysis for pump-and-treat design. Groundwater Monitoring and Remediation Summer Sain S.R. and R. Furrer (2) ombining climate model output via model correlations. Stochastic Environmental Research and Ris Assessment 24(6) OI:.7/s Thompson M. and T. Fearn (996) What exactly is fitness for purpose in analytical measurements? Analyst 2(March) Ye M. S.P. Neuman and P.. Meyer (24) Maximum Lielihood Bayesian averaging of spatial variability models in unsaturated fractured tuff. Water Resour. Res. 4 W53 doi:.29/23wr2557. Ye M. P.. Meyer and S.P. Neuman (28) On model selection criteria in multimodel analysis. Water Resour. Res. 44 W3428 doi:.29/28wr683. Yoota F. and K. Thompson (24) Value of information literature analysis: A review of applications in health ris assessment. Med. ecis. Ma