Analysis of Regression and Bayesian Predictive Uncertainty Measures
|
|
- Claud Carson
- 5 years ago
- Views:
Transcription
1 Analysis of and Predictive Uncertainty Measures Dan Lu, Mary C. Hill, Ming Ye Florida State University, Tallahassee, FL, USA U.S. Geological Survey, Boulder, CO, USA ABSTRACT Predictive uncertainty can be quantified using confidence and probability intervals constructed around predictions. Confidence intervals are based on regression inferential theory; probability intervals are based on theory. For the confidence intervals, this work considered linear and nonlinear confidence intervals obtained using methods that require tens and hundreds of model runs, respectively. The probability intervals are obtained using Markov Chain Monte Carlo (MCMC) methods that require,s of model runs. Confidence and probability intervals are conceptually different and only mathematically equivalent under certain conditions. We use simple test cases to show that for linear models, the two types of intervals are mathematically equivalent with proper choices of prior probability. However, for nonlinear models, regardless of choice of prior probability, the two types of intervals are always different. The discrepancy depends on the model total nonlinearity. Therefore, it is inappropriate to use the two intervals to validate each other, as has been done in previous practice. INTRODUCTION Groundwater modeling is often used to predict effects of future anthropomorphic or natural occurrences. Since modeling predictions are inherently uncertain, quantification of predictive uncertainty is necessary. Confidence intervals and probability intervals constructed around the predictions can be used as measures of predictive uncertainty. Confidence intervals are based on inferential statistical theory from regression; linear and nonlinear intervals can be calculated. Probability intervals are based on theory. Markov chain Monte Carlo (MCMC) has been popular for estimating the probability intervals. While comparative studies of the two types of predictive uncertainty measures have been conducted (e.g., Vrugt and Bouten, ; Gallagher and Doherty, 7), underlying theoretical differences remain unclear. The purpose of this work is to compare the two kinds of predictive uncertainty measures by investigating their theoretical differences. To illustrate these differences, we consider a set of simple test cases with linear and nonlinear models. CONFIDENCE INTERVALS AND PROBABILITY INTERVALS Confidence intervals, from the frequentist point of view, represent the percent of the time in repeated sampling that the confidence intervals contain true predictions. To understand this better, consider the procedure of evaluating the confidence intervals. It involves first sampling N sets of observations based on distribution of errors and then calculating the confidence intervals (with confidence level -)for a certain prediction function based on the generated N sets of observations. For the N intervals, (- % of the intervals contain the true value. For linear intervals, the portions that the true value is either larger than the upper or smaller than the lower confidence limits are equal, being /. For nonlinear intervals the portions are not necessarily equal. The probability intervals, inferred from theory, represent posterior probability that the predictions lies in the interval. In statistics, a prediction is thought of as a random variable with its own distribution. The posterior distribution summarizes the state of knowledge about the unknown prediction conditional on the prior and current data. The narrower the distribution is, the greater our knowledge about the prediction. The amount is measured by the probability interval, which is a probabilistic region around posterior statistics such as posterior mean. They are calculated here using Markov Chain Monte Carlo (MCMC) methods that generate the entire posterior probability distribution from which the intervals are determined.
2 RELATIONSHIP BETWEEN CONFIDENCE AND PROBABILITY INTERVALS First consider a linear model, y Xβε, with n observations in the vector y, p unknown true parameters in the vector β, true random errors in the vector. The random error is assumed to be multivariate Gaussian, i.e., ε Nn (, C), where C ω and is the weight matrix used in objective function of inverse modeling. The estimates of β are multivariate Gaussian, i.e., ˆ * T β N p ( β, ( X ωx) ), where X is sensitivity matrix. Consider a linear prediction function g( β) Zβ. Using regression theory, the ( ) % confidence interval on the prediction (assuming that the model correctly represents reality) is given for two circumstances with unknown and known σ. When σ is unknown, and it is estimated by the calculated variance ˆ T s ( yxβ) ω( yxβ ˆ)/( n p), the distribution of g( β ˆ) is t-distribution, and the confidence interval is (Hill and Tiedeman, 7) ˆ T T / g( β) t /( n p)[ s Z ( X ωx) Z ] () where t -/ (n-p) is a t statistic with significance level and degrees of freedom equal to (n-p). When σ is known, the distribution of g( β ˆ) is normal distribution, the confidence interval is (McClave and Sincich, ) ˆ T T / g( β) z /[ Z ( X C X) Z ] () where z / is the z statistic with significance level. In statistics with noninformative priors for which p( β) constant and p( ) /, the posterior distribution of g( β ) is multivariate t-distribution. Thus, the ( ) % probability intervals for g( β) are the same as those of equation () derived from regression theories. In the same context, for informative conjugate prior with p( β) N p ( β p,c p ), and assume σ is known, the posterior distribution of g( β ) is multivariate normal. Thus, the ( ) % probability interval for g( β ) (assuming that the model correctly represents reality) is given as (McLaughlin and Townley, 996) ' T T / g( β p) z /[ Z ( X C XCp ) Z ] (3) As C p I, equation (3) reduces to equation (). The only difference is that the prediction is evaluated ' for β p the posterior mean, as determined from theory, instead of ˆβ the least square estimate, as determined from regression theory. For a linear problem, the two quantities of parameters are the same and equations () and (3) produce the same intervals. For a nonlinear model y f ( β) ε with parameters β, errors ε Nn(, C) with known C, based on theorem, with noninformative prior, the posterior density of parameter β is (Berger, 985) exp[log p( y β)] p( β y) (4) exp[log p( y)] dβ Consider a Taylor series expansion of log p( y β ) about ˆβ to the second order term, where ˆβ maximizes the log likelihood, log p( y β ). Then equation (4) is approximated by:
3 where ˆ ˆ T exp log ( ) ( ) ( ˆ)( ˆ p y β ββ I β ββ) p( β y) ˆ exp log ( ) ( ˆ T ) ( ˆ)( ˆ p y β ββ I β ββ) d β exp ( ˆ T ) I( ˆ)( ˆ ββ β ββ) p/ ˆ / ( ) I( β) ˆ log p( y β) I( β) T ββ β β ˆ β Xβ ), the posterior density ˆ ˆ p( β y) Nn β, [ I( β)] is the Fisher information matrix. When the model is linear (i.e., f ( ) is exact with [ ( ˆ T I β)] XC X. In this case, the probability interval of g( β ) from posterior distribution is mathematically equivalent with its confidence interval in regression as shown in equation (). However, if the model is highly nonlinear as indicated by large total nonlinearity, ignoring the higher order terms can cause significant error. In this case, confidence and probability intervals can be very different. The difference depends on the size of the higher order terms, which is reflected in the skew of the distribution. In addition to the linear confidence intervals above, nonlinear confidence intervals are also available from regression theories (Vecchia and Cooley, 987; Cooley, 4; Hill and Tiedeman, 7) that should be able to account for higher order terms resulted from model linearization. Nonlinear intervals can be calculated using likelihood method of Vecchia and Cooley (987). It determines the minimum and maximum values of prediction over a confidence region on the parameter set. The confidence region is defined in p-dimensional parameter space and has a specified probability of containing the true set of parameter values, as illustrated in Figure. (5) SA Figure : Geometry of a nonlinear confidence interval on prediction g(b). The parameter confidence region (shaded area), contours of constant g(b) (dashed lines), and locations of the minimum (g(b)=c, with b=b L ) and maximum (g(b)=c 4, with b=b U ) values of the prediction on the confidence region are shown. The lower and upper limits of the nonlinear confidence interval on prediction g(b) are thus c and c 4, respectively. (Adapted from Hill and Tiedeman, 7, Figure 8.3.) The method for computing nonlinear confidence intervals involves first defining the (-)-percent parameter confidence region. This region is defined as the set of parameter values for which the objective-function values, S(b), satisfy the following condition: ' / S( b) S( b ) s t ( n p) (6) Nonlinear intervals are also shown in the results below for simple test cases.
4 SIMPLE TEST CASES To compare the predictive uncertainty measures of confidence intervals in regression and probability intervals from theory, we apply the three measures, linear and nonlinear confidence intervals and probability intervals, to two simple test cases. In both test cases, we employ MCMC implemented in MICA code (Doherty, 3) to calculate the probability intervals. Linear Test Case Linear model y axbε, with parameters a= and b=3 and true errors i N(,) conjugate prior of the two parameters with C p. We consider I. Twenty data (x=,, ) are used to calibrate model and the calibrated model is used to predict the point at x=3. Nonlinear Test Case In the nonlinear test problem, the model is y x/ asin( abx) ε. All the other conditions are the same as those of the linear test problem. Cumulative distribution function F(a).8.6. Linear Model (a) Parameter a Cumulative distribution function F(a).8.6. Nonlinear Model (d) Parameter a Cumulative distribution function F(b).8.6. (b) Parameter b Cumulative distribution function F(b).8.6. (e) Parameter b Cumulative distribution function F(y).8.6. (c) Prediction y Cumulative distribution function F(y).8.6. (f) Prediction y Figure : Cumulative distribution functions of parameters and prediction based on regression and theory for parameter a and b, and prediction y in both linear simple test case (a, b, and c); and nonlinear simple test case (d, e, and f). Figure 3: The nonlinear confidence interval limits (red dots), the minimum and maximum values of prediction (red lines), the confidence region of parameter set bounded by the objective function goal (black contour); the probability interval limits (blue dot), where the upper.5% and lower.5% prediction values include parameter samples indicated by green dots from MCMC, and the median 95% prediction values include the samples indicated by yellow dots. Figure plots the cumulative distribution functions (CDFs) of the parameters and prediction for the linear and nonlinear test cases. The left panel of Figure confirms that the distributions of parameters and prediction from regression and theory are identical in the linear model case, as the mathematical theory above indicates. Therefore, for the linear model, the confidence and probability intervals are equivalent. However, for the nonlinear model, due to nonzero higher order derivatives of the likelihood function that are discarded in equation (5), these two intervals are distinct. In this case, the probability interval is smaller than the linear confidence interval, as shown in the right panel of Figure. And it is also smaller than the nonlinear confidence interval as illustrated in Figure 3. In Figure 3, the
5 black ellipse represents the 95% confidence region of the true parameters, the black star at the center of the ellipse. The red lines are model evaluations that intersect with the ellipse, and the intersections are the maximum and maximum values of the prediction (specific to the confidence region). The yellow and green dots are parameter samples obtained from MCMC simulation. Model predictions of these samples are first sorted and the threshold parameters values of the.5% and 97.5% percentiles of the predictions are identified. Their corresponding model evaluations are plotted in blue lines in Figure 3. Figure 3 shows the discrepancy between nonlinear confidence interval determined by the minimum and maximum values of prediction over a confidence region on the parameter set and probability interval from MCMC samples. CONCLUSIONS This work includes theoretical analysis and numerical experiments (using simple test cases) for comparing the confidence intervals based on regression theory and probability intervals based on theory. For linear models, the two types of intervals are mathematically and numerically equivalent only with noninformative prior information. However, for the nonlinear models, the confidence intervals and probability intervals are distinct mathematically and numerically. Their discrepancy depends on the model total nonlinearity. For groundwater models that are always nonlinear, it is not appropriate to validate the confidence intervals and probability intervals for each other. ACKNOWLEDGMENTS The authors thank John Doherty for providing the MICA code of MCMC simulation. This work was supported in part by NSF-EAR grant 974 and DOE-SBR grant DE-SC687. REFERENCES Berger, J.O., 985. Statistical decision theory and analysis, nd edition, Springer. Cooley, R. L., 4. A theory for modelling groundwater flow in heterogeneous media, U. S. Geological Survey Professional Paper 979. Doherty, J., 3. MICA: model-independent Markov Chain Monte Carlo analysis, Watermark Numerical Computing, Brisbane, Australia. Gallagher M., Doherty J., 7. Parameter estimation and uncertainty analysis for a watershed model, Environmental Modelling and Software,, -. Hill, M.C., Tiedeman C., 7. Effective calibration of ground water models, with analysis of data, sensitivities, predictions, and uncertainty, John Wiley, New York. McClave, J.T., Sincich T.,. Statistics, 8 th edition, Prentice Hall. McLaughlin, D., Townley L.R., 996. A reassessment of the groundwater inverse problem, Water Resour. Res., 3(5), 3-6. Vecchia, A.V., Cooley R.L., 987. Simultaneous confidence and prediction intervals for nonlinear regression models with application to a groundwater flow model, Water Resour. Res., 3(7), Vrugt, J. A., Bouten W.,. Validity of first-order approximations to describe parameter uncertainty in soil hydraulic models, Soil Sci. Soc. Am. J. 66:74-75.
Analysis of regression confidence intervals and Bayesian credible intervals for uncertainty quantification
WATER RESOURCES RESEARCH, VOL. 48,, doi:10.1029/2011wr011289, 2012 Analysis of regression confidence intervals and Bayesian credible intervals for uncertainty quantification Dan Lu, 1 Ming Ye, 1 and Mary
More informationEstimation of Operational Risk Capital Charge under Parameter Uncertainty
Estimation of Operational Risk Capital Charge under Parameter Uncertainty Pavel V. Shevchenko Principal Research Scientist, CSIRO Mathematical and Information Sciences, Sydney, Locked Bag 17, North Ryde,
More informationXiaoqing Shi Ming Ye* Stefan Finsterle Jichun Wu
Special Section: Model-Data Fusion in the Vadose Zone Xiaoqing Shi Ming Ye* Stefan Finsterle Jichun Wu Evalua ng predic ve performance of regression confidence intervals and Bayesian credible intervals
More informationfor Complex Environmental Models
Calibration and Uncertainty Analysis for Complex Environmental Models PEST: complete theory and what it means for modelling the real world John Doherty Calibration and Uncertainty Analysis for Complex
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationStatistical Methods for Particle Physics Lecture 4: discovery, exclusion limits
Statistical Methods for Particle Physics Lecture 4: discovery, exclusion limits www.pp.rhul.ac.uk/~cowan/stat_aachen.html Graduierten-Kolleg RWTH Aachen 10-14 February 2014 Glen Cowan Physics Department
More informationBayesian Inference: Concept and Practice
Inference: Concept and Practice fundamentals Johan A. Elkink School of Politics & International Relations University College Dublin 5 June 2017 1 2 3 Bayes theorem In order to estimate the parameters of
More informationDevelopment of Stochastic Artificial Neural Networks for Hydrological Prediction
Development of Stochastic Artificial Neural Networks for Hydrological Prediction G. B. Kingston, M. F. Lambert and H. R. Maier Centre for Applied Modelling in Water Engineering, School of Civil and Environmental
More informationSampling: A Brief Review. Workshop on Respondent-driven Sampling Analyst Software
Sampling: A Brief Review Workshop on Respondent-driven Sampling Analyst Software 201 1 Purpose To review some of the influences on estimates in design-based inference in classic survey sampling methods
More informationModeling Uncertainty in the Earth Sciences Jef Caers Stanford University
Probability theory and statistical analysis: a review Modeling Uncertainty in the Earth Sciences Jef Caers Stanford University Concepts assumed known Histograms, mean, median, spread, quantiles Probability,
More informationUniversität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen. Bayesian Learning. Tobias Scheffer, Niels Landwehr
Universität Potsdam Institut für Informatik Lehrstuhl Maschinelles Lernen Bayesian Learning Tobias Scheffer, Niels Landwehr Remember: Normal Distribution Distribution over x. Density function with parameters
More informationBAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS
BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS Andrew A. Neath 1 and Joseph E. Cavanaugh 1 Department of Mathematics and Statistics, Southern Illinois University, Edwardsville, Illinois 606, USA
More informationMAXIMUM LIKELIHOOD, SET ESTIMATION, MODEL CRITICISM
Eco517 Fall 2004 C. Sims MAXIMUM LIKELIHOOD, SET ESTIMATION, MODEL CRITICISM 1. SOMETHING WE SHOULD ALREADY HAVE MENTIONED A t n (µ, Σ) distribution converges, as n, to a N(µ, Σ). Consider the univariate
More informationFinite Population Correction Methods
Finite Population Correction Methods Moses Obiri May 5, 2017 Contents 1 Introduction 1 2 Normal-based Confidence Interval 2 3 Bootstrap Confidence Interval 3 4 Finite Population Bootstrap Sampling 5 4.1
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationBayesian Econometrics
Bayesian Econometrics Christopher A. Sims Princeton University sims@princeton.edu September 20, 2016 Outline I. The difference between Bayesian and non-bayesian inference. II. Confidence sets and confidence
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationDeciding, Estimating, Computing, Checking
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationDeciding, Estimating, Computing, Checking. How are Bayesian posteriors used, computed and validated?
Deciding, Estimating, Computing, Checking How are Bayesian posteriors used, computed and validated? Fundamentalist Bayes: The posterior is ALL knowledge you have about the state Use in decision making:
More informationIntroduction to Probability and Statistics (Continued)
Introduction to Probability and Statistics (Continued) Prof. icholas Zabaras Center for Informatics and Computational Science https://cics.nd.edu/ University of otre Dame otre Dame, Indiana, USA Email:
More informationQuantile POD for Hit-Miss Data
Quantile POD for Hit-Miss Data Yew-Meng Koh a and William Q. Meeker a a Center for Nondestructive Evaluation, Department of Statistics, Iowa State niversity, Ames, Iowa 50010 Abstract. Probability of detection
More informationParameter Estimation. William H. Jefferys University of Texas at Austin Parameter Estimation 7/26/05 1
Parameter Estimation William H. Jefferys University of Texas at Austin bill@bayesrules.net Parameter Estimation 7/26/05 1 Elements of Inference Inference problems contain two indispensable elements: Data
More informationStatistical Practice
Statistical Practice A Note on Bayesian Inference After Multiple Imputation Xiang ZHOU and Jerome P. REITER This article is aimed at practitioners who plan to use Bayesian inference on multiply-imputed
More informationSTA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
More informationσ(a) = a N (x; 0, 1 2 ) dx. σ(a) = Φ(a) =
Until now we have always worked with likelihoods and prior distributions that were conjugate to each other, allowing the computation of the posterior distribution to be done in closed form. Unfortunately,
More informationBayesian inference for sample surveys. Roderick Little Module 2: Bayesian models for simple random samples
Bayesian inference for sample surveys Roderick Little Module : Bayesian models for simple random samples Superpopulation Modeling: Estimating parameters Various principles: least squares, method of moments,
More informationBayesian Modeling of Accelerated Life Tests with Random Effects
Bayesian Modeling of Accelerated Life Tests with Random Effects Ramón V. León Avery J. Ashby Jayanth Thyagarajan Joint Statistical Meeting August, 00 Toronto, Canada Abstract We show how to use Bayesian
More informationBayesian Inference in GLMs. Frequentists typically base inferences on MLEs, asymptotic confidence
Bayesian Inference in GLMs Frequentists typically base inferences on MLEs, asymptotic confidence limits, and log-likelihood ratio tests Bayesians base inferences on the posterior distribution of the unknowns
More informationStatistical techniques for data analysis in Cosmology
Statistical techniques for data analysis in Cosmology arxiv:0712.3028; arxiv:0911.3105 Numerical recipes (the bible ) Licia Verde ICREA & ICC UB-IEEC http://icc.ub.edu/~liciaverde outline Lecture 1: Introduction
More informationPARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.
PARAMETER ESTIMATION: BAYESIAN APPROACH. These notes summarize the lectures on Bayesian parameter estimation.. Beta Distribution We ll start by learning about the Beta distribution, since we end up using
More informationMultivariate statistical methods and data mining in particle physics
Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general
More informationWhen using physical experimental data to adjust, or calibrate, computer simulation models, two general
A Preposterior Analysis to Predict Identifiability in Experimental Calibration of Computer Models Paul D. Arendt Northwestern University, Department of Mechanical Engineering 2145 Sheridan Road Room B214
More informationInference when identifying assumptions are doubted. A. Theory B. Applications
Inference when identifying assumptions are doubted A. Theory B. Applications 1 A. Theory Structural model of interest: A y t B 1 y t1 B m y tm u t nn n1 u t i.i.d. N0, D D diagonal 2 Bayesian approach:
More informationTheory and Methods of Statistical Inference. PART I Frequentist theory and methods
PhD School in Statistics cycle XXVI, 2011 Theory and Methods of Statistical Inference PART I Frequentist theory and methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More informationPART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
More informationStatistical Methods in Particle Physics Lecture 1: Bayesian methods
Statistical Methods in Particle Physics Lecture 1: Bayesian methods SUSSP65 St Andrews 16 29 August 2009 Glen Cowan Physics Department Royal Holloway, University of London g.cowan@rhul.ac.uk www.pp.rhul.ac.uk/~cowan
More informationStatistical Methods in Particle Physics
Statistical Methods in Particle Physics Lecture 11 January 7, 2013 Silvia Masciocchi, GSI Darmstadt s.masciocchi@gsi.de Winter Semester 2012 / 13 Outline How to communicate the statistical uncertainty
More informationBayesian inference. Fredrik Ronquist and Peter Beerli. October 3, 2007
Bayesian inference Fredrik Ronquist and Peter Beerli October 3, 2007 1 Introduction The last few decades has seen a growing interest in Bayesian inference, an alternative approach to statistical inference.
More informationPrimer on statistics:
Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood
More informationTheory and Methods of Statistical Inference
PhD School in Statistics cycle XXIX, 2014 Theory and Methods of Statistical Inference Instructors: B. Liseo, L. Pace, A. Salvan (course coordinator), N. Sartori, A. Tancredi, L. Ventura Syllabus Some prerequisites:
More informationA BAYESIAN MATHEMATICAL STATISTICS PRIMER. José M. Bernardo Universitat de València, Spain
A BAYESIAN MATHEMATICAL STATISTICS PRIMER José M. Bernardo Universitat de València, Spain jose.m.bernardo@uv.es Bayesian Statistics is typically taught, if at all, after a prior exposure to frequentist
More informationFULL LIKELIHOOD INFERENCES IN THE COX MODEL
October 20, 2007 FULL LIKELIHOOD INFERENCES IN THE COX MODEL BY JIAN-JIAN REN 1 AND MAI ZHOU 2 University of Central Florida and University of Kentucky Abstract We use the empirical likelihood approach
More informationEffect of correlated observation error on parameters, predictions, and uncertainty
WATER RESOURCES RESEARCH, VOL. 49, 6339 6355, doi:10.1002/wrcr.20499, 2013 Effect of correlated observation error on parameters, predictions, and uncertainty Claire R. Tiedeman 1 and Christopher T. Green
More informationLecture 5. G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1
Lecture 5 1 Probability (90 min.) Definition, Bayes theorem, probability densities and their properties, catalogue of pdfs, Monte Carlo 2 Statistical tests (90 min.) general concepts, test statistics,
More informationA Note on Bayesian Inference After Multiple Imputation
A Note on Bayesian Inference After Multiple Imputation Xiang Zhou and Jerome P. Reiter Abstract This article is aimed at practitioners who plan to use Bayesian inference on multiplyimputed datasets in
More informationTheory and Methods of Statistical Inference. PART I Frequentist likelihood methods
PhD School in Statistics XXV cycle, 2010 Theory and Methods of Statistical Inference PART I Frequentist likelihood methods (A. Salvan, N. Sartori, L. Pace) Syllabus Some prerequisites: Empirical distribution
More informationBAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA
BAYESIAN METHODS FOR VARIABLE SELECTION WITH APPLICATIONS TO HIGH-DIMENSIONAL DATA Intro: Course Outline and Brief Intro to Marina Vannucci Rice University, USA PASI-CIMAT 04/28-30/2010 Marina Vannucci
More informationMachine Learning 4771
Machine Learning 4771 Instructor: Tony Jebara Topic 11 Maximum Likelihood as Bayesian Inference Maximum A Posteriori Bayesian Gaussian Estimation Why Maximum Likelihood? So far, assumed max (log) likelihood
More informationThe Bayesian Approach to Multi-equation Econometric Model Estimation
Journal of Statistical and Econometric Methods, vol.3, no.1, 2014, 85-96 ISSN: 2241-0384 (print), 2241-0376 (online) Scienpress Ltd, 2014 The Bayesian Approach to Multi-equation Econometric Model Estimation
More informationINVERTED KUMARASWAMY DISTRIBUTION: PROPERTIES AND ESTIMATION
Pak. J. Statist. 2017 Vol. 33(1), 37-61 INVERTED KUMARASWAMY DISTRIBUTION: PROPERTIES AND ESTIMATION A. M. Abd AL-Fattah, A.A. EL-Helbawy G.R. AL-Dayian Statistics Department, Faculty of Commerce, AL-Azhar
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationInference when identifying assumptions are doubted. A. Theory. Structural model of interest: B 1 y t1. u t. B m y tm. u t i.i.d.
Inference when identifying assumptions are doubted A. Theory B. Applications Structural model of interest: A y t B y t B m y tm nn n i.i.d. N, D D diagonal A. Theory Bayesian approach: Summarize whatever
More informationBayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features. Yangxin Huang
Bayesian Inference on Joint Mixture Models for Survival-Longitudinal Data with Multiple Features Yangxin Huang Department of Epidemiology and Biostatistics, COPH, USF, Tampa, FL yhuang@health.usf.edu January
More informationBayesian Inference and MCMC
Bayesian Inference and MCMC Aryan Arbabi Partly based on MCMC slides from CSC412 Fall 2018 1 / 18 Bayesian Inference - Motivation Consider we have a data set D = {x 1,..., x n }. E.g each x i can be the
More informationA Bayesian Treatment of Linear Gaussian Regression
A Bayesian Treatment of Linear Gaussian Regression Frank Wood December 3, 2009 Bayesian Approach to Classical Linear Regression In classical linear regression we have the following model y β, σ 2, X N(Xβ,
More informationProbing the covariance matrix
Probing the covariance matrix Kenneth M. Hanson Los Alamos National Laboratory (ret.) BIE Users Group Meeting, September 24, 2013 This presentation available at http://kmh-lanl.hansonhub.com/ LA-UR-06-5241
More informationParameter estimation and forecasting. Cristiano Porciani AIfA, Uni-Bonn
Parameter estimation and forecasting Cristiano Porciani AIfA, Uni-Bonn Questions? C. Porciani Estimation & forecasting 2 Temperature fluctuations Variance at multipole l (angle ~180o/l) C. Porciani Estimation
More informationBayesian Methods in Multilevel Regression
Bayesian Methods in Multilevel Regression Joop Hox MuLOG, 15 september 2000 mcmc What is Statistics?! Statistics is about uncertainty To err is human, to forgive divine, but to include errors in your design
More informationFundamental Probability and Statistics
Fundamental Probability and Statistics "There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are
More informationA MultiGaussian Approach to Assess Block Grade Uncertainty
A MultiGaussian Approach to Assess Block Grade Uncertainty Julián M. Ortiz 1, Oy Leuangthong 2, and Clayton V. Deutsch 2 1 Department of Mining Engineering, University of Chile 2 Department of Civil &
More informationCSC 2541: Bayesian Methods for Machine Learning
CSC 2541: Bayesian Methods for Machine Learning Radford M. Neal, University of Toronto, 2011 Lecture 10 Alternatives to Monte Carlo Computation Since about 1990, Markov chain Monte Carlo has been the dominant
More informationBasics of Uncertainty Analysis
Basics of Uncertainty Analysis Chapter Six Basics of Uncertainty Analysis 6.1 Introduction As shown in Fig. 6.1, analysis models are used to predict the performances or behaviors of a product under design.
More informationBivariate Degradation Modeling Based on Gamma Process
Bivariate Degradation Modeling Based on Gamma Process Jinglun Zhou Zhengqiang Pan Member IAENG and Quan Sun Abstract Many highly reliable products have two or more performance characteristics (PCs). The
More informationPIRLS 2016 Achievement Scaling Methodology 1
CHAPTER 11 PIRLS 2016 Achievement Scaling Methodology 1 The PIRLS approach to scaling the achievement data, based on item response theory (IRT) scaling with marginal estimation, was developed originally
More informationNew Bayesian methods for model comparison
Back to the future New Bayesian methods for model comparison Murray Aitkin murray.aitkin@unimelb.edu.au Department of Mathematics and Statistics The University of Melbourne Australia Bayesian Model Comparison
More informationNonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems
Nonlinear Model Reduction for Uncertainty Quantification in Large-Scale Inverse Problems Krzysztof Fidkowski, David Galbally*, Karen Willcox* (*MIT) Computational Aerospace Sciences Seminar Aerospace Engineering
More informationA Statistical Input Pruning Method for Artificial Neural Networks Used in Environmental Modelling
A Statistical Input Pruning Method for Artificial Neural Networks Used in Environmental Modelling G. B. Kingston, H. R. Maier and M. F. Lambert Centre for Applied Modelling in Water Engineering, School
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationBayesian Prediction of Code Output. ASA Albuquerque Chapter Short Course October 2014
Bayesian Prediction of Code Output ASA Albuquerque Chapter Short Course October 2014 Abstract This presentation summarizes Bayesian prediction methodology for the Gaussian process (GP) surrogate representation
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 7 Approximate
More informationUsing training sets and SVD to separate global 21-cm signal from foreground and instrument systematics
Using training sets and SVD to separate global 21-cm signal from foreground and instrument systematics KEITH TAUSCHER*, DAVID RAPETTI, JACK O. BURNS, ERIC SWITZER Aspen, CO Cosmological Signals from Cosmic
More informationImperfect Data in an Uncertain World
Imperfect Data in an Uncertain World James B. Elsner Department of Geography, Florida State University Tallahassee, Florida Corresponding author address: Dept. of Geography, Florida State University Tallahassee,
More informationML estimation: Random-intercepts logistic model. and z
ML estimation: Random-intercepts logistic model log p ij 1 p = x ijβ + υ i with υ i N(0, συ) 2 ij Standardizing the random effect, θ i = υ i /σ υ, yields log p ij 1 p = x ij β + σ υθ i with θ i N(0, 1)
More informationEstimation of reliability parameters from Experimental data (Parte 2) Prof. Enrico Zio
Estimation of reliability parameters from Experimental data (Parte 2) This lecture Life test (t 1,t 2,...,t n ) Estimate θ of f T t θ For example: λ of f T (t)= λe - λt Classical approach (frequentist
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationA Likelihood Ratio Test
A Likelihood Ratio Test David Allen University of Kentucky February 23, 2012 1 Introduction Earlier presentations gave a procedure for finding an estimate and its standard error of a single linear combination
More informationFlexible Regression Modeling using Bayesian Nonparametric Mixtures
Flexible Regression Modeling using Bayesian Nonparametric Mixtures Athanasios Kottas Department of Applied Mathematics and Statistics University of California, Santa Cruz Department of Statistics Brigham
More informationProbabilistic Machine Learning. Industrial AI Lab.
Probabilistic Machine Learning Industrial AI Lab. Probabilistic Linear Regression Outline Probabilistic Classification Probabilistic Clustering Probabilistic Dimension Reduction 2 Probabilistic Linear
More informationBayesian Dynamic Linear Modelling for. Complex Computer Models
Bayesian Dynamic Linear Modelling for Complex Computer Models Fei Liu, Liang Zhang, Mike West Abstract Computer models may have functional outputs. With no loss of generality, we assume that a single computer
More informationBayesian Inference. Chapter 1. Introduction and basic concepts
Bayesian Inference Chapter 1. Introduction and basic concepts M. Concepción Ausín Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master
More informationCE 3710: Uncertainty Analysis in Engineering
FINAL EXAM Monday, December 14, 10:15 am 12:15 pm, Chem Sci 101 Open book and open notes. Exam will be cumulative, but emphasis will be on material covered since Exam II Learning Expectations for Final
More informationOutline Lecture 2 2(32)
Outline Lecture (3), Lecture Linear Regression and Classification it is our firm belief that an understanding of linear models is essential for understanding nonlinear ones Thomas Schön Division of Automatic
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationReliability Monitoring Using Log Gaussian Process Regression
COPYRIGHT 013, M. Modarres Reliability Monitoring Using Log Gaussian Process Regression Martin Wayne Mohammad Modarres PSA 013 Center for Risk and Reliability University of Maryland Department of Mechanical
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationOn the Optimal Scaling of the Modified Metropolis-Hastings algorithm
On the Optimal Scaling of the Modified Metropolis-Hastings algorithm K. M. Zuev & J. L. Beck Division of Engineering and Applied Science California Institute of Technology, MC 4-44, Pasadena, CA 925, USA
More informationBest Linear Unbiased Prediction: an Illustration Based on, but Not Limited to, Shelf Life Estimation
Libraries Conference on Applied Statistics in Agriculture 015-7th Annual Conference Proceedings Best Linear Unbiased Prediction: an Illustration Based on, but Not Limited to, Shelf Life Estimation Maryna
More informationPractical Bayesian Quantile Regression. Keming Yu University of Plymouth, UK
Practical Bayesian Quantile Regression Keming Yu University of Plymouth, UK (kyu@plymouth.ac.uk) A brief summary of some recent work of us (Keming Yu, Rana Moyeed and Julian Stander). Summary We develops
More informationFast Likelihood-Free Inference via Bayesian Optimization
Fast Likelihood-Free Inference via Bayesian Optimization Michael Gutmann https://sites.google.com/site/michaelgutmann University of Helsinki Aalto University Helsinki Institute for Information Technology
More informationStatistics for the LHC Lecture 2: Discovery
Statistics for the LHC Lecture 2: Discovery Academic Training Lectures CERN, 14 17 June, 2010 indico.cern.ch/conferencedisplay.py?confid=77830 Glen Cowan Physics Department Royal Holloway, University of
More informationResearch Article A Nonparametric Two-Sample Wald Test of Equality of Variances
Advances in Decision Sciences Volume 211, Article ID 74858, 8 pages doi:1.1155/211/74858 Research Article A Nonparametric Two-Sample Wald Test of Equality of Variances David Allingham 1 andj.c.w.rayner
More informationObtaining Uncertainty Measures on Slope and Intercept
Obtaining Uncertainty Measures on Slope and Intercept of a Least Squares Fit with Excel s LINEST Faith A. Morrison Professor of Chemical Engineering Michigan Technological University, Houghton, MI 39931
More informationApproximate Bayesian computation for spatial extremes via open-faced sandwich adjustment
Approximate Bayesian computation for spatial extremes via open-faced sandwich adjustment Ben Shaby SAMSI August 3, 2010 Ben Shaby (SAMSI) OFS adjustment August 3, 2010 1 / 29 Outline 1 Introduction 2 Spatial
More informationA novel determination of the local dark matter density. Riccardo Catena. Institut für Theoretische Physik, Heidelberg
A novel determination of the local dark matter density Riccardo Catena Institut für Theoretische Physik, Heidelberg 28.04.2010 R. Catena and P. Ullio, arxiv:0907.0018 [astro-ph.co]. Riccardo Catena (ITP)
More informationModelling Operational Risk Using Bayesian Inference
Pavel V. Shevchenko Modelling Operational Risk Using Bayesian Inference 4y Springer 1 Operational Risk and Basel II 1 1.1 Introduction to Operational Risk 1 1.2 Defining Operational Risk 4 1.3 Basel II
More informationDavid Giles Bayesian Econometrics
David Giles Bayesian Econometrics 1. General Background 2. Constructing Prior Distributions 3. Properties of Bayes Estimators and Tests 4. Bayesian Analysis of the Multiple Regression Model 5. Bayesian
More information(5) Multi-parameter models - Gibbs sampling. ST440/540: Applied Bayesian Analysis
Summarizing a posterior Given the data and prior the posterior is determined Summarizing the posterior gives parameter estimates, intervals, and hypothesis tests Most of these computations are integrals
More informationTutorial 4: Power and Sample Size for the Two-sample t-test with Unequal Variances
Tutorial 4: Power and Sample Size for the Two-sample t-test with Unequal Variances Preface Power is the probability that a study will reject the null hypothesis. The estimated probability is a function
More information