Bayesian sensitivity analysis of a cardiac cell model using a Gaussian process emulator Supporting information
|
|
- Piers Shepherd
- 6 years ago
- Views:
Transcription
1 Bayesian sensitivity analysis of a cardiac cell model using a Gaussian process emulator Supporting information E T Y Chang 1,2, M Strong 3 R H Clayton 1,2, 1 Insigneo Institute for in-silico Medicine, University of Sheffield, Sheffield, UK. 2 Department of Computer Science, University of Sheffield, Sheffield, UK. 3 School of Health and Related Research, University of Sheffield, Sheffield, UK. r.h.clayton@sheffield.ac.uk Introduction In this supporting information, we include details of the mathematics that underpin the approach taken in this study. A much more in-depth coverage of Gaussian process GP) emulators is given in the MUCM webpages and the toolkit that has been developed there page=metahomepage.html. Our aim here is to describe the pathway through this material that was followed for the present study. We have used notation that is consistent with the MUCM pages. Vector and matrix quantities are indicated by bold type, and posterior estimates given design data are indicated with an asterisk. Emulator construction Each emulator was described by a GP, composed of a mean function and a covariance function, GPx) = hx) T β + σ 2 cx, x ). 1) We used a linear form for the mean function with hx) = 1, x) T, hx) T β = β 0 + β 1 x β P x P, 2) and a Gaussian form for the covariance [ P { } ] 2 xp,1 x p,2 ) cx 1, x 2 ) = exp. 3) p=1 δ p 1
2 This choice enabled direct calculation of variance based sensitivity indices. In these expressions x = x 1, x 2,..., x P ) are P inputs parameters), the emulator hyperparameters β and δ are vectors of length P, and σ 2 is a scalar. These quantities were obtained by fitting to the design data, assuming weak prior information on β and σ 2 [2]. In this approach, we first obtained a best estimate for δ, ˆδ, from the maximum of the posterior log likelihood given the design data D and outputs fd), and and a prior estimate of δ, δ 0. Best estimates of β and σ 2, were then obtained from ˆδ. The value for ˆδ was chosen to be the value that optimised the posterior distribution πδ, given the design data D and fd), and assuming a prior distribution of δ, π δ δ) ˆσ 2 ) n q)/2 A 1/2 H T AH 1/2 4) Where N was the number of design points 200), Q = P + 1, H an N Q matrix given by H = [hx 1 ), hx 2 ),..., hx N )] T, 5) A an N N matrix 1 cx 1, x 2 ) cx 1, x N ) cx 2, x 1 ) 1. A =., 6)... cx N, x 1 ) 1 c the covariance function given by Equation 3, x 1, x 2,..., x N rows of the design data D, and ˆσ 2 = 1 N Q 2) fd)t { A 1 A 1 H H T A 1 H ) 1 H T A 1} fd). 7) The value of ˆδ was obtained by maximising equation 4 using the design data as described in the implementation details below. This enabled ˆσ 2 to be calculated from equation 7, and ˆβ was then calculated from ˆβ = H T A 1 H ) 1 H T A 1 fd), 8) The set of hyper parameters ˆδ, ˆσ 2, and ˆβ, together with the mean function given by equation 2 and the covariance function equation 3 specified each emulator. 2
3 Emulator validation Once the emulators were constructed, they could be used to estimate the output e.g. AP D 90 ) for an input x using the posterior mean and variance of the emulator output given the design data. An asterisk is used to denote a posterior output,. The posterior mean of each emulator output was given by a linear combination of the inputs modified by a term describing the fit to the design data m x) = hx) T ˆβ + cx) T A 1 fd) H ˆβ ), 9) and the posterior variance by v x, x ) = ˆσ 2{ cx, x ) cx) T A 1 cx ) + hx) T cx) T A 1 H ) H T A 1 H ) 1 hx ) T cx ) T A 1 H ) T }. 10) In these equations the linear mean function hx) was given by equation 2, the covariance matrix cx) that depends on ˆδ by equation 3, and the matrices H and A by equations 5 and 6 respectively. These functions were used to validate the emulators against test data D t and corresponding outputs fd t ) obtained from N t = 20 additional runs of the simulator. For each row j = 1,..., N T of D t with test inputs x t j, and test output yt j = fxt j ), the standardised error between the predicted emulator output m x t ) and the observed simulator output y t j was calculated from E j = yt j m x t j ) v x t j, xt j ). 11) The Mahanalobis distance for the complete set of test data was a measure of overall agreement between the predicted and test data, and was calculated from MD = fd t ) m ) T V ) 1 fd t ) m ), 12) where m was an N t 1 vector m x t 1), m x t 2),..., m x t N t )), and V is an N t N t matrix, where each element V i, j) is v x t i, xt j ). The theoretical distribution for MD has a mean of N t and a variance of 2N t N t + N Q 2)/N Q 4) [1]. 3
4 Following validation, a new version of each emulator was built by combining the design data {D, fd)} and test data {D t, fd t )} and using the N + N t = 220 sets of input and output data to obtain updated values of the emulator hyperparameters ˆδ, ˆβ and ˆσ2. The updated emulator was used for all subsequent analysis. Uncertainty in emulator output If the probability density function of uncertain inputs is given by ωx), then the posterior expectation of the emulator output with random inputs X is E [E[fX)]] = m x)ωx) dx, 13) and the variance of this expectation was V ar [E[fX)]] = v x, x )ωx)ωx ) dxdx. 14) The expected variance of the emulator output was given by E [V ar[fx)]] = I 1 V ar [E[fx))]) + I 2 E [E[fx))]) 2 ) 15) where I 1 = v x, x)ωx) dx 16) and I 2 = m x) 2 ωx) dx. 17) Our choice of linear mean, weak prior, and Gaussian correlation function for the GP enabled the direct calculation of these integrals, provided that the probability density function of the inputs ωx) was multivariate Gaussian with specified mean m and variance var. The expectation of the emulator output was then 4
5 E [E[fx))] = R h T ˆβ + Rt T e, 18) Where e was an N 1 vector e = A 1 fd H ˆβ) 19) R h a 1 Q vector R h = 1, m), 20) where m was a 1 P vector of the mean values of each input, and R t a 1 N vector, where the k th element of N) was given by R t k) = B 1/2 2C + B 1/2 Qk m k exp ) ). 21) 2 In this expression C was a diagonal P P prior correlation matrix where Ci, j) = ˆδi) for i = j, and Ci, j) = 0 otherwise, B the precision matrix of the input distribution ωx), the inverse of a diagonal covariance matrix CV where CV i, j) = vari) for i = j, and Ci, j) = 0 otherwise, and vari) was the i th element of a P 1 variance vector var corresponding to the variance in input i. The function Q k m k ) yielded a single scalar Q k m k) = 2 m k x k ) T C m k x k ) + m k m) T B m k m) 22) where m k was a P 1 vector given by m k = 2C + B) 1 2Cx k + Bm), 23) and where x k was the k th row of inputs with P elements) in the design data used to build the emulator. The variance of the expectation of the emulator output V ar [E[fx]] was given by V ar [E[fx)]] = ˆσ 2 [ U R T t A 1 R t + R h G T R t ) T WR h G T R t ) ]. 24) In this expression U = B B2 1/2 25) 5
6 where B2 was a 2P 2P matrix, B21...P, 1...P ) = 2C + B, B2P P, P P ) = 2C + B, B21...P, P P ) = 2C, B2P P, 1...P ) = 2C, 26) G an N Q matrix, G = A 1 H, 27) and W a Q Q matrix, W = H T A 1 H. 28) The expectation of the variance of the emulator output E [V ar[fx]] could then be calculated using equation 15, where I 1 = ˆσ 2 [ 1 trace A 1 R tt ) + trace W Rhh 2R ht G + G T R t tg ))] 29) and I 2 = ˆβ T R hh ˆβ + 2 ˆβT R ht e + e T R tt e. 30) In these expressions R tt was an N N matrix, where entry k, l) was given by R tt k, l) = B 1/2 4C + B 1/2 exp Q kl m kl)/2), 31) where m kl was a P 1 vector m kl = 4C + B) 1 2Cx k + 2Cx l + Bm), 32) 6
7 and Q kl m kl) a scalar given by Q kl m kl) = 2m kl x k ) T Cm kl x k ) + 2m kl x l ) T Cm kl x l ) +m kl m) T Bm kl m). 33) R ht was a Q N matrix, where the k th column was given by R ht k) = R t k)f, 34) where F was a Q 1 vector with entries 1, m k1)...m kp )), where m k was given by equation 23. Finally, R hh was a Q Q matrix R h h = R T h R h, where R h was given by equation 20. In the manuscript, Table 2 shows E [E[fx))], V ar [E[fx))], and E [V arfx))]/e [E[fx)]] 100 for each of the eight emulators, obtained by setting the mean vector m for the inputs to be 0.5, 0.5, 0.5, 0.5, 0.5, 0.5), and the variance vector var to be 0.04, 0.04, 0.04, 0.04, 0.04, 0.04). The distributions of AP D 90 in Figure 5a) were obtained from E [E[fx))] and E [V arfx))] calculated using an identical mean vector with all entries set to 0.5, and a variance vector where all elements were set to except for the element corresponding to G K, which was set to 0.01, 0.02, 0.05, and 0.1. Calculation of mean effects The mean effect M w x w ) shows how the emulator output averaged over uncertain inputs changes when input x w is given a fixed value. The posterior expectation of M w x w ) is a scalar, and is given by M w x w ) = R w ˆβ + Tw e. 35) This equation had the same form as equation 18. R w was a 1 Q vector with R w 1) = 1, and R w i) = x w for i = w, and the mean value of input i mi) otherwise for i = 2...Q. T w was a 1 N vector, where the k th element was given by 7
8 [ T w k) = i w B 1/2 ii exp 2C ii + B ii ) 1/2 exp 1 ) ) ] 2Cii B ii x i,k m i ) 2 2 2C ii + B ii ) 1 2 x w x w,k ) T 2C ww x w x w,k ), 36) where x i/w,k was the value of the i/w th of P ) input on the k th of N) of the design data used to build the emulator, and all other quantities are defined above. Calculation of sensitivity indices from the GP emulators For each emulator, the sensitivity index of each input was calculated from E [V w ]/E [V ar[fx)]]. The total variance in emulator output E [V ar[fx)]]was calculated as described in equation 15, and the variance in the emulator output when input x w was fixed E [V w ], the sensitivity variance, was calculated from E [V w ] = E [E[E[fx x w )] 2 ]] E [E[fx)] 2 ]. 37) The first term in this equation was given by E [E[E[fx x w )] 2 ]] = ˆσ 2 { U w trace [ W Q w S w A 1 H H T A 1 S T w + H T A 1 P w A 1 H )]} + e T P w e + 2 ˆβ T S w e + ˆβ T Q w ˆβ, 38) where U w was a scalar U w = ) 1/2 B ii, 39) B ii + 4C ii i w P w was an N N matrix, where the k, l) th element was 8
9 [ B ii P w k, l) = 2C ii + B ii i w exp 1 ) 2Cii B ii [xi,k m i ) 2 + x i,l m i ) 2])] 2 2C ii + B ii { ) 1/2 B ww exp 1 4C ww + B ww 2 1 4C ww + B ww ) [4C 2 wwx w,k x w,l ) 2 + 2C ww B ww xw,k m w ) 2 + x w,l m w ) 2) ])} 40) Q w is a Q Q matrix, which was assembled in the following steps Q w 1, 1) = 1 Q w 1, 2...Q) = m1...p ) Q w 2...Q, 1) = m1...p ) Q w 2...Q, 2...Q) = m m T Q w w + 1, w + 1) = Q w w + 1, w + 1) + B 1 ww, 41) and S w a Q N matrix, where the k, l) th element was S w k, l) = E[h k x)] 1 i P ) B 1/2 [ ii exp 1 )] 2Cii B ii x 2C ii B ii ) 1/2 i,l m i ) 2 2 2C ii + B ii 1 if k = 1 E[h k x)] = if k w m k 42) 43) 2C kk x k,l +B kk m k 2C kk +B kk if k = w Implementation All of the code for this study was implemented in Matlab, using expressions detailed in the MUCM toolkit The code was tested against the numerical examples provided in the toolkit. Each emulator was fitted using design data and outputs obtained from N = 200 runs of the LR1991 9
10 simulator, so for each emulator D was an N P 200 3) matrix, and fd) an N ) vector. The design data were used to construct H. An initial estimate of σ 2 was then obtained from equation 7. Following the MUCM toolkit, we re-parameterised equation 4 with τ = 2 log e δ). This removes a constraint on the optimisation because δ ranges from 0 whereas τ can range from. Matrix A was thus calculated from equations 6 and 3 with δ = expτ/2), and the Nelder-Mead algorithm as implemented in the Matlab function fminsearch was then used to find the minimum of π τ) = π δ expτ/2)) 44) with a tolerance of 10 5 and a maximum number of iterations set to A vector of initial values for the correlation length hyperparameters, δ 0, was required for the Nelder-Mead minimisation. We found the fit of the emulator to the test data was sensitive to the choice of δ 0. Good sets of initial estimates found by trial and error) were 0.1, 1.0, 1.0, 1.0, 1.0, 1.0) for the emulator for maximum dv /dt, 0.1, 1.0, 1.0, 0.1, 1.0, 1.0) for peak V m, the variance of each column of D for the emulators for dome voltage, AP D 90, and resting voltage, and 1.0, 1.0, 1.0, 1.0, 1.0, 1.0) for the remaining emulators. References [1] Leonardo S. Bastos and Anthony OHagan. Diagnostics for Gaussian Process Emulators. Technometrics, 514): , November [2] M. C. Kennedy and A O Hagan. Predicting the output from a complex computer code when fast approximations are available. Biometrika, 87:1 13,
Gaussian Processes for Computer Experiments
Gaussian Processes for Computer Experiments Jeremy Oakley School of Mathematics and Statistics, University of Sheffield www.jeremy-oakley.staff.shef.ac.uk 1 / 43 Computer models Computer model represented
More informationIntroduction to emulators - the what, the when, the why
School of Earth and Environment INSTITUTE FOR CLIMATE & ATMOSPHERIC SCIENCE Introduction to emulators - the what, the when, the why Dr Lindsay Lee 1 What is a simulator? A simulator is a computer code
More informationParameter estimation and prediction using Gaussian Processes
Parameter estimation and prediction using Gaussian Processes Yiannis Andrianakis and Peter G. Challenor August, 9 Abstract This is a report on methods of parameter estimation and prediction for Gaussian
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
More informationRecursive Estimation
Recursive Estimation Raffaello D Andrea Spring 08 Problem Set 3: Extracting Estimates from Probability Distributions Last updated: April 9, 08 Notes: Notation: Unless otherwise noted, x, y, and z denote
More informationLearning Gaussian Process Models from Uncertain Data
Learning Gaussian Process Models from Uncertain Data Patrick Dallaire, Camille Besse, and Brahim Chaib-draa DAMAS Laboratory, Computer Science & Software Engineering Department, Laval University, Canada
More informationUncertainty in energy system models
Uncertainty in energy system models Amy Wilson Durham University May 2015 Table of Contents 1 Model uncertainty 2 3 Example - generation investment 4 Conclusion Model uncertainty Contents 1 Model uncertainty
More informationCOM336: Neural Computing
COM336: Neural Computing http://www.dcs.shef.ac.uk/ sjr/com336/ Lecture 2: Density Estimation Steve Renals Department of Computer Science University of Sheffield Sheffield S1 4DP UK email: s.renals@dcs.shef.ac.uk
More informationData assimilation with and without a model
Data assimilation with and without a model Tim Sauer George Mason University Parameter estimation and UQ U. Pittsburgh Mar. 5, 2017 Partially supported by NSF Most of this work is due to: Tyrus Berry,
More informationData assimilation with and without a model
Data assimilation with and without a model Tyrus Berry George Mason University NJIT Feb. 28, 2017 Postdoc supported by NSF This work is in collaboration with: Tim Sauer, GMU Franz Hamilton, Postdoc, NCSU
More informationAn adaptive kriging method for characterizing uncertainty in inverse problems
Int Statistical Inst: Proc 58th World Statistical Congress, 2, Dublin Session STS2) p98 An adaptive kriging method for characterizing uncertainty in inverse problems FU Shuai 2 University Paris-Sud & INRIA,
More information(Extended) Kalman Filter
(Extended) Kalman Filter Brian Hunt 7 June 2013 Goals of Data Assimilation (DA) Estimate the state of a system based on both current and all past observations of the system, using a model for the system
More informationUnderstanding the uncertainty in the biospheric carbon flux for England and Wales
Understanding the uncertainty in the biospheric carbon flux for England and Wales John Paul Gosling and Anthony O Hagan 30th November, 2007 Abstract Uncertainty analysis is the evaluation of the distribution
More informationPILCO: A Model-Based and Data-Efficient Approach to Policy Search
PILCO: A Model-Based and Data-Efficient Approach to Policy Search (M.P. Deisenroth and C.E. Rasmussen) CSC2541 November 4, 2016 PILCO Graphical Model PILCO Probabilistic Inference for Learning COntrol
More information1 Data Arrays and Decompositions
1 Data Arrays and Decompositions 1.1 Variance Matrices and Eigenstructure Consider a p p positive definite and symmetric matrix V - a model parameter or a sample variance matrix. The eigenstructure is
More informationEstimation Theory. as Θ = (Θ 1,Θ 2,...,Θ m ) T. An estimator
Estimation Theory Estimation theory deals with finding numerical values of interesting parameters from given set of data. We start with formulating a family of models that could describe how the data were
More informationVariational Principal Components
Variational Principal Components Christopher M. Bishop Microsoft Research 7 J. J. Thomson Avenue, Cambridge, CB3 0FB, U.K. cmbishop@microsoft.com http://research.microsoft.com/ cmbishop In Proceedings
More informationBayesian Dynamic Linear Modelling for. Complex Computer Models
Bayesian Dynamic Linear Modelling for Complex Computer Models Fei Liu, Liang Zhang, Mike West Abstract Computer models may have functional outputs. With no loss of generality, we assume that a single computer
More informationKalman Filtering in a Mass-Spring System
Department of Mathematics Center for Quantitative Sciences in Biomedicine North Carolina State University July 30, 2016 Practical Overview Goal: Implement Kalman filter for linear system 1. Solve a linear
More informationGlobal Optimisation with Gaussian Processes. Michael A. Osborne Machine Learning Research Group Department o Engineering Science University o Oxford
Global Optimisation with Gaussian Processes Michael A. Osborne Machine Learning Research Group Department o Engineering Science University o Oxford Global optimisation considers objective functions that
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationAn EM algorithm for Gaussian Markov Random Fields
An EM algorithm for Gaussian Markov Random Fields Will Penny, Wellcome Department of Imaging Neuroscience, University College, London WC1N 3BG. wpenny@fil.ion.ucl.ac.uk October 28, 2002 Abstract Lavine
More informationLecture 4: Dynamic models
linear s Lecture 4: s Hedibert Freitas Lopes The University of Chicago Booth School of Business 5807 South Woodlawn Avenue, Chicago, IL 60637 http://faculty.chicagobooth.edu/hedibert.lopes hlopes@chicagobooth.edu
More informationBig model configuration with Bayesian quadrature. David Duvenaud, Roman Garnett, Tom Gunter, Philipp Hennig, Michael A Osborne and Stephen Roberts.
Big model configuration with Bayesian quadrature David Duvenaud, Roman Garnett, Tom Gunter, Philipp Hennig, Michael A Osborne and Stephen Roberts. This talk will develop Bayesian quadrature approaches
More informationAn introduction to Bayesian statistics and model calibration and a host of related topics
An introduction to Bayesian statistics and model calibration and a host of related topics Derek Bingham Statistics and Actuarial Science Simon Fraser University Cast of thousands have participated in the
More informationAdaptive ensemble Kalman filtering of nonlinear systems
Adaptive ensemble Kalman filtering of nonlinear systems Tyrus Berry George Mason University June 12, 213 : Problem Setup We consider a system of the form: x k+1 = f (x k ) + ω k+1 ω N (, Q) y k+1 = h(x
More informationPart IB Statistics. Theorems with proof. Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua. Lent 2015
Part IB Statistics Theorems with proof Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
More informationMachine Learning - MT & 5. Basis Expansion, Regularization, Validation
Machine Learning - MT 2016 4 & 5. Basis Expansion, Regularization, Validation Varun Kanade University of Oxford October 19 & 24, 2016 Outline Basis function expansion to capture non-linear relationships
More information6.4 Kalman Filter Equations
6.4 Kalman Filter Equations 6.4.1 Recap: Auxiliary variables Recall the definition of the auxiliary random variables x p k) and x m k): Init: x m 0) := x0) S1: x p k) := Ak 1)x m k 1) +uk 1) +vk 1) S2:
More informationConvergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit
Convergence of Square Root Ensemble Kalman Filters in the Large Ensemble Limit Evan Kwiatkowski, Jan Mandel University of Colorado Denver December 11, 2014 OUTLINE 2 Data Assimilation Bayesian Estimation
More informationStatistics 910, #15 1. Kalman Filter
Statistics 910, #15 1 Overview 1. Summary of Kalman filter 2. Derivations 3. ARMA likelihoods 4. Recursions for the variance Kalman Filter Summary of Kalman filter Simplifications To make the derivations
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationModelling and Control of Nonlinear Systems using Gaussian Processes with Partial Model Information
5st IEEE Conference on Decision and Control December 0-3, 202 Maui, Hawaii, USA Modelling and Control of Nonlinear Systems using Gaussian Processes with Partial Model Information Joseph Hall, Carl Rasmussen
More informationState Estimation with a Kalman Filter
State Estimation with a Kalman Filter When I drive into a tunnel, my GPS continues to show me moving forward, even though it isn t getting any new position sensing data How does it work? A Kalman filter
More information5.2 Expounding on the Admissibility of Shrinkage Estimators
STAT 383C: Statistical Modeling I Fall 2015 Lecture 5 September 15 Lecturer: Purnamrita Sarkar Scribe: Ryan O Donnell Disclaimer: These scribe notes have been slightly proofread and may have typos etc
More informationarxiv: v1 [stat.me] 24 May 2010
The role of the nugget term in the Gaussian process method Andrey Pepelyshev arxiv:1005.4385v1 [stat.me] 24 May 2010 Abstract The maximum likelihood estimate of the correlation parameter of a Gaussian
More informationPreliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com
1 School of Oriental and African Studies September 2015 Department of Economics Preliminary Statistics Lecture 2: Probability Theory (Outline) prelimsoas.webs.com Gujarati D. Basic Econometrics, Appendix
More informationAppendix of the paper: Are interest rate options important for the assessment of interest rate risk?
Appendix of the paper: Are interest rate options important for the assessment of interest rate risk? Caio Almeida,a, José Vicente b a Graduate School of Economics, Getulio Vargas Foundation b Research
More informationProbabilistic & Unsupervised Learning
Probabilistic & Unsupervised Learning Gaussian Processes Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London
More informationLecture 10: Parameter Estimation for ODE Models
Computational Biology Group (CoBI), D-BSSE, ETHZ Lecture 10: Parameter Estimation for ODE Models Prof Dagmar Iber, PhD DPhil BSc Biotechnology 2016/17 December 7, 2016 2 / 93 Contents 1 Review of Previous
More informationBayes Decision Theory
Bayes Decision Theory Minimum-Error-Rate Classification Classifiers, Discriminant Functions and Decision Surfaces The Normal Density 0 Minimum-Error-Rate Classification Actions are decisions on classes
More informationThe Variational Kalman Smoother
The Variational Kalman Smoother Matthew J. Beal and Zoubin Ghahramani Gatsby Computational Neuroscience Unit {m.beal,zoubin}@gatsby.ucl.ac.uk http://www.gatsby.ucl.ac.uk May 22, 2000. last revision April
More informationPREDICTION FROM THE DYNAMIC SIMULTANEOUS EQUATION MODEL WITH VECTOR AUTOREGRESSIVE ERRORS
Econometrica, Vol. 49, No. 5 (September, 1981) PREDICTION FROM THE DYNAMIC SIMULTANEOUS EQUATION MODEL WITH VECTOR AUTOREGRESSIVE ERRORS BY RICHARD T. BAILLIE' The asymptotic distribution of prediction
More informationEE 565: Position, Navigation, and Timing
EE 565: Position, Navigation, and Timing Kalman Filtering Example Aly El-Osery Kevin Wedeward Electrical Engineering Department, New Mexico Tech Socorro, New Mexico, USA In Collaboration with Stephen Bruder
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 254 Part V
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationComputer Vision Group Prof. Daniel Cremers. 9. Gaussian Processes - Regression
Group Prof. Daniel Cremers 9. Gaussian Processes - Regression Repetition: Regularized Regression Before, we solved for w using the pseudoinverse. But: we can kernelize this problem as well! First step:
More informationMATRICES. a m,1 a m,n A =
MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of
More informationENGR352 Problem Set 02
engr352/engr352p02 September 13, 2018) ENGR352 Problem Set 02 Transfer function of an estimator 1. Using Eq. (1.1.4-27) from the text, find the correct value of r ss (the result given in the text is incorrect).
More informationDiagnostics for Linear Models With Functional Responses
Diagnostics for Linear Models With Functional Responses Qing Shen Edmunds.com Inc. 2401 Colorado Ave., Suite 250 Santa Monica, CA 90404 (shenqing26@hotmail.com) Hongquan Xu Department of Statistics University
More informationPhysics 403. Segev BenZvi. Parameter Estimation, Correlations, and Error Bars. Department of Physics and Astronomy University of Rochester
Physics 403 Parameter Estimation, Correlations, and Error Bars Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Best Estimates and Reliability
More informationGenerative classifiers: The Gaussian classifier. Ata Kaban School of Computer Science University of Birmingham
Generative classifiers: The Gaussian classifier Ata Kaban School of Computer Science University of Birmingham Outline We have already seen how Bayes rule can be turned into a classifier In all our examples
More informationCovariance function estimation in Gaussian process regression
Covariance function estimation in Gaussian process regression François Bachoc Department of Statistics and Operations Research, University of Vienna WU Research Seminar - May 2015 François Bachoc Gaussian
More informationBayesian Inference for the Multivariate Normal
Bayesian Inference for the Multivariate Normal Will Penny Wellcome Trust Centre for Neuroimaging, University College, London WC1N 3BG, UK. November 28, 2014 Abstract Bayesian inference for the multivariate
More informationPrinciples of DCM. Will Penny. 26th May Principles of DCM. Will Penny. Introduction. Differential Equations. Bayesian Estimation.
26th May 2011 Dynamic Causal Modelling Dynamic Causal Modelling is a framework studying large scale brain connectivity by fitting differential equation models to brain imaging data. DCMs differ in their
More informationUncertainty propagation in a sequential model for flood forecasting
Predictions in Ungauged Basins: Promise and Progress (Proceedings of symposium S7 held during the Seventh IAHS Scientific Assembly at Foz do Iguaçu, Brazil, April 2005). IAHS Publ. 303, 2006. 177 Uncertainty
More informationMIXTURE MODELS AND EM
Last updated: November 6, 212 MIXTURE MODELS AND EM Credits 2 Some of these slides were sourced and/or modified from: Christopher Bishop, Microsoft UK Simon Prince, University College London Sergios Theodoridis,
More informationComputer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression
Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.
More informationAMS-207: Bayesian Statistics
Linear Regression How does a quantity y, vary as a function of another quantity, or vector of quantities x? We are interested in p(y θ, x) under a model in which n observations (x i, y i ) are exchangeable.
More informationCS 532: 3D Computer Vision 6 th Set of Notes
1 CS 532: 3D Computer Vision 6 th Set of Notes Instructor: Philippos Mordohai Webpage: www.cs.stevens.edu/~mordohai E-mail: Philippos.Mordohai@stevens.edu Office: Lieb 215 Lecture Outline Intro to Covariance
More informationIntroduction to Gaussian Processes
Introduction to Gaussian Processes Neil D. Lawrence GPSS 10th June 2013 Book Rasmussen and Williams (2006) Outline The Gaussian Density Covariance from Basis Functions Basis Function Representations Constructing
More informationLabor-Supply Shifts and Economic Fluctuations. Technical Appendix
Labor-Supply Shifts and Economic Fluctuations Technical Appendix Yongsung Chang Department of Economics University of Pennsylvania Frank Schorfheide Department of Economics University of Pennsylvania January
More informationMachine learning - HT Maximum Likelihood
Machine learning - HT 2016 3. Maximum Likelihood Varun Kanade University of Oxford January 27, 2016 Outline Probabilistic Framework Formulate linear regression in the language of probability Introduce
More informationMultivariate Bayesian Linear Regression MLAI Lecture 11
Multivariate Bayesian Linear Regression MLAI Lecture 11 Neil D. Lawrence Department of Computer Science Sheffield University 21st October 2012 Outline Univariate Bayesian Linear Regression Multivariate
More informationEfficiency and Reliability of Bayesian Calibration of Energy Supply System Models
Efficiency and Reliability of Bayesian Calibration of Energy Supply System Models Kathrin Menberg 1,2, Yeonsook Heo 2, Ruchi Choudhary 1 1 University of Cambridge, Department of Engineering, Cambridge,
More informationvariability of the model, represented by σ 2 and not accounted for by Xβ
Posterior Predictive Distribution Suppose we have observed a new set of explanatory variables X and we want to predict the outcomes ỹ using the regression model. Components of uncertainty in p(ỹ y) variability
More informationMATH5745 Multivariate Methods Lecture 07
MATH5745 Multivariate Methods Lecture 07 Tests of hypothesis on covariance matrix March 16, 2018 MATH5745 Multivariate Methods Lecture 07 March 16, 2018 1 / 39 Test on covariance matrices: Introduction
More informationSupplementary Note on Bayesian analysis
Supplementary Note on Bayesian analysis Structured variability of muscle activations supports the minimal intervention principle of motor control Francisco J. Valero-Cuevas 1,2,3, Madhusudhan Venkadesan
More informationRidge Regression Revisited
Ridge Regression Revisited Paul M.C. de Boer Christian M. Hafner Econometric Institute Report EI 2005-29 In general ridge (GR) regression p ridge parameters have to be determined, whereas simple ridge
More informationHierarchy. Will Penny. 24th March Hierarchy. Will Penny. Linear Models. Convergence. Nonlinear Models. References
24th March 2011 Update Hierarchical Model Rao and Ballard (1999) presented a hierarchical model of visual cortex to show how classical and extra-classical Receptive Field (RF) effects could be explained
More informationA new Hierarchical Bayes approach to ensemble-variational data assimilation
A new Hierarchical Bayes approach to ensemble-variational data assimilation Michael Tsyrulnikov and Alexander Rakitko HydroMetCenter of Russia College Park, 20 Oct 2014 Michael Tsyrulnikov and Alexander
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationPh.D. Qualifying Exam Monday Tuesday, January 4 5, 2016
Ph.D. Qualifying Exam Monday Tuesday, January 4 5, 2016 Put your solution to each problem on a separate sheet of paper. Problem 1. (5106) Find the maximum likelihood estimate of θ where θ is a parameter
More informationNonparametric Drift Estimation for Stochastic Differential Equations
Nonparametric Drift Estimation for Stochastic Differential Equations Gareth Roberts 1 Department of Statistics University of Warwick Brazilian Bayesian meeting, March 2010 Joint work with O. Papaspiliopoulos,
More informationThe Bayesian approach to inverse problems
The Bayesian approach to inverse problems Youssef Marzouk Department of Aeronautics and Astronautics Center for Computational Engineering Massachusetts Institute of Technology ymarz@mit.edu, http://uqgroup.mit.edu
More informationPractical Bayesian Optimization of Machine Learning. Learning Algorithms
Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that
More information1 Kalman Filter Introduction
1 Kalman Filter Introduction You should first read Chapter 1 of Stochastic models, estimation, and control: Volume 1 by Peter S. Maybec (available here). 1.1 Explanation of Equations (1-3) and (1-4) Equation
More informationBayesian Inference. Chapter 9. Linear models and regression
Bayesian Inference Chapter 9. Linear models and regression M. Concepcion Ausin Universidad Carlos III de Madrid Master in Business Administration and Quantitative Methods Master in Mathematical Engineering
More informationLinear Regression (9/11/13)
STA561: Probabilistic machine learning Linear Regression (9/11/13) Lecturer: Barbara Engelhardt Scribes: Zachary Abzug, Mike Gloudemans, Zhuosheng Gu, Zhao Song 1 Why use linear regression? Figure 1: Scatter
More informationEstimating percentiles of uncertain computer code outputs
Appl. Statist. (2004) 53, Part 1, pp. 83 93 Estimating percentiles of uncertain computer code outputs Jeremy Oakley University of Sheffield, UK [Received June 2001. Final revision June 2003] Summary. A
More informationThe Expectation-Maximization Algorithm
1/29 EM & Latent Variable Models Gaussian Mixture Models EM Theory The Expectation-Maximization Algorithm Mihaela van der Schaar Department of Engineering Science University of Oxford MLE for Latent Variable
More informationSpatially Adaptive Smoothing Splines
Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia speckman@statmissouriedu September 11, 23 Banff 9/7/3 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, =
More informationSTAT 135 Lab 13 (Review) Linear Regression, Multivariate Random Variables, Prediction, Logistic Regression and the δ-method.
STAT 135 Lab 13 (Review) Linear Regression, Multivariate Random Variables, Prediction, Logistic Regression and the δ-method. Rebecca Barter May 5, 2015 Linear Regression Review Linear Regression Review
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationData Analysis and Manifold Learning Lecture 6: Probabilistic PCA and Factor Analysis
Data Analysis and Manifold Learning Lecture 6: Probabilistic PCA and Factor Analysis Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture
More informationLINEAR MODELS FOR CLASSIFICATION. J. Elder CSE 6390/PSYC 6225 Computational Modeling of Visual Perception
LINEAR MODELS FOR CLASSIFICATION Classification: Problem Statement 2 In regression, we are modeling the relationship between a continuous input variable x and a continuous target variable t. In classification,
More informationST 740: Linear Models and Multivariate Normal Inference
ST 740: Linear Models and Multivariate Normal Inference Alyson Wilson Department of Statistics North Carolina State University November 4, 2013 A. Wilson (NCSU STAT) Linear Models November 4, 2013 1 /
More informationCovariance Matrix Simplification For Efficient Uncertainty Management
PASEO MaxEnt 2007 Covariance Matrix Simplification For Efficient Uncertainty Management André Jalobeanu, Jorge A. Gutiérrez PASEO Research Group LSIIT (CNRS/ Univ. Strasbourg) - Illkirch, France *part
More informationNovember 2002 STA Random Effects Selection in Linear Mixed Models
November 2002 STA216 1 Random Effects Selection in Linear Mixed Models November 2002 STA216 2 Introduction It is common practice in many applications to collect multiple measurements on a subject. Linear
More informationLecture 13: Data Modelling and Distributions. Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1
Lecture 13: Data Modelling and Distributions Intelligent Data Analysis and Probabilistic Inference Lecture 13 Slide No 1 Why data distributions? It is a well established fact that many naturally occurring
More informationApplication of the Ensemble Kalman Filter to History Matching
Application of the Ensemble Kalman Filter to History Matching Presented at Texas A&M, November 16,2010 Outline Philosophy EnKF for Data Assimilation Field History Match Using EnKF with Covariance Localization
More informationDiscovery Guide. Beautiful, mysterious woman pursued by gunmen. Sounds like a spy story...
Dv G W C T Gp, A T Af Hk T 39 Sp. M Mx Hk p j p v, f M P v...(!) Af Hk T 39 Sp, B,,, UNMISSABLE! T - f 4 p v 150 f-p f x v. Bf, k 4 p v 150. H k f f x? D,,,, v? W k, pf p f p? W f f f? W k k p? T p xp
More informationSupplementary materials for Multivariate functional response regression, with application to fluorescence spectroscopy in a cervical pre-cancer study
Supplementary materials for Multivariate functional response regression, with application to fluorescence spectroscopy in a cervical pre-cancer study Hongxiao Zhu a,, Jeffrey S. Morris b, Fengrong Wei
More informationPhysics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
More information. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
More informationThe Gaussian distribution
The Gaussian distribution Probability density function: A continuous probability density function, px), satisfies the following properties:. The probability that x is between two points a and b b P a
More informationImproving power posterior estimation of statistical evidence
Improving power posterior estimation of statistical evidence Nial Friel, Merrilee Hurn and Jason Wyse Department of Mathematical Sciences, University of Bath, UK 10 June 2013 Bayesian Model Choice Possible
More informationNew Insights into History Matching via Sequential Monte Carlo
New Insights into History Matching via Sequential Monte Carlo Associate Professor Chris Drovandi School of Mathematical Sciences ARC Centre of Excellence for Mathematical and Statistical Frontiers (ACEMS)
More information