Introduction to Smoothing spline ANOVA models (metamodelling)
|
|
- Shona Harrell
- 5 years ago
- Views:
Transcription
1 Introduction to Smoothing spline ANOVA models (metamodelling) M. Ratto DYNARE Summer School, Paris, June 215. Joint Research Centre Serving society Stimulating innovation Supporting legislation 1
2 Introduction to ANOVA Models Consider the mapping Y f( x) where x is a vector of input variables x ( x1, x2,..., x k ) and Y is the output. x is the i-th element of x varying in 1 i x i x Ω f(x) Y 2
3 Specification of the input factors x Ω Marginal p.d.f. p( x) pi( xi) x 1 x 2 x 3 Marginal p.d.f. + correlation structure x 4 Joint p.d.f. p( x1, x2,..., x k ) Rank correlation, Dependence-trees, diagonal band, copulas MCMC (Gibbs, Metropolis, ) 3
4 Propagation of uncertainty Input Model Output px ( ) Y f( x ) py ( ) x 1 x 2 x 3 x 4 x k 1 2 n x i : input factors y 4
5 Model approximation (meta-modelling) x f(x) Y Realistic Computer models are very costly in terms of run time. We need a way to characterize uncertainty and perform SA based on a limited number of model runs. We aim to identify a 'simple' relationship between Xi's and Y that fits well the original model and is less computationally demanding. 5
6 Model approximation (meta-modelling) A Meta-model is a simpler model that mimics the larger computational model. Evaluations of Meta-Models are much faster. 6
7 Model approximation (meta-modelling) Local approximation methods take f and its derivatives at a base point X and construct a function that matches the properties of f in the nearby region (Taylor series). 7
8 Model approximation (meta-modelling) 1. Linear Regression 2. Quadratic Response Surface Regression Options 1 and 2 work pretty well in some cases. However many realistic models are highly nonlinear and/or periodic and these methods fail. 3. Gaussian Process or Spatial Models 4. Nonparametric Regression Models Both 3 and 4 work fairly well for a small number of inputs. Variable selection is helpful for a modest number of inputs. 8
9 ANOVA Models ANOVA models define a decomposition of Y=f(x) into main effects and interactions k Y f( x) f f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i This is also called the High Dimensional Model Representation (HDMR) E.g., if k=3: f( x) f f ( x ) f ( x ) f ( x ) f ( x, x ) f ( x, x ) f ( x, x ) f ( x, x, x ) This decomposition is non-unique for general joint pdf s of x. The total number of summands in the ANOVA is 2 k 9
10 Properties of the ANOVA decomposition (orthogonal case) k Y f( x) f f x f x, x... f x, x,..., x If each term is chosen with zero mean f ( x ) dp( x ), x i 1,2,..., k... i i i i i i ij i j 1,2,..., k 1 2 k i 1 i j i f ( x, x ) dp( x ) dp( x ), x, x i j ij i j i j i j Ω f ( x, x,..., x ) dp( x ) dp( x )... dp( x ) k 1 2 k 1 2 k 1
11 Properties of the ANOVA decomposition (orthogonal case) k Y f( x) f f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i then the ANOVA decomposition has TWO properties f x dp( x) f E( Y) Ω All the summands are orthogonal: i i j j if,...,,..., 1 s 1 l f f dp( x) k i1,..., is j1,..., jl Ω It follows that the ANOVA decomposition is unique and each term can be defined as 11
12 Properties of the ANOVA decomposition k Y f( x) E( Y) f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i Ω x i f x dp( x x ) f f x E( Y x ) i i i i xi xj Ω, f x dp( x x x ) f f ( x ) f ( x ) f ( x, x ) i j i i j j ij i j E( Y x, x ) i j 12
13 ANOVA k Y E( Y) f x f x, x... f x, x,..., x i i ij i j 1,2,..., k 1 2 k i 1 i j i Being the terms orthogonal, we can square and integrate the eq above over Ω and decompose the variance of f(x) into terms of increasing dimensionality (ANOVA) k V( Y) V V V... V 1,2,..., i ij ijk k i 1 i j i j k V 2 i fi ( xi) dxi 2 V f i1,..., idx dx dx i, i,... i i i... i 1 2 s s 1 2 s k 1 S S S... S 1,2,..., i ij ijk k i 1 i j i j k 13
14 Main effects If we were to approximate f(x) with a function g(x i ) 2 L E[( f( x) g( x i )) ] g( x ) E( Y x ) L L i L E[ Var( Y X )] min i i min f i =E(Y X i ) has the minimum loss L among univariate functions 14
15 Main effects V Var[ E( Y x )] i Var( Y) E[ Var( Y x i )] when we approximate f(x) with a function g(x i ), g*=e(y X i ) has the minimum loss L i 2 L E[( f( x) g( x)) ] g( x) f L V( Y) g( x) f ( x) L g( x ) E( Y x ) L L i L E[ Var( Y X )] min i i min 15
16 Main effects f ij =E(Y X i,x j ) has the minimum loss L among bivariate functions and so on Var(fi)/var(Y): also called non-parameteric R-squared; Pearson s correlation ratio; 16
17 Main effects f ( X, X ) X X X X X i ~ N(,1) 17
18 ANOVA model estimation Tensor product decomposition (reproducing kernel Hilbert space, RKHS) k k F Hj {1} Hj ( Hj Hi ) j 1 j 1 j i Orthogonal functional decomposition F q {1} j 1 F j 18
19 ANOVA model estimation Smoothing methods to estimate ANOVA decompositions, truncated at the 2 nd -3 rd order terms: SMOOTHING SPLINES ANOVA MODELS Smoothing the y vs x mapping (think of an HPfilter), that provides efficient convergence properties to the true ANOVA decomposition. [this is one possible methods, other are RBF s, kernel regressions, ] 19
20 ANOVA model estimation Denote the generic mapping as Y = f(x), where X [, 1] k and k is the number of parameters. The simplest example of smoothing spline mapping estimation of z is the additive model: g( X) g g x k i 1 i i 2
21 ANOVA model estimation To estimate g we can use a multivariate smoothing spline minimization problem, that is, given j, find the minimizer g(x) of: 1 N N 2 k ( y 1 2 n g( Xn)) λj [ gj( X j)] dx j n1 j1 where a Monte Carlo sample of dimension N is assumed. This minimization problem requires the estimation of the k hyperparameters j (also denoted as smoothing parameters): GCV, GML, etc. (see e.g. Wahba, 199; Gu, 22). 21
22 ANOVA model estimation We re-formulate the additive model for the general case with interactions as to find the minimizer g(x) of: 1 1 j y g λ P g N N q 2 ( n ( Xn)) n1 j1θj 2 F where the q-dimensional vector of j smoothing parameters needs to be optimized somehow. 22
23 ANOVA model estimation Wahba (199), Gu (22), Storlie et al. (27): en-bloc (like HP); Ratto et al., 24-27: based on recursive filtering and smoothing estimation: SDP modelling. (like KF-smoothing version of the HP); Liu et al. (22-26): polynomial basis expansion. 23
24 SDP modelling SDP modeling is one class of non-parametric smoothing approach first suggested by Young (1993). The estimation is performed with the help of the `classical' recursive (non-numerical) Kalman filter and associated fixed interval smoothing algorithms and has been applied for sensitivity analysis in Ratto et al. (24-27). 24
25 Mapping/sensitivity strategies OAT (Taylor): truncation; mapping by knowing all derivatives in a base point decomposition with infinite terms GSA (ANOVA): non-parametric regression/smoothing mapping on a spacefilling MC sample decomposition with finite terms 25
26 DSGE models Let us consider a generic DSGE model: y t endogenous variables, u t exogenous shocks X structural parameters. X can be characterised by plausible ranges, expressed in terms of prior distributions, or by a posterior distribution, as a result of an estimation. 26
27 DSGE models The model behaviour is a function of the values assumed for X within the prior or posterior space of structural parameters. Let Y be a generic 'output' of the model: a multiplier, a measure of fit, an IRF. Y depends on the values of X Y=f(X 1,, X k ), f non-linear analytic form is unknown 27
28 Mapping the reduced form of RE models Relationship between the reduced form of a rational expectation model and the structural coefficients. let the reduced form be y t =Ty t-1 +Bu t, 'outputs' Y of our analysis will be the entries in the transition matrix T(X 1,,X k ) or in the matrix B(X 1,,X k ). 28
29 Lubik Schorfheide (25) We analyse the reduced form coefficients describing the relationship between t vs e R,t We sample the structural coefficients from posterior ranges obtained after estimating the model using data for Canada. 29
30 Lubik Schorfheide (25) 3
31 Lubik Schorfheide (25) log( ( vs e )) π t Factorisation R,t Y exp( f f... f e) 1 exp( e) exp( f ) k j j k k k Y exp( e) exp( f ) g ε j j j Correction [new in DYNARE 4.5] j 31
32 LS25: pie vs e_r [-log(y)] 2 rho R, Si=.69 2 psi1, Si=. 1 k, Si=. 1 rho y s, Si= rho q, Si=. 2 tau, Si=. 1 rr, Si=. 1 rho A, Si= rho ies, Si=. p 1 1 psi2, Si=. 1 psi3, Si=. 1 alpha, Si=
33 LS25: R vs e_r [-log(y)].5 k, Si=.72.2 rho R, Si=.12.1 psi1, Si=.8.5 psi2, Si= psi3, Si=.1.2 tau, Si=. rho ies, Si=. p.1.1 rho A, Si= alpha, Si=..1.1 rr, Si=. rho s, Si=. y.1.1 rho q, Si=
34 Lubik Schorfheide (25) 1 Reduced form GSA - Log-transformed elements Values psi1 psi2 psi3 rho_r alpha rr k tau rho_q rho_a rho_ys rho_pies 34
35 Toolbox documentation The mapping of the reduced form soultion forces the use of samples from prior ranges or prior distributions, i.e.: pprior=1 (default); ppost= (default); [unless neighbourhood_width is applied] 35
36 Toolbox documentation 36
37 ss_anova Toolbox Download here the Recursive SS-ANOVA Toolbox for MATLAB. (MATLAB version < 7.5) 37
38 Using Monte Carlo filtering Store the sample of the state space A,B matrices; For 1-step ahead irf, perform the MCF sensitivity tests for B(i,j) within/outside the specified ranges. 38
39 Using Monte Carlo filtering: y ( π vs R ) [ 1,] t t1 rho_r psi1 k 1 rho_r psi1 k inside threshold outside threshold
40 Using Monte Carlo filtering: y ( π vs R ) [, ] t t1 1 rho_r 1 k psi Monte Carlo Filtering for pie vs R 1-dec 2-dec 3-dec 4-dec 5-dec 6-dec 7-dec 8-dec 9-dec 1-dec
41 Mapping reduced form solution 41
Dynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationTAKEHOME FINAL EXAM e iω e 2iω e iω e 2iω
ECO 513 Spring 2015 TAKEHOME FINAL EXAM (1) Suppose the univariate stochastic process y is ARMA(2,2) of the following form: y t = 1.6974y t 1.9604y t 2 + ε t 1.6628ε t 1 +.9216ε t 2, (1) where ε is i.i.d.
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationSpatially Adaptive Smoothing Splines
Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia speckman@statmissouriedu September 11, 23 Banff 9/7/3 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, =
More informationStatistical Inference and Methods
Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 31st January 2006 Part VI Session 6: Filtering and Time to Event Data Session 6: Filtering and
More informationPolynomial chaos expansions for sensitivity analysis
c DEPARTMENT OF CIVIL, ENVIRONMENTAL AND GEOMATIC ENGINEERING CHAIR OF RISK, SAFETY & UNCERTAINTY QUANTIFICATION Polynomial chaos expansions for sensitivity analysis B. Sudret Chair of Risk, Safety & Uncertainty
More informationDependence. Practitioner Course: Portfolio Optimization. John Dodson. September 10, Dependence. John Dodson. Outline.
Practitioner Course: Portfolio Optimization September 10, 2008 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y ) (x,
More informationDependence. MFM Practitioner Module: Risk & Asset Allocation. John Dodson. September 11, Dependence. John Dodson. Outline.
MFM Practitioner Module: Risk & Asset Allocation September 11, 2013 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y
More informationCOMP 551 Applied Machine Learning Lecture 20: Gaussian processes
COMP 55 Applied Machine Learning Lecture 2: Gaussian processes Instructor: Ryan Lowe (ryan.lowe@cs.mcgill.ca) Slides mostly by: (herke.vanhoof@mcgill.ca) Class web page: www.cs.mcgill.ca/~hvanho2/comp55
More informationx. Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ 2 ).
.8.6 µ =, σ = 1 µ = 1, σ = 1 / µ =, σ =.. 3 1 1 3 x Figure 1: Examples of univariate Gaussian pdfs N (x; µ, σ ). The Gaussian distribution Probably the most-important distribution in all of statistics
More informationLabor-Supply Shifts and Economic Fluctuations. Technical Appendix
Labor-Supply Shifts and Economic Fluctuations Technical Appendix Yongsung Chang Department of Economics University of Pennsylvania Frank Schorfheide Department of Economics University of Pennsylvania January
More informationImplementation and Evaluation of Nonparametric Regression Procedures for Sensitivity Analysis of Computationally Demanding Models
Implementation and Evaluation of Nonparametric Regression Procedures for Sensitivity Analysis of Computationally Demanding Models Curtis B. Storlie a, Laura P. Swiler b, Jon C. Helton b and Cedric J. Sallaberry
More informationComputer Vision Group Prof. Daniel Cremers. 9. Gaussian Processes - Regression
Group Prof. Daniel Cremers 9. Gaussian Processes - Regression Repetition: Regularized Regression Before, we solved for w using the pseudoinverse. But: we can kernelize this problem as well! First step:
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationSobol-Hoeffding Decomposition with Application to Global Sensitivity Analysis
Sobol-Hoeffding decomposition Application to Global SA Computation of the SI Sobol-Hoeffding Decomposition with Application to Global Sensitivity Analysis Olivier Le Maître with Colleague & Friend Omar
More informationNonparmeteric Bayes & Gaussian Processes. Baback Moghaddam Machine Learning Group
Nonparmeteric Bayes & Gaussian Processes Baback Moghaddam baback@jpl.nasa.gov Machine Learning Group Outline Bayesian Inference Hierarchical Models Model Selection Parametric vs. Nonparametric Gaussian
More informationESTIMATION of a DSGE MODEL
ESTIMATION of a DSGE MODEL Paris, October 17 2005 STÉPHANE ADJEMIAN stephane.adjemian@ens.fr UNIVERSITÉ DU MAINE & CEPREMAP Slides.tex ESTIMATION of a DSGE MODEL STÉPHANE ADJEMIAN 16/10/2005 21:37 p. 1/3
More informationComputer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression
Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.
More informationLocal and Global Sensitivity Analysis
Omar 1,2 1 Duke University Department of Mechanical Engineering & Materials Science omar.knio@duke.edu 2 KAUST Division of Computer, Electrical, Mathematical Science & Engineering omar.knio@kaust.edu.sa
More informationGaussian process regression for Sensitivity analysis
Gaussian process regression for Sensitivity analysis GPSS Workshop on UQ, Sheffield, September 2016 Nicolas Durrande, Mines St-Étienne, durrande@emse.fr GPSS workshop on UQ GPs for sensitivity analysis
More informationCS 7140: Advanced Machine Learning
Instructor CS 714: Advanced Machine Learning Lecture 3: Gaussian Processes (17 Jan, 218) Jan-Willem van de Meent (j.vandemeent@northeastern.edu) Scribes Mo Han (han.m@husky.neu.edu) Guillem Reus Muns (reusmuns.g@husky.neu.edu)
More information19 : Slice Sampling and HMC
10-708: Probabilistic Graphical Models 10-708, Spring 2018 19 : Slice Sampling and HMC Lecturer: Kayhan Batmanghelich Scribes: Boxiang Lyu 1 MCMC (Auxiliary Variables Methods) In inference, we are often
More informationROBOTICS 01PEEQW. Basilio Bona DAUIN Politecnico di Torino
ROBOTICS 01PEEQW Basilio Bona DAUIN Politecnico di Torino Probabilistic Fundamentals in Robotics Gaussian Filters Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile
More information9.2 Support Vector Machines 159
9.2 Support Vector Machines 159 9.2.3 Kernel Methods We have all the tools together now to make an exciting step. Let us summarize our findings. We are interested in regularized estimation problems of
More informationSolving Boundary Value Problems (with Gaussians)
What is a boundary value problem? Solving Boundary Value Problems (with Gaussians) Definition A differential equation with constraints on the boundary Michael McCourt Division Argonne National Laboratory
More informationExtracting Rational Expectations Model Structural Matrices from Dynare
Extracting Rational Expectations Model Structural Matrices from Dynare Callum Jones New York University In these notes I discuss how to extract the structural matrices of a model from its Dynare implementation.
More informationMultivariate Random Variable
Multivariate Random Variable Author: Author: Andrés Hincapié and Linyi Cao This Version: August 7, 2016 Multivariate Random Variable 3 Now we consider models with more than one r.v. These are called multivariate
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationMultivariate Distribution Models
Multivariate Distribution Models Model Description While the probability distribution for an individual random variable is called marginal, the probability distribution for multiple random variables is
More informationKernel-based Approximation. Methods using MATLAB. Gregory Fasshauer. Interdisciplinary Mathematical Sciences. Michael McCourt.
SINGAPORE SHANGHAI Vol TAIPEI - Interdisciplinary Mathematical Sciences 19 Kernel-based Approximation Methods using MATLAB Gregory Fasshauer Illinois Institute of Technology, USA Michael McCourt University
More informationReduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation
Reduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation Curtis B. Storlie a a Los Alamos National Laboratory E-mail:storlie@lanl.gov Outline Reduction of Emulator
More informationBayesian Inference. Chapter 4: Regression and Hierarchical Models
Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Advanced Statistics and Data Mining Summer School
More informationLMS Algorithm Summary
LMS Algorithm Summary Step size tradeoff Other Iterative Algorithms LMS algorithm with variable step size: w(k+1) = w(k) + µ(k)e(k)x(k) When step size µ(k) = µ/k algorithm converges almost surely to optimal
More informationMulti-Linear Mappings, SVD, HOSVD, and the Numerical Solution of Ill-Conditioned Tensor Least Squares Problems
Multi-Linear Mappings, SVD, HOSVD, and the Numerical Solution of Ill-Conditioned Tensor Least Squares Problems Lars Eldén Department of Mathematics, Linköping University 1 April 2005 ERCIM April 2005 Multi-Linear
More informationIntroduction to Probabilistic Graphical Models: Exercises
Introduction to Probabilistic Graphical Models: Exercises Cédric Archambeau Xerox Research Centre Europe cedric.archambeau@xrce.xerox.com Pascal Bootcamp Marseille, France, July 2010 Exercise 1: basics
More informationAn Introduction to Graphs in Dynare
An Introduction to Graphs in Dynare Johannes Pfeifer. University of Cologne June 9, 7 First version: March 3 Abstract This paper provides an introduction to the graphs displayed by Dynare in its stoch_simul,
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationInversion Base Height. Daggot Pressure Gradient Visibility (miles)
Stanford University June 2, 1998 Bayesian Backtting: 1 Bayesian Backtting Trevor Hastie Stanford University Rob Tibshirani University of Toronto Email: trevor@stat.stanford.edu Ftp: stat.stanford.edu:
More informationThe Metropolis-Hastings Algorithm. June 8, 2012
The Metropolis-Hastings Algorithm June 8, 22 The Plan. Understand what a simulated distribution is 2. Understand why the Metropolis-Hastings algorithm works 3. Learn how to apply the Metropolis-Hastings
More informationIntroduction to Gaussian Processes
Introduction to Gaussian Processes Neil D. Lawrence GPSS 10th June 2013 Book Rasmussen and Williams (2006) Outline The Gaussian Density Covariance from Basis Functions Basis Function Representations Constructing
More informationChapter 6. Maximum Likelihood Analysis of Dynamic Stochastic General Equilibrium (DSGE) Models
Chapter 6. Maximum Likelihood Analysis of Dynamic Stochastic General Equilibrium (DSGE) Models Fall 22 Contents Introduction 2. An illustrative example........................... 2.2 Discussion...................................
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationFast evaluation of mixed derivatives and calculation of optimal weights for integration. Hernan Leovey
Fast evaluation of mixed derivatives and calculation of optimal weights for integration Humboldt Universität zu Berlin 02.14.2012 MCQMC2012 Tenth International Conference on Monte Carlo and Quasi Monte
More informationL09. PARTICLE FILTERING. NA568 Mobile Robotics: Methods & Algorithms
L09. PARTICLE FILTERING NA568 Mobile Robotics: Methods & Algorithms Particle Filters Different approach to state estimation Instead of parametric description of state (and uncertainty), use a set of state
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationLecture 8: Bayesian Estimation of Parameters in State Space Models
in State Space Models March 30, 2016 Contents 1 Bayesian estimation of parameters in state space models 2 Computational methods for parameter estimation 3 Practical parameter estimation in state space
More informationIntroduction to Gaussian Processes
Introduction to Gaussian Processes Iain Murray murray@cs.toronto.edu CSC255, Introduction to Machine Learning, Fall 28 Dept. Computer Science, University of Toronto The problem Learn scalar function of
More informationIn manycomputationaleconomicapplications, one must compute thede nite n
Chapter 6 Numerical Integration In manycomputationaleconomicapplications, one must compute thede nite n integral of a real-valued function f de ned on some interval I of
More informationMarkov Chain Monte Carlo Methods for Stochastic
Markov Chain Monte Carlo Methods for Stochastic Optimization i John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U Florida, Nov 2013
More informationGAUSSIAN PROCESS REGRESSION
GAUSSIAN PROCESS REGRESSION CSE 515T Spring 2015 1. BACKGROUND The kernel trick again... The Kernel Trick Consider again the linear regression model: y(x) = φ(x) w + ε, with prior p(w) = N (w; 0, Σ). The
More informationSpatial Process Estimates as Smoothers: A Review
Spatial Process Estimates as Smoothers: A Review Soutir Bandyopadhyay 1 Basic Model The observational model considered here has the form Y i = f(x i ) + ɛ i, for 1 i n. (1.1) where Y i is the observed
More informationF denotes cumulative density. denotes probability density function; (.)
BAYESIAN ANALYSIS: FOREWORDS Notation. System means the real thing and a model is an assumed mathematical form for the system.. he probability model class M contains the set of the all admissible models
More informationMarkov Chain Monte Carlo Methods for Stochastic Optimization
Markov Chain Monte Carlo Methods for Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge U of Toronto, MIE,
More information. Frobenius-Perron Operator ACC Workshop on Uncertainty Analysis & Estimation. Raktim Bhattacharya
.. Frobenius-Perron Operator 2014 ACC Workshop on Uncertainty Analysis & Estimation Raktim Bhattacharya Laboratory For Uncertainty Quantification Aerospace Engineering, Texas A&M University. uq.tamu.edu
More informationGaussian processes. Chuong B. Do (updated by Honglak Lee) November 22, 2008
Gaussian processes Chuong B Do (updated by Honglak Lee) November 22, 2008 Many of the classical machine learning algorithms that we talked about during the first half of this course fit the following pattern:
More informationGaussian kernel GARCH models
Gaussian kernel GARCH models Xibin (Bill) Zhang and Maxwell L. King Department of Econometrics and Business Statistics Faculty of Business and Economics 7 June 2013 Motivation A regression model is often
More informationParticle Filtering Approaches for Dynamic Stochastic Optimization
Particle Filtering Approaches for Dynamic Stochastic Optimization John R. Birge The University of Chicago Booth School of Business Joint work with Nicholas Polson, Chicago Booth. JRBirge I-Sim Workshop,
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationAUTOMOTIVE ENVIRONMENT SENSORS
AUTOMOTIVE ENVIRONMENT SENSORS Lecture 5. Localization BME KÖZLEKEDÉSMÉRNÖKI ÉS JÁRMŰMÉRNÖKI KAR 32708-2/2017/INTFIN SZÁMÚ EMMI ÁLTAL TÁMOGATOTT TANANYAG Related concepts Concepts related to vehicles moving
More informationPractical Bayesian Optimization of Machine Learning. Learning Algorithms
Practical Bayesian Optimization of Machine Learning Algorithms CS 294 University of California, Berkeley Tuesday, April 20, 2016 Motivation Machine Learning Algorithms (MLA s) have hyperparameters that
More informationPrediction of Data with help of the Gaussian Process Method
of Data with help of the Gaussian Process Method R. Preuss, U. von Toussaint Max-Planck-Institute for Plasma Physics EURATOM Association 878 Garching, Germany March, Abstract The simulation of plasma-wall
More informationPart 6: Multivariate Normal and Linear Models
Part 6: Multivariate Normal and Linear Models 1 Multiple measurements Up until now all of our statistical models have been univariate models models for a single measurement on each member of a sample of
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationIdentification Analysis of DSGE models with DYNARE
Identification Analysis of DSGE models with DYNARE FP7 Funded, Project MONFISPOL Grant no.: 225149. Marco Ratto European Commission, Joint Research Centre with the contribution of Nikolai Iskrev Bank of
More informationCopula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011
Copula Regression RAHUL A. PARSA DRAKE UNIVERSITY & STUART A. KLUGMAN SOCIETY OF ACTUARIES CASUALTY ACTUARIAL SOCIETY MAY 18,2011 Outline Ordinary Least Squares (OLS) Regression Generalized Linear Models
More informationGaussian Process Regression
Gaussian Process Regression 4F1 Pattern Recognition, 21 Carl Edward Rasmussen Department of Engineering, University of Cambridge November 11th - 16th, 21 Rasmussen (Engineering, Cambridge) Gaussian Process
More informationA Process over all Stationary Covariance Kernels
A Process over all Stationary Covariance Kernels Andrew Gordon Wilson June 9, 0 Abstract I define a process over all stationary covariance kernels. I show how one might be able to perform inference that
More informationKernel Methods. Machine Learning A W VO
Kernel Methods Machine Learning A 708.063 07W VO Outline 1. Dual representation 2. The kernel concept 3. Properties of kernels 4. Examples of kernel machines Kernel PCA Support vector regression (Relevance
More informationcomponent risk analysis
273: Urban Systems Modeling Lec. 3 component risk analysis instructor: Matteo Pozzi 273: Urban Systems Modeling Lec. 3 component reliability outline risk analysis for components uncertain demand and uncertain
More informationProbabilistic Fundamentals in Robotics. DAUIN Politecnico di Torino July 2010
Probabilistic Fundamentals in Robotics Gaussian Filters Basilio Bona DAUIN Politecnico di Torino July 2010 Course Outline Basic mathematical framework Probabilistic models of mobile robots Mobile robot
More informationInterpreting Regression Results
Interpreting Regression Results Carlo Favero Favero () Interpreting Regression Results 1 / 42 Interpreting Regression Results Interpreting regression results is not a simple exercise. We propose to split
More informationDynamic Macro 1 / 114. Bayesian Estimation. Summer Bonn University
Dynamic Macro Bayesian Estimation Petr Sedláček Bonn University Summer 2015 1 / 114 Overall plan Motivation Week 1: Use of computational tools, simple DSGE model Tools necessary to solve models and a solution
More informationNonlinear Multivariate Statistical Sensitivity Analysis of Environmental Models
Nonlinear Multivariate Statistical Sensitivity Analysis of Environmental Models with Application on Heavy Metals Adsorption from Contaminated Wastewater A. Fassò, A. Esposito, E. Porcu, A.P. Reverberi,
More information10-701/ Recitation : Kernels
10-701/15-781 Recitation : Kernels Manojit Nandi February 27, 2014 Outline Mathematical Theory Banach Space and Hilbert Spaces Kernels Commonly Used Kernels Kernel Theory One Weird Kernel Trick Representer
More informationA Search and Jump Algorithm for Markov Chain Monte Carlo Sampling. Christopher Jennison. Adriana Ibrahim. Seminar at University of Kuwait
A Search and Jump Algorithm for Markov Chain Monte Carlo Sampling Christopher Jennison Department of Mathematical Sciences, University of Bath, UK http://people.bath.ac.uk/mascj Adriana Ibrahim Institute
More informationChapter 5. Chapter 5 sections
1 / 43 sections Discrete univariate distributions: 5.2 Bernoulli and Binomial distributions Just skim 5.3 Hypergeometric distributions 5.4 Poisson distributions Just skim 5.5 Negative Binomial distributions
More informationBayesian Inference. Chapter 4: Regression and Hierarchical Models
Bayesian Inference Chapter 4: Regression and Hierarchical Models Conchi Ausín and Mike Wiper Department of Statistics Universidad Carlos III de Madrid Master in Business Administration and Quantitative
More informationPreliminary Examination in Numerical Analysis
Department of Applied Mathematics Preliminary Examination in Numerical Analysis August 7, 06, 0 am pm. Submit solutions to four (and no more) of the following six problems. Show all your work, and justify
More informationModelling Dependence with Copulas and Applications to Risk Management. Filip Lindskog, RiskLab, ETH Zürich
Modelling Dependence with Copulas and Applications to Risk Management Filip Lindskog, RiskLab, ETH Zürich 02-07-2000 Home page: http://www.math.ethz.ch/ lindskog E-mail: lindskog@math.ethz.ch RiskLab:
More informationState Space Representation of Gaussian Processes
State Space Representation of Gaussian Processes Simo Särkkä Department of Biomedical Engineering and Computational Science (BECS) Aalto University, Espoo, Finland June 12th, 2013 Simo Särkkä (Aalto University)
More informationCSE446: non-parametric methods Spring 2017
CSE446: non-parametric methods Spring 2017 Ali Farhadi Slides adapted from Carlos Guestrin and Luke Zettlemoyer Linear Regression: What can go wrong? What do we do if the bias is too strong? Might want
More informationCOMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017
COMS 4721: Machine Learning for Data Science Lecture 10, 2/21/2017 Prof. John Paisley Department of Electrical Engineering & Data Science Institute Columbia University FEATURE EXPANSIONS FEATURE EXPANSIONS
More informationBayesian Inference for DSGE Models. Lawrence J. Christiano
Bayesian Inference for DSGE Models Lawrence J. Christiano Outline State space-observer form. convenient for model estimation and many other things. Bayesian inference Bayes rule. Monte Carlo integation.
More informationFundamentals of Data Assimila1on
014 GSI Community Tutorial NCAR Foothills Campus, Boulder, CO July 14-16, 014 Fundamentals of Data Assimila1on Milija Zupanski Cooperative Institute for Research in the Atmosphere Colorado State University
More informationLearning gradients: prescriptive models
Department of Statistical Science Institute for Genome Sciences & Policy Department of Computer Science Duke University May 11, 2007 Relevant papers Learning Coordinate Covariances via Gradients. Sayan
More informationGaussian Process Approximations of Stochastic Differential Equations
Gaussian Process Approximations of Stochastic Differential Equations Cédric Archambeau Dan Cawford Manfred Opper John Shawe-Taylor May, 2006 1 Introduction Some of the most complex models routinely run
More informationReliability Monitoring Using Log Gaussian Process Regression
COPYRIGHT 013, M. Modarres Reliability Monitoring Using Log Gaussian Process Regression Martin Wayne Mohammad Modarres PSA 013 Center for Risk and Reliability University of Maryland Department of Mechanical
More informationBAYESIAN DECISION THEORY
Last updated: September 17, 2012 BAYESIAN DECISION THEORY Problems 2 The following problems from the textbook are relevant: 2.1 2.9, 2.11, 2.17 For this week, please at least solve Problem 2.3. We will
More informationLecture 4: Dynamic models
linear s Lecture 4: s Hedibert Freitas Lopes The University of Chicago Booth School of Business 5807 South Woodlawn Avenue, Chicago, IL 60637 http://faculty.chicagobooth.edu/hedibert.lopes hlopes@chicagobooth.edu
More informationECO 513 Fall 2009 C. Sims HIDDEN MARKOV CHAIN MODELS
ECO 513 Fall 2009 C. Sims HIDDEN MARKOV CHAIN MODELS 1. THE CLASS OF MODELS y t {y s, s < t} p(y t θ t, {y s, s < t}) θ t = θ(s t ) P[S t = i S t 1 = j] = h ij. 2. WHAT S HANDY ABOUT IT Evaluating the
More informationProblem 1: Toolbox (25 pts) For all of the parts of this problem, you are limited to the following sets of tools:
CS 322 Final Exam Friday 18 May 2007 150 minutes Problem 1: Toolbox (25 pts) For all of the parts of this problem, you are limited to the following sets of tools: (A) Runge-Kutta 4/5 Method (B) Condition
More informationTSRT14: Sensor Fusion Lecture 8
TSRT14: Sensor Fusion Lecture 8 Particle filter theory Marginalized particle filter Gustaf Hendeby gustaf.hendeby@liu.se TSRT14 Lecture 8 Gustaf Hendeby Spring 2018 1 / 25 Le 8: particle filter theory,
More informationCMPE 58K Bayesian Statistics and Machine Learning Lecture 5
CMPE 58K Bayesian Statistics and Machine Learning Lecture 5 Multivariate distributions: Gaussian, Bernoulli, Probability tables Department of Computer Engineering, Boğaziçi University, Istanbul, Turkey
More informationWinter 2019 Math 106 Topics in Applied Mathematics. Lecture 1: Introduction
Winter 2019 Math 106 Topics in Applied Mathematics Data-driven Uncertainty Quantification Yoonsang Lee (yoonsang.lee@dartmouth.edu) Lecture 1: Introduction 19 Winter M106 Class: MWF 12:50-1:55 pm @ 200
More informationComputing first and second order approximations of DSGE models with DYNARE
Computing first and second order approximations of DSGE models with DYNARE Michel Juillard CEPREMAP Computing first and second order approximations of DSGE models with DYNARE p. 1/33 DSGE models E t {f(y
More informationPreliminary statistics
1 Preliminary statistics The solution of a geophysical inverse problem can be obtained by a combination of information from observed data, the theoretical relation between data and earth parameters (models),
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 7 Approximate
More information