Inversion Base Height. Daggot Pressure Gradient Visibility (miles)
|
|
- Dulcie McDonald
- 6 years ago
- Views:
Transcription
1 Stanford University June 2, 1998 Bayesian Backtting: 1 Bayesian Backtting Trevor Hastie Stanford University Rob Tibshirani University of Toronto trevor@stat.stanford.edu Ftp: stat.stanford.edu: pub/hastie WWW: These transparencies are available via ftp: ftp://stat.stanford.edu/pub/hastie/bayes.ps
2 Stanford University June 2, 1998 Bayesian Backtting: 2 In a Nutshell y = f1 + f2 + : : : + f p + " Backtting cycles around and replaces each current function P estimate f j by f j S j (y k6=j f k), where S j is a smoothing operator; Gibbs sampling cycles around and obtains a new realization of f j via f j S j (y P k6=j f k) + S 1=2 j z, where z is a vector of N(0; 1) variates and is the standard deviation.
3 Stanford University June 2, 1998 Bayesian Backtting: 3 ' $ Example: Air Pollution Inversion Base Height Daggot Pressure Gradient Inversion Base Temperature Visibility (miles) Four environmental variables in a air pollution study.
4 Stanford University June 2, 1998 Bayesian Backtting: 4 Outline Smoothing splines a Bayesian version. Additive Models and backtting Bayesian backtting Example: Bone mineral density Priors, variance components and DF GAMs and MetropolisHastings
5 Stanford University June 2, 1998 Bayesian Backtting: 5 Smoothing Splines y i = f(x i ) + " i ; " i N(0; 2 ) X R(f; y) = (y i f(x i )) 2 + i Z [f 00 (x)] 2 dx Solution is natural cubic spline, with knots at unique values of x Solutions vary between linear ts ( = 1) and interpolating ts ( = 0) Finite dimensional solution minimizes with solution (y f) T (y f) + f T Kf; f = (I + K) 1 y = S()y
6 Stanford University June 2, 1998 Bayesian Backtting: 6 Bayesian Smoothing Splines y = f + " f N(0; 2 K ) where K is partially improper (linear functions have innite variances) " N(0; 2 I) (y; f) (y f) T (y f)=2 2 + f T Kf=2 2 fjy N(S()y; S() 2 ), where = 2 = 2 f = S()y + S() 1 2 z where z is an Nvector of N(0; 1) variates.
7 Stanford University June 2, 1998 Bayesian Backtting: 7 Picture the Prior f N(0; 2 K ), or equivalently, f = N, and N(0; 2 D) number number number number eigenfunction number number number number Eigenfunctions of the Prior Covariance Prior Variance 10^3 10^0 10^2 10^ Number
8 Stanford University June 2, 1998 Bayesian Backtting: 8 Backtting Additive Models y = f 1 (x 1 ) + f 2 (x 2 ) + + f p (x p ) + " Estimating Equations f 1 (x 1 ) = S 1 (y f 2 (x 2 ) f p (x p )) f 2 (x 2 ) = S 2 (y f 1 (x 1 ) f p (x p )). f p (x p ) = S p (y f 1 (x 1 ) f 2 (x 2 ) ) where S j are: univariate regression smoothers such as smoothing splines, lowess, kernel linear regression operators yielding polynomial ts, piecewise polynomials,... more complicated operators: surface smoothers for 2nd order interactions, and random eects shrinkage operators. We use GaussSeidel or \backtting" to solve these estimating equations.
9 Stanford University June 2, 1998 Bayesian Backtting: 9 Justication? Example: penalized least squares Minimize nx i=1 y i px j=1 2+ px f j (x ij ) j=1 j Z (f 00 j (t)) 2 dt m f 1 = S 1 ( 1 )(y X j6=1 f j ) f 2 = S 2 ( 2 )(y X j6=2 f j ). f p = S p ( p )(y X j6=p f j ) where S j ( j ) denotes a smoothing spline using variable x j and penalty coecient j. Each smoothing spline t is O(N) computations, hence so is entire t.
10 Stanford University June 2, 1998 Bayesian Backtting: 10 Bayesian Additive Splines f j N(0; 2 j K j ) f j jy N(G j y; C j 2 ) where G j and C j are ugly and O(N 3 ) computations. You don't believe me? See Wahba (1990). Backtting computes ^f j = G j y eciently in O(N) computations, but not C j. Current GAM software in Splus approximates C j by C 0 j + S j, where C 0 j is the exact posterior covariance operator for the linear part of f j, but S j is the exact posterior covariance operator for the nonlinear part of a univariate spline problem.
11 Stanford University June 2, 1998 Bayesian Backtting: 11 Gibbs sampling saves the day! L(f j jy; f k ; k 6= j) = L(f j jy X k6=j f k ; ff k ; k 6= jg) = N(S j (y X k6=j f k ); S j 2 ) By replacing the backtting operator f j S j (y P k6=j f k) by the univariate Bayesian spline posterior sampler P f j S j (y k6=j f k) + S 1=2 j z we generate a Markov chain, whose stationary distribution coincides with (f1; f2; : : : ; f p jy) Carter and Kohn (1994, Biometrika), and Kohn and Wong (1998, manuscript) propose a similar algorithm using a state space approach.
12 Stanford University June 2, 1998 Bayesian Backtting: 12 Other operators We can use the Bayesian Backtting algorithm with operators other than smoothing splines S j. General nonparametric smoother: S j denes the smoothing operation, with implicit prior f j N(0; (S j I) 2 ): The operator S 1=2 j is found by a simple Taylor series expansion. Fixed linear eects: S j = X j (X T j X j) 1 X T j. This results from the model f j = X j j with j N(0; 2 D) and D diagonal, and! 1. Then S 1=2 j = S j and is easily applied. For the intercept term, P for example, we simply obtain p N(ave[y 1 f j]; 2 =n). Random linear eects: S j = X j (X T X j j ) 1 X T j. This results from f j = X T j j with j N(0; ).
13 Stanford University June 2, 1998 Bayesian Backtting: 13 ' $ Example: Growth Curves y ij = f(t ij ) + x T i E + V i + " ij Age Spinal Bone Mineral Density Age Spinal Bone Mineral Density Asian Black Hispanic White Ethnic Group Age Random Individual Level Effect
14 Stanford University June 2, 1998 Bayesian Backtting: 14 Functionals of posteriors Posterior Distributions of Functionals Derivative of Posterior Mean Age The location of the maximum derivative (center of growth spurt) is not too convincing. We now attempt a more realistic model
15 Stanford University June 2, 1998 Bayesian Backtting: 15 Computing S 1 2 z Smoothing Splines Writing f(x) = P M j=1 b j(x) j in terms of the natural spline basis, the posterior distribution for f has the form f jy N(Sy; S 2 ) = N(B^; B^ B T ) ^ = (B T B + ) 1 B T y ^ = 2 (B T B + ) 1 The Cholesky squareroot of the last expression is computed routinely in the smoothing spline computations, and so is available for our purposes. General Smoothers S 1=2 = S 1 2 S(S I) S(S I) S(S I)3
16 Stanford University June 2, 1998 Bayesian Backtting: 16 ' $ y ij = f(t ij i ) + x T i E + V i + " ij Age Spinal Bone Mineral Density Principal Curve Age Mean Age Curve Girl Average Age Random Level Shifts Girl Average Age Random Age Shifts
17 Stanford University June 2, 1998 Bayesian Backtting: 17 Age Shifted Mean Curves Level Shifted Mean Curves Age Adjusted Age Shrunken Level Fits Girl Average Age
18 Stanford University June 2, 1998 Bayesian Backtting: 18 ' $ Theta V Girl Average Age Spinal BMD
19 Stanford University June 2, 1998 Bayesian Backtting: 19 Estimating 2, 2,... A fully hierarchical Bayesian approach puts priors on these hyperparameters, and generates them along with the other posterior realizations. Empirical Bayes procedures maximize the marginal likelihood of y to estimate the hyperparameters. Very similar to REML Restricted Maximum Likelihood Estimation. This highlights the formal equivalence of additive spline models and mixedeects models: Y = N N : : : N p p + " Crossvalidation, GCV and Cp use prediction error to guide selection.
20 Stanford University June 2, 1998 Bayesian Backtting: 20 Priors for 2, 2,... p( 2 j ) 1= 2 j or 1= 2 j exp( = 2 j ) with = 10e 10. p( 2 ) 1= 2. Wong and Kohn (1998), Carter and Kohn (1994) These lead to Inverse Gaussian posteriors: p( 2 j jy; 2 ; ff j g p 1 ) = p( 2 j jf j) = IG(n=2; 1 2 f T j K j f j + j ) p( 2 jrest) = IG(n=2; jjejj 2 ) where e = y P j f j. These are generated within each cycle of the Gibbs algorithm, along with the functions. O(N) computations.
21 Stanford University June 2, 1998 Bayesian Backtting: 21 Posterior for df The eective degrees of freedom are dened as df = trs(), where = 2 = 2. Inversion Base Height Daggot Pressure Gradient Degrees of Freedom Degrees of Freedom Iteration Iteration Inversion Base Temperature Visibility (miles) Degrees of Freedom Degrees of Freedom Iteration Iteration
22 Stanford University June 2, 1998 Bayesian Backtting: 22 Prior for df? Using the priors for 2 and 2, we can induce a prior for df for any sequence of x values Percent of Total Prior Degrees of Freedom Actually, this gure is based on log 2 U[ 25; 25]. When 25! 1, we get point masses of 1 at 2 and N! 2
23 Stanford University June 2, 1998 Bayesian Backtting: 23 Bayes vs Bootstrap Suppose 2 is known, and we add residuals r N(0; I 2 ) to ^f and ret. For a single smoothing spline: Bootstrap: f N(S 2 y; S 2 2 ) Bayes: f jy N(Sy; S 2 ) S > S 2, and the Bayes posterior intervals are wider than the bootstrap intervals; they include an average bias component. For an additive spline model: Bootstrap: f j = N(A j Ay; A 2 j 2 ) Bayes: f j N(A j y; (I A j )S j (I S j ) 2 ) Bayes Bootstrap
24 Stanford University June 2, 1998 Bayesian Backtting: 24 Generalized Additive Models Suppose instead we have a GAM, such as an additive logistic regression model: LogitP (Y = 1jx) = X j f j (x j ) where the f j can be functions, random eects or \xed" eects, each with their (Gaussian) priors N(0; K j ) and hyperparameters j. Similar to Zeger and Karim (1991, JASA), we propose a MetropolisHastings scheme for updating the functions: At the current state, approximate the likelihood by a Gaussian, thus creating a working response z i and weights w i. Generate a new realization f 0 to replace j f j from this Gaussian approximation, which we denote by q(f j ; f j). 0
25 Stanford University June 2, 1998 Bayesian Backtting: 25 Move to f 0 j with probability min 0 (f )q(f 0 ; f) (f)q(f; f 0 ) ; 1 where (f) denotes the posterior. Again, all the computations can be performed in O(N) operations per update for smoothing splines, random eects and xed eects. This allows for estimation of mixed eects GLMs, with both the usual random eects as well as nonparametric smoothers, in a seamless fashion. The End
Spatially Adaptive Smoothing Splines
Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia speckman@statmissouriedu September 11, 23 Banff 9/7/3 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, =
More informationFahrmeir: Recent Advances in Semiparametric Bayesian Function Estimation
Fahrmeir: Recent Advances in Semiparametric Bayesian Function Estimation Sonderforschungsbereich 386, Paper 137 (1998) Online unter: http://epub.ub.uni-muenchen.de/ Projektpartner Recent Advances in Semiparametric
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 9: Basis Expansions Department of Statistics & Biostatistics Rutgers University Nov 01, 2011 Regression and Classification Linear Regression. E(Y X) = f(x) We want to learn
More informationGeneralized Additive Models
Generalized Additive Models The Model The GLM is: g( µ) = ß 0 + ß 1 x 1 + ß 2 x 2 +... + ß k x k The generalization to the GAM is: g(µ) = ß 0 + f 1 (x 1 ) + f 2 (x 2 ) +... + f k (x k ) where the functions
More information* * * * * * * * * * * * * * * ** * **
Generalized Additive Models Trevor Hastie y and Robert Tibshirani z Department of Statistics and Division of Biostatistics Stanford University 12th May, 1995 Regression models play an important role in
More informationStatistics 203: Introduction to Regression and Analysis of Variance Course review
Statistics 203: Introduction to Regression and Analysis of Variance Course review Jonathan Taylor - p. 1/?? Today Review / overview of what we learned. - p. 2/?? General themes in regression models Specifying
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationIntegrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University
Integrated Likelihood Estimation in Semiparametric Regression Models Thomas A. Severini Department of Statistics Northwestern University Joint work with Heping He, University of York Introduction Let Y
More informationPenalized Splines, Mixed Models, and Recent Large-Sample Results
Penalized Splines, Mixed Models, and Recent Large-Sample Results David Ruppert Operations Research & Information Engineering, Cornell University Feb 4, 2011 Collaborators Matt Wand, University of Wollongong
More informationCombining estimates in regression and. classication. Michael LeBlanc. and. Robert Tibshirani. and. Department of Statistics. cuniversity of Toronto
Combining estimates in regression and classication Michael LeBlanc and Robert Tibshirani Department of Preventive Medicine and Biostatistics and Department of Statistics University of Toronto December
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationStatistics 203: Introduction to Regression and Analysis of Variance Penalized models
Statistics 203: Introduction to Regression and Analysis of Variance Penalized models Jonathan Taylor - p. 1/15 Today s class Bias-Variance tradeoff. Penalized regression. Cross-validation. - p. 2/15 Bias-variance
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationAbstract Nonparametric regression techniques such as spline smoothing and local tting depend implicitly on a parametric model. For instance, the cubic
Penalized Regression with Model-Based Penalties Nancy E. Heckman and J. O. Ramsay June 15, 1998 The authors wish to acknowledge the support of the Natural Sciences and Engineering Research Council of Canada
More informationLikelihood Ratio Tests. that Certain Variance Components Are Zero. Ciprian M. Crainiceanu. Department of Statistical Science
1 Likelihood Ratio Tests that Certain Variance Components Are Zero Ciprian M. Crainiceanu Department of Statistical Science www.people.cornell.edu/pages/cmc59 Work done jointly with David Ruppert, School
More informationBayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
More informationAn Introduction to GAMs based on penalized regression splines. Simon Wood Mathematical Sciences, University of Bath, U.K.
An Introduction to GAMs based on penalied regression splines Simon Wood Mathematical Sciences, University of Bath, U.K. Generalied Additive Models (GAM) A GAM has a form something like: g{e(y i )} = η
More informationLinear Models for Regression
Linear Models for Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationComments on \Wavelets in Statistics: A Review" by. A. Antoniadis. Jianqing Fan. University of North Carolina, Chapel Hill
Comments on \Wavelets in Statistics: A Review" by A. Antoniadis Jianqing Fan University of North Carolina, Chapel Hill and University of California, Los Angeles I would like to congratulate Professor Antoniadis
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationIntroduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
More informationComputer Vision Group Prof. Daniel Cremers. 4. Gaussian Processes - Regression
Group Prof. Daniel Cremers 4. Gaussian Processes - Regression Definition (Rep.) Definition: A Gaussian process is a collection of random variables, any finite number of which have a joint Gaussian distribution.
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationNONLINEAR CLASSIFICATION AND REGRESSION. J. Elder CSE 4404/5327 Introduction to Machine Learning and Pattern Recognition
NONLINEAR CLASSIFICATION AND REGRESSION Nonlinear Classification and Regression: Outline 2 Multi-Layer Perceptrons The Back-Propagation Learning Algorithm Generalized Linear Models Radial Basis Function
More informationChoosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation
Choosing the Summary Statistics and the Acceptance Rate in Approximate Bayesian Computation COMPSTAT 2010 Revised version; August 13, 2010 Michael G.B. Blum 1 Laboratoire TIMC-IMAG, CNRS, UJF Grenoble
More informationLikelihood NIPS July 30, Gaussian Process Regression with Student-t. Likelihood. Jarno Vanhatalo, Pasi Jylanki and Aki Vehtari NIPS-2009
with with July 30, 2010 with 1 2 3 Representation Representation for Distribution Inference for the Augmented Model 4 Approximate Laplacian Approximation Introduction to Laplacian Approximation Laplacian
More informationReduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation
Reduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation Curtis B. Storlie a a Los Alamos National Laboratory E-mail:storlie@lanl.gov Outline Reduction of Emulator
More informationComputer Vision Group Prof. Daniel Cremers. 9. Gaussian Processes - Regression
Group Prof. Daniel Cremers 9. Gaussian Processes - Regression Repetition: Regularized Regression Before, we solved for w using the pseudoinverse. But: we can kernelize this problem as well! First step:
More informationDirect Learning: Linear Classification. Donglin Zeng, Department of Biostatistics, University of North Carolina
Direct Learning: Linear Classification Logistic regression models for classification problem We consider two class problem: Y {0, 1}. The Bayes rule for the classification is I(P(Y = 1 X = x) > 1/2) so
More informationContents. Part I: Fundamentals of Bayesian Inference 1
Contents Preface xiii Part I: Fundamentals of Bayesian Inference 1 1 Probability and inference 3 1.1 The three steps of Bayesian data analysis 3 1.2 General notation for statistical inference 4 1.3 Bayesian
More informationPattern Recognition and Machine Learning
Christopher M. Bishop Pattern Recognition and Machine Learning ÖSpri inger Contents Preface Mathematical notation Contents vii xi xiii 1 Introduction 1 1.1 Example: Polynomial Curve Fitting 4 1.2 Probability
More informationSTA 4273H: Sta-s-cal Machine Learning
STA 4273H: Sta-s-cal Machine Learning Russ Salakhutdinov Department of Computer Science! Department of Statistical Sciences! rsalakhu@cs.toronto.edu! h0p://www.cs.utoronto.ca/~rsalakhu/ Lecture 2 In our
More informationAnalysing geoadditive regression data: a mixed model approach
Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression
More informationLogistic Regression. Seungjin Choi
Logistic Regression Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationRecap. HW due Thursday by 5 pm Next HW coming on Thursday Logistic regression: Pr(G = k X) linear on the logit scale Linear discriminant analysis:
1 / 23 Recap HW due Thursday by 5 pm Next HW coming on Thursday Logistic regression: Pr(G = k X) linear on the logit scale Linear discriminant analysis: Pr(G = k X) Pr(X G = k)pr(g = k) Theory: LDA more
More informationCSci 8980: Advanced Topics in Graphical Models Gaussian Processes
CSci 8980: Advanced Topics in Graphical Models Gaussian Processes Instructor: Arindam Banerjee November 15, 2007 Gaussian Processes Outline Gaussian Processes Outline Parametric Bayesian Regression Gaussian
More informationA Modern Look at Classical Multivariate Techniques
A Modern Look at Classical Multivariate Techniques Yoonkyung Lee Department of Statistics The Ohio State University March 16-20, 2015 The 13th School of Probability and Statistics CIMAT, Guanajuato, Mexico
More informationGaussian process for nonstationary time series prediction
Computational Statistics & Data Analysis 47 (2004) 705 712 www.elsevier.com/locate/csda Gaussian process for nonstationary time series prediction Soane Brahim-Belhouari, Amine Bermak EEE Department, Hong
More informationSpatial Process Estimates as Smoothers: A Review
Spatial Process Estimates as Smoothers: A Review Soutir Bandyopadhyay 1 Basic Model The observational model considered here has the form Y i = f(x i ) + ɛ i, for 1 i n. (1.1) where Y i is the observed
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.utstat.utoronto.ca/~rsalakhu/ Sidney Smith Hall, Room 6002 Lecture 3 Linear
More informationLatent Variable Models for Binary Data. Suppose that for a given vector of explanatory variables x, the latent
Latent Variable Models for Binary Data Suppose that for a given vector of explanatory variables x, the latent variable, U, has a continuous cumulative distribution function F (u; x) and that the binary
More informationBayesian Linear Regression
Bayesian Linear Regression Sudipto Banerjee 1 Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota, U.S.A. September 15, 2010 1 Linear regression models: a Bayesian perspective
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationDefault Priors and Effcient Posterior Computation in Bayesian
Default Priors and Effcient Posterior Computation in Bayesian Factor Analysis January 16, 2010 Presented by Eric Wang, Duke University Background and Motivation A Brief Review of Parameter Expansion Literature
More informationOdds ratio estimation in Bernoulli smoothing spline analysis-ofvariance
The Statistician (1997) 46, No. 1, pp. 49 56 Odds ratio estimation in Bernoulli smoothing spline analysis-ofvariance models By YUEDONG WANG{ University of Michigan, Ann Arbor, USA [Received June 1995.
More informationRegularization Paths
December 2005 Trevor Hastie, Stanford Statistics 1 Regularization Paths Trevor Hastie Stanford University drawing on collaborations with Brad Efron, Saharon Rosset, Ji Zhu, Hui Zhou, Rob Tibshirani and
More informationA short introduction to INLA and R-INLA
A short introduction to INLA and R-INLA Integrated Nested Laplace Approximation Thomas Opitz, BioSP, INRA Avignon Workshop: Theory and practice of INLA and SPDE November 7, 2018 2/21 Plan for this talk
More informationECE521 week 3: 23/26 January 2017
ECE521 week 3: 23/26 January 2017 Outline Probabilistic interpretation of linear regression - Maximum likelihood estimation (MLE) - Maximum a posteriori (MAP) estimation Bias-variance trade-off Linear
More informationMachine Learning Lecture 5
Machine Learning Lecture 5 Linear Discriminant Functions 26.10.2017 Bastian Leibe RWTH Aachen http://www.vision.rwth-aachen.de leibe@vision.rwth-aachen.de Course Outline Fundamentals Bayes Decision Theory
More informationNonparametric Bayesian Methods - Lecture I
Nonparametric Bayesian Methods - Lecture I Harry van Zanten Korteweg-de Vries Institute for Mathematics CRiSM Masterclass, April 4-6, 2016 Overview of the lectures I Intro to nonparametric Bayesian statistics
More informationAn Algorithm for Bayesian Variable Selection in High-dimensional Generalized Linear Models
Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session CPS023) p.3938 An Algorithm for Bayesian Variable Selection in High-dimensional Generalized Linear Models Vitara Pungpapong
More informationCOS 424: Interacting with Data
COS 424: Interacting with Data Lecturer: Rob Schapire Lecture #14 Scribe: Zia Khan April 3, 2007 Recall from previous lecture that in regression we are trying to predict a real value given our data. Specically,
More informationGaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency
Gaussian processes for spatial modelling in environmental health: parameterizing for flexibility vs. computational efficiency Chris Paciorek March 11, 2005 Department of Biostatistics Harvard School of
More informationEstimation of cumulative distribution function with spline functions
INTERNATIONAL JOURNAL OF ECONOMICS AND STATISTICS Volume 5, 017 Estimation of cumulative distribution function with functions Akhlitdin Nizamitdinov, Aladdin Shamilov Abstract The estimation of the cumulative
More informationIntroduction to Smoothing spline ANOVA models (metamodelling)
Introduction to Smoothing spline ANOVA models (metamodelling) M. Ratto DYNARE Summer School, Paris, June 215. Joint Research Centre www.jrc.ec.europa.eu Serving society Stimulating innovation Supporting
More informationSpatial smoothing using Gaussian processes
Spatial smoothing using Gaussian processes Chris Paciorek paciorek@hsph.harvard.edu August 5, 2004 1 OUTLINE Spatial smoothing and Gaussian processes Covariance modelling Nonstationary covariance modelling
More informationRelevance Vector Machines
LUT February 21, 2011 Support Vector Machines Model / Regression Marginal Likelihood Regression Relevance vector machines Exercise Support Vector Machines The relevance vector machine (RVM) is a bayesian
More informationADVANCED MACHINE LEARNING ADVANCED MACHINE LEARNING. Non-linear regression techniques Part - II
1 Non-linear regression techniques Part - II Regression Algorithms in this Course Support Vector Machine Relevance Vector Machine Support vector regression Boosting random projections Relevance vector
More informationOutline Lecture 2 2(32)
Outline Lecture (3), Lecture Linear Regression and Classification it is our firm belief that an understanding of linear models is essential for understanding nonlinear ones Thomas Schön Division of Automatic
More informationBANA 7046 Data Mining I Lecture 6. Other Data Mining Algorithms 1
BANA 7046 Data Mining I Lecture 6. Other Data Mining Algorithms 1 Shaobo Li University of Cincinnati 1 Partially based on Hastie, et al. (2009) ESL, and James, et al. (2013) ISLR Data Mining I Lecture
More informationMultivariate statistical methods and data mining in particle physics
Multivariate statistical methods and data mining in particle physics RHUL Physics www.pp.rhul.ac.uk/~cowan Academic Training Lectures CERN 16 19 June, 2008 1 Outline Statement of the problem Some general
More informationGeneralized Additive Models (GAMs)
Generalized Additive Models (GAMs) Israel Borokini Advanced Analysis Methods in Natural Resources and Environmental Science (NRES 746) October 3, 2016 Outline Quick refresher on linear regression Generalized
More informationBayesian Estimation and Inference for the Generalized Partial Linear Model
Bayesian Estimation Inference for the Generalized Partial Linear Model Haitham M. Yousof 1, Ahmed M. Gad 2 1 Department of Statistics, Mathematics Insurance, Benha University, Egypt. 2 Department of Statistics,
More informationPart 8: GLMs and Hierarchical LMs and GLMs
Part 8: GLMs and Hierarchical LMs and GLMs 1 Example: Song sparrow reproductive success Arcese et al., (1992) provide data on a sample from a population of 52 female song sparrows studied over the course
More informationPackage FDRreg. August 29, 2016
Package FDRreg August 29, 2016 Type Package Title False discovery rate regression Version 0.1 Date 2014-02-24 Author James G. Scott, with contributions from Rob Kass and Jesse Windle Maintainer James G.
More informationStatistical Data Mining and Machine Learning Hilary Term 2016
Statistical Data Mining and Machine Learning Hilary Term 2016 Dino Sejdinovic Department of Statistics Oxford Slides and other materials available at: http://www.stats.ox.ac.uk/~sejdinov/sdmml Naïve Bayes
More informationAutomatic Local Smoothing for Spectral Density. Abstract. This article uses local polynomial techniques to t Whittle's likelihood for spectral density
Automatic Local Smoothing for Spectral Density Estimation Jianqing Fan Department of Statistics University of North Carolina Chapel Hill, N.C. 27599-3260 Eva Kreutzberger Department of Mathematics University
More informationStat260: Bayesian Modeling and Inference Lecture Date: February 10th, Jeffreys priors. exp 1 ) p 2
Stat260: Bayesian Modeling and Inference Lecture Date: February 10th, 2010 Jeffreys priors Lecturer: Michael I. Jordan Scribe: Timothy Hunter 1 Priors for the multivariate Gaussian Consider a multivariate
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationA general mixed model approach for spatio-temporal regression data
A general mixed model approach for spatio-temporal regression data Thomas Kneib, Ludwig Fahrmeir & Stefan Lang Department of Statistics, Ludwig-Maximilians-University Munich 1. Spatio-temporal regression
More informationCan we do statistical inference in a non-asymptotic way? 1
Can we do statistical inference in a non-asymptotic way? 1 Guang Cheng 2 Statistics@Purdue www.science.purdue.edu/bigdata/ ONR Review Meeting@Duke Oct 11, 2017 1 Acknowledge NSF, ONR and Simons Foundation.
More informationGraphical Models for Collaborative Filtering
Graphical Models for Collaborative Filtering Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012 Sequence modeling HMM, Kalman Filter, etc.: Similarity: the same graphical model topology,
More informationMachine Learning Lecture 7
Course Outline Machine Learning Lecture 7 Fundamentals (2 weeks) Bayes Decision Theory Probability Density Estimation Statistical Learning Theory 23.05.2016 Discriminative Approaches (5 weeks) Linear Discriminant
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationGaussian Processes for Regression. Carl Edward Rasmussen. Department of Computer Science. Toronto, ONT, M5S 1A4, Canada.
In Advances in Neural Information Processing Systems 8 eds. D. S. Touretzky, M. C. Mozer, M. E. Hasselmo, MIT Press, 1996. Gaussian Processes for Regression Christopher K. I. Williams Neural Computing
More information1 Bayesian Linear Regression (BLR)
Statistical Techniques in Robotics (STR, S15) Lecture#10 (Wednesday, February 11) Lecturer: Byron Boots Gaussian Properties, Bayesian Linear Regression 1 Bayesian Linear Regression (BLR) In linear regression,
More informationRegression, Ridge Regression, Lasso
Regression, Ridge Regression, Lasso Fabio G. Cozman - fgcozman@usp.br October 2, 2018 A general definition Regression studies the relationship between a response variable Y and covariates X 1,..., X n.
More informationStat 5101 Lecture Notes
Stat 5101 Lecture Notes Charles J. Geyer Copyright 1998, 1999, 2000, 2001 by Charles J. Geyer May 7, 2001 ii Stat 5101 (Geyer) Course Notes Contents 1 Random Variables and Change of Variables 1 1.1 Random
More informationEfficient Bayesian Multivariate Surface Regression
Efficient Bayesian Multivariate Surface Regression Feng Li (joint with Mattias Villani) Department of Statistics, Stockholm University October, 211 Outline of the talk 1 Flexible regression models 2 The
More informationLecture 2: Linear Models. Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011
Lecture 2: Linear Models Bruce Walsh lecture notes Seattle SISG -Mixed Model Course version 23 June 2011 1 Quick Review of the Major Points The general linear model can be written as y = X! + e y = vector
More informationBacktting algorithms for total-variation and empirical-norm penalized additive modeling with high-dimensional data
The ISI's Journal for the Rapid Dissemination of Statistics Research (wileyonlinelibrary.com) DOI: 0.00X/sta.0000.................................................................................................
More informationRestricted Likelihood Ratio Tests in Nonparametric Longitudinal Models
Restricted Likelihood Ratio Tests in Nonparametric Longitudinal Models Short title: Restricted LR Tests in Longitudinal Models Ciprian M. Crainiceanu David Ruppert May 5, 2004 Abstract We assume that repeated
More informationMachine Learning. Lecture 4: Regularization and Bayesian Statistics. Feng Li. https://funglee.github.io
Machine Learning Lecture 4: Regularization and Bayesian Statistics Feng Li fli@sdu.edu.cn https://funglee.github.io School of Computer Science and Technology Shandong University Fall 207 Overfitting Problem
More informationModelling Survival Data using Generalized Additive Models with Flexible Link
Modelling Survival Data using Generalized Additive Models with Flexible Link Ana L. Papoila 1 and Cristina S. Rocha 2 1 Faculdade de Ciências Médicas, Dep. de Bioestatística e Informática, Universidade
More informationTowards a Bayesian model for Cyber Security
Towards a Bayesian model for Cyber Security Mark Briers (mbriers@turing.ac.uk) Joint work with Henry Clausen and Prof. Niall Adams (Imperial College London) 27 September 2017 The Alan Turing Institute
More informationCovariance Matrix Simplification For Efficient Uncertainty Management
PASEO MaxEnt 2007 Covariance Matrix Simplification For Efficient Uncertainty Management André Jalobeanu, Jorge A. Gutiérrez PASEO Research Group LSIIT (CNRS/ Univ. Strasbourg) - Illkirch, France *part
More informationOr How to select variables Using Bayesian LASSO
Or How to select variables Using Bayesian LASSO x 1 x 2 x 3 x 4 Or How to select variables Using Bayesian LASSO x 1 x 2 x 3 x 4 Or How to select variables Using Bayesian LASSO On Bayesian Variable Selection
More informationAlternatives. The D Operator
Using Smoothness Alternatives Text: Chapter 5 Some disadvantages of basis expansions Discrete choice of number of basis functions additional variability. Non-hierarchical bases (eg B-splines) make life
More informationProbabilistic Matrix Factorization
Probabilistic Matrix Factorization David M. Blei Columbia University November 25, 2015 1 Dyadic data One important type of modern data is dyadic data. Dyadic data are measurements on pairs. The idea is
More informationtype of the at; and an investigation of duration of unemployment in West Germany from 1980 to 1995, with spatio-temporal data from the German Federal
Bayesian Inference for Generalized Additive Mixed Models Based on Markov Random Field Priors Ludwig Fahrmeir and Stefan Lang University of Munich, Ludwigstr. 33, 80539 Munich email: fahrmeir@stat.uni-muenchen.de
More informationBoosting Methods: Why They Can Be Useful for High-Dimensional Data
New URL: http://www.r-project.org/conferences/dsc-2003/ Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003) March 20 22, Vienna, Austria ISSN 1609-395X Kurt Hornik,
More informationLecture 6: Bayesian Inference in SDE Models
Lecture 6: Bayesian Inference in SDE Models Bayesian Filtering and Smoothing Point of View Simo Särkkä Aalto University Simo Särkkä (Aalto) Lecture 6: Bayesian Inference in SDEs 1 / 45 Contents 1 SDEs
More informationSTA414/2104. Lecture 11: Gaussian Processes. Department of Statistics
STA414/2104 Lecture 11: Gaussian Processes Department of Statistics www.utstat.utoronto.ca Delivered by Mark Ebden with thanks to Russ Salakhutdinov Outline Gaussian Processes Exam review Course evaluations
More informationLinear Regression and Its Applications
Linear Regression and Its Applications Predrag Radivojac October 13, 2014 Given a data set D = {(x i, y i )} n the objective is to learn the relationship between features and the target. We usually start
More informationBiostatistics Advanced Methods in Biostatistics IV
Biostatistics 140.754 Advanced Methods in Biostatistics IV Jeffrey Leek Assistant Professor Department of Biostatistics jleek@jhsph.edu Lecture 12 1 / 36 Tip + Paper Tip: As a statistician the results
More informationStatistical Methods for Data Mining
Statistical Methods for Data Mining Kuangnan Fang Xiamen University Email: xmufkn@xmu.edu.cn Support Vector Machines Here we approach the two-class classification problem in a direct way: We try and find
More informationMarkov Chain Monte Carlo Algorithms for Gaussian Processes
Markov Chain Monte Carlo Algorithms for Gaussian Processes Michalis K. Titsias, Neil Lawrence and Magnus Rattray School of Computer Science University of Manchester June 8 Outline Gaussian Processes Sampling
More informationLinear Regression Linear Regression with Shrinkage
Linear Regression Linear Regression ith Shrinkage Introduction Regression means predicting a continuous (usually scalar) output y from a vector of continuous inputs (features) x. Example: Predicting vehicle
More information