Spatially Adaptive Smoothing Splines
|
|
- Christian Harrison
- 5 years ago
- Views:
Transcription
1 Spatially Adaptive Smoothing Splines Paul Speckman University of Missouri-Columbia September 11, 23 Banff 9/7/3
2 Ordinary Simple Spline Smoothing Observe y i = f(t i ) + ε i, = 1,, n ε i iid N(, σ 2 ) t i [, 1] WLOG f Only known to be smooth min f n (y i f(t i )) 2 + λ i=1 1 f (p) (t) 2 dt Silverman s equivalent kernel: ˆf(t) 1 n n ( t ti w i=1 λ 1/(2p+1) ) y i Banff 9/7/3 1
3 Example: Heaviside function 18 true data ( ) and noisy data () f true (x) and f noisy (x) n = x Banff 9/7/3 2
4 16 true data ( ) and best fit Smooth Spl f true (x) and f SS (x) x Banff 9/7/3 3
5 Spatially adaptive Idea: Somehow adjust penalty for roughness of f min f n (y i f(t i )) 2 + i=1 1 λ(t)f (p) (t) 2 dt for a good choice of λ(t) > Banff 9/7/3 4
6 Problem: choose λ(t) from the data Two proposals Frequentist: Pintore, Speckman, Holmes (23) Use GCV Bayesian w/ Dongchu Sun Use discretized approximation to smoothing spline Type of stochastic volatility model In progress Related work Local bandwidth kernel smoothers Local GCV: Cummins, Filloon, Nychka (21) Adaptive P-splines: Ruppert and Carroll Banff 9/7/3 5
7 Toward frequentist exact solution min f n (y i f(t i )) 2 + i=1 Special case of L-spline with Lf = λd p f: 1 λ(t)f (p) (t) 2 dt min f n (y i f(t i )) 2 + i=1 1 (Lf(t)) 2 dt Reproducing kernel: (eg, Gu or Heckman and Ramsay) K λ (s, t) = G(s, u) = 1 1 G(s, u)g(t, u) du λ(u) (s u)p 1 + (p 1)! Banff 9/7/3 6
8 Solution satisfies ˆf(t) = n c j K λ (t, t j ) + j=1 f = Σ λ c + Td Σ λ = [K λ (t i, t j )] n n 1 λ (f (p) ) 2 = f Σ λ f p 1 j= Problem becomes min y Σ λ c Td 2 + f Σ λ f f One common solution: Factorize [ ] R T = [Q 1 Q 2 ] Q = [Q 1 Q 2 ] orthogonal d j φ j (t) Banff 9/7/3 7
9 Solution (see Wahba or Gu) ˆf = Aλ y I A λ = nq 2 (Q 2M λ Q 2 ) 1 Q 2 M λ = Σ λ + ni So ˆf = y nq2 (Q 2M λ Q 2 ) 1 Q 2y Banff 9/7/3 8
10 Special case: piecewise constant λ We fix = τ < τ 1 < < τ K < τ K+1 = 1 Assume each τ k = t i, some i Piecewise constant: λ(t) = λ k = e γ k, t [τ k 1, )τ k ) Explicit (but messy) formula for Σ λ for p = 2 ˆf λ is a polynomial spline with multiple knots at τ k Banff 9/7/3 9
11 Choosing appropriate λ by GCV We use Generalized Cross Validation with extra cost term: Cost κ 1 A λ does not diagonalize V (λ 1,, λ K ) = n (I A λ)y 2 (tr(i κa λ )) 2 Brute force: Matlab, Nelder-Mead optimization Seems to work for K 2 Banff 9/7/3 1
12 Examples Heaviside: K = 5 16 true data ( ) and best fit Spat Adapt Spl with 5 jumps f true (x) and f SAS(5) (x) x Banff 9/7/3 11
13 Bayesian interpretation: Wahba, 1978 Reasonable prior X(t) is mean zero Gaussian process with covariance K λ (s, t) Diffuse prior on X(),, D p 1 X() Then posterior is spatially adaptive spline: normal with mean A λ y and covariance σ 2 A λ (Wahba, 1978) Nychka showed that average frequentist coverage tends to have right level Banff 9/7/3 12
14 Bayes credibility interval 2 Spat Adapt Spl with 5 jumps ( ) and 95% Confidence Intervals 15 f SAS(5) (x) and 95% confidence intervals x Banff 9/7/3 13
15 Doppler function f(t) = (t(1 t)) 1/2 sin 2π(1 + a)/(t + a), a = 5, n = 128, S-N = 7 8 true data ( ) and noisy data () 6 4 f true (x) and f noisy (x) x Banff 9/7/3 14
16 Doppler: ordinary smoothing 5 true data ( ) and best fit Smooth Spl f true (x) and f SS (x) x Banff 9/7/3 15
17 Doppler:K = 5 5 true data ( ) and best fit Spat Adapt Spl with 5 jumps f true (x) and f SAS(5) (x) x Banff 9/7/3 16
18 Doppler:K = 1 5 true data ( ) and best fit Spat Adapt Spl with 1 jumps f true (x) and f SAS(1) (x) x Banff 9/7/3 17
19 Doppler:K = 2 6 true data ( ) and best fit Spat Adapt Spl with 2 jumps 4 2 f true (x) and f SAS(2) (x) x Banff 9/7/3 18
20 Doppler:K = 5, Bayesian interval 6 Spat Adapt Spl with 5 jumps ( ) and 95% Confidence Intervals 4 f SAS(5) (x) and 95% confidence intervals x Banff 9/7/3 19
21 Experiments with κ: Heaviside, K = 1, κ = 1 18 true data ( ) and Spat Adapt Spl with 1 jumps ( ), κ = f true (x) and f SAS(1) (x), κ = x Banff 9/7/3 2
22 Heaviside, K = 1, κ = true data ( ) and Spat Adapt Spl with 1 jumps ( ), κ = f true (x) and f SAS(1) (x), κ = x Banff 9/7/3 21
23 Heaviside, K = 1, κ = true data ( ) and Spat Adapt Spl with 1 jumps ( ), κ = f true (x) and f SAS(1) (x), κ = x Banff 9/7/3 22
24 Conclusions for this part κ = 14 works pretty well in at least one example K is another regularizing parameter K = 5 or 1 seemed to work well We can get empirical Bayes credibility intervals for f Much more research needed Banff 9/7/3 23
25 Bayesian model Our simplified approach Discretize Use MCMC to get posterior quantities Banff 9/7/3 24
26 Bayesian model for spline smoothing (Wahba, 1978) Prior for f: D p X(t) = b dw (t) Diffuse prior: X(), DX(),, D p 1 X() Posterior for f with this prior: normal Mean = smoothing spline with λ = σ 2 /b Variance = σ 2 A λ Kohn and coauthors have championed this model; Hastie and Tibshirani (22) Banff 9/7/3 25
27 Discretize the prior: assume t i s equally spaced (Whitaker) x i = x i x i 1 2 x i = x i x i 1 etc Assume p x i iid N(, b), i = p + 1,, n x 1, x 1,, p 1 x p diffuse Equivalent forms: Bx N(, bi n p ) p(x) b (n p)/2 exp ( 1 ) 2b x B Bx Banff 9/7/3 26
28 (Partially informative prior) B B is Beran s annihilator matrix For moderate n, fit is indistinguishable from regular smoothing spline Banff 9/7/3 27
29 Rich class of priors on function space Equivalent? min f,γ n (y i f(t i )) 2 + i=1 1 e γ(t) f (p) (t) 2 dt + η 1 γ (q) (t) 2 dt Banff 9/7/3 28
30 Bayes solution Inverse gamma prior on σ 2 Inverse gamma prior on η MCMC: Markov chain Monte Carlo simulation to estimate posterior quantities Gibbs sampling is relatively simple: x y, γ, η, σ 2 has multivariate normal smoothing spline posterior σ 2 y, x, γ, η is inverse gamma η y, x, γ, σ 2 is inverse gamma γ i γ i, y, x, σ 2, η does not have nice form and we now sample one component at a time Mixing can be slow, especially for p > 2 Banff 9/7/3 29
31 Example: Doppler, n = 5, nonadaptive x Banff 9/7/3 3
32 Example: Doppler, n = 5, adaptive delta_k Banff 9/7/3 31
33 Computational issues: p = 3 would be better MCMC convergence issues What is the continuous version of this process? Ideas extend to other penalities such as L 1 In principle extends to higher dimensions Data dependent penalties open many new possibilities Banff 9/7/3 32
Inversion Base Height. Daggot Pressure Gradient Visibility (miles)
Stanford University June 2, 1998 Bayesian Backtting: 1 Bayesian Backtting Trevor Hastie Stanford University Rob Tibshirani University of Toronto Email: trevor@stat.stanford.edu Ftp: stat.stanford.edu:
More informationIntegrated Likelihood Estimation in Semiparametric Regression Models. Thomas A. Severini Department of Statistics Northwestern University
Integrated Likelihood Estimation in Semiparametric Regression Models Thomas A. Severini Department of Statistics Northwestern University Joint work with Heping He, University of York Introduction Let Y
More informationBayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
More informationModelling geoadditive survival data
Modelling geoadditive survival data Thomas Kneib & Ludwig Fahrmeir Department of Statistics, Ludwig-Maximilians-University Munich 1. Leukemia survival data 2. Structured hazard regression 3. Mixed model
More informationPriors for Bayesian adaptive spline smoothing
Ann Inst Stat Math ) 64:577 63 DOI.7/s463--3-6 Priors for Bayesian adaptive spline smoothing Yu Ryan Yue Paul L. Speckman Dongchu Sun Received: September 9 / Revised: July / Published online: 8 January
More informationIntroduction to Smoothing spline ANOVA models (metamodelling)
Introduction to Smoothing spline ANOVA models (metamodelling) M. Ratto DYNARE Summer School, Paris, June 215. Joint Research Centre www.jrc.ec.europa.eu Serving society Stimulating innovation Supporting
More informationOdds ratio estimation in Bernoulli smoothing spline analysis-ofvariance
The Statistician (1997) 46, No. 1, pp. 49 56 Odds ratio estimation in Bernoulli smoothing spline analysis-ofvariance models By YUEDONG WANG{ University of Michigan, Ann Arbor, USA [Received June 1995.
More informationAnalysing geoadditive regression data: a mixed model approach
Analysing geoadditive regression data: a mixed model approach Institut für Statistik, Ludwig-Maximilians-Universität München Joint work with Ludwig Fahrmeir & Stefan Lang 25.11.2005 Spatio-temporal regression
More informationPenalized Splines, Mixed Models, and Recent Large-Sample Results
Penalized Splines, Mixed Models, and Recent Large-Sample Results David Ruppert Operations Research & Information Engineering, Cornell University Feb 4, 2011 Collaborators Matt Wand, University of Wollongong
More informationMCMC Sampling for Bayesian Inference using L1-type Priors
MÜNSTER MCMC Sampling for Bayesian Inference using L1-type Priors (what I do whenever the ill-posedness of EEG/MEG is just not frustrating enough!) AG Imaging Seminar Felix Lucka 26.06.2012 , MÜNSTER Sampling
More informationA STATISTICAL TECHNIQUE FOR MODELLING NON-STATIONARY SPATIAL PROCESSES
A STATISTICAL TECHNIQUE FOR MODELLING NON-STATIONARY SPATIAL PROCESSES JOHN STEPHENSON 1, CHRIS HOLMES, KERRY GALLAGHER 1 and ALEXANDRE PINTORE 1 Dept. Earth Science and Engineering, Imperial College,
More informationDoubly Penalized Likelihood Estimator in Heteroscedastic Regression 1
DEPARTMENT OF STATISTICS University of Wisconsin 1210 West Dayton St. Madison, WI 53706 TECHNICAL REPORT NO. 1084rr February 17, 2004 Doubly Penalized Likelihood Estimator in Heteroscedastic Regression
More informationSTA414/2104 Statistical Methods for Machine Learning II
STA414/2104 Statistical Methods for Machine Learning II Murat A. Erdogdu & David Duvenaud Department of Computer Science Department of Statistical Sciences Lecture 3 Slide credits: Russ Salakhutdinov Announcements
More informationWeb Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D.
Web Appendix for Hierarchical Adaptive Regression Kernels for Regression with Functional Predictors by D. B. Woodard, C. Crainiceanu, and D. Ruppert A. EMPIRICAL ESTIMATE OF THE KERNEL MIXTURE Here we
More informationFunctional Estimation in Systems Defined by Differential Equation using Bayesian Smoothing Methods
Université Catholique de Louvain Institut de Statistique, Biostatistique et Sciences Actuarielles Functional Estimation in Systems Defined by Differential Equation using Bayesian Smoothing Methods 19th
More informationAn Introduction to GAMs based on penalized regression splines. Simon Wood Mathematical Sciences, University of Bath, U.K.
An Introduction to GAMs based on penalied regression splines Simon Wood Mathematical Sciences, University of Bath, U.K. Generalied Additive Models (GAM) A GAM has a form something like: g{e(y i )} = η
More informationEffective Computation for Odds Ratio Estimation in Nonparametric Logistic Regression
Communications of the Korean Statistical Society 2009, Vol. 16, No. 4, 713 722 Effective Computation for Odds Ratio Estimation in Nonparametric Logistic Regression Young-Ju Kim 1,a a Department of Information
More informationSupplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements
Supplement to A Hierarchical Approach for Fitting Curves to Response Time Measurements Jeffrey N. Rouder Francis Tuerlinckx Paul L. Speckman Jun Lu & Pablo Gomez May 4 008 1 The Weibull regression model
More informationBayesian Modeling of Conditional Distributions
Bayesian Modeling of Conditional Distributions John Geweke University of Iowa Indiana University Department of Economics February 27, 2007 Outline Motivation Model description Methods of inference Earnings
More informationSTAT 518 Intro Student Presentation
STAT 518 Intro Student Presentation Wen Wei Loh April 11, 2013 Title of paper Radford M. Neal [1999] Bayesian Statistics, 6: 475-501, 1999 What the paper is about Regression and Classification Flexible
More informationDensity Estimation. Seungjin Choi
Density Estimation Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr http://mlg.postech.ac.kr/
More informationFahrmeir: Recent Advances in Semiparametric Bayesian Function Estimation
Fahrmeir: Recent Advances in Semiparametric Bayesian Function Estimation Sonderforschungsbereich 386, Paper 137 (1998) Online unter: http://epub.ub.uni-muenchen.de/ Projektpartner Recent Advances in Semiparametric
More informationKernel density estimation in R
Kernel density estimation in R Kernel density estimation can be done in R using the density() function in R. The default is a Guassian kernel, but others are possible also. It uses it s own algorithm to
More informationUncertainty Quantification for Inverse Problems. November 7, 2011
Uncertainty Quantification for Inverse Problems November 7, 2011 Outline UQ and inverse problems Review: least-squares Review: Gaussian Bayesian linear model Parametric reductions for IP Bias, variance
More informationNonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University
Nonstationary spatial process modeling Part II Paul D. Sampson --- Catherine Calder Univ of Washington --- Ohio State University this presentation derived from that presented at the Pan-American Advanced
More informationAn example of Bayesian reasoning Consider the one-dimensional deconvolution problem with various degrees of prior information.
An example of Bayesian reasoning Consider the one-dimensional deconvolution problem with various degrees of prior information. Model: where g(t) = a(t s)f(s)ds + e(t), a(t) t = (rapidly). The problem,
More informationEcon 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines
Econ 2148, fall 2017 Gaussian process priors, reproducing kernel Hilbert spaces, and Splines Maximilian Kasy Department of Economics, Harvard University 1 / 37 Agenda 6 equivalent representations of the
More informationAlternatives. The D Operator
Using Smoothness Alternatives Text: Chapter 5 Some disadvantages of basis expansions Discrete choice of number of basis functions additional variability. Non-hierarchical bases (eg B-splines) make life
More informationSpatial Process Estimates as Smoothers: A Review
Spatial Process Estimates as Smoothers: A Review Soutir Bandyopadhyay 1 Basic Model The observational model considered here has the form Y i = f(x i ) + ɛ i, for 1 i n. (1.1) where Y i is the observed
More informationHypothesis Testing in Smoothing Spline Models
Hypothesis Testing in Smoothing Spline Models Anna Liu and Yuedong Wang October 10, 2002 Abstract This article provides a unified and comparative review of some existing test methods for the hypothesis
More informationLecture 8: The Metropolis-Hastings Algorithm
30.10.2008 What we have seen last time: Gibbs sampler Key idea: Generate a Markov chain by updating the component of (X 1,..., X p ) in turn by drawing from the full conditionals: X (t) j Two drawbacks:
More informationIntroduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones
Introduction to machine learning and pattern recognition Lecture 2 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 Last week... supervised and unsupervised methods need adaptive
More informationCPSC 540: Machine Learning
CPSC 540: Machine Learning MCMC and Non-Parametric Bayes Mark Schmidt University of British Columbia Winter 2016 Admin I went through project proposals: Some of you got a message on Piazza. No news is
More informationSpatial Statistics with Image Analysis. Outline. A Statistical Approach. Johan Lindström 1. Lund October 6, 2016
Spatial Statistics Spatial Examples More Spatial Statistics with Image Analysis Johan Lindström 1 1 Mathematical Statistics Centre for Mathematical Sciences Lund University Lund October 6, 2016 Johan Lindström
More informationSequential Monte Carlo Methods for Bayesian Computation
Sequential Monte Carlo Methods for Bayesian Computation A. Doucet Kyoto Sept. 2012 A. Doucet (MLSS Sept. 2012) Sept. 2012 1 / 136 Motivating Example 1: Generic Bayesian Model Let X be a vector parameter
More informationA short introduction to INLA and R-INLA
A short introduction to INLA and R-INLA Integrated Nested Laplace Approximation Thomas Opitz, BioSP, INRA Avignon Workshop: Theory and practice of INLA and SPDE November 7, 2018 2/21 Plan for this talk
More informationNonparametric Bayesian Methods - Lecture I
Nonparametric Bayesian Methods - Lecture I Harry van Zanten Korteweg-de Vries Institute for Mathematics CRiSM Masterclass, April 4-6, 2016 Overview of the lectures I Intro to nonparametric Bayesian statistics
More informationBayesian Regularization
Bayesian Regularization Aad van der Vaart Vrije Universiteit Amsterdam International Congress of Mathematicians Hyderabad, August 2010 Contents Introduction Abstract result Gaussian process priors Co-authors
More informationLecture 2: From Linear Regression to Kalman Filter and Beyond
Lecture 2: From Linear Regression to Kalman Filter and Beyond January 18, 2017 Contents 1 Batch and Recursive Estimation 2 Towards Bayesian Filtering 3 Kalman Filter and Bayesian Filtering and Smoothing
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT68 Winter 8) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
More informationFundamental Issues in Bayesian Functional Data Analysis. Dennis D. Cox Rice University
Fundamental Issues in Bayesian Functional Data Analysis Dennis D. Cox Rice University 1 Introduction Question: What are functional data? Answer: Data that are functions of a continuous variable.... say
More informationBeyond Mean Regression
Beyond Mean Regression Thomas Kneib Lehrstuhl für Statistik Georg-August-Universität Göttingen 8.3.2013 Innsbruck Introduction Introduction One of the top ten reasons to become statistician (according
More informationAlternatives to Basis Expansions. Kernels in Density Estimation. Kernels and Bandwidth. Idea Behind Kernel Methods
Alternatives to Basis Expansions Basis expansions require either choice of a discrete set of basis or choice of smoothing penalty and smoothing parameter Both of which impose prior beliefs on data. Alternatives
More informationGaussian Processes. Le Song. Machine Learning II: Advanced Topics CSE 8803ML, Spring 2012
Gaussian Processes Le Song Machine Learning II: Advanced Topics CSE 8803ML, Spring 01 Pictorial view of embedding distribution Transform the entire distribution to expected features Feature space Feature
More informationRiemann Manifold Methods in Bayesian Statistics
Ricardo Ehlers ehlers@icmc.usp.br Applied Maths and Stats University of São Paulo, Brazil Working Group in Statistical Learning University College Dublin September 2015 Bayesian inference is based on Bayes
More informationReduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation
Reduction of Model Complexity and the Treatment of Discrete Inputs in Computer Model Emulation Curtis B. Storlie a a Los Alamos National Laboratory E-mail:storlie@lanl.gov Outline Reduction of Emulator
More informationDirect Learning: Linear Classification. Donglin Zeng, Department of Biostatistics, University of North Carolina
Direct Learning: Linear Classification Logistic regression models for classification problem We consider two class problem: Y {0, 1}. The Bayes rule for the classification is I(P(Y = 1 X = x) > 1/2) so
More informationStatistics & Data Sciences: First Year Prelim Exam May 2018
Statistics & Data Sciences: First Year Prelim Exam May 2018 Instructions: 1. Do not turn this page until instructed to do so. 2. Start each new question on a new sheet of paper. 3. This is a closed book
More informationI N S T I T U T D E S T A T I S T I Q U E B I O S T A T I S T I Q U E E T S C I E N C E S A C T U A R I E L L E S (I S B A)
I N S T I T U T D E S T A T I S T I Q U E B I O S T A T I S T I Q U E E T S C I E N C E S A C T U A R I E L L E S I S B A UNIVERSITÉ CATHOLIQUE DE LOUVAIN D I S C U S S I O N P A P E R 2011/01 1101 BAYESIAN
More informationEconometrics I, Estimation
Econometrics I, Estimation Department of Economics Stanford University September, 2008 Part I Parameter, Estimator, Estimate A parametric is a feature of the population. An estimator is a function of the
More informationNonparametric Drift Estimation for Stochastic Differential Equations
Nonparametric Drift Estimation for Stochastic Differential Equations Gareth Roberts 1 Department of Statistics University of Warwick Brazilian Bayesian meeting, March 2010 Joint work with O. Papaspiliopoulos,
More informationKneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach"
Kneib, Fahrmeir: Supplement to "Structured additive regression for categorical space-time data: A mixed model approach" Sonderforschungsbereich 386, Paper 43 (25) Online unter: http://epub.ub.uni-muenchen.de/
More informationAutomatic Smoothing and Variable Selection. via Regularization 1
DEPARTMENT OF STATISTICS University of Wisconsin 1210 West Dayton St. Madison, WI 53706 TECHNICAL REPORT NO. 1093 July 6, 2004 Automatic Smoothing and Variable Selection via Regularization 1 Ming Yuan
More informationConstrained Gaussian processes: methodology, theory and applications
Constrained Gaussian processes: methodology, theory and applications Hassan Maatouk hassan.maatouk@univ-rennes2.fr Workshop on Gaussian Processes, November 6-7, 2017, St-Etienne (France) Hassan Maatouk
More informationAdvanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering
Advanced Computational Methods in Statistics: Lecture 5 Sequential Monte Carlo/Particle Filtering Axel Gandy Department of Mathematics Imperial College London http://www2.imperial.ac.uk/~agandy London
More informationStatistical inference on Lévy processes
Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop 2013 26th March Outline
More informationCheng Soon Ong & Christian Walder. Canberra February June 2018
Cheng Soon Ong & Christian Walder Research Group and College of Engineering and Computer Science Canberra February June 2018 (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 254 Part V
More informationMalliavin Calculus in Finance
Malliavin Calculus in Finance Peter K. Friz 1 Greeks and the logarithmic derivative trick Model an underlying assent by a Markov process with values in R m with dynamics described by the SDE dx t = b(x
More informationVast Volatility Matrix Estimation for High Frequency Data
Vast Volatility Matrix Estimation for High Frequency Data Yazhen Wang National Science Foundation Yale Workshop, May 14-17, 2009 Disclaimer: My opinion, not the views of NSF Y. Wang (at NSF) 1 / 36 Outline
More informationA general mixed model approach for spatio-temporal regression data
A general mixed model approach for spatio-temporal regression data Thomas Kneib, Ludwig Fahrmeir & Stefan Lang Department of Statistics, Ludwig-Maximilians-University Munich 1. Spatio-temporal regression
More informationA new iterated filtering algorithm
A new iterated filtering algorithm Edward Ionides University of Michigan, Ann Arbor ionides@umich.edu Statistics and Nonlinear Dynamics in Biology and Medicine Thursday July 31, 2014 Overview 1 Introduction
More informationBasis Penalty Smoothers. Simon Wood Mathematical Sciences, University of Bath, U.K.
Basis Penalty Smoothers Simon Wood Mathematical Sciences, University of Bath, U.K. Estimating functions It is sometimes useful to estimate smooth functions from data, without being too precise about the
More informationPhysics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
More informationGaussian processes for inference in stochastic differential equations
Gaussian processes for inference in stochastic differential equations Manfred Opper, AI group, TU Berlin November 6, 2017 Manfred Opper, AI group, TU Berlin (TU Berlin) inference in SDE November 6, 2017
More informationConsistent high-dimensional Bayesian variable selection via penalized credible regions
Consistent high-dimensional Bayesian variable selection via penalized credible regions Howard Bondell bondell@stat.ncsu.edu Joint work with Brian Reich Howard Bondell p. 1 Outline High-Dimensional Variable
More informationOn fixed effects estimation in spline-based semiparametric regression for spatial data
Libraries Conference on Applied Statistics in Agriculture 015-7th Annual Conference Proceedings On fixed effects estimation in spline-based semiparametric regression for spatial data Guilherme Ludwig University
More informationERRATA. for Semiparametric Regression. Last Updated: 30th September, 2014
1 ERRATA for Semiparametric Regression by D. Ruppert, M. P. Wand and R. J. Carroll Last Updated: 30th September, 2014 p.6. In the vertical axis Figure 1.7 the lower 1, 2 and 3 should have minus signs.
More informationStat 451 Lecture Notes Numerical Integration
Stat 451 Lecture Notes 03 12 Numerical Integration Ryan Martin UIC www.math.uic.edu/~rgmartin 1 Based on Chapter 5 in Givens & Hoeting, and Chapters 4 & 18 of Lange 2 Updated: February 11, 2016 1 / 29
More informationNonparametric Bayesian Methods (Gaussian Processes)
[70240413 Statistical Machine Learning, Spring, 2015] Nonparametric Bayesian Methods (Gaussian Processes) Jun Zhu dcszj@mail.tsinghua.edu.cn http://bigml.cs.tsinghua.edu.cn/~jun State Key Lab of Intelligent
More informationBayesian Analysis of Multivariate Smoothing Splines
Bayesian Analysis of Multivariate Smoothing Splines Dongchu Sun, Department of Statistics, University of Missouri, Columbia, MO 65211, USA Shawn Ni, Department of Economics, University of Missouri, Columbia,
More informationMarkov Chain Monte Carlo methods
Markov Chain Monte Carlo methods By Oleg Makhnin 1 Introduction a b c M = d e f g h i 0 f(x)dx 1.1 Motivation 1.1.1 Just here Supresses numbering 1.1.2 After this 1.2 Literature 2 Method 2.1 New math As
More informationMultivariate Normal & Wishart
Multivariate Normal & Wishart Hoff Chapter 7 October 21, 2010 Reading Comprehesion Example Twenty-two children are given a reading comprehsion test before and after receiving a particular instruction method.
More informationThe Metropolis-Hastings Algorithm. June 8, 2012
The Metropolis-Hastings Algorithm June 8, 22 The Plan. Understand what a simulated distribution is 2. Understand why the Metropolis-Hastings algorithm works 3. Learn how to apply the Metropolis-Hastings
More informationGeneralized Cross Validation
26 november 2009 Plan 1 Generals 2 What Regularization Parameter λ is Optimal? Examples Dening the Optimal λ 3 Generalized Cross-Validation (GCV) Convergence Result 4 Discussion Plan 1 Generals 2 What
More informationComputational statistics
Computational statistics Markov Chain Monte Carlo methods Thierry Denœux March 2017 Thierry Denœux Computational statistics March 2017 1 / 71 Contents of this chapter When a target density f can be evaluated
More informationOn Bayesian Computation
On Bayesian Computation Michael I. Jordan with Elaine Angelino, Maxim Rabinovich, Martin Wainwright and Yun Yang Previous Work: Information Constraints on Inference Minimize the minimax risk under constraints
More informationPART I INTRODUCTION The meaning of probability Basic definitions for frequentist statistics and Bayesian inference Bayesian inference Combinatorics
Table of Preface page xi PART I INTRODUCTION 1 1 The meaning of probability 3 1.1 Classical definition of probability 3 1.2 Statistical definition of probability 9 1.3 Bayesian understanding of probability
More informationFactorization of Seperable and Patterned Covariance Matrices for Gibbs Sampling
Monte Carlo Methods Appl, Vol 6, No 3 (2000), pp 205 210 c VSP 2000 Factorization of Seperable and Patterned Covariance Matrices for Gibbs Sampling Daniel B Rowe H & SS, 228-77 California Institute of
More informationBayesian Methods for Machine Learning
Bayesian Methods for Machine Learning CS 584: Big Data Analytics Material adapted from Radford Neal s tutorial (http://ftp.cs.utoronto.ca/pub/radford/bayes-tut.pdf), Zoubin Ghahramni (http://hunch.net/~coms-4771/zoubin_ghahramani_bayesian_learning.pdf),
More informationBayesian Nonparametrics
Bayesian Nonparametrics Peter Orbanz Columbia University PARAMETERS AND PATTERNS Parameters P(X θ) = Probability[data pattern] 3 2 1 0 1 2 3 5 0 5 Inference idea data = underlying pattern + independent
More informationScalable MCMC for the horseshoe prior
Scalable MCMC for the horseshoe prior Anirban Bhattacharya Department of Statistics, Texas A&M University Joint work with James Johndrow and Paolo Orenstein September 7, 2018 Cornell Day of Statistics
More informationStatistical Inference and Methods
Department of Mathematics Imperial College London d.stephens@imperial.ac.uk http://stats.ma.ic.ac.uk/ das01/ 31st January 2006 Part VI Session 6: Filtering and Time to Event Data Session 6: Filtering and
More informationAnalysis Methods for Supersaturated Design: Some Comparisons
Journal of Data Science 1(2003), 249-260 Analysis Methods for Supersaturated Design: Some Comparisons Runze Li 1 and Dennis K. J. Lin 2 The Pennsylvania State University Abstract: Supersaturated designs
More informationSTA 294: Stochastic Processes & Bayesian Nonparametrics
MARKOV CHAINS AND CONVERGENCE CONCEPTS Markov chains are among the simplest stochastic processes, just one step beyond iid sequences of random variables. Traditionally they ve been used in modelling a
More informationStochastic optimal control with rough paths
Stochastic optimal control with rough paths Paul Gassiat TU Berlin Stochastic processes and their statistics in Finance, Okinawa, October 28, 2013 Joint work with Joscha Diehl and Peter Friz Introduction
More informationStochastic Collocation Methods for Polynomial Chaos: Analysis and Applications
Stochastic Collocation Methods for Polynomial Chaos: Analysis and Applications Dongbin Xiu Department of Mathematics, Purdue University Support: AFOSR FA955-8-1-353 (Computational Math) SF CAREER DMS-64535
More informationCalibrating Environmental Engineering Models and Uncertainty Analysis
Models and Cornell University Oct 14, 2008 Project Team Christine Shoemaker, co-pi, Professor of Civil and works in applied optimization, co-pi Nikolai Blizniouk, PhD student in Operations Research now
More informationLecture 7 and 8: Markov Chain Monte Carlo
Lecture 7 and 8: Markov Chain Monte Carlo 4F13: Machine Learning Zoubin Ghahramani and Carl Edward Rasmussen Department of Engineering University of Cambridge http://mlg.eng.cam.ac.uk/teaching/4f13/ Ghahramani
More informationBayesian inverse problems with Laplacian noise
Bayesian inverse problems with Laplacian noise Remo Kretschmann Faculty of Mathematics, University of Duisburg-Essen Applied Inverse Problems 2017, M27 Hangzhou, 1 June 2017 1 / 33 Outline 1 Inverse heat
More informationLikelihood Ratio Tests. that Certain Variance Components Are Zero. Ciprian M. Crainiceanu. Department of Statistical Science
1 Likelihood Ratio Tests that Certain Variance Components Are Zero Ciprian M. Crainiceanu Department of Statistical Science www.people.cornell.edu/pages/cmc59 Work done jointly with David Ruppert, School
More informationAn Introduction to Functional Data Analysis
An Introduction to Functional Data Analysis Chongzhi Di Fred Hutchinson Cancer Research Center cdi@fredhutch.org Biotat 578A: Special Topics in (Genetic) Epidemiology November 10, 2015 Textbook Ramsay
More informationDynamic System Identification using HDMR-Bayesian Technique
Dynamic System Identification using HDMR-Bayesian Technique *Shereena O A 1) and Dr. B N Rao 2) 1), 2) Department of Civil Engineering, IIT Madras, Chennai 600036, Tamil Nadu, India 1) ce14d020@smail.iitm.ac.in
More informationMarkov Chain Monte Carlo (MCMC)
Markov Chain Monte Carlo (MCMC Dependent Sampling Suppose we wish to sample from a density π, and we can evaluate π as a function but have no means to directly generate a sample. Rejection sampling can
More informationBayesian Analysis of RR Lyrae Distances and Kinematics
Bayesian Analysis of RR Lyrae Distances and Kinematics William H. Jefferys, Thomas R. Jefferys and Thomas G. Barnes University of Texas at Austin, USA Thanks to: Jim Berger, Peter Müller, Charles Friedman
More information1. SS-ANOVA Spaces on General Domains. 2. Averaging Operators and ANOVA Decompositions. 3. Reproducing Kernel Spaces for ANOVA Decompositions
Part III 1. SS-ANOVA Spaces on General Domains 2. Averaging Operators and ANOVA Decompositions 3. Reproducing Kernel Spaces for ANOVA Decompositions 4. Building Blocks for SS-ANOVA Spaces, General and
More informationOn Conditional Variance Estimation in Nonparametric Regression
On Conditional Variance Estimation in Nonparametric Regression Siddhartha Chib Edward Greenberg January 2011, November 2011 Abstract In this paper we consider a nonparametric regression model in which
More informationReview. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Review DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall16 Carlos Fernandez-Granda Probability and statistics Probability: Framework for dealing with
More informationBayesian Estimation and Inference for the Generalized Partial Linear Model
Bayesian Estimation Inference for the Generalized Partial Linear Model Haitham M. Yousof 1, Ahmed M. Gad 2 1 Department of Statistics, Mathematics Insurance, Benha University, Egypt. 2 Department of Statistics,
More informationBayesian linear regression
Bayesian linear regression Linear regression is the basis of most statistical modeling. The model is Y i = X T i β + ε i, where Y i is the continuous response X i = (X i1,..., X ip ) T is the corresponding
More informationEstimation of cumulative distribution function with spline functions
INTERNATIONAL JOURNAL OF ECONOMICS AND STATISTICS Volume 5, 017 Estimation of cumulative distribution function with functions Akhlitdin Nizamitdinov, Aladdin Shamilov Abstract The estimation of the cumulative
More information