1 / 14 R-INLA Sam Clifford Su-Yun Kang Jeff Hsieh Bayesian Research and Analysis Group 30 August 2012
What is R-INLA? R package for Bayesian computation Integrated Nested Laplace Approximation MCMC free Wrapper for GMRFlib http://www.r-inla.org Håvard Rue and Sara Martino. INLA Functions which allow to perform a full Bayesian analysis of structured additive models using Integrated Nested Laplace Approximaxion, 0.0 edition, December 2010. URL http://www.r-inla.org/ Håvard Rue and Turid Follestad. GMRFLib: Fast and exact simulation of Gaussian Markov random fields on graphs. Technical report, Norwegian University of Science and Technology, 2007. URL http://www.math.ntnu.no/hrue/gmrflib/ 2 / 14
Overview Replace all variables with their Laplace approximation Construct GMRF of posterior from prior likelihood GMRF prior conjugate to GMRF likelihood Solve with Newton method Håvard Rue, Sara Martino, and Nicolas Chopin. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society B, 71:319 392, 2009 3 / 14
4 / 14 Laplace Approximation consider probability density p(x; θ) dx Replace with MVN (x; x, Q) where x maximises the log likelihood Q the Hessian of log likelihood at x Everything is now Gaussian
Gaussian Markov Random Field Multivariate normal density { } π (x) = Q (2π) n exp (x µ)t Q (x µ) 2 Conditional independence Q ij = 0 Many models and priors can be represented as GMRF linear model penalised random walk Besag and Besag, York and Mollie stochastic PDE over finite element basis AR(1) Håvard Rue and Leonhard Held. Gaussian Markov random fields: theory and applications. CRC Press, 2005 Stefan Lang and Andreas Brezger. Bayesian P-Splines. Journal of Computational and Graphical Statistics, 13(1):183 212, 2004 Julian Besag. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B, 36: 192 236, 1974 Julian Besag, Jeremy York, and Annie Mollié. Bayesian image restoration, with two applications in spatial statistics. Annals of the Institute of Statistical Mathematics, 43:1 20, 1991 Finn Lindgren, Håvard Rue, and Johan Lindström. An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society B, 73, 2011 5 / 14
6 / 14 Newton solver Posterior GMRF as surface to be maximised Newton-Raphson solver seeks global maximum GMRF mean is location of maximum value GMRF precision is Hessian at maximum value
7 / 14 Posterior inference GMRF mean is location of maximum value GMRF precision is Hessian at maximum value Integrate to obtain marginals of parameters Easy because everything is Gaussian
8 / 14 Posterior summaries 95% region of HPD Kullback-Leibler divergence DIC, log likelihood, etc.
Prior precisions GMRF is a conjugate prior for GMRF wide variety of likelihoods and priors example random walk of order 2 p (β λ) MVN (0, Q) p (λ) Γ (1, b). λ transformed to log λ log-gamma prior replaced with its Laplace approximation Row 5 10 15 20 5 10 15 20 Column Dimensions: 24 x 24 Figure: C for random walk penalty of order 2 with 24 unique covariate values 6 4 2 0 2 4 Stefan Lang and Andreas Brezger. Bayesian P-Splines. Journal of Computational and Graphical Statistics, 13(1):183 212, 2004 9 / 14
10 / 14 Defining custom precision matrices generic0 most general, Q = τc ( ) generic1 Q = τ I β λmax C generic2 for two level hierarchical model expression build a custom prior
11 / 14 Example Daily trend one week of simulated data ( ( ) ) 2πti y i N sin, 0.33 2 24 generic0 Row 5 10 6 4 2 cyclic random walk penalty of order 2 sum to zero constraint small amount added to diagonal 15 20 5 10 15 20 Column Dimensions: 24 x 24 0 2 4 (y β, λ, τ) MVN (y; Xβ, τi) MVN (β λ; 0, λc) Γ (λ; 1, b) Γ (τ; 1, b)
y 12 / 14 Example Results PostDens [Precision for the Gaussian observations] 0.00 0.15 0.30 6 8 10 12 14 16 18 f(t) 1.5 0.5 0.5 1.5 5 10 15 20 t PostDens [Precision for t] 0.000 0.004 0.008 0 200 400 600 800 1000 1200 1.5 0.5 0.5 1.5 0 50 100 150 Index Figure: σ = 0.318 (0.285, 0.356) Figure: Fitted smooth
Other cool stuff can be run on remote supercomputer GMRFlib parallelisable obtain successive starting values to aid convergence animalinla survival models SPDE spatial (spatio-temporal) models MCMC sampler included for comparison Håvard Rue and Turid Follestad. GMRFLib: Fast and exact simulation of Gaussian Markov random fields on graphs. Technical report, Norwegian University of Science and Technology, 2007. URL http://www.math.ntnu.no/hrue/gmrflib/ Sara Martino Anna Marie Holand, Ingelin Steinsland and Henrik Jensen. Animal models and integrated nested Laplace approximations. Technical Report 4/2011, Norges Teknisk-Naturvitenskapelige Universitet, 2011 Sara Martino and Håvard Rue. Complex data modeling and computationally intensive statistical methods, chapter Case Studies in Bayesian Computation using INLA. 2010 Finn Lindgren, Håvard Rue, and Johan Lindström. An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society B, 73, 2011 Håvard Rue and Leonhard Held. Gaussian Markov random fields: theory and applications. CRC Press, 2005 13 / 14
14 / 14 Code library(inla) t <- rep(1:24,7) y <- rnorm(n=168,mean=sin(t * 2 * pi / 24),sd=0.33) make.crw2 <- function(n){ Q <- toeplitz( c(6,-4,1, rep(0,n-5), 1, -4)) return(as(q, "dgcmatrix")) } Q <- make.crw2(24) # image(q,col.regions=grey.colors(16)) result1 <- inla(y f(t, model="generic0",cmatrix=q,diagonal=1e-03,constr=t), data=as.data.frame(list(y = y, t=t)),verbose=t, control.predictor=list(compute=t)) windows() par(mfrow=c(2,1)) plot(y t, pch=1,ylab="f(t)",xlab="t") lines(result1$summary.random$t[,2]) lines(result1$summary.random$t[,4],lty=2) lines(result1$summary.random$t[,6],lty=2) plot(y, pch=1,ylab="y",xlab="index") lines(result1$summary.linear.predictor[,1]) lines(result1$summary.linear.predictor[,3],lty=2) lines(result1$summary.linear.predictor[,5],lty=2)