Disease mapping with Gaussian processes

Transcription

1 EUROHEIS2 Kuopio, Finland August 2010 Aki Vehtari (former Helsinki University of Technology) Department of Biomedical Engineering and Computational Science (BECS)

2 Acknowledgments Researchers - Jarno Vanhatalo - Jouni Hartikainen - Ville Pietiläinen - Jaakko Riihimäki - Simo Särkkä Collaborators and data - The National Institute for Health and Welfare - Finnish Cancer Registry - Statistics Finland

3 Example: alcohol related diseases in Finland Collaboration: The National Institute for Health and Welfare Data: Statistics Finland About inhabited 5km 5km cells in Finland - many cells with no inhabited neighbors In about died due to alcohol diseases - expected death count less than one per cell

4 Outline Related methods and previous work What is a Gaussian process (GP) Flexibility of GP covariance functions Inference Non-Gaussian observation models Computational challenges and some solutions Examples - spatial variation of alcohol-related diseases - spatio-temporal prediction of cancer incidences

5 Spatio-temporal smoothing Assuming nearby-regions have similar latent effect Popular choices - latent Gaussian Markov random field models (e.g. CAR, BYM) - latent Gaussian process models

6 Previous work in GP Kriging since 1950 s Bayesian approach for GPs since 1970 s (Empirical) Bayesian GPs in spatial epidemiology since 1980 s Popularity growing due to algorithm development and increase in the computer speed

7 Gaussian process Infinite-dimensional generalization of multivariate Gaussian Can be used to set a prior on function space Univariate Gaussian Bivariate Gaussian Samples from GP f N(µ, σ 2 ) f N(µ, Σ) f GP(µ, Σ)

8 Spatial smoothing Observations Y X y N(f, σ 2 )

9 Spatial smoothing Observations Y X1 X2 X3 X 1) Uncertainty related to f

10 Spatial smoothing Observations Y X1 X2 X3 X 1) Uncertainty related to f 2) Uncertainty related to y

11 Spatial smoothing Observations Y X1 X2 X3 X Let s focus on f f values given similar x values seem to correlate

12 Spatial smoothing Observations Covariance k(x 2,X j ) Y X1 X2 X3 X Describe correlation with covariance function k(x i, x j )

13 Spatial smoothing Observations Covariance k(x 2,X j ) Y X1 X2 X3 X Describe correlation with covariance function k(x i, x j ) e.g. k(x i, x j ) = φ exp ( (x i x j ) 2 l 2 )

14 Spatial smoothing Observations Covariance k(x 2,X j ) Y F(X 1 ) X1 X2 X3 X F(X 2 ) Function values given X 1 and X 2 are far in X space and do not correlate

15 Spatial smoothing Observations Covariance k(x 2,X j ) Y F(X 3 ) X1 X2 X3 X F(X 2 ) Function values given similar X 2 and X 3 correlate

16 Gaussian process Joint distribution for the observations y and a new f [ ] ( [ y K (X, X) + σ N 0, 2 ]) I K (X, X ) K (X, X) K (X, X ) f

17 Gaussian process Joint distribution for the observations y and a new f [ ] ( [ y K (X, X) + σ N 0, 2 ]) I K (X, X ) K (X, X) K (X, X ) f Since a finite number of random variables, computations as for a multivariate Gaussian E[f ] = K (X, X)[K (X, X) + σ 2 I] 1 y cov[f ] = K (X, X ) K (X, X)[K (X, X) + σ 2 I] 1 K (X, X )

18 Gaussian process regression Observations f=gp(x,x obs,y obs,θ) Uncertainty in f Uncertainty in y Y X E[f ] = K (X, X)[K (X, X) + σ 2 I] 1 y cov[f ] = K (X, X ) K (X, X)[K (X, X) + σ 2 I] 1 K (X, X ) Note: linear equations but nonlinear regression due to weighting by covariance function

19 Flexibility of GP covariance functions Squared exponential Matern(5/2) Exponential Two lengthscales

20 Spatial smoothing Correlation defined, e.g., using Euclidian distance Spatial correlation decreases as distance increases

21 Flexibility of GP covariance functions Examples of spatial surfaces sampled from different GP priors

22 Flexibility of GP covariance functions Non-linearities and interactions automatically for large number of covariates 0.75 readmission probability days of preceding care age

23 Computational challenges Matrix inversion O(n 3 ) Non-Gaussian observation models Estimation of the covariance function parameters

24 Inference given a covariance matrix Given a Gaussian observation model, a covariance function K and its parameters θ: analytic solution with matrix algebra E[f θ] = K (X, X)[K (X, X) + σ 2 I] 1 y cov[f θ] = K (X, X ) K (X, X)[K (X, X) + σ 2 I] 1 K (X, X ) Requires the inversion of a n n matrix, which is generally an O(n 3 ) operation

25 Covariance matrix approximations Matrix inversion O(n 3 ) With reduced rank or projection approximations O(m 2 n) - m n - useful for global spatial phenomena Compact support covariance functions - naturally sparse covariance matrix if the correlation length is short - useful for local spatial phenomena Combined approach for good global and local performance

26 Non-Gaussian observation models With a latent variable approach E.g. count observations y Poisson(α exp(f )) f Uncertainty in f 20 Observations µ=α exp(f) Uncertainty in µ F 0 Y 10 X 0 X

27 Non-Gaussian observation models With a latent variable approach E.g. count observations y Poisson(α exp(f )) f Uncertainty in f 20 Observations µ=α exp(f) Uncertainty in µ Uncertainty in y F 0 Y 10 X 0 X

28 Inference given a non-gaussian observation model Integration over the latent variables f i required No analytic solution - Monte Carlo methods have mixing and convergence problems when n is large New coming of analytic approximations - Laplace approximation ie. a Gaussian at the mode - expectation propagation (EP) ie. a Gaussian which minimizes KL-divergence from the true distribution - variational Bayes ie. a Gaussian which minimizes KL-divergence from the approximation

29 Integration over the latent space In our spatial and spatio-temporal experiments - Expectation propagation about as good as MCMC, but much faster - Laplace approximation almost as good as EP, but faster In others experiments - variational not as good as EP

30 Inference given unknown covariance function parameters θ Estimation of θ (empirical Bayes) - optimize θ given the marginal likelihood p(d θ) (from the properties of a multivariate Gaussian) - nonlinear optimization wrt. θ

31 Inference given unknown covariance function parameters θ Estimation of θ (empirical Bayes) - optimize θ given the marginal likelihood p(d θ) (from the properties of a multivariate Gaussian) - nonlinear optimization wrt. θ Full Bayes - integrate over the posterior of θ p(θ D) p(d θ)p(θ) - no analytic solution Monte Carlo, grid, or CCD approximation reasonable performance when latent values have been marginalized with analytic approximation

32 Integration over the covariance function parameters θ In our experiments - with large n optimization of θ as good as integration integration over the latent space more important If integration needed - if computation of the marginal density is slow, the integration need to be made with as few density evaluations as possible - eg. grid based when few hyperparameters

33 Computational solutions Matrix inversion O(n 3 ) sparse approximations O(m 2 n) - reduced rank approximations compact support covariance functions - naturally sparse covariance matrix Integration over n dimensional latent space MCMC, Laplace, Expectation propagation, Variational Integration over hyperparameters Monte Carlo, grid evaluation, etc.

34 Improvement in computational speed Basic GP with MCMC vs. speeded-up GP 2006: resolution 20km (900 cells), analysis 2-3 days 2009: resolution 5km ( cells), analysis 20 min

35 Example: alcohol related diseases in Finland Collaboration: The National Institute for Health and Welfare Data: Statistics Finland About inhabited 5km 5km cells in Finland - many cells with no inhabited neighbors In about died due to alcohol diseases - expected death count less than one per cell

36 Example: alcohol related diseases in Finland Sex-age-education standardized expected death counts used to compute the raw risk Risk smoothed using GP with long and short length scale and negative-binomial observation model

37 Example: alcohol related diseases in Finland Sex-age-education standardized expected death counts used to compute the raw risk Risk smoothed using GP with long and short length scale and negative-binomial observation model Is the relative risk higher in the population centers?

38 Example: alcohol related diseases in Finland The smoothed relative risks vs. the population density

39 Example: alcohol related diseases in Finland The smoothed relative risks vs. the population density Add population density as explanatory variable

40 Example: alcohol related diseases in Finland Population density and spatial variation explain the variation in the risk Population density effect Spatial effect

41 Example: alcohol related diseases in Finland Spatial and population density explain the variation in the risk Combined effect

42 Example: alcohol related diseases in Finland With Google Maps With Google Earth

43 Example: alcohol related diseases in Finland Zoomed near Kuopio

44 Example: alcohol related diseases in Finland Jarno Vanhatalo, Ville Pietiläinen and Aki Vehtari (2010). Approximate inference for disease mapping with sparse Gaussian processes. Statistics in Medicine, 29(15): Jarno Vanhatalo, Pia Mäkelä, ja Aki Vehtari (2010). Alkoholikuolleisuuden alueelliset erot Suomessa 2000-luvun alussa. Yhteiskuntapolitiikka, 75(3): Online supplement research/mm/finnwell/alcoholfinland/ GPstuff toolbox In Finnish Newspapers: Helsingin Sanomat, Pohjalainen, Lapin Kansa

45 Compared to CAR In spatial epidemiology CAR (or BYM) is most used model Correlation defined conditionally based on a neighborhood structure discrete definition + major computational speed-up if a precision matrix is sparse due to small neighborhoods - describes only local correlation - neighborhood definition may be difficult for irregularly spaced data and high dimensional data

46 Example: alcohol related diseases in Finland Comparison to CAR Compared to CAR computed with INLA software (uses fast Integrated Nested Laplace Approximation) - CAR model lacks long range correlation part - CAR model has much higher variance, especially for cells having no or few inhabited neighbors - GP has a better predictive performance Variance CAR CAR Mean GP GP

47 Example: lung cancer women County incidences and background population for years years, 431 counties observations Data: Finnish Cancer Registry Model: GP with temporal + spatial + spatiotemporal component

48 Example: lung cancer women (a) Temporal (b) Spatial

49 Example: lung cancer women (a) 1953 (d) 1983 (b) 1963 (c) (e) 1993 (f) 2003

50 Work in progress Spatio-temporal GPs can be written as linear stochastic partial differential equations Reduces computational complexity from O(n 3 T 3 ) to O(n 3 T ), i.e. method scales linearily in T n limited as for spatial GP - few thousand with no sparse aproximations - more than ten thousand with sparse approximations Current tests with over million spatio-temporal points Makes it easier to specify non-stationary temporal dynamics, which are necessary, for example, when performing future predictions

51 Example: skin melanoma Prediction with a non-stationary spatio-temporal GP Ten year predictions (mean + 95 % CI) and observed counts in one city y Time

52 Example: skin melanoma Prediction with a non-stationary spatio-temporal GP Ten year predictions (mean + 95 % CI) and observed counts in one city y Time Age-period-cohort model in progress

53 Other applications GPs are not just for spatial statistics Length of stay in rehabilitation after hip fracture surgery Probability of readmission Clustering users of health care services Genome wide analysis Marked point processes Industrial processes