Order-q dependent stochastic processes in Bayesian nonparametric applications Department of Statistics, ITAM, Mexico BNP 2015, Raleigh, NC, USA 25 June, 2015
Contents Order-1 process Application in survival analysis Application in proteomics (DDP) Order-q process Application in time series modeling Application in multiple time series (Dependent Polya tree) Application in disease mapping Extensions
Order1 process η 1 η 2 η 3 η 4 η 5 θ 1 θ 2 θ 3 θ 4 θ 5 Dependence among {θ k } is induced through a latents {η k } Close form expressions when use conjugate distributions Want to ensure a given marginal distribution
Order1 process Nieto-Barajas & Walker (2001):
Order1 process Nieto-Barajas & Walker (2001): Beta process: {θ k } BeP 1 (a, b, c) θ 1 Be(a, b), η k θ k Bin(c, θ k ), θ k+1 η k Be(a + η k, b + c k η k ) θ k Be(a, b) marginally
Order1 process Nieto-Barajas & Walker (2001): Beta process: {θ k } BeP 1 (a, b, c) θ 1 Be(a, b), η k θ k Bin(c, θ k ), θ k+1 η k Be(a + η k, b + c k η k ) θ k Be(a, b) marginally Gamma process: {θ k } GaP 1 (a, b, c) θ 1 Ga(a, b), η k θ k Po(c k θ k ), θ k+1 η k Ga(a + c k, b + η k ) θ k Ga(a, b) marginally
Survival Analysis Hazard rate modelling If T is a discrete r.v. with support on τ k then h(t) = θ k I (t = τ k ) with {θ k } BeP 1 (a, b, c) If T is a continuous r.v. and {τ k } are a partition of IR + then h(t) = θ k I (τ k1 < t τ k ) with {θ k } GaP 1 (a, b, c)
Survival Analysis Hazard rate modelling If T is a discrete r.v. with support on τ k then with {θ k } BeP 1 (a, b, c) h(t) = θ k I (t = τ k ) If T is a continuous r.v. and {τ k } are a partition of IR + then with {θ k } GaP 1 (a, b, c) h(t) = θ k I (τ k1 < t τ k ) This is old stuff!, but what it is new is that there is an R-package called BGPhazard that implements these models
Survival Analysis: Order-1 Beta process 0 5 10 15 20 0.0 0.2 0.4 0.6 0.8 1.0 Estimate of hazard rates time Hazard rate + + + + + + + + + + + + + Hazard function Confidence band (95%) NelsonAalen based estimate
Survival Analysis: Order-1 Beta process Estimate of Survival Function 0.0 0.2 0.4 0.6 0.8 1.0 Model estimate Confidence bound (95%) KaplanMeier KM Confidence bound (95%) 0 5 10 15 20 times
Survival Analysis: Order-1 Gamma process Estimate of Survival Function 0.0 0.2 0.4 0.6 0.8 1.0 Model estimate Confidence bound (95%) KaplanMeier KM Confidence bound (95%) 0 20 40 60 80 100 120 140 times
Dependent Dirichlet Process (DDP) Nieto-Barajas & al. (2012): Use order1 Beta P. to define a DDP Time series of random prob. measures F = {F 1, F 2,...} Consider stick break. rep. F t = w th δ µth h=1, w th = θ th (1 θ tj ) j<h
Dependent Dirichlet Process (DDP) Nieto-Barajas & al. (2012): Use order1 Beta P. to define a DDP Time series of random prob. measures F = {F 1, F 2,...} Consider stick break. rep. F t = w th δ µth h=1, w th = θ th (1 θ tj ) j<h Common locations across time: µ th = µ h iid G for all t
Dependent Dirichlet Process (DDP) Nieto-Barajas & al. (2012): Use order1 Beta P. to define a DDP Time series of random prob. measures F = {F 1, F 2,...} Consider stick break. rep. F t = w th δ µth h=1, w th = θ th (1 θ tj ) j<h Common locations across time: µ th = µ h iid G for all t Dependent (unnormalized) weights: For each h {θ th } BeP 1 (1, b, c h )
Dependent Dirichlet Process (DDP) Nieto-Barajas & al. (2012): Use order1 Beta P. to define a DDP Time series of random prob. measures F = {F 1, F 2,...} Consider stick break. rep. F t = w th δ µth h=1, w th = θ th (1 θ tj ) j<h Common locations across time: µ th = µ h iid G for all t Dependent (unnormalized) weights: For each h {θ th } BeP 1 (1, b, c h ) Say that F DDP(b, G, c) and marginally F t DP(b, G)
Proteomics Study: Pathway inhibition experiment to study the effects of drug Lapatinib on ovarian cancer cell lines An ovarian cell line was initially treated with Lapatinib, and then stimulated over time T = 8 measurement time points, t d = 0, 5, 15, 30, 60, 90, 120, and 240 minutes, n = 30 proteins were probed using RPPA Difference scores (posttreatment - pretreatment intensities) were recorded
Histograms of expression scores t=0 t=5 t=15 t=30 Histogram 0 2 4 6 8 10 12 Histogram 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Histogram 0.0 0.5 1.0 1.5 2.0 2.5 Histogram 0 1 2 3 4 3.0 1.5 0.0 3.0 1.5 0.0 3.0 1.5 0.0 3.0 1.5 0.0 Y Y Y Y t=60 t=90 t=120 t=240 Histogram 0.0 0.5 1.0 1.5 2.0 2.5 Histogram 0.0 0.5 1.0 1.5 2.0 Histogram 0.0 0.5 1.0 1.5 2.0 Histogram 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.0 1.5 0.0 3.0 1.5 0.0 3.0 1.5 0.0 3.0 1.5 0.0 Y Y Y Y
Time series of expression scores Difference scores 2.5 2.0 1.5 1.0 0.5 0.0 0 50 100 150 200 Time
Proteomics The full model for the data is a random effects model y ti = x ti + u i + ɛ ti Temporal effect: x ti iid Ft and (F 1,..., F T ) DDP(b, G, c) with c th = c/ t to account for the unequally spacing of the observations Pathway effect: u i s.t. (u 1,..., u n ) CAR based on consensus interactions Measurement error: ɛ ti iid N(0, τt )
Plots of ˆF t = E(F t data) c.d.f. 0.00 0.02 0.04 0.06 0.08 0.10 0.12 3.0 2.5 2.0 1.5 1.0 0.5 0.0 0 50 100 150 200 250 Time X Increasing suppression over t 1 = 0 through t 5 = 60. From t 6 = 90 the effect is wearing off
Order2 process η 1 η 2 η 3 η 4 η 5 θ 1 θ 2 θ 3 θ 4 θ 5 Throw more arrows to induce higher order dependence There is no way to obtain a given marginal distribution: say beta or gamma Unless we include an extra latent (layer)
Order2 process ω η 1 η 2 η 3 η 4 η 5 θ 1 θ 2 θ 3 θ 4 θ 5 With this common ancestor ω we can through more arrows and still ensure a given marginal
Space and time process This idea can be use to induce time and/or spatial dependence t = 1 t = 2 t = 3 θ 1,1 (η 1,1 ) θ 1,3 (η 1,3 ) θ 1,2 (η 1,2 ) θ 1,4 (η 1,4 ) θ 2,1 (η 2,1 ) θ 2,3 (η 2,3 ) θ 2,2 (η 2,2 ) θ 2,4 (η 2,4 ) θ 3,1 θ 3,2 (η 3,1 ) (η 6 3,2 ) θ 3,3 (η 3,3 ) θ 3,4 (η 3,4 ) ω
Order-q process Jara & al. (2013): Orderq (AR) beta process: {θ t } BeP q (a, b, c) ω Be(a, b) η t ω ind Bin(c t, ω) q q θ t η Be a + η tj, b + (c tj η tj ) θ t Be(a, b) marginally j=0 j=0
Time series: θ t = Unemployement in Chile 0.0 0.1 0.2 0.3 0.4 BeP BDM 1980 1990 2000 2010 2020 Year
Dependent Polya Tree Nieto-Barajas & Quintana (2015): Use orderq Beta P. to define a DPT Time series of random prob. measures F = {F 1, F 2,...} Consider Polya Trees F t with nested partition Π t = {B tmj } and branching probs. Θ t = {θ t,m,j } with m = level, j = 1,..., 2 m
Dependent Polya Tree Nieto-Barajas & Quintana (2015): Use orderq Beta P. to define a DPT Time series of random prob. measures F = {F 1, F 2,...} Consider Polya Trees F t with nested partition Π t = {B tmj } and branching probs. Θ t = {θ t,m,j } with m = level, j = 1,..., 2 m Common nested partitions across time: Π t = Π for all t
Dependent Polya Tree Nieto-Barajas & Quintana (2015): Use orderq Beta P. to define a DPT Time series of random prob. measures F = {F 1, F 2,...} Consider Polya Trees F t with nested partition Π t = {B tmj } and branching probs. Θ t = {θ t,m,j } with m = level, j = 1,..., 2 m Common nested partitions across time: Π t = Π for all t Dependent branching probs: For each m and j {θ t,m,j } BeP q (aρ(m), aρ(m), c)
Dependent Polya Tree Nieto-Barajas & Quintana (2015): Use orderq Beta P. to define a DPT Time series of random prob. measures F = {F 1, F 2,...} Consider Polya Trees F t with nested partition Π t = {B tmj } and branching probs. Θ t = {θ t,m,j } with m = level, j = 1,..., 2 m Common nested partitions across time: Π t = Π for all t Dependent branching probs: For each m and j {θ t,m,j } BeP q (aρ(m), aρ(m), c) Say that F DPT q (Π, a, ρ, c) and marginally F t PT ρ(m) = m δ with δ {1.1, 2} (see Watson & al. 2015)
Multiple time series analysis Study: bioeconomic activity indicators (ITAEE) for the 32 States of Mexico Values are reported every 3 months from 2003 Available are 32 series of length 46 Values are transformed to constant prices of 2008 and are destationalized Took second differences to make data stationary
Original and second differences time series 70 80 90 100 110 120 20 10 0 10 2004 2006 2008 2010 2012 2014 Time 0 10 20 30 40 Time
Multiple time series analysis The model proposed for the data: X i = {X ti, t 1}, i = 1,..., n is an AR(p) process for each series X ti = β 1i X t1,i + + β pi X tp,i + ε ti, iid ε ti F t Ft, for i = 1,..., n {F 1, F 2,...} σ DPT q (Π σ, a, ρ, c) σ f (σ) Note that there is a further mixture: B mj are quantiles of N(0, σ 2 ) Dependence in {F t } dependence in {ε ti } resembling a MA(q) process, however the dependece is not necessarily exponentially decaying
Estimated {F t } Density 0.00 0.05 0.10 0.15 0.20 0.25
Spatial process
Spatial process Nieto-Barajas & Bandyopadhyay (2013): Spatial gamma process: {θ t } SGaP(a, b, c) ω Ga(a, b) η ij ω ind Ga(c ij, ω) θ i η Ga a + c ij, b + η ij j i j i i is the set of neighbours of region i θ t Ga(a, b) marginally
Disease mapping Study: Mortality in pregnant women due to hypertensive disorder in Mexico in 2009. Areas are the States Y i = Number of deaths in region i E i = At risk: Number of births (in thousands) λ i = Maternity mortality rate Zero-inflated model f (y i ) = π i I (y i = 0)+(1π i )Po(y i λ i E i ), λ i = θ i exp(β x i ) β is a vector of reg. coeff. s.t. β k N(0, σ 2 0 ) θ i SGaP(a, a, c) Six explanatory variables
Estimated mortality rate λ i 2 26 8 5 [3.05,6.33) [6.33,6.67) [6.67,7.38) [7.38,8.73) [8.73,21.07] 3 25 10 19 28 32 24 1 18 11 22 14 13 6 16 15 9 29 30 17 21 12 20 27 7 4 31 23
Estimated zero inflated prob. π i 2 26 8 5 [0,0.01) [0.01,0.04) [0.04,0.06) [0.06,0.5) [0.5,0.6] 3 25 10 19 28 32 24 1 18 11 22 14 13 6 16 15 9 29 30 17 21 12 20 27 7 4 31 23
Extensions Use same ideas with stochastic processes instead of random variables Dependent Dirichlet processes using multinomial processes as latents Dependent gamma processes using Poisson processes as latents These constructions are currently under study
References Jara, A., Nieto-Barajas, L. E. & Quintana, F. (2013). A time series model for responses on the unit interval. Bayesian Analysis 8, 723 740. Nieto-Barajas, L. E. & Bandyopadhyay, D. (2013). A zero-inflated spatial gamma process model with applications to disease mapping. Journal of Agricultural, Biological and Environmental Statistics 18, 137 158. Nieto-Barajas, L. E., Müller, P., Ji, Y., Lu, Y. & Mills, G. (2012). A time series DDP for functional proteomics profiles. Biometrics 68, 859 868. Nieto-Barajas, L. E. & Quintana, F. A. (2015). A Bayesian nonparametric dynamic AR model for multiple time series analysis. Preprint. Nieto-Barajas, L. E. & Walker, S. G. (2002). Markov beta and gamma processes for modelling hazard rates. Scandinavian Journal of Statistics 29, 413 424. Watson, J. Nieto-Barajas, L. E. & Holmes, C. (2015). Characterising variation of nonparametric random probability measures using the Kullback-Leibler divergence. Preprint.