A stochastic formulation of a dynamical singly constrained spatial interaction model

Transcription

1 A stochastic formulation of a dynamical singly constrained spatial interaction model Mark Girolami Department of Mathematics, Imperial College London The Alan Turing Institute, British Library Lloyds Register Foundation Programme on Data-Centric Engineering Brunel University London Modern Statistical Methods in Health and Environment Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

2 Louis Ellam Mark Girolami Greg Pavliotis Alan Wilson ICL & ATI ICL & ATI ICL UCL & ATI Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

3 Motivation Question: What will cities and regions look like in the future? Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

4 Urban Analytics How will cities and regions develop as time progresses? Observational data from Twitter, Android location tracking, camera monitoring, travel ticketing, demographics, etc. Simple statistical summaries and models yielding insights the hallmark of Urban Analytics. Mathematical modelling long history in urban and regional analysis and planning. Build upon these mathematical formalisms with this new found data (throwing baby out with bath water). Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

5 Complex Urban Systems We re uncertain! Urban and regional systems are complex. Actions of individuals give rise to an emergent behaviour: Phase transitions; Path dependence; and Multiple equilibria. Mechanistic models of complex systems = model error. Uncertainty should be addressed. This talk: Improving insights into urban and regional development by addressing uncertainties arising in the modelling process. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

6 The London Retail System Figure: London retail structure for 2008 (left) and 2012 (right). The locations of retail zones and residential zones are red and blue, respectively. Sizes are in proportion to floorspace and spending power, respectively. N = 625 and M = 201. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

7 The London Retail System Stratford and Shepherds Bush have risen up the rankings as a result of the opening of Westfield Stratford City and Westfield London. Knightsbridge appears to have experienced a significant reduction in town centre floorspace. Ilford and Harrow also experienced declines in town centre floorspace. The West End remains the largest centre in London. Croydon, Kingston, Romford, Canary Wharf, Camden Town, Kings Road East and Angel showed strong growth in total town centre floorspace over the period. Source: 2013 London Town Centre Health Check Analysis Report. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

8 Airports in England Figure: Airports by number of passengers for 2007 (left) and 2016 (right). The locations of airports and residential zones are red and blue, respectively. N = 33 and M = 25. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

9 Urban and Regional Modelling Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

10 Spatial Interaction Model Assume N origin zones and M destination zones. Origin quantities {O i } N i=1. Destination quantities {D j } M j=1. {T ij } N,M i,j=1 denote the flows from zone i to j, respectively. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

11 Assumptions Flows from origin zones: O i = M T ij, i = 1,..., N. j=1 Flows to destination zones: D j = N T ij, j = 1,..., M. i=1 Known as a singly-constrained or production-constrained system since {O i } N i=1 is fixed and {D j} M j=1 is undetermined. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

12 Assumptions Fixed transport benefit: N M T ij ln W j = X, i=1 j=1 where W j is the size, and X j := ln W j is the attractiveness, of zone j. Fixed transport cost: N M T ij c ij = C, i=1 j=1 where c ij is the cost of transporting a unit from zone i to zone j. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

13 A Singly-Constrained Spatial Interaction Model Maximization of an entropy function. The destination flows are given by D j = N i=1 O i exp ( α ln W j βc ij ) M k=1 exp ( α ln W k βc ik ). X j := ln W j is the attractiveness of j. c ij is the inconvenience of transporting from zone i to j. α is the attractiveness scaling parameter. β is the cost scaling parameter. αx j βc ij is the net utility from transporting from zone i to j. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

14 A Dynamical Urban and Regional Model Harris and Wilson Model The sizes of the destination zones satisfy the (modified) competitive Lotka-Volterra equations dw j dt for some initial condition W (0) = w. Definitions: ) = ɛ (D j κw j W j, j = 1,..., M, κ > 0 is the cost rate per unit size; and ɛ > 0 is the responsiveness parameter. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

15 Stochastic Urban and Regional Modelling Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

16 Potential Energy Function In what proceeds, it is more natural to work in terms of attractiveness X j := ln W j. The energy of a configuration X = (X 1,..., X M ) T is given by an interaction potential U(X ). The force acting on each site is given by a conservative vector field U(X ). Interaction Potential γ 1 U(X ) = κ M exp(x j ) α 1 j=1 }{{} Running cost N i=1 M M O i ln exp(αx j βc ij ) δ j=1 } {{ } Net utility j=1 X j }{{} Additional. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

17 More General Modelling Assumptions We assume that: 1 There exists a potential function U C 2 (R M, R) such that γ 1 U(X ) = R j (X ) κ exp(x j ) + δ j (X ). 2 The drift coefficients U(X ) are locally Lipshitz and satisfy the one-sided growth condition in that there exist K 1, K 2 > 0 such that U(X ) X K 1 + K 2 X 2, X R M. 3 U satisfies the super-linear growth condition in that there exists K 1, K 2 > 0 such that U(X ) K 1 + K 2 X, X R M. 4 The initial condition X (0) = x has finite variance E x 2 <. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

18 Key Results under the Modelling Assumptions Urban dynamics, in terms of attractiveness, satisfies the overdamped Langevin dynamics dx dt = U(X ) + 2γ 1 db, X (0) = x. dt The SDE has a unique solution X C([0, T ]; R M ) that does not explode on [0, T ], almost surely. Since δ j = δ > 0 and U C 2 with a growth condition, X (t) is ergodic with respect to a well-defined Boltzmann distribution, whose density is given by π (X ) = 1 Z exp ( γu(x ) ), Z := exp ( γu(x ) ) dx <. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

19 A Stochastic Formulation of the Spatial Interaction Model With our specification of U(X ), overdamped Langevin dynamics give the Harris and Wilson model, plus multiplicative noise. Stochastic Urban Retail Model Floorspace dynamics is a stochastic process that satisfies the following Stratonovich SDE. dw j dt ) = ɛw j (D j κw j + δ + σw j db j dt, where ( B 1,..., B M ) T is standard M-dimensional Brownian motion and σ = 2/ɛγ > 0 is the volatility parameter. Fluctuations (missing dynamics) are modelled as noise that we interpret in the Stratonovich sense. The extra parameter (or function) δ to represents local economic stimulus to prevent zones from collapsing (needed for ergodicity). Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

20 Metropolised Time-Stepping methods Metropolis-Adjusted Langevin Truncated Algorithm (MALTA) Metropolis Hastings algorithm with an Euler-Maruyama proposal, with truncated drift: ˆX := ˆX n τ U( ˆX n ) 1 τ U( ˆX n ) + 2τγ 1 ξ n, ξ n N (0, I ). Truncated drift: U(X ) τ gives an Euler-Maruyama proposal; and U(X ) > τ gives an Euler-Maruyama proposal, but with unit drift. Resulting Markov chain is geometrically ergodic with respect to π. Converges strongly to the Markov process {X (t) : t [0, T ]} on finite time intervals. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

21 Illustrative Stochastic Urban Retail Model M = 2 and N = 20: Figure: Sample path of prototype example with parameter values α = 1.8, β = 1, γ = 1, δ = 0.5 and κ = 1. Under this parameter regime the two sites do not coexist and there are frequent phase transitions. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

22 The Boltzmann Distribution Insights Probability distribution that encodes our knowledge of spatial interactions. Configurations are confined to a metastable state (path dependence). As t, system moves towards lowest energy configurations. Sampling Challenges π is high-dimensional, anisotropic, multi-modal and involves rare events. α = 1.8, β = 1, δ = 0.5 α = 1.8, β = 0, δ = 0.1 α = 1.1, β = 10, δ = 0.2 Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

23 Sampling from the Boltzmann Distribution A Simplified Initial Distribution Define the density π 0 (X ) exp( γ 0 U 0 (X )) with U 0 (X ) = { M j=1 κ exp(x j ) }{{} Running cost } M O i )X j, ( δ + 1 M j=1 }{{} Equal net utility for some γ 0 < γ. We can sample from π 0 exactly and easily. Bridging Distributions Define a temperature schedule 0 = t 0 < t 1 < < t T = 1 with bridging distributions π j = 1 π 1 t j 0 π t j, where Z j := π j dx, for j = 1,..., T. Z j Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

24 Bridging distributions Figure: {π j } T =9 j=1 from left to right, then top to bottom. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

25 The London Retail System: Invariant Distribution Figure: A sample from the invariant distribution showing behaviour in the limit t (most stable configurations). The largest zone is Kensington with 12% of total floorspace and the West End continues to be significant with 5% of total floorspace. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

26 A Statistical Model of Urban Retail Structure Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

27 Hierarchical Modelling Given observation data Y = (Y 1,..., Y M ), of log-sizes, infer the parameter values θ = (α, β) T R 2 + and corresponding latent variables X R M. Assumption (Data generating process) Assume that each observation Y 1,..., Y M is an independent and identical realization of the following hierarchical model: Y 1,..., Y M X, σ N (X, σ 2 I ), X θ π(x θ) exp( U(X ; θ)), θ π(θ). Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

28 Joint Posterior Distribution The joint posterior is given by π(x, θ Y ) = π(y X, θ)π(x, θ), π(y ) = π(y ) π(y X, θ)π(x, θ)dxdθ. We have a hierarchical prior given by π(x, θ) = π(θ) exp( U(X ; θ)), Z(θ) = Z(θ) exp( U(X ; θ))dx. The joint posterior is doubly-intractable π(x, θ Y ) = π(y X, θ) exp( U(X ; θ))π(θ). π(y )Z(θ) Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

29 Metropolis-within-Gibbs with Block Updates We are interested in low-order summary statistics of the form E X,θ Y [h(x, θ)] = h(x, θ)π(x, θ Y )dxdθ. X and θ are highly coupled, so we use Metropolis-within-Gibbs with block updates. X -update. Propose X Q X and accept with probability { min 1, π(y X, θ) exp( U(X ; θ)) q(x X } ) π(y X, θ) exp( U(X ; θ)) q(x. X ) θ-update. Propose θ Q θ and accept with probability { min 1, π(y X, θ ) Z(θ) exp( U(X ; θ )) q(θ θ } ) π(y X, θ) Z(θ ) exp( U(X ; θ)) q(θ. θ) Unfortunately the ratio Z(θ )/Z(θ) ratio is intractable! Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

30 Unbiased Estimates of the Partition Function We can use the Psuedo-Marginal MCMC framework if we have an unbiased, positive estimate of π(x θ), denoted ˆπ(X θ, u), satisfying π(x θ) = ˆπ(X θ, u)π(u θ)du. The Forward Coupling estimator (FCE) gives an unbiased estimate of 1/Z: E[S] = 1/Z. The idea is to find two sequences of consistent estimators {Y (i) }, {Ỹ (i) }, each with the same distribution, such that Y (i) and Ỹ (i 1) are coupled. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

31 Unbiased Estimates of the Partition Function Requires N estimates of 1/Z using annealed importance sampling, for a random stopping time N. Coupling between Y (i) and Ỹ (i 1) is introduced with a Markov chain that shares random numbers. Variance reduction technique. Then the unbiased estimate is given by S := Y (0) + N i=1 Y (i) Ỹ (i 1). Pr(N i) Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

32 The Sign Problem S can be negative when Y (i) < Ỹ (i 1) for many i. This is known as the sign problem. Rejecting when S is negative would introduce a bias. Instead, we note that E[h(X, θ)] = 1 π(y ) h(x, θ)π(y X, θ)ˆπ(x θ, u)π(θ)π(u)dudθdx, = h(x, θ)σ(x θ, u)ˇπ(x, θ, u Y )dudθdx σ(x θ, u)ˇπ(x, θ, u Y )dudθdx, where σ is the sign function and we have defined ˇπ(X, θ, u Y ) = π(y X, θ) ˆπ(X θ, u) π(θ)π(u) π(y X, θ) ˆπ(X θ, u) π(θ)π(u)dudθdx. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

33 Pseudo-Marginal Markov Chain We can sample from ˇπ(X, θ, u Y ) using Metropolis-within-Gibbs with block updates. X -update. Propose X Q X and accept with probability { min 1, π(y X, θ) exp( U(X ; θ)) q(x X } ) π(y X, θ) exp( U(X ; θ)) q(x. X ) (θ, u)-update. Propose (θ, u ) Q θ,u and accept with probability { min 1, π(y X, θ ) S(θ, u) exp( U(X ; θ ))π(θ )π(u ) q(θ θ } ) π(y X, θ) S(θ, u ) exp( U(X ; θ))π(θ)π(u)q(θ. θ) Posterior expectations are estimated using N i=1 E X,θ Y [h(x, θ)] = lim h(x i, θ i )σ(x i θ i, u i ) N N i=1 σ(x. i θ i, u i ) Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

34 Illustration: Airports in England Figure: Left: Trace plots of α and β. 10.1% of the σ values are negative. The posterior mean of α and β are 0.97 and 0.94, respectively, indicating a moderate attraction towards to larger but fairly local airports. Right: 5th and 95th percentiles of the latent states X. There tends to be more variability in larger airports such as Heathrow and Manchester. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

35 Conclusions and Outlook Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

36 Further work and extensions Investigation of new data assimilation methodologies to calibrate models to data available at different scales. For example: Population data; Cost matrix; or Time dependent parameters. Deployment of new methodology to a global problem to provide new insights into urban retail structure. Improved mechanistic models of urban and regional phenomena. Melding of data and models takes us beyond data analytics. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

37 Summary We have revisited the Harris and Wilson model to formally account for uncertainties in the modelling process. A stochastic extension of the model, under certain conditions, may be expressed as a Langevin diffusion. The invariant distribution provides new insights into urban and regional dynamics, and suggests the introduction of a new parameter in the model. We presented a Bayesian hierarchical model for urban and regional systems. We have demonstrated our approach using an example of airports in England. Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38

38 Further Reading / References 1 Equilibrium values and dynamics of attractiveness terms, B. Harris and A. Wilson (1978) 2 Explorations in urban and regional dynamics, J. Dearden and A. Wilson (2015) 3 Exponential convergence of Langevin distributions and their discrete approximations, G. Roberts and R. Tweedie (1996) 4 Pathwise accuracy and ergodicity of metropolized integrators for SDEs, N. Bou-Rabee and E. Vanden-Eijnden (2009) 5 On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods, A. Lyne, et al (2015). 6 Markov Chain Truncation for Doubly-Intractable Inference, C. Wei, and I. Murray (2016). Mark Girolami (Imperial, ATI) Stochastic formulation of SIM / 38