Self-exciting point process modeling of crime

Transcription

1 Self-exciting point process modeling of crime G. O. Mohler M. B. Short P. J. Brantingham F. P. Schoenberg G. E. Tita Abstract Highly clustered event sequences are observed in certain types of crime data, such as burglary and gang violence, due to crime specific patterns of criminal behavior. Similar clustering patterns are observed by seismologists, as earthquakes are well known to increase the risk of subsequent earthquakes, or aftershocks, near the location of an initial event. Space-time clustering is modeled in seismology by self-exciting point processes and the focus of this paper is to show that these methods are well suited for criminological applications. We first review self-exciting point processes in the context of seismology. Next, using residential burglary data provided by the Los Angeles Police Department, we illustrate the implementation of self-exciting point process models in the context of urban crime. For this purpose we use a fully non-parametric estimation methodology to gain insight into the form of the space-time triggering function and temporal trends in the background rate of burglary. 1 Introduction Criminological research has shown that crime can spread through local environments via a contagion-like process (Johnson 2008). For example, burglars will repeatedly attack clusters Department of Mathematics, University of California, Los Angeles Department of Mathematics, University of California, Los Angeles Department of Anthropology, University of California, Los Angeles Department of Statistics, University of California, Los Angeles Department of Criminology, University of California, Irvine 1

2 of nearby targets because local vulnerabilities are well-known to the offenders (Bernasco and Nieuwbeerta, 2005). A gang shooting may incite waves of retaliatory violence in the local set space (territory) of the rival gang (Tita and Ridgeway, 2007; Cohen and Tita, 1999). The local, contagious spread of crime leads to the formation of crime clusters in space and time. Similarly, the occurrence of an earthquake is well known to increase the likelihood of another earthquake nearby in space and time. For example, we plot in Figure 1 a histogram of the times between nearby earthquakes, pairs of earthquake events separated in space by 110 kilometers or less, for all recorded earthquakes of magnitude 3.0 or greater in Southern California during The histogram shows a spike at short times, indicating an increased likelihood of another event in the days following each earthquake. For a stationary Poisson process the distribution of times between pairs of events would be approximately uniform when the length of the time window is much larger than the longest time bin of the histogram. In the case of residential burglary, evidence indicates that an elevated risk exists for both a house that has been recently burgled and its neighboring houses (Farrell and Pease, 2001; Short et al., 2009; Johnson et al., 2007). To illustrate this point further, we plot in Figure 1 a histogram of the times between nearby burglaries, residential burglaries separated in space by 200 meters or less, for all recorded residential burglaries within an 18km by 18km region of the San Fernando Valley in Los Angeles during Again we observe a spike at short times, indicating an increased likelihood of victimization within a few hundred meters and several days of each burglary. Self-excitation is also found in gang violence data, as an event involving rival gangs can lead to retaliatory acts of violence. In Figure 2, we plot the times of all recorded violent crimes between the gang known as Locke Street and the rival gang known as Lowell Street occurring between 2000 and 2002 in the Los Angeles police district of Hollenbeck. Here we observe clear clustering patterns suggestive of self-excitation in the rate at which 2

3 Figure 1: On the left, histogram of times (less than 50 days) between burglary events separated by 200 meters or less. On the right, histogram of times (less than 300 days) between Southern California earthquake events of magnitude 3.0 or greater separated by 110 kilometers or less. the two rival gangs attack each other. Figure 2: Times of violent crimes between two rivalry gangs in Los Angeles. We propose that self-exciting point processes can be adapted for the purpose of crime modeling and are well suited to capture the spatial-temporal clustering patterns observed in crime data. More specifically, spatial heterogeneity in crime rates can be treated using background intensity estimation and the self-exciting effects detected in crime data can be modeled with a variety of kernels developed for seismological applications or using non- 3

4 parametric methods. In Section 2, we review self-exciting point processes in the context of seismological modeling. In Section 3, we present a model for residential burglary based upon non-parametric methods for Epidemic Type Aftershock-Sequences models of earthquakes. Our methodology combines the idea of stochastic declustering with Kernel Density Estimation in a novel way. We validate the methodology with a simulated point process in the Appendix. 2 Self-exciting point process models in seismology A space-time point process N(t, x, y) is typically characterized via its conditional intensity λ(t, x, y), which may be defined as the limiting expected rate of the accumulation of points around a particular spatial-temporal location, given the history H t of all points up to time t (Daley and Vere-Jones, 2003): λ(t, x, y) = lim t, x, y 0 E [ N{(t, t + t) (x, x + x) (y, y + y)} H t ] t x y. (1) In Seismology a mark M k, the magnitude of the earthquake, is associated with each event (t k, x k, y k ) and the conditional intensity often takes the form, λ(t, x, y, M) = j(m)λ(t, x, y), (2) λ(t, x, y) = µ(x, y) + g(t t k, x x k, y y k ; M k ). (3) {k: t k <t} Models of this type, referred to as Epidemic Type Aftershock-Sequences (ETAS) models, work by dividing earthquakes into two categories, background events and aftershock events. Background events occur independently according to a stationary Poisson process µ(x, y), with magnitudes distributed independently of µ according to j(m). Each of these earth- 4

5 quakes then elevates the risk of aftershocks and the elevated risk spreads in space and time according to the kernel g(t, x, y, M). Many forms for g have been proposed in the literature, though in general the kernel is chosen such that the elevated risk increases with earthquake magnitude and decreases in space and time away from each event. For example, the isotropic kernel, g(t, x, y; M) = K 0 (t + c) e α(m M 0) p (x 2 + y 2 + d), (4) q is one of a variety of kernels reviewed in (Ogata, 1998). Standard models for the background intensity µ(x, y) include spline, kernel smoothing, and Voronoi estimation (Ogata and Katsura, 1988; Silverman, 1986; Okabe et al., 2000). In the case of fixed bandwidth kernel smoothing, the background intensity is estimated by, µ(x, y) = µ u(x x k, y y k ; σ), (5) k where µ is a parameter controlling the overall background rate. The events (t k, x k, y k, M k ) are assumed to be background events and in practice can be obtained through a declustering algorithm (Zhuang, Ogata, and Vere-Jones, 2002). The appropriate selection of parameter values is as critical to the modeling process as specifying accurate forms for µ, g, and j. The distance in space and time over which the risk spreads, the percentage of background events vs. aftershocks, the dependence of the increased risk on magnitude size, etc. all can have a great impact on the predictive power of a point process model. Parameter selection for ETAS models is most commonly accomplished through maximum likelihood estimation, where the log-likelihood function (Daley and Vere- 5

6 Jones, 2003), l(θ) = k { } T log λ(t k, x k, y k ; θ) λ(t, x, y; θ)dydxdt, (6) 0 S is maximized over all parameter sets θ. Here S [0, T ] is the space-time window of observation. More recently, non-parametric methods have been introduced for self-exciting point process estimation (Zhuang, 2006; Marsan and Lenglin, 2008). Consider space-time point data {(t k, x k, y k )} N k=1 and a general self-exciting point process model of the form, λ(t, x, y) = µ(t, x, y) + g(t t k, x x k, y y k ). (7) {k: t k <t} Assuming model correctness, the probability that event i is a background event, p ii, is given by, p ii = µ(t i, x i, y i ) λ(t i, x i, y i ), (8) and the probability that event j triggered event i, p ji, is given by, p ji = g(t i t j, x i x j, y i y j ), (9) λ(t i, x i, y i ) (Zhuang, Ogata, and Vere-Jones, 2002). Let P denote the matrix with entries p ji (note that the columns sum to one). Then stochastic declustering can be used in the following way. Given an initial guess P 0 of the matrix P, we then have N(N +1)/2 probabilistic data points {(t k, x k, y k, p kk )} N k=1 and {(t i t j, x i x j, y i y j, p ji )} i>j. Given this data, a non-parametric density estimation procedure can be used to estimate µ from {(t k, x k, y k, p kk )} N k=1 and g from {(t i t j, x i x j, y i y j, p ji )} i>j, providing estimates µ 0 and g 0. We can then proceed iteratively as follows until convergence is achieved, 6

7 Step 1) Estimate µ n and g n from P n 1. Step 2) Update P n from µ n and g n using (8) and (9). For example, a simple histogram estimator is used in (Marsan and Lenglin, 2008) in step 1. 3 A self-exciting point process model of burglary For the purpose of modeling burglary we consider an unmarked self-exciting model for the conditional intensity of the form, λ(t, x, y) = ν(t)µ(x, y) + g(t t k, x x k, y y k ). (10) {k: t k <t} Here we neglect spatially-localized temporal fluctuations in the background rate and assume that the fluctuations occur globally (for example due to weather, seasonality, time of day, etc.) In the case of seismology, research over a number of decades was needed to refine the (parametric) form of the triggering function g. For this reason, non-parametric methods are appealing in the context of crime in order to quickly gain insight into the forms of ν, µ and g. For this purpose we use the iterative procedure outlined in the previous section to estimate the model, with several modifications. Because the probabilistic data {(t k, x k, y k, p kk )} N k=1 and {(t i t j, x i x j, y i y j, p ji )} i>j is both 3-dimensional and the number of data points is O(N 2 ) (where N is typically O(1000) for earthquake and crime data sets), the estimation step for µ and g is computationally expensive. The dimensionality prevents straight forward use of binning methods such as the Average Shifted Histogram (Marsan and Lenglin use a standard histogram on a coarse grid), as many bins may have extremely small, but non-zero, values (since the data is probabilistic, 7

8 the count in each bin can be less than 1). Alternatively, the large size of the data set prevents efficient use of off-grid methods such as Kernel Density Estimation. To get around these issues we use the following Monte-Carlo based iterative procedure, Step 1) Sample background events {(t b i, x b i, y b i )} N b i=1 and offspring/parent inter-point distances {(t o i, x o i, y o i )} No i=1 from P n 1. Step 2) Estimate ν n, µ n and g n from the sampled data using variable bandwidth Kernel Density Estimation. Step 3) Update P n from ν n, µ n and g n using (8) and (9). Because N b + N o = N, the size of the sampled data at each iteration allows for the use of Kernel Density Estimation. Another issue is that the number of background and offspring events, N b and N o, is changing at each iteration. Thus a fixed bandwidth for any density estimation technique (kernel smoothing, histogram, etc.) will over smooth at some iterations and undersmooth at others. Therefore we employ variable bandwidth KDE (alternatively Cross Validation could be used). We give further details of our approach and provide validation using a simulated point process in the Appendix. Results We fit the model given by Equation (10) to a dataset collected by the Los Angeles Police Department of 5376 reported residential burglaries in an 18km by 18km region of the San Fernando Valley in Los Angeles occurring during the years 2004 and Each burglary is associated with a reported time window over which it could have occurred, often a few hour span (for instance, the time span over which a victim was at work), and we define the time of burglary to be the midpoint of each burglary window. 8

9 In Figure 3, we plot the sampled inter-point distances {(t o i, x o i, y o i )} No i=1 on the 75th iteration of the stochastic declustering algorithm (see Appendix for convergence verification). The number of sampled (offspring) events is 706 (13.1% of all events) and of these events approximately 68% are exact-repeats (occurring at the same house). On the left, the spatial inter-point distances are plotted showing that elevated crime risk travels around 50m 100m from the house of an initial burglary. Note also that the risk travels vertically and horizontally (along streets), more so than it does in other directions. On the right, we plot the spatial (x-coordinate) inter-point distances against the time inter-point distances. Here exact-repeat burglaries, those occurring at the same house, are apparent along the x-axis. Figure 3: Spatial (left) and space-time (right) offspring/parent inter-point distances {(t o i, xo i, yo i )}No i=1 sampled from P 75. In Figure 4, we plot (on a logarithmic scale) the estimated marginals g 75 (t) = g 75 (t, x, y)dxdy x y and g 75 (x) = g 75 (t, x, y)dydt t y computed from the KDE estimate of g at the 75th iteration. Here the presence of exact- 9

10 repeat events can again be seen, as g 75 (x) appears to approximate a delta distribution at the origin. The spike around 1-2 days in the plot of g 75 (t) is due to the presence of fast crime sprees, where most likely the same burglar visited several neighboring houses within a time span of a few minutes to several days. There are also several bumps in the elevated risk of burglary, for example around 7 days. Here one possible explanation is that the routine of the burglar and/or the victim is such that a particular day of the week is a preferable time to commit the burglary. After 7-10 days, the elevated risk of repeat/near-repeat victimization drops to an intermediate level and stays relatively flat for a time span on the order of several hundred days before decaying back to baseline rates. These results are consistent with previous quantitative studies of exact-repeat burglaries (Short et al, 2009). Figure 4: Marginal g 75 (t) (left) and marginal g 75 (x) (right) estimated using KDE based upon offspring/parent inter-point distances sampled from P 75. In (Johnson et al, 2008), the authors discuss the need to model risk heterogeneity and in general it is a difficult task to separate clustering due to background heterogeneity and clustering due to self-excitation. One benefit of using the non-parametric approach outlined above is that temporal and spatial changes in the rate of crime are automatically separated into those stemming from exogenous effects and those due to self-excitation. In Figure 5, we plot the estimated marginals ν 75 (t) and µ 75 (x, y) estimated using KDE from {(t o i, x o i, y o i )} No i=1 10

11 at the 75th iteration. Here the estimated background rate exhibits temporal fluctuations on a time scale of months/days, separate from the fluctuations due to self-excitation. These fluctuations are likely caused by a number of factors such as seasonal, economic, and demographic changes. In seismology, declustered catalogs are of great interest as they can be used in estimating the background rate of major earthquakes. Similarly, declustered crime catalogs could be used to more accurately assess how exogenous effects play a role in changing crime rates. Figure 5: Background rate time marginal ν 75 (t) (left) and space marginal µ 75 (x, y) (right) estimated using KDE from the background events sampled from P Discussion We showed how self-exciting point processes from seismology can be used for the purpose of crime modeling. In the future it may be desirable to tailor point process models specifically for crime, taking into account the crime type and the local geography of the city. Based upon the insights provided by nonparametric estimates, parametric models can be constructed that have advantages with respect to model fitting and simulation. Background rates can also be improved by incorporating other data types (in Johnson et al, 2008 housing density is used to improve models of repeat victimization). In the case of gang violence, a hybrid network- 11

12 point process approach may be useful for capturing the self-exciting effects stemming from gang retaliations. Here increased risk may not diffuse in geographic space, but instead may travel through the network space of gang rivalry relations. The methodology used in this study can be implemented for other applications as well, for example refining point process models of earthquakes. It could potentially be adapted, more generally, to other second order models of point processes. The stochastic declustering algorithm opens up the door to a plethora of density estimation techniques (Silverman, 1986; Scott, 1992; Eggermont and LaRiccia, 2001) that could be used to explore point processes in a way parametric methods do not allow. In (Marsan and Lenglin, 2010) it is shown that the method is an Expectation-Maximization (EM) type algorithm. At the maximization step the complete data log-likelihood function decouples in terms of the background and triggering functions, which is why at each iteration the problem reduces to several decoupled density estimation problems. Several issues could potentially arise here, one being that the method could converge to a local (but not global) minimum of the observed data log-likelihood function. In numerical tests we have found > O(1000) data points are needed in 3 dimensions for the iterates to converge to the right densities. Similar to analytical results for standard density estimation, it would be useful to have convergence results relating sample size, branching ratio, domain size, and the bandwidth of the density estimators to the solution of the fixed point iteration. 5 Appendix Given point data (t k, x k, y k ) N k=1 and a self-exciting point process model of the form, λ(t, x, y) = ν(t)µ(x, y) + g(t t k, x x k, y y k ), (11) {k: t k <t} 12

13 we iterate the following until convergence: Step 1) Sample background events {(t b i, x b i, y b i )} N b i=1 and offspring/parent inter-point distances {(t o i, x o i, y o i )} No i=1 from P n 1. Step 2) Estimate ν n, µ n and g n from the sampled data. Step 3) Update P n from ν n, µ n and g n using (8) and (9). In order to estimate ν n, µ n and g n from the sampled data, we use variable bandwidth Kernel Density Estimation. To estimate g n, we first scale the data {(t o i, x o i, y o i )} No i=1 to have unit variance in each coordinate and based upon the rescaled data compute D i, the kth nearest neighbor distance (3-dimensional Euclidean distance) to data point i. We then transform the data back to its original scale and, letting σ x, σ y, and σ t be the sample standard deviation of each coordinate, estimate the triggering function as, g n (t, x, y) = 1 N N o i=1 1 σ x σ y σ t (2π) (3/2) D 3 i ( exp (x xo i ) 2 (y yo i ) 2 (t ) to i ) 2. 2σxD 2 i 2 2σyD 2 i 2 2σt 2 Di 2 The background rate is estimated similarly, where 1-dimensional and 2-dimenional Gaussian kernels are used to estimate ν n and µ n respectively. Throughout we compute D i corresponding to ν using the 100th nearest neighbor distance and in higher dimensions we use the 15th nearest neighbor distance for D i corresponding to µ and g. We validate the method by simulating (11) with ν(t)µ(x, y) = ( ) ( ) µ (2π)(4.5) exp x2 exp y2 2 2(4.5) 2 2(4.5) 2 and ( ) ( ) ( ) g(t, x, y) = θω exp ωt exp x2 exp y2 2σx 2 2σy 2 and comparing the estimates supplied by the method with the known distribution. 13

14 In Figure (6), we plot the L2 error P n P n 1 2 at the nth iteration and the number of sampled background events N b at the nth iteration against the true number of background events. Here we observe that the error converges quickly for the first 10 iterations and then stabilizes as the error introduced by estimating the point process through sampling P cannot be reduced further (unless a deterministic iterative procedure is employed). We also verify that the method applied to the 5376 burglary events in Section 3 reached convergence in Figure (6). Here we observe a similar rate of convergence for the crime data as with the simulated point process. Figure 6: L2 error P n P n 1 2 (top left) and N b, the number of sampled background events, (top right) at the nth iteration for known point process model. L2 error P n P n 1 2 (bottom left) and N b, the number of sampled background events, (bottom right) at the nth iteration for the method applied to the 5000 burglary events in Section 3. In Table 1, we list the exact parameter values used for the simulated point process and 14

15 the estimates on the 75th iteration computed using the sample variances of the coordinates of {(t o i, x o i, y o i )} No i=1 and the values of N b, N o. In Figure (7), we plot the estimated marginals of g(t, x, y) against the actual distributions. The parameter values were selected to yield point patterns with scales similar to those observed in crime data. Table 1: Parameter values w 1 σ x σ y θ µ exact estimate Figure 7: Estimated (circles) and actual (solid line) marginals of g(t, x, y) on the 75th iteration. Upper left is the marginal g(x), upper right is the marginal g(y), and lower figure is marginal g(t). 15

16 References Cited [1] Bernasco, W. and Nieuwbeerta, P. (2005). How do residential burglars select target areas? A new approach to the analysis of criminal location choice. Brit. J. Criminology 45, [2] Cohen, J. and Tita, G. (1999). Spatial Diffusion in Homicide: Exploring a General Method of Detecting Spatial Diffusion Processes. Journal of Quantitative Criminology 15 (4), [3] Daley, D. and Vere-Jones, D. (2003). An Introduction to the Theory of Point Processes, 2nd edition. New York: Springer. [4] Eggermont, P. P. B. and LaRiccia, V. N. (2001). Maximum Penalized Likelihood Estimation: Volume I: Density Estimation, New York: Springer. [5] Farrell, G. and Pease, K. (Ed.) (2001). Repeat Victimization, Criminal Justice Press, New York. [6] Johnson, S. D. (2008). Repeat burglary victimisation: a tale of two theories. Journal of Experimental Criminology 4, [7] Johnson, S. D., Bernasco, W., Bowers, K. J., Elffers, H., Ratcliffe, J., Rengert, G., and Townsley, M. (2007). Space-time patterns of resk: A cross national assessment of residential burglary victimization, J. Quant. Crim., 23, [8] Marsan D. and Lenglin, O. (2008). Extending Earthquakes Reach Through Cascading, Science, 319:

17 [9] Marsan D. and Lenglin, O. (2010). A new estimation of the decay of aftershock density with distance to the mainshock. Preprint ( [10] Ogata, Y. (1998). Spate-time point process models for earthquake occurrences, Ann. Inst. Statist. Math., 50 (2), [11] Ogata, Y. and Katsura, K. (1988). Likelihood analysis of spatial inhomogeneity for marked point patterns, Ann. Inst. Statist. Math., 40, [12] Okabe, A., Boots, B., Sugihara, K. and Chiu, S. (2000). Spatial Tessellations, 2nd ed. Wiley, Chichester. [13] Scott, D. W. (1992). Multivariate Density Estimation, New York: Wiley. [14] Short, M.B. et al.. (2009). Measuring and Modeling Repeat and Near-Repeat Burglary Effects, J. Quant. Criminol. 25 (3), 325. [15] Silverman, B. W. (1986). Density Estimation for Statistics and Data Analysis, London: Chapman and Hall. [16] Tita, G. and Ridgeway, G. (2007). The Impact of Gang Formation on Local Patterns of Crime. Journal of Research on Crime and Delinquency 44 (2), [17] Zhuang, J., Ogata, Y., and Vere-Jones, D. (2002). Stochastic Declustering of Space- Time Earthquake Occurences, JASA, 97 (458), [18] Zhuang, J. (2006). Second-order residual analysis of spatiotemporal point processes and applications to model evaluation. Journal of the Royal Statistical Society: Series B, 68 (4),