Gambling scores for earthquake predictions and forecasts

Transcription

1 Geophys. J. Int. (21) 181, doi: /j X x Gambling scores for earthquake predictions and forecasts Jiancang Zhuang Institute of Statistical Mathematics 1-3 Midori-Cho, Tachikawa-Shi, Tokyo , Japan. Accepted 29 December 22. Received 29 November 1; in original form 29 July 18 GJI Seismology 1 INTRODUCTION The preparation process of a large earthquake is complicated, and so are the so-called anomalies that appear before the occurrence of large earthquakes. There have been many reports on the precursors prior to large earthquakes. However, when such anomalies occur, the expected large earthquake may not follow. Most seismologists currently suspect that our limited geophysical observations are not sufficient to understand the underground earthquake preparation process and to make earthquake prediction deterministically. Jackson and Kagan suggested that forecasts and predictions should be rigorously distinguished (Jackson & Kagan 1999; Kagan & Jackson 2). They defined statistical forecasts in terms of the rate-density, or probability density per unit time and area, of earthquakes as a function of magnitude, while predictions are more like a decision rule, usually specified as deterministic or quasi-deterministic statements, are much more specific, implying considerably higher probability and more certainty than a forecast. A scientific claim of a prediction or a forecast should include the following elements: (1) a specific location or area, (2) a specific span of time and (3) a specific magnitude range. Currently, earthquake forecasts/predictions can be divided into three classes: (1) Binary ( Yes - and No -) predictions: which give specific statements about the occurrence of a future earthquake. (2) Alarm levels: which give some indexing values showing the relative probabilities of earthquakes under consideration. SUMMARY This paper presents a new method, namely the gambling score, for scoring the performance earthquake forecasts or predictions. Unlike most other scoring procedures that require a regular scheme of forecast and treat each earthquake equally, regardless their magnitude, this new scoring method compensates the risk that the forecaster has taken. Starting with a certain number of reputation points, once a forecaster makes a prediction or forecast, he is assumed to have betted some points of his reputation. The reference model, which plays the role of the house, determines how many reputation points the forecaster can gain if he succeeds, according to a fair rule, and also takes away the reputation points betted by the forecaster if he loses. This method is also extended to the continuous case of point process models, where the reputation points betted by the forecaster become a continuous mass on the space time magnitude range of interest. We also calculate the upper bound of the gambling score when the true model is a renewal process, the stress release model or the ETAS model and when the reference model is the Poisson model. Key words: Probabilistic forecasting; Statistical seismology. (3) Probability forecasts: which are based on some probability models and provide the expectation of earthquakes occurring in a given space time magnitude window. Another way to classify forecasts is based on whether the forecasts are announced regularly or irregularly. Binary predictions are usually tested using methods based on contingency tables. Suppose that the target region is divided into n small cells and let a: number of successful predictions of occurrences, b: number of false alarms, c: number of successful prediction of non-occurrences, and d: number of occurrences failures to prediction. The Hanssen Kuiper skill score, or R-score, is defined by R = a a + d b (1) b + c (see, e.g. Harte & ere-jones 25; Shi et al. 21). In Harte & ere-jones (25), this definition was mistyped as R = a/ (a + b) d/(c + d) (Harte 29, personal communications). If the predictions are better than a random guess, then R >. Shi et al. (21) used this principle for evaluating the annual earthquake predictions made by the China Seismological Bureau. However, there is a deficiency with R-score: as well as Yes and No, another outcome is NA (no prediction, or no comment available), which means that the forecaster provides no comments on whether or not an earthquake will occur in the given space time magnitude window. That is to say, such a test should be based on a Downloaded from by guest on 12 October C 21 The Author Journal compilation C 21RAS

2 3 2 contingency table or different 2 2 contingency table if the forecaster only gives Yes - or NA -predictions, which is much more complicated than the original 2 2case. Typical examples of alarm-level type forecasts are the methods of Pattern Informatics (PI, see e.g. Rundle et al. 22, 23; Nanjo et al. 26) and M8 (see, e.g. Keilis-Borok & Kossobokov 1987, 199). Their performance is usually evaluated by using Molchan s error diagram (Molchan 199, 1991, 1997, 23; Molchan & Kagan 1992), which can be regarded as an extension of the procedure derived from the 2 2 contingency table. For every possible threshold, let the cells with alarm levels above the threshold be announced as Yes -predictions. The curve of the portion of failures to prediction ν = d a + d against the fraction of time on prediction a + b τ = (3) a + b + c + d is called Molchan s error diagram. The resulting Molchan s error diagram provides a comprehensive summary of the performance of such alarms. In the case of complete random guess, as it is easy to see, ν = 1 τ, the diagram is a straight line joining the points (, 1) and (1, ). A better alarm strategy than a completely random guess should have a ν τ curve below this straight line while a worse alarm strategy has a curve above it. Probability forecasts are usually made by using point process models of seismicity, such as the Poisson model, renewal models, the stress release model, and the ETAS model. A space time magnitude point process can be regarded as an object which takes values randomly among the countable subset in the form {(t i, x i, m i ): i =..., 1,, 1,...}, wheret i, x i and m i are the temporal, spatial, and magnitude components, respectively. We require that the number of points in a point process falling any bounded space time magnitude volume is always finite. Such models are naturally specified by a conditional intensity function λ of time t, location x and magnitude m, which satisfies (see, e.g. Daley & ere-jones 23, chapter 7) λ(t, x, m)dt dx dm Pr N(dt dx dm) 1 H t, (4) where the notation H t is a simplification of the history of the observation of the process up to time t but not including t, i.e., the subset {(t i, x i, m i ): t i < t} of the point process N consisting of the elements with t i < t; N(dt dx dm) is the number of earthquake events falling in the volume dt dx dm = t, t + dt) x, x + dx) m, m + dm). The probability that at least 1 event occurs in the space time magnitude window T 1, T 2 B M is Pr{N(T 1, T 2 B M) > H T1 } T2 = 1 exp λ(t, x, m)dt dx dm. (5) M B T 1 A forecasting procedure based on simulations, by using the conditional intensity, can be found in ere-jones (1998). Of course, if we use the forecasted probabilities or expectations of earthquakes as alarm levels, Molchan s error diagram can be used to evaluate the performance probability forecasts. A more natural way is to use the information score or the entropy score (see, Kagan & Knopoff 1977; ere-jones 1998; Harte & ere-jones 25). Suppose that the whole space time magnitude window for a forecast is divided into M cells of equal sizes and that the forecast gives a probability p i that at least 1 event occurs in the ith space time magnitude (2) Scores for earthquake forecasts 383 cell, of interest; while the reference model, usually taken as the Poisson model, gives a probability of p i.theinformationgainoncell i against the reference model is then defined as the logarithm of the likelihood ratio B i = Y i log(p i / p i ) + (1 Y i ) log(1 p i )/(1 p i ), (6) where Y i = 1 if there is at least 1 event occurrence in the ith cell, and, Y i =, otherwise. The total information gain over all cells is defined by B = K B i = i=1 K i=1 Y i log p i + (1 Y i )log 1 p i, (7) p i 1 p i where K is the total number of cells. The expected information gain per unit space time magnitude volume is defined as K G = E B i, (8) i=1 where is the total volume of the space time magnitude range, and is the size of each cell. Such an information gain varies when the division of the interested space time magnitude range changes, but tends to a limit when the volume of each cell gets to infinitesimally small. Consider the case where the conditional intensity can be decomposed as λ(t, x, m) = λ g (t, x) h(m; t, x), (9) where λ g is the ground conditional intensity and h(m; t, x) is the conditional probability density function (p.d.f.) of the magnitude distribution at time t and location x, given the history of observations H t. Then the expected information gain per unit space time magnitude volume is λ(t, x, m) G = E log N(dt dx dm) λ (t, x, m) = E λ λ(t, x, m) (t, x, m)log dt dx dm (1) λ (t, x, m) = E λ g (t, x)log λ g(t, x) λ g (t, x) + λ (t, x, m)log dt dx h(m; t, x) h (m; t, x) dm dt dx, (11) where λ, λ and λ are the conditional intensities of the true model, the forecasting model, and the reference model, respectively, all having a similar decomposition as in (9); λ g, λ g and λ g are the corresponding ground intensities; and h, h and h are the corresponding magnitude densities. In the above, the notation of integral of a predictable function f (t, x, m) in a space time magnitude window S with respect to the measure of a point process N is interpreted as f (t, x, m) N(dt dx dm) = f (t i, x i, m i ), (12) i:(t i,x i,m i ) N providing that the realization of N in is {(t i, x i, m i ): i = 1,..., n}. (1) can be obtained from martingale theories related to the properties of the conditional intensity λ (see, e.g. Zhuang 26). The first term in (11) is the contribution to the information gain from predicting the occurrence times and locations, and the second term is the contribution from forecasting the magnitude component. In many point process models for earthquake occurrence, the Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21 RAS

3 384 J. Zhuang magnitude distribution is assumed independent from the occurrence times, that is, λ(t, x, m) = λ g (t, x) h(m), (13) where h(m) is the p.d.f. of magnitudes above a certain threshold m, usually taking the form of an exponential distribution (equivalent to the Gutenberg Richter law). If we make a further assumption that the true model, the reference model, and the forecasting model all use the same magnitude density, then the second term becomes. As the first term does not involve magnitudes, each event has approximately equal contribution to the information gain, no matter whether it is a large or small event. 2 GAMBLING SCORES As mentioned in last section, the following two issues have not been solved in these three evaluation procedures: (1) No-comment-available (NA-) predictions. There are many seismologists who only give Yes predictions, and set the default output on the others as No-comment-available (NA-) predictions. In the contingency table based tests, these NA-predictions are usually regarded as No predictions. Such a treatment results in underestimating the performance of forecasters. (2) Equal weights for all events of different magnitudes. All the above three evaluation methods treat each earthquake equally. This seems strange: for example, if Forecaster A makes a successful prediction of a M8 earthquake and misses 1 M4 events and Forecaster B forecasts these 1 M4 events nicely and misses the M8 earthquake, all three methods give a higher score to B rather to A. To solve this embarrassing situation, we need consider the risk of being failure that the forecaster has taken in each forecast. A fair scoring scheme should reward the success in a way that is compatible with the risk taken. Suppose that we have the reference model, usually the Poisson model for usual cases or Omori-Utsu formula for the case of forecasting aftershocks, which gives probability p that at least 1 event occurs in a given space time magnitude window. Thus, in the viewpoints of the reference model, the risk (probability of being failure) taken by the forecaster is 1 p if he gives a Yes - prediction or p if he gives a No -prediction. The forecaster, similar to a gambler, who starts with a certain number of reputation points, bets 1 reputation point on Yes or No according to his forecast, or bets nothing if he performs a NA-prediction. If the forecaster bets 1 reputation point of his reputation on Yes and loses, the number of his reputation points is reduced by 1; if his forecasts is successful, he should be rewarded (1 p )/p reputation points. The quantity (1 p )/p is the return (reward/bet) ratio for bets on Yes. In this way, if the reference model is correct, the expected return that he gains from this bet is E b R = Pr{Yes reference model} 1 p p Pr{No reference model} (14) =. Similarly, the return ratio for bets on No is p /(1 p ). If reference probability p of earthquake occurrence is small, the return ratio for No -prediction is also small, implying that there is almost no reward if the forecaster bet on something that is almost sure by the reference model. This rule also applies to probability forecasts. Suppose that p is the occurrence probability of an earthquake given by the forecaster. We can regard the forecaster as splitting 1 reputation point by betting p on Yes and 1 p on No. In this way, the forecaster s expected pay-off based on the reference model is still, that is, E b R = p 1 p (1 p) p p p +(1 p ) (1 p) p 1 p (15) =. In the case of negative bets, a bet of 1 on Yes means that the forecaster takes the role of the reference model, that is, he wins 1 reputation if no earthquake and loses (1 p )/p if the concerned earthquake occurs. This is equivalent to that the forecaster bets (1 p )/p on No. We can swap the roles of the forecaster and the reference model. It is not difficult to verify that the case that the reference model bets pp on Yes and p(1 p ) on No against the forecaster is equivalent to the situation that the forecaster bets pp on Yes and p (1 p) on No against the reference model. Reputation points transferred between the forecaster and the reference model are the same in these two cases. This proposed technique significantly differs from previous testing methods for earthquake forecast, especially by taking into account the difference in the occurrence rate for weak and strong events. Since large earthquakes are rare, for example, in a global catalogue M 8 events occur roughly once per year, one may argue that, if we want to test any forecast in a reasonable time period, only smaller earthquakes can be used, this method might become unpractical. This is not true. In this case, the return ratio for a success, G, just become not so big as for large earthquakes. A forecaster can still have a positive reward if his forecasts are better than that from the reference model. The more time he wins, the higher his total score. Now, let us consider the behaviour of the above scoring procedure with respect to the true model. It is reasonable that the true model includes much more complicated information than the reference model and that the reference model is an expectation of the true model when some part of the information in the true model is unavailable. Suppose that the true model gives p as the probability that at least one event occurs in the forecasting space time magnitude window of interest. The expected pay-off to the forecaster is then ER = E q 1 p X (1 X)q p, (16) where the random variable X takes a value of 1 if there are 1 or more events occurring, and otherwise, and q is the probability that the forecaster bets Yes (we assume that the forecaster always bets 1 on Yes or bets nothing). Then qp qp qp ER = E p = E p Eq. (17) If q is independent of p and p then p ER = Eq E Eq =, (18) p since p is always a conditional expectations of p under some conditions, Ep /p = 1. Also, ER requires Cov (q, p /p ) orcov (q, p ) Cov (q, p ), that is, q should be more correlated to the probability p from true model than p from the reference model. Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21RAS

4 3 EXTENDING GAMBLING SCORE TO POINT-PROCESS MODELS In continuous space time scale, point process models specified by the conditional intensity function are naturally used for describing, investigating and forecasting seismicity in a particular region. Throughout this and coming sections, we will fix λ, λ and λ,respectively, to be the conditional intensity functions of the prediction model, the baseline (reference) model and the true model. Similarly, we use λ g, λ g and λ g to denote their corresponding ground conditional intensities. We assume that the bet at Yes has a density b(t, x, m) onthe space time magnitude window of interest, say =,T S M. Then the bet on dt dx dm is b(t, x, m)dt dx dm, and the reward bet ratio is 1 Pr{N(dt dx dm) 1 H t } Pr{N(dt dx dm) 1 H t } 1 λ (t, x, m)dt dx dm λ (t, x, m)dt dx dm (19) λ(t)dt dx dm 1, where λ is the conditional intensity for the reference model. Thus the return for a bet at Yes on dt dx dm is b(t, x, m) N(dt dx dm) b(t, x, m)dt dx dm. (2) λ (t, x, m) By integrating the above quantity over, the forecaster s average profit in a unit space time magnitude volume can be expressed as P = 1 b(t, x, m) N(dt dx dm) b(t, x, m)dt dx dm, λ (t, x, m) (21) with an expectation { EP = 1 } λ E (t, x, m) b(t, x, m) λ (t, x, m) 1 dt dx dm, (22) where λ (t, x, m) is the conditional intensity of the true model. As we discussed before, λ is some conditional expectation of λ, thus λ /λ has a mean of 1, and (22) represent covariance density of b and λ /λ. By using the Cauchy Schwarz inequality, as E λ /λ = 1, EP = 1 ( ) Cov b(t, x, m), λ (t, x, m) dt dx dm λ (t, x, m) 1 λ arbar dt dx dm. (23) λ EP will be maximized when the correlation between b(t, x, m) and λ /λ is one, that is, b(t, x, m) = wλ (t, x, m)/λ (t, x, m) + c(t, x, m), where w is a constant representing the forecaster s bet strength, and c(t, x, m) is a deterministic function describing the amount that is bet regularly. In this way, the upper bound of the total return is determined by w and λ /λ, between which w is controlled by the forecaster and the basic return from λ /λ is bounded by G = max EP w = 1 E { λ (t, x, m) λ (t, x, m) 1 2 dt dx dm}. (24) Scores for earthquake forecasts THE RENEWAL PROCESSES In this and following sections, we consider several examples of temporal point-process models with an independent magnitude distribution. The first example is the renewal process. A renewal process is a generalization of the Poisson process in the waiting times. In essence, the Poisson process has independent identically distributed waiting times exponentially distributed before advancing to the next event. A renewal process is defined as a point process with the waiting times taking on a more general distribution. Suppose that the density function of the waiting times is denoted by f ( ), usually called the renewal density. In the temporal case, we assume that the conditional intensity λ(t, m) can be decomposed as two independent components, the ground conditional intensity λ g (t) and the magnitude density h(m). Denote the hazard function corresponding to f by η, thatis, η(u)du = Pr{u waitingtime < u + du waitingtime > u} = f (u)du 1 F(u) (25) where F is the cumulative probability function of f. Then ground conditional intensity is the same as the hazard function, that is, λ g (t) = f t t N(t ) 1 Ft t, (26) N(t ) where t N(t ) is the occurrence time of the last event before t. Assuming that the forecaster uses the best strategy given by (24), the maximum basic return is { 1 T λg (t) 2 G = E dt} T λ g (t) 1, (27) where λ g (t) is the conditional intensity of the reference model. Since the renewal process is stationary, 1 T λg (t) 2 G = lim T T λ g (t) 1 dt f (τ) 2 τ λg(t) 1 λ g (t) dt dτ (28) =. τ f (τ)dτ If we assume that the reference model is the Poison model of the same average occurrence rate as this renewal process, that is, λ g = E λ g. By choosing a proper time unit, we can normalize the process with unit rate, that is, E λ g (t) = 1, or, equivalently, τ f (τ)dτ = 1, and then G = = f (τ) τ λ g (t) 1 2 dt dτ f 2 (t) 1 F(t) + 1 F(t) dt 2. (29) For illustration, we consider the gamma renewal process and the log-normal renewal process. The equation defining the probability density function of a gamma-distributed random variable is e u/θ f (u; k,θ) = u k 1 foru > andk,θ >, (3) θ k Ɣ(k) with a hazard function η(u) = uk 1 e u θ θ k Ɣ ( ), (31) k, u θ where θ is called the scale parameter and k the shape parameter. If we let kθ = 1, then the rate of the renewal process is normalized Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21 RAS

5 386 J. Zhuang Figure 1. Hazard functions of the gamma renewal process (a) and of the log-normal renewal process (b), both with different parameters (k and σ equal.1,.2,.5, 1, 2, 5 and 1 from the darkest to the lightest curves). The mean rate is normalized to 1 through kθ = 1andμ = σ/2 for the gamma and log-normal renewal densities, respectively. to 1. Fig. 1(a) shows the hazard functions for the gamma densities with different values of k and kθ = 1 fixed. In Fig. 1(a), k = θ = 1 corresponds to the case of the Poisson process. When k < 1 the process tends to be a clustering process; and while k > 1, the variance of the waiting time is smaller than that for a Poisson process with the same rate, that is, the process gets regular. The log-normal distribution has the probability density function 1 f (u; μ, σ) = uσ ln(u) μ 2 2π e 2σ 2 (32) for u > with a hazard function 2 η(u) = uσ 2π 1 erf e ln(u) μ2 2σ 2 ln(u) μ σ 2, (33) where μ and σ are the mean and standard deviation of the variable s natural logarithm (by definition, the variable s logarithm is normally distributed) and erf is the error function. Since the mean of a lognormal random variable is e μ+σ 2 /2, the relation to normalize the rate of a log-normal renewal process is μ = σ 2 /2. Fig. 2 shows the hazard function corresponding to the log-normal density when σ takes different values between.1 and 1 with μ = σ 2 /2. When σ is small, the process is clustered; when σ is large, the process is regular. Fig. 2 gives the upper bounds of the basic returns based on the gamma and log-normal renewal processes when the Poisson model is the reference model. We can see that, when k = 1, the gamma renewal process is identical to the reference Poisson model, and thus, the upper bound of basic return is, while the log-normal renewal process is closest to the Poisson model at σ = 1 but still differs and G is still positive. The upper bounds of the expected return increase when the processes get more regular or more clustered. 5 STRESS RELEASE MODELS The stress release model was introduced in a series of papers by ere-jones and others (see, e.g. Zheng & ere-jones 1991, 1994; also, see, Liu et al. 1998; Shi et al. 1998; Lu et al. 1999; Lu & ere-jones 2; Bebbington & Harte 21, 23; for its extension to the linked stress release models) based on the elastic rebound theory. The information gain for earthquake forecasts based on the stress release model has been discussed by Bebbington (25). This model assumes that the stress level in a certain region is gradually built up, linearly with time by tectonic movements, and drops down suddenly in the form of earthquakes, that is, the stress level at t, X(t), can be written as X(t) = X() + ρt S(t), (34) where X() is the initial stress level at time, ρ is the constant loading rate from the tectonic movements, and S(t) is the accumulated stress from earthquakes in the period, t). We assume that the stress level reduced by the ith earthquake of magnitude m i is s i = 1.75m i, (35) which corresponds to the Benioff strain, and S(t) = s i. (36) i: t i <t The ground conditional intensity is a monotonically increasing function of the stress level, usually chosen as λ g (t) = (X) = e γ X = exp{γ X() + ρt S(t)}, (37) where γ is an amalgam of the strength and heterogeneity of the crust medium in the region. The amount of stress S i released by an earthquake has a distribution with a density function of ξ(s)and cumulative probability function ξ(s), which is independent from the stress level. If the process is assumed to be stationary, that is, X (t) has a stationary distribution with a density f X (x), the forward equation (see, Zheng 1991; Borovkov & ere-jones 2) takes a functional equationformof f X (x) = x (x) f X (y) Pr(S > y x)dy, (38) where S is the random variable for the stress released by an event, which yields (ere-jones 1988) the average occurrence rate λ = E{ X( )} = ρ ES = (y) f X (y)dy. (39) Borovkov & ere-jones (2) showed that, given the above formulation, (X) has the moments, for k = 1, 2,..., E k (X) = ρ (ργ) k 1 (k 1)! ES k 1 n=1 (1 Ee nγ S ). (4) (The original equation in Borovkov & ere-jones (2) was (ργ) k (k 1)!/ k 1 n= {1 E e nγ S }. I think that it was a typing error.) Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21RAS

6 Scores for earthquake forecasts 387 Figure 2. Upper bounds of the gambling scores per unit time for the gamma and log-normal renewal processes of unit occurrence rates. The horizontal axis represents k and σ for the gamma renewal process (solid) and log-normal renew process (dashed), respectively. According to (24) and (39), if we take the reference model as the Poisson model with the same stationary rate λ, the upper bound of the best gambling score when the true model is the stress model is 1 G = lim T T E { T λg (t) 2 dt} λ g (t) 1 = E ( λg ( ) λ 1 ) 2 = ar 2 (X) {E (x)} 2 = γ ES 1 Ee γ S 1, (41) where Ee γ S = e γ x ξ(x)dx. From (41), G is determined by γ and the distribution of S.Usually, ξ(s) is considered in three case: the Pareto distribution, the truncated Pareto distribution and the tapered Pareto distribution. Please see Appendix A for explicit formulae of E S ande e γ S. 6 THE ETAS MODEL The epidemic-type aftershock sequence (ETAS) model was introduced by Ogata in a series of papers (see, e.g. Ogata 1988, 1989, 1992) to describe multistage aftershock activity. It has been used in many studies (e.g. Ma & Zhuang 21; Helmstetter & Sornette 23; Helmstetter et al. 23; Hainzl & Ogata 25). It is a special kind of marked Hawkes s self-exciting model (Hawkes 1971a,b). This process is also a marked branching process with immigration. These immigrants, represented as the background seismicity in the earthquake context, are modelled as a Poisson process. Every event, background or not, potentially produces its own descendants. This model has been extended to the space-time version with wide application in the study of seismicity (see, e.g. Ogata 1998, 24; Zhuang et al. 22, 24; Console et al. 23; Ogata & Zhuang 26). In this paper, we only consider the temporal case. The ground conditional intensity of this model is λ g (t) = μ + κ(m i ) g(t t i ), (42) i: t i <t where t i and m i are the occurrence time and the magnitude of the ith event, μ is the constant rate of the background process, κ(m) = A e α(m m) is the mean number children produced by an event of magnitude m, m being the magnitude threshold, and g(t) = p 1 c (1 + t/c) p is the probability density of the time lags between the occurrences of the parent event and the child events. This mean intensity of this process (see, e.g. Zhuang et al. 24) is λ = Eλ g (t) = μ/(1 ρ), (43) where ρ = Eκ(m) = κ(m) h(m)dm is the mean number of M descendants in the first generation from an arbitrary earthquake, h(m) is the p.d.f. of the magnitude distribution as in (13). According to Appendix B, E λ 2 g (t) = F() = 1 F(ω)dω 2π = μ2 + 2μ λ 2π(1 ρ 2 ) + λk 2 G (ω) G(ω)dω 2π 1 ρ 2 G (ω) G(ω), (44) where F(τ) =E λ g ( ) λ g ( +τ) and F is the Fourier transformation of F, G and G are the Fourier transformation of g and its complex conjugate, ρ =E κ(m), and K 2 =E κ(m) 2. It is easy to verify that F() has upper and lower bounds satisfying μ 2 + 2μ λ 2π(1 ρ 2 ) + λk 2 ξ 2π where F() μ2 + 2μ λ 2π(1 ρ 2 ) + λk 2 ξ 2π(1 ρ 2 ), (45) ξ = G (ω) G(ω)dω = 2π g 2 2π(p 1)2 (t)dt = c(2p 1). Consider the JMA data set analysed by Zhuang et al. (24) as an example. In their results, various estimates are given: ρ = (taken as.48), c.2, p 1.1. In this study, we also estimate that E (κ(m)) 2 = 1.6, μ =.25 events per day and Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21 RAS

7 388 J. Zhuang λ =.49 events per day, then.42 < F() <.53, which yields.75 < G < This is to say, if the true model of seismicity in the central Japan region is the ETAS model and we use the best betting strategy against the Poisson reference model, then the average pay-off is approximately as much as the bet. 7 DISCUSSION How to bet Eq. (24) shows that the maximum total return from the bet b(t, x, m) is not determined by the total amount of the bet, but by other two components in the bet: one is λ /λ, and the other is w. The ratio λ /λ represents how the bet is correlated to the superiority of the true model to the reference model, and w is the the betting strength. If a negative bet is allowable, one can even bet b(t, x, m) = wλ (t, x, m)/λ (t, x, m) 1 to reap profit at the cost of a zero mean. However, in the usual case that non-negative bets are only allowed, the best betting strategy is b(t, x, m) = w λ (t, x, m) min λ (t, x, m). (46) λ (t, x, m) In the above discussion, the true model, λ (t, x, m), is always unknown. However, we can still calculate another bound by replacing λ (t, x, y) with λ(t, x, m), the conditional intensity of the forecasting model. In the probability sense, the return cannot achieve its upper bound. This does not prevent that, on some occasions, the forecaster can gain a large return ratio. The probability of such phenomena decreases when the betting time interval and range of locations and magnitudes increase. Role of magnitudes Unlike other methods, the gambling score does not give equal weight to each event. Suppose that the magnitude distribution is independent from occurrence times and locations. Then the best bet is b(t, x, m) = w λ (t, x, m) λ (t, x, m) = w λ g (t, x) λ g (t, x), implying that the integrand in (24) does not include m and the integral over magnitudes from 4 to 5, is the same as that from 8 to 9. As there are much more events in the magnitude range between 4 and 5 than between 8 and 9, we can see that a big event is more important in evaluating the score for successful yes -predictions than a small event. Confidence levels Definitely, the success of a forecaster who gains positive points depends either on the superiority of his forecasting model to the baseline model or on his lucks. To distinguish between these two cases, we need to evaluate the confidence level of their scores. Here we consider the following two cases. (i) The reference gives the same probabilities of earthquakes for each prediction. Suppose that the earthquake probability given by the reference model is p, and that there are M successes in the N bets by the forecaster. If the reference model is true, then M is a binomial random variable with parameters N and p. The singlesided α-confidence level ( <α.5) CL R (α) for the total score R can be evaluated through CL R (α) = CL M (α) (G + 1) N. because of R = M(G + 1) N, where G = (1 p )/p is the return ratio for a success bet, and CL M (α)istheα-confidence level of M. (ii) In most cases, the reference model gives different reference probabilities for each predictions. In this way, we can simulate with the reference model many times, and then evaluate the scores of the predictor under each simulation. The (1 α)-quantile of these scores can be as an estimate of the α-confidence level. 8 CONCLUDING REMARKS This paper introduces the concept of the gambling score, which is a risk (chance of failure) compatible scoring method for evaluating the forecast performance for different kinds of earthquake forecasting and predictions. In this scoring procedure, the forecaster is rewarded or punished for success or failure of earthquake prediction according the risks that he has taken, which is specified by the reference model. This method extends naturally to the continuous cases of point process models. ACKNOWLEDGMENTS This research is supported by Grant-in-Aid Nos for Scientific Research (A), and for Young Scientists (Startup), both from Ministry of Education, Science, Sports and Culture, Japan. Discussions with Rodolfo Console, David Harte, David Jackson, Yan Kagan, Maura Murru, Yosihiko Ogata and David ere- Jones are gratefully acknowledged. The author also thanks the editor, Xiaofei Chen, for his encouragements, and Yan Kagan and an anonymous referee, for their helpful comments. REFERENCES Bebbington, M. & Harte, D., 21. On the statistics of the linked stress release process, J. Appl. Probab., 38A, Bebbington, M. & Harte, D., 23. The linked stress release model for spatio-temporal seismicity: formulations, procedures and applications, Geophys. J. Int., 154, Bebbington, M.S., 25. Information gains for stress release models, Pure appl. Geophys., 162, Borovkov, K. & ere-jones, D., 2. Explicit formulae for stationary distributions of stress release processes, J. Appl. Probab., 37(2), Console, R., Murru, M. & Lombardi, A.M., 23. Refining earthquake clustering models, J. geophys. Res., 18(B1), 2468, doi:1.129/22jb213. Daley, D.D. & ere-jones, D., 23. An Introduction to Theory of Point Processes, ol. 1: Elementrary Theory and Methods, 2nd edn, Springer, New York, NY. Hainzl, S. & Ogata, Y., 25. Detecting fluid signals in seismicity data through statistical earthquake modeling, J. geophys. Res., 11, B5S7, doi:1.129/24jb3247. Harte, D. & ere-jones, D., 25. The entropy score and its uses in earthquake forecasting, Pure appl. Geophys., 162(6), Hawkes, A., 1971a. Spectra of some self-exciting and mutually exciting point processes, Biometrika, 58, Hawkes, A., 1971b. Point spectra of some mutually exciting point processes, J. R. Stat. Soc.: Ser. B (Stat. Methodol.), 33, Helmstetter, A. & Sornette, D., 23. Foreshocks explained by cascades of triggered seismicity, J. geophys. Res., 18(B1), 2457, doi:1.129/23jb2485. Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21RAS

8 Helmstetter, A., Ouillon, G. & Sornette, D., 23. Are aftershocks of large californian earthquakes diffusing? J. geophys. Res., 18(B1), 2483, doi:1.129/23jb253. Jackson, D.D. & Kagan, Y.Y., Testable earthquake forecasts for 1999, Seism. Res. Lett., 7, Kagan, Y. & Knopoff, L., Earthquake risk prediction as a stochastic process, Phys. Earth planet. Inter., 14(2), Kagan, Y.Y. & Jackson, D.D., 2. Probabilistic forecasting of earthquakes, Geophys. J. Int., 143, Kagan, Y.Y. & Schoenberg, F., 21. Estimation of the upper cutoff parameter for the tapered pareto distribution, J. Appl. Probab., 38A, Keilis-Borok,.I. & Kossobokov,.G., Periods of high probability of occurrence of the world s strongest earthquakes, Comput. Seismol., 19, Keilis-Borok,.I. & Kossobokov,.G., 199. Premonitory activation of seismic flow: algorithm m8, Phys. Earth planet. Inter., 61, Liu, J., ere-jones, D., Ma, L., Shi, Y. & Zhuang, J., The principal of coupled stress release model and its application, Acta Seismol. Sin., 11, Lu, C. & ere-jones, D., 2. Application of linked stress release model to historical earthquake data: comparison between two kinds of tectonic seismicity, Pure appl. Geophys., 157, Lu, C., Harte, D. & Bebbington, M., A linked stress release model for historical japanese earthquakes: coupling among major seismic regions, Earth Planets Space, 51, Ma, L. & Zhuang, J., 21. Relative quiescence within the jiashi swarm in china: an application of the etas point process model, J. Appl. Probab., 38(A), Molchan, G.M., 199. Strategies in strong earthquake prediction, Phys. Earth planet. Inter., 61(1.2), Molchan, G.M., Structure of optimal strategies in earthquake prediction, Tectonophysics, 193(4), Molchan, G.M., Earthquake prediction as a decision-making problem, Pure appl. Geophys., 149(1), Molchan, G.M., 23. Earthquake prediction strategies: a theoretical analysis, in Nonlinear Dynamics of the Lithosphere and Earthquake Prediction, pp , eds Keilis-Borok,.I. & Soloviev, A.A., Springer, Heidelberg. Molchan, G.M. & Kagan, Y.Y., Earthquake prediction and its optimization, J. geophys. Res., 97(B4), Nanjo, K.Z., Rundle, J.B., Holliday, J.R. & Turcotte, D.L., 26. Pattern informatics and its application for optimal forecasting of large earthquakes in Japan, Pure appl. Geophys., 163(11 12), Ogata, Y., Statistical models for earthquake occurrences and residual analysis for point processes, J. Am. Stat. Assoc., 83, Ogata, Y., Statistical model for standard seismicity and detection of anomalies by residual analysis, Tectonophysics, 169, Ogata, Y., Detection of precursory seismic quiescence before major earthquakes through a statistical model, J. geophys. Res., 97, Ogata, Y., Space-time point-process models for earthquake occurrences, Ann. Inst. Stat. Math., 5, Ogata, Y., 24. Space-time model for regional seismicity and detection of crustal stress changes, J. geophys. Res., 19(B3), B338, doi:1.129/23jb2621. Ogata, Y. & Zhuang, J., 26. Space-time etas models and an improved extension, Tectnophysics, 413(1-2), Rundle, J.B., Tiampo, K.F., Klein, W. & Martins, J.S.S., 22. Selforganization in leaky threshold systems: the influence of near-mean field dynamics and its implications for earthquakes, neurobiology, and forecastings, Proc. Natl. Acad. Sci., 99, Rundle, J.B., Turcotte, D.L., Shcherbakov, R., Klein, W. & Sammis, C., 23. Statistical physics approach to understanding the multiscale dynamics of earthquake fault systems, Rev. Geophys., 41(4), 119, doi:1.129/23rg135. Shi, Y., Liu, J., ere-jones, D., Zhuang, J. & Ma, L., Application of mechanical and statistical models to study of seismicity of synthetic C 21 The Author, GJI, 181, Journal compilation C 21 RAS Scores for earthquake forecasts 389 earthquakes and the prediction of natural ones, Acta Seismol. Sin., 11, Shi, Y., Liu, J. & Zhang, G., 21. An evaluation of chinese earthqauike prediction, , J. Appl. Probab., 38A, ere-jones, D., On the variance properties of the stress release models, Aust.N.Z.J.Stat.,3A(1), ere-jones, D., Probability and information gain for earthquake forecasting, Comput. Seismol., 3, ere-jones, D., Robinson, R. & Yang, W., 21. Remarks on the accelerated moment release model: problems of model formulation, simulation and estimation, Geophys. J. Int., 144(3), Zheng, X., Ergodic theorems for stress release processes, Stoch. Proc. Appl., 37, Zheng, X. & ere-jones, D., Application of stress release models to historical earthquakes from north china, Pure appl. Geophys., 135(4), Zheng, X. & ere-jones, D., Further applications of the stochastic stress release model to historical earthquake data, Tectonophysics, 229, Zhuang, J., 26. Second-order residual analysis of spatiotemporal point processes and applications in model evaluation, J. Roy. Stat. Soc.: Ser. B (Stat. Methodol.), 68(4), Zhuang, J., Ogata, Y. & ere-jones, D., 22. Stochastic declustering of space-time earthquake occurrences, J. Am. Stat. Assoc., 97(3), Zhuang, J., Ogata, Y. & ere-jones, D., 24. Analyzing earthquake clustering features by using stochastic reconstruction, J. geophys. Res., B5(3), B531, doi:1.129/24jb3157. APPENDIX A: EXPLICIT FORMULAE OF G FOR STRESS RELEASE MODELS According to (41), G is determined by ESandEe γ S. In the below, we consider the cases that S belongs to the Pareto, the truncated Pareto and the tapered Pareto distributions. (i) The Pareto distribution of orresponds to the case where the magnitude distribution follows the Gutenberg Richter law, or in other words, the exponential distribution. It has a probability density function ξ(s) = k 1 ( ) s k ; s S, k > 1, (A1) S S where k is linked with the Gutenberg Richter b-value by k = 4 b+1. 3 It has a mean (k 1) S /(k 2) when k > 2; and Ee γ S = S e γ s ξ(s)ds = (k 1)S k 1 γ k 1 Ɣ(1 k,γs ), (A2) where Ɣ(a, x) = t a 1 e t dt is the upper incomplete gamma x function. (ii) The truncated Pareto distribution of s corresponds to the case that the magnitude distribution follows the truncated Gutenberg Richter law. It has a probability density function in the form of ( ) k 1 s k ξ(s) = ( ) 1 k, S s S u, (A3) S S 1 u S S where S u and S are the upper and lower cut-off magnitude threshold, respectively. It has a mean ES = k 1 k 2 S S u ( S u 1 S ) 1 k ( S u S ) 1 k ; (A4) Downloaded from by guest on 12 October 218

9 39 J. Zhuang and Ee γ S = S e γ s ξ(s)ds (k 1)Sk 1 γ k 1 Ɣ(1 k,γs ) Ɣ(1 k,γs u ) =. (A5) 1 (S u /S ) 1 k (iii) The tapered Pareto (Kagan) distribution (see, Kagan & Schoenberg 21; ere-jones et al. 21) has a cumulative distribution function ( ) s k ( S s (s) = 1 exp S ), S s <, (A6) and a density ( k ξ(s) = s + 1 )( ) s k ( ) S s exp, S s <, S (A7) where is an upper cut-off parameter governing the position of the exponential taper to zero in the frequency of large events. This distribution has a mean ( ) ( ES = S + S k S S1 k c exp Ɣ 1 k, S ) ; (A8) and Ee γ S = e γ s d (s) = γ e γ s (s)ds S S ( ) S k S (1 + γ )s = γ exp ds + e γ S = γ S k S + e γ S. S ( 1 + γ Sc ) k 1 ( e S Sc Ɣ 1 k, S ) + γ S (A9) In all above three cases, G is determined by the dimensionless parameters k, γ S and S u /S,and /S. APPENDIX B: PROOF OF (44) To obtain E λ 2 g (t), let us first consider Eλ g (s 1 ) λ g (s 2 ) { s1 = E μ + κ(m) g(s 1 u 1 )N(du 1 dm) M s2 } μ + κ(m ) g(s 2 u 2 )N(du 2 dm ) M = μ 2 + I 1 + I 2 + I 3, where I 1 = μeλ g (s 2 ) = μ 2 /(1 ρ) = I 2. According to (43), { s1 I 3 = E κ(m) g(s 1 u 1 )N(du 1 dm) M s2 } κ(m ) g(s 2 u 2 )N(du 2 dm ). M (B1) Given the realization of the point process, namely, {(t i, m i ): i = 1, 2,...}, the above equation can be rewritten in the usual summation form I 3 = E κ(m i ) g(s 1 t i ) κ(m j ) g(s 2 t j ) i: t i <s 1 j: t j <s 2 = E i: t i <min(s 1,s 2 ) + E i: t i <s 1 j : j i κ(m) 2 g(s 1 t i ) g(s 2 t i ) t j < s 2 κ(m i ) κ(m j ) g(s 1 t i ) g(s 2 t j ) = J 1 + J 2. According to, for example, Zhuang (26), assuming s 1 < s 2, s1 J 1 = E κ(m) 2 g(s 1 τ) g(s 2 τ) λ(τ,m)dτ dm = K 2 λ M g(ν) g(s 2 s 1 + ν)dν, and J 2 = E κ(m) κ(m ) g(s 1 τ 1 ) g(s 2 τ 2 ) λ(τ 1, m) λ(τ 2, m )dτ 1 dm dτ 2 dm = ρ 2 E g(ν 2 ) g(ν 2 ) λ g (s 1 ν 1 )λ g (s 2 ν 2 )dν 1 dν, where K 2 =E {κ(m) 2 }, ρ =E κ(m) and λ = Eλ g ( ). When the model is stationary, E λ g (s 1 ) λ g (s 2 ) is only a function of s 2 s 2. In this case, let F(τ) =E λ g ( ) λ g ( +τ), then F satisfies F(τ) = μ 2 + 2μ λ + K 2 λ +ρ 2 g(v) g(τ + v)dv g(v 1 )g(v 2 )F(τ v 2 + v 1 )dv 1 dv 2. To solve this equation, take the non-unity Fourier transformation of the angular frequency on both sides, then F(ω) = (μ 2 + 2μ λ)δ(ω) + λk 2 G (ω) G(ω) + ρ 2 G (ω) G(ω) F(ω), where G (ω) is the complex conjugate of the Fourier transformation of g,andδ is the Dirac function. Then, F(ω) = μ2 + 2μ λ δ(ω) + λk 2 G (ω) G(ω) 1 ρ 2 1 ρ 2 G (ω) G(ω), (B2) which implies that F(τ) = μ2 + 2μ λ + λk 1 ρ 2 2 ρ 2i ζ (i+1) (τ), (B3) i= where ζ (τ) = g(v) g(τ + v)dv is the self cross-correlation of g, and ζ i represents the i-fold convolution power of ζ. Downloaded from by guest on 12 October 218 C 21 The Author, GJI, 181, Journal compilation C 21RAS