On the principle of maximum entropy and the earthquake frequency -magnitude relation

Transcription

1 Geophys. J. R. astr. SOC. (1983) 74, On the principle of maximum entropy and the earthquake frequency -magnitude relation 21 P. Y. Shen and L. Mansinha Department of Geophysics, University of Western Ontario, London, Ontario N6A 587, Canada Received 1982 December 28; in original form 1982 July 3 Summary. The entropy S for a continuous distribution p (x) is defined by where m (x) is the prior distribution representing our complete ignorance about x. The difficulty in using this definition arises from the problem of arbitrariness or subjectiveness in assigning the prior distribution m (x). Thus in maximum entropy inference, it is customary to arbitrarily adopt a uniform prior distribution and write the entropy as This expression, however, is a measure of uncertainty relative to the coordinate x so that the probability distribution p (~) generated from the principle of maximum entropy depends on the choice of x. Only when the chosen parameter actually has a uniform prior distribution, can we expect the generated distribution to conform with the empirical data. For a physical system in which the independent variable x is measured to only limited accuracy, the prior distribution m(x) can be shown to be inversely proportional to the measurement error of x. A parameter with uniform prior distribution, then, is one that can be measured with equal accuracy throughout its range. In this context, the magnitude of an earthquake is such a parameter because using this parameter in the principle of maximum entropy leads to the empirically determined Gutenberg-Richter frequency-magnitude relation. Other proposed frequency-magnitude relations can also be generated from the principle of maximum entropy by imposing appropriate constraints. However, it is emphasized that such relations are generated from the principle as null hypotheses, to be tested by the empirical regional or global seismicity data.

2 778 Introduction P. Y. Shen and L. Mansinha The principle of maximum entropy can be formalized as follows. Let x be a discrete random variable whose values (xi, i = 1,2,..., N) form a mutually exclusive and exhaustive set and let pi = pi(xi) be the associated probability distribution. Define the entropy S as (Shannon 1948) N S=- C pilnpi i= 1 and let the distribution pi be subject to the following constraints: N 1 Pi=l i=l and N gipi=gi, j=1,2,..., J i= 1 where gi is any function of x and gi its expected value. We seek the distribution pi which maximizes the entropy S, subject to these restraints. If x is a continuous random variable, equations (l), (2) and (3) are usually replaced by s=- 1 2 l: and p (x) dx = 1, p(x) In p (x) dx, [*gi(x)p(x)dx=gi, x1 < x Q x2 j=1,2,..., J Berrill & Davis (1980) recently applied the principle of maximum entropy to the determination of earthquake recurrence relationship. The well-known frequency-magnitude relation of Gutenberg & Richter (1944) is h N(M) = - bg M (7) where N(M) is the number of earthquakes with magnitude greater than or equal to M, and ag and bg are parameters describing regional seismicity. Berrill & Davis showed that (7) is a consequence of the maximum entropy principle, provided that there is no upper bound to M and a mean magnitude exists. It was also shown that under the physically reasonable assumption that the magnitude is bounded (Knopoff & Kagan 1977), the principle of maximum entropy generates the truncated exponential distribution (Riznichenko 1964; Cornell & Vanmarcke 1969): (4) (5) exp (- brm) - exp (- brm1) N(M) = T, O<M<M1 1 - exp (-brml) where T is the total number of earthquakes with 0 < M Q MI. Berrill & Davis interpreted these findings as an independent confirmation of the frequency-magnitude relationships. A

3 Maximum entropy and the frequency-magnitude relation 779 closer examination, however, indicates that the application of the maximum entropy principle is not without ambiguity and care must be exercised in drawing such conclusions. To begin with, a probability distribution can be viewed either objectively (Feller 1950) or subjectively (Jeffreys 1939). Only with a subjective view can we regard the probability distribution generated from the maximum entropy principle as justified independently of empirical verification. Secondly, the entropy defined by (4) for a continuous distribution is not a formal extension of (1). It is not invariant under coordinate transformations and the probability distribution p (x) generated from the principle depends on the choice of the independent variable x. The correct entropy is given by where m (x) is the prior probability distribution (Jaynes 1968). When we compare (9) with (4), we see that adopting definition (4) as entropy is equivalent to arbitrarily assigning a uniform prior distribution m (x). Thus only when the chosen independent variable actually has a uniform prior distribution, can we expect to generate the correct probability distribution. For physical systems not free from measurement errors, such a variable is the one that can be measured with equal accuracy throughout its range. Other choices of the independent parameter would likely lead to absurd probability distributions. The principle of maximum entropy In probability theory, there are two fundamentally different views toward the interpretation of a probability distribution. For an objectivist, the probability of an event is considered an objective property of that event which can always be measured as a frequency ratio in a random experiment. A subjectivist, on the other hand, regards the probability of an event as a formal expression of our expectation that the event will, or did, occur, based on information that is available. The mathematical formulations of the maximum entropy principle under these two views are identical. However, in the objective school, physical probability distributions are generated from the maximum entropy principle simply as null hypotheses, to be subject to experimental verification (Good 1963). A subjective (or non-physical) probability distribution, on the other hand, is considered justified independently of any physical argument and experimental verification (Jaynes 1957). Thus if the probability distribution generated from the principle of maximum entropy agrees with the empirical one, a subjectivist can interpret it as an independent confirmation. But this interpretation depends critically on the existence of a unique and invariant entropy. If the entropy lacks invariance under coordinate transformations, the probability distribution generated from the maximum entropy principle would not be unique and we must take the objective view. Shannon (1948) chose (4) as the entropy for a continuous distribution p(x). This entropy is a measure of uncertainty relative to the coordinate x and is not invariant under coordinate transformations. Shannon, however, was not concerned with this limitation because in information theory, the quantity of interest is the rate of information, or channel capacity. This quantity is the entropy difference of two distributions which are absolutely continuous with respect to each other, and consequently does not depend on the choice of parameters, In maximum entropy inference, the quantity of interest is the absolute entropy itself. Probability distributions are generated by maximizing the entropy. Thus a lack of invariance of entropy under coordinate transformations implies non-uniqueness in the derived probability distribution. As we have mentioned earlier, the correct extension of (1) to a continuous random variable is given by (9). Let us examine the intuitive meaning of

4 780 P. Y. Shen and L, Mansinha the prior distribution m (x) by considering a case in which there are no constraints on p (x) except that the integral of p(x) over all x is equal to 1. Using (9), instead of (4), for the entropy, the principle of maximum entropy generates the following probability distribution Now, the presence of no constraints means that we have no prior information about the random variable x. Thus except for a constant factor, m (x) is the distribution representing our complete ignorance about x (Jaynes 1968). Expression (9) satisfies Shannon s (1948) requirements for an information measure and is invariant under any coordinate transformations because p (x) and m (x) transform in the same way. It is an absolute measure of uncertainty about the random variable x and can be interpreted as the information needed to change the probability distribution from m (x) to p (x) (Good 1950; Kullback & Leibler ; Kullback 1959). Thus the problem of defining entropy for a continuous distribution is reduced to one of assigning objectively the prior distribution. Once the prior distribution is assigned, the principle of maximum entropy will provide a unique, parameter-independent method for selecting probability distributions. But how do we find the prior distribution that represents complete ignorance? The suggestion of T. Bayes and P. S. Laplace is that in some cases, we express complete ignorance by assigning a uniform prior probability density. Jeffreys (1939, 1957), on the other hand, suggested a dx/x prior probability for a continuous variable x known to be positive. Each of these rules has its domain of applications. The problem is that there appears to be no obvious criterion for choosing any particular prior distributions. Some ideas of utilizing transformation groups to classify parameters according to their prior distributions have recently been discussed by Jaynes (1 968). The group theoretical approach, however, is not essential when the independent variable is a physical quantity that can be measured to only limited accuracy. For such physical systems, we can divide the range of the independent variable x into intervals according to the accuracy with which x is measured so that two adjacent intervals are only just discernible. The entropy for the continuous variable can then be thought of as that of a discrete one and given by (Good 1963 ; Jaynes 1963; Purcaru 1973) S=-Zp(x)ln [p(x)ax] Ax. (10) Each of the just discernible intervals has equal prior probability (Lindley 1961 ; Good 1962). The length of these intervals, however, may vary from one part of the range to another and will certainly vary if the range is infinite. In other words, intervals of equal length may have unequal prior probabilities, depending on the variation of measurement error across the range. Let us denote the measurement error of x by u(x). Passing (10) to the continuous limit, we get s = - IX: p(x) In [p(x) u(x)l dx, x1 Q x < x2 Comparing this expression with (9), we see that m (x) = l/u(x), i.e. the prior probability density is inversely proportional to the error with which the independent parameter is measured. Let us illustrate the above results by a simple example. Consider a parameter x which is known to have a uniform distribution between, say, x1 = 0 and x2 = 1OO.x is also known to be measured to only limited accuracy, the measurement error being u(x) = xlc where X is a constant. We assume, for convenience, that X = 0.1 and consider 100 samples of x. Since x

5 Maximum entropy and the frequency-magnitude relation 781 is uniformly distributed, we would expect one sample to lie in each of the 100 intervals (0, l), (1,2),...,(99, 100). However, the samples which have values within the interval (xo- O.lxo,xo + 0.1~~) are indistinguishable, owing to the measurement error. Thus we could conceivably assign the value of xo to all these samples. For example, we could have nine samples with the value of x = 90, eight samples with x = 80, and so on. The resulting distribution, then, is not p(x) = constant but p(x) = 0.01~. This is proportional to o(x). Clearly, if we want to obtain the correct distribution, we must assign to each sample a weight in inverse proportion to the measurement error. In other words, we must assign a prior distribution of l/o(x). Notice that the important quantity here is the variation of measurement error across the range of the variable and not the absolute value of the error. In other words, the prior distribution of a physical parameter depends not on the method of measurement but on the method s capability (or incapability) of achieving uniform measurement accuracy across the parameter s range. When m (x) = constant, equation (9) reduces to (4) except for an additive constant. Thus Shannon s (1948) definition of entropy can be adopted for maximum entropy inference when the independent variable is measured to equal accuracy throughout its range. The same definition must also be used when no information is available about the accuracy of measurement (Good 1963). This is the approach generally adopted in the maximum entropy inference. Some recent examples are Rietsch (1977), Gull & Daniell (1978), Rubincam (1979, 1981) and Berrill & Davis (1980). Since Shannon s entropy is defined relative to the arbitrarily chosen independent variable, the probability distribution generated from the principle will be parameter-dependent. Thus we must take an objectivist s view and consider the generated distribution as simply a null hypothesis to be tested by the empirical data. If the generated distribution agrees with the empirical one, we interpret it as implying that the parameter chosen is the one which is measured to equal accuracy throughout its range. It is interesting to point out that although Gull & Daniell (1978) chose Shannon s definition for entropy, their constraint that the data have Gaussian errors is equivalent to the assignment of a prior distribution. Maximum entropy principle and earthquake frequency-magnitude relations In applying the maximum entropy principle to the inference of earthquake frequencymagnitude relations, Berrill & Davis (1980) chose the surface wave magnitude M as the independent parameter. Using Shannon s definition for entropy and assuming the existence of a mean magnitude ii?, they obtained the Gutenberg-Richter (1944) relation (7). This, however, does not constitute a confirmation of the Gutenberg-Richter relation. Instead, the result simply implies that the surface wave magnitude is a parameter measured to equal accuracy throughout its range. Suppose, instead of the surface wave magnitude, we have chosen to use the seismic moment Mo as the independent variable, and we assume that a mean seismic moment exists. Using Shannon s (1948) entropy would then lead to the following exponential distribution: In N(Mo) =a - b ~, (11) provided that the seismic moment is not bounded from above. Clearly, this is incompatible with the Gutenberg-Richter relation (7) because Mo and M are not linearly related. The seismic moment Mo for up to moderately high magnitude earthquakes is considered to be given by (e.g. Wyss & Brune 1968; Thatcher & Hanks 1973; Kanamori & Anderson 1975; Kanamori 1977) lnmo = a M + /3, (12)

6 782 P. Y. Shen and L. Mansinha where cr and /3 are constants (subject to regional variations). Substituting (12) into (7), we get the correct frequency-seismic moment relation (Molnar 1979): 1nN(M0)=aG --bg cr /3 bg cr The distribution (1 1) fails because in its derivation, the information expressed by equation (12) has not been incorporated. To generate (13) from the principle of maximum entropy with Mo as the independent variable, we must use Jaynes entropy and assign to Mo a prior distribution of Also, we must impose the constraint that a mean magnitude I%? exists, instead of the constraint that a mean seismic moment exists. Expressed in terms ofmo, this is The prior distribution (14) implies that the measurement error of Mo grows linearly with Mo. This is certainly compatible with the interpretation that the surface wave magnitude M can be measured to equal accuracy throughout its range, provided that (12) holds. In fact, m (Mo) is the Jacobian of the coordinate transformation from M to Mo. The above discussion exemplifies the use of Jaynes entropy (9) when measurement accuracy of the independent variable is known. In general, however, the measurement error is not known and the principle of maximum entropy must be applied objectively, using Shannon s definition (4) for entropy. This is the case in the following discussions. In generating Gutenberg-Richter relation from the principle of maximum entropy, we have imposed a lower bound on the magnitude (M 2 0), as well as the existence of a mean. The mean magnitude His the first-order moment of the random variable M. But it is not the only moment that can be assigned. In fact, if the magnitude M is not considered to be bounded from below, we must assign moments in such a way that the highest moment assigned is of even order. Otherwise, the upper bound of entropy would not be attainable. If we choose to assign moments of up to second-order, the probability density generated would be of the form p(m)aexp (an+bnm+cnm2), CN<O. (16) The seismicity data, however, are available for only a finite range of the magnitude scale. It is unreasonable to extrapolate beyond this range and consequently, moments can be assigned with any arbitrary highest order. In general, if k is the highest order, the probability density generated by the maximum entropy principle will be the exponential of a k-degree polynomial. The hypothesis that k is the highest order can be tested within the wider hypothesis that k t I is the highest order by means of the likelihood-ratio criterion (Good 1963). Note that if any of the moments is unspecified, the corresponding term in the polynomial will be missing. This results can be extended to the case where generalized moments gi as defined in equations (3) or (6) are assigned. The probability density then becomes the exponential of a generalized polynomial:

7 Maximum entropy and the frequency-magnitude relation 783 In this way, any probability distribution can be generated, provided that the generalized moments exist and, always, that the distribution conforms with the empirical data. We have already seen the assignment of a generalized moment in equation (15). Now consider the cumulative frequency-magnitude relation of Utsu (1971) and Purcaru (1975): In N(M) = au - bu M + K In (cu - M), M < cu (1 8) where K = 1 in Utsu (1971). The probability density is proportional to + (K - 1) In (cu -M)] Thus (1 8) can be generated by assigning the first-order moment (the mean magnitude) and two generalized moments g1 and gz where and g, (M) = (K - 1) In (cu -M). Notice that unlike M defined in (1 5), the two generalized moments have no clear physical interpretations. This reff ects the purely empirical nature of (1 8). As another example, consider the normal cumulative distribution of Shlien & Toksoz (1970): 1nN(M)=ns +bsm+cs~, cs < 0. (1 9) The probability density is proportional to +bsm+csm2+ln Thus (1 9) can be generated by assigning the first-and second-order momentqand ageneralized moment g1 where Notice that equation (1 9) gives a normal distribution for the cumulative frequency or exceedence frequency while equation (16) expresses a normal distribution for the frequency. Equation (16) is generated from the maximum entropy principle by assigning moments which have simple physical interpretations. To get (19), on the other hand, a physically unclear moment gl must be assigned. Equations (18), (19) and some other frequency-magnitude relations, proposed to fit data from various seismic regions of the world, have been discussed and classified, among others, by Utsu (1971) and Purcaru (1973,1975). Purcaru (1973) also computed the entropy for several distributions. In his calculation, however, the probability densities were taken as proportional to the cumulative frequencies. This is true for the Gutenberg-Richter relation but incorrect for the distributions given by (18) and (19). The last example we will consider here is the generalized Gutenberg-Richter relation, proposed recently by Lomnitz-Adler & Lomnitz (1978, 1979): In N(M) = al - cl exp (al M). The probability density for this distribution has the form p (M) a exp [al t al M - CL exp (al M)]. (20) (21)

8 784 P. Y. Shen and L. Mansinha It is interesting to note that (21) is formally similar to the probability density generated from the maximum entropy principle by assigning both the mean magnitude ff and mean seismic moment ffo. The latter is given by p (M) a exp [ao + bom - co exp (OM)]. However, the adjustable constants in (2 1) are al, al and cl. On the other hand, in (22), the constant a is fixed while ao, bo and co are adjustable. In order to conform with the seismicity data and the Gutenberg-Richter relation at low to moderate magnitudes, the constant CYL in (21) must have a value of about 0.17 (Lomnitz- Adler & Lomnitz 1979). On the other hand, the constant a in (22) is precisely the same one in equation (12) which relates seismic moment to magnitude. The empirically determined value of a is about 1.5 In 10 = except for very large earthquakes (eg. Kanamori & Anderson 1975). Equation (22) is derived from the maximum entropy principle under physically reasonable constraints (the existence of mean magnitude and mean seismic moment). The foundation of (20), on the other hand, has been questioned by several researchers (see Lomnitz- Adler & Lomnitz 1978; Jones, Mansinha & Shen 1982). Conclusions The principle of maximum entropy can be interpreted subjectively as a heuristic principle for encoding knowledge with minimum prejudice, or objectively as a heuristic principle for generating null hypotheses. Under the subjective view, the probability distributions generated from the principle are regarded as justified independently of any empirical verification. Consequently, it is imperative for the subjectivist to have an entropy which is an absolute measure of uncertainty about the distribution and invariant under any coordinate transformation. Otherwise, the probability distribution would not be uniquely determined. Such an entropy is given by (9) where the prior probability density m (x) is found to be inversely proportional to the measurement error of the variable x. However, when the measurement error of x is not known or when x is a parameter without clear physical interpretation, the entropy cannot be uniquely defined. In this case, it is necessary to adopt Shannon s (1948) definition (equation 4) for entropy. But then the probability distribution generated from the principle of maximum entropy will depend on the choice of the independent variable and consequently must be subjected to empirical verification. A case in point is the application of maximum entropy principle in the determination of earthquake frequency-magnitude relationship. The principle of maximum entropy generates the Gutenberg-Richter relation when the surface wave magnitude M is used as the independent variable. This is a fortuitous result arising from the fact that M happens to be the parameter which is measured to equal accuracy throughout its range. We have shown that, if, for example, the seismic moment were chosen as the independent variable, a frequency-magnitude relation which is incompatible with the empirical data would have been generated. Acknowledgment This work was supported by the Natural Sciences and Engineering Research Council of Canada. References Benu, J. B. & Davis, R. O., Maximum entropy and the magnitude distribution, Bull. seism. SOC. Am., 70,

9 Maxlinum entropy and the frequency-magnitude relation 785 Cornell, C. A. & Vanmarcke, E. H., The major influences on seismic risk, Proc. 4th World Conf. Earthquake Engineering, Santiago, Chile, 1, Feller, W., An Introduction to Probability Theory and its Applications, Wiley, New York. Good, I. J., Probability and the Weighing of Evidence, Griffin, London. Good, I. J., Contribution to the discussion of a paper by C. Stein, J. R. statist. SOC. B, 24, Good, I. J., Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables, Ann. math. Statist., 34, Gull, S. F. & Daniell, G. J., Image reconstruction from incomplete and noisy data, Nature, 272, Gutenberg, B. & Richter, C. F., Frequency of earthquakes in California, Bull. seism. SOC. Am., 34, Jaynes, E. T., Information theory and statistical mechanics, Phys. Rev., 106, , and 108, Jaynes, E. T., Information theory and statistical mechanics, in Statistical Physics, pp , ed. Ford, K. W., W. A. Benjamin, New York. Jaynes, E. T., Prior probabilities, IEEE Trans. Systems Science and Cybernetics, SSC-4, Jeffreys, H., Theory of Probability, Oxford University Press, London. Jeffreys, H., Scientific Inference, Cambridge University Press, London. Jones, I. F., Mansinha, L. & Shen, P. Y., On the double exponential frequency-magnitude relation of earthquakes, Bull. seism. SOC. Am., 72, Kanamori, H., The energy release in great earthquakes, J. geophys. Res., 82, Kanamori, H. & Anderson, D. L., Theoretical basis of some empirical relations in seismology, Bull. seism. SOC. Am., 65, Knopoff, L. & Kagan, Y., Analysis of the theory of extremes as applied to earthquake problems, J. geophys. Res., 82, Kullback, S., Information Theory and Statistics, Wiley, New York. Kulback, S. & Leibler, R. A., On information and sufficiency, Ann. math. Statist., 22, Lindley, D. V., The use of prior probability distributions in statistical inference and decision, Proc. 4th Berkeley Symp. Math. Statkt. Bob., 1, University of California Press. Lomnitz-Adler, J. & Lomnitz, C., A new magnitude-frequency relation, Tectonophys., 49, Lomnitz- Adler, J. & Lomnitz, C., A modified form of thegutenberg-richter magnitude-frequency relation,bull. seism. SOC. Am., 69, Molnar, P., Earthquake recurrence intervals and plate tectonics, Bull. seism. SOC. Am., 69, Purcaru, G., The informational energy and entropy in earthquake statistics and prediction of earthquakes, Riv. ital. Geofi., 22, Purcaru, G., A new magnitude-frequency relation for earthquakes and a classification of relation types, Geophys. J. R. astr. SOC., 42, Rietsch, E., The maximum entropy approach to inverse probiems,j. Geophys., 42, Riznichenko, Yu. V., Cumulative method of earthquakes for the study of the seismic activity, Izu. Akad. Nauk. SSSR. Ser. Geophys., 7, Rubincam, D. P., Information theory and the earth s density distribution, NASAIGSFC Tech. Memo Rubincam, D. P., Information theory lateral density distribution for earth inferred from global gravity field, NASAIGSFC Tech. Memo Shannon, C. E., A mathematical theory of communication, Bell Syst. tech. J., 27, and reprinted in The Mathematical Theory of Communication by C. E. Shannon and W. W. Weaver, University of Illinois Press, Urbana, Shlien, S. & Toksoz, M. N., Frequency-magnitude statistics of earthquake occurrences, Earthq. Notes, 41,5-18. Thatcher, W. & Hanks, T. C., Source parameters of Southern California earthquakes, J. geophys. Res., 78, Utsu, T., Aftershocks and earthquake statistics (111), J. Fac. Sci., Hokkaido Univ. Ser VII, 3, Wyss, M. & Brune, J. N., Seismic moment, stress and source dimensions for earthquakes in the California-Nevada region, J. geophys. Res., 73,