Space, Scale, and Scaling in Entropy Maximizing

Transcription

1 Geographcal Analyss ISSN Space, Scale, and Scalng n Entropy Maxmzng Mchael Batty Centre for Advanced Spatal Analyss (CASA), Unversty College London (UCL), London, U.K. Entropy measures were frst ntroduced nto geographcal analyss durng a perod when the concept of human systems n equlbrum was n ts ascendancy. In partcular, entropy maxmzng, n drect analogy wth equlbrum statstcal mechancs, provdes a powerful framework n whch to generate locaton and nteracton models. Ths was ntroduced and popularzed by Wlson, and t led to many dfferent extensons that elaborate the framework rather than extend t to dfferent knds of models. I revew two such extensons here: how space can be ntroduced nto the formulaton through defnng a spatal entropy and how entropy can be decomposed and nested to capture spatal varaton at dfferent scales. Two obvous drectons to ths research reman mplct. Frst, the more substantve nterpretatons of the concept of entropy for dfferent shapes and szes of geographcal systems have hardly been developed. Second, an explct dynamcs assocated wth generatng probablty dstrbutons has not been attempted untl qute recently wth respect to the search for how power laws emerge as sgnatures of unversalty n complex systems. In short, the connectons between entropy maxmzng, substantve nterpretatons of entropy measures, and the longer-term dynamcs of how equlbrum dstrbutons are reached and mantaned have not been well developed. Ths lterature gap has many mplcatons for future research, and, n concluson, I sketch the need for new and dfferent entropy measures that enable us to see how equlbrum spatal dstrbutons can be generated as the outcomes of dynamc processes that converge to a steady state. Defnng and nterpretng entropy An event occurrng wth probablty p gves us a measure of nformaton about the lkelhood of that probablty beng correct. Any event wth a very low probablty that occurs gves us a great deal of nformaton, whereas when an event wth a hgh probablty occurs, ths s less of a surprse and gves us correspondngly less nformaton. Informaton thus vares nversely wth probablty, and we can defne Correspondence: Mchael Batty, Centre for Advanced Spatal Analyss (CASA), Unversty College London (UCL), 1-19 Torrngton lace, London WC1E 6BT, U.K. e-mal: m.batty@ucl.ac.uk Submtted: March 1, Revsed verson accepted: May 4, Geographcal Analyss 42 (2010) r 2010 The Oho State Unversty 395

2 Geographcal Analyss ths as 1/p. However, f we have two ndependent events wth probabltes p 1 and p 2, f one occurs and then the other occurs, we would expect the nformaton ganed to be 1/(p 1 p 2 ) because the probablty of ther jont occurrence s p 1 p 2.Yet when an event occurs, t s reasonable to suppose that the nformaton ganed should be addtonal to any nformaton already ganed, and, thus, one mght expect the nformaton for both events to be the sum of each. Clearly, ths s not 1=p 1 þ 1=p 2 6¼ 1=ðp 1 p 2 Þ but a functon FðÞ, of whch the only soluton s the qlogarthm of the nverse of the probablty, that s, F ¼F þ F >= p 1 p 2 p 1 p 2 ð1þ logðp 1 p 2 Þ¼ logðp 1 Þ logðp 2 Þ >; In short, the nformaton ganed by the occurrence of any event s logð1=pþ ¼ logðpþ, whch also can be thought of as a measure of the uncertanty of the event occurrng or as a measure of surprse (Trbus 1969). For a seres of n events, wth probabltes p ; ¼ 1; 2;...; n, the average nformaton s the expected value of ths seres, whch can be wrtten as H ¼ n ¼1 p log p ð2þ Ths measure was frst defned n ths form by Shannon (1948) when consderng the communcaton of nformaton over a nosy channel. But the formula s central to statstcal physcs, orgnatng wth Clausus n the early 19th century, and gven specfc statstcal nterpretaton by Boltzmann and then by Gbbs as the measure for thermodynamc entropy. In partcular, the method of entropy maxmzng, whch s a major theme here, was frst assocated wth fndng the dstrbuton of partcles n a physcal context, gvng rse to the Boltzmann Gbbs dstrbuton that serves as the baselne for many of the dstrbutons of spatal actvty ntroduced here (Ben-Nam 2008). When Shannon (1948) ntroduced ths measure, he sought advce as to what to call t from John von Neumann, who had worked wth a verson of the measure n quantum physcs. Although apocryphal, von Neumann 1 reportedly sad, You should call t entropy, for two reasons. In the frst place your uncertanty functon has been used n statstcal mechancs under that name, so t already has a name. In the second place, and more mportant, no one really knows what entropy really s, so n a debate you wll always have the advantage! Ths functon has many attractve propertes for descrbng spatal dstrbutons. Here, we ntally assume that the probablty p s proportonal to some count or densty of spatal actvty, such as populaton n a zone that mght be a census tract. If all the populaton were located n a mle-hgh buldng such as the one proposed for a town of 100,000 people n 1956 by Frank Lloyd Wrght (Rybczynsk 2010), then p 5 1andp k ¼ 0; 8k 6¼, and the entropy would be at a mnmum, wth H mn 5 0. If the populaton were evenly spread throughout the tracts as 396

3 Mchael Batty Space and Scale n Entropy Maxmzng p ¼ 1=n; 8, then the entropy would be at a maxmum, wth H max ¼ log n. Many dstrbutons le between these extremes, and the constructon of a varety of related measures that make comparsons wth the maxmum s possble. For example, a measure of nformaton dfference can be constructed as I ¼H max H ¼ log n þ p log p ¼ p log p ¼ p log p ð3þ 1=n q The term on the rght-hand sde (RHS) of the second lne of equaton (3) s an nformaton dfference of the knd wdely used n lkelhood theory, frst popularzed by Kullback (1959). Snckars and Webull (1977) and Webber (1979) dscussed t n a geographcal context where fq g can be nterpreted as a pror and fp g as a posteror probablty dstrbuton. The normalzaton of I as R ¼ I=H max s called relatve redundancy, whch s a measure varyng between 0 and 1. The entropy measure n equaton (2) ncreases wth the number of events or objects makng up a dstrbuton. Ths s ntutvely acceptable because as we have more events, we have more nformaton, unless the addtonal events have zero probablty of occurrence. Ths feature s easy to show because H max ¼ log n; but t also consttutes a problem for spatal analyss because t means that we cannot compare systems wth unequal numbers of objects, or, n our case, dfferent numbers of spatal subdvsons or zones. We have to normalze the quantty n some way, such as n equaton (3), and the development of spatal entropy that I present subsequently s one strategy for dong ths. Ths lack of comparablty means that methods for dervng spatal probablty dstrbutons have been much more at the fore n geographcal analyss than more substantve nterpretatons of the entropy measure. Ths focus s unfortunate because some mportant conclusons need to be drawn about the structure of dfferent spatal systems wth respect to measures of entropy. Ths s an unfnshed quest. If we consder a hypothetcal system n whch all the populaton s pled nto one zone the mle-hgh buldng example then such a system s completely ordered; t has mnmum entropy, there s no uncertanty about ts structure, and t has no varety. To make ths knd of system possble, we would need enormous constrants on ts manufacture to the pont where everythng would have to be controlled. In contrast, systems n whch the populaton s spread out evenly have maxmum entropy and maxmum dsorder and consttute the stuaton that would emerge when the system has no constrants on the system and all persons can lve where they want. Gven enough tme, people would spread out evenly n the absence of any reason for locatng n any partcular place. What s sgnfcant about ths nterpretaton s ts drect connectons to thermodynamc entropy, where maxmum dsorder occurs when all partcles mx freely, whch occurs when temperature n a system rses and any dfferences are roned out. Ths order dsorder contnuum wth respect to H s drectly nvoked f we consder that as we put more 397

4 Geographcal Analyss and more constrants on the form of a dstrbuton we successvely reduce the entropy. In ths sense, a drect lnk exsts between the probablty dstrbutons that we observe and the model and the methods of dervng such dstrbutons usng the method of entropy maxmzaton, to whch we now turn. I frst present the method, whch relates drectly to that poneered by Wlson (1970) for urban and regonal systems, although after ths presentaton, I descrbe many new nsghts that seek to show how such methods can be extended to deal wth space, scale, and scalng. The entropy-maxmzng framework The best strategy to choose a probablty dstrbuton consstent wth nformaton we know the dstrbuton must meet s to maxmze ts entropy subject to a seres of constrants that encode the relevant nformaton. When entropy s maxmzed, the dstrbuton s the most conservatve and hence the most unnformatve we can choose. Were we to choose a dstrbuton wth lower entropy, we would be assumng nformaton that we dd not have, whle a dstrbuton wth hgher entropy would volate the known constrants. Thus, ths maxmzaton s equvalent to choosng a dstrbuton that s the most lkely or probable wthn the constrants, because t s easy to show, as Wlson (1970 and n ths ssue) does, that the maxmum entropy s an approxmaton to the probablty of a partcular macrostate occurrng among all possble arrangements (or mcrostates) of the events n queston. Unlke Wlson (1970 and n ths ssue), I demonstrate the maxmzaton for a probablty dstrbuton of the locaton of populaton p n n zones, rather than the probablty p j of nteractons between zones and j, although all my dervatons are mmedately generalzable to these more detaled specfcatons. We must frst specfy the constrants, whch we take to be functons of the probabltes that defne totals, averages, or more genercally moments of a dstrbuton. To demonstrate ths dervaton, I choose two constrants for the locaton of populaton. Frst, a normalzaton constrant ensures the probabltes sum to unty: p ¼ 1 ð4þ Second, I choose a constrant for the average cost, C, of locatng n any zone, whch s the sum of the ndvdual locatonal costs c weghted by ther probabltes of occurrence: p c ¼ C ð5þ Next I form a Lagrangan L that conssts of the entropy H reduced by the nformaton encoded nto the constrants n equatons (4) and (5), and then fnd ts maxmum wth respect to the probablty p. In other words, L ¼!! p log p ðl 0 1Þ p 1 l 1 p c C ð6þ 398

5 Mchael Batty Space and Scale n Entropy Maxmzng where the parameters l 0 1and l 1 are Lagrangan multplers that ensure the maxmzaton meets these constrants. Dfferentatng equaton (6) wth respect to each probablty p and settng the result equal to zero ¼ log p l 0 l 1 c ¼ 0 Rearrangement and exponentaton of equaton (7) gves the probablty model p ¼ expð l 0 l 1 c Þ ð8þ Note that the multpler specfed as (l 0 1) enables us to get rd of the freefloatng negatve number 1 resultng from the dfferentaton n equatons (6) and (7), thus clarfyng the ensung algebra. The model n equaton (8) has some ntrgung and appealng propertes. The values of the parameters l 0 and l 1 can be determned by solvng the model accordng to the constrant equatons (4) and (5). If we substtute equaton (8) nto (4), then expð l 0 Þ becomes a partton functon defned from expðl 0 Þ¼ 9 expð l 1 c Þ; or >= " l 0 ¼ log # ð9þ expð l 1 c Þ >; The exponental model n equaton (8) can then be more clearly wrtten as p ¼ expð l 1c Þ expð l 1 c Þ ; p ¼ 1 ð10þ and from ths we see that f the Lagrangan multpler for the average cost of locaton s redundant that s, l 1 ¼ 0 then the exponental model collapses to a unform dstrbuton where p ¼ 1=n:The last step of the dervaton s to substtute the model nto the entropy equaton H; the entropy for ths model s at ts maxmum when H max ¼ p log½expð l o l 1 c ÞŠ ¼ l 0 þ l 1C ð11þ Ths maxmum s a functon of each multpler and ts constrant, wth the mplcaton that entropy s a functon of the spread of the dstrbuton, whch s determned by the cost constrant. In ths sense, entropy can be seen as a systemwde accessblty functon n that the partton and cost relate to the spread of probabltes across the system. The exact form of the relatonshp n equaton (11) requres a lttle more nsght nto the form of ts exponental functon. To ths end, we need to antcpate the next secton n movng from a dscrete to a contnuous form of model. For the exponental functon, the summatons n equatons (4) (6) and (9) (11) can be generalzed to contnuous form by assumng that p ¼ pðx ÞDx and c ¼ cðx Þ, where pðx Þ s an approxmaton of the sze of the populaton x at the pont locaton to the 399

6 Geographcal Analyss probablty densty over the nterval or area defned by Dx,andcðx Þ s an equvalent approxmaton to the cost densty n zone. We can assume that, as Dx! 0, pðx Þ!pðxÞ and cðx Þ!cðxÞ. Thus, we can wrte and smplfy constrant equaton (4) as Z 1 Z 1 pðx ÞDx ¼ pðxþdx ¼ expð l o Þ exp½ l 1 cðxþš dx lm Dx!0 0 ¼ expð l oþ ¼ 1 ð12þ l 1 whch further smplfes to expð l 0 Þ¼l 1 and l 0 ¼ log l 1 ð13þ Now the constrant on travel cost n contnuous form can be wrtten as Z 1 pðx Þcðx ÞDx ¼ pðxþcðxþdx lm Dx!0 ¼ 0 Z l 1 exp½ l 1 cðxþšcðxþdx ¼ 1 l 1 ¼ C ð14þ From the dervatons n equatons (13) and (14), the exponental model can be stated n a much smpler form, equvalent to the Boltzmann Gbbs dstrbuton n statstcal mechancs. Notng now that expð l 0 Þ¼l 1 ¼ 1= C, the model can be wrtten n ts classc form as a densty: pðxþ ¼ 1 C exp cðxþ ð15þ C where n thermodynamcs cðxþ s the energy at locaton x and C s related to the average temperature T because C ¼ kt, where k s Boltzmann s constant. Note that, as I am usng Shannon s rather than Boltzmann s entropy, the expresson for average cost s dmensonless when k 5 1, but ths does not make any sgnfcant dfference to the nterpretaton (Ben-Nam 2008). The maxmum entropy n contnuous form s not the lmt of equaton (2) wth respect to Dx as I show here. Before I do ths demonstraton, let me state ths entropy as S ¼ Z 1 0 pðxþ log pðxþdx ð16þ Then, substtutng equaton (15) nto (16), the contnuous entropy at ts maxmum has the same form as equaton (11), whch smplfes to S ¼ ¼ Z 1 0 Z 1 0 pðxþ log pðxþdx ¼ l 0 þ l 1C pðxþ log 1 C exp cðxþ C dx ¼ log C þ 1 ¼ log l 1 þ 1 ð17þ 400

7 Mchael Batty Space and Scale n Entropy Maxmzng Therefore, the approprate measurements of entropy S (and H) vary wth the log of the average cost or temperature, and the parameters l 0 and l 1 can be approxmated from ths average cost. In the sense that average cost n the system mght be nterpreted as a knd of accessblty, entropy tself can be seen as such a measure. Batty (1983), Erlander and Stewart (1990), and Roy and Thll (2004) explore related nsghts. Our last foray nto the dervaton of ths model whch I regard as a baselne for geographcal systems that must meet some conservaton constrant such as average cost nvolves sketchng how such exponental dstrbutons can emerge from a smple dynamcs that nvolves changes to the costs of locaton between dfferent places. Let us assume that a system starts wth each place havng the average cost of locaton as C,thats,c ¼ C; 8. Also assume that each place has some sort of collectve conscousness or agent that s wllng to ncrease or decrease the cost of locaton f nstructed to do so. I desgn a smulaton where, at each tme, two places and j are chosen at random and a small fxed fracton of the cost of locaton, Dc, s transferred such that the total (and average) cost of locaton remans the same. Each tme, c ðt þ 1Þ ¼c ðtþþdc and c j ðt þ 1Þ ¼c j ðtþ Dc such that Sc k ðt þ 1Þ ¼Sc k ðtþ. Let us also assume that a locaton cannot receve a negatve cost, that a lower bound exsts for c ðtþ 0; 8; t, where ths boundary condton s absolutely essental for the generaton of the stable state that ultmately emerges. If ths process contnues for many tme steps, a dstrbuton of costs (n locatons) emerges that follows the Boltzmann Gbbs dstrbuton n equatons (10) or (15) that appears when the costs are bnned and the relatve probablty dstrbuton examned. In short, through a process of random swappng akn to energy collsons n a thermodynamc system, the system self-organzes to the exponental dstrbuton from any startng pont, whch n our case s the unform dstrbuton. Ths process s robust n that many varatons of the swappng mechansm nvolvng randomness lead to the unversal form of a negatve exponental that s due to the boundary condton and the conservaton of costs. Strctly speakng, ths process s best consdered as one where each locaton s an ndvdual engagng n the process wth the resultng probablty dstrbuton formed by collectng each of these ndvduals nto locatons. Drãgulescu and Yakovenko (2000) show many varants of the model that lead to the same ultmate form wth respect to a smple economc system where ndvduals engage n swaps nvolvng a conserved quantty such as money. They also generalze the model by relaxng the boundary constrants and embed t n a wder context where wealth that s not conserved s consdered, makng the pont that these varants also admt the generaton of other dstrbutons such as the log normal and the power law. Ths knd of model has not been explored n geographcal analyss htherto for there has been no consderaton of the dynamcs that lead to entropy maxmzng. The dynamcs that have been explored s one n whch the entropy-maxmzng soluton s embedded n a wder nonlnear dynamcs (Wlson n ths ssue). Ths dscusson ntroduces the possblty of dsaggregatng the entropy-maxmzng model to the pont where ndvduals or agents 401

8 Geographcal Analyss are the basc objects consttutng a system, thus openng the framework to much more general types and styles of smulaton such as agent-based modelng. Spatal entropy: the contnuous formulaton So far, apart from my bref dgresson n the precedng secton nto contnuous entropy, I make no formal dstncton between densty and dstrbuton. I assume mplctly that dstrbuton and densty covary, whch would be the case where each nterval Dx ¼ Dx; 8, that s, each nterval s the same sze as, for example, n a spatal system arranged on a regular grd. Many spatal models gnore the sze of the nterval completely, and operatonal models that buld on entropy maxmzng rarely factor nternal sze nto ther smulatons, whch nevtably leads to based applcatons. Yet I can easly show how nterval sze must enter an analyss explctly. As before, I frst defne each element of the probablty dstrbuton p that s the product of an approxmaton to the densty p(x ) of populaton sze x at locaton and the nterval sze Dx ; p ¼ pðx ÞDx ð18þ from whch densty s defned as pðx Þ¼ p Dx ð19þ Usng equaton (18) n the entropy H, equaton (2) can be rewrtten as H ¼ pðx ÞDx log½pðx ÞDx Š ¼ pðx Þ log½pðx ÞŠDx pðx Þ½log Dx ŠDx ð20þ When we pass to the lmt, lm Dx! 0, equaton (20) can be wrtten as lmfdx! 0g H ¼ Z 1 0 pðxþ log pðxþdx Z 1 0 pðxþ log dx ð21þ where the frst term on the RHS of equaton (21) s the contnuous Shannon entropy defned as S n equaton (17). Equaton (21) mples that H!1, as lm Dx! 0, whch s another way of sayng what I have already sad n the prevous secton, namely, f Dx ¼ =n; 8, then H log n, and ths goes to nfnty n an equvalent way. The key to augmentng the entropy-maxmzng method s to use a dscrete approxmaton to the contnuous entropy S. Usng equaton (19) n the approxmaton to S, whch s the frst term on the RHS of the second lne of equaton (20), gves H S ¼ p log p ð22þ Dx whch I defne as spatal entropy (Batty 1974; Goldman 1968). Usng equaton (22) nstead of equaton (2) n the entropy-maxmzng scheme, whch nvolves 402

9 Mchael Batty Space and Scale n Entropy Maxmzng mnmzng the Lagrangan n equaton (6) wth p =Dx for p n equaton (7), leads to the augmented Boltzmann Gbbs exponental model, the equvalent of equaton (10): p ¼ Dx expð lc Þ ð23þ Dx expð lc Þ Equaton (23) can be nterpreted as a model n whch the nterval sze has been ntroduced as a weght on the probablty and s consstent wth the contnuous verson of the Boltzmann Gbbs model when passng to the lmt Dx! 0. However, another nterpretaton exsts for ths augmented model. If we wrte the entropy H S n the expanded form of equaton (22) as H S ¼ p logp þ ¼ H þ p log Dx p logdx ð24þ then we can consder the second term on the RHS of equaton (24) the expected value of the logarthm of the nterval szes as a constrant on the dscrete entropy H. Ths s a very specfc constrant n equaton (24) n that t s smply a drect augmentaton to the dscrete entropy. Instead, we set ths as a freely varyng constrant on the dscrete entropy n the form p log Dx ¼ log ð25þ and ntroduce ths nto the Lagrangan n equaton (6), whch we now wrte as L ¼!! p log p ðl 0 1Þ p 1 l 1 p c C ð26þ l 2 p log Dx log The model that we derve from ths mnmzaton can be wrtten as p ¼ expð l 0 l 1 c l 2 log Dx Þ ð27þ whch n a more famlar form can be wrtten as p ¼ ðdx Þ l 2 expð lc Þ ðdx Þ l 2 expð lc Þ ð28þ Thus, the nterval or zone sze enters the model as a scalng factor, a knd of beneft rather than cost, n the same way such factors are ntroduced by Wlson (1970) n hs famly of spatal nteracton models. By comparng equatons (23) and (28), f the multpler l 2 s forced to be unty, then the constrant on nterval sze enters the model n exactly the same way t would f t were ncorporated nto the 403

10 Geographcal Analyss entropy n the frst place, that s, as a maxmzaton of spatal rather than dscrete entropy. Note also that, n entropy-maxmzed equatons lke (28), the sgn of the multplers s undetermned untl they are ftted to meet the constrant equatons. One further pont about ths augmented maxmzaton s that f constrant equatons n the Lagrangan or augmentatons to the entropy are of logarthmc form the relevant varables enter a model as power laws: they are scalng, and any contnuous verson of the dervaton has to be modfed to ensure that these constrants le wthn defned lmts. I return to ths pont subsequently when dealng more formally wth scalng. The standard example that Wlson (1970) uses to demonstrate the logc of entropy maxmzaton s for trp dstrbuton or spatal nteracton where the entropy s based on the probablty p j that a person makes a trp T j from an orgn zone such as a workplace to a destnaton zone j such as a resdence. An example of the unconstraned model that s subject to an equvalent cost and normalzaton constrant s derved by maxmzng H ¼ p j log p j ð29þ j subject to the followng constrants: p j ¼ 1and j p j c j ¼ C j ð30þ where c j s the cost of nteracton between zones and j, and the model s derved as p j ¼ expð l 1c j Þ expð l 1 c j Þ j ð31þ The densty equvalent s based on normalzng the probablty wth respect to the sze of the zones at each orgn and destnaton Dx and Dx j, respectvely. Followng through the same logc used to derve equaton (23) for the one-dmensonal case and usng the approprate spatal entropy wth respect to p j =ðdx Dx j Þ,we generate the equvalent nteracton model as p j ¼ Dx Dx j expð l 1 c j Þ Dx Dx j expð l 1 c j Þ j ð32þ Note that all the same conclusons about the measure of entropy and the way the model can be smplfed, as developed for the locaton model, follow for the nteracton model n equaton (32). If Dx Dx j ¼ DxDx, the model collapses to the dstrbutonal form n equaton (31), whle f l 1 ¼ 0, the model collapses to the unform dstrbuton, weghted accordng to the nterval sze for the dstrbutonal form. The way n whch attractors or benefts can be ntroduced ether as augmented 404

11 Mchael Batty Space and Scale n Entropy Maxmzng measures to the entropy or as constrants also follows, and n ths sense; equatons (23) and (32) are generc forms. Before movng on to questons of scale and aggregaton, I reterate my earler defnton of nformaton dfferences wth respect to entropy. Statstcal nformaton s defned as the dfference between two dstrbutons fp g and fq g, where fq g often s referred to as the pror and fp g, the posteror. Kullback (1959) and, n a geographcal context, Snckars and Webull (1977) and Webber (1979), among others, defne nformaton I as I ¼ p log p q ; p ¼ q ¼ 1 ð33þ I vares between zero and nfnty, zero beng the measure when p ¼ q ; 8, that s, no dfference exsts between pror and posteror dstrbutons; n short, no nformaton s ganed by movng from the pror to the posteror. If we assume that the pror probablty dstrbuton s proportonal to the nterval sze that s, q ¼ Dx ¼ Dx Dx ð34þ where s the area of the entre system then the nformaton n equaton (33) becomes I ¼ p p log ¼ log þ p log p Dx = Dx ð35þ ¼ log H S When Dx ¼ Dx; 8, equaton (35) collapses to equaton (3), whch s repeated here as I ¼ H max H ¼ log n þ p log p ð36þ Many such manpulatons of entropy and nformaton exst that all gve oblque nsghts nto the measure and the shape of the relevant dstrbutons, some of whch recur n the subsequent dscusson. To conclude ths secton, one noteworthy concern s how we mght proceed to develop substantve nterpretatons of the varous entropy measures as derved so far n ths artcle, whch does not broach any emprcal applcatons. Nevertheless, although these measures have rarely been used other than for dervaton of model structures usng entropy maxmzng, obvous and straghtforward applcatons exst n whch ther actual values lead to nterestng and nformatve nsghts nto the structure of spatal systems. The thermodynamc relatons n whch entropy s the dfference between free-energy and fxed-energy use, where free energy also can be thought of as the dfference between fxed energy and entropy, can generate many substantve nterpretatons of the extent to whch spatal structures are constraned 405

12 Geographcal Analyss by known energy use. In terms of models that are derved usng entropy maxmzaton, ther parameters and constrants can be nterpreted as a functon of ther energes (Morphet 2010). Although entropy can be nterpreted as a measure of spread or dsperson n a spatal system that tes t qute strongly to ts thermodynamc nterpretaton, ts real value s n llustratng dfferences between spatal systems, partcularly where the energy constrant n a gven system changes over tme. These energy and entropy dfferences are what are mportant, because they are ted qute strongly to measures of dfference between accessblty and utlty, and to consumer surplus n transport evaluaton (Batty 2010). It s not possble to develop these deas further here, but suffce t to say an entrely new research agenda can be formulated wth respect to the substantve meanng of entropy and related energy measures that te these quanttes back to ther more fundamental thermodynamc orgns. Consequently, far from beng of manly hstorcal nterest n spatal analyss, entropy maxmzng stll has enormous potental for generatng new nsghts nto the structure and functonng of spatal systems, whch I llustrate by dervng models that pertan to scalng that are also central to new developments n complexty theory (Batty 2009). Scale and entropy: aggregaton and constrants Shannon s entropy n equaton (2) has an exceptonally easy-to-manpulate loglnear structure and addtve form that allows t to be aggregated wth respect to groups of objects that mght pertan to some hgher level of organzaton n the system of nterest. Thel (1972) refers to ths process of aggregaton as the entropydecomposton theorem and, to llustrate t, I frst dvde the set Z of n objects, n ths case the spatal zones of the geographcal system, nto K sets, Z k ; k ¼ 1; 2; :::; K, each wth n k objects such that Sn k ¼ n. The sets are mutually exclusve and collectvely exhaustve n that Z ¼ [ K Z k and f ¼ \ K ð37þ k¼1 Z k k¼1 where f s the empty set. Note now that each probablty p 2 Z k s defned so that k ¼ p and k ¼ p ¼ 1 ð38þ 2Z k k k 2Z k Substtutng these defntons nto equatons (37) and (38), we can wrte the dscrete entropy n equaton (2) as H ¼ k log k p k log p k 2Z k k k ¼ H B þ ð39þ k H k k where H B s the between-set entropy at the hgher system level, and the second term on the RHS of the second lne of equaton (39) s the sum of the wthn-set 406

13 Mchael Batty Space and Scale n Entropy Maxmzng entropes H k weghted by ther probablty of occurrence k at the hgher level. As the sets Z k get fewer and progressvely larger from the orgnal set Z whch s tantamount to dsaggregaton of the entre set nto smaller and smaller sets the wthn-set entropes decrease n sum and the between-set entropy H E rses n value untl all that remans s one aggregated set for each object, that s, H B! H. Movng the other way, when all the objects are aggregated nto one set, then H B! 0, and S k H k! H. roofs of these assertons are gven n Thel (1972) and Webber (1979). The equvalent decomposton formula for spatal entropy as we have defned t n equaton (22) can be stated. Then, notng that k ¼ 2Z k Dx ð40þ where k s the sum of the ntervals (areas) n each aggregated set Z k, spatal entropy can be decomposed as H S ¼ k log k p k log p Dx k k k 2Z k k k k ¼ H SB þ ð41þ k H Sk k where H SB s the between-set spatal entropy, and S k H Sk s the sum of the weghted wthn-set spatal entropes. An nformaton dfference structure s bured n equaton (41), as spelled out earler for spatal entropy between equatons (33) and (36), and smlar nterpretatons apply. In developng decompostons of entropy and spatal entropy n ths fashon, the focus s on explanng the varaton n entropy at dfferent spatal scales, notng that entropes can be nested nto a herarchy of levels, that s, the between-set entropes can be further subdvded nto sets that are smaller than Z k but larger than the basc sets for each object or zone Z. These deas have been used to redstrct zones to ensure equal populatons n the case of the dscrete entropy and equal populaton denstes n the case of spatal entropy n an effort to desgn spatal systems that meet some crtera of optmalty that pertan to scale and sze (see Batty 1974, 1976). In ths artcle, I do not deal wth the effect of shape on entropy, but extensons exst to deal wth dealzed spatal systems that also ncorporate constrants on shape, such as the regularty of boundares, although developments n ths area have been lmted (Batty 1974). These decomposed entropy measures can be used n entropy maxmzaton to enable models to be derved that are constraned n dfferent ways at dfferent system levels. Let us assume that the cost constrant on probabltes pertans to the entre system, as n equaton (5), but that entropy needs to be maxmzed so that the aggregate probabltes sum to those that are fxed by the level of decomposton or aggregaton chosen, as n equaton (38). I set up the Lagrangan to maxmze equaton (39) wth respect to equatons (38) and (5) 407

14 Geographcal Analyss as follows: L ¼ k log k k k l 1 p c C k 2Z k p log p k k k ðl k 0 1Þ 2Z k p k! ð42þ and then mnmze ¼ log p l k 0 l 1c ¼ 0; 2 Z k ð43þ to derve the model that we can state as p ¼ expð l k 0 l 1c Þ; 2 Z k ð44þ We can compute the partton functon drectly by substtutng for p n equaton (38), yeldng 9 expð l k 0 Þ¼ k expð l 1 c Þ or >= 2Z k ð45þ expð l 1 c Þ l k 0 ¼ log 2Z k >; from whch the relevant exponental model n equaton (44) can be more clearly wrtten as p ¼ k k expð l 1 c Þ expð l 1 c Þ ; 2 Z k and p ¼ k 2Z 2Z k k ð46þ Note that the constrant equaton on cost s for the entre system and, as such, effectvely couples the varous models for each subset n terms of ther calbraton but not n terms of ther operaton. We need to be careful about the way these models are coupled because f no system-wde constrants exst, then the models are separable; the entropy maxmzng s separable nto K subproblems. For example, assume that the cost constrant n equaton (5) s replaced wth cost constrants that pertan to the subsets wrtten as p c ¼ C k ; 8k ð47þ 2Z k k Then, from equaton (47), that the system-wde constrant s also met as p k c ¼ kck ¼ p c ¼ C; 8k k 2Z k k k k 2Z k ð48þ 408

15 Mchael Batty Space and Scale n Entropy Maxmzng If we substtute equaton (48) n (42), notng that now we have K multplers l k 1, then the derved model has the same structure as equaton (44) but now can be wrtten, followng equaton (46), as p ¼ k expð l k 1 c Þ 2Z k expð l k 1 c Þ ; 2 Z k ð49þ Ths model s not only separable for each subset Z k, but each model also s calbrated separately wth respect to the cost constrant and determnaton of the set of multplers fl k 1g. Usng spatal entropy maxmzng adds lttle to ths logc other than ensurng that the nterval or area for each zone appears n the exponental equaton. If we follow the same process, the equvalent model to that n equaton (49) can be wrtten as p ¼ k Dx expð l k 1 c Þ 2Z k Dx expð l k 1 c Þ ; 2 Z k ð50þ where f the system-wde cost constrant n equaton (5) apples, then the only dfference s that there s one multpler, l 1,notK. To provde some sense of closure to ths argument, readers are referred to Thel (1972), who provdes many applcatons of these knds of decomposton to the measurement of varance and dfference at dfferent levels of dsaggregaton for both spatal and nonspatal systems, connectng these deas to a much wder lterature about the measurement of nequalty. Generatng spatal probablty dstrbutons So far I have defned both entropy and ts method of maxmzaton wth respect to probabltes that pertan to spatal locatons. In terms of the typcal problem, there s the assumpton that the probablty of locaton s some functon a negatve exponental n the classc Boltzmann Gbbs case of some sze varable such as cost. Implctly, n ths case, the probablty of locaton mght be proportonal to the observed populaton n any zone, and a sensble assumpton s that a hgher probablty of locaton measured by a hgher populaton s assocated wth a lower cost (or hgher beneft) of locatng n the place n queston. However, another nterpretaton exsts that s less specfc about the knds of probablty dstrbutons that emerge from entropy maxmzng and depends on how one sets up a problem. In ths secton and n the rest of ths artcle, we can assume that some measure of sze, not cost, s what a probablty dstrbuton must conserve, and that probabltes vary wth respect to ths sze varable. In short, rather than thnkng of the spatal locaton problem as one n whch the probablty of populaton locaton s related to some sze or cost, we now develop a model n whch the probablty of locaton s dependent on the actual populaton sze that s observed n the locatons n queston. Ths s the obvous way to develop entropy maxmzng for cty sze dstrbutons, a topc that has remaned qute confused snce Berry (1964) and Curry 409

16 Geographcal Analyss (1964) frst speculated about these questons over 40 years ago. Ths s also the route by whch we can connect the arguments of ths artcle to sze dstrbutons n general and to power laws n partcular. To extend entropy maxmzng n ths way, I replace the probablty p of each event wth ts frequency f ðþ. I defne a functon of the sze of the event V, whch n many of these cases lterally s the populaton sze, although t could be defned as any related measure. Then I derve the approprate dscrete probablty frequency for f(v ) by maxmzng ts entropy H defned n analogy to equaton (2) as H ¼ f ðv Þ log f ðv Þ ð51þ Ths expresson s subject to the usual normalzaton and constrants assocated wth the moments of the dstrbuton that are defned as f ðv Þ¼1; f ðv ÞV ¼ V; f ðv Þ V 2 V ¼ s 2 ; and so on ð52þ where s 2 s the varance of the dstrbuton. Ths dscusson and notaton follows Trbus (1969), although several other presentatons of ths process have more formal roots n probablty theory. A good contemporary start for these more formal presentatons between entropy, scale, and scalng can be found n the books by Sornette (2006) and Sachev, Malevergne, and Sornette (2010). The Boltzmann Gbbs negatve exponental model s stll the baselne n entropy maxmzaton because t ntroduces a constrant on the dstrbuton that s the frst moment, the average, and no others apart from the normalzaton of the probabltes. Followng the same logc used earler n equatons (4) (10) and assumng the ntervals over whch the dscrete frequency s measured are equal, that s, Dx ¼ Dx; 8 (to avod any confuson wth spatal entropy at ths stage), we maxmze equaton (51) subject to the frst two constrants shown n (52). Usng the relevant Lagrangan wth approprate multplers yelds log f ðv Þ¼ l 0 l 1 V ð53þ Ths has the classc log-lnear form that generates the Boltzmann Gbbs probablty frequency f ðv Þ¼expð l 0 l 1 V Þ ð54þ whch gves the famlar exponental form f ðv Þ¼ expð l 1V Þ expð l 1 V Þ ð55þ Equaton (55) mples that the larger the sze, the lower the probablty, whch s the same as the prevous nterpretaton wth sze equvalent to locatonal cost. Ths relatonshp s made more graphc f we rearrange equaton (53), where sze s now 410

17 Mchael Batty Space and Scale n Entropy Maxmzng a functon of frequency, as V ¼ l 0 l 1 1 l 1 log f ðv Þ ð56þ However, f the sze V s populaton as measured n terms of the number of ndvduals lvng n zone, then we cannot equate cost wth sze n any way because larger populatons are much more lkely to lve n places where the costs of locaton are lower, all other thngs beng equal. Ths feature s the confuson that has never really been resolved n generatng sze dstrbutons wth entropy-maxmzng technques. The motvaton for the earler models, such as those developed by Wlson (1970), was always to maxmze entropy wth respect to a cost constrant, whereas for the models n ths secton, the motvaton s to maxmze entropy wth respect to a sze constrant. In ths context, a perfectly reasonable assumpton s that an ndvdual locatng across a space has many more places to locate where populatons are small than places where populatons are large. It s n ths sense that frequency n ths secton dffers from probablty n the prevous sectons, although formally the algebrac expressons are dentcal. Now we can show how the negatve exponental can become a power functon f the constrant on average sze s replaced by ts geometrc equvalent, that s, f ðv Þ log V ¼ log V ð57þ where log V s the expected value of the sum of the logarthms of the szes. The logc s that agglomeraton economes of sze or dseconomes of cost or energy are perceved logarthmcally rather than absolutely, as enshrned n the Weber Fechner law (Stevens 1957). We assume ths percepton s defned for a dscrete system because dffcultes noted below arse when we examne the rank sze rule and ts consstency wth entropy maxmzaton. The contnuous verson of the model must be nvoked for purposes of smplfcaton and demonstraton. However, f we maxmze entropy subject to equaton (57) and the normalzaton constrant, the model becomes log f ðv Þ¼ l 0 l 1 log V ð58þ whch n exponental form s f ðv Þ¼expð l 0 ÞV l 1 ð59þ Equaton (59) s a power functon that, n more famlar terms, can be wrtten as f ðv Þ¼ V l 1 V l 1 ð60þ where, from equaton (60), we can wrte the model n nverse form n analogy to equaton (56) as V ¼ exp l 0 f ðv Þ l 1 1 ð61þ l 1 411

18 Geographcal Analyss In ths context, V also vares nversely wth the power of frequency. From equaton (61), whch, n turn, s derved from the assumpton about logarthmc costs made n equaton (57), we can generate the more famlar rank sze rule that has been known for well over a century, frst exploted for ncome szes by areto (1906) and then for cty szes by Zpf (1949). I explore these functons n the next secton. In maxmzng entropy wth respect to the three constrants stated n equatons (52), one notes that the thrd constrant also can be smplfed to yeldng f ðv ÞðV VÞ 2 ¼ f ðv ÞV 2 V ¼ s 2 ð62þ log f ðv Þ¼ l 0 l 1 V l 2 V 2 ð63þ In the frst exponental form, ths s f ðv Þ¼expð l 0 l 1 V l 1 V 2 Þ ð64þ whch n more famlar terms s f ðv Þ¼ expð l 1V l 1 V 2Þ expð l 1 V l 1 V 2 Þ ð65þ As Trbus (1969) shows, equaton (65) s a form of the normal dstrbuton. The entropy-maxmzng dervaton s nterestng because t makes explct the polynomal form of the normal wth the contrbuton of the mean and the varance drectly assocated wth the multplers l 1 and l 2. The parameter l 1 s negatve, makng ths exponental postve, and l 2 s postve, meanng the varance term acts as a negatve exponental. The normalty of the dstrbuton s always preserved no matter what the value of these multplers. Moreover, f l 1 l 2, the varance of the dstrbuton becomes ncreasngly smaller, whle the skewness become ncreasngly peaked. We can complete ths set of dstrbutons by assumng that the sze dstrbuton s log-normal, that s, nstead of V, we now defne sze as ts logarthm, log V. We can formally restate the constrant equatons for the log-normal as 9 f ðv Þ¼1 >= f ðv Þ log V ¼ log V ð66þ h f ðv Þðlog V Þ 2 log VÞ ¼ s 2 >; 412

19 Mchael Batty Space and Scale n Entropy Maxmzng Maxmzng equaton (51) subject to equatons (66) gves the model n fnal form as f ðv Þ¼ expð l 1 log V l 1 ðlog V Þ 2 Þ expð l 1 log V l 1 log ðv Þ 2 Þ ðv 2Þ l 2 ¼ V l 1 V l 1 ðv 2Þ l 2 ð67þ Equaton (67) mples that, f l 1 l 2, the log-normal form collapses to the nverse power law form but only for a range of the largest values of V. Ths s one of the smplest demonstratons that power laws tend to domnate n the upper or heavy tal of the log-normal dstrbuton. Agan, the same caveats apply as for the exstence of the moments for the dscrete case, whch wll always be true for the sorts of spatal systems to whch these models apply, that s, where 1 V < 1. Trbus (1969) has a relatvely straghtforward demonstraton of the propertes of the normal dstrbuton wth respect to the values of the parameters that can be determned from an approxmaton to the contnuous probablty densty functon. Approxmatng scalng: the rank sze rule and Zpf s law The negatve exponental and power law models generated n the prevous secton usng entropy maxmzng represent dscrete densty functons relatng frequency to sze. These dstrbutons already defne the form of the populaton or cty sze dstrbutons (where spatal locatons defne the locatons of dstnct ctes). However, a more popular form, partcularly for cty szes, frm szes, ncomes, and related socal phenomena nvolves rankng these szes from the largest value of V, whch I now call rank r 1, to the smallest, r n. The rank s the countercumulatve of the frequency (Adamc 2002). If we accumulate the frequences from, let us say, some value of ¼ m < n to the largest value of 5 n, then ths accumulaton would defne the rank r n m. We can only express ths formally f we consder the contnuous approxmaton to f(v )asf(v), whch s defned when Dx! 0. Let us frst take the exponental model defned n equaton (55) n ts contnuous lmt as f ðvþ expð l 1 VÞ. The ntegraton defnng the countercumulatve F(V) s FðV Þ¼ Z 1 v f ðvþdv Z 1 where FðV Þr n m ¼ r k, 5 m, andk 5 n. Thus, v expð l 1 VÞdV ¼ 1 ½expð l 1 V ÞŠ 1 v ð68þ l 1 r k expð l 1 V k Þ ð69þ 413

20 Geographcal Analyss from whch 9 log r k l 1 V k >= V k 1 log 1 >; l 1 r k ð70þ Equatons (70) defne rank as a functon of populaton and populaton as a functon of rank, whch exposes the clear log-lnear structure of the exponental rank sze relatonshp. The classc rank sze relatonshp commonly s developed wth the relatonshp between sze and frequency expressed as a power law. The contnuous lmt based on equaton (60) can be wrtten as f ðvþ V l 1, from whch we defne the countercumulatve F(V) as Z 1 Z 1 FðVÞ ¼ f ðvþdv V l 1 dv ¼ 1 l 1 þ 1 V l1þ1 1 Þ ð71þ v v v where F(V) s the rank r k as defned for the ntegraton of the exponental followng equaton (68). Ths rank can be wrtten as from whch r k V l 1þ1 k ) log r k ð1 l 1 Þ log V k V k r 1 1 l 1 k ð72þ ð73þ Equatons (73) defne rank as a functon of populaton, and populaton as a functon of rank. Equatons (72) (73) mply that these power laws are scalng, that s, f we scale sze by a as av k, then the rank does not change, whch can be demonstrated by substtutng av k for V k n any of the precedng equatons. A power law s the only functon that has ths property; hence, ts clam as a sgnature of unversalty. Usng the logarthmc mean of the sze as the major constrant n generatng dstrbutons n the nverse power or Zpf areto form s consstent wth assumng that sze (or cost) can be vewed as a regular dstorton based on human percepton. We noted ths feature prevously as the Weber Fechner law, whch pertans to how we perceve brghtness and sound. Even the way our cogntve senses respond to sze s proportonal to the logarthm, not to the actual value, of the relevant measure of ntensty (see Stevens 1957). In spatal nteracton modelng, Wlson (1970) made use of ths property to show how the orgnal gravtatonal hypothess s consstent wth models produced by entropy maxmzng, partcularly n the context of very long dstance flows, such as those measured as commodtes n trade systems, where the percepton of travel cost s more lkely to be logarthmc than absolute. The same arguments are used to ncorporate addtonal constrants that mght be 414

21 Mchael Batty Space and Scale n Entropy Maxmzng thought of as benefts rather than costs, reflectng the fact that agglomeraton economes are sometmes perceved logarthmcally. We also can generate rank sze dstrbutons for the normal and log-normal models that we derved n equatons (65) and (67), respectvely. Although pursung ths development s not very llumnatng, the log-normal s a noteworthy specal case largely because many arguments exst suggestng that cty, frm, and ncome sze dstrbutons are not consstent wth power laws, but rather are log-normal, wth the power law only applyng as an approxmaton to these dstrbutons n ther upper tal. Wrtng equaton (67), notng the sgns of the multplers as determned by Trbus (1969), expressng the frst multpler as a and the second as b, and then passng to the lmt renders f ðvþ V a V 2b, from whch we form the countercumulatve as FðV Þ¼ Z 1 v f ðvþdv Z 1 v V a V 2b dv ¼ 1 a 2b þ 1 V a 2bþ1 1 Þ ð74þ v Equaton (74) ndcates that the shape of the log-normal s completely dependent on the value of the parameters a and b. Nevertheless, we can speculate on the shape of the functon for varous ranges of sze from these values and the sze {V }. The rank and sze relatonshps, analogous to equaton (73), can be wrtten as ) log r k ðaþ1þ log V k 2b log V k 1 ð75þ V k r ðaþ1 2bÞ k If a11 2b, then for the largest values of V k the second term n the frst lne of equaton (75) domnates, mplyng that the rank sze relaton s more lke a power law n ts upper or heavy tal. The precedng development s a somewhat nformal way of demonstratng the relatonshp between nverse power and log-normal functons, and readers are referred to more consdered sources that elaborate ths relatonshp. erlne (2005) formalzes an excellent dscusson about when one s able to approxmate the heavy tal of a log-normal wth a power law that bulds on earler expostons that are part of the lterature on skewed probablty functons, as summarzed by Montroll and Schlesnger (1982). The purpose here s not to develop a treatse about the log-normal or, ndeed, about the Zpf and areto power laws, for we see that both can be derved from entropy maxmzng. Rather, power laws can emerge from two sources: (1) drectly f the constrant on the entropy s a geometrc mean and (2) when the constrants on the entropy are those that defne the log-normal but for very large values of the sze dstrbuton where the varance of the dstrbuton s also very large, effectvely meanng that the heavy tal occurs over several orders of magntude. For emprcal applcatons to cty sze dstrbutons, readers are referred to the manstream lterature where these ssues are dscussed n great detal. The recent artcle by Eeckhout (2004) s representatve. 415

22 Geographcal Analyss One last substantve ssue requres us to complete ths presentaton about how scalng dstrbutons are assocated wth entropy maxmzng. The tradtonal explanaton of how power laws come to domnate spatal and socal dstrbutons essentally s based on a generc model that leads to agglomeraton economes, n whch any object chosen at random ncreasngly s unlkely to grow to a very large scale, realzng agglomeraton economes that are assocated wth large ctes, people wth large ncomes, the domnaton of large frms, and so on. In essence, the growth or declne n sze of any object makng up such compettve systems s based on Gbrat s (1931) law of proportonate effect, n whch any object of sze V t grows or declnes to V t11 by a random amount e t, whose value s proportonate to the sze of the object already reached, that s, V tþ1 ¼ð1 þ e t ÞV t. Ths process, f operated contnually for many tme perods, leads to a dstrbuton of objects that s log-normal. If the process s constraned so that objects do not declne n sze below a certan threshold (whch s tantamount to not lettng sze become negatve), several authors show that the resultant dstrbuton s no longer log-normal but rather s scalng n the form of an nverse power functon. These conclusons have emerged from several sources n physcs (Levy and Solomon 1996), n economcs (Sachev, Malevergne, and Sornette 2010), n earth scences (Sornette 2006), and n several other areas of socal nqury (Newman 2005). Ths dynamc, referred to by Solomon (2000) as the generalzed Lotka Volterra (GLV) model, essentally llustrates that n the steady state, power laws emerge from processes n whch there s random proportonate growth aganst a background of transtons between ndvduals or places n terms of the nterest varable, be t populaton, ncome, wealth, cost, or some other sze measure. The steady-state results generated by such models also are consstent wth Boltzmann dstrbutons, as Rchmond and Solomon (2001) show, whle Foley (1994) and then Mlakovc (2003) demonstrate that entropy maxmzng can be employed drectly wth the dynamcs beng embedded as constrant equatons that the process of wealth creaton must meet. An enormous lterature now exsts that deals wth stochastc GLV types of models, whch buld on proportonate effect, leadng to lognormal and power laws. Several oblque nterpretatons of the steady states assocated wth such processes as Boltzmann Gbbs dstrbutons exst, whch Rchmond and Solomon (2001) say are... Boltzmann laws n dsguse. The earler dynamc models developed by Drãgulescu and Yakovenko (2000) also are beng extended to deal wth systems where the constrants on dstrbutons of money, wealth, and ncome all vary wth consequent dfferences n ther dstrbutons, n turn, provdng a rch source of nterpretatons for the way nequaltes emerge n economc systems (Yakovenko and Rosser 2009). Lttle of ths dscusson has yet to fnd ts way nto spatal or geographcal systems because the concern wth cty sze dstrbutons has been remarkably aspatal, n contrast to entropy maxmzng n geographcal analyss; but sgns of a convergence are begnnng to appear. Wlson s (2008) recent work, for example, seeks to generalze entropy maxmzng n a dynamcal framework that he refers to as 416