Large Devatons and Multnomal Probt Choce Emn Dokumacı and Wllam H. Sandholm July 4, 2011 Abstract We consder a dscrete choce model n whch the payoffs to each of an agent s n actons are subjected to the average of m..d. shocks, and use tools from large devatons theory to characterze the rate of decay of the probablty of choosng a gven suboptmal acton as m approaches nfnty. Our model ncludes the multnomal probt model of Myatt and Wallace (2003) as a specal case. We show that ther formula descrbng the rates of decay of choce probabltes s ncorrect, provde the correct formula, and use our large devatons analyss to provde ntuton for the dfference between the two. 1. Introducton In a paper n ths journal, Myatt and Wallace (2003) consder a model of stochastc evoluton based on the multnomal probt model. Agents n ther model optmze after ther payoffs are subjected to..d. normal shocks, and ther analyss focuses on the agents long run behavor as the varance of the shocks s taken to zero. Compared to other models of choce used n stochastc evolutonary game theory, the multnomal probt model ntroduces a novel feature: the rate of decay n the probablty of choosng a suboptmal strategy s nether ndependent of payoffs, as n the mutaton models of Kandor et al. (1993) and Young (1993), nor dependent only on the gap between ts payoff and the optmal strategy s payoff, as n the logt model of Blume (1993), but can depend on the gaps between ts payoff and those of all better performng strateges. 1 The foundaton An early verson of the analyss presented here was frst crculated n a workng paper enttled Stochastc Evoluton wth Perturbed Payoffs and Rapd Play. We thank Olv Mangasaran, Davd Myatt, and Chrs Wallace for helpful comments. Fnancal support under NSF Grants SES-0617753 and SES-0851580 s gratefully acknowledged. Wsconsn Center for Educaton Research, Unversty of Wsconsn, 1025 West Johnson Street, Madson, WI 53706, USA. e-mal: edokumac@wsc.edu; webste: http://www.ssc.wsc.edu/ edokumac. Department of Economcs, Unversty of Wsconsn, 1180 Observatory Drve, Madson, WI 53706, USA. e-mal: whs@ssc.wsc.edu; webste: http://www.ssc.wsc.edu/ whs. 1 Related models also appear n unpublshed work of U (1998).
of the analyss n Myatt and Wallace (2003) (henceforth MW) s ther Proposton 1, whch characterzes the rates of decay of multnomal probt choce probabltes as the shock varance approaches zero. Ther characterzaton s based on a drect evaluaton of the lmt of the relevant multple ntegral. In ths note, we ntroduce a model of choce n whch the payoffs to each of an agent s n actons are subject to the average of m..d. shocks. One can nterpret ths average as representng the net effect of many small payoff dsturbances. Our model comes equpped wth a natural parameterzaton of the small nose lmt: as the number of shocks grows large, the probablty of a suboptmal choce approaches zero. Usng technques from large devatons theory, we derve basc monotoncty and convexty propertes of the rates of decay of choce probabltes, and we obtan a smple characterzaton of the rates themselves. Snce the average of ndependent normal random varables s tself normally dstrbuted, MW s model of choce can be obtaned as a specal case of ours. Our analyss reveals that MW s formula for the rate of decay of multnomal probt choce probabltes s ncorrect. We derve the correct formula for the rate of decay, and we offer an ntutve explanaton for the dfference between the formulas usng the language of large devatons theory. 2. Analyss 2.1 Large Devatons and Cramér s Theorem Let {Z l } l=1 be an..d. sequence of random vectors takng values n Rn. Each random vector Z l s contnuous wth convex support, wth a moment generatng functon that exsts n a neghborhood of the orgn. Let Z m = 1 m m l=1 Z l denote the mth sample mean of the sequence {Z l }. The weak law l=1 of large numbers tells us that Z m converges n probablty to ts mean vector µ EZ l R n. We now explan how methods from large devatons theory can be used to descrbe the rate of decay of the probablty that Z m les n a gven set U R n not contanng µ. The Cramér transform of Z l, denoted R: R n [0, ], s defned by R(z) = sup λ R n (λ z Λ(λ)), where Λ(λ) = log E exp(λ Z l ). Put dfferently, R s the convex conjugate of the logarthmc moment generatng functon of Z l. It can be shown that R s a convex, lower semcontnuous, nonnegatve functon that 2
satsfes R(µ) = 0. Moreover, R s fnte, strctly convex, and contnuously dfferentable on the nteror of the support of Z l, and s nfnte outsde the support of Z l. 2 For smplcty, we henceforth assume that the components of the random vector Z l = (Z l 1,..., Zl n) are ndependent. It s easy to verfy that n ths case, the Cramér transform of Z l s R(z) = n k=1 r k (z k ), where r k : R [0, ] s the Cramér transform of component Z l k, and so satsfes r k (µ k ) = 0. Example 2.1. Suppose that Z l has a multvarate normal dstrbuton wth mean vector 0 and covarance matrx σ 2 I. Then a drect calculaton shows that the Cramér transform of component Z l s r k k(z k ) = (z k) 2, mplyng that the Cramér transform of Z l tself s R(z) = 2σ 2 n (z k ) 2 k=1. 2σ 2 Example 2.2. Suppose that the components of Z l are ndependent, each wth an exponental(λ) dstrbuton. Then r k (z k ) = λz k 1 log λz k when z k > 0 and r k (z k ) = otherwse, mplyng that R(z) = n ( ) k=1 λzk 1 log λz k when z R n ++ and that R(z) = otherwse. Cramér s Theorem states that 1 (1) lm log P( Z m U) = nf R(z) m m z U whenever U R d s a contnuty set of R, meanng that nf z nt(u) R(z) = nf z cl(u) R(z). Roughly speakng, equaton (1) says that the probablty that Z m takes a value n U s of order exp( m R(z )) (that s, that the exponental rate of decay of P( Z m U) s R(z )), where z mnmzes the rate functon R on the set U. If after a large number of trals the realzaton of Z m s n U, t s overwhelmngly lkely that ths realzaton s one that acheves as small a value of R as possble gven ths constrant; thus, the rate of decay s determned by ths smallest value. 2.2 Dscrete Choce and Unlkelhood Functons Consder an agent who must choose among a set of n actons. The payoff to acton s the sum of the fxed base payoff π and the random shock Z m, whch s tself the average of the m random varables {Z l }m. The agent chooses the acton that s optmal ex post. l=1 2 These propertes of the Cramér transform and Cramér s Theorem can be found n Secton 2.2 of Dembo and Zetoun (1998). In partcular, the fnteness, strct convexty, and smoothness of R on the nteror of ts doman follow from the assumptons that Z l has convex support and that ts moment generatng functon exsts see Exercses 2.2.24 and 2.2.39 n Dembo and Zetoun (1998). 3
The probabltes wth whch the agent chooses each acton are descrbed by the choce probablty functon C m : R n R+, n defned by n { } C m (π) = P π + Z m π j + Z m j = P ( D (π + Z m ) 0 ). j=1 In the last expresson, D R n n s the matrx 1e I, where e s the th standard bass vector and 1 the vector of ones, so that (D π) j = π π j. Defne the unlkelhood functon Υ: R n R+ n by 1 (2) Υ (π) = lm log m m Cm (π). In rough terms, equaton (2) says that C m (π) s of order exp( mυ (π)). Thus, Υ (π) s the exponental rate of decay of the choce probablty C m (π) as m grows large. By Cramér s Theorem, the unlkelhood Υ (π) can be computed as (3) Υ (π) = mn n r k (z k ) subject to D (π + z) 0. k=1 Proposton 2.3 uses (3) to derve some basc qualtatve propertes of the unlkelhood functon, and Proposton 2.4 provdes a tractable characterzaton. Proposton 2.3. () Υ (π) = 0 f and only f π + µ π j + µ j for all j. () Υ (π) s nonncreasng n π and s nondecreasng n π j for j. () Υ (π) s convex n π. Proof. Parts () and () are mmedate. Snce the objectve functon n program (3) s convex, and snce the functon defnng the program s constrants s lnear n the vector (z, π) R 2n, part () follows from Mangasaran and Rosen (1964, Lemma 1). Proposton 2.4. Suppose that C 1 (π) > 0. Then the unlkelhood functon Υ satsfes (4) Υ (π) = n r k (z k ), k=1 where (5) z j = ζ j (z ) (z + π π j ) µ j for j, 4
and where z s the unque soluton to (6) r (z ) + j r j Proof. In the Appendx. ( ζj (z )) = 0, In Proposton 2.4, the vector z represents the realzaton of the average shock vector Z m that s least unlkely among those that make acton optmal. To explan the form that z takes, t s convenent to focus on the case n whch each component Z l of the k shock vector Z l has mean µ k = 0. 3 In ths case, the proposton mples that f acton was not optmal ex ante, then the shock z must be postve, the shocks to worse-performng actons must be zero, and the shock to each better-performng acton j may be ether zero or negatve, accordng to whether or not z s large enough to compensate for the base payoff defct π j π. If t s not, the negatve payoff shock z ensures that and j j have the same ex post payoff. Fnally, the postve shock value z s chosen so that the margnal reducton n unlkelhood that would result from lowerng z s exactly offset by the margnal ncreases n unlkelhood that would result from lowerng the negatve values of z j by the same amount. 2.3 The Multnomal Probt Model Because the average of m ndependent normal random varables s tself normally dstrbuted, our dscrete choce model ncludes MW s multnomal probt model as a specal case. Indeed, because the Cramér transform for a N(0, σ 2 ) random varable s the quadratc functon r k (z k ) = (z k) 2 2σ 2, the vector z from Proposton 2.4 takes a partcularly smple form: snce r k (z k) = z k σ 2, the frst-order condton (6) requres the components of the shock vector z to have arthmetc mean zero. Ths fact and the consderatons descrbed after Proposton 2.4 lead to a smple characterzaton of the unlkelhood functon of the multnomal probt model. To present t most concsely we ntroduce a new defnton: for any set K S of cardnalty n K, we let π K = 1 n K k K π k denote the average payoff of the actons n K. 3 Ths s wthout loss of generalty, snce one can always elmnate a nonzero mean µ k by replacng component Z l k wth t wth ts demeaned verson Zl k µ k and replacng the base payoff π k wth π k + µ k. 5
Proposton 2.5. Suppose that each random vector Z l N(0, σ 2 I) s multvarate normal wth..d. components, so that the random vector Z m N(0, σ2 I) s multvarate normal wth..d. components as well. Then the unlkelhood functon Υ s gven m by (7) (8) n (z k Υ (π) = )2 2σ, where 2 k=1 z π J {} π j f j J {}, j = 0 otherwse, wth the set J S {} beng unquely determned by the requrement that (9) j J f and only f π j > π J {}. Thus J s the set of actons wth z j < 0. Proof. In the Appendx. MW analyze the rates of decay of choce probabltes n the multnomal probt model by drectly evaluatng the lmt of the relevant multple ntegral. Ther Proposton 1 states that these rates take the form descrbed n equatons (7) and (8) above, but wth the set J {} beng replaced by the set of all actons whose base payoffs are at least π. In contrast, Proposton 2.5 requres J to contan only those actons whose payoffs are suffcently larger than π to make a postve contrbuton to the average payoff of actons n J {}. Among other thngs, ths ensures that equaton (8) does not assgn any acton other than a postve payoff shock. 4 We llustrate these ponts through a smple example. Example 2.6. Let n = 3, suppose that payoff shocks are..d. standard normal, and consder a base payoff vector of π = (π 1, π 2, π 3 ) = (0, b, c) wth b > 0. If c 0, so that only acton 2 s base payoff s hgher than acton 1 s, then both MW s Proposton 1 and our Proposton 2.5 specfy the unlkelhood of choosng acton 1 as Υ 1 (π) = b2, obtaned from shock vector 4 z = ( b, b, 0). Our large devatons analyss shows that the least unlkely way to satsfy 2 2 the nequalty Z m Z m b s to have the shocks to actons 1 and 2 share the burden 1 2 equally. Now suppose nstead that c > 0, so that actons 2 and 3 both have hgher base payoffs than acton 1. In ths case, MW s Proposton 1 suggests that the unlkelhood of choosng 4 The error n MW s analyss seems to occur on p. 297. It s clamed that the ntegral n equaton (A.2) vanshes because ts ntegrand vanshes, but the requrements of the domnated convergence theorem are not verfed. 6
acton 1 s Υ 1 (π) = 1 3 (b2 bc + c 2 ), obtaned from shock vector z = ( b+c, c 2b, b 2c ). But f 3 3 3 c < b, then the base payoff defct of acton 1 relatve to acton 3, π 2 1 π 3 = c, s already fully addressed by the postve shock to acton 1, z 1 = b+c > c. Indeed, the shock to acton 3 3 specfed above, z 3 = b 2c, s postve, whch can only be counterproductve. 3 In fact, when c < b, Proposton 2.5 tells us that the optmal choce of z s stll z = 2 ( b, b, 0), for an unlkelhood of Υ 2 2 1(π) = b2. More generally, Proposton 2.5 shows that 4 when b and c are postve, ( b z, b, 0) f c < b, b 2 f c < b 2 2 2 = ( b+c, c 2b, b 2c) f c [ b, 2b],, 4 2 and Υ 3 3 3 2 1 (π) = b 2 bc+c 2 f c [ b, 2b], 3 2 ( c, 0, c ) f c > 2b, c 2 f c > 2b. 2 2 4 2.4 Exponentally Dstrbuted Payoff Shocks We have seen that when payoff shocks are normally dstrbuted, the components of the shock vector z have arthmetc mean equal to µ k = 0, so that the postve payoff shock to strategy s equal n absolute value to the sum of the negatve payoff shocks to strateges wth suffcently hgher base payoffs. For nstance, when n = 2 and π j > π, the z used to determne Υ (π) s gven by z = π j π = z. 2 j If nstead the payoff shocks follow an exponental(λ) dstrbuton, then the fact that ths dstrbuton has no left tal suggests that the shock vector z should take a less symmetrc form. Indeed, snce the relevant Cramér transform s r k (z k ) = λz k 1 log λz k, realzatons of Z m that are sgnfcantly above the mean shock λ 1 (whch thus have r (z ) λz 1) are far less uncommon than realzatons of Z m that are below λ 1 to a smlar extent (and j whch thus have r j (z j ) log λz j ); of course, negatve realzatons of Z m are mpossble. j Proposton 2.7 shows that the correct asymmetrc treatment of above- and belowaverage shocks can be expressed n a surprsngly smple form: wth exponental payoff shocks, the harmonc mean of the components of z, 5 H (z ) = n n, k=1 1 z k must be equated to mean payoff shock µ k = λ 1. Proposton 2.7. Suppose that components of the random vector Z l are ndependent, each wth 5 For nterpretaton, recall that f z 1, z 2,..., z n are vewed as the average speeds at whch a fxed dstance s traversed durng n dstnct journeys, then H (z) represents the overall average speed over the n journeys. 7
an exponental(λ) dstrbuton. Then the unlkelhood functon Υ s gven by Υ (π) = n k=1 ( λz k 1 log λz k ), where z j = ζ j (z ) (z + π π j ) λ 1 for j, and where z s unquely defned by the requrement that (10) H (z,..., ζ j(z ),...) = λ 1. Proof. Snce r k (z k ) = λz k 1 log λz k by Example 2.2, r (z k k) = λ 1 z k. Thus, equatons (5) and (6) mply that z satsfes nλ = n k=1 1 z. Rearrangng ths equaton and applyng k Proposton 2.4 proves the result. Propostons 2.5 and 2.7 reveal that the unlkelhood functons for the probt and exponental nose models dffer n two mportant respects. The dscusson above emphaszes the symmetry and asymmetry of the shock vectors z. It s at least as mportant that n the probt case, the Cramér transform R(z) s quadratc n z, whle n the exponental case, R(z) grows lnearly n the postve components of z. Ths dfference reflects the fact that the rght tal of the exponental dstrbuton s fatter than the tals of the normal dstrbuton. It mples that the probabltes of suboptmal choces tend to decay much more slowly n the sze of the sample when payoff shocks are exponentally dstrbuted rather than normally dstrbuted. A. Appendx Proof of Proposton 2.4. We begn wth a lemma. Lemma A.1. The optmal soluton to program (3), z R n, s the unque vector satsfyng (11) (12) n r k (z k ) = 0, k=1 π + z (π j + z j ) 0 for all j, (13) z j µ j 0 8 for all j, and
(14) ( z j µ j ) ( π + z (π j + z j )) = 0 for all j. The proof of Proposton 2.4 follows easly from ths lemma. Condtons (12) and (13) are equvalent to the requrement that z j (z + π π j ) µ j for j, and ntroducng condton (14) s equvalent to requrng the nequalty to always bnd, yeldng condton (5). Gven condton (5), equaton (11) s equvalent to equaton (6). Proof of Lemma A.1. Snce C 1 (π) > 0, program (3) admts a feasble soluton on the nteror of the support of Z l. Snce the Cramér transform of Z l s dfferentable n ths regon, we can solve program (3) usng the Kuhn-Tucker method. The Lagrangan for program (3) s L(z, ν) = n ( r k (z k ) ν j π + z (π j + z j ) ). k=1 j Snce the objectve functon s convex, z s the mnmzer f and only t satsfes the constrants (12) and there exst Lagrange multplers ν such that z and ν together satsfy (15) r (z ) = ν j, j (16) r j (z j ) = ν j for all j, (17) ν j 0 for all j, and (18) ν j ( π + z (π j + z j )) = 0 for all j. Condtons (15) and (16) together mply condton (11). Snce each r j s strctly convex on ts doman and s mnmzed at µ j, r j satsfes sgn(r j (z j)) = sgn(z j µ j ). Thus, condtons (16) and (17) mply condton (13), and condtons (16) and (18) mply condton (14). Proof of Proposton 2.5. We apply Proposton 2.4, usng the Cramér transform for N(0, σ 2 ) random varables, r k (z k ) = (z k) 2, ntroduced n Example 2.1. Evdently, equaton (4) becomes equaton (7), and 2σ 2 snce r (z k k) = z k, equaton (6) becomes σ 2 z ( + (z + π π j ) 0 ) = 0. j If we defne J = {j S: π j > π +z } and denote ths set s cardnalty by n, we can rewrte 9
the prevous equaton as and hence as (n + 1) z = (π j π ), j J z = 1 n + 1 j J {} (π j π ) = π J {} π. Thus J = {j S: π j > π J {} }, so J = J by condton (9). Equaton (5) thus becomes z j = (z + π π j ) 0 = ( π J {} π j ) 0, whch s equaton (8). Ths completes the proof of the proposton. References Blume, L. E. (1993). The statstcal mechancs of strategc nteracton. Games and Economc Behavor, 5:387 424. Dembo, A. and Zetoun, O. (1998). Large Devatons Technques and Applcatons. Sprnger, New York, second edton. Kandor, M., Malath, G. J., and Rob, R. (1993). Learnng, mutaton, and long run equlbra n games. Econometrca, 61:29 56. Mangasaran, O. L. and Rosen, J. B. (1964). Inequaltes for stochastc nonlnear programmng problems. Operatons Research, 12:143 154. Myatt, D. P. and Wallace, C. C. (2003). A multnomal probt model of stochastc evoluton. Journal of Economc Theory, 113:286 301. U, T. (1998). Robustness of stochastc stablty. Unpublshed manuscrpt, Bank of Japan. Young, H. P. (1993). The evoluton of conventons. Econometrca, 61:57 84. 10