Additive processes. Chapter 6

Transcription

1 Chapter 6 Additive processes Discoveries of heavy-tailed pheomea are quite ofte viewed with surprise, as if heavy-tailed distributios are merely a probabilistic curiosity. I large part, this is a cosequece of the promiece ad beauty of the cetral limit theorem, which says that sum of a large umber of radom variables will be approximately Normally distributed (whe ormalized appropriately). This result the leads to the view that the emergece of the Normal distributio is almost a rule of ature. Very ofte processes i ature ca be viewed as beig formed via some additive process, i.e., beig formed by the sum of may radom evets, ad so the Normal distributio becomes our expectatio about the world aroud us. Of course, the Normal distributio is promiet i our lives. May aspects of huma growth ad behavior are approximately Normally distributed heights, weights, test scores, etc. However, heavy-tailed distributios are also very promiet i the world aroud us, as we have see throughout this book. This seems to fly i the face of the explaatio for the promiece of the Normal distributio provided by the cetral limit theorem, ad so it makes you woder what is missig from this explaatio? Of course, o thig that is missig is that there are may other ways that thigs ca evolve besides additive processes, ad these could lead to other, maybe heavy-tailed, distributios. I fact, other types of processes do lead to distributios besides the Normal, ad we cosider two other geeral forms (multiplicative processes ad extremal processes) i the followig two chapters. But, eve i the case whe thigs evolve accordig to a additive process, it turs out that there is more to the story tha the cetral limit theorem. I particular, the typical statemet of the cetral limit theorem that we lear i itroductory probability courses is oly the baby versio there is a geeralized cetral limit theorem that highlights that oe should ot really expect additive processes to always yield distributios that are approximately Normal. More geerally, arbitrary stable distributios ca emerge, ad stable distributios ofte have heavy-tails, specifically regularly varyig tails. Thus, i some sese, ituitive expectatios about the promiece of the Normal distributio are skewed by oly havig a partial view of the cetral limit theorem. I this chapter, we seek to remedy this by studyig the geeralized cetral limit theorem ad the class stable distributios. Note that the typical treatmet of the geeralized cetral limit theorem ad the class of stable distributios is extremely techical, but we do our best to provide a elemetary treatmet here. Of course, this meas that we will ot prove results i full geerality. However, i Sectio (add ref), we give poiters to the iterested reader where the full details ca be foud. 67

2 6.1 The cetral limit theorem Our focus i this chapter is o additive processes, i.e., processes that grow via the sum of radom evets. I particular, our goal is to uderstad the behavior of S = X 1 + X X, where X i are i.i.d. The study of additive processes is a extremely classical ad importat area, ad two of the most celebrated results i probability provide us isight ito the behavior of S : the law of large umbers ad the cetral limit theorem. While you have certaily see both of these results before, we preset them i a slightly o-stadard form i the followig i order to highlight a perspective o these results that is ot ofte taught i itroductory probability courses. I particular, the law of large umbers ad the cetral limit theorem ca be thought of as the first ad secod order approximatios of the additive process S. To see this, let us first recall the statemet of the law of large umbers. Theorem 17 (The strog law of large umbers). Cosider a ifiite sequece of i.i.d. radom variables X 1, X 2,... havig E [X i ] = E [X] <. The, S a.s. E [X] as. Iformally, the law of large umbers says that, up to a first-order approximatio, S well-approximated by its mea, ad is thus liear i, i.e., S = E [X] + o(). This is a particularly ice approximatio because it is determiistic; all the radomess of the X i has disappeared ad the approximatio oly depeds o ad E [X]. Figure (add ref)highlights that it does ot take very large before S behaves early determiistically ad so the approximatio quickly becomes accurate. (TODO: add figure of growth of S for differet.) Give that that the first order approximatio of S from the law of large umbers is determiistic, it is atural to ask if it ca be made more precise, i.e., what is the secod order correctio term? Oe would hope to have a correctio term that captures the radomess of S, ad this is exactly the role of the cetral limit theorem. Theorem 18 (The cetral limit theorem). Cosider a ifiite sequece of i.i.d. radom variables X 1, X 2,... havig E [X i ] = E [X] < ad Var [X i ] = σ 2 <. The, 1 (S E [X]) d Z as, where Z Normal(0, σ 2 ). The form of the cetral limit theorem highlights clearly that it ca be viewed as a correctio term for the law of large umbers. Specifically, the term S E [X] is the error of the approximatio provided by the law of large umbers, ad so the cetral limit theorem highlights that this error is o the order of. Cosequetly, by combiig the cetral limit theorem ad the law of large umbers, we get a secod

3 order approximatio of the additive process S, where the secod term is of order, i.e., S = E [X] + Z + o( ), where Z Normal(0, σ 2 ). Of course, sice the cetral limit theorem does ot have a determiistic limit, the approximatio is o loger determiistic. However, this is atural (ad eve desirable) give the radomess of S. Figure (add ref)highlights this view of the cetral limit theorem as a correctio term for the approximatio of S. (TODO: add figure ad brief discussio) Sice uderstadig, ad geeralizig, the cetral limit theorem is core to our goals i this chapter, it is importat for us to cosider the proof. I fact, whe we move to the geeralized cetral limit theorem, the proof will be more complex, of course; but the structure will mimic the approach we take here to prove the stadard versio of the cetral limit theorem. Before givig the proof, recall that the characteristic fuctio of a radom variable X is defied as φ X (t) = E [ e itx]. The characteristic fuctio is a particularly useful quatity because it completely determies the distributio of the radom variable, i.e., it ca be iverted to fid the distributio fuctio; but it is ofte easier to work with aalytically. A particularly importat property that highlights the value of the characteristic fuctio is the fact that φ a1 X 1 +a 2 X a X (t) = φ X1 (a 1 t)φ X2 (a 2 t)... φ X (a t). As you ca imagie, this property makes the characteristic fuctio a particularly appealig tool for studyig additive processes. Proof of Theorem 18. To begi, defie Y i = (X i E [X]) ad deote the characteristic fuctio of the i.i.d. Y i by φ Y. Though we do ot kow much about the form of φ Y, we ca use the facts that φ Y (0) = ie [Y ] = 0 ad φ Y (0) = E [ Y 2] = Var [Y ] = σ 2 to write a Taylor expasio of φ Y as follows: φ Y (t) = 1 σ2 t o(t2 ), as t 0. Note that this represetatio explicitly depeds o the fact that σ 2 <. i=1 Next, defie Z = Y i as the ormalized additive process we are iterested i characterizig. We ca write the characteristic fuctio of Z as follows φ Z (t) = ( φ Y (t/ ) ) ( = 1 σ2 t 2 ) 2 + o(t2 /) e σ2 t 2 /2 as. This limit is exactly the characteristic fuctio of a Normal distributio with mea zero ad variace σ 2, as desired.

4 Sidebar: Heavy-tailed distributios ad the cetral limit theorem Note that the cetral limit theorem applies eve whe the X i are heavy-tailed, as log as the variace is fiite. So, for example, if oe cosiders a regularly varyig distributio with α, the variace is fiite ad so 1 (S E[X]) coverges to a Normal distributio. This seems atural, sice the cetral limit theorem is so familiar. However, upo more careful examiatio it is actually a bit surprisig. Recall that if X 1 ad X 2 are regularly varyig with idex α, the X 1 + X 2 RV(α), see Lemma 5. Thus, by iductio, ay fiite sum X X RV(α). So, for all, S = X X is heavy-tailed, yet as the cetral limit theorem gives a limitig distributio of a Normal, which is light-tailed. Eve more surprisigly, while all momets of the Normal distributio are fiite, S will have ifiite momets for all fiite i this case. This seems cotradictory at first; however, the reaso for the apparet cotradictio is simple. For ay fiite, the approximatio of S provided by the Normal distributio is oly accurate up to a particular poit i the tail, say for t < t. Sice the Normal is light-tailed, it caot approximate the full tail of a heavy-tailed distributio, but as, t, ad so covergece i distributio is achieved. (TODO: show pictures of this.) 6.2 Geeralizig the cetral limit theorem The stadard statemet of cetral limit theorem gives a covicig explaatio for the emergece of the Normal distributio i the world aroud us: if a process grows additively the it ca be approximated well via the Normal distributio. But, of course, there are coditios o whe this may happe. It is the relaxig of these coditios that highlights that the Normal is ot as special as the stadard statemet of the cetral limit theorem makes it appear. I particular, there are four key assumptios i the cetral limit theorem: (i) the X i are idetical; (ii) the X i are idepedet; (iii) the X i have fiite mea; ad (iv) the X i have fiite variace. It turs out that (i) ad (ii) are ot particularly crucial. Specifically, if the X i are ot idetical, the it is still possible to obtai versios of the cetral limit theorem where S coverges to a Normal distributio, e.g., the Lyapuov cetral limit theorem. Similarly, if the X i are depedet it is still possible to prove versios of the cetral limit theorem where S coverges to a Normal distributio, as log as the depedece is weak, e.g., the martigale cetral limit theorem. Of course, if the depedece is extreme, the it ca be costructed to create a arbitrary limitig distributio for S, but such costructios are quite artificial. (TODO: We could cosider addig the formal statemets of these CLT variats.) Thus, we are led to cosiderig assumptios (iii) ad (iv) that the X i have fiite mea ad variace. I both of these cases, we ca immediately see that S caot have a limit that is ormally distributed sice the Normal distributio is defied via a fiite mea ad variace. However, this does ot mea that a versio of the cetral limit theorem does ot apply here.

5 I particular, we might still hope to have a result of the form (X 1 + X X ) b a d G, (6.1) for some sequeces a ad b. Of course, i the case of the stadard cetral limit theorem a = 1/ ad b = E [X]. If we hope to obtai a geeralized form of the cetral limit theorem such as the oe above, the a first step is to uderstad what distributios may serve as limits, i.e., what distributios might G follow? To that ed, ote that the Normal distributio has some very useful properties that make it atural i the cotext of the cetral limit theorem. I particular, if X 1, X 2 are idepedet ad ormally distributed with mea µ ad variace σ 2, the X 1 + X 2 has mea 2µ ad variace 2σ 2. Ad, more geerally, for ay a 1, a 2 > 0, a 1 X 1 + a 2 X 2 is ormally distributed with mea (a 1 + a 2 )µ ad variace (a a2 2 )σ2, which implies that ) ( ) d a 1 X 1 + a 2 X 2 = ( a a2 2 X 1 + a 1 + a 2 a a2 2 µ Thus, the distributio of the sum of two Normal radom variables yields a simple liear scalig of oe of the origial radom variables. So, the Normal distributio is, i a sese, stable with respect to additio ad scalar multiplicatio. This otio of stability turs out to be more geeral tha just the Normal distributio, ad i particular the class of stable distributios is defied as follows. Defiitio 15. A distributio F is stable if, for i.i.d. radom variables X 1, X 2 with distributio F, ad ay costats a 1, a 2 > 0, there exist costats a > 0 ad b R such that a 1 X 1 + a 2 X 2 d = ax1 + b. A radom variable is stable if its distributio fuctio is stable. We have already see that the Normal distributio is a example of a stable distributio. Aother commo distributio that is stable is the Cauchy distributio. However, beyod these two examples, it is difficult to uderstad from the distributio aloe how broad the class of stable distributios is ad what properties stable distributios have. A secod, equivalet, defiitio of stable distributios provides a bit more isight ito the class. Defiitio 16. A distributio F is stable if, for ay 2 i.i.d. radom variables X 1, X 2,, X with distributio F, there exist costats c > 0, d R such that X 1 + X X d = c X 1 + d. (TODO: Either prove here that the two defs are equivalet, or tur this ito a exercise.) This secod defiitio of the class of stable distributios highlights that the otio of stability is tightly coupled to the form of the form of the cetral limit theorem, ad to its geeralized form i (6.1). I particular, it ca be show from this defiitio that stable distributios are precisely the distributios that ca serve as the limitig distributio G i (6.1).

6 Theorem 19. A distributio F is stable if ad oly there exists ad ifiite sequece of i.i.d. radom variables X 1, X 2,... ad determiistic sequeces {a }, {b } (a > 0), such that (X 1 + X X ) b a d F. I a sese, the above theorem provides a geeralized cetral limit theorem, albeit oe that is particularly vague. That is, it highlights that there is a etire class of distributios, stable distributios, that serve as limitig distributios for additive processes. So, the Normal distributio is ot as special as the stadard statemet of the cetral limit theorem makes it appear, ad the fact that it is the limitig distributio ca be explaied simply by the fact that it is a stable distributio. Proof sketch of Theorem 19. First, we show that if F is a stable distributio, the it is the limit, i distributio, of a cetered, ormalized additive process. Let {X i } i 1 deote a i.i.d. sequece of radom variables with distributio F. By Defiitio 16, for ay 2, there exist costats c > 0, d R such that I other words, for ay 2, It therefore follows trivially that as, X 1 + X X d = c X 1 + d. (X 1 + X X ) d c d = X1. (X 1 + X X ) d c d F. Next, we show that if the distributio F is the limit i distributio of a cetered, ormalized additive process, the F is stable. Accordigly, suppose that (X 1 + X X ) b a d F, where {X i } i 1 is a i.i.d. sequece of radom variable, ad {a }, {b } are determiistic sequeces satisfyig a > 0. Now, fix iteger k 2, ad defie, for m = jk, j N, Y m = (X 1 + X 2 + X m ) b m a m, Z m = (X 1 + X 2 + X m ) kb j a j. d Cosider the limit ow as m by takig j. Clearly, Y m F. O the other had, ote that Zm is the sum of k i.i.d. radom variables, each distributed as (X 1+X 2 + +X j ) b j. Oe therefore expects that Z m d F k, where F k is the distributio of X 1 + X X k. Moreover, sice Y m ad Z m differ oly a j

7 via traslatio ad scalig parameters, it ca be show that their limitig distributios also oly differ via traslatio parameters. I other words, F k ad F differ oly via traslatio ad scalig parameters. Sice this is true for all k 2, it the follows from Defiitio 16 that F is stable. (TODO: Add referece to proof with all details.) 6.3 Uderstadig stable distributios Theorem 19 highlights the importace of the class of stable distributios i the cotext of geeralizig the cetral limit theorem; however the statemet is ot particularly satisfyig because it provides little iformatio about the form of the limitig distributios or about whe differet limitig distributios will emerge. To address these issues it is ecessary to develop a better uderstadig of the class of stable distributios. So far, we have oted that the class of stable distributios icludes to commo distributios, the Normal distributio ad the Cauchy distributio. However, beyod these two distributios it is difficult to uderstad the geerality of the class from the distributio aloe. However, it turs out that the class of stable distributios ca be characterized quite cocisely. I particular, the followig represetatio theorem for the class characterizes all stable distributios via their characteristic fuctio. Note that the proof is is quite techical, ad so we omit it, but the iterested reader ca refer to (add citatio). To state the represetatio theorem recall that, for x R, sig(x) = 1 if x < 0 0 if x = 0 1 if x > 0 Theorem 20 (Represetatio theorem). A o-degeerate radom variable X is stable if ad oly if X d = az + b, where a > 0, b R, ad the radom variable Z has a characteristic fuctio of the followig form, parameterized by α (0, 2], ad β [ 1, 1]. where. φ Z (t) = exp { t α (1 iβsig(t)γ(t, α))}, (6.2) { ( ta πα ) γ(t, α) = 2 for α 1. log t for α = 1 2 π (TODO: Add diagram of where stable distributios fit wrt heavy-tailed classificatios.) The represetatio theorem gives a cocrete characterizatio of stable distributios which is crucial for uderstadig the geerality of the class. I particular, the represetatio theorem highlights that the class of stable distributios ca be parameterized via two parameters, α (0, 2], ad β [ 1, 1]. Typically, α is referred to as the stability parameter ad stable distributios with stability parameter α as referred to as α stable distributios. As we will see, α is closely tied to the tail of the distributio. O the other had, β is the skewess parameter: β = 0 yields distributios symmetric aroud zero, while β > 0 (β < 0) yields distributios skewed to the right (left) of zero. While the represetatio theorem provides a cocise characterizatio of the class of stable distributios, the fact that it is stated i terms of the characteristic fuctio makes it difficult to derive isight about the properties of stable distributio fuctios from it directly. It would be more coveiet to have a characterizatio of the class i terms of the distributio fuctio itself, as we had for the class of regularly varyig

8 distributios, subexpoetial distributios, ad log-tailed distributios. However, while the characteristic fuctios of stable distributios have closed forms, oly i a hadful of cases ca these distributios be described via their desity fuctio or distributio fuctio i closed form. We have already discussed two of these. The case of α = 2 correspods to the Normal distributio. This is easy to observe by substitutig ito (6.2), which yields φ Z (t) = e t2, which of course meas Z is ormally distributed with mea 0 ad variace 2. Similarly, it is easy to see that α = 1, β = 0 correspods to the Cauchy distributio. A third case well-kow stable distributio is the Levy distributio, which correspods to α = 1/2, β = 1. (TODO: Esure that these forms match with what we itroduce i Chapter 2.) I these three cases, the distributio fuctio ca be writte i closed form, but i geeral it is difficult to explicitly write the distributio ad desity fuctios for stable distributios. However, there are some properties that ca be characterized. Oe that is of particular iterest i the cotext of the curret book is the tail of stable distributios. I turs out that it is possible to obtai a explicit characterizatio of the tail behavior of stable distributios. I particular, the followig theorem characterizes the tail behavior of α-stable distributios, for α (0, 2). Give that, the case of α = 2 correspods to the Normal distributio, this gives a complete characterizatio of the tail behavior of stable distributios. Theorem 21. If X is α-stable for α (0, 2), there exist p, q 0 with p + q > 0 such that, as x, P (X > x) = (p + o(1))x α, P (X < x) = (q + o(1))x α. Thus, either the left tail, the right tail, or both tails of X are regularly varyig with idex α. The cosequece of this theorem is quite deep. It highlights that, withi the class of stable distributios, the oly o-degeerate light-tailed distributio is the Normal. Further, it highlights that the oly stable distributio with fiite variace is the Normal. Thus, give Theorem 19, it becomes atural that the Normal distributio emerges as the limit i the stadard cetral limit theorem. But, at the same time, it highlights that the Normal distributio is, i some sese, a corer case ad that all other limitig distributios of additive processes are heavy-tailed with ifiite variace. The proof of Theorem 21 is agai quite techical, but it is possible to give a proof of a restricted versio of result usig elemetary techiques. I fact, the proof is a good illustratio of the use of a Tauberia theorem. I particular, we ca make use the followig Tauberia theorem for characteristic fuctios, which is due to Pitma [2] (see also Page 336 of [1]). Note that this Tauberia theorem uses oly the real compoet of the characteristic fuctio, U X (t), i.e., U X (t) := Re(φ X (t)) = cos(tx)df (x). Theorem 22 (Tauberia theorem). For slowly varyig L(x), ad α (0, 2), the followig are equivalet: P ( X > x) x α L(x) as x, π 1 U X (t) 2Γ(α) si(πα/2) tα L(1/t) as t 0.

9 Like the tauberia theorem we discussed for regularly varyig distributios (Theorem (add ref)), this result coects the tail of the distributio to the behavior of a trasform aroud zero, oly i this case the trasform cosidered is the characteristic fuctio istead of the momet geeratig fuctio. Note that this Tauberia theorem applies to the tail of X rather tha X, ad so it caot be used to distiguish the behavior of the right ad left tails of the distributio, rather it just provides iformatio about the sum of the two tails. However, because of this fact, it deals oly with the real part of the characteristic fuctio, which makes it much simpler to work with aalytically. Of course, usig the Tauberia theorem i Theorem 22, we caot hope to prove the etirety of Theorem 21, istead we prove oly the followig restricted versio. Claim 1. If X is α-stable for α (0, 2), the X RV(α). Though ot as precise as Theorem 21, this restricted result already highlights the most importat aspect of Theorem 21: that α-stable distributios with α (0, 2) are heavy-tailed. Note that i order to prove the full result, the key differece from the followig is the use of a more detailed tauberia theorem tha Theorem 22, ad the added aalytic complexity that comes alog with the eed to work with the complexvalued characteristic fuctio φ X (t) istead of just the real-valued compoet U X (t). Proof of Claim 1. To begi the proof, let Z deote a α-stable radom variable whose characteristic fuctio is give by (6.2), with α (0, 2), α 1. We the have, for t > 0, φ Z (t) = exp{ t α (1 iβ ta ( ) πα 2 )}. This meas that U Z (t) = exp{ t α } cos(δt α ), where δ = β ta ( ) πα 2. We will ow show that 1 U Z (t) t α (t 0). (6.3) Note that if δ = 0, this is trivial. Ideed, i this case, U Z (t) = exp{ t α } = 1 t α + o(t α ), which of course implies (6.3). Turig ow to the case δ 0, we have Therefore, U Z (t) = [1 t α + o(t α )] cos(δt α ). 1 U Z (t) = (1 cos(δt α )) + cos(δt α )t α cos(δt α )o(t α ) = 2 si 2 (δt α /2) + cos(δt α )t α cos(δt α )o(t α ). Usig si(δt α /2) δt α /2 as t 0, we ote that si 2 (δt α /2) = o(t α ) as t 0. The above equatio therefore implies (6.3). Now that we have established (6.3), we ca use Theorem 22 to coclude that P ( Z > x) = P (Z > x) + P (Z < x) π 2Γ(α) si(πα/2) x α.

10 6.4 The geeralized cetral limit theorem The characterizatio of stable distributios i the previous sectio ow gives us the cotext we eed i order to move to the geeralized cetral limit theorem. We already kow from Theorem 19 that the limitig distributio of additive processes is stable. Combiig this with our characterizatio of stable distributios allows us to uderstad whe differet stable distributios emerge as the limitig distributio. Theorem 23 (Geeralized cetral limit theorem). Cosider a ifiite sequece of i.i.d. radom variables X 1, X 2,... with distributio F. There exist determiistic sequeces {a }, {b } (a > 0) such that (X 1 + X X ) b a d G, if ad oly if G is α-stable with α (0, 2]. Further, α = 2, i.e., G is ormally distributed, if ad oly if x x y2 df (y) is slowly varyig as x, ad α (0, 2) if ad oly if F (x) = (p + o(1))x α L(x), F ( x) = (q + o(1))x α L(x) as x, where L(x) is slowly varyig, ad p, q 0, p + q > 0. Of course the most strikig aspect of this result comes from cotrastig it with the stadard cetral limit theorem. Here, the emergece of the Normal distributio is just oe of may possible optios, ad oe that is, i some sese, a corer case sice it correspods oly to α = 2. Aother iterestig cotrast to the stadard cetral limit theorem is that the Normal distributio occurs as the limit eve i some cases where the variace is ifiite. I these cases, the ormalizig costat a must be chose to be larger tha, ad will iclude a slowly varyig fuctio to couter the growth of x x y2 df (y). For a discussio of exactly how to determie a i such cases, see (add ref). Of course, the most ovel part of the geeralized cetral limit is that it highlights that heavy-tailed stable distributios ca emerge as the limit of additive processes. I particular, if oe starts with fiite variace distributios, the Normal emerges, but if oe starts with regularly varyig distributios with ifiite variace, the heavy-tailed distributios ca emerge. Further, the statemet shows that the emergig distributio ca eve have a ifiite mea, ad so the tail ca be extremely heavy. The stregth of the statemet of the geeralized cetral limit is that it characterizes exactly whe differet stable distributios emerge i the limit. The formal termiology that is typically used to capture this is the domai of attractio, which defies the set of distributios for X i that lead to a particular limitig distributio. For example, the domai of attractio of the Normal distributio is the set of distributios where x x y2 df (y) is slowly varyig as x. The domai of attractio is also tightly coupled with the sequeces of {a }, {b } that are used for scalig the process. Though the above statemet of the geeralized cetral limit theorem give above does ot explicitly give {a }, {b }, they ca also be precisely specified. For example, i the case of α (0, 2), the scalig costats a must be chose to satisfy L(a ) lim a α = c (0, )

11 ad the traslatio costats may be chose to satisfy 0 if 0 < α < 1 b = a si(x/a )df (x) if α = 1 E [X 1 ] if 1 < α < 2. The case of α = 2 is more ivolved, ad so we refer the reader to (add ref). Fially, let us discuss the proof of the geeralized cetral limit theorem. Ufortuately, the full proof is quite techical ad so ot appropriate for iclusio here. However, to get a sese for the result, we ca prove a restricted form of it usig a approach that mimics the way the stadard cetral limit theorem is typically prove. This highlights that the geeralized cetral limit theorem is o more mysterious tha the stadard cetral limit theorem, it simply requires a bit more techical machiery. I particular, we will prove the followig restricted versio of the geeralized cetral limit theorem for the case of symmetric radom variables. The key reaso that this restrictio is simpler to prove is that the symmetry meas that the imagiary part of the characteristic fuctio disappears, which allows the use of the Tauberia theorem i Theorem 22. Also, ote that, because of the symmetry, the traslatio costats {b } = 0, ad so they do ot show up i the statemet. Claim 2 (Restricted geeralized cetral limit theorem). Suppose that {X i } i 1 are i.i.d. symmetric radom variables with distributio F, where P ( X > x) cx α, with α (0, 2). The i=1 X i 1/α d G, where G is a symmetric α-stable distributio. Proof. Note that the X i are symmetric, ad so the imagiary part of their characteristic fuctio equals zero. Thus, deotig the characteristic fuctio of the i.i.d. X i by φ X, we have that φ X = U X (t). Moreover, sice U X is a symmetric fuctio, i.e., U X ( t) = U X (t), so is φ X. The proof of the stadard cetral limit theorem begis by cosiderig the Taylor expasio of the characteristic fuctio of Y i = X i E [X]. Of course, sice X i are symmetric here, we do ot eed to cosider such a shift. But, a more fudametal differece is that, sice the Var [X i ] =, we caot write such a Taylor expasio of φ X. Istead, we ivoke Theorem 22 to obtai a similar represetatio of φ X. I particular, we have φ X (t) = 1 b t α (1 + o(1)) as t 0, cπ where b =. ad α (0, 2). Note that the correspodig represetatio i the proof of the 2Γ(α) si( πα 2 ) stadard cetral limit theorem has the form 1 σ2 t o(t 2 ), which is quite parallel to the form above if oe were to use α = 2. Give this represetatio of φ X, the proof ow mimics that for the stadard cetral limit theorem. I i=1 particular, deote Z = X i. The the characteristic fuctio of Z 1/α satisfies the followig ( ) ) φ Z (s) = φ X (s/ 1/α ) = (1 b(1 + o(1)) t α e b t α as.

12 The limit is simply a scaled versio of the caoical characteristic fuctio of the symmetric (β = 0) α-stable distributio, as desired. 6.5 A variatio: Heavy-tails i radom walks To this poit i the chapter we have cosidered a very geeral, but very abstract, versio of a additive process: S = X X. Withi this cotext, we have focused etirely o uderstadig the behavior of S as, ad we have see that if the X i have fiite variace tha the Normal distributio will emerge but that if the X i have ifiite variace heavy-tailed distributios ca emerge. Of course, additive processes come i may guises, ad depedig o the settig, oe may ask a variety of other questios about S besides simply about its behavior as. I this sectio, we will highlight that, whe differet aspects of additive processes are studied, heavy-tailed may emerge eve i cotexts where the radom process is made up of etirely light-tailed compoets. I particular, i this chapter we cosider a classical example of a additive process, radom walks. Radom walks are a a example of a very simple process that has foud applicatio i a surprisig umber of disciplies. I fact, few mathematical models have foud applicatios i as diverse a rage of areas, icludig fiace, computer sciece, physics, biology, ad more. The most simple example of a radom walk is oe i which a walker is equally likely take a sigle step i oe directio or the other at each time step. Formally, the walker that starts at 0 ad takes a sequece of idepedet steps X 1,..., X where { 1, with probability 1/2; X i = 1, with probability 1/2. The positio of the walker at time is thus simply the additive process S = X 1 + X X. (TODO: add a figure of the radom walk) This is referred to as a simple symmetric radom walk i oe-dimesio, where simple refers to the fact that the step size is oe, symmetric refers to the fact that steps up ad dow are equally likely, ad oe-dimesioal refers to the fact that the radom walk is o a lie. You have almost certaily come across this versio of a radom walk i your itroductory probability course uder the guise of a druk leavig a bar ad waderig aimlessly up ad dow the street tryig to get home. Of course, there are may more complicated versios of radom walks too. I geeral, the radom walk may be asymmetric or biased, e.g., the probability of takig a positive step may be p 1/2; the radom walk may be i more tha oe dimesio or over a geeral graph; or the radom walk might allow step sizes other tha 1, e.g., the step size could be radom. But, for this sectio, we stick to the simple, symmetric, oe-dimesioal case because this case is already sufficiet to highlight the emergece of heavy-tailed distributios. Oe of the first atural questios to ask about a radom walk is where the walker is after a certai umber of steps, i.e., what is the behavior of S. Of course, this is the same questio that we have addressed throughout this chapter for more geeral additive processes. I particular, the law of large umbers ad the cetral limit theorem are eough to give us a aswer. I fact, i the case of the simple oe-dimesioal

13 radom walk we are studyig, the X i are light-tailed ad so the stadard versio of the cetral limit theorem applies, which meas that S / coverges to a Normal distributio. However, it is ot eve ecessary to apply the cetral limit theorem sice the distributio of the positio of the radom walk ca be derived explicitly without too much trouble. (TODO: fill i derivatio) Though the positio of the radom walk at time ca be uderstood quite easily, this is oly oe of may questios oe may ask about a radom walk. For example, two other importat questios that are ofte asked are what is the maximum positio of the walker after steps? ad whe will the walker first retur to its startig poit? It is the secod of these that we focus o here. I particular, we will study the retur time of a radom walk, which is deoted by T ad defied as the first time whe the walker returs to its startig poit. The retur time of a radom walk is ofte of crucial importace. For example, (TODO: fill i motivatio) The retur time is of particular iterest i the cotext of this chapter because it is a example where a heavy-tailed distributio emerges from a additive process. Further, it is a particularly jarrig example of this because because a heavy-tailed distributio emerges eve though we are cosiderig a additive process that is defied etirely of bouded (ad thus extremely light-tailed) distributios. Specifically, the tail of the distributio of the retur time of a simple, symmetric, oe-dimesioal radom walk ca be characterized as follows. Theorem 24. Cosider a simple, symmetric, oe-dimesioal radom walk. The distributio of the retur 2/π time T satisfies P (T > x) x. This result highlights that, ot oly is the retur time heavy-tailed, it is extremely extremely heavy-tailed. I particular, it is regularly varyig with idex 1/2, which highlights that both the mea ad variace are ifiite. (TODO: add a paragraph talkig about variatios of the process that give differet tails, i.e., that affect the tail idex.) Proof of Theorem 24. For = 0, 1, 2,, let u() deote the probability that the radom walk hits zero at time, i.e., u() = P (S = 0). 1 Clearly, u() = 0 for all odd, sice the radom walk ca oly retur to zero i a eve umber of steps. Also, ote that sice the radom walk starts at zero, u(0) = 1. We will aalyse the distributio of T by relatig it to the sequece u( ), which, as we will see, ca be computed explictly. Let f() = P (T = ). As before, ote that f() = 0 for all odd. Also, ote that sice T > 0, f(0) = 0. We first relate u( ) ad f( ) as follows. For 1, if the radom walk hits zero at time 2, the the time T of first retur to zero is ecessarily 2. We ca therefore represet the probability u(2) as u(2) = P (S 2 = 0) = P (T = 2j) P (S 2 = 0 T = 2j). j=1 Moreover, ote that P (S 2 = 0 T = 2j) is simply the probability that the radom walk hits zero after startig at zero after 2 2j steps, i.e., P (S 2 = 0 T = 2j) = u(2 2j). We therefore obtai the 1 It is importat to ote that whe we cosider the evet S = 0, the radom walk could have hit zero several times before time. u() captures the probability of all such paths.

14 followig recursive relatio for 1. u(2) = f(2j)u(2 2j) (6.4) j=1 We ow use the above recursio to relate the Laplace trasform of the radom variable T to that of the fuctio u( ). Specifically, defie ψ T (s) := E [ e st ] = ψ u (s) := f(2)e 2s, =1 u(m)e ms = m=0 Now, usig (6.4), we may express ψ u as follows. ψ u (s) = u(2)e 2s =0 = 1 + =1 j=1 u(2)e 2s. =0 f(2j)u(2 2j)e 2s Iterchagig the order of the summatios above (TODO: add justificatio), which gives us ψ u (s) = 1 + e 2js f(2j) e s(2 2j) u(2 2j) j=1 = 1 + ψ T (s)ψ u (s), =j ψ T (s) = 1 1 ψ u (s). (6.5) Now that we have related the Laplace trasform of T to that of the sequece u( ), we ow compute u( ) ad its trasform explicitly. For 1, let us cosider the evet S 2 = 0. This evet simply meas that after startig at zero, the radom walk made positive steps, ad egative steps. Therefore, the total umber of paths of the radom walk that correspod to the evet S 2 = 0 equals ( ) 2. Sice the total umber of possible paths after 2 steps equals 2 2, ad each of these is equally likely, we coclude that ( 2 ) u(2) = 2 2.

15 Therefore, the trasform ψ u is give by Usig the biomial expasio ψ u (s) = 1 + (TODO: elaborate here) we may express ψ u as Therefore, usig the relatio (6.5), we obtai ( 2 ) 2 2 e 2s. =1 1 1 x = 1 + ψ u (s) = ( 2 ) 2 2 x, =1 1 1 e 2s. ψ T (s) = 1 1 e 2s. We ca ow use the Tauberia theorem (add ref)to deduce the tail behavior of T from the behavior of ψ T (s) ear the origi. As s 0, sice e 2s = 1 2s(1 + o(1)), Therefore, from Theorem (add ref), ψ T (s) = 1 1 (1 2s(1 + o(1))) = 1 2s(1 + o(1)) = 1 2s(1 + o(1)) P (T > x) 2 1. Γ(1/2) x Notig that Γ(1/2) = π, we have our desired tail charactertizatio.