MODELS FOR COMPUTER NETWORK TRAFFIC

Transcription

1 MODELS FOR COMPUTER NETWORK TRAFFIC Murad S. Taqqu Boston University Joint work with Walter Willinger, Joshua Levy and Vladas Pipiras,...

2 Web Site

3 OUTLINE Background: 1) Presentation of the Ethernet data 2) Macro model The role of heavy tails: 3) On-Off (micro model) 4) Heavy-tailed rewards

4 ETHERNET DATA time. Data set: number of Ethernet packets/unit Collected at a LAN (Local Area Network) at Bellcore on August 29, Huge data set: 27 hours, with a time resolution of 2μsec. How to analyze it? Scaling methods seem natural.

5 Packets/Time Unit Packets/Time Unit Packets/Time Unit Time Unit = 1 Seconds Time Unit = 1 Seconds Time Unit = 1 Seconds Packets/Time Unit Packets/Time Unit Packets/Time Unit Time Unit = 1 Seconds Time Unit = 1 Seconds Time Unit = 1 Seconds Packets/Time Unit Packets/Time Unit Packets/Time Unit Time Unit = 1 Second Time Unit = 1 Second Time Unit = 1 Second Packets/Time Unit Packets/Time Unit Packets/Time Unit Time Unit =.1 Second Time Unit =.1 Second Time Unit =.1 Second Packets/Time Unit Packets/Time Unit Packets/Time Unit Time Unit =.1 Second Time Unit =.1 Second Time Unit =.1 Second

6

7 FRACTIONAL BROWNIAN MOTION FBM: B H (t), t, 1/2 <H<1 1) Gaussian, mean, B H () = 2) Stationary increments 3) EB 2 H (t) =σ2 t 2H Properties: (a) EB H (s)b H (t) = σ2 2 { t 2H + s 2H t s 2H} (b) B H (at) = D a H B H (t) for all a> Notes: If H =1/2, then EBH 2 (t) =σ2 t and so FBM reduces to Brownian motion. The increments of FBM are called Fractional Gaussian Noise.

8 Systematic study of the effectiveness of the different statistical estimators M. S. Taqqu, V. Teverovsky, and W. Willinger (1995) Estimators for long-range dependence: an empirical study. Fractals, 3(4): M. S. Taqqu and V. Teverovsky (1998) Estimating long-range dependence in finite and infinite variance series. In R. Adler, R. Feldman, and M. S. Taqqu, editors, A Practical Guide to Heavy Tails: Statistical Techniques for Analyzing Heavy-Tailed Distributions, Boston, Birkhäuser. P. Abry, P. Flandrin, M. S. Taqqu and D. Veitch (2) Wavelets for the analysis, estimation and synthesis of scaling data. In K. Park and W. Willinger, editors, Self-similar Network Traffic and Performance Evaluation, Wiley.

9 Jean-Marc Bardet, Gabriel Lang, Georges Oppenheim, Anne Philippe, Stilian Stoev and Murad S. Taqqu (23) Semi-parametric estimation of the long-range dependence parameter : A survey. In: Theory and Applications of Long-range Dependence, P. Doukhan, G. Oppenheim and M. S. Taqqu editors. Birkhäuser, Boston, pages Gilles Faÿ, Eric Moulines and FrançoisRoueff, Murad S. Taqqu (28). Estimators of Long- Memory: Fourier versus Wavelets. Preprint.

10 WHY? THE ON-OFF MODEL

11

12 ON times: U 1 (j), j=1, 2,... F 1 (i.i.d.) Either F 1 (x) =P (U 1 >x) l 1 x α 1 (1 <α 1 < 2) as x,or EU1 2 <, (α 1 =2). OFF times: U 2 (j), j=1, 2,... F 2 (i.i.d.) Either F 2 (x) =P (U 2 >x) l 2 x α 2 (1 <α 2 < 2) as x,or EU2 2 <, (α 2 =2). MEANS: μ 1 = EU 1,μ 2 = EU 2.

13 ON/OFF reward: W (t) = { 1 if t ON t OFF. Many i.i.d. sources Each workstation generates its own reward process {W (m) }: {W (m) (t), t }, m=1, 2,... i.i.d. Question: How does Tt M m=1 W (m) (u) du adequately normalized behave as M, T?

14 Theorem 1 T H Tt 1 M L lim T L lim M M m=1 ( W (m) (u) EW (m) (u) ) du = B H (t) where H = 3 min(α 1,α 2 ) 2 Note: If α 1 = α 2 =2,thenH =1/2, and the limit is Brownian Motion.

15 STEPS IN THE PROOF 1. Recall that FBM is uniquely characterized by: 1) Gaussian process, mean 2) Stationary increments 3) EBH 2 (t) =t2h. Hence enough to show that V (t) Var ( t W (u)du ) t 2H as t. 2. Determine the covariance: γ(u) =EW(u)W () (EW()) 2 since V (t) =2 t ( v ) γ(u)du dv

16 3. Mean: EW(t) =1 P (t is in an ON interval) + P (t is in an OFF interval) = P (t is in an ON interval) = μ 1 μ 1 + μ 2. since we have an alternating renewal process. Covariance: γ(t) =EW(t)W () EW(t) 2 = EW(t)W () ( μ1 μ 1 + μ 2 ) 2 = P ( and t areinanoninterval) ( μ1 = P (t is ON isinon)p ( is in ON) = μ 1 μ 1 + μ 2 [ π 11 (t) μ 1 μ 1 + μ 2 ]. μ 1 + μ 2 ) 2 ( μ1 μ 1 + μ 2 ) 2

17 4. Get renewal equation for π 11 (t). 5. Solve it by using Laplace transforms: Â(s) = e st A(t)dt A(t) ast Â(s) ass. 6. π 11 (s) = γ(s) = V (s). 7. Case α 1 = α 2 = α<2. We find that V (s) C 1 s α 4 as s. (1) 8. We want to conclude: V (t) C 2 t 3 α as t (2) If (2) holds, then we are done, because 2H = 3 α. But (2) = (1) : Abelian theorem (1) = / (2) : Tauberian theorem

18 9. OK ((1) = (2)) if, in addition, lim lim inf λ 1 x inf t [1,λ] (Bingham, Goldie, Teugels, 1987). V (tx) V (x) x 3 α (3) But we need to know V (t). So go back to renewal equation in time domain to show that (3) holds.

19 If we reverse the limits, we must change the normalization accordingly and in this case L lim M L lim T 1 1 M 1/α T 1/α Tt M m=1 (W (m) (u) EW (m) (w))du = cs α (t) where S α (t) is a symmetric Lévy stable motion, a process with independent and stationary increments but with infinite variance.

20 What happens if T and M go together at infinity? Assume that M(T ) is an integer-valued function which is non-decreasing in T and which tends to infinity as T. Then, under the FGC: fast growth condition: lim T (M(T )) T α 1 =, the limit is the fractional Brownian motion B H (t), and under the SGC: slow growth condition: lim T (M(T )) T α 1 =, the limit is the Lévy stable process S α (t) This was shown by Mikosch, Resnick, Rootzén and Stegeman ( 22) in the on-off case.

21 WHY THIS BREAKDOWN? ( Case of finite variance rewards β =2) W (Tt,M)= Recall: Tt M m=1 ( W (m) (u) EW (m) (u) ) du 1 FGC: lim T lim M W (Tt,M)=B T H M 1/2 H (t) 1 SGC: lim M lim T W (Tt,M)=S T 1/α M 1/α α (t) Note: For finite T and M: EW (Tt,M) 2 < and ( EW (Tt,M) 2) 1/2 ct H M 1/2. But for SGC with M = M(T )andt, we expect the limit to be S α (t), which has infinite variance. Hence: ( EW (T,M) 2) 1/2 T 1/α M 1/α ( EW (T,M) 2) 1/2 T 1/α M 1/α T H M 1/2 T 1/α M 1/α = T (3 α)/2 M 1/2 T 1/α M 1/α This happens if M(T )/T α 1..

22 SUMMARY FGC: (Fast growth condition): lim T M(T ) T α 1. The limit is FBM : B H (t) SGC: (Slow growth condition): lim T The limit is LSM : S α (t) M(T ). T α 1 IGC: (Intermediate growth condition): lim T M(T ) T α 1 constant (, ). What is the limit?

23 INTERMEDIATE GROWTH CONDITION (Kaj and Gaigalas (22), Kaj and Taqqu (28) in the infinite source Poisson model variant). In the intermediate growth case: lim T M(T ) T α 1 constant (, ). the limit process Y α (t) has the following characteristics: all its moments are finite it is non-gaussian it has the same covariance as FBM with H = 3 α, 2 namely: EY α (s)y α (t) =C { s 2H + t 2H s t 2H} Local behavior: c α Y α ( t c ) B H(t), c.

24

25 THE INTERMEDIATE LIMIT PROCESS Y α (t) u= where: v= ( t Y α (t) = ) (N(du, ) dv) n(du, dv) 1 {u<s<v}ds u = start time of flow (job, connection) v = end time of flow t 1 {u<s<v} ds = amount of time flow spends in [,t] N(du, dv) = Poisson random measure in the (u, v) plane = number of points in the box dudv n(du, dv) =EN(du, dv) = mean measure of the random measure N(du, dv), with n(du, dv) =α(v u) α 1 dudv

26 Infinite source Poisson model An alternative to the ON/OFF model is the infinite source Poisson model. In this case sources come in at a rate of λ and each source transmits for a random time with distribution F on (x) lx α, x,1<α<2. The transmissions are assumed i.i.d. Here it is λ that plays the role of M. Let W (u) denote the number of active sources at time u>andleta(t) = t W (u)du be the cumulative load fron time to time t.

27 Under the slow growth condition one has lim T (λ(t )) = T α 1 L lim T A(Tt) EA(Tt) λ 1/α T 1/α = cs α (t) and under the fast growth condition lim T (λ(t )) T α 1 = one has L lim T A(Tt) EA(Tt) λ 1/α T (3 α)/2 = cb H (t). Observe that one has EA(t) = μλt.

28 HEAVY-TAILED REWARDS: THE TELECOM PROCESS

29

30 Finite variance rewards (β =2) R(j) i.i.d., ER(j) =,ER 2 (j) < Answer: Same limit as in the on-off case with α 1 = α 2 = α. The limit is the Gaussian FBM with H = 3 α 2 Heavy tailed rewards (1 <α<β<2) R(j) i.i.d., ER(j) =, P [ R(j) >x] lx β as x. In this case, look at: 1 T H Tt 1 M 1/β M m=1 W (m) (u) du ButwhatisH? (How to replace 2 by β in H =(3 α)/2?)

31 Answer: β =2: β<2: H = 3 α 2 H = β +1 α β What is the limit Z(t)? We know that it is β- stable process stationary increments dependent increments Conjecture : The limit Z(t) is the (moving average) β-stable analogue of FBM.

32 Fractional Brownian Motion (FBM) B H (t) = f(t, u)dm(u), where 1) f(t, u) =a[(t u) H 1/2 + ( u) H 1/2 + ] + b[(t u) H 1/2 ( u) H 1/2 ], a,b R 2) dm(u): i.i.d. noise, Gaussian. Note: B H (at) = f(at, u)dm(u) = f(at, au)dm(au) d = a H 1/2 a 1/2 B H (t) =a H B H (t) Fractional Stable Motion (FSM) Same : f(t, u)dm β(u), but now 1) f(t, u) =a[(t u) H β + ( u) H β + ] + b[(t u) H β ( u) H β ], a,b R 2) dm β (u): i.i.d. noise, β-stable, β<2.

33 The conjecture that the limit is Fractional stablemotionisfalse. THEOREM: The limit is what is now known as the Telecom process. FRACTIONAL STABLE MOTION: f(t, u)dm β(u) where M β ( ) isβ-stable. THE TELECOM PROCESS: f(t, u, v)dm β(u, v) where M β (, ) isβ-stable. M β is a β-stable random measure. For any set A, M β (A) is a random variable with characteristic function: Ee iθm β(a) = e θ β m(a) where m( ) is a non-random measure, called the control measure.

34 The Telecom process Z(t) = ( t 1 {u<s<v}ds ) M β (du, dv) u = start of the connection v = end of the connection v u duration of the connection t 1 {u<s<v} ds = lenght of the connection in [,t]. M β has control measure: m(du, dv) =α(v u) α 1 + dudv where 1 <α<β<2. The Telecom process Z(t), has stationary increments and is self-similar with index H = β +1 α β (1/β, 1).

35 Selected Bibliography Will E. Leland, Murad S. Taqqu, Walter Willinger and Daniel V. Wilson. On the self-similar nature of Ethernet traffic (Extended Version). IEEE/ACM Transactions in Networking, 2 (1994), Walter Willinger, Vern Paxson, Rudolf. H. Riedi and Murad S. Taqqu (23) Long-range dependence and data network traffic In: Theory and Applications of Long-range Dependence, P. Doukhan, G. Oppenheim and M. S. Taqqu editors. Birkhäuser, Boston (23) Ingemar Kaj and Murad S. Taqqu (28) Fractional Brownian motion and to the Telecom process: the integral representation approach (with Ingemar Kaj). In In an Out of Equilibrium 2, Vol. 2 of Progress in Probability, Maria Eulalia Vares and Vladas Sidoravicius, editors, Birkhäuser.