Semi-Markov Network Traffic Model Based on Pareto Distribution

Transcription

1 rd International Conference on Engineering Technology and Application (ICETA 2016) ISBN: Semi-Markov Network Traffic Model Based on Pareto Distribution Jinlong Fei State Key Laboratory of Maematical Engineering and Advanced Computing, Zhengzhou, Henan, China Science and Technology on Communication Security Laboratory, Chengdu, Sichuan, China Tianpeng Wang, Xinzheng He & Yuefei Zhu State Key Laboratory of Maematical Engineering and Advanced Computing, Zhengzhou, Henan, China ABSTRACT: The establishment of e network traffic model has an important significance on e research of network. On e basis of existing researches, is paper proposes e improved Semi-Markov network traffic model, and re-divides e state of network traffic into idle state, normal state and busy state, and also researches e busy state of network traffic by e use of Pareto distribution, and later researches e state transition relation of Semi-Markov process on is basis. The experimental results show at e characteristics of e network traffic under e busy state can be well in line wi e nature of Pareto distribution, and e network bandwid utilization calculated in is way is also consistent wi e actual situation, so at e model established in is paper can well describe e characteristics of network traffic. Keywords: Pareto distribution; Semi-Markov process; network traffic model; network bandwid utilization 1 INTRODUCTION Wi e sustainable development of e Internet, a variety of new network applications emerge in endlessly. On e one hand, ese applications bring convenience to people s life; on e oer hand, it causes explosive grow of network traffic, and also brings new challenges to e research of e traffic model. In order to get better network services, ere is a need to research e existing traffic behavior, and establish a better traffic model on is basis, so as to provide a reference for planning and development of network infrastructure, analysis and forecasting of network performance and management of network traffic. In e early period of e development of Internet, people proposed a lot of network traffic models based on Poisson process according to e existing researches of telephone network model [1] [2], and en found at e traditional Poisson model was unable to accurately describe e behavior of network traffic wi e deepening of e research [3], because e telephone network ust had a single application of communication, but e Internet possessed ousands of applications, and a variety of applications have eir own characteristics. Therefore, it is unable to accurately describe e characteristics of e entire network traffic by e use of Poisson model. In 1994, Leland et al [4] found out e self-similarity of e Internet traffic, so at e research of e network traffic model entered into a new phase. The researches of e self-similar network traffic model emerge in endlessly, of which e more important models are e model based on e correlation eory [5], wavelet-based model [6] [7], ON / OFF model based on heavy-tailed distribution [8], model based on multifractal eory [9] [10] and mixed model combined wi a variety of eories [11] [12] [13] [14] and so on. These models can preferably simulate e network traffic behavior, but wi a poor analyticity and relatively high computational complexity. Therefore, Huang Xiaolu, et al [15] proposes a Semi-Markov model of network traffic, and divides e network traffic into idle state, up state, down state and busy state, and analyzes e traffic behavior under ese four states and e transition relations between states. However, in is model, e researchers mainly determine at e busy state obeys e geometric Brownian motion by way of eoretical derivation, but wi e development of e network, it is found at e size of e network traffic often obeys e 359

2 [16] [17] heavy-tailed distribution rough e experiment [18] [19] [20], and e heavy-tailed distribution is also an important reason to cause e self-similarity of network traffic. Therefore, e specific distribution of e characteristics of traffic distribution under e busy state in e model described in e literature [15] is also one of e key points researched in is paper. On e oer hand, e model described in e literature [15] divides e network into four states. However, in fact, e division into up and down state does not have an effect on e establishment of e entire network traffic model, and makes e issue complicated. How to re-consider is issue is also e key point researched in is paper. Therefore, based on e models of Huang Xiaolu and so on, is paper proposes a new type of Semi-Markov model based on Pareto distribution. Pareto distribution is a typical heavy-tailed distribution, and simplifies e number of states of e model, and simplifies e original four states to ree states, and finally proves e effectiveness of e model rough experiment. 2 PARETO DISTRIBUTION AND SEMI-MARKOV PROCESS 2.1 Heavy-tailed distribution Heavy-tailed distribution is one kind of distribution of random variable in nature, broader an normal distribution, which is mainly manifested in reality as follows: a small amount of individuals makes a substantial contribution or occupies a large number of resources. For example, most of e oil is from less oil field, e frequency of catastrophe only accounts for a small portion, water content in human body accounts for about 65%. The intuitive feature of e heavy-tailed distribution is short big head and small tail leng, which is defined as follows [17] : Definition 1: Assuming at e distribution function of e random variable (X) is F(x), if for > 0, c> 0, when x, en [ ] 1 P X > x = F x cx The random variable (X) obeys e heavy-tailed distribution, where, is a heavy-tailed parameter, indicating e tailing degree of distribution. The distribution function of e heavy-tailed distribution is actually a cluster of function. The nature is different wi e difference of e heavy-tailed parameter : When 0 < <1, e distribution function has an infinite mean value and an infinite variance; when 1<<2, e distribution function has a finite mean value and a finite variance; when 2 and <2, e probability density functions are respectively expressed as fast decay and slow decay. The above definition is actually e first-order expanded form of e heavy-tailed distribution. In many researches, e second-order expanded form of e heavy-tailed distribution (F(x)) is often used. Its best effect is e second-order expanded formula β F x 1 ax 1 + bx, > 0, β > 0 Where and β are called as e tail parameters, and a and b are scaling functions; ey are obtained from e parameter estimation. The corresponding probability density function is: = + ( + β ) f x a x ab x 1 β Pareto distribution Pareto distribution is e most classic and e simplest heavy-tailed distribution, which was found out by e Italian economist, Vilfredo Pareto after observing a lot of phenomena in 1906, and initially called as e Pareto rule, and later furer summarized as Pareto distribution. Pareto rule is mainly manifested as 80/20 rule, at is, 80% of results depend on 20% of reasons. For example, 20% of e population own 80% of e weal, 80% of outcome of labor depend on 20% of previous efforts, 80% of sales volume come from 20% of customers and so on. Pareto distribution is defined as follows: Definition 2: Assuming at e distribution function of e random variable (X) is F(x), if for>0, k>0 and x k, en [ ] 1 P X > x = F x = k x The random variable (X) obeys Pareto distribution, where, is a shape parameter, indicating e heavy-tailed degree. The smaller is, e greater e heavy-tailed degree is. k is e reshold parameter, which determines e desirable minimum value of e random variable. The probability density function of Pareto distribution is: k f x x k + x =,, > Semi-Markov process Semi-Markov process [21] is a random process, which has some properties of Markov chains, but it is also different from Markov chain. In particular, e state transition of Semi-Markov process is a Markov chain, but e time interval of e state transition is a random variable wi its own distribution. The specific definition is as follows: Definition 3: Considering at e random process wi e state of 0,1,2,3 meets e following conditions: when it enters into e state i(i 0) i. The next state is, its probability of P i, of which 0 360

3 ii. When e next state is, e time interval from entering into e state i to transition from i to is a random variable, and its distribution function is F i The state at e time t is recorded as Z (t), en {Z(t), t 0} is called as Semi-Markov process. 3 SEMI-MARKOV NETWORK TRAFFIC MODEL BASED ON PARETO DISTRIBUTION 3.1 Improved semi-markov network traffic model Huang Xiaolu, et al set up two reshold values for e network traffic rate per unit time according to e characteristics of e network traffic in e literature [15], which are respectively called as e busy reshold value and idle reshold value. They also divide e network traffic into idle state, up state, down state and busy state. When e unit traffic rate is greater an e busy reshold value, it is called as e busy state; when e unit traffic rate is less an e idle reshold value, it is called as e idle state; when e unit traffic rate is between two reshold values, and e traffic rate at e next time is greater an e traffic rate at e last time, it is called as e up state, and on e contrary, e down state. The busy reshold value and idle reshold value are set up according to e specific characteristics of e network. down state. Therefore, in e research of is paper, we simplify ese four states into ree states: idle state, normal state and busy state, of which e definition of e idle state and busy state is e same wi e definition in e literature [15], while e normal state is e state of unit traffic rate between e busy reshold value and idle reshold value, at is, e up state and down state are collectively referred to as e normal state, wiout considering eir conversion relations. The state conversion relations are shown in Figure 1. Thus, e improved Semi-Markov network traffic model is described as follows: Assuming at e state space is I = {idle, norm, busy}, a set of discrete time is T = {0, 1, 2, }, and e network traffic state is Z(t) at e time of t, define = n,, n < n 1 Z t X n T t t t + {Z(t), t 0} is called as e Semi-Markov process of e network traffic change. tn ( n T ) is e changing time of e network traffic state (called as transfer time). Xn represents e network traffic state at e transition time of tn, en { X, n n T} is an embedded Markov chain. According to e definition of e Semi-Markov process, 1) The state transition probability of Semi-Markov network traffic model is Pi = P X n+ 1 = X n = i, i I, I 2) State transition interval distribution is in in ni ni ib ib bn bn nb nb Figure 1. State transition diagram of e improved Semi-Markov model. According to e above definition, in e actual operation of e network, mutual conversion between idle state and busy state may experience repeatedly up state and down state, and e duration of up state and down state may be extremely short, resulting in extremely frequent conversion between ese two states, and complication of e conversion process, so at e research of e limit probability of each state will become more complicated; on e oer hand, we mainly concern about e duration disunction distribution in e process of mutual conversion of e busy state and idle state and ese two states, but less concern about wheer e network traffic rate is in e up state or e bi bi F t = P T t X =, X = i, T = t t, i I, I i n n+ 1 n n n+ 1 n 3.2 Characteristics of network traffic state distribution Characteristics of traffic under e busy state When e system is in e busy state, e network traffic is mainly a variety of traffic generated by user behavior. According to e preceding description, and combined wi a large number of experimental observations, e user s network behavior is in line wi e heavy-tailed distribution in most cases, and Pareto distribution can be used to furer analyze and research e characteristics of network traffic under e busy state. Assuming at e network traffic rate per unit under e busy state is e random variable V busy, its distribution function is F(v), e busy reshold value of e network traffic rate is B >0, shape parameter is>0, according to e definition of Pareto distribution, its distribution function F(v) is: = = 1,, > 0 F v P V v B v v B busy 361

4 Its probability density function is: B f v v B + v =,, > 0 1 According to e characteristics of heavy-tailed distribution, when 1< <2, e distribution function has a finite mean value and an infinite variance. If e shape parameter 1 < <2, and v B, e maematical expectation of e random variable V busy is: + + ( busy ) = = B E V vf v dv vf v dv + Bv =,1 2 B + 1 dv = B < < v Characteristics of traffic under e normal state and idle state 1) Idle state According to e literature [15] and e division of e network traffic state in is paper, e network traffic under e idle state is mostly e routing information and control information exchanged between e network equipment. The information is e necessary information for maintaining normal operation of e network, wi a strong stability and cyclicality, so it can be considered at it is in line wi e normal distribution, at is 2 V ~ N µ, σ idle idle idle Where, V idle is e network traffic rate under e idle state, µ idle is e expected value of e network traffic rate under e idle state. That is, E V = µ idle idle 2) Normal state Under e normal state, e network traffic rate is between e traffic rate of e idle state and busy state. The traffic generated by e user does not occupy e absolute dominance, while e background traffic (namely, all kinds of network maintenance information under e idle state) occupies extremely low bandwid, so e network traffic rate at is time is mostly determined by e number of network users. That is, it can be considered at e network traffic rate is proportional to e number of users per unit time. The user behavior has a strong randomness. Based on e relevant eories of queuing eory [22], e network traffic is not in line wi Poisson distribution in e case of a large traffic, but e change in e number of users is a Poisson process wi an approximate parameter of λ, of which λ is e number of newly increased network users per unit time, at is, e changing rate of e number of users. According to e characteristics of e Poisson process [21], e time interval of newly increasing network users twice (namely, e time interval of two events) obeys e exponential distribution wi e mean value of 1/λ, and e user number per unit time is a reciprocal of is time interval, so e user number per unit time obeys e exponential distribution wi e mean value of λ. Assuming at e network traffic rate under e normal state is V norm, according to e proportional relationship between e number of network traffic rate under e normal state and user number per unit time, e network traffic rate under e normal state obeys e exponential distribution wi e mean value of Cλ, of which C>0 is e proportionality coefficient, namely E ( Vnorm ) = Cλ 3.3 Limit state probability According to e improved Semi-Markov network traffic model described in e Section 3.1, { X, n n T} is an embedded Markov chain of Semi-Markov process wi e change of network traffic, and its state space is I={idle, norm, busy}, and e state transition probability is P ( i, I ). According to e model state transition diagram I i ( m), set { m m 1, P > 0, I} is {2, 3, 4, } and its greatest common divisor is 1, so e Markov chain { X, n n T} is aperiodic. On e oer hand, all e states in e state space I are interoperable, so I is an irreducible closed set, and e Markov chain{ X n, n T} is irreducible. According to e literature [21], e irreducible aperiodic Markov chain under e finite state must be in e stationary distribution, so e embedded Markov chain { X, n n T} must be in e stationary distribution. Assuming at e stationary distribution is π, I, en { } π = π kpk, k I π = 1, π 0. I is e time distribution of e network traffic rate under e state, e time interval distribution of e state transition in e Semi-Markov model is Assuming at FH ( I ) F t = P T t X =, X = i, T = t t, i n n+ 1 n n n+ 1 n so FH ( t) = PkiF k ( t) k I, k The expected value (E(T)) of e network traffic rate under e state is + + = = E T tdfh t tdfh t 0 362

5 Assuming at T is e time interval of successively entering into e state twice, e Markov chain { X, n n T} is irreducible, so e improved Semi-Markov network traffic model of is paper is irreducible. According to e literature [21], if T has off-lattice distribution [21] and finite mean value, en e limit state probability (at is, when e network tends to be stable, e network traffic rate is e probability of each state) is k I ( ) π E T P = π k E Tk 3.4 Network bandwid utilization According to e characteristics of network traffic probability distribution under each state in e Section 3.2 and e limit probability of network traffic under each state in e Section 3.3, e entire network bandwid utilization BU can be easily solved when e network traffic is portrayed by e use of e improved Semi-Markov network traffic model based on Pareto distribution under e busy state. Assuming at e actual network bandwid is B a, en e value of BU is: E Vk BU = Pk B k I a 1 = Pbusyi B + Pidlei µ idle + Pnormi Cλ B 1 a Where, P k is e limit probability under e state k, E(V k ) is e expected value of e network traffic rate when e system is under e state k. The meaning of e remaining parameters in e formula is described in Section EXPERIMENTAL ANALYSIS The data set used in e experimental analysis is e public data set UNIBS [23] of Turin Polytechnic University. The data set is e traffic collection collected by a telecommunications network research group on e border router of campus network of Brescia University in ree consecutive days from September 30, The total amount of UNIBS data set is about 27GB. The school routers are connected to e Internet rough a dedicated 100Mbps link, so we take 1Mbps as a reshold value under e idle state, and take 10Mbps as a reshold value under e busy state. In is paper, on e one hand, we mainly examine e limit state probability and network utilization of e improved Semi-Markov model; on e oer hand, we need to examine wheer e characteristics of network traffic under e busy state are in line wi Pareto distribution proposed in is paper. For e characteristics of network traffic under e idle state and e normal state, its description in is paper is consistent wi at in e literature [15]. Therefore, is paper mainly concerns about e parameters µ idle and Cλ required in solving e subsequent network bandwid utilization, raer an experimental verification of e characteristics of traffic under ese two states. 4.1 Characteristics of network traffic under e busy state First, we process e raw data obtained. The specific process is as follows: Step 1: to divide UNIBS data set wi an interval of 1ms, and make statistics of e data size wiin each 1ms, at is, to calculate e data size wiin is time period according to e number of data packets wiin each 1ms, and e leng of each data packet; Step 2: to calculate e average traffic rate wiin each 1ms by e use of e data size obtained in e last step dividing1ms, and store e results; Step 3: to filter e data stored in e second step according to e reshold value under e busy state and e reshold value under e idle state, find out all e time periods of busy state, idle state and normal state, and e state transition relationship between each time period and subsequent time period, and respectively store e time period of ese ree states and e subsequent state transition relationship. After completion of e above processing steps of ree raw data, e random sampling meod can be used to extract 100,000 samples in e time period under e busy state, and record e size of traffic rate of each sample, and en divide e range of 10Mbps-100Mbps into 30 equal ranges, and make statistics of e number of samples wiin each range, and finally draw e results as a curve shown in Figure 2: Figure 2. Distribution diagram of network traffic rate under e busy state of UNIBS data set. It can be seen at e result is in line wi e dia- 363

6 gram of probability density function of e typical Pareto distribution. After curve fitting, e shape parameter is Limit state probability and network bandwid utilization The limit state probability π, and E ( T )( I ) By e use of total data stored after processing in e second step in e Section 4.1, on e one hand, it can make statistics of e total times of system state transition and e times of system transition from a state to anoer state, us solving e state transition probability of e imbedded Markov chain, Pi ( i, I ). On e oer hand, if e states in e adacent two time periods are e same, two time periods can be merged to one time period, and e state duration can be recorded as a sum of ese two time periods, and en we reconsider wheer e next adacent time period can be merged, and repeat is process until e states in any two adacent time periods are not e same, so at we can make statistics of e duration of each state in accordance wi such rules, and solve e time interval distribution of state transition in e Semi-Markov process. According to e Section 3.3, Pi and Fi ( t ), e stationary distribution of e embedded Markov chain { π, I} and e time expectation E ( T )( I ) of e system under a state can be obtained. P I can be calculated by e use of { I}. According to e above computing process, e stationary distribution, time expectation of e system under a state and e limit state probability obtained by e use of UNIBS data set are shown in Table 1: Table 1. Stationary distribution, duration expectation and limit probability under ree states of UNIBS data set. State Stationary distribution Duration expectation (Unit: s) Limit probability Idle Normal Busy By e use of data in Section 4.1 and random sampling analysis of e distribution of e network traffic rate under e idle and normal state of e system, e relevant state parameters can be easily obtained. In e experiment, e obtained parameter µ idle is Mbps, and Cλ is Mbps. Therefore, by e use of formula in Section 3.4 and data calculated above, e network bandwid utilization can be easily obtained, at is, 1 BU = Pbusyi B + Pidleiµ idle + Pnormi Cλ 21.56% B 1 a The actual statistic of experimental data used in is paper is 22.01%, indicating at e model of is paper can preferably simulate e actual state of e network traffic. 5 CONCLUSION This paper presents an improved Semi-Markov network traffic model and simplifies e state of network traffic to e idle state, normal state and busy state, us simplifying e complexity of e issue analysis. Based on e characteristics of heavy-tailed distribution for e size of e network traffic, is paper applies Pareto distribution for e analysis of e network traffic under e busy state. The experimental results show at, e distribution of e network traffic rate under e busy state can be well in line wi e characteristics of Pareto distribution, and e entire network bandwid utilization is also consistent wi e actual situation, us proving e effectiveness of e model proposed in is paper. Follow-up work can be analysis of e idle reshold value and busy reshold value from e perspective of experiment, as well as various parameters in is paper, and research of e impact of size of various parameters on e research results, in order to find out e meods of setting various parameters. ACKNOWLEDGEMENTS This paper is Supported by e National Science-technology Support Plan Proects (No. 2012BAH47B01); e Key Laboratory Open-end Fund of Information Assurance Technology (No. KJ ). REFERENCES: [1] Frost V S, Melamed B Traffic modeling for telecommunications networks. IEEE Communications Magazine, 32(3): [2] Greiner M, Jobmann M, Kluppelberg C Telecommunication traffic, queueing models, and subexponential distributions. Queueing Systems, 33(2): [3] Paxson V, Floyd S Wide Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, 3(3): [4] Leland W E, Taqqu M S, Willinger W, Wilson DV On e self-similar nature of Eernet traffic (Extended Version). IEEE/ACM Transactions on Networking, 2(1): [5] El Abdouni Khayari R, Sadre R, Haverkort B R A validation of e pseudo self-similar traffic model. International Conference on Dependable Systems and Networks, pp: [6] Flandrin P Wavelet analysis and synesis of fractional Brownian motion. IEEE Transactions on Information Theory, 38(2): [7] Riedi R H, Crouse M S, Ribeiro V J, et al A Multifractal Wavelet Model wi application to network traf- 364

7 fic. IEEE Transactions on Information Theory, 45(3): [8] Willinger W, Taqqu M S, Sherman R, et al Self-Similarity rough High-Variability: Statistical Analysis of Eernet LAN Traffic at e Source Level. IEEE/ACM Transactions on Networking, 5(1): [9] Liu J K, Shu Y T, Zhang L F, et al Traffic modeling based on FARIMA Models. IEEE Conference on Electrical and Computer Engineering, pp: [10] Shu Y T, Jin Z G, Zhang L F, et al Traffic prediction using FARIMA Models. IEEE International Conference on Communications (ICC), pp: [11] Dai Yunliang Research on Network Traffic Model Based on Multifractal Wavelet. Chengdu: University of Electronic Science and Technology. [12] Bai Jun, Xia Jingbo, Zhao Xiaohuan Self-similar network traffic prediction model based on EMD and RVM. Computer Science, 42 (01): [13] Zhang Jiukun Research on Network Traffic Flow Model and Traffic Forecasting Technology. Xi an: Xi an University of Electronic Science and Technology. [14] Cong Suo, Han Liangxiu, Liu Yan, Gao Chuanshan Multi-fractal network traffic model based on discrete wavelet transform. Journal of Communications, 24(05): [15] Huang Xiaolu, Min Yinghua, Wu Qi Semi-Markov model of network traffic. Journal of Computers, 28(10): [16] Taqqu M S, Willinger W, Sherman R Proof of a fundamental result in self-similar traffic modeling. Computer Communication Review, 27(2): [17] Willinger W, Alderson D, Doyle J C, et al More normal an normal: scaling distributions and complex systems. Proceedings of e 36 conference on Winter Simulation. Winter Simulation Conference, pp: [18] Korn F, Muukrishnan S, Wu Y H Modeling skew in data streams. SIGMOD Chicago: ACM, pp: [19] Downey A B Lognormal and pareto distributions in e Internet. Computer Communications, 28(7): [20] Chen Chu, Xu Yong, Zhang Ling Impact of heavy-tailed distribution on e nature of network traffic. Computer Application, 29 (06): [21] Sheldon M. Ross, Random Process. Gong Guanglu, translated. Beiing: China Machine Press. [22] Lu Chuanlai Queuing eory (2 nd edition). Beiing: Beiing University of Posts and Telecommunications Press. [23] UNIBS data sets [Online]