What Signal Do Packet-Pair Diperion arry? Xiliang Liu, Kaliappa Ravindran, and Dmitri Loguinov The ity Univerity of New York, New York, NY 6 xliu@gc.cuny.edu, ravi@c.ccny.cuny.edu Texa A&M Univerity, ollege Station, TX 77843 dmitri@c.tamu.edu Abtract Although packet-pair probing ha been ued a one of the primary mechanim to meaure bottleneck capacity, crotraffic intenity, and available bandwidth of end-to-end Internet path, there i till no concluive anwer a to whanformation about the path i contained in the output packet-pair diperion and how i encoded. In thi paper, we addre thi iue by deriving cloed-form expreion of packet-pair diperion in the context of a ingle-hop path and general burty crotraffic arrival. Under the aumption of cro-traffic tationarity and ASTA ampling, we examine the tatitical propertie of the information encoded in inter-packet pacing and derive the aymptotic average of the output packet-pair diperion a a cloed-form function of the input diperion. We how that thi reul different from what wa obtained in prior work uing fluid cro-traffic model and that thi dicrepancy ha a ignificanmpact on the accuracy of packet-pair bandwidth etimation. Index Term Stochatic Proce, Queuing Theory, Active Meaurement, Bandwidth Etimation. I. INTRODUTION Sending probing packet to meaure network path characteritic ha been a common practice in the Internet ince the late 98. There are two categorie of probing-baed meaurement: delay baed and diperion baed. In the firt category, path characteritic uch a per-hop capacity, queuing delay, and link utilization are inferred baed on the RTT or one-way delay of individual packet [], [6], [], [4], [2]. In the econd category, the diperion of packet-pair i traditionally ued to infer bottleneck capacity [3], [4], [5], [7], [], [3], [22], [23]; however, recent approache alo ue packet-pair/train to meaure cro-traffic and available bandwidth of an end-to-end path [2], [8], [9], [2], [8], [25], [26]. I traightforward to identify the information encoded in the delay of individual probing packet. Hence, delay-baed etimation technique are theoretically validated and meaurement difficultie are motly due to practical iue [2], [24]. On the other hand, i far more difficult to characterize the information contained in the output diperion of probing packet-pair. onequently, apart from the practical iue, diperion-baed meaurement technique are yet to be fully jutified for general cro-traffic condition. There ha been a fair amount of reearch effort to characterize the information encoded in packet-pair pacing. However, previou analyi either relied on contant-rate fluid crotraffic model [5], [7], or provided anwer only partially uitable for generic burty cro-traffic [3], [9], [2], [25]. In Supported by NSF grant R-36246, ANI-3246, NS-43494. thi paper, we provide a more accurate, yet concie characterization of packet-pair probing in the context of a ingle-hop path and non-fluid cro-traffic. We identify three tochatic procee related to cro-traffic arrival and how that packetpair probing eentially inpect the ample-path of thee three procee and contruct the output diperion ignal baed on their random ampling. We derive everal cloedform expreion to decribe thi contruction procedure and call our characterization of packet-pair probing the ampling and contructing model. Under the aumption of cro-traffic tationarity, we examine the tatitical propertie of the probing ignal encoded in inter-packet pacing and derive the aymptotic average of the output diperion a a cloed-form function of the input diperion. We how that the reult deviate from what wa previouly obtained uing contant-rate fluid cro-traffic and that thi deviation ha a ignificant advere impact on packetpair bandwidth etimation technique. We lit the terminology ued in the paper in Table I. In ection II, we ummarize related work of packet-pair analyi and report our earlier modeling attempt. In ection III, we introduce our ampling-and-contructing model to characterize packet-pair probing. Baed on our model, we examine the tatitical characteritic of encoded probing ignal in ection IV. We derive the aymptotic average of output diperion in ection V and how that deviation from the fluid reult ha an advere impact on packet-pair bandwidth etimation in ection VI. Finally, we preent our concluding remark in ection VII. A. Related Work II. BAKGROUND The earliet packet-pair analyi dated back to 988, when Jacobon [] examined the packet-pair pacing in the abence of cro-traffic and obtained the following reult = >. ) where and are the input and output pacing of the packetpair, repectively, i the probing packet ize, and i the bottleneck capacity of the path. Note that when the input pacing i mall, the output pacing contain information about. In real network, cro-traffic i often non-negligible. To take into account the effect of cro-traffic, Dovroli et al. [5]
2 TABLE I TERMINOLOGY Term Network path Hop capacity Bottleneck capacity Narrow hop ro-traffic intenity Hop available bandwidth Path available bandwidth Tight hop Encoded probing ignal Definition Sequence of interconnected FIFO tore and forward hop Tranmiion peed of the hop in bit per econd The minimum hop capacity along a network path The hop with the mallet capacity along a network path The average arrival rate of cro-traffic in ome time interval The hop reidual capacity after tranmitting cro-traffic in ome time interval The minimum hop available bandwidth along the path The hop with the minimum available bandwidth along a path The information contained in packet-pair output diperion and Melander et al. [7] tudied the relationhip between the packet-pair input and output rate uing a contant-rate fluid cro-traffic model. In a ingle-hop path, their reult can be ummarized a follow r r = r + λ r λ, 2) r r < λ where r = / and r = / are the input and output rate of packet-pair, repectively, λ i the cro-traffic intenity, and i again the hop capacity. Tranlating 2) into it pacing verion, we get = + λ λ > λ. 3) Thi how that when the input rate i higher than hop available bandwidth, there will be a determinitic multiplicative ignal λ/ and a determinitic additive ignal / encoded in the output pacing. Again note that thi reult embrace ) a a pecial cae when λ =. Applying mathematical induction to all hop along the path, we get the following relation between the input and output rate for an arbitrary multi-hop path r r = r + λ b r λ, 4) r r < λ where b i the econd minimum hop-available-bandwidth along the path, i the capacity of the tight hop, and λ i the cro-traffic intenity at the tight hop. Thi relation lead to the recent meaurement propoal TOPP [8], [6], which i a technique to infer available bandwidth and tight-link capacity. Realitic cro-traffic i alway burty and it intenity i never a time-invariant contant. Therefore, a natural quetion become how to generalize reult )-4) to accommodate burty cro-traffic. We ummarize the main reult of previou tudie next. To interpret hi Internet meaurement obervation of the probing packet RTT phae plot, Bolot 993) [3] adopted a ingle-hop path with burty cro-traffic in hi analyi. Bolot howed that the packet-pair diperion reflect the amount of traffic workload arrived at the router between the pair when the router doe nodle between their arrival. Hu et al. 23) [9] did a imilar analyi and propoed a pacing formula under the condition when the packet-pair hare the ame queueing period the ame condition ued in Bolot analyi) = + y, 5) where y i a random variable reflecting the cro-traffic intenity in the duration between the arrival of the probing pair. Both Bolot and Hu pointed out that when the input pacing i large enough, the output pacing, although random, become equal to on average. In other word, the input pacing i only contaminated by ome additive zero-mean ignal. Paztor et al. 22) [2] identified everal type of ignal encoded in the packet diperion. They differentiated packetpair falling into the ame router buy period from thoe that do not. In the former cae, a multiplicative ignal related to cro-traffic called ditribution ignature i encoded into the output pacing; while in the latter cae, an additive zero-mean white noie ignal called accumulation ignature i encoded. B. Our Earlier Attempt We now report our earlier attempt of packet-pair probing modeling inpired by conventional widom. Although it turned out to be of little ucce, a comparion of thi approach with the one preented later in thi paper help undertand the problem better. Baed on previou inight, we tried to interpret packet-pair probing in ingle-hop path with general cro-traffic uing the following ignal model = + y JQ, 6) + w DQ where w i a zero-mean random variable, y i a λ-mean random variable, and λ i the long-term average of cro-traffic intenity. JQ Joint Queuing) and DQ Dijoint Queuing) are the condition under which a packet-pair hare the ame router buy period or fall into different queueing period repectively. Thi characterization can accommodate the fluid model a a pecial cae where both random variable w and y degenerate to determinitic contant. Alo note that, in the fluid model, a packet-pair fall into the ame queuing period if and only if < / λ). In burty cro-traffic, i eay to verify that when < /, the JQ condition i alway atified. However, when > /, there i uually no determinitic relationhip between and the JQ condition. For any given > /, JQ
3 Fig.. p p y, + + w Our unifying ignal model inpired by previou work. = + y = + w become a random event that occur with certain probability. Thi can be chematically illutrated uing a unifying ignal model in Fig.. The unifying model how that the forwarding hop can be viewed a a tochatic mixture of two independent random ytem. With probability p, the input ignal pae through the JQ ytem, where multiplicative random ignal y/ and additive determinitic ignal / are tamped on. With probability p, the input ignal pae through the DQ ytem, where additive white noie ignal w i tamped on. Although thi characterization make intuitive ene, we found tha not accurate. We dicu the explanation in Section IV. III. THE SAMPLING AND ONSTRUTING NATURE OF PAKET-PAIR PROBING A. Formulation of ro-traffic Arrival In thi paper, we focu on ingle-hop packet-pair probing. We aume infinite buffer capacity, FIFO queuing, and a workconerving dicipline for the forwarding hop. For the compoite of cro-traffic and probing traffic, we aume imple traffic arrival, i.e., at mot one packet arrive at any time intance. For cro-traffic alone, we identify three ample-path which play crucial role in determining the nature of packet-pair probing. They are the ample-path of cro-traffic intenity proce, hop workload-difference proce, and available bandwidth proce. We next preent a rigorou formulation for thee three element and how the baic relationhip among them. Definition : ro-traffic i driven by the packet counting proce {Nt), t < } and the packet ize proce {S n, n < }. The cumulative traffic arrival {V t), t < } i a random proce counting the total volume of data received by the hop up to time intance t V t) = Nt) n= S n. Note that V t) and Nt) are right continuou, meaning that the packet arriving at counted in V t). Definition 2: We define {Y t), t < } a the proce indicating the average cro-traffic arrival rate in the interval t, t + ] V t + ) V t) Y t) =. and call it -interval cro-traffic intenity proce. In thi paper, packet ize and data are meaured in bit. The econd critical elemen hop workload-difference proce. Definition 3: Hop workload proce {Wt), t < } indicate the um at time intance t of ervice time of all packet in the queue and the remaining ervice time of the packen ervice. Definition 4: We define {D t), t < } a the proce indicating the difference between the hop workload at time t and t + D t) = Wt + ) Wt), and call it -interval workload-difference proce. The third important proce i the available bandwidth proce. Definition 5: Hop utilization proce {Ut), t < } i an on-off proce aociated with {Wt)} { Wt) > Ut) = 7) Wt) = and -interval hop idle proce It, t + ) = I t) = t+ t Ux)dx 8) i a proce indicating the total amount of idle time of the forwarding hop in [t, t+]. We further call time interval [t, t+ ] a hop buy period if I t) = and a hop idle period if It + ) It) =. Definition 6: We define {B t), t < } a the proce indicating the reidual bandwidth in the time interval [t, t+] B t) = t+ t Ux)dx ) = I t), 9) and call it -interval available bandwidth proce. The following theorem decribe the relationhip among the three important procee. Theorem : For all poitive t and, the following hold B t) = Y t) + D t). ) Proof: Note that the total hop idle time within the time interval [t, t + ] i I t) = B t). ) The amount of data tranmitted by the hop within the time interval [t, t + ] i V t + ) V t) + Wt) Wt + )) Thu, the hop working time i = Y t) D t). Y t) D t). 2) Since i the um of hop working time and hop idle time. Adding up ) and 2), we get = B t) D t) + Y t). 3) Rearranging 3), we get the deired reult.
4 a R a ) R a ) = lim t a +) W dt) = W d a 2 ) Intruion Reidual W d t t 2 t 3 t 4 t 5 time t 6 t 7 = Wa 2 ) Wa 2 ) = Wa 2 ) Wa 2 ). The lat equality i due to the imple arrival aumption. Since there i no cro-traffic packet arrival at time a 2 when p 2 arrive, we have Wa 2 ) = Wa 2 ). The term R a ) i the intruion reidual at time a 2 caued by the probing packet p and experienced by the packet p 2. In other word, the queuing delay of p 2 in the hop i given by Wa 2 ) = Wa + ) + R a ). 4) Fig. 2. Illutration of the intruion reidual function. A a direct reult of the obervation illutrated by Fig. 2, R a ) can be computed a follow 3 Packet-pair probing i eentially interacting with the ample-path of the procee we jut formulated. To prepare for the preentation of our main reult, we next examine certain detail about thi interaction. B. Probing Intruion of Packet-Pair We ue the triple a,, to denote a pair of probing packet p and p 2 of the ame ize. The firt element a in the triple i the arrival time of the packet p to the hop; i the inter-packet pacing; and i the probing packet ize. The arrival time of p 2 i a 2 = a +. The departure time of the probing packet from the hop are denoted by d and d 2. The output pacing i = d 2 d. In term of rate, the input and output probing rate are r = / and r = /. We ue Wt) and Ĩt) to denote the workload amplepath and the hop idle ample-path aociated with the uperpoition of cro-traffic and probing traffic. Note that traffic compoition only increae hop workload. Tha, for all t, Wt) Wt). Therefore, we define the following function to help undertand thi intruion behavior of packet probing. Definition 7: The intruive range of the probing traffic into Wt) i the et {t : Wt) > Wt)}. The intruion reidual function i W d t) = Wt) Wt). Let u next examine the propertie of function W d t) to undertand the intruion behavior of a ingle probing packet. Before the arrival of the probing packet, W d t) =. It get an immediate increment of / upon the packet arrival, where i the packet ize. In Wt) buy period, W d t) remain unchanged. In Wt) idle period, W d t) decreae linearly with lope until it become, which mark the end of the intruive range. Within the intruive range, W d t) i monotonically non-increaing. Fig. 2 illutrate thi behavior, from which we can infer that t, t 2 ), t 3, t 4 ) and t 5, t 6 ) are three buy period in Wt), wherea t 2, t 3 ), t 4, t 5 ), and t 6, t 7 ) are three idle period in Wt). Time intance t i the arrival time of the probing packet, wherea t 7 mark the end point of the intruive range 2. When Wt) i probed by a packet-pair a,,, we are intereted in the left-hand limit of W d t) at time a 2, denoted 2 Note that the probing packet depart before t 7. ) ) + + R a ) = I B a ) a ) =. 5) We are alo intereted in computing Ĩa ) when the hop i probed by packet-pair a,,, which, from the intruion behavior decribed in Fig. 2, can be expreed a following Ĩ a ) = I a ) ) + B a ) = ) +. 6) Notice that between the two term R a ) and Ĩa ), there i at mot one poitive term for any given a. When R a ) >, the two packet in the pair hare the ame hop buy period and Ĩa ) =. When Ĩa ) >, the two packet fall into different hop buy period and R a ) =. Hence, the poitivene of the two term correpond to JQ and DQ condition repectively. We are now ready to derive the relation between the input pacing and the output pacing for any individual packetpair. Thi relation i a miletone of our packet-pair analyi.. Output Packet-Pair Diperion We firt preent a corollary, which i due to the workconerving aumption. orollary : For any packet arriving into the hop at time t and departing from the hop at time t 2, the time interval [t, t 2 ] i a hop buy period. Thi corollary immediately lead to the following lemma Lemma : When a hop i probed by a packet-pair a,,, we have Ĩd, d 2 ) = Ĩa, a 2 ). Proof: Firt, due to corollary, we have Ĩa, d ) = Ĩa 2, d 2 ) =. 7) Further, notice that Ĩa, d 2 ) can be expreed in the following two way Ĩa, d 2 ) = Ĩa, d ) + Ĩd, d 2 ) = Ĩd, d 2 ) 8) Ĩa, d 2 ) = Ĩa, a 2 ) + Ĩa 2, d 2 ) = Ĩa, a 2 ) 9) ombining 8) and 9), we have Ĩd, d 2 ) = Ĩa, a 2 ). 3 I cutomary to denote maxx, ) uing X + and call it the poitive part of X.
5 Our next theorem expree the output pacing of a packetpair from two different angle. Theorem 2: When Wt) i probed by a packet pair a,,, the output pacing can be expreed a ) + = Y a ) + + B a ) ) + B a ) = + D a ) +. 2) Proof: We examine the hop activity with repect to Wt) within the time interval [d, d 2 ]. Notice that / time unit are pent on erving probing packet p 2 and that V a 2 ) V a ) = Y a ) 2) time unit are pent on erving the cro-traffic that ha arrived to the hop during the time interval a, a 2 ). Thu, the total hop working time in [d, d 2 ] i given by Y a ) +. 22) Alo notice that Ĩd, d 2 ) i the total idle time of the hop during thi time interval. Since the um of the hop working time in 22) and hop idle time mut be equal to d 2 d, we immediately have the following = d 2 d = Y a ) + + Ĩd, d 2 ). 23) Further, due to Lemma and 6), we get ) + Ĩd, d 2 ) = Ĩa, a 2 ) = Ĩa B a ) ) =. 24) Subtituting 24) back to 23), we proved the firt equality in 2). For the econd part of 2), firt notice that the total delay of p and p 2 at the hop are given by d a = Wa ) + 25) d 2 a 2 = R a ) + Wa 2 ) +. 26) Subtracting 25) from 26), we get = + R a ) + D a ). 27) Subtituting 5) into 27), we get the econd half of 2). The mot alient feature of Theorem 2 i that the reult i almot unconditional, in the ene that neither relie on any aumption on cro-traffic arrival pattern nor impoe any retriction on the input ignal. In addition, thi reult enforce uch a conceptual idea that packet-pair probing can be viewed a a ampling-and-contructing procedure a illutrated in Fig. 3. The packet-pair a,, i eentially ampling the three ample-path Y t), D t), and B t) at the time point a and then contructing the output ignal uing the three ample baed on 2). Although 2) how two different way of contructing the output ignal, they both produce the ame reult. We urely can take advantage of Theorem and rewrite 2) in a form involving only two procee e.g., Y t) and D t)). However, the preent verion i more intuitive and make later analyi eaier. Our characterization already hed Fig. 3. a,, a 3 random ample contruction,, t Y t) D t) B t) The ampling-and-contructing nature of packet pair probing. light on what the encoded probing ignal are and how they are encoded. It alo allow invetigation of their tatitical nature from an analytical angle rather than experimental obervation. IV. THE STATISTIS OF ENODED PROBING SIGNALS We ue {T n, n < },, to denote an infinite equence of packet-pair probing driven by a point proce Λt) = max{n : T n t}. We ue n to denote the output pacing of the n-th packet-pair T n,, in the probing equence. Adjacent packet-pair are ufficiently eparated, meaning that we neglect the cae where a packetpair fall into the intruive range of the preceding pair. onequently, the ampling and contructing model hold for all pair in the probing equence. Thi i a practically valid implification becaue meaurement tool [8], [26] all devie the inter-probing delay much larger than o a to keep the average probing traffic intenity mall. Given thi dicuion, the packet-pair equence will generate three dicrete-time ample-path of encoded probing ignal EPS) ample: Y T n ), D T n ), and B T n ). We now invetigate the tatitical propertie of thee EPS ample-path. A. Baic We introduce a concept imilar to probability ditribution called frequency ditribution to characterize ample-path tatitic. For the detail of thi concept, pleae refer to [9, page 46-5]. Definition 8: For continuou-time ample-path Xt), define indicator function Ψx, t) { Xt) x Ψx, t) = Xt) > x. 28) The frequency ditribution function Px) of Xt) i defined a follow auming the limit exit for x R) Px) = lim τ τ τ Ψx, t)dt. 29) For dicrete-time ample-path X n, define indicator function Ψx, n) { Xn x Ψx, n) = X n > x. 3)
6 The frequency ditribution function Px) of X n i defined imilarly k Px) = lim Ψx, n). 3) k k n= In the pirit of the probabilitic mean, we have continuoutime and dicrete-time ample-path mean a follow 4 E[Xt)] = E[X n ] = τ xdpx) = lim Xt)dt, τ τ k xdpx) = lim k k i= X i. To examine the tatitic of EPS ample-path, we impoe the another ASTA Arrival See Time Average) property on the ampling proce {T n }. ASTA guarantee the equality of the tatitic or frequency ditribution) in ampled erie to the correponding tatitic in the continuou-time amplepath being ampled [5]. With non-negligible ASTA bia, ampling-baed etimation uually fail to reach the meaurement target and not much i left for dicuion. The fact that quite a few current meaurement technique perform well without factoring ASTA into their deign ugget that ASTA bia i either not preent or i negligible in the tudied Internet environment. B. ro-traffic Aumption and Implication Under the ASTA aumption, our focu i now hifted to the ample-path tatitic of Y t), D t), and B t), which again are dependent on the probabilitic propertie of the underlying cro-traffic arrival proce. We make an ergodic tationarity aumption on cro-traffic arrival and invetigate it implication on thee ample-path. Aumption : The cumulative traffic arrival proce {V t)} ha ergodic tationary increment, i.e., for any poitive, the proce {Y t)} i an ergodic tationary proce with enemble mean λ, which i le than the hop capacity. Thi aumption impoe two retriction on the proce {Y t)}. Firt, the tationarity aumption implie that Y t) ha identical ditribution for all t. onequently, {Y t)} i alo a wide ene tationary proce 5. Second, the ergodicity aumption implie that, at any time intance t, the variance of Y t) decay to a increae. Due to Szczotka [27], [28], the hop workload proce {Wt)} will inherit the ergodic tationarity property from the traffic arrival proce. Alo becaue of the definition of workload-difference proce and Theorem, thi property i further carried over to proce {D t)} and {B t)}, whoe enemble mean are and λ repectively. The following lemma tate the implication of Aumption on the three ample-path under invetigation. They are all immediate conequence of ergodic tationarity. 4 We ue E[.] to denote the ample-path mean, or limiting time-average, intead of the enemble mean. 5 Auming econd order moment exit. Lemma 2: For any poitive, the ample-path mean of Y t), D t), and B t) are given by E[Y t)] = lim τ τ E[D t)] = lim τ E[B t)] = lim τ τ τ τ τ τ Y t)dt = λ 32) D t)dt = 33) B t)dt = λ. 34) Finally, ergodicity alo implie that the variance of B t) decay when the obervation interval become large and that thi decaying variance will be reflected on the ample-path frequency ditribution. Hence, we have the following lemma, which i intuitive and we kip the formal proof. Lemma 3: Under the aumption of the paper, the frequency ditribution function P x) of ample-path B t) approache the following tep function a get large { x < λ P x) = x λ. 35) We point out that the ergodic tationarity aumption i not a neceary condition for thee ample-path propertie. We conjecture that traffic with aymptotic tationarity alo exhibit the ame propertie including traffic driven by regenerative on-off arrival procee). However, the main goal of the paper i not to identify the weaket traffic condition that give u the deirable ample-path propertie, but rather to make a realitic traffic aumption and tudy it implication on bandwidth meaurement technique. A recent tudy howed that Internet traffic can be well modeled a a tationary proce on the timecale of hour [29]. Therefore, we tarted by auming traffic tationarity; however, later reult in thi paper are all derived baed on the ample-path propertie. They are expected to have broader applicability than thoe limited to tationary cro-traffic.. Unifying Signal Model Reviited Thank to ASTA, the three dicrete-time ample-path generated by packet-pair ampling have the following limiting time-average E[Y T n )] = λ E[D T n )] = E[B T n )] = λ. 36) Alo recall that a large obervation interval reduce the pread of the frequency ditribution for Y T n ) and B T n ) 6. We now reviit the unifying model preented in Section II and explain why i not valid. Recall that when the input ignal i very large, the ditribution of B T n ) i concentrated around λ. The poitive-part term R T n ) in the econd equality of model 2) become almot contantly. Hence, the additive zeromean term D T n ) become eaily detected from the output ignal ample n. However, thi doe not mean that there i 6 Unlike B T n) and Y T n), D T n) i not a moving average proce by nature. onequently, the frequency ditribution of D T n) gradually become inenitive to the increae of.
7 no cro-traffic intenity information captured in n. The firt equality of 2) how that Y T n ) i alway ampled by the packet-pair. I only becaue of the trong noie term ĨT n ) that the deired ignal i undetectable baed on the obervation of n. When input ignal < /, the poitive-part term ĨT n ) in the firt equality of model 2) i. Hence, additive ignal / and multiplicative λ-mean ignal Y T n ) can be eaily detected from the tatitic of n. However, the additive zeromean ignal D T n ) doe not ecape packet-pair ampling. It only become undetectable from the tatitic of n due to the trong noie term R T n ). When i neither large nor mall, both poitive-part term become prominent noie. Neither the additive zero-mean ignal nor the multiplicative λ-mean ignal can be eaily detected. However, they are alway captured by probing ampling and encoded in the output pacing. Thi i exactly the ubtlety that ecaped the unifying model 6). The unifying model how that n carrie a ample of the zero-mean ignal only under DQ condition when ĨT n) > ). However, the truth i that the output pacing alway carrie a ample of D T n ) regardle of the condition i under. I the whole et of ample D T n ) that exhibit additive zero-mean nature, not the et of ample under the DQ condition. Similarly, i the whole et of ample Y T n ) that ha a mean equal to λ, not jut the et of ample under the JQ condition. Thi explain why the unifying ignal model i not accurate. V. ASYMPTOTIS IN PAKET-PAIR SAMPLING One important application of our packet-pair probing model 2) i that allow immediate derivation of the aymptotic average of probing output pacing a a cloed-form function of the input pacing. Thi functional relation, which we call the gap repone curve, erve a a theoretical foundation for bandwidth meaurement tool uch a TOPP [8] and Spruce [26]. Previou derivation aumed contant-rate fluid crotraffic and the reult given in 2)-3). In thi ection, we reviit thi problem under the condition of ergodic tationary crotraffic arrival, which i clearly more realitic than fluid traffic. A. Aymptotic Average of Note that the output pacing alo form a dicrete-time ample-path n. The aymptotic average of i jut the ample-path mean of n. Tha, n E[ n ] = lim i. 37) n i= Our next theorem provide a cloed-form expreion for E[ n ]. Theorem 3: When a hop i probed by ASTA packet-pair equence {T n },, and auming that -interval available bandwidth ample-path B t) ha frequency ditribution function P x), then the following hold E[ n] = λ + + = + / x dp x) x dp x). 38) E[R t)] E[Ĩt)] λ α Fig. 4. The evolving trend of E[Ĩt)] and E[R t)] with repect to while keeping contant. Proof: Recall Theorem 2 and notice that due to ASTA property, the frequency ditribution of B T n ) i alo P x). Alo note that B T n ) i only ditributed in [, ]. Therefore, [ B ) ] + T n ) x E = / dp x) [ ) ] + B T n ) / x E = dp x) ombining thee reult with 2) and 36), we have the tatement of the theorem. To help undertand thi reult, we examine the two integral term E[Ĩt)] and E[R t)] in 38). Fig. 4 plot the evolving trend of the two term with repect to while keeping contant. A een in the figure, E[R t)] how a monotonically decreaing tend a increae and at ome point α it become or practically negligible. The other term E[Ĩt)] remain for / and then how a monotonically increaing trend, aymptotically approaching a linear function of. The two curve interect at the point = / λ). Thee reult can be ummarized into the following two et of formula E[Ĩt)] = < E[Ĩt)] = E[R t)] = λ E[R t)] = λ < lim E[Ĩt)] = λ lim E[R t)] = lim / E[Ĩt)] =. 39). 4) All thee propertie are provable baed on the lemma we had in the previou ection. We next prove one of them and leave the verification of the other to the reader. Theorem 4: The term E[R t)] i a continuou and monotonic decreaing function of in the range, ). It converge to a increae. Proof: Firt, note that for any > and t, we have R t) R + t). 4)
8 Aymptotic Average of Output Diperion E[ ] Fluid Lower Bound Gap Repone urve /, onet point of curve deviation α, end point of curve deviation /-λ), maximum deviation point /E[ ] Fluid Upper Bound Rate Repone urve /α, onet of deviation, end point of deviation -λ, maximum deviation point Input Packet-pair Spacing Input Rate r Fig. 5. Illutration of the gap repone curve. Fig. 6. Illutration of the rate repone curve. Thi difference define a new ample-path, whoe timeaverage can be computed a following which can be rewritten a E[R t) R + t)], 42) E[R t)] E[R + t)]. 43) Thi how that E[R t)] i a monotonic decreaing function of. Further, by taking the limit of 43) when, we have lim E[R t)] E[R + t)]) =. 44) Thi prove the continuity of E[R t)] with repect to in the range, ). Next, we how it convergence to a increae. Firt, note that lim E[R t)] = lim = lim dp x) lim Note that the firtem in 45) i zero 7 lim x dp x) x dp x).45) dp x) = lim P ) = P ) =, 46) and the econd item in 45) i alo zero lim = lim x dp x) < lim dp x) P ) = P ) =. 47) Hence, the limit of E[R t)] when i zero. B. Deviation From Fluid Model Note that the real gap repone curve 38) i different from the fluid model 3). In fact, a chematically howed in Fig. 5, the fluid model i a lower bound of the real curve. In the input diperion range /, α), the real curve poitively deviate 7 Recall that P i the tep function given in 35). from thi lower bound and reache the maximum deviation at the point = / λ), where the input rate i equal to the available bandwidth. Thi deviation i alo illutrated by the curve in the hadow area of Fig. 4 and can be expreed a x dp x) λ / x dp x) λ. 48) In contrat, note that uing the unifying ignal model 6), we can not foreee uch a deviation phenomenon. Epecially at the point = / λ) where the amount of deviation i maximized, the unifying model would, however, predict no deviation at all. It help to identify the exact value of α, which repreent the end point of the deviation range. Note that α i the minimum input pacing that caue a zero intruion reidual α = inf{ : E[R t)] = } = inf{ : P ) = }. 49) Thi require that B t) be greater than the input probing rate / for almot every t along the time axi. In other word, the input rate / mut be maller than the ditribution lower bound of -interval available bandwidth. I often not poible to atify uch a condition exactly 8, ince the convergence of P x) to the tep function in 35) might only be aymptotic and P x) may remain poitive for all x, ] regardle of the obervation interval. In that cae, α = and we can only mark it approximately a a point where the deviation become practically negligible. I often more informative to look at the rate verion of the repone curve rather than the pacing verion, becaue the rate repone curve ha a direct aociation with crotraffic intenity and available bandwidth. Plotting /E[ n ] with repect to r = / and comparing it with the fluid model 3), we get Fig. 6. The fluid rate model become an upper bound of the real repone curve. The input rate range /α, ) become the area where the real curve negatively deviate from the fluid model. 8 For intance, when cro-traffic arrival i Poion.
9. Impact of Probing Packet Size How doe the packet ize affect the repone curve or the amount of deviation from fluid model? We firt conider the cae when input rate r < λ. Notice that the econd expreion in 48) can be implified to the following form x dp x) = = r dp x) Applying integration by part, we get xdp x) = rp r) Subtituting 5) back to 5), we get x dp x) = r x dp x) ) xdp x). 5) P x)dx. 5) P x)dx. 52) When while keeping r contant, the ampling interval = /r approache infinity proportionally. Hence, we have lim x dp x) = lim P x)dx. 53) Dropping the contant /, we get a ufficient and neceary condition for the amount of deviation anput rate r < λ to vanih when lim P x)dx =. 54) Similarly, for any input rate r > A, a ufficient and neceary condition for the deviation to vanih i ) lim r r P x)dx =. 55) Thee condition require the ample-path ditribution of B t) not only to exhibit decaying variance when the obervation interval become large, but alo to how ufficient decaying peed. Our experimental obervation o far how that thee propertie are uually atified. The problem of identifying cro-traffic type in which 54) or 55) i violated remain open. Even though large probing packet ize uually implie le deviation of the real repone curve from the fluid model, we point out the exitence of uch intereting cae where certain probing packet ize can lead to a repone curve with no derivation at all. Thi happen only when cro-traffic exhibit periodic arrival pattern. Auming the workload ample-path Wt) i a periodic function that repeat itelf every T time unit, then etting probing packet ize to T λ) caue the real repone curve to coincide with the fluid model. In practice, cro-traffic i rarely periodic and we have to rely on large packet ize to reduce the amount of repone curve deviation. However, due to the limit of path MTU and the concern of packet fragmentation, probing packet ize can not be made arbitrarily large. The quetion become whether the commonly ued MTU of 5 byte i enough to reduce the amount of curve deviation to uch an extent that impact on bandwidth meaurement accuracy become inignificant? In what follow, we how that even for relatively mooth e.g., Poion) cro-traffic, the deviation phenomenon till can have ignificant advere impact on bandwidth etimation. VI. THE IMPAT OF RESPONSE URVE DEVIATION ON BANDWIDTH MEASUREMENT A. omputing Repone urve We devie an off-line algorithm to compute the ingle-hop repone curve baed on cro-traffic packet arrival trace, the probing packet ize, and the hop capacity. The trace file provide information regarding the arrival time and packet ize for every cro-traffic packet. Given a trace file with ufficiently long time interval recorded, the frequency ditribution of the aociated ample-path uch a Y t), D t), and B t)) in that finite time interval become good approximation of their limiting frequency ditribution. Our off-line algorithm approximate the ample-path mean of n for any given input pacing. Next, we briefly explain the pirit of thi algorithm. We ue Υ τ to denote a cro-traffic trace in the time interval [, τ]. Given Υ τ and hop capacity, the hop workload ample-path Wt) in the interval [, τ], denoted a Wt) τ, can be computed. The following corollary tate the function propertie of workload ample-path. orollary 2: Hop workload ample-path conit of alternating buy period and idle period. Any buy period comprie piece-wie linear egment with lope. Taking advantage of thee functional propertie and uing a proper data tructure, we can repreent Wt) τ without loing any information about the original proce. From Wt) τ, we are able to retrieve Y t), D t) and B t) for any n [, τ ]. In other word, we keep the full information about Y t) τ, D t) τ, and B t) τ in the data tructure of Wt) τ. Intead of approximating E[ n ] uing a finite number of output diperion ample, we approximate E[ t)], the correponding continuou-time ample-path mean, uing the time average of t) in a finite interval. Note that due to the ASTA aumption, E[ n] = E[ t)]. Hence, a good approximation of the latter ample-path mean alo erve a a good approximation for the former. The continuou-time ample path t) alo ha certain nice propertie a we tate in the next theorem. The proof i in contructive term, which provide a concrete idea of how our off-line algorithm i deigned. Definition 9: Event-point are the time intance at which the workload ample path witche from a buy period to an idle period or undergoe a udden increment due to packet arrival. An epoch i a time interval between two adjacent event-point. Theorem 5: The ample-path t) conit of piece-wie linear egment with poible lop, and. For any two time intance < t < t 2, t) i continuou in the interval t, t 2 ) given that t and t 2 fall into the ame epoch of Wt), and that t + and t 2 + alo fall into the ame epoch of Wt).
.5 Output rate /E[ ] mb/) 8 6 4 2 Fluid rate repone curve Rate curve meaured in imulation Rate curve computed off-line 2 4 6 8 2 4 Input rate r mb/) The ratio between input and output rate r/r.4.3.2. Real curve computed off-line Real curve meaured in imulation TOPP-tranformed fluid curve.9 2 4 6 8 2 Input rate rmb/) Fig. 7. A comparion between the fluid curve of prior work, the curve computed offline, and the one meaured in imulation. Fig. 8. Tranformed curve ued by TOPP. Proof: See the Appendix. Our algorithm compute t) τ, the ample-path t) in time interval [, τ ], baed on Υ τ,, and. Taking advantage of Theorem 5, we can repreent the ample-path information of t) τ to it full preciion in a proper data tructure. We then compute the following 9 t) τ = τ and ue it a an approximation to E[ t)] = lim τ τ τ τ t)dt, 56) u)du. 57) I clear that the preciion of thi approximation i mainly decided by τ. Thu, we can pick a large τ o that further increae of τ would make little difference. It could be ometime impractical to have uch a long trace. However, note that even when 56) i not a good approximation of E[ t)], it till repreent the correct reuln a hypothetical periodic cro-traffic that repeat itelf after every τ time unit. Thi i due to the fact than periodic cro-traffic, ample-path t) ha a limiting time-average equal to it time average in one period. In our experiment, we ue a ingle-hop path with capacity = mb/. We ue Poion cro-traffic with average arrival rate of 5 packet per econd. The packet ize i 75 byte. Hence, λ = 3. mb/. The probing packet ize i choen to be 5 byte. We compute E[ t)] for 3 input value of in [.86 m, 2 m], which correpond to 3 equally paced input rate in the range of [. mb/, 4 mb/]. We generate a -econd cro-traffic trace and find that the time average of t) in time interval [, 3] ha ufficiently converged. Even for the mallenput =.86 m, doubling the trace interval to 6 doe not produce more than % difference. To validate our algorithm, we alo ue the ame etting in NS2 imulation to meaure the repone curve. For every input rate, the ender tranmit packet-pair. The inter-probing delay i controlled by an exponentially ditributed random variable to meet the ASTA ampling condition. The average 9 Theorem 5 alo allow an efficient computation of 56) with high accuracy. inter-probing delay i choen to be. We ue the average of the output diperion to approximate E[ n]. Fig. 7 how both the rate curve computed off-line and the rate curve meaured in NS2 imulation. The figure how that the two agree and non-negligibly deviate from the fluid upper bound. The deviation i very clear even though we ue the larget probing packet ize. The figure alo how that we can obtain a much moother curve uing off-line computation than uing NS2 imulation. Next, we dicu the implication of our finding on two packet-pair bandwidth meaurement technique: TOPP and Spruce. B. TOPP We firt conider a ingle-hop path. TOPP ue the fluid rate model 2) a it meaurement rationale. It firt collect the output rate r = / for a erie of equally paced input rate in ome interval [r min, r max ]. TOPP then tranform the meaured rate repone curve to a function between r/ r and r, which admit the following piece-wie linear relation in fluid model: { r < λ r/ r = r + λ r λ. 58) TOPP identifie the econd egmenn the curve and applie linear regreion to calculate the capacity and cro-traffic intenity λ. We tranform both the computed and the meaured rate repone curve in Fig. 7 to the form of 58). A howed in Fig. 8, the deviation range appear a the econd egment. When applying linear regreion on thee egment, we get etimation reult in Table II. The table how that even if TOPP could manage to get the aymptotic rate curve, it would not achieve an accurate meaurement due to it unawarene of the deviation phenomenon. In a ingle-hop cae, note that the real curve agree with fluid model when r. Therefore, linear regreion can be applied to thi curve portion to extract the capacity We ue to denote a meaurement of E[ ] uing finite number of ample of.
TABLE II TOPP ESTIMATION RESULTS IN MB/S) λ λ Real value 3. 7. TOPP in n2 imulation 35.97 32.33 3.64 TOPP in off-line computation 35.8 32.38 3.43 average of output packet-pair diperion in a ingle-hop path, which extend previou fluid model and erve a a theoretical foundation for packet-pair bandwidth etimation. In our future work, we will apply thee reult to analyze the robutne of current available bandwidth meaurement technique to multihop queuing interference. and cro-traffic intenity λ. In a multi-hop path, unlike what can be eaily derived from the fluid model a howed in 4), the real curve become very complex and i not amenable to cloed-form analyi. However, baed on the ingle-hop reult, we can predict that one neceary condition for the curve portion that contain the information of λ and to urvive againt multi-hop interference i that the available bandwidth for all pre-tight link mut be ignificantly higher than the tight link capacity. Otherwie, when the input rate i greater than, it fall into the deviation range of ome pretight link and the probing rate get reduced on average before the packet-pair arrive at the tight hop.. Spruce Spruce aume a ingle bottleneck link whoe capacity can be etimated beforehand. Spruce end probing pair with intra-pair pacing et to /, which i the bottleneck link tranmiion delay of the probing packet. Inter-pair delay i controlled by an exponentially ditributed random variable to meet ASTA property. Each probing pair generate an available bandwidth etimate by computing A n = n ). 59) Spruce average the lat ample to obtain a final etimate for the path available bandwidth. Our next theorem tate the unbiaedne of Spruce etimator Theorem 6: Under the aumption of thi paper, in a ingle-hop path, we have E [ n )] = λ. 6) Proof: ombining 38) and the firt equality in 39), thi theorem i eaily proved. In a multi-hop path, the pre-bottleneck link introduce noie to the input ignal, which hift it from a contant to a random variable. The pot-bottleneck link introduce noie to. Thi impact can be minimized if both noie ignal are zero-mean, which happen when the non-bottleneck hop have available bandwidth ignificant higher than the bottleneck capacity. A detailed analyi of pruce robutne to multihop queueing interference require future work, for which the inight of thi paper will be helpful. VII. ONLUDING REMARKS In thi paper, we identified three important ample-path related to cro-traffic arrival and etablihed the amplingand-contructing model to characterize packet-pair probing. Thi approach uncover the full picture of encoded probing ignal and lead to a cloed-form olution to the aymptotic REFERENES [] K. G. Anagnotaki, M. B. Greenwald, and R. S. Ryger, cing: Meauring Network-Internal Delay uing only Exiting Infratructure, IEEE INFOOM, April 23. [2] S. Banerjee and A. Agrawala, Etimating Available apacity of a Network onnection, IEEE International onference on Network, Sept. 2. [3] J. Bolot, haracterizing end-to-end packet delay and lo in the internet, AM SIGOMM, 993. [4] R. arter and M. rovella, Meauring Bottleneck Link Speed in Packet- Switched Network, Internation Journal on Performance Evaluation, 2728, 996. [5]. Dovroli, P. Ramanathan, and D. Moore, What Do Packet Diperion Technique Meaure?, IEEE INFOOM, April 2. [6] A. B. Downey, Uing Pathchar to Etimate Internet Link haracteritic, AM SIGOMM, 999. [7] K. Harfouh, A. Betavro, and J. Byer, Meauring Bottleneck Bandwidth of Targeted Path Segment, INFOOM, March 23. [8] G. He and J. Hou, On Exploiting Long Range Dependence of Network Traffic in Meauring ro Traffic on an End-to-end Bai, IEEE INFOOM, March 23. [9] N. Hu and P. Steenkite, Evaluation and haracterization of Available Bandwidth Probing Technique, IEEE JSA, Augut 23. [] V. Jacobon, ongetion Avoidance and ontrol, AM SIGOMM, 988. [] V. Jacobon, Pathchar A Tool to Infer haracteritic of Internet Path, 997. Slide available from ftp://ftp.ee.bl.gov/pathchar/mritalk.pdf. [2] M. Jain and. Dovroli, End-to-end available bandwidth: meaurement methodology, dynamic, and relation with TP throughput, AM SIGOMM, Augut 22. [3] K. Lai and M. Baker, Meauring Bandwidth, IEEE INFOOM, 999. [4] K. Lai and M. Baker, Meauring link bandwidth uing a determinitic model of packet delay, AM SIGOMM, 2. [5] B. Melamed and D. Yao, The ASTA property, Advance in Queueing: Theory, Method and Open Problem, page 95 224, 995. [6] B. Melander, M. Bjorkman, and P. Gunningberg, A New End-to-End Probing and Analyi Method for Etimating Bandwidth Bottleneck, IEEE Globecom Global Internet Sympoium, November 2. [7] B. Melander, M. Bjorkman, and P. Gunningberg, Firt-ome-Firt- Served Packet Diperion and Implication for TP, IEEE GLOBE- OM, 22. [8] B. Melander, M. Bjorkman, and P. Gunningberg, Regreion-Baed Available Bandwidth Meaurement, SPETS, July 22. [9] E. Muhammad and S. Stidham. Sample-path Analyi of Queueing Sytem. Kluwer Academic Publiher, 999. [2] A. Paztor and D. Veitch, Active Probing uing Packet Quartet, AM SIGOMM IMW, 22. [2] A. Paztor and D. Veitch, On the cope of end-to-end probing method, IEEE ommunication Letter, 22. [22] A. Paztor and D. Veitch, The packet ize Dependence of packet pair like method, In Proceeding IWQoS2, 22. [23] V. Paxon, End-to-end Internet packet dynamic, IEEE/AM Tranaction on Networking, 73):277 292, 999. [24] R. S. Praad,. Dovroli, and B. A. Mah, The effect of layer-2 toreand-forward device on per-hop capacity etimation, IEEE INFOOM, 23. [25] V. Ribeiro, M. oate, R. Riedi, S. Sarvotham, B. Hendrick, and R. Baraniuk, Multifractal ro Traffic Etimation, IT Specialit Seminar on IP Traffic Meaurement, September 2. [26] J. Strau, D. Katabi, and F. Kaahoek, A meaurement tudy of available bandwidth etimation tool, AM IM, 23. [27] W. Szczotka, Stationary repreentation of queue. I., Advance in Applied Probability, 8:85 848, 986. [28] W. Szczotka, Stationary repreentation of queue. II., Advance in Applied Probability, 8:849 859, 986.
2 [29] Y. Zhang, N. Duffield, V. Paxon, and S. Shenker., On the contancy of Internet Path Propertie, AM SIGOMM Internet Meaurement Workhop, November 2. APPENDIX I PROOF OF THEOREM 5 Proof: We approach the proof in three tep. In the firt tep, we prove the econd tatemenn the theorem. Tha, we how that t) i continuou in the time interval t, t 2 ). Recalling Theorem 2, we have t) = Y t) + + Ĩt). 6) The firt two term on the right hand ide of 6) are contant with repect to t becaue of the following Y t) = V t + ) V t) = Y t ) V t) V t )) + V t + ) V t + )). Since t, t] i within a workload epoch, there i no crotraffic arrival during thi interval. Hence V t) V t ) =. The ame i alo true in the interval t +, t + ] and we have V t + ) V t + ) =. Therefore, Y t) = Y t ) i a contant with repect to t. The third term Ĩt) on the right hand ide of 6) i obviouly a continuou function of t. Hence t) i continuou in t, t 2 ). In the econd tep, we how that t) τ can be plinto a erie of conecutive continuou egment. Tha, t) τ = n i= t) ti+, 62) where t =, t n+ = τ, and < + for all i n. We now prove thi reult by contructing uch a partition. Let e be the firt event-poinn Wt) after t and e 2 be the the firt event-poinn Wt) after t +. Then t) i continuou in the interval t, mine, e 2 )) due to the reult proved in the firt tep. Suppoe that τ mine, e 2 ), then etting n = and t n+ = t 2 = τ, the partition i accomplihed. Otherwie, let t 2 = mine, e 2 ) and proceed with the partition of t) τ t 2 recurively. Thi eventually plit t) τ into the form of 62). In the third tep, we how than any continuou egment t) ti+, t) i a piece-wie linear curve with poible lope,, and. We prove thi reult by recurively identifying all the linear egment in the interval, + ). We denote by e and e 2 the firt event-point in Wt) after and + repectively. In what follow, we dicu the identification procedure in ix poible cae. In the firt cae when W ) > and W + ) >, let t = mine, e 2, + ). Notice that for any < t < t, I ) = I t) and conequently Ĩ ) = Ĩt). ombining the proof in the firt tep, we have ) = t), which mean that t) t i a linear egment with lope. In the econd cae when W ) = and W + ) =, let t = mine, e 2, + ). Uing a imilar argument, i eay to ee that t) t i alo a linear egment with lope. In the third cae when W ) >, W + ) =, and I ) /, let t = mine, e 2, + ). Notice that for any < t < t, we have I t) = I ) + t ). 63) Since I ) /, we get Ĩ ) and Ĩ t) = Ĩ ) + t ). 64) ombining 64) and the proof in the firt tep, which how the other two term on the right hand ide of 6) are contant, we have t) = ) + t ), which mean that t) t i a linear egment with lope. In the fourth cae when W ) >, W + ) =, and I ) < /, let t = mine, e 2, +, +/ I )). Notice that for any < t < t, 63) till hold, and we alo have Ĩ ) = due to the cae condition Ĩ ) = I ) ) + =. 65) Further recall that t < t +/ I ), and conequently we alo have Ĩt) = I ) + t ) ) +. 66) Therefore, Ĩ ) = Ĩt) =. ombining thi reult with 6), we have t) = ), which mean that t) t i a linear egment with lope. In the fifth cae when W ) =, W + ) >, and I ) > /, let t = mine, e 2, +, +I ) /). Notice that for any < t < t, we have I t) = I ) t ) 67) Ĩ ) = I ) ) + = I ). 68) Again recall t < t + I ) /, we have Ĩ t) = I t) ) + = I ) t ) ) + = I ) t ) = Ĩ ) t ). 69) ombing 6) and 69), we have t) = ) t ), which mean that t) t i a linear egment with lope. In the ixth cae when W ) =, W + ) >, and I ) /, let t = mine, e 2, + ). Notice that for any < t < t, we have I t) = I ) t ) 7) Ĩ t) = I t) ) + =. 7) onequently, we alo have Ĩ t) = Ĩ ) =. 72) ombing 6) and 72), we have t) = ), which mean that t) t i a linear egment with lope. Finally, if t = +, we finihed identifying all linear egment. Otherwie, we recurively identify the remaining linear egment in t) ti+ t. Notice that the ix cae we dicued above cover all poibilitie. Hence, t) ti+ t can only contain piece-wie linear egment with poible lope,, and. ombining all three tep, we proved the theorem.