Shared-State Sampling

Transcription

1 Share-State Samling Freeric Rasall, Sebastia Sallent an Jose Yufera Det. of Telematics, Technical University of Catalonia (UPC) ABSTRACT We resent an algorithm, Share-State Samling (S 3 ), for the roblem of etecting large flows in high-see networks. While evise with ifferent rinciles in min, S 3 turns out to be a generaliation of two existing algorithms tackling the same roblem: Samle-an-Hol an Multistage Filters. S 3 is foun to outerform its reecessors, with the avantage of smoothly aating to the memory technology available, to the extent of allowing a artial imlementation in DRAM. S 3 exhibits mil traeoffs between the ifferent metrics of interest, which greatly benefits the scalability of the aroach. The roblem of etecting frequent items in streams aears in other areas. We also comare our algorithm with roosals aearing in the context of atabases an regare suerior to the aforementione. Our analysis an exerimental results show that, among those evaluate, S 3 is the most attractive an scalable solution to the roblem in the context of high-see network measurements. Categories an Subject Descritors: C.2.3 [Comuter- Communication Networks]: Network Oerations traffic measurement, ientifying large flows General Terms: Algorithms, Measurement Keywors: Per-flow measurements, scalability. INTRODUCTION Network measurements are essential for a number of network management tasks like traffic engineering [5], the etection of hot-sots or DoS attacks [] or accounting. Within traffic measurements, the er-flow aroach rovies a reasonable traeoff between the amount of information acquire an its usefulness [6]. Unfortunately, the major rawback of the er-flow aroach is its lack of scalability: link sees imrove % er year, while the see of memory evices imroves at a much slower ace (7 9%) [2]. The only memory technology that can kee u with the increasing This work has been fune by grant CICYT TSI from the Sanish Ministery of Science an Eucation. Permission to make igital or har coies of all or art of this work for ersonal or classroom use is grante without fee rovie that coies are not mae or istribute for rofit or commercial avantage an that coies bear this notice an the full citation on the first age. To coy otherwise, to reublish, to ost on servers or to reistribute to lists, requires rior secific ermission an/or a fee. IMC 6, October 25 27, 26, Rio e Janeiro, Brail. Coyright 26 ACM /6/...$5.. number of flows is DRAM. However, DRAM evices are alreay too slow for existing high-see line rates. SRAM is the only memory technology allowing for er-acket uates at current line sees. Unfortunately, SRAM evices are exensive, ower-hungry an have limite caacity. This, together with the observation in several stuies that a large share of the traffic is ue to a small fraction of the total number of flows [4], have motivate the search for solutions concentrating on the measurement of large flows. Focusing on such flows not only suffices for many alications but may also allow for new ones: e.g. threshol accounting or scalable queue management [2]. Pioneering work by Estan an Varghese [2] showe that it was ossible to ientify or etect such heavy-hitters using slightly more memory than that require in case such flows were known beforehan. While subsequent research has tackle the roblem of measuring flows simultaneously consiering ifferent flow efinitions [3], the roblem for reefine flow efinitions is still oen from a research oint of view an of great ractical interest. This aer contributes with a scalable algorithm, Share-State Samling(S 3 ), for the etection of large flows in high-see networks. The aer is structure as follows. Section 2 summaries revious work focusing on those our work shall be comare to. In section 3, we resent our algorithm an we analye it in section 4. Section 5 iscusses how S 3 coul be imlemente in high-see routers. Besies being of interest er se, the iscussion shows the versatility of S 3 an hels unerstaning its scalability, an issue that we finally aress in section 6. Section 7 resents some results obtaine with software imlementations of the algorithms an real traffic traces. While the focus is on the erformance of S 3, we comare to other roosals in the literature. An exhaustive evaluation of all algorithms is not ossible for sace reasons. Nevertheless, the exeriments resente suffice for a concetual comarison ientifying the weaknesses an strengths of the algorithms. Section 8 conclues the aer. 2. RELATED WORK The roblem of etecting frequent or hot items in streams has attracte consierable attention in the ast, secially in the fiel of atabase management an ata mining. In this context, Toivonen [7] rooses algorithms that use uniform ranom samling for the iscovery of association rules, which require a single ass as comare to rior work. Manku an Motwani [7] roose two algorithms, Sticky Samling (S 2 ) an Lossy Counting (LC), for comuting frequency counts exceeing user-secifie threshols. Cormoe

2 an Muthukrishnan [] roose algorithms able to ientify hot items in atabases in a single ass while allowing for eletions. To the best of our knowlege, Samle-an- Hol () an Multistage Filters( ) [2], while showing some similarities with revious work in the context of atabases [6], were the first algorithms roose in the context of network measurements. [] [8] [9] aress relate roblems, but their focus is not on etecting large flows. In what follows, we summarie the ieas behin the roosals that we comare with, while reviewing the roblem at han. an aim at ientifying, within the total traffic resent in some measurement interval, large flows, efine as those with sie above some threshol (such as T bytes) or above some fraction of the total (e.g. = %). Flows are unerstoo as sets of ackets with common roerties. The secific roerties consiere, the so-calle flow efinition or attern, is alication-secific 2. Given a flow efinition, each flow is uniquely ientifie by the secific values the roerties within the efinition take, what is calle the flow ientifier (FID) or key. The iea in an is the following: there can be at most / large flows in a measurement interval. Thus, if these can be ientifie, small but fast memories can be use to iniviually track such flows by keeing a eicate entry in the so-calle flow memory: not only small but fast memories suffice but also, accuracy is imrove since bytes arriving after etection are exactly counte. Further, the amount of measurement ata to reort is rastically reuce. This is a araigm shift comare to revious techniques (e.g. Samle NetFlow), that use large (but slow) DRAM memories an alleviate the ga between link an memory banwith through samling. In both algorithms, flows are challenge to instantiate entries in the flow memory accoring to their sie. oes this by means of samling an is memoryless an ranom: flows must be samle once to get etecte. On the contrary, is eterministic an stateful: flows must increase some counters beyon some threshol before entering the flow memory. S 2 an LC in [7] resemble an in that, once item tyes (i.e. flows) are ientifie, eicate entries are suorte, which get uate with subsequent occurrences of such tyes. LC slits the stream into fixe-length buckets where items either uate or instantiate entries. In aition, entries for infrequent items are rune at bucket bounaries. S 2 runes entries at the en of variable-length buckets an only samle items instantiate entries, where the samling rate varies over the length of the stream. LC an S 2 relax the challenge ut on flows to enter the flow memory so as to imrove accuracy an comensate the increase number of enries occuie by means of runing. Hence, the challenge ut on flows is not only entering but also remaining at the synosis structure. While both algorithms have been suggeste to outerform an, no evaluation with real traffic traces has, to our knowlege, ever been conucte. An ieal algorithm shoul reort, at the en of a measurement interval, the FID an sie of only those flows above the threshol. Less ieal algorithms may fail in three ways: i) missing to reort some large flows (false negatives) ii) Measurement intervals can san ifferent timescales: for instance, for traffic engineering these may be in the orer of minutes to hours whereas for queue management these coul be in the orer of secons. 2 E.g., for traffic matrices estimation, flows may be efine as ackets with same source an estination refixes, whereas for ientifying TCP DoS attacks, the focus may be on TCP ackets with some flags set an istinguishe by aresses. reorting flows below the threshol (false ositives) an iii) roviing inaccurate estimates for large flows. Consequently, when assessing the erformance of this tye of algorithms the focus will be on: first, how high the etection robability for large flows is; secon, how accurately can their sies be estimate an, thir, the overall amount of memory require. Due to the resence of false ositives, more than / entries may have to be suorte in the flow memory: false ositives may revent large flows from being measure (in site of them being etecte), if these exhaust the entries available. The amount of memory require by algorithms, in excess to the minimum, is relate to their ability to avoi false ositives. In an, this eens on how much can algorithms revent small flows from entering the flow memory. Although S 2 an LC eliminate false ositives when querie, their memory requirements are etermine by the number of entries that may have active at any oint, which eens on how goo these algorithms are at exelling small flows from the flow memory. Finally, we shall also be concerne with algorithm s er-acket rocessing times, which ultimately een on the number of memory accesses require: too many accesses may reclue the imlementation of the algorithms or limit their scalability even if the fastest memories are use. 3. SHARED-STATE SAMPLING The algorithm that we shall now resent, Share-State Samling (S 3 ), is base on the following rincile: if the number of flows in a link is large an most of the traffic is ue to a small fraction of these flows then, most of the flows must be small. Therefore, making it harer for these many small flows to get cature may be a reasonable strategy to avoi false ositives. The major rawback of is the strong traeoff it exhibits between accuracy an sace requirements: to achieve reasonable accuracy, must oversamle, which causes the etection of small flows thereby increasing the memory requirements. This is so since oversamling by a factor O reuces the threshol by a factor /O. In this regar, we shoul carefully istinguish between the threshol of the roblem efinition, T, that classifies flows into small or large an the threshol of the algorithm, which is the sie of a flow beyon which the etection robability is higher than some rescribe value. In, flows of sie / bytes (with the byte samling robability) are etecte with robability.63. S 3 aims at imroving the etection curve of by making it closer to that of an ieal algorithm, which shoul be Pet ieal (v) = u(v T ), with u(v) the iscrete ste function, T the threshol an v the flow sie. The key issue is how to achieve a ecrease in the etection robability for small flows, without affecting the etection robability for large flows. S 3 works as follows. For each arriving acket, it erforms a looku in the flow memory to check whether that flow has alreay been cature. In that case, the corresoning eicate byte counter is uate. That is, as in an, ackets of flows alreay cature byass the challenge or filter. Packets of flows not (yet) etecte are samle with some robability. If a flow is samle times then, an entry is create in the flow memory. In orer to count the number of times that each flow has been samle, a set of counters are suorte. When a acket gets samle, these counters are examine: if these hol a value of, this makes

3 samles altogether an an entry for this flow is create in the flow memory; otherwise, the counters get incremente to remember the number of samles taken so far. That is, our algorithm counts the number of times flows get samle an entries in the flow memory shoul only be create for flows samle times. We refer to as the samle threshol. The way in which samles are obtaine an counte is exlaine in what follows, together with the suorting analysis. 4. ANALYSIS OF OUR ALGORITHM 4. The "single-flow" case Let us start analying what haens in case there were only one flow. Suose that the algorithm samle every byte with robability an that samles from a flow incremente some counter until samles were taken. Samling each byte with robability, the first samles will increment the counter an the th samle (if there) will get to the flow memory. Clearly, the flow will not be etecte if fewer than samles from it are taken. Hence, the etection robability for a flow of sie v bytes, P et(v), is the robability that we take at least samles ha we samle all of its bytes (the suerinex shall later become clear). Since each byte is ineenently samle, the number of samles that we woul take, N v, is a binomial r.v. an we can write! vx Pet(v) v = P {N v } = j ( ) v j () j j= Alternatively: the number of bytes that ass before we get the first samle is a geometric() r.v. Thus, the number of bytes that go before the th samle is taken, S, is the sum of i.i. geometric r.v., which has a negative binomial istribution. Therefore, the robability that a flow of sie v gets etecte is the robability that the number of bytes require to take samles be no larger than v. Think of bytes as attemts to get a samle: a flow is etecte if fewer attemts than the available, v, suffice to get samles. The robability that it takes exactly j attemts until the th samle is obtaine is P {S = j} = `j ( ) j, where j. Thus, if v <, the etection robability for a flow is an P et(v) = P {S v} = vx j=! j ( ) j (2) if v. Let us see what the role of the samle threshol is. The first observation is that, for =, only one samle is neee to cature a flow. Further, no counters are require (no revious samle nees to be remebere ) an Pet(v) in () an (2) become ( ) v, which is the etection robability with since, then, both algorithms are ientical. Clearly, the etection robability for a flow ecreases when we increase for the same, since more samles from the flow are require. This is shown in fig. (bottom-right) which lots Pet(v) for ifferent values of. The key observation is that, as increases, Pet(v) has an inflection oint as comare to the curve of, which oes not have it regarless of. Let us see what haens if is chosen eening on. If each byte is samle with robability an samles from a flow are require, the flow will be etecte, on the average, after bytes, which is times the average of a geometric() r.v. Let us see what haens (v) (logscale) P et P et (v) e-4 e-6 e-8 e- e-2 e-4 e-6 e-8 e-2 = = v (flow sie).4.2 = =2 =3 =5 = =5 = =5 = = v (flow sie) ineenent of, = = v (flow sie) Figure : Pet (v) for several values of when = ; / = 5. Bottom-right: Pet (v) for = an =... if we set the samling robability so that, regarless of, it takes, on the average, a constant amount of bytes, /, until a flow gets cature. It is straightforwar to see that has to be chosen as = (3) Choosing as in (3), the suose inflection oint will be locate at v = =, which oes not een on either. Fig. (to) shows this effect lotting Pet(v) when is chosen accoring to (3), for some values of an = 5. As can be observe, varies the steeness of Pet(v). The following terms will be use throughout the remainer of the aer. We call the location of the inflection oint, the cutoff sie, =. The etection robability for a flow of Vcut bytes is aroximately.63 for small values of. In these terms, (3) can be written as =. We call granularity,, the targete caability of etecting flows no smaller than a fraction of the total traffic in a measurement interval, C. That is, = T/C. By selectivity we mean the ower of iscriminating (etecting) flows accoring to their sie. Secifically, we mean the ability of algorithms to avoi etecting small flows while not comromising the etection of large flows. Now, the reason for Pet(v) exhibiting such behavior is the following. For a flow of sie v, E{N v} = v since N v is binomial(v,). It is well known that the most likely values of a binomial r.v. are concentrate aroun its mean. If v < an = / then, the average number of samles taken ( v ) will always be smaller than. Thus, the robability of the most likely values of N v shall not be inclue in P {N v }, thereby yieling a low etection robability. Conversely, if v >, v = ( v ) shall always excee thereby raising Pet(v, ), which will be high. As of this reliminary analysis, we will set equal to the threshol T an use to govern the selectivity of S 3. To see the gain in selectivity in S 3 comare to, fig. (bottom-left) lots Pet(v) as before, but in logarithmic scale. As can be seen, the etection robability for flows above = 5 is very high in S 3, while being several orers of magnitue smaller than that with for flows below ; alreay for small values of. Further, as increases, Pet(v) seems to aroximate the etection curve of an ieal algorithm. Finally, note that corresons to the

4 average number of samles that woul be taken from a flow of sie. 4.2 The inflection oint, the ieal algorithm We shall now justify why is an inflection oint. Secifically, the following iscussion shows that, if = /, selectivity in S 3 imroves as we increase, in that the etection robability for a flow increases if v >, ecreases if v < an remains aroximately constant if v =. Let N v, be the number of bytes samle from a flow with v bytes when each is samle with robability =. The etection robability for a flow is P {N v, }. Note that, increasing, the istribution of N v, changes. As binomial(v, ), N v, is the sum of v i.i. r.v., with mean E{N v, } = v an variance σn 2 v, = v( ). The etection robability can be written as P {N v, < }, which is equivalent to ( Nv, E{N v, } P < E{N ) v,} (4) σ Nv, σ Nv, By virtue of the central limit theorem, Z v, = (N v, E{N v, })/σ Nv, is, for large v, aroximately a stanar normal r.v., Z an (4) aroximates Φ(( E{N v, })/σ Nv, ) with Φ(x) = P {Z < x}. Letting = in the exectation an variance of N v,, Pet(v) can be aroximate by s Vcut v! Φ (5) v The square root in (5) monotonically increases in. If v <, the term in arentheses is ositive. Hence, increasing, the argument of Φ(x) increases. Since Φ(x) monotonically increases with x, Φ(x) P et (v, ) ecreases. Conversely, if v >, the argument of Φ(x) is negative an increasing in absolute value with, meaning that Φ(x) will increase in. When v =, the argument in Φ(x) is, an the etection robability slightly varies aroun.5. Lemma. In the single-flow case, S 3 aroximates an ieal algorithm as aroaches =. In other wors, lim P et(v) = u(v ) Proof. As, we ten to require samles from flows before etecting them. However, if = /, we also ten to samle every byte. Thus, when =, every byte is samle an therefore accounte: if v <, the number of samles never reaches the samle threshol (no etection); if v the threshol is always reache. 4.3 The effects of limite memory So far we have stuie the etection robability in S 3 for the case with only one flow, P et(v). In reality, we have stuie the etection robability for any number of flows ha the algorithm sufficient memory to exactly count the number of samles taken from every flow. This is so since, in that case, the etection robability for a flow eens only on its sie an on given that the samling rocess is memoryless an bytes get ineenently samle. We call this, the unlimite memory case (hence, the reason for the suerinex ). Unfortunately, the case will be that the number of flows is larger than the number of counters that can be affore. Thus, some counters will have to count the number of samles taken from several flows. This may lea to flows below the threshol being etecte: since counters are raise by other flows, the algorithm may think that a flow has been samle more times than it really were. We call this effect, interference. For large flows the imact shall be small since P et(v) may alreay be close to unity. The roblem may be for small flows, since the assumtion is that there are many such flows. In what follows we stuy what haens when counters have to be share among flows (hence, the name of our algorithm). 4.4 Keeing track of samles The way samles are counte may be critical for S 3 to erform as in the unlimite memory case. Thus, we investigate several aroaches. It turns out that suorting several stages of counters, where counters get assigne to flows by means of comuting ineenent hash functions on acket FIDs (as in ), is a goo strategy. For sace reasons, we o not rovie the full iscussion an the reaer is referre to [5]. Thus, the remainer of the aer focuses on the erformance of our algorithm when counters are arrange in m arallel stages: each samle byte will increment by a counter inexe by the result of comuting a hash function on the acket FID, at each of the stages until a samle fins all its stage counters at, in which case an entry shall be create for that flow. This comletes the efinition of the algorithm an the analysis motivating it. The next section stuies how the etection robability is when samles are counte using m stages of b counters. 4.5 P et (v) arranging counters in stages A flow must be samle at least once to get etecte. If a flow woul have been samle more than times (which can only haen if v > ), interference woul not hel since it woul be etecte anyway. If a flow woul have been samle < k < times, it woul be etecte rovie that all counters where it hashe to were, at least, k, even if v <. Consequently, if N v is the number of times a flow woul be samle, we have that: 8 < P et (v N v = k) = : k = k =,... v P {all k} k =... The etection robability for a flow of sie v is maximum if it is the last one to arrive an all other bytes, C v, enter the samler. Let S be the number of samles taken out of these C v bytes. If S = r, the number of counters at k is, at most, r. Thus, the robability that a flow k samle k times hashes to one of these counters when S = r is boune above by r r. Since the hash k b ( k)b functions are assume ineenent an the same samle bytes may increment a counter at each stage, the etection robability for a flow samle k times is boune above by ` r m when exactly S = r samles from the rest of ( k)b the traffic are taken. Thus, in general we have that: (6) C v X r mp P {all k} {S = r} (7) ( k)b r= k where S is binomial(c v, ). As such, for large C, S will be very close to (C v) < C = / with very high robability, by virtue of the law of large numbers. Thus, we can aroximate P {all k} P {all k an (7) as P {all k} min ( k)b S /} m,. Note that this suggests taking b > / for this robability to be

5 below. Consiering this aroximation, (6) an unconitioning from N v, the etection robability for a flow with v bytes is boune above by P et (v) P et(v) + (b) m µx k= m P {n = k} (8) k where µ = min(, v). That is, by the etection robability in case of unlimite memory, Pet(v), lus a term, P et (v,,, C, b, m) reresenting the increase in the etection robability cause by other traffic incrementing our flow s counters. Note that, unlike in the unlimite memory case, flows smaller than may get etecte if samle. The observation is that selectivity may no longer een solely on : it shall also een on the egree of interference between flows, which eens on the amount of memory suorte. Note the following traeoff: on one han, increasing the samle threshol, the term Pet(v) iminishes for flows below, thereby imroving the selectivity. On the other, the byte samling robability also increases, which translates into more samles from other flows increasing stage counters ( P et ), thereby otentially increasing the etection robability for small flows. This suggests the existence of an otimum value for. In aition, for P et (v) to be close to Pet(v), the term /(b) m shoul be small. In articular, note that if b the rightmost term in (8) vanishes. Increasing m, the same factor also vanishes if b > /. Let us now see the role of m an b. Let S be the number of samles taken from the rest of flows an ɛ i, i =..m the amount by which the counters (one at each of m stages) where our flow hashes to get incremente by other flows. We omit, for clarity, the subinex but recall that the istribution of the above een on. The etection robability for our flow is maximum if it is the last one to arrive an the rest of traffic enters the samler, in which case S is binomial(c v, ). The robability that a flow gets etecte is no larger than the robability that, at all stages, ɛ i + N v. Consiering that, for any k, E{ɛ i S = k} = k E{S}, we have that E{ɛi} = = b b (C v). Since N b v an S are ineenent, so are N v an ɛ i, i =..m. Hence, efining Z i = Nv+ɛ i E{N v+ɛ i } σ Nv+ɛi, the etection robability for a flow arriving last can be written as P {Z i v/vcut (C v)/(vcutb), i =..m}. Following (σ Nv 2 +σ2 ɛ ) /2 i the same line of reasoning as in section 4.2, it can be roven that the inflection oint in the etection curve at the en of the measurement interval is at V infl ( ), regarless of m. This suggests the nee to suort a number b of counters er stage at least several times / for the etection curve to be selective throughout the measurement interval. The rouct b can be written as C. If we interret C = as the minimum strength of a large flow C/b an C/b as the strength of the interference, b can be unerstoo as the signal-to-interference ratio. Fig. 2 lots (8) for ifferent stage sies when 2 stages are use. Note how, even for the worst-case etection curve given by (8), V infl well aroximates the location of the inflection oint. This is shown in fig. 2, where vertical lines are lotte at V infl for each value of b, whose location lies close to the value of v where the convexity of P et (v) changes. Note how, for b / =, the inflection oint isaears, whereas, for b /, V infl. While m oes not affect the location of the inflection oint, it shall imact on the shae of the etection curve. P et (v) (logscale) P et (v) v. b=2 unlimite memory. b=. b=5 b=2 e-4 b=3 e-5 b=5 b=7 e-6 b=x 3 e-7 unlimite memory b=3x 3 b=5x 3 e-8 b=x 4 e-9 b=5x v Figure 2: Worst-case etection robability (8) when = 5, = 5 ( = %) an m = 2, for ifferent stage sies. Secifically, the etection robability for flows much smaller than the threshol exonentially ecreases with m. This suggests that, once the number of counters er stage excees several times /, it is much more effective to a stages than to increase the sie of the existing ones. Fig. 3 shows the worst-case etection curve for a fixe stage sie of counters, when ifferent number of stages are use. Note that using m = 2 stages with counters is much more effective than using 2 stages with b = 4 counters, for the same amount of memory. The etection robability for a flow with 2 3 bytes is 7 in the first case an 4 in the secon. Finally, note how, for a large number of stages, P et (v) Pet(v + Vcut ). That b is, flows at the en of the measurement interval aear as V infl = /b = C/b bytes larger. P et (v) (logscale) P et (v) v b b unlimite memory e-5 unlimite m= memory m=2 m=3 e- m=4 m=5 m= m=5 e-5 m=2 m=3 m=5 e v Figure 3: Effect of the number of stages, m, in (8) for the same setu when stages are of b = counters. 4.6 On the number of cature flows In S 3, flows must be samle at least once to get etecte. Therefore, an uer boun for the etection robability for a flow of v bytes is the robability that it gets samle at least once, i.e. P et (v) ( ) v, which is the etection robability with for an oversamling factor an threshol. A first consequence of this observation is the following lemma, whose roof (that can be foun in [5]) we omit for sace reasons.

6 Lemma 2. For the same byte samling robability, S 3 etects, on the average, fewer flows than. The next theorem (also roven in [5]) bouns above the execte number of etecte flows in S 3 when the number of counters suorte at each stage is large enough. While tighter bouns for secific flow sie istributions may be foun, our boun is ineenent of the flow sie istribution. Theorem. With m stages of b counters each, if b > ( )/m, > then, the execte number of flows etecte by S 3 in a measurement interval is boune above by ` +, (+m)((b) m ) regarless of the flow sie istribution, acket sie istribution, number of flows an amount of total traffic in a measurement interval. Recall that, for a granularity, there can be at most / large flows. Hence, if no assumtion is to be mae about the flow sie istribution, the flow memory has to be at least / entries. Thus, the term ` + (+m)((b) m ) can be unerstoo as the factor by which we nee to inflate the flow memory beyon the minimum so that, on the average, all cature flows fin an entry available. The boun in theorem shows that, regarless of the traffic mix (e.g. number of flows), the flow memory oes not nee to be overimensione if the amount of memory suorte at the stages is sufficiently large. In this regar, if b is several times / then, (b) m (b) m, an the excess memory require ecreases exonentially in m. 4.7 S 3 relate to an While the iscussion so far has focuse on [, ], the comlete analysis coul have been formulate in terms of [/, ] since, in S 3, =. When =, S 3 behaves as with O = an T =. As aroaches, tens to unity an then every incoming byte for an unetecte flow increases stage counters until one fins all stages at =. Therefore, if samle counters are arrange in arallel stages, S 3 becomes when the samling robability becomes. Hence, an can be unerstoo as secial instances of S 3 when or take their extreme values. At one ege, this gives a ranom memoryless algorithm,. At the other, a eterministic memoryful one,. In the mile, S 3 shall iffer from in the following. In, counters may have to hol values as large as since the ecision on creating entries is taken eening on whether counters reach such value. Thus, requires log 2() bits er counter. In S 3, counters have to hol values of, at most, [2,..], thereby requiring log 2() = log 2() bits er counter. Thus, for the same amount of memory, S 3 can suort more counters than. This memory gain shall be aroximate by G m = log 2()/log 2(). Define as such, G m monotonically ecreases for [2,..]. For = (), no counters are neee an G m =. For = ( ), G m =. Finally, in, all ackets belonging to unetecte flows increment counters. Further, all ackets fining all counters at or above create an entry. In S 3, only ackets with bytes samle increase counters an ackets that woul fin all counters at o not instantiate entries unless samle. The memory gain an the fact that only samle bytes increment counters translate into smaller interference. Smaller interference (i.e. hel from other flows assing the stages) an the fact that flows must be samle at least once to get etecte, translates into better selectivity for the same amount of memory, as we will see in section 7. While we o not show it for sace reasons, it can be more rigorously roven that S 3 becomes as. This, together with the generality of theorem allows us to rove the following lemma that gives an alternative boun to that in [2] for the number of cature flows in that oes not een on the number of flows. Lemma 3. The execte number of flows assing a arallel Multistage Filter with m stages of b = V /m cut counters each is boune above by ` + (+m)( ) regarless of the flow sie istribution, number of flows an amount of traffic in a measurement interval. Proof. Since S 3 when an theorem () alies for any value of >, conition b > ( )/m yiels b > (Vcut )/m, which is always met if (b) m =. Thus, = (b) m, which yiels the result when substitute in the boun in theorem. 4.8 Accuracy of S 3 There are two sources of inaccuracy in S 3 : the uncertainty ue to samling an the effect of interference resulting from limite memory. With unlimite memory, the number of bytes that go before a flow gets etecte, S, has a negative binomial(, ) istribution. The best estimate for the sie of a large flow, v, is ˆv = E{S }+R = /+R = +R, with R the number of bytes accounte once the flow is etecte. The absolute error is ˆv v = S for a mean square error MSE = E{(ˆv v) 2 } = V ar(s ). Since S is the sum of i.i. geometric() r.v., X, we have that V ar(s ) = σx. 2 )V 2 Since =, the MSE is given by ( cut/. Defining the relative error, ɛ rel, as the square root of the MSE over the sie of a flow, the relative error is (for a flow at the threshol an unlimite memory) given by ɛ = q. Note that ɛ is, by efinition, only ue to the effect of samling. In aition, note how it vanishes when an that it can be aroximate by for large threshols when. Limite memory translates into interference at the stages, causing the remature etection of flows. Consequently, estimating the number of misse bytes with, the worst case is for a flow arriving last since it shall be overestimate at the most. Let c..c m be the values at the stage counters that the first samle from this flow fins: c min samles from it shall be require with c min = min(c..c m) since the flow has to raise all its stage counters to ( ) an get samle once again before entering the flow memory. This imlies a number of misse bytes S cmin with a negative binomial( c min, ) istribution. The MSE is therefore given by E{( S cmin ) 2 } = Vcut 2 + E{S c 2 min } 2E{S cmin }. Now, conitioning for c min an taking exectation we have that E{S cmin } = ( E{c min} )V cut an E{S c 2 min } = 2 `( E{cmin})( 2 ) + E{( c min) 2 }. Substituting in the exression for the MSE, taking the square root an iviing by, the worst-case relative error is foun to be ɛ rel = r E{cmin} ɛ 2 + E{c2 min } 2 (9)

7 Note that, as, ɛ vanishes an the relative error is given by E{c 2 min }/Vcut: there is no inaccuracy ue to samling since bytes are samle with robability an E{c 2 min} is the secon moment of the smallest increment cause by other flows that share our flow s counters. In aition, as we enlarge the stages, E{c min} an E{c 2 min} get small an ɛ rel ɛ. Lemma 4. The worst-case relative error in S 3, ɛ rel, is boune below by ɛ. Proof. From (9), ɛ rel ɛ ( E{c min} )ɛ 2 + E{c2 min } 2 ɛ 2 E{c2 min } E{c min} E{c min}. Since c min is integervalue, E{c 2 min} E{c min}, an the above is always true. The observation in lemma 4 has imortant ractical an theoretical imlications: for a certain, the relative error may ecrease with the amount of memory, but this imrovement shall, at some oint, become negligible since the relative error is boune below by ɛ. However, ɛ can be mae arbitrarily as close to as we wish 3. The following two lemmas shall be use in section 6 to stuy the scalability of S 3. Lemma 5. It suffices that E{c2 min } ɛ 2 2 ɛ 2 for the relative error in the estimates of large flows to be no larger than ɛ, where ɛ ɛ. Proof. c min is, at most, : samles beyon the ( ) th either instantiate entries or o not raise the counters. Thus, ( E{c min}/). Consiering (9), this yiels ɛ rel ɛ 2 + E{c 2 min }/2. Thus, for ɛ rel to be below ɛ it suffices that ɛ 2 + E{c 2 min}/ 2 ɛ 2. Since c min, an ɛ 2 are non-negative, this is ossible only if ɛ ɛ. Lemma 6. If P {c min ɛ} δ for some ɛ, δ < then, the relative error for a large flow is boune above by ɛ 2 + ɛ 2 ( δ) + δ Proof. c 2 min is integer-value, non-negative an restricte P to,... ( ) 2. Thus, it can be roven that E{c 2 min} = P {c 2 min i}, i =..( ) 2. Since P {c min i} is nonincreasing with i, the revious summation can be boune above by R ( ) 2 P {c 2 min x}. Since P {c min ɛ } δ P {c 2 min ɛ 2 2 } δ, if such conition is met, the revious integral (i.e. E{c 2 min}) can be boune as E{c 2 min} R ɛ R ( ) 2 δ = ɛ (( ) 2 ɛ 2 2 )δ. This imlies ɛ 2 2 E{c 2 min } ɛ 2 ( δ)+( 2 )2 δ ɛ 2 ( δ)+δ. Proof follows by virtue of lemma IMPLEMENTING S 3 Stage counters in have to be imlemente in memories as fast as line rates, since such counters may have to be uate on each acket arrival. Given its resemblance with, stages in S 3 coul be imlemente using SRAM [2]. In this section we resent some reference esign that shows how S 3 coul be imlemente using slower memories. This will reveal the flexibility of S 3 aating to the available memory technology an how S 3 coul scale even if the ga between memory an link sees increase. Two observations motivate this esign: first, the fact that, in S 3, not every arriving acket translates into a memory uate; secon, the 3 This is true, since, while in the iscussion we have assume integer values for, nothing revents us really from using real-value samle threshols. fact that, unlike in other er-acket rocessing tasks (like classification in firewalls or route lookus), the actions out of accounting o not aly to the ackets but to counters, which allows early releasing ackets before such actions are taken. In the following, we assume that several rocessing engines (µp for short) are available for the execution of measurement algorithms as well as two tyes of memory evices, fast but small ones (e.g. SRAM) an larger but slower ones (e.g. DRAM) 4. While we shall refer to these memories as SRAM or DRAM, what we shall really be stuying is the fact that S 3 smoothly relaxes the time requirements of the memories involve, no matter their technology: i.e., with SRAM we shall mean memories able to count at line rates, whereas DRAM shall refer to slower memories. Reference esign: Whenever a acket arrives (e.g. at a line car), S 3 erforms a looku in the flow memory, as an. This can involve lookus as often as ackets arrive. Thus, the flow memory nees to be imlemente in high-see memory. However, unlike in, only ackets with samle bytes cause rea/write oerations at the stages. Let us assume that, on recetion of ackets, an inut µp, create some samle hanler, S h, an release the acket from the measurement rocess 5. This woul minimise the elay that the measurement might a since creating a S h involves retrieving a small amount of acket ata, which may alreay be gathere for other uroses. These S h s coul contain the information relevant to our algorithm: the F ID (S F ID), a timestam (S tstam), the acket length (S len ) an the number of bytes samle from the acket (S bytes ). Finally, this inut µp coul ut this hanler asie in a queue at some memory evice, until another µp rea it an took over. Clearly, this queue shoul also be in fast memory since consecutive ackets coul be samle an queueing a S h shoul finish before any other S h entere. Some other rocessor (server µp ) coul be use to to rea these S h s from the queue, scheule the comutation of the hash functions on the S F ID an increment the resective stage counters, hel in a larger but slower memory evice. Let us assume that the write oerations on the counters coul be ieline, an that after such oerations the server µp coul know if all counters reache so that entries coul be create in the flow memory when aroriate. Fig. 4 shows the iea. In what follows, we stuy the feasibility of such a esign using fewer fast memory an the limitations that this may ut on the erformance of S Analysis of our reference esign A acket causes memory oerations at the stages if one or more of its bytes get samle. It can be easily roven that, the case with the worst time constraints is when traffic is comose of ackets of minimum sie, l min, the link is ermanently busy an all the traffic goes to the samler. Let t = l min be the minimum inter-arrival time, with C C l l the link caacity. A acket will cause the creation of a S h with robability s = ( ) lmin. Thus, the average 4 These assumtions are very reasonable in case of network rocessors, where we fin a core rocessor erforming control functions an a set of RISC rocessors otimie for acket-rocessing functions, which tyically run several harwareassiste threas. Further, NPs (for instance Intel s IXP2xx) are tyically esigne to interface with DRAM as well as SRAM evices. 5 A similar concet is use in NP s, where acket hanlers, tyically hel in SRAM, are create for ackets. Such hanlers are ata structures containing control information an are tyically use as ointers for tasks like queueing.

8 Figure 4: Reference esign worst-case inter-samle time is t/ s. Let us aroximate this geometric arrival rocess by assuming that the intersamle times are exonentially istribute (the continuous counterart of a geometric rocess) with the same mean, that is, by a Poisson rocess with rate λ = s/ t. With this aroximation, we are consiering a worse case since, as Poisson, more than one samle coul arrive within t. In orer to queue a S h in the buffer, the access time of the fast evice shoul be no larger than t. Let T a be the time sent by the server µp in reaing S h s from the buffer, comuting the hash functions an uating the corresoning stage counters. Since comarisons an hash functions can be efficiently imlemente in harware, T a will be on the orer of the access time of the large memory evice. Let us efine α = Ta, α > as the isarity between the access times t of the two tyes of memories. If we require the fast memory scale with the link caacity, then α = Ta = t l min C l / T a which can be unerstoo as the ratio link versus memory banwith, for the slower memory. Since it is reasonable to consier T a constant, we can moel our esign as an M/D//k system, as shown in fig. 4. With such moel, the loa of the buffer+server µp system is given by ρ = λt a = s Ta = sα. Now, for the t system to be stable it must be that ρ <. Thus, the first constraint is that we cannot oversamle at will: s <. α Thus, using memories α times slower may limit, which shall in turn limit the selectivity (for the same or ) or the granularity for a fixe selectivity. To easily see this, C l D let us aroximate s l min = lmin, where D is the uration of the measurement interval for a total of C l D bytes. The inequality bove becomes < C ld l min. In aition, α as er section 4.8 this constraint for shall limit accuracy since the worst-case relative error is above ɛ /. Now, since T a > t, some S h s may fin the server µp busy uating counters at the slow memory 6 or reviously create S h s in the queue. Since this buffer will in ractice have limite sace, it coul haen that some S h foun it full an ha to be iscare. Losing S h s woul reuce the etection robability of flows, since the algorithm woul think that flows were samle fewer times. Let us see how large shoul this buffer be for this to rarely haen. 5.. Dimensioning the buffer in fast memory In orer to imension the sie of the buffer, we can use the relations for the M/G/ for constant service times. In case of an infinite queue (i.e. M/D/ instea of M/D//k), the average waiting time for a S h woul be E{W } = ρta 2( ρ) for, by virtue of Little s formulae, an average occuation 6 In reality, the case might be that µp suort several threas an that the accesses to memory are queue at some memory interface. This is the case with Intel s IXP2xx where the so-calle micro-engines issue memory oerations to the memory controllers. Nevertheless, we moel the situation as a server being busy with a samle hanler for some time Ta. E{Q} = λe{w } = time woul be σ 2 W = eviation σ W Ta ρ ( ρ) 3 The variance of the waiting T 2 a ρ an its stanar 3( ρ) 2. Hence, the stanar eviation for ρ2. 2( ρ) T a 2 ρ ` 4 ρ ( ρ) 2 2 the number of samles in the queue can be boune by σ Q ρ. Now, since the arrival of samles is aroximately ( ρ) 3 Poisson, the PASTA roerty hols. Thus, the robability that some S h fins exactly k samles in the buffer is P {Q = k} an the robability of a S h fining more than k samles in the queue is P {Q k}. By virtue of Chebyshev s inequality, P {Q > E{Q} + nσ Q} <, n >. Hence, if the buffer n 2 is of sie B = E{Q} + σ Q ɛo then, the robability that a S h gets iscare will be < ɛ o. For small values of ɛ o, the behavior of our M/D//k may be well estimate using the relations for the M/D/. Thus, substituting each term by the values foun before, we get to B = 3ɛoρ 2 +2 ρ ρ2 2( ρ) + ρ ( ρ) 3ɛ o = 2. For ρ <, we have that 3ɛ ρ2 < ρ an B can o( ρ) be boune above as ρ( 3ɛo+2) ρ B 2 3ɛ o( ρ) ( ρ), where 3ɛ o the latter aroximation comes from consiering ɛ o. Concluing, if the buffer is, in number of S h s, as large as ρ B = ( ρ) () 3ɛ o then, the overflow robability will be smaller than ɛ o Imact of uate elay on accuracy The following imairment ue to this buffering shoul be note. Whenever a S h is queue, it takes some time until stage counters for that flow get uate. If one of these S h turns out to be the one fining all counters at, the creation of an entry for this flow will be elaye the time it took that S h to get to the counters. If subsequent ackets of that flow arrive within this time interval, they will fin no entry an will be either samle or ignore. Thus, while losing S h s affects the etection robability, elaying them may affect accuracy. Again, the worst case is when the link is always busy an all ackets arriving after the flow just etecte belong to it. If it takes, on the average, τ for a S h to get service then, the maximum number of bytes that the algorithm shall miss is, on the average, τc l. The average time it takes for a S h to get service is τ = E{W } + T a = (2 ρ)ta, 2( ρ) with W the waiting time in the buffer. Thus, the average number of bytes misse from a flow is boune above by E{v miss } (2 ρ)ta 2( ρ) C l = + α lmin. Note that this error tens to infinity as ρ tens to unity. Thus, if either the ρ 2 isarity of sees or the byte-samling robability is high, this error will be consierable. However, for e.g. ρ <.75, the error will be boune above by 2.5 α l min, which is 2.5 α minimum-sie ackets. For the same loa, B = 2 ensures an overflow robability below 6. Thus, for moerate loas (achieve reucing ), this error shoul not be a concern. In case the loa were high, our esign can be easily ugrae so as to estimate this error an boun it above as shown in [5], where we also show some numerical examle showing the otential of the solution. Final remark an section summary: At very high sees the flow memory may have to be imlemente with CAMs [2] (currently ns). In our esign, both µp s (inut an server) access the buffer of S h s. Memories simultaneously suorting rea/write oerations o actually exist [3]. For the same amount of memory, S 3 can suort

9 more stage counters than. Further, stages in S 3 can be imlemente in memories slower than link banwiths. This allows imlementing S 3 at sees where an imlementation of woul be either ifficult, costly or not ossible. Further, the use of DRAM translates into consierable savings in terms of sace an ower consumtion. Using memories α times slower limits above the samling robability, thereby theoretically limiting selectivity an the worst-case accuracy. Both can be comensate in case of DRAM using more an larger stages. Further, even for high (essimistic) values of α, S 3 can be very selective for values of (or ) much below the limit ue to α. Thus, small buffers may alreay ensure no overflow an the imact on accuracy ue to the elay may be negligible. Finally, schemes combining both memory tyes have been roose in relate contexts SCALABILITY OF S 3 We shall now see how S 3 scales an comare it to the other aroaches mentione in this aer. The roblem at han gets harer when finer granularities are targete, when higher accuracy is require or when the amount of traffic in a measurement interval increases. As a consequence, the error relative to the smallest sie of interest (rather than the absolute error, or the error relative to the total) is the natural metric of interest. Hence, the focus in this section is on the amount of memory (an number of accesses) require by algorithms to etect flows above a fraction of the total traffic C while incurring a relative error no larger than some rescribe ɛ. For the sake of comarison, we shall assume that stage counters in S 3 are of fixe sie. For its ractical relevance, we shall istinguish between entries an stage counters not only because stages coul be hel in DRAM, but also since, even if imlemente in SRAM, entries may be significantly larger than stage counters (a : relationshi is suggeste in [2]). This suggests the convenience of minimising the sie of the flow memory. S 3 requires three arameters to be set: the stage sie (b), number of stages (m) an the samle threshol (). This gives three egrees of freeom. For what accuracy concerns, the worst case is for a large flow arriving last as iscusse in section 4.8. Let c min be the smallest of the m counters, c..c m, that this flow shall fin an let us start imensioning m an b so that P {c min ɛ} δ () for some ɛ <, δ <. Assuming c..c m i.i. (which is reasonable since hash functions ten to uniformly srea flows at each stage), () is equivalent to P {c i ɛ } m δ. Now, the average number of samles taken altogether is boune above by C =. Since stage counter values are, at most,, we have that E{c i} min(, / ). Since we shall b require b > / an, we have that E{c i} /. b Thus, from the non-negativity of c i, i =..m an by virtue of Markov s inequality it suffices that ` E{c i } m δ for () ɛ to hol. Consiering the boun for E{c i}, the inequality above is satisfie if ` m δ, meaning that stages with ɛ b ` b /m counters suffice. Taking the equality, the total ɛ δ 7 For instance [4] roose a counter management algorithm(cma) for a generalurose counting architecture reviously roose in which DRAM is use to store statistics counte at line rates using counters in SRAM. We irectly count samles in DRAM instea of using small SRAM to uate a large number of counters in DRAM an thus we o not use a CMA. number of stage counters for () to hol is given by M = bm = /mm (2) ɛ δ regarless of the value of >. As a function of m, (2) reners a minimum at m = log(/δ) for any δ >, ɛ >. Taking m = log(/δ) stages means suorting stages with b = e counters, for a total of M = e log(/δ) counters. ɛ ɛ Now, by virtue of lemma 6, for the worst-case relative error to be no larger than some ɛ it suffices that ɛ 2 + ɛ 2 ( δ) + δ = ɛ 2 (3) where ɛ 2 = ɛ 2. This last conition requires (ɛ 2 +/), or (ɛ 2 +), which can always be satisfie. Solving for ɛ in (3), the number of require stage counters is given by M = e s δ (ɛ 2 ɛ 2 ) δ log( δ ) (4) Observations: i) (4) ecreases in r = (ɛ 2 ɛ 2 ), where ɛ 2 < ɛ 2. As a function of δ, (4) exhibits a minimum in (, r ). For values of ɛ of ractical significance (say between 8 an 2 ), δ r well aroximates the location of this minimum. ii) For a given ɛ 2, the only way to increase r is making ɛ 2 small, that is, increasing or, for a given. Hence, (4) is smallest when ɛ =, i.e. when =. In this case, the minimum is for δ ɛ 2 /, which gives the boun M (e/ɛ )log(/ɛ 2 ) for, that oes not een on nor on C. Recall that C increases with either the see of links or with measurement interval lengths. iii) For small ɛ 2, may have to be chosen close to for ɛ 2 to be below ɛ 2, that is, the samling robability may have to be high. However, as link sees increase (right when timing constraints get harer an there are otentially more flows), ɛ can be mae, for the same granularity, significantly smaller than ɛ for or. Thus, S 3 may meet the error boun, with roughly the same number of stage counters that, at small samling robabilities, allowing for an imlementation of the stages in DRAM. iv) The number of counters to uate er samle acket is m. Choosing m as log(/ɛ 2 ) slowly increases with ɛ. For ɛ = %, this is 2 stages an 2 for an error one hunre times smaller. Even in this worst-case analysis this shoul not reclue a DRAM imlementation of the stages, since only samle ackets shall increase counters. v) Several guarantee ɛ 2 < ɛ 2. Increasing increases the see requirements but also r ɛ 2, thereby iminishing (4)). In aition, δ r / otimies in sie. However, b = e/(ɛ) an m = log(/δ) subject to (3) allow multile setus that guarantee the error boun. Thus, in the circumstances that taking m = log(/ɛ 2 ) ut excessive time requirements for a secific memory technology an value of, larger δ an smaller ɛ can be chosen. This is suorting fewer but larger stages. Larger stages shoul not be a concern if DRAM can be use. Note the flexibility of the aroach aating to the secific see/caacity features of the memories. vi) With b e/(ɛ ), m 2log( /ɛ ) an > /ɛ 2, conition b > ( )/m in theorem translates aroximately into < (e/ɛ ) log(/ɛ2 ), which oses no ractical constraint. For instance, for ɛ = %, the constraint is < Plugging the values for b an m, the boun