Decentralized Real-Time Monitoring of Network-Wide Aggregates

Transcription

1 Decetralzed Real-Tme Motorg of Network-Wde Aggregates Rolf Stadler Mads Dam, Alberto Gozalez, Fetah Wuhb KTH Royal Isttute of Techology Stockholm, Swede Large-scale Dstrbuted Systems ad Mddleware (LADIS 8) IBM TJ Watso Research Lab, NY, Sept 15-17, 28 Outle A self-orgazg Motorg Layer Cotuous Motorg of Aggregates wth Accuracy Objectves (A-GAP) Performace comparso gossp vs. tree-based motorg (GAP vs. G-GAP) 2

2 Today s Maagemet Systems for Today s Network Techologes aalyze Maagemet System act observe Maagemet tellgece outsde maaged system. Clear separato betwee maagemet system ad maaged system, by desg. Maaged System 3 Today s Maagemet Systems for Today s Network Techologes (2) aalyze Maagemet System Motorg ad cofgurato, geerally FCAPS fuctos, performed o a per-devce bass. Successful for - small umber of compoets - small rate of chage. Maaged System 4

3 A Maagemet Layer sde the Network aalyze polces exceptos reports 5 A Motorg System for Large-scale Dyamc Evromets 1. Egeer a self-orgazg motorg layer sde the maaged system. 2. Support motorg of aggregates real-tme. across eghborhood, doma, etwork sum, max, average, percetle, hstogram, 3. Provde prmtves for pollg, cotuous motorg, detecto of threshold crossgs. 4. Support cotrollg the performace trade-offs. accuracy, overhead, executo tme, robustess 6

4 Cotuous Motorg of Aggregates wth Accuracy Objectves (A-GAP) 7 The Problem Fd a effcet soluto for cotuous motorg of aggregates large-scale l dyamc etwork evromets -Aggregato fuctos: SUM, MAX ad AVERAGE, -Sample aggregates: total umber of VoIP flows, maxmum lk utlzato, hstogram of curret load across routers a etwork doma Key Applcato Areas: Network Supervso, Qualty Assurace, Proactve Fault Maagemet 8

5 Tradeoff betwee Estmato ad Overhead Overhead Estmato Error Maagemet solutos deployed today usually provde qualtatve cotrol of the accuracy Goal: Cotrol trade-off through error objectve 9 Decetralzed -Network Aggregato Maagemet Stato Computg Aggregates Self-stablzg spag tree Global Aggregate Partal Aggregate Local varable Root 1 1 Physcal Node Aggregatg Node Leaf Node Icremetal, -etwork etwork aggregato Push-based Effcet Operato Local flters coform to error objectve Adapt dyamcally to etwork statstcs

6 Local Adaptve Flters Local varable or partal aggregate Last update value Flter wdth Flter Exceeded: 1) Trggers a update to paret 2) Flter s shfted tme Local flter o a ode Cotrols the maagemet overhead by flterg updates Drops updates wth small chage to partal aggregate Perodcally adapts to the dyamcs of etwork evromet 11 Problem Formalzato Fd flter wdths to motor aggregate for a gve accuracy objectve, wth mmal overhead Overhead: maxmum processg load ω over all maagemet processes Accuracy objectve: average error percetle error maxmum error Mmze Mmze Mmze { } Max ω { } Max ω { } Max ω s.t. E[ E root ] ε s.t. p( E root >γ) θ s.t. E root κ 12

7 A-GAP: A Dstrbuted Heurstc The global problem s mapped oto a local problem for each ode Mmze { ω π } Max s.t. ( ) π E E out ε Attempts to mmze the maxmum processg load over all odes by mmzg the load wth each ode s eghborhood Flter computato: decetralzed ad asychroous Each ode depedetly rus a cotrol cycle: every τ secods { request model varables from chldre compute ew flters ad accuracy objectves for chldre compute model varables for local ode } 13 A Stochastc Model for the Motorg Process Model based o dscrete-tme Markov chas It relates for each ode -the error of ts partal aggregate -evoluto of the partal aggregate -the rate of updates seds -the wdth of the local flter It permts to compute for each ode -the dstrbuto of estmato error -the protocol overhead Updates to paret G Node state F Flter wdth Updates from chldre λ S out E out ω S E Update rate Step szes Estmato Error Update rate (processg load) Step szes Et Estmato Error 14

8 ,35,3,25,2,15,1, ,5 2 1,5 1,5 ode 1 ode 2 ode 3 ode 4 ode 5 ode 6 ode 7 Measured Estmated Stochastc Model (leaf ode) Estmatg step sze (MLE) Evoluto of local varable Trasto Matrx Step Sze Estmato Error Maagemet Overhead X + X F + X F j = otherwse. t j = P( X P( X = ) + P( F = j ) < X < F j ) F, j j = s+ F P( X = z) P( G = s z) s > F z = s F F d + F P( Sout = s) = P( X = z) P( G = d z) s = d = F z = d F otherwse. E = G out λ = ( 1 P( S = )) out 15 A-GAP: Model-based Motorg Error Objectve Estmato Error Estmato model varables Optmzato Problem Stochastc Model of Motorg Process Overhead Updates/sec Step Szes Flter Wdths Aggregate Estmato x (t) Measuremets local varables Tree-based aggregato 16

9 A-GAP: Evaluato through Smulato Overlay topologes -Aboveet: 654 odes, 1332 lks -Grds: 25, 85, 221, 613 odes Aggregate: Number of http flows the doma Traces -From two 1 Gbt/s lks that coect Uversty of Twete to a research etwork Cotrol cycle -τ=1 sec 17 Tradeoff: Accuracy vs Overhead 6 5 ε = ARC Updates/se ec ε =2 ε =5 A-GAP T m = ε =1 ε =15.1 ε =2 Overhead decreases mootocally Overhead depeds o the chages of the aggregate, ot o ts value. A-GAP outperforms a rate-cotrol scheme (ARC) Avg Error 18

10 Scalablty alzed) Updates/sec (orm 1,8,6,4,2 Grd 25 Grd 221 Grd 613 e m Avg Error Mmum error e m creases wth the etwork sze Overhead creases learly wth etwork sze for same error objectve 19 Robustess Estmato Error Tme 175 Maxmum m Load (Updates/sec) Node A fals Ed of Traset Tme 175 Estmato error: several spkes durg sub-secod traset perod Overhead: sgle peak wth a log traset 2

11 Error Predcto by A-GAP vs Actual Error,5,4,3 At ActualError Absolute Avg Error Error Predcted by A-GAP,2, Error 4 Accurate predcto of the error dstrbuto Maxmum error >> average error (oe order of magtude) 21 A-GAP Prototype Lab testbed at KTH 16 motorg odes 16 Csco 26 Seres routers Smartbts 6 traffc geerator A-GAP mplemeted Java Maagemet Stato Aggregato Tree Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 Physcal Network 22

12 Prototype: Maagemet Stato Iterface Select Aggregato Fucto Select Accuracy Objectve Select Root Node Evoluto of the Aggregate (True Value ad A-GAP Estmato) Overhead Dstrbuto ad Evoluto Show Aggregato Tree Real-tme Estmato of Error Dstrbuto ad Trade-off 23 Smulato vs Testbed Measuremets Testbed Expermet Updates/sec Smulato Avg Error e Curves are close: dfferece overhead below 3,5% Prototype valdates smulato mode 24

13 Prototype: Error Estmato by A-GAP vs Actual Error,1,8 Measured Error Error Estmated by A-GAP,6,4,2 Absolute Avg Error, Error 3 Accurate estmato of the error dstrbuto Maxmum error >> average error (oe order of magtude) 25 Prototype: Overhead Estmato by A-GAP vs Actual Overhead 3 2,5 Updates/sec 2 1,5 1,5 ode 1 ode 2 ode 3 ode 4 ode 5 ode 6 ode 7 Measured Accurate estmato of the overhead Estmato teds to be more accurate for odes close to the root Estmated 26

14 Gossp vs. Tree-based Aggregato 27 Gossp protocols Gossp protocols are roud-based, durg each roud a ode radomly selects a subset of eghbors ad teracts wth them. Applcatos - formato dssemato - database replcato - falure detecto - resource dscovery - computg aggregates - 28

15 Computg aggregates wth gosspg Push Syopses [Kempe et al. 3] The protocol computes AVERAGE of the local varables x. After each roud a ew estmate of the aggregate s computed as s /w. Expoetal covergece for uform gossp ad complete graphs Protocol Ivarats: s = x, r, = r r, r, w Roud { 1. s = x ; 2. w = 1; 3. sed ( s, w ) to self } Roud r + 1 { * * 1. Let {( s, w )} be all pars set to l l durg roud r * * 2. s s w w = ; l l = l l 3. choose shares α for all odes j, j such that α = j, j 1 (, j * s, α, j * w 4. for all j sed α ) to each j } 29 The G-GAP protocol Roud { 1. s = x ; 2. w = 1 ; 3. L = {} ; 4. for each ode j ( rs, j, rw, j ) = (,) ; 5. for each ode j ( srs, j, srw, j ) = (,) ; 6. sed ( s, w,,,,) to self; 7. for all j sed (,,,,,) to j } Roud r+1 { 1. Let M be all messages receved by durg roud r 2. s = s( m) + ( xr, xr 1, ) m M ; w = w( m) m M 3. for all j ( acks, j, ackw, j) = (,) 4. L = L org( M) } 5. for all j Neghbors { a. ( rs, j, rw, j) = ( rs, j, rw, j) + (( rs( m), rw( m) acks( m), ackw( m))) morg : ( m) = j b. ( acks, j, ackw, j) = ( srs, j, srw, j) + (( s m ), w ( m )) morg : ( m) = j c. f (detected_falure(j)) {. ( s, w) = ( s, w) + ( rs, j, rw, j). ( rs, j, rw, j) = ( srs, j, srw, j) = (,). L = L \ j } } 6. for all j L { a. choose α such that α 1, j j, j = b. choose β, such that j β = j, j 1 ad β, = c. ( srs, j, srw, j) = β, j( α, s x), β, j( α, w 1) d. sed ( α, js, α, jw, srs, j, srw, j, acks, j, ackw, j) to j e. ( rs, j, rw, j ) = ( rs, j +α, js, rw, j + α, jw ) } 3

16 Accuracy vs. Overhead gossp- ad tree-based aggregato protocol GAP ad G-GAP 654 ode etwork GoCast overlay, coectvty 1 aggregato: AVERAGE UT trace 4 rouds/sec o falures 31 Accuracy vs. Falure Rate gossp- ad tree-based aggregato protocol GAP ad G-GAP 654 ode etwork GoCast overlay, coectvty 1 aggregato: AVERAGE UT trace 4 rouds/sec odes fal radomly, recover after 1 sec 32

17 Summary A self-orgazg motorg layer sde the maaged system -Motorg etwork-wdewde aggregates. -Pollg, cotuous motorg, threshold detecto. -Cotrollg the performace trade-offs. Cotuous motorg of aggregates wth accuracy objectves -Effcet, scalable ad adaptable motorg usg aggregato trees s feasble. -Model-based motorg allows for performace predcto. Tree-based vs. gossp-based cotuous motorg -I a tradtoal wrele etworkg scearo, tree-based aggregato outperforms gossp-based aggregato 33 Refereces F. Wuhb, M. Dam, R. Stadler: Decetralzed Detecto of Global Threshold Crossgs Usg Aggregato Trees, Computer Networks, Vol. 52, No. 9, pp , 28. A. Gozalez Preto, R. Stadler: A-GAP: A Adaptve Protocol for Cotuous Network Motorg wth Accuracy Objectves, IEEE Trasactos o Network ad Servce Maagemet (TNSM), Vol. 4, No. 1, Jue 27. F. Wuhb, M. Dam, R. Stadler, A. Clemm: Robust Motorg of Network-wde Aggregates through Gosspg, 1th IFIP/IEEE Iteratoal Symposum o Itegrated Maagemet (IM 27), Much, Germay, May 21-25, 27. K.S. Lm ad R. Stadler: Real-tme vews of etwork traffc usg decetralzed maagemet, 9th IFIP/IEEE Iteratoal Symposum o Itegrated Network Maagemet (IM 25), Nce, Frace, May 16-19,