Deay tomography for arge scae networks MENG-FU SHIH ALFRED O. HERO III Communcatons and Sgna Processng Laboratory Eectrca Engneerng and Computer Scence Department Unversty of Mchgan, 30 Bea. Ave., Ann Arbor, MI 4809- URSI Genera Assemby 00
Network Mongorng and Dagnoss Deay, Packet Loss Rate, Traffc Type,... Probems wth drect measurement (rmon): Dagnoss unavaabe or dsabed at nterna nodes. Non-cooperatve nterna nodes. A nterna nodes must be synchronzed
Network Tomography Probem End-to-End Measurements Actve vs. Passve Method Actve Method: Send probe packets Passve Method: Montor exstng fows 3
Importance of Lnk Deay Statstcs Assessment and updatng of routng/fow contro QoS assurance, especay for vdeo/audo streamng Network upgrade/mantenance pannng Securty, e.g., dstrbuted Dena-of-Servce (DoS) attacks 4
Probem Formuaton: Genera Notatons Logca Tree T=(V,E) V : Nodes, E : Lnks L nks, R eaf nodes/probe paths root N M : # of packets sent from root to eaf : The set of nks n probe path. (, n) X : nth probe packet deay at nk aong path, ncudng queueng deay, retransmsson deay,......... R Y (, n) and possby propagaton deay. = M X (, n) : nth End-to-end probe deay aong path. 5
Probem Formuaton: Genera Assumptons Network Assumptons N) Network topoogy known. N) Probe paths (routng tabe) known. N3) Cooperatng edge nodes are synchronzed Statstca Assumptons S) Spata Independence { (, n) X } For a gven packet aong path, mutuay ndependent. S) Tempora Independence and Statonarty M (, n) ( k X X If path and both contan nk, and jn, )..d. 6
Dscrete Deay Mode Lnk deays are dscretzed wth bn sze q (, n) Lnk deay vaues X { 0, q, q, L, qd} Lnk Deay P.M.F., d Lemma. ( (, n) ) p = P X = d The deay p.m.f. wth two bns at each nk s unquey dentfabe from end-to-end packet deays, except when the deay p.m.f. s at a nks are dentca. A = n + 3 3 p 3 + + + 3 3 p p p p ( ) ( p ) p p Q Q Q Q p p ( p ) Q p p Q + + p p ( p ) Q p p Q 3 3 Q = p ( p ) + p ( p ) Q = p ( p ) + p ( p ) 0 0 + + 7
Contnuous Deay Mode: Gaussan Mxture Arbtrary shapes of nk deay dstrbutons Let f ( x) be the nk deay p.d.f at nk. k = k m= α φ( x; θ ) m, m, : the number of mxture components. α, m : mxng probabty for the mth component. 0 α, α =, m m =, m φ( x; θ ) : Gaussan densty functon wth mean and, m k { } varance θ = µ, σ, m, m, m 0.5 φ (0, ) + 0.3 φ(, ) + 0. φ(5, 4) 8
Contnuous Deay Mode: Identfabty Probem t k, k, k = Exampe: Two eaf tree. Le 3 { } f( y,y )= φ ( y ; µ + µ, σ + σ ) { + + } φ ( y ; µ µ, σ σ ) 3 3 µ = µ + µ Y Y Y Y µ = µ + µ 3 σ = σ + σ σ = σ + σ 3 source 3 4 equatons wth 6 unknowns! recever : y recever : y 9
Mxed Fnte Mxture Mode ρ Utzaton factor of a queueng system 0 < for stabe system. P(Queue s empty) = = ρ ρ α Introduce a deta component at (or near) 0 wth probabty mass α 0 k Lnk deay p.d.f. becomes f ( x) α δ( x) + α φ( x; θ ),0 m= m, m, Suffcent condton for dentfabty (asymptotc) The deay dstrbuton defned above s dentfabe from end-to-end measurements f () α > 0 for a () A the,0 Gaussan components n nk deay dstrbutons have dstnct means and varances. k m= 0 = α m, = 0 0
Mxed Fnte Mxture Mode: Exampe f ( x ) = 0. δ( x ) + 0.9 φ( x ; 0, ) x f ( x ) = 0.3 δ( x ) + 0.7 φ( x ; 4, ) x f ( x ) = 0.03 δ( x ) + 0.7 φ( x ; 0, ) + 0.07 φ( x ; 4, ) + 0.63 φ( x ; 4, 3) x + x 3 3 3 3 3
EM Estmaton Agorthm: Notatons Assume pror knowedge of Component ndcator vector { } ( n, ) ( n, ) z = x m m, k (, n ) (, n ) (, n ) (, n =,, L, ),0,, k { } z z z z f s generated by the th component, z ( n, ) m, = 0 otherwse { } { ( n, )} { ( n, ) Unobserved data,,, } xz x= x z= z { ( n, )} Observed data y = y { xyz} Compete data,, Parameter vector Θ= αm,, θm, { }
EM Estmaton Agorthm Compete data kehood og L( xz, Θ) og L( xz, Θ ) = L { = : M n= k m= z N z ogα (, n),0,0 + ( n ogα + og φ( x ; θ )) (, n) (, ) m, m, m, Let ω = E z y ; Θ ( n, ) ( n, ) ( n, ) m, m,, Θ t ( n, ) ( n, ) ( n, ) ( n, ) Q ( θ ) = E z og φ( x ; θ ) y ; Θ m, m, m, m, t t 3
E-Step E EM Estmaton Agorthm og L( xz, Θ ) y; Θ = t L = : M n= N { } ω og α + Q ( θ, Θ ) k k ( n, ) ( n, ) m, m, m, m, m= 0 m= t M-step α θ t+ m, = : M : M n= ( n, ) m, arg max Q ( θ, ) t+ N (, n) m, θ : M n= m, N N ω = Θ t 4
Computer Experment Matab Smuaton wth 5000..d. end-to-end deays for each probe path. source Numbers of Gaussan mxture components and true/estmated deta factor α,0 3 Lnk 3 4 5 6 7 k 3 4 5 6 7 α,0 0.5 0.3 0. 0. 0.5 0.3 0. αˆ,0 0.53 0.304 0.099 0.99 0.5 0.33 0.0 5
True (sod) and estmated (dotted) Gaussan mxture components. 6
Concuson and Extensons Concusons Dscusson of dscrete and contnuous deay modes. Proposed mxed fnte Gaussan mxture mode for nk deay. EM agorthm mpementaton wth known mode orders. Extensons Unsupervsed mode order estmaton. Adaptve agorthm for parameter and mode order update. 7
References F. L. Prest, N. G. Duffed, J. Horowtz, D. Towsey, Mutcast-based nference of network-nterna deay dstrbutons, Umass CMPSCI 99-55, 999. Logca mutcast tree. Dscrete nk deays wth fnte eves. Canonca deay tree,.e., there s a nonzero probabty that a probe experences no deay n traversng each nk. Sampe-average approach. Identfbty s proved by showng bjecton mappng exsts from the nk deay dstrbutons to the probabtes of the events n whch the end-to-end deay s no greater than q for at east one recever. Contnuous mode s dscussed, but dentfabty probem s eft open. M. Coates and R. Nowak, Network tomography for nterna deay estmaton, ICASSP 00, Sat Lake Cty, May 00. Logca uncast tree. Dscrete nk deays wth fnte eves. Back-to-back packet par measurements. MLE usng EM-based agorthm. Sequenta Monte Caro trackng of tme varaton. 8