6.3 Buffer Space versus Number of Flows

Transcription

1 6.3 Buffer Space veru Number of Flow What to do when the number of flow i o large that it become impractical to allocate a eparate flow-control window for each one of them Per-detination flow merging: Sapountzi and Kateveni, IEEE Communication Magazine, Jan. 2005, pp Dynamically haring the buffer pace among the flow: the ATLAS I and the QFC flow control protocol, and it evaluation Regional Explicit Congetion Notification (RECN) Requet-Grant protocol: N. Chryo, 2005 End-to-end congetion control via requet-grant: N. Chryo, 2006 Ethernet Quantized Congetion Control Job-ize aware cheduling and buffer management Other buffer haring protocol of the mid-90 Buffer memory cot veru tranmiion throughput cot in 995 CS Copyright Univerity of Crete

2 Buffered Switching Fabric with Internal Backpreure large off-chip buffer (DRAM) mall on-chip buffer backpreure Performance of OQ at the cot of IQ, Require per-flow backpreure. 2

3 Cell Ditribution Method Aggregate traffic ditribution: Randomized routing (no backpreure) Adaptive routing (indicriminate backpreure) load balancing on the long-term only ame input different output ditribute cell re-equence cell Per-flow traffic ditribution: Per-flow round-robin (PerFlowRR) Per-flow imbalance up to cell (PerFlowIC) accurate load balancing, on a horter-term bai 3

4 Too many Flow Per-output Flow Merging N 2 per chip in the middle tage Retain the benefit of per-flow backpreure N flow per link, everywhere Re-equencing need to conider flow a they were before merging Freedom from deadlock 4

5 The ATLAS I Credit Flow-Control Protocol M. Kateveni, D. Serpano, E. Spyridaki: Credit-Flow-Controlled ATM for MP Interconnection: the ATLAS I Single-Chip ATM Switch, Proc. IEEE HPCA-4 (High Perf. Comp. Arch.), La Vega, USA, Feb. 998, pp ; Feature: identically detined traffic i confined to a ingle lane each lane can be hared by cell belonging to multiple packet A oppoed to Wormhole Routing, where: a virtual circuit (VC) i allocated to a packet and dedicated to it for it entire duration, i.e. until all flit (cell) of that packet go through identically detined packet are allowed to occupy ditinct VC CS Copyright Univerity of Crete 5

6 Q F C -like C red it P ro to co l Q u a n tu m F lo w C o n tro l (Q F C ) A llia n c e : p ro p o e d ta n d a rd fo r c re d it-b a e d flo w c o n tro l o v e r W A N A T M lin k A T L A S I: im ila r p ro to c o l, a d a p te d to h o rt lin k h a rd w a re im p le m. d e tin. 0 2 d tc r b b b p o o lc r B c re d it i i: d t 0 2 b u ffe r i B - D - b c e ll uptream w itch dow ntream w itch b o th k in d o f c re d it a re n e e d e d fo r a c e ll to d e p a rt N u m b e r o f L a n e L = (in A T L A S I: b = ) B b CS Copyright Univerity of Crete 6

7 S a tu ra tio n T h ro u g h p u t 64x64 fabric: 6-tage banyan uing 2x2 elem ent 20-cell or 20-flit burt, uniform ly detined S a t u r a t i o n T h r o u g h p u t A T L A S W o r m h o l e B u ffe r S p a c e (= L a n e ) p e r L in k c e ll o r flit (B = L, w ith b= ) CS Copyright Univerity of Crete 7

8 N o n -H o t-s p o t D e la y, in th e P re e n c e o f H o t-s p o t D e t in a t io n n o n -h o t-p o t lo a d = 0.2 ; 2 0 -ce ll/flit b u rt; 6 4 x6 4 fa b ric: 6 -tg b a n ya n w. 2 x2 e l D e l a y ( c e l l t i m e ) A T L A S, 2 h t- p A T L A S, h t- p 0 W o rm h., 2 h t- p W o rm h., h t- p W o rm h., n o h t- p A T L A S, n o h t- p N u m b e r o f L a n e (L ) (w ith b u ffe r p a ce B = 6 ce ll o r flit p e r lin k) CS Copyright Univerity of Crete 8

9 5 ATLAS I T R A N S C E I V E R S 4 im p. m a tc h Single-chip ATM Switch with Multilane Backpreure 0 Gbit/ = Mb//port Shared Buffer 0.35 µm CMOS , FORTH-ICS, Crete, GR R o u tin g T a b le V P /V C T ra n la t. T a b le L o a d M o n hdr p a ttrn m a tc h Q u e u e M a n a g. c r. r c h. d e l e r g e n S c h e d C o u n t d e l C re d it T a b le c h ip c o n tro l & m a n a g e m e n t im p. m a tc h C e ll B u ffe r & S w itc h in g P ip e lin e d M e m T R A N S C E I V E R S 0 mm

10 C o re : C e ll B u ffe r & S w itch in g H e a d e r P r., R t'n g, T ra n l. C re d it-b a e d F lo w C o n tro l D e ig n E ffo rt 5 % 2 0 % G a te 20% FF 20% 20% S R A M 20% A re a 0 % 0 % P o w e r 2 5 % Q u e u e P o in te r M a n a g 'm n t 5 % 50% 0 % S ch e d u lin g, P o p. C o u n t 3 % C trl/m g t, L o a d M o n, M ic. E la tic b u f., I/O L in k In tf. 7 % 42% 45% 25% 3 % p -y r K K K b it 2 5 % w ir in g 4% 4 5 % 2 5 % P e rip h e ry : G ig a B a u d T ra n ce ive r P a d & D rive r 0 % 0 % m m 9. W CS Copyright Univerity of Crete 0 2

11 Backpreure Cot Evaluation, veru Alternative Meaure the cot of credit flow control in ATLAS & compare to: Alternative, without internal backpreure in the fabric: large buffer (off-chip DRAM) in all witche throughout the fabric, or internal peedup in the fabric and output buffer Kornaro, Pnevmatikato, Vatolaki, Kalokerino, Xanthaki, D. Mavroidi, Serpano, Kateveni: ATLAS I: Implementing a Single-Chip ATM Switch with Backpreure, IEEE Micro Magazine, Jan/Feb. 999, CS Copyright Univerity of Crete

12 M aking Large A T M S w itc he: S w itch in g F ab ric w ith In tern al B ackp reu re in p u t in te rfa ce in p u t q u e u e (p e r-flo w ) A T L A S A T L A S A T L A S N N la rg e, o ff-ch ip b a ckp r. b a ckp re u re b a ckp re u re b u ffe r (D R A M ); to ta l th ro u g h p u t = 2 N m a ll, o n -ch ip b u ffe r CS Copyright Univerity of Crete 2

13 S w itc h in g F a b ric w ith o u t B a c k p re u re : L a rg e B u ffe r N N la rg e, o ff-ch ip b u ffe r (D R A M ); to ta l th ro u g h p u t = 2 N lo g N CS Copyright Univerity of Crete 3

14 S w itc h in g F a b ric w ith o u t B a ckp re u re 2 : In tern al S p eed u p o u tp u t q u e u e o u tp u t in te rfa ce N N m a ll, o n -ch ip b u ffe r la rg e, o ff-ch ip b u ffe r (D R A M ); to ta l th ro u g h p u t = (+ )N p eedup > ; under b ur ty o r non-uniform traffic : > >... CS Copyright Univerity of Crete 4

15 B a ckp re u re C o t/b e n e fit : N o B a c k p re u re, L a rg e O ff-c h ip B u ffe r C o re : G a te FF S R A M A re a P o w e r C e ll B u f.& S w itc h in g 20% 20% 2 0 % x0 0 % x.6 H d r, R t'n g, T ra n l. C re d it-b. F lo w C trl Q.P tr, S c h, C trl, e tc. 20% 50% 0 % 0 % x0 2 5 % x.5 E la tic b u f., I/O In tf. 4 2 % % % x0 6 % + 4 % x0 2 5 % + w irin g P e rip h e ry : O ff-c h ip 5 5 % x 2 C o m m u n ic a tio n 3 5 % x 2 C o t CS Copyright Univerity of Crete 5

16 B a ckp re u re C o t/b e n e fit 2 : N o B a c k p re u re, In te rn a l S p e e d u p, O u tp u t Q u e u e C o r e : G a te FF S R A M A r e a P o w e r C e ll B u f.& S w itc h in g % xs 20% xs 20% 0 % ++ H d r, R t'n g, T r a n l. C r e d it- b. F lo w C tr l Q.P tr, S c h, C tr l, e tc. 20% 50% 0 % 0 % x0 2 5 % xs 2 E la tic b u f., I/O In tf. 42%xS 45%xS 25%x0 4 % x % w irin g P e r ip h e r y : O ff- C h ip 5 5 % x S C o m m u n ic a tio n 3 5 % xs C o t CS Copyright Univerity of Crete 6

17 Regional Explicit Congetion Notification (RECN) Generalization & evolution of the ATLAS/QFC protocol Source-routing header decribe path through fabric Intermediate or final link congetion end back-notification All packet to congeted link confined to a ingle lane intermediate link identified via path component in header entire tree of detination in ingle lane (improvement over QFC) equivalent of lane here called Set-Aide Queue (SAQ) VOQ replaced by Single (!) Input Queue + SAQ dynamically create/delete SAQ CAM aumed to match incoming pck header veru current SAQ Duato, Johnon, Flich, Naven, Garcia, Nachiondo: A New Scalable and Cot-Effective Congetion Management Strategy for Lole Multitage Interconnection Network, HPCA-, San Francico, USA, Feb CS Copyright Univerity of Crete 7

18 Requet-Grant Protocol Conider a buffer feeding an output link, and receiving traffic from multiple ource: If credit are pre-allocated to each ource, the buffer need to be a large a one RTT-window per ource; If credit are held at buffer and only allocated to requeting ource() when thee have omething to tranmit, then a ingle RTT-window uffice for all ource! economize on buffer pace at the cot of longer latency N. Chryo, M. Kateveni: Scheduling in Switche with Small Internal Buffer, IEEE Globecom 2005, St. Loui, USA, Nov. 2005; N. Chryo, M. Kateveni: Scheduling in Non-Blocking Buffered Three- Stage Switching Fabric, IEEE Infocom 2006, Barcelona, Spain, Apr. 2006; CS Copyright Univerity of Crete 8

19 Buffered Fat-tree / Clo fabric: implified view Congetion elimination uing proactive admiion Enure all injected packet can fit in fabric-output buffer fabric output buffer never exert backpreure Output buffer credit requeted by input cheduled by output arbiter injection rate are fair end2end no parking lot problem With good traffic ditribution (invere-multiplexing) intermediate buffer never fill up a well backpreure eliminated! Minimal in-fabric packet delay (queue uually empty) & no packet drop low flow completion time CS Copyright Univerity of Crete 9

20 Hotpot Traffic: the load for two output i. x C Indicriminate bckpr: cell to non-hotpot det have very large delay Req-grant bckpr. perform a per-flow queue, but ue hared queue O(N) le queue Higher latency at low load due to requet-grant mg Need to handle contention among requet uing per-flow req. cnt CS Copyright Univerity of Crete 20

21 Ethernet Quantized Congetion Control (QCN) Congetion w. queue: every ~ 00 new frame, compute congetion feedback value (fd) fd = Qoffet + w dq if fd > 0, iue congetion notification mg (CNM) to rc NIC, identifying MAC-pair flow Non-congeted flow hare a ingle NIC queue Upon receiving a CNM for a det MAC: alloc. flow queue & rate limiter (RL) target = current RL new RL = f ( RL, fd ) Recovery upon abence of CNM: fat recovery: RL = (RL+ target) / 2 active increae: increment target (+x Mbp) Additive Increae / Multiplicative Decreae (AIMD) CS Copyright Univerity of Crete 2

22 QCN at output i unfair: who to get the next CNM? QCN CP at output veru. at input under fan-in cenario next CNM goe to the unlucky flow that ent the nxt 00-th frame QCN at input fair rate QCN at output CS Copyright Univerity of Crete 22

23 Arrival-ampling QCN at input doe not protect non-congeted flow Departure from witch input buffer (VOQ) not FIFO (order enforced by arbiter) Arrival ampling: CNM flow of nxt 00th frame flow CNM ed proportionally to arrival rate unfair with non-fifo ervice out of buffer Occupancy ampling: CNM the flow w. larget backlog (~ arrival rate minu departure rate) Occupancy ampling Arrival ampling Chryo e.a. Unbiaed QCN for calable erver-rack fabric, IBM Technical Report: CS Copyright Univerity of Crete 23

24 Small & flat FCT for cale-out latency-enitive app Traditionally, congetion control care about tput, fairne of congeted link b/w & latency of non-congeted pkt congeted pkt high latency Main perf. metric of interet when a group of erver collectively work on latency-enitive (datacenter) application flow completion time (FCT) -- mall for a many flow a poible Meaure/action TCP variant: keep in-fabric queue empty (Alizadeh, SIGCOM 200) datacenter network rethink loy (current) v. lole flow control avoid delayed TCP retranmiion: up to many milliecond to recover pkt hift alo from determinitic routing to flow-level (and more recently alo to packet-level) multipathing M. Al-Fare, SIGCOMM 08, Zat SIGCOMM 2) what about flow croing temporarily congeted path? proactive (cheduled injection) alo drawing renewed interet don t blindly inject packet--cheduled injection reduce in-fabric backlog hot (network-tack) latencie: many ten of μ; ok for previou netw. (latency 00 μ) but new net only fewμ avoid exceive packet copie (RDMA?), TCP offloading CS Copyright Univerity of Crete 24

25 Shortet-job-firt cheduling: let mall go firt Alizadeh, e.a., pfabric: Minimal Near-Optimal Datacenter Tranport Switche tore packet in per-input priority queue orted baed on the remaining ize of the correponding TCP flow but how to find the remaining ize of a TCP flow? heuritic: ue the already tranmitted ize a indication chedule firt packet from the flow with the mallet remaining ize when buffer exceed threhold, drop packet from the flow with the larget remaining ize Cot of priority queue commodity (cheap) witche have mall on-chip buffer (a few hundred of 500B frame per input) h/w comparator for a few-hundred value i feaible CS Copyright Univerity of Crete 25

26 Virtual witche in cloud (computing) datacenter Another layer of witching inide hot (likely in /w veratile/flexible) VM vnic vswitch NIC fabric NIC vswitch vnic VM Ping between two remote phyical erver: 2 μ; ping between two collocated VM: 22 μ vswitche have oft capacity link: tranfer pkt when on CPU exceive packet drop inide det hot (Crian, e.a., SIGCOMM 203) offloaded vswitch function inide hardware network adapter (SRIOV) good but not fully SDN compatible Lole vswitche (Crian, e.a. SIGCOM 203) vswitch backpr. can propagate inide phyical network exceive blocking reerve endpoint vswitch credit before injecting pkt from hot Crian, Birke, Chryo, Minkenberg, Guat, zfabric: how to virtualized lole Ethernet, IEEE Cluter 204 CS Copyright Univerity of Crete 26

27 Buffer Space for Bounded Peak-to-Average Rate Ratio Aume R peak (i) / R average (i) PAR for all flow i on a link R(i) i the rate (throughput) of flow i PAR i a contant: peak-to-average ratio bound interpretation: rate fluctuation i bounded by PAR Each flow i need a credit window of RTT R peak (i) Buffer pace for all flow i ( RTT R peak (i) ) = = RTT ( R peak (i) ) RTT ( PAR R average (i) ) = = PAR RTT ( R average (i) ) PAR ( RTT R link ) Allocate buffer pace = PAR number of window When individual flow rate change, rearrange the allocation of buffer pace between flow but mut wait for the buffer of one flow to drain before rallocating it (not obviou how to) H.T. Kung, T. Blackwell, A. Chapman: Credit-Baed Flow Control for ATM Network: Credit Update Protocol, Adaptive Credit Allocation, and Statitical Multiplexing, SIGCOMM '94, pp CS Copyright Univerity of Crete 27

28 Dynamically Sharing the Buffer Space among Flow In order to depart, a packet mut acquire both: a per-flow credit (to guard againt buffer hogging ), and a per-link credit (to enure that the hared buffer doe not overflow) Properly manage (increae or decreae) the per-flow window allocation baed on traffic circumtance: ATLAS and QFC protocol never change the per-flow window H.T.Kung protocol move allocation between flow (unclear how) other idea: ue two window ize a full one and a mall one; ue full-ize window when total buffer occupancy i below a threhold, ue mall-ize window (for all flow) above that point (flow that had already filled more than a mall window will loe their allocation on packet departure) C. Ozveren, R. Simcoe, G. Varghee: Reliable and Efficient Hop-by-Hop Flow Control, IEEE JSAC, May 995. Draining Rate Theorem: M. Kateveni: Buffer Requirement of Credit- Baed Flow Control when a Minimum Draining Rate i Guaranteed, HPCS'97; ftp://ftp.ic.forth.gr/tech-report/997/997.hpcs97.drain_cr_buf.p.gz CS Copyright Univerity of Crete 28

29 C o m m u n ic a tio n C o t v e r u B u ffe r/l o g ic C o t O n -C h ip : m illio n o f tra n i to r - h u n d re d o f p in O ff-c h ip : d a ta fr o m H o t In te r c o n n e c t '9 5 k e y n o te p e e c h, b y A. F r a e r, V P, A T & T B e ll L a b : p e e d o f tra n m i io n lin e 45 M b / c o t o f lo n g d i ta n c e x m i io n 45 $ /m ile /m o n th p e e d o f ig n a l p ro p a g a tio n 7 m ic ro e c /m ile ro u n d -trip w in d o w iz e 79 b y te /m ile c o t o f 6 M B y te D R A M $ c o t o f w in d o w iz e m e m o ry 0.5 c e n t /m ile in v e tm e n t w rite -d o w n p e rio d 36 m o n th c o t o f q u e u e m e m. p e r m o n th c e n t /m ile /m o n th ra tio : tra n m i io n /m e m o ry c o t 3 3 0,0 0 0 to CS Copyright Univerity of Crete 29

30 P e r-c o n n e ctio n Q u e u e in g & F C : H o w m a n y ``W in d o w '' o f B u ffe r S p a c e? w in d o w S iz e (V C i) = R T T * p e a k T h ro u g h p u t(v C i) w in d o w S iz e (V C i) < o r < < w in d o w L := R T T * th ro u g h p u t(l in k ) c o t(l in k ) ~ = 3 3 0,0 0 0 * c o t(w in d o w L ) lo y flo w c o n tro l u u a lly o p e ra te th e n e tw o rk w ith g o o d p u t re a c h in g u p to % o f lin k th ro u g h p u t lo le flo w c o n tro l o p e ra te u p to % lin k u tiliz a tio n th e % e x tra u tiliz a tio n w ith lo le F C i w o rth a p p ro x. 0 to 0 0 th o u a n d w in d o w L ' w o rth o f e x tra b u ffe r m e m o ry if lo le flo w c o n tro l c a n y ie ld it lin k u tiliz a tio n a d v a n ta g e w ith le th a n a fe w te n o f th o u a n d o f w in d o w L ' o f e x tra b u ffe r m e m o ry, th e n lo le flo w c o n tro l i a c le a r w in in d e e d, lo le F C c a n d o th a t, e v e n w ith q u ite le b u ffe r p a c e... CS Copyright Univerity of Crete 30