Scaling Network Verification using Symmetry and Surgery

Transcription

1 Scaling Network Verification using Symmetry and Surgery Gordon D. Plotkin Nikolaj Bjørner Nuno P. Lopes Andrey Rybalcenko George Vargese LFCS, University of Edinburg, UK Microsoft Researc, UK Microsoft Researc, USA Abstract On te surface, large data centers wit about 100,000 stations and nearly a million routing rules are complex and ard to verify. However, tese networks are igly regular by design; for example tey employ fat tree topologies wit backup routers interconnected by redundant patterns. To exploit tese regularities, we introduce network transformations: given a reacability formula and a network, we transform te network into a simpler to verify network and a corresponding transformed formula, suc tat te original formula is valid in te network if and only if te transformed formula is valid in te transformed network. Our network transformations exploit network surgery (in wic irrelevant or redundant sets of nodes, eaders, ports, or rules are sliced away) and network symmetry (say between backup routers). Te validity of tese transformations is establised using a formal teory of networks. In particular, using Van Bentem- Hennessy-Milner style bisimulation, we sow tat one can generally associate bisimulations to transformations connecting networks and formulas wit teir transforms. Our work is a development in an area of current wide interest: applying programming language tecniques (in our case bisimulation and modal logic) to problems in switcing networks. We provide experimental evidence tat our network transformations can speed up by 65x te task of verifying te communication between all pairs of Virtual Macines in a large datacenter network wit about 100,000 VMs. An all-pair reacability calculation, wic formerly took 5.5 days, can be done in 2 ours, and can be easily parallelized to complete in minutes. D.2.4 [Software Engineer- Categories and Subject Descriptors ing]: Software/Program Verification General Terms Keywords 1. Introduction Verification, Teory Network Verfication, Symmetries, Semantics Cloud services suc as Dropbox, Google, icloud, Amazon, and Azure contain up to a million inexpensive servers connected by a data center network. A single service request suc as a Web searc is split among undreds of servers tat communicate to produce te response. As our dependency on networks increases, te reliability of data center networks becomes increasingly critical. However, surveys sow [19, 31] tat network outages are quite common. Tus verification tecnologies tat proactively prevent potential network disruptions are valuable. A network, as sown in Figure 1, consists of boxes (routers, switces, firewalls, encefort referred to as routers) tat forward packets from input ports to output ports. We abstract te dataplane, or forwarding component of a router, as a set of rules tat map predicates on packet eaders (e.g., 32-bit IP addresses starting wit 101) to output ports; some rules called Access Control Lists (ACLs) use complex predicates (e.g., sources from outside Microsoft tat send SQL packets) on sets of fields to decide wic packets must be dropped. Te rules may also prescribe canges in packet eaders. Te output ports of eac router are pysically connected to te input ports of oter routers as specified by a network topology. Te network program, te composition of all te router programs following te topology, ten maps packets from entry points in te network troug te network to exit points. Tis abstraction only enables qualitative reasoning, for example about packet reacability or about looping. It abstracts away quantitative issues, suc as performance metrics (e.g., delay) and ignores te control plane tat builds te forwarding rules. Te correctness requirements [17] are also simple: tey specify te eaders tat can communicate between osts, togeter wit predicates on te pats te eaders traverse. So werein, terefore, lies te complexity tat justifies te emerging area of network verification [15 17, 22, 30]? First, manual rules added by operators interact wit te rules computed by automatic routing protocols. Second, te forwarding rules typically involve load balancing by wic a packet can be sent by many possible routes to its destination. Tird, routers sometimes rewrite packets. Fourt, wile te program structure is simple, te state space is large. Forwarding rules operate on at least te IP and TCP eaders and oter fields suc as MPLS [18]. Tus conservatively, we ave a space of 2 80 eaders, millions of rules, and a large number N of servers (as many as 1 million) tat can communicate. Cecking reacability across N 2 pairs of stations for 2 80 eaders to find a network policy violation is a scaling callenge. Early systems [22] found a single violation using SAT solvers. Later systems [15, 17] found all violations for medium sized networks using symbolic execution. NetPlumber [16] introduced efficient incremental analysis. Finally, Yang and Lam [30] used predicate abstraction to rewrite router rules in a form tat is muc more efficient to verify. However, tese improvements do not target te scaling callenge. For example, it was reported in [10] tat te cost of verifying reacability for all pairs of stations took around 1 day, even on a small university network. Verifying all pairs in a large data center in a few minutes (te time scale at wic a network can be reconfigured) seemed out of reac for earlier tecniques. 1.1 Scaling Verification Using Network Transformations Fortunately, most large networks are designed using design patterns tat enforce regularities tat we can exploit. For example, data center networks are often arranged in fat-trees (Figure 1), wic

2 Figure 1. Data center network scematic Core Aggregation Edge Leaf are leveled graps were routers at a given level are symmetrically connected to multiple routers at adjacent levels. Tis is done for load balancing and resiliency reasons. For example, in Figure 2 R 3 and R 4 are symmetrically placed, for if tey are intercanged and te ports on R 3, R 4 and te corresponding ports of teir neigbors R 1, R 2 and R 5 are correspondingly intercanged, ten te global topology stays te same. Tus, by design, one may expect te rule sets of symmetrically placed routers at a given level (e.g., R 3 and R 4) to be symmetrical. Tis suggests tat symmetrical routers suc as R 3 and R 4 can be replaced by a single equivalent (wit respect to packet forwarding) router as in Figure 2 witout canging te qualitative properties of te network. Doing so reduces te rules in te new network, and in te limit can transform a fat tree into a tin tree. R 1... Z R 2... R 3 R 4... p 1 p 2 X R 5 Y Transforms to Z R 1 R 2 Figure 2. Suppose tat intercanging R 3 and R 4 as well as corresponding marked ports on teir neigbours, e.g., p 1 and p 2 on R 5, leaves reacability te same. Ten R 3 and R 4 can be merged into a single router in te transformed network on te rigt. Furter, te ierarcical structure implies tat communication witin subtrees sould stay local. Tus two stations X and Y attaced to te same edge (or ToR, for Top of Rack switc) R 5 (see Figure 2) sould typically communicate only via R 5. Tus, for verification of te communication between X and Y, te rest of te network is irrelevant as long as one can prove tat traffic from X to Y does not leak out from te ToR R 5. Tis suggests a general notion of slicing away irrelevant rules and portions of te network. We refer to suc slicing of ports and rules as network surgery. One can also slice networks in terms of eaders. For example, if in reality te two backup routers R 3 and R 4 of Figure 2 are nearly identical except for two local addresses 1 in R 3 and 2 in R 4, we migt ope to slice te network into two equivalent networks operating on disjoint sets of eaders, one of wic as R 3 and R 4 perfectly symmetric, and one of wic as te standard topology but wit a smaller set of rules (say dealing wit only 1 and 2 ). Bot symmetry-induced and slicing-based transformations are network transformations tat transform te original network into a sequence of simpler networks wit equivalent forwarding, but wit smaller size in terms of boxes, rules, and links. If te verification X R 3 R 5 Y effort scales wit size, and te transformations are efficient, te overall verification time will be faster. To proceed, we need a model of networks and a way to write reacability specifications. To tis end we use an operational semantics treating eac router as a state macine. We ten use a standard modal logic defined on te states to describe desired beaviors. Te logic permits assertions tat certain (anonymous) transitions occur, tat a packet eader as reaced a certain port, or tat a packet eader as a certain property. To connect network transformations wit te logic we generally sow tat tey yield bisimulations (in te sense of Van Bentem and Park and Milner [28]) between a network N and its transform N. We ten employ te Van Bentem-Hennessy-Milner principle, as used in modal logic and process calculus [24, 28], to sow te validity of a transformation using bisimulation. In process calculus tis principle states tat if tere is a bisimulation between two processes P and P, ten P as a given property ϕ, if, and only if, P does. In our case, as, for example, N may ave different ports from N, it may be necessary to modify te formula ϕ asserted of N to anoter, ϕ, asserted of N (te analogous situation in process calculus would be wen comparing processes wit different action alpabets, see, e.g., [4]). We propose two different ways to employ te Van Bentem- Hennessy-Milner tecnique for network verification. One is to reduce te size of N, for example, by slicing away parts of te network irrelevant to te proposition, or else by exploiting symmetry eiter to remove essentially duplicate rules, or to merge symmetrically placed routers wit symmetrical semantics. Te oter is to reduce te number of properties to be verified, by finding symmetries between propositions about symmetrically placed parts of te network or by identifying equivalent packet eaders. In doing so, it may be useful to pass troug a sequence of suc transformations, producing a sequence N n 1 N n of bisimilar networks. Our contributions (wit a paper outline) are: 1. A Teory of Network Dataplanes: To formalize network transformation, we introduce a network model (Sections 2.1, 2.2), a modal logic (Section 2.3), and a proof tecnique based on bisimulation to verify networks (Section 3). After describing surgeries, we describe a compositional way of generating bisimulations (Section 5). A side effect of all tis macinery is te ability to formalize earlier concepts suc as slicing [15], and (a generalized version of) Yang-Lam equivalence [30]. 2. A Toolbox of Network Transformations: We provide a toolbox of network transformations (Examples 1 troug 4) based on surgery (Section 4) and eader merging and symmetry (Section 6) tat can be applied iteratively to simplify te verification task. Tere are generally bisimulations corresponding to eac. As wit earlier work (e.g., [7]), we exploit symmetry over state spaces. However, to do tis we exploit te structure of te network domain were te state consists of bot eaders and locations, and te program is distributed over boxes and rules, enabling symmetries to be constructed from bot topological and semantic components. 2. Scaling Compreensive Network Verification: We sow (Section 7) ow one migt scale compreensive network evaluation for large production data centers over all pairs of stations. In particular, for a Microsoft data center in Singapore wit 820K rules, we reduce te time to verify reacability between all pairs of VMs from 5.5 days to 2 ours on a single core using only simple transformations, a 65 reduction. Earlier work [15 17, 30] did not report te running times for reacability between all N 2 pairs. 2. Networks and Teir Logics As a foundation for our teory, we provide a formal definition of switcing networks as graps of interconnected boxes of various

3 kinds, suc as routers, switces, bridges, or firewalls. Tis is network syntax. Having a syntax available, we ten define network semantics in terms of suitable kinds of transition relations. Te semantics, in its turn, supports a suitable modal logic, wic can be used to specify network properties tat we wis to verify. 2.1 Network Syntax Networks are constructed using a box signature tat consists of a set of boxes b Box, wit eac box aving a given finite collection K fin N of ports, written b : K; we write Box for te signature. (We use a set, rater tan a sequence, of ports to prepare te ground for network surgery in wic we slice away ports to transform a network.) Given suc a box signature, a network N consists of a finite set Node N of nodes, a box assignment function β N : Node N Box N, a symmetric, 1-1, irreflexive, port connection or link relation γ N on te set rt N of network ports were: rt N = def {(n, i) n Node N, β N (n) : K, and i K} Te idea beind aving bot boxes and nodes is tat te same box (e.g., backup router) can be replicated at different points of te network; one can tink of te nodes as being box id s attaced by β N to teir boxes. We use p to range over ports, and write te pairs (n, i) as n.i. Te connection relation γ N specifies ow ports are connected to eac oter and ence encodes te network topology. It is symmetric as packets can flow in eiter direction; it is 1-1 to model pysical connection; it is irreflexive to avoid tigt loops. Not all ports need be connected to anoter port; suc unconnected ports are called external; te oters are called internal. Definitions in tis style of various forms of networks made up out of boxes wit ports can be found in oter contexts, for example saring graps [12], bigraps [25], or cyclic networks [13]; oter definitions can be found in te network literature, for example [32]. 2.2 Network Semantics For te network semantics, we first assume given a set Pac N of packet eaders ranged over by (below we may just say packet ); we call Pac N te eader space (of N ). Packet eaders are typically bit strings ence te 2 80 possibilities alluded to in te introduction but one may use any convenient set. For eac box b : K we ten ave te set of box-located packets, were suc a located packet is a pair, i, of a packet and a box location, tat is, a box port i (wit i K). In oter words, a box-located packet is a specific packet located at a specific port in te network. Next, for eac box b Box, we assume given a transition i Box, i between box-located packets. Note tat packets may be canged (rewritten) in a transition as well as teir location. Wen te signature can be understood from te context, we will generally just i i. Tese transitions are typically given by te data plane of a router, via forwarding tables, ACLs (access control lists), and packet rewriting. We do not specify ere wat tese may be, or ow tey induce transition relations (owever, apart from Section 7, our work is independent of suc considerations). Te box transitions arise from packets moving from an input port of a router, troug its internal ports, and, possibly canged, on to an output port, following its set of rules. In many switcing networks only packet locations cange, as tere are no packet rewriting rules. Tat is, packet eaders are not canged by any of te boxes in te network. Formally, for any box b:k in te network we i Box, i = = for all i, i K and,. Tis olds, for example, for te Singapore network considered in Section 7. Next, packet exit ports j may be independent of teir entry ports i. Formally, we additionally ave for any box b:k in te i 1 Box, i 2 Box, i for all i 1, i 2, i K and,. Tis appens wen te forwarding tables are centrally located and are used to route all incoming messages to exit ports, independently of teir entry port, and wen ACLs are given per exit port; te Singapore network again provides an example. In tis case te te semantics can be equivalently described in terms of rules of te form I were I K, wit one suc rule for eac, namely te one were I = def {j K i i Box, j} Wit te box transition semantics available, we define te network semantics in terms of two transition relations between states: an internal one, moving to internal network nodes, and an external one, moving to external network nodes. Te set, States N, of states of te network N consists of te network-located packets; tese are pairs, p, of a packet and a network location, tat is, a network port p. Te internal network transition n.i n.i olds if, and only if, for some i j and n.j γ n.i. In some sense we are doing two transitions in one step in tat a packet first follows a box transition at input port i to output port i in te same box, and ten moves to te input port j of te router to wic i is connected (as specified by te port connection relation γ). A similar sort-circuiting is done in te Header Space model [15]. Te external network transition n.i n.i olds i i were n.i is an external port. We write N for te total network transition relation N N. Below, wen tey can be understood from te context, we generally omit network suffixes from Node N, β N and so on. However, network suffixes are essential wen we describe transformations between networks. Te double packet/location nature of states gives te subject a special interest. Note tat our states only keep track of te location of one packet, not of several as would be natural if we were interested in concurrent aspects of network operation. Our purpose, as elsewere in te network verification literature, see, e.g., [33], is rater to verify properties suc as reacability, concerning only one packet at a time. A notion of network slice will prove important. In its most general form, a slice is a subset S of States, te set of states. We say tat a slice is a network invariant if, in addition, we p S p p) p S A less general, but useful case, already considered in [15], is were te slice is a product H P of sets H and P of packets and ports. 2.3 A Network Logic We next spell out a small natural modal logic for suc networks tat supports te application of te van Bentem-Hennessy-Milner principle wile being enoug to cover typical network requirements suc as reacability relations between osts.

4 We first assume given a collection of packet formulas α to specify properties of packets, togeter wit a satisfaction relation = α for tem, were Pac N. For example suc packet formulas may consist of wildcard expressions suc as 101xxxxx for packets wit (let s say) eigt bit destination IP addresses tat start wit 101 as in Header Space Analysis [15]. One can ten define a largely standard modal logic by te following grammar of N -formulas: Original Complex Core k l k l R 1 R 2 i j i j V 1 V 2 Core by ub surgery k H l k l R 1 R 2 i j i j V 1 V 2 ϕ ::= p ϕ ϕ ϕ ϕ Fϕ Te modal worlds are te network states. Te formula α expresses tat α olds of te state p expresses tat te state location is p; ϕ expresses tat one can reac a state were ϕ olds in zero or more internal steps from te current state; and Fϕ expresses tat one can reac a state were ϕ olds by an external step from te current state. Oter connectives are definable: te oter propositional ones; te modality, expressing necessarily reacing, aving taken zero or more internal steps; Tϕ abbreviating (ϕ Fϕ) and te corresponding Tϕ; p 1,..., p n (for n > 0) p p n. More expressive logics can be obtained, for example, by adding fixed-points µx.ϕ[x], but te simpler logic seems sufficiently expressive. Given a network N, and a semantics for it, we obtain a semantics for te logic, following te above ideas. As usual, we define a satisfaction p = N ϕ tat in network N te packet satisfies te formula ϕ wen located at port p by structural induction on formulas. Te clauses for te boolean formulas are as usual; te oters p = N α def = p = p def p = p = N ϕ def, p p = p = N Fϕ def, p p = ϕ Tere are ten two natural validity notions for a given network N (reflecting te double nature of states): one were a eader validates a formula if te eader satisfies te formula wen located at any port; and te oter were all eaders satisfy te formula wen located at any port: = N ϕ p = N ϕ and = N ϕ. = N ϕ (Tere is also a tird validity p = N ϕ, but we do not know of a use for it.) Let us give a few examples to illustrate te expressiveness of te logic. First, = N p 1 p 2, p 3 p 4)) olds if, wenever a network packet satisfying α is at port p 1 ten it can reac an external port p 4 via p 2 or p 3, being meanwile transformed into a packet satisfying α. In te same vein, suppose a network slice S is defined by a boolean combination of atomic formulas, ϕ. Ten te slice is a network invariant if, and only if, = N ϕ Tϕ olds. Suppose tat S is anoter slice, defined by a boolean combination of atomic formulas, ϕ. Ten = N ϕ Tϕ Figure 3. Replacing te core by a ub. olds if te second slice cannot be accessed from te first. Tis models te standard notion of slicing (a form of network virtualization tat requires isolation between slices) considered in [15, 17], but our formalism can express alternative notions. Continuing, suppose tat p is an internal port. Ten = N p olds if any packet wit eader satisfying α and reacing p is dropped (or loops troug p forever). Next, = p p) olds if any packet wit eader, loops troug p via some oter port, returning to p wit its eader possibly being transformed. Tis is a generic loop, in te terminology of [15]. However, if one as an invariant α available, one can detect some infinite loops: = N p p p)) olds if any packet wit eader satisfying α loops troug p, via some oter port, wit its eader, owever transformed, still satisfying α. 2.4 A First Surgery Our first example of a surgery sows ow to replace a core by a ub, wile preserving reacability formulas. Example 1: Replacing a Core by a Hub Suppose our network N consists of a core N core troug wic ToRs (or oter boxes) communicate via a single link. Suppose too tat packet eaders are not canged by any of te boxes in te network. Ten if te core is obstruction-free, in a suitable sense, reacability queries between network macines accessing te core via tose boxes old if, and only if, tey old for a greatly reduced network in wic te core is replaced by a ub: see Figure 3. As sown tere, te core network N core as external ports k and l (at nodes a and b, say). Fix a traffic set T Pac N of packet eaders; typically tis will be te packets addressed to te VMs andled by R 2. Ten we assume te core is obstruction-free for T between a.k and b.l, i.e., tat, for all T we a.k N b.l Te network N is te same as N except tat te core as been replaced by a ub H wit ports k and l. Regarding its semantics we assume only tat, for all k l In bot cases we ave boxes (ToR s, say) R 1 and R 2 (at nodes r 1 and r 2, say) to wic (for example, virtual macines) V 1 and V 2 are connected, wit ports as sown. Regarding logic, bot networks

5 ave te same packet formulas, and te evident collections of port formulas. Ten, for any and p, we ave tat te reacability formula ϕ reac, namely: α r v 2.j is satisfied p in N if, and only if, it is satisfied p in N (we are assuming tat we ave a packet formula α T defining T, i.e., suc tat T = { = α T}). Tis is easily verified by direct consideration of te semantics of te formula in eac of te two networks, coupled wit te no-obstruction assumption. For eiter N or p = ϕ reac olds if, and only if, we ave T p = r p v 2.j and we can ten calculate, for r 1.i = v r 1.i v i a.k N r 2.l v i r 2.l v i k r 2.l v r 1.i v r 1.i = v 2.j In te calculation we use te fact tat te network does not alter packet eaders and, in te tird equivalence, te fact tat N core is obstruction-free for T between a.k and b.l. 3. Network Bisimulations To sow te validity of network transformations, our proof tecnique is to define bisimulations between two networks N and N built using te same box signature, but possibly wit different box semantics, for example between te networks on te left and rigt in Figure 2. We take tese to be relations between network-located packets wic are bot bisimulations between N and N and bisimulations between N and N, eac in te usual sense (note tey are ten also bisimulations between N and ). It is N natural to consider, in particular, one-step bisimulations between N and N ; tese are bisimulations between bot N and N and N and N (and so also between N and N ). One can form relations between states from relations Pa and between te respective sets of packets and ports p def Pa p p Ten, given a relation between ports, tere is a largest relation Pa between packets wic, so combined wit, forms a (onestep) network bisimulation between N and N. We remark tat, wile tere certainly are largest suc (one-step) bisimulations, tey do not seem very useful. Te bisimulations relative to a fixed port relation are sensitive to ports visited and so do prove useful; tey are analogous to tose considered in [4]. Given a bisimulation between N and N we wis to transfer properties between te first network and te second, as is usual wit bisimulations, so tat proving a formula in te first network is equivalent to proving a transformed formula in te second network. As we ave two logics ere, speaking of different sets of packets and ports, we need to correlate formulas. To tat end we define a correlation relation For between N -formulas and N -formulas by taking ϕ For ϕ to old if, and only if, for all, p,, p we p = p = p = ϕ) We ten ave te expected proposition expressing te Van Bentem- Hennessy-Milner principle for networks: PROPOSITION 1. Te formula correlation relation For is closed under te logical connectives, tat is te following implications old: For ϕ For ϕ ϕ For ϕ ϕ For ϕ ψ For ψ ϕ ψ For ϕ ψ ϕ For ϕ ϕ For ϕ ϕ For ϕ Fϕ For Fϕ PROOF. Tis is a standard proof. Tere are five cases: 1. As never olds, we ave For. 2. Suppose tat ϕ For ϕ and p. p = N ϕ olds p = N ϕ does not old p = N ϕ does not old (as we ave ϕ For ϕ) p = N ϕ olds. 3. Suppose tat ϕ For ϕ and tat ψ For ψ p. p = N ϕ ψ olds p = N ϕ p = N ψ bot old p = N ϕ p = N ψ bot old (as we ave ϕ For ϕ and ψ For ψ) p = N ϕ ψ olds. 4. Suppose tat ϕ For ϕ and p. Assume p = N ϕ olds, in order to sow p = N ϕ does. Note tat ten, for all, p suc p p = N ϕ olds. Assuming given, p suc p p, we ave to p = N ϕ olds. As is a bisimulation between N and N p, tere are, p suc p p p. By te assumption we ten p = N ϕ, and so, as ϕ For p = N ϕ, olds as required. Te proof p = N ϕ olds p = N ϕ does is similar. 5. Tis is similar to te previous case. In case is built from relations Pa (on packets) and (on ports), as above, tere are natural sufficient conditions for te correlations to old between atomic formulas. Recall tat atomic formulas are eiter of te form α (sets of packets) p 1,..., p n (sets of ports). So, α For α olds if,. Pa = ( = α = α) p 1,..., p m p 1,..., p n olds if p, p. p p = (p {p 1,..., p m} p {p 1,..., p n }) Tese conditions are also necessary, except in evident trivial cases. We remark tat tere is no commitment to te above logic for purposes of network verification. Proposition 1 demonstrates a wide range of properties tat can be transferred between bisimilar networks. However one is free to express suc properties using watever logical or automata-teoretic means one desires. For example one can translate our logic into Datalog, so one can use [20]. 4. Network Surgery We next give tree examples of network transformations by various forms of surgery, weter at te levels of graps, eaders, or rules. Te networks are related to teir transforms by bisimulations; tese, in teir turn, allow application of appropriate versions of te Van Bentem-Hennesy-Milner principle, justified by Proposition 1. We also revisit Example 1, and sow wy te obstruction-freeness assumption made tere does not justify a elpful bisimulation. Our first example of a network transformation may appear too trivial to be useful (removing disconnected sets of nodes). How-

6 Slice irrelevant portions X R 1 Nodes 1 Nodes 2 Y Transforms to via slicing (Example 2) X R 1 Nodes 1 Nodes 2 Y Transforms to via removal (Example 1) X Nodes 1 Figure 4. Wen considering te reacability between stations in te same subtree, te remainder of te network can be sliced away (Example 3), leaving a disconnected set of Nodes tat can be removed by Example 2. ever, as Figure 4 sows, it becomes effective after te following example, a more complex surgery called slicing. Example 2: Removing Disconnected Components Wen a network N consists of two disconnected subnetworks we can verify teir properties separately. Suppose its nodes split into two disjoint subsets Node 1 and Node 2 wic are disconnected in tat for all ports n 1.i 1 and n 2.i 2: n 1 Node 1 n 1 Node 2 = n 1.i 1 γ n 2.i 2 Ten N naturally splits into two subnetworks N 1 and N 2 wit respective node sets Node 1 and Node 2 and wit β Ni j and γ Nj being te evident restrictions of β N and γ N, for j = 1, 2. Note tat a port is internal (external) in an N j iff it is in N. We ten ave tat N is bisimilar to eac of te N j. For j = 1, 2, one can take as (one-step) bisimulation n.i n.i n.i n.i n Node j Te logic for N j as as formulas tose of N tat only mention ports wit nodes in Node j. For suc formulas, Proposition 1 tells us tat tat for all packets and N j-ports n.i we n.i = N n.i = Nj ϕ We now introduce te more powerful slicing transformation illustrated in Figure 4. Wen verifying te reacability between certain pairs of stations suc as X and Y, under certain conditions it is possible to slice away irrelevant portions of te network wic we formalize as slicing. Example 3: Slicing Suppose we ave a slice of te form S = H P of a network N, were H is a subset of te set of eaders and P is a subset of te set of ports. We wis to define a corresponding slice network N S obtained by cutting off all te ports not in P and restricting te eader space to H. We restrict ourselves to te case were P is connection invariant, meaning tat for all p, p rt N, we ave: pγ N p = (p P p P ) Tus we are going to cut off te port at one end of a link if, and only if, we cut off te port at its oter end; in oter words, we are cutting off eiter external ports or wole links. For slicing to work, we will need te slice to be a network invariant, as defined above. In te case of slices of te form H P network invariance can be reprased in terms of natural no-leakage and eader invariance conditions on network boxes. For every box b:k occurring in te network (i.e., in te range of β) say tat a box port pair (i, j) K 2 is P -leaky if, and only if, for some p P and q / P we ave: n. β(n) = b p = n.i n.j γ q Y and say tat a box port pair (i, j) K 2 is P -connected if, and only if, for some p, q P we ave: n. β(n) = b p = n.i (n.j γ q (q = n.j q is external)) Ten te slice is a network invariant if, and only if, te following two conditions old: No leakage For all b:k, i K, and i j = (i, j) is not P -leaky Header invariance For all b:k, H, and P -connected pairs (i, i j = H In te case were te slice is only on eaders it evidently suffices to ceck invariance of H for all pairs of ports of all boxes occurring in te network. To define te slice network N S, we begin by defining te slice signature Box S. For every box b : K and every I K, we assume available a Box S box b I : I; intuitively, b I is obtained from b by slicing off all of b s ports tat are in K\I. We take te eader space to be H and its semantics is te restriction of tat of b to H, tat is tat, for all, H and i, i I we i Box S,b i i Te slice network N S is ten obtained as follows: its node set is tat of N ; its box assignment function is given by: β N S (n) = β N (n) {i K n.i P } were β(n) : K; and its connection relation γ N I is te restriction of γ N to N I s ports, i.e., to P. We note tat te set of ports of N S is P, as anticipated. (For n.i is suc a port iff i K and n.i P, were β(n) : K. But if n.i P ten n.i rt N and so i K; so we see tat n.i is indeed suc a port iff it is in rt N S.) Note too tat, as P is connection invariant, a port p P is an internal (external) port of N S if, and only if, it is of N. In te particular case wen we are only slicing on eaders, tat is wen P is te set of all ports, N S can be taken to be N, altoug one evidently still needs te restricted signature. We now sow tat, restricted to S, te transition relations of N and N S are te same. LEMMA Te internal (external) transition relation of N S is contained in tat of N. 2. Te restriction of te internal (external) transition relation of N to te slice S is contained in tat of N S. PROOF. 1. Suppose, first, n.i N n.i in order to sow n.i n.i. Ten for some j we i Box S, j and also n.j γ N S n.i. We ten i Box, j and n.j γ N n.i and n.i n.i. Next, suppose instead n.i N n.i. Ten we i Box S, i and n.i is an external port of N S. We ten ave i Box, i and, by te above remark on external ports of N S, tat n.i is an external port of N, and n.i n.i. 2. For te second part consider two n.i n.i in S. Ten suppose, first, n.i n.i to sow n.i N n.i. i Box, βn j and n.j γ N n.i, for some j. So, first, as n.i P n.i is in S), i is in te set I = def {i K n.i P } were β N (n) : K. Ten, as n.i P n.i S) and as P is connection invariant and

7 n.j γ N n.i, we see tat n.j P and so tat j I also olds. It follows i Box, βn (n) j. Ten, as, H, we finally i Box S, βn S j. Next as n.j γ N n.i and n.j, n.i P we furter ave tat n.j γ N S n.i. Combining te last two facts, we n.i N n.i, as required. Next, suppose instead, i Box, βn i and n.i is an external port of N. Here, n.i n.i are in S, n.i and n.i are in P ; terefore, arguing as above, we see i Box S, βn S i olds. But n.i is an external port of N S as it is an external port of N. So we find n.i N n.i, concluding te proof. Turning to logic, for te logic of N S, we take te N -formulas tat mention only ports of N S. Tere is a natural potential bisimulation S between N and N S, defined p p p p S THEOREM 1. Te relation S is a one-step bisimulation between N and N S if, and only if, S is a network invariant of N. In tat case, for any N S formula and any p S we p = N p = N S ϕ PROOF. Suppose tat S is an invariant. To sow tat S is a bisimulation between te N and N S, we suppose p p (i.e., p p S) and verify te usual two conditions. First, p p ten, as S is an invariant, we p S. So, applying Lemma 1, part 2, we ave p N p. p p p S) we ave verified te first condition. Second, p N p p S and p p, by Lemma 1, part 1. Te second condition follows. One sows S an external transition bisimulation similarly. Conversely, suppose tat S is a one-step bisimulation between N and N S. To sow S invariant, suppose p S p p. p p and so, as S is a bisimulation, we ave p N p for p p p, tat is, p p S. Tus S is indeed an invariant. Te final part follows from te assumption tat S is a bisimulation by applying Proposition 1 and te remarks made immediately tereafter. B C A D Transform: Remove rule in B, redirect in A Figure 5. Wen considering forwarding eader, redirecting traffic from A to only use C and dropping te relevant rule in B does not impact te reacability of Our next surgery (redirection), formalized in Example 4, keeps te topology uncanged but drops rules. Tis, in turn, involves redirecting te traffic covered by te dropped rules. First, notice in Figure 5 tat A, in Figure 5, forwards packets to eader to bot B and C. Tis is done for good reasons in practice suc as load balancing. B C A D However, given tat certain conditions old, to compute te reacable set of packets addressed to it suffices to redirect traffic at A to only go to C, and drop te rules dealing wit tis traffic from B. Te advantage of tis transformation is tat it can be applied repeatedly to gradually drain te rules out of backup routers, to make a muc simpler reduced network. Example 4: Redirecting Traffic Suppose we ave a network N in wic traffic T Pac N flows from a node a to a node b and ten, witout cange, to a node d, but tere is an alternative flow, in wic te same traffic moves from a to yet anoter node c from wic it also flows witout cange to d. Ten, as long as te box at d treats te traffic in te same way, by wicever of te two routes it arrives, we can transform te network by rerouting te traffic T troug b troug c instead, and removing relevant rules from te box at b. Tis results in anoter network N, wit less rules. Wile tis transformation is clearly far from general, it seems widely applicable to data center switcing networks because of te symmetry present tere. In Section 6 below we point out ow one could pick out candidate nodes a, b, c, etc., using symmetry considerations. We next present te transformation and its background assumptions precisely. Te nodes a, b, c, and d are supposed all distinct. B a A i j i j b k C k l d D c l B a A i j i j b k C k l d D Figure 6. Te formal setup for Example 4 Let te boxes at tese nodes be A : K A, B : K B, C : K C, and D : K D, respectively. Suppose tat a and b are connected by ports a.i and b.i ; tat b and d are connected by ports b.k and d.k ; tat a and c are connected by ports a.j and c.j ; and tat c and d are connected by ports c.l and d.l. Note tat we ten also ave tat i and j are distinct, as are i and k; j and l; and k and l. Tis sceme is depicted in Figure 6. Te background assumptions are tat for any T te following i q = q = k wic says tat T traffic entering at i is forwarded by B uncanged at j q = q = l wic says tat T traffic entering at j is forwarded by C uncanged at l, k l q wic says tat D treats T traffic at k and l identically. Te network N is obtained from N by replacing te boxes A : K A and B : K B by boxes A : K A and B : K B. Te transition rules of A are te same as tose of A except for transitions to te ports i or j, were we put q i q i / q j q j q i T) c l

8 Te transition rules of B are te same as tose for B except for transitions between ports i and k were we i k i k / T Turning to te bisimulation relation, we p to old if, and only if, = and one of te following two conditions also old: - p = p and, if T, ten p b.i - T and one of te following two conditions old: p = b.i p = c.j p = d.k p = d.l Note tat tis bisimulation relation is not te composition of separate relations between packets and ports. THEOREM 2. Te relation is a one-step bisimulation between N and N. PROOF. We first sow a bisimulation of te relation. We consider all te cases wen a p olds, and verify te bisimulation condition in eac. We always ave =. Suppose first tat p = p. Ten eiter / T or else p b.i. Tere are two cases: In te first case, we assume first tat p is not a port of te node a. In tis case, a -successor state p of eiter network cannot be of te b.i and so is -related to itself. Furter, bot networks ave te same -successor states, as te relevant transitions are te same in bot networks. Tis is evident in te case of te c and te d nodes were te corresponding boxes are te same in bot networks. In te case of te node b, te boxes B and B ave te same transitions, except for tose from i ; owever in tis case we ten ave / T and so B and B ave te same transitions i. As te -successor states are te same in bot networks and are -related to temselves, te bisimilarity condition olds in tis case. In te second case we assume tat p as te form a.q, and consider te possible q of te A and A successors q. Tese are te same if / T or else q i and q j. Te corresponding -successor states in te two networks are ten equal and in te relation to temselves. Oterwise, T and q = i or q = j, and we note: Every A i can be paired off wit te A j, c.j. Every A j can be paired off wit te A j, c.j ; and tis accounts for all te remaining A successors of te j wit T. Tere are no A successors q of te i. Tis verifies te bisimilarity condition in tis case. Suppose next tat p p. Ten T, and tere are two cases: In te first case p = b.i and p = c.j. So, employing te first two background assumptions, te only possible - successor state of N d.k, and only possible next suc state of N d.l. d.l, te bisimulation condition olds in tis case. In te second case, p = d.k and p = d.l. By te tird background assumption q reacable k and tose reacable l by a D transition are te same, and so too, terefore are te states of te two networks reacable p p by a -transition. Furter, for suc a q, d.q will eiter be an external port or else be connected to a port oter tan b.i (as tat port is connected to anoter port). So p is reacable by a transition p (equivalently p) ten we p. Te bisimulation condition terefore olds in tis case too. We next sow tat is a bisimulation of te relation. Te two networks ave te same relation. p wenever tere is a transition p. Te only case were relates two different states, one of wic may make a transition, d.l, wit T, and in tat case one can make a transition to a p iff te oter can, as D treats T traffic at k and l identically. Putting tese togeter, we see tat is a bisimulation of. Turning to logic we take te logic of N to be tat of N, and consider formula correlations. For packet eader formulas we evidently always ave α For α For port formulas we assume available a boolean combination ϕ T of packet eader formulas defining T. We ten ave te following p p if p is not b.i, c.j, d.k, or d.l, if p is c.j, d.k, or d.l, and ϕ p For ϕ p ϕ T c.j ) For ϕ c.j ϕ T d.l ) For ϕ T d.l ) So, in particular, if we are away from te surgery te same assertions old. More precisely, if p is not b.i, c.j, d.k, or d.l, and ϕ does not mention any of tese ports, ten, for any, we p = N p = N ϕ Tere are variations on tis transformation. Regarding te definition of A, one can instead leave its transitions to i uncanged, wen te proof tat is a one-step bisimulation relation goes troug uncanged. It may also appen tat te transitions to j are uncanged, as te ones being added are already tere. Tis is common in te core were ports suc as i and j are treated symmetrically for te usual redundancy and performance reasons. Next, let us assume, as is common in switcing networks, tat packets are forwarded witout being transformed. We can ten weaken te condition tat D treats T traffic at k and l identically. Given a subset G of ports (to be guarded ), we define a forwarding relation TG on ports. It is te least relation suc tat p TG q if, and only if, one of te following old: p = q, or p, q / G and bot of te following old T, p p q q p TG q T, q q p p p TG q Ten TG is an equivalence relation and if p G and p TG q ten p = q. Te weaker condition on D is ten tat d.k TG d.l.

9 One can define a bisimulation exactly as before, except tat one replaces te second clause in te case were T (i.e., te one tat p = d.k and p = d.l ) by p TG p. As before, te same assertions old if we are away from te surgery, i.e., te ports b.i, c.j, or any ports in te TG-relation to a different port, and so, in particular, any port in G. As before, one may also leave te transitions to i of A uncanged. Returning to Example 1, let us see wy tere need not be a relevant bisimulation between te two networks, by wic we mean a bisimulation allowing te two atomic r 1.i v 2.j to be transferred. Te problem is tat altoug a packet T can go from r 1.i to v 2.j via te core, it migt also enter te core and ten go somewere else and ten be dropped. Now consider te following reacability formula (generally stronger tan ϕ reac ) α r v 2.j Tis olds in N, but may fail to old in N. However, if tat is te case, ten, by Proposition 1, tere can be no bisimulation between N and N tat allows te two atomic formulas to be transferred. 5. Composite Bisimulations We now see ow composite bisimulations can be obtained. Tese use topological relations between networks and teir signatures to compose togeter bisimulation relations between te transition relations of boxes to form bisimulations between te networks. Tis contrasts wit te bisimulations in Section 4 wic are rater constructed in an ad oc way, according to te need at and. Suppose we ave two signatures Box and Box. Ten a signature relation between te two signatures consists of: a box relation B between Box and Box, and for eac so-related pair of boxes b:k, b:k, a box port relation b,b K K between teir ports. Suppose next tat we ave two networks N and N built over te two signatures. Ten a topological relation between N and N consists of a signature relation between te two signatures, as above, togeter wit a node relation N between teir sets of nodes suc tat: if n N n ten β N (n) B β N (n), and te network port relation between rt N and rt N defined by: n.i n.i def n N n i β N (n),β N (n) is a bisimulation of γ by γ. (Note tat ten if n.i n.i ten n.i is external iff n.i is.) Turning to te box transition relations, assume first tat we ave two eader spaces Pac N and Pac N. Ten a packet eader relation is a relation Pa between te two eader spaces, Pac N and Pac N. Assume next tat we ave a semantics for eac of te signatures, tat is, we ave families of transition relations N,b (b Box) and N,b (b Box) for te two signatures, as considered above. Ten, given a signature relation between te two signatures as above, a packet eader relation Pa is a signature bisimulation between te two signatures if, and only if te box-located packet relation b,b Blp, defined i i def Pa i b,b is a bisimulation between b and b, wenever b B b. Given a topological relation (comprising a signature relation and a node relation) and a packet relation as above, we ave a relation Pa between te two sets of packets and a relation i i between te two sets of ports. So we can define a relation Net between te corresponding located packet sets as remarked in Section 3, i.e., by p p def Pa p p We ten ave: PROPOSITION 2. If Pa is a signature bisimulation, ten Net is a one-step bisimulation relation between N and N. PROOF. Assume Pa is a signature bisimulation. We give te first alves of te proofs tat Net is an internal transition relation bisimulation and tat it is an external one; te second alves are similar. n.i n.i. Ten Pa, n N n, and i β N (n),β N (n) i, and i β N (n),β N (n) i. If, n.i n.i, i βn j and n.j γ n.i, for some j. As Blp is a bisimulation of βn (n) by β N (n), tere are, j suc i β N j j j (and so Pa and j β N (n),β N (n) j). Next, as n N n and j β N (n),β N (n) j, n.j n.j. So as n.j γ n.i and is a bisimulation of γ by γ, tere are n and j suc tat n.j γ n.j and n.i n.j. Ten, i β N j and n.j γ n.j, we ave n.i n.i. Finally, as Pa and n.i n.i we also n.i n.i, as required. If, n.i n.i, i βn i and n.i is external. So, as Blp is a bisimulation of βn (n) by β N (n), tere are, i suc i β N i β N (n),β N (n) i i (and so Pa and i β N (n),β N (n) i ). Next, as n N n and i β N (n),β N (n) i, n.i n.i. But ten as n.i is external, we see tat n.i is external too. So, i β N i, we n.i n.i Finally, as Pa and n.i n.i we also ave n.i n.i, as required. β N (n),β N (n) 6. Network Symmetries and Merging Headers We use composite symmetries in two ways. Te first is to define various notions of symmetries, particularly te topological ones tat typically old in data centers, and ten sow ow local symmetries can be found and also ow to divide out networks by symmetry groups. Te second is to construct equivalencies between packet eaders tat allow us to replace assertions about one eader by an assertion about an equivalent one, or to go furter and replace eaders by teir equivalence classes. Example 5: Network Symmetry At te most general level, a symmetry of a network N is a permutation π N of network states wic preserves and reflects bot N and N. Tat is, for all network p p, we p p π N p) N π N p ) and similarly for N. More strictly, a one step symmetry of N is required to preserve and reflect N and and N. One could also require tat te permutation is composed from separate permutations of te eaders and te network ports (note tat if te eader space is finite, ten te reflection condition is redundant). Te network structure provides a good source of exploitable symmetries, and so we focus on composite symmetries wic are simply tose composite bisimulations between a network and itself

10 were all te relations are bijections. (We note in passing te evident fact tat all tese various classes of symmetries form groups.) Let us spell tis out in functional terms. Assume given a signature Box. Ten a signature symmetry over Box consists of: a box permutation π Bo : Box = Box, and for all boxes b : K, a box port bijection π b : K = K, were π Bo(b) : K, Next, given a network N over Box, a topological symmetry of N is a signature symmetry, togeter wit a node permutation π No :Node = Node suc tat: For all n, we ave π Bo(β(n)) = β(π No(n)), and For all ports n.i and n +.i +, we ave: n.i γ n +.i + π (n.i) γ π (n +.i + ) were te port bijection π : rt = rt is defined by: π (n.i) = π No(n).π β(n) (i) (n.i rt) We can associate a grap Γ N to every network N. It as nodes tose of te network and as relation R N were: R N (n, n ) def n.i, n.i rt. n.i γ n.i Ten every topological symmetry π No is a symmetry of Γ N. Turning to te semantics, suppose we ave a eader space Pac N, and families of transition relations N,b (b Box), as usual. Ten, given a signature symmetry as above, a packet bijection π Pa : Pac N = PacN is a symmetry of te signature semantics, if, for all,, b : K, and all i, i K, we i i π Pa π b (i) π Bo (b) π Pa ( π b (i ) Figure 2 provides an example of a topological symmetry were te boxes R 3 and R 4, and teir nodes would be permuted, but all oters would be fixed, and were te ports are permuted as sown. If, furter, te two boxes contained te same rules (modulo box port bijections) ten te identity packet bijection would provide a signature semantics symmetry. Given a topological symmetry π No (comprising a signature symmetry and a node permutation) and a packet bijection π Pa as above, we can define a bijection of te located packet set by: π n.i) = def π π (n.i) and we ave te following corollary of Proposition 2: COROLLARY 1. If π Pa is a symmetry of te signature semantics, ten π Net is a one-step symmetry of N. Turning to te logic, assume tat for eac packet formula α tere is a formula α π suc tat: = α π π 1 Pa () = α We ten obtain an evident omomorpic definition of ϕ π for N - formulas ϕ, were: (@ p) π π (p) ( ϕ) π = ϕ π (ϕ ψ) π = ϕ π ψ π ( ϕ) π = ϕ π (Fϕ) π = Fϕ π Proposition 1 ten tells us, tat, for any N -formula ϕ, we p = ϕ π π (p) = ϕ π We can use symmetry considerations to pick out candidates for traffic-redirection surgery. We would like to find two symmetrically placed nodes b and c wic are candidates for redirecting traffic (from b to c), so wis to find a box symmetry π Bo and an associated topological symmetry π No wic switces a and b. Let us say tat te symmetry is local if π No leaves all te oter nodes invariant. For suc a local symmetry a and b will ave te same Γ N neigbors. If tere is only one link between any two nodes, as is common, te signature symmetry is also ten determined. So te suggestion is to look for two nodes wit te same neigbors, and use te node connections to determine te port correlations between tem (two ports are correlated if tey link to te same neigbor). One can ten coose pairs of suc ports (oter tan any ports connecting a and b) to searc for candidates for i and j, or for k and l, for traffic redirection. Figure 2 again provides an example. Te two required nodes are tose corresponding to te two boxes R 3 and R 4. We next sketc ow to quotient a network N by a group of topological symmetries. In tis way we may be able to slim fat trees, as discussed above. First, let us briefly recall te relevant background material (and see, e.g. [3, 17]). An action of a group G on a set X is a map :G X X, suc tat te following two equations old: (g g) x = g (g x) e x = x Te orbit equivalence relation is ten defined on X by: x G y def g G. g x = y and we write X/G for te set of equivalence classes [x] of elements x of X under tis equivalence relation. Under componentwise composition, te topological symmetries (π Bo, π, π No) form a group Top N, wit tree actions: on Box; on Boxrt = def {b.i b:k, i K}; and on Node, were: (π Bo, π, π No) b = def π Bo(b) (π Bo, π, π No) (b.i) = def π Bo(b).π(i) b (π Bo, π π No) n = def π No(n) Let G be a subgroup of Top N. Let Box/G, Boxrt/G, and Node/G, be te collections of orbit equivalence classes for eac of its tree actions, inerited from Top N. We define a signature on Box/G by setting [b]:{[b.i] i K}, for eac b:k (ignoring tat te [b.i] are not natural numbers). Ten we can define a quotient network N /G over te signature. It as node set Node/G, box assignment function β N /G, were β N /G ([n]) = def [β N (n)] and connection relation γ N /G were m.q γ N /G m.q def n, n, b, b, i, i. n.i γ N n.i β N (n) = b β N (n ) = b m = [n] m = [n ] q = [b.i] q = [b.i ] Assume next tat te identity packet permutation is a symmetry of te signature semantics, given any signature component of any element of G. Ten we can define a transition relation for Box/G, keeping te packet set te same as tat of N, q q def b, i, i i c = [b] q = [b.i] q = [b.i ] So we ave defined te syntax and semantics of te quotient network N /G, as desired. Tere is a one-step bisimulation between N and N /G, formed by combining te identity relation on packets and te relation on ports, were: n.i [n].[n.i] def n [n] n.i [n.i] Finally, assuming te evident logic for N /G, we p 1,..., p m For [p]