Synchronous Elastic Systems

Size: px

Start display at page:

Download "Synchronous Elastic Systems"

Allyson Baldwin
6 years ago
Views:

1 Synchronous Elastic Systems Mike Kishinevsky and Jordi Cortadella Intel Strategic CAD Labs Hillsboro, USA Universitat Politecnica de Catalunya Barcelona, Spain

2 2 Contributors to SELF research Performance analysis: Jorge JúlvezJ Theory of elastic machines: Sava Krstic and John O Leary Micro-architectural pipelining: Timothy Kam, Marc Galceran Oms Optimization: Dmitry Bufistov, Josep Carmona Bill Grundmann

3 3 Agenda I. Basics of elastic systems II. III. IV. Early evaluation and performance analysis Applications of elastic systems Demo of SELF compiler

4 4 Synchronous Stream of Clock cycle Token (of data)

5 5 Synchronous Elastic Stream Clock cycle Clock cycle Bubble (no data) Token

6 6 Synchronous Circuit Latency =

7 7 Synchronous Elastic Circuit Latency = e Latency can vary

8 8 Ordinary Synchronous System A C A C = B D B D Changing latencies changes behavior

9 Synchronous Elastic (characteristic property) e A e C e A e e C = e B e D e B e D Changing latencies does NOT change behavior = time elasticity 9

10 10 Elasticity? Elasticity refers to elasticity of time, i.e. tolerance to changes in timing parameters, not properties of materials Luca Carloni et al. in the first systematic study of such systems called them Latency Insensitive Systems Other used names: Latency tolerant systems Synchronous emulation of asynchronous systems Synchronous handshake circuits We use term synchronous elastic to link to asynchronous elastic systems that have been developed before e.g., David Muller s s pipelines of late 1950s Ivan Sutherland s s micro-pipelines 1989 Tolerate the variability of input data arrival and computation delaysd Asynchronous elastic tolerate changes in continuous time Synchronous elastic - in discrete time

11 11 Why Scalable Modular (Plug & Play) Better energy-delay trade-offs (design for typical case instead of worst case) New micro-architectural opportunities in digital design Not asynchronous: use existing design experience, CAD tools and flows... but have some advantages of asynchronous

12 12 How to Design Synchronous Elastic Systems Example of the implementation: SELF = Synchronous Elastic Flow Others are possible

13 13 Pipelined communication sender receiver What if the sender does not always send valid data?

14 14 The bit sender receiver What if the receiver is not always ready?

15 15 The bit sender receiver

16 16 The bit sender receiver

17 17 The bit sender receiver

18 18 The bit sender receiver Back-pressure

19 19 The bit sender receiver Long combinational path

20 20 Cyclic structures Combinational cycle One can build circuits with combinational cycles (constructive cycles by Berry), but synthesis and timing tools do not like them

21 21 Example: pipelined linear communication chain with transparent latches sender H L H L receiver ½cycle ½cycle Master and slave latches with independent control

22 22 Shorthand notation (clock lines not shown) D Q En En clk

23 sender SELF (linear communication) receiver En En En En V V V V S S S S 23

24 SELF sender receiver En En En En 1 V V V V S S S S 0 24

25 SELF sender receiver En En En En 1 V V V V S S S S 0 25

26 SELF sender receiver En En En En 1 V V V V S S S S 0 26

27 SELF sender receiver En En En En 1 V V V V S S S S 0 27

28 SELF sender receiver En En En En 1 V V V V S S S S 0 28

29 SELF sender receiver En En En En 0 V V V V S S S S 0 29

30 SELF sender receiver En En En En 0 V V V V S S S S 0 30

31 SELF sender receiver En En En En 0 V V V V S S S S 0 31

32 SELF sender receiver En En En En 0 V V V V S S S S 0 32

33 SELF sender receiver En En En En 0 V V V V S S S S 0 33

34 SELF sender receiver En En En En 1 V V V V S S S S 1 34

35 SELF sender receiver En En En En 1 V V V V S S S S 1 35

36 SELF sender receiver En En En En 1 V V V V S S S S 1 36

37 SELF sender receiver En En En En 1 V V V V S S S S 1 37

38 SELF sender receiver En En En En 1 V V V V S S S S 1 38

39 SELF sender receiver En En En En 1 V V V V S S S S 1 39

40 SELF sender receiver En En En En 1 V V V V S S S S 1 40

41 SELF sender receiver En En En En 1 V V V V S S S S 1 41

42 SELF sender receiver En En En En 1 V V V V S S S S 1 42

43 SELF sender receiver En En En En 1 V V V V S S S S 0 43

44 SELF sender receiver En En En En 1 V V V V S S S S 0 44

45 SELF sender receiver En En En En 1 V V V V S S S S 0 45

46 SELF sender receiver En En En En 1 V V V V S S S S 0 46

47 SELF sender receiver En En En En 1 V V V V S S S S 0 47

48 SELF sender receiver En En En En 1 V V V V S S S S 0 48

49 SELF sender receiver En En En En 1 V V V V S S S S 0 49

50 SELF sender receiver En En En En 1 V V V V S S S S 0 50

51 51 Elastic channel and its protocol not Idle * Retry Sender Receiver * not Transfer

52 52 Elastic channel protocol Sender Receiver * D D * C C C B * A Transfer Retry Idle

53 53 Basic VS block En i En i V i-1 1 V i V i-1 1 VS V i S i-1 1 Si S i-1 1 Si VS block + data-path latch = elastic HALF-buffer (EHB) EHB + EHB = elastic buffer with capacity 2

54 Control specification of the EB 54

55 Two implementations 55

56 Elastic buffer keeps data while stop is in flight W2R1 W1R2 W2R2 W1R1 EBs = FIFOs with two parameters: Forward latency Capacity Backward latency for stop propagation assumed (but need not be) equal to fwd latency Typical case: (1,2) - 1 cycle forward latency with capacity of 2 Replaces normal registers Decoupling buffers W1R1 Cannot be done with Single Edge Flops without double pumping Can use latches inside Master-Slave as shown before 56

57 57 Join + VS V 1 S 1 V S VS V 2 VS S 2

58 58 (Lazy) Fork V V 1 S 1 S V 2 S 2

59 Eager Fork S 1 V S ^ ^ V 1 V 2 S 2 59

60 60 Eager fork (another implementation) VS VS VS VS VS

61 61 Variable Latency Units [0 [0--k] k] cycles cycles go done clear V/S V/S

62 Coarse grain control 62

63 63 Elasticization Synchronous Elastic

64 CLK 64

65 65 PC FORK J O I N IF/ID ID/EX EX/MEM MEM/WB F O R J K O I N CLK

66 FORK V S J O I N V S J O I N V S V S F O R K V S CLK 66

67 1 0 FORK J O I N J O I N F O R K 1 0 CLK 67

68 Elastic control layer 1 1 Generation of gated clocks CLK 68

69 Equivalence Synchronous: stream of data D: a b c d e d f g h i j SELF: elastic stream of data D: a * b * * c d e * d f * g h * * i j V: S: Transfer sub-stream = original stream Called: transfer equivalence, flow equivalence, or latency equivalence 69

70 70 Marked Graph models of elastic systems

71 71 Modelling elastic control with Petri nets bubble data-token token data-token token bubble

72 Modelling elastic control with Petri nets bubble 2 data-tokens tokens data-token token Hiding internal transitions of elastic buffers 72

73 Modelling elastic control with Marked Graphs 73

74 Modelling elastic control with Marked Graphs Forward ( or Request) Backward ( or Acknowledgement) 74

75 75 Elastic control with Timed Marked Graphs. Continuous time = asynchronous d=250ps d=151ps Delays in time units

76 76 Elastic control with Timed Marked Graphs. Discrete time = synchronous elastic d=1 d=1 Latencies in clock cycles 1 1

77 77 Elastic control with Timed Marked Graphs. Discrete time. Multi-cycle operation d=2 d=1 2 1

78 78 Elastic control with Timed Marked Graphs. Discrete time. Variable latency operation d {1,2} d=1 e.g. discrete probabilistic distribution: average latency 0.8* *2 = 1.2 {1,2} 1

79 79 Modeling forks and joins d=1 1

80 80 Modelling combinational elastic blocks d=1 d=0 1 0

81 81 Elastic Marked Graphs An Elastic Marked Graph (EMG) is a Timed MG such that for any arc a there exists a complementary arc a satisfying the following condition a = a and a = a Initial number of tokens on a and a (M0(a)+M0(a )) = capacity of the corresponding elastic buffer Similar forms of pipelined Petri Nets and Marked Graphs have been previously used for modeling pipelining in HW and SW (e.g. Patil 1974; Tsirlin, Rosenblum 1982)

82 Reminder: Performance analysis of Marked graphs Th = operations / cycle = number of firings per time unit The throughput is given by the minimum mean-weight cycle Th=min(Th(A), Th(B), Th(C))=2/5 Th(A)=3/7 A B C Th(B)=3/5 Th(C)=2/5 Efficient algorithms: (Karp 1978), (Dasdan,Gupta 1998) 82

83 83 Early evaluation Naïve solution: introduce choice places issue tokens at choice node only into one (some) relevant path problem: tokens can arrive to merge nodes out-of of-order order later token can overpass the earlier one Solution: change enabling rule early evaluation issue negative tokens to input places without tokens, i.e. keep the same firing rule Add symmetric sub-channels with negative tokens Negative tokens kill positive tokens when meet Two related problems: Early evaluation and Exceptions (how to kill a data-token) token)

84 84 Examples of early evaluation MULTIPLEXOR a b T F c if s = T then c := a -- don t wait for b else c := b -- don t wait for a s MULTIPLIER a b * c if a = 0 then c := 0 -- don t wait for b

85 85 Petri nets Related work Extensions to model OR causality Kishinevsky et al. Change Diagrams [e.g. book of 1994] Yakovlev et al. Causal Nets 1996 Asynchronous systems Reese et al 2002: Early evaluation Brej 2003: Early evaluation with anti-tokens tokens Ampalan & Singh 2006: preemption using anti-tokens tokens

86 86 Dual Marked Graph Marking: Arcs (places) > Z (allow negative markings) Some nodes are labeled as early-enabling Enabling rules for a node: Positive enabling: M(a) > 0 for every input arc Early enabling (for early enabling nodes): M(a) > 0 for some input arcs Negative enabling: M(a) < 0 for every output arc Firing rule: the same as in regular MG

87 87 Dual Marked Graphs Early enabling can be associated with an external guard that depends on data variables (e.g., a select signal of a multiplexor) Actual enabling guards are abstracted away (unless needed) Anti-token generation: When an early enabled node fires, it generates anti-tokens in the predecessor arcs that had no tokens Anti-token propagation counterflow: When negative enabled node fires, it propagates the anti-tokens from the successor to the predecessor arcs

88 88 Dual Marked Graph model Enabled! -1

89 89 Passive anti-token token Passive DMG = version of DMG without negative enabling Negative tokens can only be generated due to early enabling, but cannot propagate Let D be a strongly connected DMG such that all cycles have positive cumulative marking Let D p be a corresponding passive DMG. If environment (consumers) never generate negative tokens, then throughput (D)( ) = throughput (D( p ) If capacity of input places for early enabling transitions is unlimited, then active anti-tokens tokens do not improve performance Active anti-tokens tokens reduce activity in the data-path (good for power reduction)

90 90 Properties of DMGs Firing invariant: Let node n be simultaneously positive (early) and negative enabled in marking M. Let M 1 be the result of firing n from M due to positive (early) enabling. Let M 2 be the result of firing n from M due to negative enabling. Then, M 1 = M 2 Token preservation. Let c be a cycle of a strongly connected DMG with initial marking M 0. For every reachable marking M : M(c) = M 0 (c) Liveness. A strongly connected passive DMG is live iff for every cycle c: M(c) > 0. For DMGs this is a sufficient condition of liveness It is also a necessary condition for positive liveness Repetitive behavior. In a SC DMG: a firing sequence s from M leads to the same marking iff every node fires in s the same number of times DMGs have properties similar to regular MGs

91 Implementing early enabling 91

92 92 How to implement anti-tokens tokens? Positive tokens Negative tokens

93 93 How to implement anti-tokens tokens? Positive tokens Negative tokens

94 How to implement anti-tokens tokens?

95 Controller for elastic buffer L H En En V L V H V V H L S S S S 95

96 96 Dual controller for elastic buffer En En V + V + S + S + V - S - V - S -

97 Dual Join and Fork 97

98 Join with early evaluation 98

99 99 Condition on Early Evaluation Function Early evaluation function makes decision based on presence of valid bits, not on their absence Formally: EE is positive unate with respect to data input Example: legal EE function for a data-path MUX (s select input)

100 100 Passive anti-token token (capacity one) Bigger capacity can be achieved by injecting anti-token up-down counters on elastic channels

101 101 Properties of elastic channels Invariants: mutually exclusive Kill (V - ) and (S + ) (V + ) and retain of a kill (S - )

102 102 DLX processor model with slow bypass Bypass α Fetch Decode Execute Memory β Write-back 1 α 1 β Throughput: Th = operations / cycle Late evaluation Th=0.5 Applying early evaluation on Execution and Write-back Th = 0.7 (α=0.3; β=0.3)

103 103 Conclusions Early evaluation can increase performance beyond the min cycle ratio The duality between positive and negative tokens suggests a clean and effective implementation Dual Marked Graphs is a formal model for analytical analysis and optimization methods

104 104 Performance analysis with early evaluation (joint work with Jorge JúlvezJ lvez)

105 105 Revisit Performance Analysis of Marked Graphs The throughput can also be computed by means of linear programming t1 p1 p2 t2 Average marking 1 mp = lim m p( τ)dτ t t t 0 t3 Throughput th = min( m p 1, mp2) th = min m [Campos, Chiola, Silva 1991] p p

106 Revisit Performance Analysis of Marked Graphs max th b p 1 p 2 a m p1 = 1 + t b t a m p2 = 0 + t a t b m p3 = 1 + t d t a m p4 = 0 + t a t c m p5 = 1 + t c t d reachability th constraints p 3 p 4 d p 5 c Th = 0.5 th m p2 // transition b th m p4 // transition c th m p5 // transition d th min(m p1, m p3 ) // transition a 106

107 107 GMG = Multi-guarded Dual Marked Graph Refinement of passive DMGs Every node has a set of guards Every guard is a set of input places (arcs) Example: t1 t3 t2 p1 p3 p2 t4 G(t4)={{p1,p3},{p2,p3}}

108 108 Early evaluation 1-α α β 1-β

109 Early Early evaluation evaluation α 1 α β 1 β α β (0.43) (0.43) (0.60) (0.60) (0.40) (0.40)

110 LP formulation for an upper bound of a throughput (by example) p 1 p 2 a α b p 3 1-α p 4 max th m p1 = 1 + t b t a m p2 = 0 + t a t b m p3 = 1 + t d t a m p4 = 0 + t a t c m p5 = 1 + t c t d th m p2 th m p4 th m p5 d p 5 c th = α m p1 + (1-α) m p3 Th = (2 - α) ) / (3 - α) 110

111 111 Averaging cycle throughput or cycle times does not work 1/2 p 1 p 2 a α b Averaging throughput of individual cycles Th = α 1/2 + (1- α) ) 2/3 = (4 - α) ) / 6 p 3 1-α p 4 2/3 d p 5 Th = (2 - α) ) / (3 - α) c Averaging effective cycle times of individual cycles 1/Th = 2α 2 + (1- α) ) 3/2 = (3 + α) ) / 2 Th = 2/(3+ α)

112 What can we do with synchronous elastic systems? 112

113 113 Variable latency units ALU ALU L = 3 L = 1 L = 2 L = 1 start done

114 Benchmark Patricia from Media Bench 1 st operand # adds # adds 2 nd operand Significant bits Statistics of operand sizes 12 bits of an adder do 95% of additions bits of adder used 114

115 Power-delay for an adder Compare 64 bits VLA and prefix adder relative delay 115

116 116 Variable-latency cache hits 12-cycle miss L2-cache 2-way associative 32KB 2-cycle hit L1-cache 1-cycle hit suggested by Joel Emer for ASIM experiment

117 117 Variable-latency cache hits 12-cycle miss L2-cache Pseudo-associative 32KB {1-2} cycle hit L1-cache 1-cycle hit Sequential access: if hit in first access L = 1, if not L=2 Trade-off: faster, or larger, or less power cache

118 118 Variable-latency cache hits 12-cycle miss L2-cache Pseudo-associative 64KB {2-3} cycle hit L1-cache 1-cycle hit Sequential access: if hit in first access L = 1, if not L=2 Trade-off: faster, or larger, or less power cache

119 Correct-by-construction pipelining Transforms: bypass retiming elasticize early enabling insert buffers and negative tokens size elastic buffer capacity ID RF E1 E2 Correct-by-construction ID 0 0 RF 1 1 E1 E2-1 SPEC IMP [Joint work with Timothy Kam and Marc Galceran] 119

120 Tree topology NoC AGENT R AGENT AGENT R R AGENT AGENT R R AGENT AGENT [In collaboration with Ken Stevens, Charles Dike, Bill Grundmann] 120

121 121 Router node interface B A Router C

122 122 NoC Router M EHB S S B A EHB M S C Relative order of tokens between agents is preserved M EHB

123 Switch and Merge 123

124 124 Correctness (short story) Developed theory of elastic machines (for late evaluation) Verify correctness of any elastic implementation = check conformance with the definition of elastic machine All SELF controllers are verified for conformance Elasticization is correct-by by-construction Theory for early evaluation and negative delays is more challenging Sketch of a theory, but no fully satisfactory compositional properties found yet Verification done on concrete systems and controllers

125 125 Summary SELF gives a low cost implementation of elastic machines Functionality is correct when latencies change New micro-architectural opportunities Compositional theory proving correctness Early evaluation - mechanism for performance and power optimization Optimization methods (that we did not discuss): Retiming and recycling, buffer optimization and pipelining Applications to design of NoC link layer

126 See reference list for some relevant publications 126

127 SELF compiler Flow graph Parameterized library of controllers Control generation Netlist of distributed controllers Verilog SMV blif Simulator Performance Backend synthesis NuSMV SIS & ABC Verification Logic synthesis 127

128 Example 128

129 Example Evaluation Throughput No early evaluation Passive anti-tokens tokens M2 W Passive anti-tokens tokens F3 W Active anti-tokens tokens

Equivalence Checking For Synchronous Elastic Circuits

Equivalence Checking For Synchronous Elastic Circuits Vidura Wijayasekara and Sudarshan K. Srinivasan Department of Electrical and Computer Engineering North Dakota State University Fargo ND 58104 Email: