CS-206 Concurrency. Lecture 8 Concurrent. Data structures. a b c d. Spring 2015 Prof. Babak Falsafi parsa.epfl.ch/courses/cs206/ remove(c) remove(b)

Transcription

1 CS-206 Concurrency Lecture 8 Concurrent a b c d Data structures Spring 2015 Prof. Babak Falsafi parsa.epfl.ch/courses/cs206/ remove(b) remove(c) Adapted from slides originally developed by Maurice Herlihy and Nir Shavit from the Art of Multiprocessor Programming, and Babak Falsafi EPFL Copyright 2015 EPFL CS-206 Spring 2015 Lec.8-1

2 Where are We? Lecture & Lab M T W T F 16-Feb 17-Feb 18-Feb 19-Feb 20-Feb 23-Feb 24-Feb 25-Feb 26-Feb 27-Feb 2-Mar 3-Mar 4-Mar 5-Mar 6-Mar 9-Mar 10-Mar 11-Mar 12-Mar 13-Mar 16-Mar 17-Mar 18-Mar 19-Mar 20-Mar 23-Mar 24-Mar 25-Mar 26-Mar 27-Mar 30-Mar 31-Mar 1-Apr 2-Apr 3-Apr 6-Apr 7-Apr 8-Apr 9-Apr 10-Apr 13-Apr 14-Apr 15-Apr 16-Apr 17-Apr 20-Apr 21-Apr 22-Apr 23-Apr 24-Apr 27-Apr 28-Apr 29-Apr 30-Apr 1-May 4-May 5-May 6-May 7-May 8-May 11-May 12-May 13-May 14-May 15-May 18-May 19-May 20-May 21-May 22-May 25-May 26-May 27-May 28-May 29-May u Concurrent data structures w Coarse-grained locking w Fine-grained locking w Lock-free data structures u Examples w Linked lists u Next week w Higher-level data structures EPFL CS-206 Spring 2015 Lec.8-2

3 Contention u When many threads compete for a lock u Prevents efficient multithreaded execution w Threads spend more time waiting for lock than doing work u Real problem in multiprocessor programming EPFL CS-206 Spring 2015 Lec.8-3

4 Today: Concurrent Objects u Adding threads should not lower throughput w Contention effects u Should increase throughput w Not possible if inherently sequential w Surprising things are parallelizable EPFL CS-206 Spring 2015 Lec.8-4

5 Coarse-Grained Synchronization u Each method locks the object w Avoid contention using queue locks w Easy to reason about w In simple cases u Example w Solaris (Oracle s OS) first version had a single lock w Every time there was an OS access, one thread could get in w What is wrong with this picture? EPFL CS-206 Spring 2015 Lec.8-5

6 Coarse-Grained Synchronization u Sequential bottleneck w Threads stand in line u Adding more threads w Does not improve throughput w Struggle to keep it from getting worse u So why even use a multiprocessor? w Well, some apps are inherently parallel EPFL CS-206 Spring 2015 Lec.8-6

7 Fine-Grained Synchronization u Instead of using a single lock u Split object into w Independently-synchronized components u Methods conflict when they access w The same component w At the same time EPFL CS-206 Spring 2015 Lec.8-7

8 Example with Linked List u Illustrate these patterns u Using a list-based Set w Common application w Building block for other apps EPFL CS-206 Spring 2015 Lec.8-8

9 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } EPFL CS-206 Spring 2015 Lec.8-9

10 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Add item to set EPFL CS-206 Spring 2015 Lec.8-10

11 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Remove item from set EPFL CS-206 Spring 2015 Lec.8-11

12 List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Is item in set? EPFL CS-206 Spring 2015 Lec.8-12

13 List Node public class Node { public T item; public int key; public Node next; } EPFL CS-206 Spring 2015 Lec.8-13

14 List Node public class Node { public T item; public int key; public Node next; } item of interest EPFL CS-206 Spring 2015 Lec.8-14

15 List Node public class Node { public T item; public int key; public Node next; } Usually hash code EPFL CS-206 Spring 2015 Lec.8-15

16 List Node public class Node { public T item; public int key; public Node next; } Reference to next node EPFL CS-206 Spring 2015 Lec.8-16

17 Reasoning about Concurrent Objects u Invariant w Property that always holds u Why do we care about invariants? w Invariant is true when object is created w Invariant truth is preserved by each method w Each step of each method EPFL CS-206 Spring 2015 Lec.8-17

18 Specifically u Invariants preserved by w add() w remove() w contains() u Example invariants to preserve for linked lists w Tail reachable from head w Sorted w No duplicates EPFL CS-206 Spring 2015 Lec.8-18

19 Sequential List Based Set add() a c d remove() a b c EPFL CS-206 Spring 2015 Lec.8-19

20 Sequential List Based Set add() a c d remove() b a b c EPFL CS-206 Spring 2015 Lec.8-20

21 Coarse-Grained Locking a b d EPFL CS-206 Spring 2015 Lec.8-21

22 Coarse-Grained Locking a b d c EPFL CS-206 Spring 2015 Lec.8-22

23 Coarse-Grained Locking a b d honk! honk! c Simple but hotspot + bottleneck EPFL CS-206 Spring 2015 Lec.8-23

24 Coarse-Grained Locking u Easy, same as synchronized methods w One lock to rule them all u Simple, clearly correct w Deserves respect! u Works poorly with contention w Queue locks help w But bottleneck still an issue EPFL CS-206 Spring 2015 Lec.8-24

25 Fine-grained Locking u Requires careful thought w Do not meddle in the affairs of wizards, for they are subtle and quick to anger u Split object into pieces w Each piece has own lock w Methods that work on disjoint pieces need not exclude each other EPFL CS-206 Spring 2015 Lec.8-25

26 Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-26

27 Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-27

28 Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-28

29 Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-29

30 Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-30

31 Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-31

32 Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-32

33 Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-33

34 Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-34

35 Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-35

36 Removing a Node a c d remove(b) Why hold 2 locks? EPFL CS-206 Spring 2015 Lec.8-36

37 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-37

38 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-38

39 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-39

40 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-40

41 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-41

42 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-42

43 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-43

44 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-44

45 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-45

46 Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-46

47 Uh, Oh a c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-47

48 Uh, Oh Bad news, c not removed a c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-48

49 Problem a b c u To delete node c w Swing node b s next field to d a b c u Problem is, w Someone deleting b concurrently could direct a pointer to c EPFL CS-206 Spring 2015 Lec.8-49

50 Insight u If a node is locked w No one can delete node s successor u If a thread locks w Node to be deleted w And its predecessor w Then it works EPFL CS-206 Spring 2015 Lec.8-50

51 Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-51

52 Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-52

53 Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-53

54 Hand-Over-Hand Again a b c d remove(b) Found it! EPFL CS-206 Spring 2015 Lec.8-54

55 Hand-Over-Hand Again a b c d remove(b) Found it! EPFL CS-206 Spring 2015 Lec.8-55

56 Hand-Over-Hand Again a c d remove(b) EPFL CS-206 Spring 2015 Lec.8-56

57 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-57

58 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-58

59 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-59

60 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-60

61 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-61

62 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-62

63 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-63

64 Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-64

65 Removing a Node a b c d Must acquire lock for b remove(c) EPFL CS-206 Spring 2015 Lec.8-65

66 Removing a Node a b c d Cannot acquire lock for b remove(c) EPFL CS-206 Spring 2015 Lec.8-66

67 Removing a Node a b c d Wait! remove(c) EPFL CS-206 Spring 2015 Lec.8-67

68 Removing a Node a b d Proceed to remove(b) EPFL CS-206 Spring 2015 Lec.8-68

69 Removing a Node a b d remove(b) EPFL CS-206 Spring 2015 Lec.8-69

70 Removing a Node a b d remove(b) EPFL CS-206 Spring 2015 Lec.8-70

71 Removing a Node a d remove(b) EPFL CS-206 Spring 2015 Lec.8-71

72 Removing a Node a d EPFL CS-206 Spring 2015 Lec.8-72

73 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } EPFL CS-206 Spring 2015 Lec.8-73

74 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Key used to order node EPFL CS-206 Spring 2015 Lec.8-74

75 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Predecessor and current nodes EPFL CS-206 Spring 2015 Lec.8-75

76 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Node search EPFL CS-206 Spring 2015 Lec.8-76

77 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Make sure locks released EPFL CS-206 Spring 2015 Lec.8-77

78 Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Everything else EPFL CS-206 Spring 2015 Lec.8-78

79 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } EPFL CS-206 Spring 2015 Lec.8-79

80 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } lock pred == head EPFL CS-206 Spring 2015 Lec.8-80

81 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Lock current EPFL CS-206 Spring 2015 Lec.8-81

82 Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Traversing list EPFL CS-206 Spring 2015 Lec.8-82

83 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-83

84 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Search key range EPFL CS-206 Spring 2015 Lec.8-84

85 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } At start of each loop: curr and pred locked EPFL CS-206 Spring 2015 Lec.8-85

86 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } foundnode = false; If item found, remove node EPFL CS-206 Spring 2015 Lec.8-86

87 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } foundnode = false; If node found, remove it EPFL CS-206 Spring 2015 Lec.8-87

88 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Unlock predecessor EPFL CS-206 Spring 2015 Lec.8-88

89 Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-89

90 Remove: searching while (curr.key <= key) { if (item == curr.item) { demote current pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-90

91 Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Find and lock new current EPFL CS-206 Spring 2015 Lec.8-91

92 Remove: searching while (curr.key <= key) { if Lock (item invariant == curr.item) restored { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-92

93 Why does this work? u To remove node e w Must lock e w Must lock e s predecessor u Therefore, if you lock a node w It can t be removed w And neither can its successor EPFL CS-206 Spring 2015 Lec.8-93

94 Adding Nodes u To add node e w Must lock predecessor w Must lock successor u Add/remove must acquire locks in the same order w What happens in the order is compromised? w E.g., remove code lock predecessor first, add successor first EPFL CS-206 Spring 2015 Lec.8-94

95 Properties to prove for add/remove u Does safetyness hold? u Does the liveness property hold? u Are the invariants maintained? EPFL CS-206 Spring 2015 Lec.8-95

96 Drawbacks u Better than coarse-grained lock w Threads can traverse in parallel u Still not ideal w Long chain of acquire/release w Inefficient EPFL CS-206 Spring 2015 Lec.8-96

97 Traffic Jam u Any concurrent data structure based on mutual exclusion has a weakness u If one thread w Enters critical section w And eats the big muffin w Cache miss, page fault, descheduled w Everyone else using that lock is stuck! w Need to trust the scheduler. EPFL CS-206 Spring 2015 Lec.8-97

98 Other patterns: Optimistic Synchronization u Search without locking u If you find it, lock an check w Ok à we are done w Opps à start over u Evaluation w Usually cheaper than locking w Mistakes are expensive EPFL CS-206 Spring 2015 Lec.8-98

99 Optimistic: Traverse without Locking a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-99

100 Optimistic: Lock and Load a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-100

101 Optimistic: Lock and Load c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-101

102 What could go wrong? a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-102

103 What could go wrong? a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-103

104 What could go wrong? a b d e remove(b) EPFL CS-206 Spring 2015 Lec.8-104

105 What could go wrong? a b d e remove(b) EPFL CS-206 Spring 2015 Lec.8-105

106 What could go wrong? a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-106

107 What could go wrong? c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-107

108 What could go wrong? a d e add(c) Uh-oh EPFL CS-206 Spring 2015 Lec.8-108

109 Check after lock u By traversing optimistically, we give up any guarantees we had about the list during traversal u The node we locked might have just been removed u Need to check the pointers to it EPFL CS-206 Spring 2015 Lec.8-109

110 What Else Could Go Wrong? a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-110

111 What Else Could Go Wrong? a b d e add(c) add(b ) EPFL CS-206 Spring 2015 Lec.8-111

112 What Else Could Go Wrong? a b d e add(c) b add(b ) EPFL CS-206 Spring 2015 Lec.8-112

113 What Else Could Go Wrong? a b d e b add(c) EPFL CS-206 Spring 2015 Lec.8-113

114 What Else Could Go Wrong? c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-114

115 Check after lock u By traversing optimistically, we give up any guarantees we had about the list during traversal u New nodes might just have been added u Need to check the pointers between the locked nodes EPFL CS-206 Spring 2015 Lec.8-115

116 Optimistic Traversal u Traverse optimistically, get the lock and then check if we can move on u Need to check after we get the lock w We know nothing about the items locked w Need to check if they are in the same situation as they were before we locked w After checking, we know that we hold the lock, and thus they cannot suffer further changes EPFL CS-206 Spring 2015 Lec.8-116

117 Other patterns: Lazy Synchronization u Postpone hard work u E.g., break remove into two parts w Logical removal à marks component to be deleted w Physical removal à do what needs to be done EPFL CS-206 Spring 2015 Lec.8-117

118 Reminder: Lock-Free Data Structures u No matter what w Guarantees minimal progress in any execution w i.e. Some thread will always complete a method call w Even if others halt at malicious times w Implies that implementation can t use locks EPFL CS-206 Spring 2015 Lec.8-118

119 Recall: Using atomics u compareandset(expectedvalue,newvalue) method w Compares a variable with an expectedvalue given as input w Sets it to newvalue if comparison is successful w Does everything atomically EPFL CS-206 Spring 2015 Lec.8-119

120 Lock-free Lists u Next logical step w Wait-free contains() w lock-free add() and remove() u How about turning adds/removes into atomics? w Use compareandset() or CAS What could go wrong with only CAS? EPFL CS-206 Spring 2015 Lec.8-120

121 Lock-free Lists a b c 0 e Use CAS to verify pointer is correct Removal Not enough! EPFL CS-206 Spring 2015 Lec.8-121

122 Problem a b c e Removal d Node added EPFL CS-206 Spring 2015 Lec.8-122

123 The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 c 01 e 0 Physical Removal CAS Mark-Bit and Pointer are CASed together (AtomicMarkableReference) d 0 Fail CAS: Node not added after logical Removal EPFL CS-206 Spring 2015 Lec.8-123

124 Solution u Use AtomicMarkableReference u Atomically w Swing reference and w Update flag u Remove in two steps w Set mark bit in next field w Redirect predecessor s pointer EPFL CS-206 Spring 2015 Lec.8-124

125 Marking a Node u AtomicMarkableReference class w Java.util.concurrent.atomic package Reference address F mark bit EPFL CS-206 Spring 2015 Lec.8-125

126 Extracting Reference & Mark Public Object get(boolean[] marked); EPFL CS-206 Spring 2015 Lec.8-126

127 Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0! (funny use of Java arrays to pass a pointer) EPFL CS-206 Spring 2015 Lec.8-127

128 Extracting Mark Only public boolean ismarked(); Value of mark EPFL CS-206 Spring 2015 Lec.8-128

129 Changing State Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); EPFL CS-206 Spring 2015 Lec.8-129

130 Changing State If this is the current reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); And this is the current mark EPFL CS-206 Spring 2015 Lec.8-130

131 Changing State then change to this new reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); and this new mark EPFL CS-206 Spring 2015 Lec.8-131

132 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); EPFL CS-206 Spring 2015 Lec.8-132

133 Changing State public boolean attemptmark( Object expectedref, boolean updatemark); If this is the current reference EPFL CS-206 Spring 2015 Lec.8-133

134 Changing State public boolean attemptmark( Object expectedref, boolean updatemark);.. then change to this new mark. EPFL CS-206 Spring 2015 Lec.8-134

135 Removing a Node a b CAS c d remove(c) EPFL CS-206 Spring 2015 Lec.8-135

136 Removing a Node failed a CAS b CAS c d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-136

137 Removing a Node a b c d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-137

138 Removing a Node a d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-138

139 Summary u Coarse-grained locking u Fine-grained locking w Basic synchronization w Optimistic synchronization w Lazy synchronization u Lock-free synchronization EPFL CS-206 Spring 2015 Lec.8-139