CS-206 Concurrency Lecture 8 Concurrent a b c d Data structures Spring 2015 Prof. Babak Falsafi parsa.epfl.ch/courses/cs206/ remove(b) remove(c) Adapted from slides originally developed by Maurice Herlihy and Nir Shavit from the Art of Multiprocessor Programming, and Babak Falsafi EPFL Copyright 2015 EPFL CS-206 Spring 2015 Lec.8-1
Where are We? Lecture & Lab M T W T F 16-Feb 17-Feb 18-Feb 19-Feb 20-Feb 23-Feb 24-Feb 25-Feb 26-Feb 27-Feb 2-Mar 3-Mar 4-Mar 5-Mar 6-Mar 9-Mar 10-Mar 11-Mar 12-Mar 13-Mar 16-Mar 17-Mar 18-Mar 19-Mar 20-Mar 23-Mar 24-Mar 25-Mar 26-Mar 27-Mar 30-Mar 31-Mar 1-Apr 2-Apr 3-Apr 6-Apr 7-Apr 8-Apr 9-Apr 10-Apr 13-Apr 14-Apr 15-Apr 16-Apr 17-Apr 20-Apr 21-Apr 22-Apr 23-Apr 24-Apr 27-Apr 28-Apr 29-Apr 30-Apr 1-May 4-May 5-May 6-May 7-May 8-May 11-May 12-May 13-May 14-May 15-May 18-May 19-May 20-May 21-May 22-May 25-May 26-May 27-May 28-May 29-May u Concurrent data structures w Coarse-grained locking w Fine-grained locking w Lock-free data structures u Examples w Linked lists u Next week w Higher-level data structures EPFL CS-206 Spring 2015 Lec.8-2
Contention u When many threads compete for a lock u Prevents efficient multithreaded execution w Threads spend more time waiting for lock than doing work u Real problem in multiprocessor programming EPFL CS-206 Spring 2015 Lec.8-3
Today: Concurrent Objects u Adding threads should not lower throughput w Contention effects u Should increase throughput w Not possible if inherently sequential w Surprising things are parallelizable EPFL CS-206 Spring 2015 Lec.8-4
Coarse-Grained Synchronization u Each method locks the object w Avoid contention using queue locks w Easy to reason about w In simple cases u Example w Solaris (Oracle s OS) first version had a single lock w Every time there was an OS access, one thread could get in w What is wrong with this picture? EPFL CS-206 Spring 2015 Lec.8-5
Coarse-Grained Synchronization u Sequential bottleneck w Threads stand in line u Adding more threads w Does not improve throughput w Struggle to keep it from getting worse u So why even use a multiprocessor? w Well, some apps are inherently parallel EPFL CS-206 Spring 2015 Lec.8-6
Fine-Grained Synchronization u Instead of using a single lock u Split object into w Independently-synchronized components u Methods conflict when they access w The same component w At the same time EPFL CS-206 Spring 2015 Lec.8-7
Example with Linked List u Illustrate these patterns u Using a list-based Set w Common application w Building block for other apps EPFL CS-206 Spring 2015 Lec.8-8
List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } EPFL CS-206 Spring 2015 Lec.8-9
List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Add item to set EPFL CS-206 Spring 2015 Lec.8-10
List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Remove item from set EPFL CS-206 Spring 2015 Lec.8-11
List-Based Sets public interface Set<T> { public boolean add(t x); public boolean remove(t x); public boolean contains(t x); } Is item in set? EPFL CS-206 Spring 2015 Lec.8-12
List Node public class Node { public T item; public int key; public Node next; } EPFL CS-206 Spring 2015 Lec.8-13
List Node public class Node { public T item; public int key; public Node next; } item of interest EPFL CS-206 Spring 2015 Lec.8-14
List Node public class Node { public T item; public int key; public Node next; } Usually hash code EPFL CS-206 Spring 2015 Lec.8-15
List Node public class Node { public T item; public int key; public Node next; } Reference to next node EPFL CS-206 Spring 2015 Lec.8-16
Reasoning about Concurrent Objects u Invariant w Property that always holds u Why do we care about invariants? w Invariant is true when object is created w Invariant truth is preserved by each method w Each step of each method EPFL CS-206 Spring 2015 Lec.8-17
Specifically u Invariants preserved by w add() w remove() w contains() u Example invariants to preserve for linked lists w Tail reachable from head w Sorted w No duplicates EPFL CS-206 Spring 2015 Lec.8-18
Sequential List Based Set add() a c d remove() a b c EPFL CS-206 Spring 2015 Lec.8-19
Sequential List Based Set add() a c d remove() b a b c EPFL CS-206 Spring 2015 Lec.8-20
Coarse-Grained Locking a b d EPFL CS-206 Spring 2015 Lec.8-21
Coarse-Grained Locking a b d c EPFL CS-206 Spring 2015 Lec.8-22
Coarse-Grained Locking a b d honk! honk! c Simple but hotspot + bottleneck EPFL CS-206 Spring 2015 Lec.8-23
Coarse-Grained Locking u Easy, same as synchronized methods w One lock to rule them all u Simple, clearly correct w Deserves respect! u Works poorly with contention w Queue locks help w But bottleneck still an issue EPFL CS-206 Spring 2015 Lec.8-24
Fine-grained Locking u Requires careful thought w Do not meddle in the affairs of wizards, for they are subtle and quick to anger u Split object into pieces w Each piece has own lock w Methods that work on disjoint pieces need not exclude each other EPFL CS-206 Spring 2015 Lec.8-25
Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-26
Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-27
Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-28
Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-29
Hand-over-Hand locking a b c EPFL CS-206 Spring 2015 Lec.8-30
Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-31
Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-32
Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-33
Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-34
Removing a Node a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-35
Removing a Node a c d remove(b) Why hold 2 locks? EPFL CS-206 Spring 2015 Lec.8-36
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-37
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-38
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-39
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-40
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-41
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-42
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-43
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-44
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-45
Concurrent Removes a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-46
Uh, Oh a c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-47
Uh, Oh Bad news, c not removed a c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-48
Problem a b c u To delete node c w Swing node b s next field to d a b c u Problem is, w Someone deleting b concurrently could direct a pointer to c EPFL CS-206 Spring 2015 Lec.8-49
Insight u If a node is locked w No one can delete node s successor u If a thread locks w Node to be deleted w And its predecessor w Then it works EPFL CS-206 Spring 2015 Lec.8-50
Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-51
Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-52
Hand-Over-Hand Again a b c d remove(b) EPFL CS-206 Spring 2015 Lec.8-53
Hand-Over-Hand Again a b c d remove(b) Found it! EPFL CS-206 Spring 2015 Lec.8-54
Hand-Over-Hand Again a b c d remove(b) Found it! EPFL CS-206 Spring 2015 Lec.8-55
Hand-Over-Hand Again a c d remove(b) EPFL CS-206 Spring 2015 Lec.8-56
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-57
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-58
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-59
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-60
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-61
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-62
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-63
Removing a Node a b c d remove(b) remove(c) EPFL CS-206 Spring 2015 Lec.8-64
Removing a Node a b c d Must acquire lock for b remove(c) EPFL CS-206 Spring 2015 Lec.8-65
Removing a Node a b c d Cannot acquire lock for b remove(c) EPFL CS-206 Spring 2015 Lec.8-66
Removing a Node a b c d Wait! remove(c) EPFL CS-206 Spring 2015 Lec.8-67
Removing a Node a b d Proceed to remove(b) EPFL CS-206 Spring 2015 Lec.8-68
Removing a Node a b d remove(b) EPFL CS-206 Spring 2015 Lec.8-69
Removing a Node a b d remove(b) EPFL CS-206 Spring 2015 Lec.8-70
Removing a Node a d remove(b) EPFL CS-206 Spring 2015 Lec.8-71
Removing a Node a d EPFL CS-206 Spring 2015 Lec.8-72
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } EPFL CS-206 Spring 2015 Lec.8-73
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Key used to order node EPFL CS-206 Spring 2015 Lec.8-74
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Predecessor and current nodes EPFL CS-206 Spring 2015 Lec.8-75
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Node search EPFL CS-206 Spring 2015 Lec.8-76
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Make sure locks released EPFL CS-206 Spring 2015 Lec.8-77
Remove method public boolean remove(item item) { int key = item.hashcode(); Node pred, curr; boolean foundnode = false; try { } finally { curr.unlock(); pred.unlock(); } return foundnode; } Everything else EPFL CS-206 Spring 2015 Lec.8-78
Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } EPFL CS-206 Spring 2015 Lec.8-79
Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } lock pred == head EPFL CS-206 Spring 2015 Lec.8-80
Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Lock current EPFL CS-206 Spring 2015 Lec.8-81
Remove method try { pred = this.head; pred.lock(); curr = pred.next; curr.lock(); } finally { } Traversing list EPFL CS-206 Spring 2015 Lec.8-82
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-83
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Search key range EPFL CS-206 Spring 2015 Lec.8-84
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } At start of each loop: curr and pred locked EPFL CS-206 Spring 2015 Lec.8-85
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } foundnode = false; If item found, remove node EPFL CS-206 Spring 2015 Lec.8-86
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } foundnode = false; If node found, remove it EPFL CS-206 Spring 2015 Lec.8-87
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Unlock predecessor EPFL CS-206 Spring 2015 Lec.8-88
Remove: searching Only one node locked! while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-89
Remove: searching while (curr.key <= key) { if (item == curr.item) { demote current pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-90
Remove: searching while (curr.key <= key) { if (item == curr.item) { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } Find and lock new current EPFL CS-206 Spring 2015 Lec.8-91
Remove: searching while (curr.key <= key) { if Lock (item invariant == curr.item) restored { pred.next = curr.next; foundnode = true; break; } pred.unlock(); pred = curr; curr = curr.next; curr.lock(); } EPFL CS-206 Spring 2015 Lec.8-92
Why does this work? u To remove node e w Must lock e w Must lock e s predecessor u Therefore, if you lock a node w It can t be removed w And neither can its successor EPFL CS-206 Spring 2015 Lec.8-93
Adding Nodes u To add node e w Must lock predecessor w Must lock successor u Add/remove must acquire locks in the same order w What happens in the order is compromised? w E.g., remove code lock predecessor first, add successor first EPFL CS-206 Spring 2015 Lec.8-94
Properties to prove for add/remove u Does safetyness hold? u Does the liveness property hold? u Are the invariants maintained? EPFL CS-206 Spring 2015 Lec.8-95
Drawbacks u Better than coarse-grained lock w Threads can traverse in parallel u Still not ideal w Long chain of acquire/release w Inefficient EPFL CS-206 Spring 2015 Lec.8-96
Traffic Jam u Any concurrent data structure based on mutual exclusion has a weakness u If one thread w Enters critical section w And eats the big muffin w Cache miss, page fault, descheduled w Everyone else using that lock is stuck! w Need to trust the scheduler. EPFL CS-206 Spring 2015 Lec.8-97
Other patterns: Optimistic Synchronization u Search without locking u If you find it, lock an check w Ok à we are done w Opps à start over u Evaluation w Usually cheaper than locking w Mistakes are expensive EPFL CS-206 Spring 2015 Lec.8-98
Optimistic: Traverse without Locking a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-99
Optimistic: Lock and Load a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-100
Optimistic: Lock and Load c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-101
What could go wrong? a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-102
What could go wrong? a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-103
What could go wrong? a b d e remove(b) EPFL CS-206 Spring 2015 Lec.8-104
What could go wrong? a b d e remove(b) EPFL CS-206 Spring 2015 Lec.8-105
What could go wrong? a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-106
What could go wrong? c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-107
What could go wrong? a d e add(c) Uh-oh EPFL CS-206 Spring 2015 Lec.8-108
Check after lock u By traversing optimistically, we give up any guarantees we had about the list during traversal u The node we locked might have just been removed u Need to check the pointers to it EPFL CS-206 Spring 2015 Lec.8-109
What Else Could Go Wrong? a b d e add(c) Aha! EPFL CS-206 Spring 2015 Lec.8-110
What Else Could Go Wrong? a b d e add(c) add(b ) EPFL CS-206 Spring 2015 Lec.8-111
What Else Could Go Wrong? a b d e add(c) b add(b ) EPFL CS-206 Spring 2015 Lec.8-112
What Else Could Go Wrong? a b d e b add(c) EPFL CS-206 Spring 2015 Lec.8-113
What Else Could Go Wrong? c a b d e add(c) EPFL CS-206 Spring 2015 Lec.8-114
Check after lock u By traversing optimistically, we give up any guarantees we had about the list during traversal u New nodes might just have been added u Need to check the pointers between the locked nodes EPFL CS-206 Spring 2015 Lec.8-115
Optimistic Traversal u Traverse optimistically, get the lock and then check if we can move on u Need to check after we get the lock w We know nothing about the items locked w Need to check if they are in the same situation as they were before we locked w After checking, we know that we hold the lock, and thus they cannot suffer further changes EPFL CS-206 Spring 2015 Lec.8-116
Other patterns: Lazy Synchronization u Postpone hard work u E.g., break remove into two parts w Logical removal à marks component to be deleted w Physical removal à do what needs to be done EPFL CS-206 Spring 2015 Lec.8-117
Reminder: Lock-Free Data Structures u No matter what w Guarantees minimal progress in any execution w i.e. Some thread will always complete a method call w Even if others halt at malicious times w Implies that implementation can t use locks EPFL CS-206 Spring 2015 Lec.8-118
Recall: Using atomics u compareandset(expectedvalue,newvalue) method w Compares a variable with an expectedvalue given as input w Sets it to newvalue if comparison is successful w Does everything atomically EPFL CS-206 Spring 2015 Lec.8-119
Lock-free Lists u Next logical step w Wait-free contains() w lock-free add() and remove() u How about turning adds/removes into atomics? w Use compareandset() or CAS What could go wrong with only CAS? EPFL CS-206 Spring 2015 Lec.8-120
Lock-free Lists a b c 0 e Use CAS to verify pointer is correct Removal Not enough! EPFL CS-206 Spring 2015 Lec.8-121
Problem a b c e Removal d Node added EPFL CS-206 Spring 2015 Lec.8-122
The Solution: Combine Bit and Pointer Logical Removal = Set Mark Bit a 0 b 0 c 01 e 0 Physical Removal CAS Mark-Bit and Pointer are CASed together (AtomicMarkableReference) d 0 Fail CAS: Node not added after logical Removal EPFL CS-206 Spring 2015 Lec.8-123
Solution u Use AtomicMarkableReference u Atomically w Swing reference and w Update flag u Remove in two steps w Set mark bit in next field w Redirect predecessor s pointer EPFL CS-206 Spring 2015 Lec.8-124
Marking a Node u AtomicMarkableReference class w Java.util.concurrent.atomic package Reference address F mark bit EPFL CS-206 Spring 2015 Lec.8-125
Extracting Reference & Mark Public Object get(boolean[] marked); EPFL CS-206 Spring 2015 Lec.8-126
Extracting Reference & Mark Public Object get(boolean[] marked); Returns reference Returns mark at array index 0! (funny use of Java arrays to pass a pointer) EPFL CS-206 Spring 2015 Lec.8-127
Extracting Mark Only public boolean ismarked(); Value of mark EPFL CS-206 Spring 2015 Lec.8-128
Changing State Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); EPFL CS-206 Spring 2015 Lec.8-129
Changing State If this is the current reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); And this is the current mark EPFL CS-206 Spring 2015 Lec.8-130
Changing State then change to this new reference Public boolean compareandset( Object expectedref, Object updateref, boolean expectedmark, boolean updatemark); and this new mark EPFL CS-206 Spring 2015 Lec.8-131
Changing State public boolean attemptmark( Object expectedref, boolean updatemark); EPFL CS-206 Spring 2015 Lec.8-132
Changing State public boolean attemptmark( Object expectedref, boolean updatemark); If this is the current reference EPFL CS-206 Spring 2015 Lec.8-133
Changing State public boolean attemptmark( Object expectedref, boolean updatemark);.. then change to this new mark. EPFL CS-206 Spring 2015 Lec.8-134
Removing a Node a b CAS c d remove(c) EPFL CS-206 Spring 2015 Lec.8-135
Removing a Node failed a CAS b CAS c d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-136
Removing a Node a b c d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-137
Removing a Node a d remove(c) remove(b) EPFL CS-206 Spring 2015 Lec.8-138
Summary u Coarse-grained locking u Fine-grained locking w Basic synchronization w Optimistic synchronization w Lazy synchronization u Lock-free synchronization EPFL CS-206 Spring 2015 Lec.8-139