Announcements Maps performance tips On server: Maintain DB connections, prepared statements (per thread/request!) Use Spark.before, Spark.after to open and close. Use ThreadLocal<T>, or pass the connection around. On client: Minimize redraws (not on every mouse event!). http://benalman.com/projects/jquery-throttle-debounce-plugin/ Java profiler is jvisualvm, consider YourKit. There is a JS profiler in the Chrome dev tools. John Jannotti (cs32) Design with Threads April 5, 2018 1 / 31
Design with Threads John Jannotti /course/cs0320/www/docs/lectures/ April 5, 2018 John Jannotti (cs32) Design with Threads April 5, 2018 2 / 31
A mental model for synchronized blocks 1 s y n c h r o n i z e d ( bank ) { 2 double b a l a n c e = bank. get ( John ) ; 3 b a l a n c e += 2 3 1. 2 3 ; 4 bank. put ( John, b a l a n c e ) ; 5 } becomes (CAPITALS == built in magic ) 1 LOCK l o c k = bank.getlock ( ) ; 2 t r y { 3 l o c k. OBTAIN( ) 4 double b a l a n c e = bank. get ( John ) ; 5 b a l a n c e += 2 3 1. 2 3 ; 6 bank. put ( John, b a l a n c e ) ; 7 } f i n a l l y { 8 l o c k. RELEASE ( ) ; 9 } John Jannotti (cs32) Design with Threads April 5, 2018 3 / 31
Java s synchronized methods synchronized keyword in front of method. 1 s y n c h r o n i z e d v o i d d e p o s i t ( S t r i n g name, double amount ) 2 double b a l a n c e = t a b l e. get ( name ) ; 3 b a l a n c e += amount ; 4 t a b l e. put ( name, b a l a n c e ) ; 5 } Equivalent to synchronized (this) for body. 1 v o i d d e p o s i t ( S t r i n g name, double amount ) { 2 s y n c h r o n i z e d ( t h i s ) { 3 double b a l a n c e = t a b l e. get ( name ) ; 4 b a l a n c e += amount ; 5 t a b l e. put ( name, b a l a n c e ) ; 6 } 7 } What about a static synchronized method? John Jannotti (cs32) Design with Threads April 5, 2018 4 / 31
Examples List add()/remove() in synchronized regions Can you safely do unsynchronized lookups in ArrayList? Iteration? What is the difference between Vector & ArrayList? Prefer Collections.synchronizedList(alist); Maps, Sets, and other collections Unsynchronized by default. Map sm = Collections.synchronizedMap(m); Consider java.util.concurrent.concurrent(hash)map Avoids corruption, but { get();...; put(); } still isn t atomic! John Jannotti (cs32) Design with Threads April 5, 2018 5 / 31
Dining Philosophers Five diners (threads) at round table. Need 2 forks to eat. Only 5 on table. Each picks up left (acquires lock). Then the right (acquires lock). Then eats (does work). Then puts both down (releases locks). What can go wrong? How can you ensure it won t? John Jannotti (cs32) Design with Threads April 5, 2018 6 / 31
Deadlocks Thread 1 Thread 2 Lock A...... Lock B Lock B... wait Lock A wait wait Doesn t just affect locks Wait for I/O from another thread/process. Trying to acquire a resource from a pool. Think about it when waiting on anything. John Jannotti (cs32) Design with Threads April 5, 2018 7 / 31
Avoiding Deadlocks (and Lock Contention) Limit the number of locks in your programs By minimizing writable sharing, not by crossing your fingers! Try to avoid the need to hold multiple locks simultaneously. If you must hold multiple locks, order them. Hold locks briefly (as measured by time and code) Keep synchronized regions short. Minimize calls inside locked regions. Avoid I/O inside locked regions. Briefly is about helping you understand your code and when locks are held, not about lower the probability of bad luck. John Jannotti (cs32) Design with Threads April 5, 2018 8 / 31
Using Threads Getting the most out of threads is tricky. Figuring out where to use threads is hard. Some things are obvious. But most potential uses aren t. Bottom line Use threads where it makes design easier, or you know you have a performance problem. Minimize (mutable) data sharing. Consider message passing designs John Jannotti (cs32) Design with Threads April 5, 2018 9 / 31
Read/Write Locks java.util.concurrent.locks.readwritelock readlock(), writelock() Multiple simultaneous readers allowed. Only one writer (with no reader) allowed. Useful for read-heavy workloads (a common situation) John Jannotti (cs32) Design with Threads April 5, 2018 10 / 31
Thread Coordination Suppose you build your KD-Tree with multiple threads. At each split, if there s enough work, fork a thread. When can you do a lookup? (That is, when are the threads done?) Can be coordinated with synchronized regions. There are more direct constructs, and you should use them. John Jannotti (cs32) Design with Threads April 5, 2018 11 / 31
Fork/Join Basic operations for run-to-completion threads. Fork: create and start a new thread Join: wait for a thread to exit Structured in some languages Fork/join are visible in the lexical structure of the program. A forked section might look like a special block. Java is unstructured Fork: Thread t = new Thread(work); t.start() Join: t.join() or t.join(timeout) How would you join multiple threads? PS. Java 7 introduced a Fork/Join Framework that you might consider. John Jannotti (cs32) Design with Threads April 5, 2018 12 / 31
Monitors Used when conditions control access, not just mutual exclusion. We ll describe what the Java language provides. But also be aware of what the standard libraries offer. java.util.concurrent com.google.common.util.concurrent Java provides basic control primitives Waiting (for condition to hold) Notifying (of change in condition) Java does not provide the parenthetical semantics on these operations. John Jannotti (cs32) Design with Threads April 5, 2018 13 / 31
Wait-Notify A thread wants to yield CPU until an event occurs Coordination without termination (unlike join()) Wait for another thread to provide needed data. Wait for user to push a button. Java provides this in a limited way. Event must be explicitly noticed by another thread. That other thread needs to tell the waiting thread about it. Java Implementation Wait by calling o.wait() or o.wait(timeout) Another thread must call o.notifyall() (on the same object). Or o.notify() (in any doubt? use notifyall) stackoverflow.com/questions/37026/java-notify-vs-notifyall-all-overagain John Jannotti (cs32) Design with Threads April 5, 2018 14 / 31
Wait-Notify to implement a Monitor Details are slightly tricky Java allows wait() on an object, not a condition obj.wait() must occur while synchronized on obj. Notify must occur on the same object. And also while synchronized by that object. Simplified code pattern 1 s y n c h r o n i z e d ( o b j ) { 2 w h i l e (! c o n d i t i o n ) { 3 o b j. w a i t ( ) ; 4 } 5 } John Jannotti (cs32) Design with Threads April 5, 2018 15 / 31
Producer-Consumer Consumer Gets next item from queue. Processes that item. 1 w h i l e ( t r u e ) { 2 s y n c h r o n i z e d ( queue ) { 3 w h i l e ( queue. isempty ( ) ) { 4 t r y { 5 queue. w a i t ( ) ; 6 } 7 c atch ( I n t e r r u p t e d E x c e p t i o n e ) { } 8 } 9 next = queue. remove ( ) ; 0 } 1 next. p r o c e s s ( ) ; 2 } John Jannotti (cs32) Design with Threads April 5, 2018 16 / 31
Producer-Consumer Producer Adds items to queue 1 w h i l e ( t r u e ) { 2 item = <get next item> 3 s y n c h r o n i z e d ( queue ) { 4 queue. add ( item ) ; 5 queue. n o t i f y ( ) ; 6 } 7 } John Jannotti (cs32) Design with Threads April 5, 2018 17 / 31
Building up from Primitives Now you know all the primitives you need. But actually, you should rarely use them! Get to know the java.util.concurrent package. Executors more flexible than raw Threads Concurrent Data Structures A variety of lock types / synchronization Written by smart, focused people. We ll motivate some of these utilities and implement some (perhaps poorly) so you never should! John Jannotti (cs32) Design with Threads April 5, 2018 18 / 31
Raw threads can be inflexible Consider a Web Server or Ray Tracing Engine. Both are embarassingly parallel Just start one thread per client connection! Just break the image into tiles! And yet... Too many clients... nobody served. How many tiles? John Jannotti (cs32) Design with Threads April 5, 2018 19 / 31
Separate specification of work from thread management You already wrote your code as Runnables. Instead of (new Thread(r)).start(); Generalize with an Executor. 1 E x e c u t o r e = <Get E x e c u t o r Implementation >; 2 e. e x e c u t e ( r ) ; Allows flexible execution. Serial, Pools, Scheduled... How would you implement a ThreadPoolExecutor? What does ThreadPoolExecutor.execute(Runnable) do? What will be shared among threads, and how will you protect it? John Jannotti (cs32) Design with Threads April 5, 2018 20 / 31
WorkerThread run() 1 p u b l i c v o i d run ( ) { 2 w h i l e ( t r u e ) { 3 Runnable t a s k = n u l l ; 4 s y n c h r o n i z e d ( work queue ) { 5 w h i l e ( work queue. isempty ( ) ) { 6 t r y { 7 work queue. w a i t ( ) ; 8 } c atch ( I n t e r r u p t e d E x c e p t i o n e ) { } 9 } 0 t a s k = work queue. remove ( ) ; 1 } 2 t a s k. run ( ) ; / Why not two l i n e s up? / 3 } 4 } John Jannotti (cs32) Design with Threads April 5, 2018 21 / 31
Handling Errors What if the internal run() throws an exception? Do you want to stop the worker? Do you want to record the report/problem? 1 t r y { 2 t a s k. run ( ) ; 3 } c a t c h ( Throwable t ) { 4 // l o g an e r r o r ( and s t a c k t r a c e?) 5 } Be sure to catch Throwable. Runnable doesn t support returning values (or errors). John Jannotti (cs32) Design with Threads April 5, 2018 22 / 31
Adding to the queue() 1 p u b l i c v o i d e x e c u t e ( Runnable t a s k ) { 2 s y n c h r o n i z e d ( work queue ) { 3 work queue. add ( t a s k ) ; 4 work queue. n o t i f y A l l ( ) ; 5 } 6 } Together, these are the Monitor code pattern. Specifically Producer/Consumer. John Jannotti (cs32) Design with Threads April 5, 2018 23 / 31
Producer/Consumer The monitor code fragment is not very clear. A BlockingQueue is a nicer abstraction. 1 p u b l i c v o i d run ( ) { 2 w h i l e ( t r u e ) { 3 t r y { 4 Runnable t a s k = work queue. t a k e ( ) ; 5 t a s k. run ( ) ; / No w o r r i e s about b e i n g slow. / 6 } c a tch ( I n t e r r u p t e d E x c e p t i o n e ) { } 7 } 8 } 9 p u b l i c v o i d e x e c u t e ( Runnable t a s k ) { 0 t r y { 1 work queue. put ( t a s k ) ; 2 } c atch ( I n t e r r u p t e d E x c e p t i o n e ) { } 3 } John Jannotti (cs32) Design with Threads April 5, 2018 24 / 31
Task completion and returning values There s no longer a Thread to join(). How can we tell if our tasks are complete? Do you care about individual tasks? Or maybe just that all of them are done. Ideas? John Jannotti (cs32) Design with Threads April 5, 2018 25 / 31
Several designs Query the queue size (all). Only if you know all tasks submitted. Queue would need to lie about the currently running task. Use a CountDownLatch (n). Provide a better Runnable implementation that wraps Runnables. Make a class that knows it has been run. void execute(donerunnable task) task.isdone() JDK: void execute(runnablefuture<t> task) Use a richer execute() interface. Future<T> submit(callable<t> task) Future and Callable are real. Speculate on their interface? Available from an ExecutorService John Jannotti (cs32) Design with Threads April 5, 2018 26 / 31
Concurrent Datastructures Suppose you want to use a Map or Queue in several threads. You could use Collections.synchronizedMap (et al.) But Those make every method synchronized And still require external synchronization for common ops. 1 s y n c h r o n i z e d (map) { 2 i f (! map. c o n t a i n s K e y ( key ) ) 3 v a l u e = <some l e n g t h y computation >; 4 r e t u r n map. put ( key, v a l u e ) ; 5 e l s e 6 r e t u r n map. get ( key ) ; 7 } ConcurrentHashMap offers putifabsent and even computeifabsent. John Jannotti (cs32) Design with Threads April 5, 2018 27 / 31
Concurrency utilities to the rescue ConcurrentMap, ConcurrentQueue interfaces. ConcurrentHashMap, ConcurrentLinkedQueue... classes. Allow multiple readers (and sometimes writers) oldval = map.putifabsent(key, value); newval = map.computeifabsent(key, Tile::new) John Jannotti (cs32) Design with Threads April 5, 2018 28 / 31
Simple atomic ooperations A common shared variable is a statistics counter. Multiple threads need to increment atomically. java.util.concurrent.atomic.atomicinteger John Jannotti (cs32) Design with Threads April 5, 2018 29 / 31
Immutable Objects Consider a Complex number package. 1 p u b l i c v o i d add ( o t h e r ) { 2 r e a l += o t h e r. r e a l ; 3 imag += o t h e r. imag ; 4 } vs 1 p u b l i c Complex add ( Complex o t h e r ) { 2 r e t u r n new Complex ( r e a l + o t h e r. r e a l, 3 imag + o t h e r. imag ) ; 4 } John Jannotti (cs32) Design with Threads April 5, 2018 30 / 31
Synchronization and Files Files are not inherently thread-safe. I/O may not be atomic. POSIX specifies some guarantees, but libraries obfuscate details. And file position is shared across threads Try to access files with only one thread Or use synchronization (on the same object!) Not like this: synchronized (new File("safe.txt")) {... } Don t worry about pure-read workloads. Log files are problematic (maybe you don t mind). OS supports opening in append mode. new FileWriter(..., true) John Jannotti (cs32) Design with Threads April 5, 2018 31 / 31