Event-based concurrency control - the Software Languages Lab

Event-based Concurrency Control Tom Van Cutsem Software Languages Lab Vrije Universiteit Brussel

Copyright is held by the author/owner(s). OOPSLA'09, October 25–29, 2009, Orlando, FL, USA. !!!!! ACM 09/10. OOPSLA 2009 Tutorial

1

Tom Van Cutsem

Goals Composing concurrent tasks Overview of existing models, their benefits and drawbacks Propose events as an alternative to the predominant model of multithreading Show that event-driven programming can be generalized to exploit multiple CPUs/cores OOPSLA 2009 Tutorial

2

Tom Van Cutsem

Agenda Before break: Threads Actors After break: Event-driven programming (Communicating) Event Loops OOPSLA 2009 Tutorial

3

Tom Van Cutsem

Why concurrency? to express independent tasks to deal effectively with I/O: Files, Sockets, ... for interactiveness (GUI, Games) distributed systems are inherently concurrent for efficiency (Scientific apps, web servers)

OOPSLA 2009 Tutorial

4

Tom Van Cutsem

Parallel vs Concurrent Programming Parallel programming: efficiency Matrix multiplication, FFT, search, solving PDEs, monte carlo, ... Concurrent programming: architectural reasons UI, I/O, ensuring responsiveness, distributed computing, etc. 5

OOPSLA 2009 Tutorial

Tom Van Cutsem

Threads (& Locks) Why threads are a bad idea (for most purposes) John Ousterhout Invited Talk at the 1996 USENIX Technical Conference

Concurrent Programming in Java: Design Principles and Patterns Doug Lea OOPSLA 2009 Tutorial

6

Tom Van Cutsem

Threads Multiple independent control flows Scheduler determines interleaving (implicit) Communicate by synchronously reading & writing shared data Synchronization via locks and condition variables

OOPSLA 2009 Tutorial

7

Tom Van Cutsem

Preemptive Scheduling A thread: may be preempted by any other thread at any time => inconsistent state, non-determinism must never explicitly yield control to another thread => automatic context switching

OOPSLA 2009 Tutorial

8

Tom Van Cutsem

Threads Threads

...

OOPSLA 2009 Tutorial

shared state (memory, files,...)

Tom Van Cutsem

9

Threads are for Wizards

OOPSLA 2009 Tutorial

10

(Ousterhout, 1995)

Tom Van Cutsem

The Problem with Threads seemingly straightforward adaptation of sequential programming model but: huge amount of non-determinism programmer’s job is to prune unwanted nondeterminism The Problem with Threads Edward Lee IEEE Computer, Vol. 39, No. 5, pp. 33–42, May 2006

OOPSLA 2009 Tutorial

Tom Van Cutsem

11

Example: concurrent increments Threads

...

OOPSLA 2009 Tutorial

c.inc()

12

c

Tom Van Cutsem

Unsynchronized Counter final Counter c = new Counter(); ! ! ! ! Thread[] threads = new Thread[MAX_THREADS]; for (int i = 0; i < MAX_THREADS; i++) { ! threads[i] = new Thread(new Runnable() { ! ! public void run() { ! ! ! for (int j = 0; j < NUM_INCS; j++) { ! ! ! ! c.inc(); ! ! ! } ! ! } ! }); ! threads[i].start(); } ! ! // wait for all threads to finish for (int j = 0; j < threads.length; j++) { ! threads[j].join(); }

OOPSLA 2009 Tutorial

class Counter private int public void val = val } }

{ val = 0; inc() { + 1;

MAX_THREADS = 10 NUM_INCS = 100.000 inc() -> 1.000.000x c.val -> 827.674 time -> 16 millisec

Tom Van Cutsem

13

Runtime view T1

...

c.inc() c.inc()

c

val = 5 6 7

T2

T1

T2

iload_0 5 iinc

6

istore_0 inc() { val = val + 1; }

iinc

iload_0 iinc istore_0

OOPSLA 2009 Tutorial

iload_0 6 7

istore_0

time 14

Tom Van Cutsem

Race Conditions T1

...

c.inc() c.inc()

c

val = 5 6 6

T2

T1

T2

iload_0 5 iinc

6 iload_0 5

inc() { val = val + 1; }

istore_0 iinc

iload_0 iinc istore_0

6

istore_0

time

OOPSLA 2009 Tutorial

Tom Van Cutsem

15

Race Conditions T1

...

c.inc() c.inc()

c

val = 5 6 6

T2

T1

T2

iload_0 5 iinc

6 iload_0 5

inc() { val = val + 1; } iload_0 iinc istore_0 Outcome depends

OOPSLA 2009 Tutorial

istore_0 iinc

6

istore_0

on thread scheduler! time 16

Tom Van Cutsem

Race Conditions

When program output depends unexpectedly upon the arbitrary ordering of concurrent activities

OOPSLA 2009 Tutorial

Tom Van Cutsem

17

Synchronized Counter final Counter c = new Counter(); ! ! ! ! Thread[] threads = new Thread[MAX_THREADS]; for (int i = 0; i < MAX_THREADS; i++) { ! threads[i] = new Thread(new Runnable() { ! ! public void run() { ! ! ! for (int j = 0; j < NUM_INCS; j++) { c.inc(); ! ! ! } ! ! } ! }); ! threads[i].start(); } ! ! // wait for all threads to finish for (int j = 0; j < threads.length; j++) { ! threads[j].join(); }

OOPSLA 2009 Tutorial

class Counter private int public void val = val } }

{ val = 0; inc() { + 1;

MAX_THREADS = 10 NUM_INCS = 100.000 inc() -> 1.000.000x c.val -> 827.674 time -> 16 millisec

18

Tom Van Cutsem

Synchronized Counter final Counter c = new Counter(); ! ! ! ! Thread[] threads = new Thread[MAX_THREADS]; for (int i = 0; i < MAX_THREADS; i++) { ! threads[i] = new Thread(new Runnable() { ! ! public void run() { ! ! ! for (int j = 0; j < NUM_INCS; j++) { synchronized (c) { c.inc(); } ! ! ! } ! ! } ! }); ! threads[i].start(); } ! ! // wait for all threads to finish for (int j = 0; j < threads.length; j++) { ! threads[j].join(); }

OOPSLA 2009 Tutorial

class Counter private int public void val = val } }

{ val = 0; inc() { + 1;

MAX_THREADS = 10 MAX_THREADS = 10 NUM_INCS = 100.000 NUM_INCS = 100.000 inc() -> 1.000.000x -> 1.000.000x c.valinc() -> 827.674 c.val -> millisec 1.000.000 time -> 16 time -> 159 millisec

Tom Van Cutsem

19

Locking T1

...

c.inc()

monitorenter

c

iload_0 5

c.inc()

T2

T2

T1 val = 5 6 7

iinc

6 monitorenter

istore_0 monitorexit synchronized(c) { val = val + 1; }

monitorenter iload_0 iinc istore_0 monitorexit

monitorenter iload_0 6 iinc istore_0

time OOPSLA 2009 Tutorial

20

7

monitorexit Tom Van Cutsem

Locking Requires Cooperation All involved threads must acquire the lock! A single thread that forgets to take the lock may concurrently enter the critical section Locking protocols

OOPSLA 2009 Tutorial

Tom Van Cutsem

21

One Forgetful Thread final Counter c = new Counter(); ! ! ! ! Thread[] threads = new Thread[MAX_THREADS-1]; for (int i = 0; i < threads.length; i++) { ! threads[i] = new Thread(new Runnable() { ! ! public void run() { ! ! ! for (int j = 0; j < NUM_INCS; j++) { ! ! ! ! synchronized (c) { c.inc(); } ! ! ! } ! ! } ! }); ! threads[i].start(); } Thread forgetful = new Thread(new Runnable() { public void run() { for (int j = 0; j < NUM_INCS; j++) { c.inc(); } } }); forgetful.start(); OOPSLA 2009 Tutorial

22

class Counter private int public void val = val } }

{ val = 0; inc() { + 1;

MAX_THREADS = 10 NUM_INCS = 100.000 inc() -> 1.000.000x c.val -> 985.724 time -> 242 millisec

Tom Van Cutsem

One Forgetful Thread T1

...

c.inc()

Tf

Tf

T1

c.inc()

monitorenter iload_0 5

c val = 5 6 7

iinc

6 iload_0 5

istore_0 synchronized(c) { val = val + 1; }

monitorenter iload_0 iinc istore_0 monitorexit

val = val + 1;

iload_0 iinc istore_0

OOPSLA 2009 Tutorial

monitorexit iinc

6

istore_0

time Tom Van Cutsem

23

Enforcing synchronization final Counter c = new Counter(); ! ! ! ! Thread[] threads = new Thread[MAX_THREADS-1]; for (int i = 0; i < threads.length; i++) { ! threads[i] = new Thread(new Runnable() { ! ! public void run() { ! ! ! for (int j = 0; j < NUM_INCS; j++) { ! ! ! ! synchronized (c) { c.inc(); } ! ! ! } ! ! } ! }); ! threads[i].start(); } Thread forgetful = new Thread(new Runnable() { public void run() { for (int j = 0; j < NUM_INCS; j++) { c.inc(); } } }); forgetful.start(); OOPSLA 2009 Tutorial

24

class Counter { private int val = 0; public synchronized void inc() { val = val + 1; } }

Tom Van Cutsem

Enforcing synchronization final Counter c = new Counter(); ! ! ! ! Thread evenIncT = new Thread(new Runnable() { public void run() { for (int j = 0; j < NUM_INCS; j++) { c.inc(); c.inc(); } } }); evenIncT.start(); Thread inspectorT = new Thread(new Runnable() { boolean sawOdd = false; public void run() { for (int j = 0; j < NUM_INCS; j++) { sawOdd = sawOdd | (c.count() % 2 == 1); } } }); inspectorT.start(); OOPSLA 2009 Tutorial

class Counter { private int val = 0; public synchronized void inc() { val = val + 1; } public synchronized int count() { return val; } }

sawOdd = true

Tom Van Cutsem

25

Enforcing synchronization final Counter c = new Counter(); ! ! !evenIncT ! Thread = new Thread(new Runnable() { public void run() { for (int j = 0; j < NUM_INCS; j++) { synchronized (c) { c.inc(); c.inc(); } } } }); evenIncT.start(); Thread inspectorT = new Thread(new Runnable() { boolean sawOdd = false; public void run() { for (int j = 0; j < NUM_INCS; j++) { sawOdd = sawOdd | (c.count() % 2 == 1); } } }); inspectorT.start(); OOPSLA 2009 Tutorial

26

class Counter { private int val = 0; public synchronized void inc() { val = val + 1; } public synchronized int count() { return val; } }

sawOdd = false

Tom Van Cutsem

Condition Variables Make threads wait for each other (without “busy waiting”) In Java: all objects are condition variables wait:

suspend thread until notified

notify:

wake up arbitrary waiting thread

notifyAll:

wake up all waiting threads

OOPSLA 2009 Tutorial

Tom Van Cutsem

27

A cell object Consumer

Producer

c.get() c.put(42) 42 c

OOPSLA 2009 Tutorial

28

Tom Van Cutsem

A cell object class Cell { private int content = 0; private boolean isEmpty = true; public synchronized void put(int v) { while (!isEmpty) { try { this.wait(); } catch (InterruptedException e) { } } ... isEmpty = false; public synchronized int get() { this.notifyAll(); while (isEmpty) { content = v; try { } this.wait(); ... } catch (InterruptedException e) { } } isEmpty = true; this.notifyAll(); return content; } }29 Tom Van Cutsem OOPSLA 2009 Tutorial

Producers & Consumers final Cell c = new Cell();

Thread producer = new Thread(new Runnable() { public void run() { for (int i = 0; i < n; i++) { c.put(produce(i)); } } }); Thread consumer = new Thread(new Runnable() { public void run() { for (int i = 0; i < n; i++) { consume(c.get()); } } }); OOPSLA 2009 Tutorial

30

Tom Van Cutsem

c

waiting for lock cer

du pro

waiting for notify um

s con

Trace

er

producer

consumer c.get() lock(c)

class Cell { private int content = 0; 42 private boolean isEmpty = true; false; true

c.put(42) lock(c) isEmpty?

public synchronized void put(int v) { while (!isEmpty) { this.wait(); } isEmpty = false; this.notifyAll(); content = v; }

OOPSLA 2009 Tutorial

lock(c) !isEmpty? isEmpty = false this.notifyAll() lock(c)

public synchronized int get() { while (isEmpty) { this.wait(); } isEmpty = true; this.notifyAll(); return content; } }

this.wait()

content = 42 unlock(c) lock(c) isEmpty?

31

time

isEmpty = true Tom Van Cutsem

Deadlocks class Counter { private int val = 0; public void inc(n) { val = val + n; } } final Counter c = new Counter(); final Cell cell = new Cell();

t1 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(1); } cell.put(10); } }) OOPSLA 2009 Tutorial

t2 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(cell.get()); } } })

32

Tom Van Cutsem

Deadlocks T1

t1 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(1); } cell.put(10); } })

T2

lock(counter) lock(counter) counter.inc(1) unlock(counter) lock(counter) cell.get() wait() cell.put(10)

t2 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(cell.get()); } } })

OOPSLA 2009 Tutorial

notifyAll() counter.inc(10)

time Tom Van Cutsem

33

Deadlocks T1

t1 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(1); } cell.put(10); } })

lock(counter) lock(counter)

cell.get() wait()

t2 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(cell.get()); } } })

OOPSLA 2009 Tutorial

T2

time 34

Tom Van Cutsem

Deadlocks T1

t1 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(1); } cell.put(10); } })

T2 lock(counter)

lock(counter)

cell.get() wait()

Deadlock occurrence depends on thread scheduler!

t2 = new Thread(new Runnable() { public void run() { synchronized (counter) { counter.inc(cell.get()); } } })

OOPSLA 2009 Tutorial

time 35

Tom Van Cutsem

Beware! Here be dragons threa ds Preemption: unit of concurrent interleaving is (bytecode) instruction or even smaller => not visible in the code Locking protocol requires cooperation from all threads => scattered throughout code Locking too little => race conditions Locking too much => deadlocks OOPSLA 2009 Tutorial

36

Tom Van Cutsem

Some advantages Synchronous communication does not disrupt sequential control flow Can exploit true multiprocessor concurrency (one thread per physical CPU/core) OS Support (but often heavyweight and platform-dependent)

OOPSLA 2009 Tutorial

37

Tom Van Cutsem

... and some more disadvantages Not easily distributable: shared-memory assumption Limited scalability: context switch for preemptively scheduled threads is heavyweight Overhead of managing thread state on stack But...

Why events are a bad idea (for high-concurrency servers) von Behren, Condit and Brewer Proceedings of the 9th USENIX Conference on Hot Topics in Operating Systems, 2003

OOPSLA 2009 Tutorial

38

Tom Van Cutsem

Best Practices Keep critical sections as small as possible Reduce shared state to a minimum Avoid calls to unknown code while holding locks Confine conditional synchronization to highlevel abstractions (e.g. a bounded buffer) Instead of spawning a large number of threads, better to use an event loop (e.g. managing client socket connections) OOPSLA 2009 Tutorial

39

Tom Van Cutsem

Actors

Concurrent Object-oriented Programming Gul Agha In Communications of the ACM, Vol 33 (9), p. 125, 1990 OOPSLA 2009 Tutorial

40

Tom Van Cutsem

The Actor Model Hewitt, Baker, Clinger, Agha, ... (MIT, late 1970s) (formed direct motivation to build Scheme!) Fundamental model of concurrent computation Designed for open distributed systems Functional and stateful (imperative) variants

OOPSLA 2009 Tutorial

Tom Van Cutsem

41

Actors private state

actor

asynchronous messaging

OOPSLA 2009 Tutorial

42

Tom Van Cutsem

Functional Actors An actor has: A mailbox: buffer of incoming messages A behaviour: a script to process incoming messages

“object + methods”

Acquaintances: references to other actors “object references”

OOPSLA 2009 Tutorial

43

Tom Van Cutsem

Functional Actors In response to a message, an actor can: create new actors send messages (asynchronously) become a new behavior

OOPSLA 2009 Tutorial

44

Tom Van Cutsem

Functional Actors become: specify replacement behaviour original and replacement behaviour process messages in parallel (pipelining!)

b b’

b’’

Tom Van Cutsem

45

OOPSLA 2009 Tutorial

Functional Actors become: specify replacement behaviour original and replacement behaviour process messages in parallel

b b’

b’’

OOPSLA 2009 Tutorial

46

Tom Van Cutsem

(Weak) Guarantees Messages not necessarily received in order of sending time Every message is eventually delivered A

B

msg1 msg2

OOPSLA 2009 Tutorial

Tom Van Cutsem

47

Example: a counter actor Functional def makeCounter(n) { behaviour { def inc() { become makeCounter(n+1) } def dec() { become makeCounter(n-1) } def read(customer) { customer