Using Design Patterns and Frameworks to Develop Object-Oriented ...

8 downloads 139 Views 4MB Size Report
Using Design Patterns and. Frameworks to Develop. Object-Oriented Communication. Systems. Douglas C. Schmidt www.cs.wustl.edu schmidt.
Using Design Patterns and Frameworks to Develop Object-Oriented Communication Systems

Motivation 



Douglas C. Schmidt www.cs.wustl.edu/schmidt/ [email protected] Washington University, St. Louis



Developing ecient, robust, extensible, portable, and reusable communication software is hard It is essential to understand successful techniques that have proven e ective to solve common development challenges Design patterns and frameworks help to capture, articulate, and instantiate these successful techniques

2

1

Observations 

Developers of communication software confront recurring challenges that are largely application-independent {





Design Patterns 

e.g., service initialization and distribution, error handling, ow control, event demultiplexing, concurrency control

Successful developers resolve these challenges by applying appropriate design patterns However, these patterns have traditionally been either: 1. Locked inside heads of expert developers 2. Buried in source code 3

Design patterns represent solutions to problems that arise when developing software within a particular context {



==

problem/solution pairs in a

Patterns capture the static and dynamic structure and collaboration among key participants in software designs {



i.e., \Patterns context"

They are particularly useful for articulating how and why to resolve non-functional forces

Patterns facilitate reuse of successful software architectures and designs 4

Proxy Pattern

Graphical Notation

1: METHOD

: BROKER

CALL

4: METHOD RETURN

: QUOTER PROXY

OBJECT

PROCESS THREAD

2: FORWARD

: CLASS

REQUEST

3: RESPONSE

CLASS

CLIENT : QUOTER

TEMPLATE CLASS

CLASS UTILITY

NETWORK CLASS CATEGORY

INHERITS

INSTANTIATES

SERVER

ABSTRACT CLASS

A 

CONTAINS

USES

Intent: provide a surrogate for another object that controls access to it 5

6

Frameworks More Observations 

Patterns enable reuse of architecture and design knowledge, but not code (directly)

A framework is: {

Reuse of patterns alone is not sucient {







\An integrated collection of components that collaborate to produce a reusable architecture for a family of related applications"

Frameworks di er from conventional class libraries: 1. Frameworks are \semi-complete" applications

To be productive, developers must also reuse detailed designs, algorithms, interfaces, implementations, etc.

2. Frameworks address particular application domains 3. Frameworks provide \inversion of control"



Application frameworks are an e ective way to achieve broad reuse of software

7



Typically, applications are developed by inheriting from and instantiating framework components 8

Tutorial Outline

Di erences Between Class Libraries and Frameworks



NETWORKING

APPLICATION SPECIFIC LOGIC

MATH

INVOKES

USER INTERFACE

EVENT LOOP

ADTS



DATA BASE

Outline key challenges for developing communication software Present the key reusable design patterns and framework components in high-performance Web clients and servers {

Both single-threaded and various multi-threaded solutions are presented

{

The patterns and frameworks covered generalize to other communication software systems

(A)

CLASS LIBRARY ARCHITECTURE NETWORKING

MATH INVOKES

APPLICATION SPECIFIC LOGIC

ADTS

(B)

USER INTERFACE

CALL BACKS

EVENT LOOP

DATABASE

APPLICATION FRAMEWORK ARCHITECTURE

 e.g., ORBs, video-on-demand, medical imaging 

Discuss lessons learned from using patterns and frameworks on production software systems {

e.g., telecom, avionics, medical systems 10

9

Stand-alone vs. Distributed Application Architectures

Concurrency vs. Parallelism SERVER

PRINTER

maxfdp1

COMPUTER

read_fds

CLIENT

CD ROM

WORK REQUEST

FILE SYSTEM

WORK CLIENT

REQUEST

(1) STAND-ALONE APPLICATION ARCHITECTURE NAME SERVICE

TIME SERVICE

CYCLE SERVICE

SERVER

CPU1

FI LE

PRINT SERVICE

CD ROM PRINTER

FILE SYSTEM

11

CPU2

CPU3

CPU4

CLIENT

WORK REQUEST

CLIENT

(2) DISTRIBUTED APPLICATION ARCHITECTURE

CLIENT

CONCURRENT SERVER

SERVICE SERVICE

WORK REQUEST

CLIENT

DISPLAY

NETWORK

WORK REQUEST

WORK REQUEST

WORK REQUEST CLIENT

WORK REQUEST CLIENT

PARALLEL SERVER 12

Sources of Complexity 



Distributed application development exhibits both inherent and accidental complexity

Sources of Complexity (cont'd) 

Inherent complexity results from fundamental challenges, e.g., {

{

Accidental complexity results from limitations with tools and techniques, e.g., {

Distributed systems  Latency

Low-level tools

 e.g., Lack of type-secure, portable, re-entrant, and extensible system call interfaces and component libraries

 Error handling

{

Inadequate debugging support

 Service partitioning and load balancing

{

Widespread use of algorithmic decomposition

Concurrent systems  Race conditions

 Fine for explaining network programming concepts and algorithms but inadequate for developing large-scale distributed applications

 Deadlock avoidance

{

 Fair scheduling

Continuous rediscovery and reinvention of core concepts and components

 Performance optimization and tuning 14

13

OO Contributions 



Communication software has traditionally been performed using low-level OS mechanisms, e.g., {

fork/exec

{

Shared memory

{

Signals

{

Sockets and select

{

POSIX pthreads, Solaris threads, Win32 threads

Concurrent Web Client/Server Example 



OO design patterns and frameworks elevate focus to application concerns, e.g., {

Service functionality and policies

{

Service con guration

{

Concurrent event demultiplexing and event handler dispatching

{

Service concurrency and synchronization

The following example illustrates a concurrent OO architecture for a high-performance Web client/server Key system requirements are: 1. Robust implementation of HTTP protocol {

2. Extensible for use with other protocols {

15

i.e., resilient to incorrect or malicious Web clients/servers e.g., DICOM, HTTP 1.1, SFP

3. Leverage multi-processor hardware and OS software {

e.g., Support various concurrency models 16

General Web Client/Server Interactions 1: GET ~schmidt HTTP/1.0

WWW CLIENT

WWW SERVER

2: index.html

Web Server Software Architecture HTTP Handler Sock Stream

PROTOCOL HANDLERS

HTML

HTTP Handler

Sock Stream

Sock Stream

Event Dispatcher

PARSER

GUI

HTTP Handler

HTTP Acceptor Sock Acceptor

DISPATCHER REQUESTER

 GRAPHICS ADAPTER

OS KERNEL

OS KERNEL

OS I/O SUBSYSTEM

OS I/O SUBSYSTEM

NETWORK ADAPTERS

NETWORK ADAPTERS

NETWORK

Event Dispatcher {

COMMUNICATION PROTOCOL (E.G., HTTP)



HTTP Handlers {



Encapsulates Web server concurrency and dispatching strategies Parses HTTP headers and processes requests

HTTP Acceptor {

Accepts connections and creates HTTP Handlers

17

Design Patterns in the Web Server Implementation

18

Tactical Patterns 

Proxy {

Thread Pool

Thread-per Request

Acceptor 

Connector

Active Object

Thread-per Session

Half-Sync/ Half-Async Asynchronous Completion Token



Double Checked Locking



Reactor/ Proactor

STRATEGIC PATTERNS TACTICAL PATTERNS State

Strategy

Abstract Factory

Adapter

Singleton

19

Strategy {

Service Configurator

\De ne a family of algorithms, encapsulate each one, and make them interchangeable"

Adapter {

\Convert the interface of a class into another interface client expects"

Singleton {



\Provide a surrogate or placeholder for another object to control access to it"

\Ensure a class only has one instance and provide a global point of access to it"

State {

\Allow an object to alter its behavior when its internal state changes" 20

Concurrency Patterns

Event Handling Patterns 

Reactor {





{



\Eciently associates state with the completion of asynchronous operations"

\Decouples method execution from method invocation and simpli es synchronized access to shared resources by concurrent threads"

Half-Sync/Half-Async {

\Decouples asynchronous event demultiplexing and event handler completion dispatching from service(s) performed in response to events"

Asynchronous Completion Token

Active Object {

\Decouples synchronous event demultiplexing and event handler initiation dispatching from service(s) performed in response to events"

Proactor {





\Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming e ort without degrading execution eciency"

Double-Checked Locking Optimization Pattern {

\Ensures atomic initialization of objects and eliminates unnecessary locking overhead on each access"

21

22

Concurrency Architecture Patterns 



{

\Allows each client connection to run concurrently"

 Suited for HTTP 1.1, but not HTTP 1.0 23



\Decouples active connection establishment from the service performed once the connection is established"

Acceptor {

\Allows up to N requests to execute concurrently within a pool of threads "

Thread-per-Connection

Connector {

\Allows each client request to run concurrently in a separate thread"

Thread Pool {





Thread-per-Request {



Service Initialization Patterns

\Decouples passive connection establishment from the service performed once the connection is established"

Service Con gurator {

\Decouples the behavior of network services from point in time at which services are con gured into an application"

24

Alternative Web Server Concurrency Patterns

Selecting the Server's Concurrency Architecture 

Problem {







A very strategic design decision for high-performance Web servers is selecting an ecient concurrency architecture

Forces



{

No single concurrency architecture is optimal

{

Key factors include OS/hardware platform and workload

The following are the key concurrency pattern alternatives: 1. Reactive 2. Thread-per-request 3. Thread-per-connection

Solution {

The following example illustrates the design patterns (and framework components) in an OO implementation of a concurrent Web Server

Understand key alternative concurrency patterns

4. Synchronous Thread Pool 5. Asynchronous Thread Pool

25

Thread-per-Request Web Server

Reactive Web Server HTTP Handler

6: PROCESS

2: HANDLE INPUT HTTP 3: CREATE HANDLER Handler 4: ACCEPT CONNECTION 5: SPAWN THREAD HTTP Handler HTTP HTTP Acceptor Handler

2: HANDLE INPUT 3: CREATE HANDLER 4: ACCEPT CONNECTION 5: ACTIVATE HANDLER

HTTP Handler

HTTP Acceptor

Reactor

26

Reactor

6: PROCESS HTTP REQUEST

HTTP REQUEST

SERVER

SERVER

1: CONNECT CLIENT

CLIENT

1: CONNECT CLIENT

CLIENT

27

CLIENT

CLIENT

28

Thread-per-Connection Web Server 3: SPAWN THREAD

2: CREATE, ACCEPT,

PER CONNECTION

HTTP Acceptor

2: ACCEPT CONNECTION 3: MORPH INTO HANDLER

HTTP Handler

Reactor

4: PROCESS HTTP REQUEST

Event Dispatcher

HTTP Handler

AND ACTIVATE HTTP_HANDLER

HTTP HTTP Handler Handler HTTP Handler

Handle-based Synchronous Thread Pool Web Server

HTTP Handler

HTTP Acceptor

HTTP Acceptor

1: HTTP

REQUEST

CLIENT

SERVER

4: PROCESS

HTTP REQUEST

1: HTTP

REQUEST

CLIENT

CLIENT

CLIENT

SERVER

CLIENT

CLIENT

29

Asynchronous Thread Pool Web Server

Queue-based Synchronous Thread Pool Web Server Message Queue

Active Object

4: DEQUEUE & Active Object

PROCESS REQUEST

Active Object

Active Object

5: PROCESS HTTP REQUEST

30

2: HANDLE INPUT 3: ENQUEUE REQUEST HTTP Handler HTTP Handler HTTP Handler

HTTP Acceptor

HTTP Handler

I/O Completion Port

6: DEQUEUE COMPLETION & PROCESS HTTP REQUEST Handler HTTP Handler

Reactor

REQUEST

CLIENT

Proactor

3: HTTP

REQUEST

SERVER CLIENT

Async Async Async Read Accept Write Async

Accept

HTTP Handler

7: PROCESS HTTP REQUEST

1: HTTP

1: INITIATE ASYNC ACCEPT 2: RUN EVENT LOOP 4: ACCEPT COMPLETES 5: QUEUE COMPLETION

SERVER CLIENT

CLIENT

CLIENT

31

CLIENT

32

The ADAPTIVE Communication Environment (ACE) SELF-CONTAINED DISTRIBUTED SERVICE COMPONENTS

TOKEN SERVER

LOGGING SERVER

JAWS ADAPTIVE WEB SERVER

GATEWAY SERVER

NAME SERVER

SERVICE HANDLER

CONNECTOR

C++

PROCESS/ THREAD MANAGERS

SYNCH WRAPPERS

C APIS

PROCESSES/ THREADS

LOG MSG SPIPE SAP

STREAM PIPES

PROCESS/THREAD SUBSYSTEM

SOCK_SAP/ TLI_SAP

SOCKETS/

TLI



svc_run

FIFO SAP

REACTOR/ PROACTOR

OS ADAPTATION LAYER NAMED SELECT/ PIPES

POSIX

AND

svc_run

svc_run

CORBA HANDLER

(ASX)

IO COMP

COMMUNICATION SUBSYSTEM

GENERAL

WWW SERVER svc_run

ADAPTIVE SERVICE EXECUTIVE

WRAPPERS

THE ACE ORB (TAO)

TIME SERVER

ACCEPTOR

FRAMEWORKS

MIDDLEWARE APPLICATIONS

Architecture of Our WWW Server

SERVICE CONFIGURATOR

DYNAMIC LINKING

MEM MAP

MEMORY MAPPING

SYSV

: Options

s

WRAPPERS

SYSTEM

V IPC

VIRTUAL MEMORY SUBSYSTEM

WIN32

: HTTP : Msg Processor Queue

SHARED MALLOC

: HTTP Handler

: HTTP Handler

SERVICES

: Reactor

A set of C++ wrappers and frameworks based on common communication software design patterns

: HTTP Handler : HTTP Acceptor

33

34

Demultiplexing and Dispatching Events 

{





The Reactor Pattern

Problem Web servers must process several di erent types of events simultaneously



Intent {

Forces {

Multi-threading is not always available

{

Multi-threading is not always ecient

{

Tightly coupling general event processing with server-speci c logic is in exible



Solution {

\Decouples synchronous event demultiplexing and event handler initiation dispatching from service(s) performed in response to events"

This pattern resolves the following forces for synchronous event-driven software: {

How to demultiplex multiple types of events from multiple sources of events synchronously and eciently within a single thread of control

{

How to extend application behavior without requiring changes to the event dispatching framework

Use the Reactor pattern to decouple generic event processing from server-speci c processing 35

36

Collaboration in the Reactor Pattern

Structure of the Reactor Pattern APPLICATION-

INDEPENDENT

DEPENDENT

Event_Handler

n

handle_input() handle_output() handle_signal() handle_timeout() get_handle() A

1

Initiation Dispatcher

1

handle_events() register_handler(h) remove_handler(h)

n

HTTP Acceptor n 1

1

schedule_timer(h) cancel_timer(h) expire_timers(h)

Handles 1

main program INITIALIZE

callback : Initiation Concrete Event_Handler Dispatcher Reactor() register_handler(callback)

REGISTER HANDLER

get_handle()

EXTRACT HANDLE

handle_events()

START EVENT LOOP

select()

FOREACH EVENT DO

Timer_Queue

1

1



HTTP Handler INITIALIZATION MODE

APPLICATION-

n

Participants in the Reactor pattern

EVENT HANDLING MODE

select (handles); foreach h in handles { if (h is output handler) table[h]->handle_output () ; if (h is input handler) table[h]->handle_input (); if (h is signal handler) table[h]->handle_signal (); } timer_queue->expire_timers ();

handle_input()

DATA ARRIVES

handle_output()

OK TO SEND

handle_signal()

SIGNAL ARRIVES

handle_timeout()

TIMER EXPIRES REMOVE HANDLER

remove_handler(callback) handle_close()

CLEANUP

37

38

An Integrated Reactive/Active Web Server

REGISTERED OBJECTS 5: recv(request) 6: process(request)

Event Handler

HTTP Handler

Event Handler

Event Handler

4: handle_input()

APPLICATION LEVEL

Event Handler

HTTP Handler

HTTP Acceptor

1: handle_input()

FRAMEWORK LEVEL

HTTP Handler

REGISTERED OBJECTS

2: accept() 3: make_handler()

Initiation Dispatcher

Reactor OS EVENT DEMULTIPLEXING INTERFACE

KERNEL LEVEL

KERNEL LEVEL

FRAMEWORK LEVEL

APPLICATION LEVEL

A Single-threaded Reactive Web Server

39

4: getq(msg) 5:svc(msg)

svc_run

svc_run svc_run

HTTP Handler HTTP Handler Message HTTP Event HTTP Queue Processor Handler Handler Event Handler Event 2: recv_request(msg) Handler3: putq(msg) 1: handle_input() Initiation Dispatcher

Reactor OS EVENT DEMULTIPLEXING INTERFACE

40

The HTTP Handler Public Interface 

The HTTP Handler is the Proxy for communicating with clients {

Along with Reactor, this class implements the asynchronous task part of Half-Sync/Half-Async

The HTTP Handler Protected Interface 

protected: // Reactor notifies when client's timeout. virtual int handle_timeout (const Time_Value &, const void *) { // Remove from the Reactor. Reactor::instance ()->remove_handler (this, READ_MASK); }

// Reusable base class. template class HTTP_Handler : public Svc_Handler { public: // Entry point into HTTP_Handler, called by // HTTP_Acceptor. virtual int open (void *) { // Register with Reactor to handle client input. Reactor::instance ()->register_handler (this, READ_M

}

The following methods are invoked by callbacks from the Reactor

// Reactor notifies when HTTP requests arrive. virtual int handle_input (HANDLE);

// Register timeout in case client doesn't // send any HTTP requests. Reactor::instance ()->schedule_timer (this, 0, ACE_Time_Value (HTTP_CLIENT_TIMEOUT));

// Receive/frame client HTTP requests (e.g., GET). int recv_request (Message_Block &*);

};

42

41

Integrating Multi-threading 

Problem {





Multi-threaded Web servers are needed since Reactive Web servers are often inecient, nonscalable, and non-robust

Forces {

Multi-threading can be very hard to program

{

No single multi-threading model is always optimal

Solution {

Use the Active Object pattern to allow multiple concurrent server operations using an OO programming style 43

The Active Object Pattern 

Intent {



\Decouples method execution from method invocation and simpli es synchronized access to shared resources by concurrent threads"

This pattern resolves the following forces for concurrent communication software: {

How to allow blocking read and write operations on one endpoint that do not detract from the quality of service of other endpoints

{

How to simplify concurrent access to shared state

{

How to simplify composition of independent services

44

Structure of the Active Object Pattern loop { m = actQueue.remove() dispatch (m) }

Client Interface

INVISIBLE TO CLIENTS



Scheduler

3: dispatch()

Activation dispatch() get_request'() Queue set_request'() insert() head_request'() 1 remove() 1 1 2: insert(get_request') 1 n 1 Resource Representation

Method Objects

METHOD OBJECT CONSTRUCTION

client

CREATE METHOD OBJECT RETURN RESULT HANDLE

REGISTERED OBJECTS

APPLICATION LEVEL

insert(m1')

DEQUEUE NEXT METHOD OBJECT

remove(m1')

EXECUTE

dispatch(m1')

RETURN RESULT

reply_to_future()

The Scheduler determines the sequence that Method Objects are executed

Using the Active Object Pattern in the Web Server

FRAMEWORK LEVEL

cons(m1') future()

INSERT IN PRIORITY QUEUE

45

KERNEL LEVEL

: Client : Activation : Represent: Scheduler Interface Queue ation

m1()

INVOKE

SCHEDULING/ EXECUTION

VISIBLE TO CLIENTS

1: get_request()

COMPLETION

get_request() set_request() head_request()

Collaboration in the Active Object Pattern

4: getq(msg) 5:svc(msg)

svc_run

svc_run

46

The HTTP Processor Class 

Processes HTTP requests using the \Thread Pool" concurrency model {

svc_run

Implement the synchronous task portion of the Half-Sync/Half-Async pattern class HTTP_Processor : public Task { public: // Singleton access point. static HTTP_Processor *instance (void);

HTTP Message HTTP HTTP Handler Queue Handler Processor HTTP Event Handler Event Handler Handler Event 2: recv_request(msg) Handler 3: putq(msg)

// Pass a request to the thread pool. virtual int put (Message_Block *); // Event loop for the pool thread virtual int svc (int) { Message_Block *mb = 0; // Message buffer.

1: handle_input() Initiation Dispatcher

// Wait for messages to arrive. for (;;) { getq (mb); // Inherited from class Task; // Identify and perform WWW Server // request processing here... } protected: HTTP_Processor (void); // Constructor.

Reactor OS EVENT DEMULTIPLEXING INTERFACE

47

48

Using the Singleton Pattern 

The HTTP Processor is implemented as a Singleton that is created \on demand"





Problem {

HTTP_Processor * HTTP_Processor::instance (void) { // Beware of race conditions! if (instance_ == 0) instance_ = new HTTP_Processor;

}

Subtle Concurrency Woes with the Singleton Pattern



return instance_;

Constructor creates the thread pool 

HTTP_Processor::HTTP_Processor (void) { // Inherited from class Task. activate (THR_NEW_LWP, num_threads); }

The canonical Singleton implementation has subtle \bugs" in multi-threaded applications

Forces {

Too much locking makes Singleton too slow

{

Too little locking makes Singleton unsafe

:::

Solution {

Use the Double-Checked Locking optimization pattern to minimize locking and ensure atomic initialization

49

The Double-Checked Locking Optimization Pattern 

Intent {



\Ensures atomic initialization of objects and eliminates unnecessary locking overhead on each access"

This pattern resolves the following forces: 1. Ensures atomic initialization or access to objects, regardless of thread scheduling order 2. Keeps locking overhead to a minimum {



:::

50

Using the Double-Checked Locking Optimization Pattern for the Web Server if (instance_ == NULL) { mutex_.acquire (); if (instance_ == NULL) instance_ = new HTTP_Processor; mutex_.release (); } return instance_;

HTTP Processor static instance() static instance_

e.g., only lock on creation

Note, this pattern assumes atomic memory access

Mutex

:::

51

52

Integrating Reactive and Multi-threaded Layers 

Problem {





Half-Sync/Half-Async Pattern 

Justifying the hybrid design of our Web server can be tricky

Forces {

Engineers are never satis ed with the status quo ;-)

{

Substantial amount of time is spent re-discovering the intent of complex concurrent software design

{



\Decouples synchronous I/O from asynchronous I/O in a system to simplify programming e ort without degrading execution eciency"

This pattern resolves the following forces for concurrent communication systems: {

How to simplify programming for higher-level communication tasks

 These are performed synchronously {

Solution {

Intent

How to ensure ecient lower-level I/O communication tasks

 These are performed asynchronously

Use the Half-Sync/Half-Async pattern to explain and justify our Web server concurrency architecture

54

53

Collaborations in the Half-Sync/Half-Async Pattern

SYNC

SYNC TASK 1

External Event Source

TASK 3

SYNC QUEUEING ASYNC PHASE PHASE PHASE

ASYNCHRONOUS TASK LAYER

QUEUEING LAYER

SYNCHRONOUS TASK LAYER

Structure of the Half-Sync/Half-Async Pattern

SYNC TASK 2 1, 4: read(data) MESSAGE QUEUES

3: enqueue(data)

EXTERNAL EVENT RECV MSG PROCESS MSG ENQUEUE MSG DEQUEUE MSG EXECUTE TASK

Async Task

Message Queue

Sync Task

notification() read(msg) work() enqueue(msg) read(msg) work()

ASYNC TASK

2: interrupt



EXTERNAL EVENT SOURCES

55

This illustrates input processing (output processing is similar) 56

Using the Half-Sync/Half-Async Pattern in the Web Server

ASYNC TASK LEVEL

QUEUEING LEVEL

SYNC TASK LEVEL

svc_run

svc_run

svc_run

Joining Async and Sync Tasks in the Web Server 

4: getq(msg) 5:svc(msg) Message Queue

The following methods form the boundary between the Async and Sync layers int HTTP_Handler::handle_input (void) { Message_Block *mb = 0;

HTTP Processor

HTTP HTTP HTTP HandlerHandler Handler

2: recv_request(msg) Event Event Event 3: putq(msg) HandlerHandler Handler

}

1: handle_input()

// Try to receive and frame message. if (recv_request (mb) == HTTP_REQUEST_COMPLETE) { Reactor::instance ()->remove_handler (this, READ_MASK); Reactor::instance ()->cancel_timer (this); // Insert message into the Queue. HTTP_Processor::instance ()->put (mb); }

// Task entry point. HTTP_Processor::put (Message_Block *msg) { // Insert the message on the Message_Queue // (inherited from class Task). putq (msg); }

Reactor

58

57

Optimizing Our Web Server for Asynchronous Operating Systems 

Problem {



Synchronous multi-threaded solutions are not always the most ecient



Intent {

Forces {

Purely asynchronous I/O is quite powerful on some OS platforms

 e.g., Windows nt 4.x {



The Proactor Pattern

Good designs should be adaptable to new contexts

Solution {



\Decouples asynchronous event demultiplexing and event handler completion dispatching from service(s) performed in response to events"

This pattern resolves the following forces for asynchronous event-driven software: {

How to demultiplex multiple types of events from multiple sources of events asynchronously and eciently within a minimal number of threads

{

How to extend application behavior without requiring changes to the event dispatching framework

Use the Proactor pattern to maximize performance on Asynchronous OS platforms 59

60

Structure of the Proactor Pattern APPLICATION-

APPLICATION-

INDEPENDENT

DEPENDENT

HTTP Handler

overlapped_result = GetQueuedCompleteStatus(); overlapped_result->complete()

Async Write HTTP Acceptor

Event_Handler

n

1

handle_accept() handle_read_file() handle_write_file() handle_timeout() get_handle() A 1

Completion Dispatcher

1

handle_events() register_handle()

n

Async Op

Async Accept

open() cancel()

n

Proactive Initiator Asynchronous operation initiated Operation performed asynchronously

1

1

Handles

Timer_Queue schedule_timer(h) cancel_timer(h) expire_timer(h) 1

1



Collaboration in the Proactor Pattern

Operation completes

Asynchronous Operation Asynchronous Processor Operation

Completion Dispatcher

Completion Handler

register (operation, handler, dispatcher) execute

dispatch

handle event

Completion Handler notified

Participants in the Proactor pattern 61

62

Client Connects to a Proactive Web Server Web Server

4: connect Web Browser

7: create HTTP Handler 8: read (connection, Handler, Dispatcher)

Acceptor 6: accept complete

Completion Dispatcher 3: handle events

1: accept connections

Client Sends Request to a Proactive Web Server 1: GET /etc/passwd Web Browser

2: accept (Acceptor, Dispatcher)

Operating System 5: accept complete

63

Web Server HTTP Handler

4: parse request

6: write (File, Conn., 8: write Handler, Dispatcher) 5: read (File) 3: read complete complete 7: write complete Completion Operating File Dispatcher System System 2: read complete

64

The Acceptor Pattern

Structuring Service Initialization 

Problem {







{

The communication protocol used between clients and the Web server is often orthogonal to the initialization protocol 

Forces {

Low-level connection establishment APIs are tedious, error-prone, and non-portable

{

Separating initialization from use can increase software reuse substantially

\Decouples passive initialization of a service from the tasks performed once the service is initialized"

This pattern resolves the following forces for network servers using interfaces like sockets or TLI: 1. How to reuse passive connection establishment code for each new service 2. How to make the connection establishment code portable across platforms that may contain sockets but not TLI, or vice versa

Solution {

Intent

3. How to ensure that a passive-mode descriptor is not accidentally used to read or write data

Use the Acceptor pattern to decouple passive service initialization from run-time protocol

4. How to enable exible policies for creation, connection establishment, and concurrency 66

65

Structure of the Acceptor Pattern

Collaboration in the Acceptor Pattern

HTTP Handler

HTTP Acceptor

Event Dispatcher

S FIE TI O N

ENDPOINT INITIALIZATION PHASE

CREATE & ACTIVATE

peer_acceptor_ handle_input()

SERVICE INITIALIZATION PHASE

peer_stream_ open()

acc : peer_acceptor_ sh: : Initiation : SOCK Acceptor Svc_Handler Dispatcher Acceptor open() open()

Server

is a factory that creates, connects, and activates a Svc Handler

 Acceptor

67

SERVICE PROCESSING PHASE

HTTP Handler

INITIALIZE PASSIVE ENDPOINT REGISTER HANDLER EXTRACT HANDLE START EVENT LOOP FOREACH EVENT DO CONNECTION EVENT

CREATE, ACCEPT, AND ACTIVATE OBJECT REGISTER HANDLER FOR CLIENT I/O EXTRACT HANDLE DATA EVENT PROCESS MSG CLIENT SHUTDOWN SERVER SHUTDOWN

register_handler(acc) get_handle() handle_events() select() handle_input() sh = make_svc_handler() accept_svc_handler (sh) activate_svc_handler (sh) register_handler(sh) get_handle() handle_input() svc() handle_close() handle_close()

68

Using the Acceptor Pattern in the Web Server HTTP Handler

HTTP Acceptor

HTTP Handler

Acceptor

Svc Handler

1: handle_input() 2: sh = make_svc_handler() 3: accept_svc_handler(sh) 4: activate_svc_handler(sh) PASSIVE LISTENER

Svc Handler

ACTIVE CONNECTIONS

Reactor

The Acceptor Class 

The Acceptor class implements the Acceptor pattern // Reusable Factor template class Acceptor : public Service_Object // Subclass of Event_Handler. { public: // Notified by Reactor when clients connect. virtual int handle_input (void) { // The strategy for initializing a SVC_HANDLER. SVC_HANDLER *sh = new SVC_HANDLER; peer_acceptor_.accept (sh->peer ()); sh->open (); } // ...

HTTP Handler Svc Handler

HTTP Handler Svc Handler

protected: // IPC connection factory. SOCK_Acceptor peer_acceptor_; }

70

69

The HTTP Acceptor Class Interface 

Putting the Pieces Together at Run-time 

{

The HTTP Acceptor class accepts connections and initializes HTTP Handlers class HTTP_Acceptor : public Acceptor // Inherits handle_input() strategy from Acceptor. { public: // Hook called automatically when HTTP_Acceptor // is dynamically linked. virtual int init (int argc, char *argv[]); // Hook called automatically when HTTP_Acceptor is // dynamically unlinked. virtual int fini (void);

}

// ...

Problem





Forces {

Certain server con guration decisions can't be made eciently until run-time

{

Forcing users to pay for components they don't use is undesirable

Solution {

71

Prematurely committing ourselves to a particular Web server con guration is in exible and inecient

Use the Service Con gurator pattern to assemble the desired Web server components dynamically 72

Structure of the Service Con gurator Pattern

{



\Decouples the behavior of communication services from the point in time at which these services are con gured into an application or system"

This pattern resolves the following forces for highly exible communication software: {

How to defer the selection of a particular type, or a particular implementation, of a service until very late in the design cycle

 i.e., at installation-time or run-time { {

How to build complete applications by composing multiple independently developed services How to optimize, recon gure, and control the behavior of the service at run-time

CONFIGURATION LAYER

Intent

REACTIVE LAYER



APPLICATION LAYER

The Service Con gurator Pattern

Concrete Service Object

Service Config

Service Object suspend() resume() init() A fini() info()

1 1 1

n 1

Service Repository

Event Handler n

1

Reactor

73

Collaboration in the Service Con gurator Pattern main()

CONFIGURATION MODE

CONFIGURE FOREACH SVC ENTRY DO DYNAMICALLY LINK SERVICE INITIALIZE SERVICE REGISTER SERVICE EXTRACT HANDLE

: Reactor

: Service : Service Config Repository

Service_Config() process_directives() link_service()

FOREACH EVENT DO INCOMING EVENT SHUTDOWN EVENT CLOSE SERVICE UNLINK SERVICE

Using the Service Con gurator Pattern in the Web Server SERVICE CONFIGURATOR RUNTIME

Service Repository

TP WWW Server Service Object

init(argc, argv) register_handler(svc) get_handle() insert()

STORE IN REPOSITORY START EVENT LOOP

EVENT HANDLING MODE

svc : Service_Object

74

Service Config

run_event_loop()

Reactor

Reactive WWW Server SHARED OBJECTS

Service Object

TPR WWW Server Service Object

handle_events() handle_input() handle_close()



remove_handler(svc) fini()

unlink_service()

remove()



75

Existing Web server is based on Half-Sync/HalfAsync pattern Other versions could be single-threaded, could use other concurrency strategies, and other protocols 76

The HTTP Acceptor Class Implementation // Initialize service when dynamically linked.

Con guring the Web Server with the Service Con gurator 

int HTTP_Acceptor::init (int argc, char *argv[]) { Options::instance ()->parse_args (argc, argv);

% cat ./svc.conf dynamic TP_WWW_Server Service_Object * www_server.dll:make_TP_WWW_Server() "-p $PORT -t $THREADS"

// Set the endpoint into listener mode. Acceptor::open (local_addr); // Initialize the communication endpoint. Reactor::instance ()->register_handler (this, ACCEPT_MASK)



// Terminate service when dynamically unlinked. int HTTP_Acceptor::fini (void) { // Unblock threads in the pool so they will // shutdown correctly. HTTP_Processor::instance ()->close ();

extern "C" Service_Object *make_TP_WWW_Server (void); Service_Object *make_TP_WWW_Server (void) { return new HTTP_Acceptor; // ACE dynamically unlinks and deallocates this object. }

// Wait for all threads to exit. Thread_Manager::instance ()->wait ();

78

77

The OO Architecture of the JAWS Framework

Main Program for Web Server Dynamically con gure and execute the Web Server {

Strategy



Note that this is totally generic!

State Singleton Cached Virtual Filesystem

Asynchronous Completion Token Protocol Handler

// Initialize the daemon and dynamically // configure the service. daemon.open (argc, argv);

Protocol Filter

Tilde Expander

Acceptor

int main (int argc, char *argv[]) { Service_Config daemon;

~ /home/...

Event Dispatcher

Adapter

// Loop forever, running services and handling // reconfigurations.

Protocol Pipeline Framework Streams

daemon.run_event_loop ();

}

Reactor/Proactor I/O Strategy Framework

Strategy

}

Factory function that dynamically allocates a Half-Sync/Half-Async Thread Pool Web Server

Concurrency Strategy Framework Service Configurator

/* NOTREACHED */

 79

www.cs.wustl.edu/~jxh/research/ 80

Active Object

}

The concurrent Web Server is con gured and initialized via a con guration script

Applying Patterns to CORBA ORBs

Web Server Optimization Techniques

CLIENT

SERVICE CONFIGURATOR



Use lightweight concurrency



Minimize locking

ABSTRACT FACTORY



Apply le caching and memory mapping

STRATEGY



Use \gather-write" mechanisms



Minimize logging



Pre-compute HTTP responses



Avoid excessive time calls



Optimize the transport interface

CONNECTOR

THREAD-SPECIFIC STORAGE





ACCEPTOR

WRAPPER FACADES OS KERNEL



Bene ts of Design Patterns



ACTIVE OBJECT

REACTOR

Design patterns enable large-scale reuse of software architectures Patterns explicitly capture expert knowledge and design tradeo s Patterns help improve developer communication Patterns help ease the transition to objectoriented technology

83

OS KERNEL

www.cs.wustl.edu/~schmidt/ORB-patterns.ps.gz

81



SERVANT

82

Drawbacks to Design Patterns 

Patterns do not lead to direct code reuse



Patterns are deceptively simple



Teams may su er from pattern overload





Patterns are validated by experience and discussion rather than by automated testing Integrating patterns into a software development process is a human-intensive activity 84

Suggestions for Using Patterns E ectively 

Do not recast everything as a pattern {









Lessons Learned using OO Frameworks

Instead, develop strategic domain patterns and reuse existing tactical patterns



Institutionalize rewards for developing patterns Directly involve pattern authors with application developers and domain experts



Bene ts of frameworks {

Enable direct reuse of code (cf patterns)

{

Facilitate larger amounts of reuse than standalone functions or individual classes

Drawbacks of frameworks {

High initial learning curve

 Many classes, many levels of abstraction

Clearly document when patterns apply and do not apply

{

The ow of control for reactive dispatching is non-intuitive

{

Veri cation and validation of generic components is hard

Manage expectations carefully 85

Patterns and Framework Literature 

{ { 

Gamma et al., \Design Patterns: Elements of Reusable OO Software" AW, 1994 Pattern Languages of Program Design series by AW, 1995,1997 Siemens, Pattern-Oriented Software Architecture, Wiley, 1996

{

October '96 \Communications of the ACM" (eds: Douglas C. Schmidt, Ralph Johnson, and Mohamed Fayad) October '97 \Communications of the ACM" (eds: Douglas C. Schmidt and Mohamed Fayad)

Magazines {





Special Issues in Journals {



Conferences and Workshops on Patterns

Books {

86

C++ Report and JOOP, columns by Coplien, Vlissides, Vinoski, Schmidt, and Martin 87



Pattern Language of Programs Conferences {

September, 1998, Monticello, Illinois, USA

{

st-www.cs.uiuc.edu/users/patterns/patterns.html

The European Pattern Languages of Programming conference {

July, 1998, Kloster Irsee, Germany

{

www.cs.wustl.edu/~schmidt/patterns.html

USENIX COOTS {

April 27,30, 1998, Santa Fe, New Mexico

{

www.usenix.org/events/coots98/ 88

Obtaining ACE and JAWS 

The ADAPTIVE Communication Environment (ACE) is an OO toolkit designed according to key network programming patterns {



All source code for ACE and JAWS is freely available {



JAWS is both a Web server framework and a high-performance Web server

www.cs.wustl.edu/~schmidt/ACE.html

Mailing lists * [email protected] * [email protected] * [email protected] * [email protected]



Newsgroup {

comp.soft-sys.ace 89