Using Design Patterns and. Frameworks to Develop. Object-Oriented
Communication. Systems. Douglas C. Schmidt www.cs.wustl.edu schmidt.
Using Design Patterns and Frameworks to Develop Object-Oriented Communication Systems
Motivation
Douglas C. Schmidt www.cs.wustl.edu/schmidt/
[email protected] Washington University, St. Louis
Developing ecient, robust, extensible, portable, and reusable communication software is hard It is essential to understand successful techniques that have proven e ective to solve common development challenges Design patterns and frameworks help to capture, articulate, and instantiate these successful techniques
2
1
Observations
Developers of communication software confront recurring challenges that are largely application-independent {
Design Patterns
e.g., service initialization and distribution, error handling, ow control, event demultiplexing, concurrency control
Successful developers resolve these challenges by applying appropriate design patterns However, these patterns have traditionally been either: 1. Locked inside heads of expert developers 2. Buried in source code 3
Design patterns represent solutions to problems that arise when developing software within a particular context {
==
problem/solution pairs in a
Patterns capture the static and dynamic structure and collaboration among key participants in software designs {
i.e., \Patterns context"
They are particularly useful for articulating how and why to resolve non-functional forces
Patterns facilitate reuse of successful software architectures and designs 4
Proxy Pattern
Graphical Notation
1: METHOD
: BROKER
CALL
4: METHOD RETURN
: QUOTER PROXY
OBJECT
PROCESS THREAD
2: FORWARD
: CLASS
REQUEST
3: RESPONSE
CLASS
CLIENT : QUOTER
TEMPLATE CLASS
CLASS UTILITY
NETWORK CLASS CATEGORY
INHERITS
INSTANTIATES
SERVER
ABSTRACT CLASS
A
CONTAINS
USES
Intent: provide a surrogate for another object that controls access to it 5
6
Frameworks More Observations
Patterns enable reuse of architecture and design knowledge, but not code (directly)
A framework is: {
Reuse of patterns alone is not sucient {
\An integrated collection of components that collaborate to produce a reusable architecture for a family of related applications"
Frameworks di er from conventional class libraries: 1. Frameworks are \semi-complete" applications
To be productive, developers must also reuse detailed designs, algorithms, interfaces, implementations, etc.
2. Frameworks address particular application domains 3. Frameworks provide \inversion of control"
Application frameworks are an e ective way to achieve broad reuse of software
7
Typically, applications are developed by inheriting from and instantiating framework components 8
Tutorial Outline
Di erences Between Class Libraries and Frameworks
NETWORKING
APPLICATION SPECIFIC LOGIC
MATH
INVOKES
USER INTERFACE
EVENT LOOP
ADTS
DATA BASE
Outline key challenges for developing communication software Present the key reusable design patterns and framework components in high-performance Web clients and servers {
Both single-threaded and various multi-threaded solutions are presented
{
The patterns and frameworks covered generalize to other communication software systems
(A)
CLASS LIBRARY ARCHITECTURE NETWORKING
MATH INVOKES
APPLICATION SPECIFIC LOGIC
ADTS
(B)
USER INTERFACE
CALL BACKS
EVENT LOOP
DATABASE
APPLICATION FRAMEWORK ARCHITECTURE
e.g., ORBs, video-on-demand, medical imaging
Discuss lessons learned from using patterns and frameworks on production software systems {
e.g., telecom, avionics, medical systems 10
9
Stand-alone vs. Distributed Application Architectures
Concurrency vs. Parallelism SERVER
PRINTER
maxfdp1
COMPUTER
read_fds
CLIENT
CD ROM
WORK REQUEST
FILE SYSTEM
WORK CLIENT
REQUEST
(1) STAND-ALONE APPLICATION ARCHITECTURE NAME SERVICE
TIME SERVICE
CYCLE SERVICE
SERVER
CPU1
FI LE
PRINT SERVICE
CD ROM PRINTER
FILE SYSTEM
11
CPU2
CPU3
CPU4
CLIENT
WORK REQUEST
CLIENT
(2) DISTRIBUTED APPLICATION ARCHITECTURE
CLIENT
CONCURRENT SERVER
SERVICE SERVICE
WORK REQUEST
CLIENT
DISPLAY
NETWORK
WORK REQUEST
WORK REQUEST
WORK REQUEST CLIENT
WORK REQUEST CLIENT
PARALLEL SERVER 12
Sources of Complexity
Distributed application development exhibits both inherent and accidental complexity
Sources of Complexity (cont'd)
Inherent complexity results from fundamental challenges, e.g., {
{
Accidental complexity results from limitations with tools and techniques, e.g., {
Distributed systems Latency
Low-level tools
e.g., Lack of type-secure, portable, re-entrant, and extensible system call interfaces and component libraries
Error handling
{
Inadequate debugging support
Service partitioning and load balancing
{
Widespread use of algorithmic decomposition
Concurrent systems Race conditions
Fine for explaining network programming concepts and algorithms but inadequate for developing large-scale distributed applications
Deadlock avoidance
{
Fair scheduling
Continuous rediscovery and reinvention of core concepts and components
Performance optimization and tuning 14
13
OO Contributions
Communication software has traditionally been performed using low-level OS mechanisms, e.g., {
fork/exec
{
Shared memory
{
Signals
{
Sockets and select
{
POSIX pthreads, Solaris threads, Win32 threads
Concurrent Web Client/Server Example
OO design patterns and frameworks elevate focus to application concerns, e.g., {
Service functionality and policies
{
Service con guration
{
Concurrent event demultiplexing and event handler dispatching
{
Service concurrency and synchronization
The following example illustrates a concurrent OO architecture for a high-performance Web client/server Key system requirements are: 1. Robust implementation of HTTP protocol {
2. Extensible for use with other protocols {
15
i.e., resilient to incorrect or malicious Web clients/servers e.g., DICOM, HTTP 1.1, SFP
3. Leverage multi-processor hardware and OS software {
e.g., Support various concurrency models 16
General Web Client/Server Interactions 1: GET ~schmidt HTTP/1.0
WWW CLIENT
WWW SERVER
2: index.html
Web Server Software Architecture HTTP Handler Sock Stream
PROTOCOL HANDLERS
HTML
HTTP Handler
Sock Stream
Sock Stream
Event Dispatcher
PARSER
GUI
HTTP Handler
HTTP Acceptor Sock Acceptor
DISPATCHER REQUESTER
GRAPHICS ADAPTER
OS KERNEL
OS KERNEL
OS I/O SUBSYSTEM
OS I/O SUBSYSTEM
NETWORK ADAPTERS
NETWORK ADAPTERS
NETWORK
Event Dispatcher {
COMMUNICATION PROTOCOL (E.G., HTTP)
HTTP Handlers {
Encapsulates Web server concurrency and dispatching strategies Parses HTTP headers and processes requests
HTTP Acceptor {
Accepts connections and creates HTTP Handlers
17
Design Patterns in the Web Server Implementation
18
Tactical Patterns
Proxy {
Thread Pool
Thread-per Request
Acceptor
Connector
Active Object
Thread-per Session
Half-Sync/ Half-Async Asynchronous Completion Token
Double Checked Locking
Reactor/ Proactor
STRATEGIC PATTERNS TACTICAL PATTERNS State
Strategy
Abstract Factory
Adapter
Singleton
19
Strategy {
Service Configurator
\De ne a family of algorithms, encapsulate each one, and make them interchangeable"
Adapter {
\Convert the interface of a class into another interface client expects"
Singleton {
\Provide a surrogate or placeholder for another object to control access to it"
\Ensure a class only has one instance and provide a global point of access to it"
State {
\Allow an object to alter its behavior when its internal state changes" 20
Concurrency Patterns
Event Handling Patterns
Reactor {
{
\Eciently associates state with the completion of asynchronous operations"
\Decouples method execution from method invocation and simpli es synchronized access to shared resources by concurrent threads"
Half-Sync/Half-Async {
\Decouples asynchronous event demultiplexing and event handler completion dispatching from service(s) performed in response to events"
Asynchronous Completion Token
Active Object {
\Decouples synchronous event demultiplexing and event handler initiation dispatching from service(s) performed in response to events"
Proactor {
\Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming e ort without degrading execution eciency"
Double-Checked Locking Optimization Pattern {
\Ensures atomic initialization of objects and eliminates unnecessary locking overhead on each access"
21
22
Concurrency Architecture Patterns
{
\Allows each client connection to run concurrently"
Suited for HTTP 1.1, but not HTTP 1.0 23
\Decouples active connection establishment from the service performed once the connection is established"
Acceptor {
\Allows up to N requests to execute concurrently within a pool of threads "
Thread-per-Connection
Connector {
\Allows each client request to run concurrently in a separate thread"
Thread Pool {
Thread-per-Request {
Service Initialization Patterns
\Decouples passive connection establishment from the service performed once the connection is established"
Service Con gurator {
\Decouples the behavior of network services from point in time at which services are con gured into an application"
24
Alternative Web Server Concurrency Patterns
Selecting the Server's Concurrency Architecture
Problem {
A very strategic design decision for high-performance Web servers is selecting an ecient concurrency architecture
Forces
{
No single concurrency architecture is optimal
{
Key factors include OS/hardware platform and workload
The following are the key concurrency pattern alternatives: 1. Reactive 2. Thread-per-request 3. Thread-per-connection
Solution {
The following example illustrates the design patterns (and framework components) in an OO implementation of a concurrent Web Server
Understand key alternative concurrency patterns
4. Synchronous Thread Pool 5. Asynchronous Thread Pool
25
Thread-per-Request Web Server
Reactive Web Server HTTP Handler
6: PROCESS
2: HANDLE INPUT HTTP 3: CREATE HANDLER Handler 4: ACCEPT CONNECTION 5: SPAWN THREAD HTTP Handler HTTP HTTP Acceptor Handler
2: HANDLE INPUT 3: CREATE HANDLER 4: ACCEPT CONNECTION 5: ACTIVATE HANDLER
HTTP Handler
HTTP Acceptor
Reactor
26
Reactor
6: PROCESS HTTP REQUEST
HTTP REQUEST
SERVER
SERVER
1: CONNECT CLIENT
CLIENT
1: CONNECT CLIENT
CLIENT
27
CLIENT
CLIENT
28
Thread-per-Connection Web Server 3: SPAWN THREAD
2: CREATE, ACCEPT,
PER CONNECTION
HTTP Acceptor
2: ACCEPT CONNECTION 3: MORPH INTO HANDLER
HTTP Handler
Reactor
4: PROCESS HTTP REQUEST
Event Dispatcher
HTTP Handler
AND ACTIVATE HTTP_HANDLER
HTTP HTTP Handler Handler HTTP Handler
Handle-based Synchronous Thread Pool Web Server
HTTP Handler
HTTP Acceptor
HTTP Acceptor
1: HTTP
REQUEST
CLIENT
SERVER
4: PROCESS
HTTP REQUEST
1: HTTP
REQUEST
CLIENT
CLIENT
CLIENT
SERVER
CLIENT
CLIENT
29
Asynchronous Thread Pool Web Server
Queue-based Synchronous Thread Pool Web Server Message Queue
Active Object
4: DEQUEUE & Active Object
PROCESS REQUEST
Active Object
Active Object
5: PROCESS HTTP REQUEST
30
2: HANDLE INPUT 3: ENQUEUE REQUEST HTTP Handler HTTP Handler HTTP Handler
HTTP Acceptor
HTTP Handler
I/O Completion Port
6: DEQUEUE COMPLETION & PROCESS HTTP REQUEST Handler HTTP Handler
Reactor
REQUEST
CLIENT
Proactor
3: HTTP
REQUEST
SERVER CLIENT
Async Async Async Read Accept Write Async
Accept
HTTP Handler
7: PROCESS HTTP REQUEST
1: HTTP
1: INITIATE ASYNC ACCEPT 2: RUN EVENT LOOP 4: ACCEPT COMPLETES 5: QUEUE COMPLETION
SERVER CLIENT
CLIENT
CLIENT
31
CLIENT
32
The ADAPTIVE Communication Environment (ACE) SELF-CONTAINED DISTRIBUTED SERVICE COMPONENTS
TOKEN SERVER
LOGGING SERVER
JAWS ADAPTIVE WEB SERVER
GATEWAY SERVER
NAME SERVER
SERVICE HANDLER
CONNECTOR
C++
PROCESS/ THREAD MANAGERS
SYNCH WRAPPERS
C APIS
PROCESSES/ THREADS
LOG MSG SPIPE SAP
STREAM PIPES
PROCESS/THREAD SUBSYSTEM
SOCK_SAP/ TLI_SAP
SOCKETS/
TLI
svc_run
FIFO SAP
REACTOR/ PROACTOR
OS ADAPTATION LAYER NAMED SELECT/ PIPES
POSIX
AND
svc_run
svc_run
CORBA HANDLER
(ASX)
IO COMP
COMMUNICATION SUBSYSTEM
GENERAL
WWW SERVER svc_run
ADAPTIVE SERVICE EXECUTIVE
WRAPPERS
THE ACE ORB (TAO)
TIME SERVER
ACCEPTOR
FRAMEWORKS
MIDDLEWARE APPLICATIONS
Architecture of Our WWW Server
SERVICE CONFIGURATOR
DYNAMIC LINKING
MEM MAP
MEMORY MAPPING
SYSV
: Options
s
WRAPPERS
SYSTEM
V IPC
VIRTUAL MEMORY SUBSYSTEM
WIN32
: HTTP : Msg Processor Queue
SHARED MALLOC
: HTTP Handler
: HTTP Handler
SERVICES
: Reactor
A set of C++ wrappers and frameworks based on common communication software design patterns
: HTTP Handler : HTTP Acceptor
33
34
Demultiplexing and Dispatching Events
{
The Reactor Pattern
Problem Web servers must process several di erent types of events simultaneously
Intent {
Forces {
Multi-threading is not always available
{
Multi-threading is not always ecient
{
Tightly coupling general event processing with server-speci c logic is in exible
Solution {
\Decouples synchronous event demultiplexing and event handler initiation dispatching from service(s) performed in response to events"
This pattern resolves the following forces for synchronous event-driven software: {
How to demultiplex multiple types of events from multiple sources of events synchronously and eciently within a single thread of control
{
How to extend application behavior without requiring changes to the event dispatching framework
Use the Reactor pattern to decouple generic event processing from server-speci c processing 35
36
Collaboration in the Reactor Pattern
Structure of the Reactor Pattern APPLICATION-
INDEPENDENT
DEPENDENT
Event_Handler
n
handle_input() handle_output() handle_signal() handle_timeout() get_handle() A
1
Initiation Dispatcher
1
handle_events() register_handler(h) remove_handler(h)
n
HTTP Acceptor n 1
1
schedule_timer(h) cancel_timer(h) expire_timers(h)
Handles 1
main program INITIALIZE
callback : Initiation Concrete Event_Handler Dispatcher Reactor() register_handler(callback)
REGISTER HANDLER
get_handle()
EXTRACT HANDLE
handle_events()
START EVENT LOOP
select()
FOREACH EVENT DO
Timer_Queue
1
1
HTTP Handler INITIALIZATION MODE
APPLICATION-
n
Participants in the Reactor pattern
EVENT HANDLING MODE
select (handles); foreach h in handles { if (h is output handler) table[h]->handle_output () ; if (h is input handler) table[h]->handle_input (); if (h is signal handler) table[h]->handle_signal (); } timer_queue->expire_timers ();
handle_input()
DATA ARRIVES
handle_output()
OK TO SEND
handle_signal()
SIGNAL ARRIVES
handle_timeout()
TIMER EXPIRES REMOVE HANDLER
remove_handler(callback) handle_close()
CLEANUP
37
38
An Integrated Reactive/Active Web Server
REGISTERED OBJECTS 5: recv(request) 6: process(request)
Event Handler
HTTP Handler
Event Handler
Event Handler
4: handle_input()
APPLICATION LEVEL
Event Handler
HTTP Handler
HTTP Acceptor
1: handle_input()
FRAMEWORK LEVEL
HTTP Handler
REGISTERED OBJECTS
2: accept() 3: make_handler()
Initiation Dispatcher
Reactor OS EVENT DEMULTIPLEXING INTERFACE
KERNEL LEVEL
KERNEL LEVEL
FRAMEWORK LEVEL
APPLICATION LEVEL
A Single-threaded Reactive Web Server
39
4: getq(msg) 5:svc(msg)
svc_run
svc_run svc_run
HTTP Handler HTTP Handler Message HTTP Event HTTP Queue Processor Handler Handler Event Handler Event 2: recv_request(msg) Handler3: putq(msg) 1: handle_input() Initiation Dispatcher
Reactor OS EVENT DEMULTIPLEXING INTERFACE
40
The HTTP Handler Public Interface
The HTTP Handler is the Proxy for communicating with clients {
Along with Reactor, this class implements the asynchronous task part of Half-Sync/Half-Async
The HTTP Handler Protected Interface
protected: // Reactor notifies when client's timeout. virtual int handle_timeout (const Time_Value &, const void *) { // Remove from the Reactor. Reactor::instance ()->remove_handler (this, READ_MASK); }
// Reusable base class. template class HTTP_Handler : public Svc_Handler { public: // Entry point into HTTP_Handler, called by // HTTP_Acceptor. virtual int open (void *) { // Register with Reactor to handle client input. Reactor::instance ()->register_handler (this, READ_M
}
The following methods are invoked by callbacks from the Reactor
// Reactor notifies when HTTP requests arrive. virtual int handle_input (HANDLE);
// Register timeout in case client doesn't // send any HTTP requests. Reactor::instance ()->schedule_timer (this, 0, ACE_Time_Value (HTTP_CLIENT_TIMEOUT));
// Receive/frame client HTTP requests (e.g., GET). int recv_request (Message_Block &*);
};
42
41
Integrating Multi-threading
Problem {
Multi-threaded Web servers are needed since Reactive Web servers are often inecient, nonscalable, and non-robust
Forces {
Multi-threading can be very hard to program
{
No single multi-threading model is always optimal
Solution {
Use the Active Object pattern to allow multiple concurrent server operations using an OO programming style 43
The Active Object Pattern
Intent {
\Decouples method execution from method invocation and simpli es synchronized access to shared resources by concurrent threads"
This pattern resolves the following forces for concurrent communication software: {
How to allow blocking read and write operations on one endpoint that do not detract from the quality of service of other endpoints
{
How to simplify concurrent access to shared state
{
How to simplify composition of independent services
44
Structure of the Active Object Pattern loop { m = actQueue.remove() dispatch (m) }
Client Interface
INVISIBLE TO CLIENTS
Scheduler
3: dispatch()
Activation dispatch() get_request'() Queue set_request'() insert() head_request'() 1 remove() 1 1 2: insert(get_request') 1 n 1 Resource Representation
Method Objects
METHOD OBJECT CONSTRUCTION
client
CREATE METHOD OBJECT RETURN RESULT HANDLE
REGISTERED OBJECTS
APPLICATION LEVEL
insert(m1')
DEQUEUE NEXT METHOD OBJECT
remove(m1')
EXECUTE
dispatch(m1')
RETURN RESULT
reply_to_future()
The Scheduler determines the sequence that Method Objects are executed
Using the Active Object Pattern in the Web Server
FRAMEWORK LEVEL
cons(m1') future()
INSERT IN PRIORITY QUEUE
45
KERNEL LEVEL
: Client : Activation : Represent: Scheduler Interface Queue ation
m1()
INVOKE
SCHEDULING/ EXECUTION
VISIBLE TO CLIENTS
1: get_request()
COMPLETION
get_request() set_request() head_request()
Collaboration in the Active Object Pattern
4: getq(msg) 5:svc(msg)
svc_run
svc_run
46
The HTTP Processor Class
Processes HTTP requests using the \Thread Pool" concurrency model {
svc_run
Implement the synchronous task portion of the Half-Sync/Half-Async pattern class HTTP_Processor : public Task { public: // Singleton access point. static HTTP_Processor *instance (void);
HTTP Message HTTP HTTP Handler Queue Handler Processor HTTP Event Handler Event Handler Handler Event 2: recv_request(msg) Handler 3: putq(msg)
// Pass a request to the thread pool. virtual int put (Message_Block *); // Event loop for the pool thread virtual int svc (int) { Message_Block *mb = 0; // Message buffer.
1: handle_input() Initiation Dispatcher
// Wait for messages to arrive. for (;;) { getq (mb); // Inherited from class Task; // Identify and perform WWW Server // request processing here... } protected: HTTP_Processor (void); // Constructor.
Reactor OS EVENT DEMULTIPLEXING INTERFACE
47
48
Using the Singleton Pattern
The HTTP Processor is implemented as a Singleton that is created \on demand"
Problem {
HTTP_Processor * HTTP_Processor::instance (void) { // Beware of race conditions! if (instance_ == 0) instance_ = new HTTP_Processor;
}
Subtle Concurrency Woes with the Singleton Pattern
return instance_;
Constructor creates the thread pool
HTTP_Processor::HTTP_Processor (void) { // Inherited from class Task. activate (THR_NEW_LWP, num_threads); }
The canonical Singleton implementation has subtle \bugs" in multi-threaded applications
Forces {
Too much locking makes Singleton too slow
{
Too little locking makes Singleton unsafe
:::
Solution {
Use the Double-Checked Locking optimization pattern to minimize locking and ensure atomic initialization
49
The Double-Checked Locking Optimization Pattern
Intent {
\Ensures atomic initialization of objects and eliminates unnecessary locking overhead on each access"
This pattern resolves the following forces: 1. Ensures atomic initialization or access to objects, regardless of thread scheduling order 2. Keeps locking overhead to a minimum {
:::
50
Using the Double-Checked Locking Optimization Pattern for the Web Server if (instance_ == NULL) { mutex_.acquire (); if (instance_ == NULL) instance_ = new HTTP_Processor; mutex_.release (); } return instance_;
HTTP Processor static instance() static instance_
e.g., only lock on creation
Note, this pattern assumes atomic memory access
Mutex
:::
51
52
Integrating Reactive and Multi-threaded Layers
Problem {
Half-Sync/Half-Async Pattern
Justifying the hybrid design of our Web server can be tricky
Forces {
Engineers are never satis ed with the status quo ;-)
{
Substantial amount of time is spent re-discovering the intent of complex concurrent software design
{
\Decouples synchronous I/O from asynchronous I/O in a system to simplify programming e ort without degrading execution eciency"
This pattern resolves the following forces for concurrent communication systems: {
How to simplify programming for higher-level communication tasks
These are performed synchronously {
Solution {
Intent
How to ensure ecient lower-level I/O communication tasks
These are performed asynchronously
Use the Half-Sync/Half-Async pattern to explain and justify our Web server concurrency architecture
54
53
Collaborations in the Half-Sync/Half-Async Pattern
SYNC
SYNC TASK 1
External Event Source
TASK 3
SYNC QUEUEING ASYNC PHASE PHASE PHASE
ASYNCHRONOUS TASK LAYER
QUEUEING LAYER
SYNCHRONOUS TASK LAYER
Structure of the Half-Sync/Half-Async Pattern
SYNC TASK 2 1, 4: read(data) MESSAGE QUEUES
3: enqueue(data)
EXTERNAL EVENT RECV MSG PROCESS MSG ENQUEUE MSG DEQUEUE MSG EXECUTE TASK
Async Task
Message Queue
Sync Task
notification() read(msg) work() enqueue(msg) read(msg) work()
ASYNC TASK
2: interrupt
EXTERNAL EVENT SOURCES
55
This illustrates input processing (output processing is similar) 56
Using the Half-Sync/Half-Async Pattern in the Web Server
ASYNC TASK LEVEL
QUEUEING LEVEL
SYNC TASK LEVEL
svc_run
svc_run
svc_run
Joining Async and Sync Tasks in the Web Server
4: getq(msg) 5:svc(msg) Message Queue
The following methods form the boundary between the Async and Sync layers int HTTP_Handler::handle_input (void) { Message_Block *mb = 0;
HTTP Processor
HTTP HTTP HTTP HandlerHandler Handler
2: recv_request(msg) Event Event Event 3: putq(msg) HandlerHandler Handler
}
1: handle_input()
// Try to receive and frame message. if (recv_request (mb) == HTTP_REQUEST_COMPLETE) { Reactor::instance ()->remove_handler (this, READ_MASK); Reactor::instance ()->cancel_timer (this); // Insert message into the Queue. HTTP_Processor::instance ()->put (mb); }
// Task entry point. HTTP_Processor::put (Message_Block *msg) { // Insert the message on the Message_Queue // (inherited from class Task). putq (msg); }
Reactor
58
57
Optimizing Our Web Server for Asynchronous Operating Systems
Problem {
Synchronous multi-threaded solutions are not always the most ecient
Intent {
Forces {
Purely asynchronous I/O is quite powerful on some OS platforms
e.g., Windows nt 4.x {
The Proactor Pattern
Good designs should be adaptable to new contexts
Solution {
\Decouples asynchronous event demultiplexing and event handler completion dispatching from service(s) performed in response to events"
This pattern resolves the following forces for asynchronous event-driven software: {
How to demultiplex multiple types of events from multiple sources of events asynchronously and eciently within a minimal number of threads
{
How to extend application behavior without requiring changes to the event dispatching framework
Use the Proactor pattern to maximize performance on Asynchronous OS platforms 59
60
Structure of the Proactor Pattern APPLICATION-
APPLICATION-
INDEPENDENT
DEPENDENT
HTTP Handler
overlapped_result = GetQueuedCompleteStatus(); overlapped_result->complete()
Async Write HTTP Acceptor
Event_Handler
n
1
handle_accept() handle_read_file() handle_write_file() handle_timeout() get_handle() A 1
Completion Dispatcher
1
handle_events() register_handle()
n
Async Op
Async Accept
open() cancel()
n
Proactive Initiator Asynchronous operation initiated Operation performed asynchronously
1
1
Handles
Timer_Queue schedule_timer(h) cancel_timer(h) expire_timer(h) 1
1
Collaboration in the Proactor Pattern
Operation completes
Asynchronous Operation Asynchronous Processor Operation
Completion Dispatcher
Completion Handler
register (operation, handler, dispatcher) execute
dispatch
handle event
Completion Handler notified
Participants in the Proactor pattern 61
62
Client Connects to a Proactive Web Server Web Server
4: connect Web Browser
7: create HTTP Handler 8: read (connection, Handler, Dispatcher)
Acceptor 6: accept complete
Completion Dispatcher 3: handle events
1: accept connections
Client Sends Request to a Proactive Web Server 1: GET /etc/passwd Web Browser
2: accept (Acceptor, Dispatcher)
Operating System 5: accept complete
63
Web Server HTTP Handler
4: parse request
6: write (File, Conn., 8: write Handler, Dispatcher) 5: read (File) 3: read complete complete 7: write complete Completion Operating File Dispatcher System System 2: read complete
64
The Acceptor Pattern
Structuring Service Initialization
Problem {
{
The communication protocol used between clients and the Web server is often orthogonal to the initialization protocol
Forces {
Low-level connection establishment APIs are tedious, error-prone, and non-portable
{
Separating initialization from use can increase software reuse substantially
\Decouples passive initialization of a service from the tasks performed once the service is initialized"
This pattern resolves the following forces for network servers using interfaces like sockets or TLI: 1. How to reuse passive connection establishment code for each new service 2. How to make the connection establishment code portable across platforms that may contain sockets but not TLI, or vice versa
Solution {
Intent
3. How to ensure that a passive-mode descriptor is not accidentally used to read or write data
Use the Acceptor pattern to decouple passive service initialization from run-time protocol
4. How to enable exible policies for creation, connection establishment, and concurrency 66
65
Structure of the Acceptor Pattern
Collaboration in the Acceptor Pattern
HTTP Handler
HTTP Acceptor
Event Dispatcher
S FIE TI O N
ENDPOINT INITIALIZATION PHASE
CREATE & ACTIVATE
peer_acceptor_ handle_input()
SERVICE INITIALIZATION PHASE
peer_stream_ open()
acc : peer_acceptor_ sh: : Initiation : SOCK Acceptor Svc_Handler Dispatcher Acceptor open() open()
Server
is a factory that creates, connects, and activates a Svc Handler
Acceptor
67
SERVICE PROCESSING PHASE
HTTP Handler
INITIALIZE PASSIVE ENDPOINT REGISTER HANDLER EXTRACT HANDLE START EVENT LOOP FOREACH EVENT DO CONNECTION EVENT
CREATE, ACCEPT, AND ACTIVATE OBJECT REGISTER HANDLER FOR CLIENT I/O EXTRACT HANDLE DATA EVENT PROCESS MSG CLIENT SHUTDOWN SERVER SHUTDOWN
register_handler(acc) get_handle() handle_events() select() handle_input() sh = make_svc_handler() accept_svc_handler (sh) activate_svc_handler (sh) register_handler(sh) get_handle() handle_input() svc() handle_close() handle_close()
68
Using the Acceptor Pattern in the Web Server HTTP Handler
HTTP Acceptor
HTTP Handler
Acceptor
Svc Handler
1: handle_input() 2: sh = make_svc_handler() 3: accept_svc_handler(sh) 4: activate_svc_handler(sh) PASSIVE LISTENER
Svc Handler
ACTIVE CONNECTIONS
Reactor
The Acceptor Class
The Acceptor class implements the Acceptor pattern // Reusable Factor template class Acceptor : public Service_Object // Subclass of Event_Handler. { public: // Notified by Reactor when clients connect. virtual int handle_input (void) { // The strategy for initializing a SVC_HANDLER. SVC_HANDLER *sh = new SVC_HANDLER; peer_acceptor_.accept (sh->peer ()); sh->open (); } // ...
HTTP Handler Svc Handler
HTTP Handler Svc Handler
protected: // IPC connection factory. SOCK_Acceptor peer_acceptor_; }
70
69
The HTTP Acceptor Class Interface
Putting the Pieces Together at Run-time
{
The HTTP Acceptor class accepts connections and initializes HTTP Handlers class HTTP_Acceptor : public Acceptor // Inherits handle_input() strategy from Acceptor. { public: // Hook called automatically when HTTP_Acceptor // is dynamically linked. virtual int init (int argc, char *argv[]); // Hook called automatically when HTTP_Acceptor is // dynamically unlinked. virtual int fini (void);
}
// ...
Problem
Forces {
Certain server con guration decisions can't be made eciently until run-time
{
Forcing users to pay for components they don't use is undesirable
Solution {
71
Prematurely committing ourselves to a particular Web server con guration is in exible and inecient
Use the Service Con gurator pattern to assemble the desired Web server components dynamically 72
Structure of the Service Con gurator Pattern
{
\Decouples the behavior of communication services from the point in time at which these services are con gured into an application or system"
This pattern resolves the following forces for highly exible communication software: {
How to defer the selection of a particular type, or a particular implementation, of a service until very late in the design cycle
i.e., at installation-time or run-time { {
How to build complete applications by composing multiple independently developed services How to optimize, recon gure, and control the behavior of the service at run-time
CONFIGURATION LAYER
Intent
REACTIVE LAYER
APPLICATION LAYER
The Service Con gurator Pattern
Concrete Service Object
Service Config
Service Object suspend() resume() init() A fini() info()
1 1 1
n 1
Service Repository
Event Handler n
1
Reactor
73
Collaboration in the Service Con gurator Pattern main()
CONFIGURATION MODE
CONFIGURE FOREACH SVC ENTRY DO DYNAMICALLY LINK SERVICE INITIALIZE SERVICE REGISTER SERVICE EXTRACT HANDLE
: Reactor
: Service : Service Config Repository
Service_Config() process_directives() link_service()
FOREACH EVENT DO INCOMING EVENT SHUTDOWN EVENT CLOSE SERVICE UNLINK SERVICE
Using the Service Con gurator Pattern in the Web Server SERVICE CONFIGURATOR RUNTIME
Service Repository
TP WWW Server Service Object
init(argc, argv) register_handler(svc) get_handle() insert()
STORE IN REPOSITORY START EVENT LOOP
EVENT HANDLING MODE
svc : Service_Object
74
Service Config
run_event_loop()
Reactor
Reactive WWW Server SHARED OBJECTS
Service Object
TPR WWW Server Service Object
handle_events() handle_input() handle_close()
remove_handler(svc) fini()
unlink_service()
remove()
75
Existing Web server is based on Half-Sync/HalfAsync pattern Other versions could be single-threaded, could use other concurrency strategies, and other protocols 76
The HTTP Acceptor Class Implementation // Initialize service when dynamically linked.
Con guring the Web Server with the Service Con gurator
int HTTP_Acceptor::init (int argc, char *argv[]) { Options::instance ()->parse_args (argc, argv);
% cat ./svc.conf dynamic TP_WWW_Server Service_Object * www_server.dll:make_TP_WWW_Server() "-p $PORT -t $THREADS"
// Set the endpoint into listener mode. Acceptor::open (local_addr); // Initialize the communication endpoint. Reactor::instance ()->register_handler (this, ACCEPT_MASK)
// Terminate service when dynamically unlinked. int HTTP_Acceptor::fini (void) { // Unblock threads in the pool so they will // shutdown correctly. HTTP_Processor::instance ()->close ();
extern "C" Service_Object *make_TP_WWW_Server (void); Service_Object *make_TP_WWW_Server (void) { return new HTTP_Acceptor; // ACE dynamically unlinks and deallocates this object. }
// Wait for all threads to exit. Thread_Manager::instance ()->wait ();
78
77
The OO Architecture of the JAWS Framework
Main Program for Web Server Dynamically con gure and execute the Web Server {
Strategy
Note that this is totally generic!
State Singleton Cached Virtual Filesystem
Asynchronous Completion Token Protocol Handler
// Initialize the daemon and dynamically // configure the service. daemon.open (argc, argv);
Protocol Filter
Tilde Expander
Acceptor
int main (int argc, char *argv[]) { Service_Config daemon;
~ /home/...
Event Dispatcher
Adapter
// Loop forever, running services and handling // reconfigurations.
Protocol Pipeline Framework Streams
daemon.run_event_loop ();
}
Reactor/Proactor I/O Strategy Framework
Strategy
}
Factory function that dynamically allocates a Half-Sync/Half-Async Thread Pool Web Server
Concurrency Strategy Framework Service Configurator
/* NOTREACHED */
79
www.cs.wustl.edu/~jxh/research/ 80
Active Object
}
The concurrent Web Server is con gured and initialized via a con guration script
Applying Patterns to CORBA ORBs
Web Server Optimization Techniques
CLIENT
SERVICE CONFIGURATOR
Use lightweight concurrency
Minimize locking
ABSTRACT FACTORY
Apply le caching and memory mapping
STRATEGY
Use \gather-write" mechanisms
Minimize logging
Pre-compute HTTP responses
Avoid excessive time calls
Optimize the transport interface
CONNECTOR
THREAD-SPECIFIC STORAGE
ACCEPTOR
WRAPPER FACADES OS KERNEL
Bene ts of Design Patterns
ACTIVE OBJECT
REACTOR
Design patterns enable large-scale reuse of software architectures Patterns explicitly capture expert knowledge and design tradeo s Patterns help improve developer communication Patterns help ease the transition to objectoriented technology
83
OS KERNEL
www.cs.wustl.edu/~schmidt/ORB-patterns.ps.gz
81
SERVANT
82
Drawbacks to Design Patterns
Patterns do not lead to direct code reuse
Patterns are deceptively simple
Teams may su er from pattern overload
Patterns are validated by experience and discussion rather than by automated testing Integrating patterns into a software development process is a human-intensive activity 84
Suggestions for Using Patterns E ectively
Do not recast everything as a pattern {
Lessons Learned using OO Frameworks
Instead, develop strategic domain patterns and reuse existing tactical patterns
Institutionalize rewards for developing patterns Directly involve pattern authors with application developers and domain experts
Bene ts of frameworks {
Enable direct reuse of code (cf patterns)
{
Facilitate larger amounts of reuse than standalone functions or individual classes
Drawbacks of frameworks {
High initial learning curve
Many classes, many levels of abstraction
Clearly document when patterns apply and do not apply
{
The ow of control for reactive dispatching is non-intuitive
{
Veri cation and validation of generic components is hard
Manage expectations carefully 85
Patterns and Framework Literature
{ {
Gamma et al., \Design Patterns: Elements of Reusable OO Software" AW, 1994 Pattern Languages of Program Design series by AW, 1995,1997 Siemens, Pattern-Oriented Software Architecture, Wiley, 1996
{
October '96 \Communications of the ACM" (eds: Douglas C. Schmidt, Ralph Johnson, and Mohamed Fayad) October '97 \Communications of the ACM" (eds: Douglas C. Schmidt and Mohamed Fayad)
Magazines {
Special Issues in Journals {
Conferences and Workshops on Patterns
Books {
86
C++ Report and JOOP, columns by Coplien, Vlissides, Vinoski, Schmidt, and Martin 87
Pattern Language of Programs Conferences {
September, 1998, Monticello, Illinois, USA
{
st-www.cs.uiuc.edu/users/patterns/patterns.html
The European Pattern Languages of Programming conference {
July, 1998, Kloster Irsee, Germany
{
www.cs.wustl.edu/~schmidt/patterns.html
USENIX COOTS {
April 27,30, 1998, Santa Fe, New Mexico
{
www.usenix.org/events/coots98/ 88
Obtaining ACE and JAWS
The ADAPTIVE Communication Environment (ACE) is an OO toolkit designed according to key network programming patterns {
All source code for ACE and JAWS is freely available {
JAWS is both a Web server framework and a high-performance Web server
www.cs.wustl.edu/~schmidt/ACE.html
Mailing lists *
[email protected] *
[email protected] *
[email protected] *
[email protected]
Newsgroup {
comp.soft-sys.ace 89
