Download - Enterprise Integration Patterns

6 downloads 298 Views 421KB Size Report
Microsoft's Web Services Extensions (WSE). HOT. HOT ... Channel. Router. Translator. Endpoint. Endpoint. Monitoring. Mes
Enterprise Integration Patterns Asynchronous Messaging Architectures in Practice

Translator Test Message

Splitter

Aggregator Enricher

Gregor Hohpe

Integration Challenges • Users want to execute business functions that span multiple applications • Requires disparate applications to be connected to a common integration solution • However: – – – –

Networks are slow Networks are unreliable No two applications are alike Change is Inevitable

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

Isolated Systems

Unified Access 2

Message-Oriented Middleware Message-oriented architectures provide loose coupling and reliability System A

Channel

System B

Message

¾ Remove location • Channels are separate from dependencies applications • Channels are asynchronous & ¾ Remove temporal dependencies reliable ¾ Remove data format • Data is exchanged in selfdependencies contained messages Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

3

Thinking Asynchronously Order Mgmt Web Site

Shipping

Inventory

New Order

Order Mgmt Web Site

Shipping

Inventory

New Order Confirm New Order Confirm

Idle New Order Confirm Confirm

Synchronous Enterprise Integration Patterns

Asynchronous Copyright © 2003 Gregor Hohpe

4

Many Products & Implementations • Message-oriented middleware (MOM) – IBM WebSphere MQ – Microsoft MSMQ – Java Message Service (JMS) Implementations

• EAI Suites – TIBCO, WebMethods, SeeBeyond, Vitria, ...

• Asynchronous Web services

HOT HOT

– WS-ReliableMessaging, ebMS – Sun’s Java API for XML Messaging (JAXM) – Microsoft’s Web Services Extensions (WSE)

The Underlying Design Principles Are the Same! Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

5

Catalog of 65 Patterns 1. Request-Reply Example

Message Construction Command Message RPC Message Query Message Document Message Event Message Reply Message Return Address Correlation Identifier Message Sequence Message Expiration Canonical Data Model Format Indicator

2. Order Management Example

Message Routing Content-Based Router Message Filter Recipient List Splitter Aggregator Resequencer Distribution w. Aggr. Resp. Auction Routing Table ProcessManager

Message Transformation Data Enricher Content Filter Check Luggage

Endpoint

Endpoint Message

Application A

Messaging Endpoints Messaging Adapter Polling Consumer Event-Driven Consumer Transactional Client Competing Consumers Message Dispatcher Message Selector Idempotent Receiver Messaging Mapper

Enterprise Integration Patterns

Channel

Router

Translator

Application B

Messaging Channels Point-to-Point Channel Publish-Subcr. Channel Durable Subscriber Datatype Channel Invalid Message Channel Dead Letter Channel Guaranteed Messaging Channel Adapter

Monitoring

Copyright © 2003 Gregor Hohpe

Systems Management

3. Bonus

Control Bus Message Header Envelope Wrapper Message History Message Store Channel Purger Test Message

6

Pattern: Request-Reply Consumer

Request

Provider

Request Channel

Reply Channel

• • • •

Reply

Service Provider and Consumer (similar to RPC) Channels are unidirectional Two asynchronous Point-To-Point Channels Separate request and reply messages

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

7

Multiple Consumers Requests

Requests

Provider

Request Channel

Consumer Consumer 11 Consumer Consumer 22

Reply Channel 1 Reply Channel 2

?? Replies

• Each consumer has its own reply queue • How does the provider know where to send the reply? – Could send to all consumers Æ very inefficient – Hard code Æ violates principle of context-free service

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

8

Pattern: Return Address Reply Channel 1

Reply Channel 2

Provider

Request Channel

Consumer Consumer 11 Consumer Consumer 22

Reply Channel 1 Reply Channel 2

Replies

• Consumer specifies Return Address (reply channel) in the request message • Service provider sends reply message to specified channel

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

9

Multiple Service Providers Provider Provider 11 Request Channel

Consumer Consumer

Provider Provider 22

Reply Channel

• Request message can be consumed by more than one service provider • Point-to-Point Channel supports Competing Consumers, only one service receives each request message • Channel queues up pending requests Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

10

Multiple Service Providers Consumer

Service 1 (slow)

Request 1 Request 2 Reply 2

Reply 1

Enterprise Integration Patterns

Service 2 (fast)

• Reply messages get out of sequence • How to match request and reply messages? – Only send one request at a time Æ very inefficient – Rely on natural order Æ bad assumption

Copyright © 2003 Gregor Hohpe

11

Pattern: Correlation Identifier Consumer 1

2

1

2

Message Identifier Request Channel

Correlate Request & Reply

2

1

2

1

1

Provider Provider 11

2

2

Provider Provider 22 1

Response Channel

Correlation Identifier

• Equip each message with a unique identifier – Message ID (simple, but has limitations) – GUID (Globally Unique ID) – Business key (e.g. Order ID)

• Provider copies the ID to the reply message • Consumer can match request and response Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

12

Pattern: Pipes-And-Filters Pipe

Pipe Decrypt Decrypt

Incoming Order

Filter

Pipe Authenticate Authenticate

Filter

Pipe De-Dup De-Dup

Filter

‘Clean’ Order

• Connect individual processing steps (filters) with message channels (pipes) – Pipes decouple sender and receiver – Participants are unaware of intermediaries – Compose patterns into larger solutions

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

13

Multiple Specialized Providers Order Messages

Order Order Entry Entry

??

Widget Widget Inv. Inv. Gadget Gadget Inv. Inv.

• Each provider can only handle specific type of message • Route request to the “appropriate” provider based on the content of the request message – Do not want to burden sender with decision (decoupling) – Letting each consumer “pick out” desired messages requires distributed coordination Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

14

Pattern: Content-Based Router Order Messages

Widget Widget Inv. Inv.

Order Order Entry Entry ContentBased Router

Gadget Gadget Inv. Inv.

• Insert a Content-Based Router • Message routers forward incoming messages to different output channels • Message content not changed • Mostly stateless, but can be stateful (e.g. de-duper) Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

15

Composite Message Order Message

Order Order Entry Entry

??

Widget Widget Inv. Inv. Gadget Gadget Inv. Inv.

• How can we process a message if it contains multiple elements, each of which may have to be processed in a different way? – Treat each element independently – Need to avoid missing or duplicate elements – Make efficient use of network resources Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

16

Pattern: Splitter Order Message

Widget Widget Inv. Inv.

Order Order Entry Entry

?? Splitter

Order Order Item 1 Item 2

Gadget Gadget Inv. Inv.

• Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item.

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

17

Composite: Splitter & Router Order Item 1

Order Message

Widget Widget Inv. Inv.

Order Order Entry Entry Splitter

Order Order Item 1 Item 2

Gadget Gadget Inv. Inv.

Router Order Item 2

• Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item. • Then use a Content-Based Router to route the individual messages to the proper destination Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

18

Producing a Single Response Order Item 1

Response 1

Confirmed Order

Widget Widget Inv. Inv.

??

Gadget Gadget Inv. Inv. Order Item 2

Billing Billing

Response 2

• How to combine the results of individual, but related messages so that they can be processed as a whole? – – – –

Messages out of order Message delayed Which messages are related? Avoid separate channel for each system

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

19

Pattern: Aggregator Order Item 1

Response 1

Confirmed Order

Widget Widget Inv. Inv.

Billing Billing Gadget Gadget Inv. Inv. Order Item 2

Aggregator Response 2

• Use a stateful filter, an Aggregator, to collect and store individual messages until a complete set of related messages has been received. – Aggregator publishes a single message distilled from the individual messages. – Correlation – Completeness Condition – Aggregation Algorithm Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

20

Pattern: Scatter-Gather • Send a message to a dynamic set of recipients, and return a single message that incorporates the responses. Scatter-Gather Pub-Sub Channel

Quote Request

“Best” Quote

Enterprise Integration Patterns

Vendor Vendor A A

Quote

Vendor Vendor B B Vendor Vendor C C

Aggregator

Copyright © 2003 Gregor Hohpe

21

Composing Patterns • Receive an order, get best offer for each item from vendors, combine into validated order. Scatter-Gather Pub-Sub Channel

New Order

Splitter

Validated Order

Aggregator

Enterprise Integration Patterns

Quote Request for each item

“Best” Quote for each item

Vendor Vendor A A

Quote

Vendor Vendor B B Vendor Vendor C C

Aggregator

Copyright © 2003 Gregor Hohpe

22

System Management • Messaging systems are asynchronous and distributed – Multiple platforms – Difficult to detect errors – Difficult to configure (property file hell)

• How can we effectively administer a messaging system that is distributed across multiple platforms and a wide geographic area?

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

23

Pattern: Control Bus Application Message Flow

Control Bus Management Console

• • • • • •

Configuration Heartbeat Test messages Exceptions / logging Statistics / Quality-of-Service (QoS) Live console

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

24

How To Inspect Messages? Point-To-Point Channel

Sender Sender

Receiver Receiver Message

?? Management Console

• Cannot add another receiver because it would consume the message • Cannot switch to Publish-Subscribe-Channel because Point-To-Point Channel may already have Competing Consumers

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

25

Pattern: Wire Tap Wire Tap

Sender Sender

Receiver Receiver Message

Message Management Console

• Simple Router that duplicates message to two output channels • Also known as Tee • Some side effects: Message ID changes, latency

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

26

Track Messages Wire Tap

Wire Tap

Component Component

Analysis

• E.g., message run time, message volume • Missed messages if channels or component unreliable

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

27

What if Return Address is Used? Wire Tap

Consumer Consumer Consumer Consumer

Request Channel

Provider

Reply Channels

??

• Provider routes reply message to dynamic channel • Cannot dynamically inject Wire Tap

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

28

Pattern: Smart Proxy Smart Proxy

Consumer Consumer

Request Channel

Provider

Fixed Reply Channel

Consumer Consumer Reply Channels

• A Smart Proxy stores original Return Address and replaces it with a fixed channel address • Intercepts reply messages and forwards them to correct channel • Allows analysis of request and reply messages Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

29

Pattern: Test Message Test Message Injector

Test Message Separator Processor Processor

Appl. Appl. Msg. 1 Msg. 2

Test Message

9 Test Data Generator

Appl. Msg. 1

Test Result

Appl. Msg. 2

Test Data Verifier

Control Bus

Management Console

• Inject application specific test messages • Extract result from regular message flow • Compare result against predefined (or computed) result Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

30

In Summary… • Visual and verbal language to describe integration solutions • Combine patterns to describe larger solutions • No fancy tools – whiteboard or PowerPoint • No vendor jargon • Not a precise specification language – (e.g., see OMG UML Profile for EAI)

• Not a new “methodology” • Each pattern describes trade-offs and considerations not included in this overview Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

31

Resources • Book (late October): – Enterprise Integration Patterns – Addison-Wesley, 0-321-20068-3

• Contact – Gregor Hohpe – [email protected]

• Web Site – – – –

http://www.eaipatterns.com Pattern catalog Bibliography, related papers [email protected]

• www.thoughtworks.com

Enterprise Integration Patterns

Copyright © 2003 Gregor Hohpe

32