Building a Configurable Publish/Subscribe Notification Service by
Cristian Gabriel Fiorentino Submitted to
Departamento de Computaci´on y Sistemas Facultad de Ciencias Exactas in partial fulfillment of the requirement for the degree of
Systems Engineer at
Universidad Nacional del Centro de la Provincia de Buenos Aires
Advisor:
Mariano Cilia
Date:
May, 2006
Tandil, Argentina 2006
Abstract The convergence of technologies and information-driven applications require a middleware that support data streams. This middleware needs to interpret, aggregate, filter and analyze streams of messages usually in a distributed environment to adequately make messages flush from any origin to interested destinations. Publish/Subscribe (pub/sub) middleware basically deals with these issues decoupling producers and consumers of data. But existent pub/sub solutions are monolithic and only include a subset of the required features. Moreover their functionalities are in some cases impossible or difficult to extend. The problem arises when users want to find a middleware that completely fulfills their application requirements. In this work we propose Franscesca: a framework that allows the configuration/adaptation of a pub/sub solution based on a reusable and easily extensible set of components.
iii
iv
Acknowledgements First of all I want to thank my Advisor Prof. Mariano Cilia. He has not only exceptionally introduced and guided me into technological and academic issues, but he is the kind of person that knows how to give lessons of work, improvement, simplicity, and effort. I am not only so glad to have met so a surprisingly professional, but I feel I have found a friend. To him, to whom I think is a great example to follow. Second, I want to thank the Databases and Distributed Systems Group (DVS) from the Technische Universitt Darmstadt (TUD), Germany, and specially to Prof. Alex Buchmann, who allowed me to visit Germany and his research group. There I had the opportunity to begin this work being in touch with Ph.D. students who also helped and introduced me on related topic, and that also made my stay in Darmstadt one of the best experiences of my life. Special thanks go to Ludger Fiege, for his unconditional support. Thanks to Universidad Nacional del Centro de la Provincia de Buenos Aires (UNICEN), specially to the Faculty of Sciences and to its professors who have formed me in my career. Also I want to thank Tandil, the city where I have studied, and to all my partners and friends I did during my stay there. Special thanks to Pablo Guerrero and to Jose and Mario Antollini. I want to thank all my friends in Mar del Plata, the city that saw me born. To Matias and Emanuel Martinez and David Ochoa, friends of life. Special thanks to my sister Romina and my grandmother Lola, the ones that are always when I need them. With deep feelings I want to give very special thanks to the friend of my heart Ana Maria Angel Mafla. The best thanks that comes from my soul are to my parents Mario and Lydia, the persons I love most in life. To whom I will be forever in gratitude for their admirable effort, that made it possible to overcome my studies; for their incomparable and noninterchangeable example of life they give me; and for their unconditional help and tireless guiding words to encourage me to reach my dreams. For them I dedicate all this, and all that from this starting point I do reach to attain. Thanks to God, for all the things he has given me and to my people, to whom I still ask to give me forces to continue loving life and enjoying my work. Tandil, May 2006 Cristian Gabriel Fiorentino
v
vi
Contents 1 Introduction 1.1 New Technology Trends and the Request/Replay Paradigm 1.2 Event-based Paradigm . . . . . . . . . . . . . . . . . . . . . 1.3 Notification Services . . . . . . . . . . . . . . . . . . . . . . 1.4 Motivation: Why a Configurable NS? . . . . . . . . . . . . . 1.5 Proposal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Organization of this work . . . . . . . . . . . . . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
1 1 2 2 3 4 5 6
2 Background & Related Work 2.1 Overview . . . . . . . . . . . . . 2.2 Notification Services . . . . . . . 2.2.1 Rebeca [1] . . . . . . . . . 2.2.2 Scribe [2] . . . . . . . . . 2.3 Peer-to-peer (P2P) Technologies 2.3.1 Pastry [3] . . . . . . . . . 2.3.2 Bamboo [4] . . . . . . . . 2.3.3 Bit Zipper Rendezvous [5] 2.3.4 Spanning-tree Overlay [6] 2.3.5 Common API [7] . . . . . 2.4 Component-based Infrastructures
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
7 7 8 8 9 9 9 11 11 12 13 13
Service Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
15 15 15 16 16
. . . . . . . . . .
19 19 20 21 22 22 23 23 23 23 24
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
3 Our Approach: A Pub/Sub Notification 3.1 System Requirements . . . . . . . . . . 3.1.1 Functional Requirements . . . . 3.1.2 Non-Functional Requirements . . 3.2 Proposed Approach . . . . . . . . . . . . 4 Framework Architecture 4.1 Main Architecture . . . . . 4.2 Components’ View . . . . . 4.2.1 Network Layer . . . 4.2.2 Topology Layer . . . 4.2.3 Routing Layer . . . 4.2.4 Broker Layer . . . . 4.2.5 Application Layer . 4.3 Data/Control Flow View . . 4.3.1 Pipelined Approach 4.3.2 Services . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . vii
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
viii
CONTENTS
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
25 32 32 34
5 Configurations & Management 5.1 Configuration Capabilities . . . . . . . . . . . 5.1.1 Layer Selection . . . . . . . . . . . . . 5.1.2 Component Type . . . . . . . . . . . . 5.1.3 Functionality Selection . . . . . . . . . 5.1.4 Component Settings . . . . . . . . . . 5.2 Configuration Verification . . . . . . . . . . . 5.2.1 Layering Verifications . . . . . . . . . 5.2.2 Component Type Verification . . . . . 5.2.3 Functionality Architecture Consistence 5.2.4 Component Settings . . . . . . . . . . 5.3 Components Descriptions . . . . . . . . . . . 5.4 Management Design . . . . . . . . . . . . . . 5.4.1 Management Design Details . . . . . . 5.4.2 Configuring Verification Design . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
. . . . . . . . . . . . . .
37 37 37 38 38 38 39 39 39 40 40 40 41 42 43
6 Deployment & Runtime 6.1 Deployment . . . . . . . . . . . . . . . . . . . 6.1.1 Container-based Deployment Options 6.1.2 Other Deployment Decisions . . . . . 6.2 Runtime Environment . . . . . . . . . . . . . 6.2.1 OSGi Platform . . . . . . . . . . . . . 6.2.2 Using OSGi . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
45 45 45 46 46 47 47
7 Detailed Design 7.1 Subsystems Design . . . . . . . . . . . . . . . 7.1.1 Framework Subsystem . . . . . . . . . 7.1.2 Layers Subsystem . . . . . . . . . . . 7.1.3 Management Subsystem . . . . . . . . 7.2 Class Diagrams . . . . . . . . . . . . . . . . . 7.2.1 Components . . . . . . . . . . . . . . 7.2.2 Events . . . . . . . . . . . . . . . . . . 7.2.3 Messages . . . . . . . . . . . . . . . . 7.2.4 Management . . . . . . . . . . . . . . 7.3 Interaction Diagrams . . . . . . . . . . . . . . 7.3.1 Event Composition and Transmission 7.3.2 Components Wiring . . . . . . . . . . 7.3.3 Event Creation and Dissemination . . 7.3.4 Events Handling . . . . . . . . . . . . 7.3.5 Components Configuration . . . . . . 7.4 Specific Components Creation . . . . . . . . . 7.4.1 Define component type . . . . . . . . 7.4.2 Include Configurable Strategies . . . . 7.4.3 Identify Data Agreements . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . .
49 49 49 51 51 51 51 54 57 60 61 61 64 65 67 68 68 69 69 69
4.4 4.5
4.3.3 Interfaces . . . . . . . . . Deployment View . . . . . . . . . 4.4.1 Local and Remote Clients Processes View . . . . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
CONTENTS
ix
7.4.4 7.4.5 7.4.6
Implement Main Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Specify Component Configurations and Subscriptions . . . . . . . . . . . . . 72 Create Component Bundle . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
8 Case Study: Using Franscesca Encapsulating Rebeca 8.1 Rebeca Encapsulation . . . . . . . . . . . . . . . . . . . . . 8.1.1 Rebeca Design . . . . . . . . . . . . . . . . . . . . . 8.1.2 Encapsulation Decisions . . . . . . . . . . . . . . . . 8.1.3 Encapsulation Implementation . . . . . . . . . . . . 8.2 Framework Usage & Configuration . . . . . . . . . . . . . . 8.2.1 NS-Creation User Interface . . . . . . . . . . . . . . 8.2.2 Life Cycle Management Interface (OSGi GUI Shell) 8.2.3 NS Configuring User Interface . . . . . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
75 75 75 77 77 82 83 84 85
9 Conclusions & Future Work 87 9.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 9.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 A Interface Verification A.1 Routing Layer . . . A.1.1 Merging . . A.1.2 Bit Zipper . A.2 Topology Layer . . A.2.1 Static Tree A.2.2 DHT . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
91 91 91 92 93 93 93
B Container Selection 97 B.1 Containers: A Brief Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 B.2 Containers Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
x
CONTENTS
List of Figures 1.1 1.2 1.3
Notification Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . NS Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1
P2P Topology Network Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1
Our Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.1 4.2 4.3 4.4 4.5 4.6 4.7
Architecture . . . . . . . . Layers and components . Interfaces . . . . . . . . . Global Deployment View Deployment View . . . . . Remote Application . . . Processes View . . . . . .
5.1
Management Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 7.11 7.12
Packages . . . . . . . . . . . . . . . Component classes . . . . . . . . . Event classes . . . . . . . . . . . . Messages . . . . . . . . . . . . . . Management . . . . . . . . . . . . Layering Model . . . . . . . . . . . Pipeline Model . . . . . . . . . . . Components Wiring . . . . . . . . Event Creation and Dissemination Events Handling . . . . . . . . . . Components Configuration . . . . Components Implementation . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
. . . . . . . . . . . .
50 52 55 58 60 62 63 64 66 67 68 71
8.1 8.2 8.3 8.4 8.5 8.6
Rebeca Design . . . . . . . . . . Rebeca Code Classification . . . Encapsulations . . . . . . . . . . NS-Creation Interface . . . . . . Life Cycle Management Interface NS-Configuration Interface . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
76 78 79 83 84 86
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
xi
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
3 4 5
20 21 25 32 33 34 34
xii
LIST OF FIGURES
List of Tables 4.1 4.2 4.3
Layers Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Upwards Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Downwards Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.1 5.2 5.3 5.4
Network Components Topology Components Routing Components . Broker Components .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
40 41 41 41
7.1 7.2 7.3 7.4 7.5
Application Agreements Broker Broker . . . . . . Routing Agreements . . Topology Agreements . Network Agreements . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
70 70 70 70 70
A.1 Routing Interface Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 A.2 Topology Interface Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 B.1 Container Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
xiii
xiv
LIST OF TABLES
Listings 7.1 7.2 7.3 8.1 8.2 8.3 8.4 8.5
Bundle Activator . . . . . Manifest . . . . . . . . . . Metadata.xml . . . . . . . Application Configuration Broker Configuration . . . Routing Configuration . . Topology Configuration . Network Configuration . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
xv
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
73 73 74 79 80 81 82 82
xvi
LISTINGS
Chapter 1
Introduction To adequately introduce the matters of this work it is essential to primarily give the foundations of Notification Services (NS). Based on this, the state of art of this kind of middleware research and an explanation of modern application requirements, we will go consecutively deeper in detail to reach to understand the problem domain this work deals with. To do so, first an explanation of new technology trends and how they have been mistakenly based on the request/replay paradigm is presented. Later, the Event-based paradigm is explained as a better approach that actually fits with nowadays technology convergence and modern application requirements. Later, Notifications Services are defined as a solution that offers an implementation of this paradigm for distributed applications. Having understood all this, the problematic on using existent NS’s that this work refers to is presented (Motivation). Finally, the basic solution approach is sketched, to be explained in detail in the overall course of this work.
1.1
New Technology Trends and the Request/Replay Paradigm
Since more than a decade that commercial applications do apply the Client/Server architectural style. They do this by basically relying on software infrastructures (middleware) that rely on the request/reply paradigm. It must be observed that under this paradigm clients are responsible for initiating the interaction while servers (in a passive role) just answer to those requests. But this paradigm seems to be inadequate to fit the requirements of a new technology trend. For instance, from the technology point of view, nowadays there is a convergence of technologies, that go from the sensor miniaturization, intelligent environment adaptation systems, and the initiative of autonomous systems that are able to detect and correct failures or anomalies. Under the business trends we can cite the event-driven supply chain management, or the trends in business areas like zero-latency enterprises, real-time enterprises, just to name a few. This technology convergence shows that the needed software infrastructure must be adapted to new requirements that impose these new applications. They are basically characterized by monitoring/observing situations of interest and then reacting properly. The adequate use of such a paradigm has an important impact on the utilization of resources. This can be clearly seen if we analyze a simple example based on the traditional request/replay paradigm. Lets imagine a scenario of e-commerce with the interaction of two enterprises. In this case, application A is interesting in some situations related with the execution of application B. Traditionally, application A would issue requests to application B to determine if the situation of 1
2
CHAPTER 1. INTRODUCTION
interest has happen. Given that application A is decoupled from application B, then it will have to repeatedly send requests till to observe the change of state of interest. The frequency with which that request is sent has an important impact in the resource use, because if it is made very frequently, CPU cycles and network utilization are wasted unnecessary. On the other hand, if the frequency is too low, we could miss the observation of the specific situation. It is evident that the use of the request/reply paradigm is not appropriate for this kind of applications, ending up with an over-dimension of resources. Other well known examples, such as Internet auctions (like ebay), newspapers and news agencies follow the same pattern. The reason of its use is due to that the users computers have a pre-installed web browser that only is able to work based to the request/reply paradigm. In other words, it is at the client-side where the conversation is initiated, either explicitly by the user or by a refresh/reload command. This leads us to an obvious inefficient use of resources.
1.2
Event-based Paradigm
Intuitively, the problematic presented above could be solved by making application B to send a message to application A when the situation of interest is detected. This has various implications. The first is that application B must be modified to consider the situations of interest of application A (i.e. implementing the observer pattern [8, 9]). This obviously restricts the interaction to only these two applications, then the incorporation of new ones would imply code modification for application B, obtaining then highly coupled applications. The use of this pattern can work in the context of one application, but not in the case of the application interaction in a distributed system. Moreover, the expressiveness about the interests would be represented and embedded into the programming code of B, with the consequent complications in case of a change of interests of application A. In the event-based paradigm, the producer of data is responsible to initiate the interaction when a situation of interest is observed (i.e., new offers/bids in case of online auctions or news in case of online newspapers). Also in this paradigm, the incorporation of new applications do not imply code modification for another applications. This is due that new application requirements are expressed dynamically to the overall system. In this way, refereed new technologies trends behavior perfectly fits the event-driven paradigm. Consequently, the traditionally used middleware must be extended to also support this paradigm. But doing so, the underlying infrastructure needs to deal with event/data dissemination, from the places where they are produced to the destinations where interested consumers are located. From an abstract point of view, the middleware must allow the flow of data/events offering also adequate primitives to be able to be interpreted, filtered, combined and analyzed.
1.3
Notification Services
In distributed systems, the responsibility of data dissemination is delegated to the middleware, more specifically to a Notification Service (NS). A particular case of this middleware, but with growing popularity nowadays, are the publish/subscribe notification services. Here the interaction among participants is initiated by the observer of state changes (also known as event/data producer or publisher, or server in client/server terms). Consumers (also known as subscribers) explicit their interests expressed in a subscription language. The publish/subscribe service plays the role of a
1.4. MOTIVATION: WHY A CONFIGURABLE NS?
3
mediator, and is then the responsible of making information flow from the producer to one o more interested consumers. In this context, a NS offers asynchronous communication, which decouples naturally producers and consumers, making them anonymous, and allowing a dynamic number of participants. The NS obviously is not restricted to a single computer, and its tasks can/must be distributed to various computers, playing the role known as broker. Figure 1.1 shows a simple pub/sub NS middleware implemented as an interconnected set of brokers. The figure also shows a common NS interaction among participants. At any time some applications can subscribe to certain elements of data (playing the role of subscribers), while others applications publish some data (known as publishers). If published data matches with any subscription, then the NS is in charge of notifying interested subscribers about this data.
Figure 1.1: Notification Service The pub/sub infrastructure then needs to deal with the dissemination of data in a distributed environment from where it is produced to the destinations where interested consumers are located. To do so, streaming data needs to be interpreted, aggregated, filtered, analyzed, reacted to and eventually disposed of.
1.4
Motivation: Why a Configurable NS?
Today, there are various of such commercial products: WebSphereMQ [10] (formerly MQ-Series), TIB-Rendezvous [11], and various JMS [12, 13] compliant implementations (Sun [14], JBoss [15], OpenJMS [16], ActiveMQ [17]) that deal with these issues. In the academia there are several projects (see Chapter 2 for references) that focus on one or the other issue of interest, like efficient dissemination, addressing models, message correlation, mobility, scalability, fault-tolerance, access control, data integration, security, privacy protection, transactions, caching, etc. But unfortunately, application requirements that need a publish/subscribe infrastructure not always coincide with the functionalities/characteristics that existent solutions offer. Specific requirements of a given application may need different combination of NS features. Figure 1.2 shows some examples of the most important required features (or configuration parameters) that NS’s might support. Still, we must think that for many of them there exist various alternatives. For instance, addressing models include five classifications, fault tolerance can be achieved in many ways (replication, different topologies, etc.), and there exists many network protocols. This leads to a situation where some requirements cannot be fulfilled by the infrastructure. Given that existent infrastructures are proprietary and not extensible, not covered features cannot
4
CHAPTER 1. INTRODUCTION
be included into products to fulfill applications needs. Since these infrastructures are also monolithic solutions, they also include functionalities that are not necessary to specific applications, and they are also impossible to extract.
Management Monitoring
Addressing Model
Global-time Aware
!
!
!
!
Security
Application Config.
NS
!
!!
Extensible Funtionality Deployment Patforms
Transactions
!
FaultTolerant
!
Performance
!
Adaptable Brokers
Routing Algorithm
Reliability
!
!
Catching
Scalable
!
!
Data Integration
Network Protocol
!
NS Size
!
!
!
!
Figure 1.2: NS Requirements As can be seen many combinations are possible. To give some real world ones we can enumerate: a news ticker application basically requires topic-oriented addressing, efficient data dissemination and scalability, but may not include transaction support. On the other side, supply-chain management based on the AutoID Infrastructure requires content-based addressing, transactions, security, fault-tolerance, data integration and scalability while other aspects like access control or caching are optional. As an additional problem to solve is that not all combinations are able to achieve, technologically impossible, or they are difficult to generate. For instance some routing algorithms only work on certain kind of topologies; the desire of monitoring capabilities fault-tolerance and security and the requirement of a lightweight NS (to be deployed in small machines); and other well known tradeoffs like the one between performance and scalability. Today there is no product nor prototype that can offer all desired features and that they can be adapted to application requirements. Moreover, these products are closed enough so that you can not extend with the absent features to fulfill application requirements.
1.5
Proposal
This work intends to give a solution to the problem presented above. One utopia would be to have a NS for each set of requirements. Although this looks like impossible, if we could have this catalog of NS’s, still this would not be enough. Applications change over time: aspects like scalability, the emerging technologies, and still changing application requirements will need middleware to be readapted to correctly fulfill the new necessities. Due to the all the previous arguments, we can notice that a solution to this problem is Configurability. If we could create a Framework for creating and re-configuring Notifications Services
1.6. TERMINOLOGY
5
we would be on the right track. A framework like this must offer: an organized separation of NS functionality, the unmistakable places where these functionalities must be situated, abstract interfaces between these functionalities, and a global organization to manage all these. Joined with this, an extensible catalog of building blocks separated by different functionality is needed. In such an approach, as sketched in Figure 1.3, a NS instance can be created by just selecting the desired functionalities and placing them in predefined framework points, to be automatically coordinated by an organization component.
Provided by Framework User Defined
Catalogue “B”
NS instance
Catalogue “C”
Catalogue “D”
A1
B1
C1
D1
A2
B2
C2
D2
A3
B3
C3
D3
C1
D2
NS Framework
Building blocks
Catalogue “A”
Organization
A2
B3
Organization
Figure 1.3: Framework Based on experience on building notification services from the DVS research group1 [1, 6, 5] specific features from them [18, 19, 20, 21, 22], and the experience of others [4, 3, 23, 24, 2, 25, 26, 27]; in this work a framework for building and re-configuring pub/sub notification services is developed. It allows the combination of required features of a notification service and as a result the middleware is “shaped” according to the corresponding application requirements. As an additional result, it offers a testbed where different ideas can be experimented and proven together with other approaches.
1.6
Terminology
In this work the terms Publish, Publication, Publisher, Subscribe, Subscription, Subscriber and Notification, Notifier are used in the context of the publish/subscribe event-based paradigm. The term Event indicates framework flow of data/control. These events are treated by event handlers to be mapped to defined component interfaces. Also the term Event is used to refer to application specific occurrences (data transmitted in publications). The term message is applied to the main units of data used in the framework. In this way we can say that framework events transmit messages. Message or messaging can also be used in the context of MOM (Message Oriented Middleware), a superclass of the NS middleware. Although 1
Technische Universitt Darmstadt (TUD) https://www.dvs1.informatik.tu-darmstadt.de/
6
CHAPTER 1. INTRODUCTION
this usage is mostly omitted in this work for the confusing overlapping among the terms message and event. The term interface is used in the context of the layering architectural style expressing local specific calls to consecutive layers. The term protocol is used in this same context indicating the messaging interaction between peer remote layers.
1.7
Organization of this work
In Chapter 2 we present the fundamentals of pub/sub notification services including an overview of related projects, and we also include related work on service-based infrastructures. Chapter 3 enumerates system requirements and sketches our proposal. In Chapter 4 we present our main design decisions including architecture, interfaces, and pre-defined components. Chapter 5 presents framework configuration and management capabilities and design. Chapter 6 describes deployment strategies together with the use of runtime environments. Chapter 7 sketches the low level framework design. On Chapter 8 the framework is used and validated encapsulating the Rebeca NS. Finally in Chapter 9 we present conclusions and future work.
Chapter 2
Background & Related Work We may synthetically define a Notification Service as a middleware that gives an implementation for using the event/based paradigm. In a notification service there are different participants: applications, that basically produce and/or consume messages, and brokers, that help to disseminate messages across the network between producers and consumers. Each broker maintains a routing table that determines in which directions a message needs to be forwarded. This table needs to be maintained up-to-date with respect to active consumers and their subscriptions. Within brokers we basically distinguish in three groups: local brokers that constitute the clients’ access point to the middleware and they are usually part of the communication library loaded into the clients. A local broker is connected to at most one border broker. Border brokers form the boundary of the distributed communication middleware and maintain connections to local brokers. Inner brokers are connected to other inner or border brokers and do not maintain any connections to the applications. There are different features of pub/sub notification services that have been and are being investigated in the academia as for example, efficient dissemination [28, 29, 1, 30, 5, 6], message addressing models [22, 21, 31, 30], message correlativity [32], mobility support [18, 19], various aspect of security [20], message persistence [1], fault tolerance [5, 6], access control [20], data integration [33], privacy protection [20], transaction [1], caching [1], etc. But it is important to notice that either commercial products and research prototypes just implement a reduced subset of these functionalities. As the main goal of this work is to create a framework for constructing NS’s, here we must go in some detail with various of its features. To do so, first an overview of related projects and researches is given. Later, we go deeper in some specific Notification Services projects, some Peerto-peer Technologies and an overview of Component-oriented Infrastructures is given. All of them necessary to understand for later use in chapters.
2.1
Overview
In recent years, academia and industry have concentrated on publish/ subscribe mechanisms because they offer loosely coupled exchange of asynchronous notifications, facilitating extensibility and flexibility. These approaches have evolved from restricted channels to more flexible subscription mechanisms. For instance, subject-based addressing [34] define an uniform name space for messages and their destinations. Here, to every message a subject is attached in order to find matchings with subscriptions, also expressed with subjects. 7
8
CHAPTER 2. BACKGROUND & RELATED WORK
To improve expressiveness of the subscription model the content-based approach was proposed where predicates on the content of a notification can be used for subscriptions. This approach is more flexible but requires a more complex infrastructure [28]. Many projects in this category (like Rebeca [1], Siena [25], JEDI [26], Gryphon [35, 27]) concentrate on scalability issues in wide-area networks and on efficient algorithms and techniques for matching and routing notifications to reduce network traffic [30, 29, 32]. Most of these approaches use simple Boolean expressions as subscription patterns and assume homogeneous name spaces. Still more the Concept-based addressing model [33] has been created to extend content-based filtering to heterogeneous environments. In this approach not only the content of the message matters to express subscriptions but also its semantics. More recently a new generation of publish/subscribe systems built on top of an overlay network has emerged. They mostly pursue wide-area scalability based on a topic-oriented addressing model. This is the case of Scribe [2] which is implemented on top of Pastry [3]. The mapping of topics onto multicast groups is done by simply hashing the topic name. Hermes [24] uses a similar approach but tries to get around the limitations of topic-based publish/subscribe by implementing a so-called “type and attribute based” publish/subscribe model. It extends the expressiveness of subscriptions and aims to allow multiple inheritance in event types. FioranoMQ [36] is a grid-enabled, peer-topeer JMS messaging platform, but offering only Topic-based addressing. A pair of content-based addressing on top of a dynamic peer-to-peer networks have been proposed: Bit Zipper Rendezvous [5] and the Spanning-tree Overlay proposed in [6] where the efficient routing of notifications takes advantage of the topology graph underneath. These works combines the high expressiveness of content-based subscriptions and the scalability and fault tolerance of a peer-to-peer system (detail on them are given in Section 2.3). Most of the projects mentioned above are monolithic and they provide a static set of features. Few of them are in some sense extensible while the extensions are hard to develop. What is needed is the possibility to easily combine features in order to completely fulfill application requirements. This can be achieved if pub/sub notification services are built based on extensible set of components that rely on an appropriate architecture/infrastructure.
2.2 2.2.1
Notification Services Rebeca [1]
Rebeca (recursive acronym for Rebeca Event-Based Electronic Commerce Architecture) is a NS framework created to test content-based routing algorithms and performance. It offers a flexible filter mechanism that is used to implement subscriptions and matching algorithms. It also uses advertisements, a mechanism that allows to optimize routing tables, allowing brokers to know what kind of events are going to be published. Its topology is a static tree of nodes, which was selected for simplicity to avoid transition replication. It do not offer failure detection nor recovery mechanisms, only allowing the network to be statically generated to later be efficiently used by its interesting content-based routing algorithms. New functionalities were introduced over time: transactional events, concept-based addressing, caching, mobility and a JMS-like [12, 13] interface. Now we go in a little more detail to some Rebeca’s routing algorithms, because they will be named and used in later chapters. • Flooding: In this algorithm, the routing table of each broker is initialized to a constant set. Hence, a broker will forward a notification received from a local client to all neighbors and a notification received from a peer broker to all other neighbors. It implies that every
2.3. PEER-TO-PEER (P2P) TECHNOLOGIES
9
broker processes any published notification. Flooding is the only routing strategy that does not require the remote routing configuration to be updated. Therefore, no admin messages are exchanged. • Simple Routing: This routing strategy uses filter forwarding to update the routing configuration in reaction to subscribing and un-subscribing clients: new and cancelled subscriptions are simply flooded into the broker network such that they reach every broker. In the next algorithms similarities between filters are used to optimize the transmission. • Identity-Based Routing: Here the main idea is that a subscription is not forwarded to a neighbor if an identical subscription that has not been cancelled was forwarded to that neighbor. An un-subscription is not forwarded to a neighbor if there is another subscription of a local client or another neighbor that is identical to the former. • Covering-Based Routing: The basic idea of covering is the following: a subscription is not forwarded to a neighbor if a subscription that covers the former was forwarded to that neighbor that has not been cancelled. If a subscription is forwarded, the receiving broker deletes all routing entries whose filters are covered by the new subscription (and that refer to the same destination) in order to get rid of the obsolete routing entries. • Merging-Based Routing: This algorithm can be implemented on top of covering-based routing. Merging-based routing does not merely rely on the filters which have been issued by the clients. Instead, if a merging-based routing algorithm is applied, a broker can merge the filters of existing routing entries and forward this merger to a subset of its neighbors.
2.2.2
Scribe [2]
Scribe is a publish/subscribe NS, that organizes data placement for improving routing strategies. It is constructed over a P2P network to inherit all its routing advantages, but it is limited to work only as a topic-based dissemination. As previously mentioned, this dissemination method offers little expressivity. It is build on top of Pastry, the chosen P2P overlay network. The way it work is generating a multicast tree, joining Pastrys routing to a rendezvous associated with every topic.
2.3 2.3.1
Peer-to-peer (P2P) Technologies Pastry [3]
Pastry is a scalable, distributed object location and routing substrate for wide-area peer-to-peer applications. Pastry performs application-level routing and object location in a potentially very large overlay network of nodes connected via the Internet. Each node in the Pastry network has an unique identifier (nodeId). When presented with a message and a key, a Pastry node efficiently routes the message to the node with a nodeId that is numerically closest to the key, among all currently live Pastry nodes. The expected number of routing steps is O(log N), where N is the number of Pastry nodes in the network. Each Pastry node keeps track of its immediate neighbors in the nodeId space. Pastry takes into account network locality; it seeks to minimize the distance messages travel, according to a scalar proximity metric like the number of IP routing hops. Pastry is completely decentralized,
10
CHAPTER 2. BACKGROUND & RELATED WORK
scalable, and self-organizing; it automatically adapts to the arrival, departure and failure of nodes, and notifies applications of new node arrivals, node failures and recoveries. The nodeId is used to indicate a nodes position in a circular nodeId (128-bit node identifier). In each routing step, a message is forwarded to a relatively closer node with a nodeId that shares a longer common prefix or is numerically closer to the key than the local node. Since only local information is used, Pastry minimizes the distance of the next routing step with no sense of global direction. The expected distance travelled by a messages during each successive routing step is exponentially increasing. To support this routing procedure, each Pastry node maintains a routing table, a neighborhood set and a leaf set. Each entry in the routing table contains the IP address of one of potentially many nodes whose nodeId have the appropriate prefix; in practice, a node is chosen that is close to the present node, according to the proximity metric. Lets go now into a little more detail with some routing issues:
• Routing algorithm: Given a message, the node first checks to see if the key falls within the range of nodeIds covered by its leaf set. If so, the message is forwarded directly to the destination node, namely the node in the leaf set whose nodeId is closest to the key (possibly the present node). If the key is not covered by the leaf set, then the routing table is used and the message is forwarded to a node that shares a common prefix with the key by at least one more digit. This simple routing procedure always converges. Routing steps takes logarithmical steps, assuming accurate routing tables and no recent node failures. • Initialization: A new node X can be located automatically by an old node A, for instance, using expanding ring IP multicast, or be obtained by the system administrator through outside channels. Node X then asks A to route a special join message with the key equal to A. Like any message, Pastry routes the join message to the existing node Z whose id is numerically closest to. Since node A is assumed to be in proximity to the new node X, Xs neighborhood is initialized the same as As neighborhood set. Moreover, A has the closest existing nodeId to X, thus its leaf set is the basis for Xs leaf set. The total cost for a node join, in terms of the number of messages exchanged, is O(log 2b N). • Failure-Departure: To replace a failed node in the leaf set, its neighbor in the nodeId space contacts the live node with the largest index on the side of the failed node, and asks that node for its leaf table. The failure of a node that appears in the routing table of another node is detected when that node attempts to contact the failed node and there is no response. This event does not normally delay the routing of a message, since the message can be forwarded to another node. However, a replacement entry must be found to preserve the integrity of the routing table. Each node attempts to contact each member of the neighborhood set periodically to see if it is still alive. If a member is not responding, the node asks other members for their neighborhood tables and checks the distance of each of the newly discovered nodes. • Proximity: Pastrys notion of network proximity is based on a scalar proximity metric, such as the number of IP routing hops or geographic distance. It is assumed that the application provides a function that allows each Pastry node to determine the distance of a node with a given IP address to itself.
2.3. PEER-TO-PEER (P2P) TECHNOLOGIES
2.3.2
11
Bamboo [4]
A DHT (Distributed Hash Table) that handles high levels of churn (process of node arrival and departure). In simulated networks without churn, Bamboo achieves lookup performance comparable with Pastry. Bamboo achieves this goal through the following three features of its design: 1. Static resilience to failures: can continue to perform lookups after node failures, routing around them even before recovery begins. 2. Timely, accurate failure detection: A combination of active probing and recursive routing allows Bamboo to efficiently detect failures. 3. Congestion-aware recovery mechanisms: Bamboo integrates new nodes and recovers from the failure of old ones in a congestion-aware manner. In its implementation documentation authors argues that semantics of TCP are inappropriate for a DHT; instead, what is needed is message-based, unreliable, unordered, but congestioncontrolled communication. Bamboo uses the time between the moment it sends a message and the receipt of the corresponding ACK to maintain an exponentially weighted average round trip time. Computes round-trip timeouts (RTOs) to decide when to retransmit a packet. Construction of a Bamboo network and the maintenance thereof use the same algorithms. Originally, Bamboo used the Pastry join algorithm. To join an existing network, a Pastry node A needs the IP address and port of another node G, to which it asks to obtain Routing table info. In Bamboo every period a node sends its entire leaf set to a randomly-chosen neighbor in the set, which in turn replies with the contents of its leaf set (Pastry only does the push but not this pull). For maintaining routing tables, it does the tuning of global routing tables, and does a called local tuning (like Pastry). Since a node has only O(log N) neighbors, the use of recursive routing allows a Bamboo node to only communicate with a small number of peers, and active probing is feasible. Bamboo nodes acquire and maintain neighbor state by several congestion-aware, continuous optimization processes. This process is divided into separate concerns, first producing correctness through maintenance of the leaf sets, then producing efficiency by filling the routing table, and finally tuning routing table entries for proximity. In all cases, the rate at which these messages are sent is capped and is further scaled back in response to congestion or load. Also, Bamboo nodes actively probe their neighbors to compute good timeout values, and by using recursive routing they avoid sending messages to nodes for which they do not have such timeouts.
2.3.3
Bit Zipper Rendezvous [5]
This is an optimal Data Placement for General P2P Queries. As P2P offers a rendezvous point where data and query meet, here the intention is to intelligently place data items at nodes (which involves choosing rendezvous nodes and routing to them). Pairs of queries and values are evaluated by participating nodes. Peer-to-peer based Bit Zipper Rendezvous partitions query-value pairs as opposed to values only. Where flooding to N nodes used to be the only fall-back, the Bit Zipper approach is a replacement algorithm needing only O(log N). In a distributed publish-subscribe network, subscriptions are defined as left parameters located at some nodes. Right parameters are notifications that are forwarded to consumers in case of matching subscriptions. A parameter goes to several partitions, two parameters go to only one
12
CHAPTER 2. BACKGROUND & RELATED WORK
place (i.e., a node). But these two ones are left and right (subscription and notification). So a subscription may be evaluated in different nodes depending the notification, and vice versa. When a left or right parameter reaches, its seed is computed depending of the query and data it brings, and the L(s) or R(s) (left and right sides) is computed to find the actual rendezvous nodes that it must reach the message. Actually, to reach this set of nodes, it is not sent a message per node because some intermediate ones are reached on the pass of the message in the DHT routing. The routing is done by a certain kind of prefix algorithms. In this way, once obtaining the key K where to go, the actual destination node is reached by a distributed algorithm that calculates with recursively routing. Calculating m = k= L(l) + R(r) we obtain the partition key for both seeds, that is, the partition where the pair is placed. After reaching it, only remains to analyze filters to know if the subscription (parameter: “l”) matches the notification (parameter: “r”).
2.3.4
Spanning-tree Overlay [6]
This approach takes the node graph topology and the peer-to-peer network and merge them with the high expressivity of routing in a content-based publish/subscribe system. To avoid introducing routing cycles within a more general redundant graph, it selects for each event (publication or subscription) a spanning sub-tree of the entire graph to be transmitted. To manage congestion balance, it makes use of a different tree for every broker to deliver the same notification. Anyway broker failures may happen, and it is assumed here.
Figure 2.1: P2P Topology Network Example On each edge it associates exactly one content-based filter which determines if an event notification should be forwarded. Notifications go through only one tree, while subscriptions need to go to all possible trees. Filter updates follows a broadcast algorithm attenuated by filters on the edges. This routing algorithm presents an invariant that ensures that published notifications follow a path where filters are always subsets of the one before, so obtaining always good performance in the global routing algorithm. Its concepts were proven over Rebeca and a simple peer-to-peer network called Chord [37], that connects nodes into a ring with distances of the type { 12 , 14 ,.., 21i } so assuring logarithmical paths respect to the number of nodes existent in the network. Figure 2.1 shows a simplified example of a Chord P2P topology network. On transmitting events, when publications reach a node, it is told that is responsible of notifying them to all interested parties in the range: (key(origin),
2.4. COMPONENT-BASED INFRASTRUCTURES
13
range). To do so, the node must check that range to all the nodes that are attached to it and that are responsible of that range. Then, following this idea recursively, a binomial tree like path is generated for each message transmission. A similar idea is used for subscriptions.
2.3.5
Common API [7]
The proposal for a common API for P2P [7] is an interesting interface standardization that defines a common API for structured peer-to-peer overlays and the main abstractions that can be built on them. It defines some abstractions that can be built upon a basic key-based routing (KBR) API. These abstractions include Distributed Hash Tables (DHT), Group Anycast and Multicast (CAST), and Decentralized Object Location and Routing (DOLR). It is presented now an overview of the most important KBR interfaces. For message routing: • void route(key K, msg M, nodehandle FirstHop, boolean Hint): forwards a message towards the root of the key K. Optimal Hint specifies a node that should be used to as a first hop in routing the message. Either the key K or hint may be NULL, but not both. • void forward(key K, msg M, nodehandle NextHopNode): up-call invoked at each node that forwards message M, including the source node and the key’s root node (before deliver is invoked). It does informs that M with key K is to be forwarded to NextHopNode. The application may modify any parameter or determinate message settings or the value NextHopNode to be NULL. Modifying the NextHopNode argument the router behavior is overridden. • void deliver(key K, msg M): invoked on the node that is the root for key K upon the arrival of message M. It is provided as a convenience for applications. For Routing state:
(not involving communication with peer nodes)
• update(nodehandle Node, bool Joined): up-call invoked to inform the application that Node has either Joined or left the neighbor set of the local node. The intention in this NS Framework approach, is to use such a KBR API, but assuring that interfaces also works for non-P2P overlays.
2.4
Component-based Infrastructures
The enterprise Java world has done much work on building alternatives to the mainstream J2EE technologies [38]. Part of it is a reaction to the heavyweight complexity in the mainstream J2EE. So a number of other frameworks have focused on offering lightweight containers. They also have emerged with the intention to simplify component implementations, basing them on POJOs (Plain Old Java Objects) [39]. The aim is to come back from complex code that tended to mix business code with specific technological issues. The terms component and service are used sometimes indistinctively in this kind of infrastructures. A component means a glob of software that is intended to be used, without change but
14
CHAPTER 2. BACKGROUND & RELATED WORK
allowing its configuration, by applications that are out of the control of the writers of the component. A service is similar to a component in that it is used by foreign applications, but they are intended to be used remotely and not just locally like components. Component containers are founded on the principles of Inversion of Control (IoC) [38]. The important issue in all of this is to ensure that the configuration of services is separated from their use. Although globally used, Inversion of Control is a too generic and confusing term. As a result, IoC advocates have settled on the name Dependency Injection as a more specific one. There exist various forms of dependency injection: Constructor Injection, Setter Injection, and Interface Injection. But Dependency Injection is not the only way of removing the dependency from the application class to the plugin implementation. The other pattern you can use to do this is Service Locator, much used in J2EE technologies. The basic idea behind a service locator is to have an object that knows how to get hold of all of the services that an application might need (a registry). From a more practical point of view, some of these containers offer management tools, but others just include the ability to dynamically wire components in some programmatically or descriptively way. Some Component-based Infrastructures offer various desired management services for many applications and framework developments: components’ life-cycle management, remote and easy deployment, configurability, manageability, and monitoring capabilities. Various Component-based infrastructures will be analyzed in more detail later to find out the one that better fits with the requirements of this framework. Here are just presented some of them. Pico-Container [40] is a small, light weight container but it does not provide components’ management nor runtime configuration. DustDevil (a limited container implemented in Bamboo) is also light weight and has a nice desired communication structure but does not cover most of the previously mentioned requirements. JMX (Java Management eXtension) [41] is a little complex to use and heavy weight (not specially suited for small devices). The OSGi (Open Services Gateway Initiative) specification [42] (and more specifically one of its implementations [43]) offers a container and management environment for managing service life-cycle. OSGi has a small footprint and is compliant with the J2ME [44] specification.
Chapter 3
Our Approach: A Pub/Sub Notification Service Framework This chapter intends to give a summary of Franscesca’s main Requirements and to explain the main Proposed Approach to achieve them. Franscesca (FraNSCeSCA) is an acronym that stands for “FRAmework for a Notification Service ComponEnt-baSed Configurable Architecture”.
3.1
System Requirements
The system that is intended to be achieved, and that is explained in this work, can be synthesized in the next paragraph: “A framework for Notification Services that is able to be constructed respect to user needs, selecting among a predefined and extensible set of services, that mainly include broker type, routing algorithm and network structure. The resulting NS must be then flexibly configurable and easily to deploy.” Based on various and previously mentioned experiences on building notification services [1, 33, 6, 18, 19, 21, 20, 3, 2, 45] here are enumerated the main functional and non-functional requirements that from our perspective need to be included in the resulting system.
3.1.1
Functional Requirements
• Clear Design: one of the main requirements of such a framework is to offer a clear design to generate different NS alternatives with several features, different strategies and algorithms. • Library of components: offer a set of (extensible) building blocks including: brokers offering existent addressing models and filtering strategies. Include routing algorithms applying to them and working over different topologies, from static overlays to P2P networks. The topology must be constructed with independence of the network protocol. • Components design: distinguish between different framework components that offer clear interfaces, basing them on different design behaviors and efficiency issues (i.e. filters, strategies). Components must be reusable and independent of any technology or runtime-environment. 15
16 CHAPTER 3. OUR APPROACH: A PUB/SUB NOTIFICATION SERVICE FRAMEWORK
• Configuration capabilities: allow dynamic parameter configuration of components, either manually or automatically by monitoring agents listening system information. Relate these parameters with QoS (Quality of Service) parameters definitions. • Component Life-cycle management: be able to add/remove, start/stop the notification service or any of its individual components remotely at runtime. In this way to be able to change the services offered by the NS dynamically. Reach to be as flexible as to generate various NS’s into the same machine, still reusing same components to be applied to any of them. • Instantiation and deployment: offer an User Interface that given the selection of components needed, returns an instantiated Notification Service ready to deploy and use. The obtained NS must be multi-platform, so being able to be deployed in different systems like mainframes, PCs and Mobile devices like PDA, telephones, and embedded systems. • Clients Applications: applications using any NS generated with this framework must make use of a clear NS interface, and be able to connect to brokers locally or remotely, offering in this way different client deployments. • Monitoring and verification: offer different access points to let the easy introduction of monitoring and configuration verification functionalities.
3.1.2
Non-Functional Requirements
As in every framework, reusability is an issue. Here, in particular, we want to reuse a predefined set of specific functionality, like de/serlialization of messages, routing algorithms, or topologyrelated strategies. But also extensibility, in the sense of adding new features, plays a fundamental role since optimizations and new routing strategies appear frequently. As notification services are usually distributed in a network of brokers, easy and remote deployment is a key factor. Moreover, in such a distributed NS configuration and monitoring capabilities are required in order to tune parameters or replace components if needed. In this way we claim for a manageable and usable framework. In most cases the user wants the resulting NS to be efficient and scalable. The possibility to allow the user to find a trade-off between these two factors is desired. Additionally, the possibility to build an adaptable NS where load- or fault-related adaptations can be managed. In many cases, the underlying infrastructure needs to run diverse and concurrent notification services just to fulfill the requirements of diverse applications, resulting on an unreasonable use of resources. This leads to an approach that where the rational use of resources needs to be taken into account.
3.2
Proposed Approach
This work focuses on a framework that facilitates the combination of various features of a publish/subscribe infrastructure to be able to adequately and fully satisfy application requirements. This can be achieved if the publish/subscribe notification service is constructed based on a reusable and extensible set of components, relying on an appropriate architecture. In this way, an application developer can shape a NS according to his specific application requirements. Preliminary results [46] have shown that these blocks can be classified in four big groups: the own NS characteristics, the routing strategies, the network topology, and the communication
3.2. PROPOSED APPROACH
17
protocol. To reach that conclusion, a deep study and classification of existing publish/subscribe infrastructures was needed, covering their algorithms and mechanisms. This made it possible to abstract functionalities and standardize a general interface for each of them, and so to be able to combine them as needed. Additionally, it is pretended to offer orthogonal features to the specific NS functionality. As previously mentioned some examples of them are monitoring capabilities, configurability and system administration. Also, and considering that the NS’s are normally distributed in a broker network, it is required to deploy and manage these brokers remotely. The resulting infrastructure to be developed will also serve as a testbed to experiment with new and efficient routing algorithms, scalability issues, fault tolerance or transaction, among others; offering an appropriate framework for future investigations in this wide and growing computer science area. We want then to provide a framework to build pub/sub notification services according to applications’ requirements, selecting from a set of predefined NS characteristics like the underlying NS topology, routing strategies, message serialization and also defining other features like how to deal with failures, caching, security or access control. According to all this, our goal is to offer an environment that supports the development of such systems in the following three steps which are also sketched in Figure 3.1. (1) NS Specification
Elementary Building Blocks
Resulting NS
Adm User Framework Configs. Services QoS
(2) NS Deployment
(3) NS Management
NS
NS
Deployment Tool
NS
NS
Figure 3.1: Our Approach 1. Based on the application requirements, the desired set of functions and mechanisms are selected from pre-established framework building blocks (components). This components catalog can be simply extended by following clearly established interfaces. Selected components plus some Quality of Service (QoS) decisions are then introduced through a configuration tool. After a generate operation, the desired building blocks are encapsulated as a NS instance that is ready for the next phase. 2. The resulting NS is deployed by means of a deployment tool on the desired nodes, that can
18 CHAPTER 3. OUR APPROACH: A PUB/SUB NOTIFICATION SERVICE FRAMEWORK
be placed remotely from the user administrator. 3. After the NS is deployed on the desired nodes, they can be monitored allowing adaptation to new components, calibration and tuning of the whole service where QoS parameters can be influenced. In the following chapters we present the proposed framework architecture, how to deal with configuration and management, and we explain how resulting NS is deployed and run.
Chapter 4
Framework Architecture This chapter explains Franscesca’s Architecture, exposing all necessary architectural Views needed to understand it: Components’ View, Data/Control Flow View, Deployment View and Processes View. Management design details are not presented here, and is included in Chapter 5.
4.1
Main Architecture
There have been first analyzed and identified possible interactions among elementary building blocks of functionality based on a study [46] that considered all those projects mentioned in Section 2. Having grouped those blocks (that are called here components), and according to the analysis of their functionality and interaction patterns, the appropriate architecture style for this framework has been found: the layered [47] approach. Figure 4.1 shows an overview of the framework base architecture. Layered systems are the ones in which components are assigned to layers to control intercomponent interaction. Layers should be selected to achieve cohesive independent functionalities with an equilibrated granularity among them. In the pure version of this style, each layer communicates only with its intermediate message. The goal is to achieve modifiability and portability. Each successive layer is built on its predecessor, hiding the lower layer and providing some services that the upper layer makes use of. In our approach there have been defined five layers. The Broker layer encapsulates the main NS functionality and also offers an interface to be used by any interested Application layer. Below the broker layer, various and well defined NS aspects were identified. First and most proximal to the broker layer is the Routing one, which manages the different routing algorithms to be used by the NS. But routing strategies (or routers) must work over some kind of node structure and organization, and it is how the Topology layer appears. Finally, other low level Network features have been encapsulated in the lowest layer, including mostly the selection of protocols and serialization strategies. Based on the analyzed interaction patterns, there have been accordingly defined the interfaces between layers. The communication among layers and components can be synchronous or asynchronous. Messages can be passed between layers as events by using the corresponding layer interface. Arrows in the picture shows this. As mentioned before, a pub/sub NS relies on a distributed network of nodes that cooperate. The way this network is modelled has an enormous impact on the design of the infrastructure. 19
20
CHAPTER 4. FRAMEWORK ARCHITECTURE
Application Layer
Management
Broker Layer
Routing Layer
Topology Layer
Network Layer
Figure 4.1: Architecture So let’s focuss our attention on the topology layer for a while. In this layer we assume different Overlay networks, which can be seen as in the most generic case, where main characteristics of P2P (like self-organization and healing, robustness, load balancing, and scalability) are considered. Nevertheless, other simpler and restricted overlay can still be modelled. Founded on [7] and own analysis it has been defined a common overlay interface that condenses the kernel functionality of different overlay networks. From the runtime perspective we focus our approach on a “distributed” runtime environment that serves as a container for the resulting NS. This environment should support remote deployment and configuration, and also monitoring capabilities on the running NS. All these Management functionalities are designed to be orthogonal to the main stack of layers, and are encapsulated in the vertical components of Figure 4.1. These logical layers defined in the architecture are internally composed of various components. These components may be of different types: the ones owning the main layers goals and the ones offering just supporting functionality to the formers. Combining components from different layers is not trivial and there are obviously combinations that are inadequate. Dependencies among components are also represented within the framework, and will be explained in more detail later in this chapter.
4.2
Components’ View
In this section the responsibility of every layer is presented in more detail and is followed with a description of the possible components inside each of them. Figure 4.2 shows a lower level version of the main architecture. In this picture we can see some of the presented components for each layer and how are they attached to conform services. Also in this figure we can see a simplified version of the main interfaces for each layer. Interfaces will be explained in more detail in subsection 4.3. From the point of view of this section it is important to notice the different kind of components illustrated in Figure 4.2. Squares refers to main components holding the principal layer functional-
4.2. COMPONENTS’ VIEW
21
ity and creating the specific layer protocol events. Rounded squares are filters which were created to achieve just layer optimizations. These components are light weight and do not create later protocols events, limiting them to just change existent protocol information. Finally, we can see ovals which represent strategies. This kind of components offer just several aids to other components resolving any particular, but still configurable issue (i.e. scheduling algorithms, message serialization, etc). In Figure 4.2 we may notice the way in which all these kinds of components can be attached. Main components can be attached to any kind of components: to other main component in different layers (i.e. Rebeca Routing) or in the same layer (i.e. Static Tree), to filters (i.e. Concept-based Broker) and to strategies (i.e. Network TCP). Filters need to be attached to at least one main component (i.e. Filter in broker layer) and can have its own strategies. Any number of correlative filters attached one below the other are allowed. Strategies, then, can be attached to any other component (i.e. Serializer), but are the only case in which their attachments do not follow the main NS flow of data and control. ServiceA
ServiceB
AppComp1
Concept-B Broker notify
Bit Zipper
Spanning Tree P2P
forward deliver
Routing
Channels
Network (UDP)
receive
DHT Topology
Rebeca Routing
Ants
Static tree
send
Scopes
Content-B Broker
route
Management
Filter pub sub adv
Broker
pub sub newPub
notify
Application
AppComp2
Serializer
Network (TCP)
Network
Figure 4.2: Layers and components
4.2.1
Network Layer
Since this layer is at the bottom of the stack, its responsibility is oriented to the mediation with the physical network. On one side, messages coming from the topology layer need to be serialized and transmitted to a specific network address by using a determined protocol. On the other side, incoming messages from the protocol-side need to be de-serialized and passed to the upper layer. In this layer, different components can be used in order to apply diverse serialization or compression
22
CHAPTER 4. FRAMEWORK ARCHITECTURE
approaches and even different protocols.
4.2.2
Topology Layer
The main responsibility of this layer is to maintain the status information of the overlay upto-date considering, for instance, dynamic changes like node joins and leaves, or errors. These situations must generate management events that need to be handled accordingly. This layer is also responsible to send messages to specific destinations into the overlay network. Possible components within this layer are: • DHT(Distributed Hash Tables): is a specific kind of overlay network that provides selfstability (dealing with failures and dynamic changes on the participant nodes), high scalability, and good distributed object location in wide-area peer-to-peer applications. Examples of them are Pastry and Bamboo. • Static tree: is a simple overlay networks type. It is conformed as a static set of nodes, with fixed network physical addresses. The tree topology is related to avoid transition cycles over the full network. This topology lacks of aspects like self-stability needed to manage network changes and failures. • Channels: may be defined as a (dynamic) set of paths, through which messages are transmitted. They can be thought as shortcuts within the network (reaching more directly different destinations). This component can be used as a main topology component or to be combined with other components within the same layer to improve performance and organization.
4.2.3
Routing Layer
This layer performs the main message routing decisions. According to the addressing model, it must decide where to transmit specific events. Here, subscriptions, publications and advertisings are differentiated to decide when to change routing tables and to transmit messages. The Routing Layer basically maintains message destinations according to the available subscription information. The destination types may vary with respect to the selected service, for example they may be specific node addresses or just DHT keys. Within this layer, diverse components (or routers) applying different routing strategies can be used, like the ones mentioned below: • Various Content-based routing algorithms [1] like the ones provided by Rebeca: Flooding, Simple Routing, Identity Routing, Covering and Merging. • P2P routing algorithms like Bit Zipper or the Spanning Tree P2P approach presented in Chapter 2, relying subscription information to P2P rendezvous points. • The Ants approach [23], where the routing strategy is based on real ant behavior. In this approach routes are generated by passing ants that emit and set “pheromone” in paths so other ants can smell it, and then know the way to go. First, ants go randomly by paths, but later on, the best paths are naturally obtained by the paths with most pheromone on them. This strategy was simply traduced to a message routing algorithm that first floods the broker network, and then fills routing tables (according to some probabilities) to better route messages. • The use of Scopes [31] which is an interesting routing strategy based on a selection criteria to restrict/group the set of message destinations.
4.3. DATA/CONTROL FLOW VIEW
4.2.4
23
Broker Layer
This Layer basically offers the pub/sub functionality to the application layer. In this layer can be found components encapsulating different implementations of NS brokers offering any existent addressing model: Channel-based, Subject-based, Topic-based, Context-based and Concept-based. Attached to these brokers it will be common to find components offering different event filtering, in order to optimize message routing (i.e. event correlation filters and time-aware filters). This layer handles remote client connections and serves them with the same main Broker interface functionalities offered to local applications. As mentioned before, brokers are classified on a) inner brokers (also called routers), that manage the connections to other inner brokers and can apply correlation filters to the flow of messages b) border brokers, that handles connections with clients (which may offer, for instance, a store and forward functionality) and c) local broker, that basically manages the pub/sub interactions between the border broker and the client. Any kind of events like subscriptions and notification can reach the broker to be treated. Nevertheless, not all messages reach this layer due to shortcuts in the lower layers. For instance, routing decisions can be made in the Routing layer without the broker noticing about it. Also in the Topology layer a similar situation may occur.
4.2.5
Application Layer
This layer is the one that includes application’s logic and is making use of the middleware. It accesses the NS by calling the pub/sub interface always on the broker layer. There is at least two typical cases where this layer can be placed. The first one is in the case of P2P applications where the application is also a peer node that runs broker functionality (see Figure 4.2). The other case is characterized by applications acting as an end consumer and/or producer of messages. In this particular case, the topology and routing layers are empty. This case will be explained in more detail in the subsection 4.4.
4.3
Data/Control Flow View
The flow of data and control can be analyzed in three ways in our framework: the way in which components are attached and organized to communicate among them (pipelined approach); the way in which the flow of data among components is controlled (services); and the interfaces defined for each layer, which adequately order and restrict interaction among components.
4.3.1
Pipelined Approach
It has been intended to design interfaces adequately so to decouple totally components and layers. To do so it has been first thought of an hypothetical layer, identifying all the possible interactions among components. They include message passing between layers, configuration and management messages, and get/set state ones. Later, it was taken into account the way to achieve and implement interaction protocols. They mostly includes either synchronous and asynchronous messages. For them, it has been defined a well designed set of Service Access Points (SAPs) for each layer, offering well-known points to access layers and its services. Having a layer control component holding common layer functionality, has the advantages that
24
CHAPTER 4. FRAMEWORK ARCHITECTURE
the layer may contain and control access to components. The problem with this approach is the overhead in each message passing or method call. This is due to the intermediate layer component that performs administration and distribution of services in each layer. A second and better solution was take into account, to create a very small and simple notification service solution that administrate messages passed among all components at all layers. That give us flexibility and high decoupling among components. But there is also the same overhead problem (there is overhead in every message passing) and still worse, a single notification service may be a bottleneck for the overall infrastructure. So still a better solution has been found. This one refers to attach components dynamically as they appear in the environment. In this way, as a new component is started, it is attached directly to related components. In this approach a kind of subscription to components is done, in which the small underlying service that manages components only works sporadically on the arrival or departure of components. But the most important advantage with this approach is that we are generating a pipelining of components defining a straight communication between layers, but still allowing the configuration of desired functionality at runtime. The only drawback with this approach is that (as more than one component can be attached to a component) a last decision to what component to send messages must be taken inside the one that is treating it. Anyway, the multiplicity of related components will not commonly be of more than two or three, and this configuration capability can be managed as desired. This topic is related and better explained in the next section.
4.3.2
Services
Within a single runtime environment we can find various and concurrent components in the same layer. So being able to generate many combinations among them. But not all the combinations are possible. For example, a Merging Routing strategy in the Routing Layer cannot be attached to a DHT component in the Topology Layer. So, in order to be able to configure all, and only the possible combinations in an easy way, services are defined. The combination of components crossing all layers is called here a service. Different services (or NS instances) can run concurrently within a single runtime environment. Moreover, and with the purpose of better using resources, a single component in a layer may serve various services within the same runtime environment. Services may be seen then as internal channels of attached components across the whole broker layer structure. Services abstracts the use of a set of components at all layer. This allows us to configure various brokers in different ways at different nodes, offering still more flexibility. For example, at a machine, a routing component may be shared by different brokers, but in other the same components may be replicated. Service does not define the way these services are implemented in each node instance, but define adequately the components functionality they must hold. In Figure 4.2 two services that bundle different components across layers is presented. ServiceA represents the BitZipper approach [5] relying on P2P infrastructure by using an UDP network connection component. On the other hand, ServiceB is characterized by a Rebeca routing strategy [1] relying on a static network topology combined with the use of channels and a traditional TCP network connection.
4.3. DATA/CONTROL FLOW VIEW
4.3.3
25
Interfaces
The design of a common set of interfaces between layers was crucial. They make it possible to add and change functionality in every layer with minimal impact (if any) on others components. Because of the nature of the system we are building (which is basically message passing), we can not restrict to synchronous interactions. Consequently, we provide two different kinds of interfaces. Within the asynchronous interfaces we apply an event interaction. These interfaces are particular for each layer and they are distinguished in upwards messages, downwards messages and connection-related messages. Figure 4.3 sketches these asynchronous interactions that may happen between layers. Synchronous interfaces define a direct communication between layers and components. They are basically defined with the purpose to get or set state of components (in the same or neighbor layers), and also to let the management functionality to get/set configuration parameters in layer components.
Figure 4.3: Interfaces The interfaces have been created respect to a deep study and abstraction of different algorithms
26
CHAPTER 4. FRAMEWORK ARCHITECTURE
and strategies in all layers. This analysis is showed in Appendix A, where interfaces are verified to fit and work correctly with the most representative Framework building blocks. In Figure 4.3 we may synthetically see the events1 that may happen between every kind of layers (vertical arrows), and some of the routines that are called for their treatment (horizontal arrows). Discriminated with column colors interface types are showed: protocol, interface and connection. Protocol interfaces achieve the main layer business functionality and are responsible of sending messages to remote peer entities. Interfaces messages are generated in one layer just to inform any occurred situation of interest to a neighbor layer (or layer component). Finally connection events are created to create connection among remote entities and layers components. Dashed arrows shows the common flow and correlation of events between layers. In this way it may be observed the pipelined way in which components may be organized along every layer (upward and downward), and the shortcuts -in which not all messages cover the whole layers paths- offered by the layered approach that provides less computing effort (i.e. an upwards Forward message in the routing layer do not need to traverse the whole layering, but it is directly mapped to a downwards RouteCont message in the same layer). The tunnels showed in the center of Figure 4.3 represent a layering shortcut to transmit events. They are used to send events to and from remote client applications (see Section 4.4.1). In this approach the layering stack is not the same and it is necessary to traverse the complete layering one to receive and sent events to border brokers. In the Routing layer it is mostly applied the common API presented in Chapter 2. In this way we show that it may be used (with little modifications) over any DHT overlay network implementation, and any other different kind of overlays like a static tree of nodes. This section is organized into a pair of subsections that refers to Asynchronous and Synchronous Interfaces. Asynchronous Interface Definition Below, the full set of asynchronous interfaces for all layer is presented organized into two tables, one for upwards interfaces (Table 4.2) and the other for downwards ones (Table 4.3). Into these tables interfaces are also discriminated into the mentioned types: protocol, interface and connection. Layer Application Broker Routing Topology Network
Entity Application Broker RoutingEntity (Overlay) Node Machine
Destination ApplicationAddress BrokerAddress RoutingAddress Key Address
Information ApplicationInfo BrokerInfo RoutingInfo NodeInfo MachineInfo
Table 4.1: Layers Terminology Prior to do that, it is explained some terminology used to abstract similar meanings in different layers (for details about message types refer to Section 7.2.3). In every layer we can distinguish network entities, but on each of them their behavior is much different. Also each of these entities has a related destination (or address) that is different for each layer. Specific information about these entities that includes Type, Identifier, Address, and any other important data are encapsulated in Info (or information) tags. Let’s assume then the terms showed in Table 4.12 to be used in the overall document. 1 In this point we can refer to events and component interfaces indistinctively because they both have the same parameters. Events do the data flow and interfaces are the place where these events are handled. 2 All definitions are abstract and do not refer to any particular technology.
Broker
Application
UPWARDS
onRremoveDestination(NSDestination): interface designed to inform the broker layer that a Destination (i.e a topic) has been removed.
onAddDestination(NSDestination): interface designed to inform the broker layer that a Destination (i.e a topic) has been created.
onRemoveSubscription(Subscription): interface designed to inform the broker layer that a subscription has been removed.
onAddSubscription(Subscription): interface designed to inform the broker layer that a subscription has been created.
onRemoveAdvertisement(Advertisement): interface designed to inform the broker layer that an advertisement has been removed.
onAddAdvertisement(Advertisement): interface designed to inform the broker layer that an Advertisement has been created.
newPublication(Publication): interface called in Broker Layer when a Publication matches broker subscriptions respect to any filtering mechanism.
handleException(BrokerException): an optional interface designed to receive any broker exception (i.e. full queue, informing that it can not receive more events till new advice).
notify(ApplicationEvent): interface called by the broker on the appearance of an Application Event that matches application subscription. This is the main application interface to listen events from the framework asynchronously.
Protocol
confirm(RoutingEvent, eventState): interface created to treat received events that inform about the correctness on the processing (or transmission) of any RoutingEvent.
update(RoutingInfo, joined): to handle events received in the broker layer after any routing layer change or failure detected. Inform about the left or join of node in the network.
confirm(BrokerEvent, eventState): interface created to treat received events that inform about the correctness on the processing (or transmission) of any BrokerEvent.
update(BrokerInfo, joined): to handle events received in the application layer after any broker layer change or failure detected. Indicates which Broker has left or joined the network.
Interface
connConfirm(origin, dest, connStatus): confirmation about the connection status of origin (local) and destination (remote) brokers.
connIndication(BrokerInfo, requireConn): requirement of connection/disconnection to the local broker from a remote Broker.
Connections
Network
Topology
Routing
UPWARDS
confirm(NetworkEvent, eventState): interface created to treat received events that inform about the correctness on the processing (or transmission) of any NetworkEvent.
update(MachineInfo, bool joined): to handle events received in the topology layer after any network layer change or failure detected. Indicates which Machine has left or joined the network.
confirm(TopologyEvent, eventState): interface created to treat received events that inform about the correctness on the processing (or transmission) of any TopologyEvent.
update(NodeInfo, joined): administration up-call invoked by Routing layer (specifically by DHT components), that informs about changes produced in the (P2P) overlay network to let the Routing layer to change its state (RoutingTables). NodeInfo contains keys and host-address data of the node.
Interface
Table 4.2: Upwards Interfaces
receive(NetworkMessage): abstract definition of an interface that receives a network layer message after a network read. Received messages all upper level messages and include framework and interface parameters (i.e. Service ID).
receive(TopologyMessage): interface created to handle events received from the Network layer that must reach the Topology layer.
deliver(RoutingMessage): invoked only in the destination node (the root for key K in case of DHT) upon the arrival of message RoutingMessage. Provided as a convenience for applications.
forward(RoutingMessage, Key, nextNode): up-call invoked at each node that forwards message RoutingMessasge, including the source node and the key’s root node (before deliver is invoked). Informs that RoutingMessage with a destination Key is to be forwarded to nextNode. Application may modify any parameter or terminate message setting nextNode to NULL. Modifying the nextNode argument the router behavior is overridden.
Protocol
connConfirm(origin, dest, connStatus): confirmation about the connection status of origin (local) and destination (remote) nodes.
connIndication(NodeInfo, requireConn): requirement of connection/disconnection to the local node from a remote Node.
connConfirm(origin, dest, connStatus): confirmation about the connection status of origin (local) and destination (remote) routers.
connIndication(RoutingInfo, requireConn): requirement of connection/disconnection to the local router from a remote router.
Connections
Routing
Broker
DOWNWARDS
removeDestination(NSDestination): used to remove a destination (i.e. a topic, a channel, etc.).
addDestination(NSDestination): used to hold a new destination (i.e. a topic, a channel, etc.).
route(Publication): call to route any publication (based on any addressing model) respect to any routing strategy.
removeAdvertisement(Advertisement): interface called by the broker to remove an Advertisement.
addAdvertisement(Advertisement): interface called by the broker to actually manage a new Advertisement.
removeSubscription(Subscription): interface called by the broker to remove an existent Subscription.
addSubscription(Subscription): interface called by the broker to add a new Subscription. This subscription holds all necessary filters and destinations.
eliminatePublishAdv(Advertisement): used to delete advertisements.
createPublishAdv(Advertisement): to allow the application to advertise what type of publication is going to perform.
eliminateDestination(NSDestination): used to delete destinations.
createDestination(NSDestination): used to create a new subscription destination (i.e. a topic, a channel, etc.).
unSubscribe(Subscription): used for deleting a subscription.
subscribe(Subscription, ApplicationInfo): main broker interface used by applications to subscribe to certain Applications Events respect to any addressing model. ApplicationInfo gives details about the applications that is performing the subscription.
publish(ApplicationEvent, Publication): main broker interface used by applications publish Application Events. In case of label-based addressing models, Publication is used to hold the publication type.
Protocol
response(BrokerEvent, eventState): interface defined to handle an event received to inform about the processing state of any upcall event (required for some reliability protocols).
change(BrokerInfo, bool joined): interface defined to handle an event sent to this layer to inform about any specific Broker update.
response(AppEvent, eventState): interface defined to handle an event received to inform about the processing state of any upcall event (required for some reliability protocols).
change(ApplicationInfo, joined): interface defined to handle an event sent to this layer to inform about any specific Application update.
Interface
connResp(origin, dest, connStatus): response to a connection/disconnection to the destination (local) broker required by an origin (remote) broker.
connReq(BrokerInfo, requireConn): outgoing ask for a connection/disconnection to a neighbor Broker. It may refer to connect to a specific address or just to create attachment to Brokers that are to be connected previously by their respective overlay nodes.
Connections
Network
Topology
DOWNWARDS
connRequest(NodeInfo, requireConn): outgoing ask for a connection/disconnection to a neighbor Node. connResponse(origin, dest, connStatus): response to a connection/disconnection to the destination (local) broker required by an origin (remote) broker.
response(TopologyEvent, eventState): interface defined to handle an event received to inform about the processing state of any upcall event (required for some reliability protocols).
connResponse(origin, dest, connStatus): response to a connection/disconnection to the destination (local) router required by an origin (remote) router.
connRequest(RoutingInfo, requireConn): outgoing ask for a connection/disconnection to a neighbor router.
Connections
change(NodeInfo, bool joined): interface defined to handle an event sent to this layer to inform about any specific Node update.
response(RoutingEvent, eventState): interface defined to handle an event received to inform about the processing state of any upcall event (required for some reliability protocols).
change(RoutingInfo, bool joined): interface defined to handle an event sent to this layer to inform about any specific Routing update.
Interface
Table 4.3: Downwards Interfaces
send(Message, Address): main interface used to send any Message to a specific address based on any underlying technology (i.e. IP port and address).
routeCont(RoutMsg, Key, nextNode, fwd): to resend a message to the destination Key after an up-call. Include a next node if desired to begin routing. Either Key or nextNode may be NULL, but not both. Parameter forward is set to true is it is desired to up-call forward messages in intermediate nodes.
routeInit(RoutMsg, Key, fwd): routes a RoutMsg through any overlay network. In the case of a DHT it sends message to the node numerically closest to the Key and forwards a message towards the Key (the root of the of the key K, that has been obtained by any P2P routing algorithm, i.e. Bit Zipper).
Protocol
4.4. DEPLOYMENT VIEW
31
Synchronous Interface Definition This is a direct communication (but still component independent) between layers’ components. It is used to obtain or set state between components in the same or different layers, and also to get/set configuration parameters determined by the management functionality. The getter/setter interfaces to administrate component’s state are simple and are defined as follows: • set”Parameter”(value): sets a specific parameter value. • get”Parameter”(): obtains a parameter value. Although for each layer there exists a specific set of interfaces for configuring its components, there is a common management interface defined for all framework components: • getServiceIds(): gets the services that the component is actually serving; • setServiceId(serviceId): sets a new service that the component must serve; • getServiceOrder(serviceId): gets the order in which the component is working in the service; • setServiceOrder(serviceId, order): sets a new order in which the component must workin the service; • getMaximunEvents(): obtains the maximum number of events that the component is queuing; • setMaximunEvents(max): sets the maximum number of events that the component is able to queue. • getQualityOfService(parameter): used to obtain any specific QoS value. • setQualityOfService(parameter, value): used to set QoS parameter values on any component. With the purpose of wiring components, a pair of interfaces are defined to manage services. These interfaces are used by the wiring service to notify components about the arrival or departure of related components into the runtime environment. • addComponent(compId, serviceId, layer, confData): adds a new component with a determined compId, serviceId, layer and configuration data. • removeComponent(compId): remove a component from the layer. The framework must detect and inform about any inconsistence, and treat the transitive connection/disconnection of components in different layers leaving alive those components which are still linked with other ones.
32
CHAPTER 4. FRAMEWORK ARCHITECTURE
Router
Broker
Broker-Client
Client
Figure 4.4: Global Deployment View
4.4
Deployment View
Figure 4.4 shows a distribution of components running in a pub/sub system. In this figure we can see an interconnected broker network (inside the square), and a set of client applications connected to them. Notice that in this case the broker network is cyclic, so the underlaying Topology must support it. Common clients connect to borders broker that connect them to the overall network. The particularity of this picture is the introduction of P2P applications clients. This kind of clients place a layer above the broker one, and offers an interesting deployment option to which we will go deeper later on. By now it is important to notice that the same NS node into the same runtime environment acts both as broker and as client. Also in this figure we can notice Routers. Routers are lower level Brokers, that only refer their work on routing messages respect to the established routing strategy. Routers do not manage clients and conform an interesting deployment option for internal brokers, that only include Network, Topology and Routing layers. From another point of view, let’s analyze the different distribution of components respect to the different architectural layers. Figure 4.5 intends to show all this. In the lower level (Network) we see different interconnected machines running remotely (or locally in different virtual machines). The layer above it (Topology) shows a ring of interconnected topology nodes that run on some of these machines. There can be one of such nodes per network layer machine, or we can find various nodes running on only one machine. Over this layer, different routers running different routing algorithms may be established. Finally we reach the Broker layer, where we notice that over all this layering deployment flexibility, we may have various running Brokers. There exist then a network of Brokers. In Figure 4.5 there is defined a pair of different interconnected set of Brokers that serves different applications.
4.4.1
Local and Remote Clients
Figure 4.2 shows how clients are also brokers. This may be useful for P2P applications, in which participants act as clients and brokers at the same time. This structure allows to minimize the code needed to be running, so not being necessary to run both client and broker separately, but together and more efficiently.
4.4. DEPLOYMENT VIEW
33
NS Brokers
Broker Layer
Router Strategies
Routing Layer
Overlay Nodes
Topology Layer
Physical Machines
Internet Internet
Network Layer
Figure 4.5: Deployment View
But this may not be always the desired case. So for many NS-based applications, clients must be independent of the whole broker network. This new kind of client is showed in Figure 4.6. In this new approach, a second and more simple layer structure is defined for clients. But still maintaining the same broker layering structure as in Figure 4.2, of course lacking the top most Application Layer, the Routing layer and the Topology layer. This layer configuration is easy to achieve, due to the service configuration defined before. In this new Client layering, Application and Network layers are the same as in the previous Broker one. The new Layer here in the Local Broker. Its responsibility is to communicate with the border Broker to send subscriptions and publications and receive notifications. The principal components defined here is the Local Broker, which is in essence a simplified version of the Broker layer component. In Figure 4.6 we may also see that the vertical Management component is present so to administrate remote clients in the same way as brokers. But a third, and still, more independent client approach is a variation of the last one in which the client is just a process running remotely which communicates to the border broker. That means, that the the management component is lacking and that this kind of clients must be deployed manually. To allow all this configuration flexibility, interfaces were planned to work for all cases. The most important decision is to treat the pub/sub/adv messages downwards and upwards at the same time in the Broker Layer. That means that brokers may receive this kind of notifications, either from the layer above (Application) or the layer below (Routing), treating them in the same but directing application responses to different directions.
34
CHAPTER 4. FRAMEWORK ARCHITECTURE
App1
App2
notif
pub sub adv Broker
send
receive
connect
Local Broker connected
Management
notify
pub sub adv
Application
Network (TCP) Network
Figure 4.6: Remote Application
4.5
Processes View
Conf. Conf. Manag. Manag.
Component Connections
This subsection intends to show the running behavior of any broker generated with this NS framework. Picture 4.7 shows a kind of photograph of an executing runtime environment which holds two NS’s. There we can see the threads running in the system represented as cycling arrow. A main thread is the one of the runtime environment, which manages components’ life cycle.
Context-B Context-B
Broker
Bit Bit Zipper Zipper Life LifeCycle Cycle Manag Manag. .
Subj.-B Subj.-B
Merge Merge
Flooding Flooding
Runtime Environment
Routing
DHT DHT
Channel Channel
StaticTree StaticTree
Topology
Conf. Conf. Check Check
Commb Commb
Network
Figure 4.7: Processes View As the runtime environment is responsible of starting/stoping components, the picture show with arrows that all component threads are hold by it. The runtime environment not only manages
4.5. PROCESSES VIEW
35
the main NS layer components, but also the configuring, management and checking configuration components. Each component in the infrastructure is an event handler that receive messages and enqueue them for later treatment. In this way components can be seen as threads which dequeue messages and execute them respect to scheduling decisions.
36
CHAPTER 4. FRAMEWORK ARCHITECTURE
Chapter 5
Configurations & Management This chapter gives details of the most important framework requirement: Configurability. Framework configuration capabilities and the Architecture-Based NS Configuration Analysis are explained. Also the overall Management design is presented here.
5.1
Configuration Capabilities
The configuration capabilities offered in this NS framework proposal can be differentiated in four ways: the layers selection, the component types, the functionality selection and the component’s settings.
5.1.1
Layer Selection
The layers selection are the highest level decisions. In this level of abstraction, the main NS nodes’ behavior are specified. We can create three different kinds of layer configurations: • Brokers: where the node is defined to include only Broker functionality. In this configuration, the included layers are: Network, Topology, Routing and Broker. • Routers: this kind of node includes only Routing functionality and is the one to be used for inner Brokers. In this configuration, the included layers are: Network, Topology and Routing. • Clients: in which the node plays only the role of a remote client application that is connected to any border broker. The layers included in this configuration are: Network, Broker and Application. • Client - Brokers: here the intention is to combine both Client and Broker functionalities at the same node. So the set of layers included here are: Network, Topology, Routing, Broker and Application. Also, for any of these configurations we can decide if to include or not the Runtime Environment to support life cycle configuration Management. 37
38
5.1.2
CHAPTER 5. CONFIGURATIONS & MANAGEMENT
Component Type
Into any layer we may include several components offering different functionalities that can be dependent or independent among them. Components can be defined to be of any of any of three types: main components, filters and strategies. These types are independent of components’ functionality, but they classify component behavior into the overall architecture. The wiring of these different kind of components if very flexible, but there exists some logic restrictions explained in Section 4.2.
5.1.3
Functionality Selection
One of the main goals of this framework is to offer a library of building blocks, or ready to use components. In this way, for each layer we can include the desired functionalities. But obviously, some components wiring can not be allowed. One illustrating example is the case of the Bit Zipper routing algorithm that was specifically created for a specific type of topology, the P2P overlay. This means that it must be restricted the use of such a routing algorithms only with a specific type of overlay networks. Anyway various P2P overlays implementations can be attached to this routing algorithm. Also some components could have been designed to require some kind of functionality to support them. Let’s think in the case of a Network component for which we want to test various serialization strategies. In this case, this Network component will not work without the inclusion if any of these strategies. All these restrictions must be guaranteed to be satisfied. To achieve all these mentioned decisions, components specify subscriptions to other component types. In this way components own a functionality class and explicit they required and desired types to which they can be attached to. Required types refers to components that must be included with no exception, and desired types indicates just the possible components to which they can be connected to (so to be notified when these components are started in the runtime environment). This subscription is not static, and it can be modified at runtime to accept new kinds of components.
5.1.4
Component Settings
This configuration capability is the lower level one. It refers to change specific parameter values into any existent component. There are a set of common of parameters defined for every component. But also there exist specific parameters for individual components. The intention is then to allow the dynamic setting of any of these parameters. This configuration is not as simple as to only offering a way to change values of possible parameters. It also can be used for changing configurations. In order to organize and abstract the selection of desired functionalities in the NS Services were introduces. Services are defined as a complete set of components including all layers. A service refers then to a specific node or broker running within the runtime environment with specific functionality. Components can work on one on various services. And each component has an order (or position) into every service in which it is included.
5.2. CONFIGURATION VERIFICATION
5.2
39
Configuration Verification
Based on the previously explained configuration capabilities, there can be defined four different abstract levels of configuration verifications. These verification must be achieved always on creating or changing Notification Services configurations. In this way we can assure architecture coherence, restricting the way in which the architectural design is permitted to evolve over time. Based on [48, 49] and my Architecture-Based NS Configuration analysis [50, 51] (synthetically explained below) it has been generate a strategy to formally interpret verifications. The main idea is to map the whole NS Framework architecture into some ADL (Architectural Description Language) architectural styles or families, and to define on them desired constraints. Most of the restrictions to be verified in every category (showed in Section 5.1) have been found to be nicely mapped to ADL architecture constraints. Later, on creating or updating NS instances, ADL systems are dynamically generated. Then, the application checks configured system against predefined constraints. The application returns compiler similar errors and warnings messages informing the user about the feasibility of his configurations. In the following subsections, details about each of these kinds of restrictions and their verifications are given. The selected ADL to implement the verifier application has been ACME [52] due to its simplicity and the existence of related tools. In particular it has been used the ACMELib library [53] that includes the ArmaniParser [54] to express constraints. Armani constraints are defined in a SQL-like language that uses first-order predicates to express and package design constraints.
5.2.1
Layering Verifications
This upper level verification refers to maintain main framework architecture coherence. Basically, the intention is to do not allow skipping layers in the stack sequence, so to let only the connection of components to others residing on same or contiguous layers. But as explained before, different layer configurations are allowed. So to achieve this verification we must watch to a pair of parameters: (1) the layer in which components are hold and (2) the predefined NS node type (in which different layer sequences are allowed). Parameter (1) can be simply expressed in terms of architectural constraints adding a property to each individual style component referring to the layer in which it is included. Parameter (2) is included in the system definition as a global property.
5.2.2
Component Type Verification
As previously explained, there exits three different types of components: main components, filters and strategies. Although these types are flexible to wire there exist some restrictions. So it is necessary to maintain consistence on its rules on wiring components. Main components can be attached to any kind of components: to other main component in different layers or in the same layer, to filters and to strategies. Filters need to be attached to at least one main component and can have its own strategies. Any number of correlative filters attached one below the other are allowed. Strategies, then, can be attached to any main or filter component, but are the only case in which their attachments do not follow the main NS flow of data and control. Restrictions for this verification can be expressed in the overall architecture family specifications
40
CHAPTER 5. CONFIGURATIONS & MANAGEMENT
as predicate constraints. These constraints are checked on system component connections taking into account component type properties.
5.2.3
Functionality Architecture Consistence
Previous restrictions are based mostly on framework behavior. In the following we refer to particular functionalities. Specific framework building blocks (or functionalities) can internally refer to individual architectures (i.e. P2P overlay component). And for some cases, these functionalities must follow certain characteristics on wiring components. These restrictions can be clearly mapped to architectural family constraints making use previously defined properties in each layer: component class and the set of subscription including the classes of components to which they intend to be attached to. This verification is said to be the more complex because transitive verifications can be required. For example, let’s think of a component that has two possible attachments, a desired one A and a required one B. It is possible to attach it first to A, and to have attached to it various pipelined filters. Despite of this, the indirect attachment to B must be guaranteed, needing to achieve then transitive verifications.
5.2.4
Component Settings
Setting specific internal parameters into components can require to verify injected values respect to some boundaries. For instance, a common component parameter is the size of the queue for managing Events. This value can vary respect to specified limits. These limits can be expresses as simple constraints in the architecture family description. Then, on configuring a system, these values are made explicit and consequently checked to predefined family parameter boundaries. The simple setting of service parameters into a component require a rather complex verification. Changing/adding these services values, implicates a verification of consistence into the whole NS. On setting service values on any component, global verifications for the entire system must be done to check that all defined services contains all required components.
5.3
Components Descriptions
In this section we describe, for all analyzed components, particular settings of previously explained properties. These properties describe components and contain particular information needed to include these components in the framework. This information is used either to automatically generate components wiring, to define internal component behavior (without coding), and to check relationships constraints based on the strategies presented in this chapter. Using a table for each layer, there are showed components’ specific component type, functionality class and subscriptions. Comp. Name TCP UDP Serializ A
Comp. Type main main strategy
Funct. Class protocol protocol serialization
Subscriptions req: N.serialization.* req: N.serialization.* -
Table 5.1: Network Components
5.4. MANAGEMENT DESIGN
Component Name Pastry Bamboo Static Tree Dyn. Channels Channels
41
Comp. Type main main main filter main
Funct. Class DHT DHT static channel channel
Subscriptions des: N.* des: N.protocol.UDP des: N.protocol.TCP, T.channels.DynChannels des: N.protocol.TCP des: N.protocol.TCP
Table 5.2: Topology Components Comp. name Simple Routing Merging Bit Zipper Spanning tree Ants Scopes
Comp. Type main main main main main main
Funct. Class rebeca rebeca P2P P2P ants scopes
Subscriptions des: T.static des: T.static des: T.DHT des: T.DHT des: T.* des: T.*
Table 5.3: Routing Components Comp. Name Rebeca Scribe Concept-Based Time Filter Order Filter
Comp. Type main main main filter filter
Funct. Class Context-B Subject-B Concept-B filter filter
Subscriptions des: R.rebeca.*; B.filter.* des: R.*; B.filter.* req: R.Context-B.*,R.Subject-B.*; B.filter.* B.filter.* B.filter.*
Table 5.4: Broker Components
For illustrating purposes, let’s examine Table 5.1 that refers to the Network Layer. This table contains three components. TCP and UDP components are two main components included in the protocol functionality class. They also present a subscription: req: N.serialization.*. This one express that to work they require the wiring to any component (* ) with serialization functionality class, of the network later (N ). The component SerializA is a strategy component included in the serialization class with no subscription.
5.4
Management Design
Figure 5.1 sketches a running notification service broker with its defined layered design. Attached to it is the Management component, which administrates framework component life cycle, monitoring and configuration capabilities. Changing configurations to the NS can be achieved manually through the Configuration component or automatically by the Management component reading monitoring data. Also, as mentioned above, configurations must be verified in different ways to satisfy several constraints. So an extension to the main management functionality has been included to the main design. The following sections give Management Design Details and explain Architecture Verification component.
42
CHAPTER 5. CONFIGURATIONS & MANAGEMENT
Management
Broker Layer
Comp Config Lyfe Cycle Manag
Main Manager
output
Application Layer
Monitoring monit
Config. Data
input
Manual Configuring
Running NS Broker(s)
Routing Layer
update
Topology Layer Network Layer
Event Translation Layer Style Checker Main Checker Configuring Verification
Comp Type Checker
output
Settings Checker
Figure 5.1: Management Design
5.4.1
Management Design Details
Having explained Framework architecture and having formalized Configuration Capabilities now it is time to explain in more detail the Management crossing functionality. This crossing functionality is structured as a vertical layer perpendicular to the main NS layering stack. The management component is responsible of various functionalities: Life Cycle Management, Component Configuration, and Monitoring Capabilities. The design of the components that achieve this functionality are explained in detail below. • Manual Configuring: offers an user interface to deploy a packaged NS, or to update it. Updating a NS includes adding or removing components from a deployed NS. This interface also allows to re-configure any deployed component parameters. This component communicates with Main Manager component that actually achieves the work calling to the Life Cycle Management and Component Configuration functionalities respectively. • Monitoring: this functionality refers to listen to several events generated in the NS. This data can later be showed as output, or more complexly be analyzed to re-configure the system. For example, monitoring could check for event queue sizes in components, and in case to exceed pre-established boundaries could take some action as reconfiguring the system. In the same way as the Manual Configuring component, these actions are actually performed calling the Main Manager component which mediates and organize communication to Life Cycle Management and Component Configuration functionalities. • Main Manager: this main logical component include main framework organization functionality. It includes, the container of components joined with their life cycle management, their
5.4. MANAGEMENT DESIGN
43
parameter configuration behavior, and maintain the broker structure of wired components. Internally it is conformed with the following components: – Component Configuration: this component manages how to change NS parameters values like, queue sizes, services offered by a component, or any internal component state. They are achieved acceding to each component configuration interface. This is mostly related to the get/set interfaces previously mentioned. – Life Cycle Management: The system offers life cycle management in terms of dynamically deployable components. So allowing the user to install, start and stop services, and making sure that after it related components are notified of changes and updated because of them. – Configuration Data: a repository aimed to contain configuration decisions, and to log NS monitored activities. NS/Functionalities Component Creator There exists an additional independent facility designed to instantiate the desired notification service respect to the selected functionalities. This facility also allows to archive new functionalities to be included in existent NS configurations. In both cases a resulting package is created that is ready to be deployed in the desired (possibly remote) runtime environment.
5.4.2
Configuring Verification Design
The inclusion of the architecture-based configurability verification to the NS framework has been planned and designed to work independently of the overall Framework. In this way, this verification can be plugged and unplugged as desired so not to interfere with framework performance. We have then the possibility to attach a Verification component to the Management System. In this way, any system configuration event is sent to this new component to verify consistence respect to the system architecture style information. These events are sent translated to a more abstract event language to be understood by any verification strategy. If any verification component is attached to the Management component, configuration is just not checked. Any Verification (or Checking) component can be attached to the Management based on simple interface. This allows to include simple stand alone verifications, or more complex ones as the one showed in Figure 5.1. In this case, the Architecture-based verification has a main Checking component that discriminates events and generate various kinds of verifications (the ones explained in Section 5.2). These verification then do their work analyzing events respect to Architecture information and some dynamic data if needed. After verifications are made, some kind of output is generated. In this case, the main Checking component receives all simple stand alone verification outputs, and generates a global one. This global output is presented as Warnings or Errors to be taken into account by Managers that are introducing configuration changes.
44
CHAPTER 5. CONFIGURATIONS & MANAGEMENT
Chapter 6
Deployment & Runtime Several Notification Service Deployment strategies are explained in this chapter, and details about the way in which they are to be included into different runtime environments is given.
6.1
Deployment
As can be clearly seen from the interfaces (Figure 4.3), they basically model three main roles within NS participants. The upward flow of data (left-side of the figure) basically represents consumers of messages, since messages are injected from the network layer and they leave the stack at the top (application layer). The downward flow (right-side of the figure) characterizes producers of messages since the flow begins at the top of the stack (application layer) and leaves it at the bottom. The third case characterizes participants that acts as brokers within the NS network. This is clearly visible in the figure by the upward flow (up to the broker layer) and the downward flow (up to the network layer) forming an inverted U. It must be noticed that NS participants can play just one of these roles or simply all roles simultaneously. The identification of these flows within the layered architecture leads us to apply an idea for deployment purposes. As previously mentioned it has been adopted a solution that is based on subscribing to components instead of subscribing to certain messages. The most important advantage of this approach is that we are offering a pipeline of components that may ne adapted at runtime. Then, according to specific services, a sequence of components can be built which basically represents the previously mentioned flow of messages. Depending on the roles established for participants, different sequences can be built for that purposes. At deployment-time, we build then pipelines of components (with limited control flow just to spring some of them if necessary) that basically process messages. This pipeline is ready to be deployed on a runtime environment.
6.1.1
Container-based Deployment Options
We claim for cohesive configurable components, that share a common interface and that may be tied together to offer a desired set of services. Also life cycle management is needed and inversion of control characteristics are desired. Containers offers these characteristics, so here we try to find the best approach to deploy our system on them. It has been analyzed several options, and the better way to use them to support our solution: 45
46
CHAPTER 6. DEPLOYMENT & RUNTIME
• SEDA [45]: stands for Staged Event Driven Architecture. It offers independent components called stages that self manage threads. The communication between them are based on event queues. In a first approach it was thought to be a good idea to follow this architecture, but it lacks of component independence because in the most simple use, every stage must know the queues where to send messages, offering low cohesion, and then little configurability. Also it does not offer component deployment and configuration as desired, and tightly components must be manually coded. • Each layer container: In this approach every layer is implemented as a components container. It offers a good and organized design and good layer independence, using existent small containers that may not have dynamic deployment of components. But the problem with this approach is the bad use of resources because a container per layer must be maintained, and also introduces overhead in the inter-layer communication requiring inter-container communication. Moreover, following this idea, it would be also necessary to establish a global container which would act like a container of containers. This idea was discarded. • Global container: This last approach is referred on holding only a container for the whole layer components. In this way, all of them are contained by the same environment. Here we obtain more direct communication between components gaining performance. Also, as all the components are treated in the same way, we obtain an easily understood framework deployment. For all these reasons this is the selected approach for our framework.
6.1.2
Other Deployment Decisions
An important deployment decision followed in this framework is the possibility to deploy the system using any existent container solution. This offers the possibility to change the platform as desired reusing main component functionality. This is possible if interfaces are well defined and placed in well cohesive components that do not depend on any technologies. Of course it also depends on containers that allow a good separation between interfaces and component association. Moreover, it is interesting the possibility to deploy the system without a container. In this way we obtain a low-weight NS but that must be deployed manually. This deployment approach do not offer configuration neither life cycle configuration, but if offers the lowest overhead. This last approach is possible to generate with our architecture due to the defined interfaces and the cohesive encapsulation of functionality. To generate such a deployment, having the main component functionality encapsulations, it only remains to attach components manually before running the overall system. Another important deployment issue is the ability to establish various NS’s into the same runtime environment. In this way to be able to reuse any running component to several brokers as desired. Details of this approach have been presented in Chapter 4.
6.2
Runtime Environment
We have selected OSGi as the container solution for our Framework development. This selection refers to an analysis of different environments based on a set of desired parameters for our system. This analysis is presented in Appendix B. OSGi offers most of our required management functionalities: components life-cycle management, remote configuration and deployment, and it is designed
6.2. RUNTIME ENVIRONMENT
47
to be light weight, multi-platform and easy to use. In this section we first give a deeper overview of the OSGi platform, and later an explanation of how it is applied to this framework.
6.2.1
OSGi Platform
OSGi (Open Services Gateway Initiative) [42] is defined as a standardized, component oriented, computing environment for networked services. It offers a container and management environment that hold services respect to a common interface. These services may be started or stopped at any time using any available deployment tool. This services are deployed as bundles. Bundles are JAR files containing all service classes and configuration files. OSGi is compliant with J2ME [44] (Java 2 Platform Micro Edition) which addresses the needs of small mobile devices like phones, PDA, and embedded devices. OSGi supports us with most of the desired services that such an environment may offer. It is a light weight container and it offers a flexible component life cycle management, allowing in an easy way to add/remove new services and to configure them also at runtime. All internal OSGi services (i.e. configuration and deployment tools) are also implemented as bundles. Two interesting and useful of such services here are the Service Binder [55] and the Configuration Service [42], both of them explained below. • Service Binder: Service Binder provides a way to relate components based on subscriptions. These subscriptions hold the kind of components that any other components are interested to communicate to. Subscriptions organize a set of component characteristics into a configuration file based on a pretty expressive language. On the arrival of a component into the container, the Service Binder is charged on notifying this new presence to all components that have subscribed to it. In this way, Service Binder separates components unification logic from the business one, removing it from code and incorporating it in a configuration file. In this way, Service Binder allows actually to generate POJO [39] codification, binding the code needed to encapsulate components to be placed into a container. • Configuration Service: An important claimed issue in this framework is configurability. The Configuration Service offers the possibility to access to interfaces making use of reflection technologies and so gaining flexibility. Also it offers a configuration tool to get/set parameter values in a vertical manner, so allowing to fill or force internal states.
6.2.2
Using OSGi
Our approach using OSGi is to start any configured layer component as services. Making use of OSGi’s management functionality we are able to start/stop and configure components as desired. The wiring between components is based on Service Binder. Then the system may dynamically update to new installed functionalities. Also dynamically updating Service Binder subscriptions we gain more flexibility and runtime configurability. Installing a whole NS is simple by relying on the OSGi platform. We obtain the desired NS as a deployable component that contains all desire services (in the form bundles). With this, the deployment of the NS can be performed (locally or remotely). After deployment, any component can be managed and monitored. In this way the system supports dynamic updates of new and pre-established functionalities. The presented configuration get/set interfaces are pretended to be managed by the OSGi Configuration Service. Using it, Brokers can be tuned as desired at runtime.
48
CHAPTER 6. DEPLOYMENT & RUNTIME
Chapter 7
Detailed Design This chapter presents lower level design details than given in Chapter 4 about the framework. Subsystem design is showed, and Class and Interaction Diagrams are included to understand framework internal behavior. Also specific component design and implementation issues are explained here.
7.1
Subsystems Design
The framework implementation is organized into three logic subsystems. The intention is to separate functionality as possible to obtain package independence. These subsystems are showed in Figure 7.1: framework, layers and management. Details about them are given below.
7.1.1
Framework Subsystem
The purpose of this package is to contain all framework classes. These classes are immovable and are independent of any container or technology used. The sub-packages included in this subsystem are: • Component: includes the main hierarchical structure defined to represent components into the framework independently from any container. Java interfaces are defined and main abstract components are implemented. – AbstractComponent: Includes all defined interfaces and abstract classes. – LayerComponent: Includes ready-to-extend abstract classes for each layer. • Connections: Includes classes that achieve event wiring. • FrameworkMessages: inside this sub-package are placed abstract messages classes. These messages are the ones that are passed to remote peer layers, and they are integrating part of events. • LayersInterface: this sub-package holds (organized into layers) the abstract interfaces defined in Section 4.3.3. For each layer, these interfaces are mapped to Java interfaces and events classes. – Events: define the events to be passed among layers. 49
50
CHAPTER 7. DETAILED DESIGN
Figure 7.1: Packages – Interfaces: define the interfaces to be implemented by component developers to handle prior defined events. • Utils: Includes all necessary auxiliary classes to be used into the framework like Queues and XML [56] parsers. • FrameworkEvent: contains the classes that represent the events that occurs in the context of the framework. • Services: includes all necessary classes needed to implement services (or NS brokers instances). • NS-Abstractions: This package offers a set of abstract NS concepts to be used by any layer component implementation. The sub-packages included inside it are: – FrameworkInfo: includes, for the framework and for each layer, necessary specific information needed to be passed to other entities. – Destination: each layer represent some kind of destination abstraction. These concepts are organized here to be adequately used by each layer. – NSMessage: includes the full set of Message interfaces, that is an important part of the common data model.
7.2. CLASS DIAGRAMS
51
• LayersConfigurationInterface: Includes interfaces for each layer that refer to configuration activities. These interfaces are implemented by the framework.
7.1.2
Layers Subsystem
The intention of this subsystem is to offer an organized set of packages where the extensible framework building blocks are to be placed. This organization separates business logic implementation and containers based classes. This subsystem contains a sub-package for every layer. These sub-packages include all implemented components for each specific layer. Into any of these layers we have a lower level subpackages structure to place different component types: main components, filters and strategies. All new packages must be placed following this organization. Also there is a defined a set of internal packages to be used into any components, these ones are explained below: • MainClass: this package in prepared to just hold the main component class. • ContainerBased: here are hold (possibly various) container specific encapsulations classes and property files for the component. • NSImplementations: (optional) to include specific NS abstractions implementations to extend the framework. • OtherClases: just to include the rest of the classes that implement the component. That is, the new component ones or legacy code.
7.1.3
Management Subsystem
This subsystem contains all necessary classes to implement the crossing Management functionality. Management components are treated and implemented in the same way as the main NS functionalities ones. This means that these components will also be included into a specific container. So, although separated here, the main internal structure of management is related with framework components. The sub-packages included in this subsystem are: • Component: includes all management components: Manager, Configurator, Monitor and Architectural Style Checker • ManagementInterface: includes interfaces defined for each these components.
7.2
Class Diagrams
This section explains class design of the main framework entities. The selected class diagrams to be showed are: Components Design, Events Design, Messages Design and Management Design.
7.2.1
Components
Figure 7.2 shows framework components low level design. It includes component class hierarchy, including extensions of parent classes and interface implementations. Also relevant relationship with other classes are showed.
52
CHAPTER 7. DETAILED DESIGN
Figure 7.2: Component classes • FrameworkComponent: is the main framework component interface. Included methods are simple and abstract enough to be implemented by any NS components, and allow abstract communication among components. • FrameworkListener: interface designed to hold methods needed to implement the decoupled wiring of components. • SimpleComponent: this abstract class is a light weight component with all required framework message dispatcher functionality. This component is the base class to be extended by any framework component. This class manages event queues and call template methods to handle different kinds of FrameworkEvents.
7.2. CLASS DIAGRAMS
53
• StrategyComponent: a light weight special type of component that is responsible of some strategy action. This component do not maintain connections. It offers an abstract interface to allow other components to ask to perform its functionality. • ManagementComponent: a component ready to be implemented by any management functionality component. As it can be seen in the class diagram, Management components are treated in the same way as layer components, but with different message handling and proprietary wiring policy with other components. • CommonConfigurationInterface: an interface that resumes needed methods to get/set parameters that are shared by all layer components. • UserToolkit: an interface that holds all required functionalities that a layer component developer require to interact with the framework to implement business functionality. • LayerComponent: an abstract class that includes all necessary interfaces for any layer (main or filter type) component. This component is independent of any particular layer functionality and is planned to be extended by all of them (except for strategy components). This component can be set to have a filter or main component type. It also includes Connections functionality to administrate local services and components wiring. It also manages FrameworkEvent handling, calling to template methods to handle specific LayerEvents. – ApplicationComponent: a component that handles this specific layer LayerEvents and that call template methods to actually implement the handling. This component is ready to be extended by any application component, either main component or filter type. – BrokerComponent: a component that handles this specific layer LayerEvents and that call template methods to actually implement the handling. This component is ready to be extended by any broker component, either main component or filter type. – RoutingComponent: a component that handles this specific layer LayerEvents and that call template methods to actually implement the handling. This component is ready to be extended by any routing component, either main component or filter type. – TopologyComponent: a component that handles this specific layer LayerEvents and that call template methods to actually implement the handling. This component is ready to be extended by any topology component, either main component or filter type. – NetworkComponent: a component that handles this specific layer LayerEvents and that call template methods to actually implement the handling. This component is ready to be extended by any network component, either main component or filter type. • ComponentType: an interface for specifying component type behavior. This separation of type is achieved following a strategy pattern design for including main type and filter functionalities. – FilterType: a component type class that implements filter behavior for creating events. – MainComponentType: a component type class that implements main component behavior for creating events. • ConnectionsInterface: this interface holds connection and service management behavior to be implemented in different ways as desired. This class also maintain local knowledge about the layer and the order in which the component (that holds the connection object) is placed into the service.
54
CHAPTER 7. DETAILED DESIGN
• Connections: this class implements the ConnectionsInterface interface and actually holds layer component connections. It also includes all attached strategy components). Also it manages these connections respect to the commended services that the components must service to.
• BrokerDownwardsInterface: This interface persuade Broker components to implement the defined downwards interfaces for the broker layer. Similarly, there exists this same kind of interfaces for the rest of the layers. • BrokerUpwardsInterface: This interface persuade Broker components to implement the defined upwards interfaces for the broker layer. Similarly, there exists this same kind of interfaces for the rest of the layers.
7.2.2
Events
Figure 7.3 shows framework event low level design. This figure includes main framework event hierarchy and shows how Layer events and messages are attached to them. • FrameworkEvent: is the base class for all events passed through all framework components. Various kind of events can be distinguished from this. The main ones are configuration events and inter-layer events. But there exists also monitoring events, config check events and any future events needed to be include. • ConfigurationEvent: describes an event sent to/from the management functionality to manage changes in the NS configuration. – GetEvent: a configuration event that aims to obtain some particular component’s internal state. – SetEvent: a configuration events that aims to change some particular component’s internal state. – UnsetEvent: a configuration events that aims to delete some particular component’s internal state. – AnswerEvent: an answer sent from any layer component to the management functionality. Particularly, this kind of event is created to respond to GetEvent ones. • ConfigCheckEvent: an event sent to any configuration checker component on the occurrence of a configuration activity. This event carry particular configuration events to be analyzed. • MonitoringEvent: specific events sent from any component to the monitoring functionality. • InterlayerEvent: is the framework event used to communicate components in the layering stack. This kind of events contains a LayerEvent and Framework parameters necessary to be sent to the next layer (or component). Including framework data parameters into this kind of FrameworkEvent allows LayerEvent no refer to framework information, but only to business data. • FrameworkInfo: framework information mostly includes the service identifier in which the event works, the direction of the event and the pre-established framework configuration type.
7.2. CLASS DIAGRAMS
55
Figure 7.3: Event classes
56
CHAPTER 7. DETAILED DESIGN
• LayerEvent: into this classification are included all specific events that flow across the layering stack and that refer to the interfaces defined in Section 4.3.3. This kind of events contain interface parameters needed to be passed to the next layer (or component) to perform the indicated action. • ProtocolEvent: into this event classification are contained all events that require sending data to remote peer layers. This means that this kind of events are an integrant part of the overall layering protocol that follow the main layering functionality. This event hold messages, that are the units of data that inform peer layer the action to take. The classification of this kind of events is divided into the existent layers: – ApplicationEvent: from this classification extend all events that are handled in the application layer. – BrokerEvent: from this classification extend all events that are handled in the broker layer. – RoutingEvent: from this classification extend all events that are handled in the routing layer. – TopologyEvent: from this classification extend all events that are handled in the topology layer. – NetworkEvent: from this classification extend all events that are handled in the network layer. • ConnectionEvent: this classification of events achieves connection behavior. This class extends from ProtocolEvent because it follows the same protocol working strategy. Connect request and connect indication events are extended from this event class, and also connect response and connect confirm to allow to generate a confirmed connection protocol. These same four specializations are used for all layers just changing message data in each of them. • InterfaceEvent: this event classification refers to events that accomplish interface (not protocol) functionality. This means that this kind of events are generated in one (producer) layer and they are just consumed in the following (consumer) layer, and do not require protocol behavior like sending message to peer remote layers. The specializations of this class of events is explained below: – TunnelEvent: this kind of event is able to carry any other LayerEvent’s across layers that do not handle them. The behavior of a components on the arriving of these tunnel events is to handle the internal event if the component manages it, or to send the tunnel event to the next attached component in the actual service.1 – Update: event used to inform the layer above about updated entities or failures detected. – Change: event used to inform the layer below about any generated changes on the actual entity (node, broker, etc.). – Response: this event is used to inform the layer below about the correctness on the processing of a previously sent layer event. – Confirm: this event is used to inform the layer above about the correctness on the processing (and transmission) of an event. 1 The framework offers a similar behavior for synchronic getter/setter interfaces into components. But in this case the “tunnel” is achieved calling the next component get/set interface if the actual one do not contain the required parameter name.
7.2. CLASS DIAGRAMS
57
• FrameworkMessage: this class holds the behavior all of the messages being passed through all components in the layering stack, but abstracting message implementation (or message model). In this way, for the user point of view, the management of XML [56] or object-based messages is the same. • Message: the base Java interface for every layer specific message. This message structure is explained in Section 7.2.3. • PipelineMessage: a FrameworkMessage extension that contains only a single document to represent each specific message in all layers. In this way, into each layer the same object is modified by one implementation of the actual message interface (see Section 7.2.3). • LayeringMessage: a FrameworkMessage extension that implements an object structure to represent the actual layer message. In each layer this structured object contains an implementation of the actual message interface (see Section 7.2.3).
7.2.3
Messages
Events transmit messages. A particular event can send various kinds of messages. Figure 7.4 shows the abstract messages defined to be used in this framework. As can be seen messages are defined as interfaces. This allows to generate different implementations of same behavior data units. But this do not mean that users must implement these interfaces, actually the Framework do it. Also defining interfaces allows to apply same behavior to single separated messages or apply them all to a same object that implements the overall messages behavior. Both approaches refer to the way they are applied to LayeringMessages and to Pipeline Messages respectively. Messages still use other lower level data types like LayerInfo and Destination abstractions that contain specific information for each layer. In the following are explained framework message hierarchy. • Message: the base message interface that allow to set any generic or layer specific parameter names and values. • ApplicationMessage: a particular classification of a message for the Application layer. – Subscription: a subscription with needed data from the application point of view. It allows to include legacy subscription or to create them applying individual abstract filters. – Publication: a publication with needed data from the application point of view. It allows to include filter data in the case the addressing model is less expressive than in the content-based approach. – Advertisement: an advertisement with needed data from the application point of view. Also it allows its conformation by adding any variety of filters. – NSDestination: a destination (i.e. topic, channel, etc.) with needed data from the application point of view. – ApplicationEvent: an abstraction that encapsulates events that the application desire to publish. – SimpleApplicationEvent: an application event containing only one event. – MultipleApplicationEvent: an application event containing multiple events. Using this kind of events, publications can contain various application events. It is a convenience for many applications.
58
CHAPTER 7. DETAILED DESIGN
Figure 7.4: Messages
7.2. CLASS DIAGRAMS
59
• BrokerMessage: a particular classification of a message for the Broker layer. – Subscription: a subscription with needed data from the broker point of view. It involves a transformation of the application subscription plus additional broker data. – Publication: a publication with needed data from the broker point of view. It may involve a transformation of the application publication plus additional broker data. – Advertisement: an advertisement with needed data from the broker point of view. It involves a transformation of the application advertisement plus additional broker data. – NSDestination: a destination (i.e. topic, channel, etc.) with needed data from the broker point of view. – BrokerException: on any exceptional situation or error occurrence the broker must inform other brokers or applications about the happening situation. An example can be a full queue exception that indicates neighbor brokers to stop sending messages till next advice. • RoutingMessage: a particular classification of a message for the Routing layer. – Subscription: a subscription with needed data from the routing point of view. It involves a transformation of the broker subscriptions plus additional routing data. – Publication: a publication with needed data from the routing point of view. It can involve a transformation of the broker publication plus additional routing data. – Advertisement: an advertisement with needed data from the routing point of view. It involves a transformation of the broker advertisement plus additional routing data. – NSDestination: a destination (i.e. topic, channel, etc.) with routing data from the broker point of view. – Blank: A blank routing message indicates that routing work must be executed “downwards”. This means that first the broker layer must analyze the incoming event and later to call to routing functionality.2 • TopologyMessage: a particular classification of a message for the Topology layer. – UpdateMessage: a specific topology method used to inform (usually to neighbor nodes) any happened situation like errors or the need to restructure the nodes network. This kind of messages is to be used by self changing overlays like DHT’s. – TransmitionMessage: a message that encapsulates information from other layers, and that the topology layer is charged to transmit. • NetworkMessage: a particular classification of a message for the Network layer. – Nack: a simple network message that inform about the not properly reception of messages. This message is required by some reliability protocols. – Ack: a simple network message that inform about the properly reception of messages. This message is required by most of the reliability protocols. – TransmitionMessage: a message that encapsulates information from other layers, and that the network layer is charged to transmit. 2
This behavior is offered to facilitate encapsulation of NS solutions that were designed to work in this downwards way.
60
CHAPTER 7. DETAILED DESIGN
• ConnectionMessage: a particular classification of a message for connection purposes. – ConnectRequestMessage: a message that holds required data to start a connection protocol. This message is used in either ConnectRequest and ConnectIndication events. – ConnectResponseMessage: a message that holds data to respond to a connect requirement. This message is used in either ConnectResponse and ConnectConfirm events.
7.2.4
Management
Figure 7.5 shows framework management low level design. This figure shows the main management functionalities and how they relate with each other.
Figure 7.5: Management • ManagerInterface: as explained in Section 5.4 there exists a main management component that administrates and implements main management functionality. This interface holds all needed methods to achieve this. • Manager: just a ManagerInterface implementation. The manager has access to handle components life cycle and internal configurations. • ConfigCheckInterface: an interface that guide all configuration checkers. This one can be implemented by any simple checker, or any more complex architecture verifier. • ServiceChecker: a configuration verifier that just check for service integrity. • ArchStyleChecker: a more complex configuration verifier that achieves all verifications explained in Section 5.2. This component holds and uses other smaller checkers to commit its work, integrating their results.
7.3. INTERACTION DIAGRAMS
61
• ConfiguratorInterface: an interface that formalizes manual configuration for the whole NS components. • Configurator: an implementation of the ConfiguratorInterface. • MonitorInterface: an interface that formalizes monitoring capabilities for the whole NS components. • Monitor: just an implementation of the MonitorInterface.
7.3
Interaction Diagrams
To understand framework running behavior this section explain interaction among framework elements. There have been selected the most important scenarios that bring a better comprehension of this work.
7.3.1
Event Composition and Transmission
In Section 4.3.3 there have been explained the designed interfaces between layers. Also in Section 7.2.2 it has been showed the event class design. But no details has been presented about how events and messages are treated on reaching each consecutive layer. Also little has been explained about the way in which a layer X message in broker A reaches layer X in a peer broker B. So in this section we present an explanation of interface and protocol interaction. Event structure can be deduced from the event low level design (7.2.2). FrameworkEvent, and most specifically its extended class InterlayerEvent, is composed of a LayerEvent and framework data. These are the main element needed to communicate two component into any layers. A LayerEvent (and most specifically a ProtocolEvent) contains a FrameworkMessage, and includes interface data to be consumed by the next layer in the stack. Finally, FrameworkMessages holds specific layer X and prior layer messages, and they include additional data like event type description, destination info. In Franscesca there are defined two models for managing messages: Layering and Pipeline. Layering model refers to treat messages in each layer as different objects. These objects are then composed in each layer, conforming a composite of messages in each successive layer. The Pipeline model uses a single element to map messages in all layers. In each layer, this element is modified to add new information, but not overwriting previous added data. This approach is adequate to be used to implement XML-based [56] messages. In this approach all information can be nonintrusive integrated in the same document, to be transmitted later as a single element by adequate transmission protocols (i.e. SOAP [57]). Layering Model The name of this model refers to the way messages are composed and decomposed in a layering architecture. Figure shows event/message main structure for this model and intends to explain how each of the components are created, used and combined to reach desired destinations. The explained case refers to a local layer component “A” that wants to send a message to a peer layer component remotely named “F”. This example is explained in six steps, each one representing a message passing between two layers.
62
CHAPTER 7. DETAILED DESIGN
FrameworkEvent (InterlayerEvent) LayerEvent (ProtocolEvent) LayeringMessage (layer X) Interface Framework Message (layer X-1)
Event Type
Specific Message (Layer X)
Framework Parameters
Parameters
Destina tion
LayerEvent (Layer X+1)
a aa a
a
b b bb b
b
A
F
a aa
a B
E
a
e
D
b
d
e
e
e
d
d
d
c
c
a a
b
b c c cc c
c
b
b C c c cc ccc
c
Figure 7.6: Layering Model 1. Layer component A creates a complete InterlayerEvent. There is a lack of the X-1 layer Framework Message because there is no upper layer above it. The InterlayerEvent is sent to component B in the layer below. 2. Component B consumes A’s Framework Parameters and Interface Parameters. Later in created its own InterlayerEvent and introduce A’s ProtocolMessage as its X-1 layer Framework Message. B sends its new InterlayerEvent to component C. 3. Component C consumes B’s Framework Parameters and Interface Parameters. As C is the last components in the stack it do not need to create its own InterlayerEvent component. It is sufficient to it to create a LayerEvent, introduce B’s ProtocolMessage as its X-1 layer Framework Message, and to send it through the network to reach the remote Broker. 4. The lowest component in the remote broker, D, receives C’s LayerEvent, consumes its InterfaceParameters. Then D create its own InterlayerEvent, and as its ProtocolMessage it includes C’s X-1 layer Framework Message. D send his InterlayerEvent to the layer above: E. 5. E consumes D’s Framework Parameters and Interface Parameters. Then it process its peer component B’s ProtocolMessage. E does separate B’s X-1 layer Framework Message and creates its own InterlayerEvent to the upper layer. 6. The last layer in the sequence, F, consumes Es Framework Parameters and Interface Parameters. And at last it is ready to process the message that component A has sent to it. In all this explanation, there is no use of the X+1 layer event. The reason is that this is an intermediate event used when more than one component is included into the same layer. An example using this approach is given in the Pipeline Model below.
7.3. INTERACTION DIAGRAMS
63
Pipeline Model This model name (pipeline) refers to the way in which components behave in this approach as transformers (or filters) of the same data. In Figure we can notice the differences of this pipeline model with the layering one. First of all we must notice that the event structure is not modified. Changes are included in the message internal structure. There is no more a composite message but a single plain message to perform the same functionality. FrameworkEvent (InterlayerEvent) LayerEvent (ProtocolEvent) PipelineMessage (layer X) Interface
Framework Parameters
Parameters
...
Unique Message LayerEvent (Layer X+1)
a
a
a
a’
a b
a
a’’
b’ b
A a
a B
a
D
a
a
c
c b
a
c
c
b
a’
b
b
b’ b
C
a’’ b’ bc
Figure 7.7: Pipeline Model This scenario merges the pipeline model with the use of Filter Components. Merging both strategies we obtain a lower weight event/message structure. As previously mentioned, Filter behavior is restricted to modify existent events and not creating new ones. This is particularly interesting to use when we have more than component per layer, so reducing the number of events created. The Pipeline model is not referred to reuse and transform the same event, but the same message. Left side of Figure shows two layers. The upper one (A) with a main component and the lower one (B) with a main component and a filter component below. Both components in the same layer (B) use same parameter information. On receiving an event, main component of layer B process it and generates a new layer x+1 event (b). But it do not create a new message, it modifies the same received message (transformation a -> a’. Then it sends the same As event, with the new (intermediate) event attached to it. On receiving this same event, the lower (filter) component of layer B do not create a new event, but modify the intermediate one. Neither it creates a new message, but it modifies the existent one (transformation a’ -> a”). As it is the last component in the layer, it creates a new FrameworkEvent, including Framework and Interface parameters and placing the new layer event. Afterwards, as in this particular case this is the last layer in the stack, it includes the FrameworkEvent parameters in the single plain message to be transmitted. In this way, the single message is also used to transmit framework and layers parameter non-intrusively to
64
CHAPTER 7. DETAILED DESIGN
the destination Broker. Right side of Figure shows how the lower peer layer (now layer C) filter component use transmitted framework and interface parameters (b and b’). Also we can notice how this lower (filter) component on noticing the no existence of a framework event (because it is the first in he stack) it creates one. Also this component consumes a” data, removes it from the message, and lets it available to the component above just the a’ version of the message. Similarly it does the same consuming possibly interface parameters (transformation b’ -> b). On receiving the event, the main component now consumes a’ and the modified layer parameters. As the previous component has not created an intermediate event (because it is a filter) the main component creates a new event to be sent to the upper layer including the a version of the message and new layer and framework parameters (c). Finally the component in layer D is ready to consume message a.
7.3.2
Components Wiring
Figure 7.8 shows an scenario where a new functionality (a component) is introduced in the framework. This interaction diagram shows how management components treats this new configuration, how interested components are notified about this new apparition, and how local wiring is achieved.
Figure 7.8: Components Wiring Prior to add a new component, an user administrator sets component bundles configurations to inform the framework the services that the component will serve to and the position where to wire it. Then, he uses the container shell (or any other graphical interface) to introduce the new components to the NS. On doing this, the container is responsible of instantiating the component and to inform interested partners about this new appearance.
7.3. INTERACTION DIAGRAMS
65
As mentioned before, to notify interested components the Service Binders is used. Using it, a better separation about business and managing logic is obtained. Service Binder reads component configurations, and search for subscribed components. Then, using the preestablished addComponent method it inform all interested components about the new component. All notified components then check the relationship with this new component and establish the wiring. This wiring is achieved holding a reference to the new component in its Connections object. The Connection object then maintains the relationship among services and attached components to be used later on message processing. Subscriptions are made from upper components to below ones. So, on being notified, a component must call the recently introduced lower component to also hold the connection. In this way a bidirectional connections is generated, allowing both upper and lower event flows.
7.3.3
Event Creation and Dissemination
Except for configuration getter/setter interfaces, main framework interaction is managed as asynchronous events. This section explains the way in which layer events are created in one component and are distributed to other components. It is important to notice here that on implementing a new component, the developer only has to refer to code business logic functionality into the defined layer interfaces (see Section 7.4.4 for details). Framework is charged of generating framework events, administrating components types, managing different different message models and sending events correctly to attached components. Figure 7.9 shows an abstract behavior of any specific component and framework work on processing an event: from the moment when it is created in one component, till it is injected to be processed in another component. The object ComponentA, is an instance of a RebecaBroker component, whose interface has been implemented by a framework developer (user). In one of the implemented interfaces methods, the user requires to create an specific layer event and to disseminate it. To do so, the user is supplied with an interface that concentrates all required functionality to interact with the framework: the UserToolkit. After the call to these interfaces, activities are generated and managed by the framework and its configured internal state. To implement a component abstract method, the developer writes some user code. Also, if it has previously been defined, he makes use of Strategy getter/setter methods through the Toolkit interface. At some point, the developer will require to create a specific Message instance, and he does so asking UserToolkit’s MessageFactory method. This method return the required message respect to the message model selected. After filling message parameters, the developer require to create a new event. To do so he ask the UserToolkit’s EventFactory method, passing it the event type an the previously created message as parameters. The framework then create a FrameworkMessage (if it has not been previously created respect to the message model selected) and later it instantiates the event respect to the component type behavior. After filling event interface parameters, the developer is ready to deliver the event. To do this, he again call an UserToolkit interfaces, deciding if to transfer it into the actual flow direction (calling send method), or maybe to realize a callback (in this last case the event is sent in the other direction maybe because it is a response or it is a layer stack shortcut). To actually deliver the new event, the framework ask the Connections interface about service next step. If the next component is included in the same layer as the actual one, it reuses the previous FrameworEvent and adds to it the new LayerEvent created. If the next step is on a different layer, the framework creates a new FrameworkEvent, organize into it the created messages and events and adds framework parameters. Finally, the framework pass the event to the next component calling the main framework event
66
CHAPTER 7. DETAILED DESIGN
Figure 7.9: Event Creation and Dissemination
7.3. INTERACTION DIAGRAMS
67
handler method. On receiving the new event, the new component enqueue the event for later processing.
7.3.4
Events Handling
This scenario (showed in Section 7.10) can be seen as the continuation of the previous explained one. As mentioned in Section 7.3.3), after an event is received in a component, it is enqueued for later processing. This section explain framework work to reach to handle the entire framework event, including framework parameters, interface parameters, and the specific layer event.
Figure 7.10: Events Handling As mentioned in Section 4.5, all components (that extends from the Simple Component class) hold an individual thread for message handling. This thread is maintained asleep while its queue is empty, and dequeue events3 to be processed if any exists. From the moment an event is dequeued it is handled in different places respect to the event type and the event part being analyzed. There are accordingly defined different methods to handle events in the different component abstractions. The main strategy used to handle them is based on event selection and template methods. The first template method used handles different FrameworEvent types. It is designed in this way due to different components handle different kind of events (i.e. a LayerComponent particularly handle InterlayerEvents). At this point event framework parameters are processed. Later, using the same behavior, different next level specifications of layer components (RoutingComponent in the example) handle different LayerEvents. For this reason, a new template method is called on them to select the specific handler for the event. Finally, on obtaining it, the proper method that handle the received layer event is called, discriminating in this point messages and different layer parameters. As can be also observed, an UserToolkit parameter is passed in each method call from the LayerComponent processing, where it is instantiated4 . If the handler requires additional event information or need to modify it (being a filter component) it can ask the Toolkit interface for it. 3
Actually the event dispatcher dequeue events in a FIFO order, although it is the intention in the future to allow to select the desired scheduling algorithm. 4 Actually, Toolkit objects do not really exist, for performance purposes it is just an interface implemented by the LayerComponents. But from the user point of view this appears to be a single object instance
68
CHAPTER 7. DETAILED DESIGN
7.3.5
Components Configuration
This scenario shows how component parameters are re-configured. The interaction diagram showed in Figure 7.11 shows a manual configuration, although this behaves similarly in case of being automatically reconfigured after monitoring activities.
Figure 7.11: Components Configuration An administrator can change component configuration, introducing new values to parameters through an user interface. This interface communicates to a Management component that interprets configurations and call to various services (also it can be called by other entities like monitoring). On doing so, the Management call to configuration checkers, if any exists. These checkers analyze the new configuration respect to the actual architectural state, and throw an output of the results. They do not modify the configuration process, and they can either be called prior to actually accomplish the configuration. After analyzing the new configuration, the Management component discriminates individual configurations, and iteratively call the the ComponentConfigurator components that actually achieves the work of configuring. This ComponentConfigurator has a reference to all components, and selecting the desired one, it calls the specific getter/setter methods to materialize the configuration respect to the introduced data.
7.4
Specific Components Creation
Adding new functionality components (or building blocks) to the framework may include implementing code from scratch or reusing and encapsulate legacy code. In both cases, implementing a new component is a six steps process: 1. Define component type. 2. Decide if to include configurable strategies.
7.4. SPECIFIC COMPONENTS CREATION
69
3. Identify Data Agreements. 4. Create main class implementing component interfaces. 5. Define component configurations and subscriptions. 6. Create component bundle (optional).
7.4.1
Define component type
Setting component type is the first action to take due to they define the main component behavior. Also different classes and different configuration values are applied to them. Main Components types and Filter types are extended from the same abstract class. But they may instantiate different Type objects in their constructor methods (Filter or MainComponent). It is important to notice that this work is transparent to developer eyes, and that the framework does it respect to descriptive component configurations. Strategy components are a rather different component and are explained in next Section 7.4.2.
7.4.2
Include Configurable Strategies
In many cases, on implemented a certain functionality, it is possible to find certain sub-routines that can be implemented in various ways, respect to different technologies, algorithms, etc. The framework allows the developer to include these strategies as lightweight but still configurable components. To implement this type of components the developer must extend the StrategyComponent class, and implement just the requires getter/setter methods. Then, to use this functionality, the developer (not being aware of where this component is placed) just ask the UserToolkit interface to obtain a specific parameter result from a particular Strategy component.
7.4.3
Identify Data Agreements
To allow to interconnect components using different technologies and proprietary data types, the framework must provide a common set of data formats and types. The messages showed in Section 7.2.3 are abstract enough to include any data type. But at the moment to implement the adapter in every new component5 , the developer must know the type and format of the common data. As previously mentioned, components have a Component Class. Components from the same class should be directly interchangeable. In order to do that, they should follow the same agreements. For this reason, for each layer there have been identified some component classes, and for each of them there have been defined agreements. Components classes are extensible, and new agreements could be specified to be known and used by other developers. The following tables (one per layer) explain agreements showing interested parameters for each component class, and their data format and data types.
7.4.4
Implement Main Class
Components (excluding strategy types) must extends the specific abstract classes that refers to the layer in which the functionality is included, implementing of course all abstract methods. All 5
Although in fact the adapter can be omitted and the framework forced to transfer proprietary message for performance purposes.
70
CHAPTER 7. DETAILED DESIGN
Class
Parameter
Format
Data Type
Object XML
Event type Event type
Object reference XML ref
Object XML
Table 7.1: Application Agreements Class
Parameter
Format
Data Type
Channel-B Object-B Subject-B Content-B Concept-B
Filter Filter Filter Filter Filter
channel name object instance channel path set of name value pairs semantic expression
String Object String (allow wildcards) Collection of (String + String) String (XML)
Table 7.2: Broker Broker Class
Parameter
Format
Data Type
P2P Ants Rebeca
destination destination destination
NodeKey clannelId Event Processor
String String Event Processor class
Table 7.3: Routing Agreements Class
Parameter
Format
Data Type
Static Network P2P Channel
destination (key) destination (key) destination (key)
node id long number channelID
String String String
Table 7.4: Topology Agreements Class
Parameter
Format
Data Type
TCP UDP SOAP/HTTP
destination destination destination
IP address + port IP address + port URI + HTTP (GET or POST)
String + String String + String String + String
Table 7.5: Network Agreements
components of the same layer offers then the same defined interface (see Figure 7.12). This means that if there exists a pipeline of attached components in the same layer, one component will call the next one with the same interface that it has been called before. As shown in Figure 7.12, it is the responsibility of the last component in one layer to call the next layer first component using its specific interfaces. This behavior is transparent to framework users, and has not to be coded on creating new components. Actually the Connection object attached to each component is responsible of doing it. A component developer only has to process received events (implementing abstract methods), and to generate new ones to be sent to other components. In this way, Framework behavior is transparent to his eyes. When it its time to create and deliver a new event, the developer only need to produce a new Message, and call the UserToolkit’s createEvent and sendEvent methods. After this last method is called, the framework is responsible of managing the event and to send it other components. Having defined it descriptively, the framework binds component type main behavior and the message management model one.
7.4. SPECIFIC COMPONENTS CREATION
71
Layer A
Interface Layer “B”
Encapsulation Legacy Legacy Code Code
Layer B
Legacy Code
Interface Layer “B”
New NewCode Code Legacy components Interface Layer “C”
Layer C
Figure 7.12: Components Implementation Encapsulating Legacy Code The main component class is responsible of managing component functionality using framework data. If existent code is to be reused and encapsulated, the main component then acts also as a bridge (or adapter) that maps legacy functionality to framework terms. Defined framework messages (Section 7.2.3) and data agreements (defined in Section 7.4.3) allow to generate this mapping in two ways (or models): Transform original data types into framework types: in this first approach the main component acts then as an adapter from legacy data to a common one defined in the framework. This approach requires more processing resources but allows to obtain more reusable components to be combined with a wider set of functionalities. Simply to encapsulate original messages into framework ones: this second approach is more lightweight but limits the combination of the component to a more closer related components set. Anyway, if we are searching for performance this second approach is a good option. There are also other considerations to take in account on encapsulating legacy code. Figure 7.12 shows an organized way to encapsulate existent code. In the most generic case, framework component interfaces are to be implemented using more than one piece of code. These pieces of code could be just Java objects, or any other language code6 . In this way, the main class is responsible of holding legacy component and call their specific interfaces accordingly. Although yet being Java code, legacy components could not to be implemented as we desire (i.e. taking maybe unnecessary parameters, not returning a value or not doing just the function we 6
It is possible to call legacy code written in other languages (like C, C++, or just Assembler) using standard programming interfaces like Java Native Interface (JNI) [58]
72
CHAPTER 7. DETAILED DESIGN
desire, but only a part of it or a more complex one). For this reason, and to homogenize behavior it is recommended (if possible) to make these legacy code to implement our desired interface. This “visitor” interface could be designed to be general enough for any case receive an Object (possibly a Collection) as parameter and returning also an Object (that refers to the actual message to be transmitted). Also, the implementation of this desired interface should be responsible of managing legacy code to achieve the desired functionality. In this way, main classes are more decoupled from the legacy implementation, just referring their work to organization and encapsulation concerns.
7.4.5
Specify Component Configurations and Subscriptions
Configurations and subscriptions specify certain internal component behaviors once components are introduced into the framework. Referring on the terms explained in Sections 5.2 and 5.3, and other necessary parameters, the parameters to be defined in this step are: • The layer in which this component is included. • The type of component you are creating. • The component identifier. • The functionality class of the component. • The Service that this component will serve to, and the service order in which this component is intended to be placed. • The message model to be managed by the component. • The set of wiring restrictions (or subscriptions). These configurations are set descriptively in a configuration file. To organize all settings in a same file, the Service Binder XML configuration file has been extended to include them. But it is important to notice that it is not required to make use of the container neither Service Binder to configure components in this way. The non-intrusive extensibility of XML allows to treat same documents with different functionalities. An example of this document is showed in next section Listing 7.3.
7.4.6
Create Component Bundle
To create a bundle refers to include the container-behavior for the the component that is been created. This separation of business and management logic allows to reuse implemented components into different containers solution, or still to use no container at all. As previously mentioned, and based on the containers selection study (see Appendix B), here we only refer to create OSGi (Oscar) bundles. Creating an OSGi bundle using the ServiceBinder functionality refers mostly to generate some configurations files than to generate code. Let’s explain it in steps: 1. The first thing to do is to create an interface through which the bundle functionality is to be called by other services. This interface is provided by the framework in the FrameworkComponent class (see 7.2.1). This interface is then to be implemented by all framework components.
7.4. SPECIFIC COMPONENTS CREATION
73
Listing 7.1: Bundle Activator 1
import o r g . ungoverned . g r a v i t y . s e r v i c e b i n d e r . G e n e r i c A c t i v a t o r ;
3
p u b l i c c l a s s MyActivator e x t e n d s G e n e r i c A c t i v a t o r {
5
}
2. Create an empty class that extends a specific Oscar BundleActivator class: GenericActivator. 3. Implement the class that implements the interfaces defined in step 1. But this step has been previously done, and refers to implementing the main class, as explained in Section 7.4.4. 4. Create an OSGi manifest file. This file describes the bundle service, not only for the container but to users. As showed in Listing 7.2, this file includes the definition of the Bundle Activator file (line 1); the required imported packages (line 3); and the exported package that includes the main service interfaces (line 5). For human understanding it provides: the Bundle name (line 7); the bundle description (line 9-10); the bundle vendor (line 12); its actual version (line 14); and due to the inclusion of the service binder, it allows to include where to search the service manifest file (line 16). Listing 7.2: Manifest 1
Bundle−A c t i v a t o r : NS Framework . l a y e r s . b r o k e r L a y e r . component . r e b e c a . c o n t a i n e r B a s e d . MyActivator
3
Import−Package : o r g . ungoverned . g r a v i t y . s e r v i c e b i n d e r
5
Export−Package : NS Framework . framework . component . main
7
Bundle−Name : Rebeca Broker
9
Bundle−D e s c r i p t i o n : A s e r v i c e t h a t implements main Rebeca b r o k e r functionality
11
Bundle−Vendor : C r i s t i a n F i o r e n t i n o 13
Bundle−V e r s i o n : 1 . 0 . 0 15
Metadata−L o c a t i o n : NS Framework/ l a y e r s / b r o k e r L a y e r / component / r e b e c a / c o n t a i n e r B a s e d / metadata . xml
5. Create a ServiceBinder metadata file. This metadata file (showed as example in Listing 7.3) gives the container the required information to instantiate the service, and additionally specifies various properties to be used by the framework: line 5 specifies the main service class; line 7 inform about the main interface that the main class implements; lines 22-25 contains service properties (most of the ones explained in Sections 5.2 and 5.3). Line 27 specifies the kind of components (subscriptions) that the service being created intend to be connected to. Lines 29-33 specifies subscription parameters. The filter in line 29 specifies subscription restrictions using a LDAP [59] (Lightweight Directory Access Protocol) search filter syntax. Cardinality and the wiring policy can be specified in lines 30 and 31 respectively. Finally lines 32 and 33 inform about the methods that main services classes must contain to be notified on starting desired services. 6. Compile the classes. 7. Generate a jar file containing all previously mentioned elements: the bundle activator, the interface implementation, and both configuration files (manifest.mf and metadata.xml). The
74
CHAPTER 7. DETAILED DESIGN
Listing 7.3: Metadata.xml 2
4
6
< s e r v i c e i n t e r f a c e =”NS Framework . framework . component . main . FrameworkComponent” /> 8 10 12 14 16 18 20 22 24
One o f : MAIN FILTER STRATEGY −−> name=”compType” v a l u e=”MAIN” t y p e=” s t r i n g ”/> Java c l a s s name ( u p p e r c a s e ) −−> name=”compName” v a l u e=”REBECA” t y p e=” s t r i n g ”/> Any ( u p p e r c a s e ) −−> name=”compID” v a l u e=”REBECABROKER” t y p e=” s t r i n g ”/> Depends on Layer ( u p p e r c a s e ) −−> name=”compClass ” v a l u e=”CONTENT−B” t y p e=” s t r i n g ”/> One o f : APPLICATION BROKER ROUTING TOPOLOGY NETWORK ( u p p e r c a s e ) name=”compLayer ” v a l u e=”BROKER” t y p e=” s t r i n g ”/> One o f : PROTOCOL PIPELINE ( u p p e r c a s e ) −−> name=”messageType ” v a l u e=”PROTOCOL” t y p e=” s t r i n g ”/>
−−>
−−>
26
30 32 34
36
bundle activator is placed in the root of the jar file. The interface implementation (main class), and other related files must be organized following the package structure mentioned in Section 7.1. The manifest file must be placed into a META-INF directory on the jar root. Finally, the metadata file can be placed anywhere in the jar file, while correctly specifying it in the manifest file.
Chapter 8
Case Study: Using Franscesca Encapsulating Rebeca This chapter explain how Franscesca has been used, encapsulating Rebeca functionality into framework components, to implement a full NS. This not only gives an example of usage, but allows to validate framework interfaces and functionality separation. Joined with this example, explanations of main framework tools for using the framework are also given.
8.1
Rebeca Encapsulation
To validate Franscesca, and to show an example of its usage, a complete Notification Service has been implemented using most of this framework capabilities. This implementation makes use of most of the Rebeca [1] (see Chapter 2 for details) Notification Service functionalities. These functionalities have been encapsulated as framework components. On just doing so, these functionalities gain the advantages offered by this framework: NS life cycle management, remote deployment, runtime configurability and adaptation, extensibility, different message models, clear design, etc. (Performance comparisons using this framework are left out from the scope of this work, and are to be included in future works). To be able to do this mapping from Rebeca’s design into Franscesca’s components, an internal analysis on the former had to be done. This analysis is explained in the following subsection joined with several considerations that compare both approaches. Later it is explained the way in which encapsulations are actually achieved and implemented.
8.1.1
Rebeca Design
In a similar or different, most of the Notifications Services should refer their design on these internal functionalities. Every NS is based on at least one Broker addressing model, a related routing algorithm, and some kind of topology using some network protocol. Rebeca has been designed mainly to test interesting Content-based routing strategies and performance. Also some extensions have been been included into it enhancing its main functionality. Figure 8.1 synthetically intends to show main Rebeca design. The Rebeca-based middleware is conformed of a Broker network. More specifically it is composed of a type of broker called Router. Clients connect to these Routers through so called Lo75
76
CHAPTER 8. CASE STUDY: USING FRANSCESCA ENCAPSULATING REBECA
Router
Local Broker A
EventBroker EventBroker
RoutingEngine RoutingEngine
Local Broker B
EventBroker EventBroker
EventRouter EventRouter EvRoutingConn EvRoutingConn
EventTransport EventTransport
EventTransport EventTransport
Connection
Dependence
EventTransport EventTransport
Communication
Figure 8.1: Rebeca Design calBrokers. Figure 8.1 shows two LocalBrokers connected to a Router. To this same Router, any number of other Routers can also be attached to, conforming a static tree of routers (Rebeca do not allow cycles in its topology). The LocalBroker is composed of an EventBroker that uses an EventTransport to establish connection and to send and receive Events. The Router is composed by four main components: the EventRouter, that manages connections; the RoutingEngine that abstracts main routing strategies and contains needed Routing Tables; and two dynamic components that are created on establishing connections. Synthetically explained, inside the Router the behavior is as follows. After any connection is established by the EventRouter, it creates a bidirectional EventRouterConnection. This component is composed of an EventTransport that is the one that actually performs event communication. The EventTransport then listen to incoming Events that are later handled by the EventRouterConnection. This last component then deliver the event to the RoutingEngine to be processed. After processing it, and on generating new Events, the RoutingEngine call the processEvent() method of some specific EventRouterConnection(s). On doing so, this last component is responsible of transmitting the events, making use again of its EventTransport. As a result of this analysis, we can list several considerations that mostly refer to differences between the Rebeca and Franscesca NS approaches: • In Rebeca there is no topology or overlay abstraction. There actually exist a static Router network playing the role of topology. But its main component, the EventRouter, overlaps the Routing and Topology concepts respect to Franscesca’s definitions. • Rebeca do not allow Brokers to be established local to the EventRouter not requiring a network connection. (It allow LocalBrokers to be connected directly to a RoutingEngine but
8.1. REBECA ENCAPSULATION
77
with the aim to realize a local publish/subscribe system). • Another difference between Rebeca and Franscesca is that in the former remote clients (through LocalBrokers) connect to Routers. In Franscesca, remote clients (through equally named LocalBrokers) connect to Brokers (and not to Routers). This is due that LocalBrokers and common Brokers are hold in the same layer (Broker Layer), and they regard to the architectural layering style policies of communicating remotely only among peer layer components. But it is not due only to refer to theory concerns, but to homogenize remote and local clients (see Section 4.4.1), and to allow stand alone remote clients to be set independently of Broker Layer message filtering strategies (that are to be placed only into Border Brokers).
8.1.2
Encapsulation Decisions
The next step to take was to investigate Rebeca implementation classes, so to recognize the ones that achieve Rebeca functionalities. Later, and having in mind the division of concerns of the presented framework, it was the intention to classify Rebeca classes into the ones that better fit with each Franscesca layer. The selected approach for this purpose is showed in Figure 8.2. This figure shows a simplified but demonstrative picture of this work (Rebeca consists of near 200 Java classes, but the most relevant ones are showed). In this figure, sets represent Franscesca layers, and into them are included the classes that are to be placed into them. Set intersection intend to explain differences in this lower level analysis: the EventRouting class intersects Topology and Routing Layer behavior; also there is a Broker and Network layer intersection presented on the definition of an EventBroker subclass for a specific Network layer protocol (TCPEventBroker). Making a summary from the overall analysis, respect to Rebeca and Franscesca similarities and differences we can cite: Rebeca applications have been found to correspond nicely to Franscesca application design and used interfaces; a still more clear correspondence has been found respect to the Routing design; network design also fits into the same definitions; but some distinctions have been detected respect to the Broker and Topology design. Nevertheless, explained differences have not restricted to generate the encapsulation at all. Franscesca has been designed to be flexible enough to overcome these and still other difficulties. As a result, we are now able to present the concluding encapsulations approach, resuming explained Java classes sets into Franscesca components organized into respective layers. Figure 8.3 shows then the components that are to be implemented encapsulating Rebeca code: a Rebeca Content-based Broker, five different (configurable) Routing Algorithms offered by Rebeca, a Static Tree Overlay, and a TCP socket-based Network protocol. Additionally, it includes a generic application layer component for testing purposes that also is based on Rebeca code. Also to present a filter component type example, it is included a Channel filter that generates optimizations on achieving transmission. In the following are given details, for each layer, about the way encapsulations have been implemented and how differences have been solved.
8.1.3
Encapsulation Implementation
Applications Layer Rebeca offers some EventProducer and EventConsumer example hierarchies classes that allow to test the NS. Consumers implement a defined EventProcess interface, that joined with Filters (like
78
CHAPTER 8. CASE STUDY: USING FRANSCESCA ENCAPSULATING REBECA
Figure 8.2: Rebeca Code Classification
8.1. REBECA ENCAPSULATION
79
pub sub newPub
Application
notify
Application
Rebeca Context-B notify
Identity
Merging
Static tree send
receive
Channels Topology
Covering
forward deliver
Simple
route
Management
Flooding Routing
pub sub adv
Broker
Network (TCP) Network
Figure 8.3: Encapsulations subscriptions and advertisements) conform the tuples hold by routing tables. Both components that hold the encapsulation for consumer and producer respectively only refer to map Rebeca subscriptions and publications filters into Framework definitions. Also the framework is charged on assigning a destination to the running applications. From the defined application layer common interfaces, in these encapsulations are implemented: the Notify() method for obvious reasons; no exceptions are sent from Brokers to Applications in Rebeca so the interface defined for this purpose, HandleException(), is not used. Also, no reliability protocols are implemented in Rebeca, so defined interfaces (Response(), Confirm()) do not need to be implemented. Neither connection adaptation is managed in this NS, so Change() and Update() interfaces do not need to be implemented. The configured properties and subscriptions for both components are very similar, just differing on component identifier and component name. These values are explained in Listing 8.1. Listing 8.1: Application Configuration 1 3 5 7 9 11 13 15
< s e r v i c e i n t e r f a c e =”NS Framework . framework . component . main . FrameworkComponent” />
4 6 8 10
12 14 16
4 6 8 10
12 14 16
4 6 8 10
12 14 16
4 6 8 10
12 14 16
