Coordination Middleware for Decentralized Applications in Dynamic Networks Kurt Schelfthout
∗
K.U.Leuven Dept. of Computer Science - Distrinet Celestijnenlaan 200A 3001 Leuven, Belgium
K.U.Leuven Dept. of Computer Science - Distrinet Celestijnenlaan 200A 3001 Leuven, Belgium
[email protected]
[email protected]
ABSTRACT The Ph.D. work presented in this paper describes novel middleware abstractions for the support of decentralized applications in dynamic networks. Decentralized applications are characterized by the absence of an application component that has global control; a network is dynamic if its composition changes frequently and unexpectedly over time. In such a domain, application components are necessarily spread over the network nodes and need to coordinate among each other to achieve the application’s functionality. The goal of the Ph.D. research is to support this coordination by suitable middleware abstractions. We describe two prototypes that were built with this goal in mind. First, a middleware supporting views, abstractions for representing and maintaining context information in a mobile ad hoc network is presented. A second middleware, that enables protocol-based interaction in mobile networks by supporting roles as a first order abstraction, is described. The application of this second middleware in a real world case study of automatic guided vehicle control is presented, showing its usefulness. Ongoing and further research in this area is discussed.
Categories and Subject Descriptors D.2.11 [Software Engineering]: Software Architectures—Domainspecific architectures ; C.2.4 [Computer-Communication networks]: Distributed Systems—Distributed Applications
Keywords middleware, mobile networks, roles, coordination
1.
Tom Holvoet
INTRODUCTION
Recent years have seen the advent of distributed applications, such as applications in mobile and ad hoc networks, that share a ∗advisor
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. 2nd International Doctoral Symposium on Middleware ’05 November 28December 2, 2005 Grenoble , France Copyright 2005 ACM 1-59593-267-4/05/11 ...$5.00.
combination of two properties: they are decentralized, and they are deployed in a dynamic network. As will be showed shortly, the combination of these two properties makes for a challenging application domain, especially with respect to the concern of coordination: application components are necessarily spread over the network and need to coordinate among each other in order to reach the overall goal of the application. To motivate the goal of this Ph.D. work - the development of appropriate middleware abstractions for coordination of application components - we discuss both dynamic networks and decentralized applications in more detail. A network is dynamic if its composition changes frequently over time. One reason for these dynamics can be that the network nodes are mobile, and application components are then typically coordinating with a changing subset of nodes that catches their interest. For example, a robot avoiding collisions is only interested in coordinating with robots that are close to it. Furthermore, due to the unreliability of the nodes themselves, a node may virtually disappear at any time, e.g. when its battery is down. Another cause of network dynamics is the unreliability or the constraints imposed by the communication infrastructure. For example, in an ad hoc network, senders have a limited communication range, and may move in and out of each other’s range at any time. An application is decentralized if there is no application component that has global control over the application. All components work together as peers: they are collectively responsible for the functioning of the application. One reason for choosing such a decentralized architecture, can be that deployment of the application requires decentralization: there is simply no fixed infrastructure available to deploy a critical component, e.g. in an ad hoc network. As another reason, scalability and anonymity requirements may impose a decentralized architecture, such as in an anonymous information sharing application (e.g. Freenet [25]). In an application domain where these two properties are combined, the coordination among application components becomes at the same time more complicated (due to the dynamics) and more important (due to the decentralization). One can thus expect that the application developer is supported in his or her task by an appropriate coordination middleware. This eliminates tedious and repetitive tasks, provides a good model around which to structure (part of) the application, and prevents many common mistakes and pitfalls by providing a good general purpose implementation for common operations. Thus, the goal of this research is to propose useful abstractions for the coordination of application components, and to support these abstractions through a suitable coordination middleware. The work presented here focusses specifically on decentralized
applications in mobile networks and mobile ad hoc networks. Specific application examples include vehicle control, traffic control, collaborative applications in ad hoc networks and active network management. In Sect. 3.3, we describe a real world application involving the control of automatic guided vehicles in more detail. The paper proceeds as follows. First we describe existing coordination middleware approaches in our chosen application domain in Sect. 2. Then, we describe two coordination middleware prototypes that were built in the course of this research, as well as a case study in which the middleware is applied, in Sect. 3. Finally, we draw conclusions.
2.
STATE OF THE ART
Existing coordination middleware for mobile networks can be roughly divided in two families: publish/subscribe middleware [12] and tuplespace-based middleware [13]. In the publish/subscribe (P/S) family, publishers send notifications of state changes to a list of subscribers. Subscribers are not known in advance but let the P/S middleware know of their interest in certain notifications through subscriptions. Typically, subscriptions are defined by a filter on the notifications’ contents. Research has discussed extensions of the principle of publish/subscribe for mobile networks, and for dealing with unreliable connections. In the Java Messaging Service [22], dealing with disconnections is done through durable subscriptions , which are stored and can be activated by re-subscribing. JEDI [4] deals with disconnections and mobility by defining an explicit moveIn and moveOut operator. A fixed dispatching server then buffers relevant subscriptions and forwards them upon reconnection. This is of course problematic in the case where clients do not know whether they are going to move or not. Mobility extensions of Siena [8] follow a similar approach. Elvin [23] offers a proxy server that maintains a connection with publishers on behalf of a mobile client. Clients then need to reconnect to this proxy server in order to receive buffered events. [3] describes that, due to the appearance and reappearance of hosts, there is a phase immediately after reconnection in which a mobile subscriber needs to subscribe to events and wait until some events get fired until it can reassess the current state of the network. Buffers and virtual clients are introduced to subscribe to past events and events in future locations respectively. In [5] locationdependent subscriptions are introduced to exploit the event-based paradigm in mobile applications. Location-dependency refines a subscription to accept only events related to a mobile user’s current location. STEAM [12] also deals with mobility by adding the possibility to filter notifications on the publisher’s location, in addition to the contents. Scopes [6] are an extension for publish/subscribe systems that structures publishers and subscribers by reducing visibility. Only subscribers in the same scope as a publisher can see events from that publisher. A scope can also be dynamic by marking all events in a certain session with a tag - this can be viewed as tagging all the events in an interaction. Scopes have also been applied to sensor networks, [21], as a generic abstraction for a group of nodes. In the tuplespace (TS) family, application components manipulate a shared collection of data objects, called tuples, to communicate. Typical operations on a tuplespace are putting, taking and reading tuples. The latter two operations also use a filter (in TS, called a template) on the tuples’ contents to determine which tuples to read or take. In a mobile network, a single shared space does not exist, so it is distributed over the nodes. Extensions of TS for mobile networks support additional mechanisms to gather tuples from tuplespaces on remote nodes. For example, LIME [13] makes the tuplespaces on any given node accessible to components
on connected nodes. EgoSpaces [9] allows application components to specify exactly from which nodes tuples must be gathered using “views”. A view declaratively describes a set of tuplespaces on remote nodes, by specifying a constraint on the location of the nodes. An example is “tuplespaces from all nodes that are less than 20 meters away”. An application component can then manipulate the tuplespaces in the view as if it was one shared tuplespace. For example, a component can execute a “take” on a view defined as “tuplespaces from all nodes that are less than 20 meters away”. The middleware then transparently searches the network for tuplespaces on nodes matching the view definition, executes the take operation on each of those, and finally returns the result to the application component. TOTA [11] takes a different approach: instead of gathering tuples through views, an application component can inject a tuple in its local tuplespace, together with some propagation rules. The TOTA middleware then transparently propagates the tuples to neighboring nodes, and maintains the distributed tuple as the topology of the network changes.
3.
TWO MIDDLEWARE APPROACHES
This section describes two middleware approaches that support coordination in dynamic networks. The first proposes an abstraction for gathering information from application components on neighboring network nodes in mobile ad hoc networks, allowing application components to be context-aware. The second approach is a continuation and generalization of the first, and proposes a middleware that supports the execution of interaction protocols in mobile networks. Afterwards, we discuss validation in a real world case: automatic guided vehicle control.
3.1
Views: Abstractions for Context-Awareness
An application component deployed on a node in a mobile ad hoc network (MANET) needs information from application components on other nodes in order to fulfill its task - it needs information about its context. However, due to the physical mobility of the nodes, and the limited communication range of an ad hoc network, any node may leave or enter in a node’s transmission range at any time, and so a component’s context changes continuously. Consequently, a coordination middleware (called ObjectPlaces) was proposed that supports a first order abstraction of an application component’s context, called a view [17]. For our purposes, an application component’s context is the aggregate of all available information on currently reachable nodes in the MANET. An application component’s context thus changes because (1) information on a reachable node is changed, or (2) the set of reachable nodes changes. In ObjectPlaces, an application component can gather context by defining a view. A view is built by the middleware based on a declarative specification that describes how far over the network the view reaches, and in what information the viewing component is interested in. The most important property of a view is that it is an actively maintained structure, that changes as the component’s context changes. The component can listen for new context events, e.g. the arrival of a new node carrying interesting information. Technically, the middleware works as follows. Each application component can maintain “viewable data” in a local collection of objects, which is basically a tuplespace variant [1]. The objects in this local tuplespace represent information that is of interest to others, and so contribute to the overall context information available. Other application components can gather copies of objects from tuplespaces on nodes in their neighborhood using a view. A view is a local collection of objects, reflecting the contents of multiple tuplespaces on reachable nodes in the network based on
(a)
(b)
(c)
Figure 1: Circles denote nodes, ovals denote tuplespaces, and the rectangle denotes a view. The other shapes represent data objects. (a) A viewing node declares a new view. The circle denotes the bound on the distance metric from which the middleware will gather data for this particular view. (b) The middleware has gathered the appropriate data, and the view is built. (c) A new node enters the zone, so the view is updated with the new data object that the node carries. a declarative specification. This collection is continuously updated by the middleware, both with respect to changing contents of the tuplespaces in the view, as with respect to changes in the network topology. A view thus represents the context of the viewing component. To build a view, a component specifies two parameters. First, a distance metric and a bound, that determine how far over the network the clients view will reach. An example is a physical distance with a bound of three meters: the view will span tuplespaces on nodes within three meters from the node where the view is built. Second, an application gives an object template that constrains what objects will be included in the view. Given these parameters, the middleware searches the network for nodes satisfying the constraints given by the application, gathers copies of the relevant data objects and returns them to the application in the form of a local collection of objects, which can take the form of any convenient data structure. Additionally, the view is actively maintained by the middleware as the network changes or as the content of the tuplespaces involved in the view changes. A graphical representation of this process can be found in Fig. 1. In order to support this middleware, a distributed protocol was developed that enables the construction of a view in a mobile ad hoc network. The protocol basically builds a minimal spanning tree over the part of the network delimited by the distance metric of the view, and uses this tree to route the gathered data to the builder of the view (which is the root of the spanning tree). Additionally, the correctness and performance of the middleware was evaluated in a simple abstract scenario. See [17] for more details. Contributions. A view is a convenient and general-purpose abstraction for coordination in MANETs, since it is an active representation of a component’s context. Both the contents and the representation of a view can be tailored to the needs of a particular application component. The concept of view is an extension of an earlier, similar proposal in [16]. The difference is that a view here is (1) actively maintained, and (2) can be tailored to an application component’s wishes (i.e. can take the form of any convenient data structure). If the middleware does not support an active context representation, the application programmer is forced to program such a representation using polling.
3.2
Roles for Protocol-based Coordination
Closer examination of the view-based middleware, and confrontation with the real world problem described in Sect. 3.3, revealed that our existing coordination middleware for mobile networks focusses on applications where the main problem is gathering (or spreading) information. However, in a broader class of applications, coordination is concerned with maintaining behavioral relationships between application components. This relation needs to be maintained by the exchange of multiple, related messages, i.e. a protocol. To support protocol-based coordination, information spreading and gathering is a necessary, but not sufficient condition. For example, consider collision avoidance between between automatic vehicles. Collision avoidance needs complex coordination mechanisms: it is a mutual exclusion problem, for which a distributed protocol is needed. In such problems, existing coordination middleware for mobile networks can fruitfully be used as a discovery mechanism, but after that, the protocol itself is handled “outside” the middleware. This has as an unfortunate effect that all support from the middleware for dealing with the dynamics in the network is lost at the protocol level. To mitigate this situation, a new middleware (although based on the views-based middleware) was developed that offers roles as first order abstractions of the behavior of one side in an interaction protocol. This is inspired by ongoing work in object-oriented software engineering, where roles are used to model object collaborations. This has lead to usages of roles in framework design [15], implementation support [26], patterns [7] and languages [24]. Roles are also used in protocol-based interaction in multi-agent systems [10] [20], allowing the description of inter-agent interaction separately from agent behavior. In the middleware presented here, a role in an interaction can be activated manually by application components, or is activated automatically by the middleware if a node needs to participate in an interaction started on another node. More in detail, when an application component activates a role, the middleware finds components on other nodes that need to play one of the other roles in the interaction, and activates them. This activation is (similarly to views) based on a distance constraint; for example, for an interaction consisting of an initiator and a participant, the initiator role can activate participant roles on nodes within a certain distance. After
(a)
(b)
(c)
Figure 2: (a) A role, called initiator, is activated. The circle denotes the zone in which participants for this particular interaction should be activated. (b) The middleware activated the necessary participants in the protocol. The protocol can begin execution. (c) A new node enters the zone, so a new participant role is instantiated on that node. The initiator role is notified of this new participant, and the new participant starts partaking in the protocol as well. activation, the roles can start to execute the interaction protocol on behalf of the application components. The middleware monitors the network actively: when nodes leave or enter the zone of interest, roles are activated or safely deactivated respectively. Also, the roles already in the interaction are notified of departure or arrival of roles, so they can react appropriately (e.g. by re-sending messages, or updating their information). The process of activating roles is illustrated in Fig. 2. Note that this middleware is more general than the views-based middleware: instead of searching the network for nodes offering certain data, the network is now searched for nodes offering certain roles. It is thus possible to define an interaction that supports the gathering of data objects, similarly to building a view. Roles also need to coordinate locally on one node, both with other roles and with application components on that node. For this local coordination, a tuplespace is used where components and roles can put, take and read necessary information. A tuplespace was chosen because it hides the identity of the processes involved, and provides 1-to-n communication. Both are necessary properties since roles may be activated and deactivated at any time, so uncoupling between roles and between roles and application components is desirable. A prototype of this middleware was implemented in C# on .NET. More details about its working, as well as implementation issues can be found in [18]. The middleware was applied in a real world case study involving automatic guided vehicle (AGV) control, discussed next. We also discuss contributions of this second middleware at the end of the next section.
3.3
AGV Case Study
Approximately one year and a half ago, our research group started a project in close cooperation with Egemin 1 , a manufacturer of AGV transportation systems. An AGV transportation system uses unmanned vehicles that are custom made to be able to transport various kinds of loads, from raw materials to completed products. AGVs can move through a warehouse by following a predefined path on the factory floor, guided by a laser navigation system, or by magnets or cables that are fixed in the floor. An AGV can pick up a load at a certain location and drop it at another location. An AGV 1
http://www.egemin.com
can also park at particular locations when it is idle and charge its battery at a charging station. The main functionality the AGV system should perform is handling transports, i.e. moving loads from one place to another. Transports are typically generated by logistic management programs, other logistic machines or operators. AGVs are traditionally controlled by a central server that gives orders to the AGVs through a wireless network. The goal of the research project is to decentralize the control to the vehicles themselves. This eliminates the need of a central server, giving an important relative cost benefit for installations with a small number of AGVs. Also, it shifts the load of the central server to the AGVs (which need to be equipped with processors anyway in order to steer the AGV), so improving usage of computing resources, which should give better scalability. The AGV application constitutes an interesting case study for the application of coordination middleware for mobile networks. First, the moving AGVs constitute a mobile (and thus dynamic) network. Second, an important part of the AGV application deals with complex and real world coordination problems: AGVs must for example avoid collisions with other AGVs, detect and solve deadlocks, and divide transports among them. More information about the AGV case can be found in [27], and demonstration movies of the prototype implementation can be found at http://emc2. egemin.com. The proposed role-based middleware is used for supporting several necessary interactions in the AGV case: a mutual exclusion protocol for collision avoidance (based on [14]), deadlock detection and resolution (based on [2], and assignment of transports to AGVs, based on the Contract Net protocol [19]. Contributions. To the best of our knowledge, there is no similar proposal to use roles in order to support protocol-based coordination in mobile networks. In our experience, the middleware provides invaluable support for the design of interactions for two main reasons. First, it makes dealing with mobility easier by activating and deactivating necessary roles in the interaction automatically. Second, because new roles are activated for each interaction session, several concurrent interaction sessions are cleanly separated. As a result, tedious and repetitive tasks such as management of the ids of current interaction partners, is taken care of by the middleware.
4.
CONCLUSIONS AND FUTURE WORK
The goal of the presented Ph.D. research is to propose generic, useful abstractions for the development of decentralized applications in dynamic networks. Focussing on one of the most difficult concerns in such domains, coordination, we have proposed two such concepts: views and roles. For each, a prototype middleware was built and evaluated in a real world application. At the moment, we are working on further refinements of the role-based middleware, most notably to allow the application developer to specify policies that determine how an interaction reacts to changes in its composition (e.g. a new role on a new node enters the interaction). These should be specified as a separate concern from the interaction itself, increasing reusability and adaptivity of the interaction. Another track that is currently pursued is the description of the interface of the roles towards other roles and towards application components with which it is composed. Currently, dependencies between roles are solved by “manually” exchanging data objects through a shared tuplespace, but it would be better if a structured approach were supported that provides some guarantees concerning correctness of the composition.
Acknowledgements This research is supported by the Flemish Institute for Advancement of Research in Industry.
5.
REFERENCES
[1] N. Carriero, D. Gelernter, and J. Leichter. Distributed data structures in linda. In Proc. 13th ACM Symposium on Principles of Programming Languages, 1986. [2] K. Chandy, J. Misra, and L. Haas. Distributed deadlock detection. ACM Transactions on Computer Systems, 1(2):143–156, 1983. [3] M. Cilia, L. Fiege, C. Haul, A. Zeidler, and A. P. Buchmann. Looking into the past: enhancing mobile publish/subscribe middleware. In Proceedings of the 2nd international workshop on Distributed event-based systems, 2003. [4] G. Cugola and H.-A. Jacobsen. Using publish/subscribe middleware for mobile systems. ACM SIGMOBILE Mobile Computing and Communications Rev., 6(4):25 – 33, 2002. [5] L. Fiege, F. C. G¨artner, O. Kasten, and A. Zeidler. Supporting mobility in content-based publish/subscribe middleware. In Proceedings of the ACM/IFIP/USENIX International Middleware Conference, 2003. [6] L. Fiege, M. Mezini, G. M¨uhl, and A. P. Buchmann. Engineering event-based systems with scopes. In Proceedings of the European Conference on Object-Oriented Programming (ECOOP), 2002. [7] M. Fowler. Dealing with roles. supplemental information to Analysis Patterns, Addison- Wesley, 1997. [8] P. Inverardi, M. Caporuscio, and P. Pelliccione. Formal analysis of clients mobility in the siena publish/subscribe middleware. Technical report, Department of Computer Science, University of L’Aquila, 2002. [9] C. Julien and G.-C. Roman. Supporting context-aware interaction in dynamic multi-agent systems. In Environments for Multi-Agent Systems, First International Workshop, Revised Selected Papers, LNAI 3374, 2004. [10] E. Kendall. Role modeling for agent system analysis, design, and implementation. IEEE Concurrency, Agents and Multi-Agent Systems, 8(2):34–41, 2000.
[11] M. Mamei and F. Zambonelli. Self-maintained distributed tuples for field-based coordination in dynamic networks. In The 19th Symposium on Applied Computing (SAC 04), 2004. [12] R. Meier and V. Cahill. Exploiting proximity in event-based middleware for collaborative mobile applications. In Proceedings of the 4th IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS’03). Springer-Verlag Heidelberg, Germany, 2003. [13] A. Murphy, G. P. Picco, and G.-C. Roman. Lime: a middleware for physical and logical mobility. In Proc. of the 21th International Conference on Distributed Computing Systems (ICDCS-21), May 2001. [14] G. Ricart and A. K. Agrawala. An optimal algorithm for mutual exclusion in computer networks. Commun. ACM, 24(1):9–17, 1981. [15] D. Riehle and T. Gross. Role model based framework design and integration. In Proceedings of the 1998 Conference on Object-Oriented Programming Systems, Languages, and Applications, 1998. [16] G.-C. Roman, C. Julien, and Q. Huang. Network abstractions for context-aware mobile computing. In Proceedings of 24th International Conference on Software Engineering, pages 363–373, 2002. [17] K. Schelfthout and T. Holvoet. Views: Customizable abstractions for context-aware applications in MANETs. In Workshop on Software Engineering for Large-Scale Multi-Agent Systems, 2005. [18] K. Schelfthout, D. Weyns, and T. Holvoet. Middleware for protocol-based coordination in dynamic networks. In Proc. of 3rd International Workshop on Middleware for Pervasive and Ad-Hoc Computing (MPAC05), 2005. [19] R. G. Smith. The contract net protocol: High level communication and control in a distributed problem solver. IEEE Transactions on Computers, C-29(12):1104–1113, 1980. [20] E. Steegmans, D. Weyns, T. Holvoet, and Y. Berbers. A design process for adaptive behavior of situated agents. In Agent-Oriented Software Engineering V. Springer-Verlag, 2005. [21] J. Steffan, L. Fiege, M. Cilia, and A. Buchmann. Scoping in wireless sensor networks: A position paper. In Proceedings of Workshop on Middleware for Pervasive and Ad Hoc Computing, 2004. [22] Sun Microsystems, Inc. Java message service spec. 1.1, 2002. [23] P. Sutton, R. Arkins, and B. Segall. Supporting disconnectedness - transparent information delivery for mobile and invisible computing. In Proc. of CCGrid, 2001. [24] T. Tamai, N. Ubayashi, and R. Ichiyama. An adaptive object model with dynamic role binding. In Proceedings of International Conference on Software Engineering, 2005. [25] The Freenet Homepage. http://freenetproject.org, 2002. [26] M. VanHilst and D. Notkin. Using role components to implement collaboration-based designs. In Proceedings of the 11th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pages 359–369, 1996. [27] D. Weyns, K. Schelfthout, T.Holvoet, and T. Lefever. Decentralized control of E’GV transportation systems. In Proceedings of AAMAS 2005 - Industry Track, Utrecht, The Netherlands, 2005.
