As mechatronic systems are often safety-critical, the inherent flexibility of such .... ample taken from a self-optimizing version of the software for the railcab ...
Multi-Agent System Design for Safety-Critical Self-Optimizing Mechatronic Systems with UML Holger Giese, Sven Burmester , Florian Klein , Daniela Schilling , and Matthias Tichy Software Engineering Group, University of Paderborn, Warburger Str. 100, D-33098 Paderborn, Germany
[hg|burmi|fklein|das|mtt]@upb.de Abstract. One of the concepts proposed for controlling and optimizing the complex mechatronic systems that will emerge when mechatronic components formerly operating in isolation become interconnected is agent-oriented software. As mechatronic systems are often safety-critical, the inherent flexibility of such software seems to be at odds with the need for thorough validation and verification, however. The presented approach resolves this conflict by means of a specific UML based design approach for safety-critical multi-agent macro- and micro-architectures. The macro-architecture ensures that all local interactions between agents are governed by a set of social rules, termed as culture, specified by means of patterns and roles. Employing compositional verification techniques, we can then preclude hazards addressed by the cultures for the entire system. In the micro-architectural view, agents can then be specified by refining the verified roles. Within the scope of correct refinements, the agents may employ selfoptimization techniques without invalidating the guarantees about safe system behavior made by the patterns.
1 Introduction The design of mechatronic products [1], characterized by a tight integration between mechanical, electrical, and software engineering, is a research area rapidly gaining attention in these disciplines. The specified behavior of individual mechatronic components is traditionally brought about by software operating at a local level. A fascinating new idea is to interconnect such local operating islands into networked mechatronic products, more aware of their context and thus capable of autonomous and flexible reactions to changes in environmental and operational conditions. To handle the complexity of such systems, the proposed approach relies on selfoptimization, characterized by the fundamental concepts of adaptation, planning and learning. On the other hand, mechatronic systems are often subject to real-time and safety-critical requirements that make a proper analysis indispensable. The worst-case real-time characteristics w.r.t. deadlines must be predictable, and appropriate means for validation and/or verification are required in order to assess the safety hazards the projected system might contain. The presented approach addresses the apparent contradiction between striving for predictability and self-optimization by introducing a specific style of multi-agent macro- and micro-architectures.
This work was developed in the course of the Special Research Initiative 614 - Self-optimizing Concepts and Structures in Mechanical Engineering - University of Paderborn, and was published on its behalf and funded by the Deutsche Forschungsgemeinschaft. Supported by the International Graduate School of Dynamic Intelligent Systems.
Adapting agent-oriented designs to dynamic real-world contexts is currently a very active research area (see e.g. [2]). Generally recognized benchmarks such as the RoboCup [3] demand real-time reactions in a complex environment and have inspired sophisticated and flexible solutions. In contrast however, artificial intelligence techniques as well as dynamic software reconfiguration are not recommended in standards for safety-critical systems such as the IEC 61508 (cf. [4, p. 90]). For many industrial applications, failure of a critical component is simply not an option, which makes the potential emergence of unpredicted behaviors a rather undesirable property. Consequently, our approach may appear to be rather restrictive with respect to the autonomous behavior of each individual agent. As we cannot allow self-optimization to have uncontrolled effects on safety-critical aspects of the system, the behavioral norms governing these aspects are designed strictly offline (see e.g. [5], [6], [7] in contrast to e.g. [8], [9]). The macro-architecture of a multi-agent system usually just provides a foundation for the proper coordination of the agents (cf. [10, 11]). The macro-architecture we propose additionally ensures that the interaction between agents conforms to a set of patterns and roles specified by the local culture. The individual patterns are specified using an extended real-time variant of UML. They allow automated compositional verification1 , precluding possible hazards. The state explosion problem is avoided by limiting descriptions to interactions with local scope, abstracting from complex system behaviors by means of non-deterministic choices and bounds. Still, our approach is not confined to primitive behaviors: it is non-restrictive and offers the option to add arbitrary layers of increasingly complex refining patterns to the verified design. The resulting structure and relationships between the behavioral patterns are explicitly documented, again using a fairly standard UML notation. Note that we employ UML as a formal specification language with precise semantics: Prominently using story patterns [13], an extended type of UML collaboration diagram, the open source UML CASE tool Fujaba2 is able to generate executable programs from purely graphical specifications. By introducing the appropriate stereotypes, we therefore strive to provide a precise but accessible description of a multi-agent system’s social structure that is grounded in established modeling techniques from the software engineering domain. Due to its high level focus, our notation should not be seen as competing against Agent UML ([14] and [15]). The ability to refine a verified design without invalidating its guarantees concerning safe system behavior is the foundation of the micro-architecture. Within its bounds, UML modeling techniques permit the development of flexible agents. Necessarily though, restrictions to the influence of adaptation, planning, and learning on the system’s runtime behavior apply. Furthermore, as these activities are often time-consuming and require unpredictable amounts of computation time, we need a strict separation between them and the safety-critical real-time behavior. The low-level hard real-time control system guarantees proper emergency reactions, not unlike reflexes in the human nervous system. The unpredictable high-level components of each agent are free 1
2
A detailed description of our verification technique and an extensive review of related work both concerning the modeling of real-time systems and compositional verification can be found in [12]. www.fujaba.de
to modify the real-time behavior, as long as the changes respect the behavioral frame set by the macro-architecture. Our approach to decoupling real-time control and self-optimization is similar to CIRCA [16]. The architecture of CIRCA consists of a reactive real-time subsystem dealing with control-level problems and an unpredictable AI subsystem reasoning about task-level problems. While CIRCA places the focus on the safe operation of a single autonomous agent, the presented approach additionally addresses the macro-architecture level of multi-agent systems and ensures that interactions can only result in configurations classified as safe. We illustrate the proposed macro- and micro-architecture using a simplified example taken from a self-optimizing version of the software for the railcab research project3 . The railcab system employs a passive track system and intelligent shuttles that operate individually and make independent and decentralized operational decisions. The project’s vision is to combine the comforts of individual traffic, such as flexible scheduling, on-demand availability and customized cars, with the cost and resource effectiveness of public transportation. The control infrastructure of the shuttle-based transportation system is based on satellite positioning and a wireless communication network for communication between shuttles and stationary installations. Within this paper, we focus on the possiblity of safety-critical communication failures and provisions for appropriate fail-safe behaviors: Shuttles register with so-called registries that keep all shuttles in the local scope updated about other shuttles’ velocities and positions. In case of a registry failure, emergency braking maneuvers seem to be the only reasonable reaction. If a single shuttle fails, the remaining shuttles are informed via the periodic position updates and may then react accordingly. In the remainder of the paper, we first review the proposed macro-architecture style. In Section 3, the micro-architecture and its relation to the macro-architecture are presented. The paper closes with some final remarks and an outlook on future work.
2 Macro-architecture The proposed macro-architecture style provides a comprehensive perspective for viewing mechatronic systems and enables a top-down approach to their specification. In order to handle the inherent complexity, system level requirements are subsequently decomposed and translated into operative goals which are assigned to an appropriate number of communities charged with their implementation. Communities are governed by a set of verifiable patterns and corresponding roles defined by their culture. The rules expressed by the patterns ensure the desired behavior at the local level, while the social structure guarantees their proper composition and overall compatibility. In the following subsections, we first define the required physical and social structures and then proceed with the detailed presentation of one of the patterns used by the communities we introduce. 2.1 Ontology, requirements, topology and decomposition As the first step of the design process, a physical model of the system environment is defined using traditional software engineering formalisms, i.e. UML structure diagrams 3
http://www-nbp.upb.de/en/index.html
(see Figure 1). This model serves as an ontology for the further discussion of requirements and behaviors. Directly using the elements of the domain model to express goals and constraints has the double advantage of precise semantics and making a seamless translation into runtime checks trivial. This could be done in conjunction with more abstract requirements engineering methodologies such as KAOS [17] - nothing precludes their use for the initial analysis and decomposition. For our example, a more straightforward approach is sufficient. While the railcab projects also aims at energy efficiency and optimized traffic flow, the safety of the passengers is of paramount importance and therefore the focus of this section. The patterns introduced below are built around provisions for verifiable fail-safe behavior, even though they may also pertain to optimization.
Fig. 1. UML class diagram defining the physical model’s ontology
The topology of the railway network is defined by a set of directed track sections and the nodes that connect them. Shuttles are seen as physical agents operating within this environment. They interact with the distributed traffic controllers, the registries, via a wireless link. At the system level, there also exists a global topology server responsible for initially informing the shuttles about the track layout, a process that we shall not deal with in detail here. On the basis of this physical model, safety-critical hazards that are to be precluded can now be defined in terms of its elements using story patterns. A collision is imminent when two shuttles on the same or adjacent track sections show a velocity differential that is large in comparison to the difference between their relative positions. It has occurred when the distance between two shuttles’ absolute positions no longer exceeds their respective physical dimensions. A more complex goal like striving for energy efficiency could be equated to minimizing the decrease in battery charge while traversing a specific track section. As a direct global implementation of complex system goals is often impractical, our approach allows for the decomposition of a system into manageable subsections. Each physical area of responsibility corresponds to a community of agents on the conceptual level. Once a desirable property has been proven for the individual communities, compositional rules ensure it equally holds for their conjunction. If used at the system level, the projected cooperative method of collision avoidance would require every shuttle to continuously keep track of the position of every other shuttle in the system. Aside from theoretical issues concerning its verification,
affiliation area
affiliation area
critical area
critical area
Node
Track Section
Registry
Fig. 2. Instance diagram presenting a subsection of the system’s topology
this solution is not technically feasible in terms of bandwidth and memory consumption for the shuttles’ embedded controller. By dividing the system into smaller control areas, we can focus on keeping each individual area collision-free. Each control area is a physical subsection of the overall system, containing all tracks, registries and shuttles within. On a conceptual level, physical presence in an area establishes membership in the corresponding community entrusted with its safety. To avoid safety issues with shuttles passing between control areas, each area defines an affiliation area, which triggers membership within the community, and a critical area, within which the guarantees made by the community hold. While the affiliation areas overlap, the system’s division into critical areas is both disjunctive and total (see dotted lines in Figure 2). Thus, a (domain specific) proof for the critical areas extends to the entire system Railcab System
System Area
CA1 : Control Area affiliation area T1 : TrackSection
Area I
Area II
Area III
Fig. 3.1. Instance diagram presenting the social structure of the system
located on
is member S1 : Shuttle
Control Area
Fig. 3.2. Definition of community types
Introducing the system area as an encompassing community containing all agents, it is possible to construct a hierarchical instance diagram structuring all community instances (see Figure 3.1). For the discussion of realistically sized systems, it seems preferable to be able to reason in terms of a smaller number of abstract community types (system/control area), however. These templates define the common physical properties of a set of community instances, notably the physical requirements for membership in such a community. These conditions are formalized by the story pattern in Figure 3.2. Story patterns illustrate graph grammar formulae and may be used as instance situation templates that are matched against the system state at runtime. They can express simple structural constraints, but are actually expressive enough to precisely describe the effects of functions. The actual behavioral patterns and invariants a community guarantees, e.g. the presence of a topology server reacting to requests
about the track layout in the system area, are adopted by means of the commitment to a culture. 2.2 Specification of communities Controlled Area
{true}
Registration
1
Registry
0..n
Shuttle
{instantiation delay < 40ms}
.
{|T1.length-S1.relativePosition|+S2.relativePosition| < dmin} S1 : Shuttle
Distance Coordination
S2 : Shuttle
+located on T1 : TrackSection
{instantiation delay < 40ms}
+located on :Node
:TrackSection
{|S1.relativePosition-S2.relativePosition| < d min} S1 : Shuttle
+ends at
S2 : Shuttle
+begins at :TrackSection +located on
+located on
Fig. 4. Social diagram defining a control area’s culture
Cultures provide an abstract definition of a community’s internal workings. They define structural invariants in the form of compulsory roles. These roles are implied in different behavioral patterns which may be governed by sets of specific associated preconditions and impose certain constraints. They are documented by a novel kind of diagram, the social diagram. It closely builds on established UML notations and merely introduces culture as an extension of the UML subsystem stereotype. Figure 4 displays the definition of the culture of a controlled area using a centralized registry. The culture calls for the presence of exactly one registry and an arbitrary number of shuttles in the community. Each member of the community is unconditionally required to execute the registration pattern. The pattern imposes temporal constraints concerning the acceptable delay between the fulfillment of the necessary preconditions and its successful instantiation. In conjunction with an appropriate layout of the affiliation area, this ensures that no shuttle can reach the critical area without first properly registering with the centralized control. This behavior is extended by the distance coordination pattern [12], which guarantees that shuttles will properly function together as convoys. On a purely functional level, it ensures that shuttles in close proximity establish and maintain a safe distance. At the same time, it induces shuttles to behave in an energy-efficient manner by reducing their overall air resistance. The pattern is triggered whenever the distance between two shuttles traveling in the same direction, either on the same or on adjacent tracks, falls below a certain minimum, as formalized by the attached story patterns. Note that the distance coordination pattern depends on the registration pattern. The distance coordination pattern extends the context information available to a shuttle with position and velocity data about other shuttles, whereas the registration pattern only depends on the data locally available to a shuttle and the initial topology of the system. Using
bootstrapping in this manner, additional more complex aspects such as traffic flow can be addressed by using the information made available by lower level patterns. 2.3 Specification of patterns In the previous design steps, we defined a set of communities, their cultures, patterns defining possible interactions between community members and rules governing their instantiation. Below, we describe a detailed UML specification of these interactions, using the registration pattern as an example. The obligation to register before entering a critical area and the continuously broadcast updates enable the shuttles to recognize possible dangers and conflicts in time to resolve conflicts and avert dangers. Communication between the registry and the registered shuttles is therefore crucial to the safety of the system. This Inverse Look-Up approach differs from look-up services [18], where clients search the network for a specific service, as these do not address the need for fail-safe behavior in case of service and network failures. In our case, a major network failure triggers a general emergency braking maneuver. Default
/registry.register
Registration /registry.register
when(approach CriticalArea(registry))
waitForAnswer
registry.acceptDereg /
after(20)
register
when(problemsResolved()) registry.reject / when(approach CriticalArea(registry)) when(approachCriticalArea(registry))
registry.accept / registry.data
MissingRegistration Controlled
Deregistration
after(20) / registry.data
/registry.deregister
waitForAnswer
after(20)
deregister
when(exitCriticalArea(registry)) / registry.deregister
ProceedData
registry.rejectDereg /
ConnectionLost
after(21)
registry.data / registry.data; updateData()
registry.data / updateData()
when(emergency()) /
Emergency
when(!emergency()) /
Fig. 5. State chart describing the shuttle role of the registration pattern
The shuttle role behavior of the registration pattern is depicted in Figure 5, using statecharts. When the pattern is instantiated, the shuttle can non-deterministically decide when to start the registration process. To do so, it sends a register message to the registry, which causes the shuttle to enter the Registration-state and wait for the registry’s reply. After 20 msec without a response or after receiving a reject message from the registry, the shuttle changes back to state Register and retries. The method approachCriticalArea(registry : Registry) : Boolean, described by the story diagram in Figure 6, determines when the shuttle has to initiate an emergency braking maneuver to avoid entering the critical area of a registry with which it is not currently registered. This may either occur because of failed registration attempts or because such attempts have not yet been initiated. The shuttle then switches to state MissingRegistration. The structural constraint defined by the pattern matches when a shuttle (this) is located on a track section t1 in a registry’s affiliation area and at least one of that track’s successors t2 is part of its critical area. As t1.length is the length
n
Fig. 6. Story diagram for method approachCriticalArea(registry : Registry) : Boolean
of the current track and this.relativePosition is the distance the shuttle covered on this track, t1.length - this.relativePosition is the distance remaining until the next track (and consequently the critical area) begins. When this distance falls within a safety-margin of the stopping distance, calculated based on the current velocity by the shuttle’s internal method stoppingDistance(), the critical point has been passed and the method returns true. After receiving an accept message from the registry, the shuttle switches to state Controlled. It then periodically sends its velocity and positioning data. In turn, it receives the other shuttles’ data, broadcast by the registry every 20 msec. If no data is received, state ConnectionLost is entered. When the shuttle internally detects an emergency (possibly based on the data received from the registry) it switches to Emergency. In both states, the shuttle performs an emergency braking maneuver and waits until the problem is solved. When the shuttle exits the critical area (which is described by a story diagram similar to Figure 6), it starts the deregistration process, which resembles the registration process. active shuttle.register /
Inactive
after(2000) / shuttle.reject
after(20) / setData(shuttle,null)
Registering
/ shuttle.accept
Default shuttle.data(p) / setData(shuttle,p)
/ shuttle.reject shuttle.deregister / / shuttle.acceptDereg
Deregistering
after(2000) / shuttle.rejectDereg
after(20) / shuttle.data(getData())
Start / shuttle.reject
after(20) / shuttle.data(getData())
Broadcasting / shuttle.data(getData())
Fig. 7. Registry role of the registration pattern
The corresponding registry role is shown in Figure 7. Whenever the registry receives a register event, it can non-deterministically decide whether to accept or reject the request. While managing a shuttle (being in the active-state), the registry expects to receive updated velocity and positioning data every 20 msec, which is then stored internally. If no update is received (e.g. because the connection between shuttle and registry is lost), the failure is recorded and published (setData(shuttle,null)) likewise. Meanwhile, the stored data is sent to all shuttles every 20 msec. As a registry habitually manages multiple shuttles, distributing the information to all registered shuttles via
multicast is useful in order to save bandwith. Thus, the sending intervals of all registryroles needs to be synchronized. This is realized by allowing the registry to postpone the first transmission for up to 20 msec (cf. second branch of state active in Figure 7), waiting for the next general broadcast event. The statecharts from Figures 5 and 7 specify the role behavior of the Shuttle and the Registry in the registration pattern. The correctness of the patterns can be verified by compositional model checking as described in [12]. The specification of the roles still leaves several degrees of freedom due to non-determinism (e.g. in the substate registering of Figure 7), which permits the utilization of self-optimization within the agents at the micro-architecture level.
3 Micro-architecture This section discusses a generic micro-architecture that offers support for the adaptation of agent behavior within the limits of the patterns described in the macro-architecture. Basically, our architecture has a hard real-time (HRT) part and a self-optimizing (SO) part. For a successful integration of self-optimizing concepts into HRT mechatronic systems, their real-time behavior must not be compromised, i.e. verified properties must hold even if the self-optimizing part behaves erroneously. Therefore, the two parts must be decoupled, only allowing the SO part to change specifically designated parts of the HRT system. Statecharts detailing an individual agent’s behavior can be derived by refining the specified pattern roles. The resulting behavior remains safe as long as all applied refinements are correct (cf. [12]). The designated parts of a statechart where self-optimization is safely possible are states with alternative outgoing transitions. At least one of these transitions must be a safe default transition that fires if a certain timeout occurs. The SO part may choose any of these outgoing transitions as well as a valid firing time. If not, the timeout transition will fire in order to adhere to the pattern and uphold the verified properties. During verification, it is assumed that the transitions are nondeterministically chosen. Thus, the verified properties hold regardless of which transition is chosen by the SO part at runtime. When aiming to optimize the flow-rate in the railcab system, it might be reasonable for a registry to start rejecting some registrations if there already are jams in its controlled area. Hence, the registry may change its behavior w.r.t. the acceptance of the registration attempts. The shuttles may in turn choose for themselves at which time they attempt to register with the registry. Some may choose to register while still at a long distance from the critical area in order to be able to reroute if their registration is rejected multiple times. The Registering state in Figure 7 reflects this situation and therefore has more than one outgoing triggerless transition. In the remainder of this section, we will address how self-optimization can alter the behavior of the registry w.r.t. the registration attempts of shuttles. The cooperation between HRT and SO parts required to realize self-optimization in an agent at the micro-architecture level can be achieved in two different ways: (1) Via direct communication, the SO part can control the HRT part and determine both which transition is chosen and the transition parameters. The SO part has to process the data within the time frame given by timeout transitions. (2) Alternatively, the SO part can
adapt the default behavior of the HRT part of the system by adjusting guards in the statechart of the HRT part. Self−Optimizing Behavior Planning
Learning
Adaption
PlanRequest Plan
Adaption Feedback Hard Real−Time Behavior shuttle1.register /
Inactive
after(2000) / shuttle1.reject
Registering
/ shuttle1.reject
Fig. 8. Micro architecture
To support these cooperation models, our micro-architecture contains several components (see Figure 8). As described above, the system is divided into a SO and a HRT part. The HRT part is specified by statecharts or other modeling approaches suitable to meet hard real-time properties. The SO part consists of a planning, an adaptation, and a learning component. The planning component controls the statecharts, while the adaptation component adapts their long-term behavior. The learning component is used to learn from the past of the system and to change future executions of the system accordingly. In this context, the past is determined by the previous execution traces of the statechart. Based on these execution traces, e.g. using case-based reasoning, the SO part can infer the adaptions needed to optimize the behavior of the agent by either adjusting its default behavior or changing the behavior of the planning component. The HRT part communicates with the planning component using a non-blocking event queue. Therefore, the standard mechanisms of UML statecharts can be used to integrate the planning and HRT parts. Based on the information guaranteed to be available by the registration pattern, the planning component may compute the expected traffic and can thus reject or accept the registration of additional shuttles. In an improved version, a registration fee might be employed to control the traffic and to avoid jams instead of simply rejecting shuttle registrations. To offer support for the adaptation of statecharts we have developed the following approaches: (1) As a first basic idea, runtime parameters that are used within the guards can be adapted. For example, a parameter min importance in a subexpression (shuttle.importance>min importance) of a guard expresses that, in a specific situation, only shuttles with an importance higher than min importance can enter the critical area. (2) For a probabilistic extension, we are following [19] to support adding probability values to transitions. During execution, a random value will be used to choose which transition fires. In our example, this probability value will be adjusted at runtime w.r.t. the number of shuttles in the controlled area (e.g., probabil-
ity reject=shuttleCount/maxShuttles). (3) Finally, fuzzy logic systems [1, p. 164ff] can
be integrated into statecharts. As the worst case execution time (WCET) for the evaluation of a fuzzy logic system can be determined, it is possible to evaluate the fuzzy logic system in the context of the HRT part of the system. For example, the traffic density in the controlled area may be classified into the fuzzy sets empty, middle, and full during the fuzzification step. The parameters used in this fuzzification as well as parameters used in the fuzzy rules and in the defuzzification step can be adapted at runtime without changing the WCET for the evaluation of the fuzzy logic system. The presented concepts can be employed to systematically realize the required selfoptimization by means of adaptation, planning and learning. The restriction to valid refinements ensures that the behavior required by each agent role as specified at the macro-architecture level is always respected. Thus, the safety analysis undertaken at the macro-architecture level still holds for the refined multi-agent system that additionally incorporates self-optimization.
4 Conclusion and Future Work As outlined in the introduction, the nature of mechatronic systems requires that the worst-case behavior w.r.t. real-time and safety aspects can be predicted. On the other hand, the vision of self-optimizing networked mechatronic systems requires very flexible agents that adapt their autonomous behavior at runtime. The presented approach proposes a macro- and micro-architecture style that helps to overcome this contradiction by means of communities and their cultures, at the price of a restricted notion of agents and their cooperation. The specific cultures can be characterized by the standard software engineering techniques of the UML with minor extensions. Within each agent, the proposed micro-architecture only permits local behaviors that are safe refinements of the behaviors specified at the macro architecture level. Therefore, self-optimization in form of adaptation, planning, and learning does not invalidate the verified role behavior, and the macro- and micro-architecture are decoupled. Today, the behavioral norms established through the role protocols of the patterns can be employed for automatic compositional verification (cf. [12]). In the future, this might provide the basis for the verification of the dynamic part of the system. The presented basic UML extensions such as story patterns and story diagrams are already supported by the CASE tool Fujaba, including Java code generation. We further plan to provide the same level of tool support for the newly presented concepts such as social diagrams.
References 1. Dawson, D., Seward, D., Bradley, D., Burge, S.: Mechatronics and the Design of Intelligent Machines and Systems. Stanley Thornes (2000) 2. Hannebauer, M., Wendler, J., Pagello, E., eds.: Balancing Reactivity and Social Deliberation in Multi-Agent Systems: From RoboCup to Real-World Applications. Volume 2013 of LNAI. Springer (2001) 3. Kitano, H., Kuniyoshi, Y., Noda, I., Asada, M., Matsubara, H., Osawa, E.I.: Robocup: A challenge problem for AI. AI Magazine (1997) 18(1):73–85 Springer.
4. Herrmann, D.S.: Software Safety and Reliability : Techniques, Approaches, and Standards of Key Industrial Sectors. IEEE Computer Press (1999) 5. Shoham, Y., Tennenholtz, M.: On the synthesis of useful social laws for artificial agent societies. In: Proceedings of the 10th National Conference on Artificial Intelligence (AAAI92), San Diego, CA. (1992) 276–281 6. Goldman, R.P., Rosenschein, J.S.: Emergent coordination through the use of cooperative state-changing rules. In: Proceedings of the 12th International Workshop on Distributed Artificial Intelligence (IWDAI-93), Hidden Valley, PA. (1993) 171–186 7. Conte, R., Castelfranchi, C.: Simulative understanding of norm functionalities in social groups. In: Pre-proceedings of the International Symposium Simulating Societies-93, Certosa di Pontignano, Siena, Italy. (1993) 8. Walker, A., Wooldridge, M.: Understanding the emergence of conventions in multi-agent systems. In Lesser, V., ed.: Proceedings of the First International Conference on Multi– Agent Systems, San Francisco, CA, MIT Press (1995) 384–389 9. Shoham, Y., Tennenholtz, M.: Emergent conventions in multi-agent systems. In C. Rich, W., Nebel, B., eds.: Proceedings of Knowledge Representation and Reasoning (KR&R-92). (1992) 225–231 10. Finin, T., Labrou, Y., Mayfield, J.: KQML as an agent communication language. In Bradshaw, J.M., ed.: Software Agents. AAAI/MIT Press (1997) 11. FIPA: FIPA 97 specification - part 2: Agent communication language (1997) FIPA. Foundation for Intelligent Physical Agents. 12. Giese, H., Tichy, M., Burmester, S., Sch¨afer, W., Flake, S.: Towards the Compositional Verification of Real-Time UML Designs. In: Proc. of the European Software Engineering Conference (ESEC), Helsinki, Finland. (2003) 13. Fischer, T., Niere, J., Torunski, L., Z¨undorf, A.: Story Diagrams: A new Graph Rewrite Language based on the Unified Modeling Language. In Engels, G., Rozenberg, G., eds.: tagt6. LNCS 1764, Springer (1998) 14. Odell, J., Parunak, H., Bauer, B.: Extending UML for Agents. In Wagner, G., Lesperance, Y., Yu, E., eds.: Agent-Oriented Information Systems Workshop at the 17th National Conference on Artifical Intelligence (AAAI2000), Austin, TX, USA (2000) 3–17 15. Bauer, B., M¨uller, J., Odell, J.: Agent UML: A Formalism for Specifying Multiagent Interaction. In Ciancarini, P., Wooldridge, M., eds.: Workshop on Agent-Oriented Software Engineering (Held at the ICSE2000), Springer Verlag (2001) 91–103 16. Musliner, D., Durfee, E., Shin, K.: CIRCA: a cooperative intelligent real-time control architecture. IEEE Transactions on Systems, Man and Cybernetics 23 (1993) 1561–1574 17. Dardenne, A., van Lamsweerde, A., Fickas, S.: Goal-Directed Requirements Acquisition. Science of Computer Programming (1993) 20:3–5–0 18. Waldo, J.: The jini architecture for network-centric computing. Communications of the ACM 42 (1999) 76–82 19. Jansen, D.N., Hermanns, H., Katoen, J.P.: A Probabilistic Extension of UML Statecharts. In: Formal Techniques in Real-Time and Fault-Tolerant Systems. Proceedings of the 7th International Symposium, FTRTFT 2002, Oldenburg, Germany,. (2002) 355–374
