A mechanism for trusted agent-based workflow transport Dimosthenis Kaponis1 , Lloyd Kamara1 , Jeremy Pitt1 , and Keith Clark2 1
Intelligent & Interactive Systems Group, EEE Department, Imperial College London, SW7 2BT {d.kaponis, l.kamara, j.pitt}@imperial.ac.uk http://www.iis.ee.imperial.ac.uk 2 Department of Computing, Imperial College London, SW7 2BZ
[email protected]
Abstract. Electronic workflow facilitates transactions between systems and enterprises. In this paper we employ socio-cognitive theory and technologies in the context of an agent-based workflow engine to facilitate partner selection and build trust among electronic workflow participants. These elements form useful and effective additions to the workflow context, by capturing business policy aspects. We present a modular framework used in distributed workflow that enables the application of principles of electronic contracts and security. We demonstrate how support for expert agents can be used with machine-based learning and trust building between entities.
1
Introduction
Electronic Workflow is defined as ‘the automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules’ [1]. The participants may be human or artificial, as noted by Huhns and Singh [2], who define a workflow as ‘a composite activity consisting of tasks involving a number of humans, databases, and specialized applications’. Workflow Management Systems (WfMS) are predominantly of a centralised nature: information is kept in a database (most commonly a relational or objectrelational database system) that a server manages and connects to. Clients connect to this server to make their requests and retrieve the associated data. These systems make use of database technology to store and maintain the workflow information/data and network technology to disseminate this information to the appropriate workflow clients. The use of agent technology to improve the performance of workflow systems through de-centralisation and scalability [3] and co-ordinate [4] [5] [6] their interactions has been a feature of research over the past decade. In that time, extensive work has also taken place on decentralised system security [7] [8] and the related concept of trust [9] [10] [11] [12] [13]. In this paper, electronic contracts, the socio-cognitive concept of trust and machine learningbased interaction analysis are brought together under a framework consisting
of expert agents. Moreover, a reference implementation of some of the expert agents designed in the context of the IST B-MAN Project is also presented. Section 2 provides some background information as well as the motivation for this work. Section 3 gives an overview of the B-MAN system and its architecture. The Contract Management approach taken in B-MAN is introduced in Section 4, whereas the Machine-Learning approach that provides the engine for Trust related data generation is presented in Section 5. Section 6 provides the scope of future work and, finally, Section 7 provides the conclusions drawn from this work.
2
Background and motivation
2.1
Agent-based workflow
The use of agent technology to further automate workflow activities has been investigated by several researchers in the past decade. For example, Chang and Scott in [14], adopt the concept of ‘agent-based workflow’ which incorporates mobile agents, with support for the interfaces defined and promoted by the Workflow Management Consortium (WfMC). The Intelligent business systems research group/project of British Telecom (BT) has actively investigated the concept of agent-enhanced workflow [15]. The European Institute for Research and Strategic Studies in Telecommunications GmbH (EURESCOM) has investigated the use of agent technology in distributed workflow management, with emphasis on telecommunications scenarios [4]. Meng et al. [5] consider workflow and agent mobility, identifying the need for strengthening of the relationships between webbased services in order to advance e-businesses. Jeong-Joon Yoo et al. [3] propose a workflow system model based on mobile agents. They suggest that the hosting of workflow definitions by mobile agents reduces some of the task-processing by the workflow engines. Berry and Drabble [6] consider workflow enhancement from a more general AI perspective. They extend the Workflow concept to support operation in information, surveillance and reconnaissance (ISR) domains, necessary for the real-time, reactive, military applications they have in mind. Similar concepts are expressed in [16]. The work referenced above give an indication of the variety of approaches and applications making use of agents and workflow and is by no means exhaustive. The B-MAN Project3 advances the state of the art in this area by addressing issues including distribution and lack of centralisation of distributed systems, the intelligence and flexibility of agent systems. B-MAN integrates socio-cognitive and electronic contract technologies that reduce risk and facilitate secure interactions in electronic workflow societies. 2.2
Learning and Trust building
Trust based systems have been studied in the context of several fields, ranging from Computer Science [7, 8, 17], Psychology [10, 11, 13] and Law [18, 19]. There 3
http://www.b-man.org
is a multitude of methods and protocols with which trust-building is achieved in a society of agents. It is not uncommon to have multiple methods employed at once. Some of them include peer review, reputation, institution-based trust and collaborative interactions. Trust building between enterprise entities interacting with one another, requires a method which is inherently linked to the performance of the interactions at hand. In addition, some of the trust-building methods mentioned above are unsuitable, by nature, to the often confidentiality-bound interactions related to business process structure as well as their enactment. When modelling a business process using a graph theoretic approach, it is sometimes undesirable to have information flow to form a fully connected graph, whereby all parties are aware of the activities (or sub-activities) taking place in a remote site. The required graph should be partially connected where each edge of the graph would represent a contract between two parties (vertices). Partial connectivity is effectively the use of different contracts enabling different levels of disclosure of business process structure or work taking place at any given moment in time. In this manner, loops inside the graph would indicate a contractual level trust relationships between the enterprises at the vertices of the loop. This type of requirement excludes peer-review methods which are founded on complete visibility of peer-actions (transparency) within a society. Enterprise workflow applications are, by definition, burdened with increased de-centralisation regarding the ontological and procedural details of business processes in use. No central database or organisation can provide comprehensive lists of the business processes involved in arbitrarily large numbers of transactions; no central point can, in the framework within which enterprises interact today, exist, that assures in a globally acceptable way the trustworthiness of an enterprise (irrespective of how trustworthiness is represented); there are multiple reasons for this, both technical and political which, although very interesting and challenging, are beyond the scope of this paper. Consequently, centralised institution based trust ‘distribution’ and management is inappropriate in the applied context in which B-MAN lies. Reputation, data persistence and fusion in conjunction with some Machine Learning algorithms, are two ways that individual entities or group of entities can share between them without having to disclose the group or the information exchanged to the whole society at any given time, yet still achieve information gathering and distribution within the society. Reputation is the chosen learning method in the B-MAN reference implementation described in the subsequent sections. 2.3
The B-MAN Project
The IST-funded4 B-MAN Project aims to design, implement and promote the use of a next-generation agent-based workflow management system. The novel4
IST Project 32285
ties of the system lie both in the ground-up approach in creating an agent-based, distributed workflow engine and in the use of a framework providing a suitable interface for higher-cognitive functionality expert agents. The architecture and an overview of the B-MAN System are presented in Section 3. 2.4
Example scenario
In order to facilitate the understanding and enhance our presentation of the system in the sections below, we are describing a basic scenario which is used throughout the paper. A small cookie producing company called “Cookies’R’Us” has multiple suppliers for the ingredients required for the production of the cookies. For one of the ingredients, say flour, there are three possible suppliers. It is assumed that all companies are using the B-MAN WfMS as described in the following sections.
3
The B-MAN Architecture
The B-MAN prototype system can be logically divided into two main components: the distributed Workflow Management System (dWfMS), which replaces traditional WfMS engines and the Mutual Trusted Business Integrator (MTBI), which serves as the transport, integration and security component. It is the MTBI that this paper aims to describe in detail. In B-MAN, an enterprise is not restricted to one physical or network location, but can be dispersed globally. Each self-contained organisational or other functional entity of the enterprise is called a B-MAN Node (BN). The B-MAN Node, in turn, consists of a dWfMS which further distributes the work and the associated processing within the BN, and a Gateway Node (GW), a MTBI sub-component that serves as the entry point to the BN. The set of the BNs belonging to the same legal entity form a virtual node which represents the enterprise network as a whole and is called the Enterprise Node (EN). Each EN has one Portal Node (PN), which serves, among other roles, as the entry point to the EN (section 3.1). The existence of several ENs creates a B-MAN ecosystem, whereby interactions take place both within and between the distributed ENs. For example, “Cookies’R’Us” could have two organisational units: the company headquarters, or offices, where financial services, marketing and sales are located and the cookie factory and packaging facility, which are at a different geographical location. Those two organisational units, although related, could be represented by two B-MAN Nodes. Internet connectivity and Secure Socker Layer (SSL) ensure that for the purposes of the enterprise, the Nodes are securely connected at all times despite the fact that they are in completely different physical locations. For the scope of the B-MAN network, the two nodes form the “Cookies’R’Us” Enterprise Node. Workflow requests for the factory BN, could originate at the headquarters BN, or they could originate from a completely different BN, e.g. a customer ordering some cookies.
The system design follows a hybrid distributed system model whereby communications are initially taking place between PNs and subsequently, upon successful completion of the handshaking protocol required, on a peer-to-peer basis between the appropriate GWs. Communications between all the nodes is performed using SSL.
5 Kc
Kd 2
A
Ka Ke
1 3
6 Kb Kg
4 Portal
Kf Gateway
B
Kf Workflow Engine
Fig. 1. Illustration of the architecture of the system.
In Figure 1, PNs (circles) communicate amongst themselves (inter-EN communication), but also with GWs (diamonds) of the same organisation (intra-EN communication). The following narrative further illustrates system operation, each step corresponding to a numbered arrow in the figure. – At (1) each of the Portals acquire (via human intervention) a copy of a new contract specifying an interaction between the two organisations. In our example this represents our cookie company signing a new contract with a flour manufacturer Flour Inc. – At some future moment in time, a Gateway decides to communicate with an external organisation. This could mean that the factory in the cookie company needs more flour. At this moment the Expert Agent framework (Section 3.3) handles the Trust querying process. A definite destination EN — out of the now four possible ones — is selected.
– The originating GW sends that request to its Portal Ka (2). The Portal communicates with the Remote Portal Kb (3) providing a Meta-Request5 . – If all Portal-level checks are successful, Kb identifies the destination Gateway (that will eventually provide the requested service) — here Ke — and provides the Ke certificate as well as its location to Ka . Ka in turn provides the certificate for the originating Gateway Kc – Finally Ke is also provided with Kc ’s certificate (not shown in the figure) so as to allow it to communicate with it. – At this point (5), the involved Gateways can communicate with each other directly. Portals do not have a direct role in the inter-organisational communication beyond handshaking and routing the requests. In interactions not covered by the above scenario, the Portal has different, yet important roles in information dispersion, fusion and orchestration of trust within the organisation, although, again, the core of the processing still takes place in the Gateway Nodes. 3.1
Portal Node
In B-MAN, workflow is open, as long as the underlying network allows communication between nodes. Decentralisation of enterprises is possible and it is common for an organisation to have its nodes physically distributed in remote locations and depending on the use of (insecure) networks, such as the Internet to function. A Portal Node(PN) refers to a special kind of entity in the context of a B-MAN Enterprise, which provides the entry point to the EN. Each EN has its own PN serving as an entry point to the associated domain — a point of reference which other enterprises can utilise to access it. Although the work-related communication is taking place solely between Gateway Nodes (see Section 3.2), initial communication (handshake), inter-organisational routing and preliminary security checks take place through the Portals. Portals also serve as coordinators both at a software agent level as well as a human controller level (as a central location through which monitoring and control can take place. Finally, PNs are used to gather and broadcast trust important information, such as trust and security updates and bulletins. 3.2
Gateway Node
Distribution of processing and delegation of responsibility for each transaction to the most local resource means that Gateway Nodes are responsible for most of the processing and analysis taking place on the system’s traffic. In terms of the transport mechanism, Gateway Nodes (GW) are the starting and ending 5
A Meta-Request contains the subset of of the information provided in the full workflow request relevant to identifying the service required as well as authenticate it
points of a request’s journey. They directly interface with local workflow engines; they also package new requests and transport them to other GWs within the organisation or the PNs from where they are sent to other organisations. GWs keep records of where requests went and when they were sent: This information is passed on to the expert agent modules which process this information and in-time provide increased value over business partners. They also screen incoming requests coming from other GWs (internal or external), validating them against the predefined contracts, checking their cryptographic validity and providing them with a unique identifier. Each GW can communicate with either the organisation’s PN or other GWs – internal or external to the organisation. Each arriving request, generates an ‘event’. Events trigger specific sequences of actions, such as notifying the expert agents that have registered interest in them. 3.3
The Expert Agent Framework
B-MAN’s transport mechanism is fully modular to allow future extensions in its functionality. Semantically, processing modules (called ‘Expert Agents’ in B-MAN nomenclature) are split in the following categories: 1. Observer, where the expert agent, in the context of the system, merely absorbs the meta-information generated by request reception and/or transmission by the Gateway; 2. Consultant, where the expert agent, in addition to observing, can be queried on demand by the system regarding specific requests; 3. Controller, where the expert agent, in addition to observing, is always queried by the system regarding specific requests and whose output always affects the path of that request. Expert agents follow predefined structure guidelines and comply with exported interfaces. Upon start-up of the system the expert agents are identified in the file-system and registered with the system. In all three types there can be vectors of such expert agents present; In the cases of Consultants and Controllers, multiple plug-ins could potentially lead to conflict of opinion. For this reason, weighting of expert agent opinions is performed at the system-configuration level and a human-controller is responsible for configuring the system by providing weights for each expert agent’s output.
4
Contracts Management
The two ways in which ‘trust’ is embedded into the B-MAN system are Contracts (which provide a safety-latch by minimising risk) and through Expert Agents (as described later). Contracts can provide a reduced-risk operating environment for the participating ENs, yet still offer opportunities for trust-information gathering to take place. It is envisioned that in the future, electronic contracts will be dynamically
negotiated and agreed upon between agents on behalf of organisations. Within the context of B-MAN, contracts are not dynamic: They are put in place by human users who previously agree to the terms and conditions and the service details. A B-MAN contract, consists of three layers: – General information – Contract Template – Terms and Conditions The General Information section contains IP locations of the PNs of the two ENs, as well as Cryptographic keys for those PNs. They also contain other housekeeping information such as the name of the contract, signing date; information that would exist in legal documents today. The Contract Template specifies the format and options that can or must be filled in for all requests. Validation involves checking that the request corresponds to the respective contract and that any specified attributes are indeed wellformed and meaningful within the context of the agreement. This facilitates the generation of requests by the WFMS set of components and provides several facilities for error-trapping and non-repudiation in the case of malicious/faulty requests. Provision is made in Contract Templates for three types of parameter. Range parameters describe a numerical interval between which the specified value must fall. Primitive parameters are single attributes of Boolean, float or String-type. Enumeration parameters describe a set of String values a parameter can have. Each of the above parameters has an associated (String) label which is used to identify and refer to the ‘dimension’ in question. Finally the Conditions and Terms section provides the majority of the constraints and rules of the contract as is explained in Section 4.1. Once a contract is put in place in the B-MAN system, it is used as the basis for workflow transfer between nodes. Task Requests are generated by the Workflow Management Engine. A Task Request sent by the system will largely depend on the Contract itself. The reason for this is that the structure of the request is determined by the Contract Template. By ‘filling’ the gaps of the Contract Template, the WFMS instantiates the Contract Template and creates a request. Recalling the ‘Cookie’ scenario, consider the contract between the cookie company and one of the flour partners. For each request for a flour shipment, the quality of the flour, the quality and the quantity of the flour are requested. This information will be ‘published’ in the contract template. Specification of some or all of those parameters may be mandatory, if a default quality or quantity policy is not in place. Labels may be attached to parameters, such as ‘Quantity (kg)’ being attached to a range ‘1−−100’. The ‘Quality’ parameter, could be attached to an enumeration ‘{coarse, fine, extra − fine}’. In the Task Request, a valid value for each parameter would be selected and the set of those parameters would be the Instantiated Contract Template.
In our implementation, both the Contracts and the Instantiated Contract Template are expressed using the Extensible Markup Language6 . This welldefined and well-established standard for structured documentation is particularly suited to the selected task, due to its capacity for capturing the concepts and relationships that need to be expressed and for the availability of supporting technology facilitating integration with other software tools used in the implementation. The variety of XML processing and binding implementations and technologies available make it easy to validate Requests as well as bind the information contained therein in objects, making it accessible to other parts of the program.
4.1
Contract Semantics — Conditions and Terms
Another part of the contract, which is outside of the contract template (that is, it is not instantiated by every request) contains the associated conditions and terms (ct). These describe further allowances and constraints that, respectively, can or must characterise the interactions arising during the performance of a requested task. This includes, for example, absolute limits on the number of unit orders permitted under the contract agreement, temporal and iterative restrictions (for specific processes). The ct fields can be likened to the terms and conditions associated with ‘real-life’ contracts. The existence of a comprehensive list of possible constraints as well as the combination of those constraints with specific references to variables defined in the Contract Template results in a very flexible method for defining the permissions involved and provides both fine and coarse granularity depending on usage. One key concept in the execution of task request is the agreed level of monitoring. During execution of a task, the task originator may want to query the contractor as to the task’s current state. These requirements can be represented through monitoring attributes. The first of these, allowsMonitoring, indicates whether or not the agreed contract permits monitoring at all. If set to true, the monitoring type specified through the monitoringType field provides details of the exact monitoring agreement: the specifics of monitoring are established on a per-contract basis. The final monitoring-related attributed is thirdPartyMonitoring, which denotes whether or not the contractor extends the same monitoring privileges afforded the task-originator to any party contracting the taskoriginator. This, in effect, allows a ‘chain of monitoring’ to be established in supply-chain scenarios, where there is a hierarchic dependency and, arguably, a need for hierarchic transparency in the context of task performance. The monitoringType attribute may be further used to convey any nuances surrounding the transfer of monitoring rights to ‘higher-level’ entities. 6
http://www.w3.org/XML/
5
Experience through Machine Learning: Building Trust ratings
Trust building is a higher-level function that can take advantage of the ObserverConsultant-Controller paradigm in use in the MTBI. A Consultant module can be informed of the events generated and process them using a Machine Learning technique. The interfacing to the WfMS (and potentially to a human being) allows for both supervised and unsupervised learning to take place. For example, human employees could serve as tutors in a neural network-based Controller module. Within the B-MAN Project, the ‘environment’ is static hence the participants and their relationships are largely constant. Information between entities is not exchanged; however, due to the distribution of the system, information gathered by different nodes at different times and based on different contracts is very useful and is exchanged when required. For each outgoing request sent by a node and which at some points completes or terminates, operational feedback is sent by the Workflow Engine to the MTBI. Operational Feedback includes the results of a previously sent request (such as completed or failed) as well as more finely-grained result-sets such as ‘delayed’, ‘ok, but bad quality’ etc. This information is used to reinforce or degrade the value of each participant in a given role. Over time, such ratings are able to ‘map’ the performance and quality of services provided by several participants throughout an organisation. By sharing the information within the organisation, both future contract and requests can be made more effective and efficient. In the current implementation of the system a TD-Reinforcement Learning mechanism is being employed to keep and evolve ratings of business partner performance over time. The system functions by rating business partners’ service with the partner assuming a specific ‘role’. For example, if SweetsIngredients Inc. was providing both sugar and flour to the company, there would be two distinct and unrelated ratings for those two ‘roles’ of SweetsIngredients Inc.: that of the sugar-provider and that of the flour-provider. An example of the existing functionality follows: In our scenario, “Cookies’R’Us” has a unique name for the flour ordering process/task, CookieFlour. This characterisation is esoteric – it is not shared with any other organisation or body. The business process engaged in producing the cookies, of which ordering the flour is part, does not know anything about the specific partners with which the organisation has contracts. The business process is static and could be determined long before the contracts with the current suppliers were signed, whereas the providers may change at any time. The binding element is the name of the service requested. The Trust and Learning module of the MTBI component, keeps reinforcement learning matrices for each possibly requested service. For example, in the aforementioned scenario, it would keep a reinforcement learning matrix for CookieFlour, Cocoa (or other flavour) and sugar. Each matrix M has x columns, where x = the number of providers
for this service7 . In order to turn the, currently one dimensional matrix into a two-dimensional one, an additional factor is required per service. This factor is currently a measure called The Importance-Factor (I) which is derived using the following formula: I = V × IT × IQ where V is the normalised volume (or quantity) requested, It is the normalised Temporal importance factor and Iq is the normalised Quality importance factor. In the current implementation of B-MAN, this Quality and Temporal information for the Importance Factor is not provided to the MTBI sub-system, until the completion of the task. Since the components of the formula presented above are normalised, maximum values are required. Those are initialised with ‘best-guess’ values before the first-run of the system and always keep the maximum requested value by updating the maxima whenever a higher value is encountered. The distillation of the above factors into the Importance Factor is enhancing the convergence rate of the system to its optimal performance. 5.1
Alternative approaches within the current Reinforcement Learning scheme
Alternative approaches within the current TD-Reinforcement based learning system, could include service-specific factors, and further categorisation, leading to the use of multi-dimensional matrices. Preliminary experimentation in a research environment with such an approach showed varied results; in the cases where one specific combination of parameters was the norm, convergence on that part of the matrix was rapid with satisfactory optimality approaches. In other cases, the matrix did not converge in a considerable amount of time (or at all), due to the high dispersion of the data gathered.
6
Extensions and Future Work
6.1
Experimentation with different algorithms
As the testing phase of the B-MAN project approaches, it becomes desirable to provide implementations of different learning algorithms. Similar approaches and variations in the current reinforcement algorithm are the primary goals. The simplest modification would consist of improving the current policy π, which currently exists in the form of the Importance-Factor mentioned in the previous section. Pursuit Reinforcement Learning, TD(λ) and combinations of reinforcement learning are also candidates for the prototype implementation. 7
The largely static environment in which B-MAN is meant to operate, alleviates the possibly critical performance issues that could arise in more dynamic environments, that would require resizing and normalisation of the matrices at run-time.
In the longer term, the combinatorial use of Neural Networks as the value function in conjunction with TD-Reinforcement Learning, in an arrangement similar to that of Tesauro in [20]. 6.2
Multiple learning Expert Agents
As outlined earlier in this paper, the framework under which this mechanism was established is modular, and permits the use of multiple ‘learning’ Expert Agents, each contributing to an overall ‘rating’ of each potential partner. Work is underway to examine the combination of different learning modules, each with different approaches to the learning and prioritisation problem. The final decision the system makes is currently based on a weighted sum, although different formulas are being examined. For example, for three Consultant Pn expert agents xi with weights wi , the output the system would use would be i=1 wi × xi and x can range between -128 and 127 (one signed byte). Changes in the weights can radically change the way the system performs. In some cases, conflicts can exist between expert agent modules and the weights are responsible for maintaining a balance between each expert agent’s verdict. 6.3
Automation of the weight-configuration and the feedback
Users of the system are currently responsible for configuring the system and thus providing the weights, although in the future a more automated method of dynamically configuring the system’s weights can be implemented based on previous performance of the expert agents.
7
Conclusions
Workflow systems are systems involving large amounts of data in the form of requests and responses linking resources — artificial or human — so that they can communicate towards the completion of one or more tasks. The B-MAN Project sets out an approach towards designing workflow systems that is distinct from contemporary industrial approaches. The B-MAN approach builds upon the work done in the last decade concerning the unison of agent and workflow technology. The result is a workflow management system making use of information localisation, distributed processing of information and agent-based ‘intelligent’ processing. In this paper, we have covered the transport component of this system in some detail. We have shown how this component functions as a link between higher-cognition modules, called Expert Agents, that further enhance system functionality. We have introduced some of the work done in the resulting prototype implementation which enables trust quantification through the combination of reinforcement learning techniques as well as electronic contracts. We plan to continue development of the system so as to enhance the learning algorithms in the context of workflow and improve on the automation of the system as whole.
8
Acknowledgements
This work has been undertaken in the context of the EU-funded B-MAN Project ((IST-2001-32285). The authors wish to express their gratitude to the reviewers for their comments.
References 1. Coalition, W.M.: The workflow management coalition specification — terminology & glossary. Technical Report WFMC-TC-1011, Workflow Management Coalition (1996) http://www.wfmc.org/standards/docs/TC-1011 term glossary.pdf. 2. Huhns, M.N., Singh, M.P.: Workflow agents. IEEE Internet Computing 2 (1998) ˜ 94–96 http://www.ai.sri.com/berry/publications/AAAI99-workshop.ps. 3. Yoo, J.J., Lee, D., Suh, Y.H., Lee, D.I.: Scalable workflow system model based on mobile agents. In S.-T. Yuan, M.Y., ed.: Proceedings of the 4th Pacific Rim International Workshop on Multi-Agents. Volume 2132., Springer-Verlag (2001) 222–236 4. Blavette, V.: Communications management process integration using software agents. Technical report, EURESCOM (1999) http://www.eurescom.de/public/projects/P800-series/p815/default.asp. 5. Meng, J., Helal, S., Su, S.: An ad-hoc workflow system architecture based on mobile agents and rule-based processing. In: Proceedings of the International Conference on Parallel and Distributed Computing Techniques and Applications. (2000) http://www.harris.cise.ufl.edu/projects/publications/adhocWF.pdf. 6. Berry, P.M., Drabble, B.: Swim: An ai-based system for organisation management. In: Proceedings of the 2nd NASA International Workshop and planning and scheduling for space, San Francisco, California (2000) http://www.ai.sri.com/ berry/publications/NASA-AIPS-00.ps. 7. Blaze, M., Feigenbaum, J., Ioannidis, J., Keromytis, A.D.: The role of trust management in distributed systems security. In: Secure Internet Programming: Security Issues for Mobile and Distributed Objects. Number 1603 in Lecture Notes in Computer Science. Springer (1999) 185–210 8. Wong, H.C., Sycara, K.P.: Adding security and trust to multiagent systems. Applied Artificial Intelligence 14 (2000) 927–941 9. Castelfranchi, C., Falcone, R.: On the specification of heterogeneous computational systems (2002) 10. Castelfranchi, C., Falcone, R.: Social trust: A cognitive approach. In Castelfranchi, C., Tan, Y.H., eds.: Trust and Deception in Virtual Societies. Kluwer Academic Press (2000) 55–90 11. Castelfranchi, C., Falcone, R.: Principles of trust for MAS: Cognitive anatomy, social importance, and quantification. In Demazeau, Y., ed.: Proceedings of the Third International Conference on Multi-Agent Systems, IEEE Computer Society (1998) 72–79 12. Jonker, C.M., Treur, J.: Formal analysis of models for the dynamics of trust based on experiences. In: In Multi-Agent System Engineering: Proceedings of the 9th European Workshop on Modelling Autonomous Agents in a Multi-Agent World. Volume 1647 of Lecture Notes in Artificial Intelligence., Springer-Verlag (1999) 221–232
13. Falcone, R., Castelfranchi, C.: The socio-cognitive dynamics of trust: Does trust create trust? In Falcone, R., Tan, Y.H., Singh, M., eds.: Trust in Cyber-Societies. Number 2246 in Lecture Notes in Artificial Intelligence (2001) 14. Chang, W., Scott, C.T.: Agent - based workflow: Trp support environment ( tse ). In: Proceedings of the Fifth International World Wide Web Conference. (1996) 15. Shepherdson, J., Thompson, S., Odgers, B.: Cross organisational workflow co-ordinated by software agents. In: Work Acticity and Collaboration Workshop paper (WACC’99), BT Laboratories, UK (1999) http://more.btexact.com/projects/ibsr/papers/wacc99.pdf. 16. Myers, K.L., Berry, P.M.: At the boundary of workflow and ai. In: AAAI-99 Workshop on Agent-Based Systems ins the Business Context,. (1999) 17. Jones, A.J.I., Firozabadi, B.S.: On the characterisation of a trusting agent — aspects of a formal approach. In Castelfranci, C., Tan, Y., eds.: Trust and Deception in Virtual Societies. Kluwer Academic Publishers (2001) 157–168 18. Conte, R., Falcone, R., Sartor, G.: Introduction: Agents and norms: How to fill the gap? Artificial Intelligence and Law 7 (1999) 1–15 19. Weitzenb¨ oeck, E.M.: Electronic agents and the formation of contracts. International Journal of Law and Information Technology 9 (2001) 204–234 20. Tesauro, G.: Td-gammon, a self-teaching backgammon program achieves masterlevel play. Neural Computation 6 (1994) 246–49
