Parallelism and messaging services in a J2EE-based e-commerce

Parallelism and messaging services in a J2EE-based e-commerce brokering platform. Experiences in European projects Francisco Valera*, Aymeric de Solages†, Enrique Vázquez and Luis Bellido‡ * Universidad Carlos III of Madrid Department of Telematic Engineering Leganés, Madrid, Spain [email protected]

† France Telecom R&D ‡ Universidad Politécnica of Madrid Electronic commerce & banking Dpt. Department of Telematic Engineering Systems Caen, France Madrid, Spain [email protected] {enrique,lbt}@it.uc3m.es

Abstract This article presents different experiences carried out in the connectivity layer of several e-commerce mediation and brokering projects: ABS and ABROSE (supported by the European Programme on Advanced Communications, Technologies, and Services, ACTS) and Smart-EC project (supported by the European Programme on Information Society Technologies, IST). Project prototypes were implemented using Java, but in order to allow distributed communications, different techniques were used. While in the first two projects CORBA was considered the best choice for the communication infrastructure, the arrival of J2EE at the same time as the beginning of Smart-EC, led this project to move from CORBA to this new technology. This article will begin by summarising the first two approaches in order to finally concentrate on J2EE communication issues: parallelism, synchronous and asynchronous communications, blocking calls, deadlocks, control flow, multi-threading, etc. The last part of the article presents Smart-EC project as a use case, showing some different communication problems that were found and the different solutions that were taken.

Keywords

J2EE, Java, CORBA, communications, synchronism, asynchronism, deadlock, brokering, mediation.

1. INTRODUCTION This article presents the connectivity layer of several e-commerce mediation and brokering European projects from the point of view of distributed communications management. All the projects have been implemented using Java but since their distribution requirements were different (and technological solutions have continuously been enhanced), different communication techniques were used in each case. While in the first two projects (ABS [1] and ABROSE [2]) CORBA was considered the best choice for the communication infrastructure, the arrival of J2EE at the same time as the beginning of Smart-EC [12] (year 2000), led the project to move from CORBA to this new technology. This article will briefly describe the first two approaches in order to finally concentrate on J2EE communication issues (parallelism, synchronous and asynchronous communications, blocking calls, deadlocks, control flow, multi-threading, etc.) offering diverse experiences through some comparisons and showing some strengths and weaknesses of the different methodologies. As mentioned, Smart-EC began to use J2EE since this technology was first published, and so the article is also considering Smart-EC as a J2EE use case, presenting both its J2EE-based modular architecture as well as its module communication scheme.

International Conference on Electronic Commerce Research (ICECR-5)

Parallelism and messaging services in a J2EE-based e commerce brokering platform

2

2. COMMUNICATION ISSUES Nowadays, communication related details are normally solved by underlying platforms, so that developers do not usually have to bother about these particulars (sockets complexity or communication protocol details, for instance are typically hidden by well-known APIs). However, one of the main problems in this area, apart from performance or security related issues, is inter-module communication management. This section describes the different models and situations that have appeared in both the ABS and the ABROSE projects and introduces next section where other distribution alternatives have been adopted in Smart-EC: ● ABS: it was a highly distributed system that gave the possibility to run every system module on different server machines placed on different networks. The architecture was designed so that every single module could be a CORBA object. ● ABROSE: the system was basically designed to run in two main domains: the broker domain, and the user domain. As it was not so much distributed, it was decided to have just two CORBA objects (one for the broker and one for the user) responsible for managing and performing distributed communications. ● Smart-EC: the whole system (design, architecture, communications, etc.) is based on J2EE. One of the main analyzed points in this article is the treatment that communications receive from these viewpoints in J2EE. As we will see, while CORBA invocations can be easily defined as non-blocking calls, so that it is not difficult to have such things as asynchronous or parallel communications, Java threads usage is banned at developers’ level, and its RMIbased distributed architecture is devoted to blocking invocations.

2.1

Java Communication Model

This section has been divided following the typical program-to-program communication terminology for interprocess coordination in time: synchronous communications (when a process runs only as a result of an invocation from some other process or because some other process has been completed) and asynchronous communications (where processes operate independently of other processes). Java communications have been built up following the synchronous paradigm both for the classical method invocation between classes through local object references and also for remote method invocations (RMI) through remote object reference. Apart from this, Java has also been offering for a long time support for asynchronous communications (also known as messaging) in order to allow loosely coupled applications to be easier developed. 2.1.1

Synchronous communications

In a broad sense, we can define that two entities in a system are communicating synchronously, when one of the entities, the client, sends a request message to the other entity, the server, and waits for a response from the server, which will be the result of the actions performed by the server and requested by the client. In a Java system, synchronous communications can be used at different levels: ● Local object method invocation. Local java objects residing in the same Java Virtual Machine (JVM) communicate synchronously using method invocation. In this case, the client object uses an object reference variable to call the server object method directly. The object reference variable points directly at an object in the heap.



3

● Sockets. Sockets are a flexible and sufficient mechanism for general communication between entities, especially when the system is distributed, that is, the entities reside in different address spaces or different machines. However, when sockets are used an application protocol is needed to define the coding of the messages that are exchanged. ● Remote Procedure Calls (RPC) abstract the use of sockets so the programmer sees a local procedure call, when in fact, the arguments of the call are encoded and sent to a remote server. However, RPC does not suit very well a distributed object system, because the entities that communicate are objects residing in different address spaces, and objects normally communicate through method invocation. ● Remote method invocation (RMI). In this case, the idea is that the programmer sees a method invocation on an object, when in fact, it is not a local object, but a remote one. The parameters need to be encoded and sent to the remote methods, as well as the response from the remote methods need to be encoded and sent back to the client. The advantage is that distributed programs can be developed with the same syntax and semantics used for nondistributed programs. Synchronous communication has the advantage that it is quite straightforward and it is easy to understand and to use. The main disadvantage is that waiting for the server to complete an action might be a waste of time, especially if the client can complete other tasks that do not require the answer from the server. Besides, if the server receives several requests from different clients, the problem is bigger if the server processes and answers each request sequentially. To solve this problem, multi-threading is normally used, both in the client and the server. Even if the communication only supports single threaded behaviour, as in the case of RMI, the programmers can use Java’s threading primitives to create new threads when new requests arrive. In a distributed Java system, distributed objects have the potential to operate concurrently, since they are executing across different machines or different address spaces. In the case of different machines, there can be really true parallelism, since the different machines are physically operating in parallel. An object in a machine can operate as client, server, or quite often, both as client and server, and a server might receive requests from different clients concurrently. Synchronization between client and server objects in this context of concurrent and parallel execution, can lead to the problems normally found in concurrent systems, such as deadlock situations. [10] presents a scenario that exemplifies the problem, and proposes a methodology to detect these problems using UML. Because of its success, Java RMI deserves special attention. While there are other RMI systems, such as CORBA, Java RMI has been designed specifically for Java. The Java RMI architecture defines how objects behave, how and when exceptions can occur, how memory is managed, and how parameters are passed to, and returned from, remote methods [9]. In Java RMI, in order to have an object with remote access, a remote interface has to be defined, declaring the remote methods that are accessible. Both client and server need to know this remote interface, and the server object needs to implement the methods. A remote object is accessed via an object reference variable which points to a proxy stub implementation of the remote interface. That stub contains information that allows it to connect to the remote object, which contains the implementation of the methods. The proxy stub is generated by the RMI compiler from the remote service implementation class. Even if RMI is a good abstraction to work with objects remotely, working with local objects or with remote objects has some differences that need to be known by the programmer. How the parameters are passed in method invocations is one of them. In Java, primitive parameters are passed by value, and objects and arrays are passed by reference. In RMI, primitive parameters are also passed by value. However, when passing an object, there are two alternatives: object parameters and remote object parameters. In the first case, a local object is passed in a remote method invocation, and in this case, the object is passed by value, i.e., a copy of the object is passed to the remote object. Similarly, when a remote method returns an object, a copy of the whole object is returned to the calling program. Sending an object to a remote machine can be complicated, since the object can reference to other Java objects in a complex graph-like structure. Since the local and remote machines do not share heap memory, RMI must send the referenced object and all the objects it references along with it. The technology used to transform an object into a stream



4

of bytes that can be sent over the network is called Object serialization. An object that needs to be passed to a remote method must be serializable, that is, the object must implement the java.io.Serializable interface. So when an object is passed to a remote machine, the object is serialized by the local machine, sent over the network and de-serialized in the memory of the remote machine to get the copy of the object that can be used by a Java program. This mechanism can be very inefficient, in particular with complex object graphs, because it involves network latency and overhead of the client and server software stacks. The remote object parameters mechanism is a way of implementing pass-by-reference in RMI: instead of serializing the whole object, a remote reference to the object is sent. This will only work with remote objects, and, in fact, what is sent over the wire is the remote proxy for that object. 2.1.2

Asynchronous communications

As it has been seen, one of the most important problems of typical Java synchronous communications (local object method reference invocation and also remote method invocation) is that due to their very connection oriented viewpoint they enforce a strict request/reply sequence. In Java language, these restrictions have traditionally been solved using different execution threads so that in order not to keep the client application waiting for a response to come back before performing any other action, it is possible to carry out different actions at the same time (concurrently). Another common way to allow an application to perform additional tasks instead of waiting for tasks to be completed is to use asynchronous communications (starting new tasks after the previous request is made but before its corresponding response has come back). Applications that have several unordered actions to be done (normally including user inputs, lengthy remote operations like database queries, complex calculations, etc.) may usually benefit from this approach. This kind of mechanism, allows loosely coupled distributed communications: components may send messages to different destinations (without waiting for a response) and may receive messages from other senders as they arrive (without the need of requesting messages to receive them). Apart from allowing the problem of blocking calls to be solved, there are some other reasons to use asynchronous solutions. For instance: ● Providers do not need to implement components with specific interfaces (no remote method signatures need to be known in advance), so they can be replaced much more easily and the implemented platform results to be more flexible and modular (more open to reuse). As long as components can read and understand the message, the platform on which they reside and the languages in which they are written are unimportant. ● In case of application, system, or network failure, communications between applications are not compromised, they are just delayed (and moreover, failures in one part of the application are less likely to affect other unrelated parts of the application). In Java, the responsibility for managing all the complexity of asynchronous communications (unlike RMI, this scheme is not so transparent and does not look as a local call at all) relies on JMS [8], the Java Messaging Service. It allows Java application programmer to create, send, receive and read messages without having to care about low level details like reliability, queue management and the like. The first JMS specification was published in 1998 and although it was initially an external API and was not part of Java core classes it was finally included in Java 2 core packages. As it will be explained later we could really took advantage of this API, when it was finally fixed as a mandatory feature of J2EE (in version 1.3), and the last prototype implemented in Smart-EC is already using it.



2.1.3

5

Java/CORBA communications comparisons

When ABS and ABROSE prototypes were implemented, Java remote communications were not mature enough to support all the details that a big application required (RMI for instance, had just appeared and it could not still interact with existing Web browsers properly. Both projects developed the idea of electronic brokering in electronic commerce (we will later go into this concept with the complete description of Smart-EC project) from the point of view of a Web-based Java client accessing a Javabased broker modular architecture (this is between years 1997 and 2000 when heavy clients with complete Java programs running in the browser Java virtual machine were very common, see Figure 1, although it seems that nowadays this practice is becoming obsolete and light HTML clients are again the most accepted choice, passing the business logic to the server side). As we have said, ABS was decided to be a very distributed system where every single server module could be run in a different machine. In order to develop such scenario, CORBA was chosen as the proper technology to communicate server modules and also to carry out client-server communications, because it was then showing a very good integration with Java and was providing a very good set of facilities. ABROSE on the other hand was not such a distributed system, so although CORBA was needed to communicate with clients, communications in the server were done exactly the same way as they were done in the client (Java local invocations). For inter-domain communications CORBA was used, but different approaches were taken. ABROSE prototype was using a centralized approach, with two modules (one for each domain) responsible for sending and receiving data from the other domain. That way, these two modules are the only modules that need to speak CORBA. As soon as users need to send any request to the broker, they invoke a method in their local communication module. This module contacts its broker counterpart, which delivers the query to the corresponding module inside the broker domain.

Figure 1: ABROSE graphical interface based on a Java applet



6

Another possibility (the one that was used in ABS) would be to make every module responsible for its own communications. As every module in the broker domain is a CORBA object and communications between them are done through CORBA instead of invocations through Java references, every module in the client side could communicate at its own risk with the corresponding broker module. With this distributed possibility, every CORBA module would have to define its programming interfaces using IDL and from the beginning of the implementation stage it would be stated which methods and parameters are accessible from other modules. This may imply a stronger effort in the design stage, but it makes system integration easier because it can be assured that at least module interconnections are homogeneously performed. CORBA utilization, however, implies a speed execution decrease compared to that in ABROSE scheme, where every broker module is executed in the same JVM and method invocations are immediately performed. With ABROSE scheme, there are only two components that must use CORBA, and so communications can be easier managed because the rest of the modules stay absolutely unaware of communications complexity (connections, sending and receiving data, error controls, and so forth). From the point of view of synchronism and deadlock, whenever a Java module was required not to get blocked the typical threading mechanism was used so as to keep the main application free. Using CORBA, there is also another possibility, because deferred synchronous invocations can also be used. These invocations are defined in IDL using the ‘oneway’ identifier and with the only condition of having a void attribute as returning value, they return the control flow to the caller method right away. For more details about Java-CORBA communications issues read [15].

2.2

J2EE Communication Model

The adoption of multi-tier architectures is a common practice when complex applications need to deal on one side with back office systems and databases and at the same time support a diversity of assorted clients (Web browsers, hand held devices, etc.). These architectures can provide encapsulation (functionalities inside an API), separation of concerns (layer specialization) and layer reusability, offering a more flexible and maintainable system. There are many possible multi-tier supporting platforms (CORBA, DCOM, etc.), but for Internet web-based solutions such as e-commerce actual applications, the market is showing a notorious trend towards J2EE [13]. In Smart-EC a three-tier architecture based on J2EE is being used (see also section 3.3). This section will briefly go through a general description of the different communication mechanisms present in J2EE technology, relating them with the already seen Java communication model (and following the same synchronous/asynchronous schema). Section 3.4 will later present communications in Smart-EC as a particular use case of all these subjects. 2.2.1


An enterprise bean (session bean or entity) bean provides services to clients through the bean’s interfaces. There are two kind of enterprise bean access from a client in J2EE: remote access and local access. In J2EE 1.3 there was only one kind of access: remote access by using RMI. Remote and remote home interfaces are RMI interfaces, and in the EJB architecture, one of the tasks of the EJB Container is to provide the classes that implement the remote and remote home interfaces. One of the advantages of using RMI is that these objects are accessible from a client through the standard Java RMI API. However, the inefficiencies of passing objects by value are also present. One example of the problems in efficiency caused by the remote interfaces is the access to entity beans. Entity beans represent an object persistently stored in a database. A possible way to program a distributed application is that all the components that need to handle a persistent object, access directly the entity bean. The entity bean will have “set” and “get” methods for the attributes of the object that it represents and the clients use these methods to change and consult the values of the attributes. Since the clients are in fact handling a remote object (the entity bean), every time a “get” or a “set” method is used, it is in fact a remote operation, and RMI inefficiencies come into play. The solution to this is



7

already a standard solution, also called a pattern. It consists of using Value Objects that agglutinate all or a subgroup of the attributes of the persistent object, so instead of accessing each attribute separately, the client gets or sets all these values in just one remote method invocation to the entity bean, passing as a parameter or receiving as a return value the Value Object. This is normally combined with another standard solution, the session façade pattern. The session façade pattern consists of centralizing all the access to the entity beans of the application in a session bean, which resides in the same physical machine as the entity beans, to reduce the many remote references and method invocations between client and server that would lead to network performance problems. See [14]. In J2EE 2.0, a new kind of access to enterprise beans is introduced: local access. Local access is only possible when the client and the enterprise bean are located in the same Java virtual machine. In this case a local client accesses a session or an entity bean through the bean’s local interface and local home interface. The container provides classes that implement the bean’s local and local home interfaces. The objects that implement the local home interface and local interface are local Java objects. The behaviour when passing arguments is the same as in Java, that is, the arguments and results of the locally invoked methods are passed by reference. Local access is an improvement over the use of remote access, especially when there are problems of efficiency. The drawback is that a local client and the server object must reside in the same JVM. Going back to the example of the access to entity beans, an efficient and flexible way of handling persistent objects in a J2EE application would be the combination of a session façade session bean that provides the methods to store and retrieve the objects, and a set of entity beans representing the persistent objects, that would be accessed through local interfaces. Both façade session bean and entity beans would reside in the same machine, and the façade session bean would have remote interface to allow the access from remote clients in different machines. This approach was taken in the second prototype of Smart-EC, which used J2EE 2.0., obtaining improvements in performance over the use of a session façade with entity beans with remote interfaces. Synchronous communications between enterprise beans have the same advantage as using RMI in Java: the programmer sees objects and method invocations on the objects. The details of the communications are abstracted, and the application can be distributed over different machines without changes, if remote interfaces are used. However, this way of communicating is blocking: the client has to wait for the server response before carrying out any other actions. Besides, multi-threading is not possible in the J2EE platform (in fact, there are different threads for the different requests from external clients, but this is controlled by the application server). The next section deals with this problem, and the solution provided in J2EE in detail. 2.2.2


The J2EE platform is designed to allow the rapid development of business applications in a modular way. Developers create business-level components that carry out tasks on behalf of a single client and take advantage of general standardized services made available by the J2EE platform provider. The J2EE platform transparently handles the details of concurrent access to common data—mainly by offering the so-called “entity beans”—when several clients access the platform, whereas the business developer creates a business application as if one client at a time only had access to it. Doing so, the J2EE platform frees developers from worrying about deadlocks and race conditions. Most business processes are sequential. Although J2EE easily manages multiple client requests, business-level components cannot create threads. This limitation does not really hamper business-oriented developments because most business applications are mono-thread (although not all, and Smart-EC is a good counter-example.) To put it technically, all J2EE methods are synchronous, or blocking: when a method is called, the control flow of the calling component is handed over to the called component, and the calling component is blocked until a response is received, which in turn results in the control flow being transferred back to the calling component. As a result, one and only one component is running at a given point in time that is responsible for carrying forward the user request. As a result, there is one and only one control flow in the system for each client request sent to the J2EE platform, and the only source of new threads is external: a new thread is created only when a new request is made from outside the platform: there are as many threads running on the platform, as there are client requests being handled.



8

The Java Messaging Service (JMS), a tool for business-level parallelism. A first way of circumventing this limitation that was being thought of was to create fake requests, responsible for performing subtasks on behalf of the initial user request and running in parallel with it. But we discovered there exists a smarter way of achieving a similar result: the already mentioned Java Messaging Service. Denying the use of threads based on business practices seems indeed arbitrary and sometimes too restrictive. Actually, the main reason underlying this choice was merely technical: it lay in the tremendous difficulty of defining a standard combining both high reliability and developer-friendliness—two of the main features of J2EE—when parallelism comes in. High reliability is achieved through a heavy use of transactions (in the computer-science sense of the word, not the commercial one), and multithread transactions complexity cannot easily be concealed from the end-developer. We will see in what sense JMS allows us to walk around this limitation. JMS compared with a corporate e-mail system. JMS can be thought of as a business-level way of introducing multithreading in a programming environment such as J2EE. The support of JMS has been made a mandatory feature for version 1.3 of J2EE. The role of JMS is best explained when compared with a corporate e-mail system: components exchange messages asynchronously by depositing them in the message queue of the receiving component, very much like company employees send e-mail messages to their colleagues that are deposited in the recipient’s mail box, the real life analogue of a message queue. Messages are not deposited directly into the recipient’s mail box, but entrusted to a mail server, the role of which is played here by a so-called JMS provider. How does JMS circumvent the above mentioned multithreading limitation. A regular method call can be seen as an exchange of two messages, the first one from caller to callee (the method call with parameters), and the second one from callee to caller (the method response). These messages possibly pertain to a single encompassing transaction. This is not the case for JMS messages: only exchanges between a component and the JMS provider (the mail server, entrusted with all messages) are subject to a transactional treatment. Such an exchange is only a part of the overall message transmission. As commitment only occurs when all exchanges pertaining to a JMS transaction have been made, one should never use a transaction encompassing a sent and a received message if the latter is expected as a response of the former. This limitation of the reliability requirements makes life easier to J2EE platform developers. See [8] for more details. How MDBs provide parallelism. Message driven beans (MDB) receive and process messages asynchronously: the calling bean does not need to wait for the response before carrying out other tasks, but only has to make a message queue available for getting a future response: this way, some parallel processing is made possible. We will see a detailed example of this with the Transaction Handler of the Smart-EC platform. A message queue as the only MDB interface. An MDB is entirely message-driven, and consequently has no home nor remote interface attached that would allow to define exported methods. As seen, calling a method results in a synchronous message exchange and is not suitable for asynchronous communications. Instead of receiving method calls, an MDB has a message queue attached to it, that allows any bean intending to delegate a task to that MDB to deposit a message describing that task, so that the processing of this task can be differed to a later time. How to simulate logical entities through physical but stateless MDBs. An MDB has no internal state and this feature makes it very similar to a stateless beans. In other words, an MDB processes a request (in the form of a message) and forgets about it. When sending a message to such and MDB, you never know which instance receives and processes it, and actually you do not need to bother about it, as they are all equivalent. This also means that you cannot rely on a physical bean instance to embody in the long run a logical entity you’ve defined: you can only simulate a logical entity by supplying in each message sent the logical entity ID and a serialized version of its previous internal state. The calling component has to keep track of these data between calls on behalf of the (logical) called component. It might seem strange to define a component that is not lastingly mapped to a physical one in memory, but it’s the only way to get on with stateless beans, and in the mean time we’ve now got the possibility of parallelism: when a calling component entrusts an MDB instance with a task through a message, it does not need to wait for the task to be completed or even started: the caller keeps its own control flow and a new one is created for the callee. The caller can continue



9

working, and in particular delegate other tasks to other MDB instances: we have got parallelism within the processing of a single user request. Another consequence of all this is that all the MDBs of a given class actually share the same receiving queue. In other words, the queue receiving the message driving the so-called message-driven beans is attached to the MDB class, and not to its instances.

3. USE CASE: SMART-EC 3.1

Project description

The J2EE synchronous and asynchronous communication mechanisms presented in the previous section have been used in the implementation of the Smart-EC electronic brokering system. Smart-EC [12] (Support for Mediation And bRokering for Electronic Commerce) is a project partly funded by the European Commission under the Information Society Technologies Programme (IST-1999-10130). SchlumbergerSema (Spain), Technical University of Madrid (Spain), National Technical University of Athens (Greece), Laboratoire d’Informatique, Robotique, et Microélectronique de Montpellier (France), France Télécom R&D (France), AQL (France), Tradezone International Ltd. (UK), and Anaya Multimedia (Spain) participated in the project. The brokering systems available in the Internet often deal with simple services separately, or with a service package offered by one provider. Smart-EC focuses on the provision of complex services that require coordinated accesses to different providers through the Internet. The project, which ended on June 2002, has implemented a prototype that is able to analyse a request for a complex service, divide it into simple components, and combine several services or products, offered by different providers, into a solution that satisfies the complex request. The broker helps the customer to refine or change the request until a suitable solution is found. After the customer agreement, it confirms the transaction to the selected providers. As a result, the customer can obtain a complex service, involving several suppliers, in the same way as a simple one.

Figure 2: Smart-EC system graphical interface



10

The main interest of Smart-EC was to design a brokering engine capable of handling the complexity of the division and the combination of services in both the request building and the solution building phases. The user interface was kept simple, without fancy graphics or natural language support. Basically, a service (simple or complex) is specified by a number of attributes, each of them with a set of possible values. If a customer submits a complex service request that specifies the desired attribute values, the Smart-EC engine is able to divide it into several requests, one for each simple service that will be needed, with attribute values derived from those specified by the customer. Then the system searches the best offers for each service, and, if the customer accepts, it contacts each selected provider to get all the components needed to fulfil the request. The customer can specify a priority level and a range of acceptable values for each attribute. The system grades available service offers depending on how well they match the values given by the customer, or, if no suitable offers are found, indicates to the customer which attribute values should be changed. If possible, the system proposes advantageous alternatives to the customer, for example a service with better characteristics than the ones requested at the same price, or a good offer that a provider has announced to start in a few days. The intelligent brokering functions illustrated above are habitually provided by human agents with a good knowledge of the service sector of interest. The objective of Smart-EC was to automate these functions as much as possible. To achieve this goal, the Smart-EC system uses a general service model, complemented with domain-specific ontologies. The service model defines general concepts like customer, provider, attribute, value, request, offer, order, etc., and the logical relations among them. A domain-specific ontology defines concepts that are particular to a given service domain. For example, the ontology used to demonstrate the Smart-EC prototype describes publishing services. It includes a formal definition of concepts like book, author, language, translator, page, binding type, printing technology, etc., and the logic relationships that exist among them, for example: what simple services make up a complex one (e.g. producing a book involves content creation, page design, printing, binding, etc.), the attributes of each service (e.g. a book has page size, number of pages, paper quality, cover material, etc.), the constraints and possible values for each attribute (e.g. cover material can be paperback, hardcover, leather, imitation leather or cloth), and the relationships between the values of the attributes in a complex service and the values of the corresponding attributes for the simple services that compose it (e.g. how the price for a complex service is split between the simple services). See [4] for details. The ontology is the only part of Smart-EC that is specific for a given service sector. The user interface and the system architecture are general, so the broker can be used in different sectors, provided that the necessary knowledge about them is obtained and written as logical relations in the formal language used by the system [17].

3.2

Objectives of the project

In addition to the general goal of automating the provision of complex services over the Internet, the Smart-EC project had a number of specific objectives, including: ● System customisation for multi-device access (Web, WAP, PDAs, WebTV, etc.) based on XML. ● Evaluation of ontology capabilities in e-commerce environments. ● Evaluation of the J2EE technology for e-commerce platform design and implementation. ● Evaluation of the intelligent mobile agent technology and its suitability in the field of e-commerce and telecommunications. ● Test the integration of various technologies: J2EE, XML, ontologies, mobile agents, etc. This paper focuses on J2EE issues. Our experience with mobile agents is reported in [16] and [11].



3.3

11

System architecture

Figure 3 shows the general architecture of the Smart-EC system.

WEB INTERFACE

G E N E R I C

S E R V L E T

ADMINIST. MANAGER

SYSTEM CONTROLLER

USER HANDLER

CUSTOMER HANDLER PROVIDER HANDLER

PROVIDER

COMPLEX SERVICE HANDLER TRANSACTION HANDLER REMOTE PROVIDER HANDLER

LOGIN

WAP INTERFACE

SOLUTION BUILDER

ONTOLOGY HANDLER

DATA CONTROLLER

ONTOLOGY ODS

DB

MODELS

UTILS

Figure 3: Smart-EC system architecture

The Generic Servlet module is in charge of the user interface to access the system. It exchanges messages in XML (eXtensible Markup Language) [5] to communicate with the System Controller module. The use of XML to represent the information is combined with style sheets written in XSL (eXtensible Stylesheet Language) to define the format in which the information is presented to the user (a customer or a provider). This XML/XSL combination allows the system interface to be generic and easily configurable for different types of terminals, both fixed and mobile. Adapting the interface to a new type of terminal is reduced to designing a new XSL style sheet. The System Controller receives messages from the Generic Servlet, identifies the action invoked by the user, translates the content of each XML message into an internal data structure, and passes it to the appropriate module depending on the action requested: the Login module, Customer Handler, Provider Handler, etc. The Customer Handler manages the registration of customers, update of customer profile, and service requests. The module guides a customer in the process of making a request, offering two options: make a new request or select one request from a repository of requests (modifying some values). In order to make a new request, the customer will be able to navigate through the catalogue of services, which is displayed as a tree. If a complex service is selected, the customer can go down the tree to see its components. Alternatively, the customer can define a request for the complex service by filling the values of the complex service attributes, and the system will transparently break it down in other simpler services. The Customer Handler relies on the User Handler to display the tree of services, and the Ontology Handler, which is in charge of the access to the ontology. The Provider Handler is in charge of the communication with the providers and it relies too on the User Handler to display the tree of services. A provider is able to navigate through the tree of services and register offers for services, in a similar way as a customer can navigate and select services to make requests. Normally, the registration of offers will be done by automatic procedures between the providers systems and Smart-EC. The project has distinguished several types of providers with which Smart-EC is able to communicate [18]. The Remote Provider Handler manages the connection to remote providers to check the availability of offers on line and to update their catalogue of registered offers.



12

The Complex Service Handler module manages the division and combination of services in the process of building a solution. It takes the request that the customer has built with the help of the Customer Handler, and uses the Ontology Handler to break down or/and aggregate the services in the request. The results are one or several requests that the Complex Service Handler will then pass to the Solution Builder module, which is responsible for looking for the offers that match each request The Solution Builder looks for the registered offers that better suit a request. This is basically done by comparing the values of the attributes defined in a request to the values that are defined in the offers registered for a given service. Some of the attributes are generic (for example, price or duration), while others are specific to each service (for example, source and target language, or subject of the text, for a publishing translation service). For each attribute, the customer can specify a single value or a range of acceptable values (for example, price less than 100), and a priority level: normal, important, or compulsory. The process of comparing requests with offers is quite flexible. The system will return a solution consisting of offers which will have a grade depending on how well they verify the conditions of the request, giving more weight to the attributes that are more important for the customer. The system can also suggest the modification of the request, according to the registered offers, so the customer can find advantageous offers that would not be presented for the original request. For this purpose, the Solution Builder expands the ranges of attribute values contained in the request, so that offers with attribute values that do not exactly match the customer request, but are “close enough”, are considered by the module. This expansion is smaller for important attributes than for normal attributes. For compulsory attributes no expansion is made, and the offer is discarded if the value given in the offer is outside the range of acceptable values given in the request. In any case, non-compliant attribute values are explicitly marked and are taken into account when grading the offers, as explained below. Therefore, an offer that is not discarded by the Solution Builder in the first place may be discarded later if its grade is too low compared to other offers. The customer defines how many solutions must be found for a request, and can choose one of the two search modes implemented in the prototype: exhaustive or fast. In the exhaustive search mode, the Solution Builder will evaluate every offer registered for the service, and select those that get the highest grades. In the fast search mode, the Solution Builder will search for the first available offers whose attributes are on line with attributes priorities set by the customer in the request. The Solution Builder will begin to search into the preferred providers and will stop the searching process as soon as there are enough acceptable offers. This searching method is faster, since it will not search for every offer, but on the other hand, selected offers may not be the best ones. It is also possible for the customers to select, at the time they make the request, if the system should contact the provider to verify the availability of an offer, that is, if the provider did not run out of stock. While this may increase the delay for obtaining results, it avoids the presentation of offers that would need to be discarded later. For this task, the Solution Builder relies on the Remote Provider Handler module, which is in charge of the communication with the remote providers. The Solution Builder manages request cancellations. It checks a message queue in order to verify if a cancel order has been received (e.g. the customer has cancelled the request, or has closed the browser), and if so, it will stop the processing of the request and return an empty list of offers. The Solution Builder grades each offer depending on how the attribute values of the offer comply with the values specified in the request, taking into account the attribute priorities (for example, a non-compliant value in an important attribute implies a lower offer grade compared with a non-compliant value in a normal attribute). The selection and grading of the offers also takes into account the customer preferences, which can be explicitly specified in the customers’ profile (for example, preferred providers), or learned by the system from previous interactions with the customer. The Solution Builder passes the selected offers to the Complex Service Handler. Each of them contains information about the provider that registered the offer, the attribute values, the grade assigned by the Solution Builder, and an indication telling if the availability of the offer has been verified or not. In addition, some attributes of the offer may be marked by the Solution Builder, for example an attribute that does not comply with the value, or range of values, given in the request, or an attribute with multiple values, one of which must be selected by the customer before ordering the service. The Complex Service Handler combines the offers received from the Solution Builder in order to make up one or more solutions that are proposed to the customer. The presentation of the solutions will depend on the customer



13

preferences, for example, only the best solution or several solutions ordered from better (larger grade) to worse, the level of detail of the list of solutions, etc. When the found solution or set of solutions is not satisfactory, the customer can modify the conditions in the request and repeat the process of offer searching. When an acceptable solution is obtained, the customer can choose it and request the system to execute a transaction in which all the providers involved in the provision of the complex service are contacted. The Transaction Handler module is in charge of reserving the service form each provider. If any of the reservations fail, the rest of them are cancelled. This module uses the messaging services provided by JMS, as discussed in section 3.4.2. The architecture of the system is completed by the following modules: Ontology Handler, in charge of accessing the ontology, Data Controller, in charge of accessing the database of the system (Oracle), Login, in charge of verifying the user identity, Administration Manager, managing the user accounts and service definition, and utils, implementing general functions. Finally, the Models module defines the basic data structures of the Smart-EC service model [4].

3.4

Communications in Smart-EC

As we have seen, Smart-EC platform design and implementation has been totally based on J2EE technology and methodology. According to the previous presentation about its architecture, this section has been divided in two different parts in order to treat both synchronous and asynchronous communications between the different modules of the system. This section will cover the different communication methods that have been applied inside Smart-EC server in order to cope with the diverse problems that have appeared when all the modules were deployed together. Communications with the client have not been treated here (as they were in the case of ABROSE and ABS), since in Smart-EC the client was chosen to be a light one basing it only on HTML and WML, so that classical HTTP communication was used. External communications with Grasshopper agent platform (placed in external providers) are not treated here either. This communication issues are based on a proprietary mechanism and have been covered in [11] and [16]. 3.4.1


The architecture of Smart-EC is strongly based on session beans. The majority of the modules described in 3.3 are session beans that provide a remote interface, or at least, they provide a session bean to be accessed from the rest of the modules. Therefore, the communication between these modules is through RMI, so it is blocking and not very efficient. Besides, J2EE defines that the only one thread can be executing an instance of a session bean. If a client sends a request to an instance which is already executing another request, the client will get an exception to the second request (thrown by the container). This simplifies things for the programmer, because concurrency problems such as deadlocks are not possible, but in 3.4.2 we see that in some cases, to carry out a single request, it would be desirable to perform several tasks in parallel. This is not possible for an instance of a session bean. Also, with this approach, there is the problem of the response time. The Generic Servlet, in charge of the user interface, (which is not a session bean, but a servlet) Generic Servlet uses a remote method to send an XML stream to the SystemController identifying the actions of the user. As a result, the System Controller will need to return another XML stream back, blocking the Generic Servlet while it manages to compute the result. Of course, in order to compute the result, and depending on the content of the XML stream, the SystemController will need to invoke methods not only on local object, as any Java program, but also on other session beans: Login, Customer Handler, Provider Handler, etc. This will be again a remote method invocation, on top of RMI. Then, the next session bean will proceed in the same way: local invocations on local objects and remote invocation on remote objects, etc. As it can be seen, a user request involves a sequence of many remote method invocations. Even in the case that all the components are running on the same machine, these remote method invocations are being transported on top of RMI, so there is the overhead associated to this protocol.



14

In Smart-EC, in order to reduce this overhead, special care has been taken when designing the domain objects that are passed as parameters. These objects, representing users, offers, requests, etc., are defined in the models module. The objective was to reduce network traffic as much as possible. We applied the Value Object pattern, in order to reduce the number of method invocations to get the needed information from remote objects (see 2.2.1) and designed a set of classes with minimal functionality, to avoid the possible overhead associated to loading the class file for the objects. The storage of persistent objects has also been designed in order to reduce remote communication overhead. The DataController provides the functionality needed to handle the persistent objects of the system (users, offers, saved requests) following the model of a session façade and several entity beans 2.2.1. The entity beans are accessed through local interfaces, thus avoiding the use of remote communications and also simplifying the access to the data, since the communication between session bean and entity beans is not constrained by the use of value objects. All these efforts in order to reduce the overhead of the synchronous remote communications are well worth it. Our experience in Smart-Ec has proven it. In our first prototype, there were significant performance problems that made us realize that a re-design was needed, taking more into account the problem of the remote communications. In our second prototype, we have improved the response time for the user, due to the reduction of the overhead for remote communications, the use of local interfaces to access the entity beans, and the parallelisation of the tasks carried out by some of the modules, that are explained in the next section. 3.4.2


Motivation Contrary to most business applications, a negotiation process involving several providers to fulfil a single request like Smart-EC’s is highly parallelisable and can fully take advantage of the multithreading capabilities provided by JMS. It is true that involved threads need some kind of synchronization to assure the overall consistency of the purchased complex service, but they are independent otherwise. The negotiation model presented in the sequel assures the overall atomicity of the complex service offered as the main consistency requirement. This means that all of the simple services needed to fulfil the complex service must be purchased, or—if it is not possible— none of them. Negotiation is carried out by the so-called Transaction Handler of the Smart-EC platform.

Transaction Handler TN

RPH

TN

RPH

TN

RPH

TN

RPH

TN

RPH

link to the external world

Pa

ra ll e l

pr e

re s

er v

at io

n

#1

Smart-EC platform

CH

start global transaction

TC ral Pa p lel a erv es re r n ti o #n

Figure 4: Transaction Handler architecture


link to the external world


15

Transactions Handler Figure 4 depicts the internal architecture of the Transaction Handler and its interaction with the other modules within the platform. The Transaction Handler is made up of a single Transaction Controller (TC), which is called by the Customer Handler (CH), and of several Transaction Negotiators (TNs) interacting each with one Remote Provider Handler (RPH), the platform internal representative (or proxy) of a real provider site. Application level two-step commitment protocol. The negotiation model used here for achieving atomicity is similar to the standardized two-step commitment protocol defined as a kind of fault-tolerant mechanism for transactions in a distributed system (see for instance [7]), but used here at the application level. The main idea behind this protocol is that every party involved in a transaction keeps a copy of its old state until it is notified that all other parties got the needed information to compute their respective new states. One party collects ready signals from the other ones, and notifies them back when all parties are ready. The notification comes in the form of a commit signal, whence the name. If ready or commit signals cannot be sent, rollback signals are sent instead. Detailed two-step protocol. The TH performs a two-step negotiation with each provider, and launches a new thread for each of them. The first step is a reservation (or pre-reservation) step and is performed independently by each Transaction Negotiator. When performed successfully, the TN sends a ready signal to the TC and waits, entering the synchronization state where all negotiations must stop until all reservations are completed. The completion is notified by the TC through a commit signal allowing the confirmation step to be performed. If completion could not occur, a rollback signal (TC_rollback) is sent instead, for instance if the TC previously received another rollback signal (RPH_rollback) from one of the TNs, instead of a ready signal. TN negotiation behavior is summarized on figure 5. A transition is labelled with ‘message received causing the transition / action taken ^message sent back’, the receiver being the TN and the sender being the TC. P denotes the provider proposing three methods: reserve, confirm, cancel called by the TN on the RPH. Shared negotiation model. It is clear that the negotiation model must be somehow shared by the platform and the involved providers to make things works. In other words, providers must implement a cancellable reservation mechanism, very much like in the real life. Any kind of payment model can be attached to the negotiation model. For instance, a price could be attached to the reservation as a compensation for blocking the resource, possibly deducted from the service price if confirmation occurs.

[ service unavailable

OR server unreachable

] ^ RPH_rollback start

begin reservation

service offered

cancellation in progress

reservation in progress do/P.reserve() [ service available

do/P.cancel()

] ^ready

cancel [ all-or-nothing loosened ]

TC_rollback [ other service RPH_rollback ] service reserved when[timeout - x]/TC.warn() give up

commit [ all reservations done ] / P.confirm()^confirm service bought

Figure 5: Transaction Negotiator behaviour as a finite automaton


end


16

The Transaction Handler as a whole is method driven. The Transaction Controller carries the external interface of the Transaction Handler, offered to the rest of the Smart-EC platform and is implemented as a regular stateless bean— therefore making the whole TH look like a regular “method driven” bean. This makes the use of MDBs transparent to the rest of the platform while increasing its overall efficiency. Transaction Negotiators were implemented as Message Driven Beans, so as to allow several instances of them to run in parallel for processing the same user request. As already discussed, every new MDB started represents a new flow of control launched, in addition to the initial one which remains owned from beginning to end by the Transaction Controller when the Transaction Handler is called. How to regulate the number of control flows. The number of control flows owned by the Transaction Handler at a given point in time does not reflect the number of messages sent to the Transaction Negotiators, as some of them are waiting in the Transaction Negotiators’ queue. Instead, it can be computed as: NRemaining messages = Nsent messages – Nthoroughly processed messages Ncontrol flow = NRemaining messages – NMessages waiting in queue It is important to note that the J2EE platform allows you to set a maximum for Ncontrol flow, and this has an obvious consequence on the overall platform throughput. A trade-off has to be found between the desired efficiency and the server central memory capacity. As an example, one might be willing to set Ncontrol flow ≤ 10. Logical Transaction Negotiators and physical MDBs. Let’s derive what was said about MDBs and parallelism in the case of the Transaction Handler. The Transaction Negotiators share a common queue (or mail-box). The Transaction Controller (TC) sends messages to a Transaction Negotiator (TN) via a message queue. As already stated, Transaction Negotiators are MDBs, and MDBs are stateless, which means that they process a request and forget about it. The message queue is therefore owned by the Transaction Negotiators’ class, and not by its instances, and that all Transaction Negotiators actually share the same message queue, which can thus be created once and for all at deployment time. Transaction Negotiators are logical entities that can be identified with an ID plus an internal state and they are only temporarily attached to an MDB instance, when a message is sent by the TC to a logical TN. The TC has to keep track of the internal states of all non-active TNs on their behalf. When the TC sends a message for a task to be performed, the TC has to specify, in addition to the task details, the ID of the targeted Transaction Negotiator (logical entity) and its current internal state. When the task has been performed, and before quitting, the TN sends back to the TC a message containing, in addition to the task results, its ID and new internal state, to be stored by the TC. What is the point in using Transaction Negotiators, then, if the TC has to keep track of everything on TN's behalf? It is true that the TN is not materialized by a permanent class instance, but when carrying out a task, the instance responsible for it has a separate control flow: when the TN processes a received message, the TC can do something else: it does not have to wait for any method return. In particular, it can get in touch successively with other not yet running Transaction Negotiators, which results in several Transaction Negotiators plus the TC running in parallel.

Solution Builder As it has been previously explained, the Solution Builder is the module responsible for managing the solution build-up control flow so it is then one of the main responsible of all the resources that are engaged during this process, as it is going to be seen. Whenever the Solution Builder receives an invocation in order to find several offers, the first thing to be done is to allocate memory space so as to place the resulting offer or set of offers. After some checking has been performed with the incoming request, the database is contacted to retrieve an initial number of offers to solve the request.



17

It must then first be checked that offers are complete, because the system architecture there is the possibility of registering uncompleted offers. That way the user is forced to contact online providers every time while these are not required to keep all their updated offers at Smart-EC local database. After some more test to check offers correctness, it is also possible (if the user selected so in the graphical interface) to verify if the offer is ready at the current moment, so that the user is not receiving any unavailable offer, and so that the Transaction Handler is not initializing any unneeded transaction. The last and most important step is the offer grading. All the offers are carefully checked attribute by attribute to be able to test how well do they request (taking into account the priority already set by the users) and receiving an according score. Supposing there are not offers good enough to build the requested solution, some more offers will be retrieved and the whole process will be repeated for the new offers. The whole process finishes when the final solution is built and returned to the user. All these steps are absolutely transparent to the user and take place without contacting him at all. It also means that if the user does not want to continue with the request or wants to exit the application, the system will internally keep on doing what it was programmed to do without freeing any kind of resources until the process have finished. The problem is that different Java invocations (both local and remote invocations) are successively taking place and so, different modules are being blocked waiting for a response, without being able to do any other action. Even by the definition of a cancellation method in the Solution Builder so that it could be called in order to stop any further processing, the problem would not be solved due to the different synchronous calls that have been made: the Solution Builder is blocked waiting for answers to arrive. In ABS or ABROSE one of the possibilities we have seen was the usage of deferred synchronous calls (oneway methods) or the definition of new threads to delegate different tasks and stay free in order to receive some more incoming invocations. However, we have already mentioned that these solutions cannot be applied here because Java invocations are always blocking ones and because there is no thread management possibilities within J2EE. JMS was again the chosen option although it was also presenting notorious inconveniences. The typical way to use JMS in J2EE is basically to declare queues that are going to receive messages in the J2EE server and then associate the corresponding beans with these queues so that the server can properly deliver messages to the right beans (the Solution Builder would then be periodically checking its related queue searching for a cancellation message that would cause the offer retrieving and grading process to be automatically stopped and all resources to be freed). The problem is that whenever a user is accessing the system a new set of beans is defined for him (different instances of the Solution Builder and of the other beans are then created). This means that different queues would then be needed to be defined so that different instances could receive different messages (and each Solution Builder could distinguish which request is to be cancelled). But this would not be a scalable solution, since a lot of queues would be required. In spite of this, when a queue is associated with a certain bean, it is possible to define the so called ‘selectors’. That way the beans are only going to receive messages from the queue if they are matching the corresponding selector (the request identifier for example, defined as a message property). Apart from all this problematic, there is another important issue to be commented: if all the different modules are blocked because of their successive synchronous calls, how is it possible for them to send the cancellation message (or whatever message is to be sent)? In Smart-EC, this fact has not been completely implemented from the point of view client functionality, since it was not really offering any new feature from the brokering viewpoint and it would cause the system to be much more complex in order to solve the deadlock situation already mentioned (it was not the same with the functionality that the Transaction Handler was required to provide). From a technical point of view however, it was indeed an interesting problem, so it was finally decided to implement the functionality outside the client possibilities. Request cancellation can then be performed



18

from the Administrator Manager console (a Java application that is not blocked by the control flow of the user request) and that way every resource can be cleanly released.

4. CONCLUSION This article has presented different communication mechanisms available for distributed communications within the framework of different European electronic commerce brokering research projects. The first part of the article has covered generic issues about synchronous and asynchronous communications and they have been compared with the schema available in CORBA showing as use case the communication method used in ABS and ABROSE projects. For these two projects, it was decided to use CORBA in order to allow distributed communications to be performed, since Java remote communication mechanism was not very mature yet. With the development of J2EE all these issues have been facilitated and Smart-EC project is using this technology throughout the platform. The article has described Smart-EC project both because of its brokering interesting capabilities and because of its value as J2EE use case, since this Sun technology was launched at the same time as Smart-EC an so it has been possible to follow its evolution very closely. Finally, from the communications viewpoint, the article has shown how is it possible to provide parallelism in a J2EE environment where threads management has been restricted, through the usage of JMS, an asynchronous communication API developed for Java applications and lately included with J2EE last stable distribution.

ACKNOWLEDGEMENTS This work has been partially financed by the European Commission through projects ABS and ABROSE (ACTS program) and Smart-EC (IST program).

References [1]

ABS. Architecture for Information Brokerage Services. ACTS-206. [September 2002]

[2]

ABROSE. Agent Based Brokerage Services in Electronic Commerce. ACTS-316. Available [Internet]: [07 de septiembre de 2002]

[3]

ACTS. Advanced Communications Technologies and [September 2002]

[4]

Bellido, L., E. Vázquez, F. Valera. Modelling complex services in an e-commerce brokering system. IADIS International Conference. WWW/Internet 2002. 13-15 November 2002. Lisbon, Portugal

[5]

Bergholz, A. Extending Your Markup -- An XML Tutorial, IEEE Internet Computing, 4(4):74-79, 2000

[6]

IST. Information Society Technologies Programme. Available [Internet]: [September 2002]

[7]

ITU-T X.851. Information technology - Open Systems Interconnection - Service definition for the Commitment, Concurrency and Recovery service element. December 1997.


Services.

Available

Available

[Internet]:

[Internet]:


19

[8]

JavaTM Message Service Tutorial. Available [Internet]: < http://java.sun.com/products/jms/tutorial> [September 2002]

[9]

jGuru. Fundamentals of RMI Short Course. Available [September 2002]

[10]

Kaveh, N. and Emmerich, W. Deadlock Detection in Distributed Object Systems. In V. Gruhn (ed). Proc. of the Joint 8th European Software Engineering Conference (ESEC) and 9th ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE-9), Vienna, Austria. pp. 44-51. ACM Press. 2001. Available [Internet]: [September 2002]

[11]

Sygkouna, I., M. Strimpakou, F. Valera, A. Kaltabani, L. Bellido, E. Vazquez, M. Anagnostou. Seamless Incorporation of Agents in an E-Commerce Intermediation Platform. Fourth International Workshop on Mobile Agents for Telecommunication Applications, MATA’02. 23-24 October 2002. Universitat Pompeu Fabra. Barcelona, Spain.

[12]

SMARTEC. Support for Mediation And bRokering for Electronic Commerce. IST-1999-10130. Available [Internet]: [September 2002]

[13]

Sun Microsystems Inc. (2001). JavaTM 2 Platform, Enterprise Edition v 1.3. Available [Internet]: [September 2002]

[14]

Sun Microsystems Inc. (2002). Sun Java Center J2EE patterns, Available [September 2002]

[15]

Valera, F., J. E. López de Vergara, J. I. Moreno, V. A. Villagrá and J. J. Berrocal. Communication Management Experiences in E-Commerce. Communications of the ACM (ISSN: 0001-0782). 44 (4): 63-71. April 2001.

[16]

Valera F., A. Kaltabani, E. Vázquez, L. Bellido and M. Anagnostou. Middleware and mobile agents experiences in e-commerce. 6th Internacional Scientific Conference on Work With Display Units, WWDU'02. May 22-25, 2002. Berchtesgaden. Germany. ISBN: 3-925251-08-1. (pág: 536-538).

[17]

Vázquez, E., L. Bellido and F. Valera (editors). SMART-EC architecture v2. Smart-EC Project. Technical report D42. 2001. Available [Internet]: [September 2002]

[18]

Vázquez, E. and F. Valera (editors). SMART-EC architecture v1. Smart-EC Project. Technical report D41. 2000. Available [Internet]: [September 2002]


[Internet]:

[Internet]:

Parallelism and messaging services in a J2EE-based e-commerce

Parallelism and messaging services in a J2EE-based e-commerce

Suggest Documents