.
F
E
A
T
U
R
E
DASHER: The Dasher project focuses on the
A Prototype for Federated E-Commerce Services
procurement subset of e-commerce. It aims to
C URT P OWLEY, D AVID B ENJAMIN , D AVID G ROSSMAN , R OBERT N ECHES , PAUL P OSTEL , E RNESTO B RODERSOHN , R UPAL FADIA , Q UAN Z HU , AND P ETER W ILL University of Southern California Information Sciences Institute
provide a common service infrastructure that supports distributed applications and federations of loosely affiliated software components.
T
raditionally, distributed applications emphasize a high level of integration under control of a central authority. These applications typically map onto private networks or proprietary value-added networks (VANs). Experience has shown that such applications are difficult to build, maintain, and enhance. In contrast, federated applications promise parallel development of individual sites, flexible adaptation to changing environments, and smooth upgrade to new technologies. “Federated” refers to union by alliance rather than integration through central authority. Distributed Web applications, based upon server/browser middleware that supports HTTP, HTML, Java, and CGI, are instances of federated applications. Ideally, federated applications will simplify implementation, maintenance, and enhancement of individual sites of distributed enterprises. In addition, federated approaches facilitate a competitive market of component services, in which custom systems could be assembled to the requirements of a given organization by populating a skeleton system with selections from alternative component offerings.1 In this article, we introduce our vision of an architecture for federated services and discuss our prototype implementation, Dasher (Defense Acquisition System for High-performance Services), which was developed to demonstrate and test this vision in the context of electronic commerce services. The Dasher effort leverages a longstanding electronic procurement application called FAST,2 which has evolved during continuous operation since 1987. The FAST implementation intertwines multiple procurement services, which are controlled asynchronously during operation. Dasher modularizes and
1089 -7801/ 97/$10.00 ©1997 IEEE
62
IEEE INTERNET COMPUTING
.
D
restructures FAST to investigate a federated systems approach.
A
S
H
E
R
P
R
O
T
O
T
Y
P
E
User Buyer
Order system BACKGROUND . The University of Southern California Web browser E-mail Fax Phone Information Sciences Institute (ISI) has a long history of research and development in intelligent logistics and electronic comE-mail Web server Fax Fax merce. Its efforts began in 1979 with the MOSIS3 project, an online broker of VLSI FAST fabrication services. In 1986, ISI began FAST Phone Phone operator developing FAST, the first online broker for Vendor off-the-shelf components over the Internet. Dialog ISI continues to run MOSIS. It recently Supervisor manager spun off FAST as a separate commercial Online (Appscan) order enterprise, FASTXchange, Inc.,* which Order system Sourcing Quoting operates custom versions for large corporate management E-mail customers, a public-access version called Diana for consumer and small-to-midsize corporate buyers, and a specialized version Figure 1. System diagram of FAST operations. for government use. Over $14 million in orders to over 4,000 different vendors have been processed by these systems. rate set of input, output, and intermediate state tables for FAST provides Internet access to buying organizations each subservice. The main polling loop, written in C, uses and “come-as-you-are” access to thousands of vendors by an SQL interface to scan the relevant tables for each service, multiple communication paths, including e-mail, X12 elec- determines the state of each pending process, and takes the tronic data interchange, agent-based interfaces, and auto- appropriate action. FAST subservices interact by writing matically generated fax. It provides e-commerce services such data to one another’s tables. The polling loop uses a dialogas sourcing, quoting, and ordering. It also supports operator management program that has scripts allowing it to log in interaction, access to external online data services, and access and interact with various proprietary external subservices. to several vendors’ online ordering and inventory systems. Leveraging our experience with FAST, we have focused System Interfaces Dasher on procurement aspects of e-commerce. Rather than FAST deals implicitly with four essential interfaces that for try to reimplement all of FAST in our Dasher prototype, we federation must be explicit. These interfaces are formalized have concentrated on inserting module boundaries using in our proposed architecture. Figure 2 shows the resulting distributed-object computing technology in conjunction system diagram for future federated services. As in FAST, the with code reuse. Modularizing the system into smaller com- two main blocks are the requester and provider subsystems. ponents organized along functional boundaries, with welldefined APIs at a high level of abstraction, ideally makes it ■ Remote service interconnection. This interface allows easier to enhance and maintain. Additionally, this approach objects to access remote objects while making the underadvances our longer term goals of supporting custom enterlying communication methods as transparent as possible. prise systems via component swapping, and integration We assume that services provide meta-service information that includes a service access model, enabling across enterprises via component interoperability. This artiprospective requesters to access the service without hardcle reports our progress in identifying the requirements of coding any of the details. federated applications and in proposing, developing, and ■ Local service interconnection. With local service intertesting solutions. connection, interfaces among local services can be made to look as similar as possible to remote service interfaces. PROPOSED ARCHITECTURE Making local services look like remote services is a useFOR FEDERATED SERVICES ful discipline for a company building multiple linked Figure 1 shows a simplified system diagram of FAST proservices because the resulting modularity simplifies curement operations. The three main blocks are the buyer, maintenance and configurability to meet specific the FAST provider, and the vendor subsystems. The requester needs. provider, which is built on an Oracle database, uses a sepa-
IEEE INTERNET COMPUTING
http://computer.org/internet/ NOVEMBER • DECEMBER 1997 63
.
F
E
A
T
U
R
E
Legacy system
Security
User 1
User 2
four key interfaces. The main difference is that the requester does not have a metaservice block.
Wrapper Local service interconnect
User and flow management Workflow interconnect
Persistent storage interconnect
Requester organization
Messages
The architecture also needs to specify the content of messages that are sent between the blocks in Figure 2. Most important among these are the messages sent to and from each service. We identify four main classes of messages: ■
Remote service interconnect
Security
Service A
Service B
Dialog mgmt Workflow manager
Persistent storage interconnect
Metaservice Local service interconnect Human 1
Human 2
Workflow interconnect
■
Service provider organization
Figure 2. Our proposed architecture for federated services. ■ ■
■
Persistent storage. Persistent storage gives each service its own private storage, while retaining as much commonality among services as possible. In particular, the storage for each service should be in two parts: a serviceindependent part (for example, routing data) and a service-dependent part (for example, price quotes). Local workflow management. Federated systems need to recognize the complexity of managing services by providing a comprehensive subsystem that tracks classes of activities and time schedules, advising programs and humans in real time regarding optional actions and relative priorities.
The provider uses a security module to transform an external service request into a local request. In performing a specific service, each service module accesses its local store and, if necessary, generates subservice requests on the local interconnection. The workflow manager coordinates and directs all local services. If no local service can satisfy the service requests, they go out on the remote service interconnect, possibly through a dialog manager, to black-box legacy services outside of the architecture. Each provider maintains a metaservice information block, which provides preliminary information to any remote requester interested in accessing services at this site. The requester side is very similar to the service side, reflecting the small distinction between a requester who is initiating a request for a service and a provider who is delegating requests for subservices. The requester has the same
NOVEMBER • DECEMBER 1997
64
http://computer.org/internet/
■
Meta-information. Meta-information establishes a format for describing a module’s capabilities. Query capabilities should include service index terms, access method, security method, pricing and availability, output format, billing and payment practice, special qualifications, and means of delivery. Additionally, a query should be able to obtain a model of the service in terms of the essential aspects of the three other classes of messages below. Registration. The registration process enables a particular entity to invoke a service at some time in the future. A registration request is addressed to a service name and establishes details regarding the user, security, routing path for the response, pricing, timing, and miscellaneous constraints. Invocation. The invocation process is directed to a particular service name. The arguments include authorization, tied back to the registration process (for example, by a user name and password); an action such as “invoke service” or “evaluate a what-if request”; a unique identifier (supplied by the user and/or the service); event handling (poll or post); return path; pricing and timing information; miscellaneous constraints; and service arguments. The unique identifier and event-handling fields are required because many services are long-lived and asynchronous. For instance, it may take a week to obtain one price quote, two weeks to obtain three quotes, and an infinite time to obtain four quotes. In such situations, the service would provide some immediate response to the requester, but the full service response would come later in a follow-up message. Follow-up. The follow-up request is directed to a particular service name. The query again provides authorization, unique identifier if appropriate, return path, and requested action. The actions can be short (have you completed?), long (what is the result?), summary (what services have you performed over the past month?), or cancellation.
This general architecture for federated services represents our long-term goal. As a first step toward implementing it, we built part of the architecture shown in Figure 3. In designing and implementing the prototype, we divided the effort into two parts: the services themselves and the remote interconnection system.
IEEE INTERNET COMPUTING
.
D
A
S
H
E
R
P
R
O
T
O
T
Y
P
E
Service
Remote service interconnect Trigger Dialog management
Process
CORBA
Local service interconnect Sourcing
Quoting
Figure 4. Design of a general service.
Persistent storage interconnect
from the terminal, formatted into a message using a template, and e-mailed back to the requester. In the Dasher prototype, templates are used for parsing and creating messages. Figure 3. The Dasher system. Design of Sourcing
DASHER: SOURCING AND QUOTING SERVICES Internally, services are structured as shown in Figure 4. A service handles requests by using an object-oriented state-transition model, with chunks of code activated by triggers associated with states. An Oracle relational database stores information about the request as transitions occur through stages of processing. External data services are accessed automatically as needed, using a CORBA interface where possible. The data stored in relational tables represent objects. For example, vendors are represented in two tables: vendor contains the vendor name and vendor_terms contains terms that describe the items sold by the vendors. Each vendor is represented by a row in the first table and several associated rows in the second. Structuring the data as objects enables a smooth interface between the internal representation of the data in the service and our object-based service interconnection scheme. The flow of processing is controlled by defining state fields (columns) for certain tables and associating triggers with them. A trigger associates an action with a specified condition in a table. When a row of data achieves a predefined state, the trigger causes execution of a block of code, which can be written in SQL, Perl, C shell, C, and/or C++. Typically, the resulting processing can update the row of the table that caused the trigger, update a table downstream in the processing flowpath, or initiate a request to a process external to the service. Thus, a sequence of triggered processes systematically transitions a request through a sequence of states until the request is completed. A local program called Appscan, developed as part of FAST, is used as an intermediary between services and external data services. Appscan is a dialog-management system that simulates a user interacting with a remote third-party computer service designed for online human interaction (a similar commercial application is Edify*). Using a script tailored for the specific online service, Appscan logs into the online system, navigates through its forms, and enters the request using the data in its message. The output is read
IEEE INTERNET COMPUTING
Sourcing identifies potential suppliers (sources) of a part or service. Currently, our prototype only finds sources for parts. The core input to the Sourcing service is a description of a part, structured as a set of attribute-value pairs. Typical attributes include part number, manufacturer, federal supply class, and national stock number. In addition to the part description, the incoming request includes information about the requester, such as the route for returning the response. When the input request is inserted into the input table, the state field in the newly created rows are set to NEW, triggering SQL code that attempts to identify an external data service that can be queried to find sources. The SQL code compares the attributes describing the part with the attributes required by several external sourcing services. If a match is found, the attributes needed for each query are inserted into a set of query tables and the appropriate state values set. Perl code is triggered, which extracts information from the query tables and uses a template to create a message to Appscan. When a response is received, Sourcing uses Perl code to parse Appscan’s response using a template, inserts the results into a set of tables, and sets the appropriate state values. SQL code is then triggered, which constructs the responses for transmission back to the Sourcing service requester. Design of Quoting
The Quoting service obtains price and availability of parts. The core input to Quoting is a part description, just as in Sourcing. There may also be constraints, for example, that quotes be obtained only from small businesses or sources meeting certain quality standards. When Quoting receives the input, it sends a request to the Sourcing service using CORBA, which asynchronously returns a list of candidate vendors for the part. While waiting for the Sourcing response, the Quoting service can continue with other work. When potential vendors are identified, Quoting selects the subset of vendors that have an automated online quote capability. Currently, Dasher’s Quoting and Sourcing services handle only those requests that can be fully automated.
http://computer.org/internet/ NOVEMBER • DECEMBER 1997 65
.
F
E
A
T
U
R
E
Web client
E-mail customer
Both of the two main modules, Sourcing and Quoting, have CORBA interfaces for receiving Web requests, and for X12 translator communicating with each Web server other. (CORBA is not yet fully implemented for the Web interface to Quoting. However, this technology has already been Sourcing Quoting tested and is fully operational for Sourcing.) The Web-client requests for Sourcing or Quoting are forwarded to the Web server, which submits Appscan them to Sourcing or Quoting CORBA client/server through the CORBA interface. USC Information Sciences Institute E-mail There is also an ANSI X12 Telnet or dial-in interface to Sourcing through HTTP Third-party CORBA (for a detailed descriplegacy systems tion of the ANSI X12 interface, see the sidebar ANSI X12 EDI Figure 5. Dasher prototype architecture. Platforms located at ISI are contained within the Message Format). The Sourcing module best demonstrates our shaded box, while modules that can or do run on external platforms are outside the box. desired architecture with three implemented connections to its It does not duplicate the extensive support tools FAST pro- CORBA interface: ANSI X12, Web, and Quoting. The same vides to its small staff of operators who handle requests that API is used for all three of these Sourcing requesters. Requests cannot be automated. from the three modules are indistinguishable to the Sourcing For each vendor, a template is used to construct an e-mail module except for the return routing information. message to Appscan. Appscan logs into the online vendor’s Both Quoting and Sourcing use external third-party legacomputer, obtains up-to-date price and availability of the cy systems that require communication by telnet or modem part, and e-mails the response to Quoting. Quoting inter- dial-in. We use Appscan to bridge these systems. Since prets the Appscan response with a template, records the part Appscan was developed as part of FAST, it has an e-mail information in the database, and sends a quote response interface rather than a CORBA interface. If upgraded for back to the requester. our prototype, Appscan would present a CORBA interface to Quoting and Sourcing, making possible a single interface DASHER: THE SERVICE INTERCONNECTION mechanism to our service modules. We chose CORBA as the principal method of remote and Except for Quoting and Sourcing, all the modules shown local service interconnection because it supports detail trans- run on different machines and communicate via a local area parency and interface standardization, and it has a reason- network within the ISI box, or via the Internet. All are able chance of becoming a prevalent industry standard.4 designed to communicate using the Internet, but this capaCORBA is designed to provide interoperability between bility is not yet fully tested. Quoting and Sourcing share the applications that may be written in different languages and same machine and database. However, with the exception of may run on different platforms. Conceptually, a heteroge- a few tables of static data (for example, ISI staff names) and neous network of clients and servers implemented in differ- the table of vendors, they each use their own set of tables, and ent environments can interoperate with CORBA shoulder- data is shared between them only through the CORBA intering the burden of lower-level details such as choosing a face. These two services could be run on separate machines transport mechanism and establishing a network connection. with separate databases by simply duplicating the static-data tables and adding a separate vendor table to Quoting. Overall Design Not shown in Figure 5 is the Monitoring module, creatFigure 5 gives a more detailed view of the architecture of our ed to monitor the execution of Sourcing and Quoting prototype implementation, illustrating how interconnected requests. This tool can be used either at the systems level for modules communicate. debugging purposes, or at the applications level for moni-
NOVEMBER • DECEMBER 1997
66
http://computer.org/internet/
IEEE INTERNET COMPUTING
.
D
A
S
H
E
R
P
R
O
T
O
T
Y
P
E
ANSI X12 EDI MESSAGE FORMAT The ANSI X12 standard1 specifies a format for electronic data interchange (EDI) of business data, including procurement transactions such as request for quote and purchase order. The structure of messages transmitted between business partners is hierarchically decomposed as follows: ■
■
■
■
Interchange—message routing and control header followed by transaction sets grouped by business function to aid routing within the recipient’s organization. Transaction set—information for a single, specific business transaction. Each of the hundreds of types of transaction sets specifies an ordering of segments, commonly including loops (repeating subsequences). Segment—intermediate unit of information, such as an address, which is in turn composed of assorted data elements. Most segments recur in multiple types of transactions. Data element—a qualifier, a value, or text to be interpreted according to the type of its segment and its position within the segment.
While ANSI X12 precisely specifies the potential syntax of messages, the meaning depends on interpretation of the textual names of the segments and their elements. Consequently, to understand each other’s messages, partnerships must first negotiate agreement on the specific syntax to be followed and the interpretation of each field. Then, each partner must implement a translator between this agreed-upon format and its internal business data representation.
toring the progress of a transaction. We implemented this service using Java applets and Sun’s Java-to-DatabaseConnectivity (JDBC) API.* The monitor simplifies tracing and debugging tasks by presenting a sequence of boxes on the Web page that change color. From an applications level, this indicates the percentage of the Sourcing or Quoting request completed. When Sourcing completes, the monitor flashes a message to the user and displays sourcing results when clicked. From a systems level, each box corresponds to a process that must be executed to complete the request. Thus, when there is a problem, this correspondence can be used to locate which process in Sourcing or Quoting to investigate. We believe that this monitor technology has important potential for visualization and for metalevel control of federated applications. The same technology can also be applied to other tasks such as metering or performance measurement. Operation
To explain in more detail how the prototype modules interoperate, we trace the execution of a Quoting request through the Dasher prototype shown in Figure 5.
IEEE INTERNET COMPUTING
Large companies can define a single convention for anyone wishing to do business with them. However, small companies have relatively high overhead for electronic commerce because they must conform to the conventions established by their more powerful partners and because the translator setup for one partnership often has little carryover to the next. Some translator software vendors supply plug-ins for some large company and government conventions, but this solution is proprietary and requires redundant efforts by each vendor to implement and maintain the plug-ins. Further, differences in semantic interpretation may not be addressed, particularly with respect to connecting to internal business data. The X12 standard does not specify the message transport mechanism. This is often a value-added network, but may be other means, such as Internet e-mail, FTP, or HTTP. Dasher is developing a model-based ANSI X12 translation service to reduce the overall effort for EDI and enable rapid partnering. It interprets a company-specific translation model to map into a hub representation with concise semantics. Using the sending company’s model to translate into the hub representation, and the receiving company’s model to translate to a converted message, any company could exchange X12 transactions with all others that have registered their translation model. REFERENCE 1. Electronic Data Interchange X12 Standards, Rel. 3070, Data Interchange Standards Assoc., Alexandria, Va., Dec. 1996.
Phase 1. A request is initiated when a user references the URL of a Web server HTML document for quote requests. Using standard HTTP, the Web browser displays an input form. The user enters attributes of the desired part and submits the Quoting request. The Web server receives the request and passes it to a CORBA client using the CGI protocol. The CORBA client calls a CORBA-server function to request a quote. A CORBA process called an Object-Request Broker (ORB) transparently intercepts the call and passes it to the CORBA server for the Quoting module. The CORBA server inserts the quote request into a table in the relational database for Quoting. Because the CORBA API is object based, the database insertion must map the objects into relational tables. We chose Intellicorp’s PowerModel5 for the conversion tool because it also provides useful development and rapid prototyping capabilities. For example, building on top of PowerModel, we created a development environment that automatically reads table definitions and generates much of the code used to communicate between services. The insertion of the Quoting request into the database triggers the Quoting process, which runs using the state-transi-
http://computer.org/internet/ NOVEMBER • DECEMBER 1997 67
.
F
E
A
T
U
R
E
tion model until it triggers its CORBA client to query Sourcing for a list of potential suppliers. The CORBA client uses PowerModel to extract the part description from the database and makes a call to the Sourcing service CORBA server. Phase 2. Sourcing’s CORBA server inserts the request into its
database via PowerModel, which triggers the Sourcing module to identify which external databases will accept the part attributes as input. Sourcing e-mails a request to the Appscan service. Using either the telnet protocol on the Internet or a modem dial-in, Appscan logs into the external databases, e-mails the identified vendors to Sourcing, and Sourcing processes these responses for return to Quoting. The CORBA client extracts them from the database using PowerModel and returns them by calling Quoting’s CORBA server. There is an interesting issue involving the use of a clientserver model such as CORBA’s for the interaction between two modules such as Quoting and Sourcing. A call from a client to a server can be thought of as a function call: the client passes arguments to the server, which subsequently returns results. In this model, the calling client waits, or blocks, for the results of the call. However, since Quoting and Sourcing are intended to be autonomous agents, it is inefficient and undesirable to have Quoting wait for results from Sourcing, especially since Sourcing must first wait for results from Appscan. To make the interaction between Quoting and Sourcing asynchronous, Sourcing sends its response in a separate client call to Quoting, with the results passed as arguments in the call. The separation of a request from a response is often necessary in a collection of federated services, especially when the servicing of a request may involve recursive interaction with other services. Another consideration that arises from the separation of the service request from the service response is that the requester must save the context of the call, and there must be some way for the requester to associate the response with the call. We handle this problem in two ways. ■
■
First, the service requester specifies a request label to the service provider, and this request label is returned to the requester in the asynchronous response. The request label is treated as a “black box” by the service provider, while the requester uses it as a unique label identifying a response. Second, the service provider returns a “ticket” to the requester, which can be used later to refer to the request. The ticket is treated as a “black box” by the requester, while the provider uses it as a unique label identifying a request.
Phase 3. Quoting determines which of the vendors report-
ed by Sourcing can be contacted automatically, then requests quotes from them using Appscan. Appscan is used here in
NOVEMBER • DECEMBER 1997
68
http://computer.org/internet/
the same manner as by Sourcing in phase 2. All current connections to online vendors are by modem dial-in, either directly to the vendor or through a VAN interface to the vendor. Currently, we have tested this interface with only three online vendors, although Appscan is configured to talk to seven vendor computers. The Monitoring module tracks the progress of Quoting, and the user is notified when quotes are received from the vendors. Phase 4. Details of the quotes are retrieved by the user and
displayed using a Web browser. LESSONS LEARNED AND KEY ISSUES Although we based the design and implementation of the Dasher prototype on the experience acquired over many years on FAST, our experience has already exposed a number of necessary further improvements, which can be separated into two categories: services and systems. Services
Our experience in modularizing the services for Dasher has revealed shortcomings in several areas. Inquiries may sometimes generate too many responses. For instance, a Sourcing service may be able to identify thousands of sources for a common part when the requester wants at most 10. What is needed is a means for requesters to express optimization criteria and a means for services to reason with them. Preferences could be formalized into requester-supplied metrics that could then be used by the service to rank its potential responses. Selection.
Obviously, a much greater range of business interactions could be supported if negotiation between services was available as an alternative to simple request-and-respond transactions. Two broad classes of issues must be addressed. The first issue is the coverage and semantics of protocols for proposing and resolving alternatives to requested actions. The fundamental requirement is a machine-interpretable language and message-passing protocol by which communicating services can scope the variations associated with a request, or offered in a response. This enables requesters to publicize constraints and responders to return counteroffers structured for evaluation. The language and protocols could handle a large number of cases by expressing classes of variation and parameterized limitations within classes. Examples include quality of service, schedule and format, specification change, and tradeoffs and limitations. A second issue relates to the activity management protocols and processes for controlling information flow and time allotments around a negotiation. These include cutoff times and procedures for obtaining responses, evaluating them, and generating counterproposals, as well as the routing of inforNegotiation.
IEEE INTERNET COMPUTING
.
D
mation to automated or human components that perform steps in those procedures.
A
S
H
E
R
P
R
O
T
O
T
Y
P
Other services needing improvements include data reliability (to ensure that the latest data is the most accurate); routing (to direct the response to the desired interface); functional sourcing (to enable users to locate products by type, rather than by specific part number); and third-party qualification (to ensure trustworthiness of the service).
compiled and linked with third-party precompiled software modules from libraries. Processes are executed by complex third-party interpreters. Systems have multiple layers with calls passing between independently written and running process layers. Groups of processes interoperate on the same platform, on a LAN, and across the Internet. Difficulties involve compatibility, reliability, system management, and troubleshooting across modules, versions, and platforms. Moreover, inconsistencies are subtle and execution is difficult to trace. Finally, there is typically no single expert. In our prototype system, the interacting processes include an Oracle database relying on Unix system calls, a CORBA ORB, several Perl programs interpreted by a Perl interpreter, C shell processes, several e-mail processes, a Web server, and a Web client. The failure or misbehavior of one process locally or remotely can cause the system to fail with confusing symptoms. During development, porting, and operation of the Dasher system, we frequently encountered significant difficulties stemming from such complexity. These problems cannot be neatly classified as an error in a specific module. Besides dealing with actual faults, it is also important to understand the intended behavior of middleware, and to design user applications for proper interaction with it. Ironically, a number of potentially tricky faults arise from fault tolerance. For example, the CORBA standard requires operation of an ORB to route client requests to an appropriate server. In the implementation of CORBA that we use, if an ORB fails, another ORB on the LAN (if one exists) will automatically and transparently pick up the failed ORB’s responsibilities. This can introduce dependencies on processes outside the presumed system boundaries without operator knowledge. One approach to reducing the operational complexity of a federated system is to monitor and control its behavior, as discussed earlier. Commercial systems and standards that can be used for this purpose include network monitor standards such as SNMP7 and CORBA-based monitors for distributed systems such as OrbixManager* from Iona Technologies. Potential problems in building a monitor for a federated system include
Systems
■
Issues related to timing are endemic in federated systems. For example, should a service return information to a requester incrementally as it arrives, or collect partial results until ready for a complete delivery? A common set of timing problems involve uncertainty about completeness of responses—for example, how long to wait for responses before closing a request, and how to handle responses received after a transaction is declared complete. Our research group is exploring analogies between the control issues in coordinating relatively low-level software components and the approach to negotiation outlined above. Timing.
Modeling of external entities. This feature would support organization-based authorization to access service information. For example, in a given company the production planning department might check on Quoting status to find the availability of a part, while the purchasing department might access the same service request to determine the price. To handle these situations, Quoting would need to save state information about the status of the request relative to a model of the organization. Another consideration is that data about companies and people is often highly dynamic, and organizational structure is often Byzantine. Hence, operational overhead is much lower if flexibility is designed into the database schema.
Human involvement. Although our goal is to reduce
human involvement, we have learned from FAST operations that some human interaction is critical, and we suspect will always be required.
The long-term goal that motivates our work is to develop federated system technologies for building and maintaining large systems composed of loosely affiliated, easily interconnected, independently developed software components. The term “megaprogramming” refers to the practice of building and developing large-scale software systems.6 For federated systems, megaprogramming is very difficult. Ideally, the interfaces between software units are well defined and focused and the composed system is robust, fault tolerant, and maintainable. In practice, these traits are hard to achieve. In large tightly coupled systems, software is
IEEE INTERNET COMPUTING
■
E
extending local monitors to work with loosely coupled, independent systems across the Internet and, integrating application-level modules with monitoring middleware to ensure proper behavior.
Solutions should enable a module to publicize techniques for detection of proper operation on the part of that module. Other modules could use this information to monitor and detect failure of the module, and take appropriate action. A related task would be to develop and standardize error interaction between modules, so the overall system behavior in the event of local failures better flags the troublespots and
http://computer.org/internet/ NOVEMBER • DECEMBER 1997 69
.
F
E
A
T
U
R
E
produces more graceful degradation. These methods could also be used to evaluate what part each module played in the system failure. CONCLUSION At the very highest level, the overall Dasher project is concerned with federated Web applications and information repositories relating to parts and services. To achieve its objectives, Dasher’s research includes work on middleware and a framework for modular services, as well as on readily usable and configurable services that can be put to use in multiple information repository applications. Users want to access parts and services with appropriate capabilities for particular tasks that they have in mind. They may also want to negotiate with vendors of those goods or services to obtain customizations. Thus, the global concerns of the effort are to ■ ■ ■ ■
increase the speed with which customized, task-oriented information repositories can be formed; simplify the search process; develop novel mechanisms that let users negotiate with service providers; and provide mechanisms that assist users in combining materiel and services to perform their tasks.
The techniques being developed by Dasher are now being cast at this more abstract level to increase its utility in applications that require accessing and combining information services. It is too early to report much of the work in these four areas beyond what we have described in this article. We have made some recent progress on component techniques for helping users find out where they should be focusing attention in a large set of documents.8 We are now embedding those techniques in a suite of interactive tools for helping users to make sense of large information spaces. We expect the implementation of this information-space analysis tool suite to provide another test of our federated framework approach. Problems of portability, compatibility, and system fragility do not magically evaporate by application of a federated approach. Although the approach eliminates a number of the problems, it introduces others, all primarily due to interactions between modules. Thus, the ultimate success of federated applications depends on offering improved development environments and monitoring tools that can prevent or catch these problems. As those tools progress, it will be possible to introduce richer services, such as negotiation mechanisms. It will also lead the community closer to a marketplace of services for creating custom systems from assemblies of competing componentware. ■
NOVEMBER • DECEMBER 1997
70
http://computer.org/internet/
ACKNOWLEDGMENTS Other contributors to FAST include Craig Milo Rogers, Anna-Lena Neches, César Piña, Robert Wormuth, and Eric Andersen. In addition to the co-authors, Brian Harp, Carole Sumler, Yann Philippe, and Ranjeet Shetye contributed to the Dasher project. The authors thank M. Goncalo Quadros, Paul Finster, and the reviewers for their careful review. Information concerning Dasher can be found at http://www.isi.edu/dasher/. Our research was supported by the Defense Advanced Research Projects Agency, Information Technology Office, under contract 95-CDABT 63-95-C-0101. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the DARPA or the US government. REFERENCES 1. R. Neches et al., “Electronic Commerce on the Internet,” white paper to the Federal Electronic Commerce Action Team, 7 Mar. 1994. Also available at http://www.isi.edu/dasher/Internet-Commerce.html. 2. A.-L. Neches, “FAST—A Research Project in Electronic Commerce,” Electronic Markets, No. 9-10, Oct. 1993, pp. 25–27. 3. C. Tomovich, “MOSIS—A Gateway to Silicon,” IEEE Circuits and Devices, Vol. 4, No. 2, Mar. 1988. Also available at http://www.isi. edu/mosis/. 4. J. Siegel, CORBA Fundamentals and Programming, John Wiley & Sons, New York, N.Y., 1996. 5. PowerModel User’s Guide, Intellicorp, Mountain View, Calif., Mar. 1996. 6. G. Wiederhold, P. Wegner, and S. Ceri, “Toward Megaprogramming,” Comm. ACM, Vol. 35, No. 11, pp. 89–99, Nov. 1992. 7. W. Stallings, SNMP, SNMPv2, and RMON: Practical Network Management, Don Mills, ed., Addison-Wesley, Reading, Mass., 1996. 8. Q. Zhu et al., “Searching for Parts and Services on the Web,” Proc. Int’l Symp. on Research, Development, and Practice in Digital Libraries, Tsukuba Science City, Japan, 18–21 Nov. 1997, pp. 123–130. Curt Powley is a computer scientist at the USC Information Sciences Institute. His research interests include the design and development of Internet-based modular and interoperable services, techniques for supporting communities of Internet users, and methods for navigation and information filtering on the Web. He received his PhD in computer science from UCLA in 1993. David Benjamin is a research scientist at USC’s Information Sciences Institute. His research interests include internetworked activity management for rapid definition, rapid deployment, and agile management of distributed, highly usable enterprise applications. He received his BS and MS in information and computer science from the University of California, Irvine in 1981 and 1984. David Grossman was a project leader at the USC Information Sciences Institute. Prior to joining ISI, he was a researcher and senior manager at the IBM Yorktown Research Lab. He is currently involved in starting a new company, NetEarnings, Inc., to provide financial
IEEE INTERNET COMPUTING
.
D
services to small businesses via the Internet. Grossman has a BS and PhD in physics from Harvard University. Robert Neches is the senior project leader for electronic commerce in the Enterprise Integration Systems Division at the USC Information Sciences Institute (ISI), as well as research associate professor in the USC Computer Science Department. Neches received a PhD from Carnegie Mellon University in 1981.
A
S
H
E
R
P
R
O
T
O
T
Y
P
GET CONNECTED with a new publication from IEEE Computer Society
IEEE Internet Computing V V O O LL U U M M EE
II
• •
N U M B E R
I
• •
J A N U A R Y
• •
F E B R U A R Y Y
1 1 9 9 9 9 7 7
ENGINEERING INTERNET and the
Paul Postel is vice president of engineering at FAST’s commercial instantiation, FASTXchange, Inc. He was a member of the FAST Project Research and Development team at the USC Information Sciences Institute from 1991 to early 1997. He has a BS in computer science from California State University, Sacramento.
is a bimonthly magazine that targets engineers and computer scientists using
Ernesto Brodersohn is a senior consultant at Cambridge Technology Partners in Los Angeles, California. His interests are distributed systems, object-oriented systems, the Internet, and distributed artificial intelligence. He received a Masters in computer science from USC and a bachelors degree with honors in computer engineering at the Instituto Tecnolia. Rupal Fadia is a senior engineer working in the Manufacturing Applications Division of Oracle Corporation. Prior to that, she worked as a research assistant at ISI, concentrating on the design of the Dasher project’s relational database. She earned an MS in computer science from USC in August 1996. Quan Zhu is a computer scientist at the USC Information Sciences Institute. His current research focus is in information retrieval and its application to next-generation search techniques on the World Wide Web. He has a BS in physics from the University of Science and Technology in China, and a PhD in physics from Rice University. Peter Will is the director of the Enterprise Integration Systems at the USC Information Sciences Institute. He has worked at Hewlett Packard, Schlumberger, and IBM. He is one of the pioneers of image processing, 3D vision, geometric solid modeling, and robotics, with work dating back to the 1960s in these fields. He has a BS and PhD in electrical engineering from the University of Aberdeen, Scotland. Readers can contact the authors at the Information Sciences Institute, 4676 Admiralty Way, Marina del Rey, CA 90292. E-mail
[email protected].
the ever-expanding
PLUS: George Gilder on the Bandwidth of Plenty
technologies and
Arthur Van Hoff on Java
resources of
Interactive Distance Learning R
the Internet.
®
http://computer.org/internet/
Features ❖ Peer-reviewed articles report the latest developments in Internet-based applications and enabling technologies ❖ Essays, interviews, and roundtable discussions address the Internet’s impact on engineering practice and society. ❖ Columnists provide tutorials and expert commentary on a range of topics. ❖ A companion webzine, IC Online, with unique content and links to other useful sites. To subscribe ❖ Send check, money order, credit card number to IEEE Computer Society, 10662 Los Vaqueros Circle, PO Box 3014, Los Alamitos, CA 90720-1314. ❖ CS member fees (member #_______________) ❖ $29 for six paper issues ❖ $23 for electronic copies ❖ $33 for combined paper/electronic delivery
❖ Nonmember fees: contact customer service at 1-800-CSBOOKS.
URLs FOR THIS ARTICLE
_________________________________________________
*Edify • www.edify.com/ *FASTXchange • www.FASTXchange.com/ Information Handling Services • www.ihs.com/ *Orbix Manager • www.iona.com/Products/Services/ OrbixManager/index.html *Sun JDBC API • java.sun.com/products/jdbc/index.html
_________________________________________________
Name Company Name
_________________________________________________ Address
_________________________________________________ City
State
Zip Code
_________________________________________________ Country
IEEE INTERNET COMPUTING
THE INSTITUTE OF ELECTRICAL AND ELECTRONICS ENGINEERS, INC. ®
E
